Skip to content

gh-154481: Fix reading multi-frame Zstandard members in zipfile#154537

Open
lpyu001 wants to merge 1 commit into
python:mainfrom
lpyu001:zip
Open

gh-154481: Fix reading multi-frame Zstandard members in zipfile#154537
lpyu001 wants to merge 1 commit into
python:mainfrom
lpyu001:zip

Conversation

@lpyu001

@lpyu001 lpyu001 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix zipfile reading of Zstandard-compressed archive members containing multiple concatenated frames.

ZstdDecompressor intentionally stops after one frame and stores any remaining compressed input in unused_data. ZipExtFile previously treated the decompressor's eof state as the end of the entire ZIP member. As a result, later frames were ignored, which could truncate the returned data or cause an incorrect CRC-32 error.

Fix

When a Zstandard frame ends, ZipExtFile now:

  1. Retrieves any remaining compressed input from unused_data.
  2. Creates a new ZstdDecompressor for the next frame.
  3. Continues with unused_data, or reads more compressed data when the frame boundary falls between archive reads.
  4. Marks the ZIP member as finished only after all compressed input and buffered unused data have been consumed.

This keeps the single-frame semantics of ZstdDecompressor unchanged and handles concatenated frames at the zipfile streaming layer.

Tests

The regression test creates a valid Zstandard-compressed ZIP member containing two frames and covers:

  • Both frames being available in the same archive read.
  • A frame boundary occurring between two archive reads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant