regadas opened a new pull request, #18997:
URL: https://github.com/apache/iceberg/pull/18997

   Closes #18996
   
   External callers streaming file changes from newly committed manifests with 
column projection cannot filter rewritten manifests by entry status because 
`ManifestReader.iterator()` returns `liveEntries()` (`ADDED` and `EXISTING`) 
while `entries()` and `ManifestEntry` are package-private. Expose 
`addedFiles()` and `deletedFiles()` on `ManifestReader` returning 
`CloseableIterable<F>` with defensive container copies respecting `select(...)` 
stats dropping while keeping `ManifestEntry` internal, and switch 
`CherryPickOperation.validateReplacedPartitions` to use `addedFiles()`. Also 
include `column_sizes` in `ManifestReader`'s droppable stats set so 
`select("column_sizes")` retains `columnSizes()`, and return `filePath` 
directly in `BaseFile.location()` so omitting `file_path` from `select(...)` 
returns `null` instead of throwing a `NullPointerException`.
   
   Test plan:
   - Verify `addedFiles()` and `deletedFiles()` filter mixed-status data and 
delete manifests across format versions 1–4, detach reused containers across 
`next()`, and preserve inherited metadata and projected fields under 
`select(...)`.
   - Verify `CloseableIterable.concat` over `addedFiles()` opens at most one 
stream concurrently, skips remaining manifests on early break, and closes 
streams cleanly.
   - Verify `select("file_path", "column_sizes")` retains `columnSizes()` while 
dropping unselected stats maps.
   
   ---
   **AI Disclosure**
   - Model: Gemini 4 Carbon (`gemini-4-carbon`)
   - Platform/Tool: Pi coding agent
   - Human Oversight: [unknown - human to fill in]
   - Prompt Summary: Expose streaming, projection-aware status accessors 
(`addedFiles` / `deletedFiles`) on `ManifestReader`, update 
`CherryPickOperation` to use `addedFiles()`, fix `column_sizes` selection and 
`BaseFile.location()` null handling, and add unit tests (code and tests in 
`ManifestReader`, `CherryPickOperation`, `BaseFile`, `TestManifestReader`, and 
`TestManifestReaderStats` generated with AI assistance; problem framing and API 
scope directed by the author).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to