wgtmac opened a new pull request, #937: URL: https://github.com/apache/iceberg-cpp/pull/937
Extract shared manifest/residual evaluator caching and entry-filtering rules (ignore_existing_, file filter, custom predicate) into a single FilePlanningStream::Evaluators helper, and collapse the previously separate sequential vs. executor-backed state machines in FilePlanningStream into one FilePlanningStream::EntryStream, with batch size 1 without an executor and kManifestReadBatchSize with one. ReadEntries()/Plan()/Entries() are rebuilt on top of the same EntryStream instead of duplicating manifest iteration and filtering via a separate ParallelCollect-based loop, removing the FIXME about porting Iceberg Java's ParallelIterable design. Also: - Dedupe data manifests (matching Java's ManifestGroup, which stores them in a Set) so a manifest referenced by overlapping snapshots is only scanned once. - Update FilterManifestEntries() doc comment: the predicate is now always invoked sequentially, even when an executor is configured. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
