fallintoplace opened a new pull request, #2543: URL: https://github.com/apache/iceberg-go/pull/2543
**What** - Read only filter columns when generating position deletes. **Why** - Unrelated payload columns add read and decode work. **Implementation** - Project predicate field IDs and fix nested Parquet struct pruning. - Reuse the reader when predicate fields are missing. - Prefer retained physical fields to keep row counts without reading dropped payloads. **Benchmark** - Complete local `Delete`: 8,192 rows and about 64 MiB of uncompressed payload. Memory catalog. Includes output files, manifests and commit. - Median of 5 runs, 10 deletes each, `-cpu=4`, Apple M1 Pro. Apache main `f515561a` to this change: - 29.62 to 3.59 ms/op (8.25x faster); 349.04 to 9.67 MB/op (97.2% less). - Reproduce: `go test ./table -run '^$' -bench '^BenchmarkMergeOnReadDeleteProjection$' -benchmem -benchtime=10x -count=5 -cpu=4`. **References** - [Spec: column projection](https://github.com/apache/iceberg/blob/83bf0ffab77bb45468348fa3f1e9f0e5801e4f62/format/spec.md#column-projection) - [Java: Spark scan projection](https://github.com/apache/iceberg/blob/83bf0ffab77bb45468348fa3f1e9f0e5801e4f62/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java#L329-L347) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
