fallintoplace opened a new pull request, #1766: URL: https://github.com/apache/iceberg-go/pull/1766
## What changed\n\n- Reuse the input record when a partition contains the full batch.\n- Use Arrow slices when the selected rows are one contiguous range.\n- Check the complete row range and bounds before taking the fast path.\n- Keep the existing path for scattered rows.\n\n## Why\n\n always built an Int64 index array and ran . This copied every selected column even when the result was the full batch or one contiguous range.\n\n## Benchmark\n\nApple M1 Pro, 65,536 rows, median of 3 runs:\n\n\n\n| Case | main | this PR |\n| --- | ---: | ---: |\n| 1 column, full | 649,907 ns/op, 1,610,469 B/op, 47 allocs/op | 49,908 ns/op, 0 B/op, 0 allocs/op |\n| 1 column, contiguous half | 319,600 ns/op, 815,066 B/op, 47 allocs/op | 18,402 ns/op, 288 B/op, 4 allocs/op |\n| 8 columns, full | 2,360,602 ns/op, 5,351,519 B/op, 236 allocs/op | 35,814 ns/op, 0 B/op, 0 allocs/op |\n| 8 columns, contiguous half | 1,217,218 ns/op, 2,721,096 B/op, 236 allocs/op | 18,282 ns/op, 1,760 B/op, 19 allocs/op |\n \nScattered selections still use . Their allocation counts are unchanged and runtime stayed within normal benchmark variance.\n\n## Tests\n\n- ok github.com/apache/iceberg-go (cached) ok github.com/apache/iceberg-go/catalog (cached) ? github.com/apache/iceberg-go/catalog/catalogtest [no test files] ok github.com/apache/iceberg-go/catalog/glue (cached) ok github.com/apache/iceberg-go/catalog/hadoop (cached) ok github.com/apache/iceberg-go/catalog/hive (cached) ? github.com/apache/iceberg-go/catalog/internal [no test files] ok github.com/apache/iceberg-go/catalog/rest (cached) ok github.com/apache/iceberg-go/catalog/rest/internal/planfake (cached) ok github.com/apache/iceberg-go/catalog/sql (cached) ok github.com/apache/iceberg-go/cmd/iceberg (cached) ok github.com/apache/iceberg-go/codec (cached) ok github.com/apache/iceberg-go/config (cached) ok github.com/apache/iceberg-go/encryption (cached) ok github.com/apache/iceberg-go/internal (cached) ? github.com/apache/iceberg-go/internal/awsconfig [no test files] ? github.com/apache/iceberg-go/internal/datafileavro [no test files] ? github.com/apache/iceberg-go/internal/recipe [no test files] ok github.com/apache/iceberg-go/io (cached) ok github.com/apache/iceberg-go/io/gocloud (cached) ok github.com/apache/iceberg-go/metrics (cached) ok github.com/apache/iceberg-go/puffin (cached) ok github.com/apache/iceberg-go/table (cached) ok github.com/apache/iceberg-go/table/compaction (cached) ok github.com/apache/iceberg-go/table/dv (cached) ok github.com/apache/iceberg-go/table/internal (cached) ok github.com/apache/iceberg-go/table/substrait (cached) ok github.com/apache/iceberg-go/udf (cached) ? github.com/apache/iceberg-go/utils [no test files] ok github.com/apache/iceberg-go/view (cached) ok github.com/apache/iceberg-go/view/internal (cached) ? github.com/apache/iceberg-go/website/gen [no test files]\n- ok github.com/apache/iceberg-go/table 6.905s\n- \n- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
