fallintoplace opened a new pull request, #1914:
URL: https://github.com/apache/iceberg-go/pull/1914
## What changed
- **`Partitions()` now uses the borrowed partition map for aggregate-tree
lookups.**
- **The coerced map and positional record are built only when a partition is
first seen.**
- **Binary values are copied before they are retained by an aggregate.**
- Added coverage for binary, NaN, and binary ownership behavior.
- Added a before/after benchmark for repeated partitions.
## Why
`InspectTable.Partitions` was materializing several partition
representations for every manifest entry. Most files share an existing
partition, so that work was repeated and then discarded.
The new path keeps the existing partition evolution behavior. It also keeps
the public `DataFile.Partition()` fallback for custom data file implementations.
## Benchmark
- **Command:** `go test ./table -run=^\\$
-bench=BenchmarkInspectPartitionAggregation -benchmem -benchtime=200ms -count=5`
- **Machine:** Apple M1 Pro, darwin/arm64
- **Scope:** aggregation loop only. Manifest decoding and Arrow output are
outside the timed section.
- **Median of five samples:**
- **10,000 files / 100 partitions / 1 int32 field:** `2.01 ms, 2.78 MB,
30.8k allocs` -> `0.395 ms, 59.0 KB, 911 allocs` (**80% lower time, 98% fewer
bytes**)
- **100,000 files / 100 partitions / 8 int32 fields:** `53.5 ms, 38.7 MB,
304k allocs` -> `25.8 ms, 295 KB, 4.11k allocs` (**52% lower time, 99% fewer
bytes**)
- **100,000 files / 100 partitions / 32 binary fields:** `610 ms, 732 MB,
13.6M allocs` -> `182 ms, 2.89 MB, 33.8k allocs` (**70% lower time, 99.6% fewer
bytes**)
## Tests
- `go test ./... -count=1`
- `go vet ./table`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
