fallintoplace opened a new pull request, #1940: URL: https://github.com/apache/iceberg-go/pull/1940
## What changed - add `WithRewriteManifestClusterBy` for opt-in clustering - group live data entries by **cluster key + partition spec** - roll each key at the configured manifest target size - keep the default size-only path unchanged - clean open output manifests when a callback or writer fails ## Why Clustering can keep files with the same query-relevant key together. This can reduce unrelated manifests during scan planning. ## Benchmark Command: `go test ./table -run '^$' -bench '^BenchmarkManifestMergeModes$' -benchmem -benchtime=1s -count=5`\n\nApple M1 Pro, 64 one-entry input manifests, 8 output groups:\n\n- size-only: **16.85–16.94 ms/op**\n- cluster-by: **17.10–17.26 ms/op**\n- both: about **192.9k allocs/op**\n\nThe small rewrite-time cost is expected. The goal of this option is better manifest locality for reads.\n\n## Tests\n\n- `go test ./table/...`\n- `go vet ./table/...`\n- `go test -race ./table -run 'TestRewriteManifests(ClusterBy|CleansOrphansOnInvalidClusterKey)' -count=1`\n -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
