Dem1241 opened a new pull request, #2368: URL: https://github.com/apache/iceberg-go/pull/2368
fixes #2367 partitioned writes built a separate location provider per partition with its data path set to `<table>/data/<partition>`, so the object storage hash always ended up under the partition directory. `write.object-storage.partitioned-paths=false` had no effect (`data/category=x/<hash>-file.parquet`), and with it on the layout was `data/<partition>/<hash>/file` where java writes `data/<hash>/<partition>/file`. the same override also meant a custom `write.data.path` was ignored for partitioned writes. the writer now passes `<partition>/<file>` to the table's own provider, and `objectStoreLocationProvider.NewDataLocation` follows java's `ObjectStoreLocationProvider`: hash right under the data path, partition dirs kept only when partitioned paths are on. `TestPartitionedDataLocations` covers simple and object storage layouts, with and without partition paths and with a custom `write.data.path`. it fails without the change and passes with it. `TestLocationProviderLiteralCustomPaths` now expects partitioned names to be hashed under object storage. `go test ./...`, `-race` on the table package and golangci-lint pass. i didn't run the docker integration suite. heads up that this moves where partitioned files land with the default `partitioned-paths=true` (hash before the partition dir now, matching java). existing files are tracked by full path so reads aren't affected, but happy to split that part out if you'd rather keep the current layout. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
