xixipi-lining opened a new issue, #1155:
URL: https://github.com/apache/iceberg-go/issues/1155
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
When appending data to a partitioned table with the table property
write.object-storage.enabled set to true , the object storage layout is not
applied. Instead of generating data file paths with entropy hash prefixes
(e.g., <hash>-<filename>.parquet ), the writer falls back to the default
Hadoop-style layout, writing directly to the partition directory (e.g.,
<table_location>/data/<partition_path>/<filename>.parquet ).
This issue does not affect unpartitioned tables.
#### To Reproduce
1. Create a partitioned Iceberg table.
2. Set the table property write.object-storage.enabled to true .
3. Append data to the table using the iceberg-go SDK (e.g., via
Transaction.Append ).
4. Observe the final data file paths in the storage. The files are written
without the expected hash prefixes.
#### Root Cause Analysis
The bug is caused by property state loss in table/rolling_data_writer.go .
When writerFactory dynamically generates a LocationProvider for a
specific partition, it drops all the original table properties.
In func (w *writerFactory) partitionLocProvider(partitionPath string) :
partitionDataPath := w.rootURL.JoinPath("data", partitionPath).String()
// BUG: A new empty map is passed, losing all original table properties
// including 'write.object-storage.enabled'.
loc, err := LoadLocationProvider(w.rootLocation,
iceberg.Properties{WriteDataPathKey: partitionDataPath})
Because LoadLocationProvider receives a fresh properties map containing
only WriteDataPathKey , it evaluates ObjectStoreEnabledKey as false and
incorrectly falls back to simpleLocationProvider .
#### Expected behavior
The writerFactory should preserve and pass the original table properties
(or at least the relevant object-storage properties) down to
LoadLocationProvider when creating partition location providers, so that
objectStoreLocationProvider is correctly instantiated and hashes are added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
