smaheshwar-pltr commented on code in PR #3730:
URL: https://github.com/apache/iceberg-rust/pull/3730#discussion_r4643857679


##########
crates/iceberg/src/writer/file_writer/location_generator.rs:
##########
@@ -203,6 +207,52 @@ impl LocationGenerator for ObjectStorageLocationGenerator {
     }
 }
 
+/// Generates data file paths using the layout configured in table properties.
+///
+/// It uses [`ObjectStorageLocationGenerator`] when
+/// 
[`write.object-storage.enabled`](crate::spec::TableProperties::PROPERTY_WRITE_OBJECT_STORAGE_ENABLED)
+/// is `true`, and [`DefaultLocationGenerator`] otherwise.
+#[derive(Clone, Debug)]
+pub struct TableLocationGenerator {

Review Comment:
   Thank you - on custom providers, I think they'd come in through the catalog 
builder like [storage 
factories](https://github.com/apache/iceberg-rust/blob/219078c9d8d82cdc431af77a430e07c8fe56bd65/crates/iceberg/src/catalog/mod.rs#L157)
 with the catalog handing those to each `Table`, not `TableMetadata`. So I've 
moved the entry point to `Table::location_generator()`.
   
   To check, I POC-ed what custom providers might look like 
[here](https://github.com/smaheshwar-pltr/iceberg-rust/compare/location-generator/1-table-accessor...location-generator/2-custom-providers)
 on top of this API choice i.e. `Table::location_generator()` doesn't change 
meaning e.g. DataFusion picks it up with no changes. (The wiring is 
`LocationGeneratorFactory` passed in through 
`CatalogBuilder::with_location_generator_factory`)
   
   Separately, I do wonder why location generators aren't trait objects like 
[`Storage`](https://github.com/apache/iceberg-rust/blob/219078c9d8d82cdc431af77a430e07c8fe56bd65/crates/iceberg/src/io/storage/mod.rs#L72)
 - i.e. if we were building this from scratch, we could pass `Arc<dyn 
LocationGenerator>` around instead with this PR's `Table::location_generator()` 
returning that too. https://github.com/apache/iceberg-rust/pull/3738 is what 
that might look like 
([this](https://github.com/smaheshwar-pltr/iceberg-rust/compare/location-generator/dyn-trait-poc...location-generator/dyn-trait-poc-custom-providers)
 is with custom providers on top).
   
   That's breaking so I've kept this PR additive for now, but I am curious for 
thoughts on that alternative design.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to