rdettai edited a comment on pull request #950: URL: https://github.com/apache/arrow-datafusion/pull/950#issuecomment-907820496
@yjshen @houqp you are right, I think that maybe with the delimiter argument added to `ObjectStore.list()`, the TableProvider would be able to get all the information it needs to efficiently prune the partitions in a raw hive partitioning setup! This way we leave the ObjectStore abstraction with only core object store capabilities 😄 I am still wondering how we could replicate the hive partitioning support for `TableProvider`s relative to different file formats (`ParquetTable`, `CsvFile`, `NdJsonFile`). Actually, same would apply for a HiveCatalogTable that could actually reference various file formats. It makes me feel that something is wrong with how the different abstractions are organized. This is being discussed in #133. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
