yjshen commented on pull request #950: URL: https://github.com/apache/arrow-datafusion/pull/950#issuecomment-907836854
Thanks @rdettai @houqp for the optional delimiter. I was thinking `list` API could deal with multiple partition columns by listing one directory at a time for filesystems (instead of listing leaf files recursively for non-partitioned tables), therefore could achieve a lazy listing when implementing ObjectStore. After taking both partitioned and non-partitioned tables in one remote storage into consideration, this original `list` signature would lead to two `ObjectStore` implementations for one storage system, one for partitioned and one for non-partitioned. It's wired. I like the idea of an optional delimiter because it brings a lot of flexibility. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
