tustvold opened a new issue, #2445: URL: https://github.com/apache/arrow-datafusion/issues/2445
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** `LocalFileSystem` interprets the prefix passed to `ObjectStore::list_file` as the path to a directory, and then proceeds to enumerate this directory recursively. `S3FileSystem`, however, interprets the prefix as a string prefix. The distinction arises if you consider a file structure like ``` foo/a.txt foo/b.txt ``` If called with a prefix of `fo`, `LocalFileSystem` will return an error, whereas `S3FileSystem` will return both files. **Describe the solution you'd like** I personally would expect something called `ObjectStore` to behave like an object store, and not a filesystem. In particular I would expect it to behave like a KV store without any notion of directories. I would therefore suggest: * Remove FileSystem from the naming of the implementations * Map an object store to filesystem semantics, as opposed to mapping filesystem semantics to object storage **Describe alternatives you've considered** We could instead call the trait something like `FileSystem` and give is file system like semantics **Additional context** I noticed this whilst reviewing https://github.com/apache/arrow-datafusion/pull/2394 - it seems off to me that we should need to split based on path delimiters given object stores don't have such a concept. FYI @matthewmturner @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
