pgwhalen opened a new issue, #8637: URL: https://github.com/apache/arrow-datafusion/issues/8637
### Is your feature request related to a problem or challenge? After I upgraded my application from 28.0.0 to 34.0.0, I noticed that as a result of https://github.com/apache/arrow-datafusion/pull/7390, it is no longer possible to register your own listing table with a custom file format, because the code that supports writing assumes a fixed list of [FileTypes](https://github.com/apache/arrow-datafusion/blob/03c2ef46f2d88fb015ee305ab67df6d930b780e2/datafusion/common/src/file_options/file_type.rs#L45). Specifically, the when implementing [FileFormat](https://github.com/apache/arrow-datafusion/blob/03c2ef46f2d88fb015ee305ab67df6d930b780e2/datafusion/core/src/datasource/file_format/mod.rs#L109), the enum must be provided. ### Describe the solution you'd like Ideally `FileFormat` would not have to return a `FileType`, and the ability to write to listing tables would be: - optional - not coupled to an enumerated list of file types As it stands currently, it looks like this method is only used to help out code that supports writing, so perhaps it can be refactored to be more coupled with that functionality. I don't have any specific proposal at the moment, but it seems achievable (if it's something maintainers would be okay with). ### Describe alternatives you've considered Several alternatives seem to work for me: - Give up on `ListingTable` as an extensible API and just use `TableProvider` instead. This would be disappointing, because it does provide a lot of nice functionality that would have to reimplemented. - Provide a garbage `FileType` in `FileFormat::file_type` because it doesn't seem to cause problems unless you're trying to write. This is what I'm doing for now. ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
