tychoish commented on issue #8345: URL: https://github.com/apache/arrow-datafusion/issues/8345#issuecomment-1836858807
I guess my first question is "are `FileType` and `FileFormat` actually meaningfully distinct?" Do we imagine that there are cases where you'd want to have different `FileType` values but the same `FileFormat`? Seem's unlikely to me but I don't know. Second very high level question: is there anything to be done in `ListingTable` separetly from either "`FileType` as a trait" or flattening `FileType` into `FileFormat`. Secondly, (and this is somewhat orthogonal) but for the project that brought me to this point; I'm leaning towards just to implementing `StreamingTable` (rather than `FileFormat` or a TableProvier from the ground up, something approximating it?) For things like JSON and CSV this seems maybe more ideal? Though of course, I'm new and don't know all of the innards. Obviously this is very format specific. Would love to hear additional -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
