westonpace commented on issue #14854: URL: https://github.com/apache/datafusion/issues/14854#issuecomment-2695595567
> much of it behind ListingTable and all related interfaces. Its also available everywhere you have a Session or access to a RuntimeEnv. Ah, this makes sense. `ListingTable` and friends would definitely need an object store. > Its also available everywhere you have a Session or access to a RuntimeEnv Hmm, this one is a little more troubling in my mind. What should this object store be? Why should it be needed? Or, to focus on the purposes of this discussion, what APIs will it need? > The only notable exception I am aware of is ParquetExec, which can be constructed with a custom AsyncFileReader that dispatches IO to an abstraction of its choosing. I would expect most file readers to have their own I/O abstractions. Lance has it's own, for example, which requires the equivalent of something like `preadv`. I think attempting to unify an object store trait across the I/O needs of all possible file readers could be troublesome. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
