alamb commented on issue #14444: URL: https://github.com/apache/datafusion/issues/14444#issuecomment-2640733024
> [@alamb](https://github.com/alamb) Can you point me at whatever tool you used to generate those compile timing graphs? Those look like something I'd absolutely adopt in a bunch of projects. (and to be clear, that was @waynexia who made that chart for https://github.com/apache/datafusion/issues/14256-- I just copy pasted it here) > I'm still ramping up on learning DataFusion internals as I add my own extensions and one thing that's been nagging at me is that it almost feels like the built-in data source providers are "cheating" because they get to live in core. Moving them all to separate crates that have to use the same interfaces as external datasources is something I've been contemplating suggesting for awhile so its great to see this already happening. Awesome -- indeed this is the case BTW there is a major refactor for datasources done by @mertak-synnada @ozankabak and others that (just) merged: - https://github.com/apache/datafusion/pull/14224 Among other things it makes it easier to re-use common datasource features like pushdown, limit, etc Now that that is in, we will be able to push this project along like a 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
