timsaucer opened a new issue, #15907: URL: https://github.com/apache/datafusion/issues/15907
### Is your feature request related to a problem or challenge? Currently when you add the `datafusion` crate, it pulls in many dependencies that are not needed for all use cases. We have two specific projects in mind: - In comet the work is done at the level of the Physical plan. It would be convenient to not have to pull in the sql parsing or the logical plan. - One customer has a use case where they will not be doing any sql parsing, and do not need any sql support. Specifically, they are trying to build in web assembly and the full imports are causing large bloat of the generated binaries. The purpose of this issue is to discuss other use cases and where we may these flags in. ### Describe the solution you'd like Add a few feature flags so the dependency graph is greatly reduced. From early discussions in the datafusion community meeting these might be `sql` and `logical_plan` but you might imagine others. ### Describe alternatives you've considered _No response_ ### Additional context During the community meeting, @mbutrovich suggested that databases typically have these general steps we can break down into - Parsing SQL - Building logical plans - Optimizing logical plans - Building physical plans - Optimizing physical plans - Execution I hope I captured correctly, but he suggested we could create two feature flags, one for the parsing SQL stage and one for the building / optimizing logical plans. Since these stages are done in order, if someone opted in to `sql` they would likely need everything that follows. These are the notes I tried to capture from the meeting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
