hu6360567 opened a new issue #970: URL: https://github.com/apache/arrow-datafusion/issues/970
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** I'm implementing a distributed query system, similar with Ballista, but data sources are changing over time, located at MySQL/Postresql Servers. Query is parsed and optimized into LogicalPlan at scheduler, and distributed to different executors which have access to the sepcific data sources. The key problem is that not acceptable to load entire table from DBMS to memory and execute the plan, when the table is huge. execute LogicalPlan directly in the DBMS can reduce the size of in-memory data from the botton of all computation process. **Describe the solution you'd like** One possible solution is define a new DBExecuteContext which associate with a sql connection, that can convert LogicalPlan back to Plain SQL and execute it directly to the associated sql connection. `libpg_parser` provids the ablity convert from AST back to plain SQL query, but `sqlparser-rs` hasn't. **Describe alternatives you've considered** Not yet. **Additional context** Add any other context or screenshots about the feature request here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
