crepererum opened a new pull request, #6138: URL: https://github.com/apache/arrow-datafusion/pull/6138
# Which issue does this PR close? \- # Rationale for this change Plan nodes w/ multiple children concurrently. This is helpful if planning requires IO (e.g. file or network access), you have different tables, and you do not want to wait for DataFusion to plan one after the other. The main goal is to reduce planning time for `UNION` and `JOIN` queries. # What changes are included in this PR? New config variable `physical_planning_fanout` (we can bikeshed the naming) and a some stream magic to make planning concurrent. # Are these changes tested? Existing tests pass. # Are there any user-facing changes? Faster planning. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
