crepererum opened a new pull request, #10009: URL: https://github.com/apache/arrow-datafusion/pull/10009
## Which issue does this PR close? \- ## Rationale for this change The state is initialized ONCE for all partitions. However this may take a short while (on a very busy system 1ms or more). It is quite likely that multiple threads call `execute` at the same time, because we have just fanned out to the number "target partitions" which is likely set to the number of CPU cores which now all try to start to execute the plan at the same time. The solution is to not waste CPU circles in some futex lock but to tell the async runtime (= tokio) that we are performing work and the other threads should rather do something useful. ## What changes are included in this PR? This mostly just moves code around, no functional change intended. ## Are these changes tested? Existing tests still pass. ## Are there any user-facing changes? Faster query exec. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
