ahmed-mez commented on PR #18906: URL: https://github.com/apache/datafusion/pull/18906#issuecomment-3571486037
I augmented the reproducer test case with some stats to clarify the benefit: - In chunked, we dramatically reduced poll times (23ms in vs 2.88s previously) and provided many more yield points (i.e, opportunities for other async tasks to get scheduled by the runtime) - Total execution time is essentially the same (2.87s vs 2.88s), there is no performance overhead. Note: The first poll in chunked (2.21s) includes input processing (building the hash table). This is unavoidable and the same in both approaches. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
