zhuqi-lucas commented on PR #15380: URL: https://github.com/apache/datafusion/pull/15380#issuecomment-2812031488
Thank you @Dandandan @2010YOUY01 for review. I am trying to clean up code and do more testing, it seems somethings the result is unstable for testing about balance the sort in place size and the streaming merge count. The result will diff when running in different env and computer if we use a fixed sort-place-size. And i am thinking, if we can have a thread pod for sort and merge in future, and the thread number is partition number, and make sort and merge all async, and each sort/merge task can take from the thread pool and make sure it always running for the best performance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
