berkaysynnada commented on PR #16322: URL: https://github.com/apache/datafusion/pull/16322#issuecomment-2965238217
Sorry for the late reply. @pepijnve Your diagnosis is spot on, and the proposed fix totally makes sense. I honestly can’t recall why I added the wake there but not in CongestedStream, most likely I just overlooked it. It's a really impactful gain, thank you. One small note regarding this PR: if the pending rotation somehow breaks, since SortPreservingMergeStream never yields, tests relying on tokio::timeout (like test_spm_congestion) may fail to trigger the timeout signal. That said, I don’t think it’s a big deal to address right now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
