pepijnve commented on PR #16196: URL: https://github.com/apache/datafusion/pull/16196#issuecomment-2925152088
> This is exactly what I want to avoid -- there are too many execs and we shouldn't have a structure where this is repeated everywhere in code. Isn't that somewhat unavoidable when you're dealing with a cooperative scheduler? There's no way for tokio to preempt. Only those streams that call `poll_next` themselves in a loop, and as a consequence may block for an extended period of time, would need to do this. Are there that many of those? The solution that moves the responsibility to all sources may seem like the lesser evil because there might be fewer of those, but to me that design still feels wrong because rather than yielding where it's obvious that you should, your trying to trigger a yield at a distance in a non-obvious way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
