tustvold commented on issue #13692: URL: https://github.com/apache/datafusion/issues/13692#issuecomment-2527819635
The performance implications definitely concern me, I have a nagging suspicion block_in_place spawns a thread... An arguably better solution would be to instead spawn the CPU bound work to a threadpool like rayon, but this would require any arguments to be `'static`. However, this may not be as much of an issue for sync code as it is for async.... I'll update the ticket... Such a solution would have a number of compelling properties > I'm wondering if we should rather write our own async scheduler for the async compute graph that can better deal with the DF workload This sounds a lot like https://github.com/apache/datafusion/issues/2199, I am not entirely sure such an approach is feasible at this point. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
