ion-elgreco commented on issue #5882: URL: https://github.com/apache/arrow-rs/issues/5882#issuecomment-2291503520
> > I definitely think it is a massive footgun that DataFusion's async APIs lure people into performing IO on the same runtime as CPU-bound tasks. My ideal outcome would be that you could just use DataFusion from a regular tokio threadpool, and it would spawn any potentially CPU-bound work off to its internally managed threadpool, but that is likely a non-trivial undertaking. > > Another way to help people avoid said footgun is with documentation / examples. Maybe we can make an example in datafusion showing how to use a second thread pool. Would be easy to miss though, if it can be part of the API it makes it easier to not fall in this trap -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
