alamb commented on PR #14286: URL: https://github.com/apache/datafusion/pull/14286#issuecomment-2613975012
Is there anyone willing to test out this approach in their application so we can get some confirmation that it avoids some of the timeouts reported while doing heavy processing of data from object store? Some people I think who have reported seeing this issue: - @djanderson I think you have seen issues - https://github.com/datafusion-contrib/datafusion-dft/pull/248#issuecomment-2491819394 - @rohitrastogi https://github.com/datafusion-contrib/datafusion-dft/pull/248#issuecomment-2519678264 - @ion-elgreco saw it on https://github.com/apache/arrow-rs/issues/5882 - @JanKaul mentioned he saw something similar at the Amsterdam Meetup I am willing to help make a draft PR to whatever repo to hook `DedicatedExecutor` up, if someone is willing to then test on a case that was seeing object store timeouts before Also FYI @ozankabak as we discussed this topic recently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
