timsaucer commented on issue #16833: URL: https://github.com/apache/datafusion/issues/16833#issuecomment-3196790329
I don't have much time to dive deeper, but `write_deltalake` is taking a `ArrowStreamExportable` so my suspicion is that we should be doing a normal multithreaded collect operation on the datafusion side and then sending it out, but I'd have to spend more time looking deeper. It's not super clear right now where you're seeing the single threading happening. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
