cisaacson commented on issue #6967: URL: https://github.com/apache/arrow-rs/issues/6967#issuecomment-2587211454
Thanks @tustvold. I don't think this is the issue, the flight-sql server responds right away, then on the client side is where the timeout is triggered. I have now tracked it down to building the logical plan in DataFusion, part of our initial request. There must be something with the timeout setting for `tonic` that is causing the client to cancel the request. FYI the error message that DataFusion generates is very long (lists all columns for a data source, which can be a lot in our case). Could that be part of the issue? However, I do think we should have 2 tokio runtimes on the server side, one for our I/O tasks, the other for DataFusion. The doc is helpful, but how would we point DataFusion at the second runtime? In addition we are seeing all cores used for each query, it's overkill for what we do. How would we set the number of partitions to use? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
