ozankabak commented on PR #8802: URL: https://github.com/apache/arrow-datafusion/pull/8802#issuecomment-1883633767
@alamb, would the current approach work well in a setting with a small number of threads and large batch sizes? The scenario that came up today when we discussed this internally was running a deep query (i.e. with a lot of operators) in such a setting. We were wondering if the pull mechanism would get disrupted if serializations would take too long in such an environment. Maybe the answer is no (and we should measure to see what happens), but I'd love to hear your thought process on such a scenario. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
