peter-toth commented on PR #13177: URL: https://github.com/apache/datafusion/pull/13177#issuecomment-2453107732
Are you refering to visiting / transforming a node's children in parallel on multiple threads? Unfortunately, I'm not sure either if this idea was discussed earlier. I remember I was thinking about it when we consolidated the tree traversal semantics and formed the current APIs. But as far as I can recall I didn't find that many `TreeNode` API usecases that would have benefited from parallel processing and if they did then would have required significant changes. But I might be wrong and others have more insights on existing rules / transformations in DataFusion. (I'm more familiar with Spark honestly, and there are no parallel tree processing APIs there.) Anyways, if we wanted to offer such APIs I would not change the current ones, but add completely new ones. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
