maropu commented on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL URL: https://github.com/apache/spark/pull/20303#issuecomment-458414803 Thanks for the great work, @carsonwang! I left some comments below (I'm reading the proposal and the code now...): 1. Since the behaviour of this new execution model is totally different from the existing one, could we add this new impl. while keeping the existing impl.? For example, the old one is named `AdaptiveExecutionV1` and the other named `AdaptiveExecutionV2`. 2. Any benchmark result of this new impl.? The new one always overcomes the existing one? (If so, it might not be worth keeping the old implementation...) 3. I feel the `QueryStage` name is a bit confusing because Spark already uses the `Stage` terminology in RDDs. So, it's a bit unclear that just a stage means which one. Any chance to rename to better one? e.g., `QueryFragment` or something.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
