tgravescs commented on pull request #32195: URL: https://github.com/apache/spark/pull/32195#issuecomment-823625374
so I was looking at this today and I think its fairly easy fix to just to change it to store the plan after its been through all the rules. `val queryStage = context.stageCache.getOrElseUpdate(newStage.plan.canonicalized, newStage) ` instead of the `val queryStage = context.stageCache.getOrElseUpdate(e.canonicalized, newStage)` The only downside I can think of is if there is something that is matching on the original plan but after running the rules makes it not canonically the same but really is, it would take a bit longer to have to tranverse down the plan and apply all the rules. But that kind of seems like a bug to me anyway. If anyone sees an issue with that let me know, otherwise either @andygrove can update this or I can put up a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
