venkata91 commented on pull request #34122: URL: https://github.com/apache/spark/pull/34122#issuecomment-1026209657
> I was thinking more about this ... should we always mark DETERMINATE stages with setShuffleMergeAllowed(false) when they complete ? > It makes things more cleaner, and in line with expectation whether mergers were used for first execution or not. > > I dont feel very strongly about it though ... > Thoughts ? @mridulm This makes sense to me. Only thing is it can affect the tests as `shuffleMergeFinalized` also checks for `shuffleMergeEnabled` but that would return false and in turn return `true` always, but from your recent [comment](https://github.com/apache/spark/pull/34122#discussion_r796060753) that should not be the case anymore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
