mridulm commented on issue #54723: URL: https://github.com/apache/spark/issues/54723#issuecomment-4597264068
Driver invalidates the cached map status, and increments the epoch, when the 'previous' status is no longer valid. While there is a case to be made for this when parent stage is `DETERMINATE`, we will need to think through whether any other corner cases apply in this context. I would expect this to be fairly infrequent - and typically the right thing to do. Want to understand better why this is surfacing for you typically @EnricoMi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
