HeartSaVioR opened a new pull request #35512: URL: https://github.com/apache/spark/pull/35512
### What changes were proposed in this pull request? This PR proposes to add the context of current challenge on fixing distribution of stateful operator, even the distribution is a sort of "broken" now. This PR addresses the review comment https://github.com/apache/spark/pull/35419#discussion_r801343068 ### Why are the changes needed? In SPARK-38124 we figured out the existing long-standing problem in stateful operator, but it is not easy to fix since the fix may break the existing query if the fix is not carefully designed. Anyone should also be pretty much careful when touching the required distribution. We want to document this explicitly to help others to be careful whenever someone is around the codebase. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Code comment only changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
