HeartSaVioR commented on issue #26108: [SPARK-26154][SS] Streaming left/right outer join should not return outer nulls for already matched rows URL: https://github.com/apache/spark/pull/26108#issuecomment-577432819 > As written in the migration guide, the situation is the same in 3.0.0 with old stored data, isn't it? End users might not be mad if "major" version requires them to lose something. They still have Spark 2.x version line to deny the change and take the risk. If we put this to 2.4.5, they have no way to deny. > Also, although we backport this with stateFormatVersion=1, it works as a legacy buggy mode, doesn't it? I guess not. Spark will "fail" the query when Spark detects that the query is using state format version 1 and stream-stream outer join is being used. That's what I said "destructive".
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
