HeartSaVioR edited a comment on issue #23757: [SPARK-23408][SS][BRANCH-2.3] Synchronize successive AddData actions in Streaming*JoinSuite URL: https://github.com/apache/spark/pull/23757#issuecomment-462472158 > This is a straight back-port of https://github.com/apache/spark/pull/20650/files?w=1 , and indeed it looks like the same change we see here with https://github.com/apache/spark/pull/23757/files?w=1 , OK. It can be merged first without other changes, pending tests? Yes (and you can also resolve SPARK-24211). And then SPARK-23491 and SPARK-23416 can be cherry-picked from origin PRs since they're clean cherry-pick and we already ran tests in this PR before revising.(and you can also resolve SPARK-24239) > Maybe someone can help me understand this. The diff at > https://github.com/apache/spark/pull/23757/files?w=1 > looks pretty different from > https://github.com/apache/spark/pull/23757/files It is just same as what #20650 was. You could check the difference between https://github.com/apache/spark/pull/23757/files?w=1 vs https://github.com/apache/spark/pull/23757/files The code change add indentation(s) to huge code block: `w=1` option ignores differences of indentation(s) whereas no option takes them all diff. You could find the difference from searching `logInfo(s"Processing test stream action: $action")` from PR with and without option. BTW, after fixing this I'm seeing very high change of failures from FileBasedDataSourceSuite - as I commented earlier it might need ORC version upgrade to fix some of related issues, so while we seemed to upgrade the ORC dependency from bugfix version (2.4.0 -> 2.4.1) I'm not sure we could do in 2.3 version line. cc. @dongjoon-hyun Could you help explaining how we dealt with FileBasedDataSourceSuite and relevant test failures? I'm not sure I understand all of changes.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
