GitHub user joseph-torres reopened a pull request: https://github.com/apache/spark/pull/19239
[SPARK-22017] Take minimum of all watermark execs in StreamExecution. ## What changes were proposed in this pull request? Take the minimum of all watermark exec nodes as the "real" watermark in StreamExecution, rather than picking one arbitrarily. ## How was this patch tested? new unit test You can merge this pull request into a Git repository by running: $ git pull https://github.com/joseph-torres/spark SPARK-22017 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/19239.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #19239 ---- commit 4a7f53fdab1e5e640e156a4a3d2ba27837195195 Author: Jose Torres <j...@databricks.com> Date: 2017-09-13T21:49:23Z Implement multiple watermark StreamExecution support. commit 9b9cd19106fae9a2de268eb2b559ca1bf159e9c2 Author: Jose Torres <j...@databricks.com> Date: 2017-09-14T18:30:40Z partially fix test commit 6a4c80b696f42a445c7f846fada3f823e04bd3ab Author: Jose Torres <j...@databricks.com> Date: 2017-09-14T21:52:16Z Finish rewriting test commit 484940e5eb4d1eac1c5ec81f475681c9241bbab2 Author: Jose Torres <j...@databricks.com> Date: 2017-09-14T22:24:36Z make IncrementalExecution.offsetSeqMetadata non-private commit 032f55503c8d424390da1ff85054e3a01e7489eb Author: Jose Torres <j...@databricks.com> Date: 2017-09-14T23:22:22Z properly name test dataframes commit d7f5f60c6be5bf228c960c3549eb81ed869f0227 Author: Jose Torres <j...@databricks.com> Date: 2017-09-14T23:39:22Z Combine test helper functions. commit 2f07f90423d87985322975f8ad5aef8f70f28066 Author: Jose Torres <j...@databricks.com> Date: 2017-09-15T01:18:12Z Key watermarks by relative position rather than attribute. commit 8b605384d77fdeb63b28feabee74284a5ab1409a Author: Jose Torres <j...@databricks.com> Date: 2017-09-15T02:05:14Z Address test comments. ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org