Repository: spark Updated Branches: refs/heads/branch-2.2 1388fdd70 -> 421d8ecb8
[SPARK-20957][SS][TESTS] Fix o.a.s.sql.streaming.StreamingQueryManagerSuite listing ## What changes were proposed in this pull request? When stopping StreamingQuery, StreamExecution will set `streamDeathCause` then notify StreamingQueryManager to remove this query. So it's possible that when `q2.exception.isDefined` returns `true`, StreamingQueryManager's active list still has `q2`. This PR just puts the checks into `eventually` to fix the flaky test. ## How was this patch tested? Jenkins Author: Shixiong Zhu <[email protected]> Closes #18180 from zsxwing/SPARK-20957. (cherry picked from commit bc537e40ade0658aae7c6b5ddafb4cc038bdae2b) Signed-off-by: Tathagata Das <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/421d8ecb Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/421d8ecb Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/421d8ecb Branch: refs/heads/branch-2.2 Commit: 421d8ecb8a2ec1327b50932e6016d5f837b88da9 Parents: 1388fdd Author: Shixiong Zhu <[email protected]> Authored: Mon Jun 5 14:34:10 2017 -0700 Committer: Tathagata Das <[email protected]> Committed: Mon Jun 5 14:37:59 2017 -0700 ---------------------------------------------------------------------- .../apache/spark/sql/streaming/StreamingQueryManagerSuite.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/421d8ecb/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala ---------------------------------------------------------------------- diff --git a/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala index b49efa6..2986b7f 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala @@ -78,9 +78,9 @@ class StreamingQueryManagerSuite extends StreamTest with BeforeAndAfter { eventually(Timeout(streamingTimeout)) { require(!q2.isActive) require(q2.exception.isDefined) + assert(spark.streams.get(q2.id) === null) + assert(spark.streams.active.toSet === Set(q3)) } - assert(spark.streams.get(q2.id) === null) - assert(spark.streams.active.toSet === Set(q3)) } } --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
