Repository: spark Updated Branches: refs/heads/branch-2.0 6fb1f735f -> 578e40e3e
[SPARK-16963][SQL] Fix test "StreamExecution metadata garbage collection" ## What changes were proposed in this pull request? A follow up PR for #14553 to fix the flaky test. It's flaky because the file list API doesn't guarantee any order of the return list. ## How was this patch tested? Jenkins Author: Shixiong Zhu <[email protected]> Closes #15661 from zsxwing/fix-StreamingQuerySuite. (cherry picked from commit 79fd0cc0584e48fb021c4237877b15abbffb319a) Signed-off-by: Shixiong Zhu <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/578e40e3 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/578e40e3 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/578e40e3 Branch: refs/heads/branch-2.0 Commit: 578e40e3e3fda49a391a8373d0094e6131e97ca9 Parents: 6fb1f73 Author: Shixiong Zhu <[email protected]> Authored: Thu Oct 27 12:32:58 2016 -0700 Committer: Shixiong Zhu <[email protected]> Committed: Thu Oct 27 12:33:07 2016 -0700 ---------------------------------------------------------------------- .../scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/578e40e3/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala ---------------------------------------------------------------------- diff --git a/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala index dad4104..464c443 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala @@ -265,7 +265,7 @@ class StreamingQuerySuite extends StreamTest with BeforeAndAfter with Logging { AssertOnQuery("metadata log should contain only two files") { q => val metadataLogDir = new java.io.File(q.offsetLog.metadataPath.toString) val logFileNames = metadataLogDir.listFiles().toSeq.map(_.getName()) - val toTest = logFileNames.filter(! _.endsWith(".crc")) // Workaround for SPARK-17475 + val toTest = logFileNames.filter(! _.endsWith(".crc")).sorted // Workaround for SPARK-17475 assert(toTest.size == 2 && toTest.head == "1") true } --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
