[
https://issues.apache.org/jira/browse/SPARK-27032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-27032.
----------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 23937
[https://github.com/apache/spark/pull/23937]
> Flaky test:
> org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.HDFSMetadataLog:
> metadata directory collision
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-27032
> URL: https://issues.apache.org/jira/browse/SPARK-27032
> Project: Spark
> Issue Type: Test
> Components: Spark Core, Tests
> Affects Versions: 3.0.0
> Reporter: Sean Owen
> Assignee: Sean Owen
> Priority: Critical
> Fix For: 3.0.0
>
>
> Locally and on Jenkins, I've frequently seen this test fail:
> {code}
> Error Message
> The await method on Waiter timed out.
> Stacktrace
> org.scalatest.exceptions.TestFailedException: The await method on
> Waiter timed out.
> at org.scalatest.concurrent.Waiters$Waiter.awaitImpl(Waiters.scala:406)
> at org.scalatest.concurrent.Waiters$Waiter.await(Waiters.scala:540)
> at
> org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.$anonfun$new$19(HDFSMetadataLogSuite.scala:158)
> at
> org.apache.spark.sql.execution.streaming.HDFSMetadataLogSuite.$anonfun$new$19$adapted(HDFSMetadataLogSuite.scala:133)
> ...
> {code}
> See for example
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-2.7/6057/testReport/
> There aren't obvious errors or problems with the test. Because it passes
> sometimes, my guess is that the timeout is simply too short or the test too
> long. I'd like to try reducing the number of threads/batches in the test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
