Imran Rashid created SPARK-23962: ------------------------------------ Summary: Flaky tests from SQLMetricsTestUtils.currentExecutionIds Key: SPARK-23962 URL: https://issues.apache.org/jira/browse/SPARK-23962 Project: Spark Issue Type: Improvement Components: SQL, Tests Affects Versions: 2.4.0 Reporter: Imran Rashid Attachments: unit-tests.log
I've seen {{org.apache.spark.sql.execution.metric.SQLMetricsSuite.SortMergeJoin metrics}} fail https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4150/testReport/org.apache.spark.sql.execution.metric/SQLMetricsSuite/SortMergeJoin_metrics/ with {noformat} Error Message org.scalatest.exceptions.TestFailedException: 2 did not equal 1 Stacktrace sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException: 2 did not equal 1 at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:528) at org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1560) at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:501) at org.apache.spark.sql.execution.metric.SQLMetricsTestUtils$class.getSparkPlanMetrics(SQLMetricsTestUtils.scala:146) at org.apache.spark.sql.execution.metric.SQLMetricsSuite.getSparkPlanMetrics(SQLMetricsSuite.scala:33) at org.apache.spark.sql.execution.metric.SQLMetricsTestUtils$class.testSparkPlanMetrics(SQLMetricsTestUtils.scala:187) at org.apache.spark.sql.execution.metric.SQLMetricsSuite.testSparkPlanMetrics(SQLMetricsSuite.scala:33) at org.apache.spark.sql.execution.metric.SQLMetricsSuite$$anonfun$7$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SQLMetricsSuite.scala:188) at org.apache.spark.sql.test.SQLTestUtilsBase$class.withTempView(SQLTestUtils.scala:260) ... {noformat} I believe this is because {{SQLMetricsTestUtils.currentExecutionId()}} is racing with the listener bus. I'll attach trimmed logs as well -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org