vrozov opened a new pull request, #53053:
URL: https://github.com/apache/spark/pull/53053
### What changes were proposed in this pull request?
1. Update SBT scalatest settings to use named file reporter (same as maven
scalatest plugin)
2. Update SBT scalatest settings to use the same directories for junit-style
xml files as maven scalatest plugin
3. Use without color (W), show all durations (D), show full stack traces
(F), show reminder of failed and canceled tests without stack traces (I) for
the named file scalatest reporter both in SBT and maven.
### Why are the changes needed?
Unify scalatest settings between SBT and maven. Produce `SparkTestSuite.txt`
in the following format:
```
Run starting. Expected test count is: 15
ExecutorSuite:
- SPARK-15963: Catch `TaskKilledException` correctly in Executor.TaskRunner
(1 second, 768 milliseconds)
- SPARK-19276: Handle FetchFailedExceptions that are hidden by user
exceptions (938 milliseconds)
- Executor's worker threads should be UninterruptibleThread (380
milliseconds)
- SPARK-19276: OOMs correctly handled with a FetchFailure (91 milliseconds)
- SPARK-23816: interrupts are not masked by a FetchFailure (78 milliseconds)
- Gracefully handle error in task deserialization (15 milliseconds)
- Heartbeat should drop zero accumulator updates (248 milliseconds)
- Heartbeat should not drop zero accumulator updates when the conf is
disabled (68 milliseconds)
- SPARK-39696: Using accumulators should not cause heartbeat to fail (1
second, 58 milliseconds)
- Send task executor metrics in DirectTaskResult (114 milliseconds)
- Send task executor metrics in TaskKilled (108 milliseconds)
- Send task executor metrics in ExceptionFailure (93 milliseconds)
- SPARK-34949: do not re-register BlockManager when executor is shutting
down (67 milliseconds)
- SPARK-33587: isFatalError *** FAILED *** (2 milliseconds)
false equaled false (ExecutorSuite.scala:508)
org.scalatest.exceptions.TestFailedException:
at
org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at
org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at
org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231)
at
org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:1295)
at
org.apache.spark.executor.ExecutorSuite.testThrowable$1(ExecutorSuite.scala:508)
at
org.apache.spark.executor.ExecutorSuite.$anonfun$new$38(ExecutorSuite.scala:528)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:256)
at
org.apache.spark.executor.ExecutorSuite.$anonfun$new$33(ExecutorSuite.scala:527)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:68)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:154)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at
org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:226)
at
org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at
org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at
org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at
org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at
org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:68)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:68)
at
org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:323)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at
org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at
org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at
org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at
org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at
org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:68)
at
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:68)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:47)
at
org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1321)
at
org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1315)
at scala.collection.immutable.List.foreach(List.scala:323)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1315)
at
org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:992)
at
org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:970)
at
org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1481)
at
org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:970)
at org.scalatest.tools.Runner$.main(Runner.scala:775)
at org.scalatest.tools.Runner.main(Runner.scala)
- SPARK-40235: updateDependencies is interruptible when waiting on lock (18
milliseconds)
Run completed in 6 seconds, 86 milliseconds.
Total number of tests run: 15
Suites: completed 1, aborted 0
Tests: succeeded 14, failed 1, canceled 0, ignored 0, pending 0
*** 1 TEST FAILED ***
ExecutorSuite:
- SPARK-33587: isFatalError *** FAILED *** (2 milliseconds)
false equaled false (ExecutorSuite.scala:508)
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Run test from maven and sbt
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
