Github user squito commented on the issue:
https://github.com/apache/spark/pull/17297
@sitalkedia I took a closer look -- I think this is from
"o.a.s.InternalAccumulatorSuite: 'internal accumulators in resubmitted
stages'". From the console output on jenkins, that was the last test run, and
also I can log into the box and see the log4j output from the tests in
'core/target/unit-test.log', and it shows that is the last test and it ends
with:
```
17/03/24 13:44:19.537 dag-scheduler-event-loop ERROR
DAGSchedulerEventProcessLoop: DAGSchedulerEventProcessLoop failed; shutting
down SparkContext
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.spark.MapOutputTrackerMaster$$anonfun$getEpochForMapOutput$1.apply(MapOutputTracker.scala:431)
at
org.apache.spark.MapOutputTrackerMaster$$anonfun$getEpochForMapOutput$1.apply(MapOutputTracker.scala:430)
at scala.Option.flatMap(Option.scala:171)
at
org.apache.spark.MapOutputTrackerMaster.getEpochForMapOutput(MapOutputTracker.scala:430)
at
org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:1298)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1731)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1689)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1678)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/03/24 13:44:19.540 dispatcher-event-loop-11 INFO
MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/03/24 13:44:19.546 stop-spark-context INFO MemoryStore: MemoryStore
cleared
17/03/24 13:44:19.546 stop-spark-context INFO BlockManager: BlockManager
stopped
17/03/24 13:44:19.546 stop-spark-context INFO BlockManagerMaster:
BlockManagerMaster stopped
17/03/24 13:44:19.546 dispatcher-event-loop-16 INFO
OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:
OutputCommitCoordinator stopped!
17/03/24 13:44:19.547 stop-spark-context INFO SparkContext: Successfully
stopped SparkContext
17/03/24 14:02:19.934 metrics-console-reporter-1-thread-1 ERROR
ScheduledReporter: RuntimeException thrown from ConsoleReporter#report.
Exception was suppressed.
java.lang.NullPointerException
at
org.apache.spark.deploy.master.MasterSource$$anon$2.getValue(MasterSource.scala:35)
at
org.apache.spark.deploy.master.MasterSource$$anon$2.getValue(MasterSource.scala:34)
at
com.codahale.metrics.ConsoleReporter.printGauge(ConsoleReporter.java:239)
...
```
with those NPE's repeatedly thrown every 15 minutes.
Normally that 'unit-tests.log' file is available as an archived artifact
from jenkins, but I guess it doesn't show up if the test times out, so I don't
think there is anyway for you to get at this info yourself. Hopefully you can
reproduce and debug locally?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
