Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/7770#issuecomment-127099556
It looks like this is currently failing a bunch of different tests due to a
NPE in TorrentBroadcast:
```scala
[info] - function current_date *** FAILED *** (20 milliseconds)
[info] org.apache.spark.SparkException: Job aborted due to stage failure:
Task serialization failed: java.lang.NullPointerException
[info]
org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:80)
[info]
org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
[info]
org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:63)
[info] org.apache.spark.SparkContext.broadcast(SparkContext.scala:1295)
[info]
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitMissingTasks(DAGScheduler.scala:844)
[info]
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:756)
[info]
org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:741)
[info]
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1436)
[info]
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1428)
[info]
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1417)
[info] org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
```
It looks like that line of TorrentBroadcast could be failing on
`SparkEnv.get()`, which is a little surprising given that `SparkEnv` is now
effectively a global rather than a thread-local.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
