[
https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118616#comment-14118616
]
Thomas Graves commented on SPARK-3351:
--------------------------------------
Note that this failure scenario also can cause the staging directory to be
cleaned up but yarn will retry the application master attempt and fail since
its been deleted.
> Yarn YarnRMClientImpl.shutdown can be called before register - NPE
> ------------------------------------------------------------------
>
> Key: SPARK-3351
> URL: https://issues.apache.org/jira/browse/SPARK-3351
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.2.0
> Reporter: Thomas Graves
>
> If the SparkContext exits while its in the
> applicationmaster.waitForSparkContextInitialized then the
> YarnRMClientImpl.shutdown can be called before register and you get a null
> pointer exception on the uihistoryAddress.
> 14/09/02 18:59:21 INFO ApplicationMaster: Finishing ApplicationMaster with
> FAILED (diag message: Timed out waiting for SparkContext.)
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.proto.YarnServiceProtos$FinishApplicationMasterRequestProto$Builder.setTrackingUrl(YarnServiceProtos.java:2312)
> at
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.FinishApplicationMasterRequestPBImpl.setTrackingUrl(FinishApplicationMasterRequestPBImpl.java:121)
> at
> org.apache.spark.deploy.yarn.YarnRMClientImpl.shutdown(YarnRMClientImpl.scala:73)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster.finish(ApplicationMaster.scala:140)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:178)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:113)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
