Su Qilong created SPARK-33669:
---------------------------------
Summary: Wrong error message from YARN application state monitor
when sc.stop in yarn client mode
Key: SPARK-33669
URL: https://issues.apache.org/jira/browse/SPARK-33669
Project: Spark
Issue Type: Bug
Components: YARN
Affects Versions: 3.0.1, 2.4.3
Reporter: Su Qilong
For YarnClient mode, when stopping YarnClientSchedulerBackend, it first tries
to interrupt Yarn application monitor thread. In MonitorThread.run() it catches
InterruptedException to gracefully response to stopping request.
But client.monitorApplication method also throws InterruptedIOException when
the hadoop rpc call is calling. In this case, MonitorThread will not know it is
interrupted, a Yarn App failed is returned with "Failed to contact YARN for
application xxxxx; YARN application has exited unexpectedly with state xxxxx"
is logged with error level. which confuse user a lot.
We Should take considerate InterruptedIOException here to make it the same
behavior with InterruptedException.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
