Thejdeep Gudivada created SPARK-36784:
-----------------------------------------
Summary: Handle DNS failures in Executor without causing
FetchFailed
Key: SPARK-36784
URL: https://issues.apache.org/jira/browse/SPARK-36784
Project: Spark
Issue Type: Bug
Components: Shuffle
Affects Versions: 2.3.0
Reporter: Thejdeep Gudivada
DNS failures caused on the executor nodes causing shuffle nodes to be added to
the exclude list. We should handle this failure and not cause a FetchFailed
exception in such cases.
This helps in two things :
1. Stage won't be resubmitted due to FetchFailedException
2. Spark's exclude listing also won't exclude the shuffle service node when the
problem is
indeed with the current executor host.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
