Lianhui Wang created SPARK-5759:
-----------------------------------
Summary: ExecutorRunnable should catch YarnException while
NMClient start container
Key: SPARK-5759
URL: https://issues.apache.org/jira/browse/SPARK-5759
Project: Spark
Issue Type: Bug
Components: YARN
Reporter: Lianhui Wang
some time since some of reasons, it lead to some exception while NMClient start
container.example:we do not config spark_shuffle on some machines, so it will
throw a exception:
java.lang.Error: org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException:
The auxService:spark_shuffle does not exist.
because YarnAllocator use ThreadPoolExecutor to start Container, so we can not
find which container or hostname throw exception. I think we should catch
YarnException in ExecutorRunnable when start container. if there are some
exceptions, we can know the container id or hostname of failed container.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
