> On 10 Apr 2015, at 13:40, Lorenz Knies <m...@l1024.org> wrote: > > i would consider it a bug, that the "Yarn application state monitor” thread > dies on an, i think even expected (at least in the java methods called > further down the stack), exception. > > What do you think? Is it a problem, that we compiled against hadoop 2.5?
Code path still exists in Hadoop 2.6, so no. Looks more like YarnClientSchedulerBackend.asyncMonitorApplication should be catching the IOE (RM failure triggered) and retrying. why not file a JIRA on it, in SPARK That said, loss of RM is pretty dramatic in a Hadoop cluster, because unless you have RM HA enabled, the restarted RM will have no idea what is running, the node managers will kill the processes spawned in the containers, etc etc. Your app was probably doomed anyway.
