Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/11692#issuecomment-196901240
So you are saying that if spark context in yarn client mode is cleanly
exiting while the RM is switching to the standby node, the call to
getApplicationReport in Yarn can internally retry and sleep, since sc.stop()
was called it ends up calling the scheduler backend stop interrupting the
monitoring thread.
The MonitorThread has a catch for interruptedException and should be
printing an info message " Interrupting monitor thread", you are seeing this?
Interrupted Exception is not a NonFatal error so it shouldn't be catching
it:
From scaladoc:
Extractor of non-fatal Throwables. Will not match fatal errors like
VirtualMachineError (for example, OutOfMemoryError and StackOverflowError,
subclasses of VirtualMachineError), ThreadDeath, LinkageError,
InterruptedException, ControlThrowable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
