[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252608#comment-13252608
]
Ravi Prakash commented on MAPREDUCE-4135:
-----------------------------------------
This is a particularly vexing problem. :( I can think of two ways out of this
1. Swallow IllegalStateExceptions thrown by removeShutdownHook in FileSystem.
The javadoc states:
{quote}public boolean removeShutdownHook(Thread hook)
Throws:
IllegalStateException - If the virtual machine is already in the process of
shutting down {quote}
So if we are getting this exception, it MEANS we are already in the process of
shutdown, so we CANNOT, try what we may, removeShutdownHook. If Runtime had a
method Runtime.isShutdownInProgress(), we could have checked for it before the
removeShutdownHook call. As it stands, there is no such method. In my opinion,
this would be a good patch regardless of the needs for this JIRA.
2. Not send SIGTERMs from the NM to the MR-AM in the first place. Rather we
should expose a mechanism for the NM to politely tell the AM its no longer
needed and should shutdown asap. Even after this, if an admin were to kill the
MRAppMaster with a SIGTERM, the JobHistory would be lost defeating the purpose
of 3614
> MRAppMaster throws IllegalStateException while shutting down
> ------------------------------------------------------------
>
> Key: MAPREDUCE-4135
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Attachments: MAPREDUCE-4135.patch, MAPREDUCE-4135.patch
>
>
> Always MRAppMaster throws IllegalStateException in the stderr while shutting
> down. It doesn't look good having this exception in the stderr file of
> MRAppMaster container.
> {code:xml}
> WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
> org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
> Exception in thread "Thread-1" java.lang.IllegalStateException: Shutdown in
> progress
> at
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
> at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
> at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
> at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
> at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
> at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
