[
https://issues.apache.org/jira/browse/HADOOP-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847500#action_12847500
]
Ted Yu commented on HADOOP-4829:
--------------------------------
I saw the following after applying this patch:
Exception in thread "Processor Shutdown Hook" java.lang.IllegalStateException:
Shutdown in progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:1408)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1440)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1417)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:202)
Here is the modified Cache.remove() method where IllegalStateException is
handled:
synchronized void remove(Key key, FileSystem fs) {
if (map.containsKey(key) && fs == map.get(key)) {
map.remove(key);
toAutoClose.remove(key);
if (map.isEmpty() && !clientFinalizer.isAlive()) {
try
{
if
(!Runtime.getRuntime().removeShutdownHook(clientFinalizer)) {
LOG.info("Could not cancel cleanup
thread, though no " +
"FileSystems are open");
}
}
// ignore IllegalStateException because Shutdown in
progress
catch (java.lang.IllegalStateException ise)
{
}
}
}
}
> Allow FileSystem shutdown hook to be disabled
> ---------------------------------------------
>
> Key: HADOOP-4829
> URL: https://issues.apache.org/jira/browse/HADOOP-4829
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Affects Versions: 0.18.1
> Reporter: Bryan Duxbury
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-4829-0.18.3.patch, hadoop-4829-v2.txt,
> hadoop-4829-v3.txt, hadoop-4829.txt
>
>
> FileSystem sets a JVM shutdown hook so that it can clean up the FileSystem
> cache. This is great behavior when you are writing a client application, but
> when you're writing a server application, like the Collector or an HBase
> RegionServer, you need to control the shutdown of the application and HDFS
> much more closely. If you set your own shutdown hook, there's no guarantee
> that your hook will run before the HDFS one, preventing you from taking some
> shutdown actions.
> The current workaround I've used is to snag the FileSystem shutdown hook via
> Java reflection, disable it, and then run it on my own schedule. I'd really
> appreciate not having to do take this hacky approach. It seems like the right
> way to go about this is to just to add a method to disable the hook directly
> on FileSystem. That way, server applications can elect to disable the
> automatic cleanup and just call FileSystem.closeAll themselves when the time
> is right.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
