ClientFinalizer shutdown hook in FileSystem caused problems

[Jian Fang]

Hi,

We have some customized code to flush data during hadoop shutdown process,
but we run into the following errors with Hadoop 2.0.3-alpha

Exception in thread "Thread-3" java.lang.ExceptionInInitializerError
        at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2323)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2295)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:319)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:165)
        ...our custom code to check file system to flush
Caused by: java.lang.IllegalStateException: Shutdown in progress
        at
java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:39)
        at java.lang.Runtime.addShutdownHook(Runtime.java:192)
        at
org.apache.hadoop.util.ShutdownHookManager.<clinit>(ShutdownHookManager.java:47)

Seems the Cache class in the FileSystem.java has the following logic


         // now insert the new file system into the map
        if (map.isEmpty() ) {
          ShutdownHookManager.get().addShutdownHook(clientFinalizer,
SHUTDOWN_HOOK_PRIORITY);
        }

What happened is that the above code was called in the shutdown process,
thus, led to java.lang.IllegalStateException.

I wonder why the shutdown hook was implemented in this way, why it was not
added when the Cache class was initialized? How should I fix this problem
then?

Thanks,

Jian