[
https://issues.apache.org/jira/browse/HBASE-19954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368420#comment-16368420
]
Ted Yu edited comment on HBASE-19954 at 2/18/18 1:54 AM:
---------------------------------------------------------
Did some debugging by installing hadoop-common of hadoop3 with additional
logging into local maven repo.
{code}
2018-02-17 16:14:14,573 INFO [Time-limited test]
util.ShutdownHookManager(286): clearing hooks
2018-02-17 16:14:14,588 INFO [Time-limited test]
hbase.HBaseTestingUtility(1114): Minicluster is down
2018-02-17 16:14:14,627 INFO [Time-limited test] hbase.ResourceChecker(172):
after: fs.TestBlockReorder#testBlockLocationReorder Thread=110 (was 8)
{code}
Note the above was the first test in TestBlockReorder where the {{hooks}} Set
of hadoop ShutdownHookManager was cleared (first line).
The 'Failed suppression' exception happened in the second subtest where the
FileSystem$Cache$ClientFinalizer instance was no longer in the Set.
I dumped the contents of the {{hooks}} Set at time of the exception and saw
fsdataset.impl.BlockPoolSlice instances but no ClientFinalizer instance.
After poking around hadoop ShutdownHookManager, I don't see bug.
was (Author: yuzhih...@gmail.com):
Did some debugging by installing hadoop-common with additional logging into
local maven repo.
{code}
2018-02-17 16:14:14,573 INFO [Time-limited test]
util.ShutdownHookManager(286): clearing hooks
2018-02-17 16:14:14,588 INFO [Time-limited test]
hbase.HBaseTestingUtility(1114): Minicluster is down
2018-02-17 16:14:14,627 INFO [Time-limited test] hbase.ResourceChecker(172):
after: fs.TestBlockReorder#testBlockLocationReorder Thread=110 (was 8)
{code}
Note the above was the first test in TestBlockReorder where the {{hooks}} Set
of hadoop ShutdownHookManager was cleared (first line).
The 'Failed suppression' exception happened in the second subtest where the
FileSystem$Cache$ClientFinalizer instance was no longer in the Set.
I dumped the contents of the {{hooks}} Set at time of the exception and saw
fsdataset.impl.BlockPoolSlice instances but no ClientFinalizer instance.
After poking around hadoop ShutdownHookManager, I don't see bug.
> ShutdownHook should check whether shutdown hook is tracked by
> ShutdownHookManager
> ---------------------------------------------------------------------------------
>
> Key: HBASE-19954
> URL: https://issues.apache.org/jira/browse/HBASE-19954
> Project: HBase
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
> Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19954.v1.txt
>
>
> Currently ShutdownHook#suppressHdfsShutdownHook() does the following:
> {code}
> synchronized (fsShutdownHooks) {
> boolean isFSCacheDisabled =
> fs.getConf().getBoolean("fs.hdfs.impl.disable.cache", false);
> if (!isFSCacheDisabled &&
> !fsShutdownHooks.containsKey(hdfsClientFinalizer)
> && !ShutdownHookManager.deleteShutdownHook(hdfsClientFinalizer)) {
> {code}
> There is no check that ShutdownHookManager still tracks the shutdown hook,
> leading to potential RuntimeException (as can be observed in hadoop3 Jenkins
> job).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
