[
https://issues.apache.org/jira/browse/HDFS-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-4274:
--------------------------------
Attachment: HDFS-4274.2.patch
Here is the new version of the patch:
# During {{DataNode}} shutdown, change {{DataBlockScanner#run}} to shut down
every allocated {{BlockPoolSliceScanner}}, which in turn closes the
verification logs. This part is the same as the earlier patch.
# {{DataBlockScanner#shutdown}} also joins to the background
blockScannerThread. There is a slightly tricky bit in here about avoiding a
deadlock. I explained it in comments. This is necessary to make shutdown
synchronous, so that the caller knows deterministically that shutdown has
completed. In the case of {{MiniDFSCluster}}, it means that it is safe to
delete the working directories, because nothing will be holding locks.
# Change {{BlockPoolSliceScanner#scan}} to use {{Thread#isInterrupted}} instead
of {{Thread#interrupted}}. {{BlockPoolSliceScanner#scan}} runs within the
blockScannerThread, which depends on seeing the interrupted status of the
thread to know when to exit. (See the while loop condition in
{{DataBlockScanner#run}}.) When {{BlockPoolSliceScanner#scan}} was calling
{{Thread#interrupted}}, it would clear the current thread's interrupted status,
so that the while loop in {{DataBlockScanner#run}} would never see it, and
therefore would never exit. This was occasionally causing tests to enter an
infinite loop on Windows.
With this patch in place, I am consistently seeing good test results on Mac and
Windows, with no tests blocked on a prior {{MiniDFSCluster}} holding a lock on
a verification log.
> BlockPoolSliceScanner does not close verification log during shutdown
> ---------------------------------------------------------------------
>
> Key: HDFS-4274
> URL: https://issues.apache.org/jira/browse/HDFS-4274
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 3.0.0, trunk-win
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HDFS-4274.1.patch, HDFS-4274.2.patch
>
>
> {{BlockPoolSliceScanner}} holds open a handle to a verification log. This
> file is not getting closed during process shutdown.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
