[
https://issues.apache.org/jira/browse/HADOOP-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HADOOP-5890:
--------------------------------
Attachment: hadoop-5890.txt
Attached patch starts off sleeping 2ms and then exponentially decays by a
factor of 1.5x each time. Feel free to twiddle the ratio before committing if
you so desire.
No additional tests since this is trivial and in the code path for every MiniMR
test.
> Use exponential backoff on Thread.sleep during DN shutdown
> ----------------------------------------------------------
>
> Key: HADOOP-5890
> URL: https://issues.apache.org/jira/browse/HADOOP-5890
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-5890.txt
>
>
> Tests waste a lot of time in DataNode.shutdown. Typical logs look like:
> {code}
> 2009-05-21 17:13:20,177 INFO datanode.DataNode (DataNode.java:shutdown(637))
> - Waiting for threadgroup to exit, active threads is 0
> 2009-05-21 17:13:20,177 INFO datanode.DataBlockScanner
> (DataBlockScanner.java:run(620)) - Exiting DataBlockScanner thread.
> 2009-05-21 17:13:21,117 INFO datanode.DataNode (DataNode.java:shutdown(637))
> - Waiting for threadgroup to exit, active threads is 0
> {code}
> In this example (and very commonly) the DataBlockScanner thread exits within
> 5-10ms after the first wait. The DN then sleeps an entire second before
> succeeding in shutting down.
> Using exponential backoff from a short value like 2ms up to a maximum of
> 1000ms would solve this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
