[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450862#comment-13450862
]
Suresh Srinivas commented on HDFS-3705:
---------------------------------------
Thanks for helping with testing HDFS-3703. Your help is much appreciated.
I may not have understood parts of your previous comment.
bq. If I take HBase & HDFS as they are today, it's more a less a yes: Most of
the time, people configure HBase with a timeout of 45s. So if HDFS does 30s, it
has the right state when HBase starts the recovery. So done and HDFS-3705 is
useless. However, even today, I've seen people claiming a configuration with a
10 seconds timeout.
I understood the first part. As long as HDFS has marked nodes stale before
HBase starts recovery, change from HDFS-3703 is sufficient. I did not
understand the second part related to 10 seconds. Are you saying, people may
want more aggressive recovery time for HBase than say 30s that HDFS may be
configured with?
My takeaway from your comments is, even though HDFS supports this
functionality, HBase might need even more aggressive timeout. This timeout may
not be suitable for other applications using HDFS. Hence having client side
capability that HBase can use is still needed. Is that correct?
> Add the possibility to mark a node as 'low priority' for read in the DFSClient
> ------------------------------------------------------------------------------
>
> Key: HDFS-3705
> URL: https://issues.apache.org/jira/browse/HDFS-3705
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs client
> Affects Versions: 1.0.3, 2.0.0-alpha, 3.0.0
> Reporter: nkeywal
> Fix For: 3.0.0
>
> Attachments: hdfs-3705.sample.patch, HDFS-3705.v1.patch
>
>
> This has been partly discussed in HBASE-6435.
> The DFSClient includes a 'bad nodes' management for reads and writes.
> Sometimes, the client application already know that some deads are dead or
> likely to be dead.
> An example is the 'HBase Write-Ahead-Log': when HBase reads this file, it
> knows that the HBase regionserver died, and it's very likely that the box
> died so the datanode on the same box is dead as well. This is actually
> critical, because:
> - it's the hbase recovery that reads these log files
> - if we read them it means that we lost a box, so we have 1 dead replica out
> the the 3.
> - for all files read, we have 33% of chance to go to the dead datanode
> - as the box just died, we're very likely to get a timeout exception so we're
> delaying the hbase recovery by 1 minute. For HBase, it means that the data is
> not available during this minute.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
