[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424919#comment-13424919
]
nkeywal commented on HDFS-3705:
-------------------------------
Attached a possible interface on branch 2.
- allow the client to plug in an algo to reoder the location
- I tried to use the Replica interface (initial Todd proposition), it's not
directly feasible because we need the exact location and it's not available in
this interface. May be it is with trunk
- One of the function (MD5MD5CRC32FileChecksum) is available as a static
function, so we don't have a Configuration object. As such we can't plug our
interface.
- We need to be plugged before the 'open' because the open can connects to the
different datanodes if the file is opened for writing somewhere else. So we
need to reorder the locations before these calls take place.
> Add the possibility to mark a node as 'low priority' for read in the DFSClient
> ------------------------------------------------------------------------------
>
> Key: HDFS-3705
> URL: https://issues.apache.org/jira/browse/HDFS-3705
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs client
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: nkeywal
> Attachments: hdfs-3705.sample.patch
>
>
> This has been partly discussed in HBASE-6435.
> The DFSClient includes a 'bad nodes' management for reads and writes.
> Sometimes, the client application already know that some deads are dead or
> likely to be dead.
> An example is the 'HBase Write-Ahead-Log': when HBase reads this file, it
> knows that the HBase regionserver died, and it's very likely that the box
> died so the datanode on the same box is dead as well. This is actually
> critical, because:
> - it's the hbase recovery that reads these log files
> - if we read them it means that we lost a box, so we have 1 dead replica out
> the the 3.
> - for all files read, we have 33% of chance to go to the dead datanode
> - as the box just died, we're very likely to get a timeout exception so we're
> delaying the hbase recovery by 1 minute. For HBase, it means that the data is
> not available during this minute.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
