[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460910#comment-13460910
]
Jing Zhao commented on HDFS-3912:
---------------------------------
Nicalos:
So based on your prior comments, we rethink the strategy that dynamically
changes the stale interval for writing. One problem for this strategy is that
after a datanode is marked as stale, since the stale interval may increase as a
result of the increase of the number of the stale datanodes, the same datanode
may be marked as healthy (i.e., non-stale) at once.
In the current solution, we try to provide a simpler solution. The stale
interval now is a fixed value after loading from the configuration. For read,
the strategy is the same with HDFS-3703. And for write, we add a switch flag
(only for write) so that when certain proportion of datanodes are marked as
stale, the stale datanodes can also be included as writing targets. Users can
specify this proportion through configuration. For example, if the proportion
is set to 0.5 in the beginning, when more than half of the datanodes have been
marked as stale in the cluster, we stop avoiding stale nodes for writing. And
when some of the datanodes come back, we continue avoiding stale nodes for
writing.
> Detecting and avoiding stale datanodes for writing
> --------------------------------------------------
>
> Key: HDFS-3912
> URL: https://issues.apache.org/jira/browse/HDFS-3912
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
>
> 1. Make stale timeout adaptive to the number of nodes marked stale in the
> cluster.
> 2. Consider having a separate configuration for write skipping the stale
> nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
