[
https://issues.apache.org/jira/browse/HDFS-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087409#comment-17087409
]
Yang Yun edited comment on HDFS-15278 at 4/20/20, 9:35 AM:
-----------------------------------------------------------
Thanks [~liuml07] for the review.
Yes, our use case can be satisfied if enable the
CreateFlag::IGNORE_CLIENT_LOCALITY, Thanks! The pain is that all user have to
change their code with this flag.
Currently, this feature is triggered only by setReplication to 1. With history
chosen information, I think that it's more easy to implement the creating 1
replica file.
Update to patch HDFS-15278.003.patch, define the sequentialBlocksDispersed as
final.
was (Author: hadoop_yangyun):
Thanks [~liuml07] for the review.
Yes, our use case can be satisfied if enable the
CreateFlag::IGNORE_CLIENT_LOCALITY, Thanks! The pain is that all user have to
change their code with this flag.
Currently, this feature is triggered only by setReplication to 1. With history
chosen information, I think the it's more easy to implement the creating 1
replica file.
Update to patch HDFS-15278.003.patch, define the sequentialBlocksDispersed as
final.
> After execute ‘-setrep 1’, make sure that blocks of the file are dispersed
> across different datanodes
> -----------------------------------------------------------------------------------------------------
>
> Key: HDFS-15278
> URL: https://issues.apache.org/jira/browse/HDFS-15278
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15278.001.patch, HDFS-15278.002.patch,
> HDFS-15278.003.patch
>
>
> After execute ‘-setrep 1’, many of blocks of the file may locate on same
> machine. Especially the file is written on one datanode machine. That causes
> data hot spots and is hard to fix if this machine is down.
> Add a chosen history to make sure that blocks of the file are dispersed
> across different datanodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
