Tao Jie created HADOOP-15887:
--------------------------------
Summary: Add an option to avoid writing data locally in Distcp
Key: HADOOP-15887
URL: https://issues.apache.org/jira/browse/HADOOP-15887
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 3.0.0, 2.8.2
Reporter: Tao Jie
Assignee: Tao Jie
When copying large amount of data from one cluster to another via Distcp, and
the Distcp jobs run in the target cluster, the datanode local usage would be
imbalanced. Because the default placement policy chooses the local node to
store the first replication.
In https://issues.apache.org/jira/browse/HDFS-3702 we add a flag in DFSClient
to avoid replicating to the local datanode. We can make use of this flag in
Distcp.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
