Zheng Shao created HADOOP-14086: ----------------------------------- Summary: Improve DistCp Speed for small files Key: HADOOP-14086 URL: https://issues.apache.org/jira/browse/HADOOP-14086 Project: Hadoop Common Issue Type: Improvement Components: tools/distcp Affects Versions: 2.6.5 Reporter: Zheng Shao Assignee: Zheng Shao Priority: Minor
When using distcp to copy lots of small files, NameNode naturally becomes a bottleneck. The current distcp code did *not* optimize to reduce the NameNode calls. We should restructure the code to reduce the number of NameNode calls as much as possible to speed up the copy of small files. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org