The distcp command is working slower in hadoop 0.20.2678219V3 when compared to
hadoop-0.18.3-2486615
----------------------------------------------------------------------------------------------------
Key: HADOOP-5693
URL: https://issues.apache.org/jira/browse/HADOOP-5693
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.20.0
Environment: cluster is brought up with hod in both 0.18.3 and 0.20
versions
Reporter: Iyappan Srinivasan
1st scenario :
Source cluster (3 slaves) is hadoop-0.18.3-2486615 and destination cluster is
hadoop-0.20.2678219V3 (3 slaves).
a) The source has 3 files in a directory equaling 0.4 GB
b) distcp is used to copy these contents to a destination directory.
(bin/hadoop --config ~/cluster_dir distcp -update -delete
hftp://<IPFQDN>:<port>//dir1/file1 hdfs://<IPFQDN:PORT>//copied/)
c) It takes 36 seconds
2nd scenario :
Source cluster is hadoop-0.20.2678219V3. and destination cluster (3 slaves) is
hadoop-0.18.3-2486615.
a) The source has 3 files in a directory equaling 0.4 GB
b) distcp is used to copy these contents to a destination directory.
c) It takes 20 seconds
>From this we can conclude, In the latest 0.20.2678219V3, distcp has
>considerabley slowed down ( around 90%).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
