Re: Copy Vs DistCP

2013-04-14 Thread Mathias Herberts
and will not copy files locally. -- 麦树荣 *From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs DistCP I have few questions regarding the usage of DistCP for copying files in the same cluster. 1

Re: Copy Vs DistCP

2013-04-14 Thread Ted Dunning
between clusters, and cp command is a hadoop internal parallel process and will not copy files locally. -- 麦树荣 *From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs DistCP I have few questions

Re: Copy Vs DistCP

2013-04-14 Thread Mathias Herberts
This is absolutely true. Distcp dominates cp for large copies. On the other hand cp dominates distcp for convenience. In my own experience, I love cp when copying relatively small amounts of data (10's of GB) where the available bandwidth of about a GB/s allows the copy to complete in less

Re: Copy Vs DistCP

2013-04-14 Thread Ted Dunning
On Sun, Apr 14, 2013 at 10:33 AM, Mathias Herberts mathias.herbe...@gmail.com wrote: This is absolutely true. Distcp dominates cp for large copies. On the other hand cp dominates distcp for convenience. In my own experience, I love cp when copying relatively small amounts of data

Re: Copy Vs DistCP

2013-04-13 Thread Ted Dunning
. -- 麦树荣 *From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs DistCP I have few questions regarding the usage of DistCP for copying files in the same cluster. 1) Which one is better within a same cluster and what factors (like

Re: Copy Vs DistCP

2013-04-12 Thread Lance Norskog
:2013-04-11%C2%A006:20 *To:* user@hadoop.apache.org mailto:user@hadoop.apache.org *Subject:* Copy Vs DistCP I have few questions regarding the usage of DistCP for copying files in the same cluster

Re: Copy Vs DistCP

2013-04-11 Thread Hemanth Yamijala
树荣 *From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs DistCP I have few questions regarding the usage of DistCP for copying files in the same cluster. 1) Which one is better within a same cluster and what factors (like

Re: Copy Vs DistCP

2013-04-11 Thread Azuryy Yu
command is a hadoop internal parallel process and will not copy files locally. -- 麦树荣 *From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs DistCP I have few questions regarding the usage

Re: Copy Vs DistCP

2013-04-11 Thread KayVajj
cluster while using distCP between clusters, and cp command is a hadoop internal parallel process and will not copy files locally. -- 麦树荣 *From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs DistCP

Re: Copy Vs DistCP

2013-04-11 Thread Azuryy Yu
*From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs DistCP I have few questions regarding the usage of DistCP for copying files in the same cluster. 1) Which one is better within a same cluster and what factors (like file

Copy Vs DistCP

2013-04-10 Thread KayVajj
I have few questions regarding the usage of DistCP for copying files in the same cluster. 1) Which one is better within a same cluster and what factors (like file size etc) wouldinfluence the usage of one over te other? 2) when we run a cp command like below from a client node of the cluster

Re: Copy Vs DistCP

2013-04-10 Thread 麦树荣
@hadoop.apache.orgmailto:user@hadoop.apache.org Subject: Copy Vs DistCP I have few questions regarding the usage of DistCP for copying files in the same cluster. 1) Which one is better within a same cluster and what factors (like file size etc) wouldinfluence the usage of one over te other? 2) when

Re: Copy Vs DistCP

2013-04-10 Thread Jay Vyas
DistCP is a full blown mapreduce job (mapper only, where the mappers do a fully parallel copy to the detsination). CP appears (correct me if im wrong) to simply invoke the FileSystem and issues a copy command for every source file. I have an additional question: how is CP which is internal to a

Re: Copy Vs DistCP

2013-04-10 Thread KayVajj
internal parallel process and will not copy files locally. -- 麦树荣 *From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs DistCP I have few questions regarding the usage of DistCP for copying files

Re: Copy Vs DistCP

2013-04-10 Thread KayVajj
in the same cluster while using distCP between clusters, and cp command is a hadoop internal parallel process and will not copy files locally. -- 麦树荣 *From:* KayVajj vajjalak...@gmail.com *Date:* 2013-04-11 06:20 *To:* user@hadoop.apache.org *Subject:* Copy Vs