Yes. I need two different copy. And I tried Chris's solution, distcp indeed works. Thank you all
On Sun, Jan 10, 2016 at 3:00 PM, Chris Nauroth <[email protected]> wrote: > Yes, certainly, if you only need it in one spot, then -mv is a fast > metadata-only operation. I was under the impression that Gavin really > wanted to achieve 2 distinct copies. Perhaps I was mistaken. > > --Chris Nauroth > > From: sandeep vura <[email protected]> > Date: Sunday, January 10, 2016 at 6:23 AM > To: Chris Nauroth <[email protected]> > Cc: Gavin Yue <[email protected]>, "[email protected]" < > [email protected]> > Subject: Re: how to quickly fs -cp dir with thousand files? > > Hi Chris, > > Instead of copying files . Use mv command . > > > - hadoop fs -mv /user/hadoop/file1 /user/hadoop/file2 > > > Sandeep.v > > > On Sat, Jan 9, 2016 at 9:55 AM, Chris Nauroth <[email protected]> > wrote: > >> DistCp is capable of running large copies like this in distributed >> fashion, implemented as a MapReduce job. >> >> http://hadoop.apache.org/docs/r2.7.1/hadoop-distcp/DistCp.html >> >> A lot of the literature on DistCp talks about use cases for copying >> across different clusters, but it's also completely legitimate to run >> DistCp within the same cluster. >> >> --Chris Nauroth >> >> From: Gavin Yue <[email protected]> >> Date: Friday, January 8, 2016 at 4:45 PM >> To: "[email protected]" <[email protected]> >> Subject: how to quickly fs -cp dir with thousand files? >> >> I want to cp a dir with over 8000 files to another dir in the same hdfs. >> but the copy process is really slow since it is copying one by one. >> Is there a fast way to copy this using Java FileSystem or FileUtil api? >> >> Thanks. >> >> >
