Thanks Kan, Then I think DistCp documentation needs to be corrected.
http://hadoop.apache.org/common/docs/r0.20.2/distcp.html For copying between two different versions of Hadoop, one will usually use HftpFileSystem. This is a read-only FileSystem, so DistCp must be run on the destination cluster (more specifically, on TaskTrackers that can write to the destination cluster). Each source is specified as hftp://<dfs.http.address>/<path> (the default dfs.http.address is <namenode>:50070). - Ravi On Tue, Jan 4, 2011 at 2:50 PM, Kan Zhang <k...@yahoo-inc.com> wrote: > Ravi, HFTP only supports READ operation for now. > > > > On 1/4/11 2:10 PM, "Ravi Phulari" <iphul...@gmail.com> wrote: > > Hello Hadoopers, > I need to distcp data across two clusters. For security reasons I can not > use hdfs based distcp. > HFTP based distcp is failing with following Ioexception. > > Stack trace. > > Copy failed: java.io.IOException: Not supported > at > org.apache.hadoop.hdfs.HftpFileSystem.delete(HftpFileSystem.java:360) > at org.apache.hadoop.tools.DistCp.fullyDelete(DistCp.java:939) > at org.apache.hadoop.tools.DistCp.copy(DistCp.java:655) > at org.apache.hadoop.tools.DistCp.run(DistCp.java:857) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > at org.apache.hadoop.tools.DistCp.main(DistCp.java:884) > > I am using following command for distcp. > > Hadoop distcp hftp://nn1.hadoop1:50070/data > hftp://nn2.hadoop2:50070/user/hadoop/ > Hadoop distcp /data/logs hftp://nn2.hadoop2:50070/user/hadoop/ > > Any idea why this distcp could be failing. > I don’t see any logs in JT and NN. > > Any help will be greatly appreciated. > > - > Thanks, > Ravi > >
