Are you able to distcp folders that don't have special characters? What are the versions of the two clusters and are you running on the destination cluster if there's a mis-match? If there is you'll need to use hftp:
http://hadoop.apache.org/common/docs/current/distcp.html#cpver On Wed, May 18, 2011 at 12:44 PM, sonia gehlot <[email protected]>wrote: > Hi Guys > > I am trying to copy hadoop data from one cluster to another but I am keep > on > getting this error *Server returned HTTP response code: 500 for URL* > * > * > My distcp command is: > scripts/hadoop.sh distcp > > hftp://c13-hadoop1-nn-r0-n1:50070/user/dwadmin/live/data/warehouse/facts/page_events/ > *day=2011-05-17* hdfs://phx1-rb-dev40-pipe1.cnet.com:9000/user/sgehlot > > In here I have *day=2011-05-17* in my file path > > I found this online: https://issues.apache.org/jira/browse/HDFS-31 > > Is this issue is still exists? Is this could be the reason of my job > failure? > > Job Error log: > > 2011-05-18 11:34:56,505 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: > Initializing JVM Metrics with processName=MAP, sessionId= > 2011-05-18 11:34:56,713 INFO org.apache.hadoop.mapred.MapTask: > numReduceTasks: 0 > 2011-05-18 11:34:57,039 INFO org.apache.hadoop.tools.DistCp: FAIL > > day=2011-05-17/_logs/history/c13-hadoop1-nn-r0-n1_1291919715221_job_201012091035_41977_dwadmin_CopyFactsToHive%3A+page_events+day%3D2011-05-17 > : java.io.IOException: *Server returned HTTP response code: 500 for URL*: > > http://c13-hadoop1-wkr-r10-n4.cnet.com:50075/streamFile?filename=/user/dwadmin/live/data/warehouse/facts/page_events/day=2011-05-17/_logs/history/c13-hadoop1-nn-r0-n1_1291919715221_job_201012091035_41977_dwadmin_CopyFactsToHive%253A+page_events+day%253D2011-05-17&ugi=sgehlot,user > at > > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1313) > at org.apache.hadoop.hdfs.HftpFileSystem.open(HftpFileSystem.java:157) > at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:398) > at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:410) > at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:537) > at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:306) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > 2011-05-18 11:35:06,118 WARN org.apache.hadoop.mapred.TaskTracker: Error > running child > java.io.IOException: Copied: 0 Skipped: 5 Failed: 1 > at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:572) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > 2011-05-18 11:35:06,124 INFO org.apache.hadoop.mapred.TaskRunner: Runnning > cleanup for the task > > Any help is appreciated. > > Thanks, > Sonia >
