[
https://issues.apache.org/jira/browse/MAPREDUCE-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205055#comment-13205055
]
Jason Lowe commented on MAPREDUCE-3845:
---------------------------------------
This seems related to the same type of problem as in HADOOP-7999 which should
be fixed in 0.23. I'm unable to reproduce the problem, as I get the same
output from both commands:
{noformat}
$ ls
hadoop-0.23.1.tar.gz
$ tar xf hadoop\-0.23.1.tar.gz
$ cd hadoop\-0.23.1
$ ./bin/hadoop distcp
12/02/09 17:40:23 ERROR tools.DistCp: Invalid arguments:
java.lang.IllegalArgumentException: Target path not specified
at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:86)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:95)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:357)
Invalid arguments: Target path not specified
usage: distcp OPTIONS [source_path...] <target_path>
OPTIONS
-async Should distcp execution be blocking
-atomic Commit all changes or none
-bandwidth <arg> Specify bandwidth per map in MB
-delete Delete from target, files missing in source
-f <arg> List of files that need to be copied
-filelimit <arg> (Deprecated!) Limit number of files copied to <= n
-i Ignore failures during copy
-log <arg> Folder on DFS where distcp execution logs are
saved
-m <arg> Max number of concurrent maps to use for copy
-mapredSslConf <arg> Configuration for ssl config file, to use with
hftps://
-overwrite Choose to overwrite target files unconditionally,
even if they exist.
-p <arg> preserve status (rbugp)(replication, block-size,
user, group, permission)
-sizelimit <arg> (Deprecated!) Limit number of files copied to <= n
bytes
-skipcrccheck Whether to skip CRC checks between source and
target paths.
-strategy <arg> Copy strategy to use. Default is dividing work
based on file sizes
-tmp <arg> Intermediate work path to be used for atomic
commit
-update Update target, copying only missingfiles or
directories
$ ./bin/mapred distcp
12/02/09 17:40:29 ERROR tools.DistCp: Invalid arguments:
java.lang.IllegalArgumentException: Target path not specified
at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:86)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:95)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:357)
Invalid arguments: Target path not specified
usage: distcp OPTIONS [source_path...] <target_path>
OPTIONS
-async Should distcp execution be blocking
-atomic Commit all changes or none
-bandwidth <arg> Specify bandwidth per map in MB
-delete Delete from target, files missing in source
-f <arg> List of files that need to be copied
-filelimit <arg> (Deprecated!) Limit number of files copied to <= n
-i Ignore failures during copy
-log <arg> Folder on DFS where distcp execution logs are
saved
-m <arg> Max number of concurrent maps to use for copy
-mapredSslConf <arg> Configuration for ssl config file, to use with
hftps://
-overwrite Choose to overwrite target files unconditionally,
even if they exist.
-p <arg> preserve status (rbugp)(replication, block-size,
user, group, permission)
-sizelimit <arg> (Deprecated!) Limit number of files copied to <= n
bytes
-skipcrccheck Whether to skip CRC checks between source and
target paths.
-strategy <arg> Copy strategy to use. Default is dividing work
based on file sizes
-tmp <arg> Intermediate work path to be used for atomic
commit
-update Update target, copying only missingfiles or
directories
$
{noformat}
Normally the distcp jar is located via TOOL_PATH which is setup in
hadoop-config.sh if not overridden in the environment. In both the {{hadoop}}
and {{mapred}} cases I'm seeing TOOL_PATH set via hadoop-config.sh. It'd be
interesting to see how one script is picking up the proper TOOL_PATH but the
other isn't (e.g.: prepend {{bash -x}} to each shell command and compare the
output)
> hadoop distcp fails to run with java.lang.NoClassDefFoundError
> --------------------------------------------------------------
>
> Key: MAPREDUCE-3845
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3845
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.23.1
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.23.1
>
>
> Here's how to reproduce:
> {noformat}
> $ cd /tmnp
> $ curl
> http://people.apache.org/~acmurthy/hadoop-0.23.1-rc0/hadoop-0.23.1.tar.gz |
> tar xzvf -
> $ cd hadoop-0.23.1
> $ ./bin/hadoop distcp
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/tools/DistCp
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.tools.DistCp
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Could not find the main class: org.apache.hadoop.tools.DistCp. Program will
> exit.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
