[
https://issues.apache.org/jira/browse/HDFS-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16291840#comment-16291840
]
Bharat Viswanadham edited comment on HDFS-12916 at 12/15/17 12:31 AM:
----------------------------------------------------------------------
[~aw] Thanks for your comments.
bq. In hadoop 3.x when HADOOP_OPTIONAL_TOOLS are loaded, they expect their
dependencies from the other directories to already be in the classpath. I'd be
greatly surprised if they find them from the shaded jar since I'm pretty sure
we do class hiding.
I have included Hadoop tools lib folder, so which will be adequate to get all
its dependencies. I have just ran distcp for sample, need to run other commands
to know more if they cause issue.
bq. But no matter what you do, everyone is already trained to use 'hadoop
classpath'. Until that gets changed, anything else that is done is moot.
if I override all home variables(like hdfs_home, mapred_home) to same client
location in hadoop-layout.sh, it will have only shaded client jars. so i think
this way we can address hadoop classpath problem.
was (Author: bharatviswa):
bq. In hadoop 3.x when HADOOP_OPTIONAL_TOOLS are loaded, they expect their
dependencies from the other directories to already be in the classpath. I'd be
greatly surprised if they find them from the shaded jar since I'm pretty sure
we do class hiding.
I have included Hadoop tools lib folder, so which will be adequate to get all
its dependencies. I have just ran distcp for sample, need to run other commands
to know more if they cause issue.
bq. But no matter what you do, everyone is already trained to use 'hadoop
classpath'. Until that gets changed, anything else that is done is moot.
if I override all home variables(like hdfs_home, mapred_home) to same client
location in hadoop-layout.sh, it will have only shaded client jars. so i think
this way we can address hadoop classpath problem.
> HDFS commands throws error, when only shaded clients in classpath
> -----------------------------------------------------------------
>
> Key: HDFS-12916
> URL: https://issues.apache.org/jira/browse/HDFS-12916
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
>
> [root@n001 hadoop]# bin/hdfs dfs -rm /
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/htrace/core/Tracer$Builder
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:303)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:389)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.htrace.core.Tracer$Builder
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 4 more
> cc [~busbey]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
