[
https://issues.apache.org/jira/browse/MAPREDUCE-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated MAPREDUCE-7471:
---------------------------------
Summary: Hadoop mapred minicluster command line fails with class not found
(was: Hadoop mapred minicluster command line fail with class not found)
> Hadoop mapred minicluster command line fails with class not found
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-7471
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7471
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 3.3.5
> Reporter: Duo Zhang
> Priority: Major
>
> If you run
> ./bin/mapred minicluster
> It will fail with
> {noformat}
> Exception in thread "Listener at localhost/35325"
> java.lang.NoClassDefFoundError: org/mockito/stubbing/Answer
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.isNameNodeUp(MiniDFSCluster.java:2648)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.isClusterUp(MiniDFSCluster.java:2662)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:1510)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:989)
> at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:588)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:530)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:160)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:132)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:320)
> Caused by: java.lang.ClassNotFoundException: org.mockito.stubbing.Answer
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 9 more
> {noformat}
> This line
> https://github.com/apache/hadoop/blob/835403d872506c4fa76eb2d721f2d91f413473d5/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java#L2648
> This is because we rely on mockito in NameNodeAdapter but we do not have
> mockito on our classpath, at least in our published hadoop-3.3.5 binary.
> And there is another problem that, if we do not run the above command in the
> HADOOP_HOME directory, i.e, in another directory by typing the absolute path
> of the mapred command, it will fail with
> {noformat}
> Exception in thread "main" java.lang.NoClassDefFoundError: org/junit/Assert
> at
> org.apache.hadoop.test.GenericTestUtils.assertExists(GenericTestUtils.java:336)
> at
> org.apache.hadoop.test.GenericTestUtils.getTestDir(GenericTestUtils.java:280)
> at
> org.apache.hadoop.test.GenericTestUtils.getTestDir(GenericTestUtils.java:289)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.getBaseDirectory(MiniDFSCluster.java:3069)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster$Builder.<init>(MiniDFSCluster.java:239)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:157)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:132)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:320)
> Caused by: java.lang.ClassNotFoundException: org.junit.Assert
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 8 mor
> {noformat}
> This simply because this line
> https://github.com/apache/hadoop/blob/835403d872506c4fa76eb2d721f2d91f413473d5/hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh#L601
> We should add the $HADOOP_TOOLS_HOME prefix for the default value of
> HADOOP_TOOLS_DIR.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
