Ted Yu created HBASE-20124:
------------------------------
Summary: Make hbase-spark module work with hadoop3
Key: HBASE-20124
URL: https://issues.apache.org/jira/browse/HBASE-20124
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
The following error can be observed when running tests in hbase-spark module
against hadoop3:
{code}
HBaseDStreamFunctionsSuite:
*** RUN ABORTED ***
java.lang.NoClassDefFoundError: org/apache/hadoop/ipc/ExternalCall
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
at
org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:746)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:640)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:979)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
...
Cause: java.lang.ClassNotFoundException: org.apache.hadoop.ipc.ExternalCall
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
at
org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:746)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
{code}
The dependency tree shows mixture of hadoop 2.7.4 and hadoop3 for the
hbase-spark module.
This should be addressed by adding proper profile in pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
