Hi Hubert, In the original log file, I see hbase-server and hbase-common are on the spark command:
--jars */usr/lib/hbase/lib/hbase-common-1.4.2.jar*,*/usr/lib/hbase/lib/hbase-server-1.4.2.jar*,/usr/lib/hbase/lib/hbase-client-1.4.2.jar,/usr/lib/hbase/lib/hbase-protocol-1.4.2.jar,/usr/lib/hbase/lib/hbase-hadoop-compat-1.4.2.jar,/usr/lib/hbase/lib/htrace-core-3.1.0-incubating.jar,/usr/lib/hbase/lib/metrics-core-2.2.0.jar,/usr/lib/hbase/lib/hbase-hadoop-compat-1.4.2.jar,/usr/lib/hbase/lib/hbase-hadoop2-compat-1.4.2.jar, Did you add other jars? Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: [email protected] Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: [email protected] Join Kylin dev mail group: [email protected] hubert stefani <[email protected]> 于2019年1月11日周五 下午6:20写道： > indeed. By adding the hbase-server and hbase common jars the pb seems to > be fixed. > > > Le vendredi 11 janvier 2019 à 11:11:40 UTC+1, ShaoFeng Shi < > [email protected]> a écrit : > > It seems missing some HBase class; The HFile.class should be in > hbase-server-<version>.jar, not sure whether it is EMR's package issue. You > can unzip the HBase jar files to see which jar has the class, and then add > it to spark/lib folder. > > java.lang.NoClassDefFoundError: Could not initialize class > org.apache.hadoop.hbase.io.hfile.HFile > at > org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2$1.getNewWriter(HFileOutputFormat2.java:305) > at > org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2$1.write(HFileOutputFormat2.java:229) > at > org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2$1.write(HFileOutputFormat2.java:167) > at org.apache.spark.internal.io > .HadoopMapReduceWriteConfigUtil.write(SparkHadoopWriter.scala:356) > at org.apache.spark.internal.io > .SparkHadoopWriter$$anonfun$4.apply(SparkHadoopWriter.scala:130) > at org.apache.spark.internal.io > .SparkHadoopWriter$$anonfun$4.apply(SparkHadoopWriter.scala:127) > at > org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1415) > > > > Best regards, > > Shaofeng Shi 史少锋 > Apache Kylin PMC > Work email: [email protected] > Kyligence Inc: https://kyligence.io/ > > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html > Join Kylin user mail group: [email protected] > Join Kylin dev mail group: [email protected] > > > > > hubert stefani <[email protected]> 于2019年1月11日周五 下午4:52写道： > > > hello, > > > > we are testing the 2.6 RC and we are facing a systematic issue when > > building cubes with spark engine (even with sample cube), whereas the > > MapReduce engin succeeds. > > > > The job process fails at step #8 Step Name: Convert Cuboid Data to HFile > > with the following error (full log is available as attachment): > > > > ClassNotFoundException: org.apache.hadoop.hbase.metrics.MetricRegistry > > > > We run kylin on AWS EMR 5.13 (it failed also with 5.17). > > > > Do you have any idea of the reasons why it happens ? > > Hubert > >
