Thanks. I tried to put the jars in HDFS an set kylin.job.mr.lib.dir, but
this was not my case. I figured out that my problem was actually caused
by HBASE_CLASSPATH, which I changed in my hbase_env.sh with export
HBASE_CLASSPATH=some directory. Actually this should be HBASE_CLASSPATH=
${HBASE_CLASSPATH}:some directory. Li Yang's suggestion helped me:
http://apache-kylin.74782.x6.nabble.com/NoClassDefFoundError-org-apache-hive-hcatalog-mapreduce-HCatInputFormat-td1156.html
Best regards,
Jie
Am 03.03.2016 um 02:23 schrieb Dong Li:
Hi Jie,
This is caused by jar missing for hive dependency.
Please reference this thread:
http://apache-kylin.74782.x6.nabble.com/org-apache-hadoop-hive-ql-metadata-HiveException-td3048.html#a3051
Thanks,
Dong Li
Original Message
Sender:Jie [email protected]
Recipient:[email protected]
Date:Thursday, Mar 3, 2016 00:40
Subject:ava.lang.NoClassDefFoundError:
org/apache/hive/hcatalog/mapreduce/HCatInputFo,rmat
Dear Kylin developers, I have a cluster environment (3 zookeeper machine, one
yarn RM and one backup RM, 3 node managers) and install hive-hcatalog on all
these nodes under the same directory. While building the sample cube I got the
error at step 2: Caused by: java.lang.NoClassDefFoundError:
org/apache/hive/hcatalog/mapreduce/HCatInputFormat at
org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:101)
at
org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:77)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at
org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at
org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
... 6 more I see that the mapred and hadoop application classpath both have
this jar: hive-hcatalog g/share/hcatalog/hive-hcatalog-core-0.13.1.jar. The
path is correct. I do not know why the class FactDistinctColumnsJob.setupMapper
cannot find it. The suggested answers I found from Kylin mails to the users do
not help me. This actually happened in the setMapper function, so I guess Kylin
looks for the missing class only on the machine where I started Kylin (the same
machine where Hive and Hbase run). There is a message before the error:
[pool-7-thread-2]:[2016-03-02
17:10:20,450][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.cleanupTempConfFile(AbstractHadoopJob.java:401)]
- tempMetaFileString is : null But I think this is not related. You said that
hcatalog must be available on all machines. Does Kylin read any configuration
files to get the machines? I worked already two days on this problem but cannot
solve. I need your help. Thank you in advance. Best regards, Jie
