Today I start a Kylin 1.5.4.1-hbase1.x package on a EMR cluster (separately request a client node), got this NoClassDefFoundError, finally solved by changing the /usr/lib/hbase/conf/hbase-env.sh; I recorded it in the FAQ page, just for your reference:
https://kylin.apache.org/docs15/gettingstarted/faq.html 2016-10-20 9:19 GMT+08:00 张磊 <121762...@qq.com>: > find-hive-dependency.sh is ok > I say find-hbase-dependency.sh this script, > when i mkdir -p /usr/lib/oozie/lib, the variable HBASE_ENV_INIT is set true > i do not edit /usr/lib/hbase/conf/hbase-env.sh and Change the first line > from > export HBASE_CLASSPATH=/etc/hadoop/conf > to > export HBASE_CLASSPATH=$HBASE_CLASSPATH:/etc/hadoop/conf > > > > > ------------------ 原始邮件 ------------------ > 发件人: "ShaoFeng Shi";<shaofeng...@apache.org>; > 发送时间: 2016年10月19日(星期三) 下午5:38 > 收件人: "dev"<dev@kylin.apache.org>; > > 主题: Re: java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/ > mapreduce/HCatInputFormat > > > > In 1.5.4 kylin change to load hcat from /usr/lib/hive-hcatalog, you can try > to modify the file to > https://github.com/apache/kylin/blob/kylin-1.5.4.1/build/bin/find-hive- > dependency.sh#L89 > > 2016-10-19 15:48 GMT+08:00 张磊 <121762...@qq.com>: > > > it is ok,thanks > > if my emr cluster has a directory /usr/lib/oozie/lib,HBASE_ENV_ > INIT=true, > > i do not change /usr/lib/hbase/conf/hbase-env.sh, i do not konw why you > > judge by /usr/lib/oozie/lib directory > > > > > > if [ -n is_aws ] && [ -d "/usr/lib/oozie/lib" ]; then > > export HBASE_ENV_INIT="true" > > fi > > > > > > > > > > > > ------------------ 原始邮件 ------------------ > > 发件人: "liuyiming.vip";<liuyiming....@gmail.com>; > > 发送时间: 2016年10月19日(星期三) 下午2:47 > > 收件人: "dev"<dev@kylin.apache.org>; > > > > 主题: Re: java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/ > > mapreduce/HCatInputFormat > > > > > > > > Edit /usr/lib/hbase/conf/hbase-env.sh > > Change the first line from > > export HBASE_CLASSPATH=/etc/hadoop/conf > > > > to > > export HBASE_CLASSPATH=$HBASE_CLASSPATH:/etc/hadoop/conf > > > > 2016-10-19 14:24 GMT+08:00 张磊 <121762...@qq.com>: > > > > > Hi when i build the sample kylin_sales_cube, in the kylin.log,i find > > > java.lang.NoClassDefFoundError. but in the log i find some messages, i > > do > > > not konw why? > > > > > > > > > /usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-2. > > 1.0-amzn-0.jar > > > is loaded, why class is not foung? > > > > > > > > > 016-10-18 23:17:59,902 INFO [pool-8-thread-3] > > > common.AbstractHadoopJob:203 : Hive Dependencies After Filtered: > > > /usr/lib/hive/lib/hive-metastore-2.1.0-amzn-0.jar,/ > > > usr/lib/hive/lib/hive-exec-2.1.0-amzn-0.jar,/usr/lib/hive- > > > hcatalog/share/hcatalog/hive-hcatalog-core-2.1.0-amzn-0.jar > > > > > > > > > 2016-10-18 23:17:59,907 INFO [pool-8-thread-3] > > > common.AbstractHadoopJob:371 : Job 'tmpjars' updated -- > > > file:/usr/lib/hive/lib/hive-metastore-2.1.0-amzn-0.jar, > > > file:/usr/lib/hive/lib/hive-exec-2.1.0-amzn-0.jar,file:/ > > > usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-2. > 1.0-amzn-0.jar > > > > > > > > > 2016-10-18 23:17:59,934 ERROR [pool-8-thread-3] > > > threadpool.DefaultScheduler:140 : ExecuteException > > > job:845380a7-63b7-4315-8d71-6bb290018790 > > > org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job. > > exception.ExecuteException: > > > java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/ > > > mapreduce/HCatInputFormat > > > at org.apache.kylin.job.execution.AbstractExecutable. > > > execute(AbstractExecutable.java:123) > > > at org.apache.kylin.job.impl.threadpool.DefaultScheduler$ > > > JobRunner.run(DefaultScheduler.java:136) > > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > > > ThreadPoolExecutor.java:1142) > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > > > ThreadPoolExecutor.java:617) > > > at java.lang.Thread.run(Thread.java:745) > > > Caused by: org.apache.kylin.job.exception.ExecuteException: java.lang. > > NoClassDefFoundError: > > > org/apache/hive/hcatalog/mapreduce/HCatInputFormat > > > at org.apache.kylin.job.execution.AbstractExecutable. > > > execute(AbstractExecutable.java:123) > > > at org.apache.kylin.job.execution.DefaultChainedExecutable. > > doWork( > > > DefaultChainedExecutable.java:57) > > > at org.apache.kylin.job.execution.AbstractExecutable. > > > execute(AbstractExecutable.java:113) > > > ... 4 more > > > Caused by: java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/ > > > mapreduce/HCatInputFormat > > > at org.apache.kylin.source.hive.HiveMRInput$ > > HiveTableInputFormat. > > > configureJob(HiveMRInput.java:89) > > > at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob. > > > setupMapper(FactDistinctColumnsJob.java:123) > > > at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob. > > > run(FactDistinctColumnsJob.java:103) > > > at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:88) > > > at org.apache.kylin.engine.mr.common.MapReduceExecutable. > > > doWork(MapReduceExecutable.java:120) > > > at org.apache.kylin.job.execution.AbstractExecutable. > > > execute(AbstractExecutable.java:113) > > > ... 6 more > > > Caused by: java.lang.ClassNotFoundException: org.apache.hive.hcatalog. > > > mapreduce.HCatInputFormat > > > at org.apache.catalina.loader.WebappClassLoaderBase.loadClass( > > > WebappClassLoaderBase.java:1858) > > > at org.apache.catalina.loader.WebappClassLoaderBase.loadClass( > > > WebappClassLoaderBase.java:1701) > > > ... 12 more > > > > > > > > > > -- > > With Warm regards > > > > Yiming Liu (刘一鸣) > > > > > > -- > Best regards, > > Shaofeng Shi 史少锋 > -- Best regards, Shaofeng Shi 史少锋