Kylin runs good with EMR 5.5 on AWS Oregon zone, I verfied that today; also Spark engine runs well.
2017-05-16 23:02 GMT+08:00 ShaoFeng Shi <[email protected]>: > Hi Lei, > > How did you create the EMR cluster? did you select "HCatalog" component? > > 2017-05-15 17:38 GMT+08:00 ShaoFeng Shi <[email protected]>: > >> ok the script looks good, thanks lei; I will check that. >> >> 2017-05-15 13:51 GMT+08:00 张磊 <[email protected]>: >> >>> The script of /usr/bin/hbase is: >>> >>> >>> #!/bin/bash >>> >>> >>> BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default} >>> [ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hbase ] && . >>> ${BIGTOP_DEFAULTS_DIR}/hbase >>> >>> >>> # Autodetect JAVA_HOME if not defined >>> . /usr/lib/bigtop-utils/bigtop-detect-javahome >>> >>> >>> export HADOOP_CONF=${HADOOP_CONF:-/etc/hadoop/conf} >>> export ZOOKEEPER_HOME=${ZOOKEEPER_HOME:-/usr/lib/zookeeper} >>> export HBASE_CLASSPATH=$HADOOP_CONF:$HADOOP_HOME/*:$HADOOP_HOME/lib >>> /*:$ZOOKEEPER_HOME/*:$ZOOKEEPER_HOME/lib/*:$HBASE_CLASSPATH >>> >>> >>> exec /usr/lib/hbase/bin/hbase "$@" >>> >>> >>> >>> >>> >>> ------------------ 原始邮件 ------------------ >>> 发件人: "ShaoFeng Shi";<[email protected]>; >>> 发送时间: 2017年5月15日(星期一) 中午1:03 >>> 收件人: "dev"<[email protected]>; >>> >>> 主题: Re: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/sess >>> ion/SessionState >>> >>> >>> >>> ok.. seems different regions are different. Did you check the first in >>> Kylin FAQ: https://kylin.apache.org/docs20/gettingstarted/faq.html >>> >>> 2017-05-15 9:42 GMT+08:00 张磊 <[email protected]>: >>> >>> > region：us-west-1 >>> > version：EMR 5.5.0 >>> > Hadoop：Amazon 2.7.3 >>> > Application：Hive 2.1.1, Tez 0.8.4, HBase 1.3.0, ZooKeeper 3.4.10, Spark >>> > 2.1.0, HCatalog 2.1.1 >>> > >>> > >>> > >>> > ------------------ 原始邮件 ------------------ >>> > 发件人: "ShaoFeng Shi";<[email protected]>; >>> > 发送时间: 2017年5月15日(星期一) 上午9:39 >>> > 收件人: "dev"<[email protected]>; >>> > >>> > 主题: Re: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/ >>> > session/SessionState >>> > >>> > >>> > >>> > Hi Lei, >>> > >>> > Which AWS region are you using? I tested that in the Seoul, wondering >>> > whethere that is the version/location difference. >>> > >>> > 2017-05-15 9:32 GMT+08:00 张磊 <[email protected]>: >>> > >>> > > @史少锋 >>> > > Hi shaofeng >>> > > I install kylin 2.0 in EMR5.5.0 follow the 【技术贴】使用Apache Kylin和AWS >>> > > EMR进行云上大数据OLAP分析 >>> > > But also found this error, why？ >>> > > >>> > > >>> > > 2017-05-14 18:23:57,597 ERROR [pool-9-thread-2] >>> > > threadpool.DefaultScheduler:146 : ExecuteException >>> > > job:5559cf6c-6942-4df2-8700-6aa73a8765dd >>> > > org.apache.kylin.job.exception.ExecuteException: >>> org.apache.kylin.job. >>> > exception.ExecuteException: >>> > > java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/ >>> > > session/SessionState >>> > > at org.apache.kylin.job.execution.AbstractExecutable. >>> > > execute(AbstractExecutable.java:134) >>> > > at org.apache.kylin.job.impl.threadpool.DefaultScheduler$ >>> > > JobRunner.run(DefaultScheduler.java:142) >>> > > at java.util.concurrent.ThreadPoolExecutor.runWorker( >>> > > ThreadPoolExecutor.java:1142) >>> > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( >>> > > ThreadPoolExecutor.java:617) >>> > > at java.lang.Thread.run(Thread.java:745) >>> > > Caused by: org.apache.kylin.job.exception.ExecuteException: >>> java.lang. >>> > NoClassDefFoundError: >>> > > org/apache/hadoop/hive/ql/session/SessionState >>> > > at org.apache.kylin.job.execution.AbstractExecutable. >>> > > execute(AbstractExecutable.java:134) >>> > > at org.apache.kylin.job.execution.DefaultChainedExecutable. >>> > doWork( >>> > > DefaultChainedExecutable.java:64) >>> > > at org.apache.kylin.job.execution.AbstractExecutable. >>> > > execute(AbstractExecutable.java:124) >>> > > ... 4 more >>> > > Caused by: java.lang.NoClassDefFoundError: >>> org/apache/hadoop/hive/ql/ >>> > > session/SessionState >>> > > at org.apache.kylin.source.hive.H >>> iveClientFactory.getHiveClient( >>> > > HiveClientFactory.java:26) >>> > > at org.apache.kylin.source.hive.HiveMRInput$ >>> > > RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:256) >>> > > at org.apache.kylin.source.hive.HiveMRInput$ >>> > > RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:300) >>> > > at org.apache.kylin.job.execution.AbstractExecutable. >>> > > execute(AbstractExecutable.java:124) >>> > > ... 6 more >>> > > Caused by: java.lang.ClassNotFoundException: >>> org.apache.hadoop.hive.ql. >>> > > session.SessionState >>> > > at org.apache.catalina.loader.Web >>> appClassLoaderBase.loadClass( >>> > > WebappClassLoaderBase.java:1858) >>> > > at org.apache.catalina.loader.Web >>> appClassLoaderBase.loadClass( >>> > > WebappClassLoaderBase.java:1701) >>> > > ... 10 more >>> > >>> > >>> > >>> > >>> > -- >>> > Best regards, >>> > >>> > Shaofeng Shi 史少锋 >>> > >>> >>> >>> >>> -- >>> Best regards, >>> >>> Shaofeng Shi 史少锋 >>> >> >> >> >> -- >> Best regards, >> >> Shaofeng Shi 史少锋 >> >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > -- Best regards, Shaofeng Shi 史少锋
