Hi, I am trying to set up a Kylin server on an AWS EMR edge node. When doing that the web URL throws the error that "The origin server did not find a current representation for the target resource or is not willing to disclose that one exists".
The tomcat localhost log file shows a NoClassDefFoundError for the class org/apache/hadoop/hive/metastore/api/NoSuchObjectException But I see this class in the hive library /usr/lib/hive/lib/hive-metastore-2.3.5-amzn-0.jar and the path is exported as HIVE_LIB and as HIVE_HOME (used HIVE_LIB based on suggestion here https://issues.apache.org/jira/browse/KYLIN-2511). This appears to be a case where some CLASSPATH somewhere is not correctly including this path. So, I tried modifying the kylin.properties file to add a property called kylin.job.mr.lib.dir pointing to the /usr/lib/hive/lib/ but that didnt help either. (This too was based on another suggested workaround here). Can someone point out what I am missing? Here are the full details: Hadoop version is Hadoop 2.8.5-amzn-4 Kylin version is 3.0.1 Hive 2.3.5-amzn-0 I have an AWS EMR Cluster and a separate EC2 machine that is set up as an edge node. From this edge node I am able to run hive queries to connect to the EMR Master node. Hbase also works on this node. On this edge node, I are trying to start a Kylin server. I followed the instructions given here: https://kylin.apache.org/docs/install/kylin_aws_emr.html <https://kylin.apache.org/docs/install/kylin_aws_emr.html> I exported the following variables: export KYLIN_HOME=/usr/local/kylin/apache-kylin-3.0.1-bin-hbase1x #export HADOOP_HOME=/usr/lib/hadoop/etc/hadoop export HADOOP_HOME=/usr/lib/hadoop export HBASE_HOME=/usr/lib/hbase/ export HIVE_HOME=/usr/lib/hive/lib/ export HADOOP_CONF_DIR=/etc/hadoop/conf export HIVE_LIB=/usr/lib/hive/lib export HIVE_CONF=/etc/hive/conf export HCAT_HOME=/usr/lib/hive-hcatalog The command I run is: $KYLIN_HOME/bin/kylin.sh start After starting it, I get this message: /A new Kylin instance is started by hadoop. To stop it, run 'kylin.sh stop' Check the log at /usr/local/kylin/apache-kylin-3.0.1-bin-hbase1x/logs/kylin.log Web UI is at http://ip-xxxxxxxxxxxxxxxxxxxxx.compute.internal:7070/kylin/ When I navigate to the URL, I get the following error. "The origin server did not find a current representation for the target resource or is not willing to disclose that one exists." When I look into the log file which is at $KYLIN_HOME/tomcat/logs/localhost.2020-04-08.log I see the following error: SEVERE: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping': Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/NoSuchObjectException at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443) at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325) at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5157) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5680) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1018) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:994) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652) at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1127) at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:2021) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/NoSuchObjectException at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.getDeclaredMethods(Class.java:1975) at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:613) at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:524) at org.springframework.core.MethodIntrospector.selectMethods(MethodIntrospector.java:68) at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.detectHandlerMethods(AbstractHandlerMethodMapping.java:230) at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.initHandlerMethods(AbstractHandlerMethodMapping.java:214) at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.afterPropertiesSet(AbstractHandlerMethodMapping.java:184) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping.afterPropertiesSet(RequestMappingHandlerMapping.java:127) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624) ... 25 more Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.NoSuchObjectException at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1928) -- Sent from: http://apache-kylin.74782.x6.nabble.com/
