Re:Kylin GUI error "Cannot get HiveTableMeta" on AWS EMR with Glue as hive metastore

[Xiaoxiang Yu]

Thank you for update on this. And appreciate Kaige 's contribution.










--

Best wishes to you ! 
From ：Xiaoxiang Yu





At 2020-04-10 22:38:00, "mvishnubhatta"  
wrote:
>Great. Thank you. I tried out Kaige's suggestion on
>https://issues.apache.org/jira/browse/KYLIN-3685 that you pointed me to and
>that worked for me.
>
>--
>Sent from: http://apache-kylin.74782.x6.nabble.com/

Re: Re:Kylin GUI error "Cannot get HiveTableMeta" on AWS EMR with Glue as hive metastore

[mvishnubhatta]

Great. Thank you. I tried out Kaige's suggestion on
https://issues.apache.org/jira/browse/KYLIN-3685 that you pointed me to and
that worked for me.

--
Sent from: http://apache-kylin.74782.x6.nabble.com/

Re:Kylin GUI error "Cannot get HiveTableMeta" on AWS EMR with Glue as hive metastore

[Xiaoxiang Yu]

For 3.0.2, we are trying to fix a few security issues, and code change is under 
review. 
I think we maybe start to plan a release RC for 3.0.2 in next two week, depend 
on the review result.










--

Best wishes to you ! 
From ：Xiaoxiang Yu





At 2020-04-10 09:21:59, "mvishnubhatta"  
wrote:
>Ah, that makes sense. I didn't look closely at the "31" in the link. I will
>try out your suggestions, but is there any planned timeline for 3.0.2 or
>3.1?
>
>Appreciate your quick response.
>
>--
>Sent from: http://apache-kylin.74782.x6.nabble.com/

Re: Re:Kylin GUI error "Cannot get HiveTableMeta" on AWS EMR with Glue as hive metastore

[mvishnubhatta]

Ah, that makes sense. I didn't look closely at the "31" in the link. I will
try out your suggestions, but is there any planned timeline for 3.0.2 or
3.1?

Appreciate your quick response.

--
Sent from: http://apache-kylin.74782.x6.nabble.com/

Re:Kylin GUI error "Cannot get HiveTableMeta" on AWS EMR with Glue as hive metastore

[Xiaoxiang Yu]

Hi, 
Glad to hear you that you have fix previous issue and thank you for your 
update, that is great. For the AWS Glue support with Kylin, that feature will 
be introduced in future release(should be 3.0.2 and 3.1.0), please check 
https://issues.apache.org/jira/browse/KYLIN-4206, the doc 
https://kylin.apache.org/docs31/install/kylin_aws_emr.html is for Kylin 3.1, 
and it is not released yet.


For now, you can build binary package from master branch and do a test, 
this is my https://github.com/hit-lacus/hit-lacus.github.io/issues/81 .
Besides, another committer Kaige has another suggestion(but I didn't test 
it), you can check it:  https://issues.apache.org/jira/browse/KYLIN-3685 .







--

Best wishes to you ! 
From ：Xiaoxiang Yu





At 2020-04-10 05:29:04, "mvishnubhatta"  
wrote:
>Hi,
>
>I am trying to set up Kylin on an EMR cluster (trying both on a master node
>and on an edge node). While the GUI shows up, I get a "Cannot get
>HiveTableMeta" error when trying to load a hive table.
>
>The hive metastore is AWS Glue.
>
>Curiously, the  most recent version of the document
>   does not say
>anything about Glue)
>
>Looking at an older version of the document
>  , I tried the
>following:
>
>cp /usr/lib/hive/auxlib/aws-glue-datacatalog-hive2-client.jar
>$KYLIN_HOME/lib/
>cp
>/usr/share/aws/hmclient/lib/aws-glue-datacatalog-client-common-1.11.0-SNAPSHOT.jar
>$KYLIN_HOME/lib/
>#Modify kylin.properties to uncomment the entry
>kylin.source.hive.client=cli
>#Modify kylin.properties to add an entry that says 
>kylin.source.hive.metadata-type=gluecatalog
>
>But I still get the same error on the GUI.
>
>I feel this Kylin server is not even aware that Glue is the hive catalog
>since I cannot find any reference to Glue in any of the logs or error
>messages.
>
>Any ideas on how to set up the hive metastore correctly?
>
>Hadoop version is Hadoop 2.8.5-amzn-4
>Kylin version is 3.0.1
>Hive 2.3.5-amzn-0
>
>I exported the following variables:
>export KYLIN_HOME=/usr/local/kylin/apache-kylin-3.0.1-bin-hbase1x
>#export HADOOP_HOME=/usr/lib/hadoop/etc/hadoop
>export HADOOP_HOME=/usr/lib/hadoop
>export HBASE_HOME=/usr/lib/hbase/
>export HIVE_HOME=/usr/lib/hive/
>export HADOOP_CONF_DIR=/etc/hadoop/conf
>export HIVE_LIB=/usr/lib/hive/lib
>export HIVE_CONF=/etc/hive/conf
>export HCAT_HOME=/usr/lib/hive-hcatalog 
>
>The kylin.log file throws this error:
>2020-04-09 21:17:21,243 ERROR [http-bio-7070-exec-1]
>controller.TableController:130 : Failed to load Hive Table
>java.lang.RuntimeException: cannot get HiveTableMeta
>at
>org.apache.kylin.source.hive.HiveMetadataExplorer.loadTableMetadata(HiveMetadataExplorer.java:68)
>at
>org.apache.kylin.rest.service.TableService.extractHiveTableMeta(TableService.java:211)
>at
>org.apache.kylin.rest.service.TableService.loadHiveTablesToProject(TableService.java:137)
>at
>org.apache.kylin.rest.controller.TableController.loadHiveTables(TableController.java:114)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at
>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>at
>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>at java.lang.reflect.Method.invoke(Method.java:498)
>at
>org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
>at
>org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
>at
>org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
>at
>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
>at
>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
>at
>org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
>at
>org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
>at
>org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
>at
>org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
>at
>org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
>at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
>at
>org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
>at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
>at