Hi krishna,
I guess that you set EMR to use AWS Glue catalog as Hive metadata and Kylin
is missing the AWS lib com.amazonaws.glue in Kylin's classpath. You should find
the lib in the EMR cluster and add it to your class path(maybe $KYLIN_HOME/lib)
is OK.
If you cannot find the right jar, you may package it manually, repo should
be this
https://github.com/awslabs/aws-glue-data-catalog-client-for-apache-hive-metastore.
Maybe ask EMR customer service for help should be considered.
==============================================================
Xiaoxiang Yu
Best wishes to you!
> 在 2019年7月16日，10:40，Krishna Bandaru <[email protected]> 写道：
>
> hi I created Kylin cluster with HA(3 masters and 2 cores)
> <Screen Shot 2019-07-15 at 10.36.18 PM.png>
> java.lang.RuntimeException: java.io.IOException: MetaException(message:Unable
> to instantiate a metastore client factory
> com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
> to: java.lang.ClassNotFoundException: Class
> com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
> found)
> at
> org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:97)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:122)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:100)
> at
> org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
> at
> org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:111)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: MetaException(message:Unable to instantiate a
> metastore client factory
> com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
> to: java.lang.ClassNotFoundException: Class
> com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
> found)
> at
> org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97)
> at
> org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
> at
> org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:94)
> ... 10 more
> Caused by: MetaException(message:Unable to instantiate a metastore client
> factory
> com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
> to: java.lang.ClassNotFoundException: Class
> com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
> found)
> at
> org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClientFactory(HiveUtils.java:525)
> at
> org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:506)
> at
> org.apache.hive.hcatalog.common.HiveClientCache.getNonCachedHiveMetastoreClient(HiveClientCache.java:99)
> at
> org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:318)
> at
> org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:315)
> at
> com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4791)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3584)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2372)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2335)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2250)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3985)
> at
> com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4788)
> at
> org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:315)
> at
> org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:277)
> at
> org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558)
> at
> org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
> at
> org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88)
> at
> org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
> ... 12 more
>
>
> I am getting below error please help me to get rid off this.
> even though i set
> <property>
> <name>hive.aux.jars.path </name>
>
> <value>file:///usr/lib/hive/auxlib/aws-glue-datacatalog-hive2-client.jar</value>
> </property>
> in hive-site.xml i am getting above error while cube designing in step 2(#2
> Step Name: Extract Fact Table Distinct Columns)
>
> thanks,
> krishna
