rajgowtham24 opened a new issue #2086:
URL: https://github.com/apache/hudi/issues/2086
Hi,
I am trying to achieve two identical copy of the Hudi Tables in different
buckets.
For the above scenario, I have copied the contents of the hudi table data
files into another bucket and trying to run the hive sync through
run_sync_tool.sh and i'm getting the below error.
Environment Details
emr-6.0.0
Hudi Version - 0.5.0
Could anyone please take a look and suggest an alternate approach as well.
Thanks!
2020-09-11 12:57:52,736 ERROR [main] metadata.HiveUtils
(HiveUtils.java:createMetaStoreClientFactory(498)) - Unable to instantiate a
metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
found
java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2541)
at
org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClientFactory(HiveUtils.java:491)
at
org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:480)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4371)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4351)
at
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4607)
at
org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:287)
at
org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:270)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:443)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:371)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:351)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:327)
at
org.apache.hudi.hive.HoodieHiveClient.<init>(HoodieHiveClient.java:111)
at org.apache.hudi.hive.HiveSyncTool.<init>(HiveSyncTool.java:60)
at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:197)
2020-09-11 12:57:52,756 WARN [main] metadata.Hive
(Hive.java:registerAllFunctionsOnce(273)) - Failed to register all functions.
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
found)
at
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4610)
at
org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:287)
at
org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:270)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:443)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:371)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:351)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:327)
at
org.apache.hudi.hive.HoodieHiveClient.<init>(HoodieHiveClient.java:111)
at org.apache.hudi.hive.HiveSyncTool.<init>(HiveSyncTool.java:60)
at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:197)
Caused by: MetaException(message:Unable to instantiate a metastore client
factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
found)
at
org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClientFactory(HiveUtils.java:499)
at
org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:480)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4371)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4351)
at
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4607)
... 9 more
Exception in thread "main" org.apache.hudi.hive.HoodieHiveSyncException:
Failed to create HiveMetaStoreClient
at
org.apache.hudi.hive.HoodieHiveClient.<init>(HoodieHiveClient.java:113)
at org.apache.hudi.hive.HiveSyncTool.<init>(HiveSyncTool.java:60)
at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:197)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unable
to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
found)
at
org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:275)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:443)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:371)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:351)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:327)
at
org.apache.hudi.hive.HoodieHiveClient.<init>(HoodieHiveClient.java:111)
... 2 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
found)
at
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4610)
at
org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:287)
at
org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:270)
... 7 more
Caused by: MetaException(message:Unable to instantiate a metastore client
factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due
to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not
found)
at
org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClientFactory(HiveUtils.java:499)
at
org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:480)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4371)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4351)
at
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4607)
... 9 more
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
