amaranathv opened a new issue #716: Class not found error in mapr platform URL: https://github.com/apache/incubator-hudi/issues/716 I am getting below error while trying to do the quick start in our cluster environment. I can able to do it successfully in mapr LIVE environment but not in our company environment. Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_table(Ljava/lang/String;Ljava/lang/String;Lorg/apache/hadoop/hive/metastore/api/Table;Z)V I know there is some classpath issue.Can you please through some light on this one. Error Trace Log : $ ./run_sync_tool.sh --user ***** --pass **** --database default --jdbc-url "jdbc:hive2://dbsls0306:10255/" --base-path file:///tmp/hoodie/sample-table/ --table hoodie_test --partitioned-by field1 ls: cannot access /opt/mapr/hive/hive-2.1/lib/hive-jdbc-*standalone*.jar: No such file or directory Running Command : java -cp /opt/mapr/hive/hive-2.1/lib/hive-metastore-2.1.1-mapr-1803.jar::/opt/mapr/hive/hive-2.1/lib/hive-service-2.1.1-mapr-1803.jar::/opt/mapr/hive/hive-2.1/lib/hive-exec-2.1.1-mapr-1803.jar::/opt/mapr/hive/hive-2.1/lib/hive-service-2.1.1-mapr-1803.jar::/opt/mapr/hive/hive-2.1/lib/hive-jdbc-2.1.1-mapr-1803.jar::/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/*:/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/mapr/user/avenka23/hoodie/incubator-hudi/hoodie-hive/../packaging/hoodie-hive-bundle/target/hoodie-hive-bundle-0.4.8-SNAPSHOT.jar com.uber.hoodie.hive.HiveSyncTool --user avenka23 --pass Amarq22019 --database default --jdbc-url jdbc:hive2://dbsls0306:10255/ --base-path file:///tmp/hoodie/sample-table/ --table hoodie_test --partitioned-by field1 2019-06-05 10:06:57,699 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2019-06-05 10:06:57,910 INFO [main] util.FSUtils (FSUtils.java:getFs(104)) - Hadoop Configuration: fs.defaultFS: [maprfs:///], Config:[Configuration: core-default.xml, org.apache.hadoop.conf.CoreDefaultProperties, core-site.xml, mapred-default.xml, org.apache.hadoop.mapreduce.conf.MapReduceDefaultProperties, mapred-site.xml, yarn-default.xml, org.apache.hadoop.yarn.conf.YarnDefaultProperties, yarn-site.xml, hdfs-default.xml, hdfs-site.xml], FileSystem: [org.apache.hadoop.fs.LocalFileSystem@e15b7e8] 2019-06-05 10:06:57,916 INFO [main] conf.HiveConf (HiveConf.java:findConfigFile(181)) - Found configuration file file:/mapr/datalake/uhclake/hive/conf/hive-site.xml 2019-06-05 10:06:58,106 WARN [main] conf.HiveConf (HiveConf.java:initialize(3818)) - HiveConf of name hive.exec.warehousedir does not exist 2019-06-05 10:06:58,107 WARN [main] conf.HiveConf (HiveConf.java:initialize(3818)) - HiveConf of name hive.server2.authentication.pam.profiles does not exist 2019-06-05 10:06:58,107 WARN [main] conf.HiveConf (HiveConf.java:initialize(3818)) - HiveConf of name hive.server2.authentication.pam.services does not exist 2019-06-05 10:06:58,121 INFO [main] table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(84)) - Loading HoodieTableMetaClient from file:///tmp/hoodie/sample-table/ 2019-06-05 10:06:58,122 INFO [main] util.FSUtils (FSUtils.java:getFs(104)) - Hadoop Configuration: fs.defaultFS: [maprfs:///], Config:[Configuration: core-default.xml, org.apache.hadoop.conf.CoreDefaultProperties, core-site.xml, mapred-default.xml, org.apache.hadoop.mapreduce.conf.MapReduceDefaultProperties, mapred-site.xml, yarn-default.xml, org.apache.hadoop.yarn.conf.YarnDefaultProperties, yarn-site.xml, hdfs-default.xml, hdfs-site.xml], FileSystem: [org.apache.hadoop.fs.LocalFileSystem@e15b7e8] 2019-06-05 10:06:58,128 INFO [main] table.HoodieTableConfig (HoodieTableConfig.java:<init>(69)) - Loading dataset properties from file:/tmp/hoodie/sample-table/.hoodie/hoodie.properties 2019-06-05 10:06:58,136 INFO [main] table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(94)) - Finished Loading Table of type MERGE_ON_READ from file:///tmp/hoodie/sample-table/ 2019-06-05 10:06:58,136 INFO [main] table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(96)) - Loading Active commit timeline for file:///tmp/hoodie/sample-table/ 2019-06-05 10:06:58,170 INFO [main] timeline.HoodieActiveTimeline (HoodieActiveTimeline.java:<init>(77)) - Loaded instants java.util.stream.ReferencePipeline$Head@eadd4fb 2019-06-05 10:06:58,172 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:<init>(99)) - Creating hive connection jdbc:hive2://dbsls0306:10255/ 2019-06-05 10:06:58,174 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:createHiveConnection(475)) - Getting Hive Connection from Datasource HiveDataSource{driverClassName='org.apache.hive.jdbc.HiveDriver', driverClassLoader=null, url='jdbc:hive2://dbsls0306:10255/default'} 2019-06-05 10:06:58,188 INFO [main] jdbc.Utils (Utils.java:parseURL(321)) - Supplied authorities: dbsls0306:10255 2019-06-05 10:06:58,189 INFO [main] jdbc.Utils (Utils.java:parseURL(440)) - Resolved authority: dbsls0306:10255 2019-06-05 10:06:58,579 INFO [main] jdbc.Utils (Utils.java:parseURL(321)) - Supplied authorities: dbsls0306:10255 2019-06-05 10:06:58,580 INFO [main] jdbc.Utils (Utils.java:parseURL(440)) - Resolved authority: dbsls0306:10255 2019-06-05 10:06:58,675 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:createHiveConnection(478)) - Successfully got Hive Connection from Datasource HiveDataSource{driverClassName='org.apache.hive.jdbc.HiveDriver', driverClassLoader=null, url='jdbc:hive2://dbsls0306:10255/default'} 2019-06-05 10:06:58,707 INFO [main] hive.metastore (HiveMetaStoreClient.java:open(433)) - Trying to connect to metastore with URI thrift://dbsls0306.uhc.com:11018 2019-06-05 10:06:58,722 INFO [main] hive.metastore (HiveMetaStoreClient.java:open(478)) - Opened a connection to metastore, current connections: 1 2019-06-05 10:06:58,768 INFO [main] hive.metastore (HiveMetaStoreClient.java:open(530)) - Connected to metastore. 2019-06-05 10:06:58,794 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(88)) - Trying to sync hoodie table hoodie_test with base path file:///tmp/hoodie/sample-table/ of type MERGE_ON_READ 2019-06-05 10:07:19,417 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:getDataSchema(328)) - Found the last compaction commit as Optional[[20190605095831__commit__COMPLETED]] 2019-06-05 10:07:19,419 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:getDataSchema(340)) - Found the last delta commit Optional.empty 2019-06-05 10:07:19,828 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:readSchemaFromDataFile(425)) - Reading schema from file:/tmp/hoodie/sample-table/2015/03/16/47b45fa6-9a6e-4a67-a22c-ffd7c28da02f-0_0-41-67_20190605095831.parquet SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. 2019-06-05 10:07:22,900 INFO [main] util.SchemaUtil (SchemaUtil.java:getSchemaDifference(64)) - Getting schema difference for {_hoodie_commit_time=STRING, fare=DOUBLE, _row_key=STRING, end_lat=DOUBLE, begin_lon=DOUBLE, rider=STRING, _hoodie_file_name=STRING, _hoodie_partition_path=STRING, field1=STRING, partition=STRING, driver=STRING, _hoodie_record_key=STRING, _hoodie_commit_seqno=STRING, begin_lat=DOUBLE, end_lon=DOUBLE, timestamp=DOUBLE} {`_hoodie_commit_time`=string, `_hoodie_commit_seqno`=string, `_hoodie_record_key`=string, `_hoodie_partition_path`=string, `_hoodie_file_name`=string, `_row_key`=string, `begin_lat`=double, `begin_lon`=double, `driver`=string, `end_lat`=double, `end_lon`=double, `fare`=double, `partition`=string, `rider`=string, `timestamp`=double} 2019-06-05 10:07:22,919 INFO [main] util.SchemaUtil (SchemaUtil.java:getSchemaDifference(115)) - Difference between schemas: SchemaDifference{deleteColumns=[], updateColumnTypes={}, addColumnTypes={}} 2019-06-05 10:07:22,919 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncSchema(147)) - No Schema difference for hoodie_test 2019-06-05 10:07:22,919 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(98)) - Schema sync complete. Syncing partitions for hoodie_test 2019-06-05 10:07:23,147 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(104)) - Last commit time synced was found to be null 2019-06-05 10:07:23,147 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:getPartitionsWrittenToSince(553)) - Last commit time synced is not known, listing all partitions in file:///tmp/hoodie/sample-table/,FS :org.apache.hadoop.fs.LocalFileSystem@e15b7e8 2019-06-05 10:07:23,240 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(107)) - Storage partitions scan complete. Found 3 2019-06-05 10:07:23,654 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(163)) - New Partitions [] 2019-06-05 10:07:23,654 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:addPartitionsToTable(128)) - No partitions to add for hoodie_test 2019-06-05 10:07:23,654 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(166)) - Changed Partitions [2016/03/15, 2015/03/16, 2015/03/17] 2019-06-05 10:07:23,654 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:updatePartitionsToTable(144)) - Changing partitions 3 on hoodie_test 2019-06-05 10:07:23,658 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(456)) - Executing SQL ALTER TABLE default.hoodie_test PARTITION (field1='2016-03-15') SET LOCATION 'file:/tmp/hoodie/sample-table/2016/03/15' 2019-06-05 10:07:24,473 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(456)) - Executing SQL ALTER TABLE default.hoodie_test PARTITION (field1='2015-03-16') SET LOCATION 'file:/tmp/hoodie/sample-table/2015/03/16' 2019-06-05 10:07:25,245 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(456)) - Executing SQL ALTER TABLE default.hoodie_test PARTITION (field1='2015-03-17') SET LOCATION 'file:/tmp/hoodie/sample-table/2015/03/17' Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_table(Ljava/lang/String;Ljava/lang/String;Lorg/apache/hadoop/hive/metastore/api/Table;Z)V at com.uber.hoodie.hive.HoodieHiveClient.updateLastCommitTimeSynced(HoodieHiveClient.java:588) at com.uber.hoodie.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:111) at com.uber.hoodie.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:72) at com.uber.hoodie.hive.HiveSyncTool.main(HiveSyncTool.java:189)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
