Hi Qian, I have faced this error before. It generally happens if your hive server and client(in this case hudi) are on different versions. What version of hive server are you using, because hudi is compiled against Hive 2.3.X.
Regards, Gurudatt On Wednesday, October 23, 2019, Qian Wang <[email protected]> wrote: > Hi, > > I tried using Datasource Write to sync to Hive, following is my code: > dfInput > .write.format(“org.apache.hudi”) > .mode(“append”) > .option("path", outputBase) > .option(HoodieWriteConfig.TABLE_NAME, “hudi_upsert_mor”) > .option(DataSourceWriteOptions.RECORDKEY_FIELD_OPT_KEY, "item_id") // This is the record key > .option(DataSourceWriteOptions.PRECOMBINE_FIELD_OPT_KEY, "timestamp") // use to combine duplicate records in input/with disk val > .option(DataSourceWriteOptions.OPERATION_OPT_KEY, DataSourceWriteOptions.UPSERT_OPERATION_OPT_VAL) // upsert > .option(DataSourceWriteOptions.STORAGE_TYPE_OPT_KEY, DataSourceWriteOptions.MOR_STORAGE_TYPE_OPT_VAL) // Hoodie Table Type > .option(DataSourceWriteOptions.PARTITIONPATH_FIELD_OPT_KEY, “hour”) > .option(DataSourceWriteOptions.TABLE_NAME_OPT_KEY, "test_item_upsert") // Used by hive sync and queries > .option(DataSourceWriteOptions.INSERT_DROP_DUPS_OPT_KEY, "true") // hoodie.datasource.write.insert.drop.duplicates > .option(DataSourceWriteOptions.HIVE_SYNC_ENABLED_OPT_KEY, “true”) > .option(DataSourceWriteOptions.HIVE_DATABASE_OPT_KEY, “hudi_poc”) > .option(DataSourceWriteOptions.HIVE_TABLE_OPT_KEY, “test_item_upsert”) > .option(DataSourceWriteOptions.HIVE_URL_OPT_KEY, "jdbc:hive2://zk-1.vip.hadoop.com:2181”) > .option(DataSourceWriteOptions.HIVE_PARTITION_FIELDS_OPT_KEY, “hour”) > .save() > > What are the difference among HoodieWriteConfig.TABLE_NAME, DataSourceWriteOptions.TABLE_NAME_OPT_KEY, DataSourceWriteOptions.HIVE_TABLE_OPT_KEY? > Also, I got following error, how can I solve this? > BTW, this Hive is authenticated by Kerberos. > > 19/10/22 15:41:47 ERROR HiveConnection: Error opening session > org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=default}) > > 19/10/22 15:41:47 ERROR Utils: Unable to read HiveServer2 configs from ZooKeeper > org.apache.hudi.hive.HoodieHiveSyncException: Cannot create hive connection jdbc:hive2://zk-1.vip.hadoop.com:2181 > > Caused by: java.sql.SQLException: Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper > at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:221) > at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176) > at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) > at java.sql.DriverManager.getConnection(DriverManager.java:664) > at java.sql.DriverManager.getConnection(DriverManager.java:247) > at org.apache.hudi.hive.HoodieHiveClient.createHiveConnection(HoodieHiveClient.java:570) > ... 77 more > Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 uri from ZooKeeper > at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getNextServerUriFromZooKeeper(ZooKeeperHiveClientHelper.java:86) > at org.apache.hive.jdbc.Utils.updateConnParamsFromZooKeeper(Utils.java:532) > at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:217) > ... 82 more > Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Tried all existing HiveServer2 uris from ZooKeeper. > > > Best, > Qian >
