[
https://issues.apache.org/jira/browse/HUDI-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264868#comment-17264868
]
Trevorzhang commented on HUDI-1528:
-----------------------------------
21/01/14 20:21:00 INFO hive.HoodieHiveClient: Creating table with CREATE
EXTERNAL TABLE IF NOT EXISTS `datalake_taxi`.`xx_biz_operation_t_city2`(
`_hoodie_commit_time` string, `_hoodie_commit_seqno` string,
`_hoodie_record_key` string, `_hoodie_partition_path` string,
`_hoodie_file_name` string, `ums_id_` string, `ums_ts_` string, `ums_op_`
string, `uuid` string, `org_code` string, `org_name` string, `parent_code`
string, `center` string, `flag` int, `create_time` string, `update_time`
string) PARTITIONED BY (`ds` string) ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS
INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION
'hdfs://xx.3.98.183:8020/data_lake/jqxx/data/xx_biz_operation.t_city'
21/01/14 20:21:00 INFO hive.HoodieHiveClient: Creating table with CREATE
EXTERNAL TABLE IF NOT EXISTS `datalake_taxi`.`xx_biz_operation_t_city2`(
`_hoodie_commit_time` string, `_hoodie_commit_seqno` string,
`_hoodie_record_key` string, `_hoodie_partition_path` string,
`_hoodie_file_name` string, `ums_id_` string, `ums_ts_` string, `ums_op_`
string, `uuid` string, `org_code` string, `org_name` string, `parent_code`
string, `center` string, `flag` int, `create_time` string, `update_time`
string) PARTITIONED BY (`ds` string) ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS
INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION
'hdfs://xx.3.98.183:8020/data_lake/jqxx/data/xx_biz_operation.t_city'21/01/14
20:21:00 INFO hive.HoodieHiveClient: Executing SQL *CREATE EXTERNAL TABLE IF
NOT EXISTS `datalake_taxi`.`xx_biz_operation_t_city2`*( `_hoodie_commit_time`
string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string,
`_hoodie_partition_path` string, `_hoodie_file_name` string, `ums_id_` string,
`ums_ts_` string, `ums_op_` string, `uuid` string, `org_code` string,
`org_name` string, `parent_code` string, `center` string, `flag` int,
`create_time` string, `update_time` string) PARTITIONED BY (`ds` string) ROW
FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
'hdfs://xx.3.98.183:8020/data_lake/jqxx/data/xx_biz_operation.t_city'21/01/14
20:21:00
INFO hive.HiveSyncTool: Schema sync complete. Syncing partitions for
xx_biz_operation_t_city2
21/01/14 20:21:00 INFO hive.HiveSyncTool: Last commit time synced was found to
be null21/01/14 20:21:00
INFO common.AbstractSyncHoodieClient: Last commit time synced is not known,
listing all partitions in
hdfs://xx.3.98.183:8020/data_lake/jqxx/data/xx_biz_operation.t_city,FS
:DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_1717621797_1, ugi=lingqu
(auth:SIMPLE)]]21/01/14 20:21:00
INFO hive.HiveSyncTool: Storage partitions scan complete. Found
21/01/14 20:21:00 *ERROR* hive.HiveSyncTool: Got runtime exception when hive
syncingorg.apache.hudi.hive.HoodieHiveSyncException: Failed to sync partitions
for table xx_biz_operation_t_city2 at
org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:206) at
org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:142) at
org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:94) at
org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:226)
Caused by:
*NoSuchObjectException*(message:datalake_taxi.xx_biz_operation_t_city2 table
*not found*) at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_result$get_partitions_resultStandardScheme.read(ThriftHiveMetastore.java)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_result$get_partitions_resultStandardScheme.read(ThriftHiveMetastore.java)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_result.read(ThriftHiveMetastore.java)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partitions(ThriftHiveMetastore.java:2289)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partitions(ThriftHiveMetastore.java:2274)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.listPartitions(HiveMetaStoreClient.java:1384)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:154)
at com.sun.proxy.$Proxy18.listPartitions(Unknown Source) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2562)
at com.sun.proxy.$Proxy18.listPartitions(Unknown Source) at
org.apache.hudi.hive.HoodieHiveClient.scanTablePartitions(HoodieHiveClient.java:236)
at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:196)
> hudi-sync-tool error
> --------------------
>
> Key: HUDI-1528
> URL: https://issues.apache.org/jira/browse/HUDI-1528
> Project: Apache Hudi
> Issue Type: Bug
> Components: Hive Integration
> Reporter: Trevorzhang
> Assignee: Trevorzhang
> Priority: Major
> Fix For: 0.8.0
>
>
> When using hudi-sync-tools to synchronize to a remote hive, hivemetastore
> will throw exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
