rex xiong created HUDI-4447:
-------------------------------
Summary: Hive Sync fails fails when performing delete table data
operation
Key: HUDI-4447
URL: https://issues.apache.org/jira/browse/HUDI-4447
Project: Apache Hudi
Issue Type: Bug
Components: meta-sync
Environment: Spark3.2.1 & Hudi 0.11.1
Reporter: rex xiong
Assignee: rex xiong
Fix For: 0.12.0
currently, sync meta would fail if performing delete table data operation,
because sync without setting database name/fields/extractor class, etc.This
causes two problems：
1）sync schema will recreate table in default database
2）sync will use wrong extractor class(HiveStylePartitionValueExtractor) when
syncing non partitioned table with metadata table enabled
{code:java}
spark-sql> delete from test_db.hudi_mor_none_part_table_321_0111 where id=1;
org.apache.hudi.exception.HoodieException: Could not sync using the meta sync
class org.apache.hudi.hive.HiveSyncTool
at
org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:61)
at
org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2(HoodieSparkSqlWriter.scala:626)
at
org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2$adapted(HoodieSparkSqlWriter.scala:625)
at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
....
Caused by: org.apache.hudi.exception.HoodieException: Got runtime exception
when hive syncing hudi_mor_none_part_table_321_0111
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:143)
at
org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:59)
... 93 more
Caused by: org.apache.hudi.hive.HoodieHiveSyncException: Failed to sync
partitions for table hudi_mor_none_part_table_321_0111_ro
at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:418)
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:232)
at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:156)
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:140)
... 94 more
Caused by: java.lang.IllegalArgumentException: Partition path is not in the
form partition_key=partition_value.
at
org.apache.hudi.hive.HiveStylePartitionValueExtractor.extractPartitionValuesInPath(HiveStylePartitionValueExtractor.java:37)
at
org.apache.hudi.hive.AbstractHiveSyncHoodieClient.getPartitionEvents(AbstractHiveSyncHoodieClient.java:81)
at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:396)
... 97 more
org.apache.hudi.exception.HoodieException: Could not sync using the meta sync
class org.apache.hudi.hive.HiveSyncTool
at
org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:61)
at
org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2(HoodieSparkSqlWriter.scala:626)
at
org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2$adapted(HoodieSparkSqlWriter.scala:625)
at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
at
org.apache.hudi.HoodieSparkSqlWriter$.metaSync(HoodieSparkSqlWriter.scala:625)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
