Ma Jian created HUDI-8001:
-----------------------------
Summary: Insert overwrite failed due to missing 'path' property
when using Spark 3.5.1 and Hudi 1.0.0
Key: HUDI-8001
URL: https://issues.apache.org/jira/browse/HUDI-8001
Project: Apache Hudi
Issue Type: Bug
Reporter: Ma Jian
The issue with Spark 3.5.1 arises because the {{InsertIntoHoodieTableCommand}}
chain calls the initialization of the {{HoodieFileIndex}} class. For v1 tables,
the path is stored in
{{{}CatalogTable#CatalogStorageFormat#storageProperties{}}}, but not in
{{{}CatalogTable#properties{}}}. When Spark reloads the table, it removes the
path key from {{{}CatalogTable#CatalogStorageFormat#storageProperties{}}}.
Consequently, {{InsertIntoHoodieTableCommand}} in Hudi cannot retrieve the path
from either {{CatalogTable#CatalogStorageFormat#storageProperties}} or
{{CatalogTable#properties}} during {{{}deduceOverwriteConfig{}}}. This absence
of the path key in {{combinedOpts}} leads to an error when initializing
{{{}HoodieFileIndex{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
