[
https://issues.apache.org/jira/browse/HUDI-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-8001:
---------------------------------
Labels: pull-request-available (was: )
> Insert overwrite failed due to missing 'path' property when using Spark 3.5.1
> and Hudi 1.0.0
> --------------------------------------------------------------------------------------------
>
> Key: HUDI-8001
> URL: https://issues.apache.org/jira/browse/HUDI-8001
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ma Jian
> Priority: Major
> Labels: pull-request-available
>
> The issue with Spark 3.5.1 arises because the
> {{InsertIntoHoodieTableCommand}} chain calls the initialization of the
> {{HoodieFileIndex}} class. For v1 tables, the path is stored in
> {{{}CatalogTable#CatalogStorageFormat#storageProperties{}}}, but not in
> {{{}CatalogTable#properties{}}}. When Spark reloads the table, it removes the
> path key from {{{}CatalogTable#CatalogStorageFormat#storageProperties{}}}.
> Consequently, {{InsertIntoHoodieTableCommand}} in Hudi cannot retrieve the
> path from either {{CatalogTable#CatalogStorageFormat#storageProperties}} or
> {{CatalogTable#properties}} during {{{}deduceOverwriteConfig{}}}. This
> absence of the path key in {{combinedOpts}} leads to an error when
> initializing {{{}HoodieFileIndex{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
