voon created HUDI-6202:
--------------------------
Summary: Add extra validation before properties is used
Key: HUDI-6202
URL: https://issues.apache.org/jira/browse/HUDI-6202
Project: Apache Hudi
Issue Type: Improvement
Reporter: voon
Assignee: voon
In the issue linked below, we found that tables that were upgraded from Hudi
0.7 do not have the *hoodie.datasource.write.hive_style_partitioning* key in
the table configs or {*}hoodie.properties{*}.
[https://github.com/apache/hudi/issues/8653]
As such, when using lazy partition pruning + tryListByPartitionPathPrefix, hudi
might construct a partition path that does not exist, causing a partition to be
created.
While this ticket is not aimed at fixing the issue, we propose that an
additional validation on whether the key exists is performed before the value
of the config is used.
{*}TLDR{*}: Instead of blindly falling back to defaults, we should check if the
config key exists in table properties before ACTUALLY using it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
