[
https://issues.apache.org/jira/browse/HUDI-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17530274#comment-17530274
]
Ethan Guo commented on HUDI-3974:
---------------------------------
We cannot differentiate whether the preCombine field is set by user or write
config with default value after HoodieWriteConfig is constructed, thus we may
not leverage write config to set the right preCombine field in the
hoodie.properties. The problem should be fix at upper layer in HUDI-3979.
> Fix upgrade step wrt precombine field
> -------------------------------------
>
> Key: HUDI-3974
> URL: https://issues.apache.org/jira/browse/HUDI-3974
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.11.0
>
>
> This is related to HUDI-3972
> Details about the bug: if someone is not explicitly setting preCombine field,
> hudi still falls backs to the default value of "ts" and this goes into
> hoodie.properties as well. So, when reading MOR table, since we are
> projecting just the required columns, we also add preCombine field to it and
> if the original table does not have "ts", basic read fails (even if there are
> no log files, but just base files in MOR).
> For the upgrade step, we need to update the preCombine field in the
> hoodie.properties. If the user does not set the preCombine field, it should
> be removed from the hoodie.properties; if the users sets the field, it should
> be updated in the hoodie.properties.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
