[
https://issues.apache.org/jira/browse/HUDI-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-8420:
--------------------------------------
Fix Version/s: 1.0.2
> Return all fields as mandatory for custom merging
> -------------------------------------------------
>
> Key: HUDI-8420
> URL: https://issues.apache.org/jira/browse/HUDI-8420
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Y Ethan Guo
> Assignee: Jonathan Vexler
> Priority: Major
> Fix For: 1.0.1, 1.0.2
>
>
> [https://github.com/apache/hudi/pull/11943#discussion_r1805674206]
> The default implementation of {{getMandatoryFieldsForMerging}} should return
> all columns since custom merging may need all columns regardless of the
> request schema. Only the {{DefaultSparkRecordMerger}} and
> {{{}OverwriteWithLatestSparkRecordMerger{}}}(and counterpart classes for
> other engines) should overwrite the implementation to return record key and
> optional preCombine field for better performance with projection.
>
> Right now the user must provide {{getMandatoryFieldsForMerging}} to make sure
> the functionality is correct, which is OK as long as we update the docs
> (HUDI-8421). For better dev experience, we should not require user to
> override this unless the user understands the tradeoff here.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
