Hi, in this case you need to design your own logic to handle merging. Please check OverwriteWithLatestAvroPlayload class. You can write your own one and pass it as DataSourceWriteOptions.PAYLOAD_CLASS_OPT_KEY to Hudi.
On Mon, Feb 24, 2020 at 12:25 PM selvaraj periyasamy < [email protected]> wrote: > Hi, I am experimenting Hudi 0.5.0 version for some of the update use cases. > > Our flow is as below > > RDBMS -> CDC Log -> Hive -> COW table. > > CCDC log for update would have the value only for the primary key columns + > updated columns. Remaining column values are null. While upserting values > on CCOW table, would need to update only the column values , which are > updated and retain the values for other columns. When I tested, Hudi > updates remaining column values as null since log has null values. > > Is there a way to merge rows for columns which are having values during > update? > > Thanks, > Selva >
