Thanks for all your inputs . I am able to plugin the custom class and do cell level merge for my usecase.
Thanks, Selva On Tue, Feb 25, 2020 at 5:50 PM leesf <[email protected]> wrote: > You would pass it via option, like > option(DataSourceWriteOptions.PAYLOAD_CLASS_OPT_KEY(), > EmptyHoodieRecordPayload.class.getName()) > > selvaraj periyasamy <[email protected]> 于2020年2月26日周三 > 上午2:24写道： > > > OverwriteWithLatestAvroPayload is used for Delta Streamer. Is there a way > > for DataSource Writer? > > > > please correct me , if I am wrong. > > > > Thanks, > > Selva > > > > > > On Mon, Feb 24, 2020 at 1:15 PM Gary Li <[email protected]> > wrote: > > > > > Hi, in this case you need to design your own logic to handle merging. > > > Please check OverwriteWithLatestAvroPlayload class. You can write your > > own > > > one and pass it as DataSourceWriteOptions.PAYLOAD_CLASS_OPT_KEY to > Hudi. > > > > > > On Mon, Feb 24, 2020 at 12:25 PM selvaraj periyasamy < > > > [email protected]> wrote: > > > > > > > Hi, I am experimenting Hudi 0.5.0 version for some of the update use > > > cases. > > > > > > > > Our flow is as below > > > > > > > > RDBMS -> CDC Log -> Hive -> COW table. > > > > > > > > CCDC log for update would have the value only for the primary key > > > columns + > > > > updated columns. Remaining column values are null. While upserting > > values > > > > on CCOW table, would need to update only the column values , which > are > > > > updated and retain the values for other columns. When I tested, Hudi > > > > updates remaining column values as null since log has null values. > > > > > > > > Is there a way to merge rows for columns which are having values > during > > > > update? > > > > > > > > Thanks, > > > > Selva > > > > > > > > > >
