deepakpanda93 commented on issue #16109: URL: https://github.com/apache/hudi/issues/16109#issuecomment-4854923421
This is resolved as of Hudi 1.1.0. The failure was specific to the Spark record merger (HoodieSparkRecordMerger) on the global-index + update-partition-path path — the default Avro merger worked, as confirmed on the original [#9190]. In 1.1.0 that path was reworked to be engine-agnostic: HUDI-9602 / [#13600](https://github.com/apache/hudi/pull/13600) ("Use BufferedRecordMerger for dedup and global index path"). HoodieIndexUtils.mergeForPartitionUpdatesAndDeletionsIfNeeded now performs the partition-update merge through a HoodieReaderContext + record merge mode rather than the Avro-vs-Spark record-merger split that caused this, so the Spark record type no longer hits a merger-specific failure. (Explicit record-merger impls are also deprecated in favor of hoodie.record.merge.mode as of 1.1.0, HUDI-9713.) Please retest your global-index + update-partition-path upsert with the Spark record type on 1.1.x. Closing as fixed in 1.1.0 — if you still see it fail there, please reopen with the repro and stack. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
