[ https://issues.apache.org/jira/browse/HUDI-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pratyaksh Sharma reassigned HUDI-2719: -------------------------------------- Assignee: Pratyaksh Sharma (was: Nishith Agarwal) > Add back support for copying over extra metadata from previous commit > ---------------------------------------------------------------------- > > Key: HUDI-2719 > URL: https://issues.apache.org/jira/browse/HUDI-2719 > Project: Apache Hudi > Issue Type: Improvement > Reporter: sivabalan narayanan > Assignee: Pratyaksh Sharma > Priority: Major > Fix For: 0.11.0 > > > Add back support for copying over extra metadata from previous commit . > We had a config named 'hoodie.write.meta.key.prefixes'. when this config is > added, hudi copies over any extra metadata from previous commit with matching > key prefixes as per `hoodie.write.meta.key.prefixes`. for eg: if > `hoodie.write.meta.key.prefixes` set to `abc` and if there are other config > options as "abc.key1" and "abc.key2", these two will be copied over from > previous commit metadata and get overridden with current writer's commit > metadata. > > Our initial understanding is that this was added to support copying over > delta streamer checkpoint for multi -writer scenairo. But since the config is > not very straight forward to use, we simplified and introduced a new config > as per [https://github.com/apache/hudi/pull/3820.] > Added the old one "hoodie.write.meta.key.prefixes" as an alternative. Since > multi-writer in itself is still evolving, we felt simplifying the config for > deltastreamer checkpoint makes sense. > But if someone out there was using the older config to copy some other extra > metadata for debugging or some other purposes, we need to add the support > back. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)