nsivabalan commented on PR #6216: URL: https://github.com/apache/hudi/pull/6216#issuecomment-1205950765
we added support to update hoodie.properties in a live env. mainly to update some table propertlies like metadata related props (list of partitions in metadata table). So, here is how upgrade works so that its fault tolerant and recoverable. orig.hoodie.properties Step1: take back up. cp orig.hoodie.properties backup.hoodie.properties. Step2: delete orig.hoodie.properties Step3: create new hoodie.properties in memory w/ any new properties required. create orig.hoodie.properties. Step4: delete backup.hoodie.properties. b/w step2 and step3, readers will read backup.hoodie.properties. Above is designed such that, if there is a crash at any point, we are safe and restarting the pipeline would suffice. ref: https://github.com/apache/hudi/blob/a75cc02273ae87c383ae1ed46f95006c366f70fc/hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java#L344 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
