waitingF opened a new pull request, #8807: URL: https://github.com/apache/hudi/pull/8807
### Change Logs In our company, hudi is used as a platform. We provide deltastreamer (run in continuous mode) to write to a large number of sources (including mysql & tidb) as a long time service. We often need to update the hudi configuration, but we don’t want to restart deltastreamer to achieve it, which may be too heavy for our job scheduler server based on livy/yarn. Therefore, we provide deltastreamer the configuration hot update function. It is possible to update some common parameters instantly, and these parameters will take effect at the next sync of deltastreamer. Here are some parameters can be updated: - hoodie.bulkinsert.shuffle.parallelism (used only in bulkinsert) - hoodie.upsert.shuffle.parallelism - hoodie.deltastreamer.kafka.source.maxEvents - hoodie.memory.merge.max.size - hoodie.memory.compaction.max.size - hoodie.datasource.hive_sync.* - hoodie.compact.inline.max.delta.commits - hoodie.compaction.strategy - hoodie.compaction.target.io The flowchart of deltastreamer before and after supporting hot update is as follows:  In continuous mode, before each sync, will try update properties if needed ### Impact none if not configure the hot update class ### Risk level (write none, low medium or high below) low ### Documentation Update - add a parameter `--config-hot-update-strategy-class` to set the update class. Default empty('') means no update class and will not turn the feature on. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
