hudi-bot opened a new issue, #17349: URL: https://github.com/apache/hudi/issues/17349
*Problem Statement:* When running the Hudi streamer with the [https://hudi.apache.org/releases/release-0.14.0#dynamic-configuration-updates] feature configured, the streamer restarted after a configuration change as expected; however, metrics reporting stopped. I believe the sequence of events are as follows: Hudi calls [streamSync.close()|https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/HoodieStreamer.java?ref_type=heads#L733] which ends up [shutting down metrics reporting|https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java?ref_type=heads#L1195]. Since the metrics instances are all [cached in a static field|https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/Metrics.java#L47], when a new {{StreamSync}} instance is created, it creates a new HoodieMetrics instance which [reuses|https://code.8labs.io/third-party/github.com/apache/hudi/-/tree/release-0.14.1-scwx/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/HoodieMetrics.java#L95] the stopped {{Metric s}} instance. [#10132|https://github.com/apache/hudi/pull/10132] is a fix for a similar issue, but not related to the streamer. (That patch is already in hudi 0.14.) *Reproduce Steps:* Steps to reproduce the behavior: * Create an implementation of the ConfigurationHotUpdateStrategy class * Start the hudi streamer with the {{--config-hot-update-strategy-class}} option configure with your hot update strategy class * Verify metrics reporting is working * Change a configuration property so that your {{ConfigurationHotUpdateStrategy.updateProperties}} method returns updated properties * Wait for the streamer to re-initialize with the updated properties * Observe that metrics reporting no longer works *Expected behavior* After the streamer re-initializes metrics reporting continues to work. *Github Issue:* * [https://github.com/apache/hudi/issues/12282] * [https://github.com/apache/hudi/issues/12282#issuecomment-2536237910] ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-8778 - Type: Sub-task - Parent: https://issues.apache.org/jira/browse/HUDI-9113 - Affects version(s): - 0.14.1 - 0.15.0 - Fix version(s): - 1.1.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
