mstahlswrx opened a new issue, #12282:
URL: https://github.com/apache/hudi/issues/12282

   **Describe the problem you faced**
   
   When running the Hudi streamer with the 
https://hudi.apache.org/releases/release-0.14.0#dynamic-configuration-updates 
feature configured, the streamer restarted after a configuration change as 
expected; however, metrics reporting stopped.
   
   I believe the sequence of events are as follows:
   
   Hudi calls 
[streamSync.close()](https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/HoodieStreamer.java?ref_type=heads#L733)
 which ends up [shutting down metrics 
reporting](https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java?ref_type=heads#L1195).
 Since the metrics instances are all [cached in a static 
field](https://code.8labs.io/third-party/github.com/apache/hudi/-/blob/release-0.14.1-scwx/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/Metrics.java#L47),
 when a new `StreamSync` instance is created, it creates a new HoodieMetrics 
instance which 
[reuses](https://code.8labs.io/third-party/github.com/apache/hudi/-/tree/release-0.14.1-scwx/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/HoodieMetrics.java#L95)
 the stopped `Metrics` insta
 nce.
   
   https://github.com/apache/hudi/pull/10132 is a fix for a similar issue, but 
not related to the streamer. (That patch is already in hudi 0.14.)
   
   **To Reproduce**
   
   Steps to reproduce the behavior:
   
   1. Create an implementation of the ConfigurationHotUpdateStrategy class
   2. Start the hudi streamer with the `--config-hot-update-strategy-class` 
option configure with your hot update strategy class
   3. Verify metrics reporting is working
   4. Change a configuration property so that your 
`ConfigurationHotUpdateStrategy.updateProperties` method returns updated 
properties
   5. Wait for the streamer to re-initialize with the updated properties
   4. Observe that metrics reporting no longer works
   
   **Expected behavior**
   
   After the streamer re-initializes metrics reporting continues to work.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to