Duansg opened a new pull request, #3761: URL: https://github.com/apache/hertzbeat/pull/3761
## What's changed? > Optimizing the streaming parsing performance of single metrics in Prometheus In the current parsing logic, when `protocol = http` and `parseType = prometheus`, `org.apache.hertzbeat.collector.collect.http.HttpCollectImpl#parseResponseByPrometheusExporter` processes individual metric information from the configuration template. However, in its processing logic, it requires aggregating all metric information—even if your metric appears in the first row of the exposed metrics. Therefore, I believe it is sufficient to focus solely on the current metric information. Other metric information can be parsed on demand according to configuration, thereby avoiding unnecessary memory consumption from currently irrelevant metrics. For details: - [x] 1. Add streaming parsing for single metrics in `OnlineParser` - [x] 2. `parseResponseByPrometheusExporter` Code Structure Optimization - [x] 3. Add test cases and pass historical test cases. ## Checklist - [x] I have read the [Contributing Guide](https://hertzbeat.apache.org/docs/community/code_style_and_quality_guide) - [ ] I have written the necessary doc or comment. - [x] I have added the necessary unit tests and all cases have passed. ## Add or update API - [ ] I have added the necessary [e2e tests](https://github.com/apache/hertzbeat/tree/master/e2e) and all cases have passed. > Parsing time In actual measurements of `812KB`'s metric data, if the parsed metric is the last row, even though stream parsing progresses incrementally, the overall parsing time still increases by `110ms+`. ``` // old 2025-09-09 21:14:49 [552711657229568-custom1-promhttp_metric_handler_requests_total-8920] INFO org.apache.hertzbeat.collector.dispatch.MetricsCollect - [Collect Success, Run 563ms, All 564ms]. // new 2025-09-09 21:20:52 [552711657229568-custom1-promhttp_metric_handler_requests_total-2122] INFO org.apache.hertzbeat.collector.dispatch.MetricsCollect - [Collect Success, Run 432ms, All 432ms]. ``` > Memory usage Sample Data: With unchanged collection settings, memory changes recorded at each collection point 5 minutes after startup. - old <img width="407" height="66" alt="Snipaste_2025-09-09_21-42-52" src="https://github.com/user-attachments/assets/b653fa8b-8d0f-44dd-bf04-38f0fd4f7124"; /> - new <img width="385" height="71" alt="Snipaste_2025-09-09_21-35-03" src="https://github.com/user-attachments/assets/fdc810d0-02bf-433d-9df2-7467e7abb862"; /> > Metric rendering - Custom Template <img width="1496" height="429" alt="Snipaste_2025-09-09_21-50-29" src="https://github.com/user-attachments/assets/90726381-3b21-4c24-af4e-554c17219925"; /> - Existing Templates <img width="1330" height="833" alt="Snipaste_2025-09-09_21-49-55" src="https://github.com/user-attachments/assets/b10db89c-34f3-487a-a137-eb056ce5bce1"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@hertzbeat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: notifications-unsubscr...@hertzbeat.apache.org For additional commands, e-mail: notifications-h...@hertzbeat.apache.org
