rangareddy commented on issue #5326: URL: https://github.com/apache/hudi/issues/5326#issuecomment-5102636010
This issue was reviewed as part of the JIRA-migrated backlog triage. Findings: This is now supported. Custom labels can be attached to pushgateway metrics, which is what was asked for here. Your 2022-04-17 diagnosis was right: `PushGatewayReporter` used `DropwizardExports`, which emits no additional tags. That has since changed. On current Hudi, `PushGatewayReporter` carries a `Map<String, String> labels` field and passes it as the Prometheus grouping key -- `pushGatewayClient.pushAdd(collectorRegistry, jobName, labels)` (`hudi-common/src/main/java/org/apache/hudi/metrics/prometheus/PushGatewayReporter.java:114`) -- plus per-metric label resolution in `handleLabeledMetrics()` (line 142). The label convention is implemented in `hudi-common/src/main/java/org/apache/hudi/metrics/MetricUtils.java`: a metric name may carry labels appended after a `;`, with labels comma-separated and key/value colon-separated: ``` metric_name;job_name:xxx,group:a ``` So emitting `commit_totalBytesWritten;job_name:xxx` produces the metric with a `job_name="xxx"` label, which distinguishes metrics from different jobs exactly as you wanted. Validated against the release tags -- the `labels` field is absent in `release-0.12.0` and `release-0.14.0` and present from **`release-1.0.0`** onward. So you need 1.0.0 or later. Closing as fixed. If the label convention does not work for your setup on 1.0.0+, please reopen with the reporter config and the metric names you emit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
