j1wonpark opened a new pull request, #4084:
URL: https://github.com/apache/amoro/pull/4084
## Why are the changes needed?
Currently, the Prometheus exporter exposes all registered metrics (~61
metrics) unconditionally. As the number of tables grows, the volume of metrics
increases significantly. Users need the ability to selectively enable/disable
metric categories to reduce noise and focus on the metrics that matter to their
use case.
## Brief change log
- Added `MetricCategory` enum in `amoro-metrics-prometheus` module to define
5 metric categories (`self-optimizing`, `optimizer-group`, `orphan-files`,
`ams-jvm`, `table-summary`) with their corresponding metric name prefixes
- Modified `MetricsCollector` to accept a set of disabled categories and
filter them out in `isValidMetric()`
- Modified `PrometheusMetricsReporter` to parse category settings from
`properties` in `open()` and pass them to `MetricsCollector`
- Updated `metric-reporters.yaml` with category filtering configuration
examples
- Updated Helm chart `values.yaml` and `amoro-configmap_test.yaml` to
reflect the new configuration options
### Configuration example:
```yaml
metric-reporters:
- name: prometheus-exporter
enabled: true
properties:
port: 7001
category.self-optimizing.enabled: "true"
category.optimizer-group.enabled: "true"
category.orphan-files.enabled: "false"
category.ams-jvm.enabled: "true"
category.table-summary.enabled: "false"
```
All categories are enabled by default for backward compatibility.
## How was this patch tested?
- [x] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
- `MetricCategoryTest` (10 tests): category matching, properties parsing,
case-insensitive values, invalid values
- `MetricsCollectorFilterTest` (5 tests): no filter, single/multiple
disabled categories, unknown metrics not filtered, orphan-files multi-prefix
filtering
- [ ] Add screenshots for manual tests if appropriate
- [x] Run test locally before making a pull request
- `./mvnw test -pl amoro-metrics/amoro-metrics-prometheus -am` — 15 tests
passed
- `helm unittest charts/amoro` — 74 tests passed
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs (metric-reporters.yaml
comments and Helm values.yaml comments)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
