Izeren opened a new pull request, #27989: URL: https://github.com/apache/flink/pull/27989
## What is the purpose of the change Part 3 of [FLIP-553](https://cwiki.apache.org/confluence/display/FLINK/FLIP-553). Large OpenTelemetry metric attribute values can be rejected by OTel collectors as oversized payloads. This PR adds two configuration options to the OpenTelemetry metric reporter to keep attribute values within safe bounds and to track post-truncation metric name collisions without unbounded heap growth. ## Brief change log - New option `transform.attribute-value-length-limits.<attribute-name>` (with `*` as the global key) truncates metric attribute values to the configured length. `0` drops the attribute, negative values disable the limit for that attribute, and the `*` global key applies to any attribute without an explicit entry. - New option `transform.collision-tracking-max-slots` (default `50000`) bounds the memory footprint of the post-truncation collision tracker via an access-ordered LRU. `0` disables collision tracking. Malformed or negative values `WARN` and fall back to the default — a typo should not tear down metric reporting at JobManager startup. - New package-private `MetricAttributeTransformer` applies the limits and tracks collisions. Serialized via the existing reporter lock (`@NotThreadSafe`). - `OpenTelemetryMetricReporter.open()` builds the transformer from reporter config; `notifyOfAddedMetric` applies it before registration. - Options live in `OpenTelemetryReporterOptions`, which is shared with the trace and event reporters. Their descriptions explicitly state that these two options apply only to the metric reporter and are ignored by the trace and event reporters. - Empty-string attribute values are preserved under any non-zero configured limit. ## Verifying this change This change added tests and can be verified as follows: - Unit tests in `MetricAttributeTransformerTest` covering: empty attribute values preserved under non-zero limits, access-ordered LRU hot-slot survives eviction, bounded slot tracking under `maxSlots * 4` distinct-slot load, malformed `NumberFormatException` path falls back to default, negative `max-slots` falls back to default, global vs per-attribute limit interaction, `0` drop semantics, and negative-limit disable semantics. - Unit tests in `OpenTelemetryMetricReporterTest` covering the reporter-level wiring. - Integration test `OpenTelemetryMetricReporterITCase#testAttributeValueTruncation` verifying truncated attribute values reach the exporter. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no (the new `ConfigOption`s are `@PublicEvolving`; the transformer class is package-private) - The serializers: no - The runtime per-record code paths (performance sensitive): no (metric reporter registration path only) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? docs (option descriptions regenerated into `docs/layouts/shortcodes/generated/open_telemetry_reporter_configuration.html`) and JavaDocs on `MetricAttributeTransformer` and the new `ConfigOption`s in `OpenTelemetryReporterOptions`. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Opus 4.7) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
