weiqingy opened a new pull request, #28878: URL: https://github.com/apache/flink/pull/28878
## What is the purpose of the change This is the first of four pull requests implementing [FLIP-485: Add UDF Metrics](https://cwiki.apache.org/confluence/spaces/FLINK/pages/373885706/FLIP-485+Add+UDF+Metrics), which was accepted on 2026-08-01 with 3 binding +1 and no -1 ([vote thread](https://lists.apache.org/thread/symqpswsohl2s5wmtkcw0jjp1w5dot0n)). It supersedes the single reference-implementation draft in #28692, which is now closed. FLIP-485 adds opt-in, per-operator observability for SQL/Table user-defined functions so operators can see inside UDF "black boxes" when debugging latency or errors, and so autoscaling can get a reliable "the problem is in user code" signal. This sub-task ([FLINK-40292](https://issues.apache.org/jira/browse/FLINK-40292)) adds only the shared runtime helper, `UdfMetrics`. It owns metric registration, the sampling decision, timing, and exception counting, and it is the piece that both the synchronous and the asynchronous instrumentation paths use. It has no caller yet. The first caller arrives in [FLINK-40293](https://issues.apache.org/jira/browse/FLINK-40293), which adds the configuration options and instruments synchronous scalar and table UDF calls. Splitting it out keeps the sampling and registration logic reviewable on its own. The remaining PRs in the series: 2. [FLINK-40293](https://issues.apache.org/jira/browse/FLINK-40293): `table.exec.udf-metric-enabled` and `table.exec.udf-metric.sample-interval`, plus synchronous scalar and table instrumentation. 3. [FLINK-40294](https://issues.apache.org/jira/browse/FLINK-40294): asynchronous scalar and table instrumentation. 4. [FLINK-40295](https://issues.apache.org/jira/browse/FLINK-40295): documentation. ## Brief change log - Add `UdfMetrics` in `flink-table-runtime` (`org.apache.flink.table.runtime.operators.metrics`, beside `SimpleGauge`). It registers `udfProcessingTime` and `udfExceptionCount` under `addGroup("udf", udfName)` on the operator metric group, so the full identifier is `<operator_name>.udf.<udf_name>.<metric>`. - `udfProcessingTime` is a `DescriptiveStatisticsHistogram` recording per-invocation nanoseconds. It is sampled with the same counter-based scheme as state latency tracking (FLINK-21736): only every Nth invocation is timed, including the `interval == 1` "measure every call" case. - `udfExceptionCount` is a `ThreadSafeSimpleCounter` and is incremented on every exception, not sampled. - The histogram is safe to update from an async callback thread; the sampling counter is only advanced on the task thread at dispatch. ## Verifying this change This change added tests and can be verified as follows: - `UdfMetricsTest` covers metric registration and naming, the sampling decision across the interval boundary, the `sample-interval = 1` case, the counter reset boundary, timing recorded into the histogram, and exception counting. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no. Nothing calls this class yet. The call sites, and their gating, arrive in PRs 2 and 3. - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes, as the first step of FLIP-485 - If yes, how is the feature documented? JavaDocs here. The user-facing documentation in `docs/content{,.zh}/docs/ops/metrics.md` lands in [FLINK-40295](https://issues.apache.org/jira/browse/FLINK-40295). --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Anthropic Claude Opus 4.8 and Claude Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
