weiqingy opened a new pull request, #28879: URL: https://github.com/apache/flink/pull/28879
## What is the purpose of the change This is the second of four pull requests implementing [FLIP-485: Add UDF Metrics](https://cwiki.apache.org/confluence/spaces/FLINK/pages/373885706/FLIP-485+Add+UDF+Metrics), accepted on 2026-08-01 ([vote thread](https://lists.apache.org/thread/symqpswsohl2s5wmtkcw0jjp1w5dot0n)). It follows #28878. It adds the two configuration options and the first end-to-end slice of the feature: synchronous scalar and table UDF calls are instrumented at code generation, using the `UdfMetrics` helper added in [FLINK-40292](https://issues.apache.org/jira/browse/FLINK-40292) (#28878). Two metrics are registered on the executing operator's `OperatorMetricGroup`, scoped as `<operator_name>.udf.<udf_name>.<metric>`: - `udfProcessingTime`, a Histogram of per-invocation UDF time. - `udfExceptionCount`, a Counter of exceptions that escape user code. The feature is off by default (`table.exec.udf-metric-enabled = false`) with zero overhead when disabled. The instrumentation is emitted at code generation only when the option is on, so the generated operator is byte-identical to today when it is off. When on, only every Nth invocation is timed (`table.exec.udf-metric.sample-interval`, default 100), while exceptions are counted on every invocation. This PR is stacked on #28878 and is kept as a draft until that one merges, so its diff currently also shows the `UdfMetrics` commit. Asynchronous instrumentation follows in [FLINK-40294](https://issues.apache.org/jira/browse/FLINK-40294), and the user-facing documentation in [FLINK-40295](https://issues.apache.org/jira/browse/FLINK-40295). ## Brief change log - Add two `@PublicEvolving` options to `ExecutionConfigOptions`: `table.exec.udf-metric-enabled` (default false) and `table.exec.udf-metric.sample-interval` (default 100), plus the regenerated config docs. - Instrument the generated `eval` call site for synchronous scalar and table UDFs in `BridgingFunctionGenUtil`, bracketing it with sampled timing and exception counting. - Cache one `UdfMetrics` handle per `(operator, udf name)` in `CodeGeneratorContext`, so repeated call sites of the same function in one operator share a handle instead of the second registration being dropped. - Meter only user functions on the modern `BridgingSqlFunction` stack, via an opt-in name passed from `BridgingSqlFunctionCallGen`. Lookup-join, ML-predict, vector-search, legacy `CallGen`s, and `PROCESS_TABLE` functions are not metered. ## Verifying this change This change added tests and can be verified as follows: - `UdfMetricsITCase` (integration, using `InMemoryReporter`) covers synchronous scalar and table functions, metric naming and scope, exception counting, the enabled/disabled gate, that a lookup join is not metered, that repeated call sites of one function share a single handle while distinct functions get separate ones, and that `udfProcessingTime` reflects a real induced delay rather than only a sample count. - `ConfigOptionsDocsCompletenessITCase` and the regenerated config docs verify the two new options are documented. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes, two new `@PublicEvolving` `ConfigOption`s in `ExecutionConfigOptions`. - The serializers: no - The runtime per-record code paths (performance sensitive): yes. The instrumentation wraps the generated UDF call site. It is gated at code generation, so the generated code is byte-identical when the feature is disabled, and counter-sampled when enabled, so the non-sampled path costs a single integer increment. - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? JavaDocs and the generated config option docs here. The `docs/content{,.zh}/docs/ops/metrics.md` section lands in [FLINK-40295](https://issues.apache.org/jira/browse/FLINK-40295). --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Anthropic Claude Opus 4.8 and Claude Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
