Weiqing Yang created FLINK-40293:
------------------------------------
Summary: Add UDF metric config options and instrument sync scalar
and table UDF calls
Key: FLINK-40293
URL: https://issues.apache.org/jira/browse/FLINK-40293
Project: Flink
Issue Type: Sub-task
Components: Table SQL / API, Table SQL / Planner
Reporter: Weiqing Yang
This is the second of four sub-tasks implementing FLIP-485.
FLIP:
https://cwiki.apache.org/confluence/spaces/FLINK/pages/373885706/FLIP-485+Add+UDF+Metrics
Vote thread: https://lists.apache.org/thread/symqpswsohl2s5wmtkcw0jjp1w5dot0n
Add the two configuration options and the first end-to-end slice of the feature:
synchronous scalar and table UDF calls instrumented at code generation.
Configuration, both {{@PublicEvolving}} additions to ExecutionConfigOptions:
* {{table.exec.udf-metric-enabled}}, default false, enables the feature.
* {{table.exec.udf-metric.sample-interval}}, default 100, measures every Nth
invocation.
Instrumentation:
* Wrap the generated {{eval}} call site for synchronous scalar and table UDFs in
BridgingFunctionGenUtil with sampled timing and exception counting via the
UdfMetrics
helper.
* Cache one handle per (operator, udf name) in CodeGeneratorContext, so
repeated call
sites of the same function within one operator share a handle. Without
sharing, the
second registration is dropped and only one call site's timings survive.
* Meter only user functions on the modern BridgingSqlFunction stack, through an
opt-in
name passed from BridgingSqlFunctionCallGen. Lookup-join, ML-predict,
vector-search,
legacy CallGens, and PROCESS_TABLE functions are not metered.
The instrumentation is emitted at code generation only when the feature is
enabled, so
the generated operator is byte-identical to today when it is disabled. When
enabled, the
non-sampled path costs a single integer increment. Exceptions are counted on
every
invocation and are not sampled.
Covered by a new UdfMetricsITCase using InMemoryReporter: sync scalar and table
functions, metric naming and scope, exception counting, the enabled/disabled
gate, a
lookup join not being metered, handle sharing across repeated call sites versus
separate
handles for distinct functions, and udfProcessingTime reflecting a real induced
delay.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
