[
https://issues.apache.org/jira/browse/FLINK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiqing Yang updated FLINK-41585:
---------------------------------
Priority: Major (was: Not a Priority)
> Refine UDF metrics registration
> -------------------------------
>
> Key: FLINK-41585
> URL: https://issues.apache.org/jira/browse/FLINK-41585
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner, Table SQL / Runtime
> Affects Versions: 2.4.0
> Reporter: Weiqing Yang
> Assignee: Weiqing Yang
> Priority: Major
> Fix For: 2.4.0
>
>
> With {{table.exec.udf-metric-enabled}} set to true:
> # *Evaluation outside a running operator fails.* Code generated for constant
> folding, partition pruning, watermark generators pushed into a source and
> {{SpecializedFunction}} expression evaluators registers metrics, but is
> opened without a runtime context. A user-defined function or {{STARTSWITH}}
> in a partition filter fails planning with {{IllegalStateException: The
> runtime context has not been initialized}}. Calls with constant arguments are
> not folded. A user-defined function in a pushed-down {{WATERMARK}}
> expression, or called by a {{SpecializedFunction}} evaluator, fails the job
> at runtime.
> # *Built-in functions get UDF metric groups.* Built-ins with a runtime class,
> such as {{TYPEOF}} and {{ARRAY_CONTAINS}}, and the internal functions used
> for {{UNNEST}} and {{INTERSECT ALL}}, register {{udf.*}} groups.
> # *Calls are not reported when a function is used in several parts of one
> operator.* For example in a correlate's table function arguments and its
> condition, in a lookup join's pre-filter and join condition, or in {{DEFINE}}
> and {{MEASURES}} of {{MATCH_RECOGNIZE}}. Each part registers {{udf.<name>}}
> on the same operator metric group, only the first registration is kept, and a
> {{Name collision}} WARN is logged.
> {code:sql}
> SET 'table.exec.udf-metric-enabled' = 'true';
> -- 1: fails planning (part_src is partitioned by p)
> SELECT a FROM part_src WHERE myudf(p) = 'A';
> -- 2: registers udf.TYPEOF
> SELECT TYPEOF(id) FROM src;
> -- 3: some calls of probeudf are not reported
> SELECT x FROM src, LATERAL TABLE(tf(probeudf(id))) AS T(x) WHERE probeudf(x)
> > 0;
> {code}
> *Proposed fix*
> # Do not register metrics in code generated for constant folding, partition
> pruning, pushed-down watermark generators and expression evaluators.
> # Do not register metrics for built-in functions.
> # Register each function once per operator metric group, and share it across
> the operator's generated classes.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)