[ 
https://issues.apache.org/jira/browse/FLINK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Weiqing Yang updated FLINK-41585:
---------------------------------
    Priority: Major  (was: Not a Priority)

> Refine UDF metrics registration
> -------------------------------
>
>                 Key: FLINK-41585
>                 URL: https://issues.apache.org/jira/browse/FLINK-41585
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Planner, Table SQL / Runtime
>    Affects Versions: 2.4.0
>            Reporter: Weiqing Yang
>            Assignee: Weiqing Yang
>            Priority: Major
>             Fix For: 2.4.0
>
>
> With {{table.exec.udf-metric-enabled}} set to true:
> # *Evaluation outside a running operator fails.* Code generated for constant 
> folding, partition pruning, watermark generators pushed into a source and 
> {{SpecializedFunction}} expression evaluators registers metrics, but is 
> opened without a runtime context. A user-defined function or {{STARTSWITH}} 
> in a partition filter fails planning with {{IllegalStateException: The 
> runtime context has not been initialized}}. Calls with constant arguments are 
> not folded. A user-defined function in a pushed-down {{WATERMARK}} 
> expression, or called by a {{SpecializedFunction}} evaluator, fails the job 
> at runtime.
> # *Built-in functions get UDF metric groups.* Built-ins with a runtime class, 
> such as {{TYPEOF}} and {{ARRAY_CONTAINS}}, and the internal functions used 
> for {{UNNEST}} and {{INTERSECT ALL}}, register {{udf.*}} groups.
> # *Calls are not reported when a function is used in several parts of one 
> operator.* For example in a correlate's table function arguments and its 
> condition, in a lookup join's pre-filter and join condition, or in {{DEFINE}} 
> and {{MEASURES}} of {{MATCH_RECOGNIZE}}. Each part registers {{udf.<name>}} 
> on the same operator metric group, only the first registration is kept, and a 
> {{Name collision}} WARN is logged.
> {code:sql}
> SET 'table.exec.udf-metric-enabled' = 'true';
> -- 1: fails planning (part_src is partitioned by p)
> SELECT a FROM part_src WHERE myudf(p) = 'A';
> -- 2: registers udf.TYPEOF
> SELECT TYPEOF(id) FROM src;
> -- 3: some calls of probeudf are not reported
> SELECT x FROM src, LATERAL TABLE(tf(probeudf(id))) AS T(x) WHERE probeudf(x) 
> > 0;
> {code}
> *Proposed fix*
> # Do not register metrics in code generated for constant folding, partition 
> pruning, pushed-down watermark generators and expression evaluators.
> # Do not register metrics for built-in functions.
> # Register each function once per operator metric group, and share it across 
> the operator's generated classes.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to