[ 
https://issues.apache.org/jira/browse/FLINK-41585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Weiqing Yang updated FLINK-41585:
---------------------------------
    Description: 
With {{table.exec.udf-metric-enabled}} set to true:

# *Evaluation outside a running operator fails.* Code generated for constant 
folding, partition pruning, watermark generators pushed into a source and 
{{SpecializedFunction}} expression evaluators registers metrics, but is opened 
without a runtime context. A user-defined function or {{STARTSWITH}} in a 
partition filter fails planning with {{IllegalStateException: The runtime 
context has not been initialized}}. Calls with constant arguments are not 
folded. A user-defined function in a pushed-down {{WATERMARK}} expression, or 
called by a {{SpecializedFunction}} evaluator, fails the job at runtime.
# *Built-in functions get UDF metric groups.* Built-ins with a runtime class, 
such as {{TYPEOF}} and {{ARRAY_CONTAINS}}, and the internal functions used for 
{{UNNEST}} and {{INTERSECT ALL}}, register {{udf.*}} groups.
# *Calls are not reported when a function is used in several parts of one 
operator.* For example in a correlate's table function arguments and its 
condition, in a lookup join's pre-filter and join condition, or in {{DEFINE}} 
and {{MEASURES}} of {{MATCH_RECOGNIZE}}. Each part registers {{udf.<name>}} on 
the same operator metric group, only the first registration is kept, and a 
{{Name collision}} WARN is logged.

{code:sql}
SET 'table.exec.udf-metric-enabled' = 'true';
-- 1: fails planning (part_src is partitioned by p)
SELECT a FROM part_src WHERE myudf(p) = 'A';
-- 2: registers udf.TYPEOF
SELECT TYPEOF(id) FROM src;
-- 3: some calls of probeudf are not reported
SELECT x FROM src, LATERAL TABLE(tf(probeudf(id))) AS T(x) WHERE probeudf(x) > 
0;
{code}

*Proposed fix*
# Do not register metrics in code generated for constant folding, partition 
pruning, pushed-down watermark generators and expression evaluators.
# Do not register metrics for built-in functions.
# Register each function once per operator metric group, and share it across 
the operator's generated classes.


  was:
Release testing of FLIP-485 (FLINK-41450) found three problems with 
{{table.exec.udf-metric-enabled}} set to true. They are all in UDF metrics 
registration, so they are fixed together in one pull request.

# Code evaluated outside a running operator calls {{getRuntimeContext()}}: 
planning fails when a UDF or a built-in such as {{STARTSWITH}} is in a 
partition filter, constant calls are no longer folded, and a 
{{SpecializedFunction}} evaluator fails the job.
# Built-in functions such as {{TYPEOF}} and {{ARRAY_CONTAINS}}, and the 
internal UNNEST function, get UDF metric groups.
# A UDF used in more than one part of one operator (correlate, lookup join, 
MATCH_RECOGNIZE) loses the samples of all but one part.

h3. 1. Code evaluated outside a running operator fails

*What happens*

With {{table.exec.udf-metric-enabled}} set to true, three kinds of evaluation 
outside a running operator break:

* *Partition pruning fails the query.* A filter on a partition column that 
calls a user-defined function, or a built-in such as {{STARTSWITH}}, fails at 
planning, in streaming and batch:
{noformat}
java.lang.IllegalStateException: The runtime context has not been initialized.
    at 
org.apache.flink.api.common.functions.AbstractRichFunction.getRuntimeContext(AbstractRichFunction.java:51)
    at PartitionPruner$13.open(Unknown Source)
    at 
org.apache.flink.table.planner.plan.utils.PartitionPruner$.prunePartitions(PartitionPruner.scala:129)
    at 
org.apache.flink.table.planner.plan.rules.logical.PushPartitionIntoTableSourceScanRule.lambda$onMatch$3(PushPartitionIntoTableSourceScanRule.java:160)
{noformat}
With metrics disabled the same query plans and prunes to the matching partition.
* *Constant folding is skipped.* A call with constant arguments, such as 
{{myudf('abc')}}, is no longer folded. The planner logs a WARN with the same 
exception on every reduction attempt (6 to 13 times for one small query) and 
evaluates the call per row. The reducer gives up on the whole batch of 
constants, so other constants in the same projection, for example 
{{UPPER('xyz')}}, are not folded either.
* *A {{SpecializedFunction}} evaluator fails the job.* A user function whose 
{{specialize}} calls another user-defined function through 
{{SpecializedContext#createEvaluator}} fails at runtime with the same 
{{IllegalStateException}}, because the generated evaluator is opened without a 
runtime context ({{DefaultExpressionEvaluator.java:62}}). With metrics disabled 
the job runs.

*How to reproduce*

Reproduced on master in planner tests, using the {{values}} test connector with 
{{'partition-list' = 'p:A;p:B'}} for the partitioned table. {{myudf}} is a 
deterministic {{ScalarFunction}} that returns {{s.toUpperCase() + "!"}}.
{code:sql}
SET 'table.exec.udf-metric-enabled' = 'true';
-- part_src (a INT, p STRING) PARTITIONED BY (p)
EXPLAIN SELECT a, p FROM part_src WHERE myudf(p) = 'A!';        -- fails, see 
above
EXPLAIN SELECT a, p FROM part_src WHERE STARTSWITH(p, 'A');     -- fails the 
same way
EXPLAIN SELECT a, p FROM part_src WHERE UPPER(p) = 'A';         -- plans and 
prunes (control)
-- src (a INT, b STRING)
EXPLAIN SELECT myudf('abc') AS c, a FROM src;                   -- not folded, 
WARN logged
{code}
With metrics disabled, the first two plan with only partition {{p=A}} scanned, 
and {{myudf('abc')}} is folded to {{'ABC!'}}.

*Cause*

With metrics enabled, {{CodeGeneratorContext#addReusableUdfMetrics}} adds 
{{UdfMetrics.register(getRuntimeContext().getMetricGroup(), ...)}} to the 
generated {{open()}}. {{ExpressionReducer}} ({{ExpressionReducer.scala:111}}) 
and {{PartitionPruner}} ({{PartitionPruner.scala:129}}) call {{open()}} on 
generated functions while planning, and {{DefaultExpressionEvaluator}} opens 
its generated evaluator at runtime; none of them has a runtime context. The 
reducer catches the exception and leaves the expressions unreduced. The pruner 
does not catch it, so planning fails.

*Proposed fix*

Do not instrument calls generated for these paths. The planner paths use a 
{{ConstantCodeGeneratorContext}} ({{ExpressionReducer.scala:78}}, 
{{PartitionPruner.scala:96}}) and the evaluator uses an 
{{EvaluatorCodeGeneratorContext}} ({{BridgingFunctionGenUtil.scala:1002}}), so 
{{BridgingFunctionGenUtil#udfMetricsTermIfEnabled}} can skip the metrics for 
these two contexts. Plan-time evaluation is not part of the running job, so it 
should not report job metrics. FLINK-19112 drew the same line for function 
metric groups during optimization. Add planner tests for both paths with 
metrics enabled.

h3. 2. Built-in functions are metered

*What happens*

With {{table.exec.udf-metric-enabled}} set to true, built-in functions that 
have a runtime implementation class get a UDF metric group, as if they were 
user-defined functions. This includes the internal functions the planner adds 
for {{UNNEST}} and {{INTERSECT ALL}}, so a query that calls no function at all 
can get a group such as {{udf.$UNNEST_ROWS$1}}. The metrics docs and the option 
description say these metrics are registered for SQL/Table user-defined 
functions.

*How to reproduce*

Reproduced on master in a planner integration test with an in-memory metric 
reporter, streaming mode, sample interval 1:
{code:sql}
SET 'table.exec.udf-metric-enabled' = 'true';
-- src (id INT, arr ARRAY<INT>, s STRING)
INSERT INTO sink SELECT TYPEOF(id), ARRAY_CONTAINS(arr, id), UPPER(s) FROM src;
INSERT INTO sink2 SELECT id, x FROM src CROSS JOIN UNNEST(arr) AS T(x);
INSERT INTO sink3 SELECT id FROM src INTERSECT ALL SELECT id FROM src;
{code}
Registered groups:
{noformat}
Calc:      udf.TYPEOF, udf.ARRAY_CONTAINS   (no group for UPPER)
Correlate: udf.$UNNEST_ROWS$1
Correlate: udf.$REPLICATE_ROWS$1
{noformat}
With {{table.exec.udf-metric-enabled}} set to false, no group is registered. A 
user-defined function in the same query gets only its own group, as expected.

*Cause*

Built-in functions with a runtime class are generated through 
{{BridgingSqlFunctionCallGen}}, the same entry as user-defined functions, and 
that entry always passes a metric name 
({{BridgingSqlFunctionCallGen.scala:82}}). Built-ins that {{ExprCodeGenerator}} 
handles explicitly, such as {{UPPER}}, do not take this path. On release-2.4 
about 60 public built-ins are affected (scalar or table built-ins with a 
runtime class, for example the {{ARRAY_\*}} and {{MAP_\*}} functions, 
{{TYPEOF}} and {{STARTSWITH}}), plus the internal {{$UNNEST_ROWS$1}}, 
{{$UNNEST_ROWS_WITH_ORDINALITY$1}} and {{$REPLICATE_ROWS$1}}.

Besides the extra groups and the sampling cost, this makes problem 1 reachable 
with built-ins alone, for example {{STARTSWITH}} in a partition filter.

*Proposed fix*

Do not pass a metric name in {{BridgingSqlFunctionCallGen}} when the function 
definition is a {{BuiltInFunctionDefinition}}. That also covers the internal 
UNNEST and REPLICATE_ROWS functions. Add a test that a query using only 
built-ins registers no UDF metric group.

h3. 3. A UDF used in several parts of one operator loses samples

*What happens*

With {{table.exec.udf-metric-enabled}} set to true, a user-defined function 
that is called both in the arguments of a table function and in the condition 
of the same correlate reports only part of its calls. The second registration 
of {{udf.<name>}} on the operator's metric group is rejected, and every sample 
and exception recorded through it is lost:
{noformat}
Name collision: Group already contains a Metric with the name 
'udfProcessingTime'. Metric will not be reported.
Name collision: Group already contains a Metric with the name 
'udfExceptionCount'. Metric will not be reported.
{noformat}

*How to reproduce*

Reproduced on master in a planner integration test with an in-memory metric 
reporter, sample interval 1, 3 source rows. {{probetableudf}} emits each input 
twice.
{code:sql}
SET 'table.exec.udf-metric-enabled' = 'true';
INSERT INTO sink SELECT x FROM src, LATERAL TABLE(probetableudf(probeudf(id))) 
AS T(x) WHERE probeudf(x) > 0;
{code}
The plan has one {{Correlate}} with 
{{invocation=\[probetableudf(probeudf($cor0.id))\]}} and 
{{condition=\[>(probeudf($0), 0)\]}}.

|| Case || Expected {{udf.probeudf}} || Reported ||
| invocation and condition | 9 samples | 6 samples, plus the two WARNs above |
| invocation and condition, the function throws once on the invocation side | 6 
samples, 1 exception | 4 samples, 0 exceptions |
| invocation only | 3 samples | 3 samples |
| condition only | 6 samples | 6 samples |

Counters inside the functions confirm that all calls are made. Only the metrics 
are lost.

*Cause*

The correlate's condition is generated into a separate collector class with its 
own {{CodeGeneratorContext}} ({{CorrelateCodeGenerator.scala:196}}). 
{{CodeGeneratorContext#addReusableUdfMetrics}} registers each function name 
once per context, so the operator class and the collector class each call 
{{UdfMetrics.register}} for the same name on the same operator metric group. 
The collector opens first, and the operator's registration is the one rejected.

Other operators that generate several classes on one metric group lose samples 
the same way when one function appears in more than one of them. Reproduced on 
master with sample interval 1:

|| Operator || Query shape || Calls || Reported samples ||
| Lookup join | {{cudf}} in the pre-filter and in the remaining join condition 
| 5 | 2 |
| MATCH_RECOGNIZE | {{cudf}} in {{DEFINE}} and in {{MEASURES}} | 6 | 3 |

Multi-way joins and delta joins generate their conditions the same way (code 
reading, not run).

*Proposed fix*

Register each function name once per operator rather than once per generated 
class: {{UdfMetrics.register}} returns the existing handle when the same 
operator metric group already has one for that name, so every generated class 
of the operator records into one histogram and one counter. Add tests for the 
correlate and lookup join cases above.



> Refine UDF metrics registration
> -------------------------------
>
>                 Key: FLINK-41585
>                 URL: https://issues.apache.org/jira/browse/FLINK-41585
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Planner, Table SQL / Runtime
>    Affects Versions: 2.4.0
>            Reporter: Weiqing Yang
>            Assignee: Weiqing Yang
>            Priority: Not a Priority
>             Fix For: 2.4.0
>
>
> With {{table.exec.udf-metric-enabled}} set to true:
> # *Evaluation outside a running operator fails.* Code generated for constant 
> folding, partition pruning, watermark generators pushed into a source and 
> {{SpecializedFunction}} expression evaluators registers metrics, but is 
> opened without a runtime context. A user-defined function or {{STARTSWITH}} 
> in a partition filter fails planning with {{IllegalStateException: The 
> runtime context has not been initialized}}. Calls with constant arguments are 
> not folded. A user-defined function in a pushed-down {{WATERMARK}} 
> expression, or called by a {{SpecializedFunction}} evaluator, fails the job 
> at runtime.
> # *Built-in functions get UDF metric groups.* Built-ins with a runtime class, 
> such as {{TYPEOF}} and {{ARRAY_CONTAINS}}, and the internal functions used 
> for {{UNNEST}} and {{INTERSECT ALL}}, register {{udf.*}} groups.
> # *Calls are not reported when a function is used in several parts of one 
> operator.* For example in a correlate's table function arguments and its 
> condition, in a lookup join's pre-filter and join condition, or in {{DEFINE}} 
> and {{MEASURES}} of {{MATCH_RECOGNIZE}}. Each part registers {{udf.<name>}} 
> on the same operator metric group, only the first registration is kept, and a 
> {{Name collision}} WARN is logged.
> {code:sql}
> SET 'table.exec.udf-metric-enabled' = 'true';
> -- 1: fails planning (part_src is partitioned by p)
> SELECT a FROM part_src WHERE myudf(p) = 'A';
> -- 2: registers udf.TYPEOF
> SELECT TYPEOF(id) FROM src;
> -- 3: some calls of probeudf are not reported
> SELECT x FROM src, LATERAL TABLE(tf(probeudf(id))) AS T(x) WHERE probeudf(x) 
> > 0;
> {code}
> *Proposed fix*
> # Do not register metrics in code generated for constant folding, partition 
> pruning, pushed-down watermark generators and expression evaluators.
> # Do not register metrics for built-in functions.
> # Register each function once per operator metric group, and share it across 
> the operator's generated classes.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to