dylanhz created FLINK-40075:
-------------------------------
Summary: Optimize WELFORD_M2 aggregate accumulator by using
RowData instead of STRUCTURED POJO accumulator
Key: FLINK-40075
URL: https://issues.apache.org/jira/browse/FLINK-40075
Project: Flink
Issue Type: Improvement
Components: Table SQL / Runtime
Reporter: dylanhz
{{WelfordM2AggFunction}} is used for variance-related aggregates such as
{{{}VAR_POP{}}}, {{{}VAR_SAMP{}}}, {{{}STDDEV_POP{}}}, and {{{}STDDEV_SAMP{}}}.
Currently its accumulator is a structured POJO type. Using {{RowData}} as the
accumulator could avoid unnecessary structured object conversion in generated
aggregate code and improve performance on hot aggregate paths.
This should be investigated as a separate optimization because changing the
accumulator from {{STRUCTURED<WelfordM2Accumulator>}} to {{ROW<n, mean, m2>}}
may affect compiled plan and streaming state/savepoint compatibility.
*Validation*
The implementation should include:
* variance/stddev aggregate tests,
* compiled plan restore coverage,
* state/savepoint or serializer compatibility checks,
* performance evidence for the reduced conversion overhead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
