[
https://issues.apache.org/jira/browse/FLINK-15004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16986742#comment-16986742
]
godfrey he edited comment on FLINK-15004 at 12/3/19 9:34 AM:
-------------------------------------------------------------
[~ykt836], yes, `ndv` had been considered before: if `ndv` is unknown, the
planner will choose two-phase aggregate.
`DistinctRowCount` metadata handler could return null which means unknown,
while `RowCount` metadata handler always returns primitive type and planner
does not know whether the inputs have real row count or just use default value.
this issue mainly solves the following scenarios: `ndv` is known, while row
count is unknown.
was (Author: godfreyhe):
[~ykt836], yes, `ndv` had been considered before: if `ndv` is unknown, the
planner will choose two-phase aggregate.
`DistinctRowCount` metadata handler could return null which means unknown,
while `RowCount` metadata handler always returns primitive type and planner
does not know whether the inputs have real row count or just use default value.
the issue mainly solves the following scenarios: `ndv` is known, while row
count is unknown.
> Choose two-phase Aggregate if the statistics is unknown
> -------------------------------------------------------
>
> Key: FLINK-15004
> URL: https://issues.apache.org/jira/browse/FLINK-15004
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner
> Affects Versions: 1.9.0, 1.9.1
> Reporter: godfrey he
> Assignee: godfrey he
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, blink planner will use default rowCount value (defined in
> {{FlinkPreparingTableBase#DEFAULT_ROWCOUNT}} ) when the statistics is
> unknown, and maybe choose one-phase Aggregate. The job will hang if the data
> is skewed. So It's better to use two-phase Aggregate for execution stability
> if the statistics is unknown.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
