Shant Hovsepian created IMPALA-10136:
----------------------------------------
Summary: Cardinality estimates for aggregation operations don't
consider conjuncts on grouping expressions correctly
Key: IMPALA-10136
URL: https://issues.apache.org/jira/browse/IMPALA-10136
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.4.0
Reporter: Shant Hovsepian
Assignee: Shant Hovsepian
ComputeStats() in the PlanNode calls estimateNumGroups() for the
AggregationNode to calculate the cardinality of a grouping expression. Then in
a later step applyConjunctsSelectivity() is called to adjust the cardinality
based on the available conjuncts. However with aggregation operations certain
conjuncts i.e. those from the HAVING clause or conjuncts on the grouping
expressions affect the number of groups produced.
ndv(day) = 11
count(alltypesagg) = 10280
{code:java}
Query: explain select day, count(*) from alltypesagg where day=2 group by 1
+------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=4.06MB Threads=4 |
| Per-Host Resource Estimates: Memory=52MB |
| Codegen disabled by planner |
| |
| PLAN-ROOT SINK |
| | |
| 04:EXCHANGE [UNPARTITIONED] |
| | |
| 03:AGGREGATE [FINALIZE] |
| | output: count:merge(*) |
| | group by: `day` |
| | row-size=12B cardinality=11 |
| | |
| 02:EXCHANGE [HASH(`day`)] |
| | |
| 01:AGGREGATE [STREAMING] |
| | output: count(*) |
| | group by: `day` |
| | row-size=12B cardinality=11 |
| | |
| 00:SCAN HDFS [functional.alltypesagg] |
| partition predicates: `day` = 2 |
| HDFS partitions=1/11 files=1 size=74.48KB |
| row-size=4B cardinality=1.00K |
+------------------------------------------------------------+
Fetched 24 row(s) in 0.02s
{code}
Given the predicate day=1 applies to the grouping expression the cardinality of
the aggregation node should b 1 as opposed to 11.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
