Abhishek Rawat created IMPALA-15380:
---------------------------------------
Summary: Estimate UNNEST node cardinality using sampled collection
sizes
Key: IMPALA-15380
URL: https://issues.apache.org/jira/browse/IMPALA-15380
Project: IMPALA
Issue Type: Task
Reporter: Abhishek Rawat
Assignee: Xuebin Su
{{PlannerContext.java uses a constant AVG_COLLECTION_SIZE which can result in
over/under estimating cardinality. }}
{code:java}
// Assumed average number of items in a nested collection, since we currently
have no
// statistics on nested fields. The motivation for this constant is to avoid
// pathological plan choices that could result from a SubplanNode having an
unknown
// cardinality (due to UnnestNodes not knowing their cardinality), or from a
ScanNode
// significantly underestimating its output cardinality because intermediate
collections
// are not accounted for at all. For example, we will place a table ref plan
with a
// SubplanNode on the build side of a join due to an unknown cardinality if
the other
// input is a base table scan with stats.
// The constant value was chosen arbitrarily to not be "too high" or "too
low".
// TODO: Compute stats for nested types and pick them up here.
public static final long AVG_COLLECTION_SIZE = 10; {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
