Paul Rogers created IMPALA-8036:
-----------------------------------
Summary: Bad join cardinality estimation for missing stats
Key: IMPALA-8036
URL: https://issues.apache.org/jira/browse/IMPALA-8036
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.1.0
Reporter: Paul Rogers
Impala relies on stats. In the {{JoinNode.getJoinCardinality()}} we have the
following code:
{code:java}
long probeCard = getChild(0).cardinality_;
long buildCard = getChild(1).cardinality_;
if (probeCard == -1 || buildCard == -1) {
// Assume FK/PK with a join selectivity of 1.
return probeCard;
}
{code}
That is, if we don’t have a good estimate for either of the join inputs, we
just assume that all rows on the probe side will match one row on the build
side, which is overly optimistic.
The problem arises when a table cannot report is table cardinality, {{|T|}}, or
all predicates applied to the table are those for which Impala declines to
estimate selectivity (See IMPALA-7601.) Hower, many systems do estimate such
selectivity and Impala should also.
The table cardinality can be roughly estimated based on file size as described
in IMPALA-7608.
Once these are done, the planner will produce better estimates for the
occasional cases in which stats are not available, or the user uses an odd
predicate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
