[
https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948296#comment-16948296
]
Zoltan Haindrich commented on HIVE-22238:
-----------------------------------------
I went after this - but forgot to write an update here...so what's happening is
somewhat both, now I think that the rescaling is accurate and I agree with the
logic...but when calcite pushes the filter predicates to the other branch as
well it ends up downscaling by the same factor again - hence my patch have
solved some case...I'll try to get back to this sooner than later :)
my current idea is to somehow identify that the FK column in question is not
filtered so far - so that we may downscale it by the PK factor
> PK/FK selectivity estimation underscales estimations
> ----------------------------------------------------
>
> Key: HIVE-22238
> URL: https://issues.apache.org/jira/browse/HIVE-22238
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Reporter: Zoltan Haindrich
> Assignee: Zoltan Haindrich
> Priority: Major
> Attachments: HIVE-22238.01.patch
>
>
> at [this
> point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182]
> the parent operators rownum is scaled according to pkfkselectivity
> however [pkfkselectivity is
> computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157]
> on a whole subtree.
> Scaling it by that amount will count in estimation already used when
> parentstats was calculated...so depending on the number of upstream joins -
> this may lead to severe underestimations
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
