[
https://issues.apache.org/jira/browse/HIVE-30208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-30208:
----------------------------------
Labels: pull-request-available (was: )
> Estimate multi-key join cardinality with a configurable key independence
> factor
> -------------------------------------------------------------------------------
>
> Key: HIVE-30208
> URL: https://issues.apache.org/jira/browse/HIVE-30208
> Project: Hive
> Issue Type: Improvement
> Components: CBO
> Reporter: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
>
> When a join has several equi-join keys, CBO (HiveRelMdSelectivity) and the
> physical planner (StatsRulesProcFactory.JoinStatsRule) have to combine the
> keys' NDVs (number of distinct values) into the NDV of the composite key.
> Today there are only two options, chosen by a boolean setting:
> hive.cbo.stats.correlated.multi.key.joins /
> hive.stats.correlated.multi.key.joins
> - = true (default): keys are assumed fully correlated and only the largest
> NDV is used.
> - = false: exponential back-off: ndv1 * ndv2^(1/2) * ndv3^(1/4) *...
> The default "fully correlated" estimate ignores every key except one, so the
> denominator is too small and multi-key joins are heavily overestimated.
> Example: on a 10TB TPC_DS dataset TPC-DS q29
> (store_sales x store_returns on customer, item and ticket number): 748.6M
> rows estimated vs 11.0M actual.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
