[
https://issues.apache.org/jira/browse/CALCITE-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17649537#comment-17649537
]
Yunhong Zheng commented on CALCITE-4351:
----------------------------------------
Hi, [~fan_li_ya], and [~julianhyde] , is there a solution to this problem? For
RelMdUtil#numDistinctVals, Math.log(1.0 - 1.0 / dSize) will return zero easily
for same large databases, like TPC-DS benchmark. Return zero will affect the
selection of the join strategies (turn HashJoin to NestedLoopJoin) in some
cases. So I think it need some protective measures for boundary cases. Like if
'res = (1.0 - Math.exp(expo)) * dSize' return zero, we need return dSize
instead of zero
)
> RelMdUtil#numDistinctVals always returns 0 for large inputs
> -----------------------------------------------------------
>
> Key: CALCITE-4351
> URL: https://issues.apache.org/jira/browse/CALCITE-4351
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.26.0
> Reporter: Caizhi Weng
> Priority: Major
>
> Previous implementation of {{RelMdUtil#numDistinctVals}} uses the
> approximation {{ln(1 + x) ~= x}} when {{x}} is small.
> However CALCITE-4132 remove this approximation to make the result more
> accurate. This causes the function to calculate an incorrect result for large
> inputs (for example, when {{domainSize = 1e18}} and {{numSelected = 1e10}}
> the result is 0) due to precision problems.
> What I would suggest is to treat small and large inputs in different ways.
> For small inputs we use the new, more precise function and for large inputs
> we use the old, approximated function.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
