[
https://issues.apache.org/jira/browse/IMPALA-15305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Efimov reassigned IMPALA-15305:
-----------------------------------------
Assignee: Aleksandr Efimov
> Calcite planner: export NDV as a Substrait approx_count_distinct
> ----------------------------------------------------------------
>
> Key: IMPALA-15305
> URL: https://issues.apache.org/jira/browse/IMPALA-15305
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Aleksandr Efimov
> Assignee: Aleksandr Efimov
> Priority: Major
>
> NDV was left out of the mapping table added by IMPALA-15297 on the grounds
> that approx_count_distinct is the obvious counterpart but the rename would
> assert something about the estimator that a name alone does not back.
> Checking rather than assuming settles it. Substrait says of
> approx_count_distinct that it calculates the approximate number of distinct
> values "using HyperLogLog". Impala's ndv is HyperLogLog too, sized by
> ComputeHllLengthFromScale. Both name the same algorithm, so the mapping
> asserts nothing that is not already true of either side.
> Map it, and wire an aggregate converter to carry it: the provider had scalar
> and window converters until now.
> The two-argument ndv, which sets the precision, needs nothing to stay
> refused. Substrait has one variant of approx_count_distinct and it takes one
> operand, so a call carrying a scale matches nothing and the export says so.
> There is no way to drop the precision without noticing, and nothing in the
> mapping arranges that.
> No TPC-DS query is affected: ndv is an Impala function rather than standard
> SQL, so the corpus stays at 102 of 110. It is common in real Impala queries,
> which is the reason to carry it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
