[
https://issues.apache.org/jira/browse/SPARK-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhenhua Wang updated SPARK-19350:
---------------------------------
Description:
Currently, LocalLimit/GlobalLimit/Sample propagates the same row count and
column stats from its child, which is incorrect.
We can get the correct rowCount in Statistics for Limit/Sample whether cbo is
enabled or not. And column stats should not be propagated because we don't know
the distribution of columns after Limit or Sample.
was:Now we can easily get rowCount in Statistics for Limit and Sample whether
cbo is enabled or not. But column stats should not be propagated because we
don't know the distribution of columns after Limit or Sample.
> Cardinality estimation of Limit and Sample
> ------------------------------------------
>
> Key: SPARK-19350
> URL: https://issues.apache.org/jira/browse/SPARK-19350
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Zhenhua Wang
>
> Currently, LocalLimit/GlobalLimit/Sample propagates the same row count and
> column stats from its child, which is incorrect.
> We can get the correct rowCount in Statistics for Limit/Sample whether cbo is
> enabled or not. And column stats should not be propagated because we don't
> know the distribution of columns after Limit or Sample.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
