[ 
https://issues.apache.org/jira/browse/SPARK-30516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hu Fuwang updated SPARK-30516:
------------------------------
    Description: Currently, FileScan.estimateStatistics does not take 
partitionFilters and partition number into account, which may lead to bigger 
sizeInBytes. It should be reasonable to change it to involve partitionFilters 
and partition number when estimating the statistics.  (was: Currently, 
FileScan.estimateStatistics will not take partitionFilters into account, which 
may lead to bigger sizeInBytes. It should be reasonable to change it to involve 
partitionFilters and partition numbers when estimating the statistics.)

> FileScan.estimateStatistics does not take partitionFilters and partition 
> number into account
> --------------------------------------------------------------------------------------------
>
>                 Key: SPARK-30516
>                 URL: https://issues.apache.org/jira/browse/SPARK-30516
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: Hu Fuwang
>            Priority: Major
>
> Currently, FileScan.estimateStatistics does not take partitionFilters and 
> partition number into account, which may lead to bigger sizeInBytes. It 
> should be reasonable to change it to involve partitionFilters and partition 
> number when estimating the statistics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to