Hu Fuwang created SPARK-30516:
---------------------------------
Summary: FileScan.estimateStatistics does not take
partitionFilters and partition number into account
Key: SPARK-30516
URL: https://issues.apache.org/jira/browse/SPARK-30516
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.1.0
Reporter: Hu Fuwang
Currently, FileScan.estimateStatistics will not take partitionFilters into
account, which may lead to bigger sizeInBytes.
It should be reasonable to change it to involve partitionFilters and partition
numbers when estimating the statistics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
