Hu Fuwang created SPARK-30516: --------------------------------- Summary: FileScan.estimateStatistics does not take partitionFilters and partition number into account Key: SPARK-30516 URL: https://issues.apache.org/jira/browse/SPARK-30516 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.1.0 Reporter: Hu Fuwang
Currently, FileScan.estimateStatistics will not take partitionFilters into account, which may lead to bigger sizeInBytes. It should be reasonable to change it to involve partitionFilters and partition numbers when estimating the statistics. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org