This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new f0fa8b119b4 [SPARK-38962][SQL] Fix wrong computeStats at
DataSourceV2Relation
f0fa8b119b4 is described below
commit f0fa8b119b42eb1e0623b8918e33a14f6ae80a51
Author: ulysses-you <[email protected]>
AuthorDate: Wed Apr 20 14:09:10 2022 +0800
[SPARK-38962][SQL] Fix wrong computeStats at DataSourceV2Relation
### What changes were proposed in this pull request?
Use `Scan` to match `SupportsReportStatistics`.
### Why are the changes needed?
The interface `SupportsReportStatistics` should be mixed in `Scan` rather
than `ScanBuilder`
### Does this PR introduce _any_ user-facing change?
almost no, it is just dead code in normal useage
### How was this patch tested?
an obvious bug and it's a dead code in test.
Closes #36276 from ulysses-you/statistics.
Authored-by: ulysses-you <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
---
.../spark/sql/execution/datasources/v2/DataSourceV2Relation.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
index 6b0760ca163..61fe3602bb6 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
@@ -80,7 +80,7 @@ case class DataSourceV2Relation(
s"BUG: computeStats called before pushdown on DSv2 relation: $name")
} else {
// when not testing, return stats because bad stats are better than
failing a query
- table.asReadable.newScanBuilder(options) match {
+ table.asReadable.newScanBuilder(options).build() match {
case r: SupportsReportStatistics =>
val statistics = r.estimateStatistics()
DataSourceV2Relation.transformV2Stats(statistics, None,
conf.defaultSizeInBytes)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
