Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/22009#discussion_r208348226
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/SupportsReportStatistics.java
---
@@ -20,18 +20,18 @@
import org.apache.spark.annotation.InterfaceStability;
/**
- * A mix in interface for {@link DataSourceReader}. Data source readers
can implement this
- * interface to report statistics to Spark.
+ * A mix in interface for {@link BatchReadSupport}. Data sources can
implement this interface to
+ * report statistics to Spark.
*
- * Statistics are reported to the optimizer before any operator is pushed
to the DataSourceReader.
- * Implementations that return more accurate statistics based on pushed
operators will not improve
- * query performance until the planner can push operators before getting
stats.
+ * Currently statistics are reported to the optimizer before any operator
is pushed to the data
--- End diff --
Nit: don't use "currently" in docs because it can become out of date and
cause confusion. Instead, use "as of <version>" to be clear what "currently"
means.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
