This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 14ab957 [MINOR][SQL] Add comments for filters values and return
values of Row.get()/apply()
14ab957 is described below
commit 14ab957af3d605baadcdbdb26369912d460d6189
Author: Max Gekk <[email protected]>
AuthorDate: Thu Apr 23 04:23:33 2020 +0000
[MINOR][SQL] Add comments for filters values and return values of
Row.get()/apply()
### What changes were proposed in this pull request?
- Document row field values of `DATE` and `TIMESTAMP` type returned by
`Row.get()` and `Row.apply`.
- Refer to `Row.get()` from the description of filter values
### Why are the changes needed?
Reflect current behaviour of Row's method `apply()` and `get()` in comments
to inform users about different return types that are depended on the SQL
config settings `spark.sql.datetime.java8API.enabled`.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
Run `$ ./dev/scalastyle`
Closes #28300 from MaxGekk/doc-filter-date-time.
Authored-by: Max Gekk <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit e7856a7902e9b0d3d3e75fbbf1fce3557ffffb21)
Signed-off-by: Wenchen Fan <[email protected]>
---
sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala | 14 ++++++++++----
.../main/scala/org/apache/spark/sql/sources/filters.scala | 3 ++-
2 files changed, 12 insertions(+), 5 deletions(-)
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
index 9a7e077..4487a2d 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
@@ -165,8 +165,11 @@ trait Row extends Serializable {
* StringType -> String
* DecimalType -> java.math.BigDecimal
*
- * DateType -> java.sql.Date
- * TimestampType -> java.sql.Timestamp
+ * DateType -> java.sql.Date if spark.sql.datetime.java8API.enabled is
false
+ * DateType -> java.time.LocalDate if spark.sql.datetime.java8API.enabled
is true
+ *
+ * TimestampType -> java.sql.Timestamp if
spark.sql.datetime.java8API.enabled is false
+ * TimestampType -> java.time.Instant if
spark.sql.datetime.java8API.enabled is true
*
* BinaryType -> byte array
* ArrayType -> scala.collection.Seq (use getList for java.util.List)
@@ -190,8 +193,11 @@ trait Row extends Serializable {
* StringType -> String
* DecimalType -> java.math.BigDecimal
*
- * DateType -> java.sql.Date
- * TimestampType -> java.sql.Timestamp
+ * DateType -> java.sql.Date if spark.sql.datetime.java8API.enabled is
false
+ * DateType -> java.time.LocalDate if spark.sql.datetime.java8API.enabled
is true
+ *
+ * TimestampType -> java.sql.Timestamp if
spark.sql.datetime.java8API.enabled is false
+ * TimestampType -> java.time.Instant if
spark.sql.datetime.java8API.enabled is true
*
* BinaryType -> byte array
* ArrayType -> scala.collection.Seq (use getList for java.util.List)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
index 319073e..7533793 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
@@ -25,7 +25,8 @@ import
org.apache.spark.sql.connector.catalog.CatalogV2Implicits.parseColumnPath
////////////////////////////////////////////////////////////////////////////////////////////////////
/**
- * A filter predicate for data sources.
+ * A filter predicate for data sources. Mapping between Spark SQL types and
filter value
+ * types follow the convention for return type of
[[org.apache.spark.sql.Row#get(int)]].
*
* @since 1.3.0
*/
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
