zhengruifeng commented on code in PR #58385:
URL: https://github.com/apache/spark/pull/58385#discussion_r3899999175
##########
sql/api/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -14258,6 +14637,11 @@ object functions {
* @since 2.1.0
* @return
* Returns a column of the type given by the schema (a struct, array, or
map).
+ *
+ * @note
+ * Affected by these public SQL configurations:
+ * - `spark.sql.columnNameOfCorruptRecord`
+ * - `spark.sql.session.timeZone`
Review Comment:
Confirmed. Foldable DDL strings flow through ExprUtils.evalTypeExpr ->
DataType.fromDDL, where timestamp alias resolution reads
spark.sql.timestampType. I added it to the unresolved String/Column schema
forms and variant_get/try_variant_get, while leaving already-resolved
DataType/StructType overloads unchanged. Fixed in 0114665fab2.
##########
sql/api/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -13155,20 +13477,29 @@ object functions {
* @since 3.5.0
* @return
* Returns a column that evaluates to a timestamp.
+ *
+ * @note
+ * Affected by these public SQL configurations:
+ * - `spark.sql.ansi.enabled`
Review Comment:
Confirmed. to_timestamp_ntz uses the session zone when the no-format path
casts TimestampType and when the formatted path converts DateType via
daysToMicros. I added spark.sql.session.timeZone to both Scala overloads and
the shared Python docstring in 0114665fab2.
##########
sql/api/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -14955,6 +15354,11 @@ object functions {
* @since 2.4.0
* @return
* Returns a column that evaluates to a string.
+ *
+ * @note
+ * Affected by these public SQL configurations:
+ * - `spark.sql.session.timeZone`
Review Comment:
Confirmed. SchemaOfJsonEvaluator and SchemaOfCsvEvaluator construct their
options with UTC, and SchemaOfXml does the same, so I removed
spark.sql.session.timeZone from these schema inference docs. I retained
spark.sql.timestampType and added spark.sql.columnNameOfCorruptRecord for
schema_of_xml because XmlOptions reads that config and malformed XML inference
uses the corrupt-record field. Fixed in 0114665fab2.
##########
sql/api/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -16067,6 +16598,10 @@ object functions {
* @since 3.5.0
* @return
* Returns a column that evaluates to a timestamp.
+ *
+ * @note
+ * Affected by these public SQL configurations:
+ * - `spark.sql.session.timeZone`
Review Comment:
Confirmed. The three-argument ConvertTimezone uses the supplied source zone
and does not reference CurrentTimeZone; only the two-argument constructor
injects CurrentTimeZone. I removed the config note from only the three-argument
Scala overload in 0114665fab2. The Python docstring remains shared by both call
forms.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
