cloud-fan commented on a change in pull request #32949:
URL: https://github.com/apache/spark/pull/32949#discussion_r659867783
##########
File path: docs/sql-migration-guide.md
##########
@@ -83,7 +83,9 @@ license: |
- In Spark 3.2, `TRANSFORM` operator can support
`ArrayType/MapType/StructType` without Hive SerDe, in this mode, we use
`StructsToJosn` to convert `ArrayType/MapType/StructType` column to `STRING`
and use `JsonToStructs` to parse `STRING` to `ArrayType/MapType/StructType`. In
Spark 3.1, Spark just support case `ArrayType/MapType/StructType` column as
`STRING` but can't support parse `STRING` to `ArrayType/MapType/StructType`
output columns.
- - In Spark 3.2, the unit-to-unit interval literals like `INTERVAL '1-1' YEAR
TO MONTH` are converted to ANSI interval types: `YearMonthIntervalType` or
`DayTimeIntervalType`. In Spark 3.1 and earlier, such interval literals are
converted to `CalendarIntervalType`. To restore the behavior before Spark 3.2,
you can set `spark.sql.legacy.interval.enabled` to `true`.
+ - In Spark 3.2, the unit-to-unit interval literals like `INTERVAL '1-1' YEAR
TO MONTH` and the unit list interval literals like `INTERVAL '3' DAYS '1' HOUR`
are converted to ANSI interval types: `YearMonthIntervalType` or
`DayTimeIntervalType`. In Spark 3.1 and earlier, such interval literals are
converted to `CalendarIntervalType`. To restore the behavior before Spark 3.2,
you can set `spark.sql.legacy.interval.enabled` to `true`.
+
+ - In Spark 3.2, the unit list interval literals support two category of
units. The one is year-month units (YEAR and MONTH) and the other is day-time
units (DAY, HOUR, MINUTE and SECOND). Units are allowd to be listed within the
same category like `5 YEARS 8 MONTHS` or `1 DAY 2 MINUTE`. In Spark 3.1 and
earlier, WEEK, MILLISECOND, MICROSECOND and NANOSECOND are also supported as
units, and any units are allowed to be listed in a literal. To restore the
behavior before Spark 3.2, you can set `spark.sql.legacy.interval.enabled` to
`true`.
Review comment:
Do we have to forbid `WEEK, MILLISECOND, MICROSECOND and NANOSECOND`? I
think the only limitation is people can't mix year-month and day-time fields
now, but I don't see why we need to forbid `WEEK, MILLISECOND, MICROSECOND and
NANOSECOND`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
