[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r321595840 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -186,23 +186,23 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- FROM TIMESTAMP_TBL -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; - --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +-- [SPARK-28767] ParseException: no viable alternative at input 'year' +set spark.sql.parser.ansi.enabled=false; Review comment: > I'd like to quote it, to not distract people from the timestamp tests @dongjoon-hyun I hope you will be not so unhappy if I use backquotes again here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r321370816 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -186,23 +186,23 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- FROM TIMESTAMP_TBL -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; - --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +-- [SPARK-28767] ParseException: no viable alternative at input 'year' +set spark.sql.parser.ansi.enabled=false; Review comment: We can quote or set the variable. Please, take a look at the comments: https://github.com/apache/spark/pull/25410/files/af51e524d90253d26dc848d4776328c5f8359d88#r314593244 . Do you think it is better to use backquotes instead of setting the variable? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add
`date_part` function for timestamps/dates
URL: https://github.com/apache/spark/pull/25410#discussion_r320836537
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
##
@@ -1963,3 +1963,91 @@ case class Epoch(child: Expression, timeZoneId:
Option[String] = None)
defineCodeGen(ctx, ev, c => s"$dtu.getEpoch($c, $zid)")
}
}
+
+object DatePart {
+
+ def parseExtractField(
+ extractField: String,
+ source: Expression,
+ errorHandleFunc: => Unit): Expression =
extractField.toUpperCase(Locale.ROOT) match {
+case "MILLENNIUM" | "MILLENNIA" | "MIL" | "MILS" => Millennium(source)
+case "CENTURY" | "CENTURIES" | "C" | "CENT" => Century(source)
+case "DECADE" | "DECADES" | "DEC" | "DECS" => Decade(source)
+case "YEAR" | "Y" | "YEARS" | "YR" | "YRS" => Year(source)
+case "ISOYEAR" => IsoYear(source)
+case "QUARTER" | "QTR" => Quarter(source)
+case "MONTH" | "MON" | "MONS" | "MONTHS" => Month(source)
+case "WEEK" | "W" | "WEEKS" => WeekOfYear(source)
+case "DAY" | "D" | "DAYS" => DayOfMonth(source)
+case "DAYOFWEEK" => DayOfWeek(source)
+case "DOW" => Subtract(DayOfWeek(source), Literal(1))
+case "ISODOW" => Add(WeekDay(source), Literal(1))
+case "DOY" => DayOfYear(source)
+case "HOUR" | "H" | "HOURS" | "HR" | "HRS" => Hour(source)
+case "MINUTE" | "M" | "MIN" | "MINS" | "MINUTES" => Minute(source)
+case "SECOND" | "S" | "SEC" | "SECONDS" | "SECS" => Second(source)
+case "MILLISECONDS" | "MSEC" | "MSECS" | "MILLISECON" | "MSECONDS" | "MS"
=>
+ Milliseconds(source)
+case "MICROSECONDS" | "USEC" | "USECS" | "USECONDS" | "MICROSECON" | "US"
=>
+ Microseconds(source)
+case "EPOCH" => Epoch(source)
+case other =>
+ errorHandleFunc.asInstanceOf[Expression]
Review comment:
I changed type of `errorHandleFunc` to `Nothing`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add
`date_part` function for timestamps/dates
URL: https://github.com/apache/spark/pull/25410#discussion_r316817684
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##
@@ -1409,48 +1409,7 @@ class AstBuilder(conf: SQLConf) extends
SqlBaseBaseVisitor[AnyRef] with Logging
* Create a Extract expression.
*/
override def visitExtract(ctx: ExtractContext): Expression = withOrigin(ctx)
{
-ctx.field.getText.toUpperCase(Locale.ROOT) match {
- case "MILLENNIUM" | "MILLENNIA" | "MIL" | "MILS" =>
-Millennium(expression(ctx.source))
- case "CENTURY" | "CENTURIES" | "C" | "CENT" =>
-Century(expression(ctx.source))
- case "DECADE" | "DECADES" | "DEC" | "DECS" =>
-Decade(expression(ctx.source))
- case "YEAR" | "Y" | "YEARS" | "YR" | "YRS" =>
-Year(expression(ctx.source))
- case "ISOYEAR" =>
-IsoYear(expression(ctx.source))
- case "QUARTER" | "QTR" =>
-Quarter(expression(ctx.source))
- case "MONTH" | "MON" | "MONS" | "MONTHS" =>
-Month(expression(ctx.source))
- case "WEEK" | "W" | "WEEKS" =>
-WeekOfYear(expression(ctx.source))
- case "DAY" | "D" | "DAYS" =>
-DayOfMonth(expression(ctx.source))
- case "DAYOFWEEK" =>
-DayOfWeek(expression(ctx.source))
- case "DOW" =>
-Subtract(DayOfWeek(expression(ctx.source)), Literal(1))
- case "ISODOW" =>
-Add(WeekDay(expression(ctx.source)), Literal(1))
- case "DOY" =>
-DayOfYear(expression(ctx.source))
- case "HOUR" | "H" | "HOURS" | "HR" | "HRS" =>
-Hour(expression(ctx.source))
- case "MINUTE" | "M" | "MIN" | "MINS" | "MINUTES" =>
-Minute(expression(ctx.source))
- case "SECOND" | "S" | "SEC" | "SECONDS" | "SECS" =>
-Second(expression(ctx.source))
- case "MILLISECONDS" | "MSEC" | "MSECS" | "MILLISECON" | "MSECONDS" |
"MS" =>
-Milliseconds(expression(ctx.source))
- case "MICROSECONDS" | "USEC" | "USECS" | "USECONDS" | "MICROSECON" |
"US" =>
-Microseconds(expression(ctx.source))
- case "EPOCH" =>
-Epoch(expression(ctx.source))
- case other =>
-throw new ParseException(s"Literals of type '$other' are currently not
supported.", ctx)
Review comment:
@maropu @dongjoon-hyun If you think the commit
https://github.com/apache/spark/commit/d793f4958f3a9adc69f7a1e04f7fc7a994d5e15a
is necessary for merging the PR, I will apply the patch.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r316801765 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,22 +187,21 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, Review comment: This is because `spark.sql.parser.ansi.enabled` is set to `false` by default: ``` spark-sql> set spark.sql.parser.ansi.enabled=true; spark.sql.parser.ansi.enabledtrue spark-sql> select 1 as year; Error in query: no viable alternative at input 'year'(line 1, pos 12) == SQL == select 1 as year ^^^ ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314947710 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,22 +187,21 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, Review comment: I reverted the backquotes back and opened JIRA for the issue: https://issues.apache.org/jira/browse/SPARK-28767 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add
`date_part` function for timestamps/dates
URL: https://github.com/apache/spark/pull/25410#discussion_r314946561
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##
@@ -1409,48 +1409,7 @@ class AstBuilder(conf: SQLConf) extends
SqlBaseBaseVisitor[AnyRef] with Logging
* Create a Extract expression.
*/
override def visitExtract(ctx: ExtractContext): Expression = withOrigin(ctx)
{
-ctx.field.getText.toUpperCase(Locale.ROOT) match {
- case "MILLENNIUM" | "MILLENNIA" | "MIL" | "MILS" =>
-Millennium(expression(ctx.source))
- case "CENTURY" | "CENTURIES" | "C" | "CENT" =>
-Century(expression(ctx.source))
- case "DECADE" | "DECADES" | "DEC" | "DECS" =>
-Decade(expression(ctx.source))
- case "YEAR" | "Y" | "YEARS" | "YR" | "YRS" =>
-Year(expression(ctx.source))
- case "ISOYEAR" =>
-IsoYear(expression(ctx.source))
- case "QUARTER" | "QTR" =>
-Quarter(expression(ctx.source))
- case "MONTH" | "MON" | "MONS" | "MONTHS" =>
-Month(expression(ctx.source))
- case "WEEK" | "W" | "WEEKS" =>
-WeekOfYear(expression(ctx.source))
- case "DAY" | "D" | "DAYS" =>
-DayOfMonth(expression(ctx.source))
- case "DAYOFWEEK" =>
-DayOfWeek(expression(ctx.source))
- case "DOW" =>
-Subtract(DayOfWeek(expression(ctx.source)), Literal(1))
- case "ISODOW" =>
-Add(WeekDay(expression(ctx.source)), Literal(1))
- case "DOY" =>
-DayOfYear(expression(ctx.source))
- case "HOUR" | "H" | "HOURS" | "HR" | "HRS" =>
-Hour(expression(ctx.source))
- case "MINUTE" | "M" | "MIN" | "MINS" | "MINUTES" =>
-Minute(expression(ctx.source))
- case "SECOND" | "S" | "SEC" | "SECONDS" | "SECS" =>
-Second(expression(ctx.source))
- case "MILLISECONDS" | "MSEC" | "MSECS" | "MILLISECON" | "MSECONDS" |
"MS" =>
-Milliseconds(expression(ctx.source))
- case "MICROSECONDS" | "USEC" | "USECS" | "USECONDS" | "MICROSECON" |
"US" =>
-Microseconds(expression(ctx.source))
- case "EPOCH" =>
-Epoch(expression(ctx.source))
- case other =>
-throw new ParseException(s"Literals of type '$other' are currently not
supported.", ctx)
Review comment:
I agree it would be nice to point out the exact position of the error but I
don't like this ad-hoc solution. There are a few similar places in
`datetimeExpressions` and others where we could throw `ParseException` instead
of just `AnalysisException`. I believe we need a common approach. Why do you
think that propagation of the parsing context or positions look not good? We
could propagate such parameters to all expression constructors.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314601960 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,22 +187,21 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, Review comment: @dongjoon-hyun Pushed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add
`date_part` function for timestamps/dates
URL: https://github.com/apache/spark/pull/25410#discussion_r314599378
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##
@@ -1409,48 +1409,7 @@ class AstBuilder(conf: SQLConf) extends
SqlBaseBaseVisitor[AnyRef] with Logging
* Create a Extract expression.
*/
override def visitExtract(ctx: ExtractContext): Expression = withOrigin(ctx)
{
-ctx.field.getText.toUpperCase(Locale.ROOT) match {
- case "MILLENNIUM" | "MILLENNIA" | "MIL" | "MILS" =>
-Millennium(expression(ctx.source))
- case "CENTURY" | "CENTURIES" | "C" | "CENT" =>
-Century(expression(ctx.source))
- case "DECADE" | "DECADES" | "DEC" | "DECS" =>
-Decade(expression(ctx.source))
- case "YEAR" | "Y" | "YEARS" | "YR" | "YRS" =>
-Year(expression(ctx.source))
- case "ISOYEAR" =>
-IsoYear(expression(ctx.source))
- case "QUARTER" | "QTR" =>
-Quarter(expression(ctx.source))
- case "MONTH" | "MON" | "MONS" | "MONTHS" =>
-Month(expression(ctx.source))
- case "WEEK" | "W" | "WEEKS" =>
-WeekOfYear(expression(ctx.source))
- case "DAY" | "D" | "DAYS" =>
-DayOfMonth(expression(ctx.source))
- case "DAYOFWEEK" =>
-DayOfWeek(expression(ctx.source))
- case "DOW" =>
-Subtract(DayOfWeek(expression(ctx.source)), Literal(1))
- case "ISODOW" =>
-Add(WeekDay(expression(ctx.source)), Literal(1))
- case "DOY" =>
-DayOfYear(expression(ctx.source))
- case "HOUR" | "H" | "HOURS" | "HR" | "HRS" =>
-Hour(expression(ctx.source))
- case "MINUTE" | "M" | "MIN" | "MINS" | "MINUTES" =>
-Minute(expression(ctx.source))
- case "SECOND" | "S" | "SEC" | "SECONDS" | "SECS" =>
-Second(expression(ctx.source))
- case "MILLISECONDS" | "MSEC" | "MSECS" | "MILLISECON" | "MSECONDS" |
"MS" =>
-Milliseconds(expression(ctx.source))
- case "MICROSECONDS" | "USEC" | "USECS" | "USECONDS" | "MICROSECON" |
"US" =>
-Microseconds(expression(ctx.source))
- case "EPOCH" =>
-Epoch(expression(ctx.source))
- case other =>
-throw new ParseException(s"Literals of type '$other' are currently not
supported.", ctx)
Review comment:
This will work for `extract` but how about the `date_part()` function
itself? The same code should present for the function, shouldn't it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314597849 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,22 +187,21 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, Review comment: @dongjoon-hyun I removed backquotes only around names that are not functions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314596606 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,22 +187,21 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, Review comment: I would guess, `year` belongs to a functions as well. Probably, this confuses the parser? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314596606 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,22 +187,21 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, Review comment: I would guess, `year` belongs to a function as well. Probably, this confuses the parser? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314596136 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,22 +187,21 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, Review comment: @dongjoon-hyun When I removed them, I got the error: ``` -- !query 13 SELECT '' AS `54`, d1 as `timestamp`, date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01' -- !query 13 schema struct<> -- !query 13 output org.apache.spark.sql.catalyst.parser.ParseException no viable alternative at input 'year'(line 2, pos 30) == SQL == SELECT '' AS `54`, d1 as `timestamp`, date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, --^^^ date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01' ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314595461 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,22 +187,21 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, ---date_part( 'usec', d1) AS usec ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; - --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'isoyear', d1) AS isoyear, date_part( 'week', d1) AS week, ---date_part( 'dow', d1) AS dow ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, Review comment: Thanks. I will remove the backquoting. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add
`date_part` function for timestamps/dates
URL: https://github.com/apache/spark/pull/25410#discussion_r314587020
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##
@@ -1409,48 +1409,7 @@ class AstBuilder(conf: SQLConf) extends
SqlBaseBaseVisitor[AnyRef] with Logging
* Create a Extract expression.
*/
override def visitExtract(ctx: ExtractContext): Expression = withOrigin(ctx)
{
-ctx.field.getText.toUpperCase(Locale.ROOT) match {
- case "MILLENNIUM" | "MILLENNIA" | "MIL" | "MILS" =>
-Millennium(expression(ctx.source))
- case "CENTURY" | "CENTURIES" | "C" | "CENT" =>
-Century(expression(ctx.source))
- case "DECADE" | "DECADES" | "DEC" | "DECS" =>
-Decade(expression(ctx.source))
- case "YEAR" | "Y" | "YEARS" | "YR" | "YRS" =>
-Year(expression(ctx.source))
- case "ISOYEAR" =>
-IsoYear(expression(ctx.source))
- case "QUARTER" | "QTR" =>
-Quarter(expression(ctx.source))
- case "MONTH" | "MON" | "MONS" | "MONTHS" =>
-Month(expression(ctx.source))
- case "WEEK" | "W" | "WEEKS" =>
-WeekOfYear(expression(ctx.source))
- case "DAY" | "D" | "DAYS" =>
-DayOfMonth(expression(ctx.source))
- case "DAYOFWEEK" =>
-DayOfWeek(expression(ctx.source))
- case "DOW" =>
-Subtract(DayOfWeek(expression(ctx.source)), Literal(1))
- case "ISODOW" =>
-Add(WeekDay(expression(ctx.source)), Literal(1))
- case "DOY" =>
-DayOfYear(expression(ctx.source))
- case "HOUR" | "H" | "HOURS" | "HR" | "HRS" =>
-Hour(expression(ctx.source))
- case "MINUTE" | "M" | "MIN" | "MINS" | "MINUTES" =>
-Minute(expression(ctx.source))
- case "SECOND" | "S" | "SEC" | "SECONDS" | "SECS" =>
-Second(expression(ctx.source))
- case "MILLISECONDS" | "MSEC" | "MSECS" | "MILLISECON" | "MSECONDS" |
"MS" =>
-Milliseconds(expression(ctx.source))
- case "MICROSECONDS" | "USEC" | "USECS" | "USECONDS" | "MICROSECON" |
"US" =>
-Microseconds(expression(ctx.source))
- case "EPOCH" =>
-Epoch(expression(ctx.source))
- case other =>
-throw new ParseException(s"Literals of type '$other' are currently not
supported.", ctx)
Review comment:
... but I can pass `ctx: ParserRuleContext` to `DatePart`'s constructor. I
am just not sure that is is good practice. WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314586539 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1963,3 +1963,77 @@ case class Epoch(child: Expression, timeZoneId: Option[String] = None) defineCodeGen(ctx, ev, c => s"$dtu.getEpoch($c, $zid)") } } + +@ExpressionDescription( + usage = "_FUNC_(field, source) - Extracts a part of the date/timestamp.", + arguments = """ +Arguments: + * field - selects which part of the source should be extracted. Supported string values are: +["MILLENNIUM", "MILLENNIA", "MIL", "MILS", + "CENTURY", "CENTURIES", "C", "CENT", + "DECADE", "DECADES", "DEC", "DECS", + "YEAR", "Y", "YEARS", "YR", "YRS", + "ISOYEAR", + "QUARTER", "QTR", + "MONTH", "MON", "MONS", "MONTHS", + "WEEK", "W", "WEEKS", + "DAY", "D", "DAYS", + "DAYOFWEEK", "DOW", "ISODOW", "DOY", + "HOUR", "H", "HOURS", "HR", "HRS", + "MINUTE", "M", "MIN", "MINS", "MINUTES", + "SECOND", "S", "SEC", "SECONDS", "SECS", + "MILLISECONDS", "MSEC", "MSECS", "MILLISECON", "MSECONDS", "MS", + "MICROSECONDS", "USEC", "USECS", "USECONDS", "MICROSECON", "US", + "EPOCH"] Review comment: Sounds good. I will do that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add
`date_part` function for timestamps/dates
URL: https://github.com/apache/spark/pull/25410#discussion_r314586294
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##
@@ -1409,48 +1409,7 @@ class AstBuilder(conf: SQLConf) extends
SqlBaseBaseVisitor[AnyRef] with Logging
* Create a Extract expression.
*/
override def visitExtract(ctx: ExtractContext): Expression = withOrigin(ctx)
{
-ctx.field.getText.toUpperCase(Locale.ROOT) match {
- case "MILLENNIUM" | "MILLENNIA" | "MIL" | "MILS" =>
-Millennium(expression(ctx.source))
- case "CENTURY" | "CENTURIES" | "C" | "CENT" =>
-Century(expression(ctx.source))
- case "DECADE" | "DECADES" | "DEC" | "DECS" =>
-Decade(expression(ctx.source))
- case "YEAR" | "Y" | "YEARS" | "YR" | "YRS" =>
-Year(expression(ctx.source))
- case "ISOYEAR" =>
-IsoYear(expression(ctx.source))
- case "QUARTER" | "QTR" =>
-Quarter(expression(ctx.source))
- case "MONTH" | "MON" | "MONS" | "MONTHS" =>
-Month(expression(ctx.source))
- case "WEEK" | "W" | "WEEKS" =>
-WeekOfYear(expression(ctx.source))
- case "DAY" | "D" | "DAYS" =>
-DayOfMonth(expression(ctx.source))
- case "DAYOFWEEK" =>
-DayOfWeek(expression(ctx.source))
- case "DOW" =>
-Subtract(DayOfWeek(expression(ctx.source)), Literal(1))
- case "ISODOW" =>
-Add(WeekDay(expression(ctx.source)), Literal(1))
- case "DOY" =>
-DayOfYear(expression(ctx.source))
- case "HOUR" | "H" | "HOURS" | "HR" | "HRS" =>
-Hour(expression(ctx.source))
- case "MINUTE" | "M" | "MIN" | "MINS" | "MINUTES" =>
-Minute(expression(ctx.source))
- case "SECOND" | "S" | "SEC" | "SECONDS" | "SECS" =>
-Second(expression(ctx.source))
- case "MILLISECONDS" | "MSEC" | "MSECS" | "MILLISECON" | "MSECONDS" |
"MS" =>
-Milliseconds(expression(ctx.source))
- case "MICROSECONDS" | "USEC" | "USECS" | "USECONDS" | "MICROSECON" |
"US" =>
-Microseconds(expression(ctx.source))
- case "EPOCH" =>
-Epoch(expression(ctx.source))
- case other =>
-throw new ParseException(s"Literals of type '$other' are currently not
supported.", ctx)
Review comment:
`ParseException` requires either `ctx: ParserRuleContext` or `val start:
Origin, val stop: Origin` that are not available to me at the point. And it is
still `ParseException` in the output:
https://github.com/apache/spark/pull/25410/files#diff-6f4edc80e2cc973e748705e85a6053b4R514
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add
`date_part` function for timestamps/dates
URL: https://github.com/apache/spark/pull/25410#discussion_r314189408
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
##
@@ -1963,3 +1963,64 @@ case class Epoch(child: Expression, timeZoneId:
Option[String] = None)
defineCodeGen(ctx, ev, c => s"$dtu.getEpoch($c, $zid)")
}
}
+
+@ExpressionDescription(
+ usage = "_FUNC_(field, source) - Extracts a part of the date/timestamp.",
+ arguments = """
+Arguments:
+ * field - selects which part of the source should be extracted.
Supported string values are:
+["MILLENNIUM", "CENTURY", "DECADE", "YEAR", "QUARTER", "MONTH",
+ "WEEK", "DAY", "DAYOFWEEK", "DOW", "ISODOW", "DOY",
+ "HOUR", "MINUTE", "SECOND"]
+ * source - a date (or timestamp) column from where `field` should be
extracted
+ """,
+ examples = """
+Examples:
+ > SELECT _FUNC_('YEAR', TIMESTAMP '2019-08-12 01:00:00.123456');
+ 2019
+ > SELECT _FUNC_('week', timestamp'2019-08-12 01:00:00.123456');
+ 33
+ > SELECT _FUNC_('doy', DATE'2019-08-12');
+ 224
+ """,
+ since = "3.0.0")
+case class DatePart(field: Expression, source: Expression, child: Expression)
+ extends RuntimeReplaceable {
+
+ def this(field: Expression, source: Expression) {
+this(field, source, {
+ if (!field.foldable) {
+throw new AnalysisException("The field parameter needs to be a
foldable string value.")
Review comment:
According to PostgreSQL docs
https://www.postgresql.org/docs/11/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT:
>_source_ must be a value expression ... the _**field**_ parameter
needs to be **a string value**
Accepting _field_ as an expression is undocumented feature. We could support
that separately if it is needed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314188108 ## File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/timestamp.sql ## @@ -187,12 +187,11 @@ SELECT '' AS date_trunc_week, date_trunc( 'week', timestamp '2004-02-29 15:44:17 -- WHERE d1 BETWEEN timestamp '1902-01-01' --AND timestamp '2038-01-01'; --- [SPARK-28420] Date/Time Functions: date_part --- SELECT '' AS "54", d1 as "timestamp", ---date_part( 'year', d1) AS year, date_part( 'month', d1) AS month, ---date_part( 'day', d1) AS day, date_part( 'hour', d1) AS hour, ---date_part( 'minute', d1) AS minute, date_part( 'second', d1) AS second ---FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; +SELECT '' AS `54`, d1 as `timestamp`, +date_part( 'year', d1) AS `year`, date_part( 'month', d1) AS `month`, +date_part( 'day', d1) AS `day`, date_part( 'hour', d1) AS `hour`, +date_part( 'minute', d1) AS `minute`, date_part( 'second', d1) AS `second` +FROM TIMESTAMP_TBL WHERE d1 BETWEEN '1902-01-01' AND '2038-01-01'; -- SELECT '' AS "54", d1 as "timestamp", --date_part( 'quarter', d1) AS quarter, date_part( 'msec', d1) AS msec, Review comment: I uncommented those 2 queries This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
MaxGekk commented on a change in pull request #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#discussion_r314188026 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1963,3 +1963,64 @@ case class Epoch(child: Expression, timeZoneId: Option[String] = None) defineCodeGen(ctx, ev, c => s"$dtu.getEpoch($c, $zid)") } } + +@ExpressionDescription( + usage = "_FUNC_(field, source) - Extracts a part of the date/timestamp.", + arguments = """ +Arguments: + * field - selects which part of the source should be extracted. Supported string values are: +["MILLENNIUM", "CENTURY", "DECADE", "YEAR", "QUARTER", "MONTH", + "WEEK", "DAY", "DAYOFWEEK", "DOW", "ISODOW", "DOY", + "HOUR", "MINUTE", "SECOND"] Review comment: I documented all values for consistency. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
