[spark] branch master updated (726f6d3 -> 18e8d1d)
This is an automated email from the ASF dual-hosted git repository. lixiao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 726f6d3 [SPARK-30184][SQL] Implement a helper method for aliasing functions add 18e8d1d [SPARK-30307][SQL] remove ReusedQueryStageExec No new revisions were added by this update. Summary of changes: .../execution/adaptive/AdaptiveSparkPlanExec.scala | 17 +-- .../adaptive/DemoteBroadcastHashJoin.scala | 4 +- .../adaptive/LogicalQueryStageStrategy.scala | 4 +- .../adaptive/OptimizeLocalShuffleReader.scala | 56 ++ .../sql/execution/adaptive/QueryStageExec.scala| 116 - .../adaptive/ReduceNumShufflePartitions.scala | 22 ++-- .../spark/sql/execution/exchange/Exchange.scala| 2 +- .../ReduceNumShufflePartitionsSuite.scala | 9 +- .../adaptive/AdaptiveQueryExecSuite.scala | 9 +- 9 files changed, 108 insertions(+), 131 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (dea1823 -> 726f6d3)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from dea1823 [SPARK-30309][SQL] Mark `Filter` as a `sealed` class add 726f6d3 [SPARK-30184][SQL] Implement a helper method for aliasing functions No new revisions were added by this update. Summary of changes: .../sql/catalyst/analysis/FunctionRegistry.scala | 70 ++ .../aggregate/ApproximatePercentile.scala | 6 +- .../catalyst/expressions/aggregate/Average.scala | 4 +- .../expressions/aggregate/CentralMomentAgg.scala | 6 +- .../sql/catalyst/expressions/aggregate/First.scala | 6 +- .../sql/catalyst/expressions/aggregate/Last.scala | 6 +- .../expressions/aggregate/UnevaluableAggs.scala| 10 ++-- .../sql/catalyst/optimizer/finishAnalysis.scala| 4 +- .../analysis/ExpressionTypeCheckingSuite.scala | 4 +- .../resources/sql-tests/results/group-by.sql.out | 2 +- .../results/postgreSQL/window_part4.sql.out| 16 ++--- .../sql-tests/results/udf/udf-group-by.sql.out | 12 ++-- 12 files changed, 61 insertions(+), 85 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (dea1823 -> 726f6d3)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from dea1823 [SPARK-30309][SQL] Mark `Filter` as a `sealed` class add 726f6d3 [SPARK-30184][SQL] Implement a helper method for aliasing functions No new revisions were added by this update. Summary of changes: .../sql/catalyst/analysis/FunctionRegistry.scala | 70 ++ .../aggregate/ApproximatePercentile.scala | 6 +- .../catalyst/expressions/aggregate/Average.scala | 4 +- .../expressions/aggregate/CentralMomentAgg.scala | 6 +- .../sql/catalyst/expressions/aggregate/First.scala | 6 +- .../sql/catalyst/expressions/aggregate/Last.scala | 6 +- .../expressions/aggregate/UnevaluableAggs.scala| 10 ++-- .../sql/catalyst/optimizer/finishAnalysis.scala| 4 +- .../analysis/ExpressionTypeCheckingSuite.scala | 4 +- .../resources/sql-tests/results/group-by.sql.out | 2 +- .../results/postgreSQL/window_part4.sql.out| 16 ++--- .../sql-tests/results/udf/udf-group-by.sql.out | 12 ++-- 12 files changed, 61 insertions(+), 85 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated: [SPARK-30236][SQL][DOCS][2.4] Clarify date and time patterns supported in the related functions
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new bbeb191 [SPARK-30236][SQL][DOCS][2.4] Clarify date and time patterns
supported in the related functions
bbeb191 is described below
commit bbeb1911c8c8291cf8ebed85ecf6590d850616bd
Author: John Ayad
AuthorDate: Thu Dec 19 12:39:10 2019 -0800
[SPARK-30236][SQL][DOCS][2.4] Clarify date and time patterns supported in
the related functions
### What changes were proposed in this pull request?
Link to appropriate Java Class with list of date/time patterns supported
### Why are the changes needed?
Avoid confusion on the end-user's side of things, as seen in questions like
[this](https://stackoverflow.com/questions/54496878/date-format-conversion-is-adding-1-year-to-the-border-dates)
on StackOverflow
### Does this PR introduce any user-facing change?
Yes, Docs are updated.
### How was this patch tested?
Built docs:
Closes #26867 from johnhany97/SPARK-30236-2.4.
Authored-by: John Ayad
Signed-off-by: Dongjoon Hyun
---
.../catalyst/expressions/datetimeExpressions.scala | 46 --
1 file changed, 42 insertions(+), 4 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
index 45e17ae..8b3f532 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
@@ -542,6 +542,12 @@ case class WeekOfYear(child: Expression) extends
UnaryExpression with ImplicitCa
// scalastyle:off line.size.limit
@ExpressionDescription(
usage = "_FUNC_(timestamp, fmt) - Converts `timestamp` to a value of string
in the format specified by the date format `fmt`.",
+ arguments = """
+Arguments:
+ * timestamp - A date/timestamp or string to be converted to the given
format.
+ * fmt - Date/time format pattern to follow. See
`java.text.SimpleDateFormat` for valid date
+ and time format patterns.
+ """,
examples = """
Examples:
> SELECT _FUNC_('2016-04-08', 'y');
@@ -583,7 +589,14 @@ case class DateFormatClass(left: Expression, right:
Expression, timeZoneId: Opti
* Deterministic version of [[UnixTimestamp]], must have at least one
parameter.
*/
@ExpressionDescription(
- usage = "_FUNC_(expr[, pattern]) - Returns the UNIX timestamp of the given
time.",
+ usage = "_FUNC_(timeExp[, format]) - Returns the UNIX timestamp of the given
time.",
+ arguments = """
+Arguments:
+ * timeExp - A date/timestamp or string which is returned as a UNIX
timestamp.
+ * format - Date/time format pattern to follow. Ignored if `timeExp` is
not a string.
+ Default value is "-MM-dd HH:mm:ss". See
`java.text.SimpleDateFormat`
+ for valid date and time format patterns.
+ """,
examples = """
Examples:
> SELECT _FUNC_('2016-04-08', '-MM-dd');
@@ -622,7 +635,14 @@ case class ToUnixTimestamp(
* second parameter.
*/
@ExpressionDescription(
- usage = "_FUNC_([expr[, pattern]]) - Returns the UNIX timestamp of current
or specified time.",
+ usage = "_FUNC_([timeExp[, format]]) - Returns the UNIX timestamp of current
or specified time.",
+ arguments = """
+Arguments:
+ * timeExp - A date/timestamp or string. If not provided, this defaults
to current time.
+ * format - Date/time format pattern to follow. Ignored if `timeExp` is
not a string.
+ Default value is "-MM-dd HH:mm:ss". See
`java.text.SimpleDateFormat`
+ for valid date and time format patterns.
+ """,
examples = """
Examples:
> SELECT _FUNC_();
@@ -777,6 +797,12 @@ abstract class UnixTime
*/
@ExpressionDescription(
usage = "_FUNC_(unix_time, format) - Returns `unix_time` in the specified
`format`.",
+ arguments = """
+Arguments:
+ * unix_time - UNIX Timestamp to be converted to the provided format.
+ * format - Date/time format pattern to follow. See
`java.text.SimpleDateFormat`
+ for valid date and time format patterns.
+ """,
examples = """
Examples:
> SELECT _FUNC_(0, '-MM-dd HH:mm:ss');
@@ -1304,6 +1330,12 @@ case class ToUTCTimestamp(left: Expression, right:
Expression)
a date. Returns null with invalid input. By default, it follows casting
rules to a date if
the `fmt` is omitted.
""",
+ arguments = """
+Arguments:
+ * da
[spark] branch master updated (ab87bfd -> dea1823)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from ab87bfd [SPARK-29450][SS] Measure the number of output rows for streaming aggregation with append mode add dea1823 [SPARK-30309][SQL] Mark `Filter` as a `sealed` class No new revisions were added by this update. Summary of changes: sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (ab87bfd -> dea1823)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from ab87bfd [SPARK-29450][SS] Measure the number of output rows for streaming aggregation with append mode add dea1823 [SPARK-30309][SQL] Mark `Filter` as a `sealed` class No new revisions were added by this update. Summary of changes: sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (2af5237 -> ab87bfd)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 2af5237 [SPARK-29918][SQL][FOLLOWUP][TEST] Fix arrayOffset in `RecordBinaryComparatorSuite` add ab87bfd [SPARK-29450][SS] Measure the number of output rows for streaming aggregation with append mode No new revisions were added by this update. Summary of changes: .../execution/streaming/statefulOperators.scala| 1 + .../sql/streaming/StreamingAggregationSuite.scala | 65 +- 2 files changed, 65 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (ab8eb86 -> 2af5237)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from ab8eb86 Revert "[SPARK-29629][SQL] Support typed integer literal expression" add 2af5237 [SPARK-29918][SQL][FOLLOWUP][TEST] Fix arrayOffset in `RecordBinaryComparatorSuite` No new revisions were added by this update. Summary of changes: .../apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (ab8eb86 -> 2af5237)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from ab8eb86 Revert "[SPARK-29629][SQL] Support typed integer literal expression" add 2af5237 [SPARK-29918][SQL][FOLLOWUP][TEST] Fix arrayOffset in `RecordBinaryComparatorSuite` No new revisions were added by this update. Summary of changes: .../apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
