Repository: spark
Updated Branches:
  refs/heads/master 8f0c35a4d -> d720a4019


[SPARK-17297][DOCS] Clarify window/slide duration as absolute time, not 
relative to a calendar

## What changes were proposed in this pull request?

Clarify that slide and window duration are absolute, and not relative to a 
calendar.

## How was this patch tested?

Doc build (no functional change)

Author: Sean Owen <so...@cloudera.com>

Closes #15142 from srowen/SPARK-17297.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d720a401
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d720a401
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d720a401

Branch: refs/heads/master
Commit: d720a4019460b6c284d0473249303c349df60a1f
Parents: 8f0c35a
Author: Sean Owen <so...@cloudera.com>
Authored: Mon Sep 19 09:38:25 2016 +0100
Committer: Sean Owen <so...@cloudera.com>
Committed: Mon Sep 19 09:38:25 2016 +0100

----------------------------------------------------------------------
 R/pkg/R/functions.R                                  |  8 ++++++--
 .../main/scala/org/apache/spark/sql/functions.scala  | 15 +++++++++++----
 2 files changed, 17 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/d720a401/R/pkg/R/functions.R
----------------------------------------------------------------------
diff --git a/R/pkg/R/functions.R b/R/pkg/R/functions.R
index ceedbe7..4d94b4c 100644
--- a/R/pkg/R/functions.R
+++ b/R/pkg/R/functions.R
@@ -2713,11 +2713,15 @@ setMethod("from_unixtime", signature(x = "Column"),
 #' @param x a time Column. Must be of TimestampType.
 #' @param windowDuration a string specifying the width of the window, e.g. '1 
second',
 #'                       '1 day 12 hours', '2 minutes'. Valid interval strings 
are 'week',
-#'                       'day', 'hour', 'minute', 'second', 'millisecond', 
'microsecond'.
+#'                       'day', 'hour', 'minute', 'second', 'millisecond', 
'microsecond'. Note that
+#'                       the duration is a fixed length of time, and does not 
vary over time
+#'                       according to a calendar. For example, '1 day' always 
means 86,400,000
+#'                       milliseconds, not a calendar day.
 #' @param slideDuration a string specifying the sliding interval of the 
window. Same format as
 #'                      \code{windowDuration}. A new window will be generated 
every
 #'                      \code{slideDuration}. Must be less than or equal to
-#'                      the \code{windowDuration}.
+#'                      the \code{windowDuration}. This duration is likewise 
absolute, and does not
+#'                      vary according to a calendar.
 #' @param startTime the offset with respect to 1970-01-01 00:00:00 UTC with 
which to start
 #'                  window intervals. For example, in order to have hourly 
tumbling windows
 #'                  that start 15 minutes past the hour, e.g. 12:15-13:15, 
13:15-14:15... provide

http://git-wip-us.apache.org/repos/asf/spark/blob/d720a401/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/functions.scala 
b/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
index 18e736a..960c87f 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
@@ -2606,12 +2606,15 @@ object functions {
    *                   The time column must be of TimestampType.
    * @param windowDuration A string specifying the width of the window, e.g. 
`10 minutes`,
    *                       `1 second`. Check 
[[org.apache.spark.unsafe.types.CalendarInterval]] for
-   *                       valid duration identifiers.
+   *                       valid duration identifiers. Note that the duration 
is a fixed length of
+   *                       time, and does not vary over time according to a 
calendar. For example,
+   *                       `1 day` always means 86,400,000 milliseconds, not a 
calendar day.
    * @param slideDuration A string specifying the sliding interval of the 
window, e.g. `1 minute`.
    *                      A new window will be generated every 
`slideDuration`. Must be less than
    *                      or equal to the `windowDuration`. Check
    *                      [[org.apache.spark.unsafe.types.CalendarInterval]] 
for valid duration
-   *                      identifiers.
+   *                      identifiers. This duration is likewise absolute, and 
does not vary
+    *                     according to a calendar.
    * @param startTime The offset with respect to 1970-01-01 00:00:00 UTC with 
which to start
    *                  window intervals. For example, in order to have hourly 
tumbling windows that
    *                  start 15 minutes past the hour, e.g. 12:15-13:15, 
13:15-14:15... provide
@@ -2660,11 +2663,15 @@ object functions {
    *                   The time column must be of TimestampType.
    * @param windowDuration A string specifying the width of the window, e.g. 
`10 minutes`,
    *                       `1 second`. Check 
[[org.apache.spark.unsafe.types.CalendarInterval]] for
-   *                       valid duration identifiers.
+   *                       valid duration identifiers. Note that the duration 
is a fixed length of
+   *                       time, and does not vary over time according to a 
calendar. For example,
+   *                       `1 day` always means 86,400,000 milliseconds, not a 
calendar day.
    * @param slideDuration A string specifying the sliding interval of the 
window, e.g. `1 minute`.
    *                      A new window will be generated every 
`slideDuration`. Must be less than
    *                      or equal to the `windowDuration`. Check
-   *                      [[org.apache.spark.unsafe.types.CalendarInterval]] 
for valid duration.
+   *                      [[org.apache.spark.unsafe.types.CalendarInterval]] 
for valid duration
+   *                      identifiers. This duration is likewise absolute, and 
does not vary
+   *                     according to a calendar.
    *
    * @group datetime_funcs
    * @since 2.0.0


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

Reply via email to