Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20400#discussion_r165840412
--- Diff: python/pyspark/sql/window.py ---
@@ -208,20 +236,27 @@ def rangeBetween(self, start, end):
and "5" means the five off after the current row.
We recommend users use ``Window.unboundedPreceding``,
``Window.unboundedFollowing``,
- and ``Window.currentRow`` to specify special boundary values,
rather than using integral
- values directly.
+ ``Window.currentRow``,
``pyspark.sql.functions.unboundedPreceding``,
+ ``pyspark.sql.functions.unboundedFollowing`` and
``pyspark.sql.functions.currentRow``
+ to specify special boundary values, rather than using integral
values directly.
:param start: boundary start, inclusive.
- The frame is unbounded if this is
``Window.unboundedPreceding``, or
+ The frame is unbounded if this is
``Window.unboundedPreceding``,
+ a column returned by
``pyspark.sql.functions.unboundedPreceding``, or
any value less than or equal to max(-sys.maxsize,
-9223372036854775808).
:param end: boundary end, inclusive.
- The frame is unbounded if this is
``Window.unboundedFollowing``, or
+ The frame is unbounded if this is
``Window.unboundedFollowing``,
+ a column returned by
``pyspark.sql.functions.unboundedFollowing``, or
any value greater than or equal to min(sys.maxsize,
9223372036854775807).
"""
- if start <= Window._PRECEDING_THRESHOLD:
- start = Window.unboundedPreceding
- if end >= Window._FOLLOWING_THRESHOLD:
- end = Window.unboundedFollowing
+ if isinstance(start, (int, long)) and isinstance(end, (int, long)):
+ if start <= Window._PRECEDING_THRESHOLD:
+ start = Window.unboundedPreceding
+ if end >= Window._FOLLOWING_THRESHOLD:
+ end = Window.unboundedFollowing
+ elif isinstance(start, Column) and isinstance(end, Column):
+ start = start._jc
+ end = end._jc
return WindowSpec(self._jspec.rangeBetween(start, end))
--- End diff --
`doctest.testmod(optionflags=doctest.NORMALIZE_WHITESPACE)` and let's
remove `<BLANKLINE>` at
https://github.com/apache/spark/pull/20400/files#r165254861
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
