HyukjinKwon commented on code in PR #36063:
URL: https://github.com/apache/spark/pull/36063#discussion_r846943785
##########
python/pyspark/pandas/window.py:
##########
@@ -1749,6 +1752,188 @@ def var(self) -> FrameLike:
return super().var()
+class ExponentialMovingLike(Generic[FrameLike], metaclass=ABCMeta):
+ def __init__(
+ self,
+ window: WindowSpec,
+ com: Optional[float] = None,
+ span: Optional[float] = None,
+ halflife: Optional[float] = None,
+ alpha: Optional[float] = None,
+ min_periods: Optional[int] = None,
+ ):
+ if (min_periods is not None) and (min_periods < 0):
+ raise ValueError("min_periods must be >= 0")
+ if min_periods is None:
+ min_periods = 0
+ self._min_periods = min_periods
+
+ self._window = window
+ # This unbounded Window is later used to handle 'min_periods' for now.
+ self._unbounded_window =
Window.orderBy(NATURAL_ORDER_COLUMN_NAME).rowsBetween(
Review Comment:
Likewise, this is actually pretty critical that we can't specify partitions
while I don't have other better ideas. We should better document it somewhere
that the performance is expected to be slow, and OOM can happen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
