zhengruifeng commented on code in PR #55828:
URL: https://github.com/apache/spark/pull/55828#discussion_r3278143360
##########
python/pyspark/sql/functions/builtin.py:
##########
@@ -6464,6 +6464,84 @@ def rank() -> Column:
return _invoke_function("rank")
+@_try_remote_functions
+def counter_diff(value: "ColumnOrName", startTime: Optional["ColumnOrName"] =
None) -> Column:
+ """
+ Window function: computes the differences between consecutive cumulative
counter values in a
+ time series, thereby converting the counter from the cumulative to the
delta format.
+
+ Gracefully handles counter resets by returning NULL. Counter resets are
detected when the
+ counter value decreases, or when the start time advances between rows.
+
+ Use the PARTITION BY clause of the window to separate independent
counters. This is done by
+ specifying all columns which uniquely identify a time series. These are
typically the counter
+ name and any attributes tied to the counter.
+
+ Use the ORDER BY clause of the window to order the observations by the
associated timestamp
+ in ascending order.
+
+ .. versionadded:: 4.2.0
Review Comment:
this PR is not in branch-4.2, shall we:
1, update the version to 4.3, or
2, backport the PR to branch-4.2?
@cloud-fan @pnikic-db
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
