kennknowles commented on code in PR #40294:
URL: https://github.com/apache/beam/pull/40294#discussion_r4259520014
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Watch.java:
##########
@@ -769,6 +779,40 @@ public Growth<InputT, OutputT, KeyT>
withTimestampCursor(Duration allowedLatenes
return
toBuilder().setTimestampCursorAllowedLateness(allowedLateness).build();
}
+ /**
+ * Limits how many previously unseen outputs from one poll are kept as
checkpointed pending
+ * work.
+ *
+ * <p>If an incomplete poll contains more than {@code maxPendingResults}
distinct unseen
+ * outputs, the oldest outputs are emitted first and the rest are omitted
from that round. The
+ * poll's watermark is held at or before the omitted outputs so the
function will be called
+ * again to retrieve them. A {@link PollResult#complete} is never limited
because its contract
+ * promises that the function will not be called again.
+ *
+ * <p>This option must only be used with a {@link PollFn} that reports
omitted outputs again in
+ * subsequent polls. Otherwise those outputs cannot be recovered. Without
this option, all
+ * unseen outputs from each poll continue to be kept, preserving the
existing behavior. This
+ * limits the persisted pending outputs, not the result list returned by
{@link PollFn} or the
+ * temporary hashes used to preserve duplicate-key semantics while
selecting the oldest outputs.
+ *
+ * <p>While a poll is truncated, its termination condition is deferred
until a later poll is not
+ * truncated, so omitted outputs are not silently lost. This option is
therefore intended for a
+ * snapshot-style {@link PollFn} that repeats a finite set of outputs
until they are observed. A
+ * function that continuously returns more distinct unseen outputs than
this limit can keep the
+ * input polling indefinitely even if its termination condition would
otherwise stop it.
+ *
+ * <p>Combine this with {@link #withTimestampCursor()} to time-bound the
completed-key state
+ * when output timestamps advance past the retention floor. It does not
impose a count bound on
+ * completed keys that share a timestamp still at or above that floor.
+ */
+ public Growth<InputT, OutputT, KeyT> withMaxPendingResults(int
maxPendingResults) {
Review Comment:
This should be `@Nullable Integer` or otherwise have a way of explicitly
indicating it is unset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
