tomstepp commented on code in PR #33591:
URL: https://github.com/apache/beam/pull/33591#discussion_r1927419071
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/io/UnboundedSource.java:
##########
@@ -93,6 +93,21 @@ public boolean requiresDeduping() {
return false;
}
+ /**
+ * If offsetBasedDeduplicationEnabled returns true, then the UnboundedSource
needs to provide the
+ * following:
+ *
+ * <ul>
+ * <li>UnboundedReader which provides offsets that are unique for each
element and
+ * lexicographically ordered.
+ * <li>CheckpointMark which provides an offset greater than all elements
read and less than or
+ * equal to the next offset that will be read.
+ * </ul>
+ */
+ public boolean offsetBasedDeduplicationEnabled() {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
