tomstepp commented on code in PR #33591:
URL: https://github.com/apache/beam/pull/33591#discussion_r1925655873
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/io/UnboundedSource.java:
##########
@@ -93,6 +93,21 @@ public boolean requiresDeduping() {
return false;
}
+ /**
+ * If isOffsetDeduplication returns true, then the UnboundedSource needs to
+ * provide the following:
+ *
+ * <ul>
+ * <li>UnboundedReader which provides offsets that are unique for each
+ * element and lexicographically ordered.</li>
+ * <li>CheckpointMark which provides an offset greater than all elements
+ * read and less than or equal to the next offset that will be
read.</li>
+ * </ul>
+ */
+ public boolean isOffsetDeduplication() {
Review Comment:
Updated to offsetBasedDeduplicationEnabled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
