[
https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878506#comment-15878506
]
Munagala V. Ramanath commented on APEXMALHAR-2419:
--------------------------------------------------
To clarify, the point of the logs above is manifold:
1. The semantics of getLargestCompletedWindow() call are unclear -- what does
"completed"
mean ? We have well defined meanings for "committed" and "checkpointed"
windows but not
for "completed"; the only reasonable meaning I can think of is: a window in
which endWindow()
has been called. It appears that operators are using this call with this
expectation.
2. If the above interpretation is correct, we expect this call to return a new
value after each
endWindow() call but that is not happening -- the value is constant for the
entire duration of a
single run.
2. The operators that use WindowDataManager need to know when they are
processing
replayed windows and when replay is finished. Given the current behavior,
there is currently
no way for operators to determine when replay is done using
WindowDataManager.
> KafkaSinglePortExactlyOnceOutputOperator fails on recovery
> ----------------------------------------------------------
>
> Key: APEXMALHAR-2419
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2419
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: Munagala V. Ramanath
> Assignee: Sandesh
>
> The KafkaSinglePortExactlyOnceOutputOperator fails on recovery with the
> message: "Violates Exactly once. Not all the tuples received after operator
> reset."
> This is because of this check in endWindow():
> {code}
> if (!partialWindowTuples.isEmpty() && windowId >
> windowDataManager.getLargestCompletedWindow()) {
> throw new RuntimeException("Violates Exactly once. Not all the tuples
> received after operator reset.");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
