[
https://issues.apache.org/jira/browse/APEXCORE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15514117#comment-15514117
]
Tushar Gosavi commented on APEXCORE-532:
----------------------------------------
If newly added operator is attached to some operator which was already exists
in the DAG,
- It starts from activation checkpoint
- bufferserver subscriber use activation checkpoint id to set the initial
baseSeconds (setup of BufferServerSubscriber).
- when new data is received, we get partital windowId form tuple, or it with
baseSeconds and
construct full windowId.
baseSeconds is expected to set to correct value if operator starts from known
checkpoint,
during initial dag deployment input operators sends resetWindow tuple, which
sets baseSeconds
for all downstream operators. In this case operator was added in the end and
was started
from activation checkpoint. but no resetWindow tuple was received to set its
baseSeconds to
correct value.
In this case the starting checkpoint is different than windowId, may be we
should use two fields in
deploy info
- one for checkpoint from where to load the state
- starting baseSeconds which is taken from existing upstream to correctly set
the windowIds in absense of resetWindow tuple.
> New dynamically added operator does not start with correct windowId.
> --------------------------------------------------------------------
>
> Key: APEXCORE-532
> URL: https://issues.apache.org/jira/browse/APEXCORE-532
> Project: Apache Apex Core
> Issue Type: Bug
> Reporter: Tushar Gosavi
> Priority: Critical
>
> During dynamic DAG change, If new operator is added and connected to existing
> operator, it does not starts with correct windowId. The baseSeconds is set to
> 0 causing windowId management problems at master effectively halting purge
> from buffer server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
