Tushar Gosavi commented on APEXCORE-532:

If newly added operator is attached to some operator which was already exists 
in the DAG,
- It starts from activation checkpoint
- bufferserver subscriber use activation checkpoint id to set the initial 
baseSeconds (setup of BufferServerSubscriber).
- when new data is received, we get partital windowId form tuple, or it with 
baseSeconds and
  construct full windowId.

baseSeconds is expected to set to correct value if operator starts from known 
during initial dag deployment input operators sends resetWindow tuple, which 
sets baseSeconds
for all downstream operators. In this case operator was added in the end and 
was started
from activation checkpoint. but no resetWindow tuple was received to set its 
baseSeconds to
correct value.

In this case the starting checkpoint is different than windowId, may be we 
should use two fields in
deploy info
- one for checkpoint from where to load the state
- starting baseSeconds which is taken from existing upstream to correctly set 
the windowIds in absense of resetWindow tuple.

> New dynamically added operator does not start with correct windowId.
> --------------------------------------------------------------------
>                 Key: APEXCORE-532
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-532
>             Project: Apache Apex Core
>          Issue Type: Bug
>            Reporter: Tushar Gosavi
>            Priority: Critical
> During dynamic DAG change, If new operator is added and connected to existing 
> operator, it does not starts with correct windowId. The baseSeconds is set to 
> 0 causing windowId management problems at master effectively halting purge 
> from buffer server.

This message was sent by Atlassian JIRA

Reply via email to