[
https://issues.apache.org/jira/browse/TEZ-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130701#comment-14130701
]
Bikas Saha commented on TEZ-1539:
---------------------------------
TaskStateChangeNotification should be a separate jira. Its not following the
VertexStatusUpdateEvent/ENUM pattern that is being followed by
VertexStatusChangeNotification. There is no good reason for these similar
things to follow different code patterns. Nor are there specific tests for
TaskStateChangeNotification.
For the rest of the code related to this jira, there are a lot of changes which
overall look fine. But given the number of buffers and if else conditions do
you think the 2 test cases part of this patch are providing sufficient
coverage? There arent any e2e tests covering initializer events being generated
and nothing in our code/test code uses initializer events. Maybe if that had
been there then we may have realized the need for the current changes when we
had initially put in these new events. So adding them now would be a useful way
to ascertain that things work as expected before we put this in the wild and
have it used e2e for the first time by a user.
> Allow a FIRE_ONCE_ON_SUCCESS model for events generated by user code
> --------------------------------------------------------------------
>
> Key: TEZ-1539
> URL: https://issues.apache.org/jira/browse/TEZ-1539
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-1539.1.wip.txt, TEZ-1539.2.txt, TEZ-1539.3.txt
>
>
> Specifically for InputInitalizerEvents and VertexManagerEvents.
> Pasting comment from TEZ-1447
> In a majority of cases, events generated by different attempts of the same
> task will be identical - in which case just making use of the event generated
> by the first successful attempt is adequate. Doing something like this manes
> that users don't worry about retries, indices etc - and can just rely on
> receiving a set of events which are to be processed once the vertex succeeds.
> If different attempts of the same workload generate different events -
> processing is likely to be incorrect, since it's very possible for all data
> to be processed (VERTEX successful), then a failure and retry - which
> generates a different event. The initializer doesn't even run at this point,
> since it's already done it's work and is complete. Handling such scenarios,
> likely involves re-running the entire initializer and re-starting the vertex
> which processed the event from scratch. In situations like this, where data
> generated may be different, the best bet is for speculation to be disabled
> (when it's supported), and max-attempts to be set to 1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
