[
https://issues.apache.org/jira/browse/TEZ-2633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644917#comment-14644917
]
Bikas Saha commented on TEZ-2633:
---------------------------------
Attaching a patch with the changes. The patch is mainly a refactoring and
updating some functionality based on that refactoring so that new tests can be
added to verify the changes. VertexManagers can now receive attempt info on
their callbacks and can forward scheduling dependencies (if based on attempts)
when they schedule tasks. This will be consumed later on in TEZ-2646. Backwards
compatibility has been maintained. [~rajesh.balamohan] please review, specially
the changes in the different vertex managers.
The other thing that needs to be more widely discussed here is how we expose
vertex/task/attempt identification. We could directly expose our internal ids
or we could wrap them and use their functionality without exposing them
directly. This may be related to some of the needs to expose similar
information to other plugins created under TEZ-2003. I have gone with the
wrapper approach for the following reasons
1) It is generally not a good idea to expose internal ids and implicit logic
encoded in them. This can be a compatibility and maintainability issue going
forward.
2) Creating an interface for users and wrapping internal ids under them allows
reuse of the all the functionality in the ids that we want to not duplicate.
3) External interface can expose vertex and dag names, with are useful from the
user code point of view (because they specify and use those names in their
code. They may not know the exact integer identifier for the dag or vertex
before runtime.
[~hitesh] [~sseth] [~rajesh.balamohan] [~pramachandran] Thoughts?
> Allow VertexManagerPlugins to receive and report based on attempts instead of
> tasks
> -----------------------------------------------------------------------------------
>
> Key: TEZ-2633
> URL: https://issues.apache.org/jira/browse/TEZ-2633
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-2633.1.patch
>
>
> If the same event is sent from an attempt and its retry then there is no way
> to differentiate between them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
