[
https://issues.apache.org/jira/browse/GIRAPH-574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604468#comment-13604468
]
Eli Reisman commented on GIRAPH-574:
------------------------------------
Think aggregators, master compute, other things? also...
> Move Giraph Master node functionality to AppMaster or launch directly from
> AppMaster in YARN profile
> ----------------------------------------------------------------------------------------------------
>
> Key: GIRAPH-574
> URL: https://issues.apache.org/jira/browse/GIRAPH-574
> Project: Giraph
> Issue Type: Improvement
> Reporter: Eli Reisman
> Priority: Minor
>
> As folks read the Giraph on YARN code it is inevitable it will occur to
> someone "Well, if the job fails when the ApplicationMaster fails, could we
> move some or all of our Master task functions there and just call it master?"
> Yes. In two ways.
> One, we launch a dedicated master process marked as such with setup
> responsibilities, and we assess from the app master how the launch went. We
> keep launching "masters" until one takes. Then, we launch the workers.
> Another is to simply run MasterThread and associated stuff from the App
> Master directly, and when we know its up and running properly, only then does
> app master launch the workers.
> The YARN app master can be rebooted is designed to be a place for
> fault-tolerant "master node" stuff to happen. However, I think a larger
> purpose is to act as a meta-master for launching a DAG of jobs within the run
> of a single app master lifecycle. Or the app master cant act as any of these
> things, or something else I haven't thought of. The architecture is fairly
> malleable.
> This is not a requirement for us, and maybe not a good idea at all. This is
> just a placeholder JIRA to discuss and collect ideas since as I said above
> someone is going to bring it up ;)
> Thank you for reading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
