Eli Reisman created GIRAPH-571:
----------------------------------
Summary: Giraph on YARN could launch a job-local ZK instance from
the AppMaster
Key: GIRAPH-571
URL: https://issues.apache.org/jira/browse/GIRAPH-571
Project: Giraph
Issue Type: Improvement
Components: zookeeper
Affects Versions: 0.2.0
Reporter: Eli Reisman
Once GIRAPH-13 is in, we can think differently about a lot of things if we
choose too.
for one thing, we have had problems launching job-local ZK instances. We could
(for YARN) move that functionality to the App Master, having it launch a
container just for ZK and populating the Configuration's giraph.zkList setting
so when the MRv1 ZK manager code sees the Conf, it will think we already have a
non-job-local ZK at zkList's host and port, and will just connect instead of
starting another local instance, making the whole affair transparent to
existing Giraph code.
Not important, but the YARN patch is currently defaulting to only execute jobs
with a non-local ZK instance already running, and giraph.zkList populated with
its host:port.
Its quite possible when we get our MRv1 job local ZK working again, we can
remove this and it will work right out of the box, there's no reason it won't.
But managing extraneous services (especially those that hold up the job setup
like launching a ZK) is what the YARN AppMaster is all about anyway. i haven't
been able to get our local ZK instance to launch outside of test cases for a
while now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
