Hey Bikas, One other question: does the RM recover from ZK if I restart it? It seems like, when I bounce an RM, it restarts with a blank slate, even when configured to talk to ZooKeeper. Is this also not implemented yet?
Cheers, Chris On 9/19/12 11:04 AM, "Bikas Saha" <[email protected]> wrote: >Sorry about that. There is some code in the RM that's supposed to be used >to save state but its not used. So neither state preserving restart works >nor failover. > >I have been guilty of being tardy on this project but I hope to be working >on the restart piece fairly soon. Please do watch MAPREDUCE-4326 (this >should soon move to the YARN sub-project) for details. > >Bikas > >-----Original Message----- >From: Chris Riccomini [mailto:[email protected]] >Sent: Wednesday, September 19, 2012 10:21 AM >To: [email protected] >Subject: Re: More than one RM in YARN? > >Hey Bikas, > >Correct. > >Initially, I'd hoped that NMs failed over to appropriate RMs via ZK >notifications. When that didn't seem to be implemented, I hoped that RMs, >at least kept in sync with each other via ZK, so that I could use a round >robin DNS (or HW load balancer) for the RMs. When that didn't seem to be >implemented, I decided to just use DNS and a manual failover. > >Simple is best, I suppose. > >Cheers, >Chris > >On 9/19/12 10:18 AM, "Bikas Saha" <[email protected]> wrote: > >>If I understand you correctly, your target scenario is failover? >> >>Its good to know that you have something that works for you right now. >> >>Bikas >> >>-----Original Message----- >>From: Chris Riccomini [mailto:[email protected]] >>Sent: Wednesday, September 19, 2012 10:06 AM >>To: [email protected] >>Subject: Re: More than one RM in YARN? >> >>Hey Bikas, >> >>Ideally, I'd like to run them concurrently. What I'm discovering is >>that this is not (currently) possible, so I'm settling for a manual >>fail over using DNS, which should suffice for the time being. >> >>Cheers, >>Chris >> >>On 9/19/12 10:05 AM, "Bikas Saha" <[email protected]> wrote: >> >>>Chris, >>> >>>Could you please elaborate a bit on the use case you are targeting? Do >>>you want to run multiple active RM's in the cluster? And they are >>>being used concurrently? Or are you talking about failover scenarios? >>> >>>Bikas >>> >>>-----Original Message----- >>>From: Harsh J [mailto:[email protected]] >>>Sent: Tuesday, September 18, 2012 9:37 PM >>>To: [email protected] >>>Subject: Re: More than one RM in YARN? >>> >>>Chris, >>> >>>Not yet, it is coming: >>>https://issues.apache.org/jira/browse/MAPREDUCE-4326 should go in >>>first, followed by the HA work after it via >>>https://issues.apache.org/jira/browse/MAPREDUCE-4345 (Latter isn't a >>>dupe of the former exactly). >>> >>>On Wed, Sep 19, 2012 at 6:12 AM, Chris Riccomini >>><[email protected]> >>>wrote: >>>> Hey Guys, >>>> >>>> Is anyone running more than one RM on a YARN cluster (backed by ZK)? >>>We're looking at this, and I want to know if there's a standard way to >>>do it. Are people doing round robin DNS, load balancers, etc? Is it >>>possible to configure node managers to have a list of RMs rather than >>>a >>single one? >>>> >>>> Thanks! >>>> Chris >>> >>> >>> >>>-- >>>Harsh J
