Thanks a lot for the responses. I now have a better understanding. To answer to Jay's question , I think ZK can be used as for coordination service for a distributed program (you have built it on top of exposed granular api's) and it doesn't have features like resource management (including allocation of resources based on requests) of cluster nodes , which yarn has.
Rahul On Thu, May 30, 2013 at 5:59 PM, Jay Vyas <[email protected]> wrote: > What is the separation of concerns between YARN and Zookeeper? That is, > where does YARN leave off and where does Zookeeper begin? Or is there some > overlap.... > > > On Thu, May 30, 2013 at 2:42 AM, Krishna Kishore Bonagiri < > [email protected]> wrote: > >> Hi Rahul, >> >> It is at least because of the reasons that Vinod listed that makes my >> life easy for porting my application on to YARN instead of making it work >> in the Map Reduce framework. The main purpose of me using YARN is to >> exploit the resource management capabilities of YARN. >> >> Thanks, >> Kishore >> >> >> On Wed, May 29, 2013 at 11:00 PM, Rahul Bhattacharjee < >> [email protected]> wrote: >> >>> Thanks for the response Krishna. >>> >>> I was wondering if it were possible for using MR to solve you problem >>> instead of building the whole stack on top of yarn. >>> Most likely its not possible , thats why you are building it . I wanted >>> to know why is that ? >>> >>> I am in just trying to find out the need or why we might need to write >>> the application on yarn. >>> >>> Rahul >>> >>> >>> On Wed, May 29, 2013 at 8:23 PM, Krishna Kishore Bonagiri < >>> [email protected]> wrote: >>> >>>> Hi Rahul, >>>> >>>> I am porting a distributed application that runs on a fixed set of >>>> given resources to YARN, with the aim of being able to run it on a >>>> dynamically selected resources whichever are available at the time of >>>> running the application. >>>> >>>> Thanks, >>>> Kishore >>>> >>>> >>>> On Wed, May 29, 2013 at 8:04 PM, Rahul Bhattacharjee < >>>> [email protected]> wrote: >>>> >>>>> Hi all, >>>>> >>>>> I was going through the motivation behind Yarn. Splitting the >>>>> responsibility of JT is the major concern.Ultimately the base (Yarn) was >>>>> built in a generic way for building other generic distributed applications >>>>> too. >>>>> >>>>> I am not able to think of any other parallel processing use case that >>>>> would be useful to built on top of YARN. I though of a lot of use cases >>>>> that would be beneficial when run in parallel , but again ,we can do those >>>>> using map only jobs in MR. >>>>> >>>>> Can someone tell me a scenario , where a application can utilize Yarn >>>>> features or can be built on top of YARN and at the same time , it cannot >>>>> be >>>>> done efficiently using MRv2 jobs. >>>>> >>>>> thanks, >>>>> Rahul >>>>> >>>>> >>>>> >>>> >>> >> > > > -- > Jay Vyas > http://jayunit100.blogspot.com >
