Re: Abstraction layer to support both YARN and Mesos

2013-07-29 Thread Tsuyoshi OZAWA
I thought some high availability and resource isolation features in Mesos are more matured. If no one is interested in this topic, MR should go with YARN. On Fri, Jul 26, 2013 at 7:14 PM, Harsh J ha...@cloudera.com wrote: Do we have a good reason to prefer Mesos over YARN for scheduling MR

Re: Abstraction layer to support both YARN and Mesos

2013-07-29 Thread Michael Segel
Actually, I am interested. Lots of different Apache top level projects seem to overlap and it can be confusing. Its very easy for a good technology to get starved because no one asks how to combine these features in to the framework. On Jul 29, 2013, at 9:58 AM, Tsuyoshi OZAWA

Re: Abstraction layer to support both YARN and Mesos

2013-07-29 Thread Tsuyoshi OZAWA
Harsh, yes, I know what you mean :-) Never mind. We should discuss this topic with MR users. On Tue, Jul 30, 2013 at 12:08 AM, Michael Segel msegel_had...@hotmail.com wrote: Actually, I am interested. Lots of different Apache top level projects seem to overlap and it can be confusing. Its

Abstraction layer to support both YARN and Mesos

2013-07-26 Thread Tsuyoshi OZAWA
Hi, Now, Apache Mesos, an distributed resource manager, is top-level apache project. Meanwhile, As you know, Hadoop has own resource manager - YARN. IMHO, we should make resource manager pluggable in MRv2, because there are their own field users of MapReduce would like to use. I think this work

Re: Abstraction layer to support both YARN and Mesos

2013-07-26 Thread Harsh J
Do we have a good reason to prefer Mesos over YARN for scheduling MR specifically? At what times would one prefer the other? On Fri, Jul 26, 2013 at 11:43 AM, Tsuyoshi OZAWA ozawa.tsuyo...@gmail.com wrote: Hi, Now, Apache Mesos, an distributed resource manager, is top-level apache project.