On Tue, Dec 13, 2011 at 7:21 PM, Willke, Theodore L < [email protected]> wrote:
> Thanks Andy. I’d like to use MESOs to do platform design space > exploration. By this, I mean that we want to explore the effect that > varying the performance and/or quantity of compute, memory, networking, or > storage has on overall workload/framework performance. We can use this > information to create a multi-dimensional performance surface, from which > we can select the optimal/suboptimal platform configuration for a > particular workload.**** > > ** ** > > I’m hoping that MESOs can provide the “dials” we need so that we can use > one “beefy” cluster to emulate many cluster configurations. As a stretch > goal, it would be nice to be able to launch multiple copies of the same > framework at the same time on the same cluster and subject each one to a > different container performance profile, with all containers associated > with the same framework having the same profile. > What exactly do you mean by "container performance profile"? This may permit us to more rapidly explore a design space. Of course, this > proposal relies on a degree of performance control or isolation – we don’t > need to assess absolute performance though; we just need to select the best > profile.**** > > ** ** > > Questions:**** > > **· **Does the idea seem viable? > It sounds like you are trying to emulate clusters of different sized machines (i.e. how does a particular MR workload run on a cluster of 10 machines with 4cpu/2gb ram each vs. 10 machines with 8cpu/4gp ram each, etc.). Is that right? You should be able to do this in Mesos by either launching the Mesos slaves and passing different amounts of resources (subsets of the actual physical resources) to achieve different configuration profiles (subsets of the actual resources available on the slave), or by writing your own allocation module which only makes offers for a subset of the resources available on any slave. The allocation module approach seems more flexible (you can easily change the slave resource/configuration profiles on the fly) . Also, the allocation module approach is probably as easy to implement. > **** > > **· **Are you aware of anyone using MESOS in this manner? > Nope. > **** > > **· **How good is the current performance isolation between > frameworks? Only as strong as the Linux Container, right? > I believe Charles addressed this. > **** > > **· **Have you explored means of controlling memory, networking, > and/or storage performance on a Container basis? > Not sure exactly what you mean by controlling performance here? Hopefully Charles addressed this too. > **** > > ** ** > > If I/O performance provisioning has been largely ignored, we may be > interested in developing this aspect and making a contribution to the MESOS > project. > That would be great. We are always looking to expand our development community and support new adopters! > **** > > ** ** > > Cheers,**** > > Ted**** > > ** ** > > *From:* Andy Konwinski [mailto:[email protected]] > *Sent:* Monday, December 12, 2011 12:26 PM > *To:* Willke, Theodore L > *Cc:* Benjamin Hindman; Matei Zaharia; Charles Reiss > *Subject:* Re: Your MESOs talk at the Cloud ISTC launch**** > > ** ** > > Hi Ted,**** > > ** ** > > Thanks for reaching out. Yep, I can answer your questions and help you > explore using Mesos. I am available for a phone call some time this week, > however, in case you want to summarize via email, I've CC'd a few of the > other grad students that work on Mesos who might be able to answer your > questions too.**** > > ** ** > > Andy**** > > On Mon, Dec 12, 2011 at 10:58 AM, Willke, Theodore L < > [email protected]> wrote:**** > > Hi Andy,**** > > **** > > Nice talk. I enjoy flying Southwest Airlines J There’s something I’d > like to try with MESOs that may greatly benefit Intel. But I have some > questions for you to test the viability. Are you the right contact? If > so, do you have time for a brief chat by phone this week?**** > > **** > > Thanks,**** > > Ted**** > > **** > > --------------------------------**** > > Ted L. Willke, EngScD**** > > Systems Architecture Lab**** > > Intel Corporation**** > > 2111 NE 25th Ave Hillsboro, OR 97124**** > > Cell 253.209.2739**** > > **** > > ** ** >
