Just reread you post and if you require launching a big job of multiple tasks at the same time, then I believe you would just cache the offers from multiple slaves and when ready, launch your job, which would just be launching separately tasks on each of those slaves/offers you have cached.
Radek On Thu, Mar 6, 2014 at 5:45 PM, Radek Burkat <[email protected]> wrote: > From what I understand...you receive offers and at that point you have to > decide if you are going to cache them, launch a task on them, or decline > them. If you don't launch a task or don't decline, the allocator will > assume you are using the offer and count it as part of your share, and it > won't offer it to another framework. ( perhaps the allocator later can > rescind to offer, but I have not seen that happen under my conditions ) > So typically I think you want to decline it to be a good citizen. > > What happens if you start launching tasks that use say 1 cpu each of a > multi cpu slave offer, you will fragment the resources and start receiving > multiple offers for the _same_ slave. Each offer will contain a subset of > the original resources. When you get this type of fragmenting it is then > that you can launch a single task on multiple offers of the same slave at > the same time, if the sum of those resources is what you need. > > In your case you are talking about launching a "big" task on multiple > "workers", and if in workers you mean multiple slaves, I don't think it is > possible to launch a task across multiple slaves. > > Disclaimer: I'm still trying to understand mesos myself. > > Radek > > > > > On Thu, Mar 6, 2014 at 4:55 PM, Lin Zhao <[email protected]> wrote: > >> The interaction between a scheduler and Mesos as I understand is: >> >> >> while(true): //resourceOffers method >> Mesos sends a list of Offers to the scheduler. Each Offer contains >> resource info of a Mesos worker. >> Scheduler determine what to do with the list of offers >> continue >> >> My question is, when Mesos sends a new list of offers, are previous >> offers still valid? Is the scheduler being a good citizen by caching offers >> and launching it later, after other offer lists are received? Assuming no >> offerRecinced is called. >> >> The reason I ask is that I noticed in some cases Mesos may break down >> available resources in different offerings. >> >> Say there are 3 workers in the cluster (mesos1, mesos2, mesos3). After >> some time the schedule may see: >> >> *[offer1: mesos1, offer2: mesos2]* >> >> *(2 seconds later)* >> >> *[offer1: mesos3]* >> >> *(2 seconds later)* >> >> *(repeat)* >> >> >> As a schedule, I believe I can use all 3 offers to launch a big task. Is >> this true? >> >> >> -- >> Lin Zhao >> >> https://wiki.groupondev.com/Message_Bus >> 3101 Park Blvd, Palo Alto, CA 94306 >> >> >> >> > > > -- > *RADEK BURKAT **FOUNDER* > > *//pinkbike <http://www.pinkbike.com/>* > > *PHONE 1-604-351-2581* > *FAX *+1 403 668 0887 > *SKYPE radek.burkat* > > *pb <http://radek.pinkbike.com/> facebook > <http://www.facebook.com/pinkbikecom> * *twitter > <http://www.twitter.com/pinkbike>* > > *Confidentiality notice and disclaimer*: The information in this email > may be confidential and is intended only for the person or persons to whom > it is addressed. If you are not the individual to whom it is addressed, you > are warned that any disclosure, copying or dissemination of the information > is unauthorized. If you have received the email in error, please > immediately contact Pinkbike.com by telephone or email, to inform us of the > error. No liability is accepted for any unauthorized use of the information > contained in this email. > -- *RADEK BURKAT **FOUNDER* *//pinkbike <http://www.pinkbike.com/>* *PHONE 1-604-351-2581* *FAX *+1 403 668 0887 *SKYPE radek.burkat* *pb <http://radek.pinkbike.com/> facebook <http://www.facebook.com/pinkbikecom> * *twitter <http://www.twitter.com/pinkbike>* *Confidentiality notice and disclaimer*: The information in this email may be confidential and is intended only for the person or persons to whom it is addressed. If you are not the individual to whom it is addressed, you are warned that any disclosure, copying or dissemination of the information is unauthorized. If you have received the email in error, please immediately contact Pinkbike.com by telephone or email, to inform us of the error. No liability is accepted for any unauthorized use of the information contained in this email.
