Thanks Radek. This answers my question. Yes by a "big task" I mean multiple tasks that take all the resources. I was a bit confused because the offer id keeps incrementing. Now it's clear.
On Thu, Mar 6, 2014 at 5:58 PM, Radek Burkat <[email protected]> wrote: > Just reread you post and if you require launching a big job of multiple > tasks at the same time, then I believe you would just cache the offers from > multiple slaves and when ready, launch your job, which would just be > launching separately tasks on each of those slaves/offers you have cached. > > Radek > > > On Thu, Mar 6, 2014 at 5:45 PM, Radek Burkat <[email protected]> wrote: > >> From what I understand...you receive offers and at that point you have to >> decide if you are going to cache them, launch a task on them, or decline >> them. If you don't launch a task or don't decline, the allocator will >> assume you are using the offer and count it as part of your share, and it >> won't offer it to another framework. ( perhaps the allocator later can >> rescind to offer, but I have not seen that happen under my conditions ) >> So typically I think you want to decline it to be a good citizen. >> >> What happens if you start launching tasks that use say 1 cpu each of a >> multi cpu slave offer, you will fragment the resources and start receiving >> multiple offers for the _same_ slave. Each offer will contain a subset of >> the original resources. When you get this type of fragmenting it is then >> that you can launch a single task on multiple offers of the same slave at >> the same time, if the sum of those resources is what you need. >> >> In your case you are talking about launching a "big" task on multiple >> "workers", and if in workers you mean multiple slaves, I don't think it is >> possible to launch a task across multiple slaves. >> >> Disclaimer: I'm still trying to understand mesos myself. >> >> Radek >> >> >> >> >> On Thu, Mar 6, 2014 at 4:55 PM, Lin Zhao <[email protected]> wrote: >> >>> The interaction between a scheduler and Mesos as I understand is: >>> >>> >>> while(true): //resourceOffers method >>> Mesos sends a list of Offers to the scheduler. Each Offer contains >>> resource info of a Mesos worker. >>> Scheduler determine what to do with the list of offers >>> continue >>> >>> My question is, when Mesos sends a new list of offers, are previous >>> offers still valid? Is the scheduler being a good citizen by caching offers >>> and launching it later, after other offer lists are received? Assuming no >>> offerRecinced is called. >>> >>> The reason I ask is that I noticed in some cases Mesos may break down >>> available resources in different offerings. >>> >>> Say there are 3 workers in the cluster (mesos1, mesos2, mesos3). After >>> some time the schedule may see: >>> >>> *[offer1: mesos1, offer2: mesos2]* >>> >>> *(2 seconds later)* >>> >>> *[offer1: mesos3]* >>> >>> *(2 seconds later)* >>> >>> *(repeat)* >>> >>> >>> As a schedule, I believe I can use all 3 offers to launch a big task. Is >>> this true? >>> >>> >>> -- >>> Lin Zhao >>> >>> https://wiki.groupondev.com/Message_Bus >>> 3101 Park Blvd, Palo Alto, CA 94306 >>> >>> >>> >>> >> >> >> -- >> *RADEK BURKAT **FOUNDER* >> >> *//pinkbike <http://www.pinkbike.com/>* >> >> *PHONE 1-604-351-2581* >> *FAX *+1 403 668 0887 >> *SKYPE radek.burkat* >> >> *pb <http://radek.pinkbike.com/> facebook >> <http://www.facebook.com/pinkbikecom> * *twitter >> <http://www.twitter.com/pinkbike>* >> >> *Confidentiality notice and disclaimer*: The information in this email >> may be confidential and is intended only for the person or persons to whom >> it is addressed. If you are not the individual to whom it is addressed, you >> are warned that any disclosure, copying or dissemination of the information >> is unauthorized. If you have received the email in error, please >> immediately contact Pinkbike.com by telephone or email, to inform us of the >> error. No liability is accepted for any unauthorized use of the information >> contained in this email. >> > > > > -- > *RADEK BURKAT **FOUNDER* > > *//pinkbike <http://www.pinkbike.com/>* > > *PHONE 1-604-351-2581* > *FAX *+1 403 668 0887 > *SKYPE radek.burkat* > > *pb <http://radek.pinkbike.com/> facebook > <http://www.facebook.com/pinkbikecom> * *twitter > <http://www.twitter.com/pinkbike>* > > *Confidentiality notice and disclaimer*: The information in this email > may be confidential and is intended only for the person or persons to whom > it is addressed. If you are not the individual to whom it is addressed, you > are warned that any disclosure, copying or dissemination of the information > is unauthorized. If you have received the email in error, please > immediately contact Pinkbike.com by telephone or email, to inform us of the > error. No liability is accepted for any unauthorized use of the information > contained in this email. > -- Lin Zhao https://wiki.groupondev.com/Message_Bus 3101 Park Blvd, Palo Alto, CA 94306
