Hi, I think the garbage collector of Mesos agent will remove the directory of the finished task. Thanks!
From: Arkal Arjun Rao [mailto:[email protected]] Sent: Friday, February 12, 2016 4:22 PM To: [email protected] Subject: Re: Updated agent resources with every offer. Hi Vinod, Thanks for the reply. I think I understand what you mean. Could you clarify these follow-up questions? 1. So if I did write to the sandbox, mesos would know and send the correct offer? 2. And if so, and this might be hacky, if i bind mounted my docker folder (where all cached images are stored) into a sandbox directory, do you think Mesos will register the correct state of the disk in the offer? (Suppose I were to spawn a possibly persistent job that requests 0 cores, 0 memory and 0gb and use it's sandbox) Thanks again, Arjun On Fri, Feb 12, 2016 at 4:08 PM, Vinod Kone <[email protected]<mailto:[email protected]>> wrote: If your job is writing stuff outside the sandbox it is up to your framework to do that resource accounting. It is really tricky for Mesos to do that. For example, the second job might be launched even before the first one finishes. On Fri, Feb 12, 2016 at 3:46 PM, Arkal Arjun Rao <[email protected]<mailto:[email protected]>> wrote: Hi All, I'm new to Mesos and I'm working on a framework that strongly considers the disk value in an offer before making a decision. My jobs don't run in the agent's sandbox and may use docker to pull images from my dockerhub and run containers on input data downloaded from S3. My jobs clean up after themselves but do not delete the cached docker images after they complete so a later job can use them directly without the delay of downloading the image again. I cannot predict how much a job will leave behind. Leaving behind files after the job means that the disk space available for the next job is less than the disk value the current job had when it started. However the offer made to the master does not appear to update the disk parameter before making the new offer. Is there any way to get the executor driver to update the value passed in the disk field of resource offers? Here's a Stack overflow with more details http://stackoverflow.com/questions/35354841/setup-mesos-to-provide-up-to-date-disk-in-offers Thanks in advance, Arjun Arkal Rao PhD Candidate, Haussler Lab, UC Santa Cruz, USA -- Arjun Arkal Rao PhD Student, Haussler Lab, UC Santa Cruz, USA [email protected]<mailto:[email protected]>
