You can try to use Mesos containerizer to launch your Docker containers https://github.com/apache/mesos/blob/master/docs/containerizer.md https://github.com/apache/mesos/blob/master/docs/container-image.md
, and build a custom memory isolator to set the soft memory limit for your task. - Jie On Fri, Sep 23, 2016 at 3:38 PM, Jacopo Sabbatini <dedal...@gmail.com> wrote: > Hi all, > > I’m exploring Mesos as a scheduler for batch jobs running in docker > containers. > > When accepting offers and submitting tasks Mesos seems to run the > container with the hard limit `—memory` and I was wondering if it possible > to run them with soft memory limit instead, the `—memory-reservation` > parameter. > > I’ll explain the use case. In my company we have data intensive jobs that > could run for hours. These jobs could have memory spikes and we would like > not to waste hours of computation just for a brief memory spike. Our > current scheduler let jobs go over their allocated memory unless there is > overall memory pressure in the machine, i.e. jobs don’t get killed unless > they all go over their resource at the same time. > > What would be a viable solution to this problem in Mesos? > Oversubscription? Custom executor? > > Thanks >
