Hi all, I’m exploring Mesos as a scheduler for batch jobs running in docker containers.
When accepting offers and submitting tasks Mesos seems to run the container with the hard limit `—memory` and I was wondering if it possible to run them with soft memory limit instead, the `—memory-reservation` parameter. I’ll explain the use case. In my company we have data intensive jobs that could run for hours. These jobs could have memory spikes and we would like not to waste hours of computation just for a brief memory spike. Our current scheduler let jobs go over their allocated memory unless there is overall memory pressure in the machine, i.e. jobs don’t get killed unless they all go over their resource at the same time. What would be a viable solution to this problem in Mesos? Oversubscription? Custom executor? Thanks
