Hi, I'm new to Mesos and have some questions about resource management. I want to understand how Mesos limits resources used by each executors, given resources defined in TaskInfo. I did some tests and have seen different behavior for different types of resources. It appears that Mesos caps CPU usage for the executors, but doesn't limit the memory accessible to each executor.
I created an example java framework, which is largely taken from the mesos example: https://gist.github.com/lin-zhao/8544495 Basically, 1. the Scheduler launches tasks with *2* cpus, and *128 mb* memory. 2. The executor launches java with *-Xms 1500m* and *-Xmx 1500m*. 3. The java executor creates a byte array that uses *512 MB* memory. 4. The java executor starts 3 threads that loops forever, which potentially uses *3* full cpus. The framework is launched in a 3 slave Mesos (v0.14.2) cluster and finished without error. CPU: on the slaves, the cpu usage for the TestExecutor process is capped at 199%, indicating that Mesos does cap CPU usage. When the executor are assigned 1 cpu instead of 2, the cpu usage is capped at 99%. Memory: There is no error thrown. The executors used > 512 MB memory and get away with it. Can someone confirm this? I haven't tested the other resource types (ports, disk). Is the behavior documented somewhere? -- Lin Zhao https://wiki.groupondev.com/Message_Bus 3101 Park Blvd, Palo Alto, CA 94306 Temporarily based in NY 33 W 19th St.
