*[lin@mesos2 ~]$ cat
/cgroup/mesos/framework_201401171812-2907575306-5050-19011-0019_executor_default_tag_72c003a3-f213-479e-a7e3-9b86930703a7/memory.limit_in_bytes
*
*9223372036854775807*
*[lin@mesos2 ~]$ cat
/cgroup/mesos/framework_201401171812-2907575306-5050-19011-0019_executor_default_tag_72c003a3-f213-479e-a7e3-9b86930703a7/memory.usage_in_bytes
*
*584146944*
*[lin@mesos2 ~]$ cat
/cgroup/mesos/framework_201401171812-2907575306-5050-19011-0019_executor_default_tag_72c003a3-f213-479e-a7e3-9b86930703a7/memory.max_usage_in_bytes
*
*585809920*
Hmm the limit is weird. Can you find anything wrong about the way my mem is
defined?
.addResources(Resource.newBuilder()
.setName("mem")
.setType(Value.Type.SCALAR)
.setScalar(Value.Scalar.newBuilder().setValue(128)))
On Tue, Jan 21, 2014 at 2:02 PM, Vinod Kone <[email protected]> wrote:
> Mesos uses
> cgroups<https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt>to limit
> cpu and memory.
>
> It is indeed surprising that your executor in not OOMing when using more
> memory than requested.
>
> Can you tell us what the following values look like in the executor's
> cgroup? These are the values the kernel uses to decide whether the cgroup
> is hitting its limit.
>
> cat
> /cgroup/mesos/framework_<foo>_executor_<bar>_<uuid>/memory.limit_in_bytes
>
> cat
> /cgroup/mesos/framework_<foo>_executor_<bar>_<uuid>/memory.usage_in_bytes
>
> cat
> /cgroup/mesos/framework_<foo>_executor_<bar>_<uuid>/memory.max_usage_in_bytes
>
>
>
> @vinodkone
>
>
> On Tue, Jan 21, 2014 at 9:58 AM, Lin Zhao <[email protected]> wrote:
>
>> Hi,
>>
>> I'm new to Mesos and have some questions about resource management. I
>> want to understand how Mesos limits resources used by each executors, given
>> resources defined in TaskInfo. I did some tests and have seen different
>> behavior for different types of resources. It appears that Mesos caps CPU
>> usage for the executors, but doesn't limit the memory accessible to each
>> executor.
>>
>> I created an example java framework, which is largely taken from the
>> mesos example:
>>
>> https://gist.github.com/lin-zhao/8544495
>>
>> Basically,
>>
>> 1. the Scheduler launches tasks with *2* cpus, and *128 mb* memory.
>> 2. The executor launches java with *-Xms 1500m* and *-Xmx 1500m*.
>> 3. The java executor creates a byte array that uses *512 MB* memory.
>> 4. The java executor starts 3 threads that loops forever, which
>> potentially uses *3* full cpus.
>>
>> The framework is launched in a 3 slave Mesos (v0.14.2) cluster and
>> finished without error.
>>
>> CPU: on the slaves, the cpu usage for the TestExecutor process is capped
>> at 199%, indicating that Mesos does cap CPU usage. When the executor are
>> assigned 1 cpu instead of 2, the cpu usage is capped at 99%.
>>
>> Memory: There is no error thrown. The executors used > 512 MB memory and
>> get away with it.
>>
>> Can someone confirm this? I haven't tested the other resource types
>> (ports, disk). Is the behavior documented somewhere?
>>
>> --
>> Lin Zhao
>>
>> https://wiki.groupondev.com/Message_Bus
>> 3101 Park Blvd, Palo Alto, CA 94306
>>
>> Temporarily based in NY
>> 33 W 19th St.
>>
>>
>
--
Lin Zhao
https://wiki.groupondev.com/Message_Bus
3101 Park Blvd, Palo Alto, CA 94306
Temporarily based in NY
33 W 19th St.
