Thank Vinod, Yeah, I wasn't at all clear on how to set up cgroups for Mesos.
I'll try shutting down cgconfig, cleaning out /cgroup, and seeing what happens when I start the slave. If that doesn't work, I'll grab 0.180-rc5 and give that a spin. Tony On Fri, Mar 28, 2014 at 1:35 PM, Vinod Kone <[email protected]> wrote: > Hey Tony, > > I don't think Mesos would work nicely if you have multiple hierarchies > (one per controller) pre 0.18.0. One solution is to un mount and remove all > the cgroup hierarchies and let Mesos take control of it. > > Alternatively, since 0.18.0, we did a major refactor around cgroups > isolation and it works with multiple hierarchies. So you might want to give > it (0.18.0-rc5) a try. > > > @vinodkone > > > On Fri, Mar 28, 2014 at 11:05 AM, Tony Landells <[email protected]> wrote: > >> Hi, >> >> I'm trying to get Mesos up as a PoC at work to see if it's something we >> could use effectively. >> >> I'm running SL 6.4, installed libcgroup and ran cgconfig which creates >> the cgroup mounts under /cgroup: >> >> [[email protected] ~]$ cat /proc/mounts | grep cgroup >>> cgroup /cgroup/cpuset cgroup rw,relatime,cpuset 0 0 >>> cgroup /cgroup/cpu cgroup rw,relatime,cpu 0 0 >>> cgroup /cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0 >>> cgroup /cgroup/memory cgroup rw,relatime,memory 0 0 >>> cgroup /cgroup/devices cgroup rw,relatime,devices 0 0 >>> cgroup /cgroup/freezer cgroup rw,relatime,freezer 0 0 >>> cgroup /cgroup/net_cls cgroup rw,relatime,net_cls 0 0 >>> cgroup /cgroup/blkio cgroup rw,relatime,blkio 0 0 >> >> >> I'm using the mesos_0.17.0_x86_64.rpm package downloaded from >> mesosphere.io<https://urldefense.proofpoint.com/v1/url?u=http://mesosphere.io&k=ppTtPKfaP99D%2F%2BB0F9qgBA%3D%3D%0A&r=x28Jl7CpmSGov0XieaueXQ%3D%3D%0A&m=4tTsrmnaFBZhAevZGQMu7MERiC%2BYL3QdI7iTMUIFqik%3D%0A&s=1534ed849c30c9cc1f489ea1b0dab2858f004a45101fc4b70503337848dd597b>and >> I can't start a slave fine if I use process isolation, but if I try >> cgroups I get the following in /var/log/messages: >> >> 2014-03-28T10:46:05.194958-07:00 tlandells-mesos02 mesos-slave[6242]: >>> I0328 10:46:05.194540 6242 main.cpp:118] Build: 2014-02-27 15:07:00 by root >>> 2014-03-28T10:46:05.194976-07:00 tlandells-mesos02 mesos-slave[6242]: >>> I0328 10:46:05.194810 6242 main.cpp:120] Creating "cgroups" isolator >>> 2014-03-28T10:46:05.195267-07:00 tlandells-mesos02 mesos-slave[6242]: >>> I0328 10:46:05.195050 6242 main.cpp:132] Starting Mesos slave >>> 2014-03-28T10:46:05.195657-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@712: >>> Client environment:zookeeper.version=zookeeper C client 3.4.5 >>> 2014-03-28T10:46:05.195667-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@716: >>> Client >>> environment:host.name<https://urldefense.proofpoint.com/v1/url?u=http://host.name&k=ppTtPKfaP99D%2F%2BB0F9qgBA%3D%3D%0A&r=x28Jl7CpmSGov0XieaueXQ%3D%3D%0A&m=4tTsrmnaFBZhAevZGQMu7MERiC%2BYL3QdI7iTMUIFqik%3D%0A&s=e22c1990921e7300f1890ae19be345c0374fbe0fee025f89892acefe879af22d> >>> =tlandells-mesos02.dev.box.net. >>> 2014-03-28T10:46:05.195671-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@723: >>> Client >>> environment:os.name<https://urldefense.proofpoint.com/v1/url?u=http://os.name&k=ppTtPKfaP99D%2F%2BB0F9qgBA%3D%3D%0A&r=x28Jl7CpmSGov0XieaueXQ%3D%3D%0A&m=4tTsrmnaFBZhAevZGQMu7MERiC%2BYL3QdI7iTMUIFqik%3D%0A&s=0f5039fb6bfae82a50cc0ad2d01e809483cd318b3a96daf432d471f773a44430> >>> =Linux >>> 2014-03-28T10:46:05.195674-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@724: >>> Client environment:os.arch=2.6.32-358.18.1.el6.x86_64 >>> 2014-03-28T10:46:05.195678-07:00 tlandells-mesos02 mesos-slave[6242]: >>> I0328 10:46:05.195552 6242 slave.cpp:111] Slave started on 1)@ >>> 10.3.18.184:5051<https://urldefense.proofpoint.com/v1/url?u=http://10.3.18.184:5051&k=ppTtPKfaP99D%2F%2BB0F9qgBA%3D%3D%0A&r=x28Jl7CpmSGov0XieaueXQ%3D%3D%0A&m=4tTsrmnaFBZhAevZGQMu7MERiC%2BYL3QdI7iTMUIFqik%3D%0A&s=56e0996f695897eda97193ede5e6c3bc983f8c41eb5d70f9c7c1bd3e9838cfa6> >>> 2014-03-28T10:46:05.195681-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@725: >>> Client environment:os.version=#1 SMP Tue Aug 27 14:23:09 CDT 2013 >>> 2014-03-28T10:46:05.195773-07:00 tlandells-mesos02 mesos-slave[6242]: >>> I0328 10:46:05.195708 6242 slave.cpp:211] Slave resources: cpus(*):2; >>> mem(*):2808; disk(*):43161; ports(*):[31000-32000] >>> 2014-03-28T10:46:05.196105-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@733: >>> Client >>> environment:user.name<https://urldefense.proofpoint.com/v1/url?u=http://user.name&k=ppTtPKfaP99D%2F%2BB0F9qgBA%3D%3D%0A&r=x28Jl7CpmSGov0XieaueXQ%3D%3D%0A&m=4tTsrmnaFBZhAevZGQMu7MERiC%2BYL3QdI7iTMUIFqik%3D%0A&s=6c62d6ae4f01b70ca800eb1f50f3535de8fc467b57fdb6199cc7f1aad0789c99> >>> =tlandells >>> 2014-03-28T10:46:05.196115-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@741: >>> Client environment:user.home=/root >>> 2014-03-28T10:46:05.196119-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@753: >>> Client environment:user.dir=/home/tlandells >>> 2014-03-28T10:46:05.196122-07:00 tlandells-mesos02 mesos-slave[6242]: >>> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@zookeeper_init@786: >>> Initiating client connection, host=tlandells-mesos01.dev.box.net:2181, >>> tlandells-mesos02.dev.box.net:2181,tlandells-mesos03.dev.box.net:2181sessionTimeout=10000 >>> watcher=0x7fc602fc8cb0 sessionId=0 >>> sessionPasswd=<null> context=0x7fc5d8003b90 flags=0 >>> 2014-03-28T10:46:05.197040-07:00 tlandells-mesos02 mesos-slave[6242]: >>> I0328 10:46:05.196420 6269 cgroups_isolator.cpp:223] Using /cgroup as >>> cgroups hierarchy root >>> 2014-03-28T10:46:05.197058-07:00 tlandells-mesos02 mesos-slave[6242]: >>> Failed to mount cgroups hierarchy at '/cgroup' because we could not remove >>> existing directory: Directory not empty >> >> >> The command being invoked by /usr/bin/mesos-init-wrapper is: >> >>> /usr/local/sbin/mesos-slave --master=zk:// >>> tlandells-mesos01.dev.box.net:2181,tlandells-mesos02.dev.box.net:2181, >>> tlandells-mesos03.dev.box.net:2181/mesos --log_dir=/var/log/mesos >>> --isolation=cgroups >> >> >> I haven't worked with cgroups before so I'm not clear if it's okay to >> have Mesos using the default mounts or whether I should have something >> special setup. I mean it's clear that Mesos wouldn't be able to empty out >> /cgroup because it contains all the cgroup mounts... >> >> I saw a similar thread in the archives from about 6 months ago where the >> issue was apparently a clash between Mesos and Docker, but I'm only trying >> to run Mesos. >> >> The only thing that might be funky is that these are VMware instances I'm >> currently using for testing. >> >> Does anyone have any experience running the cgroups isolator in SL (or >> even RH, since it's basically just a rebadge) who can tell me what I'm >> doing wrong? >> >> Thanks, >> Tony >> > >
