Hi, I'm trying to get Mesos up as a PoC at work to see if it's something we could use effectively.
I'm running SL 6.4, installed libcgroup and ran cgconfig which creates the cgroup mounts under /cgroup: [[email protected] ~]$ cat /proc/mounts | grep cgroup > cgroup /cgroup/cpuset cgroup rw,relatime,cpuset 0 0 > cgroup /cgroup/cpu cgroup rw,relatime,cpu 0 0 > cgroup /cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0 > cgroup /cgroup/memory cgroup rw,relatime,memory 0 0 > cgroup /cgroup/devices cgroup rw,relatime,devices 0 0 > cgroup /cgroup/freezer cgroup rw,relatime,freezer 0 0 > cgroup /cgroup/net_cls cgroup rw,relatime,net_cls 0 0 > cgroup /cgroup/blkio cgroup rw,relatime,blkio 0 0 I'm using the mesos_0.17.0_x86_64.rpm package downloaded from mesosphere.ioand I can't start a slave fine if I use process isolation, but if I try cgroups I get the following in /var/log/messages: 2014-03-28T10:46:05.194958-07:00 tlandells-mesos02 mesos-slave[6242]: I0328 > 10:46:05.194540 6242 main.cpp:118] Build: 2014-02-27 15:07:00 by root > 2014-03-28T10:46:05.194976-07:00 tlandells-mesos02 mesos-slave[6242]: > I0328 10:46:05.194810 6242 main.cpp:120] Creating "cgroups" isolator > 2014-03-28T10:46:05.195267-07:00 tlandells-mesos02 mesos-slave[6242]: > I0328 10:46:05.195050 6242 main.cpp:132] Starting Mesos slave > 2014-03-28T10:46:05.195657-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@712: Client > environment:zookeeper.version=zookeeper C client 3.4.5 > 2014-03-28T10:46:05.195667-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@716: Client > environment:host.name=tlandells-mesos02.dev.box.net. > 2014-03-28T10:46:05.195671-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@723: Client > environment:os.name=Linux > 2014-03-28T10:46:05.195674-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@724: Client > environment:os.arch=2.6.32-358.18.1.el6.x86_64 > 2014-03-28T10:46:05.195678-07:00 tlandells-mesos02 mesos-slave[6242]: > I0328 10:46:05.195552 6242 slave.cpp:111] Slave started on 1)@ > 10.3.18.184:5051 > 2014-03-28T10:46:05.195681-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@725: Client > environment:os.version=#1 SMP Tue Aug 27 14:23:09 CDT 2013 > 2014-03-28T10:46:05.195773-07:00 tlandells-mesos02 mesos-slave[6242]: > I0328 10:46:05.195708 6242 slave.cpp:211] Slave resources: cpus(*):2; > mem(*):2808; disk(*):43161; ports(*):[31000-32000] > 2014-03-28T10:46:05.196105-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@733: Client > environment:user.name=tlandells > 2014-03-28T10:46:05.196115-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@741: Client > environment:user.home=/root > 2014-03-28T10:46:05.196119-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@753: Client > environment:user.dir=/home/tlandells > 2014-03-28T10:46:05.196122-07:00 tlandells-mesos02 mesos-slave[6242]: > 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@zookeeper_init@786: > Initiating client connection, host=tlandells-mesos01.dev.box.net:2181, > tlandells-mesos02.dev.box.net:2181,tlandells-mesos03.dev.box.net:2181sessionTimeout=10000 > watcher=0x7fc602fc8cb0 sessionId=0 > sessionPasswd=<null> context=0x7fc5d8003b90 flags=0 > 2014-03-28T10:46:05.197040-07:00 tlandells-mesos02 mesos-slave[6242]: > I0328 10:46:05.196420 6269 cgroups_isolator.cpp:223] Using /cgroup as > cgroups hierarchy root > 2014-03-28T10:46:05.197058-07:00 tlandells-mesos02 mesos-slave[6242]: > Failed to mount cgroups hierarchy at '/cgroup' because we could not remove > existing directory: Directory not empty The command being invoked by /usr/bin/mesos-init-wrapper is: > /usr/local/sbin/mesos-slave --master=zk:// > tlandells-mesos01.dev.box.net:2181,tlandells-mesos02.dev.box.net:2181, > tlandells-mesos03.dev.box.net:2181/mesos --log_dir=/var/log/mesos > --isolation=cgroups I haven't worked with cgroups before so I'm not clear if it's okay to have Mesos using the default mounts or whether I should have something special setup. I mean it's clear that Mesos wouldn't be able to empty out /cgroup because it contains all the cgroup mounts... I saw a similar thread in the archives from about 6 months ago where the issue was apparently a clash between Mesos and Docker, but I'm only trying to run Mesos. The only thing that might be funky is that these are VMware instances I'm currently using for testing. Does anyone have any experience running the cgroups isolator in SL (or even RH, since it's basically just a rebadge) who can tell me what I'm doing wrong? Thanks, Tony
