The failing test is an integration test that seems to be triggering a kernel bug. All other tests are passing so you should just ignore it for now; Mesos will run.
There is some discussion here resulting from this same test failing: https://lkml.org/lkml/2013/7/28/74 Does running the test leave an unkillable process as described in the post? If so, can you check the /proc/<pid>/stack and /proc/<pid>/status? Thanks, Ian On Aug 1, 2013, at 1:18 PM, Kevin Lyda <ke...@ie.suberic.net> wrote: > Not running systemd. Ubuntu uses upstart. > > cgroups are mounted by /etc/init/cgconfig.conf which is installed by > cgroup-bin (which I need for lscgroup). > > I turned that off by doing: > > service cgconfig stop > echo manual > /etc/init/cgconfig.override > > The echo line turns it off permanently. > > OK, ran make check and am back to the original error! Hurrah! > > [----------] Global test environment tear-down > [==========] 240 tests from 46 test cases ran. (91790 ms total) > [ PASSED ] 239 tests. > [ FAILED ] 1 test, listed below: > [ FAILED ] CgroupsIsolatorTest.ROOT_CGROUPS_BalloonFramework > > 1 FAILED TEST > make[3]: *** [check-local] Error 1 > make[3]: Leaving directory `/usr/src/mesos/build/src' > make[2]: *** [check-am] Error 2 > make[2]: Leaving directory `/usr/src/mesos/build/src' > make[1]: *** [check] Error 2 > make[1]: Leaving directory `/usr/src/mesos/build/src' > make: *** [check-recursive] Error 1 > > # lscgroup > cgroups can't be listed: Cgroup is not mounted > > # mount > /dev/sda1 on / type ext4 (rw,errors=remount-ro) > proc on /proc type proc (rw,noexec,nosuid,nodev) > sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) > none on /sys/fs/fuse/connections type fusectl (rw) > none on /sys/kernel/debug type debugfs (rw) > none on /sys/kernel/security type securityfs (rw) > udev on /dev type devtmpfs (rw,mode=0755) > devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) > tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755) > none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880) > none on /run/shm type tmpfs (rw,nosuid,nodev) > /dev/sdd1 on /u4 type ext4 (rw) > /dev/sdb1 on /u2 type ext4 (rw) > /dev/sdc1 on /u3 type ext4 (rw) > /dev/sda6 on /u1 type ext4 (rw) > > > On Thu, Aug 1, 2013 at 8:25 PM, Ian Downes <idow...@twitter.com> wrote: >> Looks like cgroups are mounted on /sys/fs/cgroup so I'd expect the tests to >> fail again. Are you running systemd? If the tests do fail, and you're not >> running systemd, you should try manually umount'ing cgroups and re-running >> the tests (no need to rebuild, just run `make check` again). >> >> Ian >> On Aug 1, 2013, at 12:18 PM, Kevin Lyda <ke...@ie.suberic.net> wrote: >> >>> [Initial reply not sent to list] >>> >>> Will remove lxc and make sure cgroups are unmounted. This is not in a >>> VM; it's on an x86_64 machine. Replying via phone so will get the >>> other info in a bit. >>> >>> [Followup] >>> >>> Right, removed lxc. Noted that there's a cgroups-lite package that >>> might be an issue. Hm. No, that mounts cgroups. >>> >>> OK, lscgroup is in the cgroup-bin package. That also mounts cgroups. >>> But perhaps it isn't busy w/o lxc installed? >>> >>> Trying a build now. Output of requested commands follows; will report >>> build results when they're done. >>> >>> # lscgroup >>> cpu:/ >>> cpu:/sysdefault >>> cpuacct:/ >>> cpuacct:/sysdefault >>> devices:/ >>> devices:/sysdefault >>> memory:/ >>> memory:/sysdefault >>> freezer:/ >>> freezer:/sysdefault >>> # uname -a >>> Linux node29 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 >>> 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux >>> # mount >>> /dev/sda1 on / type ext4 (rw,errors=remount-ro) >>> proc on /proc type proc (rw,noexec,nosuid,nodev) >>> sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) >>> none on /sys/fs/fuse/connections type fusectl (rw) >>> none on /sys/kernel/debug type debugfs (rw) >>> none on /sys/kernel/security type securityfs (rw) >>> udev on /dev type devtmpfs (rw,mode=0755) >>> devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) >>> tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755) >>> none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880) >>> none on /run/shm type tmpfs (rw,nosuid,nodev) >>> /dev/sdd1 on /u4 type ext4 (rw) >>> /dev/sdb1 on /u2 type ext4 (rw) >>> /dev/sdc1 on /u3 type ext4 (rw) >>> /dev/sda6 on /u1 type ext4 (rw) >>> cgroups on /sys/fs/cgroup type tmpfs (rw,uid=0,gid=0,mode=0755) >>> >>> Thanks for the feedback, >>> >>> Kevin >>> >>> >>> On Thu, Aug 1, 2013 at 7:43 PM, Ian Downes <idow...@twitter.com> wrote: >>>> The documentation is out of date and lxc has been deprecated; cgroups are >>>> used directly instead. We'll be updating the documentation shortly to >>>> reflect this. >>>> >>>> Installing lxc possibly mounted cgroups and led to the slew of busy >>>> errors. Could you please post the output of `mount` and, if you have >>>> libcgroup installed, `lscgroup`, ideally before and after lxc is >>>> installed? And also `uname -a`. >>>> >>>> The failed BalloonFramework test may be related to a bug that we're >>>> investigating. Are you running this inside a VM by any chance? >>>> >>>> Ian >>>> >>>> On Aug 1, 2013, at 8:59 AM, Kevin Lyda <ke...@ie.suberic.net> wrote: >>>> >>>>> Running make check on master from github (bfdfdce) and I'm getting a >>>>> failure: >>>>> >>>>> [ OK ] AllocatorZooKeeperTest/0.SlaveReregistersFirst (2098 ms) >>>>> [----------] 2 tests from AllocatorZooKeeperTest/0 (4242 ms total) >>>>> >>>>> [----------] Global test environment tear-down >>>>> [==========] 240 tests from 46 test cases ran. (91604 ms total) >>>>> [ PASSED ] 239 tests. >>>>> [ FAILED ] 1 test, listed below: >>>>> [ FAILED ] CgroupsIsolatorTest.ROOT_CGROUPS_BalloonFramework >>>>> >>>>> 1 FAILED TEST >>>>> make[3]: *** [check-local] Error 1 >>>>> make[3]: Leaving directory `/usr/src/mesos/build/src' >>>>> make[2]: *** [check-am] Error 2 >>>>> make[2]: Leaving directory `/usr/src/mesos/build/src' >>>>> make[1]: *** [check] Error 2 >>>>> make[1]: Leaving directory `/usr/src/mesos/build/src' >>>>> make: *** [check-recursive] Error 1 >>>>> >>>>> This is on an up-to-date Ubuntu 12.04.2. Ah. I realise I hadn't >>>>> installed lxc. OK, trying again. >>>>> >>>>> Nope. Now the errors are worse (built from a clean tree, same commit): >>>>> >>>>> [----------] Global test environment tear-down >>>>> [==========] 239 tests from 45 test cases ran. (91234 ms total) >>>>> [ PASSED ] 216 tests. >>>>> [ FAILED ] 23 tests, listed below: >>>>> [ FAILED ] SlaveRecoveryTest/1.RecoverSlaveState, where TypeParam = >>>>> mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.RecoverStatusUpdateManager, where >>>>> TypeParam = mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.ReconnectExecutor, where TypeParam = >>>>> mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.RecoverUnregisteredExecutor, where >>>>> TypeParam = mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.RecoverTerminatedExecutor, where >>>>> TypeParam = mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.CleanupExecutor, where TypeParam = >>>>> mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.RemoveNonCheckpointingFramework, >>>>> where TypeParam = mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.NonCheckpointingFramework, where >>>>> TypeParam = mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.NonCheckpointingSlave, where >>>>> TypeParam = mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.KillTask, where TypeParam = >>>>> mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.GCExecutor, where TypeParam = >>>>> mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] SlaveRecoveryTest/1.ShutdownSlave, where TypeParam = >>>>> mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] IsolatorTest/1.Usage, where TypeParam = >>>>> mesos::internal::slave::CgroupsIsolator >>>>> [ FAILED ] CgroupsIsolatorTest.ROOT_CGROUPS_BalloonFramework >>>>> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_Busy >>>>> [ FAILED ] >>>>> CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_SubsystemsHierarchy >>>>> [ FAILED ] >>>>> CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_MountedSubsystems >>>>> [ FAILED ] >>>>> CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_CreateRemove >>>>> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_Listen >>>>> [ FAILED ] CgroupsAnyHierarchyWithCpuAcctMemoryTest.ROOT_CGROUPS_Stat >>>>> [ FAILED ] >>>>> CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Freeze >>>>> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Kill >>>>> [ FAILED ] >>>>> CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Destroy >>>>> >>>>> 23 FAILED TESTS >>>>> make[3]: *** [check-local] Error 1 >>>>> make[3]: Leaving directory `/usr/src/mesos/build/src' >>>>> make[2]: *** [check-am] Error 2 >>>>> make[2]: Leaving directory `/usr/src/mesos/build/src' >>>>> make[1]: *** [check] Error 2 >>>>> make[1]: Leaving directory `/usr/src/mesos/build/src' >>>>> make: *** [check-recursive] Error 1 >>>>> >>>>> Kevin >>>>> >>>>> -- >>>>> Kevin Lyda >>>>> Galway, Ireland >>>>> US Citizen overseas? We can vote. >>>>> Register now: http://www.votefromabroad.org/ >>>> >>> >>> >>> >>> -- >>> Kevin Lyda >>> Galway, Ireland >>> US Citizen overseas? We can vote. >>> Register now: http://www.votefromabroad.org/ >> > > > > -- > Kevin Lyda > Galway, Ireland > US Citizen overseas? We can vote. > Register now: http://www.votefromabroad.org/
