[
https://issues.apache.org/jira/browse/MESOS-449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643336#comment-13643336
]
Thomas Marshall commented on MESOS-449:
---------------------------------------
So, adding an os::sleep(Seconds(0.1)) to the end of
CgroupsAnyHierachyTest::TearDown seems to fix the problem, which indicates that
my initial guess that we're not doing proper cleanup is correct. However,
that's obviously not an ideal solution; it would be better to wait for whatever
event signals that the clean up is done.
Currently, we wait until the cgroup directory has been removed, but that is
apparently not good enough. As best I can tell (not a cgroups expert) the final
event is the os writing the cgroups changes to /proc/cgroups, so we can watch
for that change and only move to the next test once it has happened.
I've written a patch that stores the number of cgroups each subsystem is
attached to, as indicated by /proc/cgorups, during SetUp, and then haTearDown
wait until these numbers are the same as they were during SetUp. This could be
problematic if something else in the system si modifying the cgroups while the
test is running, but I believe that its the best solution.
> CgroupsTests are flaky on Ubuntu
> --------------------------------
>
> Key: MESOS-449
> URL: https://issues.apache.org/jira/browse/MESOS-449
> Project: Mesos
> Issue Type: Bug
> Reporter: Thomas Marshall
>
> On Ubuntu 12.04, I get failures similar to the following:
> [----------] 1 test from CgroupsNoHierarchyTest
> [ RUN ]
> CgroupsNoHierarchyTest.ROOT_CGROUPS_NOHIERARCHY_MountUnmountHierarchy
> ../../src/tests/cgroups_tests.cpp:250: Failure
> cgroups::mount(TEST_CGROUPS_HIERARCHY, "cpu,memory"): 'cpu' is already
> attached to another hierarchy
> [ FAILED ]
> CgroupsNoHierarchyTest.ROOT_CGROUPS_NOHIERARCHY_MountUnmountHierarchy (0 ms)
> [----------] 1 test from CgroupsNoHierarchyTest (0 ms total)
> [----------] 1 test from CgroupsAnyHierarchyWithCpuAcctMemoryTest
> [ RUN ] CgroupsAnyHierarchyWithCpuAcctMemoryTest.ROOT_CGROUPS_Stat
> ../../src/tests/cgroups_tests.cpp:116: Failure
> cgroups::mount(TEST_CGROUPS_HIERARCHY, subsystems): 'memory' is already
> attached to another hierarchy
> -------------------------------------------------------------
> We cannot run any cgroups tests that require
> a hierarchy with subsystems 'cpuacct,memory'
> because we failed to find an existing hierarchy
> or create a new one. You can either remove all existing
> hierarchies, or disable this test case
> (i.e., --gtest_filter=-CgroupsAnyHierarchyWithCpuAcctMemoryTest.*).
> -------------------------------------------------------------
> ../../src/tests/cgroups_tests.cpp:147: Failure
> exists: '' is not a valid hierarchy
> [ FAILED ] CgroupsAnyHierarchyWithCpuAcctMemoryTest.ROOT_CGROUPS_Stat (0 ms)
> [----------] 1 test from CgroupsAnyHierarchyWithCpuAcctMemoryTest (0 ms total)
> [----------] 3 tests from CgroupsAnyHierarchyWithCpuMemoryFreezerTest
> [ RUN ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Freeze
> ../../src/tests/cgroups_tests.cpp:116: Failure
> cgroups::mount(TEST_CGROUPS_HIERARCHY, subsystems): 'cpu' is already attached
> to another hierarchy
> -------------------------------------------------------------
> We cannot run any cgroups tests that require
> a hierarchy with subsystems 'cpu,memory,freezer'
> because we failed to find an existing hierarchy
> or create a new one. You can either remove all existing
> hierarchies, or disable this test case
> (i.e., --gtest_filter=-CgroupsAnyHierarchyWithCpuMemoryFreezerTest.*).
> -------------------------------------------------------------
> ../../src/tests/cgroups_tests.cpp:147: Failure
> exists: '' is not a valid hierarchy
> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Freeze
> (1 ms)
> [ RUN ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Kill
> ../../src/tests/cgroups_tests.cpp:116: Failure
> cgroups::mount(TEST_CGROUPS_HIERARCHY, subsystems): 'cpu' is already attached
> to another hierarchy
> -------------------------------------------------------------
> We cannot run any cgroups tests that require
> a hierarchy with subsystems 'cpu,memory,freezer'
> because we failed to find an existing hierarchy
> or create a new one. You can either remove all existing
> hierarchies, or disable this test case
> (i.e., --gtest_filter=-CgroupsAnyHierarchyWithCpuMemoryFreezerTest.*).
> -------------------------------------------------------------
> ../../src/tests/cgroups_tests.cpp:147: Failure
> exists: '' is not a valid hierarchy
> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Kill (1
> ms)
> [ RUN ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Destroy
> ../../src/tests/cgroups_tests.cpp:116: Failure
> cgroups::mount(TEST_CGROUPS_HIERARCHY, subsystems): 'cpu' is already attached
> to another hierarchy
> -------------------------------------------------------------
> We cannot run any cgroups tests that require
> a hierarchy with subsystems 'cpu,memory,freezer'
> because we failed to find an existing hierarchy
> or create a new one. You can either remove all existing
> hierarchies, or disable this test case
> (i.e., --gtest_filter=-CgroupsAnyHierarchyWithCpuMemoryFreezerTest.*).
> -------------------------------------------------------------
> ../../src/tests/cgroups_tests.cpp:147: Failure
> exists: '' is not a valid hierarchy
> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Destroy
> (0 ms)
> [----------] 3 tests from CgroupsAnyHierarchyWithCpuMemoryFreezerTest (2 ms
> total)
> [----------] Global test environment tear-down
> [==========] 19 tests from 6 test cases ran. (12691 ms total)
> [ PASSED ] 14 tests.
> [ FAILED ] 5 tests, listed below:
> [ FAILED ]
> CgroupsNoHierarchyTest.ROOT_CGROUPS_NOHIERARCHY_MountUnmountHierarchy
> [ FAILED ] CgroupsAnyHierarchyWithCpuAcctMemoryTest.ROOT_CGROUPS_Stat
> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Freeze
> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Kill
> [ FAILED ] CgroupsAnyHierarchyWithCpuMemoryFreezerTest.ROOT_CGROUPS_Destroy
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
