-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68941/#review209312
-----------------------------------------------------------
Bad patch!
Reviews applied: [68941, 68929]
Failed command: python support/apply-reviews.py -n -r 68929
Error:
Traceback (most recent call last):
File "support/apply-reviews.py", line 35, in <module>
import urllib.request
ImportError: No module named request
Full log: https://builds.apache.org/job/Mesos-Reviewbot/23458/console
- Mesos Reviewbot
On Oct. 8, 2018, 6:56 a.m., Gilbert Song wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68941/
> -----------------------------------------------------------
>
> (Updated Oct. 8, 2018, 6:56 a.m.)
>
>
> Review request for mesos, Greg Mann, Jie Yu, Meng Zhu, Qian Zhang, and Vinod
> Kone.
>
>
> Bugs: MESOS-9295
> https://issues.apache.org/jira/browse/MESOS-9295
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Added an unit test for agent recovery with new cgroup subsystems.
>
>
> Diffs
> -----
>
> src/tests/containerizer/cgroups_isolator_tests.cpp
> 368ab93ff255e44b4063a184a7d302adf60c408e
>
>
> Diff: https://reviews.apache.org/r/68941/diff/2/
>
>
> Testing
> -------
>
> make check
>
> Verified the test without the fix would fail:
> ```
> [ RUN ]
> CgroupsIsolatorTest.ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems
> I1005 16:29:56.479673 19406 cluster.cpp:173] Creating default 'local'
> authorizer
> I1005 16:29:56.486516 19423 master.cpp:413] Master
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58 (core-dev) started on 10.0.49.2:46083
> I1005 16:29:56.486575 19423 master.cpp:416] Flags at startup: --acls=""
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="hierarchical"
> --authenticate_agents="true" --authenticate_frameworks="true"
> --authenticate_http_frameworks="true" --authenticate_http_readonly="true"
> --authenticate_http_readwrite="true" --authentication_v0_timeout="15secs"
> --authenticators="crammd5" --authorizers="local"
> --credentials="/tmp/g03iWo/credentials" --filter_gpu_resources="true"
> --framework_sorter="drf" --help="false" --hostname_lookup="true"
> --http_authenticators="basic" --http_framework_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --memory_profiling="false"
> --min_allocatable_resources="cpus:0.01|mem:32" --port="5
050" --quiet="false" --recovery_agent_removal_limit="100%"
--registry="in_memory" --registry_fetch_timeout="1mins"
--registry_gc_interval="15mins" --registry_max_agent_age="2weeks"
--registry_max_agent_count="102400" --registry_store_timeout="100secs"
--registry_strict="false" --require_agent_domain="false" --role_sorter="drf"
--root_submissions="true" --version="false"
--webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/g03iWo/master"
--zk_session_timeout="10secs"
> I1005 16:29:56.491964 19423 master.cpp:465] Master only allowing
> authenticated frameworks to register
> I1005 16:29:56.491993 19423 master.cpp:471] Master only allowing
> authenticated agents to register
> I1005 16:29:56.492008 19423 master.cpp:477] Master only allowing
> authenticated HTTP frameworks to register
> I1005 16:29:56.492033 19423 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/g03iWo/credentials'
> I1005 16:29:56.492864 19423 master.cpp:521] Using default 'crammd5'
> authenticator
> I1005 16:29:56.493099 19423 authenticator.cpp:520] Initializing server SASL
> I1005 16:29:56.494115 19423 http.cpp:1037] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-readonly'
> I1005 16:29:56.494555 19423 http.cpp:1037] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-readwrite'
> I1005 16:29:56.494699 19423 http.cpp:1037] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-scheduler'
> I1005 16:29:56.495041 19423 master.cpp:602] Authorization enabled
> I1005 16:29:56.507594 19437 master.cpp:2083] Elected as the leading master!
> I1005 16:29:56.507642 19437 master.cpp:1638] Recovering from registrar
> I1005 16:29:56.511662 19442 registrar.cpp:383] Successfully fetched the
> registry (0B) in 3.270912ms
> I1005 16:29:56.512141 19442 registrar.cpp:487] Applied 1 operations in
> 161134ns; attempting to update the registry
> I1005 16:29:56.516079 19442 registrar.cpp:544] Successfully updated the
> registry in 3.751936ms
> I1005 16:29:56.516386 19442 registrar.cpp:416] Successfully recovered
> registrar
> I1005 16:29:56.517609 19448 master.cpp:1752] Recovered 0 agents from the
> registry (123B); allowing 10mins for agents to reregister
> W1005 16:29:56.524356 19406 process.cpp:2810] Attempted to spawn already
> running process [email protected]:46083
> I1005 16:29:56.526252 19406 containerizer.cpp:305] Using isolation {
> environment_secret, volume/sandbox_path, docker/runtime, cgroups/mem,
> filesystem/linux, volume/image, network/cni, volume/host_path }
> I1005 16:29:56.539021 19406 linux_launcher.cpp:144] Using
> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
> sh: hadoop: command not found
> I1005 16:29:56.623289 19406 fetcher.cpp:66] Skipping URI fetcher plugin
> 'hadoop' as it could not be created: Failed to create HDFS client: Hadoop
> client is not available, exit status: 32512
> I1005 16:29:56.628630 19406 provisioner.cpp:281] Provisioner backend
> 'overlay' is not supported on
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/provisioner':
> Backend 'overlay' is not supported due to missing d_type support on the
> underlying filesystem
> I1005 16:29:56.628705 19406 provisioner.cpp:298] Using default backend 'copy'
> I1005 16:29:56.658514 19406 cluster.cpp:485] Creating default 'local'
> authorizer
> I1005 16:29:56.660748 19461 slave.cpp:267] Mesos agent started on
> @10.0.49.2:46083
> I1005 16:29:56.660833 19461 slave.cpp:268] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://";
> --appc_store_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="false"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authentication_timeout_max="1mins" --authentication_timeout_min="5secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_destroy_timeout="1mins" --cgroups_enable_cfs="false"
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
> --cgroups_root="mesos_test_a0422359-e2a8-4897-82b9-201252686cc1"
> --container_disk_watch_interval="15secs" --containerizers="mesos"
> --credential="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/credential"
> --default_role="*" --disallow_sharing_agent_pid_namespace="false"
> --disk_watch_interval="1mins" --docker="docker" --
docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io";
--docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock"
--docker_stop_timeout="0ns"
--docker_store_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/store/docker"
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins"
--executor_reregistration_timeout="2secs"
--executor_shutdown_grace_period="5secs"
--fetcher_cache_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/fetch"
--fetcher_cache_size="2GB" --fetcher_stall_timeout="1mins"
--frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
--gc_non_executor_container_sandboxes="false" --help="false"
--hostname_lookup="true" --http_command_executor="false"
--http_credentials="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/http_credentials"
--
http_heartbeat_interval="30secs" --image_providers="docker"
--initialize_driver_logging="true"
--isolation="filesystem/linux,docker/runtime,cgroups/mem" --launcher="linux"
--launcher_dir="/home/gilbert/mesos/build/src" --logbufsecs="0"
--logging_level="INFO" --max_completed_executors_per_framework="150"
--memory_profiling="false" --network_cni_metrics="true"
--oversubscribed_resources_interval="15secs" --perf_duration="10secs"
--perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
--quiet="false" --reconfiguration_policy="equal" --recover="reconnect"
--recovery_timeout="15mins" --registration_backoff_factor="10ms"
--resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]"
--revocable_cpu_low_priority="true"
--runtime_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm"
--sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true"
--systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/sys
tem" --version="false"
--work_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn"
--zk_session_timeout="10secs"
> I1005 16:29:56.661639 19461 credentials.hpp:86] Loading credential for
> authentication from
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/credential'
> I1005 16:29:56.662113 19461 slave.cpp:300] Agent using credential for:
> test-principal
> I1005 16:29:56.662161 19461 credentials.hpp:37] Loading credentials for
> authentication from
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/http_credentials'
> I1005 16:29:56.662405 19461 http.cpp:1037] Creating default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> I1005 16:29:56.662797 19406 scheduler.cpp:189] Version: 1.8.0
> I1005 16:29:56.663552 19461 disk_profile_adaptor.cpp:80] Creating default
> disk profile adaptor module
> I1005 16:29:56.666966 19466 scheduler.cpp:355] Using default 'basic' HTTP
> authenticatee
> I1005 16:29:56.667768 19421 scheduler.cpp:538] New master detected at
> [email protected]:46083
> I1005 16:29:56.668624 19461 slave.cpp:615] Agent resources:
> [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}]
> I1005 16:29:56.669221 19461 slave.cpp:623] Agent attributes: [ ]
> I1005 16:29:56.669273 19461 slave.cpp:632] Agent hostname: core-dev
> I1005 16:29:56.669821 19427 task_status_update_manager.cpp:181] Pausing
> sending task status updates
> I1005 16:29:56.674229 19434 state.cpp:66] Recovering state from
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta'
> I1005 16:29:56.674888 19436 slave.cpp:6909] Finished recovering checkpointed
> state from
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta',
> beginning agent recovery
> I1005 16:29:56.675580 19437 task_status_update_manager.cpp:207] Recovering
> task status update manager
> I1005 16:29:56.676358 19432 containerizer.cpp:727] Recovering Mesos containers
> I1005 16:29:56.677166 19443 linux_launcher.cpp:286] Recovering Linux launcher
> I1005 16:29:56.677981 19446 containerizer.cpp:1053] Recovering isolators
> I1005 16:29:56.682600 19422 containerizer.cpp:1092] Recovering provisioner
> I1005 16:29:56.684402 19423 metadata_manager.cpp:254] No images to load from
> disk. Docker provisioner image storage path
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/store/docker/storedImages'
> does not exist
> I1005 16:29:56.684974 19433 provisioner.cpp:494] Provisioner recovery complete
> I1005 16:29:56.686478 19443 composing.cpp:339] Finished recovering all
> containerizers
> I1005 16:29:56.686728 19450 slave.cpp:7138] Recovering executors
> I1005 16:29:56.687239 19450 slave.cpp:7291] Finished recovery
> I1005 16:29:56.688735 19457 task_status_update_manager.cpp:181] Pausing
> sending task status updates
> I1005 16:29:56.688730 19450 slave.cpp:1254] New master detected at
> [email protected]:46083
> I1005 16:29:56.688978 19450 slave.cpp:1319] Detecting new master
> I1005 16:29:56.689049 19454 http.cpp:1177] HTTP POST for
> /master/api/v1/scheduler from 10.0.49.2:38752
> I1005 16:29:56.691651 19454 master.cpp:2502] Received subscription request
> for HTTP framework 'default'
> I1005 16:29:56.691812 19454 master.cpp:2155] Authorizing framework principal
> 'test-principal' to receive offers for roles '{ * }'
> I1005 16:29:56.693610 19460 master.cpp:2637] Subscribing framework 'default'
> with checkpointing enabled and capabilities [ MULTI_ROLE,
> RESERVATION_REFINEMENT ]
> I1005 16:29:56.696306 19460 master.cpp:9883] Adding framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default) with roles { } suppressed
> I1005 16:29:56.697732 19427 slave.cpp:1346] Authenticating with master
> [email protected]:46083
> I1005 16:29:56.697902 19427 slave.cpp:1355] Using default CRAM-MD5
> authenticatee
> I1005 16:29:56.698575 19429 authenticatee.cpp:97] Initializing client SASL
> I1005 16:29:56.698640 19420 hierarchical.cpp:306] Added framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:56.698917 19429 authenticatee.cpp:121] Creating new client SASL
> connection
> I1005 16:29:56.768946 19452 master.cpp:9653] Authenticating
> [email protected]:46083
> I1005 16:29:56.769793 19453 authenticator.cpp:98] Creating new server SASL
> connection
> I1005 16:29:56.836316 19456 authenticatee.cpp:213] Received SASL
> authentication mechanisms: CRAM-MD5
> I1005 16:29:56.836383 19456 authenticatee.cpp:239] Attempting to authenticate
> with mechanism 'CRAM-MD5'
> I1005 16:29:56.836539 19456 authenticator.cpp:204] Received SASL
> authentication start
> I1005 16:29:56.836781 19456 authenticator.cpp:326] Authentication requires
> more steps
> I1005 16:29:56.836979 19457 authenticatee.cpp:259] Received SASL
> authentication step
> I1005 16:29:56.837357 19446 authenticator.cpp:232] Received SASL
> authentication step
> I1005 16:29:56.837566 19446 authenticator.cpp:318] Authentication success
> I1005 16:29:56.837731 19450 authenticatee.cpp:299] Authentication success
> I1005 16:29:56.838047 19458 master.cpp:9685] Successfully authenticated
> principal 'test-principal' at [email protected]:46083
> I1005 16:29:56.838515 19454 slave.cpp:1446] Successfully authenticated with
> master [email protected]:46083
> I1005 16:29:56.839573 19465 master.cpp:6605] Received register agent message
> from [email protected]:46083 (core-dev)
> I1005 16:29:56.839889 19465 master.cpp:3964] Authorizing agent providing
> resources 'cpus:2; mem:1024; disk:1024; ports:[31000-32000]' with principal
> 'test-principal'
> I1005 16:29:56.841236 19465 master.cpp:6787] Registering agent at
> [email protected]:46083 (core-dev) with id
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0
> I1005 16:29:56.842396 19460 registrar.cpp:487] Applied 1 operations in
> 257448ns; attempting to update the registry
> I1005 16:29:56.843335 19424 master.cpp:6598] Ignoring register agent message
> from [email protected]:46083 (core-dev) as registration is already in progress
> I1005 16:29:56.845530 19430 registrar.cpp:544] Successfully updated the
> registry in 3.021824ms
> I1005 16:29:56.847339 19421 master.cpp:6880] Registered agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> with cpus:2; mem:1024; disk:1024; ports:[31000-32000]
> I1005 16:29:56.847558 19436 slave.cpp:1479] Registered with master
> [email protected]:46083; given agent ID ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0
> I1005 16:29:56.847872 19420 task_status_update_manager.cpp:188] Resuming
> sending task status updates
> I1005 16:29:56.847980 19426 hierarchical.cpp:605] Added agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 (core-dev) with cpus:2; mem:1024;
> disk:1024; ports:[31000-32000] (allocated: {})
> I1005 16:29:56.849967 19436 slave.cpp:1548] Forwarding agent update
> {"operations":{},"resource_version_uuid":{"value":"mn0MFeEJQT+0k1KDp+d7UA=="},"slave_id":{"value":"ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0"},"update_oversubscribed_resources":false}
> I1005 16:29:56.851047 19438 master.cpp:9468] Sending offers [
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-O0 ] to framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default)
> I1005 16:29:56.852563 19438 master.cpp:7939] Ignoring update on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> as it reports no changes
> I1005 16:29:56.857623 19466 http.cpp:1177] HTTP POST for
> /master/api/v1/scheduler from 10.0.49.2:38754
> I1005 16:29:56.859246 19466 master.cpp:11462] Removing offer
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-O0
> I1005 16:29:56.859964 19466 master.cpp:4467] Processing ACCEPT call for
> offers: [ ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-O0 ] on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> for framework ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default)
> I1005 16:29:56.860101 19466 master.cpp:3541] Authorizing framework principal
> 'test-principal' to launch task 2dfad441-69a4-469f-978a-f2909b1a16da
> I1005 16:29:56.863557 19461 master.cpp:12209] Adding task
> 2dfad441-69a4-469f-978a-f2909b1a16da with resources cpus(allocated: *):0.1;
> mem(allocated: *):32; disk(allocated: *):32 on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:29:56.863749 19461 master.cpp:5663] Launching task group {
> 2dfad441-69a4-469f-978a-f2909b1a16da } of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default) with resources
> cpus(allocated: *):0.1; mem(allocated: *):32; disk(allocated: *):32 on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> on new executor
> I1005 16:29:56.864729 19422 slave.cpp:2014] Got assigned task group
> containing tasks [ 2dfad441-69a4-469f-978a-f2909b1a16da ] for framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:56.868407 19422 slave.cpp:2388] Authorizing task group containing
> tasks [ 2dfad441-69a4-469f-978a-f2909b1a16da ] for framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:56.868506 19422 slave.cpp:8466] Authorizing framework principal
> 'test-principal' to launch task 2dfad441-69a4-469f-978a-f2909b1a16da
> I1005 16:29:56.871137 19422 slave.cpp:2831] Launching task group containing
> tasks [ 2dfad441-69a4-469f-978a-f2909b1a16da ] for framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:56.873203 19422 paths.cpp:752] Creating sandbox
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/slaves/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0/frameworks/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000/executors/test_default_executor/runs/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> for user 'root'
> I1005 16:29:56.874781 19422 paths.cpp:755] Creating sandbox
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta/slaves/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0/frameworks/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000/executors/test_default_executor/runs/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> I1005 16:29:56.875211 19422 slave.cpp:8994] Launching executor
> 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 with resources
> [{"allocation_info":{"role":"*"},"name":"cpus","scalar":{"value":0.1},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"mem","scalar":{"value":32.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"disk","scalar":{"value":32.0},"type":"SCALAR"}]
> in work directory
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/slaves/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0/frameworks/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000/executors/test_default_executor/runs/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> I1005 16:29:56.876286 19422 slave.cpp:3509] Launching container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac for executor 'test_default_executor' of
> framework ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:56.878290 19422 slave.cpp:3028] Queued task group containing
> tasks [ 2dfad441-69a4-469f-978a-f2909b1a16da ] for executor
> 'test_default_executor' of framework ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:56.879345 19437 containerizer.cpp:1280] Starting container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:56.882952 19437 containerizer.cpp:3122] Transitioning the state
> of container 1c7a71c2-bbda-4225-93fa-942f86d74dac from PROVISIONING to
> PREPARING
> I1005 16:29:56.888557 19445 memory.cpp:478] Started listening for OOM events
> for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:56.888859 19445 memory.cpp:590] Started listening on 'low' memory
> pressure events for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:56.888999 19445 memory.cpp:590] Started listening on 'medium'
> memory pressure events for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:56.889108 19445 memory.cpp:590] Started listening on 'critical'
> memory pressure events for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:56.890103 19457 memory.cpp:198] Updated
> 'memory.soft_limit_in_bytes' to 32MB for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:56.890336 19457 memory.cpp:227] Updated 'memory.limit_in_bytes'
> to 32MB for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:56.899499 19466 linux_launcher.cpp:492] Launching container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac and cloning with namespaces CLONE_NEWNS
> I1005 16:29:56.906123 19465 containerizer.cpp:2044] Checkpointing container's
> forked pid 19570 to
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta/slaves/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0/frameworks/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000/executors/test_default_executor/runs/1c7a71c2-bbda-4225-93fa-942f86d74dac/pids/forked.pid'
> I1005 16:29:56.907407 19465 containerizer.cpp:3122] Transitioning the state
> of container 1c7a71c2-bbda-4225-93fa-942f86d74dac from PREPARING to ISOLATING
> I1005 16:29:56.909687 19428 containerizer.cpp:3122] Transitioning the state
> of container 1c7a71c2-bbda-4225-93fa-942f86d74dac from ISOLATING to FETCHING
> I1005 16:29:56.912649 19456 containerizer.cpp:3122] Transitioning the state
> of container 1c7a71c2-bbda-4225-93fa-942f86d74dac from FETCHING to RUNNING
> Marked '/' as rslave
> I1005 16:29:57.189963 19611 executor.cpp:201] Version: 1.8.0
> I1005 16:29:57.204545 19466 http.cpp:1177] HTTP POST for
> /agent/api/v1/executor from 10.0.49.2:38756
> I1005 16:29:57.205004 19466 slave.cpp:4607] Received Subscribe request for
> HTTP executor 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.205142 19466 slave.cpp:4670] Creating a marker file for HTTP
> based executor 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (via HTTP) at path
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta/slaves/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0/frameworks/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000/executors/test_default_executor/runs/1c7a71c2-bbda-4225-93fa-942f86d74dac/http.marker'
> I1005 16:29:57.209298 19422 memory.cpp:198] Updated
> 'memory.soft_limit_in_bytes' to 64MB for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:57.209498 19422 memory.cpp:227] Updated 'memory.limit_in_bytes'
> to 64MB for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:57.210960 19465 slave.cpp:3282] Sending queued task group
> containing tasks [ 2dfad441-69a4-469f-978a-f2909b1a16da ] to executor
> 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (via HTTP)
> I1005 16:29:57.213850 19634 default_executor.cpp:204] Received SUBSCRIBED
> event
> I1005 16:29:57.216274 19634 default_executor.cpp:208] Subscribed executor on
> core-dev
> I1005 16:29:57.216761 19634 default_executor.cpp:204] Received LAUNCH_GROUP
> event
> I1005 16:29:57.217589 19641 default_executor.cpp:428] Setting
> 'MESOS_CONTAINER_IP' to: 10.0.49.2
> I1005 16:29:57.222965 19445 http.cpp:1177] HTTP POST for
> /agent/api/v1/executor from 10.0.49.2:38758
> I1005 16:29:57.223498 19445 slave.cpp:5269] Handling status update
> TASK_STARTING (Status UUID: 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.225296 19462 task_status_update_manager.cpp:328] Received task
> status update TASK_STARTING (Status UUID:
> 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.225340 19445 http.cpp:1177] HTTP POST for /agent/api/v1 from
> 10.0.49.2:38760
> I1005 16:29:57.226431 19445 http.cpp:2408] Processing LAUNCH_NESTED_CONTAINER
> call for container
> '1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf'
> I1005 16:29:57.226475 19462 task_status_update_manager.cpp:842] Checkpointing
> UPDATE for task status update TASK_STARTING (Status UUID:
> 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.228291 19445 slave.cpp:5761] Forwarding the update
> TASK_STARTING (Status UUID: 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 to [email protected]:46083
> I1005 16:29:57.228845 19454 master.cpp:8375] Status update TASK_STARTING
> (Status UUID: 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 from agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:29:57.229161 19454 master.cpp:8432] Forwarding status update
> TASK_STARTING (Status UUID: 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.229444 19441 containerizer.cpp:1242] Creating sandbox
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/slaves/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0/frameworks/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000/executors/test_default_executor/runs/1c7a71c2-bbda-4225-93fa-942f86d74dac/containers/fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf'
> for user 'root'
> I1005 16:29:57.229817 19454 master.cpp:10932] Updating the state of task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (latest state: TASK_STARTING,
> status update state: TASK_STARTING)
> I1005 16:29:57.230021 19441 containerizer.cpp:1280] Starting container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> I1005 16:29:57.230085 19608 default_executor.cpp:204] Received ACKNOWLEDGED
> event
> I1005 16:29:57.231626 19441 containerizer.cpp:3122] Transitioning the state
> of container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> from PROVISIONING to PREPARING
> I1005 16:29:57.236529 19447 linux_launcher.cpp:492] Launching nested
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf and
> cloning with namespaces CLONE_NEWNS
> I1005 16:29:57.251343 19429 containerizer.cpp:3122] Transitioning the state
> of container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> from PREPARING to ISOLATING
> I1005 16:29:57.252596 19436 containerizer.cpp:3122] Transitioning the state
> of container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> from ISOLATING to FETCHING
> I1005 16:29:57.253917 19423 containerizer.cpp:3122] Transitioning the state
> of container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> from FETCHING to RUNNING
> I1005 16:29:57.258623 19606 default_executor.cpp:663] Finished launching
> tasks [ 2dfad441-69a4-469f-978a-f2909b1a16da ] in child containers [
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf ]
> I1005 16:29:57.258700 19606 default_executor.cpp:687] Waiting on child
> containers of tasks [ 2dfad441-69a4-469f-978a-f2909b1a16da ]
> I1005 16:29:57.259757 19615 default_executor.cpp:748] Waiting for child
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf of
> task '2dfad441-69a4-469f-978a-f2909b1a16da'
> I1005 16:29:57.261754 19438 http.cpp:1177] HTTP POST for /agent/api/v1 from
> 10.0.49.2:38762
> I1005 16:29:57.262133 19438 http.cpp:2643] Processing WAIT_NESTED_CONTAINER
> call for container
> '1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf'
> I1005 16:29:57.274417 19462 http.cpp:1177] HTTP POST for
> /master/api/v1/scheduler from 10.0.49.2:38754
> I1005 16:29:57.274698 19462 master.cpp:6241] Processing ACKNOWLEDGE call for
> status 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default) on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0
> I1005 16:29:57.275720 19450 task_status_update_manager.cpp:401] Received task
> status update acknowledgement (UUID: 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca)
> for task 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.275925 19450 task_status_update_manager.cpp:842] Checkpointing
> ACK for task status update TASK_STARTING (Status UUID:
> 4eb5fde9-4590-4b19-ab60-e1d4a3faa0ca) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.300184 19429 http.cpp:1177] HTTP POST for
> /agent/api/v1/executor from 10.0.49.2:38758
> I1005 16:29:57.300438 19429 slave.cpp:5269] Handling status update
> TASK_RUNNING (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.303266 19438 task_status_update_manager.cpp:328] Received task
> status update TASK_RUNNING (Status UUID:
> c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.303362 19438 task_status_update_manager.cpp:842] Checkpointing
> UPDATE for task status update TASK_RUNNING (Status UUID:
> c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.303864 19426 slave.cpp:5761] Forwarding the update
> TASK_RUNNING (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 to [email protected]:46083
> I1005 16:29:57.304399 19455 master.cpp:8375] Status update TASK_RUNNING
> (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 from agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:29:57.304487 19455 master.cpp:8432] Forwarding status update
> TASK_RUNNING (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.305212 19455 master.cpp:10932] Updating the state of task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (latest state: TASK_RUNNING, status
> update state: TASK_RUNNING)
> I1005 16:29:57.305565 19630 default_executor.cpp:204] Received ACKNOWLEDGED
> event
> I1005 16:29:57.307138 19406 slave.cpp:909] Agent terminating
> I1005 16:29:57.308324 19442 master.cpp:1251] Agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> disconnected
> I1005 16:29:57.308380 19442 master.cpp:3267] Disconnecting agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:29:57.308480 19442 master.cpp:3286] Deactivating agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:29:57.308667 19423 hierarchical.cpp:803] Agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 deactivated
> E1005 16:29:57.309021 19636 executor.cpp:714] End-Of-File received from
> agent. The agent closed the event stream
> I1005 16:29:57.312898 19643 default_executor.cpp:176] Disconnected from agent
> W1005 16:29:57.317960 19609 executor.cpp:666] Received '404 Not Found' () for
> SUBSCRIBE
> I1005 16:29:57.321012 19406 containerizer.cpp:305] Using isolation {
> environment_secret, volume/sandbox_path, docker/runtime, cgroups/all,
> filesystem/linux, volume/image, network/cni, volume/host_path }
> I1005 16:29:57.328989 19406 linux_launcher.cpp:144] Using
> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
> sh: hadoop: command not found
> I1005 16:29:57.348228 19464 http.cpp:1177] HTTP POST for
> /master/api/v1/scheduler from 10.0.49.2:38754
> W1005 16:29:57.348446 19464 master.cpp:6232] Cannot send status update
> acknowledgement for status c240839c-2992-449f-80c0-dc454870fc1b of task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default) to agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> because agent is disconnected
> Marked '/' as rslave
> I1005 16:29:57.429600 19406 fetcher.cpp:66] Skipping URI fetcher plugin
> 'hadoop' as it could not be created: Failed to create HDFS client: Hadoop
> client is not available, exit status: 32512
> I1005 16:29:57.431618 19406 provisioner.cpp:281] Provisioner backend
> 'overlay' is not supported on
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/provisioner':
> Backend 'overlay' is not supported due to missing d_type support on the
> underlying filesystem
> I1005 16:29:57.432054 19406 provisioner.cpp:298] Using default backend 'copy'
> W1005 16:29:57.435797 19406 cgroups.cpp:115] Cannot automatically load the
> subsystem 'freezer' because it is not supported by Mesos cgroups isolator yet
> I1005 16:29:57.436004 19406 cgroups.cpp:128] Automatically loading
> subsystems: { blkio, cpu, cpuacct, cpuset, devices, hugetlb, memory, net_cls,
> net_prio, perf_event, pids }
> I1005 16:29:57.520114 19406 cluster.cpp:485] Creating default 'local'
> authorizer
> I1005 16:29:57.521266 19450 slave.cpp:267] Mesos agent started on
> @10.0.49.2:46083
> I1005 16:29:57.521317 19450 slave.cpp:268] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://";
> --appc_store_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="false"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authentication_timeout_max="1mins" --authentication_timeout_min="5secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_destroy_timeout="1mins" --cgroups_enable_cfs="false"
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
> --cgroups_root="mesos_test_a0422359-e2a8-4897-82b9-201252686cc1"
> --container_disk_watch_interval="15secs" --containerizers="mesos"
> --credential="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/credential"
> --default_role="*" --disallow_sharing_agent_pid_namespace="false"
> --disk_watch_interval="1mins" --docker="docker" --
docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io";
--docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock"
--docker_stop_timeout="0ns"
--docker_store_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/store/docker"
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins"
--executor_reregistration_timeout="2secs"
--executor_shutdown_grace_period="5secs"
--fetcher_cache_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/fetch"
--fetcher_cache_size="2GB" --fetcher_stall_timeout="1mins"
--frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
--gc_non_executor_container_sandboxes="false" --help="false"
--hostname_lookup="true" --http_command_executor="false"
--http_credentials="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/http_credentials"
--
http_heartbeat_interval="30secs" --image_providers="docker"
--initialize_driver_logging="true"
--isolation="filesystem/linux,docker/runtime,cgroups/all" --launcher="linux"
--launcher_dir="/home/gilbert/mesos/build/src" --logbufsecs="0"
--logging_level="INFO" --max_completed_executors_per_framework="150"
--memory_profiling="false" --network_cni_metrics="true"
--oversubscribed_resources_interval="15secs" --perf_duration="10secs"
--perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
--quiet="false" --reconfiguration_policy="equal" --recover="reconnect"
--recovery_timeout="15mins" --registration_backoff_factor="10ms"
--resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]"
--revocable_cpu_low_priority="true"
--runtime_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm"
--sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true"
--systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/sys
tem" --version="false"
--work_dir="/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn"
--zk_session_timeout="10secs"
> I1005 16:29:57.521845 19450 credentials.hpp:86] Loading credential for
> authentication from
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/credential'
> I1005 16:29:57.522027 19450 slave.cpp:300] Agent using credential for:
> test-principal
> I1005 16:29:57.522047 19450 credentials.hpp:37] Loading credentials for
> authentication from
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/http_credentials'
> I1005 16:29:57.522244 19450 http.cpp:1037] Creating default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> I1005 16:29:57.522529 19450 disk_profile_adaptor.cpp:80] Creating default
> disk profile adaptor module
> I1005 16:29:57.527338 19450 slave.cpp:615] Agent resources:
> [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}]
> I1005 16:29:57.527516 19450 slave.cpp:623] Agent attributes: [ ]
> I1005 16:29:57.527657 19450 slave.cpp:632] Agent hostname: core-dev
> I1005 16:29:57.527848 19461 task_status_update_manager.cpp:181] Pausing
> sending task status updates
> I1005 16:29:57.528728 19420 state.cpp:66] Recovering state from
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta'
> I1005 16:29:57.528851 19420 state.cpp:711] No committed checkpointed
> resources found at
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta/resources/resources.info'
> I1005 16:29:57.533944 19443 slave.cpp:6909] Finished recovering checkpointed
> state from
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta',
> beginning agent recovery
> I1005 16:29:57.534601 19443 slave.cpp:7388] Recovering framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.534780 19443 slave.cpp:9112] Recovering executor
> 'test_default_executor' of framework ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.535992 19444 task_status_update_manager.cpp:207] Recovering
> task status update manager
> I1005 16:29:57.536197 19444 task_status_update_manager.cpp:215] Recovering
> executor 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.537575 19425 containerizer.cpp:727] Recovering Mesos containers
> I1005 16:29:57.537711 19425 containerizer.cpp:784] Recovering container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac for executor 'test_default_executor' of
> framework ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:57.540509 19448 linux_launcher.cpp:286] Recovering Linux launcher
> I1005 16:29:57.541021 19448 linux_launcher.cpp:331] Not recovering cgroup
> mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos
> I1005 16:29:57.541100 19448 linux_launcher.cpp:343] Recovered container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:57.541293 19448 linux_launcher.cpp:343] Recovered container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> I1005 16:29:57.541618 19421 containerizer.cpp:1053] Recovering isolators
> W1005 16:29:57.542055 19422 cgroups.cpp:347] Couldn't find the cgroup
> 'mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> in hierarchy '/sys/fs/cgroup/blkio' for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> W1005 16:29:57.542112 19422 cgroups.cpp:347] Couldn't find the cgroup
> 'mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> in hierarchy '/sys/fs/cgroup/cpu,cpuacct' for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> W1005 16:29:57.542135 19422 cgroups.cpp:347] Couldn't find the cgroup
> 'mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> in hierarchy '/sys/fs/cgroup/cpuset' for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> W1005 16:29:57.542155 19422 cgroups.cpp:347] Couldn't find the cgroup
> 'mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> in hierarchy '/sys/fs/cgroup/devices' for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> W1005 16:29:57.542176 19422 cgroups.cpp:347] Couldn't find the cgroup
> 'mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> in hierarchy '/sys/fs/cgroup/hugetlb' for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> W1005 16:29:57.542296 19422 cgroups.cpp:347] Couldn't find the cgroup
> 'mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> in hierarchy '/sys/fs/cgroup/net_cls,net_prio' for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> W1005 16:29:57.542327 19422 cgroups.cpp:347] Couldn't find the cgroup
> 'mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> in hierarchy '/sys/fs/cgroup/perf_event' for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> W1005 16:29:57.542367 19422 cgroups.cpp:347] Couldn't find the cgroup
> 'mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> in hierarchy '/sys/fs/cgroup/pids' for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:57.542619 19455 memory.cpp:478] Started listening for OOM events
> for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:57.542989 19455 memory.cpp:590] Started listening on 'low' memory
> pressure events for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:57.543119 19455 memory.cpp:590] Started listening on 'medium'
> memory pressure events for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:57.543233 19455 memory.cpp:590] Started listening on 'critical'
> memory pressure events for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:57.546376 19458 containerizer.cpp:1092] Recovering provisioner
> I1005 16:29:57.546880 19441 metadata_manager.cpp:254] No images to load from
> disk. Docker provisioner image storage path
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/store/docker/storedImages'
> does not exist
> I1005 16:29:57.547260 19443 provisioner.cpp:494] Provisioner recovery complete
> I1005 16:29:57.549708 19442 composing.cpp:339] Finished recovering all
> containerizers
> I1005 16:29:57.549911 19455 slave.cpp:7138] Recovering executors
> I1005 16:29:57.550199 19455 slave.cpp:7225] Waiting for executor
> 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (via HTTP) to subscribe
> I1005 16:29:58.243501 19434 http.cpp:1177] HTTP POST for
> /agent/api/v1/executor from 10.0.49.2:38764
> I1005 16:29:58.243890 19434 slave.cpp:4607] Received Subscribe request for
> HTTP executor 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (via HTTP)
> I1005 16:29:58.244009 19434 slave.cpp:4670] Creating a marker file for HTTP
> based executor 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (via HTTP) at path
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/meta/slaves/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0/frameworks/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000/executors/test_default_executor/runs/1c7a71c2-bbda-4225-93fa-942f86d74dac/http.marker'
> I1005 16:29:58.246625 19429 memory.cpp:198] Updated
> 'memory.soft_limit_in_bytes' to 64MB for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:29:58.248239 19632 default_executor.cpp:204] Received SUBSCRIBED
> event
> I1005 16:29:58.248303 19632 default_executor.cpp:208] Subscribed executor on
> core-dev
> I1005 16:29:58.248368 19632 default_executor.cpp:687] Waiting on child
> containers of tasks [ 2dfad441-69a4-469f-978a-f2909b1a16da ]
> I1005 16:29:58.249651 19633 default_executor.cpp:748] Waiting for child
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf of
> task '2dfad441-69a4-469f-978a-f2909b1a16da'
> I1005 16:29:58.251660 19442 http.cpp:1177] HTTP POST for /agent/api/v1 from
> 10.0.49.2:38776
> W1005 16:29:58.252976 19641 default_executor.cpp:814] Received '503 Service
> Unavailable' (Agent has not finished recovery) waiting on child container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf of
> task '2dfad441-69a4-469f-978a-f2909b1a16da'
> I1005 16:29:58.254017 19598 default_executor.cpp:1568] Established connection
> to wait for child container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf of
> task '2dfad441-69a4-469f-978a-f2909b1a16da'; Retrying the
> WAIT_NESTED_CONTAINER call in 1secs
> I1005 16:29:59.254761 19613 default_executor.cpp:748] Waiting for child
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf of
> task '2dfad441-69a4-469f-978a-f2909b1a16da'
> I1005 16:29:59.256939 19435 http.cpp:1177] HTTP POST for /agent/api/v1 from
> 10.0.49.2:38778
> W1005 16:29:59.258285 19614 default_executor.cpp:814] Received '503 Service
> Unavailable' (Agent has not finished recovery) waiting on child container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf of
> task '2dfad441-69a4-469f-978a-f2909b1a16da'
> I1005 16:29:59.259327 19611 default_executor.cpp:1568] Established connection
> to wait for child container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf of
> task '2dfad441-69a4-469f-978a-f2909b1a16da'; Retrying the
> WAIT_NESTED_CONTAINER call in 1secs
> I1005 16:29:59.551455 19430 slave.cpp:5197] Cleaning up un-reregistered
> executors
> I1005 16:29:59.551661 19430 slave.cpp:7291] Finished recovery
> I1005 16:29:59.553337 19430 slave.cpp:1254] New master detected at
> [email protected]:46083
> I1005 16:29:59.553360 19428 task_status_update_manager.cpp:181] Pausing
> sending task status updates
> I1005 16:29:59.553700 19430 slave.cpp:1319] Detecting new master
> I1005 16:29:59.558171 19422 slave.cpp:1346] Authenticating with master
> [email protected]:46083
> I1005 16:29:59.558255 19422 slave.cpp:1355] Using default CRAM-MD5
> authenticatee
> I1005 16:29:59.558544 19426 authenticatee.cpp:121] Creating new client SASL
> connection
> I1005 16:29:59.628659 19424 master.cpp:9653] Authenticating
> [email protected]:46083
> I1005 16:29:59.629145 19462 authenticator.cpp:98] Creating new server SASL
> connection
> I1005 16:29:59.697259 19446 authenticatee.cpp:213] Received SASL
> authentication mechanisms: CRAM-MD5
> I1005 16:29:59.697304 19446 authenticatee.cpp:239] Attempting to authenticate
> with mechanism 'CRAM-MD5'
> I1005 16:29:59.697412 19446 authenticator.cpp:204] Received SASL
> authentication start
> I1005 16:29:59.697468 19446 authenticator.cpp:326] Authentication requires
> more steps
> I1005 16:29:59.697562 19446 authenticatee.cpp:259] Received SASL
> authentication step
> I1005 16:29:59.697672 19446 authenticator.cpp:232] Received SASL
> authentication step
> I1005 16:29:59.697748 19446 authenticator.cpp:318] Authentication success
> I1005 16:29:59.697875 19466 authenticatee.cpp:299] Authentication success
> I1005 16:29:59.697957 19431 master.cpp:9685] Successfully authenticated
> principal 'test-principal' at [email protected]:46083
> I1005 16:29:59.698292 19437 slave.cpp:1446] Successfully authenticated with
> master [email protected]:46083
> I1005 16:29:59.700304 19458 master.cpp:6959] Received reregister agent
> message from agent ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at
> [email protected]:46083 (core-dev)
> I1005 16:29:59.700664 19458 master.cpp:3964] Authorizing agent providing
> resources 'cpus:2; mem:1024; disk:1024; ports:[31000-32000]' with principal
> 'test-principal'
> I1005 16:29:59.702517 19436 hierarchical.cpp:705] Agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 (core-dev) updated with total
> resources cpus:2; mem:1024; disk:1024; ports:[31000-32000]
> I1005 16:29:59.703006 19433 slave.cpp:1585] Re-registered with master
> [email protected]:46083
> I1005 16:29:59.703155 19443 task_status_update_manager.cpp:188] Resuming
> sending task status updates
> W1005 16:29:59.703191 19443 task_status_update_manager.cpp:195] Resending
> task status update TASK_RUNNING (Status UUID:
> c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:59.703153 19433 slave.cpp:1630] Forwarding agent update
> {"operations":{},"resource_version_uuid":{"value":"Zi/gXfjAQvSt1VyYtQbdWQ=="},"slave_id":{"value":"ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0"},"update_oversubscribed_resources":false}
> I1005 16:29:59.703436 19433 slave.cpp:5761] Forwarding the update
> TASK_RUNNING (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 to [email protected]:46083
> I1005 16:29:59.703495 19465 hierarchical.cpp:791] Agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 reactivated
> I1005 16:29:59.703779 19433 slave.cpp:4067] Updating info for framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:59.704190 19458 master.cpp:7939] Ignoring update on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> as it reports no changes
> I1005 16:29:59.704416 19458 master.cpp:8375] Status update TASK_RUNNING
> (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 from agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:29:59.704479 19458 master.cpp:8432] Forwarding status update
> TASK_RUNNING (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:59.704610 19433 task_status_update_manager.cpp:188] Resuming
> sending task status updates
> W1005 16:29:59.704641 19433 task_status_update_manager.cpp:195] Resending
> task status update TASK_RUNNING (Status UUID:
> c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:59.704887 19438 slave.cpp:5761] Forwarding the update
> TASK_RUNNING (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 to [email protected]:46083
> I1005 16:29:59.705044 19458 master.cpp:10932] Updating the state of task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (latest state: TASK_RUNNING, status
> update state: TASK_RUNNING)
> I1005 16:29:59.705271 19458 master.cpp:8375] Status update TASK_RUNNING
> (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 from agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:29:59.705366 19458 master.cpp:8432] Forwarding status update
> TASK_RUNNING (Status UUID: c240839c-2992-449f-80c0-dc454870fc1b) for task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:29:59.705773 19458 master.cpp:10932] Updating the state of task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (latest state: TASK_RUNNING, status
> update state: TASK_RUNNING)
> I1005 16:30:00.259867 19626 default_executor.cpp:748] Waiting for child
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf of
> task '2dfad441-69a4-469f-978a-f2909b1a16da'
> I1005 16:30:00.262334 19431 http.cpp:1177] HTTP POST for /agent/api/v1 from
> 10.0.49.2:38784
> I1005 16:30:00.263013 19431 http.cpp:2643] Processing WAIT_NESTED_CONTAINER
> call for container
> '1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf'
> I1005 16:30:02.505751 19430 master.cpp:9468] Sending offers [
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-O1 ] to framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default)
> I1005 16:30:02.554584 19457 http.cpp:1177] HTTP POST for
> /master/api/v1/scheduler from 10.0.49.2:38754
> I1005 16:30:02.555475 19457 master.cpp:11462] Removing offer
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-O1
> I1005 16:30:02.556067 19457 master.cpp:4467] Processing ACCEPT call for
> offers: [ ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-O1 ] on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> for framework ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default)
> I1005 16:30:02.556203 19457 master.cpp:3541] Authorizing framework principal
> 'test-principal' to launch task 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19
> I1005 16:30:02.558951 19433 master.cpp:12209] Adding task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 with resources cpus(allocated: *):0.1;
> mem(allocated: *):32; disk(allocated: *):32 on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:30:02.559180 19433 master.cpp:5663] Launching task group {
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 } of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default) with resources
> cpus(allocated: *):0.1; mem(allocated: *):32; disk(allocated: *):32 on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> on existing executor
> I1005 16:30:02.560184 19458 slave.cpp:2014] Got assigned task group
> containing tasks [ 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 ] for framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.561450 19458 slave.cpp:2388] Authorizing task group containing
> tasks [ 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 ] for framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.561545 19458 slave.cpp:8466] Authorizing framework principal
> 'test-principal' to launch task 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19
> I1005 16:30:02.563668 19458 slave.cpp:2831] Launching task group containing
> tasks [ 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 ] for framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.565382 19458 slave.cpp:3049] Queued task group containing
> tasks [ 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 ] for executor
> 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (via HTTP)
> I1005 16:30:02.567060 19455 memory.cpp:198] Updated
> 'memory.soft_limit_in_bytes' to 96MB for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:30:02.567256 19455 memory.cpp:227] Updated 'memory.limit_in_bytes'
> to 96MB for container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:30:02.568156 19460 slave.cpp:3282] Sending queued task group
> containing tasks [ 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 ] to executor
> 'test_default_executor' of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (via HTTP)
> I1005 16:30:02.571141 19609 default_executor.cpp:204] Received LAUNCH_GROUP
> event
> I1005 16:30:02.572221 19606 default_executor.cpp:428] Setting
> 'MESOS_CONTAINER_IP' to: 10.0.49.2
> I1005 16:30:02.575350 19435 http.cpp:1177] HTTP POST for
> /agent/api/v1/executor from 10.0.49.2:38766
> I1005 16:30:02.575618 19435 slave.cpp:5269] Handling status update
> TASK_STARTING (Status UUID: 568abfa7-455a-47de-88db-4c76d9633ae1) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.576120 19435 http.cpp:1177] HTTP POST for /agent/api/v1 from
> 10.0.49.2:38804
> I1005 16:30:02.576728 19443 task_status_update_manager.cpp:328] Received task
> status update TASK_STARTING (Status UUID:
> 568abfa7-455a-47de-88db-4c76d9633ae1) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.577009 19435 http.cpp:2408] Processing LAUNCH_NESTED_CONTAINER
> call for container
> '1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8'
> I1005 16:30:02.577536 19443 task_status_update_manager.cpp:842] Checkpointing
> UPDATE for task status update TASK_STARTING (Status UUID:
> 568abfa7-455a-47de-88db-4c76d9633ae1) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.578080 19441 slave.cpp:5761] Forwarding the update
> TASK_STARTING (Status UUID: 568abfa7-455a-47de-88db-4c76d9633ae1) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 to [email protected]:46083
> I1005 16:30:02.578150 19438 containerizer.cpp:1242] Creating sandbox
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_65BuJn/slaves/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0/frameworks/ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000/executors/test_default_executor/runs/1c7a71c2-bbda-4225-93fa-942f86d74dac/containers/44241a8c-49e6-4b54-880e-f6488df48bb8'
> for user 'root'
> I1005 16:30:02.578521 19433 master.cpp:8375] Status update TASK_STARTING
> (Status UUID: 568abfa7-455a-47de-88db-4c76d9633ae1) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 from agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:30:02.578821 19433 master.cpp:8432] Forwarding status update
> TASK_STARTING (Status UUID: 568abfa7-455a-47de-88db-4c76d9633ae1) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.578917 19438 containerizer.cpp:1280] Starting container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8
> I1005 16:30:02.579277 19433 master.cpp:10932] Updating the state of task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (latest state: TASK_STARTING,
> status update state: TASK_STARTING)
> I1005 16:30:02.579963 19627 default_executor.cpp:204] Received ACKNOWLEDGED
> event
> I1005 16:30:02.580211 19438 containerizer.cpp:3122] Transitioning the state
> of container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8
> from PROVISIONING to PREPARING
> I1005 16:30:02.585656 19434 linux_launcher.cpp:492] Launching nested
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8 and
> cloning with namespaces CLONE_NEWNS
> I1005 16:30:02.604260 19466 containerizer.cpp:3122] Transitioning the state
> of container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8
> from PREPARING to ISOLATING
> E1005 16:30:02.605676 19427 cgroups.cpp:693] Failed to assign pid 19691 to
> cgroup at
> '/sys/fs/cgroup/blkio/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac':
> No such file or directory
> W1005 16:30:02.606246 19464 http.cpp:2593] Failed to launch container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8:
> Collect failed: Failed to assign pid 19691 to cgroup at
> '/sys/fs/cgroup/blkio/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac':
> No such file or directory
> I1005 16:30:02.606894 19459 containerizer.cpp:2455] Destroying container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8 in
> ISOLATING state
> I1005 16:30:02.606958 19459 containerizer.cpp:3122] Transitioning the state
> of container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8
> from ISOLATING to DESTROYING
> E1005 16:30:02.607542 19625 default_executor.cpp:568] Received '400 Bad
> Request' (Collect failed: Failed to assign pid 19691 to cgroup at
> '/sys/fs/cgroup/blkio/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac':
> No such file or directory) while launching child container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8 of
> task '2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19'
> I1005 16:30:02.607585 19625 default_executor.cpp:663] Finished launching
> tasks [ 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 ] in child containers [
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8 ]
> I1005 16:30:02.607625 19625 default_executor.cpp:687] Waiting on child
> containers of tasks [ 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 ]
> I1005 16:30:02.608397 19628 default_executor.cpp:748] Waiting for child
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8 of
> task '2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19'
> I1005 16:30:02.608718 19428 linux_launcher.cpp:580] Asked to destroy
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8
> I1005 16:30:02.608880 19428 linux_launcher.cpp:622] Destroying cgroup
> '/sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/44241a8c-49e6-4b54-880e-f6488df48bb8'
> I1005 16:30:02.609771 19445 cgroups.cpp:2838] Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/44241a8c-49e6-4b54-880e-f6488df48bb8
> I1005 16:30:02.610183 19424 http.cpp:1177] HTTP POST for /agent/api/v1 from
> 10.0.49.2:38806
> I1005 16:30:02.610270 19455 cgroups.cpp:1229] Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/44241a8c-49e6-4b54-880e-f6488df48bb8
> after 379136ns
> I1005 16:30:02.610683 19424 http.cpp:2643] Processing WAIT_NESTED_CONTAINER
> call for container
> '1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8'
> I1005 16:30:02.611667 19445 cgroups.cpp:2856] Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/44241a8c-49e6-4b54-880e-f6488df48bb8
> I1005 16:30:02.611996 19444 cgroups.cpp:1258] Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/44241a8c-49e6-4b54-880e-f6488df48bb8
> after 264704ns
> I1005 16:30:02.622416 19448 http.cpp:1177] HTTP POST for
> /master/api/v1/scheduler from 10.0.49.2:38754
> I1005 16:30:02.622663 19448 master.cpp:6241] Processing ACKNOWLEDGE call for
> status 568abfa7-455a-47de-88db-4c76d9633ae1 for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default) on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0
> I1005 16:30:02.623314 19465 task_status_update_manager.cpp:401] Received task
> status update acknowledgement (UUID: 568abfa7-455a-47de-88db-4c76d9633ae1)
> for task 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.623459 19465 task_status_update_manager.cpp:842] Checkpointing
> ACK for task status update TASK_STARTING (Status UUID:
> 568abfa7-455a-47de-88db-4c76d9633ae1) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.664566 19462 containerizer.cpp:2961] Container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8 has
> exited
> I1005 16:30:02.675168 19456 containerizer.cpp:2747] Checkpointing termination
> state to nested container's runtime directory
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/containers/1c7a71c2-bbda-4225-93fa-942f86d74dac/containers/44241a8c-49e6-4b54-880e-f6488df48bb8/termination'
> I1005 16:30:02.681341 19642 default_executor.cpp:955] Child container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.44241a8c-49e6-4b54-880e-f6488df48bb8 of
> task '2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19' completed in state TASK_FAILED:
> Command terminated with signal Killed
> I1005 16:30:02.696069 19642 default_executor.cpp:976] Killing task group
> containing tasks [ 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 ]
> I1005 16:30:02.723335 19423 http.cpp:1177] HTTP POST for
> /agent/api/v1/executor from 10.0.49.2:38766
> I1005 16:30:02.723626 19423 slave.cpp:5269] Handling status update
> TASK_FAILED (Status UUID: c2a9a54e-5d45-400f-98ab-cb6d1a469b9f) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.725311 19443 memory.cpp:198] Updated
> 'memory.soft_limit_in_bytes' to 64MB for container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:30:02.726495 19453 task_status_update_manager.cpp:328] Received task
> status update TASK_FAILED (Status UUID: c2a9a54e-5d45-400f-98ab-cb6d1a469b9f)
> for task 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.726600 19453 task_status_update_manager.cpp:842] Checkpointing
> UPDATE for task status update TASK_FAILED (Status UUID:
> c2a9a54e-5d45-400f-98ab-cb6d1a469b9f) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.727046 19432 slave.cpp:5761] Forwarding the update TASK_FAILED
> (Status UUID: c2a9a54e-5d45-400f-98ab-cb6d1a469b9f) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 to [email protected]:46083
> I1005 16:30:02.727538 19447 master.cpp:8375] Status update TASK_FAILED
> (Status UUID: c2a9a54e-5d45-400f-98ab-cb6d1a469b9f) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 from agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:30:02.727638 19447 master.cpp:8432] Forwarding status update
> TASK_FAILED (Status UUID: c2a9a54e-5d45-400f-98ab-cb6d1a469b9f) for task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.728142 19447 master.cpp:10932] Updating the state of task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (latest state: TASK_FAILED, status
> update state: TASK_FAILED)
> I1005 16:30:02.728863 19608 default_executor.cpp:204] Received ACKNOWLEDGED
> event
> ../../src/tests/containerizer/cgroups_isolator_tests.cpp:2051: Failure
> Expected: v1::TASK_RUNNING
> Which is: TASK_RUNNING
> To be equal to: runningUpdate2->status().state()
> Which is: TASK_FAILED
> I1005 16:30:02.746906 19430 master.cpp:1366] Framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default) disconnected
> I1005 16:30:02.746963 19430 master.cpp:3230] Deactivating framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default)
> I1005 16:30:02.746976 19406 slave.cpp:909] Agent terminating
> I1005 16:30:02.747051 19430 master.cpp:3207] Disconnecting framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default)
> I1005 16:30:02.747128 19430 master.cpp:1381] Giving framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default) 0ns to failover
> I1005 16:30:02.747406 19440 hierarchical.cpp:420] Deactivated framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.748003 19427 master.cpp:1251] Agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> disconnected
> I1005 16:30:02.748054 19427 master.cpp:3267] Disconnecting agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:30:02.748113 19427 master.cpp:3286] Deactivating agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:30:02.748286 19457 hierarchical.cpp:803] Agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 deactivated
> I1005 16:30:02.748762 19439 containerizer.cpp:2455] Destroying container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac in RUNNING state
> I1005 16:30:02.748795 19421 master.cpp:9261] Framework failover timeout,
> removing framework ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default)
> I1005 16:30:02.748842 19439 containerizer.cpp:3122] Transitioning the state
> of container 1c7a71c2-bbda-4225-93fa-942f86d74dac from RUNNING to DESTROYING
> I1005 16:30:02.749228 19421 master.cpp:10197] Removing framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (default)
> I1005 16:30:02.749256 19439 containerizer.cpp:2455] Destroying container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf in
> RUNNING state
> I1005 16:30:02.749336 19439 containerizer.cpp:3122] Transitioning the state
> of container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> from RUNNING to DESTROYING
> I1005 16:30:02.749577 19421 master.cpp:10932] Updating the state of task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (latest state: TASK_FAILED, status
> update state: TASK_KILLED)
> I1005 16:30:02.749708 19421 master.cpp:11030] Removing task
> 2b6f0c9c-5000-432b-9ebe-e8dfbd7dcc19 with resources cpus(allocated: *):0.1;
> mem(allocated: *):32; disk(allocated: *):32 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:30:02.750182 19426 linux_launcher.cpp:580] Asked to destroy
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> I1005 16:30:02.750293 19421 master.cpp:10932] Updating the state of task
> 2dfad441-69a4-469f-978a-f2909b1a16da of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 (latest state: TASK_KILLED, status
> update state: TASK_KILLED)
> I1005 16:30:02.750313 19426 linux_launcher.cpp:622] Destroying cgroup
> '/sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf'
> I1005 16:30:02.750910 19458 cgroups.cpp:2838] Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> I1005 16:30:02.751296 19449 cgroups.cpp:1229] Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> after 317184ns
> I1005 16:30:02.751327 19421 master.cpp:11030] Removing task
> 2dfad441-69a4-469f-978a-f2909b1a16da with resources cpus(allocated: *):0.1;
> mem(allocated: *):32; disk(allocated: *):32 of framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:30:02.752017 19460 cgroups.cpp:2856] Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> I1005 16:30:02.751896 19421 master.cpp:11061] Removing executor
> 'test_default_executor' with resources
> [{"allocation_info":{"role":"*"},"name":"cpus","scalar":{"value":0.1},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"mem","scalar":{"value":32.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"disk","scalar":{"value":32.0},"type":"SCALAR"}]
> of framework ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000 on agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0 at [email protected]:46083 (core-dev)
> I1005 16:30:02.752328 19444 cgroups.cpp:1258] Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos/fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> after 242176ns
> I1005 16:30:02.753576 19455 hierarchical.cpp:359] Removed framework
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-0000
> I1005 16:30:02.866210 19446 containerizer.cpp:2961] Container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf has
> exited
> I1005 16:30:02.874135 19455 containerizer.cpp:2747] Checkpointing termination
> state to nested container's runtime directory
> '/tmp/CgroupsIsolatorTest_ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems_w2rIwm/containers/1c7a71c2-bbda-4225-93fa-942f86d74dac/containers/fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf/termination'
> I1005 16:30:02.876006 19445 linux_launcher.cpp:580] Asked to destroy
> container 1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:30:02.876114 19445 linux_launcher.cpp:622] Destroying cgroup
> '/sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac'
> I1005 16:30:02.876654 19445 cgroups.cpp:2838] Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:30:02.876652 19432 cgroups.cpp:2838] Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos
> I1005 16:30:02.877028 19427 cgroups.cpp:1229] Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac
> after 306944ns
> I1005 16:30:02.877221 19433 cgroups.cpp:1229] Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos
> after 247040ns
> I1005 16:30:02.877992 19427 cgroups.cpp:2856] Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos
> I1005 16:30:02.878149 19435 cgroups.cpp:2856] Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac
> I1005 16:30:02.878576 19457 cgroups.cpp:1258] Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac
> after 347904ns
> I1005 16:30:02.967175 19461 containerizer.cpp:2961] Container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac has exited
> I1005 16:30:02.979094 19425 cgroups.cpp:1258] Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1/1c7a71c2-bbda-4225-93fa-942f86d74dac/mesos
> after 101.027072ms
> W1005 16:30:03.004833 19456 composing.cpp:609] Attempted to destroy unknown
> container
> 1c7a71c2-bbda-4225-93fa-942f86d74dac.fc2f6bfc-6cb6-437a-841e-88cb1a50dbaf
> I1005 16:30:03.017467 19406 master.cpp:1093] Master terminating
> I1005 16:30:03.018131 19452 hierarchical.cpp:645] Removed agent
> ea0b05e9-7ba5-4dd4-b1e4-293e69cdfd58-S0
> I1005 16:30:03.084359 19422 cgroups.cpp:2838] Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1
> I1005 16:30:03.084686 19449 cgroups.cpp:1229] Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1 after
> 253952ns
> I1005 16:30:03.085131 19461 cgroups.cpp:2856] Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1
> I1005 16:30:03.085453 19437 cgroups.cpp:1258] Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_a0422359-e2a8-4897-82b9-201252686cc1 after
> 271104ns
> [ FAILED ]
> CgroupsIsolatorTest.ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems (6926
> ms)
> [----------] 1 test from CgroupsIsolatorTest (6929 ms total)
>
> [----------] Global test environment tear-down
> [==========] 1 test from 1 test case ran. (6994 ms total)
> [ PASSED ] 0 tests.
> [ FAILED ] 1 test, listed below:
> [ FAILED ]
> CgroupsIsolatorTest.ROOT_CGROUPS_AgentRecoveryWithNewCgroupSubsystems
> ```
>
>
> Thanks,
>
> Gilbert Song
>
>
