[
https://issues.apache.org/jira/browse/MESOS-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun-Hung Hsiao reassigned MESOS-6004:
--------------------------------------
Assignee: Chun-Hung Hsiao
> Tasks fail when provisioning multiple containers with large docker images
> using copy backend
> ---------------------------------------------------------------------------------------------
>
> Key: MESOS-6004
> URL: https://issues.apache.org/jira/browse/MESOS-6004
> Project: Mesos
> Issue Type: Bug
> Components: containerization, docker
> Affects Versions: 0.28.2, 1.0.0
> Environment: h4. Agent Platform
> - Ubuntu 16.04
> - AWS g2.x2large instance
> - Nvidia support enabled
> h4. Agent Configuration
> -{noformat}
> --containerizers=mesos,docker
> --docker_config=<docker auth json>
> --docker_store_dir=/mnt/mesos/store/docker
> --executor_registration_timeout=3mins
> --hostname=<aws public dns>
> --image_providers=docker
> --image_provisioner_backend=copy
> --isolation=filesystem/linux,docker/runtime,cgroups/devices,gpu/nvidia
> --switch_user=false
> --work_dir=/mnt/mesos
> {noformat}
> h4. Framework
> - custom framework written in python
> - using unified containerizer with docker images
> h4. Test Setup
> * 1 master
> * 1 agent
> * 5 tasks scheduled at the same time:
> ** resources: cpus: 0.1, mem: 128
> ** command: `echo test`
> ** docker image: custom docker image, based on nvidia/cuda ~5gb
> ** the same docker image was for all tasks, already pulled.
> Reporter: Michael Thomas
> Assignee: Chun-Hung Hsiao
> Labels: containerizer, docker, performance
>
> When scheduling more than one task on the same agent, all tasks fail a as
> containers seem to be destroyed during provisioning.
> Specifically, the errors on the agent logs are:
> {noformat}
> E0808 15:53:09.691315 30996 slave.cpp:3976] Container
> 'eb20f642-bb90-4293-8eec-6f1576ccaeb1' for executor '3' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0001 failed to start: Container is being
> destroyed during provisioning
> {noformat}
> and
> {noformat}
> I0808 15:52:32.510210 30999 slave.cpp:4539] Terminating executor ''2' of
> framework c9852a23-bc07-422d-8d69-23c167a1924d-0001' because it did not
> register within 3mins
> {noformat}
> As the default provisioning method {{copy}} is being used, I assume this is
> due to the provisioning of multiple containers taking too long and the agent
> will not wait. For large images, this method is simply not performant.
> The issue did not occur, when only one tasks was scheduled.
> Increasing the {{executor_registration_timeout}} parameter, seemed to help a
> bit as it allowed to schedule at least 2 tasks at the same time. But still
> fails with more (5 in this case)
> h4. Complete logs
> (with GLOG_v=1)
> {noformat}
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.800375
> 3738 slave.cpp:198] Agent started on 1)@172.31.23.17:5051
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.800403
> 3738 slave.cpp:199] Flags at startup:
> --appc_simple_discovery_uri_prefix="http://";
> --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticatee="crammd5"
> --authentication_backoff_factor="1secs" --authorizer="local"
> --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
> --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos,docker" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker" --docker_config="XXX"
> --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io";
> --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock"
> --docker_stop_timeout="0ns" --docker_store_dir="/mnt/t" --docker_volume_checkp
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]:
> oint_dir="/var/run/mesos/isolators/docker/volume"
> --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins"
> --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false"
> --hostname="ec2-52-59-113-0.eu-central-1.compute.amazonaws.com"
> --hostname_lookup="true" --http_authenticators="basic"
> --http_command_executor="false" --image_providers="docker"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="filesystem/linux,docker/runtime,cgroups/devices,gpu/nvidia"
> --launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos"
> --logbufsecs="0" --logging_level="INFO"
> --master="zk://172.31.19.240:2181/mesos"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --re
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: covery_timeout="15mins"
> --registration_backoff_factor="1secs" --revocable_cpu_low_priority="true"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="false" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/mnt/mesos"
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: 2016-08-09
> 10:11:41,800:3738(0x7f71172a1700):ZOO_INFO@check_events@1728: initiated
> connection to server [172.31.19.240:2181]
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.801007
> 3738 slave.cpp:519] Agent resources: gpus(*):1; cpus(*):8; mem(*):14014;
> disk(*):60257; ports(*):[31000-32000]
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.801059
> 3738 slave.cpp:527] Agent attributes: [ ]
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.801077
> 3738 slave.cpp:532] Agent hostname:
> ec2-52-59-113-0.eu-central-1.compute.amazonaws.com
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.802232
> 3772 state.cpp:57] Recovering state from '/mnt/mesos/meta'
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.803539
> 3774 status_update_manager.cpp:200] Recovering status update manager
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.803745
> 3772 containerizer.cpp:522] Recovering containerizer
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.803788
> 3770 docker.cpp:775] Recovering Docker containers
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.808550
> 3777 metadata_manager.cpp:251] Successfully loaded 2 Docker images
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.808688
> 3771 provisioner.cpp:253] Provisioner recovery complete
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: 2016-08-09
> 10:11:41,816:3738(0x7f71172a1700):ZOO_INFO@check_events@1775: session
> establishment complete on server [172.31.19.240:2181],
> sessionId=0x1566a66ab9b0010, negotiated timeout=10000
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.817003
> 3774 group.cpp:349] Group process (group(1)@172.31.23.17:5051) connected to
> ZooKeeper
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.817116
> 3774 group.cpp:837] Syncing group operations: queue size (joins, cancels,
> datas) = (0, 0, 0)
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.817147
> 3774 group.cpp:427] Trying to create path '/mesos' in ZooKeeper
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.818857
> 3774 detector.cpp:152] Detected a new leader: (id='64')
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.819005
> 3776 group.cpp:706] Trying to get '/mesos/json.info_0000000064' in ZooKeeper
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.819855
> 3775 zookeeper.cpp:259] A new leading master ([email protected]:5050)
> is detected
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.899876
> 3777 slave.cpp:4782] Finished recovery
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900573
> 3777 slave.cpp:4815] Garbage collecting old agent
> 524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S2
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900687
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S2' for gc
> 6.99998957623704days in the future
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900729
> 3777 slave.cpp:4815] Garbage collecting old agent
> c9852a23-bc07-422d-8d69-23c167a1924d-S0
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900779
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/meta/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S2' for gc
> 6.99998957520889days in the future
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900817
> 3777 slave.cpp:4815] Garbage collecting old agent
> 524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S1
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900825
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S0' for gc
> 6.99998957451259days in the future
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900899
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/meta/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S0' for gc
> 6.99998957412148days in the future
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900933
> 3777 slave.cpp:4815] Garbage collecting old agent
> 524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S3
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.900944
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S1' for gc
> 6.99998957332148days in the future
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.901010
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/meta/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S1' for gc
> 6.99998957291556days in the future
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.901051
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S3' for gc
> 6.99998957196741days in the future
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.901101
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/meta/slaves/524105e7-de7a-43f0-8b28-d3ff3e0c4a44-S3' for gc
> 6.99998957156148days in the future
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.901201
> 3775 status_update_manager.cpp:174] Pausing sending status updates
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.901203
> 3777 slave.cpp:895] New master detected at [email protected]:5050
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.901278
> 3777 slave.cpp:916] No credentials provided. Attempting to register without
> authentication
> Aug 9 10:11:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:41.901317
> 3777 slave.cpp:927] Detecting new master
> Aug 9 10:11:42 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:42.267201
> 3774 slave.cpp:1197] Re-registered with master [email protected]:5050
> Aug 9 10:11:42 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:42.267302
> 3771 status_update_manager.cpp:181] Resuming sending status updates
> Aug 9 10:11:42 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:42.267302
> 3774 slave.cpp:1233] Forwarding total oversubscribed resources
> Aug 9 10:11:42 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:42.267577
> 3774 slave.cpp:2526] Updated checkpointed resources from to
> Aug 9 10:11:48 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:48.269104
> 3777 slave.cpp:2218] Asked to shut down framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0002 by [email protected]:5050
> Aug 9 10:11:48 ip-172-31-23-17 mesos-slave[3738]: W0809 10:11:48.269186
> 3777 slave.cpp:2233] Cannot shut down unknown framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0002
> Aug 9 10:11:54 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:54.969521
> 3772 slave.cpp:1495] Got assigned task 0 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:11:54 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:54.970026
> 3772 slave.cpp:1614] Launching task 0 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:11:54 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:54.970826
> 3772 slave.cpp:5674] Launching executor 0 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 with resources cpus(*):0.1;
> mem(*):32 in work directory
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/0/runs/09a77a80-f904-4581-966a-734d24ae96f3'
> Aug 9 10:11:54 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:54.971051
> 3772 slave.cpp:1840] Queuing task '0' for executor '0' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:11:54 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:54.971212
> 3773 containerizer.cpp:781] Starting container
> '09a77a80-f904-4581-966a-734d24ae96f3' for executor '0' of framework
> 'c9852a23-bc07-422d-8d69-23c167a1924d-0003'
> Aug 9 10:11:54 ip-172-31-23-17 mesos-slave[3738]: I0809 10:11:54.973953
> 3774 provisioner.cpp:294] Provisioning image rootfs
> '/mnt/mesos/provisioner/containers/09a77a80-f904-4581-966a-734d24ae96f3/backends/copy/rootfses/ec9bb10e-6726-4872-be2b-4a72dfc0fafc'
> for container 09a77a80-f904-4581-966a-734d24ae96f3
> Aug 9 10:12:00 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:00.978344
> 3774 slave.cpp:1495] Got assigned task 1 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:00 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:00.978687
> 3774 slave.cpp:1614] Launching task 1 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:00 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:00.978976
> 3774 slave.cpp:5674] Launching executor 1 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 with resources cpus(*):0.1;
> mem(*):32 in work directory
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/1/runs/0cea1dc7-b523-4b68-851a-531ed101de5a'
> Aug 9 10:12:00 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:00.979173
> 3774 slave.cpp:1840] Queuing task '1' for executor '1' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:00 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:00.979387
> 3777 containerizer.cpp:781] Starting container
> '0cea1dc7-b523-4b68-851a-531ed101de5a' for executor '1' of framework
> 'c9852a23-bc07-422d-8d69-23c167a1924d-0003'
> Aug 9 10:12:00 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:00.981374
> 3776 provisioner.cpp:294] Provisioning image rootfs
> '/mnt/mesos/provisioner/containers/0cea1dc7-b523-4b68-851a-531ed101de5a/backends/copy/rootfses/e3af89fe-c889-45fc-929b-f008b18e5d4e'
> for container 0cea1dc7-b523-4b68-851a-531ed101de5a
> Aug 9 10:12:06 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:06.983610
> 3776 slave.cpp:1495] Got assigned task 2 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:06 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:06.983947
> 3776 slave.cpp:1614] Launching task 2 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:06 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:06.984236
> 3776 slave.cpp:5674] Launching executor 2 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 with resources cpus(*):0.1;
> mem(*):32 in work directory
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/2/runs/a4c972da-b12d-4b4b-8481-79ca320ddab7'
> Aug 9 10:12:06 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:06.984450
> 3776 slave.cpp:1840] Queuing task '2' for executor '2' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:06 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:06.984778
> 3777 containerizer.cpp:781] Starting container
> 'a4c972da-b12d-4b4b-8481-79ca320ddab7' for executor '2' of framework
> 'c9852a23-bc07-422d-8d69-23c167a1924d-0003'
> Aug 9 10:12:06 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:06.986330
> 3776 provisioner.cpp:294] Provisioning image rootfs
> '/mnt/mesos/provisioner/containers/a4c972da-b12d-4b4b-8481-79ca320ddab7/backends/copy/rootfses/feaf70a4-df43-45c9-b6be-a654c9210a54'
> for container a4c972da-b12d-4b4b-8481-79ca320ddab7
> Aug 9 10:12:12 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:12.996104
> 3771 slave.cpp:1495] Got assigned task 3 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:12 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:12.996462
> 3771 slave.cpp:1614] Launching task 3 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:12 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:12.997241
> 3771 slave.cpp:5674] Launching executor 3 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 with resources cpus(*):0.1;
> mem(*):32 in work directory
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/3/runs/d9363f0d-645b-468f-947a-8bf97d15a630'
> Aug 9 10:12:12 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:12.997406
> 3771 slave.cpp:1840] Queuing task '3' for executor '3' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:12 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:12.997640
> 3777 containerizer.cpp:781] Starting container
> 'd9363f0d-645b-468f-947a-8bf97d15a630' for executor '3' of framework
> 'c9852a23-bc07-422d-8d69-23c167a1924d-0003'
> Aug 9 10:12:12 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:12.999593
> 3777 provisioner.cpp:294] Provisioning image rootfs
> '/mnt/mesos/provisioner/containers/d9363f0d-645b-468f-947a-8bf97d15a630/backends/copy/rootfses/eeb1b066-94b8-4880-a696-090f830cf2d6'
> for container d9363f0d-645b-468f-947a-8bf97d15a630
> Aug 9 10:12:19 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:19.003924
> 3775 slave.cpp:1495] Got assigned task 4 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:19 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:19.004278
> 3775 slave.cpp:1614] Launching task 4 for framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:19 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:19.005383
> 3775 slave.cpp:5674] Launching executor 4 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 with resources cpus(*):0.1;
> mem(*):32 in work directory
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/4/runs/58c398e2-38f9-4389-87ba-8696c9a523eb'
> Aug 9 10:12:19 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:19.005681
> 3775 slave.cpp:1840] Queuing task '4' for executor '4' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:12:19 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:19.005971
> 3772 containerizer.cpp:781] Starting container
> '58c398e2-38f9-4389-87ba-8696c9a523eb' for executor '4' of framework
> 'c9852a23-bc07-422d-8d69-23c167a1924d-0003'
> Aug 9 10:12:19 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:19.007786
> 3777 provisioner.cpp:294] Provisioning image rootfs
> '/mnt/mesos/provisioner/containers/58c398e2-38f9-4389-87ba-8696c9a523eb/backends/copy/rootfses/78b7bf3a-6f3b-4f8a-b4fa-98a6ea4bf8f5'
> for container 58c398e2-38f9-4389-87ba-8696c9a523eb
> Aug 9 10:12:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:41.801560
> 3777 slave.cpp:4591] Current disk usage 43.28%. Max allowed age:
> 3.270091966898553days
> Aug 9 10:12:54 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:54.972057
> 3772 slave.cpp:4539] Terminating executor ''0' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003' because it did not register within
> 1mins
> Aug 9 10:12:54 ip-172-31-23-17 mesos-slave[3738]: I0809 10:12:54.972322
> 3775 containerizer.cpp:1622] Destroying container
> '09a77a80-f904-4581-966a-734d24ae96f3'
> Aug 9 10:13:00 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:00.979907
> 3777 slave.cpp:4539] Terminating executor ''1' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003' because it did not register within
> 1mins
> Aug 9 10:13:00 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:00.980058
> 3775 containerizer.cpp:1622] Destroying container
> '0cea1dc7-b523-4b68-851a-531ed101de5a'
> Aug 9 10:13:06 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:06.985003
> 3777 slave.cpp:4539] Terminating executor ''2' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003' because it did not register within
> 1mins
> Aug 9 10:13:06 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:06.985255
> 3774 containerizer.cpp:1622] Destroying container
> 'a4c972da-b12d-4b4b-8481-79ca320ddab7'
> Aug 9 10:13:12 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:12.998438
> 3777 slave.cpp:4539] Terminating executor ''3' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003' because it did not register within
> 1mins
> Aug 9 10:13:12 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:12.998641
> 3776 containerizer.cpp:1622] Destroying container
> 'd9363f0d-645b-468f-947a-8bf97d15a630'
> Aug 9 10:13:19 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:19.006058
> 3773 slave.cpp:4539] Terminating executor ''4' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003' because it did not register within
> 1mins
> Aug 9 10:13:19 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:19.006228
> 3773 containerizer.cpp:1622] Destroying container
> '58c398e2-38f9-4389-87ba-8696c9a523eb'
> Aug 9 10:13:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:13:41.802209
> 3775 slave.cpp:4591] Current disk usage 57.59%. Max allowed age:
> 2.269006805055764days
> Aug 9 10:14:41 ip-172-31-23-17 mesos-slave[3738]: I0809 10:14:41.803022
> 3776 slave.cpp:4591] Current disk usage 68.70%. Max allowed age:
> 1.491204118572176days
> Aug 9 10:15:01 ip-172-31-23-17 mesos-slave[3738]: E0809 10:15:01.516207
> 3775 slave.cpp:3976] Container '58c398e2-38f9-4389-87ba-8696c9a523eb' for
> executor '4' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0003 failed to
> start: Container is being destroyed during provisioning
> Aug 9 10:15:01 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:01.518277
> 3772 composing.cpp:541] Container '58c398e2-38f9-4389-87ba-8696c9a523eb' is
> already destroyed
> Aug 9 10:15:02 ip-172-31-23-17 mesos-slave[3738]: E0809 10:15:02.291764
> 3775 slave.cpp:3976] Container '09a77a80-f904-4581-966a-734d24ae96f3' for
> executor '0' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0003 failed to
> start: Container is being destroyed during provisioning
> Aug 9 10:15:02 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:02.291877
> 3775 composing.cpp:541] Container '09a77a80-f904-4581-966a-734d24ae96f3' is
> already destroyed
> Aug 9 10:15:03 ip-172-31-23-17 mesos-slave[3738]: E0809 10:15:03.022406
> 3774 slave.cpp:3976] Container 'd9363f0d-645b-468f-947a-8bf97d15a630' for
> executor '3' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0003 failed to
> start: Container is being destroyed during provisioning
> Aug 9 10:15:03 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:03.022595
> 3771 composing.cpp:541] Container 'd9363f0d-645b-468f-947a-8bf97d15a630' is
> already destroyed
> Aug 9 10:15:03 ip-172-31-23-17 mesos-slave[3738]: E0809 10:15:03.461519
> 3775 slave.cpp:3976] Container '0cea1dc7-b523-4b68-851a-531ed101de5a' for
> executor '1' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0003 failed to
> start: Container is being destroyed during provisioning
> Aug 9 10:15:03 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:03.461643
> 3774 composing.cpp:541] Container '0cea1dc7-b523-4b68-851a-531ed101de5a' is
> already destroyed
> Aug 9 10:15:04 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:04.190901
> 3776 provisioner.cpp:434] Destroying container rootfs at
> '/mnt/mesos/provisioner/containers/58c398e2-38f9-4389-87ba-8696c9a523eb/backends/copy/rootfses/78b7bf3a-6f3b-4f8a-b4fa-98a6ea4bf8f5'
> for container 58c398e2-38f9-4389-87ba-8696c9a523eb
> Aug 9 10:15:04 ip-172-31-23-17 mesos-slave[3738]: E0809 10:15:04.190904
> 3777 slave.cpp:3976] Container 'a4c972da-b12d-4b4b-8481-79ca320ddab7' for
> executor '2' of framework c9852a23-bc07-422d-8d69-23c167a1924d-0003 failed to
> start: Container is being destroyed during provisioning
> Aug 9 10:15:04 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:04.191128
> 3770 composing.cpp:541] Container 'a4c972da-b12d-4b4b-8481-79ca320ddab7' is
> already destroyed
> Aug 9 10:15:04 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:04.197221
> 3776 provisioner.cpp:434] Destroying container rootfs at
> '/mnt/mesos/provisioner/containers/09a77a80-f904-4581-966a-734d24ae96f3/backends/copy/rootfses/ec9bb10e-6726-4872-be2b-4a72dfc0fafc'
> for container 09a77a80-f904-4581-966a-734d24ae96f3
> Aug 9 10:15:04 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:04.197410
> 3776 provisioner.cpp:434] Destroying container rootfs at
> '/mnt/mesos/provisioner/containers/d9363f0d-645b-468f-947a-8bf97d15a630/backends/copy/rootfses/eeb1b066-94b8-4880-a696-090f830cf2d6'
> for container d9363f0d-645b-468f-947a-8bf97d15a630
> Aug 9 10:15:04 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:04.197510
> 3776 provisioner.cpp:434] Destroying container rootfs at
> '/mnt/mesos/provisioner/containers/0cea1dc7-b523-4b68-851a-531ed101de5a/backends/copy/rootfses/e3af89fe-c889-45fc-929b-f008b18e5d4e'
> for container 0cea1dc7-b523-4b68-851a-531ed101de5a
> Aug 9 10:15:04 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:04.197619
> 3776 provisioner.cpp:434] Destroying container rootfs at
> '/mnt/mesos/provisioner/containers/a4c972da-b12d-4b4b-8481-79ca320ddab7/backends/copy/rootfses/feaf70a4-df43-45c9-b6be-a654c9210a54'
> for container a4c972da-b12d-4b4b-8481-79ca320ddab7
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.390928
> 3776 slave.cpp:4082] Executor '3' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 has terminated with unknown status
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.395370
> 3776 slave.cpp:3211] Handling status update TASK_FAILED (UUID:
> 543e1e1e-ae47-48ab-ad64-ee82fdd27418) for task 3 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 from @0.0.0.0:0
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.402472
> 3776 slave.cpp:4082] Executor '2' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 has terminated with unknown status
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.402556
> 3776 slave.cpp:3211] Handling status update TASK_FAILED (UUID:
> 1a54664d-4157-42a3-a97f-7189c137a769) for task 2 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 from @0.0.0.0:0
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:07.406177
> 3772 containerizer.cpp:1451] Ignoring update for unknown container:
> d9363f0d-645b-468f-947a-8bf97d15a630
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.407325
> 3777 status_update_manager.cpp:320] Received status update TASK_FAILED (UUID:
> 543e1e1e-ae47-48ab-ad64-ee82fdd27418) for task 3 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:07.407598
> 3773 containerizer.cpp:1451] Ignoring update for unknown container:
> a4c972da-b12d-4b4b-8481-79ca320ddab7
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.409890
> 3774 slave.cpp:3604] Forwarding the update TASK_FAILED (UUID:
> 543e1e1e-ae47-48ab-ad64-ee82fdd27418) for task 3 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 to [email protected]:5050
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.409893
> 3777 status_update_manager.cpp:320] Received status update TASK_FAILED (UUID:
> 1a54664d-4157-42a3-a97f-7189c137a769) for task 2 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.414727
> 3774 slave.cpp:3604] Forwarding the update TASK_FAILED (UUID:
> 1a54664d-4157-42a3-a97f-7189c137a769) for task 2 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 to [email protected]:5050
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.418180
> 3777 status_update_manager.cpp:392] Received status update acknowledgement
> (UUID: 543e1e1e-ae47-48ab-ad64-ee82fdd27418) for task 3 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.419615
> 3777 status_update_manager.cpp:392] Received status update acknowledgement
> (UUID: 1a54664d-4157-42a3-a97f-7189c137a769) for task 2 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.419636
> 3774 slave.cpp:4193] Cleaning up executor '3' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.419924
> 3772 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/3/runs/d9363f0d-645b-468f-947a-8bf97d15a630'
> for gc 6.9999951410637days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.420703
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/3'
> for gc 6.99999513154074days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.420722
> 3774 slave.cpp:4193] Cleaning up executor '2' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.420897
> 3774 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/2/runs/a4c972da-b12d-4b4b-8481-79ca320ddab7'
> for gc 6.99999512931852days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.420940
> 3774 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/2'
> for gc 6.99999512869333days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.490155
> 3772 slave.cpp:4082] Executor '4' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 has terminated with unknown status
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.490289
> 3772 slave.cpp:3211] Handling status update TASK_FAILED (UUID:
> 9c5528ec-0725-42f2-a2e2-465d3c73b8d5) for task 4 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 from @0.0.0.0:0
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.490414
> 3772 slave.cpp:4082] Executor '0' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 has terminated with unknown status
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.490489
> 3772 slave.cpp:3211] Handling status update TASK_FAILED (UUID:
> 3f801210-b029-427f-af3f-e51b1dfa5cc8) for task 0 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 from @0.0.0.0:0
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:07.490687
> 3773 containerizer.cpp:1451] Ignoring update for unknown container:
> 58c398e2-38f9-4389-87ba-8696c9a523eb
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:07.490774
> 3773 containerizer.cpp:1451] Ignoring update for unknown container:
> 09a77a80-f904-4581-966a-734d24ae96f3
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.490867
> 3773 status_update_manager.cpp:320] Received status update TASK_FAILED (UUID:
> 9c5528ec-0725-42f2-a2e2-465d3c73b8d5) for task 4 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.491082
> 3773 status_update_manager.cpp:320] Received status update TASK_FAILED (UUID:
> 3f801210-b029-427f-af3f-e51b1dfa5cc8) for task 0 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.491082
> 3772 slave.cpp:3604] Forwarding the update TASK_FAILED (UUID:
> 9c5528ec-0725-42f2-a2e2-465d3c73b8d5) for task 4 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 to [email protected]:5050
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.491303
> 3774 slave.cpp:3604] Forwarding the update TASK_FAILED (UUID:
> 3f801210-b029-427f-af3f-e51b1dfa5cc8) for task 0 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 to [email protected]:5050
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.494438
> 3775 status_update_manager.cpp:392] Received status update acknowledgement
> (UUID: 9c5528ec-0725-42f2-a2e2-465d3c73b8d5) for task 4 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.494572
> 3775 slave.cpp:4193] Cleaning up executor '4' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.494757
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/4/runs/58c398e2-38f9-4389-87ba-8696c9a523eb'
> for gc 6.99999427436444days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.494817
> 3775 status_update_manager.cpp:392] Received status update acknowledgement
> (UUID: 3f801210-b029-427f-af3f-e51b1dfa5cc8) for task 0 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.494937
> 3775 slave.cpp:4193] Cleaning up executor '0' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.494823
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/4'
> for gc 6.99999427361185days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.495107
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/0/runs/09a77a80-f904-4581-966a-734d24ae96f3'
> for gc 6.99999427055407days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.495149
> 3773 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/0'
> for gc 6.99999427024296days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.591338
> 3775 slave.cpp:4082] Executor '1' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 has terminated with unknown status
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.591469
> 3775 slave.cpp:3211] Handling status update TASK_FAILED (UUID:
> 85b7c18a-c59c-4449-827c-a3bfaa2a7de8) for task 1 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 from @0.0.0.0:0
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: W0809 10:15:07.591841
> 3776 containerizer.cpp:1451] Ignoring update for unknown container:
> 0cea1dc7-b523-4b68-851a-531ed101de5a
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.591974
> 3770 status_update_manager.cpp:320] Received status update TASK_FAILED (UUID:
> 85b7c18a-c59c-4449-827c-a3bfaa2a7de8) for task 1 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.592157
> 3772 slave.cpp:3604] Forwarding the update TASK_FAILED (UUID:
> 85b7c18a-c59c-4449-827c-a3bfaa2a7de8) for task 1 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003 to [email protected]:5050
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.594518
> 3771 status_update_manager.cpp:392] Received status update acknowledgement
> (UUID: 85b7c18a-c59c-4449-827c-a3bfaa2a7de8) for task 1 of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.594636
> 3771 slave.cpp:4193] Cleaning up executor '1' of framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.594792
> 3770 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/1/runs/0cea1dc7-b523-4b68-851a-531ed101de5a'
> for gc 6.99999311635556days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.594820
> 3771 slave.cpp:4281] Cleaning up framework
> c9852a23-bc07-422d-8d69-23c167a1924d-0003
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.594861
> 3770 gc.cpp:55] Scheduling
> '/mnt/mesos/slaves/c9852a23-bc07-422d-8d69-23c167a1924d-S1/frameworks/c9852a23-bc07-422d-8d69-23c167a1924d-0003/executors/1'
> for gc 6.99999311568296days in the future
> Aug 9 10:15:07 ip-172-31-23-17 mesos-slave[3738]: I0809 10:15:07.594897
> 3770 status_update_manager.cpp:282] Closing status update streams for
> framework c9852a23-bc07-422d-8d69-23c167a1924d-0003
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
