[
https://issues.apache.org/jira/browse/MESOS-9142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jie Yu reassigned MESOS-9142:
-----------------------------
Assignee: Jie Yu
> CNI detach might fail due to missing network config file.
> ---------------------------------------------------------
>
> Key: MESOS-9142
> URL: https://issues.apache.org/jira/browse/MESOS-9142
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 1.4.1, 1.5.1, 1.6.1, 1.7.0
> Reporter: Jie Yu
> Assignee: Jie Yu
> Priority: Major
>
> Observed this in one of the scale testing:
> If the container is destroyed while in PREPARING state, the network config
> file won't be there. The CNI detach in that case should return success
> because attach is not called yet.
> {code}
> Aug 08 23:52:03 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:52:03.762208 4045 slave.cpp:3562] Launching container
> 278d88a8-2bd1-4359-8c70-25dc90879cdc for executor
> 'testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a' of framework
> 4397555f-b8fd-478c-907a-7532e2f5e32e-0001
> Aug 08 23:53:03 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:53:03.232110 4047 containerizer.cpp:1211] Starting container
> 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:53:03 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:53:03.249105 4049 provisioner.cpp:546] Provisioning image rootfs
> '/var/lib/mesos/slave/provisioner/containers/278d88a8-2bd1-4359-8c70-25dc90879cdc/backends/overlay/rootfses/25dc9ff3-a4ad-4b59-bdf7-02f7e741c729'
> for container 278d88a8-2bd1-4359-8c70-25dc90879cdc using overlay backend
> Aug 08 23:53:31 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:53:31.367316 4047 containerizer.cpp:2981] Transitioning the state
> of container 278d88a8-2bd1-4359-8c70-25dc90879cdc from PROVISIONING to
> PREPARING
> Aug 08 23:55:22 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:55:22.988091 4047 memory.cpp:479] Started listening for OOM events
> for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:55:23 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:55:23.196844 4047 memory.cpp:591] Started listening on 'low' memory
> pressure events for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:55:23 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:55:23.277029 4047 memory.cpp:591] Started listening on 'medium'
> memory pressure events for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:55:23 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:55:23.407688 4047 memory.cpp:591] Started listening on 'critical'
> memory pressure events for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:57:31 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:57:31.760859 4043 memory.cpp:199] Updated
> 'memory.soft_limit_in_bytes' to 64MB for container
> 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:57:31 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:57:31.878435 4043 memory.cpp:228] Updated 'memory.limit_in_bytes'
> to 64MB for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:57:35 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:57:35.892071 4044 cpu.cpp:101] Updated 'cpu.shares' to 112 (cpus
> 0.11) for container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 08 23:57:36 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0808 23:57:36.132098 4044 cpu.cpp:121] Updated 'cpu.cfs_period_us' to 100ms
> and 'cpu.cfs_quota_us' to 11ms (cpus 0.11) for container
> 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:01.791821 4047 isolator_module.cpp:68] Container prepare:
> container_id[value: "278d88a8-2bd1-4359-8c70-25dc90879cdc"]
> container_config[directory:
> "/var/lib/mesos/slave/slaves/4397555f-b8fd-478c-907a-7532e2f5e32e-S0/frameworks/4397555f-b8fd-478c-907a-7532e2f5e32e-0001/executors/testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a/runs/278d88a8-2bd1-4359-8c70-25dc90879cdc"
> user: "root" rootfs:
> "/var/lib/mesos/slave/provisioner/containers/278d88a8-2bd1-4359-8c70-25dc90879cdc/backends/overlay/rootfses/25dc9ff3-a4ad-4b59-bdf7-02f7e741c729"
> docker { manifest { id:
> "d7066268059eb2499922833bd058e9e0b9da8365bcae4b85ee22296b02fa0489" parent:
> "f18ee96f0b1656cab52554b270f19e8df5046d307296d2146539c04565d67747" created:
> "2018-07-06T14:14:06.393355914Z" container:
> "b449414436d7bc63cc85dcd24abde14bcec1d92f54f2316212059ee2ed7f3a65"
> container_config { Hostname: "b449414436d7" Env:
> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" User: ""
> Cmd: "/bin/sh" Cmd: "-c" Cmd: "#(nop) " Cmd: "CMD [\"/bin/sh\"]" WorkingDir:
> "" Image:
> "sha256:9025f4d6338c3425933e4734e8a27fb615d56ca8862e78bb8c4e2426f2db78bd" }
> docker_version: "17.06.2-ce" config { Hostname: "" Env:
> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" User: ""
> Cmd: "/bin/sh" WorkingDir: "" Image:
> "sha256:9025f4d6338c3425933e4734e8a27fb615d56ca8862e78bb8c4e2426f2db78bd" }
> architecture: "amd64" os: "linux" } } executor_info { executor_id { value:
> "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } resources { name: "cpus"
> type: SCALAR scalar { value: 0.11 } allocation_info { role: "slave_public" }
> } resources { name: "mem" type: SCALAR scalar { value: 64 } allocation_info {
> role: "slave_public" } } resources { name: "ports" type: RANGES ranges {
> range { begin: 19471 end: 19471 } } allocation_info { role: "slave_public" }
> } command { environment { variables { name: "MARATHON_APP_VERSION" value:
> "2018-08-08T23:45:27.052Z" } variables { name: "HOST" value: "10.0.3.181" }
> variables { name: "MARATHON_APP_RESOURCE_CPUS" value: "0.01" } variables {
> name: "MARATHON_APP_RESOURCE_GPUS" value: "0" } variables { name:
> "MESOS_TASK_ID" value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" }
> variables { name: "PORT" value: "19471" } variables { name:
> "MARATHON_APP_RESOURCE_MEM" value: "32.0" } variables { name: "PORTS" value:
> "19471" } variables { name: "MARATHON_APP_RESOURCE_DISK" value: "0.0" }
> variables { name: "PORT_80" value: "19471" } variables { name:
> "MARATHON_APP_LABELS" value: "" } variables { name: "MARATHON_APP_ID" value:
> "/testcni2" } variables { name: "PORT0" value: "19471" } } value:
> "/opt/mesosphere/packages/mesos--39041edf914496bb882949ade1e78043d53dadd0/libexec/mesos/mesos-executor"
> shell: false arguments: "mesos-executor" arguments:
> "--launcher_dir=/opt/mesosphere/active/mesos/libexec/mesos" arguments:
> "--sandbox_directory=/mnt/mesos/sandbox" arguments: "--user=root" }
> framework_id { value: "4397555f-b8fd-478c-907a-7532e2f5e32e-0001" } name:
> "Command Executor (Task: testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a)
> (Command: sh -c \'sleep 10000\')" source:
> "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" container { type: MESOS mesos
> { image { type: DOCKER docker { name: "alpine" } cached: true } }
> network_infos { labels { labels { key: "DCOS_SPACE" value: "/testcni2" } }
> ip_addresses { } name: "mesos-bridge" port_mappings { host_port: 19471
> container_port: 80 protocol: "tcp" } } } discovery { visibility: FRAMEWORK
> name: "testcni2" ports { ports { number: 19471 protocol: "tcp" labels {
> labels { key: "network-scope" value: "host" } } } } } labels { labels { key:
> "DCOS_SPACE" value: "/testcni2" } } } task_info { name: "testcni2" task_id {
> value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } slave_id { value:
> "4397555f-b8fd-478c-907a-7532e2f5e32e-S0" } resources { name: "cpus" type:
> SCALAR scalar { value: 0.01 } allocation_info { role: "slave_public" } }
> resources { name: "mem" type: SCALAR scalar { value: 32 } allocation_info {
> role: "slave_public" } } resources { name: "ports" type: RANGES ranges {
> range { begin: 19471 end: 19471 } } allocation_info { role: "slave_public" }
> } command { environment { variables { name: "MARATHON_APP_VERSION" value:
> "2018-08-08T23:45:27.052Z" } variables { name: "HOST" value: "10.0.3.181" }
> variables { name: "MARATHON_APP_RESOURCE_CPUS" value: "0.01" } variables {
> name: "MARATHON_APP_RESOURCE_GPUS" value: "0" } variables { name:
> "MESOS_TASK_ID" value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" }
> variables { name: "PORT" value: "19471" } variables { name:
> "MARATHON_APP_RESOURCE_MEM" value: "32.0" } variables { name: "PORTS" value:
> "19471" } variables { name: "MARATHON_APP_RESOURCE_DISK" value: "0.0" }
> variables { name: "PORT_80" value: "19471" } variables { name:
> "MARATHON_APP_LABELS" value: "" } variables { name: "MARATHON_APP_ID" value:
> "/testcni2" } variables { name: "PORT0" value: "19471" } } value: "sleep
> 10000" } container { type: MESOS mesos { image { type: DOCKER docker { name:
> "alpine" } cached: true } } network_infos { labels { labels { key:
> "DCOS_SPACE" value: "/testcni2" } } ip_addresses { } name: "mesos-bridge"
> port_mappings { host_port: 19471 container_port: 80 protocol: "tcp" } } }
> labels { labels { key: "DCOS_SPACE" value: "/testcni2" } } discovery {
> visibility: FRAMEWORK name: "testcni2" ports { ports { number: 19471
> protocol: "tcp" labels { labels { key: "network-scope" value: "host" } } } }
> } } command_info { environment { variables { name: "MARATHON_APP_VERSION"
> value: "2018-08-08T23:45:27.052Z" } variables { name: "HOST" value:
> "10.0.3.181" } variables { name: "MARATHON_APP_RESOURCE_CPUS" value: "0.01" }
> variables { name: "MARATHON_APP_RESOURCE_GPUS" value: "0" } variables { name:
> "MESOS_TASK_ID" value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" }
> variables { name: "PORT" value: "19471" } variables { name:
> "MARATHON_APP_RESOURCE_MEM" value: "32.0" } variables { name: "PORTS" value:
> "19471" } variables { name: "MARATHON_APP_RESOURCE_DISK" value: "0.0" }
> variables { name: "PORT_80" value: "19471" } variables { name:
> "MARATHON_APP_LABELS" value: "" } variables { name: "MARATHON_APP_ID" value:
> "/testcni2" } variables { name: "PORT0" value: "19471" } } value:
> "/opt/mesosphere/packages/mesos--39041edf914496bb882949ade1e78043d53dadd0/libexec/mesos/mesos-executor"
> shell: false arguments: "mesos-executor" arguments:
> "--launcher_dir=/opt/mesosphere/active/mesos/libexec/mesos" arguments:
> "--sandbox_directory=/mnt/mesos/sandbox" arguments: "--user=root" }
> container_info { type: MESOS mesos { image { type: DOCKER docker { name:
> "alpine" } cached: true } } network_infos { labels { labels { key:
> "DCOS_SPACE" value: "/testcni2" } } ip_addresses { } name: "mesos-bridge"
> port_mappings { host_port: 19471 container_port: 80 protocol: "tcp" } } }
> resources { name: "cpus" type: SCALAR scalar { value: 0.11 } allocation_info
> { role: "slave_public" } } resources { name: "mem" type: SCALAR scalar {
> value: 64 } allocation_info { role: "slave_public" } } resources { name:
> "ports" type: RANGES ranges { range { begin: 19471 end: 19471 } }
> allocation_info { role: "slave_public" } }]
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:01.792201 4047 container_assigner.cpp:64] Registering and
> retrieving endpoint for container_id[value:
> "278d88a8-2bd1-4359-8c70-25dc90879cdc"] executor_info[executor_id { value:
> "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" } resources { name: "cpus"
> type: SCALAR scalar { value: 0.11 } allocation_info { role: "slave_public" }
> } resources { name: "mem" type: SCALAR scalar { value: 64 } allocation_info {
> role: "slave_public" } } resources { name: "ports" type: RANGES ranges {
> range { begin: 19471 end: 19471 } } allocation_info { role: "slave_public" }
> } command { environment { variables { name: "MARATHON_APP_VERSION" value:
> "2018-08-08T23:45:27.052Z" } variables { name: "HOST" value: "10.0.3.181" }
> variables { name: "MARATHON_APP_RESOURCE_CPUS" value: "0.01" } variables {
> name: "MARATHON_APP_RESOURCE_GPUS" value: "0" } variables { name:
> "MESOS_TASK_ID" value: "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" }
> variables { name: "PORT" value: "19471" } variables { name:
> "MARATHON_APP_RESOURCE_MEM" value: "32.0" } variables { name: "PORTS" value:
> "19471" } variables { name: "MARATHON_APP_RESOURCE_DISK" value: "0.0" }
> variables { name: "PORT_80" value: "19471" } variables { name:
> "MARATHON_APP_LABELS" value: "" } variables { name: "MARATHON_APP_ID" value:
> "/testcni2" } variables { name: "PORT0" value: "19471" } } value:
> "/opt/mesosphere/packages/mesos--39041edf914496bb882949ade1e78043d53dadd0/libexec/mesos/mesos-executor"
> shell: false arguments: "mesos-executor" arguments:
> "--launcher_dir=/opt/mesosphere/active/mesos/libexec/mesos" arguments:
> "--sandbox_directory=/mnt/mesos/sandbox" arguments: "--user=root" }
> framework_id { value: "4397555f-b8fd-478c-907a-7532e2f5e32e-0001" } name:
> "Command Executor (Task: testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a)
> (Command: sh -c \'sleep 10000\')" source:
> "testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a" container { type: MESOS mesos
> { image { type: DOCKER docker { name: "alpine" } cached: true } }
> network_infos { labels { labels { key: "DCOS_SPACE" value: "/testcni2" } }
> ip_addresses { } name: "mesos-bridge" port_mappings { host_port: 19471
> container_port: 80 protocol: "tcp" } } } discovery { visibility: FRAMEWORK
> name: "testcni2" ports { ports { number: 19471 protocol: "tcp" labels {
> labels { key: "network-scope" value: "host" } } } } } labels { labels { key:
> "DCOS_SPACE" value: "/testcni2" } }].
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:01.822054 4195 container_assigner_strategy.cpp:156] New
> ephemeral-port reader for container[value:
> "278d88a8-2bd1-4359-8c70-25dc90879cdc"] created at
> endpoint[198.51.100.1:47454].
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:01.822118 4195 container_state_cache_impl.cpp:134] Writing
> container
> file[/run/dcos/mesos/isolators/com_mesosphere_MetricsIsolatorModule/containers/278d88a8-2bd1-4359-8c70-25dc90879cdc]
> with endpoint[198.51.100.1:47454]
> Aug 09 00:02:01 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:01.828858 4045 secret.cpp:296] 0 secrets have been resolved for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 09 00:02:03 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:03.230397 4047 switchboard.cpp:317] Container logger module
> finished preparing container 278d88a8-2bd1-4359-8c70-25dc90879cdc;
> IOSwitchboard server is not required
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:04.510732 4045 containerizer.cpp:2367] Destroying container
> 278d88a8-2bd1-4359-8c70-25dc90879cdc in PREPARING state
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:04.511010 4045 containerizer.cpp:2981] Transitioning the state
> of container 278d88a8-2bd1-4359-8c70-25dc90879cdc from PREPARING to DESTROYING
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:04.745018 4042 container_assigner.cpp:101] Unregistering
> container_id[value: "278d88a8-2bd1-4359-8c70-25dc90879cdc"].
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:04.746515 4195 container_assigner_strategy.cpp:202] Closing
> ephemeral-port reader for container[value:
> "278d88a8-2bd1-4359-8c70-25dc90879cdc"] at endpoint[198.51.100.1:47454].
> Aug 09 00:02:04 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:02:04.749131 4195 container_state_cache_impl.cpp:145] Removing
> container
> file[/run/dcos/mesos/isolators/com_mesosphere_MetricsIsolatorModule/containers/278d88a8-2bd1-4359-8c70-25dc90879cdc]
> Aug 09 00:02:05 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> E0809 00:02:05.153556 4048 slave.cpp:6201] Container
> '278d88a8-2bd1-4359-8c70-25dc90879cdc' for executor
> 'testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a' of framework
> 4397555f-b8fd-478c-907a-7532e2f5e32e-0001 failed to start: Container is being
> destroyed during preparing
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> E0809 00:07:08.151012 4045 slave.cpp:6308] Termination of executor
> 'testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a' of framework
> 4397555f-b8fd-478c-907a-7532e2f5e32e-0001 failed: Failed to clean up an
> isolator when destroying container: Failed to parse CNI network configuration
> file:
> '/run/mesos/isolators/network/cni/278d88a8-2bd1-4359-8c70-25dc90879cdc/mesos-bridge/network.conf':
> Failed to read CNI network configuration file:
> '/run/mesos/isolators/network/cni/278d88a8-2bd1-4359-8c70-25dc90879cdc/mesos-bridge/network.conf':
> Failed to open file: No such file or directory
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> E0809 00:07:08.151798 4046 slave.cpp:5639] Failed to update resources for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc of executor
> 'testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a' running task
> testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a on status update for terminal
> task, destroying container: Container not found
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:08.152240 4043 composing.cpp:600] Attempted to destroy unknown
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:07:08.164916 4047 gc.cpp:95] Scheduling
> '/var/lib/mesos/slave/slaves/4397555f-b8fd-478c-907a-7532e2f5e32e-S0/frameworks/4397555f-b8fd-478c-907a-7532e2f5e32e-0001/executors/testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a/runs/278d88a8-2bd1-4359-8c70-25dc90879cdc'
> for gc 1.99999809405037days in the future
> Aug 09 00:07:08 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> I0809 00:07:08.165079 4047 gc.cpp:95] Scheduling
> '/var/lib/mesos/slave/meta/slaves/4397555f-b8fd-478c-907a-7532e2f5e32e-S0/frameworks/4397555f-b8fd-478c-907a-7532e2f5e32e-0001/executors/testcni2.09d9faa5-9b66-11e8-b04f-e2837c6f5f2a/runs/278d88a8-2bd1-4359-8c70-25dc90879cdc'
> for gc 1.99999809211852days in the future
> Aug 09 00:07:26 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:26.571424 4047 cgroups.cpp:737] Skipping resource statistic for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to get usage
> for subsystem 'memory': Unknown container
> Aug 09 00:07:26 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:26.571480 4047 cgroups.cpp:737] Skipping resource statistic for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read
> 'cpu.stat': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid cgroup
> Aug 09 00:07:26 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:26.571648 4047 containerizer.cpp:2221] Skipping resource
> statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Unknown
> container
> Aug 09 00:07:27 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:27.801781 4043 cgroups.cpp:737] Skipping resource statistic for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read from
> 'blkio.time': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid
> cgroup
> Aug 09 00:07:27 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:27.801839 4043 cgroups.cpp:737] Skipping resource statistic for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to get usage
> for subsystem 'memory': Unknown container
> Aug 09 00:07:27 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:27.801862 4043 cgroups.cpp:737] Skipping resource statistic for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read
> 'cpu.stat': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid cgroup
> Aug 09 00:07:27 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:27.801961 4043 containerizer.cpp:2221] Skipping resource
> statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Unknown
> container
> Aug 09 00:07:29 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:29.182669 4048 cgroups.cpp:737] Skipping resource statistic for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read from
> 'blkio.time': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid
> cgroup
> Aug 09 00:07:29 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:29.182729 4048 cgroups.cpp:737] Skipping resource statistic for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to get usage
> for subsystem 'memory': Unknown container
> Aug 09 00:07:29 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:29.182752 4048 cgroups.cpp:737] Skipping resource statistic for
> container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Failed to read
> 'cpu.stat': 'mesos/278d88a8-2bd1-4359-8c70-25dc90879cdc' is not a valid cgroup
> Aug 09 00:07:29 ip-10-0-3-181.us-west-2.compute.internal mesos-agent[4022]:
> W0809 00:07:29.182870 4046 containerizer.cpp:2221] Skipping resource
> statistic for container 278d88a8-2bd1-4359-8c70-25dc90879cdc because: Unknown
> container
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
