[jira] [Updated] (MESOS-7350) Launching docker container with universal containerizer fails with "Failed to parse the image manifest: Docker v2 image manifest validation failed: 'signatures' field siz
[
https://issues.apache.org/jira/browse/MESOS-7350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Ustinov updated MESOS-7350:
---
Description:
I’m trying to launch docker container with universal containerizer, mesos
1.2.0. But getting error “Failed to parse the image manifest: Docker v2 image
manifest validation failed: ‘signatures’ field size must be at least one”. And
if I switch to docker containerizer, app is starting normally.
We are working with private docker registry v2 backed by nexus repository
manager 3.1.0
{code}
cat /etc/mesos-slave/docker_registry
https://docker.company.ru
cat /etc/mesos-slave/docker_config
{
"auths": {
"docker.company.ru": {
"auth": ""
}
}
}
{code}
Here agent's log:
{code}
I0405 22:00:49.860234 44856 slave.cpp:4346] Received ping from
slave-observer(7)@10.34.1.31:5050
I0405 22:00:50.327030 44865 slave.cpp:1625] Got assigned task
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for framework
5ad97c04-d982-49d3-ac4f-53c468993190-
I0405 22:00:50.327785 44865 slave.cpp:1785] Launching task
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for framework
5ad97c04-d982-49d3-ac4f-53c468993190-
I0405 22:00:50.329324 44865 paths.cpp:547] Trying to chown
'/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff'
to user 'dockdata'
I0405 22:00:50.329607 44865 slave.cpp:6896] Checkpointing ExecutorInfo to
'/export/intssd/mesos-slave/workdir/meta/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/executor.info'
I0405 22:00:50.330531 44865 slave.cpp:6472] Launching executor
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework
5ad97c04-d982-49d3-ac4f-53c468993190- with resources cpus(*)(allocated:
general_marathon_service_role):0.1; mem(*)(allocated:
general_marathon_service_role):32 in work directory
'/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff'
I0405 22:00:50.331244 44865 slave.cpp:6919] Checkpointing TaskInfo to
'/export/intssd/mesos-slave/workdir/meta/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff/tasks/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/task.info'
I0405 22:00:50.331568 44862 docker.cpp:1106] Skipping non-docker container
I0405 22:00:50.331822 44865 slave.cpp:2118] Queued task
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for executor
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework
5ad97c04-d982-49d3-ac4f-53c468993190-
I0405 22:00:50.331966 44865 slave.cpp:884] Successfully attached file
'/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff'
I0405 22:00:50.332582 44861 containerizer.cpp:993] Starting container
f82f5f69-87a3-4586-b4cc-b91d285dcaff for executor
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework
5ad97c04-d982-49d3-ac4f-53c468993190-
I0405 22:00:50.333286 44862 metadata_manager.cpp:168] Looking for image
'docker.company.ru/company-infra/kafka:0.10.2.0-16'
I0405 22:00:50.333627 44879 registry_puller.cpp:247] Pulling image
'docker.company.ru/company-infra/kafka:0.10.2.0-16' from
'docker-manifest://docker.company.rucompany-infra/kafka?0.10.2.0-16#https' to
'/export/intssd/mesos-slave/docker-store/staging/aV2yko'
E0405 22:00:50.834630 44872 slave.cpp:4642] Container
'f82f5f69-87a3-4586-b4cc-b91d285dcaff' for executor
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework
5ad97c04-d982-49d3-ac4f-53c468993190- failed to start: Failed to parse the
image manifest: Docker v2 image manifest validation failed: 'signatures' field
size must be at least one
I0405 22:00:50.835008 44853 containerizer.cpp:2069] Destroying container
f82f5f69-87a3-4586-b4cc-b91d285dcaff in PROVISIONING state
I0405 22:00:50.835127 44853 containerizer.cpp:2124] Waiting for the provisioner
to complete provisioning before destroying container
f82f5f69-87a3-4586-b4cc-b91d285dcaff
I0405 22:00:50.835273 44844 provisioner.cpp:484] Ignoring destroy request for
unknown container f82f5f69-87a3-4586-b4cc-b91d285dcaff
I0405 22:00:50.836199 44837 slave.cpp:4754] Executor
[jira] [Created] (MESOS-7350) Launching docker container with universal containerizer fails with "Failed to parse the image manifest: Docker v2 image manifest validation failed: 'signatures' field siz
Nikolay Ustinov created MESOS-7350:
--
Summary: Launching docker container with universal containerizer
fails with "Failed to parse the image manifest: Docker v2 image manifest
validation failed: 'signatures' field size must be at least one"
Key: MESOS-7350
URL: https://issues.apache.org/jira/browse/MESOS-7350
Project: Mesos
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Nikolay Ustinov
I’m trying to launch docker container with universal containerizer, mesos
1.2.0. But getting error “Failed to parse the image manifest: Docker v2 image
manifest validation failed: ‘signatures’ field size must be at least one”. And
if I switch to docker containerizer, app is starting normally.
We are working with private docker registry v2 backed by nexus repository
manager 3.1.0
Here agent's log:
{code}
I0405 22:00:49.860234 44856 slave.cpp:4346] Received ping from
slave-observer(7)@10.34.1.31:5050
I0405 22:00:50.327030 44865 slave.cpp:1625] Got assigned task
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for framework
5ad97c04-d982-49d3-ac4f-53c468993190-
I0405 22:00:50.327785 44865 slave.cpp:1785] Launching task
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for framework
5ad97c04-d982-49d3-ac4f-53c468993190-
I0405 22:00:50.329324 44865 paths.cpp:547] Trying to chown
'/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff'
to user 'dockdata'
I0405 22:00:50.329607 44865 slave.cpp:6896] Checkpointing ExecutorInfo to
'/export/intssd/mesos-slave/workdir/meta/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/executor.info'
I0405 22:00:50.330531 44865 slave.cpp:6472] Launching executor
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework
5ad97c04-d982-49d3-ac4f-53c468993190- with resources cpus(*)(allocated:
general_marathon_service_role):0.1; mem(*)(allocated:
general_marathon_service_role):32 in work directory
'/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff'
I0405 22:00:50.331244 44865 slave.cpp:6919] Checkpointing TaskInfo to
'/export/intssd/mesos-slave/workdir/meta/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff/tasks/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/task.info'
I0405 22:00:50.331568 44862 docker.cpp:1106] Skipping non-docker container
I0405 22:00:50.331822 44865 slave.cpp:2118] Queued task
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for executor
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework
5ad97c04-d982-49d3-ac4f-53c468993190-
I0405 22:00:50.331966 44865 slave.cpp:884] Successfully attached file
'/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff'
I0405 22:00:50.332582 44861 containerizer.cpp:993] Starting container
f82f5f69-87a3-4586-b4cc-b91d285dcaff for executor
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework
5ad97c04-d982-49d3-ac4f-53c468993190-
I0405 22:00:50.333286 44862 metadata_manager.cpp:168] Looking for image
'docker.company.ru/company-infra/kafka:0.10.2.0-16'
I0405 22:00:50.333627 44879 registry_puller.cpp:247] Pulling image
'docker.company.ru/company-infra/kafka:0.10.2.0-16' from
'docker-manifest://docker.company.rucompany-infra/kafka?0.10.2.0-16#https' to
'/export/intssd/mesos-slave/docker-store/staging/aV2yko'
E0405 22:00:50.834630 44872 slave.cpp:4642] Container
'f82f5f69-87a3-4586-b4cc-b91d285dcaff' for executor
'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework
5ad97c04-d982-49d3-ac4f-53c468993190- failed to start: Failed to parse the
image manifest: Docker v2 image manifest validation failed: 'signatures' field
size must be at least one
I0405 22:00:50.835008 44853 containerizer.cpp:2069] Destroying container
f82f5f69-87a3-4586-b4cc-b91d285dcaff in PROVISIONING state
I0405 22:00:50.835127 44853 containerizer.cpp:2124] Waiting for the provisioner
to complete provisioning before destroying container
f82f5f69-87a3-4586-b4cc-b91d285dcaff
I0405 22:00:50.835273 44844 provisioner.cpp:484] Ignoring destroy request for
unknown container f82f5f69-87a3-4586-b4cc-b91d285dcaff
I0405
[jira] [Created] (MESOS-7208) Persistent volume ownership is set to root when task is running with non-root user
Nikolay Ustinov created MESOS-7208:
--
Summary: Persistent volume ownership is set to root when task is
running with non-root user
Key: MESOS-7208
URL: https://issues.apache.org/jira/browse/MESOS-7208
Project: Mesos
Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Nikolay Ustinov
I’m running docker container in universal containerizer, mesos 1.1.0.
switch_user=true, isolator=filesystem/linux,docker/runtime. Container is
launched with marathon, “user”:”someappuser”. I’d want to use persistent
volume, but it’s exposed to container with root user permissions even if root
folder is created with someppuser ownership (looks like mesos do chown to this
folder).
here logs for my container:
{code}
I0305 22:51:36.414655 10175 slave.cpp:1701] Launching task
'md_hdfs_journal.23f813ab-01dd-11e7-a012-0242ce94d92a' for framework
e9d0e39e-b67d-4142-b95d-b0987998eb92-
I0305 22:51:36.415118 10175 paths.cpp:536] Trying to chown
'/export/intssd/mesos-slave/workdir/slaves/85150805-a201-4b23-ab21-b332a458fc97-S10/frameworks/e9d0e39e-b67d-4142-b95d-b0987998eb92-/executors/md_hdfs_journal.23f813ab-01dd-11e7-a012-0242ce94d92a/runs/e978d4eb-5ec1-44ad-b50a-9ae6bfe1065a'
to user 'root'
I0305 22:51:36.422992 10175 slave.cpp:6179] Launching executor
'md_hdfs_journal.23f813ab-01dd-11e7-a012-0242ce94d92a' of framework
e9d0e39e-b67d-4142-b95d-b0987998eb92- with resources cpus(*):0.1; mem(*):32
in work directory
'/export/intssd/mesos-slave/workdir/slaves/85150805-a201-4b23-ab21-b332a458fc97-S10/frameworks/e9d0e39e-b67d-4142-b95d-b0987998eb92-/executors/md_hdfs_journal.23f813ab-01dd-11e7-a012-0242ce94d92a/runs/e978d4eb-5ec1-44ad-b50a-9ae6bfe1065a'
I0305 22:51:36.424278 10175 slave.cpp:1987] Queued task
'md_hdfs_journal.23f813ab-01dd-11e7-a012-0242ce94d92a' for executor
'md_hdfs_journal.23f813ab-01dd-11e7-a012-0242ce94d92a' of framework
e9d0e39e-b67d-4142-b95d-b0987998eb92-
I0305 22:51:36.424347 10158 docker.cpp:1000] Skipping non-docker container
I0305 22:51:36.425639 10142 containerizer.cpp:938] Starting container
e978d4eb-5ec1-44ad-b50a-9ae6bfe1065a for executor
'md_hdfs_journal.23f813ab-01dd-11e7-a012-0242ce94d92a' of framework
e9d0e39e-b67d-4142-b95d-b0987998eb92-
I0305 22:51:36.428725 10166 provisioner.cpp:294] Provisioning image rootfs
'/export/intssd/mesos-slave/workdir/provisioner/containers/e978d4eb-5ec1-44ad-b50a-9ae6bfe1065a/backends/copy/rootfses/0e2181e9-1bf2-42d4-8cb0-ee70e466c3ae'
for container e978d4eb-5ec1-44ad-b50a-9ae6bfe1065a
I0305 22:51:42.981240 10149 linux.cpp:695] Changing the ownership of the
persistent volume at
'/export/intssd/mesos-slave/data/volumes/roles/general_marathon_service_role/md_hdfs_journal#data#23f813aa-01dd-11e7-a012-0242ce94d92a'
with uid 0 and gid 0
I0305 22:51:42.986593 10136 linux_launcher.cpp:421] Launching container
e978d4eb-5ec1-44ad-b50a-9ae6bfe1065a and cloning with namespaces CLONE_NEWNS
{code}
{code}
ls -la
/export/intssd/mesos-slave/workdir/slaves/85150805-a201-4b23-ab21-b332a458fc97-S10/frameworks/e9d0e39e-b67d-4142-b95d-b0987998eb92-/executors/md_hdfs_journal.23f813ab-01dd-11e7-a012-0242ce94d92a/runs/e978d4eb-5ec1-44ad-b50a-9ae6bfe1065a/
drwxr-xr-x 3 someappuser someappgroup 4096 22:51 .
drwxr-xr-x 3 root root4096 22:51 ..
drwxr-xr-x 2 root root4096 22:51 data
-rw-r--r-- 1 root root 169 22:51 stderr
-rw-r--r-- 1 root root 183012 23:00 stdout
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
