[jira] [Commented] (MESOS-6934) Support pulling Docker images with V2 Schema 2 image manifest
[ https://issues.apache.org/jira/browse/MESOS-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811150#comment-16811150 ] Gilbert Song commented on MESOS-6934: - commit db917f639e2d05fcf493e87649f42ddd2abfeae0 Author: Andrei Budnik abud...@mesosphere.com Date: Fri Apr 5 13:06:49 2019 +0200 Fixed use-after-free bug in Docker provisioner store. Deferred lambda callback of the `moveLayers()` to the `StoreProcess` to prevent use-after-free of the process object since the callback refers to the `StoreProcess` class variable `flags`. Review: https://reviews.apache.org/r/70405 > Support pulling Docker images with V2 Schema 2 image manifest > - > > Key: MESOS-6934 > URL: https://issues.apache.org/jira/browse/MESOS-6934 > Project: Mesos > Issue Type: Improvement > Components: containerization > Environment: https://reviews.apache.org/r/70288/ > https://reviews.apache.org/r/70289/ > https://reviews.apache.org/r/70290/ > https://reviews.apache.org/r/70291/ >Reporter: Ilya Pronin >Assignee: Gilbert Song >Priority: Major > Labels: containerization > Fix For: 1.8.0 > > > MESOS-3505 added support for pulling Docker images by their digest to the > Mesos Containerizer provisioner. However currently it only works with images > that were pushed with Docker 1.9 and older or with Registry 2.2.1 and older. > Newer versions use Schema 2 manifests by default. Because of CAS constraints > the registry does not convert those manifests on-the-fly to Schema 1 when > they are being pulled by digest. > Compatibility details are documented here: > https://docs.docker.com/registry/compatibility/ > Image Manifest V2, Schema 2 is documented here: > https://docs.docker.com/registry/spec/manifest-v2-2/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MESOS-9705) Operation Feedback Improvements
Greg Mann created MESOS-9705: Summary: Operation Feedback Improvements Key: MESOS-9705 URL: https://issues.apache.org/jira/browse/MESOS-9705 Project: Mesos Issue Type: Epic Reporter: Greg Mann Since the main functionality of operation feedback for non-launch operations has been completed, this epic will track improvements and bug fixes related to this code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-9693) Add master validation for SeccompInfo.
[ https://issues.apache.org/jira/browse/MESOS-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810961#comment-16810961 ] Vinod Kone commented on MESOS-9693: --- In addition to the points raised above, there is also an upgrade compatibility issue with implementing this. If a framework's task doesn't work when seccomp is enabled (e.g., a kubelet task that needs to run as unconfined so that it can launch k8s pods that are seccomp confined by docker seccomp profile), then the framework needs to be first upgraded to use seccomp unconfined option. Now if this framework was running already on non-seccomp enabled cluster, the upgraded framework needs to still keep running even with seccomp disabled. After framework upgrade, mesos agent can be upgraded to enable seccomp and this won't affect the framework. So Mesos cannot reject such a task but just ignore it. [~gilbert] [~abudnik] Should we close this as "Won't do"? > Add master validation for SeccompInfo. > -- > > Key: MESOS-9693 > URL: https://issues.apache.org/jira/browse/MESOS-9693 > Project: Mesos > Issue Type: Task >Reporter: Gilbert Song >Assignee: Andrei Budnik >Priority: Major > > 1. if seccomp is not enabled, we should return failure if any fw specify > seccompInfo and return appropriate status update. > 2. at most one field of profile_name and unconfined should be set. better to > validate in master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-9677) RPM packages should be built with launcher sealing
[ https://issues.apache.org/jira/browse/MESOS-9677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810958#comment-16810958 ] James DeFelice commented on MESOS-9677: --- It's in the el7 kernels, according to the changelog in the spec: https://git.centos.org/blob/rpms!kernel.git/c7/SPECS!kernel.spec > RPM packages should be built with launcher sealing > -- > > Key: MESOS-9677 > URL: https://issues.apache.org/jira/browse/MESOS-9677 > Project: Mesos > Issue Type: Task > Components: build >Affects Versions: 1.8.0 >Reporter: Benjamin Bannier >Priority: Major > Labels: integration, mesosphere, packaging, rpm, storage > > We should consider enabling launcher sealing in the Mesos RPM packages. Since > this feature is built conditionally, it is hard to write e.g., module code > against Mesos packages since required functions might be missing (e.g., > [https://github.com/dcos/dcos-mesos-modules/commit/8ce70e6cc789054831daa3058647e326b2b11bc9] > cannot be linked against the default RPM package anymore). The RPM's target > platform centos7 should include a recent enough kernel for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-9624) Bundle CSI spec v1.0 in Mesos.
[ https://issues.apache.org/jira/browse/MESOS-9624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810832#comment-16810832 ] Benjamin Bannier commented on MESOS-9624: - [~chhsia0] , it looks like you have landed all patches, right? Can we close this? > Bundle CSI spec v1.0 in Mesos. > -- > > Key: MESOS-9624 > URL: https://issues.apache.org/jira/browse/MESOS-9624 > Project: Mesos > Issue Type: Task > Components: storage >Reporter: Chun-Hung Hsiao >Assignee: Chun-Hung Hsiao >Priority: Critical > Labels: mesosphere, storage > > We need to bundle both CSI v0 and v1 in Mesos. This requires some redesign of > the source code filesystem layout. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-8257) Unified Containerizer "leaks" a target container mount path to the host FS when the target resolves to an absolute path
[
https://issues.apache.org/jira/browse/MESOS-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810819#comment-16810819
]
Benno Evers commented on MESOS-8257:
I removed the 1.8.0 target designation here and in the linked ticket since it
looks like there hasn't been any recent activity here, please feel free to
revert as you see fit.
> Unified Containerizer "leaks" a target container mount path to the host FS
> when the target resolves to an absolute path
> ---
>
> Key: MESOS-8257
> URL: https://issues.apache.org/jira/browse/MESOS-8257
> Project: Mesos
> Issue Type: Bug
> Components: containerization
>Affects Versions: 1.3.1, 1.4.1, 1.5.0
>Reporter: Jason Lai
>Assignee: Jason Lai
>Priority: Critical
> Labels: bug, containerization, containerizer, mountpath
>
> If a target path under the root FS provisioned from an image resolves to an
> absolute path, it will not appear in the container root FS after
> {{pivot_root(2)}} is called.
> A typical example is that when the target path is under {{/var/run}} (e.g.
> {{/var/run/some-dir}}), which is usually a symlink to an absolute path of
> {{/run}} in Debian images, the target path will get resolved as and created
> at {{/run/some-dir}} in the host root FS, after the container root FS gets
> provisioned. The target path will get unmounted after {{pivot_root(2)}} as it
> is part of the old root (host FS).
> A workaround is to use {{/run}} instead of {{/var/run}}, but absolute
> symlinks need to be resolved within the scope of the container root FS path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9677) RPM packages should be built with launcher sealing
[
https://issues.apache.org/jira/browse/MESOS-9677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810759#comment-16810759
]
Benno Evers commented on MESOS-9677:
On the `memfd_create()` manpage I can read:
{quote}
The memfd_create() system call first appeared in Linux 3.17
{quote}
According to Wikipedia, CentOS 7 uses kernels from the 3.10 series:
https://en.wikipedia.org/wiki/CentOS#Latest_version_information
So I'm not sure if it will really be safe to enable this per default on CentOS
7. [~gilbert], can you clarify this?
> RPM packages should be built with launcher sealing
> --
>
> Key: MESOS-9677
> URL: https://issues.apache.org/jira/browse/MESOS-9677
> Project: Mesos
> Issue Type: Task
> Components: build
>Affects Versions: 1.8.0
>Reporter: Benjamin Bannier
>Priority: Major
> Labels: integration, mesosphere, packaging, rpm, storage
>
> We should consider enabling launcher sealing in the Mesos RPM packages. Since
> this feature is built conditionally, it is hard to write e.g., module code
> against Mesos packages since required functions might be missing (e.g.,
> [https://github.com/dcos/dcos-mesos-modules/commit/8ce70e6cc789054831daa3058647e326b2b11bc9]
> cannot be linked against the default RPM package anymore). The RPM's target
> platform centos7 should include a recent enough kernel for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-8800) start failed after add "network/port_mapping" to isolation
[
https://issues.apache.org/jira/browse/MESOS-8800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810573#comment-16810573
]
Pavel commented on MESOS-8800:
--
Nearly the same error on 1.7.2, build with ./configure
--ENABLE_NETWORK_PORTS_ISOLATOR
--isolation=filesystem/linux,docker/runtime,posix/rlimits,network/port_mapping
{noformat}
mesos-slave: ../3rdparty/boost-1.65.0/boost/icl/concept/interval.hpp:586:
typename boost::enable_if, bool>::type
boost::icl::non_empty::exclusive_less(const Type&, const Type&) [with Type =
Interval; typename
boost::enable_if, bool>::type = bool]:
Assertion `!(icl::is_empty(left) || icl::is_empty(right))' failed.
mesos-slave[24391]: *** Aborted at 1554415003 (unix time) try "date -d
@1554415003" if you are using GNU date ***
mesos-slave[24391]: PC: @ 0x7f81feba21f7 __GI_raise
mesos-slave[24391]: *** SIGABRT (@0x5ef4) received by PID 24308 (TID
0x7f8203b8ba00) from PID 24308; stack trace: ***
mesos-slave[24391]: @ 0x7f81ff45d5e0 (unknown)
mesos-slave[24391]: @ 0x7f81feba21f7 __GI_raise
mesos-slave[24391]: @ 0x7f81feba38e8 __GI_abort
mesos-slave[24391]: @ 0x7f81feb9b266 __assert_fail_base
mesos-slave[24391]: @ 0x7f81feb9b312 __GI___assert_fail
mesos-slave[24391]: @ 0x7f8201d5c711 std::_Rb_tree<>::_M_lower_bound()
mesos-slave[24391]: @ 0x7f8201d96cfc std::_Rb_tree<>::find()
mesos-slave[24391]: @ 0x7f8201d7e19d
mesos::internal::slave::PortMappingIsolatorProcess::create()
mesos-slave[24391]: @ 0x7f8201b47f8c
std::_Function_handler<>::_M_invoke()
mesos-slave[24391]: @ 0x7f8201b36ab2
mesos::internal::slave::MesosContainerizer::create()
mesos-slave[24391]: @ 0x7f8201aac1b6
mesos::internal::slave::Containerizer::create()
mesos-slave[24391]: @ 0x55a31557011d main
mesos-slave[24391]: @ 0x7f81feb8ec05 __libc_start_main
mesos-slave[24391]: @ 0x55a315572c2c (unknown)
{noformat}
Sometimes it is preferred using network=host for container with port mapping
isolation, because apps need host IP, and port range as a resource.
Could it probably be a configuratin error issue?
> start failed after add "network/port_mapping" to isolation
> --
>
> Key: MESOS-8800
> URL: https://issues.apache.org/jira/browse/MESOS-8800
> Project: Mesos
> Issue Type: Task
>Affects Versions: 1.5.0
>Reporter: stevenlee
>Priority: Major
>
> when i add
> "cgroups/cpu,cgroups/mem,disk/du,filesystem/linux,docker/runtime,network/port_mapping"
> to islotion and add
> "cpus:24;mem:128000;ports:[31000-32000];disk:10;ephemeral_ports:[32768-57344]"
> to resources,then i systemctl start mesos-slave,report this:
> Log file created at: 2018/04/18 15:15:56
> Running on machine: test.machine.net
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> I0418 15:15:56.583765 21945 logging.cpp:201] INFO level logging started!
> I0418 15:15:56.584277 21945 main.cpp:365] Build: 2018-04-14 13:33:38 by root
> I0418 15:15:56.584306 21945 main.cpp:366] Version: 1.5.0
> I0418 15:15:56.615106 21945 systemd.cpp:240] systemd version `219` detected
> I0418 15:15:56.615202 21945 main.cpp:468] Initializing systemd state
> I0418 15:15:56.623399 21945 systemd.cpp:328] Started systemd slice
> `mesos_executors.slice`
> I0418 15:15:56.626150 21945 resolver.cpp:69] Creating default secret resolver
> I0418 15:15:56.713505 21945 containerizer.cpp:304] Using isolation \{
> environment_secret, volume/image, volume/host_path, cgroups/cpu, cgroups/mem,
> docker/runtime, volume/sandbox_path, disk/du, filesystem/linux,
> network/port_mapping }
> I0418 15:15:56.725468 21945 linux_launcher.cpp:146] Using
> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
> I0418 15:15:56.812513 21945 fetcher.cpp:69] Skipping URI fetcher plugin
> 'hadoop' as it could not be created: Failed to create HDFS client: Hadoop
> client is not available, exit status: 32512
> I0418 15:15:56.818392 21945 provisioner.cpp:299] Using default backend 'copy'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-9704) Support docker manifest v2s2 config GC.
Gilbert Song created MESOS-9704: --- Summary: Support docker manifest v2s2 config GC. Key: MESOS-9704 URL: https://issues.apache.org/jira/browse/MESOS-9704 Project: Mesos Issue Type: Improvement Components: containerization Reporter: Gilbert Song After docker manifest v2s2 support, layer GC is still properly supported. However, the manifest config is not garbage collected. Need to add the config dir to the checkpointed LAYERS_FILE to support config GC. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MESOS-9703) Support docker manifest v2s2 external urls.
Gilbert Song created MESOS-9703: --- Summary: Support docker manifest v2s2 external urls. Key: MESOS-9703 URL: https://issues.apache.org/jira/browse/MESOS-9703 Project: Mesos Issue Type: Improvement Components: containerization Reporter: Gilbert Song docker manifest v2s2 spec define external URLs. Some windows image rely on those urls to download some private layers from microsoft server. Some refactoring may be needed to get rid of the current external urls support because the uri fetcher has to parse the manifest when pulling every layer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
