Re: [atomic-devel] [atomic] Space leak issue in new installs of the *previous* Fedora Atomic Host release

[Dusty Mabe]

Adding in atomic-devel/fedora cloud list and adding some more information.

This affected the qcow/vagrant/AMI images from the following two releases
of Fedora Atomic Host:

Fedora-Atomic-25-20170228.0 - Ostree version: 25.67
Fedora-Atomic-25-20170215.1 - Ostree version: 25.59

If you booted from one of the images for one of these releases then
please run the command as shown by Colin in the previous email.

On 03/15/2017 01:04 PM, Colin Walters wrote:
> TL;DR: If you did a *new* installation of the previous Fedora Atomic Host,
> run:
> 
> ostree refs --delete fedora-atomic:fedora-atomic/25/x86_64/updates/docker-host
> 
> This command will do nothing and be safe if you *aren't* affected.
> 
> New installations as well as upgrades from older releases are not affected.
> 
> A bit more information in:
> https://pagure.io/atomic-wg/issue/251
> 
> (However, if you want to participate in development, you'll likely be rebasing
>  back/forth with the updates ref anyways, in which case there's no leak).
> 
> 

Re: [atomic-devel] Fedora Atomic Host Two Week Release Announcement

[Dusty Mabe]

On 03/15/2017 08:27 AM, nore...@fedoraproject.org wrote:
> 
> A new Fedora Atomic Host update is available via an OSTree commit:
> 
> Commit: 24d4499420ffb2cc49681020bbe5aa6780d780d2b811eab1f5ffea6446b5a4c5
> Version: 25.80
> 
> 
> Existing systems can be upgraded in place via e.g. `atomic host upgrade` or
> `atomic host deploy`.
> 
> Corresponding image media for new installations can be downloaded from:
> 
> https://getfedora.org/en/atomic/download/
> 
> Respective signed CHECKSUM files can be found here:
> https://alt.fedoraproject.org/pub/alt/atomic/stable/Fedora-Atomic-25-20170314.0/Atomic/x86_64/iso/Fedora-Atomic-25-20170314.0-x86_64-CHECKSUM
> https://alt.fedoraproject.org/pub/alt/atomic/stable/Fedora-Atomic-25-20170314.0/CloudImages/x86_64/images/Fedora-CloudImages-25-20170314.0-x86_64-CHECKSUM
> 
> For direct download, the "latest" targets are always available here:
> https://getfedora.org/atomic_iso_latest
> https://getfedora.org/atomic_qcow2_latest
> https://getfedora.org/atomic_raw_latest
> https://getfedora.org/atomic_vagrant_libvirt_latest
> https://getfedora.org/atomic_vagrant_virtualbox_latest
> 
> Filename fetching URLs are available here:
> https://getfedora.org/atomic_iso_latest_filename
> https://getfedora.org/atomic_qcow2_latest_filename
> https://getfedora.org/atomic_raw_latest_filename
> https://getfedora.org/atomic_vagrant_libvirt_latest_filename
> https://getfedora.org/atomic_vagrant_virtualbox_latest_filename
> 
> For more information about the latest targets, please reference the Fedora
> Cloud Wiki space.
> 
> https://fedoraproject.org/wiki/Cloud#Quick_Links
> 
> Do note that it can take some of the mirrors up to 12 hours to "check-in" at
> their own discretion.
> 
> Thank you,
> Fedora Release Engineering
> 

This release contains the following changes since last release from
ba95a4665776b58d342ad9cc36779f9b8fcf19c6606f8964a8ec1622cadc to
24d4499420ffb2cc49681020bbe5aa6780d780d2b811eab1f5ffea6446b5a4c5

Upgraded:
  atomic-devmode 0.3.3-1.fc25 -> 0.3.6-1.fc25
  bind99-libs 9.9.9-4.P5.fc25 -> 9.9.9-4.P6.fc25
  bind99-license 9.9.9-4.P5.fc25 -> 9.9.9-4.P6.fc25
  cockpit-bridge 131-1.fc25 -> 134-1.fc25
  cockpit-docker 131-1.fc25 -> 134-1.fc25
  cockpit-networkmanager 131-1.fc25 -> 134-1.fc25
  cockpit-ostree 131-1.fc25 -> 134-1.fc25
  cockpit-system 131-1.fc25 -> 134-1.fc25
  container-selinux 2:2.6-1.fc25 -> 2:2.10-1.fc25
  coreutils 8.25-15.fc25 -> 8.25-16.fc25
  coreutils-common 8.25-15.fc25 -> 8.25-16.fc25
  fedora-repos 25-2 -> 25-3
  freetype 2.6.5-1.fc25 -> 2.6.5-3.fc25
  gnutls 3.5.9-2.fc25 -> 3.5.10-1.fc25
  gssproxy 0.6.1-2.fc25 -> 0.7.0-1.fc25
  kernel 4.9.12-200.fc25 -> 4.9.13-201.fc25
  kernel-core 4.9.12-200.fc25 -> 4.9.13-201.fc25
  kernel-modules 4.9.12-200.fc25 -> 4.9.13-201.fc25
  krb5-libs 1.14.4-4.fc25 -> 1.14.4-7.fc25
  libseccomp 2.3.1-1.fc25 -> 2.3.2-1.fc25
  libsss_idmap 1.14.2-2.fc25 -> 1.15.1-1.fc25
  libsss_nss_idmap 1.14.2-2.fc25 -> 1.15.1-1.fc25
  libsss_sudo 1.14.2-2.fc25 -> 1.15.1-1.fc25
  nss 3.28.1-1.3.fc25 -> 3.28.3-1.0.fc25
  nss-pem 1.0.2-2.fc25 -> 1.0.3-2.fc25
  nss-softokn 3.28.1-1.0.fc25 -> 3.28.3-1.1.fc25
  nss-softokn-freebl 3.28.1-1.0.fc25 -> 3.28.3-1.1.fc25
  nss-sysinit 3.28.1-1.3.fc25 -> 3.28.3-1.0.fc25
  nss-tools 3.28.1-1.3.fc25 -> 3.28.3-1.0.fc25
  nss-util 3.28.1-1.0.fc25 -> 3.28.3-1.0.fc25
  oci-systemd-hook 0.1.4-4.git15c2f48.fc25 -> 1:0.1.5-1.git16f7c8a.fc25
  openssh 7.4p1-3.fc25 -> 7.4p1-4.fc25
  openssh-clients 7.4p1-3.fc25 -> 7.4p1-4.fc25
  openssh-server 7.4p1-3.fc25 -> 7.4p1-4.fc25
  pcre 8.40-4.fc25 -> 8.40-5.fc25
  python3-rpm 4.13.0-6.fc25 -> 4.13.0.1-1.fc25
  python3-sssdconfig 1.14.2-2.fc25 -> 1.15.1-1.fc25
  rpm 4.13.0-6.fc25 -> 4.13.0.1-1.fc25
  rpm-build-libs 4.13.0-6.fc25 -> 4.13.0.1-1.fc25
  rpm-libs 4.13.0-6.fc25 -> 4.13.0.1-1.fc25
  rpm-plugin-selinux 4.13.0-6.fc25 -> 4.13.0.1-1.fc25
  screen 4.5.0-1.fc25 -> 4.5.1-1.fc25
  selinux-policy 3.13.1-225.10.fc25 -> 3.13.1-225.11.fc25
  selinux-policy-targeted 3.13.1-225.10.fc25 -> 3.13.1-225.11.fc25
  sssd-client 1.14.2-2.fc25 -> 1.15.1-1.fc25
  vim-minimal 2:8.0.347-2.fc25 -> 2:8.0.425-1.fc25



Also, for posterity the AMIs for this release are:

Fedora-Atomic-25-20170314.0.x86_64   EC2 (ap-northeast-1) ami-b5faa8d2hvm   
standard   
Fedora-Atomic-25-20170314.0.x86_64   EC2 (ap-southeast-1) ami-c8c270abhvm   
standard   
Fedora-Atomic-25-20170314.0.x86_64   EC2 (ap-southeast-2) ami-e0191483hvm   
standard   
Fedora-Atomic-25-20170314.0.x86_64   EC2 (eu-central-1)   ami-8401d6ebhvm   
standard   
Fedora-Atomic-25-20170314.0.x86_64   EC2 (eu-west-1)  ami-42447324hvm   
standard   
Fedora-Atomic-25-20170314.0.x86_64   EC2 (sa-east-1)  ami-70e8891chvm   
standard   
Fedora-Atomic-25-20170314.0.x86_64   EC2 (us-east-1)  ami-89f55b9fhvm   
standard   
Fedora-Atomic-25-20170314.0.x86_64   EC2 (us-west-1)  ami-30025a50hvm   
standard  

Re: [atomic-devel] Status of containerizing docker and https://github.com/projectatomic/atomic-system-containers

[Giuseppe Scrivano]

Colin Walters  writes:

> Does anyone know what the status of
> https://github.com/projectatomic/atomic-system-containers
> is in general, and in particular I'm interested in the
> "containerized docker" approach.
>
> Can someone who knows a bit more about this add
> e.g. a `README.md` with getting started instructions?

We have tried different solutions to get it done, the current version in
atomic-system-containers is using a chroot for running the Docker
container, but I don't really like this approach and anyway it brings
its own set of issues as maintaining/creating the rootfs for the
container manually.

Yes, it is a bit messy, I am going to work on this and try to make it
clearer.  The gscrivano/docker-fedora and gscrivano/docker-centos
containers are based on the PR here:

  https://github.com/projectatomic/atomic-system-containers/pull/38

As soon as it gets a bit more stable, we will need to move somewhere
else than my Docker hub account, as we did for Flannel and etcd.

The biggest issue is how to support live-restore.  Docker remounts
/var/lib/docker/devicemapper/* as MS_PRIVATE when it runs, so whatever
mount it creates there, it will not be accessible once the namespace is
destroyed without some dirty tricks.
I've filed an issue for Docker upstream, as in general it is not
possible to run Docker in its own mount namespace and support
live-restore (could be useful even for things like systemd
InaccessiblePaths=):

  https://github.com/docker/docker/issues/31489

There are some workarounds in the PR, but I got it to work somehow.  The
limitation is that you can't exec in a Docker container that is living
more than two Docker containers update (as a system container keeps only
two deployments).  The reason for this limitation is that when you do
"docker exec ...", it will use the runc that was installed as part of
the system container deployment that after two updates is removed.

I wouldn't worry too much of it for now. IIUIC in libcontainerd
master there is some work to let the shim process attached to a docker
container do the exec itself, so we won't have the limitation with
upgrading the Docker system containers as won't be required to run runc
from the mount namespace the container was created.

> # atomic host status
> State: idle
> Deployments:
> ● fedora-atomic:fedora-atomic/25/x86_64/docker-host
>  Version: 25.80 (2017-03-13 23:35:50)
>   Commit: 
> 24d4499420ffb2cc49681020bbe5aa6780d780d2b811eab1f5ffea6446b5a4c5
>   OSName: fedora-atomic
> # atomic install --system gscrivano/docker-fedora
> # systemctl start docker-fedora
> ...
> Mar 15 16:02:02 localhost.localdomain runc[4867]: container_linux.go:247: 
> starting container process caused "process_linux.go:359: container init 
> caused \"rootfs_linux.go:89: jailing process inside rootfs caused 
> \\\"pivot_root invalid argument\\\"\""

looks bad, I am going to have a look.

Regards,
Giuseppe

Re: [atomic-devel] Status of containerizing docker and https://github.com/projectatomic/atomic-system-containers

[Jerry Zhang]

Hi Colin,

> Hey,
> 
> Does anyone know what the status of
> https://github.com/projectatomic/atomic-system-containers
> is in general, and in particular I'm interested in the
> "containerized docker" approach.
> 

Most of the containers that are in the repo are operational
but not fully ready for production, minus etcd and flannel
which are more stable and tested compared to the other ones.

> Can someone who knows a bit more about this add
> e.g. a `README.md` with getting started instructions?
> 
> I did find https://hub.docker.com/r/gscrivano/docker-fedora/
> which has some info, but it appears generic and not specific
> to this container.  The tradeoffs/implementation details
> of containerizing Docker in particular seem worth having
> a specific doc.
> 

Giuseppe's repo is not technically the offical repo, although
you are right in that we need better docs. Once the issues
are more flattened out I'd imagine Giuseppe would add the
docs for docker.

> (Also, that image is auto-built from github:giuseppe/atomic-oci-containers
>  which is different from the projectatomic one?)
> 

No that is the same repo. The repo in projectatomic was moved over
from giuseppe's. If you click the link it will direct you to the
projectatomic repo.

> I just tried this:
> 
> ```
> # atomic host status
> State: idle
> Deployments:
> ● fedora-atomic:fedora-atomic/25/x86_64/docker-host
>  Version: 25.80 (2017-03-13 23:35:50)
>   Commit:
>   24d4499420ffb2cc49681020bbe5aa6780d780d2b811eab1f5ffea6446b5a4c5
>   OSName: fedora-atomic
> # atomic install --system gscrivano/docker-fedora
> # systemctl start docker-fedora
> ...
> Mar 15 16:02:02 localhost.localdomain runc[4867]: container_linux.go:247:
> starting container process caused "process_linux.go:359: container init
> caused \"rootfs_linux.go:89: jailing process inside rootfs caused
> \\\"pivot_root invalid argument\\\"\""
> ```
> 
> 
> 

Try instead: https://github.com/projectatomic/atomic-system-containers/pull/38
I build that locally and docker runs fine from a system container (running
on f25 cloud):
# atomic containers list
   CONTAINER ID IMAGECOMMAND  CREATED  
STATE BACKENDRUNTIME   
   flannel  gscrivano/flannel/usr/bin/flanneld-ru 2017-03-15 16:37 
running   ostree runc  
   docker   local/docker /usr/bin/init.sh 2017-03-15 16:37 
running   ostree runc  
   etcd local/etcd   /usr/bin/etcd-env.sh 2017-03-15 16:37 
running   ostree runc

# systemctl status docker
● docker.service - Docker service
   Loaded: loaded (/etc/systemd/system/docker.service; enabled; vendor preset: 
disabled)
  Drop-In: /etc/systemd/system/docker.service.d
   └─flannel.conf
   Active: active (running) since Wed 2017-03-15 16:37:58 UTC; 7min ago

For testing convenience I've build that branch to 
https://hub.docker.com/r/jerzhang/docker/,
so you can pull with `atomic pull --storage ostree jerzhang/docker`
Please let me know if that works for you.

Regards,
Yu Qi Zhang

[atomic-devel] Status of containerizing docker and https://github.com/projectatomic/atomic-system-containers

[Colin Walters]

Hey,

Does anyone know what the status of
https://github.com/projectatomic/atomic-system-containers
is in general, and in particular I'm interested in the
"containerized docker" approach.

Can someone who knows a bit more about this add
e.g. a `README.md` with getting started instructions?

I did find https://hub.docker.com/r/gscrivano/docker-fedora/
which has some info, but it appears generic and not specific
to this container.  The tradeoffs/implementation details
of containerizing Docker in particular seem worth having
a specific doc.

(Also, that image is auto-built from github:giuseppe/atomic-oci-containers
 which is different from the projectatomic one?)

I just tried this:

```
# atomic host status
State: idle
Deployments:
● fedora-atomic:fedora-atomic/25/x86_64/docker-host
 Version: 25.80 (2017-03-13 23:35:50)
  Commit: 
24d4499420ffb2cc49681020bbe5aa6780d780d2b811eab1f5ffea6446b5a4c5
  OSName: fedora-atomic
# atomic install --system gscrivano/docker-fedora
# systemctl start docker-fedora
...
Mar 15 16:02:02 localhost.localdomain runc[4867]: container_linux.go:247: 
starting container process caused "process_linux.go:359: container init caused 
\"rootfs_linux.go:89: jailing process inside rootfs caused \\\"pivot_root 
invalid argument\\\"\""
```

Re: [atomic-devel] Buildah in projectatomic/

[Colin Walters]

On Tue, Mar 14, 2017, at 06:56 PM, Josh Berkus wrote:
> Folks,
> 
> The Buildah project (https://github.com/nalind/buildah) would like to
> move under projectatomic.

Now that we have some CI services maintained by people in this org,
part of this template should be:

 - Do you want to use https://github.com/jlebon/redhat-ci ?  (Which
   we should also consider moving under the org, though blocks on 
https://github.com/jlebon/redhat-ci/issues/29)
 - Do you want to use our Homu instance 
(https://homu-projectatomic-ci.svc.ci.openshift.org/queue/all)
which is configured to do fast-forwards so that you get both tests-on-merge
*and* your "git log" isn't the (IMO) barely legible garbage that results 
from the default
Github PR merge button when you're merging just one (or two) commits.

Re: [atomic-devel] Buildah in projectatomic/

[Micah Abbott]

On 03/14/2017 06:56 PM, Josh Berkus wrote:

Folks,

The Buildah project (https://github.com/nalind/buildah) would like to
move under projectatomic.

I'm in favor of this; not entirely sure why it wasn't there in the first
place.


+1


Objections?