Hello community, here is the log from the commit of package docker-kubic for openSUSE:Factory checked in at 2018-02-01 21:28:29 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/docker-kubic (Old) and /work/SRC/openSUSE:Factory/.docker-kubic.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "docker-kubic" Thu Feb 1 21:28:29 2018 rev:1 rq:570485 version:17.09.1_ce Changes: -------- New Changes file: --- /dev/null 2018-01-05 12:14:39.755488130 +0100 +++ /work/SRC/openSUSE:Factory/.docker-kubic.new/docker-kubic.changes 2018-02-01 21:28:32.443451910 +0100 @@ -0,0 +1,3079 @@ +------------------------------------------------------------------- +Fri Jan 26 07:15:53 UTC 2018 - [email protected] + +- Add patch to support mirroring of private/non-upstream registries. As soon as + the upstream PR (https://github.com/moby/moby/pull/34319) is merged, this + patch will be replaced by the backported one from upstream. + + private-registry-0001-Add-private-registry-mirror-support.patch + fix bsc#1074971 + +------------------------------------------------------------------- +Fri Jan 19 14:12:32 UTC 2018 - [email protected] + +- Add Obsoletes: docker-image-migrator, as the tool is no longer needed and + we've pretty much removed it from everywhere except the containers module. + bsc#1069758 + +------------------------------------------------------------------- +Fri Jan 19 07:48:10 UTC 2018 - [email protected] + +- Remove requirement on bridge-utils, which has been replaced by libnetwork in + Docker. bsc#1072798 + +------------------------------------------------------------------- +Mon Dec 18 12:32:35 UTC 2017 - [email protected] + +- Update to Docker v17.09.1_ce (bsc#1069758). Upstream changelog: + https://github.com/docker/docker-ce/releases/tag/v17.09.1-ce +- Removed patches (merged upstream): + - bsc1045628-0001-devicemapper-remove-container-rootfs-mountPath-after.patch + - bsc1066210-0001-vendor-update-to-github.com-vbatts-tar-split-v0.10.2.patch + - bsc1066801-0001-oci-add-proc-scsi-to-masked-paths.patch + +------------------------------------------------------------------- +Mon Dec 18 12:32:35 UTC 2017 - [email protected] + +- Update to Docker v17.09.0_ce. Upstream changelog: + https://github.com/docker/docker-ce/releases/tag/v17.09.0-ce +- Rebased patches: + * bsc1021227-0001-pkg-devmapper-dynamically-load-dm_task_deferred_remo.patch + * bsc1045628-0001-devicemapper-remove-container-rootfs-mountPath-after.patch + * bsc1055676-0001-daemon-oci-obey-CL_UNPRIVILEGED-for-user-namespaced-.patch + * secrets-0001-daemon-allow-directory-creation-in-run-secrets.patch + * secrets-0002-SUSE-implement-SUSE-container-secrets.patch +- Removed patches (merged upstream): + - bsc1064781-0001-Allow-to-override-build-date.patch + +------------------------------------------------------------------- +Tue Dec 5 10:58:07 UTC 2017 - [email protected] + +- Add a patch to dynamically probe whether libdevmapper supports + dm_task_deferred_remove. This is necessary because we build the containers + module on a SLE12 base, but later SLE versions have libdevmapper support. + This should not affect openSUSE, as all openSUSE versions have a new enough + libdevmapper. Backport of https://github.com/moby/moby/pull/35518. + bsc#1021227 bsc#1029320 bsc#1058173 + + bsc1021227-0001-pkg-devmapper-dynamically-load-dm_task_deferred_remo.patch + +------------------------------------------------------------------- +Mon Dec 4 12:22:29 UTC 2017 - [email protected] + +- Fix up the ordering of tests in docker.spec. This is to keep things easier to + backport into the SLE package. + +------------------------------------------------------------------- +Thu Nov 30 10:15:20 UTC 2017 - [email protected] + +- Include secrets fix to handle "old" containers that have orphaned secret + data. It's not clear why Docker caches these secrets, but fix the problem by + trashing the references manually. bsc#1057743 + * secrets-0002-SUSE-implement-SUSE-container-secrets.patch + +------------------------------------------------------------------- +Thu Nov 23 13:48:08 UTC 2017 - [email protected] + +- Replace references to /var/adm/fillup-templates with new + %_fillupdir macro (boo#1069468) + +------------------------------------------------------------------- +Tue Nov 14 22:39:56 UTC 2017 - [email protected] + +- Remove migration code for the v1.9.x -> v1.10.x migration. This has been + around for a while, and we no longer support migrating from such an old + version "nicely". Docker still has migration code that will run on + first-boot, we are merely removing all of the "nice" warnings which tell + users how to avoid issues during an upgrade that ocurred more than a year + ago. +- Drop un-needed files: + - docker-plugin-message.txt + - docker-update-message.txt + +------------------------------------------------------------------- +Tue Nov 7 16:47:01 UTC 2017 - [email protected] + +- Add a backport of https://github.com/moby/moby/pull/35424, which fixes a + security issue where a maliciously crafted image could be used to crash a + Docker daemon. bsc#1066210 CVE-2017-14992 + + bsc1066210-0001-vendor-update-to-github.com-vbatts-tar-split-v0.10.2.patch + +------------------------------------------------------------------- +Tue Nov 7 09:00:31 UTC 2017 - [email protected] + +- Add a backport of https://github.com/moby/moby/pull/35399, which fixes a + security issue where a Docker container (with a disabled AppArmor profile) + could write to /proc/scsi/... and subsequently DoS the host. bsc#1066801 + CVE-2017-16539 + + bsc1066801-0001-oci-add-proc-scsi-to-masked-paths.patch + +------------------------------------------------------------------- +Tue Oct 24 06:50:29 UTC 2017 - [email protected] + +- Correctly set `docker version` information, including the version, git + commit, and SOURCE_DATE_EPOCH (requires a backport). This should + *effectively* make Docker builds reproducible, with minimal cost. boo#1064781 + + bsc1064781-0001-Allow-to-override-build-date.patch + +------------------------------------------------------------------- +Mon Oct 16 11:06:22 UTC 2017 - [email protected] + +- Add backport of https://github.com/moby/moby/pull/35205. This used to be + fixed in docker-runc, but we're moving it here after upstream discussion. + bsc#1055676 + + bsc1055676-0001-daemon-oci-obey-CL_UNPRIVILEGED-for-user-namespaced-.patch + +------------------------------------------------------------------- +Mon Oct 9 11:36:59 UTC 2017 - [email protected] + +- Update to Docker v17.07.0_ce. Upstream changelog: + https://github.com/docker/docker-ce/releases/tag/v17.06.0-ce + https://github.com/docker/docker-ce/releases/tag/v17.07.0-ce +- Removed no-longer needed patches. + - bsc1037436-0001-client-check-tty-before-creating-exec-job.patch + - bsc1037607-0001-apparmor-make-pkg-aaparser-work-on-read-only-root.patch + - integration-cli-fix-TestInfoEnsureSucceeds.patch +- Added backport of https://github.com/moby/moby/pull/34573. bsc#1045628 + + bsc1045628-0001-devicemapper-remove-container-rootfs-mountPath-after.patch +- Rewrite secrets patches to correctly handle directories in a way that doesn't + cause errors when starting new containers. + * secrets-0001-daemon-allow-directory-creation-in-run-secrets.patch + * secrets-0002-SUSE-implement-SUSE-container-secrets.patch + +------------------------------------------------------------------- +Mon Oct 2 08:12:17 UTC 2017 - [email protected] + +- Fix bsc#1059011 + + The systemd service helper script used a timeout of 60 seconds to + start the daemon, which is insufficient in cases where the daemon + takes longer to start. Instead, set the service type from 'simple' to + 'notify' and remove the now superfluous helper script. + +------------------------------------------------------------------- +Wed Sep 27 15:04:19 UTC 2017 - [email protected] + +- fix bsc#1057743: Add a Requires: fix_bsc_1057743 which is provided by the + newer version of docker-libnetwork. This is necessary because of a versioning + bug we found in bsc#1057743. + +------------------------------------------------------------------- +Fri Sep 15 15:32:49 UTC 2017 - [email protected] + +- fix /var/adm/update-message/docker file name to be + /var/adm/update-message/docker-%{version}-%{release} + +------------------------------------------------------------------- +Wed Sep 6 11:42:31 UTC 2017 - [email protected] + +- devicemapper: add patch to make the dm storage driver remove a container's + rootfs mountpoint before attempting to do libdm operations on it. This helps + avoid complications when live mounts will leak into containers. Backport of + https://github.com/moby/moby/pull/34573. bsc#1045628 + + bsc1045628-0001-devicemapper-remove-container-rootfs-mountPath-after.patch + +------------------------------------------------------------------- +Wed Aug 30 14:58:52 UTC 2017 - [email protected] + +- Fix a regression in our SUSE secrets patches, which caused the copied files + to not carry the correct {uid,gid} mapping when using user namespaces. This + would not cause any bugs (SUSEConnect does the right thing anyway) but it's + possible some programs would not treat the files correctly. This is + tangentially related to bsc#1055676. + * secrets-0001-daemon-allow-directory-creation-in-run-secrets.patch + * secrets-0002-SUSE-implement-SUSE-container-secrets.patch + +------------------------------------------------------------------- +Wed Aug 2 13:37:16 UTC 2017 - [email protected] + +- Use -buildmode=pie for tests and binary build. bsc#1048046 bsc#1051429 + +------------------------------------------------------------------- +Wed Jul 19 18:12:26 UTC 2017 - [email protected] + +- enable deferred removal for sle12sp2 and newer (and openSUSE + equivalent. fix bsc#1021227 + +------------------------------------------------------------------- +Wed Jul 19 17:17:04 UTC 2017 - [email protected] + ++++ 2882 more lines (skipped) ++++ between /dev/null ++++ and /work/SRC/openSUSE:Factory/.docker-kubic.new/docker-kubic.changes New: ---- 80-docker.rules README_SUSE.md _service bsc1021227-0001-pkg-devmapper-dynamically-load-dm_task_deferred_remo.patch bsc1055676-0001-daemon-oci-obey-CL_UNPRIVILEGED-for-user-namespaced-.patch docker-audit.rules docker-kubic-17.09.1_ce.tar.xz docker-kubic-rpmlintrc docker-kubic.changes docker-kubic.spec docker.service private-registry-0001-Add-private-registry-mirror-support.patch secrets-0001-daemon-allow-directory-creation-in-run-secrets.patch secrets-0002-SUSE-implement-SUSE-container-secrets.patch sysconfig.docker tests.sh ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ docker-kubic.spec ++++++ # # spec file for package docker-kubic # # Copyright (c) 2018 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed # upon. The license for this file, and modifications and additions to the # file, is the same license as for the pristine package itself (unless the # license for the pristine package is not an Open Source License, in which # case the license is the MIT License). An "Open Source License" is a # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. # Please submit bugfixes or comments via http://bugs.opensuse.org/ # # nodebuginfo # Where important update information will be stored, such that an administrator # is guaranteed to see the relevant warning. %define update_messages %{_localstatedir}/adm/update-messages/%{name}-%{version}-%{release} #Compat macro for new _fillupdir macro introduced in Nov 2017 %if ! %{defined _fillupdir} %define _fillupdir /var/adm/fillup-templates %endif # Used when generating the "build" information for Docker version. The value of # git_commit_epoch is unused here (we use SOURCE_DATE_EPOCH, which rpm # helpfully injects into our build environment from the changelog). If you want # to generate a new git_commit_epoch, use this: # $ date --date="$(git show --format=fuller --date=iso $COMMIT_ID | grep -oP '(?<=^CommitDate: ).*')" '+%s' %define git_version f4ffd2511ce9 %define git_commit_epoch 1508606827 # These are the git commits required. We verify them against the source to make # sure we didn't miss anything important when doing upgrades. %define required_containerd 06b9cb35161009dcb7123345749fef02f7cea8e0 %define required_dockerrunc 3f2f8b84a77f73d38244dd690525642a72156c64 %define required_libnetwork 7b2b1feb1de4817d522cc372af149ff48d25028e %define realname docker Name: %{realname}-kubic Version: 17.09.1_ce Release: 0 Summary: The Linux container runtime License: Apache-2.0 Group: System/Management Url: http://www.docker.io # TODO(VR): check those SOURCE files below Source: %{name}-%{version}.tar.xz Source1: docker.service Source3: 80-docker.rules Source4: sysconfig.docker Source6: %{name}-rpmlintrc Source7: README_SUSE.md Source8: docker-audit.rules Source9: tests.sh # SUSE-FEATURE: Adds the /run/secrets mountpoint inside all Docker containers # which is not snapshotted when images are committed. Note that if you modify # this patch, please also modify the patch in the suse-secrets-v<version> # branch in http://github.com/suse/docker.mirror. Patch200: secrets-0001-daemon-allow-directory-creation-in-run-secrets.patch Patch201: secrets-0002-SUSE-implement-SUSE-container-secrets.patch # SUSE-BACKPORT: Backport of https://github.com/moby/moby/pull/35205. bsc#1055676 Patch400: bsc1055676-0001-daemon-oci-obey-CL_UNPRIVILEGED-for-user-namespaced-.patch # SUSE-BACKPORT: Backport of https://github.com/moby/moby/pull/35518. bsc#1021227 bsc#1029320 bsc#1058173 Patch401: bsc1021227-0001-pkg-devmapper-dynamically-load-dm_task_deferred_remo.patch # SUSE-FEATURE: Add support to mirror inofficial/private registries # (https://github.com/moby/moby/pull/34319) Patch500: private-registry-0001-Add-private-registry-mirror-support.patch BuildRequires: audit BuildRequires: bash-completion BuildRequires: ca-certificates BuildRequires: device-mapper-devel >= 1.2.68 BuildRequires: glibc-devel-static BuildRequires: libapparmor-devel BuildRequires: libbtrfs-devel >= 3.8 # enable libseccomp for sle >= sle12sp2 %if 0%{?sle_version} >= 120200 %define with_libseccomp 1 %endif # enable libseccomp for leap >= 42.2 %if 0%{?leap_version} >= 420200 %define with_libseccomp 1 %endif # enable libseccomp for Factory %if 0%{?suse_version} > 1320 %define with_libseccomp 1 %endif %if 0%{?with_libseccomp} BuildRequires: libseccomp-devel %endif BuildRequires: libtool BuildRequires: procps BuildRequires: sqlite3-devel BuildRequires: systemd-devel BuildRequires: zsh Requires: apparmor-parser Requires: ca-certificates-mozilla # Required in order for networking to work. fix_bsc_1057743 is a work-around # for some old packaging issues (where rpm would delete a binary that was # installed by docker-libnetwork). See bsc#1057743 for more details. Requires: docker-libnetwork-kubic-git = %{required_libnetwork} Requires: fix_bsc_1057743 # Containerd and runC are required as they are the only currently supported # execdrivers of Docker. NOTE: The version pinning here matches upstream's # vendor.conf to ensure that we don't use a slightly incompatible version of # runC or containerd (which would be bad). Requires: containerd-kubic-git = %{required_containerd} Requires: docker-runc-kubic-git = %{required_dockerrunc} # Provides mkfs.ext4 - used by Docker when devicemapper storage driver is used Requires: e2fsprogs Requires: git-core >= 1.7 Requires: iproute2 >= 3.5 Requires: iptables >= 1.4 Requires: procps Requires: tar >= 1.26 Requires: xz >= 4.9 Requires(post): %fillup_prereq Requires(post): udev Requires(post): shadow # Obsolete old packege without the -kubic suffix Obsoletes: docker_1_12_6 # Conflict with non-kubic package Conflicts: docker # We used to have a migration tool for the upgrade from v1.9.x to v1.10.x. # It is no longer useful, so we obsolete it. bsc#1069758 Obsoletes: docker-image-migrator # Not necessary, but must be installed when the underlying system is # configured to use lvm and the user doesn't explicitly provide a # different storage-driver than devicemapper Recommends: lvm2 >= 2.2.89 Conflicts: lxc < 1.0 BuildRoot: %{_tmppath}/%{name}-%{version}-build ExcludeArch: s390 ppc # Make sure we build with go 1.8 BuildRequires: go-go-md2man BuildRequires: golang(API) = 1.8 %description Docker complements LXC with a high-level API which operates at the process level. It runs unix processes with strong guarantees of isolation and repeatability across servers. Docker is a great building block for automating distributed systems: large-scale web deployments, database clusters, continuous deployment systems, private PaaS, service-oriented architectures, etc. %package bash-completion Summary: Bash Completion for %{realname} Group: System/Management Requires: %{name} = %{version} Supplements: packageand(docker:bash-completion) BuildArch: noarch # Obsolete old packege without the -kubic suffix Obsoletes: docker-bash-completion = 1.12.6 # Conflict with non-kubic package Conflicts: docker-bash-completion > 1.12.6 %description bash-completion Bash command line completion support for %{realname}. %package zsh-completion Summary: Zsh Completion for %{realname} Group: System/Management Requires: %{name} = %{version} Supplements: packageand(docker:zsh) BuildArch: noarch # Obsolete old packege without the -kubic suffix Obsoletes: docker-zsh-completion = 1.12.6 # Conflict with non-kubic package Conflicts: docker-zsh-completion > 1.12.6 %description zsh-completion Zsh command line completion support for %{realname}. %package test %global __requires_exclude ^libgo.so.*$ Summary: Test package for docker Group: System/Management BuildRequires: fdupes Requires: apparmor-parser Requires: bash-completion Requires: device-mapper-devel >= 1.2.68 Requires: glibc-devel-static Requires: libapparmor-devel Requires: libbtrfs-devel >= 3.8 Requires: procps Requires: sqlite3-devel Requires: golang(API) = 1.8 # Obsolete old packege without the -kubic suffix Obsoletes: docker-test = 1.12.6 # Conflict with non-kubic package Conflicts: docker-test > 1.12.6 %description test Test package for docker. It contains the source code and the tests. %prep %setup -q %if 0%{?is_opensuse} # nothing %else # PATCH-SUSE: Secrets patches. %patch200 -p1 -d components/engine %patch201 -p1 -d components/engine %endif # bsc#1055676 %patch400 -p1 -d components/engine # bsc#1021227 bsc#1029320 bsc#1058173 %patch401 -p1 -d components/engine # PATCH-SUSE: Mirror patch. %patch500 -p1 -d components/engine cp %{SOURCE7} . cp %{SOURCE9} . %build BUILDTAGS="exclude_graphdriver_aufs apparmor selinux pkcs11" %if 0%{?with_libseccomp} BUILDTAGS="seccomp $BUILDTAGS" %endif %if 0%{?sle_version} == 120000 # Provided by patch406, to allow us to build with older distros but still # have deferred removal support at runtime. We only use this when building # on SLE12. BUILDTAGS="libdm_dlsym_deferred_remove $BUILDTAGS" %endif (cat <<EOF export AUTO_GOPATH=1 export DOCKER_BUILDTAGS="$BUILDTAGS" # Until boo#1038493 is fixed properly we need to do this hack to get the # compiled-into-the-binary GOROOT. export GOROOT="$(GOROOT= go env GOROOT)" # Make sure we always build PIC code. bsc#1048046 export BUILDFLAGS="-buildmode=pie" # Specify all of the versioning information. We use SOURCE_DATE_EPOCH if it's # been injected by rpmbuild, otherwise we use the hardcoded git_commit_epoch # generated above. boo#1064781 export VERSION="$(cat ./VERSION 2>/dev/null || echo '%{version}')" export DOCKER_GITCOMMIT="%{git_version}" export GITCOMMIT="%{git_version}" export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-%{git_commit_epoch}}" export BUILDTIME="$(date -u -d "@$SOURCE_DATE_EPOCH" --rfc-3339 ns 2>/dev/null | sed -e 's/ /T/')" EOF ) > docker_build_env . ./docker_build_env # Preparing GOPATH so that the client is visible to the compiler mkdir -p src/github.com/docker/ ln -s $(pwd)/components/cli $(pwd)/src/github.com/docker/cli GOPATH=$GOPATH:$(pwd) # DOCKER ENGINE cd components/engine/ # ignore the warning that we compile outside a Docker container ./hack/make.sh dynbinary # build the tests binary GOPATH=$(pwd)/vendor:$(pwd)/.gopath/ go test \ -buildmode=pie \ -tags "$DOCKER_BUILDTAGS daemon autogen" \ -c github.com/docker/docker/integration-cli -o tests.main cd ../.. # DOCKER CLIENT cd components/cli ./scripts/build/dynbinary ./man/md2man-all.sh cd ../.. %check . ./docker_build_env # DOCKER ENGINE # go test will look in github.com/docker/docker/vendor for vendored packages but # Docker keeps them in github.com/docker/docker/vendor/src. Let's do it like # Docker does it and append github.com/docker/docker/vendor to the GOPATH so the # packages are found by go test. export GOPATH=$HOME/go/src/github.com/docker/docker/vendor:$GOPATH # Create or dir if it doesn't exist already mkdir -p $HOME/go/src/github.com/docker # Remove any existing symlinks. rm -rf $HOME/go/src/github.com/docker/* # go list -e ... doesn't seem to work with symlinks so do a full copy instead. cp -ar %{buildroot}/usr/src/docker/engine $HOME/go/src/github.com/docker/docker cd $HOME/go/src/github.com/docker/docker # We verify that all of our -git requires are correct. This is done on-build to # make sure that someone doing an update didn't miss anything. grep 'RUNC_COMMIT=%{required_dockerrunc}' hack/dockerfile/binaries-commits grep 'CONTAINERD_COMMIT=%{required_containerd}' hack/dockerfile/binaries-commits grep 'LIBNETWORK_COMMIT=%{required_libnetwork}' hack/dockerfile/binaries-commits # The command is taken from hack/make/test-unit and various test runs. # Everything that follows github.com/docker/pkg/integration-cli are packages # containing tests that cannot run in an obs build context. Some tests must be # excluded as they will always fail in our build environments. PKG_LIST=$(go list -e \ -f '{{if ne .Name "github.com/docker/docker"}} {{.ImportPath}} {{end}}' \ -tags "$DOCKER_BUILDTAGS" \ -a "${BUILDFLAGS[@]}" ... \ | grep 'github.com/docker/docker' \ | grep -Ev 'vendor/(.+/)?github.com/docker/docker' \ | grep -v 'github.com/docker/docker/vendor' \ | grep -v 'github.com/docker/docker/builder$' \ | grep -v 'github.com/docker/docker/builder/dockerfile$' \ | grep -v 'github.com/docker/docker/builder/dockerfile/parser$' \ | grep -v 'github.com/docker/docker/builder/remotecontext' \ | grep -v 'github.com/docker/docker/cmd/dockerd$' \ %ifarch s390x | grep -v 'github.com/docker/docker/container' \ %endif | grep -v 'github.com/docker/docker/daemon$' \ | grep -v 'github.com/docker/docker/daemon/graphdriver' \ | grep -Pv 'github.com/docker/docker/daemon/logger(?!/gelf)' \ | grep -v 'github.com/docker/docker/integration' \ | grep -v 'github.com/docker/docker/integration-cli' \ | grep -v 'github.com/docker/docker/man$' \ | grep -v 'github.com/docker/docker/pkg/archive$' \ | grep -v 'github.com/docker/docker/pkg/chrootarchive$' \ %if 0%{?sle_version} == 120000 | grep -v 'github.com/docker/docker/pkg/devicemapper$' \ %endif | grep -v 'github.com/docker/docker/pkg/gitutils$' \ | grep -v 'github.com/docker/docker/pkg/idtools$' \ | grep -v 'github.com/docker/docker/pkg/integration$' \ | grep -v 'github.com/docker/docker/pkg/jsonlog$' \ | grep -v 'github.com/docker/docker/pkg/mount$' \ | grep -v 'github.com/docker/docker/pkg/sysinfo$' \ | grep -v 'github.com/docker/docker/pkg/testutil' \ %if ! 0%{?with_libseccomp} | grep -v 'github.com/docker/docker/profiles/seccomp$' \ %endif | grep -v 'github.com/docker/docker/registry$' \ | grep -v 'github.com/docker/docker/volume/local$' \ ) # PLEASE KEEP THIS LIST IN ALPHABETICAL ORDER! rm ./pkg/system/rm_test.go go test -buildmode=pie -cover -ldflags -w -tags "$DOCKER_BUILDTAGS" -a -test.timeout=10m $PKG_LIST # DOCKER CLIENT find $(go env GOROOT) -type d cp -ar %{buildroot}/usr/src/docker/cli $HOME/go/src/github.com/docker/cli cd $HOME/go/src/github.com/docker/cli PKG_LIST=$(go list ./... \ | grep 'github.com/docker/cli' \ | grep -v 'github.com/docker/cli/vendor' \ | grep -Ev 'vendor/(.+/)?github.com/docker/cli' \ | grep -v 'github.com/docker/cli/cli/command/idresolver' \ | grep -v 'github.com/docker/cli/cli/command/image' \ | grep -v 'github.com/docker/cli/cli/image' \ | grep -v 'github.com/docker/cli/cmd/docker' \ | grep -v 'github.com/docker/cli/e2e' \ | grep -v 'github.com/docker/cli/cli/image' \ ) # PLEASE KEEP THIS LIST IN ALPHABETICAL ORDER! # We cannot use -buildmode=pie here becaue (for some reason) 'go test' will # produce really odd errors about packages missing (this only happens if we # have a lot of packages in the cmdline). So just avoid running these tests if # we're on ppc64le (which requires -buildmode=pie). %ifnarch ppc64le go test -cover -ldflags -w -tags "$DOCKER_BUILDTAGS" -a -test.timeout=10m $PKG_LIST %endif %install install -d %{buildroot}%{go_contribdir} install -d %{buildroot}%{_bindir} install -D -m755 components/cli/build/docker %{buildroot}/%{_bindir}/docker install -D -m755 components/engine/bundles/latest/dynbinary-daemon/dockerd %{buildroot}/%{_bindir}/dockerd install -d %{buildroot}/%{_localstatedir}/lib/docker install -Dd -m 0755 \ %{buildroot}%{_sysconfdir}/init.d \ %{buildroot}%{_sbindir} install -D -m0644 components/cli/contrib/completion/bash/docker "%{buildroot}%{_sysconfdir}/bash_completion.d/%{realname}" install -D -m0644 components/cli/contrib/completion/zsh/_docker "%{buildroot}%{_sysconfdir}/zsh_completion.d/%{realname}" # copy all for the test package install -d %{buildroot}%{_prefix}/src/docker/ cp -a components/engine/. %{buildroot}%{_prefix}/src/docker/engine cp -a components/cli/. %{buildroot}%{_prefix}/src/docker/cli # # systemd service # install -D -m 0644 %{SOURCE1} %{buildroot}%{_unitdir}/%{realname}.service ln -sf service %{buildroot}%{_sbindir}/rcdocker # # udev rules that prevents dolphin to show all docker devices and slows down # upstream report https://bugs.kde.org/show_bug.cgi?id=329930 # install -D -m 0644 %{SOURCE3} %{buildroot}%{_udevrulesdir}/80-%{realname}.rules # audit rules install -D -m 0640 %{SOURCE8} %{buildroot}%{_sysconfdir}/audit/rules.d/%{realname}.rules # sysconfig file install -D -m 644 %{SOURCE4} %{buildroot}%{_fillupdir}/sysconfig.docker # install manpages (using the ones from the engine) install -d %{buildroot}%{_mandir}/man1 install -p -m 644 components/cli/man/man1/*.1 %{buildroot}%{_mandir}/man1 install -d %{buildroot}%{_mandir}/man5 install -p -m 644 components/cli/man/man5/Dockerfile.5 %{buildroot}%{_mandir}/man5 install -d %{buildroot}%{_mandir}/man8 install -p -m 644 components/cli/man/man8/*.8 %{buildroot}%{_mandir}/man8 %fdupes %{buildroot} %pre getent group docker >/dev/null || groupadd -r docker %service_add_pre %{realname}.service %post %service_add_post %{realname}.service %{fillup_only -n docker} %preun %service_del_preun %{realname}.service %postun %service_del_postun %{realname}.service %files %defattr(-,root,root) %doc components/engine/README.md components/engine/LICENSE README_SUSE.md CHANGELOG.md %{_bindir}/docker %{_bindir}/dockerd %{_sbindir}/rcdocker %{_unitdir}/%{realname}.service %config %{_sysconfdir}/audit/rules.d/%{realname}.rules %{_udevrulesdir}/80-%{realname}.rules %{_fillupdir}/sysconfig.docker %dir %{_localstatedir}/lib/docker/ %{_mandir}/man1/docker-*.1%{ext_man} %{_mandir}/man1/docker.1%{ext_man} %{_mandir}/man5/Dockerfile.5%{ext_man} %{_mandir}/man8/dockerd.8%{ext_man} %files bash-completion %defattr(-,root,root) %config %{_sysconfdir}/bash_completion.d/%{realname} %files zsh-completion %defattr(-,root,root) %config %{_sysconfdir}/zsh_completion.d/%{realname} %files test %defattr(-,root,root) %{_prefix}/src/docker/ # exclude binaries %exclude %{_prefix}/src/docker/engine/bundles/ %exclude %{_prefix}/src/docker/cli/build/ # exclude init configurations other than systemd %exclude %{_prefix}/src/docker/engine/contrib/init/openrc %exclude %{_prefix}/src/docker/engine/contrib/init/sysvinit-debian %exclude %{_prefix}/src/docker/engine/contrib/init/sysvinit-redhat %exclude %{_prefix}/src/docker/engine/contrib/init/upstart %changelog ++++++ 80-docker.rules ++++++ # hide docker's loopback devices from udisks, and thus from user desktops SUBSYSTEM=="block", ENV{DM_NAME}=="docker-*", ENV{UDISKS_PRESENTATION_HIDE}="1", ENV{UDISKS_IGNORE}="1" SUBSYSTEM=="block", DEVPATH=="/devices/virtual/block/loop*", ATTR{loop/backing_file}=="/var/lib/docker/*", ENV{UDISKS_PRESENTATION_HIDE}="1", ENV{UDISKS_IGNORE}="1" ++++++ README_SUSE.md ++++++ # Abstract Docker is a lightweight "virtualization" method to run multiple virtual units (containers, akin to “chroot”) simultaneously on a single control host. Containers are isolated with Kernel Control Groups (cgroups) and Kernel Namespaces. Docker provides an operating system-level virtualization where the Kernel controls the isolated containers. With other full virtualization solutions like Xen, KVM, or libvirt the processor simulates a complete hardware environment and controls its virtual machines. # Terminology ## chroot A change root (chroot, or change root jail) is a section in the file system which is isolated from the rest of the file system. For this purpose, the chroot command is used to change the root of the file system. A program which is executed in such a “chroot jail” cannot access files outside the designated directory tree. ## cgroups Kernel Control Groups (commonly referred to as just “cgroups”) are a Kernel feature that allows aggregating or partitioning tasks (processes) and all their children into hierarchical organized groups to isolate resources. ## Image A "virtual machine" on the host server that can run any Linux system, for example openSUSE, SUSE Linux Enterprise Desktop, or SUSE Linux Enterprise Server. A Docker image is made by a series of layers built one over the other. Each layer corresponds to a permanent change committed from a container to the image. For more details checkout [Docker's official documentation](http://docs.docker.com/terms/image/). ## Image Name A name that refers to an image. The name is used by the docker commands. ## Container A running Docker Image. ## Container ID A ID that refers to a container. The ID is used by the docker commands. ## TAG A string associated to a Image. It commonly used to identify a specific version of a Image (like tags in version control systems). It is also possible to refer the same Image with different TAGs. ## Kernel Namespaces A Kernel feature to isolate some resources like network, users, and others for a group of processes. ## Docker Host Server The system that runs the Docker daemon, provides the images, and the management control capabilities through cgroups. # Overview Docker is a platform that allows developers and sysadmins to manage the complete lifecycle of images. Docker makes incredibly easy to build, ship and run images containing applications. Benefits of Docker: * Isolating applications and operating systems through containers. * Providing nearly native performance as Docker manages allocation of resources in real-time. * Controlling network interfaces and applying resources inside containers through cgroups. * Versioning of images. * Building images based on existing ones. * Sharining/storing on [public](http://docs.docker.com/docker-hub/) or [private](http://docs.docker.com/userguide/dockerrepos/#private-repositories) repositories. Limitations of Docker: * All Docker containers are running inside the host system's Kernel and not with a different Kernel. * Only allows Linux "guest" operating systems. * Docker is not a full virtualization stack like Xen, KVM, or libvirt. * Security depends on the host system. Refer to the [official documentation](http://docs.docker.com/articles/security/) for more details. ## Container drivers Docker has different backend drivers to handle the containers. The recommended on is [libcontainer](https://github.com/docker/libcontainer), which is also the default choice. This driver provides direct access with cgroups. The Docker packages ships also a LXC driver which handles containers using the LXC tools. At the time of writing, upstream is working on a `libvirt-lxc` driver. ## Storage drivers Docker supports different storage drivers: * `vfs`: this driver is automatically used when the Docker host filesystem does not support copy-on-write. This is a simple driver which does not offer some of the advantages of Docker (like sharing layers, more on that in the next sections). It is highly reliable but also slow. * `devicemapper`: this driver relies on the device-mapper thin provisioning module. It supports copy-on-write, hence it offers all the advantages of Docker. * `btrfs`: this driver relies on Btrfs to provide all the features required by Docker. To use this driver the `/var/lib/docker` directory must be on a btrfs filesystem. * `AUFS`: this driver relies on AUFS union filesystem. Neither the upstream kernel nor the SUSE one supports this filesystem. Hence the AUFS driver is not built into the SUSE Docker package. It is possible to specify which driver to use by changing the value of the `DOCKER_OPTS` variable defined inside of the `/etc/sysconfig/docker` file. This can be done either manually or using &yast; by browsing to: * System * /etc/sysconfig Editor * System * Management * DOCKER_OPTS menu and entering the `-s storage_driver` string. For example, to force the usage of the `devicemapper` driver enter the following text: ``` DOCKER_OPTS="-s devicemapper ``` It is recommended to have `/var/lib/docker` mounted on a different filesystem to not affect the Docker host OS in case of a filesystem corruption. # Setting up a Docker host Prepare the host: 1. Install the `docker` package. 2. Automatically start the Docker daemon at boot: `sudo systemctl enable docker` 3. Start the Docker daemon: `sudo systemctl start docker` The Docker daemon listens on a local socket which is accessible only by the `root` user and by the members of the `docker` group. The `docker` group is automatically created at package installation time. To allow a certain user to connect to the local Docker daemon use the following command: ``` sudo /usr/sbin/usermod -aG docker <username> ``` The user will be able to communicate with the local Docker daemon upon his next login. ## Networking If you want your containers to be able to access the external network you must enable the `net.ipv4.ip_forward` rule. This can be done using YaST by browsing to the `Network Devices -> Network Settings -> Routing` menu and ensuring that the `Enable IPv4 Forwarding` box is checked. This option cannot be changed when networking is handled by the Network Manager. In such cases the `/etc/sysconfig/SuSEfirewall2` file needs to be edited by hand to ensure the `FW_ROUTE` flag is set to `yes` like so: ``` FW_ROUTE="yes" ``` # Basic Docker operations Images can be pulled from [Docker's central index](http://index.docker.io) using the following command: ``` docker pull <image name> ``` Containers can be started using the `docker run` command. Please refer to the [official documentation](http://docs.docker.com/) for more details. # Building Docker containers using KIWI Starting from version 5.06.8 KIWI can be used to build Docker images. Please refer to KIWI's [official documentation](https://doc.opensuse.org/projects/kiwi/doc/#chap.lxc). The official `kiwi-doc` package contains examples of Docker images. ## Docker build system versus KIWI Docker has an [internal build system](http://docs.docker.com/reference/builder/) which makes incredibly easy to create new images based on existing ones. Some users might be confused about what to use. The right approach is to build the [base images](http://docs.docker.com/terms/image/#base-image-def) using KIWI and then use them as foundation blocks inside of your Docker's build system. That two advantages: 1. Be able to use docker specific directives (like `ENTRYPOINT`, `EXPOSE`, ...). 2. Be able to reuse already existing layers. Sharing the common layers between different images makes possible to: * Use less disk space on the Docker hosts. * Make the deployments faster: only the requested layers are sent over the network (it is like upgrading installed packages using delta rpms). * Take full advantage of caching while building Docker images: this will result in faster executions of `docker build` command. To recap: KIWI is not to be intended as a replacement for Docker's build system. It rather complements with it. ++++++ _service ++++++ <services> <service name="tar_scm" mode="disabled"> <param name="url">https://github.com/docker/docker-ce.git</param> <param name="scm">git</param> <param name="exclude">.git</param> <param name="versionformat">17.09.1_ce</param> <param name="revision">v17.09.1-ce</param> <param name="filename">docker-kubic</param> </service> <service name="recompress" mode="disabled"> <param name="file">docker-*.tar</param> <param name="compression">xz</param> </service> <service name="set_version" mode="disabled"> <param name="basename">docker</param> </service> </services> ++++++ bsc1021227-0001-pkg-devmapper-dynamically-load-dm_task_deferred_remo.patch ++++++ >From b492588a54b8efa1fba1de700cb3e0ad3fe665d9 Mon Sep 17 00:00:00 2001 From: Aleksa Sarai <[email protected]> Date: Thu, 16 Nov 2017 17:09:16 +1100 Subject: [PATCH] pkg: devmapper: dynamically load dm_task_deferred_remove dm_task_deferred_remove is not supported by all distributions, due to out-dated versions of devicemapper. However, in the case where the devicemapper library was updated without rebuilding Docker (which can happen in some distributions) then we should attempt to dynamically load the relevant object rather than try to link to it. This can only be done if Docker was built dynamically, for obvious reasons. In order to avoid having issues arise when dlsym(3) was unnecessary, gate the whole dlsym(3) logic behind a buildflag that we disable by default (libdm_dlsym_deferred_remove). SUSE-Bugs: bsc#1021227 bsc#1029320 bsc#1058173 Signed-off-by: Aleksa Sarai <[email protected]> --- hack/make.sh | 12 +- ...> devmapper_wrapper_dynamic_deferred_remove.go} | 10 +- ...mapper_wrapper_dynamic_dlsym_deferred_remove.go | 128 +++++++++++++++++++++ .../devmapper_wrapper_no_deferred_remove.go | 6 +- 4 files changed, 149 insertions(+), 7 deletions(-) rename pkg/devicemapper/{devmapper_wrapper_deferred_remove.go => devmapper_wrapper_dynamic_deferred_remove.go} (78%) create mode 100644 pkg/devicemapper/devmapper_wrapper_dynamic_dlsym_deferred_remove.go diff --git a/hack/make.sh b/hack/make.sh index bc18c066b66c..6e94824ad557 100755 --- a/hack/make.sh +++ b/hack/make.sh @@ -112,6 +112,12 @@ if [ ! "$GOPATH" ]; then exit 1 fi +# Adds $1_$2 to DOCKER_BUILDTAGS unless it already +# contains a word starting from $1_ +add_buildtag() { + [[ " $DOCKER_BUILDTAGS" == *" $1_"* ]] || DOCKER_BUILDTAGS+=" $1_$2" +} + if ${PKG_CONFIG} 'libsystemd >= 209' 2> /dev/null ; then DOCKER_BUILDTAGS+=" journald" elif ${PKG_CONFIG} 'libsystemd-journal' 2> /dev/null ; then @@ -127,12 +133,14 @@ if \ fi # test whether "libdevmapper.h" is new enough to support deferred remove -# functionality. +# functionality. We favour libdm_dlsym_deferred_remove over +# libdm_no_deferred_remove in dynamic cases because the binary could be shipped +# with a newer libdevmapper than the one it was built wih. if \ command -v gcc &> /dev/null \ && ! ( echo -e '#include <libdevmapper.h>\nint main() { dm_task_deferred_remove(NULL); }'| gcc -xc - -o /dev/null $(pkg-config --libs devmapper) &> /dev/null ) \ ; then - DOCKER_BUILDTAGS+=' libdm_no_deferred_remove' + add_buildtag libdm dlsym_deferred_remove fi # Use these flags when compiling the tests and final binary diff --git a/pkg/devicemapper/devmapper_wrapper_deferred_remove.go b/pkg/devicemapper/devmapper_wrapper_dynamic_deferred_remove.go similarity index 78% rename from pkg/devicemapper/devmapper_wrapper_deferred_remove.go rename to pkg/devicemapper/devmapper_wrapper_dynamic_deferred_remove.go index 7f793c270868..bf57371ff4cf 100644 --- a/pkg/devicemapper/devmapper_wrapper_deferred_remove.go +++ b/pkg/devicemapper/devmapper_wrapper_dynamic_deferred_remove.go @@ -1,11 +1,15 @@ -// +build linux,cgo,!libdm_no_deferred_remove +// +build linux,cgo,!static_build +// +build !libdm_dlsym_deferred_remove,!libdm_no_deferred_remove package devicemapper -// #include <libdevmapper.h> +/* +#include <libdevmapper.h> +*/ import "C" -// LibraryDeferredRemovalSupport tells if the feature is enabled in the build +// LibraryDeferredRemovalSupport tells if the feature is supported by the +// current Docker invocation. const LibraryDeferredRemovalSupport = true func dmTaskDeferredRemoveFct(task *cdmTask) int { diff --git a/pkg/devicemapper/devmapper_wrapper_dynamic_dlsym_deferred_remove.go b/pkg/devicemapper/devmapper_wrapper_dynamic_dlsym_deferred_remove.go new file mode 100644 index 000000000000..5dfb369f1ff8 --- /dev/null +++ b/pkg/devicemapper/devmapper_wrapper_dynamic_dlsym_deferred_remove.go @@ -0,0 +1,128 @@ +// +build linux,cgo,!static_build +// +build libdm_dlsym_deferred_remove,!libdm_no_deferred_remove + +package devicemapper + +/* +#cgo LDFLAGS: -ldl +#include <stdlib.h> +#include <dlfcn.h> +#include <libdevmapper.h> + +// Yes, I know this looks scary. In order to be able to fill our own internal +// dm_info with deferred_remove we need to have a struct definition that is +// correct (regardless of the version of libdm that was used to compile it). To +// this end, we define struct_backport_dm_info. This code comes from lvm2, and +// I have verified that the structure has only ever had elements *appended* to +// it (since 2001). +// +// It is also important that this structure be _larger_ than the dm_info that +// libdevmapper expected. Otherwise libdm might try to write to memory it +// shouldn't (they don't have a "known size" API). +struct backport_dm_info { + int exists; + int suspended; + int live_table; + int inactive_table; + int32_t open_count; + uint32_t event_nr; + uint32_t major; + uint32_t minor; + int read_only; + + int32_t target_count; + + int deferred_remove; + int internal_suspend; + + // Padding, purely for our own safety. This is to avoid cases where libdm + // was updated underneath us and we call into dm_task_get_info() with too + // small of a buffer. + char _[512]; +}; + +// We have to wrap this in CGo, because Go really doesn't like function pointers. +int call_dm_task_deferred_remove(void *fn, struct dm_task *task) +{ + int (*_dm_task_deferred_remove)(struct dm_task *task) = fn; + return _dm_task_deferred_remove(task); +} +*/ +import "C" + +import ( + "unsafe" + + "github.com/sirupsen/logrus" +) + +// dm_task_deferred_remove is not supported by all distributions, due to +// out-dated versions of devicemapper. However, in the case where the +// devicemapper library was updated without rebuilding Docker (which can happen +// in some distributions) then we should attempt to dynamically load the +// relevant object rather than try to link to it. + +// dmTaskDeferredRemoveFct is a "bound" version of dm_task_deferred_remove. +// It is nil if dm_task_deferred_remove was not found in the libdevmapper that +// is currently loaded. +var dmTaskDeferredRemovePtr unsafe.Pointer + +// LibraryDeferredRemovalSupport tells if the feature is supported by the +// current Docker invocation. This value is fixed during init. +var LibraryDeferredRemovalSupport bool + +func init() { + // Clear any errors. + var err *C.char + C.dlerror() + + // The symbol we want to fetch. + symName := C.CString("dm_task_deferred_remove") + defer C.free(unsafe.Pointer(symName)) + + // See if we can find dm_task_deferred_remove. Since we already are linked + // to libdevmapper, we can search our own address space (rather than trying + // to guess what libdevmapper is called). We use NULL here, as RTLD_DEFAULT + // is not available in CGO (even if you set _GNU_SOURCE for some reason). + // The semantics are identical on glibc. + sym := C.dlsym(nil, symName) + err = C.dlerror() + if err != nil { + logrus.Debugf("devmapper: could not load dm_task_deferred_remove: %s", C.GoString(err)) + return + } + + logrus.Debugf("devmapper: found dm_task_deferred_remove at %x", uintptr(sym)) + dmTaskDeferredRemovePtr = sym + LibraryDeferredRemovalSupport = true +} + +func dmTaskDeferredRemoveFct(task *cdmTask) int { + sym := dmTaskDeferredRemovePtr + if sym == nil || !LibraryDeferredRemovalSupport { + return -1 + } + return int(C.call_dm_task_deferred_remove(sym, (*C.struct_dm_task)(task))) +} + +func dmTaskGetInfoWithDeferredFct(task *cdmTask, info *Info) int { + if !LibraryDeferredRemovalSupport { + return -1 + } + + Cinfo := C.struct_backport_dm_info{} + defer func() { + info.Exists = int(Cinfo.exists) + info.Suspended = int(Cinfo.suspended) + info.LiveTable = int(Cinfo.live_table) + info.InactiveTable = int(Cinfo.inactive_table) + info.OpenCount = int32(Cinfo.open_count) + info.EventNr = uint32(Cinfo.event_nr) + info.Major = uint32(Cinfo.major) + info.Minor = uint32(Cinfo.minor) + info.ReadOnly = int(Cinfo.read_only) + info.TargetCount = int32(Cinfo.target_count) + info.DeferredRemove = int(Cinfo.deferred_remove) + }() + return int(C.dm_task_get_info((*C.struct_dm_task)(task), (*C.struct_dm_info)(unsafe.Pointer(&Cinfo)))) +} diff --git a/pkg/devicemapper/devmapper_wrapper_no_deferred_remove.go b/pkg/devicemapper/devmapper_wrapper_no_deferred_remove.go index a880fec8c499..80b034b3ff17 100644 --- a/pkg/devicemapper/devmapper_wrapper_no_deferred_remove.go +++ b/pkg/devicemapper/devmapper_wrapper_no_deferred_remove.go @@ -1,8 +1,10 @@ -// +build linux,cgo,libdm_no_deferred_remove +// +build linux,cgo +// +build !libdm_dlsym_deferred_remove,libdm_no_deferred_remove package devicemapper -// LibraryDeferredRemovalSupport tells if the feature is enabled in the build +// LibraryDeferredRemovalSupport tells if the feature is supported by the +// current Docker invocation. const LibraryDeferredRemovalSupport = false func dmTaskDeferredRemoveFct(task *cdmTask) int { -- 2.15.1 ++++++ bsc1055676-0001-daemon-oci-obey-CL_UNPRIVILEGED-for-user-namespaced-.patch ++++++ >From a24b98c0fc45d640b4eed8105033b313b8145e35 Mon Sep 17 00:00:00 2001 From: Aleksa Sarai <[email protected]> Date: Sun, 15 Oct 2017 17:06:20 +1100 Subject: [PATCH] daemon: oci: obey CL_UNPRIVILEGED for user namespaced daemon When runc is bind-mounting a particular path "with options", it has to do so by first creating a bind-mount and the modifying the options of said bind-mount via remount. However, in a user namespace, there are restrictions on which flags you can change with a remount (due to CL_UNPRIVILEGED being set in this instance). Docker historically has ignored this, and as a result, internal Docker mounts (such as secrets) haven't worked with --userns-remap. Fix this by preserving CL_UNPRIVILEGED mount flags when Docker is spawning containers with user namespaces enabled. SUSE-Bug: https://bugzilla.suse.com/show_bug.cgi?id=1055676 SUSE-Backport: https://github.com/moby/moby/pull/35205 Signed-off-by: Aleksa Sarai <[email protected]> --- daemon/oci_linux.go | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/daemon/oci_linux.go b/daemon/oci_linux.go index 0f8a392c2621..89ac627ff090 100644 --- a/daemon/oci_linux.go +++ b/daemon/oci_linux.go @@ -26,6 +26,7 @@ import ( "github.com/opencontainers/runc/libcontainer/user" specs "github.com/opencontainers/runtime-spec/specs-go" "github.com/sirupsen/logrus" + "golang.org/x/sys/unix" ) var ( @@ -469,6 +470,38 @@ func ensureSharedOrSlave(path string) error { return nil } +// Get the set of mount flags that are set on the mount that contains the given +// path and are locked by CL_UNPRIVILEGED. This is necessary to ensure that +// bind-mounting "with options" will not fail with user namespaces, due to +// kernel restrictions that require user namespace mounts to preserve +// CL_UNPRIVILEGED locked flags. +func getUnprivilegedMountFlags(path string) ([]string, error) { + var statfs unix.Statfs_t + if err := unix.Statfs(path, &statfs); err != nil { + return nil, err + } + + // The set of keys come from https://github.com/torvalds/linux/blob/v4.13/fs/namespace.c#L1034-L1048. + unprivilegedFlags := map[uint64]string{ + unix.MS_RDONLY: "ro", + unix.MS_NODEV: "nodev", + unix.MS_NOEXEC: "noexec", + unix.MS_NOSUID: "nosuid", + unix.MS_NOATIME: "noatime", + unix.MS_RELATIME: "relatime", + unix.MS_NODIRATIME: "nodiratime", + } + + var flags []string + for mask, flag := range unprivilegedFlags { + if uint64(statfs.Flags)&mask == mask { + flags = append(flags, flag) + } + } + + return flags, nil +} + var ( mountPropagationMap = map[string]int{ "private": mount.PRIVATE, @@ -575,6 +608,19 @@ func setMounts(daemon *Daemon, s *specs.Spec, c *container.Container, mounts []c opts = append(opts, mountPropagationReverseMap[pFlag]) } + // If we are using user namespaces, then we must make sure that we + // don't drop any of the CL_UNPRIVILEGED "locked" flags of the source + // "mount" when we bind-mount. The reason for this is that at the point + // when runc sets up the root filesystem, it is already inside a user + // namespace, and thus cannot change any flags that are locked. + if daemon.configStore.RemappedRoot != "" { + unprivOpts, err := getUnprivilegedMountFlags(m.Source) + if err != nil { + return err + } + opts = append(opts, unprivOpts...) + } + mt.Options = opts s.Mounts = append(s.Mounts, mt) } -- 2.15.0 ++++++ docker-audit.rules ++++++ ## # Audit rules based on CIS Docker 1.6 Benchmark v1.0.0 # https://benchmarks.cisecurity.org/tools2/docker/CIS_Docker_1.6_Benchmark_v1.0.0.pdf # Not all of these apply to SUSE. # 1.8 Audit docker daemon -w /usr/bin/docker -k docker # 1.9 Audit Docker files and directories -w /var/lib/docker -k docker # 1.10 Audit /etc/docker -w /etc/docker -k docker # 1.11 Audit Docker files and directories - docker-registry.service -w /usr/lib/systemd/system/docker-registry.service -k docker # 1.12 Audit Docker files and directories - docker.service -w /usr/lib/systemd/system/docker.service -k docker # 1.13 Audit Docker files and directories - /var/run/docker.sock -w /var/run/docker.sock -k docker # 1.14 Audit Docker files and directories - /etc/sysconfig/docker -w /etc/sysconfig/docker -k docker # 1.15 Audit Docker files and directories - /etc/sysconfig/docker-network -w /etc/sysconfig/docker-network -k docker # 1.16 Audit Docker files and directories - /etc/sysconfig/docker-registry -w /etc/sysconfig/docker-registry -k docker # 1.17 Audit Docker files and directories - /etc/sysconfig/docker-storage -w /etc/sysconfig/docker-storage -k docker # 1.18 Audit Docker files and directories - /etc/default/docker -w /etc/default/docker -k docker ## end docker audit rules ++++++ docker-kubic-rpmlintrc ++++++ addFilter ("^docker-kubic-test.x86_64: W: statically-linked-binary /usr/lib64/docker/dockerinit") addFilter ("^docker-kubic-bash-completion.noarch: W: sourced-script-with-shebang /etc/bash_completion.d/docker bash") addFilter ("^docker-kubic.x86_64: W: statically-linked-binary /usr/lib/docker/dockerinit") addFilter ("^docker-kubic.x86_64: W: unstripped-binary-or-object /usr/lib/docker/dockerinit") addFilter ("^docker-kubic.x86_64: W: no-manual-page-for-binary docker") addFilter ("^docker-kubic.x86_64: W: no-manual-page-for-binary nsinit") addFilter ("^docker-kubic-test.*") ++++++ docker.service ++++++ [Unit] Description=Docker Application Container Engine Documentation=http://docs.docker.com After=network.target containerd.socket containerd.service lvm2-monitor.service SuSEfirewall2.service Requires=containerd.socket containerd.service [Service] EnvironmentFile=/etc/sysconfig/docker # While Docker has support for socket activation (-H fd://), this is not # enabled by default because enabling socket activation means that on boot your # containers won't start until someone tries to administer the Docker daemon. Type=notify ExecStart=/usr/bin/dockerd --containerd /run/containerd/containerd.sock --add-runtime oci=/usr/sbin/docker-runc $DOCKER_NETWORK_OPTIONS $DOCKER_OPTS ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this property. TasksMax=infinity # Set delegate yes so that systemd does not reset the cgroups of docker containers # Only systemd 218 and above support this property. Delegate=yes # This is not necessary because of how we set up containerd. #KillMode=process [Install] WantedBy=multi-user.target ++++++ private-registry-0001-Add-private-registry-mirror-support.patch ++++++ ++++ 1244 lines (skipped) ++++++ secrets-0001-daemon-allow-directory-creation-in-run-secrets.patch ++++++ >From 5022c3554723040682444e324cd26ec8e2500131 Mon Sep 17 00:00:00 2001 From: Aleksa Sarai <[email protected]> Date: Wed, 8 Mar 2017 12:41:54 +1100 Subject: [PATCH 1/2] daemon: allow directory creation in /run/secrets Since FileMode can have the directory bit set, allow a SecretStore implementation to return secrets that are actually directories. This is useful for creating directories and subdirectories of secrets. Signed-off-by: Antonio Murdaca <[email protected]> Signed-off-by: Aleksa Sarai <[email protected]> --- daemon/container_operations_unix.go | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/daemon/container_operations_unix.go b/daemon/container_operations_unix.go index 954c194ea836..3ef1e0262edc 100644 --- a/daemon/container_operations_unix.go +++ b/daemon/container_operations_unix.go @@ -3,6 +3,7 @@ package daemon import ( + "bytes" "context" "fmt" "io/ioutil" @@ -13,6 +14,7 @@ import ( "github.com/docker/docker/container" "github.com/docker/docker/daemon/links" + "github.com/docker/docker/pkg/archive" "github.com/docker/docker/pkg/idtools" "github.com/docker/docker/pkg/mount" "github.com/docker/docker/pkg/stringid" @@ -216,9 +218,6 @@ func (daemon *Daemon) setupSecretDir(c *container.Container) (setupErr error) { if err != nil { return errors.Wrap(err, "unable to get secret from secret store") } - if err := ioutil.WriteFile(fPath, secret.Spec.Data, s.File.Mode); err != nil { - return errors.Wrap(err, "error injecting secret") - } uid, err := strconv.Atoi(s.File.UID) if err != nil { @@ -229,6 +228,25 @@ func (daemon *Daemon) setupSecretDir(c *container.Container) (setupErr error) { return err } + if s.File.Mode.IsDir() { + if err := os.Mkdir(fPath, s.File.Mode); err != nil { + return errors.Wrap(err, "error creating secretdir") + } + if secret.Spec.Data != nil { + // If the "file" is a directory, then s.File.Data is actually a tar + // archive of the directory. So we just do a tar extraction here. + if err := archive.UntarUncompressed(bytes.NewBuffer(secret.Spec.Data), fPath, &archive.TarOptions{ + UIDMaps: daemon.idMappings.UIDs(), + GIDMaps: daemon.idMappings.GIDs(), + }); err != nil { + return errors.Wrap(err, "error injecting secretdir") + } + } + } else { + if err := ioutil.WriteFile(fPath, secret.Spec.Data, s.File.Mode); err != nil { + return errors.Wrap(err, "error injecting secret") + } + } if err := os.Chown(fPath, rootIDs.UID+uid, rootIDs.GID+gid); err != nil { return errors.Wrap(err, "error setting ownership for secret") } -- 2.15.1 ++++++ secrets-0002-SUSE-implement-SUSE-container-secrets.patch ++++++ >From a84aa9152b50ea1fd73a7d09246ac056534d0e48 Mon Sep 17 00:00:00 2001 From: Aleksa Sarai <[email protected]> Date: Wed, 8 Mar 2017 11:43:29 +1100 Subject: [PATCH 2/2] SUSE: implement SUSE container secrets This allows for us to pass in host credentials to a container, allowing for SUSEConnect to work with containers. THIS PATCH IS NOT TO BE UPSTREAMED, DUE TO THE FACT THAT IT IS SUSE-SPECIFIC, AND UPSTREAM DOES NOT APPROVE OF THIS CONCEPT BECAUSE IT MAKES BUILDS NOT ENTIRELY REPRODUCIBLE. SUSE-Bugs: bsc#1057743 bsc#1055676 bsc#1030702 Signed-off-by: Aleksa Sarai <[email protected]> --- daemon/start.go | 5 + daemon/suse_secrets.go | 391 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 396 insertions(+) create mode 100644 daemon/suse_secrets.go diff --git a/daemon/start.go b/daemon/start.go index de32a649d7ed..2b6137d315e9 100644 --- a/daemon/start.go +++ b/daemon/start.go @@ -147,6 +147,11 @@ func (daemon *Daemon) containerStart(container *container.Container, checkpoint return err } + // SUSE:secrets -- inject the SUSE secret store + if err := daemon.injectSuseSecretStore(container); err != nil { + return err + } + spec, err := daemon.createSpec(container) if err != nil { return systemError{err} diff --git a/daemon/suse_secrets.go b/daemon/suse_secrets.go new file mode 100644 index 000000000000..9d0788f0410d --- /dev/null +++ b/daemon/suse_secrets.go @@ -0,0 +1,391 @@ +/* + * suse-secrets: patch for Docker to implement SUSE secrets + * Copyright (C) 2017 SUSE LLC. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package daemon + +import ( + "archive/tar" + "bytes" + "fmt" + "io" + "io/ioutil" + "os" + "path/filepath" + "strings" + "syscall" + + "github.com/docker/docker/container" + "github.com/docker/docker/pkg/archive" + "github.com/docker/docker/pkg/idtools" + "github.com/opencontainers/go-digest" + "github.com/sirupsen/logrus" + + swarmtypes "github.com/docker/docker/api/types/swarm" + swarmexec "github.com/docker/swarmkit/agent/exec" + swarmapi "github.com/docker/swarmkit/api" +) + +func init() { + // Output to tell us in logs that SUSE:secrets is enabled. + logrus.Infof("SUSE:secrets :: enabled") +} + +// Creating a fake file. +type SuseFakeFile struct { + Path string + Uid int + Gid int + Mode os.FileMode + Data []byte +} + +func (s SuseFakeFile) id() string { + // NOTE: It is _very_ important that this string always has a prefix of + // "suse". This is how we can ensure that we can operate on + // SecretReferences with a confidence that it was made by us. + return fmt.Sprintf("suse_%s_%s", digest.FromBytes(s.Data).Hex(), s.Path) +} + +func (s SuseFakeFile) toSecret() *swarmapi.Secret { + return &swarmapi.Secret{ + ID: s.id(), + Internal: true, + Spec: swarmapi.SecretSpec{ + Data: s.Data, + }, + } +} + +func (s SuseFakeFile) toSecretReference(idMaps *idtools.IDMappings) *swarmtypes.SecretReference { + // Figure out the host-facing {uid,gid} based on the provided maps. Fall + // back to root if the UID/GID don't match (we are guaranteed that root is + // mapped). + ctrUser := idtools.IDPair{UID: s.Uid, GID: s.Gid} + hostUser := idMaps.RootPair() + if user, err := idMaps.ToHost(ctrUser); err == nil { + hostUser = user + } + + // Return the secret reference as a file target. + return &swarmtypes.SecretReference{ + SecretID: s.id(), + SecretName: s.id(), + File: &swarmtypes.SecretReferenceFileTarget{ + Name: s.Path, + UID: fmt.Sprintf("%d", hostUser.UID), + GID: fmt.Sprintf("%d", hostUser.GID), + Mode: s.Mode, + }, + } +} + +// readDir will recurse into a directory prefix/dir, and return the set of +// secrets in that directory (as a tar archive that is packed inside the "data" +// field). The Path attribute of each has the prefix stripped. Symlinks are +// dereferenced. +func readDir(prefix, dir string) ([]*SuseFakeFile, error) { + var suseFiles []*SuseFakeFile + + path := filepath.Join(prefix, dir) + fi, err := os.Stat(path) + if err != nil { + // Ignore dangling symlinks. + if os.IsNotExist(err) { + logrus.Warnf("SUSE:secrets :: dangling symlink: %s", path) + return nil, nil + } + return nil, err + } else if !fi.IsDir() { + // Just to be safe. + logrus.Warnf("SUSE:secrets :: expected %q to be a directory, but was a file", path) + return readFile(prefix, dir) + } + path, err = filepath.EvalSymlinks(path) + if err != nil { + return nil, err + } + + // Construct a tar archive of the source directory. We tar up the prefix + // directory and add dir as an IncludeFiles specifically so that we + // preserve the name of the directory itself. + tarStream, err := archive.TarWithOptions(path, &archive.TarOptions{ + Compression: archive.Uncompressed, + IncludeSourceDir: true, + }) + if err != nil { + return nil, fmt.Errorf("SUSE:secrets :: failed to tar source directory %q: %v", path, err) + } + tarStreamBytes, err := ioutil.ReadAll(tarStream) + if err != nil { + return nil, fmt.Errorf("SUSE:secrets :: failed to read full tar archive: %v", err) + } + + // Get a list of the symlinks in the tar archive. + var symlinks []string + tmpTr := tar.NewReader(bytes.NewBuffer(tarStreamBytes)) + for { + hdr, err := tmpTr.Next() + if err == io.EOF { + break + } + if err != nil { + return nil, fmt.Errorf("SUSE:secrets :: failed to read through tar reader: %v", err) + } + if hdr.Typeflag == tar.TypeSymlink { + symlinks = append(symlinks, hdr.Name) + } + } + + // Symlinks aren't dereferenced in the above archive, so we explicitly do a + // rewrite of the tar archive to include all symlinks to files. We cannot + // do directories here, but lower-level directory symlinks aren't supported + // by zypper so this isn't an issue. + symlinkModifyMap := map[string]archive.TarModifierFunc{} + for _, sym := range symlinks { + logrus.Debugf("SUSE:secrets: archive(%q) %q is a need-to-rewrite symlink", path, sym) + symlinkModifyMap[sym] = func(tarPath string, hdr *tar.Header, r io.Reader) (*tar.Header, []byte, error) { + logrus.Debugf("SUSE:secrets: archive(%q) mapping for symlink %q", path, tarPath) + tarFullPath := filepath.Join(path, tarPath) + + // Get a copy of the original byte stream. + oldContent, err := ioutil.ReadAll(r) + if err != nil { + return nil, nil, fmt.Errorf("suse_rewrite: failed to read archive entry %q: %v", tarPath, err) + } + + // Check that the file actually exists. + fi, err := os.Stat(tarFullPath) + if err != nil { + logrus.Warnf("suse_rewrite: failed to stat archive entry %q: %v", tarFullPath, err) + return hdr, oldContent, nil + } + + // Read the actual contents. + content, err := ioutil.ReadFile(tarFullPath) + if err != nil { + logrus.Warnf("suse_rewrite: failed to read %q: %v", tarFullPath, err) + return hdr, oldContent, nil + } + + newHdr, err := tar.FileInfoHeader(fi, "") + if err != nil { + // Fake the header. + newHdr = &tar.Header{ + Typeflag: tar.TypeReg, + Mode: 0644, + } + } + + // Update the key fields. + hdr.Typeflag = newHdr.Typeflag + hdr.Mode = newHdr.Mode + hdr.Linkname = "" + return hdr, content, nil + } + } + + // Create the rewritten tar stream. + tarStream = archive.ReplaceFileTarWrapper(ioutil.NopCloser(bytes.NewBuffer(tarStreamBytes)), symlinkModifyMap) + tarStreamBytes, err = ioutil.ReadAll(tarStream) + if err != nil { + return nil, fmt.Errorf("SUSE:secrets :: failed to read rewritten archive: %v", err) + } + + // Add the tar stream as a "file". + suseFiles = append(suseFiles, &SuseFakeFile{ + Path: dir, + Mode: fi.Mode(), + Data: tarStreamBytes, + }) + return suseFiles, nil +} + +// readFile returns a secret given a file under a given prefix. +func readFile(prefix, file string) ([]*SuseFakeFile, error) { + path := filepath.Join(prefix, file) + fi, err := os.Stat(path) + if err != nil { + // Ignore dangling symlinks. + if os.IsNotExist(err) { + logrus.Warnf("SUSE:secrets :: dangling symlink: %s", path) + return nil, nil + } + return nil, err + } else if fi.IsDir() { + // Just to be safe. + logrus.Warnf("SUSE:secrets :: expected %q to be a file, but was a directory", path) + return readDir(prefix, file) + } + + stat, ok := fi.Sys().(*syscall.Stat_t) + if !ok { + logrus.Warnf("SUSE:secrets :: failed to cast file stat_t: defaulting to owned by root:root: %s", path) + } + + bytes, err := ioutil.ReadFile(path) + if err != nil { + return nil, err + } + + var suseFiles []*SuseFakeFile + suseFiles = append(suseFiles, &SuseFakeFile{ + Path: file, + Uid: int(stat.Uid), + Gid: int(stat.Gid), + Mode: fi.Mode(), + Data: bytes, + }) + return suseFiles, nil +} + +// getHostSuseSecretData returns the list of SuseFakeFiles the need to be added +// as SUSE secrets. +func getHostSuseSecretData() ([]*SuseFakeFile, error) { + secrets := []*SuseFakeFile{} + + credentials, err := readDir("/etc/zypp", "credentials.d") + if err != nil { + if os.IsNotExist(err) { + credentials = []*SuseFakeFile{} + } else { + logrus.Errorf("SUSE:secrets :: error while reading zypp credentials: %s", err) + return nil, err + } + } + secrets = append(secrets, credentials...) + + suseConnect, err := readFile("/etc", "SUSEConnect") + if err != nil { + if os.IsNotExist(err) { + suseConnect = []*SuseFakeFile{} + } else { + logrus.Errorf("SUSE:secrets :: error while reading /etc/SUSEConnect: %s", err) + return nil, err + } + } + secrets = append(secrets, suseConnect...) + + return secrets, nil +} + +// To fake an empty store, in the case where we are operating on a container +// that was created pre-swarmkit. Otherwise segfaults and other fun things +// happen. See bsc#1057743. +type ( + suseEmptyStore struct{} + suseEmptySecret struct{} + suseEmptyConfig struct{} +) + +// In order to reduce the amount of code touched outside of this file, we +// implement the swarm API for DependencyGetter. This asserts that this +// requirement will always be matched. In addition, for the case of the *empty* +// getters this reduces memory usage by having a global instance. +var ( + _ swarmexec.DependencyGetter = &suseDependencyStore{} + emptyStore swarmexec.DependencyGetter = suseEmptyStore{} + emptySecret swarmexec.SecretGetter = suseEmptySecret{} + emptyConfig swarmexec.ConfigGetter = suseEmptyConfig{} +) + +var errSuseEmptyStore = fmt.Errorf("SUSE:secrets :: tried to get a resource from empty store [this is a bug]") + +func (_ suseEmptyConfig) Get(_ string) (*swarmapi.Config, error) { return nil, errSuseEmptyStore } +func (_ suseEmptySecret) Get(_ string) (*swarmapi.Secret, error) { return nil, errSuseEmptyStore } +func (_ suseEmptyStore) Secrets() swarmexec.SecretGetter { return emptySecret } +func (_ suseEmptyStore) Configs() swarmexec.ConfigGetter { return emptyConfig } + +type suseDependencyStore struct { + dfl swarmexec.DependencyGetter + secrets map[string]*swarmapi.Secret +} + +// The following are effectively dumb wrappers that return ourselves, or the +// default. +func (s *suseDependencyStore) Secrets() swarmexec.SecretGetter { return s } +func (s *suseDependencyStore) Configs() swarmexec.ConfigGetter { return s.dfl.Configs() } + +// Get overrides the underlying DependencyGetter with our own secrets (falling +// through to the underlying DependencyGetter if the secret isn't present). +func (s *suseDependencyStore) Get(id string) (*swarmapi.Secret, error) { + logrus.Debugf("SUSE:secrets :: id=%s requested from suseDependencyGetter", id) + + secret, ok := s.secrets[id] + if !ok { + // fallthrough + return s.dfl.Secrets().Get(id) + } + return secret, nil +} + +// removeSuseSecrets removes any SecretReferences which were added by us +// explicitly (this is detected by checking that the prefix has a 'suse' +// prefix). See bsc#1057743. +func removeSuseSecrets(c *container.Container) { + var without []*swarmtypes.SecretReference + for _, secret := range c.SecretReferences { + if strings.HasPrefix(secret.SecretID, "suse") { + logrus.Warnf("SUSE:secrets :: removing 'old' suse secret %q from container %q", secret.SecretID, c.ID) + continue + } + without = append(without, secret) + } + c.SecretReferences = without +} + +func (daemon *Daemon) injectSuseSecretStore(c *container.Container) error { + newDependencyStore := &suseDependencyStore{ + dfl: c.DependencyStore, + secrets: make(map[string]*swarmapi.Secret), + } + // Handle old containers. See bsc#1057743. + if newDependencyStore.dfl == nil { + newDependencyStore.dfl = emptyStore + } + + // We drop any "old" SUSE secrets, as it appears that old containers (when + // restarted) could still have references to old secrets. The .id() of all + // secrets have a prefix of "suse" so this is much easier. See bsc#1057743 + // for details on why this could cause issues. + removeSuseSecrets(c) + + secrets, err := getHostSuseSecretData() + if err != nil { + return err + } + + idMaps := daemon.IDMappings() + for _, secret := range secrets { + newDependencyStore.secrets[secret.id()] = secret.toSecret() + c.SecretReferences = append(c.SecretReferences, secret.toSecretReference(idMaps)) + } + + c.DependencyStore = newDependencyStore + + // bsc#1057743 -- In older versions of Docker we added volumes explicitly + // to the mount list. This causes clashes because of duplicate namespaces. + // If we see an existing mount that will clash with the in-built secrets + // mount we assume it's our fault. + for _, intendedMount := range c.SecretMounts() { + mountPath := intendedMount.Destination + if volume, ok := c.MountPoints[mountPath]; ok { + logrus.Debugf("SUSE:secrets :: removing pre-existing %q mount: %#v", mountPath, volume) + delete(c.MountPoints, mountPath) + } + } + return nil +} -- 2.15.1 ++++++ sysconfig.docker ++++++ ## Path : System/Management ## Description : Extra cli switches for docker daemon ## Type : string ## Default : "" ## ServiceRestart : docker # DOCKER_OPTS="" ++++++ tests.sh ++++++ #!/bin/bash # # Script for launching the Docker integration tests # #set -x SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" DOCKER_DIR=/usr/src/docker SCRIPTS_DIR=$DOCKER_DIR/hack TESTS_EXE=$SCRIPTS_DIR/tests.main VERSION=$(cat $DOCKER_DIR/VERSION) # working dirs TESTS_DIR=/tmp/docker-int-tests BUNDLES_DIR=$TESTS_DIR/run/bundles FAKE_GOPATH=$TESTS_DIR/go # some test expect the containerd socket here CONTAINERD_SOCK=/run/containerd/containerd.sock EXPECTED_CONTAINERD_SOCK=/var/run/docker/libcontainerd/docker-containerd.sock # ... and some programs CONTAINERD_CTR=/usr/sbin/containerd-ctr EXPECTED_CONTAINERD_CTR=/usr/local/bin/docker-containerd-ctr CHECK_TIMEOUT="${CHECK_TIMEOUT:-5m}" TEST_TIMEOUT="${TEST_TIMEOUT:-60m}" TEST_ARGS="-check.vv -check.timeout=${CHECK_TIMEOUT} -test.timeout=${TEST_TIMEOUT}" TEST_SELECT= TEST_LOG=/tmp/docker-tests.log ENABLE_XUNIT=${ENABLE_XUNIT:-yes} # the sysconfig file for Docker SYSCFG_DOCKER=/etc/sysconfig/docker # some vars from the Dockerfile ENABLE_NOTARY=${ENABLE_NOTARY:-} ENABLE_REGISTRY=${ENABLE_REGISTRY:-} REGISTRY_COMMIT_SCHEMA1=ec87e9b6971d831f0eff752ddb54fb64693e51cd REGISTRY_COMMIT=47a064d4195a9b56133891bbb13620c3ac83a827 NOTARY_VERSION=v0.3.0 ################################################################################ log() { echo ">>> $@" ; } warn() { log "WARNING: $@" ; } error() { log "ERROR: $@" ; } abort() { log "FATAL: $@" ; exit 1 ; } usage() { echo "$USAGE" ; } abort_usage() { usage ; abort $@ ; } bundle() { local bundle="$1"; shift log "Making bundle: $(basename "$bundle") (in $DEST)" source "$SCRIPTS_DIR/make/$bundle" "$@" } set_opts() { OPT="$1" VALUE="$2" FILE=$3 perl -pi -e "s/^$OPT=.*$//g" $FILE echo "$OPT=\"$VALUE\"" >> $FILE } set_docker_opts() { set_opts "DOCKER_OPTS" "$DOCKER_OPTS" /etc/sysconfig/docker ; } set_containerd_opts() { set_opts "CONTAINERD_OPTS" "$CONTAINERD_OPTS" /etc/sysconfig/containerd ; } fix_expected() { EXPECTED=$1 EXISTING=$2 exp_base=$(basename $EXPECTED) exp_dir=$(dirname $EXPECTED) [ -d $exp_dir ] || mkdir -p $exp_dir rm -f $exp_dir/$exp_base (cd $exp_dir && ln -sf $EXISTING $exp_base) } save_backup() { for x in $@ ; do if [ ! -f $x ] ; then touch $x.nbak elif [ -f $x.bak ] ; then warn "$x.bak already exists: no backup will be done" else cp -f $x $x.bak fi done } restore_backup() { for x in $@ ; do if [ -f $x.nbak ] ; then rm -f $x.nbak else if [ -f $x.bak ] ; then mv -f $x.bak $x fi fi done } require_go() { go version >/dev/null 2>&1 ; } require_git() { git version >/dev/null 2>&1 ; } ################################################################################ [ -x $TESTS_EXE ] || abort "integration tests executable not found at $TESTS_EXE" [ $EUID -eq 0 ] || abort "this script must be run as root" [ -n "$VERSION" ] || abort "could not obtain version" [ -e $CONTAINERD_SOCK ] || abort "containerd socket not found at $CONTAINERD_SOCK" [ -x $CONTAINERD_CTR ] || abort "containerd-ctr not found at $CONTAINERD_CTR" if [ $# -gt 0 ] ; then # run only some specific tests TEST_SELECT="-check.f=$(echo $@ | tr ' ' '|')" fi # We want this to fail if the bundles already exist and cannot be removed. # This is to avoid mixing bundles from different versions of the code. mkdir -p $BUNDLES_DIR if [ -e "$BUNDLES_DIR/$VERSION" ] && [ -z "$KEEPBUNDLE" ]; then log "$BUNDLES_DIR/$VERSION already exists. Removing." rm -fr "$BUNDLES_DIR/$VERSION" && mkdir "$BUNDLES_DIR/$VERSION" || exit 1 echo fi DEST="$BUNDLES_DIR/$VERSION/" mkdir -p "$DEST" export DEST=$(realpath $DEST) # create a fake go path rm -rf $FAKE_GOPATH mkdir -p $FAKE_GOPATH/src/github.com/docker (cd $FAKE_GOPATH/src/github.com/docker && ln -sf $DOCKER_DIR docker) if [ -n "$ENABLE_REGISTRY" ] ; then # build the Docker registry if [ ! -x /usr/local/bin/registry-v2 ] ; then log "Building registry (commit:$REGISTRY_COMMIT)" require_git || abort "git is not installed" require_go || abort "the Go compiler is not installed" export GOPATH="$(mktemp -d)" git clone https://github.com/docker/distribution.git "$GOPATH/src/github.com/docker/distribution" (cd "$GOPATH/src/github.com/docker/distribution" && git checkout -q "$REGISTRY_COMMIT") GOPATH="$GOPATH/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH" \ go build -o /usr/local/bin/registry-v2 github.com/docker/distribution/cmd/registry (cd "$GOPATH/src/github.com/docker/distribution" && git checkout -q "$REGISTRY_COMMIT_SCHEMA1") GOPATH="$GOPATH/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH" \ go build -o /usr/local/bin/registry-v2-schema1 github.com/docker/distribution/cmd/registry chmod 755 /usr/local/bin/registry-v2 [ -x /usr/local/bin/registry-v2 ] || abort "registry-v2 could not be built" rm -rf "$GOPATH" export -n GOPATH fi fi if [ -n "$ENABLE_NOTARY" ] ; then # build the Docker notary if [ ! -x /usr/local/bin/notary-server ] ; then export GO15VENDOREXPERIMENT=1 export GOPATH="$(mktemp -d)" require_git || abort "git is not installed" require_go || abort "the Go compiler is not installed" log "Building notary (version:$NOTARY_VERSION)" git clone https://github.com/docker/notary.git "$GOPATH/src/github.com/docker/notary" (cd "$GOPATH/src/github.com/docker/notary" && git checkout -q "$NOTARY_VERSION") GOPATH="$GOPATH/src/github.com/docker/notary/vendor:$GOPATH" \ go build -o /usr/local/bin/notary-server github.com/docker/notary/cmd/notary-server GOPATH="$GOPATH/src/github.com/docker/notary/vendor:$GOPATH" \ go build -o /usr/local/bin/notary github.com/docker/notary/cmd/notary chmod 755 /usr/local/bin/notary-server [ -x /usr/local/bin/notary-server ] || abort "notary could not be built" export -n GO15VENDOREXPERIMENT rm -rf "$GOPATH" export -n GOPATH fi fi if [ -n "$ENABLE_XUNIT" ] ; then if [ ! -x /usr/local/bin/go2xunit ] ; then echo >&2 "Installing go2xunit." require_go || abort "the Go compiler is not installed" export GOPATH="$(mktemp -d)" go get -d github.com/tebeka/go2xunit cd $GOPATH/src/github.com/tebeka/go2xunit && go build -o /usr/local/bin/go2xunit . chmod 755 /usr/local/bin/go2xunit [ -x /usr/local/bin/go2xunit ] || abort "go2xunit could not be built" rm -rf "$GOPATH" export -n GOPATH fi fi # tests require this user and group /usr/sbin/groupadd -r docker >/dev/null 2>&1 || /bin/true /usr/sbin/useradd --create-home --gid docker unprivilegeduser >/dev/null 2>&1 || /bin/true # prepare some expected dirs, files, etc... fix_expected $TESTS_DIR/contrib $DOCKER_DIR/contrib fix_expected $DEST/fixtures $DOCKER_DIR/integration-cli/fixtures fix_expected $EXPECTED_CONTAINERD_SOCK $CONTAINERD_SOCK fix_expected $EXPECTED_CONTAINERD_CTR $CONTAINERD_CTR export DOCKER_TEST_HOST="tcp://127.0.0.1:2375" export PATH=/usr/local/bin:$PATH export TZ=utc export GOPATH="$FAKE_GOPATH" export DOCKER_GRAPHDRIVER="${DOCKER_GRAPHDRIVER:-vfs}" export DOCKER_USERLANDPROXY="${DOCKER_USERLANDPROXY:-true}" #export DOCKER_REMAP_ROOT=default # example usage: DOCKER_STORAGE_OPTS="dm.basesize=20G,dm.loopdatasize=200G" storage_params="" if [ -n "$DOCKER_STORAGE_OPTS" ]; then IFS=',' for i in ${DOCKER_STORAGE_OPTS}; do storage_params="--storage-opt $i $storage_params" done unset IFS fi # example usage: DOCKER_STORAGE_OPTS="dm.basesize=20G,dm.loopdatasize=200G" extra_params="" # deal with remapping save_backup /etc/subuid /etc/subgid echo "dockremap:500000:65536" > /etc/subuid echo "dockremap:500000:65536" > /etc/subgid groupadd dockremap >/dev/null 2>&1 || /bin/true useradd -g dockremap dockremap >/dev/null 2>&1 || /bin/true # make sure Docker is stopped, set our config file and then start again save_backup $SYSCFG_DOCKER cat <<SYSCFG_DOCKER_EOF > $SYSCFG_DOCKER DOCKER_OPTS="--log-level=debug \ --pidfile=$DEST/docker.pid \ -H tcp://127.0.0.1:2375 \ --storage-driver=$DOCKER_GRAPHDRIVER \ --userland-proxy=$DOCKER_USERLANDPROXY" DOCKER_NETWORK_OPTIONS="" SYSCFG_DOCKER_EOF systemctl reload-or-restart docker.service cleanup() { log "Restoring the Docker service..." restore_backup $SYSCFG_DOCKER systemctl reload-or-restart docker.service log "Removing extra files and restoring backups..." restore_backup /etc/subuid /etc/subgid rm -f $TESTS_DIR/contrib \ $DEST/fixtures \ $EXPECTED_CONTAINERD_SOCK \ $EXPECTED_CONTAINERD_CTR } trap cleanup EXIT cd $DOCKER_DIR log "Preparing the environment..." bundle .integration-daemon-start bundle .integration-daemon-setup log "Running integration tests..." export DOCKER_HOST=$DOCKER_TEST_HOST cd $DEST && $TESTS_EXE $TEST_ARGS $TEST_SELECT | tee $TEST_LOG || /bin/true if [ -n "$ENABLE_XUNIT" ] ; then log "Generating xunit logs..." go2xunit -fail -gocheck -input $TEST_LOG -output $TEST_LOG.xml fi export -n DOCKER_HOST bundle .integration-daemon-stop
