Bug#909644: Bug #909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
Control: reassign -1 runc Since it's trivial to fix in runc, I reassign this to runc now. It'll be fixed later, which is approved by release team, along with another improvement in runc. -- Shengjing Zhu
Bug#909644: Bug #909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
On 3/6/19 4:55 PM, Dmitry Smirnov wrote: > This is not such a small and definitely undesirable overhead. Its purpose is > unclear, it requires manual step and sure enough it will be forgotten. > Moreover it aims at the problem in another package. > It is not runc's job to meet Docker expectations. > Also commit ID is probably useless anyway since we should attempt to always > shipped a tagged (pre-)release. Plus we have patches that make commit ID even > more irrelevant. > > Docker problem should be fixed in Docker. > And clearly it is a Docker's problem to expect a very particular (bundled) > runc (for no good reason), and forever whine into logs about not having > that... I think applying the patch to runc build as suggested by Shengjing is a good solution for everyone, much better that what I initially proposed. It doesn't involve any git commit, which wouldn't make sense as both of you pointed out, and doesn't add any overhead. The new output for `runc --version` after this patch looks like that: $ /usr/sbin/runc --version runc version 1.0.0~rc6+dfsg1 commit: 1.0.0~rc6+dfsg1-3 spec: 1.0.1 It's a bit hacky to display the debian version on the commit line, but why not. I think it doesn't hurt either. It also prevents to add any patch to docker, as after testing I can confirm that docker doesn't complain anymore, at least for my use-case. Docker is happy enough to have a line such as `commit: 1.0.0~rc6+dfsg1-3` in the output of `runc --version`. Apparently the content of the line doesn't matter, as long as the line is there.
Bug#909644: Bug #909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
On 3/6/19 10:03 AM, Shengjing Zhu wrote: > I think the runc should be fixed. > > But I don't like the patch you suggested. It's confused to user. If > you set the git commit to the upstream one, like > ccb5efd37fb7c86364786e9137e22948751de7ed for 1.0.0-rc6, the user would > think it's 1.0.0-rc6 indeed, but apparently it's not, it's 1.0.0-rc6 > with CVE-2019-5736 patch. Indeed, you're right. > So I would suggest to use the debian package version in the commit > field. More specifically: > > diff --git a/debian/rules b/debian/rules > index 81df53b..0087b6b 100755 > --- a/debian/rules > +++ b/debian/rules > @@ -5,7 +5,11 @@ > > export DH_GOPKG := github.com/opencontainers/runc > export DH_GOLANG_INSTALL_EXTRA := libcontainer/seccomp/fixtures > + > +include /usr/share/dpkg/pkg-info.mk > + > TAGS=apparmor seccomp selinux ambient > +LDFLAGS := -X main.version=$(DEB_VERSION_UPSTREAM) -X > main.gitCommit=$(DEB_VERSION) > > %: >dh $@ --buildsystem=golang --with=golang --builddirectory=_build > @@ -33,7 +37,7 @@ override_dh_auto_configure: > # ln -svrf vendor/github.com/opencontainers/specs > _build/src/github.com/opencontainers/ > > override_dh_auto_build: > - dh_auto_build -- -tags "$(TAGS)" > + dh_auto_build -- -tags "$(TAGS)" -ldflags "$(LDFLAGS)" > > override_dh_auto_test: >DH_GOLANG_EXCLUDES="libcontainer/integration" \ Thanks for the patch, I applied it and force-pushed it to wip/909644. Feel free to merge if you like it. > And we're late to fix this before hard freeze. If we want this fix > included in buster, we should ask release team to unblock. Ok, I will do that if you upload the package then.
Bug#909644: Bug #909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
On Wednesday, 6 March 2019 12:50:54 PM AEDT Arnaud Rebillout wrote: > **Solution 1** > [...] > One is to patch docker, and fix their code so that it can handle various > runc version outputs. I prefer this solution. > **Solution 2** > > The other way is to modify the runc build so that we include the git > commit. I've pushed such a change on a branch at: > > https://salsa.debian.org/go-team/packages/runc/commit/033a60b549b0935861c92 > bf4e5118d03ef443a8f > > It's actually not that bad, no patch involved, it's really just adding > ldflags. The only downside I see is that there's an additional step when > a maintainer wants to release a new upstream version: he has to look for > the corresponding commit and add it to the file > `debian/upstream-version-gitcommits`. I provided a helper script to do > that, so it's really a small overhead. This is not such a small and definitely undesirable overhead. Its purpose is unclear, it requires manual step and sure enough it will be forgotten. Moreover it aims at the problem in another package. It is not runc's job to meet Docker expectations. Also commit ID is probably useless anyway since we should attempt to always shipped a tagged (pre-)release. Plus we have patches that make commit ID even more irrelevant. Docker problem should be fixed in Docker. And clearly it is a Docker's problem to expect a very particular (bundled) runc (for no good reason), and forever whine into logs about not having that... -- All the best, Dmitry Smirnov. --- If liberty means anything at all, it means the right to tell people what they do not want to hear. -- George Orwell signature.asc Description: This is a digitally signed message part.
Bug#909644: Bug #909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
Hi Arnaud, Thanks for the summary. On Wed, Mar 6, 2019 at 9:51 AM Arnaud Rebillout wrote: > > Dear Go team, > > So let met sum up this bug now to avoid you reading the backlog: > > In short, docker doesn't recognize the output of `runc --version`, and > then it misbehaves and flood the log forever. To be more accurate, > docker is not happy because it wants to know the git commit that was > used to build runc. This information may or may not be part of the > output of `runc --version`: it depends on how runc was built. Actually, > there are two fields, `version` and `gitcommit`, that can be passed > optionally to runc during the build (as ldflags). Therefore the output > of `runc --version` varies accordingly. > > So there's two ways out of that: > > **Solution 1** > > One is to patch docker, and fix their code so that it can handle various > runc version outputs. This is the way chosen by buildroot for example: > > https://github.com/buildroot/buildroot/blob/master/package/docker-engine/0001-Fix-faulty-runc-version-commit-scrape.patch > > Note that this patch is not the definitive solution: it just allows > docker to handle the output of runc *as built in buildroot*, and not in > every case. If we go this way, we'll do something similar, supporting > "debian's runc" output, and end up with a patch that has no chance to > make it upstream. > > Maybe the core of the issue is that the `runc --version` output is > difficult to parse properly, and that's why docker upstream don't want > to bother fixing their code... > > > **Solution 2** > > The other way is to modify the runc build so that we include the git > commit. I've pushed such a change on a branch at: > > https://salsa.debian.org/go-team/packages/runc/commit/033a60b549b0935861c92bf4e5118d03ef443a8f > > It's actually not that bad, no patch involved, it's really just adding > ldflags. The only downside I see is that there's an additional step when > a maintainer wants to release a new upstream version: he has to look for > the corresponding commit and add it to the file > `debian/upstream-version-gitcommits`. I provided a helper script to do > that, so it's really a small overhead. I think the runc should be fixed. But I don't like the patch you suggested. It's confused to user. If you set the git commit to the upstream one, like ccb5efd37fb7c86364786e9137e22948751de7ed for 1.0.0-rc6, the user would think it's 1.0.0-rc6 indeed, but apparently it's not, it's 1.0.0-rc6 with CVE-2019-5736 patch. So I would suggest to use the debian package version in the commit field. More specifically: diff --git a/debian/rules b/debian/rules index 81df53b..0087b6b 100755 --- a/debian/rules +++ b/debian/rules @@ -5,7 +5,11 @@ export DH_GOPKG := github.com/opencontainers/runc export DH_GOLANG_INSTALL_EXTRA := libcontainer/seccomp/fixtures + +include /usr/share/dpkg/pkg-info.mk + TAGS=apparmor seccomp selinux ambient +LDFLAGS := -X main.version=$(DEB_VERSION_UPSTREAM) -X main.gitCommit=$(DEB_VERSION) %: dh $@ --buildsystem=golang --with=golang --builddirectory=_build @@ -33,7 +37,7 @@ override_dh_auto_configure: # ln -svrf vendor/github.com/opencontainers/specs _build/src/github.com/opencontainers/ override_dh_auto_build: - dh_auto_build -- -tags "$(TAGS)" + dh_auto_build -- -tags "$(TAGS)" -ldflags "$(LDFLAGS)" override_dh_auto_test: DH_GOLANG_EXCLUDES="libcontainer/integration" \ > > > **What do you think?** > > I'm in favor of solution 2, it seems the path of least trouble, and is > also a correct thing to do IMHO. I'd like to have this fixed for buster, > as this issue has been opened for a long while. > > If you agree with solution 2 I'll need some DD to upload a new version > of runc with this change (sorry to bother you...). Also this bug should > be re-assigned to runc if we solve it in the runc package. > > For more details you can also look at upstream discussion: > https://github.com/moby/moby/issues/38709 > > And we're late to fix this before hard freeze. If we want this fix included in buster, we should ask release team to unblock. -- Shengjing Zhu
Bug#909644: Bug #909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
> On Mar 5, 2019, at 7:50 PM, Arnaud Rebillout > wrote: > > Dear Go team, > > So let met sum up this bug now to avoid you reading the backlog: > > In short, docker doesn't recognize the output of `runc --version`, and > then it misbehaves and flood the log forever. To be more accurate, > docker is not happy because it wants to know the git commit that was > used to build runc. This information may or may not be part of the > output of `runc --version`: it depends on how runc was built. Actually, > there are two fields, `version` and `gitcommit`, that can be passed > optionally to runc during the build (as ldflags). Therefore the output > of `runc --version` varies accordingly. > > So there's two ways out of that: > > **Solution 1** > > One is to patch docker, and fix their code so that it can handle various > runc version outputs. This is the way chosen by buildroot for example: > > https://github.com/buildroot/buildroot/blob/master/package/docker-engine/0001-Fix-faulty-runc-version-commit-scrape.patch > > Note that this patch is not the definitive solution: it just allows > docker to handle the output of runc *as built in buildroot*, and not in > every case. If we go this way, we'll do something similar, supporting > "debian's runc" output, and end up with a patch that has no chance to > make it upstream. > > Maybe the core of the issue is that the `runc --version` output is > difficult to parse properly, and that's why docker upstream don't want > to bother fixing their code... > > > **Solution 2** > > The other way is to modify the runc build so that we include the git > commit. I've pushed such a change on a branch at: > > https://salsa.debian.org/go-team/packages/runc/commit/033a60b549b0935861c92bf4e5118d03ef443a8f > > It's actually not that bad, no patch involved, it's really just adding > ldflags. The only downside I see is that there's an additional step when > a maintainer wants to release a new upstream version: he has to look for > the corresponding commit and add it to the file > `debian/upstream-version-gitcommits`. I provided a helper script to do > that, so it's really a small overhead. > > > **What do you think?** > > I'm in favor of solution 2, it seems the path of least trouble, and is > also a correct thing to do IMHO. I'd like to have this fixed for buster, > as this issue has been opened for a long while. > > If you agree with solution 2 I'll need some DD to upload a new version > of runc with this change (sorry to bother you...). Also this bug should > be re-assigned to runc if we solve it in the runc package. > > For more details you can also look at upstream discussion: > https://github.com/moby/moby/issues/38709 > > > All the best, > > Arnaud Based on the upstream bug linked it sounds like solution 2 is what they prefer, and also makes runc consistent with upstream releases. If there’s consensus I am happy to sponsor an upload of runc for you. Unfortunately, since we’re so close to the freeze it will not make it into buster unless there is an exception made. Stephen
Bug#909644: Bug #909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
Dear Go team, So let met sum up this bug now to avoid you reading the backlog: In short, docker doesn't recognize the output of `runc --version`, and then it misbehaves and flood the log forever. To be more accurate, docker is not happy because it wants to know the git commit that was used to build runc. This information may or may not be part of the output of `runc --version`: it depends on how runc was built. Actually, there are two fields, `version` and `gitcommit`, that can be passed optionally to runc during the build (as ldflags). Therefore the output of `runc --version` varies accordingly. So there's two ways out of that: **Solution 1** One is to patch docker, and fix their code so that it can handle various runc version outputs. This is the way chosen by buildroot for example: https://github.com/buildroot/buildroot/blob/master/package/docker-engine/0001-Fix-faulty-runc-version-commit-scrape.patch Note that this patch is not the definitive solution: it just allows docker to handle the output of runc *as built in buildroot*, and not in every case. If we go this way, we'll do something similar, supporting "debian's runc" output, and end up with a patch that has no chance to make it upstream. Maybe the core of the issue is that the `runc --version` output is difficult to parse properly, and that's why docker upstream don't want to bother fixing their code... **Solution 2** The other way is to modify the runc build so that we include the git commit. I've pushed such a change on a branch at: https://salsa.debian.org/go-team/packages/runc/commit/033a60b549b0935861c92bf4e5118d03ef443a8f It's actually not that bad, no patch involved, it's really just adding ldflags. The only downside I see is that there's an additional step when a maintainer wants to release a new upstream version: he has to look for the corresponding commit and add it to the file `debian/upstream-version-gitcommits`. I provided a helper script to do that, so it's really a small overhead. **What do you think?** I'm in favor of solution 2, it seems the path of least trouble, and is also a correct thing to do IMHO. I'd like to have this fixed for buster, as this issue has been opened for a long while. If you agree with solution 2 I'll need some DD to upload a new version of runc with this change (sorry to bother you...). Also this bug should be re-assigned to runc if we solve it in the runc package. For more details you can also look at upstream discussion: https://github.com/moby/moby/issues/38709 All the best, Arnaud
Bug#909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
On Fri, 25 Jan 2019 08:20:03 +0700 Arnaud Rebillout wrote: > On Thu, 24 Jan 2019 18:43:18 +0100 Marcel Partap wrote: > > > reassign runc > > > > OK coreos fixed this by building runc with a commit hash: > > > https://github.com/coreos/coreos-overlay/pull/2428/commits/a0294fbf2caf5f1a04651a22bb67357969f5831f > > > > Isn't this is quite a reasonable fix? > > > Hmm, and on the other hand, the commit you refer to is not really about patching runc, it's just about making the commit id available at compile time. It's also something reasonable. But I still believe it's easier to just patch docker.
Bug#909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
On Thu, 24 Jan 2019 18:43:18 +0100 Marcel Partap wrote: > reassign runc > > OK coreos fixed this by building runc with a commit hash: > https://github.com/coreos/coreos-overlay/pull/2428/commits/a0294fbf2caf5f1a04651a22bb67357969f5831f > > Isn't this is quite a reasonable fix? > Runc is doing the right thing, it's free to report its version the way it wants. There's no reason to patch it because one particular program expects a different output. Maybe there are other programs out there parsing the runc version string, and maybe you break them by patching runc this way. On the other end, patching docker is trivial, it's just one line to comment.
Bug#909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
On Wednesday, 26 September 2018 6:40:14 PM AEST Marcel Partap wrote: > Why would these two programs shipped in the same package not be compatible > with one another I wonder? They are compatible, but "runc" is not shipped by the docker.io package... Problem is that fork-obsessed Docker expects a particular version of runc and fails to recognise the available one and/or to parse its version output. > How can it be fixed? Someone needs to produce a patch for Docker to fix version parsing from the output of `/usr/sbin/runc -v`, I think. -- Regards, Dmitry Smirnov. --- You have enemies? Good. That means you've stood up for something, sometime in your life. -- Victor Hugo, "Villemain", 1845 (often misattributed to Winston Churchill) signature.asc Description: This is a digitally signed message part.
Bug#909644: docker.io: dockerd warning: failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1
Package: docker.io Version: 18.06.1+dfsg1-2 Severity: normal docker.io 18.06.1+dfsg1-2: the sys log is being spammed with these messages: > Sep 26 10:02:34 base dockerd[1329]: time="2018-09-26T10:02:34.612458096+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1\n" > Sep 26 10:02:54 base dockerd[1329]: time="2018-09-26T10:02:54.608852757+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1\n" > Sep 26 10:03:14 base dockerd[1329]: time="2018-09-26T10:03:14.608428914+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1\n" > Sep 26 10:03:34 base dockerd[1329]: time="2018-09-26T10:03:34.608336092+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1\n" > Sep 26 10:03:54 base dockerd[1329]: time="2018-09-26T10:03:54.608367304+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1\n" > Sep 26 10:04:14 base dockerd[1329]: time="2018-09-26T10:04:14.608181868+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1\n" > Sep 26 10:04:34 base dockerd[1329]: time="2018-09-26T10:04:34.608692071+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1\n" > Sep 26 10:04:54 base dockerd[1329]: time="2018-09-26T10:04:54.608829153+02:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.1\n" Why would these two programs shipped in the same package not be compatible with one another I wonder? How can it be fixed? -- System Information: Debian Release: buster/sid APT prefers unstable APT policy: (510, 'unstable'), (510, 'testing'), (509, 'experimental'), (500, 'stable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.18.0-1-amd64 (SMP w/12 CPU cores) Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8), LANGUAGE=en_GB.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages docker.io depends on: ii adduser 3.118 ii iptables1.6.2-1.1 ii libc6 2.27-6 ii libdevmapper1.02.1 2:1.02.145-4.1 ii libltdl72.4.6-4 ii libnspr42:4.20-1 ii libnss3 2:3.39-1 ii libseccomp2 2.3.3-3 ii libsystemd0 239-9 ii lsb-base9.20170808 ii runc1.0.0~rc5+dfsg1-3 ii tini0.18.0-1 Versions of packages docker.io recommends: ii ca-certificates 20180409 ii cgroupfs-mount 1.4 ii git 1:2.19.0-1 ii needrestart 3.3-1 ii xz-utils 5.2.2-1.3 Versions of packages docker.io suggests: pn aufs-tools ii btrfs-progs 4.17-1 ii debootstrap 1.0.109 pn docker-doc ii e2fsprogs1.44.4-2 pn rinse ii xfsprogs 4.15.1-1 pn zfs-fuse | zfsutils -- no debconf information
