Bug#837537: "Mkdir: file exists" error if any debootstrap file download failed

[Stephan Sürken]

Hi Boyuan!

On Mi, 2016-09-14 at 10:43 +0800, Boyuan Yang wrote:

(...)

> to download *once*. The dir-chroot creating process should either
> fail
> immediately or re-run the debootstrap process again with a warning
> sent to the logger,
> but it just continued as if everything is fine *until* the mkdir
> error
> happens (which is due to the unclean exit of debootstrap, maybe?)

Hmm don't know. I tested around quite a lot lately, and did not
experience such a behavior.

Could you try to reproduce this with 1.0.21? This may have some nicer
logging for these calls, and maybe an excerpt of this log might shed
some light ;).

Thx!

P.S.: Ah: <= 1.0.21 has a little bug in the service file (mini-buildd
might not use the default log level), so you might want to apply this
first (or wait for 1.0.22):

diff --git a/debian/mini-buildd.service b/debian/mini-buildd.service
index 6809dda..0a184f2 100644
--- a/debian/mini-buildd.service
+++ b/debian/mini-buildd.service
@@ -4,6 +4,7 @@ Documentation=man:mini-buildd(8)
 After=remote-fs.target
 
 [Service]
+Environment=MINI_BUILDD_OPTIONS="--verbose"
 EnvironmentFile=-/etc/default/mini-buildd
 User=mini-buildd
 ExecStart=/usr/sbin/mini-buildd $MINI_BUILDD_OPTIONS

Bug#837537: "Mkdir: file exists" error if any debootstrap file download failed

[Boyuan Yang]

Hello,

2016-09-12 20:06 GMT+08:00 Stephan Sürken :
>> I upgraded my Debian build machine to sid for latest 1.0.18 and found
>> it
>> impossible to create any dir chroots.
> (...)
>
> do I understand you correctly that the rollback mechanism can bring
> mini-buildd into that state in case debootstrap fails to work properly
> due to network problems?

Well not really. I just find it strange that when creating a dir chroot,
the debootstrap will fail silently immediately after one package falis
to download *once*. The dir-chroot creating process should either fail
immediately or re-run the debootstrap process again with a warning
sent to the logger,
but it just continued as if everything is fine *until* the mkdir error
happens (which is due to the unclean exit of debootstrap, maybe?)


Thanks,
Boyuan Yang

Bug#837537: "Mkdir: file exists" error if any debootstrap file download failed

[Stephan Sürken]

Hi Boyuan,

On Mo, 2016-09-12 at 18:08 +0800, Boyuan Yang wrote:
> Package: python-mini-buildd
> Version: 1.0.18
> Severity: normal
> 
> I upgraded my Debian build machine to sid for latest 1.0.18 and found
> it
> impossible to create any dir chroots.
(...)

do I understand you correctly that the rollback mechanism can bring
mini-buildd into that state in case debootstrap fails to work properly
due to network problems?

Thanks!

S

Bug#837537: "Mkdir: file exists" error if any debootstrap file download failed

[Boyuan Yang]

Package: python-mini-buildd
Version: 1.0.18
Severity: normal

I upgraded my Debian build machine to sid for latest 1.0.18 and found it
impossible to create any dir chroots.

The Daemon Log says:

2016-09-12 17:49:53,970 mini_buildd.misc (0576): ERROR   : Call
stderr: /bin/mkdir: cannot create directory ‘/var/lib/mini-
buildd/var/chroots/jessie/amd64/tmp’: File exists
2016-09-12 17:49:53,971 mini_buildd.misc (0646): ERROR   : Call
failed: sudo -n /bin/mkdir --verbose /var/lib/mini-
buildd/var/chroots/jessie/amd64/tmp
2016-09-12 17:49:53,972 mini_buildd.misc (0682): ERROR   : Sequence
failed at: 0 (rolling back)

[... misc normal stdout output traceback truncated ...]

2016-09-12 17:49:54,299 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Retrieving libattr1 1:2.4.47-2
2016-09-12 17:49:54,300 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Validating libattr1 1:2.4.47-2
2016-09-12 17:49:54,300 mini_buildd.misc (0576): ERROR   : Call
stdout: W: Retrying failed download of
http://ftp.us.debian.org/debian/pool/main/a/attr/libattr1_2.4.47-2_amd64.deb
2016-09-12 17:49:54,301 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Retrieving libattr1 1:2.4.47-2
2016-09-12 17:49:54,301 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Validating libattr1 1:2.4.47-2
2016-09-12 17:49:54,301 mini_buildd.misc (0646): ERROR   : Call
failed: sudo -n /usr/sbin/debootstrap --variant=buildd --keyring=/var/lib/mini-
buildd/var/chroots/jessie/amd64/keyring.gpg --arch=amd64 jessie /var/lib/mini-
buildd/var/chroots/jessie/amd64/tmp http://ftp.us.debian.org/debian/
2016-09-12 17:49:54,302 mini_buildd.misc (0682): ERROR   : Sequence
failed at: 1 (rolling back)
2016-09-12 17:49:54,316 mini_buildd.misc (0576): ERROR   : Call
stderr: umount: /var/lib/mini-buildd/var/chroots/jessie/amd64/tmp/proc:
mountpoint not found
2016-09-12 17:49:54,316 mini_buildd.misc (0576): ERROR   : Call
stderr: umount: /var/lib/mini-buildd/var/chroots/jessie/amd64/tmp/sys:
mountpoint not found
2016-09-12 17:49:54,317 mini_buildd.misc (0646): ERROR   : Call
failed: sudo -n /bin/umount -v /var/lib/mini-
buildd/var/chroots/jessie/amd64/tmp/proc /var/lib/mini-
buildd/var/chroots/jessie/amd64/tmp/sys
2016-09-12 17:49:54,347 mini_buildd.models.base  (0077): ERROR   : prepare
failed: Debian 'jessie:amd64' (overlay): Command '[u'sudo', u'-n',
u'/usr/sbin/debootstrap', u'--variant=buildd', u'--keyring=/var/lib/mini-
buildd/var/chroots/jessie/amd64/keyring.gpg', u'--arch=amd64', u'jessie',
u'/var/lib/mini-buildd/var/chroots/jessie/amd64/tmp',
u'http://ftp.us.debian.org/debian/']' returned non-zero exit status 2
[base:364]

If we try everything later, *if* any of the file fails to download, everything
would fail after the re-try (see above for "libattr1" package). The package
which fails to download may change(e.g., the example above is the libattr1
package, and the next time it could be coreutils:

*** Another attempt to build jessie dir chroot:

[ ... normal logs truncated ...]
2016-09-12 18:03:28,356 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Validating libdebconfclient0 0.192
2016-09-12 18:03:28,357 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Retrieving libcloog-isl4 0.18.2-1+b2
2016-09-12 18:03:28,357 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Validating libcloog-isl4 0.18.2-1+b2
2016-09-12 18:03:28,358 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Retrieving coreutils 8.23-4
2016-09-12 18:03:28,358 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Validating coreutils 8.23-4
2016-09-12 18:03:28,358 mini_buildd.misc (0576): ERROR   : Call
stdout: W: Retrying failed download of
http://ftp.us.debian.org/debian/pool/main/c/coreutils/coreutils_8.23-4_amd64.deb
2016-09-12 18:03:28,359 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Retrieving coreutils 8.23-4
2016-09-12 18:03:28,359 mini_buildd.misc (0576): ERROR   : Call
stdout: I: Validating coreutils 8.23-4
2016-09-12 18:03:28,360 mini_buildd.misc (0646): ERROR   : Call
failed: sudo -n /usr/sbin/debootstrap --variant=buildd --keyring=/var/lib/mini-
buildd/var/chroots/jessie/amd64/keyring.gpg --arch=amd64 jessie /var/lib/mini-
buildd/var/chroots/jessie/amd64/tmp http://ftp.us.debian.org/debian/
2016-09-12 18:03:28,360 mini_buildd.misc (0682): ERROR   : Sequence
failed at: 1 (rolling back)

)

Looks like it is a logical bug in mini-buildd. The whole debootstrap command
should not fail (or even pass) if there is an attempt to retry failed download
package.



-- System Information:
Debian Release: stretch/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.6.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=zh_CN.UTF-8,