Bug#985149: debootstrap stumbles over tabs in include parameter

[Paul Tagliamonte]

>From a quick glance, it looks like `leftoverdebs` is initially a list of
packages, but in the suite/compoenent loop, leftoverdebs is assigned to the
package sizes.

...
leftoverdebs=$(printf "$leftoverdebs"|tr ' ' '\n'|sort -u|tr '\n' ' ')

leftoverdebs="$(get_package_sizes "$m1" "$pkgdest" $leftoverdebs)"

error 1 LEFTOVERDEBS "Couldn't find these debs: %s" "$leftoverdebs"

The output number appears to be the package sizes. Very confusing output
error message indeed!

  paultag

-- 
:wq

Bug#985149: debootstrap stumbles over tabs in include parameter

[Christian Lamparter]

Hello Steve,

On 16/03/2021 15:12, Steve McIntyre wrote:

On Sat, Mar 13, 2021 at 05:56:26PM +0100, Christian Lamparter wrote:

Package: debootstrap
Version: 1.0.123
Severity: normal

Dear Maintainer,

I ran into an issue when I was using debootstrap in order to setup a chroot-
environment.

I: Retrieving InRelease
I: Checking Release signature
I: Valid Release signature (key id 6D33866EDD8FFA41C0143AEDDCC9EFBF77E11517)
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on http://deb.debian.org/debian...
E: Couldn't find these debs: 48477576

I was surprised to see that "48477576". I expected to get the
name of the package that was causing this, instead of a number.
I dug deeper to see what's causing this and I was surprised
when I found out that the parameter '--include=package'
that was passed to debootstrap was causing this.

Here's a reproducer (can be copied into bash). It fetches the
current debian stable for amd64 with "gzip" as an extra package.
(that $(printf \\t) is in this example, here so the "tab" won't
get lost when copying)


Ummm. Why are you expecting debootstrap to deal with whitespace here?
I'm not convinced this is a bug...


What threw me off is that "E: Couldn't find these debs: 48477576".
48477576 is not a package, right?.

Well, the backstory is: I came across this, because I needed some
extra, but sensible packages in my basic image for it to be useful.

this script look(ed) like this (a mailer with tab support is needed
for proper alignment)

---
PACKAGE="initramfs-tools,gzip,u-boot-tools,device-tree-compiler,\
 debian-keyring-archive,bzip2, ca-certificates,wget,\
 ..."

debootstrap --include="$PACKAGE" ...
---

But it bugged and I had no idea it was because I was using tabs to
align the extra PACKAGE in the next lines because they wouldn't fit
due to 80 cols limit.

Once I figured out that the use of tabs are the issue, I replaced
them with spaces. Then I went on to write a proper bug-report with
a testcase and PoC-patch, in case someone (like future me) stumbles
over this as well: "Check your include/excludes with a fine comb".


But back to the business at hand... Because I would need to know
why you think that a Error message like
"E: Couldn't find these debs: 48477576"
doesn't trigger a "yeah, this 48477576 looks buggy"?

Is there a deeper meaning to the numbers like 48477576?...
or:
 - 29976556? 
()
 - 91369800? 
()
 - 0? ()

that I don't get?

Cheers,
Christian

Bug#985149: debootstrap stumbles over tabs in include parameter

[Steve McIntyre]

Hi Christian,

On Sat, Mar 13, 2021 at 05:56:26PM +0100, Christian Lamparter wrote:
>Package: debootstrap
>Version: 1.0.123
>Severity: normal
>
>Dear Maintainer,
>
>I ran into an issue when I was using debootstrap in order to setup a chroot-
>environment.
>
>I: Retrieving InRelease
>I: Checking Release signature
>I: Valid Release signature (key id 6D33866EDD8FFA41C0143AEDDCC9EFBF77E11517)
>I: Retrieving Packages
>I: Validating Packages
>I: Resolving dependencies of required packages...
>I: Resolving dependencies of base packages...
>I: Checking component main on http://deb.debian.org/debian...
>E: Couldn't find these debs: 48477576
>
>I was surprised to see that "48477576". I expected to get the
>name of the package that was causing this, instead of a number.
>I dug deeper to see what's causing this and I was surprised
>when I found out that the parameter '--include=package'
>that was passed to debootstrap was causing this.
>
>Here's a reproducer (can be copied into bash). It fetches the
>current debian stable for amd64 with "gzip" as an extra package.
>(that $(printf \\t) is in this example, here so the "tab" won't
>get lost when copying)

Ummm. Why are you expecting debootstrap to deal with whitespace here?
I'm not convinced this is a bug...

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
  Armed with "Valor": "Centurion" represents quality of Discipline,
  Honor, Integrity and Loyalty. Now you don't have to be a Caesar to
  concord the digital world while feeling safe and proud.

Bug#985149: debootstrap stumbles over tabs in include parameter

[Christian Lamparter]

Package: debootstrap
Version: 1.0.123
Severity: normal

Dear Maintainer,

I ran into an issue when I was using debootstrap in order to setup a chroot-
environment.

I: Retrieving InRelease
I: Checking Release signature
I: Valid Release signature (key id 6D33866EDD8FFA41C0143AEDDCC9EFBF77E11517)
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on http://deb.debian.org/debian...
E: Couldn't find these debs: 48477576

I was surprised to see that "48477576". I expected to get the
name of the package that was causing this, instead of a number.
I dug deeper to see what's causing this and I was surprised
when I found out that the parameter '--include=package'
that was passed to debootstrap was causing this.

Here's a reproducer (can be copied into bash). It fetches the
current debian stable for amd64 with "gzip" as an extra package.
(that $(printf \\t) is in this example, here so the "tab" won't
get lost when copying)

--- reproducer.sh one-liner ---

/usr/sbin/debootstrap --foreign "--include=$(printf \\t)gzip" --arch=amd64
stable "$(mktemp -d)" "http://deb.debian.org/debian;

--- reproducer.sh EOF ---

Since I now know what to look for, I edited /usr/share/debootstrap/functions
to remove tabs from the $leftoverdebs in the download_release() function.

--- remove-tabs-from-include.patch ---
--- /usr/share/debootstrap/functions.orig   2021-03-13 17:43:18.058708393
+0100
+++ /usr/share/debootstrap/functions2021-03-13 17:43:39.792093090 +0100
@@ -804,7 +804,7 @@ download_release () {
leftoverdebs="$*"

# Fix possible duplicate package names, which would screw up counts:
-   leftoverdebs=$(printf "$leftoverdebs"|tr ' ' '\n'|sort -u|tr '\n' ' ')
+   leftoverdebs=$(printf "$leftoverdebs"|tr -d '\t'|tr ' ' '\n'|sort -u|tr
'\n' ' ')
numdebs=$(printf "$leftoverdebs"|wc -w)

for s in $SUITE $EXTRA_SUITES; do
--- remove-tabs-from-include.patch EOF ---

This "fixed" the issue for me. But I know that this is quick and dirty.

Cheers,
Christian


-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, powerpc, mips, arm64

Kernel: Linux 5.12.0-rc2-wt+ (SMP w/8 CPU threads)
Kernel taint flags: TAINT_OOT_MODULE
Locale: LANG=en_DE.UTF-8, LC_CTYPE=C.UTF-8 (charmap=locale: Cannot set
LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
UTF-8), LANGUAGE=en_US:en
Shell: /bin/sh linked to /bin/bash
Init: systemd (via /run/systemd/system)

Versions of packages debootstrap depends on:
ii  wget  1.21-1+b1

Versions of packages debootstrap recommends:
ii  arch-test   0.17-1
ii  debian-archive-keyring  2021.1.1
ii  gnupg   2.2.27-1

Versions of packages debootstrap suggests:
pn  squid-deb-proxy-client  
pn  ubuntu-archive-keyring  

-- debconf information excluded

-- debsums errors found:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = "en_US:en",
LC_ALL = (unset),
LC_CTYPE = "C.UTF-8",
LANG = "en_DE.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").