Bug#865586: live-build: binary_hdd failed with mkfs.vfat error

[Raphael Hertzog]

Hello,

On Tue, 29 Aug 2017, Matthijs Kooijman wrote:
> I also ran into this same error a while ago. It is caused by partition
> devices being created when parted creates partitions, which are not
> cleaned up when the loop devices are cleaned up. I'm attaching a patch
> that fixes this, by passing --partscan to losetup to clear out any
> lingering partition devices when the partition itself is mounted.

Thanks, applied in git.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/

Bug#865586: live-build: binary_hdd failed with mkfs.vfat error

[Matthijs Kooijman]

Package: live-build
Version: 1:20170807
Followup-For: Bug #865586

Hi,

I also ran into this same error a while ago. It is caused by partition
devices being created when parted creates partitions, which are not
cleaned up when the loop devices are cleaned up. I'm attaching a patch
that fixes this, by passing --partscan to losetup to clear out any
lingering partition devices when the partition itself is mounted.

It would be more correct to clean up the partition devices when the loop
device is removed, but I'm not quite sure whose responsibility that
would be.

Gr.

Matthijs
>From 05e0703d82e0189e40d88f5070b1c2f955a0ad9e Mon Sep 17 00:00:00 2001
From: Matthijs Kooijman 
Date: Tue, 29 Aug 2017 15:04:31 +0200
Subject: [PATCH 3/4] Pass --partscan to losetup

Recent versions of Linux, parted or some other bit of software cause
partition devices, like /dev/loop0p1 to be created when running parted
mkpart. However, these devices are not cleaned up when running
losetup -d to remove /dev/loop0 later, so they linger around and confuse
mkfs (which refuses to make a filesystem, thinking there are partitions):

	mkfs.fat 4.1 (2017-01-24)
	mkfs.vfat: Partitions or virtual mappings on device '/dev/loop0', not making filesystem (use -I to override)

To prevent this behaviour, pass --partscan to losetup when adding a new
partition, to clean up any lingering partitions. It seems losetup does not
accept --partscan when deleting a loop device, to clean up at that point, but
since binary_hdd mounts the partition last, there should not be any lingering
partition devices after live-build is done.

The --partscan option is available since util-linux 2.21 (released in 2012), so
it should be fairly safe to pass it unconditionally.
---
 functions/losetup.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/functions/losetup.sh b/functions/losetup.sh
index 0346ff677..9d9b46e5d 100755
--- a/functions/losetup.sh
+++ b/functions/losetup.sh
@@ -40,7 +40,7 @@ Losetup ()
 	FILE="${2}"
 	PARTITION="${3:-1}"
 
-	${LB_LOSETUP} --read-only "${DEVICE}" "${FILE}"
+	${LB_LOSETUP} --read-only --partscan "${DEVICE}" "${FILE}"
 	FDISK_OUT="$(${LB_FDISK} -l -u ${DEVICE} 2>&1)"
 	Lodetach "${DEVICE}"
 
@@ -50,14 +50,14 @@ Losetup ()
 	then
 		Echo_message "Mounting %s with offset 0" "${DEVICE}"
 
-		${LB_LOSETUP} "${DEVICE}" "${FILE}"
+		${LB_LOSETUP} --partscan "${DEVICE}" "${FILE}"
 	else
 		SECTORS="$(echo "$FDISK_OUT" | sed -ne "s|^$LOOPDEVICE[ *]*\([0-9]*\).*|\1|p")"
 		OFFSET="$(expr ${SECTORS} '*' 512)"
 
 		Echo_message "Mounting %s with offset %s" "${DEVICE}" "${OFFSET}"
 
-		${LB_LOSETUP} -o "${OFFSET}" "${DEVICE}" "${FILE}"
+		${LB_LOSETUP} --partscan -o "${OFFSET}" "${DEVICE}" "${FILE}"
 	fi
 }
 
-- 
2.11.0

Bug#865586: live-build: binary_hdd failed with mkfs.vfat error

[Thierry]

Hi,

it would be nice to have the simple fix suggested by Grégory DAVID
(binary_hdd-mkfs.vfat-error.patch) in stretch so that we can build hdd
images again.

Ciao,
Thierry

Bug#865586: live-build: binary_hdd failed with mkfs.vfat error

[Thierry]

Hi,

i fall into the same bug while upgrading from jessie to stretch.

Note however that, with the version of live-build shipped with stretch
(20170213) and the '--distribution' set to 'jessie', there is no problem,
so the issue seems to come from a difference of behaviour between both
version of mkfs.vfat (not a change in live-build source code), the one
shipped with stretch seems more picky.

Adding a '-I' to the options of mkfs.vfat fixes the issue, but there might
be a cleaner way, by specifying the partition, not an offset corresponding
to its location on the loop device.

Ciao,
Thierry

Bug#865586: live-build: binary_hdd failed with mkfs.vfat error

[pfoo]

Seems like I'm hitting the same issue with hdd image. iso-hybrid is working 
fine.

config (not hook or any other modification after this) :
lb config --distribution stretch --binary-images hdd --architectures amd64 
--linux-flavours amd64 \
--archive-areas "main contrib non-free" --apt-indices false --security true 
--updates true \
--mirror-bootstrap http://ftp.fr.debian.org/debian/ --mirror-binary 
http://deb.debian.org/debian/ --bootloaders syslinux \
--bootappend-live "boot=live components timezone=Europe/Paris 
locales=en_US.UTF-8 keyboard-layouts=fr hostname=rescue \
username=user noprompt noeject autologin"

Result : 
0+0 records in
0+0 records out
0 bytes (0 B) copied, 7,7539e-05 s, 0,0 kB/s
!!! The following error/warning messages can be ignored !!!
P: Mounting /dev/loop2 with offset 0
1+0 records in
1+0 records out
440 bytes (440 B) copied, 0,000428679 s, 1,0 MB/s
P: Mounting /dev/loop2 with offset 1048576
mkfs.fat 4.1 (2017-01-24)
mkfs.vfat: Partitions or virtual mappings on device '/dev/loop2', not making 
filesystem (use -I to override)
P: Begin unmounting filesystems...
P: Saving caches...
Reading package lists... Done
Building dependency tree   
Reading state information... Done

Bug#865586: live-build: binary_hdd failed with mkfs.vfat error

[Raphael Hertzog]

On Thu, 22 Jun 2017, Grégory DAVID wrote:
>With a simple live-build config,

What is your simple live-build config? Please share your config directory
and/or the command line options that you are using.

Thank you!
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/

Bug#865586: live-build: binary_hdd failed with mkfs.vfat error

[Grégory DAVID]

Package: live-build
Version: 1:20170213
Severity: important

Dear Maintainer,

   The 'lb binary_hdd' script halt with a mkfs.vfat error when build a FAT32 or 
FAT16 hdd image
   
   With a simple live-build config, I launched the whole building process that 
halt with this error (that is not shown as error):

   [2017-06-22 20:12:00] lb binary_hdd
   P: Begin building binary hdd image...
   0+0 records in
   0+0 records out
   0 bytes copied, 2.743e-05 s, 0.0 kB/s
   !!! The following error/warning messages can be ignored !!!
   P: Mounting /dev/loop1 with offset 0
   1+0 records in
   1+0 records out
   440 bytes copied, 0.0265552 s, 16.6 kB/s
   P: Mounting /dev/loop1 with offset 1048576
   mkfs.fat 4.1 (2017-01-24)
   mkfs.vfat: Partitions or virtual mappings on device '/dev/loop1', not making 
filesystem (use -I to override)
   P: Begin unmounting filesystems...
   P: Saving caches...
   Reading package lists... Done
   Building dependency tree
   Reading state information... Done


   It seems that mkfs.vfat gives the answer: '-I' in command call


-- Package-specific info:

-- System Information:
Debian Release: 9.0
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), 
LANGUAGE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages live-build depends on:
ii  debootstrap  1.0.90

Versions of packages live-build recommends:
ii  apt-utils   1.4.6
ii  cpio2.11+dfsg-6
ii  live-boot-doc   1:20170112
ii  live-config-doc 5.20170316
ii  live-manual-html [live-manual]  2:20151217.1
ii  wget1.19.1-3

live-build suggests no packages.

-- no debconf information
--- binary_hdd-orig 2017-06-22 20:16:11.927603985 +0200
+++ binary_hdd  2017-06-22 20:16:39.211437132 +0200
@@ -223,13 +223,13 @@
 
fat16)
MKFS="vfat"
-   MKFS_OPTIONS="-F 16 -n ${LB_HDD_LABEL}"
+   MKFS_OPTIONS="-F 16 -n ${LB_HDD_LABEL} -I"
MOUNT_OPTIONS=""
;;
 
fat32)
MKFS="vfat"
-   MKFS_OPTIONS="-F 32 -n ${LB_HDD_LABEL}"
+   MKFS_OPTIONS="-F 32 -n ${LB_HDD_LABEL} -I"
MOUNT_OPTIONS=""
;;