Bug#823881: dosfstools: mmd fails right after mkfs.msdos (sectors/tracks mismatch)

[Andreas Bombe]

On Wed, May 11, 2016 at 05:32:17AM +0200, Cyril Brulebois wrote:
> [ Poke Steve. ]
> 
> Andreas Bombe  (2016-05-11):
> > On Tue, May 10, 2016 at 02:15:42PM +0200, Andreas Bombe wrote:
> > > Since 416 blocks is a rather odd value, the default is used and that has
> > > changed. I think mtools is overzealous in checking those values and
> > > refusing to work. Still, it probably makes sense to use 64/32 as the
> > > default for smaller filesystem sizes (up to 512 MB possibly) and I'll
> > > prepare a version that implements this.
> > 
> > Uploading this now.
> > 
> > As far as I'm concerned, I consider this an aesthetic change. There is
> > still no guarantee that the total number of sectors is a multiple of
> > sectors per track. It just happens to work with the current values.
> 
> Steve → we probably don't want to be hardcoding such things in so many
> places right? 3 calls in src:debian-installer, plus debian-cd, and maybe
> others?

In my opinion all that effort to placate mtools is the quintessential
tail wagging the dog. I don't know the installer environment, but
disabling those checks in /etc/mtools.conf or ~/.mtoolsrc would be the
way to go.

> > If you want to make this robust, you'll either have to explicitly
> > specify matched size/sectors/heads on the command line to mkfs.msdos or
> > disable the bogus mtools check like everyone else does when encountering
> > that error.
> 
> Thanks for your input and the proposed change.
> 
> I think Steven mentioned (when we first diagnosed this) a possibly
> bogus/overzealous check on mtools side as well. You seem to agree. So,
> if this check is bogus, why not fix it/remove it upstream then?

Upstream for mtools does not seem to be particularly active, last
release was in 2013.

The problem with this check is that it is at best a heuristic. Total
sectors not being a multiple of sectors per track means that some
sectors in the last track are left unused. And nobody would just waste
some of the scarce space on a floppy, right?

That might indicate something is fishy, but it's not an actual error.
It's definitely meaningless in the linear addressing case of larger
disks where the 255/63 dummy values are used.

> > Seriously, searching for that error message in your favorite search
> > engine will give you pages upon pages of hits, all of them describing
> > how to turn it off.
> 
> Seriously, I read the man^Winfo page and implemented a workaround in
> src:debian-installer already.

I didn't mean to come across as sarcastic or whatever, I just wanted to
note how there are so many people affected by this while I couldn't find
anyone treating it as anything but a nuisance error. So yeah, the
consensus seems to be it's a bogus check.


Andreas

Bug#823881: dosfstools: mmd fails right after mkfs.msdos (sectors/tracks mismatch)

[Cyril Brulebois]

[ Poke Steve. ]

Andreas Bombe  (2016-05-11):
> On Tue, May 10, 2016 at 02:15:42PM +0200, Andreas Bombe wrote:
> > Since 416 blocks is a rather odd value, the default is used and that has
> > changed. I think mtools is overzealous in checking those values and
> > refusing to work. Still, it probably makes sense to use 64/32 as the
> > default for smaller filesystem sizes (up to 512 MB possibly) and I'll
> > prepare a version that implements this.
> 
> Uploading this now.
> 
> As far as I'm concerned, I consider this an aesthetic change. There is
> still no guarantee that the total number of sectors is a multiple of
> sectors per track. It just happens to work with the current values.

Steve → we probably don't want to be hardcoding such things in so many
places right? 3 calls in src:debian-installer, plus debian-cd, and maybe
others?

> If you want to make this robust, you'll either have to explicitly
> specify matched size/sectors/heads on the command line to mkfs.msdos or
> disable the bogus mtools check like everyone else does when encountering
> that error.

Thanks for your input and the proposed change.

I think Steven mentioned (when we first diagnosed this) a possibly
bogus/overzealous check on mtools side as well. You seem to agree. So,
if this check is bogus, why not fix it/remove it upstream then?

> Seriously, searching for that error message in your favorite search
> engine will give you pages upon pages of hits, all of them describing
> how to turn it off.

Seriously, I read the man^Winfo page and implemented a workaround in
src:debian-installer already.

I filed this against dosfstools anyway to make sure our findings /
intuitions were correct.

Thanks for the swift replies.


KiBi.


signature.asc
Description: Digital signature

Bug#823881: dosfstools: mmd fails right after mkfs.msdos (sectors/tracks mismatch)

[Andreas Bombe]

On Tue, May 10, 2016 at 02:15:42PM +0200, Andreas Bombe wrote:
> Since 416 blocks is a rather odd value, the default is used and that has
> changed. I think mtools is overzealous in checking those values and
> refusing to work. Still, it probably makes sense to use 64/32 as the
> default for smaller filesystem sizes (up to 512 MB possibly) and I'll
> prepare a version that implements this.

Uploading this now.

As far as I'm concerned, I consider this an aesthetic change. There is
still no guarantee that the total number of sectors is a multiple of
sectors per track. It just happens to work with the current values.

If you want to make this robust, you'll either have to explicitly
specify matched size/sectors/heads on the command line to mkfs.msdos or
disable the bogus mtools check like everyone else does when encountering
that error.

Seriously, searching for that error message in your favorite search
engine will give you pages upon pages of hits, all of them describing
how to turn it off.


Andreas

Bug#823881: dosfstools: mmd fails right after mkfs.msdos (sectors/tracks mismatch)

[Andreas Bombe]

On Tue, May 10, 2016 at 01:53:24AM +0200, Cyril Brulebois wrote:
> The version bump from 3.0.28-2 to 4.0-1 led to a new FTBFS on all EFI
> archs for debian-installer (amd64, arm64, i386), where the following
> operations are happening:
> | + mkfs.msdos -C ./tmp/netboot-gtk/grub_efi/efi.img 416
> | mkfs.fat 4.0 (2016-05-06)
> | + mmd -i ./tmp/netboot-gtk/grub_efi/efi.img ::efi
> | Total number of sectors (832) not a multiple of sectors per track (63)!
> | Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
> | + cleanup
> | + [ -z efi-image.7vXUPq ]
> | + rm -f efi-image.7vXUPq
> | + [ -z efi-image.u4utfz ]
> | + rm -rf efi-image.u4utfz
> | config/x86.cfg:38: recipe for target 'x86_grub_efi' failed
> 
> This can be reproduced with:
> | make -C build build_netboot-gtk USE_UDEBS_FROM=sid
> 
> after having added "set -x" at the top of build/util/efi-image in
> src:debian-installer.
> 
> After a quick look, it seems the d-i build system is only passing a
> number of blocks to mkfs.{msdos,fat}, without specifying strange
> parameters for sectors or trackers, so it looks to me that mkfs's
> or mmd's behaviour is buggy.
> 
> Incidently, 832 isn't a multiple of 63, but is a multiple of 64. Could
> there be some off-by-one somewhere?

Not an off-by-one, these are constants. Unless the media parameters can
be established (by HDIO_GETGEO or FDGETPRM ioctls) a set of defaults is
used. In 4.0 (I am also upstream) I massively simplified it to basically
use the common dummy 255/63 values unless the size matches one of the
well-known floppy sizes:

https://sources.debian.net/src/dosfstools/4.0-1/src/mkfs.fat.c/#L512 

Previously, it would set values based on well-known floppy sizes or use
64/32 as default if the target was either a file or the device major
number truncated to 8 bits matched 2 (floppy) or 7 (loop device) and
otherwise assume it's a hard disk device so use 255/63 if HDIO_GETGEO
fails:

https://sources.debian.net/src/dosfstools/3.0.28-2/src/mkfs.fat.c/#L512 


Since 416 blocks is a rather odd value, the default is used and that has
changed. I think mtools is overzealous in checking those values and
refusing to work. Still, it probably makes sense to use 64/32 as the
default for smaller filesystem sizes (up to 512 MB possibly) and I'll
prepare a version that implements this.


Andreas

Bug#823881: dosfstools: mmd fails right after mkfs.msdos (sectors/tracks mismatch)

[Cyril Brulebois]

Package: dosfstools
Version: 4.0-1
Severity: serious
Tags: d-i
Justification: d-i build failure/regression, without blatant misusage from 
d-i's side

Control: affects -1 src:debian-installer

(Please keep debian-b...@lists.debian.org and debian-...@lists.debian.org
in the loop when replying.)

Hi,

The version bump from 3.0.28-2 to 4.0-1 led to a new FTBFS on all EFI
archs for debian-installer (amd64, arm64, i386), where the following
operations are happening:
| + mkfs.msdos -C ./tmp/netboot-gtk/grub_efi/efi.img 416
| mkfs.fat 4.0 (2016-05-06)
| + mmd -i ./tmp/netboot-gtk/grub_efi/efi.img ::efi
| Total number of sectors (832) not a multiple of sectors per track (63)!
| Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
| + cleanup
| + [ -z efi-image.7vXUPq ]
| + rm -f efi-image.7vXUPq
| + [ -z efi-image.u4utfz ]
| + rm -rf efi-image.u4utfz
| config/x86.cfg:38: recipe for target 'x86_grub_efi' failed

This can be reproduced with:
| make -C build build_netboot-gtk USE_UDEBS_FROM=sid

after having added "set -x" at the top of build/util/efi-image in
src:debian-installer.

After a quick look, it seems the d-i build system is only passing a
number of blocks to mkfs.{msdos,fat}, without specifying strange
parameters for sectors or trackers, so it looks to me that mkfs's
or mmd's behaviour is buggy.

Incidently, 832 isn't a multiple of 63, but is a multiple of 64. Could
there be some off-by-one somewhere?

Switching on verbose mode (adding -v):
| + mkfs.msdos -v -C ./tmp/netboot-gtk/grub_efi/efi.img 416
| mkfs.fat 4.0 (2016-05-06)
| ./tmp/netboot-gtk/grub_efi/efi.img has 255 heads and 63 sectors per track,

| hidden sectors 0x;
| logical sector size is 512,
| using 0xf8 media descriptor, with 832 sectors;
| drive number 0x80;
| filesystem has 2 12-bit FATs and 4 sectors per cluster.
| FAT size is 1 sector, and provides 199 clusters.
| There is 1 reserved sector.
| Root directory contains 512 slots and uses 32 sectors.
| Volume ID is 0da76041, no volume label.
| + mmd -i ./tmp/netboot-gtk/grub_efi/efi.img ::efi
| Total number of sectors (832) not a multiple of sectors per track (63)!
| Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
| + cleanup
| + [ -z efi-image.AQOJ9R ]
| + rm -f efi-image.AQOJ9R
| + [ -z efi-image.dShhUx ]
| + rm -rf efi-image.dShhUx
| config/x86.cfg:38: recipe for target 'x86_grub_efi' failed

Comparing with the previous version (downgrading to 3.0.28-2):
| + mkfs.msdos -v -C ./tmp/netboot-gtk/grub_efi/efi.img 416
| mkfs.fat 3.0.28 (2015-05-16)
| ./tmp/netboot-gtk/grub_efi/efi.img has 64 heads and 32 sectors per track,
^^^
| hidden sectors 0x;
| logical sector size is 512,
| using 0xf8 media descriptor, with 832 sectors;
| drive number 0x80;
| filesystem has 2 12-bit FATs and 4 sectors per cluster.
| FAT size is 1 sector, and provides 199 clusters.
| There is 1 reserved sector.
| Root directory contains 512 slots and uses 32 sectors.
| Volume ID is 1666ec2f, no volume label.
| + mmd -i ./tmp/netboot-gtk/grub_efi/efi.img ::efi
| + mmd -i ./tmp/netboot-gtk/grub_efi/efi.img ::efi/boot
| + mcopy -i ./tmp/netboot-gtk/grub_efi/efi.img efi-image.qksN5Q/bootx64.efi 
::efi/boot/bootx64.efi
| + grub-cpmodules ./tmp/netboot-gtk/grub_efi x86_64-efi
| + exit 0

It would be great if this could be investigated shortly, as I'd like to
prepare a new d-i release soon:
  https://lists.debian.org/debian-boot/2016/05/msg00055.html


KiBi.