Re: Debian GNU/Hurd 2017 released!

[Thomas Schmitt]

Hi,

Samuel Thibault wrote:
> I rather guess the double-buffering support in grub.

But where would that be controllable ?
grub-mkimage --help has no options for that. Google does not find me
any clue about grub.cfg statements which would promise to do it.


Have a nice day :)

Thomas

Re: Debian GNU/Hurd 2017 released!

[Samuel Thibault]

Thomas Schmitt, on mar. 20 juin 2017 23:01:26 +0200, wrote:
> If i only knew what Vladimir meant with "disable double buffering
> and try again" to diagnose the menu graphics problem.
> Maybe the X extension about double frame buffering:
>   https://www.x.org/releases/X11R7.7/doc/libXext/dbelib.html

I rather guess the double-buffering support in grub.

Samuel

Re: Debian GNU/Hurd 2017 released!

[Thomas Schmitt]

Hi,

Samuel Thibault wrote:
> Files on
> https://people.debian.org/~sthibault/hurd-i386/installer/cdimage/daily/
> are now updated accordingly

Oh yes. Now it looks much better and should be digestible for partition
editors.

  
https://people.debian.org/~sthibault/hurd-i386/installer/cdimage/daily/debian-sid-hurd-i386-NETINST-1.iso

boots on my qemu-system-i386 from -hda.

No unexpectable miracle cure with menu graphics, though.

-

If i only knew what Vladimir meant with "disable double buffering
and try again" to diagnose the menu graphics problem.
Maybe the X extension about double frame buffering:
  https://www.x.org/releases/X11R7.7/doc/libXext/dbelib.html

But i fail to find info how to inquire whether my X can do it and whether
qemu's graphics window uses it.
(Further i would be reluctant to play with the X server on my real
 workstation.)

I tested with a BIOS equipped real 64 bit machine from USB stick. It shows
the GRUB menu just fine. I did not go further, because the box is full
of SATA and other witches' brew from around 2010.


Have a nice day :)

Thomas

Re: Debian GNU/Hurd 2017 released!

[Samuel Thibault]

Hello,

Thomas Schmitt, on mar. 20 juin 2017 14:32:40 +0200, wrote:
> So my proposal to Debian GNU/Hurd is to boldly add option 
>   --protective-msdos-label
> to the debian-installer run of xorriso -as mkisofs for hurd.

Files on
https://people.debian.org/~sthibault/hurd-i386/installer/cdimage/daily/
are now updated accordingly

Samuel

Re: Debian GNU/Hurd 2017 released!

[Thomas Schmitt]

Hi,

my assembler reading skills are obviously insufficient. It turns out that
the bytes in the MBR partition table range are not significant for booting
the ISO from qemu -hda.

I boldly zeroed them in a copy of the GNU/Hurd ISO and it still boots.
(To make sure that it does not boot by El Torito i copied the
 terminating volume descriptor from block 19 to block 17.)

Encouraged by this outcome, i repacked the ISO by these commands:

  # Obtain GRUB image file from start of original ISO
  dd if=debian-hurd-2017-i386-NETINST-1.iso bs=2048 count=16 of=grub_embed

  mount debian-hurd-2017-i386-NETINST-1.iso /mnt/iso

  # Most options as learned from file /mnt/iso/.disk/mkisofs
  # but adding option --protective-msdos-label to create a partition table.
  xorriso  -as mkisofs \
-o test.iso \
-J -joliet-long -r \
-V 'Debian 9.0 h-i386 n' \
--embedded-boot grub_embed \
--protective-msdos-label \
-cache-inodes \
-c boot/boot.cat \
-b boot/grub/grub_eltorito \
   -no-emul-boot -boot-load-size 4 -boot-info-table -no-pad \
/mnt/iso

The resulting test.iso boots as qemu -hda to the same GRUB menu as
the original ISO.

The partition table is what grub-mkrescue would ask for in case of
i386-pc without any efi platform:

  $ sbin/fdisk -lu test.iso
  ...
  Disklabel type: dos
  ...
  Device Boot StartEnd Sectors   Size Id Type
  test.iso1  *1 311555  311555 152.1M cd unknown

This partition is not mountable because of its start offset 1.
But that's how Vladimir wanted it for grub-mkrescue.

--
Udate after receiving reply on grub-devel:
  http://lists.gnu.org/archive/html/grub-devel/2017-06/msg00043.html

Vladimir 'phcoder' Serbinenko wrote:
> Those are only for floppies with old BIOS. If your image is over 2.88 MiB
> and thus never useful on floppies, it's safe to overwrite.

This explains why it looks like somewhat plausible executable code
and why i386-pc/boot.img of Debian 8 is in the state it is in.


--

So my proposal to Debian GNU/Hurd is to boldly add option 
  --protective-msdos-label
to the debian-installer run of xorriso -as mkisofs for hurd.

If an empty partition table is desired instead, then at least zeroize
the 64 bytes beginning at offset 446 of file "grub_embed" before it
gets handed over to xorriso.


Have a nice day :)

Thomas

Re: Debian GNU/Hurd 2017 released!

[Samuel Thibault]

Thomas Schmitt, on mar. 20 juin 2017 11:24:42 +0200, wrote:
> Did you already talk to Steve McIntyre about the appearance of amd64
> on your VM ?

IIRC I reported the issue, I don't have the reference off-hand.

Samuel

Re: Debian GNU/Hurd 2017 released!

[Thomas Schmitt]

Hi,

i wrote:
> > https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.0.0-amd64
-netinst.iso
> > With OVMF, GRUB2 is in charge and works fine.

Samuel Thibault wrote:
> That's probably sheer luck.
> [ Part 2, Image/PNG 545 KB. ]

Wow. That's really unusable.
The appearance of Hurd's GRUB menu is ill, but because of the correct
graphics every second arrow key one can navigate and trust the result
of the key even if it is not visible.

Did you already talk to Steve McIntyre about the appearance of amd64
on your VM ?


> >   -rw-r--r-- 1 root root 512 Mar 23  2015 /usr/lib/grub/i386-pc/boot.img

> Note that the script gets executed on Debian GNU/Hurd, not Debian
> GNU/Linux. Perhaps for some reason there's a bug in the Hurd version.
> ...
> Better investigate the Hurd version first before contacting them,
> perhaps it's a target-specific issue.

It looks as if already the blob boot.img on my amd64 Debian 8 has the
problem with the misused partition table byte range.
(On GNU/Hurd it could only be better than that.)
The first 512 bytes of the 2017 Hurd ISO are the same as in my locally
installed file /usr/lib/grub/i386-pc/boot.img .

The GRUB targets are the same in both cases: "i386-pc", which i understand
is initially the 16 bit aspect of x86 plus BIOS INT services, and maybe
later 32 bit code.
There are no indications that 64 and 32 bit machines are distinguished at
that level.


> Coordination between GNU projects?  You're dreaming a bit :)

rms urges us to do so. Since we don't follow his wish to use Lisp
we have to find other ways to please him. :))


Have a nice day :)

Thomas

Re: Debian GNU/Hurd 2017 released!

[Samuel Thibault]

Thomas Schmitt, on mar. 20 juin 2017 09:42:00 +0200, wrote:
> But it uses a blob as first part of that file:
> 
>   cat "/usr/lib/grub/$platform/boot.img" "$workdir/core.img" > 
> "$outdir/boot/grub/grub_embed"
> 
> I have one on my Debian 8:
>   -rw-r--r-- 1 root root 512 Mar 23  2015 /usr/lib/grub/i386-pc/boot.img

Note that the script gets executed on Debian GNU/Hurd, not Debian
GNU/Linux. Perhaps for some reason there's a bug in the Hurd version.

> Now i wonder whether i should ask Debian's GRUB2 maintainers or directly
> at grub-devel about the use cases where this would be appropriate.

Better investigate the Hurd version first before contacting them,
perhaps it's a target-specific issue.

> I could declare it a matter of coordination among GNU projects. :))

Coordination between GNU projects?  You're dreaming a bit :)

Samuel

Re: Debian GNU/Hurd 2017 released!

[Thomas Schmitt]

Hi,

i found
  
https://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/util/x86-image
which indeed produces a file grub_embed.
But it uses a blob as first part of that file:

  cat "/usr/lib/grub/$platform/boot.img" "$workdir/core.img" > 
"$outdir/boot/grub/grub_embed"

I have one on my Debian 8:
  -rw-r--r-- 1 root root 512 Mar 23  2015 /usr/lib/grub/i386-pc/boot.img

This blob already has the bytes in the partition table which cause
the insane numbers in fdisk and other partition table interpreters.
Disassembling does not look as if the code would try to hop over
the table beginning at 0x1be. I got this command on my cheat sheet:
  objdump -D -b binary -mi386 -Maddr16,data16 $mbr_file | less
but lack true assembler knowledge.

Nevertheless i can see that the MBR of grub-mkrescue has a "ret" before
the partition table begins and that its jumps go either below 0x1b0 or
above 0x1ff.


Now i wonder whether i should ask Debian's GRUB2 maintainers or directly
at grub-devel about the use cases where this would be appropriate.
I could declare it a matter of coordination among GNU projects. :))


Have a nice day :)

Thomas

Re: Debian GNU/Hurd 2017 released!

[Thomas Schmitt]

Hi,

i wrote:
> > I would need to know how the MBR and the subsequent data blocks
> > are produced.

Samuel Thibault wrote:
> The build log is available on 
> https://people.debian.org/~sthibault/hurd-i386/installer/cdimage/daily/debian-cd.log

This mentions grub_embed two times when xorriso gets its name and when
xorriso reports to have used it. But i see no hint how this files gets
created.
It would be interesting to know which documentation and which GRUB
modules lead to that binary.

I googled for "grub_embed". The best match was your changelog
entry from 2014:

  Add usb key boot support for kfreebsd & hurd images. This needs another
  grub image, grub_embed, which has to be very small. Add x86-image script,
  similar to efi-image script, with factorized grub-cpmodules part, to build
  minimal grub images with iso9660 and biosdisk support, and put GRUB modules
  on the ISO image.

I was unable to find kFreeBSD ISOs which have an MBR.

Would that "x86-image script" explain more ?
(https://sources.debian.net/src/debian-cd/unstable/ throws "Internal
 Server Error", so i cannot currently explore debian-cd myself.)


> xorriso is version 1.3.2

It's fully sufficient, although quite old.


> > > > with horizontal and vertical roll-over
> > Do you see the same effects in your tests ?

> Yes. And also with linux/kfreebsd images.

Not with amd64 (can't get i386 to boot with -bios OVMF.fd on my amd64
machine).
  
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.0.0-amd64-netinst.iso
With SeaBIOS, ISOLINUX is in charge and works fine.
With OVMF, GRUB2 is in charge and works fine.

The kFreeBSD mini ISOs i have indeed show the same display problem as
the newest Hurd NETINST.
But debian-7.9.0-kfreebsd-amd64-netinst.iso is OK with SeaBIOS.
It has GRUB 1.99 (which is a GRUB2 before its first official release).

So currently it looks like the problem is with GRUB 2.02 on (Sea)BIOS.
I will ponder how to reproduce it with grub-mkrescue.


Have a nice day :)

Thomas

Re: Debian GNU/Hurd 2017 released!

[Samuel Thibault]

Thomas Schmitt, on mar. 20 juin 2017 00:05:01 +0200, wrote:
> > Ask grub people then :)
> 
> I would need to know how the MBR and the subsequent data blocks
> are produced.

The build log is available on 

https://people.debian.org/~sthibault/hurd-i386/installer/cdimage/daily/debian-cd.log

xorriso is version 1.3.2

> > > although the menu switches by every press of an arrow key from dark purple
> > > with horizontal and vertical roll-over to grey cyan with correct geometry.
> 
> > Again, a grub issue :)
> 
> Do you see the same effects in your tests ?

Yes. And also with linux/kfreebsd images.

> (And why is your /boot/grub/grub_eltorito not needy of xorrisofs
>  option --grub2-boot-info ?)

I haven't looked into all these so I don't know, I'm just using what
happens to be there in the Debian scripts.

Samuel

Re: Debian GNU/Hurd 2017 released!

[Thomas Schmitt]

Hi,

i wrote:
> > i see a quite insane MBR partition table:

Samuel Thibault wrote:
> That's not surprising by nowadays' """ISO""" image standard: they are
> both valid as CD image, usb stick image, etc.

Sure. But normally the MBR has a valid partition table.
Not only the ISOLINUX MBR of Debian x86 ISOs but also the MBR of
grub-mkrescue.
My program xorriso writes those partition tables if it is told to do so.
But in the case of the grub_embed MBR there seems to be code where the
partition tables is supposed to be. So it would not be a good idea to
simply add option --protective-msdos-label to the xorriso -as mkisofs run.


> > One should ask the provider of grub_embed

> Ask grub people then :)

I would need to know how the MBR and the subsequent data blocks
are produced. (And for what device an MBR with garbled partition table
would be deemed suitable, if that is known.)

The code in the MBR of grub-mkrescue obviously hops onto the program in
the El Torito boot image. So it can be small enough to leave bytes
446 ff. unused.
I refer to grub-mkrescue because it is the official upstream proposal to
prepare the GRUB2 entrails and the xorrisofs options for a GRUB2 bootable
ISO.


> > although the menu switches by every press of an arrow key from dark purple
> > with horizontal and vertical roll-over to grey cyan with correct geometry.

> Again, a grub issue :)

Do you see the same effects in your tests ?
If not, with what virtual firmware do you test ?

(And why is your /boot/grub/grub_eltorito not needy of xorrisofs
 option --grub2-boot-info ?)


Have a nice day :)

Thomas

Re: Debian GNU/Hurd 2017 released!

[Samuel Thibault]

Hello,

Thomas Schmitt, on dim. 18 juin 2017 11:07:36 +0200, wrote:
>   
> http://ftp.ports.debian.org/debian-ports-cd/hurd-i386/debian-hurd-2017/debian-hurd-2017-i386-NETINST-1.iso
> 
> i see a quite insane MBR partition table:

That's not surprising by nowadays' """ISO""" image standard: they are
both valid as CD image, usb stick image, etc.

> One should ask the provider of grub_embed whether it is intentional to
> have a MBR signature in bytes 510 and 511 and to have the cleartext word
> "Floppy" in the range where the MBR is supposed to expose its partition
> table.

Ask grub people then :)

> although the menu switches by every press of an arrow key from dark purple
> with horizontal and vertical roll-over to grey cyan with correct geometry.

Again, a grub issue :)

Samuel

Re: Debian GNU/Hurd 2017 released!

[Matthias.L2]

Thank you developers for your constant hard work on Debian Hurd!

Re: Debian GNU/Hurd 2017 released!

[Arne Babenhauserheide]

Samuel Thibault  writes:

> It is with huge pleasure that the Debian GNU/Hurd team announces the
> release of Debian GNU/Hurd 2017.

That’s awesome! Thank you!

I’d just like to note that "Hurd in 140 letters command" still works:
http://www.draketo.de/english/free-software/howto-hurd-140-chars

wget http://people.debian.org/~sthibault/hurd-i386/debian-hurd.img.tar.gz;
tar xf de*hu*gz; qemu-system-x86_64 -hda de*hu*g -m 1G

> * It is now possible to run subhurds as unprivileged user, thus
> providing easy lightweight virtualization.

It would be great to have a tutorial which shows starting a subhurd very
concisely — ideally starting with the qemu image as above so other people
can follow the tutorial with minimal fuss.

Subhurds are one of the features which might actually get companies
interested in the Hurd.

Can we already run subhurds from a shared readonly qemu image with
per-subhurd unionfs translators to store all writes in separate
locations?

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken


signature.asc
Description: PGP signature

Re: Debian GNU/Hurd 2017 released!

[Thomas Schmitt]

Hi,

in

  
http://ftp.ports.debian.org/debian-ports-cd/hurd-i386/debian-hurd-2017/debian-hurd-2017-i386-NETINST-1.iso

i see a quite insane MBR partition table:

  $ /sbin/fdisk -lu debian-hurd-2017-i386-NETINST-1.iso
  ...
  Disklabel type: dos
  ...
  Device   Boot  StartEndSectors   
Size Id Type
  debian-hurd-2017-i386-NETINST-1.iso1 ?3451924861 3662313359  210388499 
100.3G  0 Empty
  debian-hurd-2017-i386-NETINST-1.iso2 ?2766929882 4653280287 1886350406 
899.5G be Solaris 
  debian-hurd-2017-i386-NETINST-1.iso3 ?  28891953 3082391602 3053499650   
1.4T  0 Empty
  debian-hurd-2017-i386-NETINST-1.iso4  4278184271 4278184271  0
 0B d4 unknown

The bytes of the MBR stem obviously from this xorrisofs option (learned
from file /.disk/mkisofs in the ISO):

  --embedded-boot boot1/boot/grub/grub_embed

One should ask the provider of grub_embed whether it is intentional to
have a MBR signature in bytes 510 and 511 and to have the cleartext word
"Floppy" in the range where the MBR is supposed to expose its partition
table.
Especially strange is that the MBR code contains lots of zeros in its
first 80 bytes. Can it be that the code should be moved ~ 80 bytes
lower and rather hop over a more plausible partition table beginning at
byte 446 ?


The MBR code itself seems to be functional. I get to a GRUB menu by
  $ qemu-system-x86_64 -enable-kvm -m 4096 -hda 
debian-hurd-2017-i386-NETINST-1.iso
although the menu switches by every press of an arrow key from dark purple
with horizontal and vertical roll-over to grey cyan with correct geometry.


This menu display defect is not bound to MBR booting. If i erase the MBR
and boot by -cdrom, i get to the same situation.
(It is not 100% sure that -hda uses MBR and -cdrom uses El Torito because
 some virtual BIOSes are known to boot El Torito from -hda and MBR from
 -cdrom. So to be sure, one has to cripple the unwanted boot starting point.
 The default virtual BIOS of Debian 8 qemu is ok in this aspect, nevertheless.)


Have a nice day :)

Thomas

Re: Debian GNU/Hurd 2017 released!

[m . s . yegeu]

Merci beaucoup to All for big work