Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-29 Thread Gregor Riepl
>> How can the installation system eat so much memory?
> 
> It shouldn't. There is even a lowmem udeb which reduces the memory footprint 
> [1].
> 
> I could even install with debian-installer on my Amiga 1200 with 64 MiB.
> 
> Please don't necessarily take tests on qemu as a reference. Rather test on 
> real
> hardware.

Ah, cool.

I probably need to reinstall soon, so should be able to test the new CD.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread Mark Cave-Ayland
On 28/04/2019 13:35, John Paul Adrian Glaubitz wrote:

> On 4/28/19 11:13 AM, Mark Cave-Ayland wrote:
>>> I'm not sure this is actually required. debian-installer does actually have 
>>> a low-mem
>>> installation mode with which I could the system even on 64 MiB.
>>
>> Oh I didn't know that! From memory when you first started the ports images 
>> the
>> minimum memory required under qemu-system-sparc64 for the ISOs was 128M, 
>> then about
>> 18 months ago it jumped to 256M and in the most recent it seems to be 512M.
> 
> I will re-test on my Amiga 1200 once time permits. I also have two Pegasus II 
> here
> which also could be useful for that test.

I should add that I suspect some of this comes from the kernel/initrd. IIRC the 
jump
from 128M to 256M occurred around the time when there were discussions on some 
of the
kernel mailing lists related to alignment of various memory areas, but it wasn't
something I really looked into at the time.

Out of interest how do you enable the low memory mode during installation?

 One other slight annoyance is that when the grub menu loads it seems to 
 hammer the
 CPU for the first 60s or so which makes selection almost impossible. I'll 
 see if I
 can find some time to look at this later.
>>>
>>> I have not seen this on real hardware. Worked just fine.
>>
>> My guess is that this is related to the way in which the keyboard/screen are 
>> being
>> polled trigging something in the emulation, so I see this as being a
>> qemu-system-sparc64 issue and not something to block the GRUB work.
> 
> Okay, good to know.
> 
 I remember there being issues translating OF
 pathnames for the OS, but was there progress using grub for HD 
 installation and also
 the installation media?
>>> The pathname translation issue was for grub-installer, i.e. the part where 
>>> the installer
>>> installs GRUB to the target hard disk.
>>
>> (goes and looks)
>>
>> Wow there's a huge backlog of grub-related messages there :/  Do you have a 
>> link to
>> the latest summary update? I can start poking at the outstanding tasks as 
>> time allows.
> 
> The main issue we have is still the bug in ofpathname. We haven't really 
> agreed on
> which utility to use. Currently we're ripping ofpath out of Yaboot which works
> fine on PowerMacs.

Okay thanks, in that case let's continue the discussion over on the debian-ppc 
lists.


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread John Paul Adrian Glaubitz
On 4/28/19 11:13 AM, Mark Cave-Ayland wrote:
>> I'm not sure this is actually required. debian-installer does actually have 
>> a low-mem
>> installation mode with which I could the system even on 64 MiB.
> 
> Oh I didn't know that! From memory when you first started the ports images the
> minimum memory required under qemu-system-sparc64 for the ISOs was 128M, then 
> about
> 18 months ago it jumped to 256M and in the most recent it seems to be 512M.

I will re-test on my Amiga 1200 once time permits. I also have two Pegasus II 
here
which also could be useful for that test.

>>> One other slight annoyance is that when the grub menu loads it seems to 
>>> hammer the
>>> CPU for the first 60s or so which makes selection almost impossible. I'll 
>>> see if I
>>> can find some time to look at this later.
>>
>> I have not seen this on real hardware. Worked just fine.
> 
> My guess is that this is related to the way in which the keyboard/screen are 
> being
> polled trigging something in the emulation, so I see this as being a
> qemu-system-sparc64 issue and not something to block the GRUB work.

Okay, good to know.

>>> I remember there being issues translating OF
>>> pathnames for the OS, but was there progress using grub for HD installation 
>>> and also
>>> the installation media?
>> The pathname translation issue was for grub-installer, i.e. the part where 
>> the installer
>> installs GRUB to the target hard disk.
> 
> (goes and looks)
> 
> Wow there's a huge backlog of grub-related messages there :/  Do you have a 
> link to
> the latest summary update? I can start poking at the outstanding tasks as 
> time allows.

The main issue we have is still the bug in ofpathname. We haven't really agreed 
on
which utility to use. Currently we're ripping ofpath out of Yaboot which works
fine on PowerMacs.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread John Paul Adrian Glaubitz
On 4/28/19 11:10 AM, Thomas Schmitt wrote:
> John Paul Adrian Glaubitz wrote:
>> You are very welcome to create a guest account for Debian Salsa
> 
> I already have one for xorriso related package preparation.

Ok, great!

>> and open
>> merge requests for the debian-installer and debian-cd projects [1].
> 
> It is more about weighing the options and their consequences. Showing
> shell command results. Convincing the maintainer.
> The proposed changes themselves are quite artless.

How about starting a discussion on the debian-boot mailing list then? I'm
also subscribed there :).

>> Note: The ia64 stuff for debian-cd was copied verbatim from arm64 (boot-ia64
>> is copied from boot-arm64 in debian-cd).
> 
> UEFI: Five CPU architectures in one specification.
> 
> Maybe if i convince you of my proposal i can use the result as lure for
> the other four EFI arches of debian-cd. :))

Sounds good :). The more unification and simplification, the better.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread Mark Cave-Ayland
On 28/04/2019 09:49, John Paul Adrian Glaubitz wrote:

> On 4/28/19 10:33 AM, Mark Cave-Ayland wrote:
 https://cdimage.debian.org/cdimage/ports/grub-test/
>>
>> I grabbed the test image from the above URL to test under 
>> qemu-system-sparc64 and
>> initially got the error below:
> 
> Please note: This image is not particularly tested with regards the installer 
> itself,
> so please let's not start a discussion regarding other issues. The images are 
> solely
> for testing the GRUB boot itself.
> 
>> After some further poking it looks like the latest images bump the minimum 
>> memory
>> requirements from 256M to 512M so qemu-system-sparc64 needs to run with -m 
>> 512.
> 
> I'm not sure this is actually required. debian-installer does actually have a 
> low-mem
> installation mode with which I could the system even on 64 MiB.

Oh I didn't know that! From memory when you first started the ports images the
minimum memory required under qemu-system-sparc64 for the ISOs was 128M, then 
about
18 months ago it jumped to 256M and in the most recent it seems to be 512M.

Certainly let's not mix this with the grub discussion: sadly I don't have 
access to
real SPARC hardware but in a separate thread I'd be interested if you had a 
moment to
test the ISOs in a lower memory LDOM so I can determine whether 
qemu-system-sparc64
behaviour matches that of real hardware. If it isn't then my job is to fix 
that. And
of course don't let this stop you going ahead with merging the GRUB patches :)

>> One other slight annoyance is that when the grub menu loads it seems to 
>> hammer the
>> CPU for the first 60s or so which makes selection almost impossible. I'll 
>> see if I
>> can find some time to look at this later.
> 
> I have not seen this on real hardware. Worked just fine.

My guess is that this is related to the way in which the keyboard/screen are 
being
polled trigging something in the emulation, so I see this as being a
qemu-system-sparc64 issue and not something to block the GRUB work.

 Patches look good, although the second patch still uses 
 /boot/grub/sparc64.elf
 instead of /boot/grub/core.img for the grub-mkimage output parameter - is 
 that correct?
>>> Good catch. Probably forgot a "git commit -a -amend" here.
>>>
>>> Btw, any suggestions for powerpc/ppc64? We are still using Yaboot here. I 
>>> assume this
>>> case will be a bit more complicated due to the fact we also need to support 
>>> PowerMacs.
>>
>> What's the current status? Sorry I had to drop out of that thread but I've 
>> recently
>> had a break because I got married :)
> 
> Congrats!
> 
>> I remember there being issues translating OF
>> pathnames for the OS, but was there progress using grub for HD installation 
>> and also
>> the installation media?
> The pathname translation issue was for grub-installer, i.e. the part where 
> the installer
> installs GRUB to the target hard disk.

(goes and looks)

Wow there's a huge backlog of grub-related messages there :/  Do you have a 
link to
the latest summary update? I can start poking at the outstanding tasks as time 
allows.


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread Thomas Schmitt
Hi,

John Paul Adrian Glaubitz wrote:
> You are very welcome to create a guest account for Debian Salsa

I already have one for xorriso related package preparation.


> and open
> merge requests for the debian-installer and debian-cd projects [1].

It is more about weighing the options and their consequences. Showing
shell command results. Convincing the maintainer.
The proposed changes themselves are quite artless.


> Note: The ia64 stuff for debian-cd was copied verbatim from arm64 (boot-ia64
> is copied from boot-arm64 in debian-cd).

UEFI: Five CPU architectures in one specification.

Maybe if i convince you of my proposal i can use the result as lure for
the other four EFI arches of debian-cd. :))


In another mail, Mark Cave-Ayland wrote:
> I've had a break because I got married :)

Congratulations !


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread John Paul Adrian Glaubitz
On 4/28/19 10:40 AM, Gregor Riepl wrote:
> That's pretty bad!
> 
> I stuffed my old Ultra 10 with 512MB, but the machine originally came with
> only 256MB.
> 
> How can the installation system eat so much memory?

It shouldn't. There is even a lowmem udeb which reduces the memory footprint 
[1].

I could even install with debian-installer on my Amiga 1200 with 64 MiB.

Please don't necessarily take tests on qemu as a reference. Rather test on real
hardware.

Adrian

> [1] https://packages.qa.debian.org/l/lowmem.html

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread John Paul Adrian Glaubitz
On 4/28/19 10:33 AM, Mark Cave-Ayland wrote:
>>> https://cdimage.debian.org/cdimage/ports/grub-test/
> 
> I grabbed the test image from the above URL to test under qemu-system-sparc64 
> and
> initially got the error below:

Please note: This image is not particularly tested with regards the installer 
itself,
so please let's not start a discussion regarding other issues. The images are 
solely
for testing the GRUB boot itself.

> After some further poking it looks like the latest images bump the minimum 
> memory
> requirements from 256M to 512M so qemu-system-sparc64 needs to run with -m 
> 512.

I'm not sure this is actually required. debian-installer does actually have a 
low-mem
installation mode with which I could the system even on 64 MiB.

> One other slight annoyance is that when the grub menu loads it seems to 
> hammer the
> CPU for the first 60s or so which makes selection almost impossible. I'll see 
> if I
> can find some time to look at this later.

I have not seen this on real hardware. Worked just fine.

>>> Patches look good, although the second patch still uses 
>>> /boot/grub/sparc64.elf
>>> instead of /boot/grub/core.img for the grub-mkimage output parameter - is 
>>> that correct?
>> Good catch. Probably forgot a "git commit -a -amend" here.
>>
>> Btw, any suggestions for powerpc/ppc64? We are still using Yaboot here. I 
>> assume this
>> case will be a bit more complicated due to the fact we also need to support 
>> PowerMacs.
> 
> What's the current status? Sorry I had to drop out of that thread but I've 
> recently
> had a break because I got married :)

Congrats!

> I remember there being issues translating OF
> pathnames for the OS, but was there progress using grub for HD installation 
> and also
> the installation media?
The pathname translation issue was for grub-installer, i.e. the part where the 
installer
installs GRUB to the target hard disk.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread Gregor Riepl
> Apr 28 08:11:44 debconf: Setting debconf/priority to medium
> Apr 28 08:11:44 kernel: [  135.647875] main-menu[242]: segfault at 8 ip
> 0100298c (rpc f80100128e08) sp 07feff83ead1 error 1 in
> main-menu[100+4000]
> 
> After some further poking it looks like the latest images bump the minimum 
> memory
> requirements from 256M to 512M so qemu-system-sparc64 needs to run with -m 
> 512.

That's pretty bad!

I stuffed my old Ultra 10 with 512MB, but the machine originally came with
only 256MB.

How can the installation system eat so much memory?



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread Mark Cave-Ayland
On 28/04/2019 08:44, John Paul Adrian Glaubitz wrote:

> On 4/28/19 9:25 AM, Mark Cave-Ayland wrote:
 Thanks everyone for the help!
>>>
>>> Attaching the two patches for debian-installer and debian-cd for reference,
>>> will commit them next week after cleaning them up and also switching ia64
>>> to GRUB as well.
>>
>> Amazing - thank you for all the hard work you've put into this! I've just 
>> had a look
>> in your home directory on people.debian.org for the GRUB test ISO but I 
>> can't see it
>> - have you removed it pending application of your patches?
> 
> I moved the images to the cdimage server (there is also a test image for ia64 
> with
> GRUB), as the space on people.debian.org is limited:
> 
>> https://cdimage.debian.org/cdimage/ports/grub-test/

I grabbed the test image from the above URL to test under qemu-system-sparc64 
and
initially got the error below:

Apr 28 08:11:31 debconf: Setting debconf/language to en
Apr 28 08:11:32 main-menu[237]: INFO: Menu item 'localechooser' selected
Apr 28 08:11:32 main-menu[242]: INFO: Falling back to the package description 
for
brltty-udeb
Apr 28 08:11:33 main-menu[242]: INFO: Menu item 'localechooser' selected
Apr 28 08:11:34 main-menu[242]: /var/lib/dpkg/status: No such file or directory
Apr 28 08:11:34 main-menu[242]: /var/lib/dpkg/status: No such file or directory
Apr 28 08:11:34 main-menu[242]: WARNING **: Configuring 
'libdebian-installer4-udeb'
failed with error code 1
Apr 28 08:11:34 main-menu[242]: WARNING **: Menu item 'localechooser' failed.
Apr 28 08:11:34 main-menu[237]: /var/lib/dpkg/status: No such file or directory
Apr 28 08:11:34 main-menu[237]: /var/lib/dpkg/status: No such file or directory
Apr 28 08:11:34 main-menu[237]: WARNING **: Configuring 'cdebconf-udeb' failed 
with
error code 1
Apr 28 08:11:34 main-menu[237]: WARNING **: Menu item 'localechooser' failed.
Apr 28 08:11:44 main-menu[242]: INFO: Modifying debconf priority limit from 
'high' to
'medium'
Apr 28 08:11:44 debconf: Setting debconf/priority to medium
Apr 28 08:11:44 kernel: [  135.647875] main-menu[242]: segfault at 8 ip
0100298c (rpc f80100128e08) sp 07feff83ead1 error 1 in
main-menu[100+4000]

After some further poking it looks like the latest images bump the minimum 
memory
requirements from 256M to 512M so qemu-system-sparc64 needs to run with -m 512.

One other slight annoyance is that when the grub menu loads it seems to hammer 
the
CPU for the first 60s or so which makes selection almost impossible. I'll see 
if I
can find some time to look at this later.

>> Patches look good, although the second patch still uses 
>> /boot/grub/sparc64.elf
>> instead of /boot/grub/core.img for the grub-mkimage output parameter - is 
>> that correct?
> Good catch. Probably forgot a "git commit -a -amend" here.
> 
> Btw, any suggestions for powerpc/ppc64? We are still using Yaboot here. I 
> assume this
> case will be a bit more complicated due to the fact we also need to support 
> PowerMacs.

What's the current status? Sorry I had to drop out of that thread but I've 
recently
had a break because I got married :)  I remember there being issues translating 
OF
pathnames for the OS, but was there progress using grub for HD installation and 
also
the installation media?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread John Paul Adrian Glaubitz
Hi!

On 4/28/19 10:20 AM, Thomas Schmitt wrote:
> Interesting. I never had an ISO for Itanium.
> Looks very similar to the Debian arm64 ISOs. Much neater MBR partition
> table than i386 and amd64 ISOs.
> 
> I still see room for improvements (for arm64 too):
> 
> - Use xorrisofs option -partition_offset 16 to get the full image size
>   including the appended partition from commands like /sbin/isosize.
> 
> - Use argument
> --interval:appended_partition_2:all::
>   to option -e, in order to point El Torito to the appended partition.
>   The image file ./CD1/boot/grub/efi.img may then be moved out of ./CD1
>   so that it does not eat room in the ISO filesystem.
> 
> Where to motivate and discuss these ?

You are very welcome to create a guest account for Debian Salsa and open
merge requests for the debian-installer and debian-cd projects [1]. I think
your expertise is very appreciated :-).

The projects can be found here:

> https://salsa.debian.org/images-team/debian-cd
> https://salsa.debian.org/installer-team/debian-installer

There is certainly a lot of room for improvement.

Note: The ia64 stuff for debian-cd was copied verbatim from arm64 (boot-ia64
is copied from boot-arm64 in debian-cd).

Adrian

> [1] https://salsa.debian.org/users/sign_in?redirect_to_referer=yes

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread Thomas Schmitt
Hi,

John Paul Adrian Glaubitz wrote:
> there is also a test image for ia64 with GRUB
> https://cdimage.debian.org/cdimage/ports/grub-test/

Interesting. I never had an ISO for Itanium.
Looks very similar to the Debian arm64 ISOs. Much neater MBR partition
table than i386 and amd64 ISOs.

I still see room for improvements (for arm64 too):

- Use xorrisofs option -partition_offset 16 to get the full image size
  including the appended partition from commands like /sbin/isosize.

- Use argument
--interval:appended_partition_2:all::
  to option -e, in order to point El Torito to the appended partition.
  The image file ./CD1/boot/grub/efi.img may then be moved out of ./CD1
  so that it does not eat room in the ISO filesystem.

Where to motivate and discuss these ?


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread John Paul Adrian Glaubitz
On 4/28/19 9:25 AM, Mark Cave-Ayland wrote:
>>> Thanks everyone for the help!
>>
>> Attaching the two patches for debian-installer and debian-cd for reference,
>> will commit them next week after cleaning them up and also switching ia64
>> to GRUB as well.
> 
> Amazing - thank you for all the hard work you've put into this! I've just had 
> a look
> in your home directory on people.debian.org for the GRUB test ISO but I can't 
> see it
> - have you removed it pending application of your patches?

I moved the images to the cdimage server (there is also a test image for ia64 
with
GRUB), as the space on people.debian.org is limited:

> https://cdimage.debian.org/cdimage/ports/grub-test/

> Patches look good, although the second patch still uses /boot/grub/sparc64.elf
> instead of /boot/grub/core.img for the grub-mkimage output parameter - is 
> that correct?
Good catch. Probably forgot a "git commit -a -amend" here.

Btw, any suggestions for powerpc/ppc64? We are still using Yaboot here. I 
assume this
case will be a bit more complicated due to the fact we also need to support 
PowerMacs.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-28 Thread Mark Cave-Ayland
On 27/04/2019 20:39, John Paul Adrian Glaubitz wrote:

> On 4/27/19 9:30 PM, John Paul Adrian Glaubitz wrote:
>> On 4/27/19 9:08 PM, Thomas Schmitt wrote:
>>> Maybe you should prepend 512 0-bytes to your cdboot.img.
>>
>> Yes, indeed that did the trick. The uploaded image now works
>> correctly and boots with GRUB on CD. We can finally get rid
>> of SILO \o/.
>>
>> Thanks everyone for the help!
> 
> Attaching the two patches for debian-installer and debian-cd for reference,
> will commit them next week after cleaning them up and also switching ia64
> to GRUB as well.

Amazing - thank you for all the hard work you've put into this! I've just had a 
look
in your home directory on people.debian.org for the GRUB test ISO but I can't 
see it
- have you removed it pending application of your patches?

Patches look good, although the second patch still uses /boot/grub/sparc64.elf
instead of /boot/grub/core.img for the grub-mkimage output parameter - is that 
correct?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 11:12 PM, Thomas Schmitt wrote:
> The ISO is now added to my collection of bootable ISOs for regression
> test. My test loop checks whether the ISOs which are repacked from
> existing ISOs still show the same boot equipment and file tree content.
> (Happens when i change boot related program parts and before releases.
>  2 hours of 3.5 GHz Xeon shoveling forth and back on about 150 GB.)

Please note: The ISO on my people.debian.org website is just a test
build. The final images will show up on cdimage.debian.org/cdimage.

> May i assume that the old debian-10.0-sparc64-NETINST-1.iso is in
> retirement for now ?
> (Please announce on debian-sparc if the genisoimage style gets revived.)

Yes, once everything is committed to debian-installer and debian-cd git.

I will make an official announcement to the list.

>> boots with GRUB on CD.
> 
> SUN disk label is actually a hard disk thing. Are there machines which
> boot from USB attached disk storage ?

I think that works as well although I only run Linux inside LDOMs, so I
just attach the ISO to a virtual device.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Thomas Schmitt
Hi,

> PS: You can fetch the image from the known location to test

Looks good. Viewing xorriso arguments in /mnt/iso/.disk/mkisofs ...
looks good too.

The ISO is now added to my collection of bootable ISOs for regression
test. My test loop checks whether the ISOs which are repacked from
existing ISOs still show the same boot equipment and file tree content.
(Happens when i change boot related program parts and before releases.
 2 hours of 3.5 GHz Xeon shoveling forth and back on about 150 GB.)


May i assume that the old debian-10.0-sparc64-NETINST-1.iso is in
retirement for now ?
(Please announce on debian-sparc if the genisoimage style gets revived.)


> boots with GRUB on CD.

SUN disk label is actually a hard disk thing. Are there machines which
boot from USB attached disk storage ?


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 9:30 PM, John Paul Adrian Glaubitz wrote:
> On 4/27/19 9:08 PM, Thomas Schmitt wrote:
>> Maybe you should prepend 512 0-bytes to your cdboot.img.
> 
> Yes, indeed that did the trick. The uploaded image now works
> correctly and boots with GRUB on CD. We can finally get rid
> of SILO \o/.
> 
> Thanks everyone for the help!

Attaching the two patches for debian-installer and debian-cd for reference,
will commit them next week after cleaning them up and also switching ia64
to GRUB as well.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
>From c7e260bbe85af8b33df390bdaff85d9c36f8f0c0 Mon Sep 17 00:00:00 2001
From: John Paul Adrian Glaubitz 
Date: Sat, 27 Apr 2019 19:33:30 +
Subject: [PATCH] sparc64: Switch bootloader for d-i from SILO to GRUB

---
 tools/boot/buster/boot-sparc64 | 123 +++--
 tools/generate_di+k_list   |   1 -
 2 files changed, 58 insertions(+), 66 deletions(-)

diff --git a/tools/boot/buster/boot-sparc64 b/tools/boot/buster/boot-sparc64
index 778aee7f..6c48f973 100755
--- a/tools/boot/buster/boot-sparc64
+++ b/tools/boot/buster/boot-sparc64
@@ -1,93 +1,86 @@
-#!/bin/bash -e
-# 
-# boot-sparc64
+#!/bin/bash
 #
-# Do install stuff for sparc64, including making first CD bootable
+# Do install stuff for sparc64, including making bootable CDs
+# Works with debian-installer
+#
+# $1 is the CD number
+# $2 is the temporary CD build dir
 
 . $BASEDIR/tools/boot/$DI_CODENAME/common.sh
 
 set -e
+#set -x
 
 N=$1
 CDDIR=$2
+INSTALLDIR=$CDDIR/install
+
+# Common mkisofs options when creating CDs
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-J -joliet-long"
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-cache-inodes"
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-l"
+
+# mkisofs options specific to sparc64
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G $CDDIR/../CD1/cdboot.img"
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-B '...'"
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "--grub2-sparc-core /boot/grub/core.img"
 
 # Exit if this is not CD#1/DVD#1
 if [ $N != "1" ]; then
-add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-J -joliet-long"
-add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-cache-inodes"
-add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-l"
 exit 0
 fi
 
 if [ "$DI_WWW_HOME" = "default" ]; then
-DI_WWW_HOME="https://d-i.debian.org/daily-images/sparc64/daily/cdrom/;
+DI_WWW_HOME="https://d-i.debian.org/daily-images/sparc64/daily;
 try_di_image_cache
 else
 DI_WWW_HOME=$(echo $DI_WWW_HOME | sed "s,%ARCH%,$ARCH,")
 fi
 
-add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G boot1/boot/isofs.b -B ..."
-add_mkisofs_opt $CDDIR/../$N.mkisofs_dirs "boot1"
-
-inst=boot1
+case "$MKISOFS" in
+*xorriso*)
+XORRISO_VER=$(xorriso_version)
+;;
+*)
+	echo "ERROR: debian-cd now depends on xorriso for making sparc64 bootable CDs."
+	exit 1;
+	;;
+esac
 
 cd $CDDIR/..
 
-# Setup directories
-mkdir -p $inst/boot
-
-silo_deb=$(find_pkg_file silo)
-if [ -z "$silo_deb" ]; then
-   echo "ERROR: silo package is required"
-   exit 1
-fi 
-# put the relevant parts of SILO boot loader
-(dpkg --fsys-tarfile $MIRROR/$silo_deb | \
-	tar xf - -C $inst/ ./boot/{isofs,second}.b)
+BOOT_IMAGES="cdrom/initrd.gz cdrom/vmlinux cdrom/debian-cd_info.tar.gz"
 
-if [ -n "$ARCHIVE_EXTRACTED_SOURCES" ]; then
-echo $silo_deb >> $CDDIR/../$N.pkgs_extracted
-find_pkg_file silo source >> $CDDIR/../$N.pkgs_extracted
-fi
+# Download boot images.
+for image in $BOOT_IMAGES; do
+if [ ! -e "$image" ]; then
+dir=$(dirname $image)
+mkdir -p $dir
+if [ -n "$LOCAL"  -a -f "${LOCALDEBS:-$MIRROR}/dists/$DI_DIST/local/installer-$ARCH/current/images/$image" ]; then
+cp "${LOCALDEBS:-$MIRROR}/dists/$DI_DIST/local/installer-$ARCH/current/images/$image" "$image"
+elif [ ! "$DI_WWW_HOME" ];then
+if [ ! "$DI_DIR" ];then
+DI_DIR="$MIRROR/dists/$DI_DIST/main/installer-$ARCH/current/images"
+fi
+cp "$DI_DIR/$image" "$image"
+else
+$WGET "$DI_WWW_HOME/$image" -O "$image"
+fi
+fi
+done
 
-# Some custom etc files
-cp -f -p $BASEDIR/data/${CODENAME}/sparc64/silo.conf $inst/boot/
-if [ -n "$KERNEL_PARAMS" ]; then
-	# Add KERNEL_PARAMS to any existing append line
-	sed -i "/^[[:space:]]*append=\"/ s|append=\"|append=\"$KERNEL_PARAMS |" \
-		$inst/boot/silo.conf
-	# If there is no default append= line, add it
-	if ! grep -q "^append=" $inst/boot/silo.conf; then
-		sed -i "/^default=/ a\append=\"$KERNEL_PARAMS\"" \
-			$inst/boot/silo.conf
-	fi
-fi
-cat $BASEDIR/data/${CODENAME}/sparc64/debian.txt \
- | sed "s/\${MEDIA_TYPE}/CDROM/" \
- | sed "s/\${DEBIAN_VERSION}/${CODENAME}/g" \
- | sed "s/\${BUILD_DATE}/${BUILD_DATE}/g" \
- > $inst/boot/debian.txt
-cp 

Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 9:08 PM, Thomas Schmitt wrote:
> Maybe you should prepend 512 0-bytes to your cdboot.img.

Yes, indeed that did the trick. The uploaded image now works
correctly and boots with GRUB on CD. We can finally get rid
of SILO \o/.

Thanks everyone for the help!

PS: You can fetch the image from the known location to test
it yourself. I just need to upload a temporary fix to
the rootskel bug now.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Thomas Schmitt
Hi,

i wrote:
> grub-mkrescue copies stuff to bytes 512 to 767 of its -G image.

It's 512 to 1023, of course.


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Thomas Schmitt
Hi,

there is a copy of the -G image file in the ISO:

  /boot/grub/sparc64-ieee1275/cdboot.img

It has only 512 bytes. Very non-zero.

But block 0 is for the SUN disk label (partition table).
libisofs overwrites it completely at ISO production time.
See
  https://sources.debian.org/src/libisofs/1.5.0-1/libisofs/system_area.c/#L776

grub-mkrescue copies stuff to bytes 512 to 767 of its -G image.
Maybe you should prepend 512 0-bytes to your cdboot.img.


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Mark Cave-Ayland
On 27/04/2019 19:26, Thomas Schmitt wrote:

> Hi,
> 
> xorriso-wise, the debian-10.0-sparc64-grub-NETINST-1.iso looks ok:
> 
>   $ xorriso -indev debian-10.0-sparc64-grub-NETINST-1.iso \
> -report_system_area plain
>   ...
>   Volume id: 'Debian 10.0 sparc64 n'
>   System area options: 0x000c
>   System area summary: SUN-SPARC-Disk-Label
>   ISO image size/512 : 443640
>   SUN SPARC disklabel: CD-ROM Disc with Sun sparc boot created by libisofs
>   SUN SPARC secs/head: 640
>   SUN SPARC heads/cyl: 1
>   SUN SPARC partmap  :   N   IdTag   PermsStartCyl   NumBlocks
>   SUN SPARC partition:   1  0x0004  0x0010   0  443640
>   SUN SPARC partition:   2  0x0002  0x0010   0  443640
>   SUN SPARC partition:   3  0x0002  0x0010   0  443640
>   SUN SPARC partition:   4  0x0002  0x0010   0  443640
>   SUN SPARC partition:   5  0x0002  0x0010   0  443640
>   SUN SPARC partition:   6  0x0002  0x0010   0  443640
>   SUN SPARC partition:   7  0x0002  0x0010   0  443640
>   SUN SPARC partition:   8  0x0002  0x0010   0  443640
>   SPARC GRUB2 core   : 1748992  453104
>   SPARC GRUB2 path   : /boot/core.img
> 
> xorriso reports /boot/core.img as "SPARC GRUB2 path", because it was
> recognized as comming from the same disk inode as /boot/grub/core.img
> and thus shares the content blocks with it:
> 
>   $ xorriso -indev debian-10.0-sparc64-grub-NETINST-1.iso \
> -find /boot/core.img -exec report_lba -- \
> -find /boot/grub/core.img -exec report_lba --
>   ...
>   Report layout: xt , Startlba ,   Blocks , Filesize , ISO image path
>   File data lba:  0 ,  854 ,  222 ,   453104 , '/boot/core.img'
>   Report layout: xt , Startlba ,   Blocks , Filesize , ISO image path
>   File data lba:  0 ,  854 ,  222 ,   453104 , '/boot/grub/core.img'
> 
> So all is well. Except the nearly zero content of bytes 512 to 767 of
> the ISO.

One interesting point: the local man page for genisoimage on my Debian stretch
machine states this:

   -G generic_boot_image
Specifies  the  path  and filename of the generic boot image to be used
when making a generic bootable CD.  The boot image will be placed on
the first 16 sectors of the CD, before the ISO9660 primary volume
descriptor.  If this option is used together with -sparc-boot, the Sun
disk label will overlay the first 512 bytes of the generic boot image.

This implies that we need to pad cdboot.img with an initial 512 bytes which 
matches
what Thomas said in one of his earlier emails:

"grub-mkrescue opens cdboot.img for reading and the image file for -G
for writing and truncates it to 0 length. Then it writes 512 zero bytes,
reads 512 bytes from cdboot.img and writes them to the -G image file."


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Thomas Schmitt
Hi,

xorriso-wise, the debian-10.0-sparc64-grub-NETINST-1.iso looks ok:

  $ xorriso -indev debian-10.0-sparc64-grub-NETINST-1.iso \
-report_system_area plain
  ...
  Volume id: 'Debian 10.0 sparc64 n'
  System area options: 0x000c
  System area summary: SUN-SPARC-Disk-Label
  ISO image size/512 : 443640
  SUN SPARC disklabel: CD-ROM Disc with Sun sparc boot created by libisofs
  SUN SPARC secs/head: 640
  SUN SPARC heads/cyl: 1
  SUN SPARC partmap  :   N   IdTag   PermsStartCyl   NumBlocks
  SUN SPARC partition:   1  0x0004  0x0010   0  443640
  SUN SPARC partition:   2  0x0002  0x0010   0  443640
  SUN SPARC partition:   3  0x0002  0x0010   0  443640
  SUN SPARC partition:   4  0x0002  0x0010   0  443640
  SUN SPARC partition:   5  0x0002  0x0010   0  443640
  SUN SPARC partition:   6  0x0002  0x0010   0  443640
  SUN SPARC partition:   7  0x0002  0x0010   0  443640
  SUN SPARC partition:   8  0x0002  0x0010   0  443640
  SPARC GRUB2 core   : 1748992  453104
  SPARC GRUB2 path   : /boot/core.img

xorriso reports /boot/core.img as "SPARC GRUB2 path", because it was
recognized as comming from the same disk inode as /boot/grub/core.img
and thus shares the content blocks with it:

  $ xorriso -indev debian-10.0-sparc64-grub-NETINST-1.iso \
-find /boot/core.img -exec report_lba -- \
-find /boot/grub/core.img -exec report_lba --
  ...
  Report layout: xt , Startlba ,   Blocks , Filesize , ISO image path
  File data lba:  0 ,  854 ,  222 ,   453104 , '/boot/core.img'
  Report layout: xt , Startlba ,   Blocks , Filesize , ISO image path
  File data lba:  0 ,  854 ,  222 ,   453104 , '/boot/grub/core.img'

So all is well. Except the nearly zero content of bytes 512 to 767 of
the ISO.


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Thomas Schmitt
Hi,

John Paul Adrian Glaubitz:
> xorriso -as mkisofs -r -checksum_algorithm_iso md5,sha1
> -V 'Debian 10.0 sparc64 n'
> -o /home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso
> -J -joliet-long -cache-inodes
> -G /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/sparc64-ieee1275/cdboot.img
> -B '...'
> --grub2-sparc-core /boot/grub/core.img
> -graft-points
> /boot/core.img=/home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/core.img
> CD1

Now you first insert .../core.img as /boot/core.img.
Then you insert it again by its role as sub-file of "CD1" as
ISO file /boot/grub/core.img.
To the latter you refer by --grub2-sparc-core .

So this is without much effect (except adding a file copy to the ISO):

  -graft-points
  /boot/core.img=/home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/core.img


> {0} ok boot cdrom
> Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args:
> WARNING: Unsupported bootblk image, can not extract fcode
> WARNING: Bootblk fcode extraction failed
> The file just loaded does not appear to be executable.

Is this the SUN firmware speaking ?

Google "sun bootblk" yields
  
https://www.thegeekdiary.com/howto-verify-if-a-bootblk-is-installed-on-the-boot-disk-sparc/?PageSpeed=noscript
where block 1 (i.e. byte 512 ff.) is inspected for "Bootblk".

So it would be about block 1 of your disk file for -G

  /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/sparc64-ieee1275/cdboot.img

That's where --grub2-sparc-core inserts its numbers.
But i guess it is about the other bytes in there.


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Thomas Schmitt
Hi,

> xorriso -as mkisofs -r -checksum_algorithm_iso md5,sha1
> -V 'Debian 10.0 sparc64 n'
> -o /home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso
> -J -joliet-long -cache-inodes
> -G /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/sparc64-ieee1275/cdboot.img
> -B ...
> --grub2-sparc-core /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/core.img
> CD1

Try

  --grub2-sparc-core /boot/grub/core.img

Reasoning:
You obviously give it the path on the source disk. But the option wants
the path in the ISO.

The disk path and the pathspec "CD1" give the impression that there
is a disk file

  CD1/boot/grub/core.img

which by the sparse form of the pathspec will be mapped to ISO path

  /boot/grub/core.img


Later mail:
I wrote:
> >   -graft-points \
> >   /boot/grub/core.img=./dummy_for_core

> What is missing and what I am testing now is the > "-graft-points"
> option which matches the core.img in the ISO with the one outside the ISO.

Not needed.
This is not a matcher but rather an enabler and a file (or tree) insertion.

Option -graft-points enables the pathspec format iso_rr_path=disk_path .

The following pathspec makes use of this:
  /boot/grub/core.img=./dummy_for_core
It inserts the disk file ./dummy_for_core into the ISO as
/boot/grub/core.img .

Your "CD1" inserts the disk tree ./CD1 into the ISO as /-directory.
(Actually it merges the content of ./CD1 with the content of /, which
 at that time is still empty. You may merge-in more disk trees.)

(That's all classical mkisofs operation. I am not happy about everything
 there, but it enables swift transition from mkisofs or genisoimage.)


The option
  -checksum_algorithm_iso md5,sha1
is without effect if you do not produce Jigdo files by option -jigdo-jigdo
et.al. It does not harm, though.


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 7:16 PM, John Paul Adrian Glaubitz wrote:
> xorriso -as mkisofs -r -checksum_algorithm_iso md5,sha1 -V 'Debian 10.0 
> sparc64 n' -o /home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso 
> -J -joliet-long -cache-inodes -G 
> /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/sparc64-ieee1275/cdboot.img 
> --grub2-sparc-core /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/core.img -B 
> ... CD1
> 
> Maybe the ... need to be quoted.

Still fails to boot:

{0} ok boot cdrom
Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args: 
WARNING: Unsupported bootblk image, can not extract fcode

WARNING: Bootblk fcode extraction failed

The file just loaded does not appear to be executable.
{0} ok

Image here:

> https://people.debian.org/~glaubitz/debian-10.0-sparc64-grub-NETINST-1.iso

Created with:

xorriso -as mkisofs -r -checksum_algorithm_iso md5,sha1 -V 'Debian 10.0 sparc64 
n' -o /home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso -J 
-joliet-long -cache-inodes -G 
/home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/sparc64-ieee1275/cdboot.img -B 
'...' --grub2-sparc-core /boot/grub/core.img -graft-points 
/boot/core.img=/home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/core.img CD1

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 7:25 PM, Mark Cave-Ayland wrote:
> I think it's trying to find /boot/grub/core.img in the ISO image built from 
> your CD1
> directory and failing - did you also update grub-mkimage in your first patch 
> so that
> the core image is output as core.img and not sparc64.elf?

Yes, of course. What is missing and what I am testing now is the "-graft-points"
option which matches the core.img in the ISO with the one outside the ISO.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 7:11 PM, Thomas Schmitt wrote:
>   $ xorriso -as mkisofs \
>   -o test.iso \
>   -V 'FAKE_SPARC_64_ISO' \
>   -J -joliet-long -cache-inodes -l \
>   -G ./dummy_for_G \
>   -B '...' \
>   --grub2-sparc-core /boot/grub/core.img \
>   -graft-points \
>   /boot/grub/core.img=./dummy_for_core
Ah, I think the last two lines are important. The path for --grub2-sparc-core
is *within* the ISO image. And then you have to match it to the outside path.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Mark Cave-Ayland
On 27/04/2019 18:12, John Paul Adrian Glaubitz wrote:

> On 4/27/19 6:47 PM, Thomas Schmitt wrote:
>> Mark Cave-Ayland wrote:
>>> In that case I think the change for Adrian's second patch should in fact be
>>> this:
>>>  add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G cdboot.img -B ...
>>> --grub2-sparc-core /boot/grub/core.img"
>>>
>>> Does that look better?
>>
>> If /boot/grub/core.img is the path of the next stage boot file in the
>> ISO filesystem, then yes.
>> grub-mkrescue names it similarly: /boot/grub/sparc64-ieee1275/core.img .
> 
> This gives me an obscure error message:
> 
> xorriso -as mkisofs -r -checksum_algorithm_iso md5,sha1 -V 'Debian 10.0 
> sparc64 n' -o /home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso 
> -J -joliet-long -cache-inodes -G 
> /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/sparc64-ieee1275/cdboot.img -B 
> ... --grub2-sparc-core /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/core.img 
> CD1
> xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.
> 
> Drive current: -outdev 
> 'stdio:/home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso'
> Media current: stdio file, overwriteable
> Media status : is blank
> Media summary: 0 sessions, 0 data blocks, 0 data, 2390g free
> xorriso : WARNING : -volid text problematic as automatic mount point name
> xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
> xorriso : NOTE : -as mkisofs: Ignored option '-cache-inodes'
> Added to ISO image: directory '/'='/home/glaubitz/tmp/sid/CD1'
> xorriso : UPDATE : 1165 files added in 1 seconds
> xorriso : UPDATE : 1165 files added in 1 seconds
> xorriso : FAILURE : Cannot find in ISO image: -boot_image grub 
> grub2_sparc_core='/home/glaubitz/tmp/sid/CD1/boot/grub/core.img'
> xorriso : NOTE : -return_with SORRY 32 triggered by problem severity FAILURE
> Makefile:489: recipe for target 'images' failed
> make: *** [images] Error 32

I think it's trying to find /boot/grub/core.img in the ISO image built from 
your CD1
directory and failing - did you also update grub-mkimage in your first patch so 
that
the core image is output as core.img and not sparc64.elf?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 7:11 PM, Thomas Schmitt wrote:
> John Paul Adrian Glaubitz wrote:
>> -J -joliet-long -cache-inodes -l
>> -G $CDDIR/../CD1/boot/grub/sparc64-ieee1275/cdboot.img -B ...
>>
>> Would it be enough to just switch to xorriso here?
> 
> I think you forgot -o result.o and -V Volume_Id.

Those are passed by the other debian-cd code.

The full command line is now:

xorriso -as mkisofs -r -checksum_algorithm_iso md5,sha1 -V 'Debian 10.0 sparc64 
n' -o /home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso -J 
-joliet-long -cache-inodes -G 
/home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/sparc64-ieee1275/cdboot.img 
--grub2-sparc-core /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/core.img -B ... 
CD1

Maybe the ... need to be quoted.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 6:47 PM, Thomas Schmitt wrote:
> Mark Cave-Ayland wrote:
>> In that case I think the change for Adrian's second patch should in fact be
>> this:
>>  add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G cdboot.img -B ...
>> --grub2-sparc-core /boot/grub/core.img"
>>
>> Does that look better?
> 
> If /boot/grub/core.img is the path of the next stage boot file in the
> ISO filesystem, then yes.
> grub-mkrescue names it similarly: /boot/grub/sparc64-ieee1275/core.img .

This gives me an obscure error message:

xorriso -as mkisofs -r -checksum_algorithm_iso md5,sha1 -V 'Debian 10.0 sparc64 
n' -o /home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso -J 
-joliet-long -cache-inodes -G 
/home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/sparc64-ieee1275/cdboot.img -B ... 
--grub2-sparc-core /home/glaubitz/tmp/sid/CD1/../CD1/boot/grub/core.img CD1
xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.

Drive current: -outdev 
'stdio:/home/glaubitz/debian-cd-test/debian-10.0-sparc64-NETINST-1.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 2390g free
xorriso : WARNING : -volid text problematic as automatic mount point name
xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
xorriso : NOTE : -as mkisofs: Ignored option '-cache-inodes'
Added to ISO image: directory '/'='/home/glaubitz/tmp/sid/CD1'
xorriso : UPDATE : 1165 files added in 1 seconds
xorriso : UPDATE : 1165 files added in 1 seconds
xorriso : FAILURE : Cannot find in ISO image: -boot_image grub 
grub2_sparc_core='/home/glaubitz/tmp/sid/CD1/boot/grub/core.img'
xorriso : NOTE : -return_with SORRY 32 triggered by problem severity FAILURE
Makefile:489: recipe for target 'images' failed
make: *** [images] Error 32

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Thomas Schmitt
Hi,

John Paul Adrian Glaubitz wrote:
> -J -joliet-long -cache-inodes -l
> -G $CDDIR/../CD1/boot/grub/sparc64-ieee1275/cdboot.img -B ...
>
> Would it be enough to just switch to xorriso here?

I think you forgot -o result.o and -V Volume_Id.

They are all in the list of emulated options (xorriso -as mkisofs -help
man xorrisofs)

A test:

  $ dd if=/dev/zero bs=512 count=2 of=dummy_for_G

  $ dd if=/dev/zero bs=512 count=10 of=dummy_for_core

  $ xorriso -as mkisofs \
  -o test.iso \
  -V 'FAKE_SPARC_64_ISO' \
  -J -joliet-long -cache-inodes -l \
  -G ./dummy_for_G \
  -B '...' \
  --grub2-sparc-core /boot/grub/core.img \
  -graft-points \
  /boot/grub/core.img=./dummy_for_core
  ...
  Written to medium : 186 sectors at LBA 0
  Writing to 'stdio:test.iso' completed successfully.

  $ xorriso -indev test.iso -report_system_area plain
  ...
  Volume id: 'FAKE_SPARC_64_ISO'
  System area options: 0x000c
  System area summary: SUN-SPARC-Disk-Label
  ISO image size/512 : 744
  SUN SPARC disklabel: CD-ROM Disc with Sun sparc boot created by libisofs
  SUN SPARC secs/head: 640
  SUN SPARC heads/cyl: 1
  SUN SPARC partmap  :   N   IdTag   PermsStartCyl   NumBlocks
  SUN SPARC partition:   1  0x0004  0x0010   0 744
  SUN SPARC partition:   2  0x0002  0x0010   0 744
  SUN SPARC partition:   3  0x0002  0x0010   0 744
  SUN SPARC partition:   4  0x0002  0x0010   0 744
  SUN SPARC partition:   5  0x0002  0x0010   0 744
  SUN SPARC partition:   6  0x0002  0x0010   0 744
  SUN SPARC partition:   7  0x0002  0x0010   0 744
  SUN SPARC partition:   8  0x0002  0x0010   0 744
  SPARC GRUB2 core   : 67584  5120
  SPARC GRUB2 path   : /boot/grub/core.img

The fact that the file core.img is named as "SPARC GRUB2 path" indicates
that its start LBA was written to the right place in the -G image.
(Numbers shown as "SPARC GRUB2 core".)


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Thomas Schmitt
Hi,

Mark Cave-Ayland wrote:
> In that case I think the change for Adrian's second patch should in fact be
> this:
>  add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G cdboot.img -B ...
> --grub2-sparc-core /boot/grub/core.img"
>
> Does that look better?

If /boot/grub/core.img is the path of the next stage boot file in the
ISO filesystem, then yes.
grub-mkrescue names it similarly: /boot/grub/sparc64-ieee1275/core.img .

Note: It won't work with genisoimage.


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 5:40 PM, Thomas Schmitt wrote:
>> Once this is done and the ISO image is generated, [...]
>> - Next check the offset and size of /boot/grub/core.img embedded in
>> sector 1 using dd and hexdump. As per Thomas' message you should find
>> the offset in bytes of the start of /boot/grub/core.img at offset 552
>> from the start of the .iso (64-bit big endian) and its size at offset
>> 560 from the start of the .iso (32-bit big endian)
> 
> This would be put there by xorriso -as mkisofs option
> 
>   --grub2-sparc-core ...path.of.file.in.ISO.filesystem...
> 
> With genisoimage it would be necessary to insert the info by
> post-processing. /usr/bin/isoinfo prints block addresses as first number
> in its "[   NN NN]" column. Byte offset is block address * 2048.

Okay, I can switch to xorriso, too. But I would assume I would have
to tune more options then.

Currently, we have:

# Common mkisofs options when creating CDs  


add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-J -joliet-long"
add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-cache-inodes"
add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-l"
(...)
add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G 
$CDDIR/../CD1/boot/grub/sparc64-ieee1275/cdboot.img -B ..."

Would it be enough to just switch to xorriso here?

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Mark Cave-Ayland
On 27/04/2019 17:21, John Paul Adrian Glaubitz wrote:

> On 4/27/19 5:35 PM, Mark Cave-Ayland wrote:
>>> I set $CDDIR to the source directory which contains core.img? How does the 
>>> initial
>>> bootloader know that it has to boot core.img?
>>
>> This is the part which Thomas explained: the offset of core.img *as embedded 
>> within
>> the ISO filesystem image* and its size are embedded at byte offsets 552 and 
>> 560
>> respectively from the start of the image. From what I can see 
>> boot.S/diskboot.S seek
>> to sector 1, read in these values, seek to the start offset and then read the
>> relevant number of bytes into memory before finally transferring control.
> 
> But how does genisoimage know that it is supposed to place "core.img" there?
> 
> This is simply what I cannot wrap my head around it. There is some hidden 
> magic
> that knows that it has to put "core.img" there, but the file is never 
> mentioned
> anywhere.

My understanding is that it's the other way around: the location of core.img is
extracted from the ISO image itself and then re-injected back into sector 1...

> FWIW, it also doesn't boot:
> 
> {0} ok boot cdrom
> Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args: 
> WARNING: Unsupported bootblk image, can not extract fcode
> 
> WARNING: Bootblk fcode extraction failed
> 
> The file just loaded does not appear to be executable.
> {0} ok
> 
> Image here: 
> https://people.debian.org/~glaubitz/debian-10.0-sparc64-grub-NETINST-1.iso
> 
> Current debian-cd patch attached.

...and the trigger for this is the --grub2-sparc-core parameter that Thomas 
pointed
out was missing from my previous email (see
https://lists.debian.org/debian-sparc/2019/04/msg00057.html for the updated
add_mkisofs_opt).


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
On 4/27/19 5:35 PM, Mark Cave-Ayland wrote:
>> I set $CDDIR to the source directory which contains core.img? How does the 
>> initial
>> bootloader know that it has to boot core.img?
> 
> This is the part which Thomas explained: the offset of core.img *as embedded 
> within
> the ISO filesystem image* and its size are embedded at byte offsets 552 and 
> 560
> respectively from the start of the image. From what I can see 
> boot.S/diskboot.S seek
> to sector 1, read in these values, seek to the start offset and then read the
> relevant number of bytes into memory before finally transferring control.

But how does genisoimage know that it is supposed to place "core.img" there?

This is simply what I cannot wrap my head around it. There is some hidden magic
that knows that it has to put "core.img" there, but the file is never mentioned
anywhere.

FWIW, it also doesn't boot:

{0} ok boot cdrom
Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args: 
WARNING: Unsupported bootblk image, can not extract fcode

WARNING: Bootblk fcode extraction failed

The file just loaded does not appear to be executable.
{0} ok

Image here: 
https://people.debian.org/~glaubitz/debian-10.0-sparc64-grub-NETINST-1.iso

Current debian-cd patch attached.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
diff --git a/tools/boot/buster/boot-sparc64 b/tools/boot/buster/boot-sparc64
index 778aee7f..1a383944 100755
--- a/tools/boot/buster/boot-sparc64
+++ b/tools/boot/buster/boot-sparc64
@@ -1,93 +1,78 @@
-#!/bin/bash -e
-# 
-# boot-sparc64
+#!/bin/bash
 #
-# Do install stuff for sparc64, including making first CD bootable
+# Do install stuff for sparc64, including making bootable CDs
+# Works with debian-installer
+#
+# $1 is the CD number
+# $2 is the temporary CD build dir
 
 . $BASEDIR/tools/boot/$DI_CODENAME/common.sh
 
 set -e
+#set -x
 
 N=$1
 CDDIR=$2
+INSTALLDIR=$CDDIR/install
+
+# Common mkisofs options when creating CDs
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-J -joliet-long"
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-cache-inodes"
+add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-l"
 
 # Exit if this is not CD#1/DVD#1
 if [ $N != "1" ]; then
-add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-J -joliet-long"
-add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-cache-inodes"
-add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-l"
 exit 0
 fi
 
 if [ "$DI_WWW_HOME" = "default" ]; then
-DI_WWW_HOME="https://d-i.debian.org/daily-images/sparc64/daily/cdrom/;
+DI_WWW_HOME="https://d-i.debian.org/daily-images/sparc64/daily;
 try_di_image_cache
 else
 DI_WWW_HOME=$(echo $DI_WWW_HOME | sed "s,%ARCH%,$ARCH,")
 fi
 
-add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G boot1/boot/isofs.b -B ..."
-add_mkisofs_opt $CDDIR/../$N.mkisofs_dirs "boot1"
-
-inst=boot1
+# case "$MKISOFS" in
+# *xorriso*)
+# XORRISO_VER=$(xorriso_version)
+# ;;
+# *)
+# 	echo "ERROR: debian-cd now depends on xorriso for making sparc64 bootable CDs."
+# 	exit 1;
+# 	;;
+# esac
 
 cd $CDDIR/..
 
-# Setup directories
-mkdir -p $inst/boot
+BOOT_IMAGES="cdrom/initrd.gz cdrom/vmlinux cdrom/debian-cd_info.tar.gz"
 
-silo_deb=$(find_pkg_file silo)
-if [ -z "$silo_deb" ]; then
-   echo "ERROR: silo package is required"
-   exit 1
-fi 
-# put the relevant parts of SILO boot loader
-(dpkg --fsys-tarfile $MIRROR/$silo_deb | \
-	tar xf - -C $inst/ ./boot/{isofs,second}.b)
+# Download boot images.
+for image in $BOOT_IMAGES; do
+if [ ! -e "$image" ]; then
+dir=$(dirname $image)
+mkdir -p $dir
+if [ -n "$LOCAL"  -a -f "${LOCALDEBS:-$MIRROR}/dists/$DI_DIST/local/installer-$ARCH/current/images/$image" ]; then
+cp "${LOCALDEBS:-$MIRROR}/dists/$DI_DIST/local/installer-$ARCH/current/images/$image" "$image"
+elif [ ! "$DI_WWW_HOME" ];then
+if [ ! "$DI_DIR" ];then
+DI_DIR="$MIRROR/dists/$DI_DIST/main/installer-$ARCH/current/images"
+fi
+cp "$DI_DIR/$image" "$image"
+else
+$WGET "$DI_WWW_HOME/$image" -O "$image"
+fi
+fi
+done
 
-if [ -n "$ARCHIVE_EXTRACTED_SOURCES" ]; then
-echo $silo_deb >> $CDDIR/../$N.pkgs_extracted
-find_pkg_file silo source >> $CDDIR/../$N.pkgs_extracted
-fi
+# Boot setup including config and help files comes from d-i.
+mkdir -pv $PWD/CD$N
+cat cdrom/debian-cd_info.tar.gz | (cd CD$N/; tar zx)
 
-# Some custom etc files
-cp -f -p $BASEDIR/data/${CODENAME}/sparc64/silo.conf $inst/boot/
-if [ -n "$KERNEL_PARAMS" ]; then
-	# Add KERNEL_PARAMS to any existing append line
-	sed -i "/^[[:space:]]*append=\"/ s|append=\"|append=\"$KERNEL_PARAMS |" \
-		$inst/boot/silo.conf
-	# If there is no default append= line, add it
-	if ! grep -q "^append=" $inst/boot/silo.conf; then
-		sed -i "/^default=/ 

Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Mark Cave-Ayland
On 27/04/2019 16:40, Thomas Schmitt wrote:

> Hi,
> 
> Mark Cave-Ayland wrote:
>> - In your second patch re-enable the -G/-B options but with -G set to
>> cdboot.img:
>> add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G cdboot.img -B ..."
> 
> Does this refer to
>   From: John Paul Adrian Glaubitz 
>   Date: Thu, 25 Apr 2019 00:05:45 +0200
>   https://lists.debian.org/debian-sparc/2019/04/msg00036.html
> ?
> 
> 
>> Once this is done and the ISO image is generated, [...]
>> - Next check the offset and size of /boot/grub/core.img embedded in
>> sector 1 using dd and hexdump. As per Thomas' message you should find
>> the offset in bytes of the start of /boot/grub/core.img at offset 552
>> from the start of the .iso (64-bit big endian) and its size at offset
>> 560 from the start of the .iso (32-bit big endian)
> 
> This would be put there by xorriso -as mkisofs option
> 
>   --grub2-sparc-core ...path.of.file.in.ISO.filesystem...
> 
> With genisoimage it would be necessary to insert the info by
> post-processing. /usr/bin/isoinfo prints block addresses as first number
> in its "[   NN NN]" column. Byte offset is block address * 2048.

Ooops yes indeed, it is necessary to explicitly enable this via 
--grub2-sparc-core.
In that case I think the change for Adrian's second patch should in fact be 
this:

 add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G cdboot.img -B ... 
--grub2-sparc-core
/boot/grub/core.img"

Does that look better?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread John Paul Adrian Glaubitz
Hi!

On 4/27/19 5:02 PM, Mark Cave-Ayland wrote:
> Adrian: based upon this I think what you need to do is:
> 
> - For consistency rename /boot/grub/sparc64.elf to /boot/grub/core.img to 
> match
>   the descriptions of the grub components in the documentation

Ok. But still generate it with grub-mkimage?

+   grub-mkimage -O sparc64-ieee1275-cdcore -p '()/boot/grub' \
+   -o $(TEMP_CD_INFO_DIR)/boot/grub/sparc64.elf \
+   $(GRUB_MODULES) $(GRUB_MODULES_CDROM)

> - Confirm that cdboot.img is the a.out executable generated by boot.S and 
> diskboot.S
>   (use dd and hexdump to check for the 0x01 0x03 0x01 0x07 signature in the 
> first 4
>   bytes)

glaubitz@kyoto:/usr/lib/grub/sparc64-ieee1275$ xxd cdboot.img |head -n2
: 0103 0107  01e0      
0010:    4000      ..@.
glaubitz@kyoto:/usr/lib/grub/sparc64-ieee1275$

Yes.

> - In your second patch re-enable the -G/-B options but with -G set to 
> cdboot.img:
>  add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G cdboot.img -B ..."

I set $CDDIR to the source directory which contains core.img? How does the 
initial
bootloader know that it has to boot core.img?

> Once this is done and the ISO image is generated, the basic setup can be 
> checked as
> follows:
> 
> - Use fdisk or similar tool to confirm that the Sun disk label exists at the
>   start of the .iso with all 8 partitions starting at sector 0 with an end 
> sector
>   representing the contents of the whole CDROM image
> 
> - Use dd and hexdump to confirm that cdboot.img appears 512 bytes into the 
> .iso
>   by checking for the 0x01 0x03 0x01 0x07 signature
> 
> - Next check the offset and size of /boot/grub/core.img embedded in sector 1 
> using
>   dd and hexdump. As per Thomas' message you should find the offset in bytes 
> of
>   the start of /boot/grub/core.img at offset 552 from the start of the .iso 
> (64-bit
>   big endian) and its size at offset 560 from the start of the .iso (32-bit 
> big
>   endian)
> 
> - If everything is correct then again using dd and hexdump you should be able 
> to
>   see the ELF signature for core.img from the offset discovered above.
> 
> Assuming all of this looks correct, then I believe that you should end up 
> with a
> bootable CDROM image using grub...

Nice. I'll give it a shot tonight and report back.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-27 Thread Mark Cave-Ayland
On 26/04/2019 18:04, Thomas Schmitt wrote:

> Hi,
> 
>> - the first 32K of an ISO9660 is undefined
> 
> That's such a negative word. :))
> It is "reserved for system use" and "not specified" by ISO 9660 / ECMA-119.
> 
> 
>> - boot.S and diskboot.S get built into cdboot.img
> 
> grub-mkrescue opens cdboot.img for reading and the image file for -G
> for writing and truncates it to 0 length. Then it writes 512 zero bytes,
> reads 512 bytes from cdboot.img and writes them to the -G image file.
> 
> 
>> - grub should run xorisso with the following parameters:
>> -G cdboot.img -B "..." --grub2-sparc-core
>> /boot/grub/sparc64-ieee1275/core.img
> 
> Actually these are for the mkisofs emulation of xorriso.
>   xorriso -as mkisofs ...options.and.pathspecs...
> 
> (See man xorrisofs for options.)
> 
> Note that
>   -B ','
> as of grub-mkrescue is not totally equivalent to your
>   -B '...'
> 
> Digging in libisoburn's xorriso/emulators.c i see that "," orders two
> appended partitions with empty disk source paths. This will cause no
> partitions to be appended.
> "..." causes 7 partitions copying the partition entry data of the entry
> before them. That would be the entry describing the ISO image as a whole.
> Again no partitions images get appended.
> 
> One should test both and inspect by SUN tools or
>   xorriso -indev $ISOIMAGE -report_system_area plain
> Probably "..." will produce a partition table with 8 entries, and ","
> will produce a table with only one entry.
> 
> 
>> - what this effectively does is:
>>   - Install a sun partition disk label to sector 0 of the ISO image
>>   - Add the ISO9660 image as the first partition (slice)
>>   - Create all 7 remaining partitions after the ISO9660 image
> 
> I expect this too.
> 
>>   - These partitions are all small and just contain cdboot.img
> 
> I expect what can be seen in debian-10.0-sparc64-NETINST-1.iso
> 
>   Volume id: 'Debian 10.0 sparc64 n'
>   ...
>   System area summary: SUN-SPARC-Disk-Label
>   ISO image size/512 : 284760
>   SUN SPARC disklabel: CD-ROM Disc with Sun sparc boot created by genisoimage
>   SUN SPARC secs/head: 640
>   SUN SPARC heads/cyl: 1
>   SUN SPARC partmap  :   N   IdTag   PermsStartCyl   NumBlocks
>   SUN SPARC partition:   1  0x0004  0x0010   0  284160
>   SUN SPARC partition:   2  0x0002  0x0010   0  284160
>   SUN SPARC partition:   3  0x0002  0x0010   0  284160
>   SUN SPARC partition:   4  0x0002  0x0010   0  284160
>   SUN SPARC partition:   5  0x0002  0x0010   0  284160
>   SUN SPARC partition:   6  0x0002  0x0010   0  284160
>   SUN SPARC partition:   7  0x0002  0x0010   0  284160
>   SUN SPARC partition:   8  0x0002  0x0010   0  284160
> 
> xorriso command -pvd_info says:
> 
>   App Id   : GENISOIMAGE ISO 9660/HFS FILESYSTEM CREATOR (C) 1993 
> E.YOUNGDALE (C) 1997-2006 J.PEARSON/J.SCHILLING (C) 2006-2007 CDRKIT TEAM
>   ...
>   Creation Time: 2019012014210500
> 
> You could put cdboot.img in there by naming it in the comma separated
> list of paths after -B. But i guess you should not.
> 
> --
> The following topics are out of my normal scope. So i can only throw in
> text snippets which may be related:
> 
>> 1) How does boot.S/diskboot.S locate core.img?
> 
> man xorrisofs says about -B:
> 
>   The pseudo disk_path  "..."  causes  that  all  empty  partition
>   entries  become  copies of the last non-empty entry. If no other
>   disk_path is given before "..." then all partitions describe the
>   ISO image. In this case, the boot loader code has to be imported
>   by option -G.
> 
> So having no boot images in the partition table is ok, in principle.
> 
>> This means that when the PROM opens the disk in boot.S, it is directly
>> opening the partition containing cdboot.img rather than the start of the
>> whole disk
> 
> It looks like all partitions lead to CD-ROM (where Nero burns).
> 
> 
>> 2) How to launch xorisso with the correct parameters?
> 
> With two "r" and only one "s". :))
> (xorriso = X/Open on Rock Ridge enhanced ISO 9660)
> 
>> grub-mkinstall
> 
> To what package does this belong ?
> Google proposes to search for "grub-install". But i don't think that this
> is for ISO 9600 production.

Hi Thomas,

Thanks again for all the information here, and my apologies for misspelling 
xorriso.

I've now double checked this against the SILO-based CDROM and I can see that 
the key
here is that the a.out loader is executed from sector 1 directly, and not the
individual slices. This means as per your partition table above we can set all 
slices
to the ISO image which also solves my question about how the bootpath lookup in
boot.S can work given that the slice is included.

Adrian: based upon this I think what you need to do is:

- For consistency rename /boot/grub/sparc64.elf to /boot/grub/core.img to match
  the 

Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-26 Thread Thomas Schmitt
Hi,

> - the first 32K of an ISO9660 is undefined

That's such a negative word. :))
It is "reserved for system use" and "not specified" by ISO 9660 / ECMA-119.


> - boot.S and diskboot.S get built into cdboot.img

grub-mkrescue opens cdboot.img for reading and the image file for -G
for writing and truncates it to 0 length. Then it writes 512 zero bytes,
reads 512 bytes from cdboot.img and writes them to the -G image file.


> - grub should run xorisso with the following parameters:
> -G cdboot.img -B "..." --grub2-sparc-core
> /boot/grub/sparc64-ieee1275/core.img

Actually these are for the mkisofs emulation of xorriso.
  xorriso -as mkisofs ...options.and.pathspecs...

(See man xorrisofs for options.)

Note that
  -B ','
as of grub-mkrescue is not totally equivalent to your
  -B '...'

Digging in libisoburn's xorriso/emulators.c i see that "," orders two
appended partitions with empty disk source paths. This will cause no
partitions to be appended.
"..." causes 7 partitions copying the partition entry data of the entry
before them. That would be the entry describing the ISO image as a whole.
Again no partitions images get appended.

One should test both and inspect by SUN tools or
  xorriso -indev $ISOIMAGE -report_system_area plain
Probably "..." will produce a partition table with 8 entries, and ","
will produce a table with only one entry.


> - what this effectively does is:
>   - Install a sun partition disk label to sector 0 of the ISO image
>   - Add the ISO9660 image as the first partition (slice)
>   - Create all 7 remaining partitions after the ISO9660 image

I expect this too.

>   - These partitions are all small and just contain cdboot.img

I expect what can be seen in debian-10.0-sparc64-NETINST-1.iso

  Volume id: 'Debian 10.0 sparc64 n'
  ...
  System area summary: SUN-SPARC-Disk-Label
  ISO image size/512 : 284760
  SUN SPARC disklabel: CD-ROM Disc with Sun sparc boot created by genisoimage
  SUN SPARC secs/head: 640
  SUN SPARC heads/cyl: 1
  SUN SPARC partmap  :   N   IdTag   PermsStartCyl   NumBlocks
  SUN SPARC partition:   1  0x0004  0x0010   0  284160
  SUN SPARC partition:   2  0x0002  0x0010   0  284160
  SUN SPARC partition:   3  0x0002  0x0010   0  284160
  SUN SPARC partition:   4  0x0002  0x0010   0  284160
  SUN SPARC partition:   5  0x0002  0x0010   0  284160
  SUN SPARC partition:   6  0x0002  0x0010   0  284160
  SUN SPARC partition:   7  0x0002  0x0010   0  284160
  SUN SPARC partition:   8  0x0002  0x0010   0  284160

xorriso command -pvd_info says:

  App Id   : GENISOIMAGE ISO 9660/HFS FILESYSTEM CREATOR (C) 1993 
E.YOUNGDALE (C) 1997-2006 J.PEARSON/J.SCHILLING (C) 2006-2007 CDRKIT TEAM
  ...
  Creation Time: 2019012014210500

You could put cdboot.img in there by naming it in the comma separated
list of paths after -B. But i guess you should not.

--
The following topics are out of my normal scope. So i can only throw in
text snippets which may be related:

> 1) How does boot.S/diskboot.S locate core.img?

man xorrisofs says about -B:

  The pseudo disk_path  "..."  causes  that  all  empty  partition
  entries  become  copies of the last non-empty entry. If no other
  disk_path is given before "..." then all partitions describe the
  ISO image. In this case, the boot loader code has to be imported
  by option -G.

So having no boot images in the partition table is ok, in principle.

> This means that when the PROM opens the disk in boot.S, it is directly
> opening the partition containing cdboot.img rather than the start of the
> whole disk

It looks like all partitions lead to CD-ROM (where Nero burns).


> 2) How to launch xorisso with the correct parameters?

With two "r" and only one "s". :))
(xorriso = X/Open on Rock Ridge enhanced ISO 9660)

> grub-mkinstall

To what package does this belong ?
Google proposes to search for "grub-install". But i don't think that this
is for ISO 9600 production.


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-26 Thread Mark Cave-Ayland
On 25/04/2019 12:58, Thomas Schmitt wrote:

> Hi,
> 
> John Paul Adrian Glaubitz wrote:
>> I have read the genisoimage manpage and I have to admit, I haven't fully 
>> understood
>> yet how it is supposed to work. It seems we have to specify the bootloader 
>> code
>> with -G, thus -G cdboot.img. But -B is apparently used to pass a whole 
>> directory
>> name which means I don't understand how genisoimage knows which is the 
>> second stage bootloader.
> 
> Dumping here what i learned when Vladimir Serbinko told me how to
> do it with xorriso underneath grub-mkrescue:
> 
> libisofs/doc/boot_sectors.txt has:
> =
>   GRUB2 SUN SPARC Core File Address
> 
> Sources:
> Mail conversations with Vladimir Serbinenko.
> 
> GRUB2 lets libisofs write after the disk label block the address and size of a
> data file in the ISO image. E.g. of /boot/grub/sparc64-ieee1275/core.img.
> This is combined with a SUN Disk Label which exposes only the single partition
> describing the overall ISO filesystem size.
> 
>   Byte Range | Value  | Meaning
>  | -- | --
>  512 -   551 | opaque | Code and data provided by GRUB2
>  ||
>  552 -   559 | offset | Start byte number of the file. 64-bit big-endian.
>  ||
>  560 -   563 |  size  | Number of bytes in the file. 32-bit big-endian.
>  ||
>  564 - 32767 | opaque | Code and data provided by GRUB2
>  ||
>  | -- | --
> 
> =
> (There is a section "SUN Disk Label and boot images for SUN SPARC" before
>  that special GRUB description.)
> 
> man xorrisofs:
> =
> 
>   -B disk_path[,disk_path ...]
> Cause one or more data files on disk to be written after the end
> of  the  ISO  image.  A  SUN Disk Label will be written into the
> first 512 bytes of the ISO  image  which  lists  this  image  as
> partition 1 and the given disk_paths as partition 2 up to 8.
> The disk files should contain suitable boot images for SUN SPARC
> systems.
> The pseudo disk_path  "..."  causes  that  all  empty  partition
> entries  become  copies of the last non-empty entry. If no other
> disk_path is given before "..." then all partitions describe the
> ISO image. In this case, the boot loader code has to be imported
> by option -G.
> 
> =
> 
> util/grub-mkrescue.c does
> =
> 
>   if (source_dirs[GRUB_INSTALL_PLATFORM_SPARC64_IEEE1275]
>   && system_area == SYS_AREA_SPARC)
> {
>   ...
>   xorriso_push ("-G");
>   xorriso_push (sysarea_img);
>   xorriso_push ("-B");
>   xorriso_push (",");
>   xorriso_push ("--grub2-sparc-core");
>   xorriso_push ("/boot/grub/sparc64-ieee1275/core.img");
> 
> =
> 
> The mkisofs emulation option --grub2-sparc-core does the stuff described
> in boot_sectors.txt.

This is a great summary! I can almost see how this works, the last piece in the
puzzle is to correlate it with what is in boot.S and diskboot.S.

What I believe should be happening is this:

- the first 32K of an ISO9660 is undefined so that a partition table can be 
added
  for multi-format disks such as this

- boot.S and diskboot.S get built into cdboot.img (see
https://www.gnu.org/software/grub/manual/grub/html_node/Images.html for the 
breakdown
of grub components) which
is an a.out executable that can be loaded/executed by the PROM

- the main grub executable gets built as core.img and is included in the ISO9660
image at /boot/grub/sparc64-ieee1275/core.img

- grub should run xorisso with the following parameters:
 -G cdboot.img -B "..." --grub2-sparc-core 
/boot/grub/sparc64-ieee1275/core.img

- what this effectively does is:
- Install a sun partition disk label to sector 0 of the ISO image
- Add the ISO9660 image as the first partition (slice)
- Create all 7 remaining partitions after the ISO9660 image
- These partitions are all small and just contain cdboot.img
- Create a special sector 1 as per above containing the offset and size of
  core.img *within the ISO9660 partition* from the start of the image in 
bytes

This gets us to the point where the PROM can read cdboot.img and start 
executing it,
so far so good.

The two parts I'm still not exactly sure about:


1) How does boot.S/diskboot.S locate core.img?

>From what I can see boot.S uses the "bootpath" property as 

Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-25 Thread John Paul Adrian Glaubitz
Hi Thomas!

On 4/25/19 1:58 PM, Thomas Schmitt wrote:
> 
> (snip)
> 
> I hope this helps to get more insight. More background might be known
> to Vladimir.

Thanks, that looks very useful. I will have a look and try to understand
how CD boot with GRUB is supposed to work. I have also posted that question
to the grub-devel mailing list, so it's probably just a matter of time
until Vladimir answers :).

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-25 Thread Thomas Schmitt
Hi,

John Paul Adrian Glaubitz wrote:
> I have read the genisoimage manpage and I have to admit, I haven't fully 
> understood
> yet how it is supposed to work. It seems we have to specify the bootloader 
> code
> with -G, thus -G cdboot.img. But -B is apparently used to pass a whole 
> directory
> name which means I don't understand how genisoimage knows which is the second 
> stage bootloader.

Dumping here what i learned when Vladimir Serbinko told me how to
do it with xorriso underneath grub-mkrescue:

libisofs/doc/boot_sectors.txt has:
=
  GRUB2 SUN SPARC Core File Address

Sources:
Mail conversations with Vladimir Serbinenko.

GRUB2 lets libisofs write after the disk label block the address and size of a
data file in the ISO image. E.g. of /boot/grub/sparc64-ieee1275/core.img.
This is combined with a SUN Disk Label which exposes only the single partition
describing the overall ISO filesystem size.

  Byte Range | Value  | Meaning
 | -- | --
 512 -   551 | opaque | Code and data provided by GRUB2
 ||
 552 -   559 | offset | Start byte number of the file. 64-bit big-endian.
 ||
 560 -   563 |  size  | Number of bytes in the file. 32-bit big-endian.
 ||
 564 - 32767 | opaque | Code and data provided by GRUB2
 ||
 | -- | --

=
(There is a section "SUN Disk Label and boot images for SUN SPARC" before
 that special GRUB description.)

man xorrisofs:
=

  -B disk_path[,disk_path ...]
Cause one or more data files on disk to be written after the end
of  the  ISO  image.  A  SUN Disk Label will be written into the
first 512 bytes of the ISO  image  which  lists  this  image  as
partition 1 and the given disk_paths as partition 2 up to 8.
The disk files should contain suitable boot images for SUN SPARC
systems.
The pseudo disk_path  "..."  causes  that  all  empty  partition
entries  become  copies of the last non-empty entry. If no other
disk_path is given before "..." then all partitions describe the
ISO image. In this case, the boot loader code has to be imported
by option -G.

=

util/grub-mkrescue.c does
=

  if (source_dirs[GRUB_INSTALL_PLATFORM_SPARC64_IEEE1275]
  && system_area == SYS_AREA_SPARC)
{
  ...
  xorriso_push ("-G");
  xorriso_push (sysarea_img);
  xorriso_push ("-B");
  xorriso_push (",");
  xorriso_push ("--grub2-sparc-core");
  xorriso_push ("/boot/grub/sparc64-ieee1275/core.img");

=

The mkisofs emulation option --grub2-sparc-core does the stuff described
in boot_sectors.txt.


(And i see nice examples of grub_util_fopen() and fwrite(3) by which
 grub-mkrescue could zero the EFI partition's partition table after
 mformat did its work ...)


I hope this helps to get more insight. More background might be known
to Vladimir.


Have a nice day :)

Thomas



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-25 Thread John Paul Adrian Glaubitz
On 4/25/19 9:30 AM, Mark Cave-Ayland wrote:
> Ah now I see - since your PROM supports gpt directly then that's how boot.S 
> works.
> Presumably then it must be diskboot.S with blocklist support that gets used 
> for older
> machines such as QEMU's sun4u?

Makes sense, yes.

> Okay so reviewing the genisoimage man page again I think you should be able 
> to use
> the blocklist approach for the CDROM: install grub into 
> /boot/grub/sparc64.elf on the
> ISO9660 partition, then generate a cdrom.img using diskboot.S that starts 
> with your
> a.out bootloader and includes the block list for sparc64.elf taken from the 
> ISO9660
> partition.

FWIW, the filename "sparc64.elf" is arbitrarily chosen by me and probably wrong
anyway. Looking at the sources for grub-install [1], the proper name would 
probably
be "boot.img".

> The part that needs to be checked here is whether diskboot.S (and indeed your 
> PROM
> disk and cdrom aliases) contain the slice because the blocklist needs to come 
> from
> the "raw" disk device since sparc64.elf is being pulled from a different 
> slice.
> Adding some code to boot.S to display bootpath will be helpful here. But then
> presumably this is the same as the blocklist configuration above anyhow?

I have read the genisoimage manpage and I have to admit, I haven't fully 
understood
yet how it is supposed to work. It seems we have to specify the bootloader code
with -G, thus -G cdboot.img. But -B is apparently used to pass a whole directory
name which means I don't understand how genisoimage knows which is the second
stage bootloader.

For SILO, the design seems more obvious with the path of the second stage boot
loader hardcoded into the source code [2]. But for GRUB, I still don't 
understand
how it's supposed to work. Maybe it has "boot.img" hardcoded somewhere and tries
to load boot.img from the partition specified with -B.

Adrian

> [1] http://git.savannah.gnu.org/cgit/grub.git/tree/util/grub-install.c#n1719
> [2] 
> https://git.kernel.org/pub/scm/linux/kernel/git/davem/silo.git/tree/first-isofs/isofs.c

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-25 Thread Mark Cave-Ayland
On 25/04/2019 07:50, John Paul Adrian Glaubitz wrote:

> On 4/25/19 8:40 AM, Mark Cave-Ayland wrote:
>> When grub is installed to disk as per your latest ISO images, what is the
>> filesystem/partition type being used?
> For GPT-based partitioning, we have a dedicated GRUB boot partition:
> 
> (parted) p
> Model: Unknown (unknown)
> Disk /dev/vdiska: 85.9GB
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
> Disk Flags: 
> 
> Number  Start   End SizeFile system Name  Flags
>  1  1049kB  1075MB  1074MB  ext3  boot, esp
>  2  1075MB  1076MB  1049kBbios_grub
>  3  1076MB  3075MB  2000MB  linux-swap(v1)
>  4  3075MB  85.9GB  82.8GB  ext4
> 
> (parted)
> 
> For machines with Sun labels, GRUB is actually installed using block lists:
> 
> (parted) p
> Model: Unknown (unknown)
> Disk /dev/vdiskb: 1100GB
> Sector size (logical/physical): 512B/8192B
> Partition Table: sun
> Disk Flags: 
> 
> Number  Start   End SizeFile system Flags
>  1  0.00B   8192MB  8192MB  ext4boot
>  2  8192MB  1016GB  1008GB  ext4
>  4  1016GB  1100GB  83.7GB  linux-swap(v1)
> 
> (parted)
> 
> See also: https://github.com/esnowberg/grub2-sparc/wiki

Ah now I see - since your PROM supports gpt directly then that's how boot.S 
works.
Presumably then it must be diskboot.S with blocklist support that gets used for 
older
machines such as QEMU's sun4u?

Okay so reviewing the genisoimage man page again I think you should be able to 
use
the blocklist approach for the CDROM: install grub into /boot/grub/sparc64.elf 
on the
ISO9660 partition, then generate a cdrom.img using diskboot.S that starts with 
your
a.out bootloader and includes the block list for sparc64.elf taken from the 
ISO9660
partition.

The part that needs to be checked here is whether diskboot.S (and indeed your 
PROM
disk and cdrom aliases) contain the slice because the blocklist needs to come 
from
the "raw" disk device since sparc64.elf is being pulled from a different slice.
Adding some code to boot.S to display bootpath will be helpful here. But then
presumably this is the same as the blocklist configuration above anyhow?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-25 Thread John Paul Adrian Glaubitz
On 4/25/19 8:40 AM, Mark Cave-Ayland wrote:
> When grub is installed to disk as per your latest ISO images, what is the
> filesystem/partition type being used?
For GPT-based partitioning, we have a dedicated GRUB boot partition:

(parted) p
Model: Unknown (unknown)
Disk /dev/vdiska: 85.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End SizeFile system Name  Flags
 1  1049kB  1075MB  1074MB  ext3  boot, esp
 2  1075MB  1076MB  1049kBbios_grub
 3  1076MB  3075MB  2000MB  linux-swap(v1)
 4  3075MB  85.9GB  82.8GB  ext4

(parted)

For machines with Sun labels, GRUB is actually installed using block lists:

(parted) p
Model: Unknown (unknown)
Disk /dev/vdiskb: 1100GB
Sector size (logical/physical): 512B/8192B
Partition Table: sun
Disk Flags: 

Number  Start   End SizeFile system Flags
 1  0.00B   8192MB  8192MB  ext4boot
 2  8192MB  1016GB  1008GB  ext4
 4  1016GB  1100GB  83.7GB  linux-swap(v1)

(parted)

See also: https://github.com/esnowberg/grub2-sparc/wiki

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-25 Thread Mark Cave-Ayland
On 25/04/2019 07:29, John Paul Adrian Glaubitz wrote:

> On 4/25/19 8:07 AM, Mark Cave-Ayland wrote:
>>> Yes, that would be most likely the sparc64.elf image that grub-mkimage 
>>> produces. But
>>> I don't know yet how to encode the boot path for that into the bootable 
>>> image.
>>
>> I'm struggling to see from the man pages the relationship between 
>> sparc64.elf and
>> cdboot.img. Presumably grub-mkimage generates sparc64.elf which is a 
>> self-contained
>> grub executable, whilst cdboot.img is the compiled version of boot.S?
> 
> Yes, sparc64.elf is the boot image created by grub-mkimage and whose bootpath 
> needs
> to be passed to the initial boot loader.

I'd expect that to be done by boot.S since the PROM on it's own can't load
sparc64.elf directly.

A quick browse of the source shows that it looks up bootpath and tries to load 
the
grub kernel from the same device, i.e. the partition or slice that cdrom and/or 
disk
in the PROM is aliased to. I can't see how that would work because that's where
boot.S was loaded from in the first place.

When grub is installed to disk as per your latest ISO images, what is the
filesystem/partition type being used?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-25 Thread John Paul Adrian Glaubitz
On 4/25/19 8:07 AM, Mark Cave-Ayland wrote:
>> Yes, that would be most likely the sparc64.elf image that grub-mkimage 
>> produces. But
>> I don't know yet how to encode the boot path for that into the bootable 
>> image.
> 
> I'm struggling to see from the man pages the relationship between sparc64.elf 
> and
> cdboot.img. Presumably grub-mkimage generates sparc64.elf which is a 
> self-contained
> grub executable, whilst cdboot.img is the compiled version of boot.S?

Yes, sparc64.elf is the boot image created by grub-mkimage and whose bootpath 
needs
to be passed to the initial boot loader.

I'm looking now at Oracle's Linux images for SPARC to figure out how they boot:

> http://ftp.icm.edu.pl/packages/linux-oracle-sparc/6.9UEK4/OL-201707062043-R6-U9-sparc-dvd.iso

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-25 Thread Mark Cave-Ayland
On 25/04/2019 06:54, John Paul Adrian Glaubitz wrote:

> On 4/25/19 7:23 AM, Mark Cave-Ayland wrote:
>> Oh wait, now I see it: SILO's isofs.b also includes a mini-ISO9660 driver to 
>> enable
>> the PROM to read from the CDROM. So given that grub on SPARC64 is already 
>> working for
>> a disk install then this aspect should already have been solved i.e. either 
>> the grub
>> kernel is located on a separate UFS slice within a sun partition or 
>> somewhere there
>> is an enhanced boot.S with another FS driver?
> 
> Yes, that would be most likely the sparc64.elf image that grub-mkimage 
> produces. But
> I don't know yet how to encode the boot path for that into the bootable image.

I'm struggling to see from the man pages the relationship between sparc64.elf 
and
cdboot.img. Presumably grub-mkimage generates sparc64.elf which is a 
self-contained
grub executable, whilst cdboot.img is the compiled version of boot.S?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread John Paul Adrian Glaubitz
On 4/25/19 7:23 AM, Mark Cave-Ayland wrote:
> Oh wait, now I see it: SILO's isofs.b also includes a mini-ISO9660 driver to 
> enable
> the PROM to read from the CDROM. So given that grub on SPARC64 is already 
> working for
> a disk install then this aspect should already have been solved i.e. either 
> the grub
> kernel is located on a separate UFS slice within a sun partition or somewhere 
> there
> is an enhanced boot.S with another FS driver?

Yes, that would be most likely the sparc64.elf image that grub-mkimage 
produces. But
I don't know yet how to encode the boot path for that into the bootable image.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread John Paul Adrian Glaubitz
On 4/25/19 6:18 AM, Mark Cave-Ayland wrote:
> FWIW I think you're pretty close here: as you say, you just need to fix the 
> bits
> which set up a sun partition label and inject the bootloader. So the bit you 
> need to
> re-enable to get this to work should be this:
> 
>   add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G boot1/boot/isofs.b -B ..."
> 
> From looking at your previous grub patches and the genisoimage man page then 
> you need
> to point this towards the a.out binary made from
> grub/grub-core/boot/sparc64/ieee1275/boot.S which I think reading through
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921249 should be cdboot.img?

Yes, that's also what I think. And, having just recently worked on this 
particular
part of GRUB uptream, I'm pretty sure we need to inject cdboot.img.

> The mkisofs above should create a sun partition layout and inject the a.out 
> binary
> into all slices so it is executed with "boot cdrom". This will then use the 
> CIF to
> locate the grub partition via the PROM bootpath property and the open the 
> ISO9660
> device in the PROM to locate the grub kernel.

The question is: We have the "sparc64.elf" GRUB boot image on the CD-ROM in the
subdirectory boot/. How do we tell the initial bootloader how to use it?

> Note: this assumes that the PROM can read ISO9660 devices directly rather than
> requiring a UFS filesystem - can all 64-bit SPARC PROMs do this? The above 
> diff
> suggests that this was the case for SILO.

I would guess so.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread Mark Cave-Ayland
On 25/04/2019 05:18, Mark Cave-Ayland wrote:

> On 24/04/2019 23:05, John Paul Adrian Glaubitz wrote:
> 
>> Okay, so I have managed to build an image now, that's the good news:
>>
>>> https://people.debian.org/~glaubitz/debian-10.0-sparc64-grub-NETINST-1.iso
>>
>> The bad news is, it doesn't boot yet:
>>
>> {0} ok boot cdrom
>> Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args: 
>> ERROR: /virtual-devices@100/channel-devices@200/disk@1: Can't open disk 
>> label package
>>
>>
>> Can't open boot device
>>
>> {0} ok
>>
>> With the attached d-i patch, debian-installer builds with GRUB instead of 
>> SILO
>> and produces a sparc64.elf boot image plus a grub.cfg configuration file.
>>
>> The other patch shows the current modifications to debian-cd.
>>
>> I assume the only thing that is missing is the correct parameter to mkisofs
>> so that it builds a bootable image, but I haven't looked that up yet. And we
>> don't need to create the directory "sparc64-ieee1275", it stays empty anyway
>> as opposed to ppc64el.
> 
> FWIW I think you're pretty close here: as you say, you just need to fix the 
> bits
> which set up a sun partition label and inject the bootloader. So the bit you 
> need to
> re-enable to get this to work should be this:
> 
>   add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G boot1/boot/isofs.b -B ..."
> 
>>From looking at your previous grub patches and the genisoimage man page then 
>>you need
> to point this towards the a.out binary made from
> grub/grub-core/boot/sparc64/ieee1275/boot.S which I think reading through
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921249 should be cdboot.img?
> 
> The mkisofs above should create a sun partition layout and inject the a.out 
> binary
> into all slices so it is executed with "boot cdrom". This will then use the 
> CIF to
> locate the grub partition via the PROM bootpath property and the open the 
> ISO9660
> device in the PROM to locate the grub kernel.
> 
> Note: this assumes that the PROM can read ISO9660 devices directly rather than
> requiring a UFS filesystem - can all 64-bit SPARC PROMs do this? The above 
> diff
> suggests that this was the case for SILO.

Oh wait, now I see it: SILO's isofs.b also includes a mini-ISO9660 driver to 
enable
the PROM to read from the CDROM. So given that grub on SPARC64 is already 
working for
a disk install then this aspect should already have been solved i.e. either the 
grub
kernel is located on a separate UFS slice within a sun partition or somewhere 
there
is an enhanced boot.S with another FS driver?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread Mark Cave-Ayland
On 24/04/2019 23:05, John Paul Adrian Glaubitz wrote:

> Okay, so I have managed to build an image now, that's the good news:
> 
>> https://people.debian.org/~glaubitz/debian-10.0-sparc64-grub-NETINST-1.iso
> 
> The bad news is, it doesn't boot yet:
> 
> {0} ok boot cdrom
> Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args: 
> ERROR: /virtual-devices@100/channel-devices@200/disk@1: Can't open disk label 
> package
> 
> 
> Can't open boot device
> 
> {0} ok
> 
> With the attached d-i patch, debian-installer builds with GRUB instead of SILO
> and produces a sparc64.elf boot image plus a grub.cfg configuration file.
> 
> The other patch shows the current modifications to debian-cd.
> 
> I assume the only thing that is missing is the correct parameter to mkisofs
> so that it builds a bootable image, but I haven't looked that up yet. And we
> don't need to create the directory "sparc64-ieee1275", it stays empty anyway
> as opposed to ppc64el.

FWIW I think you're pretty close here: as you say, you just need to fix the bits
which set up a sun partition label and inject the bootloader. So the bit you 
need to
re-enable to get this to work should be this:

  add_mkisofs_opt $CDDIR/../$N.mkisofs_opts "-G boot1/boot/isofs.b -B ..."

>From looking at your previous grub patches and the genisoimage man page then 
>you need
to point this towards the a.out binary made from
grub/grub-core/boot/sparc64/ieee1275/boot.S which I think reading through
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921249 should be cdboot.img?

The mkisofs above should create a sun partition layout and inject the a.out 
binary
into all slices so it is executed with "boot cdrom". This will then use the CIF 
to
locate the grub partition via the PROM bootpath property and the open the 
ISO9660
device in the PROM to locate the grub kernel.

Note: this assumes that the PROM can read ISO9660 devices directly rather than
requiring a UFS filesystem - can all 64-bit SPARC PROMs do this? The above diff
suggests that this was the case for SILO.


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread John Paul Adrian Glaubitz
Okay, so I have managed to build an image now, that's the good news:

> https://people.debian.org/~glaubitz/debian-10.0-sparc64-grub-NETINST-1.iso

The bad news is, it doesn't boot yet:

{0} ok boot cdrom
Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args: 
ERROR: /virtual-devices@100/channel-devices@200/disk@1: Can't open disk label 
package


Can't open boot device

{0} ok

With the attached d-i patch, debian-installer builds with GRUB instead of SILO
and produces a sparc64.elf boot image plus a grub.cfg configuration file.

The other patch shows the current modifications to debian-cd.

I assume the only thing that is missing is the correct parameter to mkisofs
so that it builds a bootable image, but I haven't looked that up yet. And we
don't need to create the directory "sparc64-ieee1275", it stays empty anyway
as opposed to ppc64el.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
>From 95d2e56c80db93aee39761b73ff6099b033e0575 Mon Sep 17 00:00:00 2001
From: John Paul Adrian Glaubitz 
Date: Wed, 24 Apr 2019 21:59:38 +0200
Subject: [PATCH] sparc64: Switch bootloader for d-i images from silo to
 grub-ieee1275

---
 build/boot/sparc64/debian.txt   | 17 -
 build/boot/sparc64/grub-cdrom.cfg   | 40 
 build/boot/sparc64/notsupported.txt |  5 ---
 build/boot/sparc64/silo.conf| 33 
 build/config/sparc64.cfg| 58 ++---
 build/config/sparc64/cdrom.cfg  |  9 ++---
 build/config/sparc64/miniiso.cfg| 30 ---
 build/config/sparc64/netboot.cfg| 19 ++
 debian/changelog|  3 ++
 debian/control  |  4 +-
 10 files changed, 105 insertions(+), 113 deletions(-)
 delete mode 100644 build/boot/sparc64/debian.txt
 create mode 100644 build/boot/sparc64/grub-cdrom.cfg
 delete mode 100644 build/boot/sparc64/notsupported.txt
 delete mode 100644 build/boot/sparc64/silo.conf
 delete mode 100644 build/config/sparc64/miniiso.cfg

diff --git a/build/boot/sparc64/debian.txt b/build/boot/sparc64/debian.txt
deleted file mode 100644
index 1bae0c634..0
--- a/build/boot/sparc64/debian.txt
+++ /dev/null
@@ -1,17 +0,0 @@
-
-  Welcome to Debian GNU/Linux ${DEBIAN_VERSION}!
-
-This is a Debian installation ${MEDIA_TYPE}, built on ${BUILD_DATE}.
-Keep it once you have installed your system, as you can boot from it
-to repair the system on your hard disk if that ever becomes necessary.
-
-WARNING: You should completely back up all of your hard disks before
-  proceeding. The installation procedure can completely and irreversibly
-  erase them! If you haven't made backups yet, remove the rescue CD from
-  the drive and press L1-A to get back to the OpenBoot prompt.
-
-Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted
-by applicable law.
-
-[ ENTER - Boot install ]   [ Type "expert" - Boot into expert mode ]
-   [ Type "rescue" - Boot into rescue mode ]
diff --git a/build/boot/sparc64/grub-cdrom.cfg b/build/boot/sparc64/grub-cdrom.cfg
new file mode 100644
index 0..41af6af63
--- /dev/null
+++ b/build/boot/sparc64/grub-cdrom.cfg
@@ -0,0 +1,40 @@
+set default=2
+set timeout=-1
+
+insmod echo
+insmod gzio
+insmod minicmd
+insmod normal
+
+menuentry "Debian GNU/Linux installer boot menu" {
+	true
+}
+
+menuentry "" {
+	true
+}
+
+function boot_one {
+	echo "Loading ..."
+	linux	${KERNEL} $options --- quiet
+	initrd	${INITRD}
+}
+
+menuentry "Default install" {
+boot_one
+}
+
+menuentry "Automated install" {
+	set options="auto=true priority=critical"
+	boot_one
+}
+
+menuentry "Expert install" {
+	set options="priority=low"
+	boot_one
+}
+
+menuentry "Rescue mode" {
+	set options="rescue/enable=true"
+	boot_one
+}
diff --git a/build/boot/sparc64/notsupported.txt b/build/boot/sparc64/notsupported.txt
deleted file mode 100644
index 871731fb3..0
--- a/build/boot/sparc64/notsupported.txt
+++ /dev/null
@@ -1,5 +0,0 @@
-
-This subarchitecture is currently not supported by Debian.
-Only 64-bit systems are supported (sparc64).
-
-The last Debian release to support sparc32 (sun4m) was Etch (4.0).
diff --git a/build/boot/sparc64/silo.conf b/build/boot/sparc64/silo.conf
deleted file mode 100644
index 307e1fe09..0
--- a/build/boot/sparc64/silo.conf
+++ /dev/null
@@ -1,33 +0,0 @@
-partition=1
-timeout=600
-message=/boot/debian.txt
-default=install
-initrd=/boot/initrd.gz
-read-write
-
-# Standard boot images
-image[sun4u]=/boot/vmlinuz-sparc64
-  label=install
-image[sun4,sun4c,sun4d,sun4m]="cat /boot/notsupported.txt"
-  label=install
-
-# Expert boots
-image[sun4u]=/boot/vmlinuz-sparc64
-  label=expert
-  append="priority=low"
-image[sun4,sun4c,sun4d,sun4m]="cat /boot/notsupported.txt"
-  label=expert
-
-# 

Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread Mark Cave-Ayland
On 24/04/2019 21:30, Frank Scheiner wrote:

> On 4/24/19 22:07, John Paul Adrian Glaubitz wrote:
>> On 4/24/19 9:46 PM, Mark Cave-Ayland wrote:
>>> For reference the CHRP bootinfo.txt isn't a configuration file, but is 
>>> actually
>>> parsed by CHRP-compliant open firmwares directly - SPARC firmwares, 
>>> including
>>> OpenBIOS don't support them. Can you explain why grub on SPARC is trying to 
>>> access
>>> this file?
>>
>> Good point, I didn't notice that there is a CHRP header and assumed that we 
>> need
>> this file on sparc and sparc64 as well.
>>
>> I haven't actually done any runtime tests yet, I replaced the configuration 
>> files
>> from ppc64el for grub-ieee1275 and adapted them for sparc64. debian-installer
>> builds fine with those changes, but as I said, I haven't tested anything yet.
>>
>> I am very welcome for any pointers though for what else is needed to use GRUB
>> for CD boot on sparc64.
> 
> `grub-mkimage` has a (target) format named "sparc64-ieee1275-cdcore".
> But I'm unsure if this is of any use for us, or needed at all, because
> if GRUB can load kernel and initramfs from disc, it should also be able
> to load its modules from disc, so a small GRUB image and separate
> modules (like for on-disk or diskless installations) could also work.
> 
> OTOH: if a "sparc64-ieee1275-cdcore" image and a corresponding grub.cfg
> works for disc booting, we could just exchange the SILO image and config
> for these.

Is it possible to see the source for sparc64-ieee1275-cdcore anywhere? A quick 
search
doesn't return anything obvious. And from memory grub is working when installed 
onto
the disk? If so, where is the source of the disk image for comparison?


ATB,

Mark.



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread John Paul Adrian Glaubitz
Closing as the bug report is obviously invalid as bootinfo.txt is not
required on sparc and sparc64.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread John Paul Adrian Glaubitz
(Leave the bug tracker, I'll close the bug)

On 4/24/19 10:43 PM, Mark Cave-Ayland wrote:
> Is it possible to see the source for sparc64-ieee1275-cdcore anywhere? A 
> quick search
> doesn't return anything obvious.

The regular grub2 packaging source is on salsa, see: 
https://salsa.debian.org/grub-team/grub

> And from memory grub is working when installed onto the disk?

GRUB has been the default disk bootloader on Debian sparc64 for a while now.

> If so, where is the source of the disk image for comparison?

There is no disk image. GRUB installed using "grub-install".

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread Frank Scheiner
On 4/24/19 22:07, John Paul Adrian Glaubitz wrote:
> On 4/24/19 9:46 PM, Mark Cave-Ayland wrote:
>> For reference the CHRP bootinfo.txt isn't a configuration file, but is 
>> actually
>> parsed by CHRP-compliant open firmwares directly - SPARC firmwares, including
>> OpenBIOS don't support them. Can you explain why grub on SPARC is trying to 
>> access
>> this file?
>
> Good point, I didn't notice that there is a CHRP header and assumed that we 
> need
> this file on sparc and sparc64 as well.
>
> I haven't actually done any runtime tests yet, I replaced the configuration 
> files
> from ppc64el for grub-ieee1275 and adapted them for sparc64. debian-installer
> builds fine with those changes, but as I said, I haven't tested anything yet.
>
> I am very welcome for any pointers though for what else is needed to use GRUB
> for CD boot on sparc64.

`grub-mkimage` has a (target) format named "sparc64-ieee1275-cdcore".
But I'm unsure if this is of any use for us, or needed at all, because
if GRUB can load kernel and initramfs from disc, it should also be able
to load its modules from disc, so a small GRUB image and separate
modules (like for on-disk or diskless installations) could also work.

OTOH: if a "sparc64-ieee1275-cdcore" image and a corresponding grub.cfg
works for disc booting, we could just exchange the SILO image and config
for these.

Cheers,
Frank



Re: Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread John Paul Adrian Glaubitz
On 4/24/19 9:46 PM, Mark Cave-Ayland wrote:
> For reference the CHRP bootinfo.txt isn't a configuration file, but is 
> actually
> parsed by CHRP-compliant open firmwares directly - SPARC firmwares, including
> OpenBIOS don't support them. Can you explain why grub on SPARC is trying to 
> access
> this file?

Good point, I didn't notice that there is a CHRP header and assumed that we need
this file on sparc and sparc64 as well.

I haven't actually done any runtime tests yet, I replaced the configuration 
files
from ppc64el for grub-ieee1275 and adapted them for sparc64. debian-installer
builds fine with those changes, but as I said, I haven't tested anything yet.

I am very welcome for any pointers though for what else is needed to use GRUB
for CD boot on sparc64.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Bug#927892: grub-ieee1275-bin: Please add bootinfo.txt for sparc and sparc64

2019-04-24 Thread John Paul Adrian Glaubitz
Source: grub2
Version: 2.02+dfsg1-17
Severity: normal
User: debian-sparc@lists.debian.org
Usertags: sparc64

Hello!

I have started working on switching the bootloader for debian-installer from
silo to grub-ieee1275 on sparc and sparc64.

For that, I have looked at the corresponding code in debian-installer for
ppc64el and noticed that the configuration file build/config/ppc64el.cfg
requires a "bootinfo.txt" configuration file which is part of the
grub-ieee1275-bin package.

Thus, in order to switch to grub-ieee1275 on sparc and sparc64, we need
to add the bootinfo.txt there as well. Could you install the bootinfo.txt
file for sparc and sparc64 as well?

Plus, it might be a good idea to adjust the GRUB version number as well
as the name of the boot script to be a generic name like debian-installer.elf
and not powerpc.elf.

Currently, the header of bootinfo.txt looks like this:


grub 2.02~beta3
grub 2.02~beta3
boot :\boot\grub\powerpc.elf


Thanks,
Adrian

--
  .''`.  John Paul Adrian Glaubitz
 : :' :  Debian Developer - glaub...@debian.org
 `. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
   `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913