Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

Hi Thomas,

On Jun 3, 2018, at 1:26 AM, Thomas Schmitt  wrote:

> Hi,
> 
> Rick Thomas wrote:
>> Instead, I used
>>   firmware-9.4.0-amd64-netinst.iso
>> to avoid any possible alpha/testing anomalies.
> 
> Normally i'd say that there is no decisive difference to expect.
> In any case copying a "netinst" ISO is much faster than a "DVD-1".
> 
> 
>> When I tried part (a) — just a simple dd — with
>> debian-9.4.0-amd64-netinst.iso, the minnowboard immediately recognized it as
>> bootable and presented it as part of the EFI menu labeled “EFI USB Device”.
> 
> Normally this difference should not appear. Very strange.
> 
> But we can now classify the demand for GPT on
>  https://minnowboard.org/tutorials/best-practice-boot-media-selection
> as spin-off of the usual rumor that EFI needs GPT.
> 
> 
>> it booted without incident and ran the installer.
>> When it came to the part where it looks for a CD, it easily found the USB
>> stick
> 
> This was expectable. As long as the ISO is marked as partition, the
> script in the initrd should find the device.
> 
> 
>> tried
>> part (a) with “firmware-buster-DI-alpha2-amd64-DVD-1.iso”.  And guess what!
>> That worked too, just the same as the other two.
> 
> Hrmpf. At least above "normally"s are not generally put in doubt.
> It would be another riddle if one ISO would fail reproducibly while
> the other succedds reproducibly.
> 
> Maybe you changed a setting in the firmware ?

In the previous experiments (where I modified the partition tables on the stick 
after dd-ing) I did try some different setting in the firmware.  But for the 
above described experiments, I reset the firmware to factory defaults before 
starting the process and made no changes thereafter.


>> or there’s something flakey about the USB ports on this device
> 
> But why did the manual start of “fs0:\efi\boot\bootx64.efi” ran the
> boot loader, loaded the kernel, and came far enough to run into the
> software shortcoming of device detection ?

I’m not absolutely sure that the manual start was necessary.  I may have gotten 
used to never seeing the “EFI USB Device” menu entry.  So when the stars 
aligned and the stick *was* seen, I didn’t notice.  So, maybe, the fact that I 
got it to work from the command line was just because it would have worked 
anyway that time from the EFI menu.

>> In any case, I apologize for all the noise.
> 
> I doubt that this was a hallucination.
> If not the hardware (stick and port) makes itself a suspect by failing
> in normal operation, then i’d bet on the firmware's settings.

I’m pretty sure that everything I’ve been seeing can be explained if we assume 
the minnowboard firmware has trouble recognizing a USB3.0 device as bootable.


> -
> 
>> Thomas, If you would like to follow up on the inability of the installer to
>> recognize itself as a suitable installation CD
> 
> I believe to understand that problem. Let's see whether debian-cd will
> find a time slot to discuss it and whether trying the devices found by
> "list-devices disk" (/dev/sda, /dev/sdb, /dev/sdc on my machine) is
> harmless enough.
> 
> It would be interesting to study the history of cdrom-detect.postinst
> (package "cdrom-detect", 1600+ commits in
> https://salsa.debian.org/installer-team/cdrom-detect/commits/master).
> cdrom-detect/blame brings me to some forth and back, which probably
> does not tell the reason for the decision not to look for "disk".
> 
> I'd need a comfortable method to look at the revison _before_ a commit
> in order to get the next older blaming.
> Or a method to see only commits which affect the interesting code parts
>   devices="$(list-devices cd; list-devices maybe-usb-floppy)"
> and
>   devices="$(list-devices usb-partition)"
>   ...
>if try_mount $device $CDFS; then
> 
> Any GitLab experts here who could help me navigate ?
> Or git experts who can augment my knowledge beyond "clone", "add",
> "commit", and "push", so that i can inspect a clone locally ?
> 
> 
> Have a nice day :)
> 
> Thomas
> 

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

On Jun 3, 2018, at 12:45 AM, deloptes  wrote:

> Rick Thomas wrote:
> 
>> So, I was beginning to wonder if I were going crazy.  In any case, I tried
>> part (a) with “firmware-buster-DI-alpha2-amd64-DVD-1.iso”.  And guess
>> what!  That worked too, just the same as the other two.
> 
> I am not surprised by anything nowdays, but I was wondering if you switched
> off the device, or just rebooted. 
> 
> regards

I actually tried it both ways.

In one case, I halted the minnowboard but left it plugged in, then I hit the 
“reset” button and it acted as I described — found the stick and was willing to 
boot from it.

In the other case, I halted the minnowboard, then unplugged it from the power 
brick, then reconnected it to the power brick. same results as above.

I’m not sure what it means, but that’s what I saw.

Rick

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Thomas Schmitt]

Hi,

Rick Thomas wrote:
> Instead, I used
>firmware-9.4.0-amd64-netinst.iso
> to avoid any possible alpha/testing anomalies.

Normally i'd say that there is no decisive difference to expect.
In any case copying a "netinst" ISO is much faster than a "DVD-1".


> When I tried part (a) — just a simple dd — with
> debian-9.4.0-amd64-netinst.iso, the minnowboard immediately recognized it as
> bootable and presented it as part of the EFI menu labeled “EFI USB Device”.

Normally this difference should not appear. Very strange.

But we can now classify the demand for GPT on
  https://minnowboard.org/tutorials/best-practice-boot-media-selection
as spin-off of the usual rumor that EFI needs GPT.


> it booted without incident and ran the installer.
> When it came to the part where it looks for a CD, it easily found the USB
> stick

This was expectable. As long as the ISO is marked as partition, the
script in the initrd should find the device.


> tried
> part (a) with “firmware-buster-DI-alpha2-amd64-DVD-1.iso”.  And guess what!
> That worked too, just the same as the other two.

Hrmpf. At least above "normally"s are not generally put in doubt.
It would be another riddle if one ISO would fail reproducibly while
the other succedds reproducibly.

Maybe you changed a setting in the firwmware ?


> or there’s something flakey about the USB ports on this device

But why did the manual start of “fs0:\efi\boot\bootx64.efi” ran the
boot loader, loaded the kernel, and came far enough to run into the
software shortcomming of device detection ?


> In any case, I apologize for all the noise.

I doubt that this was a hallucination.
If not the hardware (stick and port) makes itself a suspect by failing
in normal operation, then i'd bet on the firmware's settings.

-

> Thomas, If you would like to follow up on the inability of the installer to
> recognize itself as a suitable installation CD

I believe to understand that problem. Let's see whether debian-cd will
find a time slot to discuss it and whether trying the devices found by
"list-devices disk" (/dev/sda, /dev/sdb, /dev/sdc on my machine) is
harmless enough.

It would be interesting to study the history of cdrom-detect.postinst
(package "cdrom-detect", 1600+ commits in
https://salsa.debian.org/installer-team/cdrom-detect/commits/master).
cdrom-detect/blame brings me to some forth and back, which probably
does not tell the reason for the decision not to look for "disk".

I'd need a comfortable method to look at the revison _before_ a commit
in order to get the next older blaming.
Or a method to see only commits which affect the interesting code parts
   devices="$(list-devices cd; list-devices maybe-usb-floppy)"
and
   devices="$(list-devices usb-partition)"
   ...
if try_mount $device $CDFS; then

Any GitLab experts here who could help me navigate ?
Or git experts who can augment my knowledge beyond "clone", "add",
"commit", and "push", so that i can inspect a clone locally ?


Have a nice day :)

Thomas

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[deloptes]

Rick Thomas wrote:

> So, I was beginning to wonder if I were going crazy.  In any case, I tried
> part (a) with “firmware-buster-DI-alpha2-amd64-DVD-1.iso”.  And guess
> what!  That worked too, just the same as the other two.

I am not surprised by anything nowdays, but I was wondering if you switched
off the device, or just rebooted. 

regards

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

Well… I have some results.  Just not the kind I was expecting!  (see below)

On Jun 2, 2018, at 5:15 PM, Rick Thomas  wrote:

> I have a confession to make…
> 
> On Jun 2, 2018, at 6:09 AM, Thomas Schmitt  wrote:
> 
>> Rick Thomas reported that the boot process of
>> firmware-buster-DI-alpha2-amd64-DVD-1.iso
>> fails with
>> Incorrect CD-ROM detected
>> after it was re-partitioned to GPT.
> 
> On the advice of another comment in this thread, I didn’t use
>firmware-buster-DI-alpha2-amd64-DVD-1.iso
> for those experiments.  Instead, I used
>firmware-9.4.0-amd64-netinst.iso
> to avoid any possible alpha/testing anomalies.
> 
> I neglected to say that in my earlier report.  I’m sorry for any confusion it 
> has caused!
> 
> 
> This afternoon, I plan to make a comprehensive set of tests: 
>   (a) “as-is” no messing with the partition tables at all; just dd directly 
> to the USB stick.
>   (b) an automatically gdisk-built GPT table deleting partition 1 and leaving 
> partition 2 unmodified except for setting as type EF00.
>   (c) an automatically gdisk-built GPT table based upon the original MBR 
> table deleting partition 1 and leaving partition 2 unmodified except for 
> setting as type EF00.
>   (d) a hand-built (using disk) GPT table with partition 1 having the same 
> start/end as the partition 2 had in the original GPT table and type EF00, and 
> no partition 2.
> 
> I plan to start with
>debian-9.4.0-amd64-netinst.iso
> because it’s the smallest (to minimizing time spend dd-ing) and has the 
> fewest non-fully-supported features to confuse issues.  In particular, this 
> is not the firmware-9.4 I used above.
> 
> I’ll report back as soon as I have some results.
> 
> Thanks for all the help!
> Rick

“Curiouser and curiouser!” cried Alice.

When I tried part (a) — just a simple dd — with debian-9.4.0-amd64-netinst.iso, 
the minnowboard immediately recognized it as bootable and presented it as part 
of the EFI menu labeled “EFI USB Device”.  When I chose that option, it booted 
without incident and ran the installer.  When it came to the part where it 
looks for a CD, it easily found the USB stick (no intervention on my part 
required) and proceeded to read modules from it.  I let it go til it got to the 
partitioning phase.  I stopped it there.

So, I moved on to try the same thing (just part (a)) with 
“firmware-9.4.0-amd64-netinst.iso”.  With the same results — it gave me the EFI 
menu offering the USB stick. When I chose that, it proceeded to install, 
finding and reading the “CD” as usual.

So, I was beginning to wonder if I were going crazy.  In any case, I tried part 
(a) with “firmware-buster-DI-alpha2-amd64-DVD-1.iso”.  And guess what!  That 
worked too, just the same as the other two.

Conclusion:  Either I’m imagining things, or there’s something flakey about the 
USB ports on this device that sometimes causes a USB stick to NOT be recognized 
as a bootable device.

Thomas, If you would like to follow up on the inability of the installer to 
recognize itself as a suitable installation CD after the partition table has 
been modified, please let me know and I’ll be happy to help any way I can.

In any case, I apologize for all the noise.

Rick

PS: One other item that may shed light — the above experiments were performed 
using an 8GB Generic USB2.0 flash drive purchased thru NewEgg direct from a 
factory in China.  On the suspicion that there might be something strange about 
the USB ports on the minnowboard, tried it with a name-brand 16GB USB3.0 flash 
drive from Kingston DataTraveler.

The minnowboard has two USB ports.  The “upper” port is USB2.0 and is usually 
used for a keyboard/mouse combo.  The “lower” port is USB3.0 and recommended 
for attaching mass storage devices, such as the Debian installer.

When I plugged the USB3.0 DataTraveler into the lower (USB3.0) slot, it was NOT 
recognized as bootable.  So I swapped the plugs, plugging the keyboard/mouse 
into the lower (USB3.0) slot, and the DataTraveler into the upper (USB2.0) 
slot.  Guess what!  This time it WAS recognized as a bootable device!

So it appears that the minnowboard firmware (or USB hardware?) has trouble 
recognizing a USB3.0 device as bootable???  Something to take up with the 
Netgate minnowboard support folks, I guess.

Enjoy!

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

I have a confession to make…

On Jun 2, 2018, at 6:09 AM, Thomas Schmitt  wrote:

> Rick Thomas reported that the boot process of
>  firmware-buster-DI-alpha2-amd64-DVD-1.iso
> fails with
>  Incorrect CD-ROM detected
> after it was re-partitioned to GPT.

On the advice of another comment in this thread, I didn’t use
firmware-buster-DI-alpha2-amd64-DVD-1.iso
for those experiments.  Instead, I used
firmware-9.4.0-amd64-netinst.iso
to avoid any possible alpha/testing anomalies.

I neglected to say that in my earlier report.  I’m sorry for any confusion it 
has caused!


This afternoon, I plan to make a comprehensive set of tests: 
   (a) “as-is” no messing with the partition tables at all; just dd directly to 
the USB stick.
   (b) an automatically gdisk-built GPT table deleting partition 1 and leaving 
partition 2 unmodified except for setting as type EF00.
   (c) an automatically gdisk-built GPT table based upon the original MBR table 
deleting partition 1 and leaving partition 2 unmodified except for setting as 
type EF00.
   (d) a hand-built (using disk) GPT table with partition 1 having the same 
start/end as the partition 2 had in the original GPT table and type EF00, and 
no partition 2.

I plan to start with
debian-9.4.0-amd64-netinst.iso
because it’s the smallest (to minimizing time spend dd-ing) and has the fewest 
non-fully-supported features to confuse issues.  In particular, this is not the 
firmware-9.4 I used above.

I’ll report back as soon as I have some results.

Thanks for all the help!
Rick

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Thomas Schmitt]

Hi,

i created an ISO with this partition table:

  $ /sbin/gdisk -l test.iso
  ...
MBR: protective
  ...
GPT: present
  ...
  Number  Start (sector)End (sector)  Size   Code  Name
 1  64 7713831   3.7 GiB 0700  ISO9660
 2 7713832 7714663   416.0 KiB   EF00  Appended2

by

  mnt=/mnt/iso

  sudo mount firmware-buster-DI-alpha2-amd64-DVD-1.iso "$mnt"

  xorriso -as mkisofs \
-o test.iso \
-r \
-V 'Debian buster-DI-a2 amd64 1' \
-partition_offset 16 \
-append_partition 2 0xef "$mnt"/boot/grub/efi.img \
-appended_part_as_gpt \
"$mnt"

It will not work from DVD and not in BIOS Legacy mode. (On MS-Windows it
will not show long file names. I avoided complaints about symlinks by
avoiding option -J.)

But if started as reported by Rick Thomas:
> [...] by doing some command-line stuff in the EFI
> firmware — telling it to boot the file “fs0:\efi\boot\bootx64.efi”
then there should be a mountable Debian ISO in partition 1.
So the initrd stuff should get over the error occasion "Incorrect CD-ROM
detected".

(I meanwhile expressed my view about the ISO detection code in
  https://lists.debian.org/debian-cd/2018/06/msg0.html
)


Have a nice day :)

Thomas

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Thomas Schmitt]

Hi,

Rick Thomas reported that the boot process of
  firmware-buster-DI-alpha2-amd64-DVD-1.iso
fails with
  Incorrect CD-ROM detected
after it was re-partitioned to GPT.

It seems that the software in the initrd of a Debian isohybrid wants a
mountable partition with the ISO 9660 filesystem and file /.disk/info,
if the ISO is presented on USB stick.

But after replacing the MBR partition table by the GPT, we do not have a
mountable ISO 9660 partition.
And we cannot make one that fits, because no GPT partition may start
at block 0. And even if we force this start, the ISO partition would
overlap with the EFI partition. This is explicitely forbidden in UEFI
specs.

I will think about possibilities to repack the ISO image so that it is
representable as GPT partition and that the EFI partition sits outside
the ISO partition.


Rick, did you already try whether you can force your EFI to start with
the unchanged, MBR partitioned  firmware-buster-DI-alpha2-amd64-DVD-1.iso ?
Currently it seems that only this partition table has hope for success
with the later stages of booting in that ISO.


---
Reasoning:

The initrd works with isohybrid USB stick more or less by accident.
Not the isohybrid device is enumerated and inspected, but its partitions.
This does not really match the idea of the device on which the isohybrid
is to be presented to the computer. (It should look in partitions for FAT
and on base devices for ISO 9660.)

Confused by this, i wrote about the software in the initrd of
firmware-buster-DI-alpha2-amd64-DVD-1.iso:

>  var/lib/dpkg/info/cdrom-detect.postinst :
>   devices="$(list-devices cd; list-devices maybe-usb-floppy)"
> [...]
> either i misunderstand the scripts or script execution should not get
> there under normal circumstances.

Script bin/list-devices on my running system yields device paths of
the USB stick with the unchanged, MBR based firmware-*-DVD-1.iso

  $ bin/list-devices usb-partition
  /dev/sdc1
  /dev/sdc2

In cdrom-detect.postinst i read:

devices="$(list-devices usb-partition)"
for device in $devices; do
if try_mount $device $CDFS; then
db_set cdrom-detect/hybrid true
break 2
fi
if try_mount $device $FATFS; then
db_set cdrom-detect/usb-hdd true
break 2
fi
done

In function try_mount() of cdrom-detect.postinst :

  try_mount() {
local device=$1
local type=$2

local ret=1
if mount -t $type -o $OPTIONS $device /cdrom; then

which i replay successfully by

  # mount -t iso9660 -o ro,exec /dev/sdc1 /cdrom
  #

(Directory /cdrom gets created by cdrom-detect.postinst in line 96
  mkdir /cdrom 2>/dev/null || true
)

Now there should be log messages. Either
  CD-ROM mount succeeded: device=/dev/sdc1 fstype=iso9660
or
  CD-ROM mount failed: device=/dev/sdc1 fstype=iso9660

In case of success, the /cdrom/.disk/info decides whether Debian ISO
or not:

   if [ -e /cdrom/.disk/info ]; then
CDNAME=$(cat /cdrom/.disk/info)
log "Detected CD '$CDNAME'"

In my test it exists and the log should say

  Detected CD 'Debian GNU/Linux buster-DI-alpha2 "Buster" - Official Snapshot 
amd64 DVD Binary-1 20171205-15:32'

In case of failure there would be:

  The CD in /dev/sdc1 is not a Debian CD!

But because the GPT does not have a mountable partition with the ISO,
and the EFI partition was mountable as FAT, we get the slightly misleading
message about an unsuitable CDROM.

This message can be emitted with isohybrid USB sticks and with "usb-hdd"
sticks (which i assume are the ones with the interesting FAT filesystems).

---

Have a nice day :)

Thomas

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Thomas Schmitt]

Hi,

Rick Thomas wrote on Sun, 27 May 2018 03:28:20 -0700
> > > specifically, firmware-buster-DI-alpha2-amd64-DVD-1.iso

i wrote:
> > Point 4 of that tutorial says:
> >   "NOTE: If your USB device is not listed in the Boot Manager list,
> >confirm that it’s GPT formatted and contains a valid EFI boot
> >partition."
> > [...]
> > /sbin/gdisk test.iso

Rick Thomas wrote:
> The first problem was that the stick still wasn’t recognized as bootable.
> I was able to work around that by doing some command-line stuff in the EFI
> firmware — telling it to boot the file “fs0:\efi\boot\bootx64.efi”
> That got me into the installer, but after it did the language/locale
> picking, when the installer went looking for the CD, it complained

Would you get the same result with an unmodified ISO and that EFI
command line ?
(I.e. did the chnage from MBR to GPT do anything good ?)


> Incorrect CD-ROM detected
> The CD-ROM drive contains a CD which cannot be used for installation…

Let's see:
  wget 
https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/buster_di_alpha2/amd64/iso-dvd/firmware-buster-DI-alpha2-amd64-DVD-1.iso

The message can be found in the initrds for text and graphical install
  /install.amd/initrd.gz
  /install.amd/gtk/initrd.gz
I get from var/lib/dpkg/info/cdrom-detect.templates :
  Template: cdrom-detect/wrong-cd)
to var/lib/dpkg/info/cdrom-detect.postinst :
  devices="$(list-devices cd; list-devices maybe-usb-floppy)"
to bin/list-devices :
  for x in /sys/block/*; do
  ...
  syspaths="${syspaths:+$syspaths }$x"
  ...
  if ! $match && [ "$TYPE" = cd ]; then
  if device_info env "$devpath" | grep -q '^ID_CDROM='; then
  match=:

The function device_info is implemented alternatively

  if type udevadm >/dev/null 2>&1; then
  device_info () {
  udevadm info -q "$1" -p "$2" 2>/dev/null
  }
  elif type udevinfo >/dev/null 2>&1; then
  device_info () {
  udevinfo -q "$1" -p "$2" 2>/dev/null
  }

The initrd contains a file
  bin/udevadm
but no "udevinfo".
At least on my elderly system and with USB stick /dev/sdd it does not
report any line "ID_CDROM=" with

  bin/udevadm info -q env -p /sys/block/sdd
  
It does report quite significant things like
  ID_FS_BOOT_SYSTEM_ID=EL\x20TORITO\x20SPECIFICATION
  ID_FS_LABEL=Debian_buster-DI-a2_amd64_1
  ...
  ID_FS_TYPE=iso9660

I do get
  ID_CDROM=1
from a Blu-ray drive at /dev/sr4.

But if i compare cdrom-detect.postinst and list-devices with those in the
initrd of debian-9.3.0-amd64-netinst.iso then i cannot find any difference.

"netinst" is known to work from USB stick.
So either i misunderstand the scripts or script execution should not get
there under normal circumstances.


> Do you think it would work if I used disk to create a GPT that had partition
> 1 of they “EF00” start and end where the original partition 2 started and
> ended?

You may try. But if this brings success, then the EFI firmware is really
odd.

It may well be that EFI's inability to offer the bootable partition
is independent of the debian-installer's inability to find the USB stick
as pseudo-CDROM.


Have a nice day :)

Thomas

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

On May 27, 2018, at 5:38 AM, Thomas Schmitt  wrote:

> Hi,
> 
> i typo'ed:
>> deletion of GPT partition 2 because it overlaps
>> with the EFI partition,
> 
> Partition 1 needs to be delted. Number 2 is the EFI partition.
> 
> Further it comes to me that the gdisk session is best done one the
> USB stick with the ISO already copied onto it. This will give gdisk
> a chance to install the backup GPT at the real end of the device.
> 
> If re-usal of the nearly-GPT in the ISO yields no success, try to
> let gdisk convert the MBR partition table to GPT. For that you will
> have to copy again the original ISO onto USB stick, because the original
> MBR partition table did not survive gdisk's previous conversion effort.
> 
> 
> Have a nice day :)
> 
> Thomas

Thank you Thomas, for your detailed explanation and test procedure!

I tried your procedure (dd the iso to a stick, then use disk to modify the 
stick’s GPT).  I actually tried it twice: Once using the original GPT, second 
using the original MBR to reconstruct the GPT.
Unfortunately, it didn’t work.

The first problem was that the stick still wasn’t recognized as bootable.
I was able to work around that by doing some command-line stuff in the EFI 
firmware — telling it to boot the file “fs0:\efi\boot\bootx64.efi”
That got me into the installer, but after it did the language/locale picking, 
when the installer went looking for the CD, it complained that 
Incorrect CD-ROM detected
The CD-ROM drive contains a CD which cannot be used for installation…

Do you think it would work if I used disk to create a GPT that had partition 1 
of they “EF00” start and end where the original partition 2 started and ended?

Thanks for your help so far!
Rick

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Pascal Hambourg]

Le 27/05/2018 à 12:50, deloptes a écrit :


I have found out that not all usb drives boot well - don't know why on some
machines some drives does not boot - same image on another drive boot.
For example sandisk 4GB do not and Kingston 1GB do on same machine with same
image


I have observed a similar behaviour on one specific PC (rather old, 2005 
or so). I guess it is machine/firmware-dependent rather than USB 
drive-dependent.


Even stranger : it would only happen on a cold boot. On a warm reboot 
after running any Linux kernel, the USB drive would boot just fine. USB 
controller initialization issue, I guess.

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[deloptes]

Rick Thomas wrote:

> 
> On May 27, 2018, at 3:57 AM, deloptes  wrote:
> 
>> Rick Thomas wrote:
>> 
>>> Can you suggest any way to get around the problem?
>> 
>> Sorry to jump in, but I had a similar experience with Pi few years ago.
>> 
>> I see this board supports network boot - for experimenting it is the one
>> I usually prefer. I setup tftp/dhcp server long time ago for other
>> projects and just add the image I want to boot there (for Pi arm).
>> Advantage is I can do simple install in chroot or for arm with qemu
>> (debootstrap) and setup the dhcp so that the client picks this up.
>> This saved a lot of time when playing with Pi.
> 
> Jumping in with useful suggestions is *always* welcome!
> 
> Do you have a pointer to a step-by-step tutorial on how to set up a tftp
> server and provision it with the necessary files for booting the debian
> installer?
> 

Isn't your google not working today?

https://wiki.debian.org/PXEBootInstall

https://help.ubuntu.com/community/DisklessUbuntuHowto
http://debianaddict.com/2012/06/19/diskless-debian-linux-booting-via-dhcppxenfstftp/

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Thomas Schmitt]

Hi,

i typo'ed:
> deletion of GPT partition 2 because it overlaps
> with the EFI partition,

Partition 1 needs to be delted. Number 2 is the EFI partition.

Further it comes to me that the gdisk session is best done one the
USB stick with the ISO already copied onto it. This will give gdisk
a chance to install the backup GPT at the real end of the device.

If re-usal of the nearly-GPT in the ISO yields no success, try to
let gdisk convert the MBR partition table to GPT. For that you will
have to copy again the original ISO onto USB stick, because the original
MBR partition table did not survive gdisk's previous conversion effort.


Have a nice day :)

Thomas

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Thomas Schmitt]

Hi,

didier gaumet wrote:
> > https://minnowboard.org/tutorials/best-practice-boot-media-selection

Rick Thomas wrote:
> [...] when I dd that same .iso
> onto a USB stick, then plug the stick into the USB slot on the Turbot, will
> not be recognized as a bootable medium.

Point 4 of that tutorial says:

  "NOTE: If your USB device is not listed in the Boot Manager list,
   confirm that it’s GPT formatted and contains a valid EFI boot
   partition."

The Debian ISOs are not recognizably GPT formatted, although they contain
nearly all the bytes of a valid GPT. Their MBR partition table is not
"protective" to indicate the presence of a GPT.

Needed would be a MBR partition table with only one partition of type 0xEE
and no other partitions, deletion of GPT partition 2 because it overlaps
with the EFI partition, and the specs compliant Type GUID for the EFI
partition.


Rick Thomas wrote:
> > > It seems odd that the exact same bits should be seen as bootable when
> > > presented on a DVD, but not when presented on a USB stick…

Pascal Hambourg wrote:
> > The expected boot structures on a DVD and a
> > USB stick (or floppy, hard disk...) are completely different.

> So the UEFI firmware is more forgiving of a partition table that is “invalid
> from a strict point of view” when it’s coming from a physical DVD, than it
> is when it’s coming from a USB stick?

No. A standards compliant firmware ignores any partition table when the
medium is CD, DVD, or BD. It then rather interprets the El Torito boot
record and the boot catalog to find a suitable boot image. In case of EFI
the selected image is then used as EFI System Partition with FAT filesystem
and startup program.

The Debian ISOs for amd64 mark their EFI System Partition by MBR partition
table and by El Torito.


> Can you suggest any way to get around the problem?

Repacking the ISO by grub-mkrescue would produce a GPT.
But the GRUB part of that job is out of my scope.


As said, Debian ISOs are not very far from a real GPT.

Normally i would not trust any partition editor not to go mad on the job.
But changing stuff by hand in GPT means re-computing the CPT checksum.
This would become tricky.

So lets try gdisk. Regrettably partition 1 of the nearly-GPT is ill
beyond repair (at least by gdisk's means). So it has to be deleted.
To our luck it should not be expected by any part of the booting system.

  $ cp debian-9.3.0-amd64-netinst.iso test.iso
  $ /sbin/gdisk test.iso
  GPT fdisk (gdisk) version 0.8.10

  Partition table scan:
MBR: MBR only
BSD: not present
APM: not present
GPT: present

  Found valid MBR and GPT. Which do you want to use?
   1 - MBR
   2 - GPT
   3 - Create blank GPT

  Your answer: 2
  Using GPT and creating fresh protective MBR.
  Warning! Main partition table overlaps the first partition by 64 blocks!
  You will need to delete this partition or resize it in another utility.

  Command (? for help): d
  Partition number (1-2): 1

  Command (? for help): t
  Using 2
  Current type is 'Microsoft basic data'
  Hex code or GUID (L to show codes, Enter = 8300): ef00
  Changed type of partition to 'EFI System'

  Command (? for help): p
  Disk test.iso: 593920 sectors, 290.0 MiB
  Logical sector size: 512 bytes
  Disk identifier (GUID): 52A61562-549F-43BB-805F-66F9583C139F
  Partition table holds up to 208 entries
  First usable sector is 64, last usable sector is 593866
  Partitions will be aligned on 16-sector boundaries
  Total free space is 592971 sectors (289.5 MiB)

  Number  Start (sector)End (sector)  Size   Code  Name
 237604591   416.0 KiB   EF00  ISOHybrid1

  Command (? for help): w

  Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
  PARTITIONS!!

  Do you want to proceed? (Y/N): y
  OK; writing new GUID partition table (GPT) to test.iso.
  Warning: The kernel is still using the old partition table.
  The new table will be used at the next reboot.
  The operation has completed successfully.
  $

Now the ISO should have a GPT which marks the EFI System Partition.
Put it onto the USB stick and see what happens.

--

If this does not work, we could try to erase all partition tables and
set up a new one from info which we learned from the old one.


Have a nice day :)

Thomas

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[didier gaumet]

Le 27/05/2018 à 12:28, Rick Thomas a écrit :
> Thanks!  Yes, that confirmed what I had mostly already found by 
> experimentation.
> 
> What it didn’t explain was why the same .iso (specifically, 
> firmware-buster-DI-alpha2-amd64-DVD-1.iso) when I burn it to a DVD-R and load 
> the disk into a DVD drive that is plugged into the USB slot on the Turbot, 
> will boot into the installer just fine… But when I dd that same .iso onto a 
> USB stick, then plug the stick into the USB slot on the Turbot, will not be 
> recognized as a bootable medium.

- your installer seems an alpha, I would first try with a more robust
one (Stretch)...
- Does your USB stick only fail to boot automatically or does it also
fail to boot when you select it as the boot medium in the UEFI boot
option menu?

> I’m puzzled…  Have you tried to boot the installer from a USB stick on a 
> Turbot?  Or did you always use a physical DVD drive?  What’s the difference 
> between them that causes one to boot and the other to be ignored?

I am sorry if I gave you the false impression that I own or have used
this hardware: that is not the case...

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

On May 27, 2018, at 3:57 AM, deloptes  wrote:

> Rick Thomas wrote:
> 
>> Can you suggest any way to get around the problem?
> 
> Sorry to jump in, but I had a similar experience with Pi few years ago.
> 
> I see this board supports network boot - for experimenting it is the one I
> usually prefer. I setup tftp/dhcp server long time ago for other projects
> and just add the image I want to boot there (for Pi arm).
> Advantage is I can do simple install in chroot or for arm with qemu
> (debootstrap) and setup the dhcp so that the client picks this up. 
> This saved a lot of time when playing with Pi.

Jumping in with useful suggestions is *always* welcome!

Do you have a pointer to a step-by-step tutorial on how to set up a tftp server 
and provision it with the necessary files for booting the debian installer?

I’ve got a couple of dhcp servers running dnsmasq.  I seem to recall that 
dnsmasq is capable of serving tftp, but I don’t remember where I read about it.

Enjoy!
Rick 

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Brian]

On Sun 27 May 2018 at 03:34:40 -0700, Rick Thomas wrote:

> 
> On May 27, 2018, at 3:12 AM, Pascal Hambourg  wrote:
> 
> > Le 27/05/2018 à 00:19, Rick Thomas a écrit :
> >> Does anyone have experience installing Debian on a Minnowboard Turbot?
> > 
> > Not me.
> > 
> >> I have a dual-core, single ethernet, Turbot.  I’m able to install Debian 
> >> (Buster) from a physical DVD, but when I dd the installer .iso to a USB 
> >> stick, and try to install from the stick, the Turbot’s firmware doesn’t 
> >> see the stick as a bootable medium.
> >> If I install (using the physical DVD in a USB connected DVD drive) to a 
> >> USB stick, that USB stick (the one I installed to) will boot fine.  So 
> >> it’s not that it just can’t boot from a USB stick.
> >> The documentation isn’t entirely clear, but I *think* the Turbot’s boot 
> >> firmware is UEFI-only.  Would that make a difference?
> > 
> > Probably not, if the same image boots from a DVD.
> > 
> >> It seems odd that the exact same bits should be seen as bootable when 
> >> presented on a DVD, but not when presented on a USB stick…
> > 
> > No, it is perfectly normal. The expected boot structures on a DVD and a USB 
> > stick (or floppy, hard disk...) are completely different.
> > The Debian installer ISO images are called "hybrid" because they combine a 
> > DVD layout and a regular hard disk (or USB stick). But this comes at a 
> > price : the partition table layout could be considered invalid on a strict 
> > point of view.
> 
> Thanks, Pascal!  
> 
> So the UEFI firmware is more forgiving of a partition table that is “invalid 
> from a strict point of view” when it’s coming from a physical DVD, than it is 
> when it’s coming from a USB stick?
> 
> That would explain what I’m seeing.  Can you suggest any way to get around 
> the problem?

Successful booting from a USB drive:

 https://01.org/developerjourney/installing-ubuntu-minnowboard-max

Successful booting from a SD card:

 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=856178

-- 
Brian.

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

On May 27, 2018, at 3:50 AM, deloptes  wrote:

> Rick Thomas wrote:
> 
>> I’m puzzled…  Have you tried to boot the installer from a USB stick on a
>> Turbot?  Or did you always use a physical DVD drive?  What’s the
>> difference between them that causes one to boot and the other to be
>> ignored?
> 
> I have found out that not all usb drives boot well - don't know why on some
> machines some drives does not boot - same image on another drive boot.
> For example sandisk 4GB do not and Kingston 1GB do on same machine with same
> image

Very interesting… I’ll try a few different USB sticks and see if I can get one 
to boot.
Rick

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[deloptes]

Rick Thomas wrote:

> Can you suggest any way to get around the problem?

Sorry to jump in, but I had a similar experience with Pi few years ago.

I see this board supports network boot - for experimenting it is the one I
usually prefer. I setup tftp/dhcp server long time ago for other projects
and just add the image I want to boot there (for Pi arm).
Advantage is I can do simple install in chroot or for arm with qemu
(debootstrap) and setup the dhcp so that the client picks this up. 
This saved a lot of time when playing with Pi.

regards

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[deloptes]

Rick Thomas wrote:

> I’m puzzled…  Have you tried to boot the installer from a USB stick on a
> Turbot?  Or did you always use a physical DVD drive?  What’s the
> difference between them that causes one to boot and the other to be
> ignored?

I have found out that not all usb drives boot well - don't know why on some
machines some drives does not boot - same image on another drive boot.
For example sandisk 4GB do not and Kingston 1GB do on same machine with same
image

regards

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

On May 27, 2018, at 3:12 AM, Pascal Hambourg  wrote:

> Le 27/05/2018 à 00:19, Rick Thomas a écrit :
>> Does anyone have experience installing Debian on a Minnowboard Turbot?
> 
> Not me.
> 
>> I have a dual-core, single ethernet, Turbot.  I’m able to install Debian 
>> (Buster) from a physical DVD, but when I dd the installer .iso to a USB 
>> stick, and try to install from the stick, the Turbot’s firmware doesn’t see 
>> the stick as a bootable medium.
>> If I install (using the physical DVD in a USB connected DVD drive) to a USB 
>> stick, that USB stick (the one I installed to) will boot fine.  So it’s not 
>> that it just can’t boot from a USB stick.
>> The documentation isn’t entirely clear, but I *think* the Turbot’s boot 
>> firmware is UEFI-only.  Would that make a difference?
> 
> Probably not, if the same image boots from a DVD.
> 
>> It seems odd that the exact same bits should be seen as bootable when 
>> presented on a DVD, but not when presented on a USB stick…
> 
> No, it is perfectly normal. The expected boot structures on a DVD and a USB 
> stick (or floppy, hard disk...) are completely different.
> The Debian installer ISO images are called "hybrid" because they combine a 
> DVD layout and a regular hard disk (or USB stick). But this comes at a price 
> : the partition table layout could be considered invalid on a strict point of 
> view.

Thanks, Pascal!  

So the UEFI firmware is more forgiving of a partition table that is “invalid 
from a strict point of view” when it’s coming from a physical DVD, than it is 
when it’s coming from a USB stick?

That would explain what I’m seeing.  Can you suggest any way to get around the 
problem?

Enjoy!
Rick

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Rick Thomas]

Thanks!  Yes, that confirmed what I had mostly already found by experimentation.

What it didn’t explain was why the same .iso (specifically, 
firmware-buster-DI-alpha2-amd64-DVD-1.iso) when I burn it to a DVD-R and load 
the disk into a DVD drive that is plugged into the USB slot on the Turbot, will 
boot into the installer just fine… But when I dd that same .iso onto a USB 
stick, then plug the stick into the USB slot on the Turbot, will not be 
recognized as a bootable medium.

I’m puzzled…  Have you tried to boot the installer from a USB stick on a 
Turbot?  Or did you always use a physical DVD drive?  What’s the difference 
between them that causes one to boot and the other to be ignored?

Rick


On May 26, 2018, at 11:59 PM, didier gaumet  wrote:

> 
> maybe this will help:
> https://minnowboard.org/tutorials/best-practice-boot-media-selection
> 

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[Pascal Hambourg]

Le 27/05/2018 à 00:19, Rick Thomas a écrit :

Does anyone have experience installing Debian on a Minnowboard Turbot?


Not me.


I have a dual-core, single ethernet, Turbot.  I’m able to install Debian 
(Buster) from a physical DVD, but when I dd the installer .iso to a USB stick, 
and try to install from the stick, the Turbot’s firmware doesn’t see the stick 
as a bootable medium.

If I install (using the physical DVD in a USB connected DVD drive) to a USB 
stick, that USB stick (the one I installed to) will boot fine.  So it’s not 
that it just can’t boot from a USB stick.

The documentation isn’t entirely clear, but I *think* the Turbot’s boot 
firmware is UEFI-only.  Would that make a difference?


Probably not, if the same image boots from a DVD.


It seems odd that the exact same bits should be seen as bootable when presented 
on a DVD, but not when presented on a USB stick…


No, it is perfectly normal. The expected boot structures on a DVD and a 
USB stick (or floppy, hard disk...) are completely different.
The Debian installer ISO images are called "hybrid" because they combine 
a DVD layout and a regular hard disk (or USB stick). But this comes at a 
price : the partition table layout could be considered invalid on a 
strict point of view.

Re: Installing Debian on a Minnowboard Turbot with installer on USB stick?

[didier gaumet]

maybe this will help:
 https://minnowboard.org/tutorials/best-practice-boot-media-selection