Re: Providing (armhf) u-boot images together with d-i images?
Karsten Merker mer...@debian.org (2015-03-09): On Sun, Mar 08, 2015 at 04:07:35PM +0100, Cyril Brulebois wrote: Ian Campbell i...@debian.org (2015-01-07): On Fri, 2015-01-02 at 16:40 +0100, Cyril Brulebois wrote: Karsten Merker mer...@debian.org (2015-01-02): (Do your patches end up adding the correct Built-Using on u-boot?) No, they don't, but d-i does not do that for similar components on other platforms (syslinux/isolinux/grub) as well. Probably we should do that, but then it would have to be done for all platforms. Kibi? ISTR having proposed a patch to #700026 / #696418, but that wasn't commented upon. FWIW the patch in #700026 looks good to me as a starting point... OK, thanks. Pushed now: http://anonscm.debian.org/cgit/d-i/debian-installer.git/diff/?id=b4411bf Hello, unfortunately this does not work completely as intended - when building on a platform that does not use syslinux, such as armhf, it generates a built-using entry like syslinux (= ) (please see https://lists.debian.org/debian-boot/2015/02/msg00012.html for further information). Ah, I kind of missed that. Would have been nice to send that to the relevant bug reports, which I had a look at before sending the patch… Mraw, KiBi. signature.asc Description: Digital signature
Re: Providing (armhf) u-boot images together with d-i images?
Ian Campbell i...@debian.org (2015-01-07): On Fri, 2015-01-02 at 16:40 +0100, Cyril Brulebois wrote: Karsten Merker mer...@debian.org (2015-01-02): (Do your patches end up adding the correct Built-Using on u-boot?) No, they don't, but d-i does not do that for similar components on other platforms (syslinux/isolinux/grub) as well. Probably we should do that, but then it would have to be done for all platforms. Kibi? ISTR having proposed a patch to #700026 / #696418, but that wasn't commented upon. FWIW the patch in #700026 looks good to me as a starting point... OK, thanks. Pushed now: http://anonscm.debian.org/cgit/d-i/debian-installer.git/diff/?id=b4411bf Mraw, KiBi. signature.asc Description: Digital signature
Re: Providing (armhf) u-boot images together with d-i images?
On Fri, 2015-01-02 at 14:51 +0100, Karsten Merker wrote: On Fri, Jan 02, 2015 at 10:46:37AM +, Ian Campbell wrote: On Wed, 2014-12-31 at 01:59 +0100, Karsten Merker wrote: Talking about other compression schemes or providing multiple versions of the same file will just confuse users. It looks like someone has commented on the duplicated concat examples in v3 too. Ok, I'll change the wording for V4. My intention was to provide a generic README that works regardless of which compression options one chooses in the makefile, so that switching from .gz to .xz would be simple without having to think of updating documentation elsewhere. If I now change the README to explicitly mention only one compression type, on which one shall we settle for all installer images on armhf? Everything compressed by gzip or by xz? As usual, this is a trade-off between compression ratio, CPU cycles and memory requirements on the build hosts. I'd say go for gzip for maximum compatibility, e.g. for people making images from Windows. The previous patch produces hd-media builds for use with CD/DVD iso images (equivalent to the i386/amd64 hd-media boot.img.gz). This patch produces a netboot SD card (equivalent to the i386/amd64 netboot mini.iso), plus a netboot.tar.gz which contains everything needed to be put onto a tftp server for tftp-booting the installer (equivalent to the i386/amd64 PXE netboot.tar.gz). netboot.tar.gz sounds fine. What I don't get (see my reply to Vagrant) is what the practical difference to the user is between the hd-media builds and the netboot mini.iso ones, given that for ARM each is just an sd-card image. The netboot and hd-media versions contain different installer builds - the list of udebs in them and the order of the udebs being executed is different between the two (see http://d-i.alioth.debian.org/doc/internals/ch02.html). The netboot image has to have network access to work, it loads all additional udebs it needs and all regular packages from the network. The hd-media one is for installing from local media and can work completely offline. This distinction is made on all platforms supported by d-i and unifying those two does not seem to work (sorry, I do not remember the details; IIRC it had something to do with different and conflicting anna implementations for the two variants). Got it, thanks (I had a question in my reply to Vagrant, I won't repeat it here). For the tftpboot part, u-boot supports standard pxelinux.cfg syntax configuration files -- would it be better to just generate a suitable one of those? I have not in detail checked the pxelinux.cfg support in u-boot yet, but as we unfortunately have to build special-casing into the boot script for several platform (such as the i.MX6 console baudrate issue), I do not see how to implement that with the pxelinux.cfg syntax. Pointers welcome :). IMHO we should be moving towards using this way of doing things, which might mean fixing things like the i.MX6 issue at source. Too late for Jessie of course. BTW our kernel now supports the /chosen/stdout-path property in fdt and will automatically put a console on it. Just to make sure that I understand that correctly: with the u-boot and kernel versions currently in Jessie, there is no need to ever pass a console parameter to the kernel on any platform? Or is this something that was introduced after the u-boot v2014-10 release in Jessie? /chosen/stdout-path is more a function of the kernel and the dtb, I don't think u-boot is involved much at all, at it need not be. I suppose on platforms where there is a choice of consoles it might auto update the stdout-path to reflect where u-boot is outputting it's stuff, but you'll get some sort of sane default for the platform either way (assuming the dtb is sane). If the former, using pxelinux.cfg should indeed work everywhere; if the latter, we would have to keep using boot.scr for Jessie. I think it's the former. Inside the existing hd-media and netboot directories, a subdirectory SD-card-images gets created, which then contains the created images. These have the following sizes: option netboot full image: 18568 A10-OLinuXino-Lime.img.gz 18572 A20-OLinuXino-Lime.img.gz 18572 BananaPi.img.gz 18640 BeagleBoneBlack.img.gz 18572 Cubieboard2.img.gz 18568 Cubieboard.img.gz 18572 Cubietruck.img.gz 18564 MX53LOCO.img.gz 18568 MX6_Cubox-i.img.gz 18560 PandaBoard.img.gz 18552 Wandboard_Quad.img.gz So around 200M in total? I haven't checked but I would imagine that was an order of magnitude more than d-i produces for any other arch in the d-i builds. As I wrote in my previous mail, we could squash that down to 18M once + ~200kB per platform if we manage to unify the boot process (be it with an
Re: Providing (armhf) u-boot images together with d-i images?
On Fri, 2015-01-02 at 22:13 +0100, Karsten Merker wrote:
On Fri, Jan 02, 2015 at 02:51:18PM +0100, Karsten Merker wrote:
On Fri, Jan 02, 2015 at 10:46:37AM +, Ian Campbell wrote:
BTW our kernel now supports the /chosen/stdout-path property in fdt and
will automatically put a console on it.
Just to make sure that I understand that correctly: with the
u-boot and kernel versions currently in Jessie, there is no need
to ever pass a console parameter to the kernel on any platform?
Or is this something that was introduced after the u-boot v2014-10
release in Jessie?
Hello,
I have in the meantime run some tests and this functionality does
not seem to be available with the current versions in Jessie. If
I do not explicitly set bootargs to include a console parameter,
I get no console output, although u-boot has a valid ${console}.
The support is in linux 3.16.7-ckt2-1, which is in Jessie right now but
I don't know if it is in the kernel used by the current d-i, probably
not since that is in the process of being updated.
Even if we cannot use that functionality at the moment, I am
interested in how it works: does u-boot just copy the content of
${console} into the /chosen/stdout-path property, or is there
some additional mechanism to hand over a baudrate for serial
connections to the kernel?
I think it comes from the fdt, u-boot needn't be involved at all,
although it could be if the platform demanded it.
The stdout-path property is a device-tree path to the device to use, so
it is e.g. /smb/motherboard/iofpga@3,/uart@09 (or an
equivalent alias), it's not e.g. ttyS0 or ttyAMA0 etc.
Ian.
--
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/1420654688.11796.24.ca...@debian.org
Re: Providing (armhf) u-boot images together with d-i images?
On 2015-01-07, Ian Campbell wrote: On Fri, 2015-01-02 at 08:59 -0800, Vagrant Cascadian wrote: On 2015-01-02, Ian Campbell wrote: On Tue, 2014-12-30 at 13:42 -0800, Vagrant Cascadian wrote: On 2014-12-30, Ian Campbell wrote: But I still have a question about the hd-media sd-card images -- aren't they only useful in concert with something which adds the .iso etc to the image too? Yes, the hd-media images still need an .iso file on some other media, such as USB, SATA or an additional SD card. And therefore aren't they more usefully built along with that iso? Otherwise how do you know how big to make the partition to contain the iso at d-i build time given the varied sizes of .iso and sd-card you can buy? It would also be nice to have media with the .iso file built-in, but I don't think a requirement for it to be useful without that. I definitely see the case for moving some of the code into debian-cd, or another utility. I've considered making such a utility and shipping it in u-boot-tools, but didn't get around to it for jessie. The sd-card offsets in this patchset duplicate information in the various u-boot* README.Debian files, and it would make sense to move that into a script. This script could also generate SD images with a vmlinuz, initrd, bootscript, dtb, .iso, etc. and would be useful outside the context of debian-installer. Then, if debian-installer, debian-cd, or some other utility wanted to use it, then they could depend on u-boot-tools... live well, vagrant signature.asc Description: PGP signature
Re: Providing (armhf) u-boot images together with d-i images?
On Fri, 2015-01-02 at 16:40 +0100, Cyril Brulebois wrote: Karsten Merker mer...@debian.org (2015-01-02): (Do your patches end up adding the correct Built-Using on u-boot?) No, they don't, but d-i does not do that for similar components on other platforms (syslinux/isolinux/grub) as well. Probably we should do that, but then it would have to be done for all platforms. Kibi? ISTR having proposed a patch to #700026 / #696418, but that wasn't commented upon. FWIW the patch in #700026 looks good to me as a starting point... Ian. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1420655029.11796.25.ca...@debian.org
Re: Providing (armhf) u-boot images together with d-i images?
On 2015-01-05, Cyril Brulebois wrote: Karsten Merker mer...@debian.org (2015-01-02): Ok, I'll change the wording for V4. My intention was to provide a generic README that works regardless of which compression options one chooses in the makefile, so that switching from .gz to .xz would be simple without having to think of updating documentation elsewhere. Given discussions on this patchset are still going on, I've decided to stop waiting on it, and to upload d-i without it. Not to delay a new d-i version, but I'm wondering if it makes sense to decouple some of the parts of the patchset into a smaller patchset at this point, such as the netboot/tftpboot support with u-boot only sd images? Maybe concatenateable images? Some of these features seem only somewhat related, and it would be nice to get the uncontested bits out of the way. live well, vagrant signature.asc Description: PGP signature
Re: Providing (armhf) u-boot images together with d-i images?
On Wed, 2015-01-07 at 10:41 -0800, Vagrant Cascadian wrote: On 2015-01-07, Ian Campbell wrote: And therefore aren't they more usefully built along with that iso? Otherwise how do you know how big to make the partition to contain the iso at d-i build time given the varied sizes of .iso and sd-card you can buy? It would also be nice to have media with the .iso file built-in, but I don't think a requirement for it to be useful without that. Having a second external media isn't very common on the sorts of device we are talking about, is it? I definitely see the case for moving some of the code into debian-cd, or another utility. I've considered making such a utility and shipping it in u-boot-tools, but didn't get around to it for jessie. The sd-card offsets in this patchset duplicate information in the various u-boot* README.Debian files, and it would make sense to move that into a script. This script could also generate SD images with a vmlinuz, initrd, bootscript, dtb, .iso, etc. and would be useful outside the context of debian-installer. Then, if debian-installer, debian-cd, or some other utility wanted to use it, then they could depend on u-boot-tools... That sounds pretty sensible. Ian. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1420656404.11796.33.ca...@debian.org
Re: Providing (armhf) u-boot images together with d-i images?
On Fri, 2015-01-02 at 08:59 -0800, Vagrant Cascadian wrote: On 2015-01-02, Ian Campbell wrote: On Tue, 2014-12-30 at 13:42 -0800, Vagrant Cascadian wrote: On 2014-12-30, Ian Campbell wrote: I think the difference is one is a set of netboot images, and the other is a set of hd-media images. But what is the difference between these two things in this context? Aren't they functionally identical from the users perspective? Differnt initrd with different modules, and defaults to loading a different set of udebs. The hd-media scans for .iso files, the netboot initrd requires to download remaining udebs using network... Technically, both images could be made to do either based on bootprompt options or something, but that probably would require rewriting a bit more of d-i. [...] They're both typically used with sd-card, yes. But they differ in how they load the remaining portions of the installer, and have different modules and udebs in the initrds appropriate to the install media. Hopefully that's clear now? :) Much, thanks. But I still have a question about the hd-media sd-card images -- aren't they only useful in concert with something which adds the .iso etc to the image too? And therefore aren't they more usefully built along with that iso? Otherwise how do you know how big to make the partition to contain the iso at d-i build time given the varied sizes of .iso and sd-card you can buy? I think I'm probably just being dense here... Ian. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1420653663.11796.15.ca...@debian.org
Re: Providing (armhf) u-boot images together with d-i images?
Hi folks, Karsten Merker mer...@debian.org (2015-01-02): Ok, I'll change the wording for V4. My intention was to provide a generic README that works regardless of which compression options one chooses in the makefile, so that switching from .gz to .xz would be simple without having to think of updating documentation elsewhere. Given discussions on this patchset are still going on, I've decided to stop waiting on it, and to upload d-i without it. Mraw, KiBi. signature.asc Description: Digital signature
Re: Providing (armhf) u-boot images together with d-i images?
On Tue, 2014-12-30 at 13:42 -0800, Vagrant Cascadian wrote:
On 2014-12-30, Ian Campbell wrote:
0004-Add-SD-card-image-build-support-for-hd-media-builds-.patch
...
It seems to be creating a copy of dtbs again, please lets try
and keep it to one set of these files in the top level
component.
Well, all the dtb files need to be on the concatenateable images so that
simply dd'ing the right u-boot files onto the card will be able to
boot. If the image was only made for a specific system, then it would be
possible to only include the relevent dtb files.
I thought these were being copied into the output dir, rather than (or
as well as) being encapsulated into the image, doing the latter is fine.
Sounds like I misread the patch and it is infact only doing the latter.
0005-Add-SD-card-image-and-tftpboot-tarball-build-support.patch
What is this providing? Is it a mini.iso like netboot sd image?
I think it is basically like the mini.iso on i386/amd64, yes.
I'm not sure how this is different from what the previous
patches introduced.
I think the difference is one is a set of netboot images, and the other
is a set of hd-media images.
But what is the difference between these two things in this context?
Aren't they functionally identical from the users perspective?
For the tftpboot part, u-boot supports standard pxelinux.cfg
syntax configuration files -- would it be better to just
generate a suitable one of those?
Boot scripts allow the use of ${fdtfile}. As far as I understand, the
pxelinux.cfg syntax unfortunately lacks the ability to use u-boot
variables, and thus requires a different pxelinux.cfg for each system
that uses a different .dtb. Which doesn't seem particularly useful for
a single config for network booting...
pxelinux.cfg stanzas can include an fdtdir directory which is a path
which u-boot will search for ${fdtfile} under.
This isn't terribly well documented, but
http://patchwork.ozlabs.org/patch/423507/ improves things a lot.
Speaking of which, does the extlinux.conf stuff (which also supports
fdtdir) help with the multiple images problem at all?
extlinux.conf (and pxelinux.conf) also has the advantage over boot.scr
that it can offer multiple options (graphical vs text etc etc).
I also think more systems support booting boot scripts than u-boot's
PXE emulation at this point.
That's true.
Going forward we should try and make it our default option though I
think -- it was basically put together this way by the Fedora guys to
support exactly this sort of scenario.
I suppose my main overall concern is that this seems to be providing the
same thing 3 or 4 times and I'm not sure what the difference between
each of those things is (or perhaps as likely I've misunderstood what is
going on somewhere along the line). I was already concerned about the
proliferation of images which taking this approach was implying in
general and adding a multiplier to that just makes me more
uncomfortable.
If I'm reading it correctly(perhaps partly based on earlier descriptions
of the goal), there are several images produced:
1 u-boot only for each supported platform
2 hd-media full with u-boot + kernel + initrd + dtbs + bootscript for
each platform
3 hd-media concatenateable with kernel + initrd + dtbs + bootscript
4 netboot full with u-boot + kernel + initrd + dtbs + bootscript for
each platform (like mini.iso)
5 netboot concatenateable with kernel + initrd + dtbs + bootscript
6 netboot boot script for use with TFTP
(I've switched your bullets to numbers for ease of reference).
Are 2+4 and 3+5 not essentially the same things with different names (on
x86 the distinction is iso image vs a f/s image, but on ARM both are
sd-card images).
The netboot/hd-media without u-boot are meant to be concatenated
together with the appropriate u-boot only images... so yes, there is
significant overlap between the image types.
The full images are obviously easier for the users to use, as it
requires writing a single image, but obviously proliferate the number of
images for each new supported platform.
The concatenateable images obviously save a lot of space, at the expense
of the user having to do additional steps and more complicated
documentation (and might be difficult on non-unix-like platforms?).
It probably doesn't make sense to ship both the concatenateable and full
images. I think Karsten wanted to enable both so that they could be
tested in the daily images, and then later pick which would ship with
Jessie?
I'm not so much worried about concatenatable vs non as netboot vs
hd-media, but perhaps I just haven't grokked the difference.
Ian.
--
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive:
Re: Providing (armhf) u-boot images together with d-i images?
On 2015-01-02, Ian Campbell wrote:
On Tue, 2014-12-30 at 13:42 -0800, Vagrant Cascadian wrote:
On 2014-12-30, Ian Campbell wrote:
I think the difference is one is a set of netboot images, and the other
is a set of hd-media images.
But what is the difference between these two things in this context?
Aren't they functionally identical from the users perspective?
Differnt initrd with different modules, and defaults to loading a
different set of udebs. The hd-media scans for .iso files, the netboot
initrd requires to download remaining udebs using network...
Technically, both images could be made to do either based on bootprompt
options or something, but that probably would require rewriting a bit
more of d-i.
pxelinux.cfg stanzas can include an fdtdir directory which is a path
which u-boot will search for ${fdtfile} under.
This isn't terribly well documented, but
http://patchwork.ozlabs.org/patch/423507/ improves things a lot.
Thats really helpful, thanks!
Speaking of which, does the extlinux.conf stuff (which also supports
fdtdir) help with the multiple images problem at all?
...
extlinux.conf (and pxelinux.conf) also has the advantage over boot.scr
that it can offer multiple options (graphical vs text etc etc).
I don't think so, as netboot/hd-media are different images entirely. I
guess you could conceptually make one image with both initrds, and you
could use the menu support to select between the initrds... not sure if
that's worth doing at the moment.
I also think more systems support booting boot scripts than u-boot's
PXE emulation at this point.
That's true.
Going forward we should try and make it our default option though I
think -- it was basically put together this way by the Fedora guys to
support exactly this sort of scenario.
Seems reasonable. I'll add it to the increasing list of features I want
to get into u-boot for Jessie+1...
1 u-boot only for each supported platform
2 hd-media full with u-boot + kernel + initrd + dtbs + bootscript for
each platform
3 hd-media concatenateable with kernel + initrd + dtbs + bootscript
4 netboot full with u-boot + kernel + initrd + dtbs + bootscript for
each platform (like mini.iso)
5 netboot concatenateable with kernel + initrd + dtbs + bootscript
6 netboot boot script for use with TFTP
(I've switched your bullets to numbers for ease of reference).
Are 2+4 and 3+5 not essentially the same things with different names (on
x86 the distinction is iso image vs a f/s image, but on ARM both are
sd-card images).
They're both typically used with sd-card, yes. But they differ in how
they load the remaining portions of the installer, and have different
modules and udebs in the initrds appropriate to the install
media. Hopefully that's clear now? :)
live well,
vagrant
signature.asc
Description: PGP signature
Re: Providing (armhf) u-boot images together with d-i images?
Karsten Merker mer...@debian.org (2015-01-02): (Do your patches end up adding the correct Built-Using on u-boot?) No, they don't, but d-i does not do that for similar components on other platforms (syslinux/isolinux/grub) as well. Probably we should do that, but then it would have to be done for all platforms. Kibi? ISTR having proposed a patch to #700026 / #696418, but that wasn't commented upon. Mraw, KiBi. signature.asc Description: Digital signature
Re: Providing (armhf) u-boot images together with d-i images?
On Wed, 2014-12-31 at 01:59 +0100, Karsten Merker wrote: 0001-Add-boot-arm-u-boot-image-config.patch Seems fine. 0002-Provide-u-boot-binaries-for-armhf-systems-without-u-.patch I think this isn't actually publishing the u-boot binaries, but rather sd-card images with the u-boot inlined, is that right? Is there any use in shipping the raw binaries too? It provides both Sorry, I must have misread. 0003-Add-utils-gen-hd-image.patch I didn't review in detail, but perhaps the functionality of patch 0002 could be folded in? TBH I'm not sure what the distinction is between what 0002 does and what this script would produce (except I can see the script clearly has more functionality) It depends :-). gen-hd-image can provide different types of images - full images (SD card image including partition table, u-boot, kernel, initrd and dtbs) and concatenateable images, which is effectively a full image split up into a device-specific part (containing the partition table and u-boot) and a device-independent part (containing kernel, initrd and dtbs). If one decides to build concatenateable images, the device-specific part is mostly the same as the ready-to-copy SD card image generated by patch 2, with the difference that it contains a filled partition table with a predefined geometry and medium size. My original idea was to build u-boot-only images from patch 2 for people who want to install by TFTP or from USB-Stick using the existing hd-media tarball. This was the first thing I implemented. Vagrant proposed also building full images (u-boot, kernel, initrd and dtbs) similar to the images we build for i386/amd64, so I implemented that. As full images need a bit of space, the idea of providing concatenateable images instead of full images came up, but doing that is dependent on solving the one boot script for all platforms issue, for which we do not yet have a proper solution, therefore I implemented that as an option for testing this approach. Does using the extlinux.conf stuff (mentioned in my reply to Vagrant) help here? 0004-Add-SD-card-image-build-support-for-hd-media-builds-.patch I don't think we need all of (.gz|.bz2|.xz) (like the README suggests, although it's not clear these are all actually present). Just one would do IMHO. Only the gzipped variant is actually built, the README just covers the possible options (when calling gen-hd-image with -z/-j/-J). This README is installed in the output directory, I think? IMHO it should document only the compression type actually used in that directory and we should provide at most one version of each file (either compressed or not) using a single consistent compression algorithm for all such files. Talking about other compression schemes or providing multiple versions of the same file will just confuse users. It looks like someone has commented on the duplicated concat examples in v3 too. It seems to be creating a copy of dtbs again, please lets try and keep it to one set of these files in the top level component. The dtbs are only inside the generated images, they are not copied elsewhere as files. My mistake, I thought they were ending up unpacked in the output too. Having all of them inside the image is a necessity if we want to have the option of using concatenateable builds as in this case they are in the device-independent part. The size of the dtbs inside the image is less than 900kB, so I do not think that is much of an issue size-wise. Agreed. The previous patch produces hd-media builds for use with CD/DVD iso images (equivalent to the i386/amd64 hd-media boot.img.gz). This patch produces a netboot SD card (equivalent to the i386/amd64 netboot mini.iso), plus a netboot.tar.gz which contains everything needed to be put onto a tftp server for tftp-booting the installer (equivalent to the i386/amd64 PXE netboot.tar.gz). netboot.tar.gz sounds fine. What I don't get (see my reply to Vagrant) is what the practical difference to the user is between the hd-media builds and the netboot mini.iso ones, given that for ARM each is just an sd-card image. For the tftpboot part, u-boot supports standard pxelinux.cfg syntax configuration files -- would it be better to just generate a suitable one of those? I have not in detail checked the pxelinux.cfg support in u-boot yet, but as we unfortunately have to build special-casing into the boot script for several platform (such as the i.MX6 console baudrate issue), I do not see how to implement that with the pxelinux.cfg syntax. Pointers welcome :). IMHO we should be moving towards using this way of doing things, which might mean fixing things like the i.MX6 issue at source. Too late for Jessie of course. BTW our kernel now supports the
Re: Providing (armhf) u-boot images together with d-i images?
On Sat, 2014-12-27 at 00:00 +0100, Karsten Merker wrote: attached is V2 of the patchset. Changes since V1: I should start by saying that I'm not personally particularly interested in this functionality, but I don't want to be a blocker for people who are so since I've been asked I'll give my 2pence... I've been wondering if this image generation should be in debian installer or if it should be a separate project (similar to how debian-cd is separate). The advantage of doing it separately would be that the images can be rev'd or expanded (e.g. to add a new platform) independently from the installer releases, plus we don't need to worry so much about exploding the size of the main installer releases as much. Just an idle thought really. Please could you post the diff in the lists of files generated by the build, including the sizes of the new stuff. FWIW review would be a lot easier if the patches were patchbombed to the list in the normal git send-email way rather than as multiple attachments on a single mail (people can kill-thread if they aren't interested), but here we go: 0001-Add-boot-arm-u-boot-image-config.patch Seems fine. 0002-Provide-u-boot-binaries-for-armhf-systems-without-u-.patch There would be a lot less shell-in-make quoting faff if there was a helper under util/ with the bulk of the code in. I think this isn't actually publishing the u-boot binaries, but rather sd-card images with the u-boot inlined, is that right? Is there any use in shipping the raw binaries too? 0003-Add-utils-gen-hd-image.patch I didn't review in detail, but perhaps the functionality of patch 0002 could be folded in? TBH I'm not sure what the distinction is between what 0002 does and what this script would produce (except I can see the script clearly has more functionality) 0004-Add-SD-card-image-build-support-for-hd-media-builds-.patch I don't think we need all of (.gz|.bz2|.xz) (like the README suggests, although it's not clear these are all actually present). Just one would do IMHO. It seems to be creating a copy of dtbs again, please lets try and keep it to one set of these files in the top level component. The README should describe how to actually do the concatenation. 0005-Add-SD-card-image-and-tftpboot-tarball-build-support.patch What is this providing? Is it a mini.iso like netboot sd image? I'm not sure how this is different from what the previous patches introduced. For the tftpboot part, u-boot supports standard pxelinux.cfg syntax configuration files -- would it be better to just generate a suitable one of those? 0006-Add-additional-dependencies-needed-for-building-boot.patch Seems ok. But by having this last at least half the previous patches won't work in isolation. Better IMHO to add each dependency as the dependency is introduced. I suppose my main overall concern is that this seems to be providing the same thing 3 or 4 times and I'm not sure what the difference between each of those things is (or perhaps as likely I've misunderstood what is going on somewhere along the line). I was already concerned about the proliferation of images which taking this approach was implying in general and adding a multiplier to that just makes me more uncomfortable. Perhaps a summary of the new toplevel output directories (i.e. things added alongside netboot, hd-media, network-console etc) describing what each one is would help? Ian. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1419953311.13595.190.ca...@debian.org
Re: Providing (armhf) u-boot images together with d-i images?
On 2014-12-30, Ian Campbell wrote:
I should start by saying that I'm not personally particularly interested
in this functionality, but I don't want to be a blocker for people who
are so since I've been asked I'll give my 2pence...
Thanks for taking the time.
I've been wondering if this image generation should be in debian
installer or if it should be a separate project (similar to how
debian-cd is separate).
I overall like the idea, all the parts (kernel, initrd, dtb) are already
shipped in d-i, and those that aren't (u-boot) are easy to depend
on. The bootscripts could still go into d-i, or this image generation
utility itself...
I would also hope that these images get shipped through some official
channel(like the .iso files produced by debian-cd), in a format that's
easy for the end-user to install, though. Not sure what it takes to make
that happen.
0004-Add-SD-card-image-build-support-for-hd-media-builds-.patch
...
It seems to be creating a copy of dtbs again, please lets try
and keep it to one set of these files in the top level
component.
Well, all the dtb files need to be on the concatenateable images so that
simply dd'ing the right u-boot files onto the card will be able to
boot. If the image was only made for a specific system, then it would be
possible to only include the relevent dtb files.
0005-Add-SD-card-image-and-tftpboot-tarball-build-support.patch
What is this providing? Is it a mini.iso like netboot sd image?
I think it is basically like the mini.iso on i386/amd64, yes.
I'm not sure how this is different from what the previous
patches introduced.
I think the difference is one is a set of netboot images, and the other
is a set of hd-media images.
For the tftpboot part, u-boot supports standard pxelinux.cfg
syntax configuration files -- would it be better to just
generate a suitable one of those?
Boot scripts allow the use of ${fdtfile}. As far as I understand, the
pxelinux.cfg syntax unfortunately lacks the ability to use u-boot
variables, and thus requires a different pxelinux.cfg for each system
that uses a different .dtb. Which doesn't seem particularly useful for
a single config for network booting...
I also think more systems support booting boot scripts than u-boot's
PXE emulation at this point.
I suppose my main overall concern is that this seems to be providing the
same thing 3 or 4 times and I'm not sure what the difference between
each of those things is (or perhaps as likely I've misunderstood what is
going on somewhere along the line). I was already concerned about the
proliferation of images which taking this approach was implying in
general and adding a multiplier to that just makes me more
uncomfortable.
If I'm reading it correctly(perhaps partly based on earlier descriptions
of the goal), there are several images produced:
* u-boot only for each supported platform
* hd-media full with u-boot + kernel + initrd + dtbs + bootscript for
each platform
* hd-media concatenateable with kernel + initrd + dtbs + bootscript
* netboot full with u-boot + kernel + initrd + dtbs + bootscript for
each platform (like mini.iso)
* netboot concatenateable with kernel + initrd + dtbs + bootscript
* netboot boot script for use with TFTP
The netboot/hd-media without u-boot are meant to be concatenated
together with the appropriate u-boot only images... so yes, there is
significant overlap between the image types.
The full images are obviously easier for the users to use, as it
requires writing a single image, but obviously proliferate the number of
images for each new supported platform.
The concatenateable images obviously save a lot of space, at the expense
of the user having to do additional steps and more complicated
documentation (and might be difficult on non-unix-like platforms?).
It probably doesn't make sense to ship both the concatenateable and full
images. I think Karsten wanted to enable both so that they could be
tested in the daily images, and then later pick which would ship with
Jessie?
I'm not attached to it all happening inside of debian-installer, just
hope that whatever's done makes it easy for the users to install debian
on some of these newish accessible low-power platforms...
live well,
vagrant
signature.asc
Description: PGP signature
Re: Providing (armhf) u-boot images together with d-i images?
On 2014-12-26, Karsten Merker wrote:
On Tue, Dec 23, 2014 at 02:17:16PM -0800, Vagrant Cascadian wrote:
On 2014-12-23, Karsten Merker wrote:
I don't think it's possible yet to have the device-independent parts
working with *all* platforms (at least for Jessie), but we can get close
with a few relatively simple workarounds:
Some of the issues with bootscr.mainline_common are inconsistancies
across platforms with the u-boot console settings (particularly the baud
rate not being appended)
...
I do not know what the default environment on those platforms
looks like, but assuming that they define the fdtfile environment
variable, how about adding stanzas like
if test ${fdtfile} = am335x-boneblack.dtb ; then
(set the necessary environment variables)
fi
to bootscr.mainline_common for those special cases?
For the default serial console, yes, this could work (at least for
imx6q-wandboard.dtb and imx6q-cubox-i.dtb). For setting the framebuffer
or other console, it would still probably require manually (or through
uEnv.txt) setting the console variable.
It would get increasingly more complicated the more boards get supported
over time, but at the moment we're looking at only a handful of boards.
Alternately, could also check for specific values of console/baudrate,
which could support broader classes of systems:
# wandboard, cubox-i and possibly other imx systems may need to append
# the baud rate.
if test ${console} = ttymxc0 test -n ${baudrate} ; then
setenv console=${console},${baudrate}
fi
This way, if the user has manually configured the console value somehow
(such as console=tty0 for framebuffer), it doesn't blindly overwrite it
just because it detects a certain ${fdtfile}.
That doesn't address the conditionals that the BeagleBone Black would
need, which could potentially boot off of eMMC or an inserted SD card,
and thus ${devnum} and related variables would need to be set
conditionally...
I had planned on patching u-boot to emulate the config_distro_bootcmd
for BeagleBone Black (like with wandboard/cubox-i), but was unable to
resolve incompatible uses of ${bootpart}.
Regarding uEnv.txt: is that file automatically read by the
default environment of those platforms before executing boot.scr,
or does boot.scr have to explicitly source uEnv.txt?
The BeagleBone Black, Wandboard* and CuBox-i systems should all load
uEnv.txt before the bootscript, at least with the u-boot in Debian. I've
added some patches for uEnv.txt support and changed the order on some of
those to make sure that they worked more-or-less consistantly.
I tried getting uEnv.txt support into mainline u-boot's
config_distro_bootcmd.h, but didn't get much favorable feedback about
that:
http://lists.denx.de/pipermail/u-boot/2014-October/190843.html
Though I do think it's generally useful to be able to make generic
images and tweak them with a few uEnv.txt variables to set things that
are not detectable.
The netboot images here seem more like the mini.iso netboot images on
x86, though I've tested several armhf installs with the boot.scr,
kernel, initrd, dtb all downloaded via TFTP (more like the pxe boot
images on x86).
...
I have added support for building a netboot.tar.gz similar to the
i386/amd64 PXE one in V2 of the patchset.
Excellent!
live well,
vagrant
signature.asc
Description: PGP signature
Re: Providing (armhf) u-boot images together with d-i images?
Added debian-cd@.
Karsten Merker mer...@debian.org (2014-12-23):
On Thu, Dec 18, 2014 at 07:28:45PM +0100, Karsten Merker wrote:
On Wed, Dec 03, 2014 at 03:10:37PM -0800, Vagrant Cascadian wrote:
On 2014-12-03, Karsten Merker wrote:
several armhf systems do not have u-boot (or another firmware) in
non-volatile (i.e. ROM/Flash) memory, but instead store their
system firmware on a removable medium such as an SD card.
...
Debian provides appropriate u-boot images for several supported
systems in the u-boot-{exynos,imx,omap,sunxi} packages, but those
are not easily accessible to somebody who does not already run a
Debian/armhf system (or at least Debian on another architecture),
so I am wondering whether we should offer these u-boot images
(in unpacked form) together with the d-i images, similar to what
we do with the device-tree files extracted from the linux-image
package.
[snip]
For images such as hd-media, we'd ideally want to provide complete
u-boot + kernel + initrd ( + gtk initrd) images, which would grow each
image considerably.
[snip]
I'd be happy to put a bit more time into it, especially if we can get
support in for Jessie's debian-installer.
[snip]
just to avoid double work: I have recently started working on implementing
image building support (both for bare u-boot images as well as for full
images with u-boot + kernel + initrd) and I hope to find some time over
Christmas to finish a working prototype.
Attached is a set of patches to implement building bootable
d-i images for armhf systems. It offers various options:
- provide binary u-boot-only images for people who want to
manually install u-boot and e.g. run the rest of the setup via
tftpboot
- build full installer images in the variants netboot und
hd-media, which contain u-boot, kernel, initrd and dtbs.
- build concatenateable installer images in the variants
netboot and hd-media. These have the same contents as the
full images but are split into a device-specific and a
device-independent part to save space and must be decompressed
and then concatenated together by the user.
For testing purposes all options are enabled in the attached
patchset. In the end we should decide which ones we actually
want to enable for production use, as it does not make sense to
build both the full and concatenateable variants.
If space and bandwidth are not much of an issue, I would propose
providing the u-boot-only and full image targets, as they are
the easiest for the end user and can also be handled on Windows
systems without problems. If space and bandwidth use is too high
for the full images, we could use the concatenateable
variant, although this makes installing the images more
complicated for the end user and is only possible if we can
manage to boot _all_ armhf platforms with a single boot script,
as this boot script is in the device-independent part. The
latter is a point that still needs checking and for which I need
testers who have access to the respective systems.
I would like to commit the patches to the d-i git so that
testers can give images generated from the daily builds a try.
I would enable the u-boot-only and full targets for the daily
builds in this case. If the full targets work properly on all
platforms, we can then think about whether we would like to
replace the full variant by the concatenateable variant.
@Kibi, is that ok for you?
I'd be happy to have more feedback on this, notably from Steve on the
debian-cd@ side, and from Ian for the arm side (just to make sure).
Mraw,
KiBi.
Regards,
Karsten
P.S: I hope I have added all necessary dependencies to the
control file - checking them by building d-i in pbuilder
does not work as d-i requires network access during the
build.
--
Gem. Par. 28 Abs. 4 Bundesdatenschutzgesetz widerspreche ich der Nutzung
sowie der Weitergabe meiner personenbezogenen Daten für Zwecke der
Werbung sowie der Markt- oder Meinungsforschung.
From 43bd59363968d603d44f72db7f854575f8699750 Mon Sep 17 00:00:00 2001
From: Karsten Merker mer...@debian.org
Date: Sun, 21 Dec 2014 19:22:56 +0100
Subject: [PATCH 1/6] Add boot/arm/u-boot-image-config
The file boot/arm/u-boot-image-config contains the information
which u-boot components must be written to which offsets on the
disk to create a bootable image for various arm-based systems.
---
build/boot/arm/u-boot-image-config | 21 +
1 file changed, 21 insertions(+)
create mode 100644 build/boot/arm/u-boot-image-config
diff --git a/build/boot/arm/u-boot-image-config
b/build/boot/arm/u-boot-image-config
new file mode 100644
index 000..1d0208a
--- /dev/null
+++ b/build/boot/arm/u-boot-image-config
@@ -0,0 +1,21 @@
+# U-Boot SPL/TPL files and offsets for various platforms
+#
+# Line format (offsets are given in blocks of 512 Bytes):
+#
