Re: [OpenWrt-Devel] Is it possible to create two images for the same device with a different set of DEVICE_PACKAGES?

2020-04-10 Thread Sven Roederer 
Am Dienstag, 7. April 2020, 16:22:37 CEST schrieb Hannu Nyman:
> I do not think that there is a nice clean solution, as I do not remember
> seeing a solution of different packages for iniramfs, factory and sysupgrade
> images.
> 
> I would approach that with a two-step build process, using two .config
> recipes:
> 
> * First a build with a smaller .config recipe without that large Quantenna
> firmware. This creates the initramfs image, (which you copy to a safe place
> before the second build)
> 
> * Then a second build from a recipe including the Quantenna firmware. No
> need to do "make clean", so the second build is rather quick. That produces
> the full sysupgrade image.
> 
> In your build automation scripts, those two builds could be run
> consequtively, with a copy step between them.
> 

For our Freifunk-build we do a similar 2-step thing

* using a .config having all pacakges added modular and only the packages
for the initrd set to "y" to have them embedded
* doing a normal build produces the initrds and the imagebuilder
* with the imagebuilder we generate the final sysupgrade images with the 
full package-set

The generated initrds are just a luci-mod-failsafe wich allows flashing the 
sysupgrade via browser and of course console.

Sven





___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] Is it possible to create two images for the same device with a different set of DEVICE_PACKAGES?

2020-04-08 Thread Andre Valentin 
Hi Bjoern,

I made a patch for include/image-commands.mk that allows to building squashfs 
initrd images.
The reason for me is that the imagebuilder is unable to generate initramfs 
images.
When building the final images with the imagebuilder you can define which 
packages to include.
Your platform has to use device-tree!

That's the function:
# This take the vanilla kernel, modifies the bootarg of dtb and appends
# the filesystem to the kernel at the given initrd offset
# address (respect kernelsize!).
# Argument 1 : initrd offset
define Build/prepend-initrd-kernel
let \
initrd_offset="$(subst k,* 1024,$(word 1, $(1)))" \
rootfs_size="$$(stat -c%s $@)/4096*4096+4096" \
kernel_size="$$(stat -c%s $(KDIR)/$(KERNEL_NAME))" \
dtb_size_in="$$(stat -c%s $(KDIR)/image-$(firstword 
$(DEVICE_DTS)).dtb )" \
kernel_dtb_size="kernel_size+dtb_size" \
initrd_virtual_addr="$(KERNEL_LOADADDR)+initrd_offset" ; \
if [ $$kernel_dtb_size -ge $$initrd_offset ] ; then \
echo initrd_offset is too small for kernel ; \
exit 1 ; \
fi ; \
bootargs=$$(printf "root=/dev/ram0 rd_start=0x%x rd_size=0x%x 
ramdisk_size=0x%x $(wordlist 2,$(words $(1)),$(1))" \
$$initrd_virtual_addr $$rootfs_size $$rootfs_size ) ; \
cp $(KDIR)/$(KERNEL_NAME) $@.tmp ; \
$(LINUX_DIR)/scripts/dtc/dtc -I dtb -O dts -o - 
$(KDIR)/image-$(firstword $(DEVICE_DTS)).dtb | \
$(TOPDIR)/scripts/patch-dtb-bootargs.pl "$$bootargs" | \
$(LINUX_DIR)/scripts/dtc/dtc -I dts -O dtb -o - - >> $@.tmp ; \
dd if=$@.tmp of=$@.tmp.padded conv=sync bs=$$initrd_offset count=1 ; \
mv $@.tmp.padded $@.tmp ; \
dd if=$@ bs=$$rootfs_size count=1 conv=sync >> $@.tmp ; \
mv $@.tmp $@
endef

Kernel needs following symbols:
CONFIG_BLK_DEV_INITRD=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=1
CONFIG_BLK_DEV_RAM_SIZE=0

This is the line for image/Makefile or ...
IMAGE/tftpboot-uImage-ramdisk.bin = append-rootfs | prepend-initrd-kernel 
16777216 | lzma | uImage lzma

16777216 is the offset for the initrd in memory (take kernel size into 
account), may be lowered for you.

Kind regards,

André

Am 08.04.20 um 09:26 schrieb Bjørn Mork:
> Tomasz Maciej Nowak  writes:
>
>> In include/kernel-defaults.mk there is
>> INITRAMFS_EXTRA_FILES ?= 
>> $(GENERIC_PLATFORM_DIR)/image/initramfs-base-files.txt.
>> Maybe You could add a logic for ignore files list if they exist. No other
>> solution comes to my mind.
>
> Thanks for the suggestion.  But I believe this file is used directly as
> config to the initramfs builder in the kernel.  Extending the format
> would mean modifying usr/gen_init_cpio.c in the kernel.  That's a bit
> overkill IMHO.
>
> I am probably ending up with the simplest solution: Dropping the large
> firmware package from DEVICE_PACKAGES. Making defconfig images usable
> for first-time installation is much more important than having every
> necessary package included in the defconfig sysupgrade images.
>
>
> Bjørn
>
> ___
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
>

-- 
Mit freundlichen Grüßen
André Valentin

Systemadministration - Projektkoordination


--
MarcanT AG, Herforder Straße 163a, D - 33609 Bielefeld
Fon: +49 (521) 95945-0 | Fax: +49 (521) 95945-18
URL: http://www.marcant.net  | 
http://www.global-m2m.com 

Internet * Netzwerk * Mobile Daten

Vorstand:
Thorsten Hojas (Vorsitzender)
Marc-Henrik Delker
Dr. Anja-Christina Padberg
Handelsregister: AG Bielefeld, HRB 42260 USt-ID Nr.: DE 190203238



___
Ausserhalb unserer Geschäftszeiten (Montag bis Freitag von 8:30 Uhr bis
17:30 Uhr, ausgenommen gesetzliche Feiertage in NRW) stehen wir Ihnen
gemäß Ihrer jeweiligen Service-Level-Agreements unter der Ihnen
mitgeteilten Telefonnummer für Störungen und Notfälle zur Verfügung.
Sie können natürlich auch gerne jederzeit unter supp...@marcant.net ein
Ticket eröffnen, welches am nächsten Arbeitstag bearbeitet wird.





smime.p7s
Description: S/MIME Cryptographic Signature
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] Is it possible to create two images for the same device with a different set of DEVICE_PACKAGES?

2020-04-08 Thread Bjørn Mork 
Tomasz Maciej Nowak  writes:

> In include/kernel-defaults.mk there is
> INITRAMFS_EXTRA_FILES ?= 
> $(GENERIC_PLATFORM_DIR)/image/initramfs-base-files.txt.
> Maybe You could add a logic for ignore files list if they exist. No other
> solution comes to my mind. 

Thanks for the suggestion.  But I believe this file is used directly as
config to the initramfs builder in the kernel.  Extending the format
would mean modifying usr/gen_init_cpio.c in the kernel.  That's a bit
overkill IMHO.

I am probably ending up with the simplest solution: Dropping the large
firmware package from DEVICE_PACKAGES. Making defconfig images usable
for first-time installation is much more important than having every
necessary package included in the defconfig sysupgrade images.


Bjørn

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] Is it possible to create two images for the same device with a different set of DEVICE_PACKAGES?

2020-04-07 Thread Tomasz Maciej Nowak 
Hi Bjørn.

W dniu 07.04.2020 o 16:50, Bjørn Mork pisze:
> Hannu Nyman  writes:
> 
>> I do not think that there is a nice clean solution, as I do not
>> remember seeing a solution of different packages for iniramfs, factory
>> and sysupgrade images.
>>
>> I would approach that with a two-step build process, using two .config 
>> recipes:
>>
>> * First a build with a smaller .config recipe without that large
>> Quantenna firmware. This creates the initramfs image, (which you copy
>> to a safe place before the second build)
>>
>> * Then a second build from a recipe including the Quantenna
>> firmware. No need to do "make clean", so the second build is rather
>> quick. That produces the full sysupgrade image.
>>
>> In your build automation scripts, those two builds could be run
>> consequtively, with a copy step between them.
>>
>> That will be much easier than trying to code a logic into the actual
>> OpenWrt build Makefiles.
> 
> Yes, sure, this will work for my own use.  But it doesn't solve the
> general problem, with pre-built images involved.
> 
> What if I want to make a recipe that works on the OpenWrt Buildbots?
> The idea was to make first time installation as easy as possible, by
> providing both an image that can be installed from OEM and an image that
> enables the full hardware.
> 
> I did come up with a sort of working proof-of-concept hack, where I add
> a build rule like (yes, ugly - I'm not excpecting to push this):
> 
> define Build/filtered-initramfs
>   rm -rf $(TARGET_DIR).x
>   sed -i -e 's,CONFIG_INITRAMFS_SOURCE="$(strip $(TARGET_DIR)) 
> ,CONFIG_INITRAMFS_SOURCE="$(strip $(TARGET_DIR)).x ,' $(LINUX_DIR)/.config
>   cp -a $(TARGET_DIR) $(TARGET_DIR).x
>   rm -rf $(TARGET_DIR).x/lib/firmware/qv840 
> $(TARGET_DIR).x/usr/lib/opkg/info/qv840-firmware.*
>   $(call Image/BuildKernel/Initramfs)
> endef
> 
> 
> I was just hoping there would be nicer ways to do it.

In include/kernel-defaults.mk there is
INITRAMFS_EXTRA_FILES ?= $(GENERIC_PLATFORM_DIR)/image/initramfs-base-files.txt.
Maybe You could add a logic for ignore files list if they exist. No other
solution comes to my mind. 

> 
> 
> 
> 
> Bjørn
> 
> ___
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
> 
-- 
TMN

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] Is it possible to create two images for the same device with a different set of DEVICE_PACKAGES?

2020-04-07 Thread Bjørn Mork 
Hannu Nyman  writes:

> I do not think that there is a nice clean solution, as I do not
> remember seeing a solution of different packages for iniramfs, factory
> and sysupgrade images.
>
> I would approach that with a two-step build process, using two .config 
> recipes:
>
> * First a build with a smaller .config recipe without that large
> Quantenna firmware. This creates the initramfs image, (which you copy
> to a safe place before the second build)
>
> * Then a second build from a recipe including the Quantenna
> firmware. No need to do "make clean", so the second build is rather
> quick. That produces the full sysupgrade image.
>
> In your build automation scripts, those two builds could be run
> consequtively, with a copy step between them.
>
> That will be much easier than trying to code a logic into the actual
> OpenWrt build Makefiles.

Yes, sure, this will work for my own use.  But it doesn't solve the
general problem, with pre-built images involved.

What if I want to make a recipe that works on the OpenWrt Buildbots?
The idea was to make first time installation as easy as possible, by
providing both an image that can be installed from OEM and an image that
enables the full hardware.

I did come up with a sort of working proof-of-concept hack, where I add
a build rule like (yes, ugly - I'm not excpecting to push this):

define Build/filtered-initramfs
rm -rf $(TARGET_DIR).x
sed -i -e 's,CONFIG_INITRAMFS_SOURCE="$(strip $(TARGET_DIR)) 
,CONFIG_INITRAMFS_SOURCE="$(strip $(TARGET_DIR)).x ,' $(LINUX_DIR)/.config
cp -a $(TARGET_DIR) $(TARGET_DIR).x
rm -rf $(TARGET_DIR).x/lib/firmware/qv840 
$(TARGET_DIR).x/usr/lib/opkg/info/qv840-firmware.*
$(call Image/BuildKernel/Initramfs)
endef


I was just hoping there would be nicer ways to do it.




Bjørn

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] Is it possible to create two images for the same device with a different set of DEVICE_PACKAGES?

2020-04-07 Thread Hannu Nyman 
I do not think that there is a nice clean solution, as I do not remember 
seeing a solution of different packages for iniramfs, factory and sysupgrade 
images.


I would approach that with a two-step build process, using two .config recipes:

* First a build with a smaller .config recipe without that large Quantenna 
firmware. This creates the initramfs image, (which you copy to a safe place 
before the second build)


* Then a second build from a recipe including the Quantenna firmware. No need 
to do "make clean", so the second build is rather quick. That produces the 
full sysupgrade image.


In your build automation scripts, those two builds could be run 
consequtively, with a copy step between them.


That will be much easier than trying to code a logic into the actual OpenWrt 
build Makefiles.




___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

[OpenWrt-Devel] Is it possible to create two images for the same device with a different set of DEVICE_PACKAGES?

2020-04-07 Thread Bjørn Mork 
The device I am playing with (ZyXEL WAP6805) can be upgraded to OpenWrt
by tftp'ing an OpenWrt initramfs image to it.  But the image *must* be
less than 6815744 bytes, which is a hard coded limit in the OEM tftp
server.

The device also includes a Quantenna module, which needs a rather large
firmware image (4 MB or more). I want to include this firmware in the
OpenWrt sysupgrade images, to enable as much hardware support as
possible by default.

But if I include the Quantenna firmware in DEVICE_PACKAGES, then the
initramfs image becomes too large to be installed from OEM firmware.

Is there any way to work around this?  I have been looking at the magic
of image.mk and friends for hours now, without seeing any obvious and
clean solution.  It would be nice if it was possible to build a separate
rootfs with an explicit short package list, overriding the .config for
special purpose, size restricted, images.

Anyone have pointers to docs or relevant examples?

Or maybe some "don't do that" advice?  I have a sneaky feeling I am
looking at this from the wrong side or something...


Bjørn

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel