[OpenWrt-Devel] [RFC] use Debian like release channel identifiers?

[Paul Spooren]

Hi all,

as 19.07 is *just around the corner* I was wondering if there's a better 
way of distinguishing between versions.


Right now, I see 4 different *channels* which somewhat match the Debian 
style, therefore a possible mapping:


18.06.N -> stable
19.07-rcN -> testing
19.07-SNAPSHOT -> unstable
SNAPSHOT -> experimental

This naming could allow users to choose different "upgrade" channels, 
like for Debian, Nextcloud, Firefox, etc. They could be informed about a 
new upgrades via the LuCi interface or a cron-mail-daemon-service-thing.


The tool to check for upgrades could be build on-top of a PR[0] 
introducing JSON information of versions, targets and created images.


A Luci app is currently being created[1], I'd work on some rpcd code to 
work in the background.


It's somewhat cosmetic only but also simplifies user understanding of 
what they're using - right?


Best,
Paul

[0]: https://github.com/openwrt/openwrt/pull/2192.diff
[1]: 
https://github.com/sudhanshu16/luci/commit/750c3e22c7b4f6e98363cc16813355132558b1bf



___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [lantiq] help in supporting FRITZ!BOX 3272 (Fritz_Box_HW198))

[Enrico Mioso]

Dear Martin,
thank you very very much for being so Kind!
And thank you to all of you for your work!

Your suggestions where precious - and I will look at the referenced device.
In the meantime, I was taking a look at the flash layout. Luckily, it seems EVA 
will allow me to boot from RAM, which will be very useful.
Still, I am trying to understand as much as possible about this device before 
making any change.

First of all - I backed up all partitions using nanddump, except for those marked as 
2out of reach" in dmesg, which are not readale by nanddump.
I used something like
nanddup -f file /dev/mtd

Secondly, I was looking at EVA environment, which is as follows:

HWRevision198
HWSubRevision 1
ProductID Fritz_Box_HW198
SerialNumber  
annex A
autoload  yes
bootloaderVersion 1.1787
bootserport   tty0
country   039
cpufrequency  5
firstfreeaddress  0x81113040
firmware_info 126.06.83
firmware_version  avme
flashsize nor_size=0MB sflash_size=256KB nand_size=128MB
language  it
linux_fs_start1
maca  5C:49:79:DB:93:0C
macb  5C:49:79:DB:93:0D
macwlan   5C:49:79:DB:93:0E
macdsl5C:49:79:DB:93:0F
memsize   0x0800
modetty0  38400,n,8,1,hw
modetty1  38400,n,8,1,hw
mtd0  0x40,0x340
mtd1  0x0,0x40
mtd2  0x0,0x2
mtd3  0x2,0x3
mtd4  0x3,0x4
mtd5  0x0,0x20
my_ipaddress  192.168.178.1
promptEva_AVM
provider  wind_3272_12082013
req_fullrate_freq 25000
sysfrequency  25000
tr069_passphrase  TFX1HhthwN6T
tr069_serial  00040E-5C4979DB930C
urlader-version   2787
usb_board_mac 5C:49:79:DB:93:10
usb_device_id 0x
usb_device_name   USB DSL Device
usb_manufacturer_name  AVM
usb_revision_id   0x
usb_rndis_mac 5C:49:79:DB:93:11
wlan_key  ??

(wlankey omitted).
I am not able to understand the general syntax for mtd partitions - e.g.: it 
seems the bootloader uses completely different partition layout).
I don't think I'll need to alter those partitions, but I am better in 
understanding their definitions should something go wrong in the wrong moment. 
:)
Thanks again, and sorry for the late answer.

Enrico

P.s.: deleted quoted text to make it easier to read the mail for me...

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

[OpenWrt-Devel] [sdwalker/sdwalker.github.io] 08cab0: This week's update

[Stephen Walker]

  Branch: refs/heads/master
  Home:   https://github.com/sdwalker/sdwalker.github.io
  Commit: 08cab0a76c31d9d619bc6ef55ef77ffa113578ee
  
https://github.com/sdwalker/sdwalker.github.io/commit/08cab0a76c31d9d619bc6ef55ef77ffa113578ee
  Author: Stephen Walker 
  Date:   2019-08-25 (Sun, 25 Aug 2019)

  Changed paths:
M uscan/index-18.06.html
M uscan/index.html

  Log Message:
  ---
  This week's update



___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] Squashfs breakage lottery with UBI WAS: [PATCH RFC 2/2] amp821xx: use newly added pad-squashfs for Meraki MR24

[Russell Senior]

trimming a bit for easier reading ...

> [Fixed Mathias Email]

> > > [...]
> > > OpenWrt's mksquashfs for the rootfs (which is usually
> > > squashfs) is instructed to skip the padding via the nopad
> > > option because the rootfs will be padded by functions like
> > > pad-rootfs to the eraseblocksize which is usually much
> > > bigger at somewhere 64KiB.
> > 
> > Note also, e.g. tplink,tl-wdr3600-v1:
> >
> > [0.428860] m25p80 spi0.0: en25q64 (8192 Kbytes)
> > [0.433638] 3 fixed-partitions partitions found on MTD device spi0.0
> > [0.440112] Creating 3 MTD partitions on "spi0.0":
> > [0.444991] 0x-0x0002 : "u-boot"
> > [0.450914] 0x0002-0x007f : "firmware"
> > [0.459935] 2 tplink-fw partitions found on MTD device firmware
> > [0.465951] Creating 2 MTD partitions on "firmware":
> > [0.471047] 0x-0x001b6b1b : "kernel"
> > [0.476924] 0x001b6b1c-0x007d : "rootfs"
> > 
> > netgear,wndr3800:
> > 
> > [0.671227] m25p80 spi0.0: mx25l12805d (16384 Kbytes)
> > [0.676366] 4 fixed-partitions partitions found on MTD device spi0.0
> > [0.682724] Creating 4 MTD partitions on "spi0.0":
> > [0.687508] 0x-0x0005 : "u-boot"
> > [0.693223] 0x0005-0x0007 : "u-boot-env"
> > [0.699163] 0x0007-0x00ff : "firmware"
> > [0.707493] 2 netgear-fw partitions found on MTD device firmware
> > [0.713550] Creating 2 MTD partitions on "firmware":
> > [0.718507] 0x-0x001bd440 : "kernel"
> > [0.724195] 0x001bd440-0x00f8 : "rootfs"
> > 
> > and netgear wgt634u:
> > 
> > [1.245465] 3 bcm47xxpart partitions found on MTD device
> > physmap-flash.0
> > [1.252454] Creating 3 MTD partitions on "physmap-flash.0":
> > [1.258364] 0x-0x000a : "boot"
> > [1.286600] 0x000a-0x007e : "firmware"
> > [1.298172] 3 trx partitions found on MTD device firmware
> > [1.303639] Creating 3 MTD partitions on "firmware":
> > [1.308944] 0x001c-0x0948 : "loader"
> > [1.331486] 0x0948-0x00139800 : "linux"
> > [1.348577] 0x00139800-0x0074 : "rootfs"
> > 
> > as examples where the rootfs starts at unaligned addresses. If you
> > padded the rootfs individually, the combination of kernel+rootfs would
> > unnecessarily waste space.
> Aren't these all devices with spi-nor? They all just place the kernel +
> rootfs (squashfs) in a "firmware" mtd partitions and the mtdsplit is doing
> its magic. [...]

Yes, agreed. These are examples of why we shouldn't just remove the
-nopad, and actually mostly just my own sanity check that I remembered
this on some devices (other spi-nor devices, such as the buffalo
wzr600dhp seem to align the rootfs). Oh, and (irrelevant detail) the
wgt634u is NOR, but not spi.

> I think this is a bit out of the scope. This patch should not
> interfere with them and the unaligned squashfs rootfs starts is not a
> problem from what I can tell. That said removing the "-nopad" switch or
> adding the padding with "pad-squashfs" you made should make no difference
> in regards to the missing padding between the linux/kernel and rootfs
> since this isn't the problem.

I was under the impression that removing the -nopad switch would pad out
the root.squashfs out to a 4k boundary before concatenation, leading to
a further padding of the concatenation since the padding will (in
general) hang over a 4k boundary.

> > > But this is a problem for squashfs as rootfs in ubi
> > > partitions. Currently no explicit padding is being
> > > set and it currently works for the most time because
> > > ubi volumes are always aligned to LEBs which could
> > > be close enough for 4KiB paddings...
> > > 
> > > Digging down deeper revealed that squashfs excepts blocks
> > 
> > trivial spelling fix, that should be "accepts", I think...
> Not sure if it's trivial or not, but yes the "excepts" is wrong,
> I think it should have been "expects". But... I've still hope
> that "-nopad will be the way" so I'm prepared to just drop this
> patch again :D.
> 
> > > to be up to PAGE_SIZE. This is explained in this bug report
> > > that states that the 4k padding for ARCHs with 64KiB pages
> > > resulted in "attempt access beyond end of device" errors:
> > > 
> > 
> > AFAICT, the PAGE_SIZE on Meraki MR24 is 4k. In the kernel's
> > include/asm-generic/page.h, we have:
> The APM821xx SoC supports 4KiB, 16KiB and 64KiB page sizes.
> Ie: 
> OpenWrt currently defaults to 4KiB, but the 16KiB and 64KiB
> do provide a throughput boost and they are easily enabled
> by just editing config-default a bit.

The MR24's NAND is only 32MB (okay, that's not tiny), but I'm all for
optimizing for size.

> > When Jonas and I were discussing this, we noted that sysupgrade 

Re: [OpenWrt-Devel] Squashfs breakage lottery with UBI WAS: [PATCH RFC 2/2] amp821xx: use newly added pad-squashfs for Meraki MR24

[Christian Lamparter]

[Fixed Mathias Email]

On Sunday, August 25, 2019 8:16:48 AM CEST Russell Senior wrote:
> > On Saturday, August 24, 2019 2:18:55 AM CEST Russell Senior wrote:
> > > > What's happening here is that mksquashfs4 is being told
> > > > through the "-nopad" option to "do not pad filesystem to a
> > > > multiple of 4K" [0].
> > > 
> > > > |define Image/mkfs/squashfs |
> > > > $(STAGING_DIR_HOST)/bin/mksquashfs4 $(call
> > > > mkfs_target_dir,$(1)) $@ \ | -nopad -noappend -root-owned \ |
> > > > -comp $(SQUASHFSCOMP) $(SQUASHFSOPT) \ | -processors 1 |endef
> > > 
> > > > My guess is that this affects more than just the MR24 (I'm
> > > > looking at you too: pad2jffs and various pad-to $value)
> > > > . I've tried tracking down the change that added the "-nopad"
> > > > and found it in a commit from 2005 titled: "add some changes
> > > > from whiterussian to head" [1] [2]:
> > > 
> > > I agree that other devices where rootfs is squashfs and lives on a ubi
> > > volume are probaby also vulnerable to this problem. Regrettably, I haven't
> > > thought of any other of those devices that I have on hand to test. 
> > > 
> > > > | $(KDIR)/root.squashfs: | @mkdir -p $(KDIR)/root/jffs |-
> > > > $(STAGING_DIR)/bin/mksquashfs-lzma $(KDIR)/root $@ -noappend
> > > > -root-owned -le |+ $(STAGING_DIR)/bin/mksquashfs-lzma
> > > > $(KDIR)/root $@ -nopad -noappend -root-owned -le
> > > 
> > > 
> > > > So, this is really old...
> > > 
> > > > Question is, should we just drop the -nopad? Since as you
> > > > established, in the described corner-case (see above)
> > > > squashfs needs this 4k padding and the generated squashfs
> > > > could be considered broken otherwise?
> > > 
> > > I'm under the impression that the -nopad makes sense for NOR flash where
> > > the kernel and rootfs are glued together, padding the isolated rootfs
> > > would cause alignment problems or wasted space in the combined blobs.
> > 
> > Yes, that's the nod to padjffs2. That said,
> >  makes
> > it sound like that apart from the BLOCKSIZE, we also need to PAGE_SIZE?
> > 
> > (I think the APM821XX is a special case, since it can do 64KiB Pages
> > as well as it's 32MiB SLC NAND that uses 16 KiB erase-blocks. So a
> > PAGE can span up to 4 pages.
> > 
> > > 
> > > > (Judging from your
> > > > message, you went through the kernel code. Can you please
> > > > share the place where the lack of the padding is breaking the
> > > > fs?)
> > > 
> > > It's mostly inferred from the fact that I saw the error and kernel
> > > panic, and glancing at the kernel squashfs code. I am not pretending to
> > > understand completely how the squashfs kernel code works, but the
> > > position of the error relative to the size of the rootfs in my panic
> > > case strongly suggests it was trying to read past the end of the ubi
> > > volume.
> > > 
> > > The error comes in the kernel's fs/squashfs/block.c in the
> > > squashfs_read_data() function. There are a bunch of conditions (at least
> > > 5) within the function (see "goto read_failure;") that will lead to that
> > > message being printed.
> > > 
> > Well, that's a pity this could have saved a lot of time.
> > 
> > I've cobbered together a patch that deals with some of the
> > padding issues at "ubimkvol" and "ubinize" time. The idea
> > is that ideally we want to do the padding when we know
> > PAGE_SIZE and the BLOCKSIZE/Erasesize (MR24 blocksize in
> > image/Makefile seems wrong as well...).
> > 
> > But for now, it's set to 64KiB. If this is the way forward
> > we add enable getconf and get the PAGESIZE at runtime. If not,
> > we need to come up with something else.
> > (It's also possible to do some changes in  ubi's block code or
> > squashfs kernel code to mitigate the padding, but I don't think
> > the maintainers will even look at it).
> > 
> > 
> > Regards,
> > Christian
> > ---
> > From 803cab7d585ebb85362357d008067caf645a7f17 Mon Sep 17 00:00:00 2001
> > From: Christian Lamparter 
> > Date: Sat, 24 Aug 2019 12:55:40 +0200
> > Subject: [PATCH] base-files: pad root.squashfs to 64KiB in ubi volumes
> > 
> > SquashFS's HOWTO states in the section "Using mksquashfs"
> > 
> > that a padding is necessary "for the filesystem to be used
> > on block devices."
> > 
> > OpenWrt's mksquashfs for the rootfs (which is usually
> > squashfs) is instructed to skip the padding via the nopad
> > option because the rootfs will be padded by functions like
> > pad-rootfs to the eraseblocksize which is usually much
> > bigger at somewhere 64KiB.
> 
> Note also, e.g. tplink,tl-wdr3600-v1:
>
> [0.428860] m25p80 spi0.0: en25q64 (8192 Kbytes)
> [0.433638] 3 fixed-partitions partitions found on MTD device spi0.0
> [0.440112] Creating 3 MTD partitions on "spi0.0":
> [0.444991] 0x-0x0002 : "u-boot"
> [0.450914] 0x0002-0x007f : "firmware"
> [0.459935] 2 tplink-fw partitions 

Re: [OpenWrt-Devel] Squashfs breakage lottery with UBI WAS: [PATCH RFC 2/2] amp821xx: use newly added pad-squashfs for Meraki MR24

[Russell Senior]

Some comments inline below ...

> On Saturday, August 24, 2019 2:18:55 AM CEST Russell Senior wrote:
> > > "Christian" == Christian Lamparter  writes:
> > 
> > > I've posted a similar message to the bugreport:
> > > 
> > 
> > I should have filed the bug first and linked it in my patch.
> I think it's fine. It depends on whenever there will be a
> discussion and where it will take place... But yeah, nobody
> can tell in advance how this will go.
> 
> On Saturday, August 24, 2019 2:59:31 AM CEST Russell Senior wrote:
> > > "Russell" == Russell Senior  writes:
> > 
> > > "Christian" == Christian Lamparter  writes:
> > 
> > Russell> It's mostly inferred from the fact that I saw the error and
> > Russell> kernel panic, and glancing at the kernel squashfs code. I am
> > Russell> not pretending to understand completely how the squashfs kernel
> > Russell> code works, but the position of the error relative to the size
> > Russell> of the rootfs in my panic case strongly suggests it was trying
> > Russell> to read past the end of the ubi volume.
> > 
> > Oh, and I got important help finding this from Jonas Gorski. I was
> > remiss in not mentioning that.
> > 
> Ok, Let's add him to the CC then. As well as some of the 
> "ramips: Fix and tidy up IMAGE_SIZE #2226" and 
> "[RFC] Use DTS firmware partition to obsolete IMAGE_SIZE #2310"
> 
> https://github.com/openwrt/openwrt/pull/2226
> https://github.com/openwrt/openwrt/pull/2310
> 
> crowd. Because this will likely affect them as well... 
> But they might not know it. In any case: "Welcome everyone! :-D".
> 
> > > What's happening here is that mksquashfs4 is being told
> > > through the "-nopad" option to "do not pad filesystem to a
> > > multiple of 4K" [0].
> > 
> > > |define Image/mkfs/squashfs |
> > > $(STAGING_DIR_HOST)/bin/mksquashfs4 $(call
> > > mkfs_target_dir,$(1)) $@ \ | -nopad -noappend -root-owned \ |
> > > -comp $(SQUASHFSCOMP) $(SQUASHFSOPT) \ | -processors 1 |endef
> > 
> > > My guess is that this affects more than just the MR24 (I'm
> > > looking at you too: pad2jffs and various pad-to $value)
> > > . I've tried tracking down the change that added the "-nopad"
> > > and found it in a commit from 2005 titled: "add some changes
> > > from whiterussian to head" [1] [2]:
> > 
> > I agree that other devices where rootfs is squashfs and lives on a ubi
> > volume are probaby also vulnerable to this problem. Regrettably, I haven't
> > thought of any other of those devices that I have on hand to test. 
> > 
> > > | $(KDIR)/root.squashfs: | @mkdir -p $(KDIR)/root/jffs |-
> > > $(STAGING_DIR)/bin/mksquashfs-lzma $(KDIR)/root $@ -noappend
> > > -root-owned -le |+ $(STAGING_DIR)/bin/mksquashfs-lzma
> > > $(KDIR)/root $@ -nopad -noappend -root-owned -le
> > 
> > 
> > > So, this is really old...
> > 
> > > Question is, should we just drop the -nopad? Since as you
> > > established, in the described corner-case (see above)
> > > squashfs needs this 4k padding and the generated squashfs
> > > could be considered broken otherwise?
> > 
> > I'm under the impression that the -nopad makes sense for NOR flash where
> > the kernel and rootfs are glued together, padding the isolated rootfs
> > would cause alignment problems or wasted space in the combined blobs.
> 
> Yes, that's the nod to padjffs2. That said,
>  makes
> it sound like that apart from the BLOCKSIZE, we also need to PAGE_SIZE?
> 
> (I think the APM821XX is a special case, since it can do 64KiB Pages
> as well as it's 32MiB SLC NAND that uses 16 KiB erase-blocks. So a
> PAGE can span up to 4 pages.
> 
> > 
> > > (Judging from your
> > > message, you went through the kernel code. Can you please
> > > share the place where the lack of the padding is breaking the
> > > fs?)
> > 
> > It's mostly inferred from the fact that I saw the error and kernel
> > panic, and glancing at the kernel squashfs code. I am not pretending to
> > understand completely how the squashfs kernel code works, but the
> > position of the error relative to the size of the rootfs in my panic
> > case strongly suggests it was trying to read past the end of the ubi
> > volume.
> > 
> > The error comes in the kernel's fs/squashfs/block.c in the
> > squashfs_read_data() function. There are a bunch of conditions (at least
> > 5) within the function (see "goto read_failure;") that will lead to that
> > message being printed.
> > 
> Well, that's a pity this could have saved a lot of time.
> 
> I've cobbered together a patch that deals with some of the
> padding issues at "ubimkvol" and "ubinize" time. The idea
> is that ideally we want to do the padding when we know
> PAGE_SIZE and the BLOCKSIZE/Erasesize (MR24 blocksize in
> image/Makefile seems wrong as well...).
> 
> But for now, it's set to 64KiB. If this is the way forward
> we add enable getconf and get the PAGESIZE at runtime. If not,
> we need to come up with