Re: sysupgrade is broken
> On 21.02.2024 20:52, e9hack wrote: > > root@WLAN-DSL9:~# sysupgrade -b /var/config-backup.tar.gz > > Wed Feb 21 20:48:30 CET 2024 upgrade: Saving config files... > > tar: var/dhcp.leases: No such file or directory > > tar: var/lib/logrotate.status: No such file or directory > > tar: var/log/logrotate.log: No such file or directory > > tar: error exit delayed from previous errors > > Failed to create the configuration backup. > > I can reproduce that. The problem is caused by: > mount -t overlay overlay -o > lowerdir=/,upperdir="$tmp/upper",workdir="$tmp/work" "$dir" > > Apparently lowerdir=/ doesn't work as I expected. In $dir I can see > squashfs + overlay changes but I don't see mounts. > > root@OpenWrt:/# ls -l $dir/tmp/ > root@OpenWrt:/# ls -l $dir/rom/ > -rw-r--r--1 root root 116 Feb 19 12:53 note > root@OpenWrt:/# ls -l $dir/dev/ > crw---1 root root5, 1 Feb 19 12:53 console > > I'm not sure if there is an easy way to solve that. Anyone? Easy as "a overlay fs option to follow mounted subfilesystems"? Sorry, AFAIK, no. I commented earlier today in github (https://github.com/openwrt/openwrt/commit/4fa9aaf0bed984d200b3c48d1cc81fca7847c394) In summary, you can overlay every mounted filesystem. Alternatively, you can change the strategy and only overlay the leaf directory you actually need to modify, exclude them from main backup and append the overlay contents. Regards, Luiz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: sysupgrade is broken
On 21.02.2024 20:52, e9hack wrote: root@WLAN-DSL9:~# sysupgrade -b /var/config-backup.tar.gz Wed Feb 21 20:48:30 CET 2024 upgrade: Saving config files... tar: var/dhcp.leases: No such file or directory tar: var/lib/logrotate.status: No such file or directory tar: var/log/logrotate.log: No such file or directory tar: error exit delayed from previous errors Failed to create the configuration backup. I can reproduce that. The problem is caused by: mount -t overlay overlay -o lowerdir=/,upperdir="$tmp/upper",workdir="$tmp/work" "$dir" Apparently lowerdir=/ doesn't work as I expected. In $dir I can see squashfs + overlay changes but I don't see mounts. root@OpenWrt:/# ls -l $dir/tmp/ root@OpenWrt:/# ls -l $dir/rom/ -rw-r--r--1 root root 116 Feb 19 12:53 note root@OpenWrt:/# ls -l $dir/dev/ crw---1 root root5, 1 Feb 19 12:53 console I'm not sure if there is an easy way to solve that. Anyone? ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
sysupgrade is broken
Hi, sysupgrade is broken since commit 4fa9aaf0bed984d200b3c48d1cc81fca7847c394 base-files: sysupgrade: always setup overlay when creating backup If /etc/sysupgrade.conf contains files from /var/..., sysugrade does stop with network down. If I execute sysupgrade -b /tmp/config-backup.tar.gz, sysuggrade complains about existing files from /var/... and doesn't generate backup: root@WLAN-DSL9:~# sysupgrade -b /var/config-backup.tar.gz Wed Feb 21 20:48:30 CET 2024 upgrade: Saving config files... tar: var/dhcp.leases: No such file or directory tar: var/lib/logrotate.status: No such file or directory tar: var/log/logrotate.log: No such file or directory tar: error exit delayed from previous errors Failed to create the configuration backup. All three files does exist. Regards, Hartmut ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- Ok. Thanks for the answers. I was hoping there may be a pattern but this does not seem to be relevant. > On 2 Jan 2020, at 19:27, Petr Štetiar wrote: > > Petr Novák [2020-01-02 18:40:18]: > >> Was the Qemu test Petr S. has done been running a multi-core or single core >> emulation? > > I use 4 cores, 512M RAM as default for the QEMU machines. I'm also testing on > ipq4018, mt7621 and i.MX6 which are multicore. > > -- ynezz --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2020-01-02 18:40:18]: > Was the Qemu test Petr S. has done been running a multi-core or single core > emulation? I use 4 cores, 512M RAM as default for the QEMU machines. I'm also testing on ipq4018, mt7621 and i.MX6 which are multicore. -- ynezz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák kirjoitti 2.1.2020 klo 19.40: Q: are all the platforms where this problem has been observed multi-core (like the RPi4 or mt7621) or has this ever been experienced on a single core system? Was the Qemu test Petr S. has done been running a multi-core or single core emulation? My trusty old WNDR3700v2 is an old single-core device with Atheros AR7161. So, not only a multi-core issue, but also affecting single core devices. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- Q: are all the platforms where this problem has been observed multi-core (like the RPi4 or mt7621) or has this ever been experienced on a single core system? Was the Qemu test Petr S. has done been running a multi-core or single core emulation? Petr > On 2 Jan 2020, at 18:36, Stijn Segers wrote: > > Hannu Nyman schreef op 2 januari 2020 16:48:08 CET: >> Petr Štetiar kirjoitti 1.1.2020 klo 22.46: >>> Petr Novák [2020-01-01 21:11:30]: >>> But how come the workaround was to use an older libubox and ubus - >> was there any new check which was not there before? >>> I don't have definitive answer, as I would need RPi-4 (or any other >> real >>> hardware with Cortex-A72 core) to find the actual bit in the libubox >> which >>> caused this change in the behavior, but here is a part of the commit >>> description[1] which might help answering that: >>> >>> It seems like the recent fixes in the libubox library, particulary >> in >>> the jshn sub-component (which empowers json_dump used in the shell >>> script executed by the child process) made the execution somehow >> faster, >>> thus exposing this racy behaviour in the >> validate_firmware_image_call at >>> least on RPi-4 (Cortex-A72) target. >>> >>> As I was unable to trigger this issue even in the QEMU/Cortex-A72 I >> assume, >>> that it was simply some kind of race, needed specific timing, >> provided >>> preciously only by that RPi-4 hardware. >> >> >> I think that there may have been an older race condition behaviour that >> has >> now just surfaced better with RPi4 after the recent changes. It has >> earlier >> manifested itself sometimes with some routers, but more rarely. >> >> I have seen an occasional failure of sysupgrade in one of my routers >> since >> October (ar71xx or ath79 / WNDR3700v2). > > I've seen the same multiple times on more than one mt7621 device and I opened > FS#2696 on this. > > Granted, not the most verbose bug report. > > Stijn > > > > >> I wrote about that to the >> mailing >> list in November, although then I thought that it might be just a >> "force" >> option failure: >> http://lists.infradead.org/pipermail/openwrt-devel/2019-November/019996.html >> >> Others have seen that also, based on forum discussion: >> https://forum.openwrt.org/t/build-for-wndr3700v1-v2-wndr3800/64/295 >> >> Petr Novak describes similar thing as my error as: "it does just reboot >> but >> does not flash anything." >> >> I have tried to debug that in my WNDR3800 that has serial console >> connection, >> but have not managed to produce the error in that 3800. With 3800 the >> sysupgrade has succeeded always. However, in my 3700v2 (that has >> identical >> hardware except the RAM size) on the other side of the building, I >> still >> occasionally see the behaviour of LuCI based sysupgrade starting ok, >> but the >> router booting back to the same firmware after an invisible error. >> After that >> reboot the next sysupgrade attempt via LuCI usually works quite ok. >> (sounds >> like a sysupgrade from a recently booted system usually works, but >> sysupgrading a system after some runtime does sometimes not work.) >> >> I first thought that it was related to using force in the ar71xx/ath79 >> jump, >> but it has been present in normal sysupgrades. >> >> Possibly a manifestation of the same race condition in >> sysupgrade/procd/libubox, so hopefully your patches will fix also that. >> >> >> >> ___ >> openwrt-devel mailing list >> openwrt-devel@lists.openwrt.org >> https://lists.openwrt.org/mailman/listinfo/openwrt-devel > --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Hannu Nyman schreef op 2 januari 2020 16:48:08 CET: >Petr Štetiar kirjoitti 1.1.2020 klo 22.46: >> Petr Novák [2020-01-01 21:11:30]: >> >>> But how come the workaround was to use an older libubox and ubus - >was there >>> any new check which was not there before? >> I don't have definitive answer, as I would need RPi-4 (or any other >real >> hardware with Cortex-A72 core) to find the actual bit in the libubox >which >> caused this change in the behavior, but here is a part of the commit >> description[1] which might help answering that: >> >> It seems like the recent fixes in the libubox library, particulary >in >> the jshn sub-component (which empowers json_dump used in the shell >> script executed by the child process) made the execution somehow >faster, >> thus exposing this racy behaviour in the >validate_firmware_image_call at >> least on RPi-4 (Cortex-A72) target. >> >> As I was unable to trigger this issue even in the QEMU/Cortex-A72 I >assume, >> that it was simply some kind of race, needed specific timing, >provided >> preciously only by that RPi-4 hardware. > > >I think that there may have been an older race condition behaviour that >has >now just surfaced better with RPi4 after the recent changes. It has >earlier >manifested itself sometimes with some routers, but more rarely. > >I have seen an occasional failure of sysupgrade in one of my routers >since >October (ar71xx or ath79 / WNDR3700v2). I've seen the same multiple times on more than one mt7621 device and I opened FS#2696 on this. Granted, not the most verbose bug report. Stijn > I wrote about that to the >mailing >list in November, although then I thought that it might be just a >"force" >option failure: >http://lists.infradead.org/pipermail/openwrt-devel/2019-November/019996.html > >Others have seen that also, based on forum discussion: >https://forum.openwrt.org/t/build-for-wndr3700v1-v2-wndr3800/64/295 > >Petr Novak describes similar thing as my error as: "it does just reboot >but >does not flash anything." > >I have tried to debug that in my WNDR3800 that has serial console >connection, >but have not managed to produce the error in that 3800. With 3800 the >sysupgrade has succeeded always. However, in my 3700v2 (that has >identical >hardware except the RAM size) on the other side of the building, I >still >occasionally see the behaviour of LuCI based sysupgrade starting ok, >but the >router booting back to the same firmware after an invisible error. >After that >reboot the next sysupgrade attempt via LuCI usually works quite ok. >(sounds >like a sysupgrade from a recently booted system usually works, but >sysupgrading a system after some runtime does sometimes not work.) > >I first thought that it was related to using force in the ar71xx/ath79 >jump, >but it has been present in normal sysupgrades. > >Possibly a manifestation of the same race condition in >sysupgrade/procd/libubox, so hopefully your patches will fix also that. > > > >___ >openwrt-devel mailing list >openwrt-devel@lists.openwrt.org >https://lists.openwrt.org/mailman/listinfo/openwrt-devel ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Štetiar kirjoitti 1.1.2020 klo 22.46: Petr Novák [2020-01-01 21:11:30]: But how come the workaround was to use an older libubox and ubus - was there any new check which was not there before? I don't have definitive answer, as I would need RPi-4 (or any other real hardware with Cortex-A72 core) to find the actual bit in the libubox which caused this change in the behavior, but here is a part of the commit description[1] which might help answering that: It seems like the recent fixes in the libubox library, particulary in the jshn sub-component (which empowers json_dump used in the shell script executed by the child process) made the execution somehow faster, thus exposing this racy behaviour in the validate_firmware_image_call at least on RPi-4 (Cortex-A72) target. As I was unable to trigger this issue even in the QEMU/Cortex-A72 I assume, that it was simply some kind of race, needed specific timing, provided preciously only by that RPi-4 hardware. I think that there may have been an older race condition behaviour that has now just surfaced better with RPi4 after the recent changes. It has earlier manifested itself sometimes with some routers, but more rarely. I have seen an occasional failure of sysupgrade in one of my routers since October (ar71xx or ath79 / WNDR3700v2). I wrote about that to the mailing list in November, although then I thought that it might be just a "force" option failure: http://lists.infradead.org/pipermail/openwrt-devel/2019-November/019996.html Others have seen that also, based on forum discussion: https://forum.openwrt.org/t/build-for-wndr3700v1-v2-wndr3800/64/295 Petr Novak describes similar thing as my error as: "it does just reboot but does not flash anything." I have tried to debug that in my WNDR3800 that has serial console connection, but have not managed to produce the error in that 3800. With 3800 the sysupgrade has succeeded always. However, in my 3700v2 (that has identical hardware except the RAM size) on the other side of the building, I still occasionally see the behaviour of LuCI based sysupgrade starting ok, but the router booting back to the same firmware after an invisible error. After that reboot the next sysupgrade attempt via LuCI usually works quite ok. (sounds like a sysupgrade from a recently booted system usually works, but sysupgrading a system after some runtime does sometimes not work.) I first thought that it was related to using force in the ar71xx/ath79 jump, but it has been present in normal sysupgrades. Possibly a manifestation of the same race condition in sysupgrade/procd/libubox, so hopefully your patches will fix also that. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- OK, fair enough. In any case thanks for helping to fix this, I hope this will be pulled soon into the main repo. Petr > On 2 Jan 2020, at 08:10, Petr Štetiar wrote: > > Petr Novák [2020-01-02 08:01:54]: > >> or perhaps we can ship you a RPi4 and you can return it once you are done >> with it? > > Thanks for the offer, but I've already spent more time on this than planed and > it's not me who want to know the answer :-) > > -- ynezz --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2020-01-02 08:01:54]: > or perhaps we can ship you a RPi4 and you can return it once you are done > with it? Thanks for the offer, but I've already spent more time on this than planed and it's not me who want to know the answer :-) -- ynezz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- or perhaps we can ship you a RPi4 and you can return it once you are done with it? > On 1 Jan 2020, at 21:46, Petr Štetiar wrote: > > Petr Novák [2020-01-01 21:11:30]: > >> But how come the workaround was to use an older libubox and ubus - was there >> any new check which was not there before? > > I don't have definitive answer, as I would need RPi-4 (or any other real > hardware with Cortex-A72 core) to find the actual bit in the libubox which > caused this change in the behavior, but here is a part of the commit > description[1] which might help answering that: > > It seems like the recent fixes in the libubox library, particulary in > the jshn sub-component (which empowers json_dump used in the shell > script executed by the child process) made the execution somehow faster, > thus exposing this racy behaviour in the validate_firmware_image_call at > least on RPi-4 (Cortex-A72) target. > > As I was unable to trigger this issue even in the QEMU/Cortex-A72 I assume, > that it was simply some kind of race, needed specific timing, provided > preciously only by that RPi-4 hardware. > >> actually, it may be visible on the HDMI output - not as flexible as a serial >> console (not so easy to copy paste) but that would allow to see what is >> going on better than the ssh I was using up to now. > > I've prepared a commit[2] which is going to output that error into the syslog > instead, together with more verbose error message[3] so it's easier to track > it down next time. > > 1. > https://gitlab.com/ynezz/openwrt-procd/commit/8904b34203055bdc619cd95581a19b307b91e362 > 2. > https://gitlab.com/ynezz/openwrt-procd/commit/e87ccf2b7ae17faa2dfda470484279c1bfb51328 > 3. > https://gitlab.com/ynezz/openwrt-procd/commit/9e45a44859e81cc84dbc39c42c9dacef30b96429 > > -- ynezz --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- if this is of any help, I can give you access to one of the RPi4s via a remote SSH session > On 1 Jan 2020, at 21:46, Petr Štetiar wrote: > > Petr Novák [2020-01-01 21:11:30]: > >> But how come the workaround was to use an older libubox and ubus - was there >> any new check which was not there before? > > I don't have definitive answer, as I would need RPi-4 (or any other real > hardware with Cortex-A72 core) to find the actual bit in the libubox which > caused this change in the behavior, but here is a part of the commit > description[1] which might help answering that: > > It seems like the recent fixes in the libubox library, particulary in > the jshn sub-component (which empowers json_dump used in the shell > script executed by the child process) made the execution somehow faster, > thus exposing this racy behaviour in the validate_firmware_image_call at > least on RPi-4 (Cortex-A72) target. > > As I was unable to trigger this issue even in the QEMU/Cortex-A72 I assume, > that it was simply some kind of race, needed specific timing, provided > preciously only by that RPi-4 hardware. > >> actually, it may be visible on the HDMI output - not as flexible as a serial >> console (not so easy to copy paste) but that would allow to see what is >> going on better than the ssh I was using up to now. > > I've prepared a commit[2] which is going to output that error into the syslog > instead, together with more verbose error message[3] so it's easier to track > it down next time. > > 1. > https://gitlab.com/ynezz/openwrt-procd/commit/8904b34203055bdc619cd95581a19b307b91e362 > 2. > https://gitlab.com/ynezz/openwrt-procd/commit/e87ccf2b7ae17faa2dfda470484279c1bfb51328 > 3. > https://gitlab.com/ynezz/openwrt-procd/commit/9e45a44859e81cc84dbc39c42c9dacef30b96429 > > -- ynezz --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2020-01-01 21:11:30]: > But how come the workaround was to use an older libubox and ubus - was there > any new check which was not there before? I don't have definitive answer, as I would need RPi-4 (or any other real hardware with Cortex-A72 core) to find the actual bit in the libubox which caused this change in the behavior, but here is a part of the commit description[1] which might help answering that: It seems like the recent fixes in the libubox library, particulary in the jshn sub-component (which empowers json_dump used in the shell script executed by the child process) made the execution somehow faster, thus exposing this racy behaviour in the validate_firmware_image_call at least on RPi-4 (Cortex-A72) target. As I was unable to trigger this issue even in the QEMU/Cortex-A72 I assume, that it was simply some kind of race, needed specific timing, provided preciously only by that RPi-4 hardware. > actually, it may be visible on the HDMI output - not as flexible as a serial > console (not so easy to copy paste) but that would allow to see what is > going on better than the ssh I was using up to now. I've prepared a commit[2] which is going to output that error into the syslog instead, together with more verbose error message[3] so it's easier to track it down next time. 1. https://gitlab.com/ynezz/openwrt-procd/commit/8904b34203055bdc619cd95581a19b307b91e362 2. https://gitlab.com/ynezz/openwrt-procd/commit/e87ccf2b7ae17faa2dfda470484279c1bfb51328 3. https://gitlab.com/ynezz/openwrt-procd/commit/9e45a44859e81cc84dbc39c42c9dacef30b96429 -- ynezz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
yes, I do not have the serial console on the RPi.
But how come the workaround was to use an older libubox and ubus - was there
any new check which was not there before?
> On 1 Jan 2020, at 21:08, Petr Štetiar wrote:
>
> Petr Novák [2020-01-01 18:13:06]:
>
>> 5c6df2def2a1d26313b45cb40e2268a08c59f00 - FAILS
>> 465676c8b12618ec99122d0cc1217994de884284 - WORKS
>> 5bd12760e8fce026fd2a82d56f7b3c8dff27e096 - WORKS
>> 8e195b325a86b4dc99ee5a9ee0b637cf266f9a0a - WORKS
>
> Nice, thanks a lot for your help.
>
>> The procd after forking the validate_firmware_image process does this in the
>> parent (1) process (using strace):
>>
>> clone(child_stack=NULL, flags=SIGCHLD) = 1303
>> rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
>> close(10) = 0
>> read(9, 0x7fff5d1420, 64) = ? ERESTARTSYS (To be restarted if
>> SA_RESTART is set)
>> --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1372, si_uid=0,
>> si_status=1, si_utime=1, si_stime=0} —
>> write(6, "w", 1)= 1
>> rt_sigreturn({mask=[]}) = -1 EINTR (Interrupted system call)
>> close(9)
>
> Yep, but this EINTR leads to the "Failed to parse JSON: 4" which was missing
> in your logs, so I was confused. Then I saw `Connection to 172.30.31.233
> closed by remote host.` in one of your latest logs and I've realized, that
> you're not able to see those messages actually as those are only visible on
> the serial console...
>
> -- ynezz
--- End Message ---
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2020-01-01 18:13:06]:
> 5c6df2def2a1d26313b45cb40e2268a08c59f00 - FAILS
> 465676c8b12618ec99122d0cc1217994de884284 - WORKS
> 5bd12760e8fce026fd2a82d56f7b3c8dff27e096 - WORKS
> 8e195b325a86b4dc99ee5a9ee0b637cf266f9a0a - WORKS
Nice, thanks a lot for your help.
> The procd after forking the validate_firmware_image process does this in the
> parent (1) process (using strace):
>
> clone(child_stack=NULL, flags=SIGCHLD) = 1303
> rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> close(10) = 0
> read(9, 0x7fff5d1420, 64) = ? ERESTARTSYS (To be restarted if
> SA_RESTART is set)
> --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1372, si_uid=0,
> si_status=1, si_utime=1, si_stime=0} —
> write(6, "w", 1)= 1
> rt_sigreturn({mask=[]}) = -1 EINTR (Interrupted system call)
> close(9)
Yep, but this EINTR leads to the "Failed to parse JSON: 4" which was missing
in your logs, so I was confused. Then I saw `Connection to 172.30.31.233
closed by remote host.` in one of your latest logs and I've realized, that
you're not able to see those messages actually as those are only visible on
the serial console...
-- ynezz
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
Here are the results:
1. build with PKG_SOURCE_VERSION:=5ed190aae1b3985719046f4c744e311fc9ef18e3, test
Fails
2. build with PKG_SOURCE_VERSION:=6544e4f1fbdbb92df8a3776e449fdb5602b8ddcd, test
the commit hash does not match any version in
https://gitlab.com/ynezz/openwrt-procd/commits/wip
3. build with PKG_SOURCE_VERSION:=ff03f3ed9b6af8b209dae63f24790664c94bd916, test
the commit hash does not match any version in
https://gitlab.com/ynezz/openwrt-procd/commits/wip
and then:
5c6df2def2a1d26313b45cb40e2268a08c59f00 - FAILS
465676c8b12618ec99122d0cc1217994de884284 - WORKS
5bd12760e8fce026fd2a82d56f7b3c8dff27e096 - WORKS
8e195b325a86b4dc99ee5a9ee0b637cf266f9a0a - WORKS
I did get confused, as when it reports no error but the image is the same as
the previous one, it does just reboot but does not flash anything. but when
using a different image it did actually flash it as expected.
The procd after forking the validate_firmware_image process does this in the
parent (1) process (using strace):
clone(child_stack=NULL, flags=SIGCHLD) = 1303
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
close(10) = 0
read(9, 0x7fff5d1420, 64) = ? ERESTARTSYS (To be restarted if
SA_RESTART is set)
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1372, si_uid=0,
si_status=1, si_utime=1, si_stime=0} —
write(6, "w", 1)= 1
rt_sigreturn({mask=[]}) = -1 EINTR (Interrupted system call)
close(9)
--- End Message ---
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- OK, I have planned to do that anyway, but I have a good reason not to delay that work... > On 1 Jan 2020, at 17:14, Petr Štetiar wrote: > > Petr Novák [2020-01-01 15:14:47]: > >> The updated procd did actually allow the upgrade to proceed: > > Ok, thats interesting. > > Can you find out which commit in procd actually fixes it? It should be enough > to change PKG_SOURCE_VERSION in package/system/procd/Makefile to the > respective hash: > > 1. build with PKG_SOURCE_VERSION:=5ed190aae1b3985719046f4c744e311fc9ef18e3, > test > 2. build with PKG_SOURCE_VERSION:=6544e4f1fbdbb92df8a3776e449fdb5602b8ddcd, > test > 3. build with PKG_SOURCE_VERSION:=ff03f3ed9b6af8b209dae63f24790664c94bd916, > test > > Thanks! > > -- ynezz --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2020-01-01 15:14:47]: > The updated procd did actually allow the upgrade to proceed: Ok, thats interesting. Can you find out which commit in procd actually fixes it? It should be enough to change PKG_SOURCE_VERSION in package/system/procd/Makefile to the respective hash: 1. build with PKG_SOURCE_VERSION:=5ed190aae1b3985719046f4c744e311fc9ef18e3, test 2. build with PKG_SOURCE_VERSION:=6544e4f1fbdbb92df8a3776e449fdb5602b8ddcd, test 3. build with PKG_SOURCE_VERSION:=ff03f3ed9b6af8b209dae63f24790664c94bd916, test Thanks! -- ynezz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
The updated procd did actually allow the upgrade to proceed:
root@OpenWrt:~# sysupgrade -v
openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz
Image not in /tmp, copying...
Reading partition table from bootdisk...
Reading partition table from image...
Saving config files...
boot/config.txt
etc/config/dhcp
etc/config/dropbear
etc/config/firewall
etc/config/network
etc/config/system
etc/config/wireless
etc/dropbear/dropbear_rsa_host_key
etc/group
etc/hosts
etc/inittab
etc/opkg/keys/0b26f36ae0f4106d
etc/opkg/keys/1035ac73cc4e59e3
etc/opkg/keys/5151f69420c3f508
etc/opkg/keys/72a57f2191b211e0
etc/opkg/keys/792d9d9b39f180dc
etc/opkg/keys/9ef4694208102c43
etc/opkg/keys/b2d571e0880ff617
etc/opkg/keys/b5043e70f9a75cde
etc/opkg/keys/c10b9afab19ee428
etc/opkg/keys/dace9d4df16896bf
etc/opkg/keys/dd6de0d06bbd3d85
etc/opkg/keys/f94b9dd6febac963
etc/passwd
etc/profile
etc/rc.local
etc/shadow
etc/shells
etc/sysctl.conf
Commencing upgrade. Closing all shell sessions.
-> 7b93da68 #7b93da68 hello: {}
<- 7b93da68 # lookup: {"objpath":"system"}
-> 7b93da68 # data:
{"objpath":"system","objid":-672961887,"objtype":-575613211,"signature":{"board":{},"info":{},"reboot":{},"watchdog":{"frequency":5,"timeout":5,"magicclose":7,"stop":7},"signal":{"pid":5,"signum":5},"validate_firmware_image":{"path":3},"sysupgrade":{"path":3,"force":7,"backup":3,"prefix":3,"command":3,"options":2}}}
-> 7b93da68 # status: {"status":0}
<- 7b93da68 #d7e36aa1 invoke:
{"objid":-672961887,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/sysupgrade.img","backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}}}
-> 13c8230f #7b93da68 invoke:
{"objid":-672961887,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/sysupgrade.img","backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}},"user":"root","group":"root"}
Connection to 172.30.31.233 closed by remote host.
r11869-a176f8d3ec
Then using this current development snapshot it has failed again, and then
again with just updating procd it has worked correctly. So the problem seems to
be in procd (or interaction of procd with something else - such as libubox).
— PN
> On 1 Jan 2020, at 13:44, Petr Štetiar wrote:
>
> Petr Novák [2019-12-31 14:56:13]:
>
> I was suspecting some issue in jshn, which empowers the json_dump command, but
> it seems, that the JSON passed from /usr/libexec/validate_firmware_image back
> to validate_firmware_image_call() looks correct:
>
>> {
>> "tests": {
>> "fwtool_signature": true,
>> "fwtool_device_match": true
>> },
>> "valid": true,
>> "forceable": true,
>> "allow_backup": true
>> }
>
> but validate_firmware_image_call() somehow doesn't get/parse it and yields:
>
>> {
>> "error": {
>> "message": "Firmware image couldn't be validated"
>> }
>> }
>
> looking at the validate_firmware_image_call() I really don't see anything
> related to
> libubox which would lead to this error, so can you please try again with more
> verbose version of procd[1]? Thanks!
>
> 1.
> https://gitlab.com/ynezz/openwrt/commit/a8db973cc2bcf8d877b939801eba529f29ab7d3a
>
> -- ynezz
--- End Message ---
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2019-12-31 14:56:13]:
I was suspecting some issue in jshn, which empowers the json_dump command, but
it seems, that the JSON passed from /usr/libexec/validate_firmware_image back
to validate_firmware_image_call() looks correct:
> {
> "tests": {
> "fwtool_signature": true,
> "fwtool_device_match": true
> },
> "valid": true,
> "forceable": true,
> "allow_backup": true
> }
but validate_firmware_image_call() somehow doesn't get/parse it and yields:
> {
> "error": {
> "message": "Firmware image couldn't be validated"
> }
> }
looking at the validate_firmware_image_call() I really don't see anything
related to
libubox which would lead to this error, so can you please try again with more
verbose version of procd[1]? Thanks!
1.
https://gitlab.com/ynezz/openwrt/commit/a8db973cc2bcf8d877b939801eba529f29ab7d3a
-- ynezz
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
there you go:
root@OpenWrt:/tmp# sysupgrade -v
openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz
Reading partition table from bootdisk...
Reading partition table from image...
Saving config files...
boot/config.txt
etc/config/dhcp
etc/config/dropbear
etc/config/firewall
etc/config/network
etc/config/system
etc/config/wireless
etc/dropbear/dropbear_rsa_host_key
etc/group
etc/hosts
etc/inittab
etc/opkg/keys/0b26f36ae0f4106d
etc/opkg/keys/1035ac73cc4e59e3
etc/opkg/keys/5151f69420c3f508
etc/opkg/keys/72a57f2191b211e0
etc/opkg/keys/792d9d9b39f180dc
etc/opkg/keys/9ef4694208102c43
etc/opkg/keys/b2d571e0880ff617
etc/opkg/keys/b5043e70f9a75cde
etc/opkg/keys/c10b9afab19ee428
etc/opkg/keys/dace9d4df16896bf
etc/opkg/keys/dd6de0d06bbd3d85
etc/opkg/keys/f94b9dd6febac963
etc/passwd
etc/profile
etc/rc.local
etc/shadow
etc/shells
etc/sysctl.conf
Commencing upgrade. Closing all shell sessions.
{
"error": {
"message": "Firmware image couldn't be validated"
}
}
Command failed: Unknown error
root@OpenWrt:/tmp# ubus monitor &
root@OpenWrt:/tmp# -> d3e9aa79 #0003 status: {"status":0}
root@OpenWrt:/tmp# sysupgrade -v
openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz
Reading partition table from bootdisk...
Reading partition table from image...
Saving config files...
boot/config.txt
etc/config/dhcp
etc/config/dropbear
etc/config/firewall
etc/config/network
etc/config/system
etc/config/wireless
etc/dropbear/dropbear_rsa_host_key
etc/group
etc/hosts
etc/inittab
etc/opkg/keys/0b26f36ae0f4106d
etc/opkg/keys/1035ac73cc4e59e3
etc/opkg/keys/5151f69420c3f508
etc/opkg/keys/72a57f2191b211e0
etc/opkg/keys/792d9d9b39f180dc
etc/opkg/keys/9ef4694208102c43
etc/opkg/keys/b2d571e0880ff617
etc/opkg/keys/b5043e70f9a75cde
etc/opkg/keys/c10b9afab19ee428
etc/opkg/keys/dace9d4df16896bf
etc/opkg/keys/dd6de0d06bbd3d85
etc/opkg/keys/f94b9dd6febac963
etc/passwd
etc/profile
etc/rc.local
etc/shadow
etc/shells
etc/sysctl.conf
Commencing upgrade. Closing all shell sessions.
-> 827c4bd9 #827c4bd9 hello: {}
<- 827c4bd9 # lookup: {"objpath":"system"}
-> 827c4bd9 # data:
{"objpath":"system","objid":177377387,"objtype":1024582224,"signature":{"board":{},"info":{},"reboot":{},"watchdog":{"frequency":5,"timeout":5,"magicclose":7,"stop":7},"signal":{"pid":5,"signum":5},"validate_firmware_image":{"path":3},"sysupgrade":{"path":3,"force":7,"backup":3,"prefix":3,"command":3,"options":2}}}
-> 827c4bd9 # status: {"status":0}
<- 827c4bd9 #0a92906b invoke:
{"objid":177377387,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz","backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}}}
-> 06657534 #827c4bd9 invoke:
{"objid":177377387,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz","backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}},"user":"root","group":"root"}
{
"error": {
"message": "Firmware image couldn't be validated"
}
}
Command failed: Unknown error
<- 06657534 #827c4bd9 data:
{"objid":177377387,"data":{"error":{"message":"Firmware image couldn't be
validated"}}}
-> 827c4bd9 #0a92906b data:
{"objid":177377387,"data":{"error":{"message":"Firmware image couldn't be
validated"}}}
<- 06657534 #827c4bd9 status: {"status":9,"objid":177377387}
-> 827c4bd9 #0a92906b status: {"status":9,"objid":177377387}
cat /tmp/vfi-json.log
{
"tests": {
"fwtool_signature": true,
"fwtool_device_match": true
},
"valid": true,
"forceable": true,
"allow_backup": true
}
{
"tests": {
"fwtool_signature": true,
"fwtool_device_match": true
},
"valid": true,
"forceable": true,
"allow_backup": true
}
{
"tests": {
"fwtool_signature": true,
"fwtool_device_match": true
},
"valid": true,
"forceable": true,
"allow_backup": true
}
{
"tests": {
"fwtool_signature": true,
"fwtool_device_match": true
},
"valid": true,
"forceable": true,
"allow_backup": true
}
> On 31 Dec 2019, at 14:49, Petr Štetiar wrote:
>
> Petr Novák [2019-12-31 11:03:23]:
>
>> root@OpenWrt:/tmp# sysupgrade -v
>> openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz
>> Reading partition table from bootdisk...
>> Reading partition table from image...
>
> This output is
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2019-12-31 11:03:23]:
> root@OpenWrt:/tmp# sysupgrade -v
> openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz
> Reading partition table from bootdisk...
> Reading partition table from image...
This output is from /sbin/sysupgrade calling
/usr/libexec/validate_firmware_image:
> {
> "tests": {
> "fwtool_signature": true,
> "fwtool_device_match": true
> },
> "valid": true,
> "forceable": true,
> "allow_backup": true
> }
> Saving config files...
...snip...
> Commencing upgrade. Closing all shell sessions.
and here should be same output from procd calling
/usr/libexec/validate_firmware_image, but it is missing as probably procd
consumes the stderr output, so probably something like this is needed:
diff --git a/package/base-files/files/usr/libexec/validate_firmware_image
b/package/base-files/files/usr/libexec/validate_firmware_image
index f85fb9e4b435..754d53b2edfe 100755
--- a/package/base-files/files/usr/libexec/validate_firmware_image
+++ b/package/base-files/files/usr/libexec/validate_firmware_image
@@ -62,5 +62,6 @@ json_init
json_add_boolean valid "$VALID"
json_add_boolean forceable "$FORCEABLE"
json_add_boolean allow_backup "$ALLOW_BACKUP"
+json_dump -i >> /tmp/vfi-json.log
json_dump -i
json_set_namespace $old_ns
and then run:
$ ubus monitor &
$ sysupgrade -v openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz
$ cat /tmp/vfi-json.log
-- ynezz
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
sorry, using a pre-loaded image this time:
root@OpenWrt:/tmp# sysupgrade -v
openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz
Reading partition table from bootdisk...
Reading partition table from image...
{
"tests": {
"fwtool_signature": true,
"fwtool_device_match": true
},
"valid": true,
"forceable": true,
"allow_backup": true
}
Saving config files...
boot/config.txt
etc/config/dhcp
etc/config/dropbear
etc/config/firewall
etc/config/network
etc/config/system
etc/config/wireless
etc/dropbear/dropbear_rsa_host_key
etc/group
etc/hosts
etc/inittab
etc/opkg/keys/0b26f36ae0f4106d
etc/opkg/keys/1035ac73cc4e59e3
etc/opkg/keys/5151f69420c3f508
etc/opkg/keys/72a57f2191b211e0
etc/opkg/keys/792d9d9b39f180dc
etc/opkg/keys/9ef4694208102c43
etc/opkg/keys/b2d571e0880ff617
etc/opkg/keys/b5043e70f9a75cde
etc/opkg/keys/c10b9afab19ee428
etc/opkg/keys/dace9d4df16896bf
etc/opkg/keys/dd6de0d06bbd3d85
etc/opkg/keys/f94b9dd6febac963
etc/passwd
etc/profile
etc/rc.local
etc/shadow
etc/shells
etc/sysctl.conf
Commencing upgrade. Closing all shell sessions.
-> d4b74aaa #d4b74aaa hello: {}
<- d4b74aaa # lookup: {"objpath":"system"}
-> d4b74aaa # data:
{"objpath":"system","objid":177377387,"objtype":1024582224,"signature":{"board":{},"info":{},"reboot":{},"watchdog":{"frequency":5,"timeout":5,"magicclose":7,"stop":7},"signal":{"pid":5,"signum":5},"validate_firmware_image":{"path":3},"sysupgrade":{"path":3,"force":7,"backup":3,"prefix":3,"command":3,"options":2}}}
-> d4b74aaa # status: {"status":0}
<- d4b74aaa #0a92906b invoke:
{"objid":177377387,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz","backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}}}
-> 06657534 #d4b74aaa invoke:
{"objid":177377387,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz","backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}},"user":"root","group":"root"}
<- 06657534 #d4b74aaa data:
{"objid":177377387,"data":{"error":{"message":"Firmware image couldn't be
validated"}}}
{
"error": {
"message": "Firmware image couldn't be validated"
}
}
-> d4b74aaa #0a92906b data:
{"objid":177377387,"data":{"error":{"message":"Firmware image couldn't be
validated"}}}
Command failed: Unknown error
<- 06657534 #d4b74aaa status: {"status":9,"objid":177377387}
-> d4b74aaa #0a92906b status: {"status":9,"objid":177377387}
— PN
> On 31 Dec 2019, at 10:58, Petr Štetiar wrote:
>
> Petr Novák [2019-12-31 09:42:53]:
>
>> root@OpenWrt:~# sysupgrade -v
>> https://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz
>> wget: SSL support not available, please install one of the
>> libustream-.*[ssl|tls] packages as well as the ca-bundle and ca-certificates
>> packages.
>
> use http:// or install the recommended packages in order to be able to use
> https://
>
>> Image metadata not found
>> Use sysupgrade -F to override this check when downgrading or flashing to
>> vendor firmware
>> Reading partition table from bootdisk...
>> Reading partition table from image...
>> Invalid partition table on /tmp/image.bs
>> Failed to parse message data
>> sh: out of range
>
> The download of firmware image has failed, but the sysupgrade process
> continues, so this errors are "expected", following patch[1] should fix some
> of this.
>
> 1. https://git.openwrt.org/6dbaf71411e7f16cc1c920b6a63bbe33ef2b8921
>
> -- ynezz
--- End Message ---
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2019-12-31 09:42:53]: > root@OpenWrt:~# sysupgrade -v > https://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz > wget: SSL support not available, please install one of the > libustream-.*[ssl|tls] packages as well as the ca-bundle and ca-certificates > packages. use http:// or install the recommended packages in order to be able to use https:// > Image metadata not found > Use sysupgrade -F to override this check when downgrading or flashing to > vendor firmware > Reading partition table from bootdisk... > Reading partition table from image... > Invalid partition table on /tmp/image.bs > Failed to parse message data > sh: out of range The download of firmware image has failed, but the sysupgrade process continues, so this errors are "expected", following patch[1] should fix some of this. 1. https://git.openwrt.org/6dbaf71411e7f16cc1c920b6a63bbe33ef2b8921 -- ynezz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
here is the output from the sysupgrade with the additional change and ubus
monitor as suggested. This is tested with the development snapshot from the
30th of December: r11832-c48b571ad7
root@OpenWrt:~# sysupgrade -v
https://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-ext4-sysupgrade.img.gz
wget: SSL support not available, please install one of the
libustream-.*[ssl|tls] packages as well as the ca-bundle and ca-certificates
packages.
Image metadata not found
Use sysupgrade -F to override this check when downgrading or flashing to vendor
firmware
Reading partition table from bootdisk...
Reading partition table from image...
Invalid partition table on /tmp/image.bs
Failed to parse message data
sh: out of range
Saving config files...
boot/config.txt
etc/config/dhcp
etc/config/dropbear
etc/config/firewall
etc/config/network
etc/config/system
etc/config/wireless
etc/dropbear/dropbear_rsa_host_key
etc/group
etc/hosts
etc/inittab
etc/opkg/keys/0b26f36ae0f4106d
etc/opkg/keys/1035ac73cc4e59e3
etc/opkg/keys/5151f69420c3f508
etc/opkg/keys/72a57f2191b211e0
etc/opkg/keys/792d9d9b39f180dc
etc/opkg/keys/9ef4694208102c43
etc/opkg/keys/b2d571e0880ff617
etc/opkg/keys/b5043e70f9a75cde
etc/opkg/keys/c10b9afab19ee428
etc/opkg/keys/dace9d4df16896bf
etc/opkg/keys/dd6de0d06bbd3d85
etc/opkg/keys/f94b9dd6febac963
etc/passwd
etc/profile
etc/rc.local
etc/shadow
etc/shells
etc/sysctl.conf
Commencing upgrade. Closing all shell sessions.
-> e8d7994a #e8d7994a hello: {}
<- e8d7994a # lookup: {"objpath":"system"}
-> e8d7994a # data:
{"objpath":"system","objid":619517119,"objtype":1675602203,"signature":{"board":{},"info":{},"reboot":{},"watchdog":{"frequency":5,"timeout":5,"magicclose":7,"stop":7},"signal":{"pid":5,"signum":5},"validate_firmware_image":{"path":3},"sysupgrade":{"path":3,"force":7,"backup":3,"prefix":3,"command":3,"options":2}}}
-> e8d7994a # status: {"status":0}
<- e8d7994a #24ed14bf invoke:
{"objid":619517119,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/sysupgrade.img","backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}}}
-> 35669f7b #e8d7994a invoke:
{"objid":619517119,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/sysupgrade.img","backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}},"user":"root","group":"root"}
<- 35669f7b #e8d7994a data:
{"objid":619517119,"data":{"error":{"message":"Firmware image couldn't be
validated"}}}
{
"error": {
"message": "Firmware image couldn't be validated"
}
}
-> e8d7994a #24ed14bf data:
{"objid":619517119,"data":{"error":{"message":"Firmware image couldn't be
validated"}}}
Command failed: Unknown error
<- 35669f7b #e8d7994a status: {"status":9,"objid":619517119}
-> e8d7994a #24ed14bf status: {"status":9,"objid":619517119}
root@OpenWrt:~#
> On 30 Dec 2019, at 23:56, Petr Štetiar wrote:
>
> Petr Novák [2019-12-30 20:43:36]:
>
> Hi,
>
>> I will do my best to reproduce the issue giving more details, I will post
>> any more details here tomorrow.
>
> if I may, can you do following for the start:
>
> 1. add following change (changing it directly on device right before step 2.
> should be enough):
>
> diff --git a/package/base-files/files/usr/libexec/validate_firmware_image
> b/package/base-files/files/usr/libexec/validate_firmware_image
> index f85fb9e4b435..aed9cdfd64b5 100755
> --- a/package/base-files/files/usr/libexec/validate_firmware_image
> +++ b/package/base-files/files/usr/libexec/validate_firmware_image
> @@ -62,5 +62,6 @@ json_init
> json_add_boolean valid "$VALID"
> json_add_boolean forceable "$FORCEABLE"
> json_add_boolean allow_backup "$ALLOW_BACKUP"
> +json_dump -i >&2
> json_dump -i
> json_set_namespace $old_ns
>
> 2. run:
>
> $ ubus monitor &
> $ sysupgrade -v openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz
>
> and send the output, thanks!
>
> -- ynezz
--- End Message ---
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák [2019-12-30 20:43:36]: Hi, > I will do my best to reproduce the issue giving more details, I will post > any more details here tomorrow. if I may, can you do following for the start: 1. add following change (changing it directly on device right before step 2. should be enough): diff --git a/package/base-files/files/usr/libexec/validate_firmware_image b/package/base-files/files/usr/libexec/validate_firmware_image index f85fb9e4b435..aed9cdfd64b5 100755 --- a/package/base-files/files/usr/libexec/validate_firmware_image +++ b/package/base-files/files/usr/libexec/validate_firmware_image @@ -62,5 +62,6 @@ json_init json_add_boolean valid "$VALID" json_add_boolean forceable "$FORCEABLE" json_add_boolean allow_backup "$ALLOW_BACKUP" +json_dump -i >&2 json_dump -i json_set_namespace $old_ns 2. run: $ ubus monitor & $ sysupgrade -v openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz and send the output, thanks! -- ynezz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- Hi Petr, thanks a lot for the effort. I will do my best to reproduce the issue giving more details, I will post any more details here tomorrow. PN > On 30 Dec 2019, at 20:07, Petr Štetiar wrote: > > Petr Štetiar [2019-12-29 23:21:23]: > >> So perhaps this is something Cortex-A72 related? > > I've just tried it under QEMU 4.2.50 with: > > * machine: virt > * cpu: cortex-a72 > * rootfs: > http://downloads.openwrt.org/snapshots/targets/armvirt/64/openwrt-armvirt-64-rootfs-squashfs.img.gz >(OpenWrt SNAPSHOT, r11829-e3e939d8e6) > > and the result is same as on x86/64, I'm unable to reproduce the issue: > > root@OpenWrt:/# sysupgrade > http://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz > Downloading > 'http://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz' > Connecting to 176.9.48.73:80 > Writing to '/tmp/sysupgrade.img' > /tmp/sysupgrade.img 100% |***| 12067k 0:00:00 > ETA > Download completed (12357050 bytes) > Device linux,dummy-virt not supported by this image > Supported devices: raspberrypi,4-model-b > Image check failed. > > root@OpenWrt:/# echo raspberrypi,4-model-b > /tmp/sysinfo/board_name > > root@OpenWrt:/# sysupgrade /tmp/sysupgrade.img > Saving config files... > Commencing upgrade. Closing all shell sessions. > killall: telnetd: no process killed > Sending TERM to remaining processes ... dnsmasq netifd odhcpd ntpd ubusd > urngd logd > Sending KILL to remaining processes ... > Switching to ramdisk... > ...snip... > > So I'm wondering what is going on there. > > -- ynezz --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Štetiar [2019-12-29 23:21:23]: > So perhaps this is something Cortex-A72 related? I've just tried it under QEMU 4.2.50 with: * machine: virt * cpu: cortex-a72 * rootfs: http://downloads.openwrt.org/snapshots/targets/armvirt/64/openwrt-armvirt-64-rootfs-squashfs.img.gz (OpenWrt SNAPSHOT, r11829-e3e939d8e6) and the result is same as on x86/64, I'm unable to reproduce the issue: root@OpenWrt:/# sysupgrade http://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz Downloading 'http://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz' Connecting to 176.9.48.73:80 Writing to '/tmp/sysupgrade.img' /tmp/sysupgrade.img 100% |***| 12067k 0:00:00 ETA Download completed (12357050 bytes) Device linux,dummy-virt not supported by this image Supported devices: raspberrypi,4-model-b Image check failed. root@OpenWrt:/# echo raspberrypi,4-model-b > /tmp/sysinfo/board_name root@OpenWrt:/# sysupgrade /tmp/sysupgrade.img Saving config files... Commencing upgrade. Closing all shell sessions. killall: telnetd: no process killed Sending TERM to remaining processes ... dnsmasq netifd odhcpd ntpd ubusd urngd logd Sending KILL to remaining processes ... Switching to ramdisk... ...snip... So I'm wondering what is going on there. -- ynezz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák via openwrt-devel [2019-12-29
11:49:36]:
Hi,
thanks for reporting it and sorry for the breakage.
> Reading partition table from bootdisk...
> Reading partition table from image...
> Saving config files...
> Commencing upgrade. Closing all shell sessions.
> {
> "error": {
> "message": "Firmware image couldn't be validated"
> }
> }
> Command failed: Unknown error
Although I dont own RPi4 (or anything else with Cortex-A72), I've just tried to
reproduce it on x86/64 QEMU as it should be poor man's "similar enough" target
(64-bit/little endian), but I've failed to reproduce it with following steps:
1. I've copied target/linux/brcm2708/base-files/lib/upgrade/platform.sh to
target/linux/x86/base-files/lib/upgrade/platform.sh in order to get same
sysupgrade functionality of rpi-4 on x86/64
2. build image and run it in QEMU
3. inside QEMU I did following:
... snip ...
OpenWrt SNAPSHOT, r11829+1-e3e939d8e624 (that +1 commit is my local CI
staging tree build commit[1])
... snip ...
root@OpenWrt:/# sysupgrade
http://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz
Downloading
'http://downloads.openwrt.org/snapshots/targets/brcm2708/bcm2711/openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz'
Connecting to 176.9.48.73:80
Writing to '/tmp/sysupgrade.img'
/tmp/sysupgrade.img 100% |***| 12067k 0:00:00
ETA
Download completed (12357027 bytes)
Device qemu-standard-pc-i440fx-piix-1996 not supported by this image
Supported devices: raspberrypi,4-model-b
Reading partition table from bootdisk...
zcat: write error: Broken pipe
zcat: write: Broken pipe
Reading partition table from image...
Partition layout has changed. Full image will be written.
Image check failed.
root@OpenWrt:/# cat /tmp/sysupgrade.meta
{ "metadata_version": "1.0", "supported_devices":["raspberrypi,4-model-b"],
"version": { "dist": "OpenWrt", "version": "SNAPSHOT", "revision":
"r11829-e3e939d8e6", "target": "brcm2708/bcm2711", "board": "rpi-4" } }
root@OpenWrt:/# sha256sum /tmp/sysupgrade.img
5f809879b0d9a0791cd9329997c944a1048cd6b5f0cf59aa5fc34b5381fefb1c
/tmp/sysupgrade.img
root@OpenWrt:/# echo raspberrypi,4-model-b > /tmp/sysinfo/board_name
root@OpenWrt:/# sysupgrade /tmp/sysupgrade.img
Reading partition table from bootdisk...
zcat: write error: Broken pipe
zcat: write: Broken pipe
Reading partition table from image...
Partition layout has changed. Full image will be written.
Saving config files...
Commencing upgrade. Closing all shell sessions.
Failed to parse JSON: 4
killall: telnetd: no process killed
Sending TERM to remaining processes ... ubusd askfirst urngd logd netifd
odhcpd ntpd dnsmasq
Sending KILL to remaining processes ...
... snip ...
(BTW I've fix[2] for that "harmless" `Failed to parse JSON: 4` error message
in my procd repo.)
So perhaps this is something Cortex-A72 related?
1.
https://gitlab.com/ynezz/openwrt/commit/f0c4d73a9c36a54a36af019bd3b3e5aeb1bffcf4
2.
https://gitlab.com/ynezz/openwrt-procd/commit/ff03f3ed9b6af8b209dae63f24790664c94bd916
-- ynezz
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- Hannu was correct - the root cause are the libubox and ubus updates done recently. Taking the r11829-e3e939d8e6 as a baseline and rolling back ubus and libubox to versions before the 2019-12-26 does fix the issue. I have previously tried only to roll back the ubus but that did not work - but doing both does fix it. > On 29 Dec 2019, at 13:13, Hannu Nyman wrote: > > Petr Novák kirjoitti 29.12.2019 klo 13.49: >> I am normally building my own images, but to make sure this is easy to >> reproduce, I have recreated the problem with the most recent snapshot builds >> as well. >> > > Can you be explicit with "most recent"? > You mean r11829-e3e939d8e6 images that contain the last fix for libubox? > > > Ps. Just as reference, so far I have not seen the sysupgrade problem myself. > I have already successfully sysupgraded away from r11829-e3e939d8e6 to a > 19.07 build both on R7800 and on WNDR3700v2 > > --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- So rolling back ubus to the version before 2019-12-26 did not seem to fix the problem. --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- > On 29 Dec 2019, at 13:13, Hannu Nyman wrote: > > Petr Novák kirjoitti 29.12.2019 klo 13.49: >> I am normally building my own images, but to make sure this is easy to >> reproduce, I have recreated the problem with the most recent snapshot builds >> as well. >> > > Can you be explicit with "most recent"? > You mean r11829-e3e939d8e6 images that contain the last fix for libubox? > Yes, correct, I did a pull from git this morning and I have built an image and the problem seems to persist there - the logread issue IS fixed. > > Ps. Just as reference, so far I have not seen the sysupgrade problem myself. > I have already successfully sysupgraded away from r11829-e3e939d8e6 to a > 19.07 build both on R7800 and on WNDR3700v This may be something specific to rpi4 or the way the upgrade is done on that platform. I am now trying to make a build based on the e3e… but with rolling back ubus to the version before the 26th of December. Looking at the sysupgrade code, the ubus seems like a most likely root cause, but I want to make sure this is the case. Update in 10 mins once the build is finished and image loaded to card… --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
Petr Novák kirjoitti 29.12.2019 klo 13.49: I am normally building my own images, but to make sure this is easy to reproduce, I have recreated the problem with the most recent snapshot builds as well. Can you be explicit with "most recent"? You mean r11829-e3e939d8e6 images that contain the last fix for libubox? Ps. Just as reference, so far I have not seen the sysupgrade problem myself. I have already successfully sysupgraded away from r11829-e3e939d8e6 to a 19.07 build both on R7800 and on WNDR3700v2 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- I was also suffering from this issue Hannu has reported (logread hanging in my case), so I have tested with the most recent fix and while the logread issue is fixed, the sysupgrade issue is not. I have read the discussion about this problem before my post, sorry, perhaps I should have added that as well. I am normally building my own images, but to make sure this is easy to reproduce, I have recreated the problem with the most recent snapshot builds as well. Regarding the suggested -F workaround, that is the first thing I have tried, but it actually behaves the same, so the problem seems to be deeper and the -F does not bypass this. --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
> The problem does not seem to be with the image - older builds upgrade to the same image just fine, but the recent ones seem to fail. > > Example: taking the most recent rpi-4-squashfs-factory.img.gz from 2019-12-28 and trying to upgrade to rpi-4-squashfs-sysupgrade.img.gz from the same date does fail as indicated above. Pure guess, but your symptoms might be due to the broken ubus and/or libubox between 2019-12-26 and 2019-12-29. (ubus was broken on 26th and was fixed on 28th, libubox was broken on 26th and was fixed on 29th evening) As sysupgrade uses ubus messaging using blobmsg (from libubox) for image validation messaging, it is quite possible that your symptoms are caused by those. Your image of 28th has broken functionality. I am not sure sure what is the easiest way to sysupgrade away from the broken firmwares. Maybe using "sysupgrade -F" (--force) to bypass validation? ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] Sysupgrade possibly broken in recent development snapshots: "message": "Firmware image couldn't be validated"
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
I seem to be experiencing a problem in the recent development snapshots of
OpenWRT (running on RPi4): the sysupgrade does fail to upgrade the image.
it looks like this:
# sysupgrade openwrt-brcm2708-bcm2711-rpi-4-squashfs-sysupgrade.img.gz
Reading partition table from bootdisk...
Reading partition table from image...
Saving config files...
Commencing upgrade. Closing all shell sessions.
{
"error": {
"message": "Firmware image couldn't be validated"
}
}
Command failed: Unknown error
The problem does not seem to be with the image - older builds upgrade to the
same image just fine, but the recent ones seem to fail.
Example: taking the most recent rpi-4-squashfs-factory.img.gz from 2019-12-28
and trying to upgrade to rpi-4-squashfs-sysupgrade.img.gz from the same date
does fail as indicated above.
Thanks for any hint with a workaround or a fix.
Petr
--- End Message ---
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] sysupgrade --force broken in master?
I noticed that the sysupgrade script seems to fail to force a sysupgrade.
Trying to sysupgrade WNDR3700v2
from ar71xx r11464-cabaaf06fe
to ath79 r11469-db09335848
I tested with console sysupgrade, and that failed, too. Router reboots the
current firmware without flashing. The ssh console output shows how the
router starts the sysupgrade and notes "--force" but then no actual
sysupgrade happens, instead the router reboots and returns to the existing
firmware.
I do not current have serial console connected, so this debugging below has
been done just via SSH console
SSH console output:
OpenWrt SNAPSHOT, r11464-cabaaf06fe
-
root@router2:~# sysupgrade -v -F /tmp/WNDR3700v2-master-r11469-db09335848-201911
10-2134-sqfs-sysupgrade.bin
Device wndr3700 not supported by this image
Supported devices: netgear,wndr3700v2 wndr3700v2
Image check failed but --force given - will update anyway!
Saving config files...
etc/collectd.conf
etc/config/adblock
...
etc/uhttpd.key
etc/uhttpd.crt
Commencing upgrade. Closing all shell sessions.
...
REBOOT / reconnect ssh console
...
BusyBox v1.31.1 () built-in shell (ash)
___ __
| |.-.-.-.| | | |..| |_
| - || _ | -__| || | | || _|| _|
|___|| __|_|__|__||||__| ||
|__| W I R E L E S S F R E E D O M
-
OpenWrt SNAPSHOT, r11464-cabaaf06fe
-
==
"ubus monitor" from other SSH console:
root@router2:~# ubus monitor
-> ebae1309 #0003 status: {"status":0}
-> aae57010 #aae57010 hello: {}
<- aae57010 # lookup: {"objpath":"system"}
-> aae57010 # data:
{"objpath":"system","objid":626662651,"objtype":-172383134,"signature":{"board":{},"info":{},"reboot":{},"watchdog":{"frequency":5,"timeout":5,"magicclose":7,"stop":7},"signal":{"pid":5,"signum":5},"validate_firmware_image":{"path":3},"sysupgrade":{"path":3,"force":7,"backup":3,"prefix":3,"command":3,"options":2}}}
-> aae57010 # status: {"status":0}
<- aae57010 #255a1cfb invoke:
{"objid":626662651,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/WNDR3700v2-master-r11469-db09335848-20191110-2134-sqfs-sysupgrade.bin","force":true,"backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}}}
-> 4aede19c #aae57010 invoke:
{"objid":626662651,"method":"sysupgrade","data":{"prefix":"/tmp/root","path":"/tmp/WNDR3700v2-master-r11469-db09335848-20191110-2134-sqfs-sysupgrade.bin","force":true,"backup":"/tmp/sysupgrade.tgz","command":"/lib/upgrade/do_stage2","options":{"save_partitions":1}},"user":"root","group":"root"}
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH procd] system: reject sysupgrade of broken firmware images
On Tue, 3 Sep 2019 at 18:57, Luis Araneda wrote: > On Sun, Sep 1, 2019 at 5:09 AM Rafał Miłecki wrote: > > On Sun, 1 Sep 2019 at 06:13, Reiner Karlsberg > > wrote: > > > This needs to be handled very carefully, not to break > > > actual usage of -F. > > > I had to use -F couple of times, usually when downgrading > > > installed firmware, but with change of name over time. > > > Typical example: Change of image name for the zbt-we826. > > > Never any problem with usage of -F, though. > > > > > > But I had several problems with non-completion of > > > valid sysupgrade, which even left the system in inconsistent state, > > > as the "-f keep.tar.gz" was applied, but not the new image itself. > > > Or (silently ?) no sysupgrade done, because of mounted block device > > > or active swap file on block device, or active wifi (?). > > > Which was a PITA on remote installations. > > > > > > Question: How are sysupgrade-errors reported/to be handled, as usually > > > also a failed sysupgrade > > > triggered a reboot ? > > > documentation very unclear here, as it looks like, behaviour in case of > > > errro changed during versions of openwrt. > > > > > > Best would be "atomic sysupgrade", with standard error-code (+1) on exit > > > instead of expected reboot after success. > > > > > > I appreciate the new work on sysupgrade, but I am a bit afraid of > > > regressions. > > > > No behavior will change until you explicitly modify your target's > > /lib/upgrade/platform.sh to start calling notify_firmware_broken() for > > whatever reason (e.g. unrecognized firmware image format). > > > > I'm planning to implement more verbose sysupgrade results later. I was > > thinking about ubus method replying with a JSON containing error code > > and message. I should prepare some patch for that in a next week or > > two. > > Since you're improving sysupgrade to reject some images, I'm throwing an idea > I had some time ago (no code, sorry). > > I would be great if there is support for certain sysupgrade images to require > a settings reset (without preserving them). > The idea is to support some use cases were preserving the settings might > leave the device in a sofbrick / misconfigured state. So a reset to defaults > is mandatory, like the recent situation when migrating a configuration from > swconfig to DSA. > Sure, it could be done by migrations scripts, but they might not be 100% > reliable (cover all possible variations). > > On the implementation side, it could be done using image metadata to store an > integer with the minimum version required, which could be used by sysupgrade > to compare the locally stored value and check if a reset is mandatory or not. > (this is just one possible implementation) > > Of course, an implementation would not be useful for the current issue of > swconfig->DSA, but it might be useful in the future (who knows what might > break). I see some use-cases for that. I'll implement that. -- Rafał ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH procd] system: reject sysupgrade of broken firmware images
Hi Rafał, On Sun, Sep 1, 2019 at 5:09 AM Rafał Miłecki wrote: > On Sun, 1 Sep 2019 at 06:13, Reiner Karlsberg wrote: > > This needs to be handled very carefully, not to break > > actual usage of -F. > > I had to use -F couple of times, usually when downgrading > > installed firmware, but with change of name over time. > > Typical example: Change of image name for the zbt-we826. > > Never any problem with usage of -F, though. > > > > But I had several problems with non-completion of > > valid sysupgrade, which even left the system in inconsistent state, > > as the "-f keep.tar.gz" was applied, but not the new image itself. > > Or (silently ?) no sysupgrade done, because of mounted block device > > or active swap file on block device, or active wifi (?). > > Which was a PITA on remote installations. > > > > Question: How are sysupgrade-errors reported/to be handled, as usually also a failed sysupgrade > > triggered a reboot ? > > documentation very unclear here, as it looks like, behaviour in case of errro changed during versions of openwrt. > > > > Best would be "atomic sysupgrade", with standard error-code (+1) on exit instead of expected reboot after success. > > > > I appreciate the new work on sysupgrade, but I am a bit afraid of regressions. > > No behavior will change until you explicitly modify your target's > /lib/upgrade/platform.sh to start calling notify_firmware_broken() for > whatever reason (e.g. unrecognized firmware image format). > > I'm planning to implement more verbose sysupgrade results later. I was > thinking about ubus method replying with a JSON containing error code > and message. I should prepare some patch for that in a next week or > two. Since you're improving sysupgrade to reject some images, I'm throwing an idea I had some time ago (no code, sorry). I would be great if there is support for certain sysupgrade images to require a settings reset (without preserving them). The idea is to support some use cases were preserving the settings might leave the device in a sofbrick / misconfigured state. So a reset to defaults is mandatory, like the recent situation when migrating a configuration from swconfig to DSA. Sure, it could be done by migrations scripts, but they might not be 100% reliable (cover all possible variations). On the implementation side, it could be done using image metadata to store an integer with the minimum version required, which could be used by sysupgrade to compare the locally stored value and check if a reset is mandatory or not. (this is just one possible implementation) Of course, an implementation would not be useful for the current issue of swconfig->DSA, but it might be useful in the future (who knows what might break). Regards, Luis Araneda. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH procd] system: reject sysupgrade of broken firmware images
On Sun, 1 Sep 2019 at 06:13, Reiner Karlsberg wrote: > This needs to be handled very carefully, not to break > actual usage of -F. > I had to use -F couple of times, usually when downgrading > installed firmware, but with change of name over time. > Typical example: Change of image name for the zbt-we826. > Never any problem with usage of -F, though. > > But I had several problems with non-completion of > valid sysupgrade, which even left the system in inconsistent state, > as the "-f keep.tar.gz" was applied, but not the new image itself. > Or (silently ?) no sysupgrade done, because of mounted block device > or active swap file on block device, or active wifi (?). > Which was a PITA on remote installations. > > Question: How are sysupgrade-errors reported/to be handled, as usually also a > failed sysupgrade > triggered a reboot ? > documentation very unclear here, as it looks like, behaviour in case of errro > changed during versions of openwrt. > > Best would be "atomic sysupgrade", with standard error-code (+1) on exit > instead of expected reboot after success. > > I appreciate the new work on sysupgrade, but I am a bit afraid of regressions. No behavior will change until you explicitly modify your target's /lib/upgrade/platform.sh to start calling notify_firmware_broken() for whatever reason (e.g. unrecognized firmware image format). I'm planning to implement more verbose sysupgrade results later. I was thinking about ubus method replying with a JSON containing error code and message. I should prepare some patch for that in a next week or two. Please use "Reply to all", I almost missed your reply. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH procd] system: reject sysupgrade of broken firmware images
On 2019-09-01 00:14, Karl Palsson wrote: What's the point of "force" if it doesn't force? Are we going to add a second -F to "really force" ? Or is it going to be "oh, -F failed for some lame reason, so I'll use mtd write, and still complain anyway" Firmware can be marked as: 1) Invalid 2) Broken Force option is to bypass invalid check only. Invalid fw state may mean broken checksum or no device model match. Broken fw state should mean unrecognized fw image format. I'm not saying every target has to use that as it surely won't fit all of them nicely. I can say it's going to fit Broadcom platform very well though. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH procd] system: reject sysupgrade of broken firmware images
This needs to be handled very carefully, not to break
actual usage of -F.
I had to use -F couple of times, usually when downgrading
installed firmware, but with change of name over time.
Typical example: Change of image name for the zbt-we826.
Never any problem with usage of -F, though.
But I had several problems with non-completion of
valid sysupgrade, which even left the system in inconsistent state,
as the "-f keep.tar.gz" was applied, but not the new image itself.
Or (silently ?) no sysupgrade done, because of mounted block device
or active swap file on block device, or active wifi (?).
Which was a PITA on remote installations.
Question: How are sysupgrade-errors reported/to be handled, as usually also a
failed sysupgrade
triggered a reboot ?
documentation very unclear here, as it looks like, behaviour in case of errro
changed during versions of openwrt.
Best would be "atomic sysupgrade", with standard error-code (+1) on exit
instead of expected reboot after success.
I appreciate the new work on sysupgrade, but I am a bit afraid of regressions.
Am 01.09.2019 um 01:14 schrieb Karl Palsson:
What's the point of "force" if it doesn't force? Are we going to
add a second -F to "really force" ? Or is it going to be "oh, -F
failed for some lame reason, so I'll use mtd write, and still
complain anyway"
Cheers,
Karl P
Rafał Miłecki wrote:
From: Rafał Miłecki
This uses recently added "validate_firmware_image" to validate
passed firmware. If it happens to be invalid and marked as
impossible to force then sysupgrade simply exits with an error.
This change is needed to avoid bricking devices with some
totally broken images.
Signed-off-by: Rafał Miłecki
---
This patch depends on the:
[PATCH procd] system: add "validate_firmware_image" ubus method
---
system.c | 24
1 file changed, 24 insertions(+)
diff --git a/system.c b/system.c
index 35d5a23..7f49758 100644
--- a/system.c
+++ b/system.c
@@ -507,7 +507,18 @@ static int sysupgrade(struct ubus_context *ctx, struct
ubus_object *obj,
struct ubus_request_data *req, const char *method,
struct blob_attr *msg)
{
+ enum {
+ VALIDATION_VALID,
+ VALIDATION_FORCEABLE,
+ __VALIDATION_MAX
+ };
+ static const struct blobmsg_policy validation_policy[__VALIDATION_MAX]
= {
+ [VALIDATION_VALID] = { .name = "valid", .type =
BLOBMSG_TYPE_BOOL },
+ [VALIDATION_FORCEABLE] = { .name = "forceable", .type =
BLOBMSG_TYPE_BOOL },
+ };
+ struct blob_attr *validation[__VALIDATION_MAX];
struct blob_attr *tb[__SYSUPGRADE_MAX];
+ bool valid, forceable;
if (!msg)
return UBUS_STATUS_INVALID_ARGUMENT;
@@ -516,6 +527,19 @@ static int sysupgrade(struct ubus_context *ctx, struct
ubus_object *obj,
if (!tb[SYSUPGRADE_PATH] || !tb[SYSUPGRADE_PREFIX])
return UBUS_STATUS_INVALID_ARGUMENT;
+ if (validate_firmware_image_call(blobmsg_get_string(tb[SYSUPGRADE_PATH])))
+ return UBUS_STATUS_UNKNOWN_ERROR;
+
+ blobmsg_parse(validation_policy, __VALIDATION_MAX, validation,
blob_data(b.head), blob_len(b.head));
+
+ valid = validation[VALIDATION_VALID] &&
blobmsg_get_bool(validation[VALIDATION_VALID]);
+ forceable = validation[VALIDATION_FORCEABLE] &&
blobmsg_get_bool(validation[VALIDATION_FORCEABLE]);
+
+ if (!valid && !forceable) {
+ fprintf(stderr, "Firmware image is broken and cannot be
installed\n");
+ return UBUS_STATUS_UNKNOWN_ERROR;
+ }
+
sysupgrade_exec_upgraded(blobmsg_get_string(tb[SYSUPGRADE_PREFIX]),
blobmsg_get_string(tb[SYSUPGRADE_PATH]),
tb[SYSUPGRADE_COMMAND] ?
blobmsg_get_string(tb[SYSUPGRADE_COMMAND]) : NULL,
--
2.21.0
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH procd] system: reject sysupgrade of broken firmware images
What's the point of "force" if it doesn't force? Are we going to
add a second -F to "really force" ? Or is it going to be "oh, -F
failed for some lame reason, so I'll use mtd write, and still
complain anyway"
Cheers,
Karl P
Rafał Miłecki wrote:
> From: Rafał Miłecki
>
> This uses recently added "validate_firmware_image" to validate
> passed firmware. If it happens to be invalid and marked as
> impossible to force then sysupgrade simply exits with an error.
>
> This change is needed to avoid bricking devices with some
> totally broken images.
>
> Signed-off-by: Rafał Miłecki
> ---
> This patch depends on the:
> [PATCH procd] system: add "validate_firmware_image" ubus method
> ---
> system.c | 24
> 1 file changed, 24 insertions(+)
>
> diff --git a/system.c b/system.c
> index 35d5a23..7f49758 100644
> --- a/system.c
> +++ b/system.c
> @@ -507,7 +507,18 @@ static int sysupgrade(struct ubus_context *ctx, struct
> ubus_object *obj,
> struct ubus_request_data *req, const char *method,
> struct blob_attr *msg)
> {
> + enum {
> + VALIDATION_VALID,
> + VALIDATION_FORCEABLE,
> + __VALIDATION_MAX
> + };
> + static const struct blobmsg_policy validation_policy[__VALIDATION_MAX]
> = {
> + [VALIDATION_VALID] = { .name = "valid", .type =
> BLOBMSG_TYPE_BOOL },
> + [VALIDATION_FORCEABLE] = { .name = "forceable", .type =
> BLOBMSG_TYPE_BOOL },
> + };
> + struct blob_attr *validation[__VALIDATION_MAX];
> struct blob_attr *tb[__SYSUPGRADE_MAX];
> + bool valid, forceable;
>
> if (!msg)
> return UBUS_STATUS_INVALID_ARGUMENT;
> @@ -516,6 +527,19 @@ static int sysupgrade(struct ubus_context *ctx, struct
> ubus_object *obj,
> if (!tb[SYSUPGRADE_PATH] || !tb[SYSUPGRADE_PREFIX])
> return UBUS_STATUS_INVALID_ARGUMENT;
>
> + if
> (validate_firmware_image_call(blobmsg_get_string(tb[SYSUPGRADE_PATH])))
> + return UBUS_STATUS_UNKNOWN_ERROR;
> +
> + blobmsg_parse(validation_policy, __VALIDATION_MAX, validation,
> blob_data(b.head), blob_len(b.head));
> +
> + valid = validation[VALIDATION_VALID] &&
> blobmsg_get_bool(validation[VALIDATION_VALID]);
> + forceable = validation[VALIDATION_FORCEABLE] &&
> blobmsg_get_bool(validation[VALIDATION_FORCEABLE]);
> +
> + if (!valid && !forceable) {
> + fprintf(stderr, "Firmware image is broken and cannot be
> installed\n");
> + return UBUS_STATUS_UNKNOWN_ERROR;
> + }
> +
> sysupgrade_exec_upgraded(blobmsg_get_string(tb[SYSUPGRADE_PREFIX]),
>blobmsg_get_string(tb[SYSUPGRADE_PATH]),
>tb[SYSUPGRADE_COMMAND] ?
> blobmsg_get_string(tb[SYSUPGRADE_COMMAND]) : NULL,
> --
> 2.21.0
>
>
> ___
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
OpenPGP-digital-signature.html
Description: OpenPGP Digital Signature
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH procd] system: reject sysupgrade of broken firmware images
Hi,
> [snip]
> + blobmsg_parse(validation_policy, __VALIDATION_MAX, validation,
> blob_data(b.head), blob_len(b.head));
> +
> + valid = validation[VALIDATION_VALID] &&
> blobmsg_get_bool(validation[VALIDATION_VALID]);
> + forceable = validation[VALIDATION_FORCEABLE] &&
> blobmsg_get_bool(validation[VALIDATION_FORCEABLE]);
> +
> + if (!valid && !forceable) {
> + fprintf(stderr, "Firmware image is broken and cannot be
> installed\n");
> + return UBUS_STATUS_UNKNOWN_ERROR;
Maybe UBUS_STATUS_NOT_SUPPORTED could make sense here.
> + }
> +
> sysupgrade_exec_upgraded(blobmsg_get_string(tb[SYSUPGRADE_PREFIX]),
>blobmsg_get_string(tb[SYSUPGRADE_PATH]),
>tb[SYSUPGRADE_COMMAND] ?
> blobmsg_get_string(tb[SYSUPGRADE_COMMAND]) : NULL,
>
signature.asc
Description: OpenPGP digital signature
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH procd] system: reject sysupgrade of broken firmware images
From: Rafał Miłecki
This uses recently added "validate_firmware_image" to validate passed
firmware. If it happens to be invalid and marked as impossible to force
then sysupgrade simply exits with an error.
This change is needed to avoid bricking devices with some totally broken
images.
Signed-off-by: Rafał Miłecki
---
This patch depends on the:
[PATCH procd] system: add "validate_firmware_image" ubus method
---
system.c | 24
1 file changed, 24 insertions(+)
diff --git a/system.c b/system.c
index 35d5a23..7f49758 100644
--- a/system.c
+++ b/system.c
@@ -507,7 +507,18 @@ static int sysupgrade(struct ubus_context *ctx, struct
ubus_object *obj,
struct ubus_request_data *req, const char *method,
struct blob_attr *msg)
{
+ enum {
+ VALIDATION_VALID,
+ VALIDATION_FORCEABLE,
+ __VALIDATION_MAX
+ };
+ static const struct blobmsg_policy validation_policy[__VALIDATION_MAX]
= {
+ [VALIDATION_VALID] = { .name = "valid", .type =
BLOBMSG_TYPE_BOOL },
+ [VALIDATION_FORCEABLE] = { .name = "forceable", .type =
BLOBMSG_TYPE_BOOL },
+ };
+ struct blob_attr *validation[__VALIDATION_MAX];
struct blob_attr *tb[__SYSUPGRADE_MAX];
+ bool valid, forceable;
if (!msg)
return UBUS_STATUS_INVALID_ARGUMENT;
@@ -516,6 +527,19 @@ static int sysupgrade(struct ubus_context *ctx, struct
ubus_object *obj,
if (!tb[SYSUPGRADE_PATH] || !tb[SYSUPGRADE_PREFIX])
return UBUS_STATUS_INVALID_ARGUMENT;
+ if
(validate_firmware_image_call(blobmsg_get_string(tb[SYSUPGRADE_PATH])))
+ return UBUS_STATUS_UNKNOWN_ERROR;
+
+ blobmsg_parse(validation_policy, __VALIDATION_MAX, validation,
blob_data(b.head), blob_len(b.head));
+
+ valid = validation[VALIDATION_VALID] &&
blobmsg_get_bool(validation[VALIDATION_VALID]);
+ forceable = validation[VALIDATION_FORCEABLE] &&
blobmsg_get_bool(validation[VALIDATION_FORCEABLE]);
+
+ if (!valid && !forceable) {
+ fprintf(stderr, "Firmware image is broken and cannot be
installed\n");
+ return UBUS_STATUS_UNKNOWN_ERROR;
+ }
+
sysupgrade_exec_upgraded(blobmsg_get_string(tb[SYSUPGRADE_PREFIX]),
blobmsg_get_string(tb[SYSUPGRADE_PATH]),
tb[SYSUPGRADE_COMMAND] ?
blobmsg_get_string(tb[SYSUPGRADE_COMMAND]) : NULL,
--
2.21.0
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
