Re: [PATCH 1/2] realtek: Use firewall4
Hauke Mehrtens [2022-02-28 22:37:29]: > The realtek target is not a router, but basic device, see DEVICE_TYPE. > The basic device type does not come with firewall by default, see > include/target.mk for details. The realtek target extended > DEFAULT_PACKAGES manually with firewall. > > This changes the defaults to take firewall4 and nftables instead of > firewall and iptables. This also adds the additional package > kmod-nft-offload. > The only difference to the router type is the missing ppp and > ppp-mod-pppoe package. > > This increases the compressed image size by about 260KBytes. > > Signed-off-by: Hauke Mehrtens Acked-by: Petr Štetiar ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH 2/6] tools/dwarves: add host package
On Mon, Feb 28, 2022 at 8:17 AM Stijn Tintel wrote: > > On 28/02/2022 16:16, Ansuel Smith wrote: > >> On 27/02/2022 17:20, Ansuel Smith wrote: > >>> > > From: Tony Ambardar > > dwarves is a set of tools that use the debugging information inserted in > ELF binaries by compilers such as GCC. Utilities in the dwarves suite > include pahole, which can be used to find alignment holes in structs and > classes, and also extracts other information such as CPU cacheline > alignment, helping pack those structures to achieve more cache hits. > > These tools are also used to encode and read the BTF type information > format used with the bpf syscall, making this a Linux build dependency > when using kernel BTF information. > >>> BTW this fails to build if libdw-dev is not installed with > >>> > >>> -- Checking availability of DWARF and ELF development libraries > >>> -- Could NOT find dwarf include dir > >>> -- Could NOT find libdw include dir > >>> -- Could NOT find libdw library > >>> CMake Error at cmake/modules/FindDWARF.cmake:93 (message): > >>> Could NOT find some ELF and DWARF libraries, please install the missing > >>> packages > >>> Call Stack (most recent call first): > >>> CMakeLists.txt:60 (find_package) > >>> > >>> ERROR: tools/dwarves failed to build. > >>> > >>> Should we add it to the build prereq? > >> Please try this instead: > >> > >> diff --git a/tools/dwarves/Makefile b/tools/dwarves/Makefile > >> index b02a2398a1..e5a55706be 100644 > >> --- a/tools/dwarves/Makefile > >> +++ b/tools/dwarves/Makefile > >> @@ -12,14 +12,12 @@PKG_LICENSE:=GPL-2.0-only > >> PKG_LICENSE_FILES:=COPYING > >> > >> HOST_BUILD_PARALLEL:=1 > >> +PKG_BUILD_DEPENDS:=elfutils/host > >> > >> include $(INCLUDE_DIR)/host-build.mk > >> include $(INCLUDE_DIR)/cmake.mk > >> > >> CMAKE_HOST_OPTIONS += \ > >> - -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH \ > >> - -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \ > >> - -DCMAKE_BUILD_TYPE=Release \ > >>-D__LIB=lib \ > >>-DCMAKE_INSTALL_RPATH="$(STAGING_DIR_HOST)/lib" \ > >>-DCMAKE_SKIP_RPATH=FALSE > >> > >> Thanks, > >> Stijn > >> > > No luck... still > > > Apparently we cannot use PKG_BUILD_DEPENDS or HOST_BUILD_DEPENDS in > tools/, as this is handled in include/package-dumpinfo.mk, which is > included from include/package.mk, which we don't include in tools/... > Could use some guidance here, as I am not familiar enough with this part > of the code. depends for tools/ are specified in tools/Makefile. Which means it requires tools/elfutils > > Thanks, > Stijn > > > ___ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
OpenWrt 19.07.9 service release
Hi, The OpenWrt community is proud to announce the newest service release of OpenWrt 19.07. It fixes security issues, improves device support, and brings a few bug fixes. We recently moved our bugs to GitHub, please report issues over there (after checking that nobody else posted the same issue before). https://github.com/openwrt/openwrt/issues/ The main changes from OpenWrt 19.07.8 are: Security fixes == * hostapd: Apply SAE/EAP-pwd side-channel attack update 2 (CVE-2022-23303, CVE-2022-23304) * mbedtls: Update to version 2.16.12 to fix CVE-2021-44732 * tcpdump: fix CVE-2018-16301 * openssl: fix SM2 Decryption Buffer Overflow (CVE-2021-3711) and Read buffer overruns processing ASN.1 strings (CVE-2021-3712) Major bug fixes === * No major bug fixes in this release Device support == * uboot-lantiq: danube: fix hanging lzma kernel uncompression * ar71xx: mikrotik: rb91x: fix 10M ethernet link speed Various fixes and improvements == * Update wireless-regdb to account for new regulatory rules (6 GHz, 60 GHz, and various other fixes) * sdk: fix missing include directories * Various fixes when building with GCC 10 Core components === * Update Linux kernel from 4.14.241 to 4.14.267 * Update mac80211 from 4.19.193-1 to 4.19.221-1 * Update openssl from 1.1.1k to 1.1.1m * Update mbedtls from 2.16.10 to 2.16.12 Full release notes and upgrade instructions are available at https://openwrt.org/releases/19.07/notes-19.07.9 In particular, make sure to read the regressions and known issues before upgrading: https://openwrt.org/releases/19.07/notes-19.07.9#regressions For a very detailed list of all changes since 19.07.8, refer to https://openwrt.org/releases/19.07/changelog-19.07.9 For latest information about the 19.07 series, refer to the wiki at: https://openwrt.org/releases/19.07/ To download a OpenWrt 19.07.9 firmware image for your device, head to the Table of Hardware: https://openwrt.org/toh/start Or use the new firmware selector: https://firmware-selector.openwrt.org/ You can also navigate directly in the list of firmware images: https://downloads.openwrt.org/releases/19.07.9/targets/ As always, a big thank you goes to all our active package maintainers, testers, documenters, and supporters. Have fun! The OpenWrt Community --- To stay informed of new OpenWrt releases and security advisories, there are new channels available: * a low-volume mailing list for important announcements: https://lists.openwrt.org/mailman/listinfo/openwrt-announce * a dedicated "announcements" section in the forum: https://forum.openwrt.org/c/announcements/14 * other announcement channels (such as RSS feeds) might be added in the future, they will be listed at https://openwrt.org/contact ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH 1/2] realtek: Use firewall4
On 2/28/22 23:00, Sander Vanheule wrote:
Hi Hauke,
On Mon, 2022-02-28 at 22:37 +0100, Hauke Mehrtens wrote:
The realtek target is not a router, but basic device, see DEVICE_TYPE.
The basic device type does not come with firewall by default, see
include/target.mk for details. The realtek target extended
DEFAULT_PACKAGES manually with firewall.
This changes the defaults to take firewall4 and nftables instead of
firewall and iptables. This also adds the additional package
kmod-nft-offload.
The only difference to the router type is the missing ppp and
ppp-mod-pppoe package.
This increases the compressed image size by about 260KBytes.
Signed-off-by: Hauke Mehrtens
Commit 9e7149f729e9 ("realtek: revert to "standard" management configuration")
changed the
default port configuration for realtek devices to only have LAN ports, instead
of the
LAN/WAN VLANs that were used before. I wonder if it doesn't make more sense to
drop the
firewall package from the default now, since there is only one interface,
unless there is
a different reason to keep the firewall.
We can also remove firewall4 support from the realtek target. Probably
most people will not use it for routing and if so they can install
firewall4 manually. I just do not want to ship firewall3 by default.
Hauke
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH 1/2] realtek: Use firewall4
Hi Hauke,
On Mon, 2022-02-28 at 22:37 +0100, Hauke Mehrtens wrote:
> The realtek target is not a router, but basic device, see DEVICE_TYPE.
> The basic device type does not come with firewall by default, see
> include/target.mk for details. The realtek target extended
> DEFAULT_PACKAGES manually with firewall.
>
> This changes the defaults to take firewall4 and nftables instead of
> firewall and iptables. This also adds the additional package
> kmod-nft-offload.
> The only difference to the router type is the missing ppp and
> ppp-mod-pppoe package.
>
> This increases the compressed image size by about 260KBytes.
>
> Signed-off-by: Hauke Mehrtens
Commit 9e7149f729e9 ("realtek: revert to "standard" management configuration")
changed the
default port configuration for realtek devices to only have LAN ports, instead
of the
LAN/WAN VLANs that were used before. I wonder if it doesn't make more sense to
drop the
firewall package from the default now, since there is only one interface,
unless there is
a different reason to keep the firewall.
Best,
Sander
> ---
> target/linux/realtek/Makefile | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/target/linux/realtek/Makefile b/target/linux/realtek/Makefile
> index 704242a000a0..91af5fbcfce1 100644
> --- a/target/linux/realtek/Makefile
> +++ b/target/linux/realtek/Makefile
> @@ -18,7 +18,7 @@ endef
> include $(INCLUDE_DIR)/target.mk
>
> DEFAULT_PACKAGES += uboot-envtools ethtool kmod-gpio-button-hotplug \
> - dnsmasq firewall ip6tables iptables odhcp6c odhcpd-ipv6only \
> + dnsmasq firewall4 nftables kmod-nft-offload odhcp6c odhcpd-ipv6only \
> ip-full ip-bridge tc
>
> $(eval $(call BuildTarget))
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[PATCH 2/2] realtek: Fix tc default package
The tc package does not exits any more, it was split into tc-tiny and tc-full. Include tc-tiny by default into realtek images. This increases the compressed image size by about 260KBytes. Signed-off-by: Hauke Mehrtens --- target/linux/realtek/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/linux/realtek/Makefile b/target/linux/realtek/Makefile index 91af5fbcfce1..1be4cfe21783 100644 --- a/target/linux/realtek/Makefile +++ b/target/linux/realtek/Makefile @@ -19,6 +19,6 @@ include $(INCLUDE_DIR)/target.mk DEFAULT_PACKAGES += uboot-envtools ethtool kmod-gpio-button-hotplug \ dnsmasq firewall4 nftables kmod-nft-offload odhcp6c odhcpd-ipv6only \ - ip-full ip-bridge tc + ip-full ip-bridge tc-tiny $(eval $(call BuildTarget)) -- 2.30.2 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[PATCH 1/2] realtek: Use firewall4
The realtek target is not a router, but basic device, see DEVICE_TYPE. The basic device type does not come with firewall by default, see include/target.mk for details. The realtek target extended DEFAULT_PACKAGES manually with firewall. This changes the defaults to take firewall4 and nftables instead of firewall and iptables. This also adds the additional package kmod-nft-offload. The only difference to the router type is the missing ppp and ppp-mod-pppoe package. This increases the compressed image size by about 260KBytes. Signed-off-by: Hauke Mehrtens --- target/linux/realtek/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/linux/realtek/Makefile b/target/linux/realtek/Makefile index 704242a000a0..91af5fbcfce1 100644 --- a/target/linux/realtek/Makefile +++ b/target/linux/realtek/Makefile @@ -18,7 +18,7 @@ endef include $(INCLUDE_DIR)/target.mk DEFAULT_PACKAGES += uboot-envtools ethtool kmod-gpio-button-hotplug \ - dnsmasq firewall ip6tables iptables odhcp6c odhcpd-ipv6only \ + dnsmasq firewall4 nftables kmod-nft-offload odhcp6c odhcpd-ipv6only \ ip-full ip-bridge tc $(eval $(call BuildTarget)) -- 2.30.2 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH 2/6] tools/dwarves: add host package
On 28/02/2022 16:16, Ansuel Smith wrote: >> On 27/02/2022 17:20, Ansuel Smith wrote: >>> > From: Tony Ambardar dwarves is a set of tools that use the debugging information inserted in ELF binaries by compilers such as GCC. Utilities in the dwarves suite include pahole, which can be used to find alignment holes in structs and classes, and also extracts other information such as CPU cacheline alignment, helping pack those structures to achieve more cache hits. These tools are also used to encode and read the BTF type information format used with the bpf syscall, making this a Linux build dependency when using kernel BTF information. >>> BTW this fails to build if libdw-dev is not installed with >>> >>> -- Checking availability of DWARF and ELF development libraries >>> -- Could NOT find dwarf include dir >>> -- Could NOT find libdw include dir >>> -- Could NOT find libdw library >>> CMake Error at cmake/modules/FindDWARF.cmake:93 (message): >>> Could NOT find some ELF and DWARF libraries, please install the missing >>> packages >>> Call Stack (most recent call first): >>> CMakeLists.txt:60 (find_package) >>> >>> ERROR: tools/dwarves failed to build. >>> >>> Should we add it to the build prereq? >> Please try this instead: >> >> diff --git a/tools/dwarves/Makefile b/tools/dwarves/Makefile >> index b02a2398a1..e5a55706be 100644 >> --- a/tools/dwarves/Makefile >> +++ b/tools/dwarves/Makefile >> @@ -12,14 +12,12 @@PKG_LICENSE:=GPL-2.0-only >> PKG_LICENSE_FILES:=COPYING >> >> HOST_BUILD_PARALLEL:=1 >> +PKG_BUILD_DEPENDS:=elfutils/host >> >> include $(INCLUDE_DIR)/host-build.mk >> include $(INCLUDE_DIR)/cmake.mk >> >> CMAKE_HOST_OPTIONS += \ >> - -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH \ >> - -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \ >> - -DCMAKE_BUILD_TYPE=Release \ >>-D__LIB=lib \ >>-DCMAKE_INSTALL_RPATH="$(STAGING_DIR_HOST)/lib" \ >>-DCMAKE_SKIP_RPATH=FALSE >> >> Thanks, >> Stijn >> > No luck... still > Apparently we cannot use PKG_BUILD_DEPENDS or HOST_BUILD_DEPENDS in tools/, as this is handled in include/package-dumpinfo.mk, which is included from include/package.mk, which we don't include in tools/... Could use some guidance here, as I am not familiar enough with this part of the code. Thanks, Stijn ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH 2/6] tools/dwarves: add host package
> > On 27/02/2022 17:20, Ansuel Smith wrote: > > > > >> From: Tony Ambardar > >> > >> dwarves is a set of tools that use the debugging information inserted in > >> ELF binaries by compilers such as GCC. Utilities in the dwarves suite > >> include pahole, which can be used to find alignment holes in structs and > >> classes, and also extracts other information such as CPU cacheline > >> alignment, helping pack those structures to achieve more cache hits. > >> > >> These tools are also used to encode and read the BTF type information > >> format used with the bpf syscall, making this a Linux build dependency > >> when using kernel BTF information. > > BTW this fails to build if libdw-dev is not installed with > > > > -- Checking availability of DWARF and ELF development libraries > > -- Could NOT find dwarf include dir > > -- Could NOT find libdw include dir > > -- Could NOT find libdw library > > CMake Error at cmake/modules/FindDWARF.cmake:93 (message): > > Could NOT find some ELF and DWARF libraries, please install the missing > > packages > > Call Stack (most recent call first): > > CMakeLists.txt:60 (find_package) > > > > ERROR: tools/dwarves failed to build. > > > > Should we add it to the build prereq? > > Please try this instead: > > diff --git a/tools/dwarves/Makefile b/tools/dwarves/Makefile > index b02a2398a1..e5a55706be 100644 > --- a/tools/dwarves/Makefile > +++ b/tools/dwarves/Makefile > @@ -12,14 +12,12 @@PKG_LICENSE:=GPL-2.0-only > PKG_LICENSE_FILES:=COPYING > > HOST_BUILD_PARALLEL:=1 > +PKG_BUILD_DEPENDS:=elfutils/host > > include $(INCLUDE_DIR)/host-build.mk > include $(INCLUDE_DIR)/cmake.mk > > CMAKE_HOST_OPTIONS += \ > - -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH \ > - -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \ > - -DCMAKE_BUILD_TYPE=Release \ >-D__LIB=lib \ >-DCMAKE_INSTALL_RPATH="$(STAGING_DIR_HOST)/lib" \ >-DCMAKE_SKIP_RPATH=FALSE > > Thanks, > Stijn > No luck... still make[3]: Leaving directory '/home/ansuel/openwrt/tools/missing-macros' time: tools/missing-macros/compile#0.09#0.06#0.12 -- The C compiler identification is GNU 11.2.0 -- Detecting C compiler ABI info make[3]: Leaving directory '/home/ansuel/openwrt/tools/automake' time: tools/automake/compile#0.14#0.44#0.51 make[3]: Entering directory '/home/ansuel/openwrt/tools/libtool' make[3]: Entering directory '/home/ansuel/openwrt/tools/dosfstools' make[3]: Leaving directory '/home/ansuel/openwrt/tools/dosfstools' time: tools/dosfstools/compile#0.13#0.17#0.25 -- Detecting C compiler ABI info - done -- Check for working C compiler: /home/ansuel/openwrt/staging_dir/host/bin/gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Setting BUILD_SHARED_LIBS = ON make[3]: Leaving directory '/home/ansuel/openwrt/tools/libtool' -- Checking availability of DWARF and ELF development libraries time: tools/libtool/compile#0.15#0.27#0.34 -- Could NOT find dwarf include dir -- Could NOT find libdw include dir -- Could NOT find libdw library -- Could NOT find libelf library CMake Error at cmake/modules/FindDWARF.cmake:93 (message): Could NOT find some ELF and DWARF libraries, please install the missing packages Call Stack (most recent call first): CMakeLists.txt:60 (find_package) -- Configuring incomplete, errors occurred! ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: ubifs: handling dirty data (writing back) + power cuts
On 25.02.2022 15:17, Rafał Miłecki wrote: My actual problem is related to ubifs behaviour for power cuts happening between 5 and 35 seconds after saving a file: date > /mount/ubifs/test.txt && sleep 15 && echo CUT POWER *NOW* On the next boot test.txt exists but it's EMPTY (file size 0). For newly created files above behaviour is not the worst one - however I'd expect such file to not exist at all. The biggest problem is when dealing with existing files. In such case power cut means loosing it completely. I don't get *old* content nor *new* content. To make it clear what I meant by dealing with existing files: [fist boot] # echo foo > /mount/ubifs/test.txt # sync # cat /mount/ubifs/test.txt foo # busybox sed -i 's/foo/bar/' /mount/ubifs/test.txt && sleep 10 && echo CUT POWER NOW CUT POWER NOW [next boot] # ls -l /mount/ubifs/test.txt -rw---1 root root 0 Feb 28 02:26 /mount/ubifs/test.txt ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH 19.07] feeds.conf.default: remove freifunk feed
On 2/28/22 10:04, Paul Spooren wrote: Anyway, if the Freifunk community itself finds it to be useless, let’s drop it. I don't think it's useless to combine Freifunk and OpenWrt. I don't know why people pulled for example the freifunk luci mod out of OpenWrt Luci Feed. Now we don't benefit anymore from tree-wide changes. I had a hard time converting the mod to 19.07 and then 21.02 (thanks a lot to jow who supported me). From my side of view, we lack maintainer power, to allow us to follow each tree-wide change and apply them. However, I think OpenWrt especially benefits from all Freifunk-Communities, since we do maintenance, testing, and development. For example I am part of Freifunk Berlin (however, we have 3(?) different "firmwares" right now). Freifunk communities often rely on the "openwrt routing feed". In my view, it consists only of two real active maintenance (mwarning and me and maybe some more). Now thankfully BKPepe and neheb help to maintain the feed and get it into a good-looking shape. There is a lot of power in the community package repository. Somehow more and more stuff is maintained from my side I never planned to do so. For example, I currently maintain OLSR. I would be very happy if we could upstream again more wireless community mesh-specific packages to OpenWrt and get more help with maintenance. So that in the end, more resources are free to develop and test more. Some people in Berlin switched to an "ansiblification" of the most important backbone places in our network, that completely builds on normal OpenWrt [0] utilizing the image builders. So at least some Freifunk don't use their own make system modification. I always argue to make things more generalized so all people using OpenWrt can benefit from it, and we don't have to write the same code (or even maintain the same packages) multiple times, e.g. tunneldigger. Overall, instead of maintaining a separate feed for Freifunk where we also have to maintain the CIs and so on, I would argue for more packages in openwrt/packages and openwrt/luci. It is not about all the "uci config defaults" that are in the Freifunk repository. In my view, we need to just generate default configs differently, like we already do in ansible and feed them into the imagebuilder. It is about packages that have some more functionality, like showing mesh routing protocols on the front page, building WireGuard tunnels, etc. ... So remove the Freifunk Repository, but let's work/maintain together. Bests Nick [0] - https://github.com/freifunk-berlin/bbb-configs ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH] image: let mksquashfs4 use all processors
> On 28. Feb 2022, at 12:35, Stijn Tintel wrote: > > On 28/02/2022 13:22, Paul Spooren wrote: >> Hi team, >> >>> On 19. Feb 2022, at 19:21, Phillip Lougher >>> wrote: >>> >>> On Sat, Feb 19, 2022 at 4:01 PM Felix Fietkau wrote: On 19.02.22 16:54, Stijn Tintel wrote: > Drop the -processors argument from the mksquashfs4 call, so it will use > all available processors. This dramatically reduces the time to create > squashfs filesystems. > > The times below are observed when building an image for my main router, > the WatchGuard Firebox M300 (qoriq target): > > Before: > real4m45,973s > > After: > real0m23,497s > > Signed-off-by: Stijn Tintel We need to verify that this doesn't break reproducible builds... >>> It won't break reproducible builds. Since Mksquashfs version 4.4 the >>> ordering is guaranteed to be the same, irrespective of how many >>> threads or processors are used. >> I think OpenWrt uses `squashfskit`[1] which is a fork some squashfs-tools >> version. I think it happened due to stalled development some years ago. >> Anyway, if squashfs-tools is actively maintained we should consider moving >> back to the upstream version. >> >> Some time ago I already wanted to make the move but squashfskit uses some >> downstream patches which expose more compression options, are those now part >> of squashfs-tools[2]? If not, do we need those anyway? > Not aware if we do or don't. >> >> Thanks for all further information, I’d be happy to have upstream >> squashfs-tools with reproducible multithread support within OpenWrt! >> >> Sunshine, >> Paul >> >> [1]: https://github.com/squashfskit/squashfskit >> [2]: >> https://github.com/squashfskit/squashfskit/commit/5568b6a7d7db01c8f60a502d549a431a2b7219ae > > If you decide to switch back to squashfs-tools, and incorporate the > change to let it use all processors, we should probably respect the > number of jobs the user passes to make, and limit mksquashfs to use at > most that number of processors. This was suggested on #openwrt-devel, > after my initial submission. Unfortunately I have not found a way to do > so, as MAKE_JOBSERVER is exported in include/toplevel.mk, which is not > included in include/image.mk, and I am not familiar enough with this > part of the code to understand the impact of adding that include. I’m happy to take care of that once we figured out how to proceed with our fork. > > Stijn ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH] image: let mksquashfs4 use all processors
On 28/02/2022 13:22, Paul Spooren wrote: > Hi team, > >> On 19. Feb 2022, at 19:21, Phillip Lougher wrote: >> >> On Sat, Feb 19, 2022 at 4:01 PM Felix Fietkau wrote: >>> On 19.02.22 16:54, Stijn Tintel wrote: Drop the -processors argument from the mksquashfs4 call, so it will use all available processors. This dramatically reduces the time to create squashfs filesystems. The times below are observed when building an image for my main router, the WatchGuard Firebox M300 (qoriq target): Before: real4m45,973s After: real0m23,497s Signed-off-by: Stijn Tintel >>> We need to verify that this doesn't break reproducible builds... >>> >> It won't break reproducible builds. Since Mksquashfs version 4.4 the >> ordering is guaranteed to be the same, irrespective of how many >> threads or processors are used. > I think OpenWrt uses `squashfskit`[1] which is a fork some squashfs-tools > version. I think it happened due to stalled development some years ago. > Anyway, if squashfs-tools is actively maintained we should consider moving > back to the upstream version. > > Some time ago I already wanted to make the move but squashfskit uses some > downstream patches which expose more compression options, are those now part > of squashfs-tools[2]? If not, do we need those anyway? Not aware if we do or don't. > > Thanks for all further information, I’d be happy to have upstream > squashfs-tools with reproducible multithread support within OpenWrt! > > Sunshine, > Paul > > [1]: https://github.com/squashfskit/squashfskit > [2]: > https://github.com/squashfskit/squashfskit/commit/5568b6a7d7db01c8f60a502d549a431a2b7219ae If you decide to switch back to squashfs-tools, and incorporate the change to let it use all processors, we should probably respect the number of jobs the user passes to make, and limit mksquashfs to use at most that number of processors. This was suggested on #openwrt-devel, after my initial submission. Unfortunately I have not found a way to do so, as MAKE_JOBSERVER is exported in include/toplevel.mk, which is not included in include/image.mk, and I am not familiar enough with this part of the code to understand the impact of adding that include. Stijn ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH] image: let mksquashfs4 use all processors
Hi team, > On 19. Feb 2022, at 19:21, Phillip Lougher wrote: > > On Sat, Feb 19, 2022 at 4:01 PM Felix Fietkau wrote: >> >> On 19.02.22 16:54, Stijn Tintel wrote: >>> Drop the -processors argument from the mksquashfs4 call, so it will use >>> all available processors. This dramatically reduces the time to create >>> squashfs filesystems. >>> >>> The times below are observed when building an image for my main router, >>> the WatchGuard Firebox M300 (qoriq target): >>> >>> Before: >>> real4m45,973s >>> >>> After: >>> real0m23,497s >>> >>> Signed-off-by: Stijn Tintel >> We need to verify that this doesn't break reproducible builds... >> > > It won't break reproducible builds. Since Mksquashfs version 4.4 the > ordering is guaranteed to be the same, irrespective of how many > threads or processors are used. I think OpenWrt uses `squashfskit`[1] which is a fork some squashfs-tools version. I think it happened due to stalled development some years ago. Anyway, if squashfs-tools is actively maintained we should consider moving back to the upstream version. Some time ago I already wanted to make the move but squashfskit uses some downstream patches which expose more compression options, are those now part of squashfs-tools[2]? If not, do we need those anyway? Thanks for all further information, I’d be happy to have upstream squashfs-tools with reproducible multithread support within OpenWrt! Sunshine, Paul [1]: https://github.com/squashfskit/squashfskit [2]: https://github.com/squashfskit/squashfskit/commit/5568b6a7d7db01c8f60a502d549a431a2b7219ae ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH 2/6] tools/dwarves: add host package
On 27/02/2022 17:20, Ansuel Smith wrote: > > >> From: Tony Ambardar >> >> dwarves is a set of tools that use the debugging information inserted in >> ELF binaries by compilers such as GCC. Utilities in the dwarves suite >> include pahole, which can be used to find alignment holes in structs and >> classes, and also extracts other information such as CPU cacheline >> alignment, helping pack those structures to achieve more cache hits. >> >> These tools are also used to encode and read the BTF type information >> format used with the bpf syscall, making this a Linux build dependency >> when using kernel BTF information. > BTW this fails to build if libdw-dev is not installed with > > -- Checking availability of DWARF and ELF development libraries > -- Could NOT find dwarf include dir > -- Could NOT find libdw include dir > -- Could NOT find libdw library > CMake Error at cmake/modules/FindDWARF.cmake:93 (message): > Could NOT find some ELF and DWARF libraries, please install the missing > packages > Call Stack (most recent call first): > CMakeLists.txt:60 (find_package) > > ERROR: tools/dwarves failed to build. > > Should we add it to the build prereq? Please try this instead: diff --git a/tools/dwarves/Makefile b/tools/dwarves/Makefile index b02a2398a1..e5a55706be 100644 --- a/tools/dwarves/Makefile +++ b/tools/dwarves/Makefile @@ -12,14 +12,12 @@PKG_LICENSE:=GPL-2.0-only PKG_LICENSE_FILES:=COPYING HOST_BUILD_PARALLEL:=1 +PKG_BUILD_DEPENDS:=elfutils/host include $(INCLUDE_DIR)/host-build.mk include $(INCLUDE_DIR)/cmake.mk CMAKE_HOST_OPTIONS += \ - -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH \ - -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \ - -DCMAKE_BUILD_TYPE=Release \ -D__LIB=lib \ -DCMAKE_INSTALL_RPATH="$(STAGING_DIR_HOST)/lib" \ -DCMAKE_SKIP_RPATH=FALSE Thanks, Stijn ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH 19.07] feeds.conf.default: remove freifunk feed
Hi > On 27. 02. 22 15:29, Paul Spooren wrote: >> Isn’t it bad to backport package (or even feed) removal? > > This feed was removed before OpenWrt 21.02-rc1 was released, but it was you, > Paul, who asked if it could be backported to OpenWrt 19.07 based on this > comment [1], and a few days later, it was cherry-picked. [2] > > It could be interesting to know why it was necessary to include it in the > stable release and if you need it somehow. As said, this feed has been > present since OpenWrt 19.07.5, but given the reasons in the commit message > and as it was removed from openwrt-21.02 and master branches, it is not > necessary to have it in OpenWrt 19.07. > > If it is not used anymore, then it could be removed. I’m a fan of the Freifunk work however I don’t use it myself. Backporting it to 19.07.x allows all previous releases to add the feed and use it. I hoped to see some unification of the many Freifunk flavors by having a single “official” repository. Since that didn’t work out I’m fine that it was dropped later on. Being part of the OpenWrt project I learn every other day that “legacy” shouldn’t be broken. By removing the Freifunk feed we may break whatever special setup some community somewhere figured out, glued into some CI and scripts which silently produces images for whatever use case. Anyway, if the Freifunk community itself finds it to be useless, let’s drop it. 19.07 is EOL soonish like Petr said. > > > [1] > https://github.com/openwrt/openwrt/commit/221f97ff4737f012c90feb086bc1c2ed86c6001b#commitcomment-43920797 > > [2] > https://github.com/openwrt/openwrt/commit/2a3dbded93775aeaf28fbebbd6aada07c9f588c1 > > Regards, > Josef > ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH] firmware-utils: fix compilation with macOS
> On 28. Feb 2022, at 09:51, Rosen Penev wrote:
>
> On Mon, Feb 28, 2022 at 12:43 AM Paul Spooren wrote:
>>
>>
>>
>>> On 28. Feb 2022, at 08:34, Rosen Penev wrote:
>>>
>>> __bswap_32 is a GNU extension.
>>>
>>> Signed-off-by: Rosen Penev
>>> —
>>
>> Seems to fix the compile issue on my Mac.
>>
>> Acked-by: Paul Spooren
>>
>> If people change things in tools/, please use GitHub PRs. I added a CI to
>> compile check changes on Ubuntu/macOS since breakage is a recurring issue.
> The issue is that it would have to be implemented as a patch file,
> which is bad form to be merged.
The bump (73dfc9e7d9) could have been a PR ergo triggering the CI.
>>
>>> src/avm-wasp-checksum.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/src/avm-wasp-checksum.c b/src/avm-wasp-checksum.c
>>> index 8c112f3..41a425e 100644
>>> --- a/src/avm-wasp-checksum.c
>>> +++ b/src/avm-wasp-checksum.c
>>> @@ -156,7 +156,7 @@ int main(int argc, char *argv[])
>>> }
>>> }
>>> if (model == MODEL_X490)
>>> - crc = __bswap_32(crc);
>>> + crc = bswap_32(crc);
>>> fwrite(, sizeof(uint32_t), 1, out_fp);
>>> if (ferror(out_fp)) {
>>> fprintf(stderr, "Error writing checksum to output file: %s\n",
>>> outfile);
>>> --
>>> 2.35.1
>>>
>>>
>>> ___
>>> openwrt-devel mailing list
>>> openwrt-devel@lists.openwrt.org
>>> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH] firmware-utils: fix compilation with macOS
On Mon, Feb 28, 2022 at 12:43 AM Paul Spooren wrote:
>
>
>
> > On 28. Feb 2022, at 08:34, Rosen Penev wrote:
> >
> > __bswap_32 is a GNU extension.
> >
> > Signed-off-by: Rosen Penev
> > —
>
> Seems to fix the compile issue on my Mac.
>
> Acked-by: Paul Spooren
>
> If people change things in tools/, please use GitHub PRs. I added a CI to
> compile check changes on Ubuntu/macOS since breakage is a recurring issue.
The issue is that it would have to be implemented as a patch file,
which is bad form to be merged.
>
> > src/avm-wasp-checksum.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/src/avm-wasp-checksum.c b/src/avm-wasp-checksum.c
> > index 8c112f3..41a425e 100644
> > --- a/src/avm-wasp-checksum.c
> > +++ b/src/avm-wasp-checksum.c
> > @@ -156,7 +156,7 @@ int main(int argc, char *argv[])
> > }
> > }
> > if (model == MODEL_X490)
> > - crc = __bswap_32(crc);
> > + crc = bswap_32(crc);
> > fwrite(, sizeof(uint32_t), 1, out_fp);
> > if (ferror(out_fp)) {
> > fprintf(stderr, "Error writing checksum to output file:
> > %s\n", outfile);
> > --
> > 2.35.1
> >
> >
> > ___
> > openwrt-devel mailing list
> > openwrt-devel@lists.openwrt.org
> > https://lists.openwrt.org/mailman/listinfo/openwrt-devel
>
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH] firmware-utils: fix compilation with macOS
> On 28. Feb 2022, at 08:34, Rosen Penev wrote:
>
> __bswap_32 is a GNU extension.
>
> Signed-off-by: Rosen Penev
> —
Seems to fix the compile issue on my Mac.
Acked-by: Paul Spooren
If people change things in tools/, please use GitHub PRs. I added a CI to
compile check changes on Ubuntu/macOS since breakage is a recurring issue.
> src/avm-wasp-checksum.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/avm-wasp-checksum.c b/src/avm-wasp-checksum.c
> index 8c112f3..41a425e 100644
> --- a/src/avm-wasp-checksum.c
> +++ b/src/avm-wasp-checksum.c
> @@ -156,7 +156,7 @@ int main(int argc, char *argv[])
> }
> }
> if (model == MODEL_X490)
> - crc = __bswap_32(crc);
> + crc = bswap_32(crc);
> fwrite(, sizeof(uint32_t), 1, out_fp);
> if (ferror(out_fp)) {
> fprintf(stderr, "Error writing checksum to output file: %s\n",
> outfile);
> --
> 2.35.1
>
>
> ___
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
