> On 28. Feb 2022, at 12:35, Stijn Tintel <[email protected]> wrote: > > On 28/02/2022 13:22, Paul Spooren wrote: >> Hi team, >> >>> On 19. Feb 2022, at 19:21, Phillip Lougher <[email protected]> >>> wrote: >>> >>> On Sat, Feb 19, 2022 at 4:01 PM Felix Fietkau <[email protected]> wrote: >>>> On 19.02.22 16:54, Stijn Tintel wrote: >>>>> Drop the -processors argument from the mksquashfs4 call, so it will use >>>>> all available processors. This dramatically reduces the time to create >>>>> squashfs filesystems. >>>>> >>>>> The times below are observed when building an image for my main router, >>>>> the WatchGuard Firebox M300 (qoriq target): >>>>> >>>>> Before: >>>>> real 4m45,973s >>>>> >>>>> After: >>>>> real 0m23,497s >>>>> >>>>> Signed-off-by: Stijn Tintel <[email protected]> >>>> We need to verify that this doesn't break reproducible builds... >>>> >>> It won't break reproducible builds. Since Mksquashfs version 4.4 the >>> ordering is guaranteed to be the same, irrespective of how many >>> threads or processors are used. >> I think OpenWrt uses `squashfskit`[1] which is a fork some squashfs-tools >> version. I think it happened due to stalled development some years ago. >> Anyway, if squashfs-tools is actively maintained we should consider moving >> back to the upstream version. >> >> Some time ago I already wanted to make the move but squashfskit uses some >> downstream patches which expose more compression options, are those now part >> of squashfs-tools[2]? If not, do we need those anyway? > Not aware if we do or don't. >> >> Thanks for all further information, I’d be happy to have upstream >> squashfs-tools with reproducible multithread support within OpenWrt! >> >> Sunshine, >> Paul >> >> [1]: https://github.com/squashfskit/squashfskit >> [2]: >> https://github.com/squashfskit/squashfskit/commit/5568b6a7d7db01c8f60a502d549a431a2b7219ae > > If you decide to switch back to squashfs-tools, and incorporate the > change to let it use all processors, we should probably respect the > number of jobs the user passes to make, and limit mksquashfs to use at > most that number of processors. This was suggested on #openwrt-devel, > after my initial submission. Unfortunately I have not found a way to do > so, as MAKE_JOBSERVER is exported in include/toplevel.mk, which is not > included in include/image.mk, and I am not familiar enough with this > part of the code to understand the impact of adding that include.
I’m happy to take care of that once we figured out how to proceed with our fork. > > Stijn _______________________________________________ openwrt-devel mailing list [email protected] https://lists.openwrt.org/mailman/listinfo/openwrt-devel
