I extended the reasoning in the commit message and merged this as discussed in IRC
> On 28. Feb 2022, at 11:49, Paul Spooren <[email protected]> wrote: > > > >> On 28. Feb 2022, at 12:35, Stijn Tintel <[email protected]> wrote: >> >> On 28/02/2022 13:22, Paul Spooren wrote: >>> Hi team, >>> >>>> On 19. Feb 2022, at 19:21, Phillip Lougher <[email protected]> >>>> wrote: >>>> >>>> On Sat, Feb 19, 2022 at 4:01 PM Felix Fietkau <[email protected]> wrote: >>>>> On 19.02.22 16:54, Stijn Tintel wrote: >>>>>> Drop the -processors argument from the mksquashfs4 call, so it will use >>>>>> all available processors. This dramatically reduces the time to create >>>>>> squashfs filesystems. >>>>>> >>>>>> The times below are observed when building an image for my main router, >>>>>> the WatchGuard Firebox M300 (qoriq target): >>>>>> >>>>>> Before: >>>>>> real 4m45,973s >>>>>> >>>>>> After: >>>>>> real 0m23,497s >>>>>> >>>>>> Signed-off-by: Stijn Tintel <[email protected]> >>>>> We need to verify that this doesn't break reproducible builds... >>>>> >>>> It won't break reproducible builds. Since Mksquashfs version 4.4 the >>>> ordering is guaranteed to be the same, irrespective of how many >>>> threads or processors are used. >>> I think OpenWrt uses `squashfskit`[1] which is a fork some squashfs-tools >>> version. I think it happened due to stalled development some years ago. >>> Anyway, if squashfs-tools is actively maintained we should consider moving >>> back to the upstream version. >>> >>> Some time ago I already wanted to make the move but squashfskit uses some >>> downstream patches which expose more compression options, are those now >>> part of squashfs-tools[2]? If not, do we need those anyway? >> Not aware if we do or don't. >>> >>> Thanks for all further information, I’d be happy to have upstream >>> squashfs-tools with reproducible multithread support within OpenWrt! >>> >>> Sunshine, >>> Paul >>> >>> [1]: https://github.com/squashfskit/squashfskit >>> [2]: >>> https://github.com/squashfskit/squashfskit/commit/5568b6a7d7db01c8f60a502d549a431a2b7219ae >> >> If you decide to switch back to squashfs-tools, and incorporate the >> change to let it use all processors, we should probably respect the >> number of jobs the user passes to make, and limit mksquashfs to use at >> most that number of processors. This was suggested on #openwrt-devel, >> after my initial submission. Unfortunately I have not found a way to do >> so, as MAKE_JOBSERVER is exported in include/toplevel.mk, which is not >> included in include/image.mk, and I am not familiar enough with this >> part of the code to understand the impact of adding that include. > > I’m happy to take care of that once we figured out how to proceed with our > fork. > >> >> Stijn _______________________________________________ openwrt-devel mailing list [email protected] https://lists.openwrt.org/mailman/listinfo/openwrt-devel
