From: Alexander Kanavin <a...@linutronix.de> TL;DR version:
with this, and the previous compression level changes I am seeing drastic speedups in package_write_rpm completion times: webkitgtk goes from 78 seconds to 37 seconds glibc-locale goes from 399 seconds to 58 seconds (!) The long version: rpm uses multithreading for two purposes: - spawning compressors (which are nowadays themselves multi-threaded, so the feature is not as useful as it once was) - parallel file classification While the former behaves well on massively parallel CPUs (it was written and verified here :), the latter was then added by upstream and only benchmarked on their very old, slow laptop, apparently: https://github.com/rpm-software-management/rpm/commit/41f0e214f2266f02d6185ba11f797716de8125d4 On anything more capable it starts showing pathologic behavior, presumably from spawning massive amount of very short-lived threads, and then having to synchronize them. For example classifying glibc-locale takes 5m20s with 256 threads (default on my machine!) 1m49s with 64 threads 59s with 16 threads 48s with 8 threads Even a more typical recipe like webkitgtk is affected: 47s with 256 threads 32s with 64 threads 27s with 16 or 8 threads I have found that the optimal amount is actually four: this also means that only four compressors are running at a time, but as they're themselves using threads, and typical recipes are dominated by just two or three large packages, this does not affect overall completion time. Signed-off-by: Alexander Kanavin <a...@linutronix.de> Signed-off-by: Steve Sakoman <st...@sakoman.com> --- meta/classes/package_rpm.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass index 198a6d0afd..ec31adf967 100644 --- a/meta/classes/package_rpm.bbclass +++ b/meta/classes/package_rpm.bbclass @@ -680,6 +680,7 @@ python do_package_rpm () { cmd = cmd + " --define '_use_internal_dependency_generator 0'" cmd = cmd + " --define '_binaries_in_noarch_packages_terminate_build 0'" cmd = cmd + " --define '_build_id_links none'" + cmd = cmd + " --define '_smp_ncpus_max 4'" cmd = cmd + " --define '_source_payload %s'" % rpmbuild_compmode cmd = cmd + " --define '_binary_payload %s'" % rpmbuild_compmode cmd = cmd + " --define 'clamp_mtime_to_source_date_epoch 1'" -- 2.34.1
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#207954): https://lists.openembedded.org/g/openembedded-core/message/207954 Mute This Topic: https://lists.openembedded.org/mt/109811171/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
