Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-21 Thread Panu Matilainen
Merged #1180 into master. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/1180#event-3255406654___ Rpm-maint mailing list Rpm-mai

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-21 Thread Panu Matilainen
@pmatilai approved this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/1180#pullrequestreview-397157468___ Rpm

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-21 Thread Michal Domonkos
@dmnks pushed 1 commit. 7ab76d425d3660a3bfc83009f7cd77096bdb8881 fixup! build: prioritize large packages -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1180/files/25622ad3694c09e4e9662d387293d432d15e3

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-21 Thread Michal Domonkos
Good point on the backwards loop; it really shouldn't scream "look ma, reverse loop for no obvious reason, go figure out yourself!". Instead, I'll just make the reverse ordering explicit by simply inverting the `compareBinaries()` function, and push a fixup commit right away. -- You are receiv

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-21 Thread Panu Matilainen
Good catch on the size tag, totally managed to miss that. Looks fine by me, just one thing for consideration: since we want them largest first, might as well sort them that way too so the second for-loop doesn't need to walk backwards, which looks a bit more special than it actually is. Not that

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-20 Thread Michal Domonkos
@pmatilai Fixed and rebased. Note that I changed `RPMTAG_SIZE` to `RPMTAG_LONGSIZE` because I realized the former may not be set if the total size is greater than 32 bits (details [here](https://rpm.org/devel_doc/large_files.html)). -- You are receiving this because you are subscribed to this t

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-20 Thread Michal Domonkos
@dmnks pushed 1 commit. b5700cb7e707b4b38fa431d25f44df0ff7a5a9b2 build: prioritize large packages -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1180/files/ad337863bdeb9aed69224159e4be27ab8d2749f4..b57

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-15 Thread Panu Matilainen
FWIW, I can easily reproduce similar timings here. Using `rpmbuild -bb --noclean --short-circuit' to limit it to only the packaging phase, I get ~58s before the patch and ~44s after it. It's quite a nice win. BTW bonus points for the graphs in the commit message, they illustrate the problem rea

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-15 Thread Panu Matilainen
@pmatilai commented on this pull request. > rpmRC packageBinaries(rpmSpec spec, const char *cookie, int cheating) { rpmRC rc = RPMRC_OK; Package pkg; +Package *tasks; +int i = 0; + +for (pkg = spec->packages; pkg != NULL; pkg = pkg->next) +i++; +tasks = xcallo

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-15 Thread Panu Matilainen
@pmatilai commented on this pull request. > @@ -726,16 +727,37 @@ static rpmRC packageBinary(rpmSpec spec, Package pkg, > const char *cookie, int ch return rc; } +static int compareBinaries(const void *p1, const void *p2) { +Package pkg1 = *(Package *)p1; +Package pkg2 = *(Packa

Re: [Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-14 Thread Michal Domonkos
A testing spec file that I used to benchmark this patch: ``` %global _smp_build_nthreads 2 Name: foo Version:1.0 Release:1%{?dist} Summary:testing package for parallel building License:GPL %description %package light1 Summary:light subpackage 1

[Rpm-maint] [rpm-software-management/rpm] build: prioritize large packages (#1180)

2020-04-14 Thread Michal Domonkos
Binary packages come in different sizes and so their build time can vary greatly. Dynamic scheduling, which we currently use for parallel building, is a good strategy to combat such differences and load-balance the available CPU cores. That said, knowing that the build time of a package is propor