Re: [Rpm-maint] [rpm-software-management/rpm] Run file verification in parallel threads (#703)

2019-05-10 Thread Igor Gnatenko
And this is without your PR with dropping caches: ``` ⋊> ~/P/u/rpm on master ⨯ time ./rpm -Va >/dev/null 15:29:57 Command exited with non-zero status 1 53.21user 12.86system

Re: [Rpm-maint] [rpm-software-management/rpm] Run file verification in parallel threads (#703)

2019-05-10 Thread Igor Gnatenko
With your PR (with/without dropping caches): ``` ⋊> ~/P/u/rpm on master ⨯ time ./rpm -Va >/dev/null 15:26:03 Command exited with non-zero status 1 297.33user 12.27system

Re: [Rpm-maint] [rpm-software-management/rpm] Run file verification in parallel threads (#703)

2019-05-10 Thread Panu Matilainen
@pmatilai pushed 1 commit. b5da21861ad70b9df5c2cd5560a921251a5778d7 fixup! Run file verification in parallel threads -- You are receiving this because you are subscribed to this thread. View it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Run file verification in parallel threads (#703)

2019-05-10 Thread Panu Matilainen
Hmm, couple of tests are failing: 1) because errno isn't stored in the verify results, but that only affects the produced messages 2) ghost etc skipping isn't handled in the second loop -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it

Re: [Rpm-maint] [rpm-software-management/rpm] Run file verification in parallel threads (#703)

2019-05-10 Thread Igor Gnatenko
If you could provide some script for testing this, I can do some benchmarks. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[Rpm-maint] [rpm-software-management/rpm] Run file verification in parallel threads (#703)

2019-05-10 Thread Panu Matilainen
While we're playing with parallelising things... On my rusty old laptop, this tends to just slow things down unless things are hot in cache, in which case it roughly halves the running time. Might be worth more on SSD based setup, I'd be interested to hear results on such systems. For now, this

Re: [Rpm-maint] [rpm-software-management/rpm] Run file verification in parallel threads (#703)

2019-05-10 Thread Panu Matilainen
Dunno what there is to script. Try 'time rpm -Va' with and without, add "echo 1 > /proc/sys/vm/drop_caches" in between to see what happens with cold caches? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Correct an emitted error for built-in macro %load (#702)

2019-05-10 Thread Panu Matilainen
Thing is, for whatever reason packagers are keen to do whatever crazy things they can get away with in specs, often for no apparent reason. We don't change rpm just to appease to that stuff. Patch to document the intended and implemented behavior is welcome, and the missing \n in the message

Re: [Rpm-maint] [rpm-software-management/rpm] RPM does not de-duplicate Requires (#671)

2019-05-10 Thread Panu Matilainen
No further action -> closing -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/671#issuecomment-491262095___ Rpm-maint mailing

Re: [Rpm-maint] [rpm-software-management/rpm] RPM does not de-duplicate Requires (#671)

2019-05-10 Thread Panu Matilainen
Closed #671. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/671#event-2332894385___ Rpm-maint mailing list

Re: [Rpm-maint] [rpm-software-management/rpm] Correct an emitted error for built-in macro %load (#702)

2019-05-10 Thread Panu Matilainen
In the comment your patch removed: /* Print failure iff %{load:...} or %{!?load:...} */ -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Correct an emitted error for built-in macro %load (#702)

2019-05-10 Thread pavlinamv
Please where is this usage of %{?load:...} described (not counting macro.c)? I see %{?load:...} in spec files ruby.spec or vagrant.spec where it is used for another purpose: to be able to build SRPM on older Fedora, where the older RPM has not %load defined. And it looks that the packager

Re: [Rpm-maint] [rpm-software-management/rpm] Correct an emitted error for built-in macro %load (#702)

2019-05-10 Thread Panu Matilainen
NAK. As the comment indicates, this behavior is intentional to optionally allow silent failure, eg to try to load a macro file that is not there in all cases. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Correct an emitted error for built-in macro %load (#702)

2019-05-10 Thread Panu Matilainen
Closed #702. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/702#event-2332827595___ Rpm-maint mailing list

Re: [Rpm-maint] [rpm-software-management/rpm] Stop papering over the security disaster known as prelink (#701)

2019-05-10 Thread Panu Matilainen
@pmatilai pushed 1 commit. ec033f4329e404e8d37e8de804e3fe1006a6536f Eliminate the hysterical filesize retval from rpmDoDigest() now that we can -- You are receiving this because you are subscribed to this thread. View it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Stop papering over the security disaster known as prelink (#701)

2019-05-10 Thread Panu Matilainen
Doh, and another force-push to eliminate redundant Ftell() -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Stop papering over the security disaster known as prelink (#701)

2019-05-10 Thread Panu Matilainen
Forgot to eliminate libelf dependencies in the first round, hence the forced push. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[Rpm-maint] [rpm-software-management/rpm] Stop papering over the security disaster known as prelink (#701)

2019-05-10 Thread Panu Matilainen
Back in the turn of the century somebody thought it was a neat idea to completely compromise system security to improve program start-up start-up times a wee bit. Since then, people have thankfully started coming to their senses and removed prelink from distros entirely. Lets stop papering over

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Allow packagers to specify individual scriptlets as critical" (#700)

2019-05-10 Thread Panu Matilainen
The patch aside (which I believe is necessary at any rate), discussion on what to do about this would be welcome. RHEL carries a patch to enable a "strict mode" for scriptlet failures (ie bring back the duplicates on scriptlet-failures), and Suse seems to have something similar. -- You are

[Rpm-maint] [rpm-software-management/rpm] Fix memleak in rpmfcApplyInternal() in standalone operation (eg rpmdeps) (#699)

2019-05-10 Thread Panu Matilainen
When called in spec context, the package structs are properly freed but in rpmdeps context, commit 49f2bb7d8fd91f2d8b22bf7128fd3defe4ed5434 only added a "dirty kludgery" to make it not blow up. This causes the rpmds structures created in rpmfcApplyInternal() to leak memory. Make freePackage()

[Rpm-maint] [rpm-software-management/rpm] Revert "Allow packagers to specify individual scriptlets as critical" (#700)

2019-05-10 Thread Panu Matilainen
This alleged solution to the problem of not all scriptlet errors being reflected in our exit code is not a solution at all, it just pushes the problem to somebody elses lap and introduces inconsistency and an untracked incompatibility as well. This reverts commit

Re: [Rpm-maint] [rpm-software-management/rpm] Remove nonexistent macro "Q" from the table of builtin macros (#697)

2019-05-10 Thread Panu Matilainen
Always check for background first. This appears to have been accidentally added in commit 9343ecd94cd873e6dc1c06428975163cbb9cf9af, which should be mentioned in the commit message. Other than that it's obviously fine. -- You are receiving this because you are subscribed to this thread. Reply

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
In the commit to pass rpmts object ro rpmSpecBuild() there's a stray empty line added to build/build.c, but it seems to get lost somewhere in the birds-eye view that GH considers review mode. The birds-eye view of things is useful for, well, a birds-eye view, but bulls*** for reviewing patch

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > if (rstreqn(N, "rpmlib(", sizeof("rpmlib(")-1)) { - if (tagN != RPMTAG_REQUIRENAME) return 1; + if (tagN != RPMTAG_REQUIRENAME && + (tagN == RPMTAG_PROVIDENAME && !(Flags & RPMSENSE_RPMLIB))) + return 1; Fix the

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > rpmlog(RPMLOG_NOTICE, _("\n\nRPM build errors:\n")); rpmlogPrint(NULL); } rpmugFree(); - +if (missing_buildreqs && !rc) { + rc = RPMRC_MISSINGBUILDREQUIRES; +} +if (rc == RPMRC_FAIL) + rc = 1; Please use

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -273,20 +287,6 @@ static struct poptOption optionsTable[] = { POPT_TABLEEND }; -static int checkSpec(rpmts ts, rpmSpec spec) -{ -int rc; -rpmps ps = rpmSpecCheckDeps(ts, spec); - -if (ps) { - rpmlog(RPMLOG_ERR, _("Failed

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -273,20 +287,6 @@ static struct poptOption optionsTable[] = { POPT_TABLEEND }; -static int checkSpec(rpmts ts, rpmSpec spec) -{ -int rc; -rpmps ps = rpmSpecCheckDeps(ts, spec); - -if (ps) { - rpmlog(RPMLOG_ERR, _("Failed

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -894,6 +895,9 @@ static rpmSpec parseSpec(const char *specFile, > rpmSpecFlags flags, case PART_PREP: parsePart = parsePrep(spec); break; + case PART_BUILDREQUIRES: + parsePart =

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > goto exit; + if (what & RPMBUILD_BUILDREQUIRES) +rc = doBuildRequires(spec, test); + if ((what & RPMBUILD_CHECKBUILDREQUIRES) && + (rc == RPMRC_MISSINGBUILDREQUIRES)) + rc =

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > +/* add results to requires of the srpm */ +argvSplit(, getStringBuf(sb_stdout), "\n\r"); +outc = argvCount(output); + +if (!outc) { + rc = RPMRC_OK; + goto exit; +} + +for (int i = 0; i < outc; i++) { +

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -213,24 +278,58 @@ static rpmRC buildSpec(BTA_t buildArgs, rpmSpec spec, > int what) } else { int didBuild = (what & (RPMBUILD_PREP|RPMBUILD_BUILD|RPMBUILD_INSTALL)); + if (!spec->buildrequires && (what &

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -16,6 +16,8 @@ #include "debug.h" +#define RPMRC_MISSINGBUILDREQUIRES 11 Hmm, we're actually returning this in the API so I think it should actually go into rpmbuild.h -- You are receiving this because you are subscribed to this thread.

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > +for (int i = 0; i < outc; i++) { + parseRCPOT(spec, spec->sourcePackage, output[i], RPMTAG_REQUIRENAME, 0, 0, addReqProvPkg, NULL); +} + +rpmdsPutToHeader(*packageDependencies(spec->sourcePackage, RPMTAG_REQUIRENAME),

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -172,9 +179,67 @@ rpmRC doScript(rpmSpec spec, rpmBuildFlags what, const > char *name, return rc; } -static rpmRC buildSpec(BTA_t buildArgs, rpmSpec spec, int what) +static rpmRC doBuildRequires(rpmSpec spec, int test) +{ +StringBuf

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > + +if (!spec->buildrequires) { + rc = RPMRC_OK; + goto exit; +} + +if ((rc = doScript(spec, RPMBUILD_BUILDREQUIRES, "%generate_buildrequires", + getStringBuf(spec->buildrequires), _stdout, test))) +

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -283,7 +284,7 @@ static rpmRC buildSpec(BTA_t buildArgs, rpmSpec spec, int > what) return rc; } -rpmRC rpmSpecBuild(rpmSpec spec, BTA_t buildArgs) +int rpmSpecBuild(rpmts ts, rpmSpec spec, BTA_t buildArgs) Just noting that this is an API

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -172,6 +172,7 @@ rpmRC doScript(rpmSpec spec, rpmBuildFlags what, const > char *name, return rc; } + Stray newline added. Like said in an earlier comment, the addition of rpmtsFromPyObject() could be merged with this commit because it

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -47,7 +47,7 @@ static rpmRC doRmSource(rpmSpec spec) * @todo Single use by %%doc in files.c prevents static. */ rpmRC doScript(rpmSpec spec, rpmBuildFlags what, const char *name, - const char *sb, int test) + const

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
Argh, I didn't mean to approve the whole thing, thought I could do it for the couple of earlier commits separately. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Dynamic Build Dependencies (#593)

2019-05-10 Thread Panu Matilainen
pmatilai approved this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/593#pullrequestreview-235945251___