Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Tue, 1 Sep 2015 11:58:07 -0400 (EDT), Tomas Mlcoch wrote: > RPM itself expects epoch to be number: > https://github.com/rpm-software-management/rpm/blob/140744377b019e0de81d76d0931c32228d2ed57e/lib/rpmvercmp.c#L124-L143 > > YUM expects epoch to be integral number: > https://github.com/rpm-software-management/yum/blob/e17424b34ed8409803b5e702b24f6d0051ca99ca/yum/rpmsack.py#L922 > > Fedora's doc states that "Epoch" is numeric field: > https://fedoraproject.org/wiki/Packaging:Guidelines#Use_of_Epochs > > This rpmbuild acceptance of a non-numerical epoch is obviously a bug. > I would prefer one of my proposed solutions instead of accepting > non-numerical epoch values. > It is beyond my time and energy to "fight" bad package updates that don't fix the problem: https://bodhi.fedoraproject.org/updates/FEDORA-2015-735f979d01 What the new createrepo_c in that update does, it ignores the broken Requires altogether, skips it and doesn't add it to the repo metadata [1]. Effectively, it hides the problem even more than before when it replaced the unescaped RPM macro with a '0', changing the dependency on-the-fly, pretending that there is no problem when in fact it's broken in the binary package that cannot be installed. Hiding such breakage from the metadata hides it also from depsolvers and tools like repoclosure that work on the metadata themselves before passing a transaction to RPM. They would _not_ be able to find such a problem. IMHO, this is bad as before. I cannot estimate how often an unescaped epoch macro will break deps in the future again, but a package like that should not be published with repo metadata pretending that everything would be fine. If you fear that createrepo_c exiting with error condition *instead* would happen too often, then apparently the problem is perceived as an even worse one. Hard to believe. The corresponding RFE for "rpm" is: https://bugzilla.redhat.com/1251453 [1] --- d851a23707b6978a0f4ad1439a0b10e2f7d99942be129b12bcde0a7bcb106e0c-primary.xml.OLD 2015-10-22 00:45:03.0 +0200 +++ 944af4952f9642bfe6dc2643866de0befd3415373f459be22f898041c69ac83f-primary.xml2015-10-22 00:45:33.0 +0200 @@ -109,7 +109,6 @@ - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Thu, 22 Oct 2015 08:41:58 -0400 (EDT), Tomas Mlcoch wrote:
> 1)
> There were several points of failure:
> * Developer who made a typo in spec file which results into bad dependency.
> * rpmbuild which built an rpm with broken dependency.
> * Fedora's automated depcheck that wasn't able to find the dependency issue.
> Why still ranting about createrepo_c?
You detect invalid input data => you exit with error condition.
Easy as that.
Eventually, the problem may be detected earlier. E.g. by rpmbuild.
Would Taskotron/AutoQA be capable to detect it? It does provide means
to examine binary rpms, doesn't it? How else could it run rpmlint?
How much effort would it be to run createrepo_c on the binary rpms
and reject them if createrepo_c complains?
> 2)
> Why are you saying that createrepo_c is hiding the issue?
> That's a lie.
No, the tool is lying.
The tool creates metadata that passes repoclosure when in fact the
packages in the repo contain a dependency that cannot be resolved.
Turning a string "%{epoch}" into "0" is hiding the invalid input data.
By definition, no Epoch in versioned dependencies is equal to Epoch 0.
The tool replaces am unresolvable dependency with a resolvable one.
The new "fix" is hiding the issue, because it deletes an entire "Requires"
statement.
> After the problem was discovered, I wrote a patch
> which added a warning that informs user about such suspicious dependency.
Which user? Who would watch out for such a warning in koji, Copr and
other build systems?
> *) The approach you are fighting for - exit when a package has a
> bad dep - in the current state of things (see 1)) would be a HUGE
> SECURITY ISSUE for fedora update system.
Do you realize I've opened an RFE for "rpm" in August already?
> Anyone could build a package with intentionally broken dep and
> block ALL updates for WHOLE fedora for ALL fedora users.
Anyone? No. Only existing packagers. And for how long? And nobody would
notice and remove the package?
With that perspective, an existing packager can attack the infrastructure
in other ways, too. It's possible to break buildroot creation. It's
possible to obsolete packages you don't own.
> He would just build a package with broken dep, send it directly to
> fedora stable (skip fedora updates) repo and then.. whole
> fedora stable repo generation would be doomed and manual
> releng action would be required.
Ridiculous. Running createrepo_c on individual updates would be very
fast and would ensure they are not rejected by createrepo_c.
You're trying much to approach the problem for the wrong side, IMO.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
- Original Message - > On Tue, 1 Sep 2015 11:58:07 -0400 (EDT), Tomas Mlcoch wrote: > > > RPM itself expects epoch to be number: > > https://github.com/rpm-software-management/rpm/blob/140744377b019e0de81d76d0931c32228d2ed57e/lib/rpmvercmp.c#L124-L143 > > > > YUM expects epoch to be integral number: > > https://github.com/rpm-software-management/yum/blob/e17424b34ed8409803b5e702b24f6d0051ca99ca/yum/rpmsack.py#L922 > > > > Fedora's doc states that "Epoch" is numeric field: > > https://fedoraproject.org/wiki/Packaging:Guidelines#Use_of_Epochs > > > > This rpmbuild acceptance of a non-numerical epoch is obviously a bug. > > I would prefer one of my proposed solutions instead of accepting > > non-numerical epoch values. > > > > It is beyond my time and energy to "fight" bad package updates that don't > fix the problem: > > https://bodhi.fedoraproject.org/updates/FEDORA-2015-735f979d01 > > What the new createrepo_c in that update does, it ignores the broken > Requires altogether, skips it and doesn't add it to the repo metadata [1]. > > Effectively, it hides the problem even more than before when it replaced > the unescaped RPM macro with a '0', changing the dependency on-the-fly, > pretending that there is no problem when in fact it's broken in the binary > package that cannot be installed. > > Hiding such breakage from the metadata hides it also from depsolvers and > tools like repoclosure that work on the metadata themselves before passing > a transaction to RPM. They would _not_ be able to find such a problem. > > IMHO, this is bad as before. I cannot estimate how often an unescaped > epoch macro will break deps in the future again, but a package like that > should not be published with repo metadata pretending that everything > would be fine. If you fear that createrepo_c exiting with error condition > *instead* would happen too often, then apparently the problem is perceived > as an even worse one. Hard to believe. > > The corresponding RFE for "rpm" is: https://bugzilla.redhat.com/1251453 > > [1] Michael, few points: 1) There were several points of failure: * Developer who made a typo in spec file which results into bad dependency. * rpmbuild which built an rpm with broken dependency. * Fedora's automated depcheck that wasn't able to find the dependency issue. Why still ranting about createrepo_c? 2) Why are you saying that createrepo_c is hiding the issue? That's a lie. After the problem was discovered, I wrote a patch which added a warning that informs user about such suspicious dependency. 3) I wrote a rationale why I choose the approach where createrepo_c doesn't exit immediately with error here [2] and here [3]. You have never commented on the rationale, you are just keep saying that the approach is wrong. I'll repeat my points again: *) createrepo_c tends to be part of critical systems for delivering updates - I just cannot break whole update system and require manual releng action just because some maintainer of some foobar package did a typo in spec of package which is used by two people in the world (where one is the maintainer itself and the second one is a person who installed it by mistake). *) Dependencies like "suggests", "enhances", etc. are not important and break whole update infrastructure because there is some optional package with a broken dep that is optional is not good. *) The approach you are fighting for - exit when a package has a bad dep - in the current state of things (see 1)) would be a HUGE SECURITY ISSUE for fedora update system. Anyone could build a package with intentionally broken dep and block ALL updates for WHOLE fedora for ALL fedora users. He would just build a package with broken dep, send it directly to fedora stable (skip fedora updates) repo and then.. whole fedora stable repo generation would be doomed and manual releng action would be required. I don't know if you don't see these security aspects or if you don't consider them important. But I won't to be the one who will introduce a security issue to Fedora update infrastructure. Tomas [1] https://github.com/rpm-software-management/createrepo_c/commit/3fd1fa5020025fc62440ac8ef977fda7b087bdb5#diff-77d6596729af517434dc3b649b5251ecR386 [2] https://lists.fedoraproject.org/pipermail/devel/2015-August/213882.html [3] https://bodhi.fedoraproject.org/updates/createrepo_c-0.9.1-1.fc22#comment-342067 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
- Original Message - > On Fri, 28 Aug 2015 04:12:20 -0400 (EDT), Tomas Mlcoch wrote: > > > > On Thu, 06 Aug 2015 13:15:02 +, Igor Gnatenko wrote: > > > > > > > We discussed with Jan Silhan yesterday. It looks like something broken > > > > in > > > > createrepo/createrepo_c in F22. So it's not dnf/yum/hawkey/libsolv > > > > issue. > > > > > > > > LOG: https://ignatenkobrain.fedorapeople.org/epoch_bug.log > > > > > > > > Also CCing Jan. > > > > > > Wow. createrepo is also affected. > > > > > > createrepo_c uses strtol() to accept only numbers as Epoch. Anything > > > else strol() cannot parse is ignored and defaults to "epoch=0" in the > > > repo metadata. This means it can break for typos as well as, not just > > > an undefined macro used as Epoch in a versioned dep. > > > > > > It couldn't find a comment in the source that would tell whether this > > > is by design. > > > > > > Hi Michael, > > > > what do you suggest? > > I can think about three reasonable solutions for createrepo_c here: > > > > > > 1) Abort and print error about bad (non-numerical) epoch > > > 2) Print warning about the bad epoch > > > 3) Print warning about bad epoch and skip the broken dep > > > X) Ignore the broken package > > Why are only numerical Epoch values accepted? > > What would break, if an Epoch recognized as "bad" would be replaced > with a string such as "-BROKEN-"? epoch="-BROKEN-" in the primary metdata. > > The version tags "ver" and "rel" attributes may also be non-numerical. > Why not "epoch", too? > > Else I think rpmbuild may be able to reject more non-numerical Epochs, > but as long as it doesn't, replacing an Epoch is wrong. > > Other layers that could watch for non-numerical Epochs: rpmlint + AutoQA. > RPM itself expects epoch to be number: https://github.com/rpm-software-management/rpm/blob/140744377b019e0de81d76d0931c32228d2ed57e/lib/rpmvercmp.c#L124-L143 YUM expects epoch to be integral number: https://github.com/rpm-software-management/yum/blob/e17424b34ed8409803b5e702b24f6d0051ca99ca/yum/rpmsack.py#L922 Fedora's doc states that "Epoch" is numeric field: https://fedoraproject.org/wiki/Packaging:Guidelines#Use_of_Epochs This rpmbuild acceptance of a non-numerical epoch is obviously a bug. I would prefer one of my proposed solutions instead of accepting non-numerical epoch values. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Fri, 28 Aug 2015 04:12:20 -0400 (EDT), Tomas Mlcoch wrote: On Thu, 06 Aug 2015 13:15:02 +, Igor Gnatenko wrote: We discussed with Jan Silhan yesterday. It looks like something broken in createrepo/createrepo_c in F22. So it's not dnf/yum/hawkey/libsolv issue. LOG: https://ignatenkobrain.fedorapeople.org/epoch_bug.log Also CCing Jan. Wow. createrepo is also affected. createrepo_c uses strtol() to accept only numbers as Epoch. Anything else strol() cannot parse is ignored and defaults to epoch=0 in the repo metadata. This means it can break for typos as well as, not just an undefined macro used as Epoch in a versioned dep. It couldn't find a comment in the source that would tell whether this is by design. Hi Michael, what do you suggest? I can think about three reasonable solutions for createrepo_c here: 1) Abort and print error about bad (non-numerical) epoch 2) Print warning about the bad epoch 3) Print warning about bad epoch and skip the broken dep X) Ignore the broken package Why are only numerical Epoch values accepted? What would break, if an Epoch recognized as bad would be replaced with a string such as -BROKEN-? epoch=-BROKEN- in the primary metdata. The version tags ver and rel attributes may also be non-numerical. Why not epoch, too? Else I think rpmbuild may be able to reject more non-numerical Epochs, but as long as it doesn't, replacing an Epoch is wrong. Other layers that could watch for non-numerical Epochs: rpmlint + AutoQA. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
- Original Message - On Thu, 06 Aug 2015 13:15:02 +, Igor Gnatenko wrote: We discussed with Jan Silhan yesterday. It looks like something broken in createrepo/createrepo_c in F22. So it's not dnf/yum/hawkey/libsolv issue. LOG: https://ignatenkobrain.fedorapeople.org/epoch_bug.log Also CCing Jan. Wow. createrepo is also affected. createrepo_c uses strtol() to accept only numbers as Epoch. Anything else strol() cannot parse is ignored and defaults to epoch=0 in the repo metadata. This means it can break for typos as well as, not just an undefined macro used as Epoch in a versioned dep. It couldn't find a comment in the source that would tell whether this is by design. Hi Michael, what do you suggest? I can think about three reasonable solutions for createrepo_c here: 1) Abort and print error about bad (non-numerical) epoch -- Pros: * Clearly states that there are erroneous data on input. Cons: * For services like Koji, it would break whole repodata generation. Even a rarely used sub-package that is almost useless and used only by few users could break repodata generation and block release of critical security fixes for ALL fedora users and would require a manual fix (removal of the broken package) from releng side. * Bad dep (e.g. bad epoch) doesn't have to necessary break stuff. For example provides are not always used by other packages. Also for rich deps like suggests, enhances, etc. a bad dependency isn't a big deal and it would be foolish to stall a security update just because some package has a broken dependency because that dep may not be used at all. 2) Print warning about the bad epoch -- Pros: * One broken package doesn't block updates. Cons: * I don't see any important 3) Print warning about bad epoch and skip the broken dep -- Pros: * One broken package doesn't block updates. * Output repodata doesn't contain obviously bad data (deps) Const: * I don't see any important Option that is not acceptable: X) Ignore the broken package -- Cons: * As I said above some deps aren't used at all and leave a whole package out just because of a bad dep could make much more mess. Tomas -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On 08/28/2015 01:15 PM, Michael Schwendt wrote: The version tags ver and rel attributes may also be non-numerical. Why not epoch, too? I haven't looked into the sources, but IIRC, inside of rpm, while rel, ver etc. are strings, epoch is an integer. AFAIR, there are APIs which return the epoch as a *int, which may be NULL or a valid pointer. I.e. callers will have to special case accessing these pointers. Seems to me as if something doesn't get this right. (FWIW: All this was an FAQ 10 years ago :) ) Ralf -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Fri, 28 Aug 2015 13:59:12 +0200, Ralf Corsepius wrote: The version tags ver and rel attributes may also be non-numerical. Why not epoch, too? I haven't looked into the sources, but IIRC, inside of rpm, while rel, ver etc. are strings, epoch is an integer. See: https://lists.fedoraproject.org/pipermail/devel/2015-August/213209.html https://lists.fedoraproject.org/pipermail/devel/2015-August/213208.html Plus, RPM would not get to see repository metadata, but only downloaded packages, which contain the same bad Epoch values. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Fri, 28 Aug 2015 16:36:51 +0200, Ralf Corsepius wrote:
See:
https://lists.fedoraproject.org/pipermail/devel/2015-August/213209.html
https://lists.fedoraproject.org/pipermail/devel/2015-August/213208.html
Bugs ... an undefined epoch is supposed to be treated as 0.
No, that's the old case. No Epoch = Epoch 0.
Here it's an undefined %{epoch} macro in a versioned dependency,
making it a string that isn't expanded. It must not be treated as 0,
because that pretends the dependency resolves fine when in fact it
doesn't. The package cannot be installed. Only RPM notices it, and
the transaction check comes rather late in the game after downloading
lots of packages. Ugly.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On 08/28/2015 02:18 PM, Michael Schwendt wrote: On Fri, 28 Aug 2015 13:59:12 +0200, Ralf Corsepius wrote: The version tags ver and rel attributes may also be non-numerical. Why not epoch, too? I haven't looked into the sources, but IIRC, inside of rpm, while rel, ver etc. are strings, epoch is an integer. See: https://lists.fedoraproject.org/pipermail/devel/2015-August/213209.html https://lists.fedoraproject.org/pipermail/devel/2015-August/213208.html Bugs ... an undefined epoch is supposed to be treated as 0. I guess you also recall - the ability of some tools not to be able to cope with this was the origin why Fedora.us and early Fedoras once mandated Epoch: 0. Plus, RPM would not get to see repository metadata, but only downloaded packages, which contain the same bad Epoch values. Ralf -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On 08/28/2015 05:32 PM, Michael Schwendt wrote: On Fri, 28 Aug 2015 16:36:51 +0200, Ralf Corsepius wrote: See: https://lists.fedoraproject.org/pipermail/devel/2015-August/213209.html https://lists.fedoraproject.org/pipermail/devel/2015-August/213208.html Bugs ... an undefined epoch is supposed to be treated as 0. No, that's the old case. No Epoch = Epoch 0. Both cases are equal on the rpm level. All tools which use rpm need to set their internal representation of an epoch to 0 if rpm returns an undefined epoch. Ralf -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Fri, 28 Aug 2015 18:06:27 +0200, Ralf Corsepius wrote:
See:
https://lists.fedoraproject.org/pipermail/devel/2015-August/213209.html
https://lists.fedoraproject.org/pipermail/devel/2015-August/213208.html
Bugs ... an undefined epoch is supposed to be treated as 0.
No, that's the old case. No Epoch = Epoch 0.
Both cases are equal on the rpm level.
All tools which use rpm need to set their internal representation of an
epoch to 0 if rpm returns an undefined epoch.
It seems to me we're talking past eachother. It would be wrong to assume
that the undefined %{epoch} is supposed to 0.
If rpmbuild rejected a spec file that uses %epoch without defining an
Epoch tag, that would be fine. So far, it doesn't.
$ rpm -qpR blktap-devel-3.0.0-3.fc23.git0.9.2.x86_64.rpm |grep ep
blktap(x86-64) = %{epoch}:3.0.0-3.fc23.git0.9.2
https://lists.fedoraproject.org/pipermail/devel/2015-July/212963.html
That's what this topic is about. Substituting the unexpanded macro with
a 0 in the repo metadata tries to hide breakage under the carpet. It
breaks when passing on the packages to RPM.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Thu, 06 Aug 2015 13:15:02 +, Igor Gnatenko wrote: We discussed with Jan Silhan yesterday. It looks like something broken in createrepo/createrepo_c in F22. So it's not dnf/yum/hawkey/libsolv issue. LOG: https://ignatenkobrain.fedorapeople.org/epoch_bug.log Also CCing Jan. Wow. createrepo is also affected. createrepo_c uses strtol() to accept only numbers as Epoch. Anything else strol() cannot parse is ignored and defaults to epoch=0 in the repo metadata. This means it can break for typos as well as, not just an undefined macro used as Epoch in a versioned dep. It couldn't find a comment in the source that would tell whether this is by design. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Thu, 2015-08-06 at 18:37 +0200, Michael Schwendt wrote:
On Thu, 06 Aug 2015 13:15:02 +, Igor Gnatenko wrote:
We discussed with Jan Silhan yesterday. It looks like something broken in
createrepo/createrepo_c in F22. So it's not dnf/yum/hawkey/libsolv issue.
LOG: https://ignatenkobrain.fedorapeople.org/epoch_bug.log
Also CCing Jan.
Wow. createrepo is also affected.
createrepo_c uses strtol() to accept only numbers as Epoch. Anything
else strol() cannot parse is ignored and defaults to epoch=0 in the
repo metadata. This means it can break for typos as well as, not just
an undefined macro used as Epoch in a versioned dep.
It couldn't find a comment in the source that would tell whether this
is by design.
I'd guess it is. I'd say the root bug here is in rpmbuild, this is what
it does for Epoch itself:
case RPMTAG_EPOCH: {
SINGLE_TOKEN_ONLY;
uint32_t epoch;
if (parseUnsignedNum(field, epoch)) {
rpmlog(RPMLOG_ERR,
_(line %d: Epoch field must be an unsigned number: %
s\n),
spec-lineNum, spec-line);
goto exit;
}
...it should do the same kind of thing for epoch's within E:V-R data
too.
signature.asc
Description: This is a digitally signed message part
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Wed, 5 Aug 2015 10:26:38 -0600, Kevin Fenzi wrote: There is a dnf repoclosure plugin, but not sure how well it works off hand. It seems to be completely broken. :-( It reports lots of available shared libs as unresolved deps. - https://mschwendt.fedorapeople.org/tmp/f22-dnf-repoclosure.txt yum-utils' repoclosure - https://bugzilla.redhat.com/1251037 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
We discussed with Jan Silhan yesterday. It looks like something broken in createrepo/createrepo_c in F22. So it's not dnf/yum/hawkey/libsolv issue. LOG: https://ignatenkobrain.fedorapeople.org/epoch_bug.log Also CCing Jan. On Thu, Aug 6, 2015 at 4:03 PM Michael Schwendt mschwe...@gmail.com wrote: On Wed, 5 Aug 2015 10:26:38 -0600, Kevin Fenzi wrote: There is a dnf repoclosure plugin, but not sure how well it works off hand. It seems to be completely broken. :-( It reports lots of available shared libs as unresolved deps. - https://mschwendt.fedorapeople.org/tmp/f22-dnf-repoclosure.txt yum-utils' repoclosure - https://bugzilla.redhat.com/1251037 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct -- -Igor Gnatenko -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Thu, 6 Aug 2015 18:00:23 +, Zbigniew Jędrzejewski-Szmek wrote:
It couldn't find a comment in the source that would tell whether this
is by design.
Does this really matter? If it's by design, then the design is wrong.
If not, than the implementation is wrong.
It doesn't matter much, except that I fear dilettantism related to
handling corner-cases and lack of error handling. :-/ Not verifying
input data is the reason for many security vulnerabilities even.
I thought that in the XML repodata, epoch=-1 instead of epoch=0 would
have been possible, too, and would not resolve. (whereas no Epoch and Epoch
0
are treated as equal by definition)
However, rpmbuild doesn't care much about the EVR in versioned deps.
It accepts weird EVR specifiers, such as
Requires: badepoch = 1a:2b:3c-4-0-1-2-3
and successfully builds the package, only printing this:
# warning: line 13: Invalid version (double separator ':'):
1a:2b:3c-4-0-1-2-3: Requires: badepoch = 1a:2b:3c-4-0-1-2-3
It even happily builds a spec file that contains
Requires: badepoch = -1
and
Requires: badepoch = -1:1
and createrepo_c turns that into epoch=-1 ver=1. *sigh* ;-)
On the contrary, rpmbuild rejects spec files with a non-positive Epoch.
So, errors (even if they may be only corner-cases), propagate through the
various tools causing unpredictable symptoms.
Even after a real undefined %epoch in released packages, one can only
hope that some of this is purely theoretical:
Requires: %{depname} = %{depepoch}x:%{depminver}
It's possible today. An accidentally inserted 'x' at the wrong place,
rpmbuild would happily build it, and repoclosure would not detect it
as unresolvable, because in the metadata it becomes epoch=0.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Thu, Aug 06, 2015 at 06:37:29PM +0200, Michael Schwendt wrote: On Thu, 06 Aug 2015 13:15:02 +, Igor Gnatenko wrote: We discussed with Jan Silhan yesterday. It looks like something broken in createrepo/createrepo_c in F22. So it's not dnf/yum/hawkey/libsolv issue. LOG: https://ignatenkobrain.fedorapeople.org/epoch_bug.log Also CCing Jan. Wow. createrepo is also affected. createrepo_c uses strtol() to accept only numbers as Epoch. Anything else strol() cannot parse is ignored and defaults to epoch=0 in the repo metadata. This means it can break for typos as well as, not just an undefined macro used as Epoch in a versioned dep. It couldn't find a comment in the source that would tell whether this is by design. Does this really matter? If it's by design, then the design is wrong. If not, than the implementation is wrong. (Nevertheless, I think it's not by design, but because strtol() is so hard to use correctly. I think it's fair to say to it is designed to be used incorrectly. Systemd has a bunch of wrappers for strto*() that try to answer the question was the conversion successful? [1], and it's sad that they are needed and that they are so complex.) [1] https://github.com/systemd/systemd/blob/73974f6768e#L406 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
Hi folks,
I can't reproduce this issue.
$ sudo dnf install
https://kojipkgs.fedoraproject.org//packages/blktap/3.0.0/3.fc23.git0.9.2/x86_64/blktap-devel-3.0.0-3.fc23.git0.9.2.x86_64.rpm
https://kojipkgs.fedoraproject.org//packages/blktap/3.0.0/3.fc23.git0.9.2/x86_64/blktap-3.0.0-3.fc23.git0.9.2.x86_64.rpm
-C
Last metadata expiration check performed 1:22:41 ago on Wed Aug 5
11:51:08 2015.
The downloaded packages were saved in cache till the next successful
transaction.
You can remove cached packages by executing 'dnf clean packages'
Error: nothing provides blktap(x86-64) =
%{epoch}:3.0.0-3.fc23.git0.9.2 needed by
blktap-devel-3.0.0-3.fc23.git0.9.2.x86_64
(try to add '--allowerasing' to command line to replace conflicting packages)
Please let us know:
$ rpm -q libsolv hawkey dnf
$ sudo dnf install blktap-devel --debugsolver (it will create
'debugdata' directory which we want)
On Tue, Aug 4, 2015 at 11:47 PM, Michael Schwendt mschwe...@gmail.com wrote:
On Fri, 31 Jul 2015 03:51:12 -0400, Christopher Meng wrote:
Broken deps for x86_64
Surprisingly, the report is incomplete and doesn't find some unresolvable
dependencies. DNF doesn't either.
An undefined %{epoch} in a dependency is not found. This has been reported
to blktap: https://bugzilla.redhat.com/1248912
Note how DNF tells Dependencies resolved, but later fails during the
transaction check. How could it resolve the unexpanded %{epoch} earlier?
I'm confused as well, I never saw any problem in this package before.
Obviously. ;) If the Rawhide broken deps report had found it, breakage could
have been avoided.
A different try:
https://fedorahosted.org/rel-eng/ticket/6225
Or file it in the infrastructure tracker instead? I don't know. There are lots
of active tickets in both.
And what about DNF? Are the DNF developers interesting in looking into
it, too? Or is by design that the Dependencies resolved step doesn't
discover the unresolvable dependency?
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
--
-Igor Gnatenko
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Wed, 5 Aug 2015 13:47:42 +0300, Igor Gnatenko wrote:
I can't reproduce this issue.
Believe me, you can. You only created a completely different test-case,
which may not suffer from the same problem.
How to reproduce:
1. Use Fedora 22.
2. dnf install blktap-devel --disablerepo=updates-testing
It is important to not pull in the fixed test update for blktap.
What happens?
Dependency problem is only found during transaction check, i.e. after
the Dependencies resolved step and after downloading packages.
# dnf install blktap-devel --disablerepo=updates-testing
Last metadata expiration check performed 0:42:49 ago on Wed Aug 5 12:11:33
2015.
Dependencies resolved.
PackageArch Version RepositorySize
Installing:
blktap x86_64 3.0.0-2.fc22.git0.9.2 fedora 243 k
blktap-devel x86_64 3.0.0-2.fc22.git0.9.2 fedora21 k
Transaction Summary
Install 2 Packages
Total download size: 263 k
Installed size: 773 k
Is this ok [y/N]: y
Downloading Packages:
(1/2): blktap-devel-3.0.0-2.fc22.git0.9.2.x86_6 4.1 kB/s | 21 kB 00:05
(2/2): blktap-3.0.0-2.fc22.git0.9.2.x86_64.rpm 46 kB/s | 243 kB 00:05
Total36 kB/s | 263 kB 00:07
Running transaction check
Error: transaction check vs depsolve:
blktap(x86-64) = %{epoch}:3.0.0-2.fc22.git0.9.2 is needed by
blktap-devel-3.0.0-2.fc22.git0.9.2.x86_64
To diagnose the problem, try running: 'rpm -Va --nofiles --nodigest'.
You probably have corrupted RPMDB, running 'rpm --rebuilddb' might fix the
issue.
The downloaded packages were saved in cache till the next successful
transaction.
You can remove cached packages by executing 'dnf clean packages'
# rpm -q libsolv hawkey dnf
libsolv-0.6.11-1.fc22.x86_64
hawkey-0.5.9-3.fc22.x86_64
dnf-1.0.2-3.fc22.noarch
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Wed, Aug 5, 2015 at 1:58 PM, Michael Schwendt mschwe...@gmail.com wrote:
On Wed, 5 Aug 2015 13:47:42 +0300, Igor Gnatenko wrote:
I can't reproduce this issue.
Believe me, you can. You only created a completely different test-case,
which may not suffer from the same problem.
How to reproduce:
1. Use Fedora 22.
2. dnf install blktap-devel --disablerepo=updates-testing
Can you send debugdata from dnf?
# dnf install blktap-devel --disablerepo=updates-testing --debugsolver
and then archive 'debugdata' directory.
It is important to not pull in the fixed test update for blktap.
What happens?
Dependency problem is only found during transaction check, i.e. after
the Dependencies resolved step and after downloading packages.
# dnf install blktap-devel --disablerepo=updates-testing
Last metadata expiration check performed 0:42:49 ago on Wed Aug 5 12:11:33
2015.
Dependencies resolved.
PackageArch Version Repository
Size
Installing:
blktap x86_64 3.0.0-2.fc22.git0.9.2 fedora 243
k
blktap-devel x86_64 3.0.0-2.fc22.git0.9.2 fedora21
k
Transaction Summary
Install 2 Packages
Total download size: 263 k
Installed size: 773 k
Is this ok [y/N]: y
Downloading Packages:
(1/2): blktap-devel-3.0.0-2.fc22.git0.9.2.x86_6 4.1 kB/s | 21 kB 00:05
(2/2): blktap-3.0.0-2.fc22.git0.9.2.x86_64.rpm 46 kB/s | 243 kB 00:05
Total36 kB/s | 263 kB 00:07
Running transaction check
Error: transaction check vs depsolve:
blktap(x86-64) = %{epoch}:3.0.0-2.fc22.git0.9.2 is needed by
blktap-devel-3.0.0-2.fc22.git0.9.2.x86_64
To diagnose the problem, try running: 'rpm -Va --nofiles --nodigest'.
You probably have corrupted RPMDB, running 'rpm --rebuilddb' might fix the
issue.
The downloaded packages were saved in cache till the next successful
transaction.
You can remove cached packages by executing 'dnf clean packages'
# rpm -q libsolv hawkey dnf
libsolv-0.6.11-1.fc22.x86_64
hawkey-0.5.9-3.fc22.x86_64
dnf-1.0.2-3.fc22.noarch
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
--
-Igor Gnatenko
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Wed, 5 Aug 2015 14:00:59 +0300, Igor Gnatenko wrote: How to reproduce: 1. Use Fedora 22. 2. dnf install blktap-devel --disablerepo=updates-testing Can you send debugdata from dnf? # dnf install blktap-devel --disablerepo=updates-testing --debugsolver and then archive 'debugdata' directory. https://mschwendt.fedorapeople.org/tmp/debugdata-undef-epoch-problem.tar -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Tue, 4 Aug 2015 22:47:31 +0200 Michael Schwendt mschwe...@gmail.com wrote: Obviously. ;) If the Rawhide broken deps report had found it, breakage could have been avoided. A different try: https://fedorahosted.org/rel-eng/ticket/6225 Or file it in the infrastructure tracker instead? I don't know. There are lots of active tickets in both. Rawhide/branched broken dep reports use the 'spam-o-matic' script in the mash package: https://pagure.io/mash/blob/master/f/utils/spam-o-matic which in turn seems to import the yum 'repoclosure' util. So, I'd say try and duplicate it with repoclosure and then file against yum-utils? And what about DNF? Are the DNF developers interesting in looking into it, too? Or is by design that the Dependencies resolved step doesn't discover the unresolvable dependency? Of course the above is a short term thing, we should look at what we might replace spam-o-matic with that is dnf based. There is a dnf repoclosure plugin, but not sure how well it works off hand. kevin pgpa8LS3Tit0f.pgp Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Fri, 31 Jul 2015 03:51:12 -0400, Christopher Meng wrote:
Broken deps for x86_64
Surprisingly, the report is incomplete and doesn't find some unresolvable
dependencies. DNF doesn't either.
An undefined %{epoch} in a dependency is not found. This has been reported
to blktap: https://bugzilla.redhat.com/1248912
Note how DNF tells Dependencies resolved, but later fails during the
transaction check. How could it resolve the unexpanded %{epoch} earlier?
I'm confused as well, I never saw any problem in this package before.
Obviously. ;) If the Rawhide broken deps report had found it, breakage could
have been avoided.
A different try:
https://fedorahosted.org/rel-eng/ticket/6225
Or file it in the infrastructure tracker instead? I don't know. There are lots
of active tickets in both.
And what about DNF? Are the DNF developers interesting in looking into
it, too? Or is by design that the Dependencies resolved step doesn't
discover the unresolvable dependency?
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Fri, Jul 31, 2015 at 3:43 AM, Michael Schwendt mschwe...@gmail.com wrote:
On Thu, 30 Jul 2015 12:42:29 +, Fedora Rawhide Report wrote:
Broken deps for x86_64
Surprisingly, the report is incomplete and doesn't find some unresolvable
dependencies. DNF doesn't either.
An undefined %{epoch} in a dependency is not found. This has been reported
to blktap: https://bugzilla.redhat.com/1248912
Note how DNF tells Dependencies resolved, but later fails during the
transaction check. How could it resolve the unexpanded %{epoch} earlier?
I'm confused as well, I never saw any problem in this package before.
--
Yours sincerely,
Christopher Meng
http://cicku.me
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Undefined %epoch problem (Re: rawhide report: 20150730 changes)
On Thu, 30 Jul 2015 12:42:29 +, Fedora Rawhide Report wrote:
Broken deps for x86_64
Surprisingly, the report is incomplete and doesn't find some unresolvable
dependencies. DNF doesn't either.
An undefined %{epoch} in a dependency is not found. This has been reported
to blktap: https://bugzilla.redhat.com/1248912
Note how DNF tells Dependencies resolved, but later fails during the
transaction check. How could it resolve the unexpanded %{epoch} earlier?
$ rpm -qpR blktap-devel-3.0.0-3.fc23.git0.9.2.x86_64.rpm |grep ep
blktap(x86-64) = %{epoch}:3.0.0-3.fc23.git0.9.2
$ dnf install blktap-devel
Waiting for process with pid 2683 to finish.
Fedora - Rawhide - Developmental packages for t 1.3 MB/s | 43 MB 00:32
Last metadata expiration check performed 0:00:17 ago on Fri Jul 31 09:33:49
2015.
Dependencies resolved.
Package ArchVersion RepositorySize
Installing:
blktapx86_64 3.0.0-3.fc23.git0.9.2 rawhide 245 k
blktap-devel x86_64 3.0.0-3.fc23.git0.9.2 rawhide 21 k
Transaction Summary
Install 2 Packages
Total download size: 266 k
Installed size: 793 k
Is this ok [y/N]: y
Downloading Packages:
(1/2): blktap-devel-3.0.0-3.fc23.git0.9.2.x86_6 213 kB/s | 21 kB 00:00
(2/2): blktap-3.0.0-3.fc23.git0.9.2.x86_64.rpm 956 kB/s | 245 kB 00:00
Total 202 kB/s | 266 kB 00:01
Running transaction check
Error: transaction check vs depsolve:
blktap(x86-64) = %{epoch}:3.0.0-3.fc23.git0.9.2 is needed by
blktap-devel-3.0.0-3.fc23.git0.9.2.x86_64
To diagnose the problem, try running: 'rpm -Va --nofiles --nodigest'.
You probably have corrupted RPMDB, running 'rpm --rebuilddb' might fix the
issue.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
