Re: [Rpm-ecosystem] Locating all the RPM Ecosystem software

[Neal Gompa]

On Thu, Aug 27, 2015 at 12:24 PM, Michael Schroeder m...@suse.de wrote:

 On Thu, Aug 27, 2015 at 11:42:24AM -0400, Honza ??ilhan wrote:
  createrepo, DeltaRPM does not makes sense to me to be there. They are
 obsoleted and
  don't have a maintainers as you have said.

 Hey, I'm the author/maintainer of deltarpm. I don't mind pushing
 it to rpm-software-management, though.


​If it's still in active use, we should have it in the
rpm-software-management organization. I don't know if createrepo is still
in active use, or if createrepo_c has completely replaced it.


  libsolv has it's good place under openSUSE organization with zypper etc.
 (Although
  we can host it if Michael wants)

 But libsolv is not rpm specific...


​​That's fine. I wasn't suggesting everything move. As far as RPM-specific
stuff, hawkey serves that role fairly well. As long as there's a reference
somewhere that indicates where libsolv is, I think it's fine.


 Cheers,
   Michael.

 --
 Michael Schroeder   m...@suse.de
 SUSE LINUX GmbH,   GF Jeff Hawn, HRB 16746 AG Nuernberg
 main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Rich deps syntax finalization

[Neal Gompa]

On Mon, Aug 31, 2015 at 10:14 AM, Florian Festi  wrote:

> My thought after the discussion so far:
>
> May be no one really cares about the syntax.
> Still a lot of educating to do before rich deps go into production.
>
>
> On 08/25/2015 02:11 PM, Florian Festi wrote:
>
> > IF Operator
>
> Guess we stay with (. IF . ELSE .) - even some people are more familiar
> with (. ? . : .).
>
> > We discussed whether the operators should be upper or lower case or case
> > insensitive. So far we think *upper case* is better as is stands out
> > between the typically lower case package names. But we are interested on
> > second opinions on this, too.
>
> I am now tending to actually *use lower case*. This is what most
> programming languages do. May be familiarity beats emphasizing the
> operators with CAPS.
>
> So (. if . else .)
>
>
​This makes sense. After all, the parenthesis al​ready denotes that the
if/else clauses are meaningful.



> > AND and OR
> > ==
>
> I think we go for "and" and "or" for consistency. In the end it is
> really close with | and & to stay with dpkg. What pushed it over to and,
> or was having a different but consistent style for the rich deps - in
> opposite to macros and version comparisons.
>

​I'm okay with this.​ Will "or" have an ordering component (and thus,
short-circuit logic capability from left to right)? Or will it remain
order-less and fully evaluated each time?


>
> > NOT not?
> > 
>
> NOT is confusing people and they have problems coming up with real use
> cases. So we will go without for now and reconsider when real need arises.
>
>
​What would you consider real use cases? ​



> Speak now or forever hold your peace.
>

​I think this counts as speaking. ;)​



>
> Florian
>
> --
>
> Red Hat GmbH, http://www.de.redhat.com/ Registered seat: Grasbrunn,
> Commercial register: Amtsgericht Muenchen, HRB 153243,
> Managing Directors: Charles Cachera, Michael Cunningham, Michael
> O'Neill, Charles Peters
>
>


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] rpm-ostree's usage of libhif/hawkey merge

[Neal Gompa]

On Tue, Jan 26, 2016 at 1:12 PM, Colin Walters <walt...@verbum.org> wrote:
>
>
> On Tue, Jan 26, 2016, at 12:54 PM, Neal Gompa wrote:
>
>> Well, if you keep altering all the interfaces, especially the public
>> ones used by PackageKit and DNF, then there will be problems for
>> developers and consumers of these programs.
>
> What I'm trying to say is that given we know we need to break
> APIs for two reasons:
>
>  1) To merge libhif/hawkey
>  2) To expose more lower-level APIs for rpm-ostree
>
> (At least those two)
>
> There's going to be a lot of churn.  My offhand take here is
> that both rpm-ostree and PackageKit should embed libhif,
> and we collaborate on reviewing patches upstream, but
> accept temporary levels of forking.
>
> koschei should stick with hawkey-python for now.

If we can consider that hawkey-python is essentially frozen API wise
(or at least pretty much remaining compatible), then it may work
temporarily for Koschei and DNF, as DNF currently consumes the hawkey
Python API, rather than the C API.

>
> dnf is the hard one, particularly given the desire to better
> integrate PK and dnf (and the whole daemon vs not daemon thing).
> Maybe they could try having a branch that uses a new libhif?
> I think in practice it'd be better to have finalized some of the concepts
> of the C API before we start trying to rebase dnf on libhif.
>
> But I'm just making things up here.
>

Honestly, at this point, I'm wondering why we don't just rename libhif
to libdnf (or something else, but DNF would be the primary consumer).
Now that the hawkey/hif merger is (mostly) complete and it's one
library that was ultimately designed to be the foundations of both DNF
and PackageKit frontends, I think it would make sense to call it
libdnf from this point on. That would allow the hif backend to remain
frozen at 0.2.0 and the new integrated code could be a new library.
DNF would also eventually migrate from the hawkey python API to the
libdnf gi based API in Python.

>> That's uncalled for there.
>>
>> I have spent the last four months painstakingly working on integration
>> of DNF into Mageia[1] and I've also been working for the last three
>> months on integration of the stack for Unity Linux, and I have indeed
>> committed code to DNF and its various plugins repositories.
>
> Okay.  I had only seen you comment once or twice.  I felt your response
> was not addressing my concerns - in fact you skipped over the fact
> that I have multiple outstanding PRs to libhif that were linked in
> the original message.

Perhaps there needs to be more aggressive work on the new code to
address your concerns. Do you talk to Richard much about this?


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Sync of rpm macros between Fedora/openSUSE

[Neal Gompa]

On Mon, Apr 11, 2016 at 2:30 PM, Zbigniew Jędrzejewski-Szmek
 wrote:
> This moves those macros under the maintenance of the rpm team.
> I guess this could work for a few projects, let's say perl and python,
> but I don't see how this can scale (to perl, python, java, js, lisp, mono,
> swift, node, ocaml, octave, php, R, ruby, tcl, etc, etc, etc).
> RPM should instead provide a featureful base upon which individual
> projects can build.
>

I think the assumption here is that you could use it as a base to
start from, and that regular merging would occur.

For example, Fedora and SUSE both have GitHub organizations. The RPM
organization repository containing these macros could be forked by
both organizations. From there, each distribution would apply their
own changes to suit their needs. In addition, they can push changes
back upstream through a pull request and regularly resync with master.
This makes it much easier to keep compatibility closer together.

This mechanism would make it easy for existing distributions and new
distributions to be able to properly source what they need to support
a wide variety of software as RPM packages.

It would also keep us from sacrificing speed for compatibility, as the
distributions would be regularly forking and merging for their own
distributions.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Reproducible Builds

[Neal Gompa]

One important aspect that would make reproducible builds more
trustworthy in the RPM world would be some capability to indicate
checksums for sources and patches so that rpmbuild can verify them.
Debian already does this in the Debian Source Control (dsc) file, as
seen by this example[1]. Without some way to introduce build-time
verification of sources independent of a build system, it'd be hard to
reliably test in a simple manner that the inputs are what you expect
them to be. This is really more important during the srpm creation
stage, as that's when the archives and patches are bundled into a
unified source archive that can be used by anything (including mock)
to build packages.

[1]: http://debian.csail.mit.edu/debian/pool/main/r/rpm/rpm_4.12.0.1+dfsg1-3.dsc

On Tue, Mar 1, 2016 at 10:59 AM, Florian Festi  wrote:
> Hi!
>
> There are several RFEs and patches popping up that revolve around
> reproducible builds. Some may have noticed the recent patch adding the
> first pieces for supporting SOURCE_DATE_EPOCH[1].
>
> From the looks of it there is a quite active group within Debian working
> on the topic[2] but this topic clearly transcends single distributions.
>
> When it comes to scope it is clear that rpm cannot tackle the issue
> alone as reproducible build require changes on all kind of levels: build
> systems, build tools, implementation details but also package managers
> or to be more precise package build tools like rpmbuild. Still there are
> some things we can do to help.
>
> So I want to focus the different pieces of work on rpm(build) here. So
> far I found:
>
>  * The SOURCE_DATE_EPOCH patch mentioned above [1]
>   * Still unfinished patch for file timestamps mentioned there
>  * Setting buildhost [3]
>  * See mail above wrt deterministic archives
>
> I am pretty sure there are still pieces missing.
>
> So, my questions are: Who is actually working on reproducible builds?
> What else is missing? Are there any special needs for some build systems?
>
> Florian
>
> [1]
> https://github.com/rpm-software-management/rpm/commit/b8a54d6a1e9bb6140b6b47e23dc707e4b967537e
> https://bugzilla.redhat.com/show_bug.cgi?id=1288713
> [2] https://reproducible-builds.org/
> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1309367
>
> --
>
> Red Hat GmbH, http://www.de.redhat.com/ Registered seat: Grasbrunn,
> Commercial register: Amtsgericht Muenchen, HRB 153243,
> Managing Directors: Charles Cachera, Michael Cunningham, Michael
> O'Neill, Charles Peters
> ___
> Rpm-ecosystem mailing list
> Rpm-ecosystem@lists.rpm.org
> http://lists.rpm.org/mailman/listinfo/rpm-ecosystem



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

[Rpm-ecosystem] Eliminating %dist/%mkrel/etc and Release field hacking

[Neal Gompa]

Hello all,

Some weeks ago, I encountered some interesting situations regarding
how to deal with mass package upgrades, especially in the case of
distribution renames and ensuring that packages remain in lockstep
with the release installed. Distributions such as Fedora and Mageia
usually solved it by doing some hacking of the Release field to
include a distribution tag. Fedora uses %dist, while Mageia uses
%mkrel.

However, both do not provide a perfect method to represent the
independent package release information in a manner that would
consistently ensure upgrades work without further trickery. For
example, Fedora suggests in its guidelines to do stuff like
1%{?dist}.1 to ensure that when the release was bumped, it would be
upgraded regardless[0]. I've encountered a few cases where we still
have problems with pre-release and post-release snapshots as well,
which has led to FPC tickets and clamors for using ~ and + in the
version field for it.

After a quick search, I found out that there had been a ticket filed
about getting a consistent %dist macro or some equivalent years ago by
Rahul Sundaram[1]. A bit more searching yielded that a distribution
had solved the problem before by simply moving the dist tag
information out of the Release field.

OpenMandriva's version of RPM has a slight tweak to wire up
RPMTAG_DISTTAG to contain their tag ("omv") and added RPMTAG_DISTEPOCH
to contain the distribution release version ("2015.0" in this case).
Version comparison has been tweaked to treat RPMTAG_DISTEPOCH like a
package epoch, which prevents certain kinds of brokenness in exotic
situations, such as distribution renames.

A couple of advantages of this approach: Handling more complex
version/release schemes becomes a lot simpler, and when used properly,
it becomes a lot harder to mix packages that are of different releases
(which can break the system, especially if major ABI changes have
occurred).

The major disadvantage of this approach is that it does add to the
package header more information. However, I consider this a non-issue,
as we already are hacking the Release field to represent the
information, and it isn't represented very well. Another issue would
be that older versions of RPM wouldn't know what to do with it, and
package managers (like DNF) need to be set up to handle the extra
information.

Overall, I think it's a more solid, distribution agnostic approach to
handling distribution release information.

What does everyone else think?



[0]: 
https://fedoraproject.org/wiki/Packaging:NamingGuidelines#Minor_release_bumps_for_old_branches
[1]: http://rpm.org/ticket/859

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Fwd: [Rpm-maint] Fixing macro scoping

[Neal Gompa]

On Fri, Feb 3, 2017 at 7:14 AM, Panu Matilainen
 wrote:
>
> Stunned silence on rpm-maint, forwarding to rpm-ecosystem in hopes of a
> larger and livelier audience...
>
> - Panu -
>
>
>  Forwarded Message 
> Subject: [Rpm-maint] Fixing macro scoping
> Date: Mon, 23 Jan 2017 12:30:21 +0200
> From: Panu Matilainen 
> To: rpm-ma...@lists.rpm.org 
>
>
> Consider the following snippet, originating from
> https://bugzilla.redhat.com/show_bug.cgi?id=552944:
>
> %{!?foo: %define foo bar}
> %define dofoo() true
> echo 1: %{foo}
> %dofoo
> echo 2: %{foo}
>
> I'd assume everybody agrees both %{foo}'s should evaluate to the same value,
> but that is not the case currently. Using a cli-variant of the above:
>
> [pmatilai@sopuli rpm]$ rpm --define 'dofoo() true' --eval '%{!?foo: %define
> foo bar}' --eval '%foo' --eval '%dofoo' --eval '%foo'
> warning: Macro %foo defined but not used within scope
>
> bar
> true
> %foo
>
> The flaw here is that rpm supposedly implements block level scoping for
> macros (so in the above example, "foo" would only exist in the {!?foo:...}
> block), but doesn't actually enforce that, unless a parametric macro is
> "called". Current rpm warns about it, but warnings or not this behavior
> doesn't make the slightest sense.
>
> The question is, what do you think %{foo} should evaluate to in this case?
>
> Fixing it to honor the strict "block scoping" concept is not hard, now that
> the scoping level is honored from Lua too (see
> https://github.com/rpm-software-management/rpm/commit/1767bc4fd82bfacee622e698f9f0ae42c02126fa).
> In this case the above reproducer would emit
>
> %foo
> true
> %foo
>
> Another option is slightly changing the whole scoping notion: parametric
> macros require locally scoped macros for the automatic argument macros like
> %#, %* and %1 anyway (it's flawed too currently, see below). So perhaps the
> macro scoping should follow the current "call level", ie a macro defined
> inside a parametric macro body is local to that macro, and everything else
> is global. In this case the reproducer would emit
>
> bar
> true
> bar
>
> I have implementations for both and also a personal opinion, but I'd like to
> hear what others think.
>
> The related flaw is whether locally scoped macros should be visible to
> deeper nesting levels. Currently everything is, including those automatic
> macros:
>
> $ rpm --define '%bar() Bar %#: %{?1} %{?2}' --define '%foo() Foo %#: %{?1}
> %{?2} %bar a' --eval '%foo 1 2'
> Foo 2: 1 2 Bar 1: a 2
>
> I'd consider this a bug, %2 should not be defined within %bar() since it did
> not receive two arguments. So IMO the correct output in the above should be:
> Foo 2: 1 2 Bar 1: a
>
> But what about a macro manually %define'd within %foo() - should that be
> visible in %bar()?
>

Sorry if I sound stupid about this, but I've read this email at least
four times now, and I'm still not quite sure what you're trying to
describe...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Fwd: [Rpm-maint] Fixing macro scoping

[Neal Gompa]

On Fri, Feb 3, 2017 at 7:35 AM, Panu Matilainen
<pmati...@laiskiainen.org> wrote:
> On 02/03/2017 02:22 PM, Neal Gompa wrote:
> [...snip...]
>>
>>
>> Sorry if I sound stupid about this, but I've read this email at least
>> four times now, and I'm still not quite sure what you're trying to
>> describe...
>
>
> Right, quite possibly the question got lost in the rpm internals
> mumble-jungle.
>
> So forget the all the technical mumbling about scoping levels etc for now.
> What would you expect the following snippet to output (full reproducer spec
> is at https://bugzilla.redhat.com/show_bug.cgi?id=552944)?
>
> %{!?foo: %define foo bar}
> %define dofoo() true
> echo 1: %{foo}
> %dofoo
> echo 2: %{foo}
>

I would expect that would expand each time. My expectation is that it
should evaluate and expand correctly on each invocation.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] RPM in ALT Linux (4.0.4 vs 4.13)

[Neal Gompa]

On Sun, Sep 4, 2016 at 7:28 AM, Ivan Zakharyaschev <i...@altlinux.org> wrote:
> Hello!
>
> On Sat, 3 Sep 2016, Neal Gompa wrote:
>
>> I saw you guys listed as the most recent ones to change the rpm
>> package in ALT Linux, and I was wondering if you guys had contemplated
>> upgrading from rpm 4.0.4 to rpm 4.13?
>
>
> glebfm@ and legion@ are busy now with this.
> https://lists.altlinux.org/pipermail/devel/2016-July/201603.html
>
> They could give most details about this process.
>
> The first thing to do on this way was to rebase many ALT's features[1] onto
> rpm(-install)-4.13. (Not yet features relevant for rpm-build.)
>

I'm sorry, I'm not too familiar with ALT's rpm setup, what is rpm-install?

> [1] https://www.altlinux.org/Rpm-4.13
>
>> If so, why haven't you? What's holding you back from upgrading? I'd
>
>
> Apart from the first important step (rebasing ALT's rpm-install) which has
> been done and is ready for testing, there are things would hold us back from
> putting the new version into ALT Sisyphus:
>
> those packages which use librpm and/or access RPM's db will have to be
> adapted for the new version. (The first one, of course, is APT; then, there
> are some Perl bindings actively used by the tools for automatic package
> analysis, modification, submission; perhaps, some more, I don't know the
> list of things that hold this back well, but other involved people could
> tell you more.)
>

As for APT, the Debian guys are actually interested in working with
RPM guys to integrate the backend into mainline APT. This would be a
good time to kill two birds with one stone. Julian Klode (juliank) on
#debian-apt on OFTC has already pulled in apt-rpm git master[1] and
I've sent him a PR including patches from Fedora that make it work
with RPM 4.13[2]. If you guys still want to pursue using APT for RPM
instead of using something like DNF, then it'd be a great idea if you
guys got involved in the effort to bring apt-rpm into the current apt
master. The merge of the rpm backend feature will lead to the release
of apt 2.0. I've been talking to Michael Vogt (mvo) and Julian Klode
about it for a while now, and it'd be great to get some people who
could help with making that merge happen.

I'm somewhat surprised that there are still distros using apt-rpm by
default, but if you guys like it and want to continue using it, I
think it'd be great if you could help with getting it merged into the
apt mainline codebase.

>> like to see the ALT Linux rpm maintainer team be involved in upstream
>> rpm.org development, as I'm sure your perspective would be valuable to
>> ensure a vibrant ecosystem around rpm.
>
>
> As said, there are a few ALT-specific nice, important and non-trivial
> features in RPM, which would always require maintaining a separate fork
> unless they are taken up by another RPM project, say, the rpm-4.13 project.
> Then the forces could be joined.
>
> One of these features is the support for set-versions (the <= relation,
> which is used to constrain Requires/Provides, which would behave not like a
> linear order, but like inclusion of sets), developed by at@ in the past.
> Now, he has announced that he is developing an enhanced varaint of this
> feature and could tell the details about the current developments to those
> who are interested.
> (https://lists.altlinux.org/pipermail/devel/2016-July/201614.html : support
> for prototypes/signatures similar to C++ mangling, but for C).
>

If any of you guys who know about the extra features of your variant
of rpm can talk about them, it'd be great if they could bring them up
in the rpm-ecosystem mailing list[3] to propose for them to be merged
upstream into the rpm.org codebase. Florian Festi (the manager of the
rpm project) is always interested in receiving patches for new
features and such. Patches are accepted via rpm-maint mailing list[4]
or GitHub[5]. Discussions can also occur on IRC on Freenode in
#rpm-ecosystem and #rpm.org.

> at@ has pointed to his new work at https://github.com/svpv/rpmss --
> https://lists.altlinux.org/pipermail/devel/2016-August/201701.html . At the
> same time, at@ shared his belief that if there is some code in ALT's RPM
> which was once written and works correctly since then, there will be no need
> to put efforts into maintaining it; and so, he sees no justification in the
> complex work of rebasing onto rpm-4.13 because this would not save us any
> future efforts in maintaining ALT's RPM compared to the current situation.
> (Zero efforts if the current code of RPM works correctly.)
>

I'm somewhat biased, I guess, but as one of the folks who works on rpm
in Mageia, I personally prefer minimizing our delta with upstream so
that there's a strong point of collaboration and when new versions
arrive, we can t

Re: [Rpm-ecosystem] RPM in ALT Linux (4.0.4 vs 4.13)

[Neal Gompa]

On Sun, Sep 4, 2016 at 12:23 PM, Ivan Zakharyaschev <i...@altlinux.org> wrote:
>
> On Sun, 4 Sep 2016, Neal Gompa wrote:
>
>> On Sun, Sep 4, 2016 at 7:28 AM, Ivan Zakharyaschev <i...@altlinux.org>
>> wrote:
>
>
>>>> package in ALT Linux, and I was wondering if you guys had contemplated
>>>> upgrading from rpm 4.0.4 to rpm 4.13?
>>>
>>>
>>>
>>> glebfm@ and legion@ are busy now with this.
>>> https://lists.altlinux.org/pipermail/devel/2016-July/201603.html
>>>
>>> They could give most details about this process.
>>>
>>> The first thing to do on this way was to rebase many ALT's features[1]
>>> onto
>>> rpm(-install)-4.13. (Not yet features relevant for rpm-build.)
>>> [1] https://www.altlinux.org/Rpm-4.13
>
>
>>> As said, there are a few ALT-specific nice, important and non-trivial
>>> features in RPM, which would always require maintaining a separate fork
>
>
> The list (published under the given link on the wiki) is quite long, and
> Gleb has recently done a big job bringing them up-to-date with 4.13 (only
> the rpm-install part, not the rpm-build one):
>

I looked through the wiki page and also the git repository on ALT
Linux's gear for rpm-4.13[1], and there were some commits of concern:

* git hash: 08677107ea8efb30099b05c0c9876e0cfdbd9799: Add support of
ALT Linux traditional posttrans filetriggers
  - This commit is interesting because it's completely redundant. My
understanding is that the legacy file triggers implementation in ALT
Linux is derived from the original one developed by Mandriva. In
Mageia, we've ported all of the Mandriva style file triggers over to
the system that's part of rpm 4.13, and we'd be happy to share with
you guys the newer versions of these and help you guys get up to speed
there.

* git hash: 61a1aaa2c0795f8bd564bdbef2b45cc4f44902ac: Write to syslog
about installed/removed packages
 - Why exactly is this necessary? This isn't a huge deal, but wouldn't
apt handle this already? This also feels like it should be written as
an rpm plugin rather than be a mandatory part of rpm. Other resolvers
(like Yum/DNF and Zypper) do record this information already to some
form of database/log which is accessible at any time, so it would be
redundant in that case.

* git hash: 4487281b1ed93cc8a997adb56be51c33c52ebce9: Add
/usr/lib/rpm/macros.d/*, /etc/rpm/macros.d/* to macrofiles
 - This one seems weird, as it disables the *.attr files and
eliminates the macros file name convention we've used in Fedora,
CentOS, Mageia, and openSUSE. I suspect this patch will remain
specific to ALT Linux.

* git hash: 60ea90eeb0341c2f8c761133033cd3e015c082c7: Add scripts/find-package
 - I understand why this exists (apt metadata doesn't include file
lists, so you can't resolve them). We have the same problem in Mageia
with urpmi's hdlist2 metadata. We have patches that alter rpm's own
find-requires script to work as you need it to so that you don't have
to disable the internal generators. I've attached them to this email,
with a "series" file to indicate the order in which the patches can be
applied. These patches were written by Thierry Vignaud of Mageia for
our rpm package to solve the same issue while leveraging the new
dependency generator framework.

I'll admit, I'm not really certain about many of the other ones...
Perhaps someone else here can take a deeper review of some of the
other patches and provide some feedback?

Do you guys have an equivalent package to redhat-rpm-config[2] or
rpm-mageia-setup[3] which contains your vendor configuration for RPM?
It seems like some of this stuff (like the GROUPS file, macros, etc.)
belong there...



[1]: http://git.altlinux.org/tasks/166699/gears/560/git?p=git;a=log
[2]: http://pkgs.fedoraproject.org/cgit/rpms/redhat-rpm-config.git/
[3]: http://gitweb.mageia.org/software/rpm/rpm-setup/about/



-- 
真実はいつも一つ！/ Always, there's only one truth!


rpm-4.13-mga-nofiledeps-depgen-patches.tar.gz
Description: GNU Zip compressed data
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] RPM in ALT Linux (4.0.4 vs 4.13)

[Neal Gompa]

On Sun, Sep 4, 2016 at 6:16 PM, Ivan Zakharyaschev <i...@altlinux.org> wrote:
>
> On Sun, 4 Sep 2016, Neal Gompa wrote:
>
>>>>> The first thing to do on this way was to rebase many ALT's features[1]
>>>>> onto
>>>>> rpm(-install)-4.13. (Not yet features relevant for rpm-build.)
>>>>> [1] https://www.altlinux.org/Rpm-4.13
>
>
>> I looked through the wiki page and also the git repository on ALT
>> Linux's gear for rpm-4.13[1], and there were some commits of concern:
>>
>> * git hash: 08677107ea8efb30099b05c0c9876e0cfdbd9799: Add support of
>> ALT Linux traditional posttrans filetriggers
>>  - This commit is interesting because it's completely redundant. My
>> understanding is that the legacy file triggers implementation in ALT
>> Linux is derived from the original one developed by Mandriva. In
>
>
> I don't know enough about the history of it to tell this for sure.
>
>> Mageia, we've ported all of the Mandriva style file triggers over to
>> the system that's part of rpm 4.13, and we'd be happy to share with
>> you guys the newer versions of these and help you guys get up to speed
>> there.
>
>
> One issue is that the new rpm should be able to work with the old packages,
> published in the old distro branches, at least in order to enable an
> upgrade.
>
> Another one is that the rpm maintainer (like Gleb) can't rewrite the
> filetriggers in all the packages at once; rather he can provide the tool
> which implements the old and new features and wait for the maintainers of
> the packages who own the triggers to catch up; and ultimately, the old
> feature can be removed.
>
> I might be misunderstanding something about the mechanism.

That does make sense. Hopefully that will be properly communicated as
the migration begins. Feel free to reach out to Mageia on the dev
mailing list[1] if you'd like assistance on this.

>
>> * git hash: 61a1aaa2c0795f8bd564bdbef2b45cc4f44902ac: Write to syslog
>> about installed/removed packages
>> - Why exactly is this necessary? This isn't a huge deal, but wouldn't
>> apt handle this already? This also feels like it should be written as
>
>
> Some users invoke the bare rpm command.
>

That's fair.

>> an rpm plugin rather than be a mandatory part of rpm. Other resolvers
>> (like Yum/DNF and Zypper) do record this information already to some
>> form of database/log which is accessible at any time, so it would be
>> redundant in that case.
>
>
> What kind of plugin do you mean?
>

RPM has a plugin architecture[2] which I think could implement what
you're looking for. In fact, it looks like there already is a syslog
plugin[3]. If it doesn't do all what you need, it might be worth
extending it to emit the extra logging you want.


>> * git hash: 4487281b1ed93cc8a997adb56be51c33c52ebce9: Add
>> /usr/lib/rpm/macros.d/*, /etc/rpm/macros.d/* to macrofiles
>> - This one seems weird, as it disables the *.attr files and
>> eliminates the macros file name convention we've used in Fedora,
>> CentOS, Mageia, and openSUSE. I suspect this patch will remain
>> specific to ALT Linux.
>>
>> * git hash: 60ea90eeb0341c2f8c761133033cd3e015c082c7: Add
>> scripts/find-package
>> - I understand why this exists (apt metadata doesn't include file
>> lists, so you can't resolve them). We have the same problem in Mageia
>> with urpmi's hdlist2 metadata. We have patches that alter rpm's own
>> find-requires script to work as you need it to so that you don't have
>> to disable the internal generators. I've attached them to this email,
>> with a "series" file to indicate the order in which the patches can be
>> applied. These patches were written by Thierry Vignaud of Mageia for
>> our rpm package to solve the same issue while leveraging the new
>> dependency generator framework.
>
>
> Mageia's implementation is not in the upstream, too, is it? Which
> implementation is better: the Mageia's one or the ALT's one?
>

Our patches are not upstream because both Fedora and openSUSE use
dependency resolvers that use rpm-md metadata, which supports file
dependencies (so they don't need it). We're also going to support it
in Mageia 6 with our implementation of DNF[4], but we still keep the
patches up in order to support urpmi. In my opinion, Mageia's
implementation is better because it leverages the modern dependency
generators included in rpm 4.13. I'm not sure if file dependencies
would work if you used rpm-md repository metadata with apt-rpm, since
it does support the rpm-md repository metadata format.

>> I'll admit, I'm not really certain about many of the other ones...
>> Perhaps someone else her

Re: [Rpm-ecosystem] Special meaning of "+" (?) separator

[Neal Gompa]

On Mon, Sep 12, 2016 at 8:07 AM, Florian Festi  wrote:
> Changing the way + is treated in version compare is really a bad idea.
> So this feature would need a new char that is currently not permitted in
> versions. Candidates include: #, ^, @, §, $, ?

Why is changing '+' in version comparison a bad idea? It currently
exists as a separator that does nothing, and every usage of it I've
seen even in RPM-based distros makes the assumption that it works the
way the proposed changes will actually make it function.

If anything, this would align how the operator is used in practice,
which is always a good thing, in my book.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Special meaning of "+" (?) separator

[Neal Gompa]

On Fri, Sep 9, 2016 at 4:24 PM, Igor Gnatenko  wrote:
> Hi,
>
> during process of getting tilde approved in Fedora Packaging
> Guidelines we realized that we need some special handling for
> separator (most probably) "+".
>
> Some examples (left is what expected, right is what current situation):
> 1.0+ > 1.0  | 1.0+ == 1.0
> 1.0+20160101git < 1.0.1 | 1.0+20160101git > 1.0.1
>
> During long discussion at #rpm.org on freenode with Florian and Panu:
>
> 1. Florian didn't like to change behavior of "+" as it's already
> allowed character and people might expect it to do something
> different.
> 2. From alternative symbols we needed to choose from: "@", "#", "^"
>
> After thinking more about problem I realized that we probably just
> change behavior of "+".

Yes, this is the right way to go, as we don't need more "specialness"
and it's relatively intuitive to indicate + as the
checkout above it.

> Some questions are still in my mind:
> * vercmp: 1~ ? 1+

I'm not sure here. Normally, you use "1~" to indicate an all-inclusive
set (pre-release, release, and post-release), as opposed to "1" (which
would include only release and post-release).

Strictly speaking 1~ < 1+ in comparison, as the tilde operator pushes
it down and the plus operator would push it up.

> * How "+" should be handled in (Build)Requires?
> * BuildRequires: foo == 1+ should match 1+, 1+git, 1+whatever ?

I'm not sure exactly how this should be handled... Probably 1+?

> * BuildRequires: foo >= 1 should match 1, 1.1, 1+git, 1.1+whatever ?

Yes.

> * BuildRequires: foo < 1+ should match 1, 1~git, 0.whatever ?

Yes.

>
> Thoughts? Suggestions?
> --
> -Igor Gnatenko
> ___
> Rpm-ecosystem mailing list
> Rpm-ecosystem@lists.rpm.org
> http://lists.rpm.org/mailman/listinfo/rpm-ecosystem



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Special meaning of "+" (?) separator

[Neal Gompa]

On Sat, Sep 10, 2016 at 2:02 AM, Igor Gnatenko  wrote:
> Attaching patch to what I came up.
>
> If everything looks good, I will write same for libsolv.
>

Looks good to me, though where's the logic for the rpmlib() tag for
the behavior change?


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] LIBDNF - code reduction + important further changes

[Neal Gompa]

On Fri, Dec 1, 2017 at 8:44 AM, Richard Hughes  wrote:
> On 1 December 2017 at 08:52, Jaroslav Mracek  wrote:
>> I would prefer to apply patches  to PackageKit and so on, if you will agree?
>
> Sure, as long as there are tarball releases we can depend on in Fedora
> this is fine.
>

Likewise from the Mageia point of view.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] DNF team would like to take over libdnf ownership

[Neal Gompa]

On Mon, Oct 30, 2017 at 6:48 AM, Daniel Mach  wrote:
> Hi everyone,
> My name is Daniel Mach and I'm leading the DNF team.
>
>
> TL;DR:
>
> The DNF team wants to take libdnf over and re-start development, which would
> possibly include using C++ (only to limited and reasonable extent). The goal
> is to get existing DNF code into libdnf. The current APIs would remain
> intact.
> We're looking for your use cases.
>

Fantastic!

>
> More details follow:
>
> We are getting requests to implement more features in microdnf,
> such as modularity[1][2], unify repo cache and overall business logic.
> Porting big portions of existing DNF code (Python) into a C library
> seems to be the only feasible option to avoid code duplication.
>
> In the first place, we'd like to clarify libdnf ownership and
> responsibilities.
> Libdnf is currently co-maintained by several teams (DNF, PackageKit,
> rpm-ostree), which makes it an unmaintained orphan.
> There are gaps in roadmap, development and response time to the customer
> issues.
> The code suffers with a technical debt, especially the libhif/hawkey merge
> hasn't been finished.
>
> From organizational perspective, we are proposing following:
> * DNF team to own libdnf and be fully responsible for it.
> * DNF team to guarantee existing C and Python APIs.

This would probably be a mistake, given how the existing APIs are. I
would suggest picking which base of APIs you want to guarantee and
work from there.

> * DNF team to discuss any API changes with the stakeholders in timely
> manner.

This means you should include soname bumps every time API/ABI changes occur.

> * PackageKit to become a libdnf stakeholder.
> * rpm-ostree to become a libdnf stakeholder.
>
> We are looking for a way how to boost the development speed by using a more
> high-level programming language than C. We are considering C++ at the
> moment,
> because it offers some features we (mainly Python programmers) would
> appreciate,
> including strongly typed lists and dictionaries, less work with pointers and
> last, but not least - native objects. It also allows us to wrap and re-use
> existing C code without major changes.
>

I don't have a problem with C++. In fact, I'm more versed in C++ than
I am in C, so this is pretty nice for me. That said, I do have a
couple of concerns about it:

* Binding from C++ to languages is trickier than C, so do you plan to
provide a C API for this? Or maybe do you plan to wire in SWIG[1] to
dynamically generate them as libsolv does?
* Using C++ makes glib2 usage mostly pointless, as glib2 is by and
large replaced by the C++ STL. The only advantage of GLib2 is GObject
Introspection, but if you're using C++, you might as well do something
else for bindings generation (like SWIG).
* Please avoid Boost. The Zypper stack uses it and it's a major pain
(it's why Yocto dropped Zypper years ago).

[1]: http://www.swig.org/

> Short-term plan: Fix libdnf technical debt
> * Finish libhif and hawkey merge.
> * Remove unused code.
> * Fix code duplicates.
> * Hide private functions which were published by mistake.
>

This conflicts with guaranteeing all C/Python APIs. Again, I suggest
picking what you want to stabilize and go from there.

> Mid-term plan:
> * Merge DNF Base class with libdnf's context.
> * Re-shuffle existing C code into new structures, but still keep existing
> APIs.
> * Implement modularity[1]
>
> Long-term plan:
> * Full rewrite/cleanup
> * Move major part of DNF's code into libdnf
> * New, well-defined, fully supported API
>
> The Holy Grail:
> * libdnf becomes "The Software Management Library"
> * libdnf has fully documented and tested API
> * libdnf has bindings to infinite amount of languages :)

Though if we move from C to C++ (and consequently drop glib2
dependency), then GObject introspection goes away, which is what could
give us this (sorta). SWIG (as mentioned above) can offer this for us,
but you need to build this in from the beginning...

>
> I'd like to ask you for your feedback on the plan.
> If you have any use cases you'd like implemented, please open an issue[3] or
> a bug[4].
> I understand that the C++ part sounds controversial, but as long as we keep
> the
> C API available and intact, nothing should change for the existing users.
> Please keep the discussion technical.
>

The big question for me is if you decide to use C++, do you plan to
keep the glib2 dependency? From my point of view, glib2 is a wasteful
and unnecessary extra dependency for C++ libraries and applications.

Also, where do librepo and libcomps fit into this plan? Both libraries
are also maintained similarly to libdnf, and they've caused their own
share of issues.

In addition, it would be good if libdnf is capable of being used as a
static link library, mainly for building a statically linked version
of microdnf that would work in the initramfs for system recovery and
such.

-- 
真実はいつも一つ！/ Always, there's only one truth!

Re: [Rpm-ecosystem] rpm-4.14.0

[Neal Gompa]

On Wed, Oct 25, 2017 at 6:02 PM, Baho Utot  wrote:
> When building on a raspberry-pi 2:
>
> tar xvf rpm-4.14.0.tar.bz2
> cd rpm-4.14.0
> tar ../db-4.5.20.tar.gz
> ln -s db-4.5.10 db
> ./autogen.sh --noconfigure
> ./configure --with-crypto=openssl --without-lua
> make
>
> Fails with file not found db.h
>
> When I do the same with rpm-4.13.0.1 it works as then configure stage
> creates db.h

If those are actually your steps, I see at least two things wrong:

1. I think the minimum is BDB 4.8 (though BDB 5 is recommended)
2. The symlink you're creating isn't valid, as the path is wrong.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] DNF team would like to take over libdnf ownership

[Neal Gompa]

On Mon, Oct 30, 2017 at 3:15 PM, Daniel Mach <dm...@redhat.com> wrote:
>
>
> On Mon, Oct 30, 2017 at 12:43 PM, Neal Gompa <ngomp...@gmail.com> wrote:
>>
>> On Mon, Oct 30, 2017 at 6:48 AM, Daniel Mach <dm...@redhat.com> wrote:
>> > Hi everyone,
>> > My name is Daniel Mach and I'm leading the DNF team.
>> >
>> >
>> > TL;DR:
>> >
>> > The DNF team wants to take libdnf over and re-start development, which
>> > would
>> > possibly include using C++ (only to limited and reasonable extent). The
>> > goal
>> > is to get existing DNF code into libdnf. The current APIs would remain
>> > intact.
>> > We're looking for your use cases.
>> >
>>
>> Fantastic!
>>
>> >
>> > More details follow:
>> >
>> > We are getting requests to implement more features in microdnf,
>> > such as modularity[1][2], unify repo cache and overall business logic.
>> > Porting big portions of existing DNF code (Python) into a C library
>> > seems to be the only feasible option to avoid code duplication.
>> >
>> > In the first place, we'd like to clarify libdnf ownership and
>> > responsibilities.
>> > Libdnf is currently co-maintained by several teams (DNF, PackageKit,
>> > rpm-ostree), which makes it an unmaintained orphan.
>> > There are gaps in roadmap, development and response time to the customer
>> > issues.
>> > The code suffers with a technical debt, especially the libhif/hawkey
>> > merge
>> > hasn't been finished.
>> >
>> > From organizational perspective, we are proposing following:
>> > * DNF team to own libdnf and be fully responsible for it.
>> > * DNF team to guarantee existing C and Python APIs.
>>
>> This would probably be a mistake, given how the existing APIs are. I
>> would suggest picking which base of APIs you want to guarantee and
>> work from there.
>
> That's right, it's something close to what we're planning.
> Carefully choose what to support and what to discard,
> communicate with stakeholders, avoid problems.
>
> I just wanted to make a clear statement that users can expect
> an evolution rather than a big bang throwing away existing APIs
> and breaking everything.
>
>

That's good. Especially as we're looking at potentially using libdnf
for DrakX in Mageia (unless we decide to glue Python to Perl through
Inline::Python).

>>
>>
>> > * DNF team to discuss any API changes with the stakeholders in timely
>> > manner.
>>
>> This means you should include soname bumps every time API/ABI changes
>> occur.
>
> Either that or supporting multiple soname versions.
> I don't want to make any promises, but we're looking even into that.
>

How would that even work? Unless you're talking about symbol
versioning? I'll note that glibc is the only libc that properly
supports symbol versioning, and other libc implementations for Linux
(and other OSes) either always export only the latest or ignore symbol
versioning entirely.

>>
>>
>> > * PackageKit to become a libdnf stakeholder.
>> > * rpm-ostree to become a libdnf stakeholder.
>> >
>> > We are looking for a way how to boost the development speed by using a
>> > more
>> > high-level programming language than C. We are considering C++ at the
>> > moment,
>> > because it offers some features we (mainly Python programmers) would
>> > appreciate,
>> > including strongly typed lists and dictionaries, less work with pointers
>> > and
>> > last, but not least - native objects. It also allows us to wrap and
>> > re-use
>> > existing C code without major changes.
>> >
>>
>> I don't have a problem with C++. In fact, I'm more versed in C++ than
>> I am in C, so this is pretty nice for me. That said, I do have a
>> couple of concerns about it:
>>
>> * Binding from C++ to languages is trickier than C, so do you plan to
>> provide a C API for this? Or maybe do you plan to wire in SWIG[1] to
>> dynamically generate them as libsolv does?
>
> SWIG it is for Python (and possibly other languages in the future).
> C will probably need a hand-written wrapper.
>
>>
>> * Using C++ makes glib2 usage mostly pointless, as glib2 is by and
>> large replaced by the C++ STL. The only advantage of GLib2 is GObject
>> Introspection, but if you're using C++, you might as well do something
>> else for bindings generation (like SWIG).
>
> Replacing glib2 with C++ STL would be nice,
> but I'm su

Re: [Rpm-ecosystem] DNF team would like to take over libdnf ownership

[Neal Gompa]

On Mon, Oct 30, 2017 at 3:32 PM, Daniel Mach <dm...@redhat.com> wrote:
>
>
> On Mon, Oct 30, 2017 at 2:02 PM, Colin Walters <walt...@verbum.org> wrote:
>>
>> On Mon, Oct 30, 2017, at 07:43 AM, Neal Gompa wrote:
>> > * Using C++ makes glib2 usage mostly pointless, as glib2 is by and
>> > large replaced by the C++ STL. The only advantage of GLib2 is GObject
>> > Introspection
>>
>> Not quite; there's also the main event loop, which is also tied into
>> infrastructure like file monitoring (used today in libdnf to monitor
>> the rpmdb), and what are effectively "sub-libraries" in libglib
>> like GDBus.   For example today dnf talks to NM via pydbus:
>>
>> https://github.com/rpm-software-management/dnf/blob/master/dnf/util.py#L261
>> which is currently synchronous, but any nontrivial usage
>> requires a main loop (which pydbus uses glib's most commonly).
>>
>> And the whole "async" infrastructure in GTask is based on this.
>> Now, libdnf today is mostly synchronous, which is OK.  We could
>> keep it that way.
>
> Libdnf will most likely remain synchronous,
> but thanks for bringing this topic up.
> Making another note...
>

What prevents us from handling async things with libdnf?

>>
>> In fact for various reasons we're highly likely at some point
>> in the future to change rpm-ostree to mostly use librpm/libdnf
>> from a subprocess.
>>
>> One of those reasons is that librepo doesn't currently use any
>> of the GIO bits, notably GCancellable:
>>
>> https://github.com/projectatomic/rpm-ostree/issues/897#issuecomment-318427563
>>
>> This doesn't matter for a command line tool, but it matters a lot
>> for a daemon.  So by changing our libdnf to be in a subprocess we're
>> basically reworking the architecture to be daemon → internal command line.
>>
>> But, even if we do this, we'll need to do things like have progress
>> on downloads be emitted via some internal messaging framework
>> (probably private DBus socket).  I think that'll be fine if things are
>> generally
>> synchronous in libdnf, but if we're talking about redoing libdnf
>> I think we need to keep these high level architectural bits in
>> mind and not just focus on the programming language.
>>

If we're considering rearchitecting libdnf (albeit slowly), we should
consider making it so that things like rpm-ostree don't need to force
a subprocess model for asynchronous operations.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Trying to understand %buildsubdir and debuginfo generation

[Neal Gompa]

On Fri, Apr 27, 2018 at 5:57 AM Jeff Johnson  wrote:



> > On Apr 27, 2018, at 4:34 AM, Panu Matilainen 
wrote:
> >
> > On 04/27/2018 06:47 AM, Jason L Tibbitts III wrote
> >>
> >> Seems... similar.
> >> JJ> Talk to whomever decided that there was a need to overload %install
> >> JJ> with a test on %buildsubdir.
> >> OK, so, again, I'm trying to talk to this mailing list, which I thought
> >> provided the best chance to reach the people who know.
> >
> > Well, git blame brings this up (redhat-rpm-config):
> >
> > commit 1640cd5cac6e1308de8f303e95089053e7f5c9b5 (tag:
REDHAT_RPM_CONFIG_8_0_14)
> > Author: Tim Powers 
> > Date:   Sat Dec 14 20:37:32 2002 +
> >
> >more debug macro tweaks
> >
> > ...and that's all he wrote. Gotta love the descriptive commit messages
of that era.
> >

> Well, Tim Powers was the QA guy assigned the daunting task of ensuring
that all packages in RHEL3 rebuilt and created -debuginfo packages without
any change to spec files whatsoever on 3 architectures based on RawHide.

> That was a huge amount of effort, scripting around rpmbuild, no chained
builds or Koji with a new set of packages in RawHide every day, no mock, no
depsolver, no git, no code reviews, and on a delivery deadline.

> And you should certainly know how twisted the rpmbuild parser is/was, and
how very hacky -debuginfo production is.

> If the terseness of the checkin bothers you, well, consider how many
years it's taken to even detect the bogusness. And it's taken nearly that
many years to get rid of the insanely mysterious hack involved with
"Canonicalization shrank by 1 byte".


Well, now I feel really bad for the poor guy.

That said, it's required to make debuginfo packages get generated, as I
found out when working on OpenMandriva's rpm-openmandriva-setup:
https://github.com/OpenMandrivaSoftware/rpm-openmandriva-setup/commit/df94fda422f9bf3d9e32b6f8f59493d2d478daef



--
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Zchunk update

[Neal Gompa]

On Wed, Jun 13, 2018 at 9:16 AM Pat Riehecky  wrote:
>
> On 06/12/2018 05:52 AM, Neal Gompa wrote:
> > I wish we had a canonical location for these things that everyone
> > could reference...:(
>
> Agreed!
>
> Would it make sense to put them under doc in
> https://github.com/rpm-software-management/librepo  ?
>

Actually, I think we should have an rpm-md repository in
rpm-software-management that hosts those. That would give us also the
ability to have dedicated publishing space for documentation,
specification, etc.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Zchunk update

[Neal Gompa]

On Wed, Jun 13, 2018 at 9:24 AM Neal Gompa  wrote:
>
> On Wed, Jun 13, 2018 at 9:16 AM Pat Riehecky  wrote:
> >
> > On 06/12/2018 05:52 AM, Neal Gompa wrote:
> > > I wish we had a canonical location for these things that everyone
> > > could reference...:(
> >
> > Agreed!
> >
> > Would it make sense to put them under doc in
> > https://github.com/rpm-software-management/librepo  ?
> >
>
> Actually, I think we should have an rpm-md repository in
> rpm-software-management that hosts those. That would give us also the
> ability to have dedicated publishing space for documentation,
> specification, etc.
>

After a moment of thought, I think this is something I'll take a look
into doing. I have a decent idea of where all the various documents on
rpm-md are, and maybe I can pull them together into a unified
repository.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Required version of rpm?

[Neal Gompa]

On Tue, Jun 26, 2018 at 4:57 AM Miroslav Suchý  wrote:
>
> Dne 25.6.2018 v 13:04 Igor Gnatenko napsal(a):
> > rpmlib(RichDependencies) <= 4.12.0-1
>
> But on my F28:
>
> $ LC_ALL=C rpm -q --whatprovides 'rpmlib(RichDependencies)'
> no package provides rpmlib(RichDependencies)
>

rpmlib() dependencies are virtual, they aren't provided by anything,
but are processed during the transaction and verified.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] varlink for libdnf discussion

[Neal Gompa]

On Wed, May 2, 2018 at 1:28 PM Colin Walters  wrote:

> Let's talk about
https://github.com/varlink/documentation/wiki/Adding-varlink-to-DNF

> It relates a lot to the libdnf efforts and higher level design.  And this
discussion is also
> probably generally interesting for other librpm-based systems like
zypper, as those
> also intersect with PackageKit today.

> The whole topic of API for package management is a messy and complex one.
> However, I've been thinking about varlink more, and I like the idea
(although I confess
> to not really having tried to really use or implement it in anger, so my
opinion could change
> after that =) )

> The TL;DR of what I'm thinking here is to consider dropping the SWIG
effort from libdnf, and focus
> on varlink.  The question of implementation language (C/C++/whatever)
then becomes
> a lot more orthogonal as it's decoupled from how language bindings work.

> In terms of varlink advantages: Some aspects of libdnf are
dynamic/event-driven; I'm
> specifically thinking here of package downloads for example.  That type
of thing
> is very natural and nice in a varlink-style system.  In contrast with C++
there's
> no real standard for this - it's why both Qt and GObject have signals.

> There are clear advantages in terms of robustness for forking off a
separate process
> for package management; librpm will directly call chroot() in-process
which
> is pretty hostile to anything else you have going on.  (rpm-ostree does
not do this btw
> for client-side layering today).

> Further, the enormous size of Fedora metadata today causes lots of memory
usage
> in libsolv - this was one of the drivers behind having rpm-ostree do
exit-on-idle:
> https://github.com/projectatomic/rpm-ostree/pull/606
> which works today, but it would have worked better from the start if the
daemon
> could just stay around and all of the heavy lifting was in a subprocess.

> To repeat one of the varlink rationales; every programming language can
speak JSON.
> I suspect we don't need high performance in most cases - the only
exception I can
> think of offhand is say a `dnf search foo` type thing; but even then I
suspect there
> is a lot more things to optimize in searching over json parsing.

> varlink doesn't apply to everything; to state the obvious, there's still
a role
> for shared libraries (and language bindings to them) and still a role for
DBus, etc.
> But specifically for package management I think varlink could make a lot
of sense.

I wanted to give a lot of thought to this before I responded, because I
maintain a few projects that would be directly affected by this proposal.
Among other things, I'm the maintainer of dnfdaemon, and also I co-maintain
dnfdragora. In Mageia, we've also been working on our migration from urpmi
to DNF, and our tooling is mostly in Perl, rather than Python.

My initial thought around this was that it's absolutely nuts. You're more
or less proposing the model that is used by Docker, Kubernetes/OpenShift,
where libraries aren't actually libraries, but in fact daemons that
everyone is forced to operate through subshells and IPC.

On further reflection, I still consider it crazy... for libdnf. However, I
think it would make sense for splitting "dnf" in two. Something that has
been brought up more than a couple of times in the Fedora development IRC
channel and the mailing list is the idea of merging dnfdaemon into DNF
itself. Among other things, this would provide a means to "harden" from
events such as the terminal dying when Xorg gets into a bad state during a
live upgrade.

This model actually works fairly well for dnfdragora. During the
development of dnfdragora, there were many cases where the UI would break
and quit abruptly, but because the daemon independently was handling the
transaction, the package management action still succeeded. It doesn't
happen very often anymore, but it's still nice to have, and it makes
dnfdragora safer than dnf itself in certain circumstances. However,
dnfdaemon is very brittle because we rely on hooking into "private" APIs of
dnf to expose enough functionality to be useful. The "public" API isn't
enough. One way to fix this is to merge dnfdaemon into dnf itself, and
rewire python-dnfdaemon so that it's a client for daemonized dnf.

Now, there are still problems with this model that I'm unclear that a
varlink-enhanced dnf would solve:
* Can we survive events that cause all daemons to need to be restarted?
D-Bus doesn't handle this gracefully. I don't know if dbus-broker makes
this better...
* Can we operate in minimal contexts (where we don't have management
services or even daemons?)
* Can we deal with large quantities of information requested well? One of
the bottlenecks in dnfdaemon that there's not a great answer for is how to
deal with requesting _all available packages to install_.

To be clear, I think we should _not_ abandon having native bindings for
libdnf, because those are critical for building 

Re: [Rpm-ecosystem] lazy loading of filelists.xml to speed up dnf

[Neal Gompa]

On Wed, Aug 8, 2018 at 7:09 PM Pascal Terjan  wrote:
>
> On 7 August 2018 at 09:50, Michael Schroeder  wrote:
> > On Mon, Aug 06, 2018 at 04:36:07PM +, Zbigniew J??drzejewski-Szmek 
> > wrote:
> >> this mail is a continuation of an FPC [1] and a FESCo [2] tickets.
> >>
> >> A proposal was made is to disallow packages in Fedora from using file
> >> deps, and to optimize dnf to not load filelists.xml. File deps would
> >> still be supported, because external packages and users want to use
> >> them, but they would not be allowed for distro packages.
> >>
> >> Not downloading or loading filelists.xml which are required for file
> >> deps would provide significant bandwidth savings (~47 MB compressed)
> >> and noticeable runtime savings (~10s at dnf startup) in many common
> >> cases.
> >>
> >> So this is something that is worth exploring, but it's not clear if it
> >> is at all feasible.
> >
> > There's also something that can easily be done and would make
> > loading the filelist unneeded in most of the cases: extend the
> > primary filelist to include some whitelist of files. The whitelist
> > must also be stored in the primary data, so that the solver knows
> > what to expect.
>
> That's what Mandrake/Mandriva/Mageia/... has been doing for many
> years, there is a small file-deps file containing the ones we end up
> generating, mostly from scriptlets IIRC, and we end up with provides
> added for those in the main metadata when generating it. Then file
> lists are lazily loaded when people want to query them but not used
> for dependency resolution.
>
> $ GET 
> http://ftp.free.fr/mirrors/mageia.org/distrib/cauldron/x86_64/media/media_info/file-deps
> /bin/csh
> /bin/grep
> /bin/perl
> /usr/bin/ln
> /usr/bin/rm
> /sbin/service
> /usr/bin/chattr
> /usr/bin/guile
> /usr/bin/openssl
> /usr/bin/pear
> /usr/bin/texhash
> /usr/bin/tr
> /usr/bin/which
> /usr/sbin/groupadd
> /usr/sbin/groupdel
> /usr/sbin/useradd
> /usr/sbin/userdel
>

So the primary.xml already includes all that. If you actually look in
the primary.xml.gz files in the Mageia rpm-md data, those are already
there. The problem is that there are people who actually request files
outside of the base whitelist as a means to be able to request
"things" without knowing how they are packaged, because the file path
is the consistent thing across distros. This is supported in YUM and
DNF, just slightly differently.

In this case, the wish is to restore the YUM behavior. The idea is
that stacking this on top of the Zchunk deltarepo extension will yield
incredible boosts for everything.


--
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Some points about zchunk

[Neal Gompa]

On Thu, Jul 12, 2018 at 3:27 PM Jonathan Dieter  wrote:
>
> On Wed, 2018-07-11 at 07:55 -0400, Neal Gompa wrote:
> > On Wed, Jul 11, 2018 at 7:30 AM Michael Schroeder  wrote:
> > >
> > > On Wed, Jul 11, 2018 at 12:23:47PM +0100, Jonathan Dieter wrote:
> > > > That's something I didn't think of, and you're absolutely right.
> > > >
> > > > So, to summarize, we'll leave the gzip entry as "primary", so legacy
> > > > systems will still download without any problems, and create a new
> > > > "primary@zchunk" that librepo will automagically download if zchunk is
> > > > supported?
> > >
> > > I think there should be some librepo option to enable this. Be it
> > > something boolean or a fancy string list that can be used for
> > > some future compression method.
> > >
> > > But I'm just one person. Can someone else on this list please speak
> > > up and tell us if we want to do something stupid or not? Maybe the
> > > librepo maintainer? Or the libdnf maintainer? Anybody?
> > >
> >
> > While I agree that we should have an attribute that declares the
> > compression format for fetching, I wonder how legacy clients would
> > handle it? There must be a reason why we haven't done it so far after
> > going from gzip to bzip2 to lzma to xz?
> >
> > It's not unprecedented for there to be stuff like this (we handle the
> > sqlite data variant with _db). We could, for consistency, do
> > "_zckxml" and have librepo fetch that when it's available.
>
> I'd go with _zck since  is, by default, xml, but, other than
> that, I think what you (and Michael) are suggesting makes sense.
>
> Michael, I agree that there should be some way to manually enable
> zchunk (with a method that will work with future algorithms) in
> librepo.
>
> I don't mind writing the code, but I'd like to hear some consensus on
> the method and element name.

I'm okay with this, but does this mean we could also have (cheaply)
zck of arbitrary xml files appended using tools like modifyrepo_c,
then?



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Some points about zchunk

[Neal Gompa]

On Tue, Jul 10, 2018 at 7:18 AM Michael Schroeder  wrote:
>
> On Mon, Jul 09, 2018 at 09:32:13PM +0100, Jonathan Dieter wrote:
> > I had originally planned to do something along these lines (I think I
> > used primary-zck rather than primary@zchunk), but realized that this
> > pushed the "choose best format" code into the top-level tools, rather
> > than leaving the decision in librepo.
>
> But doesn't it in the top-level tools? How can librepo decide that
> it's ok to use zchunk if the top-level tool can't deal with it?
> IMHO the top-level tool has to tell librepo what compression/format it
> understands.
>

Currently, it doesn't work that way. The assumption is that support is
always equal. This is a flawed assumption, but that's what it is.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Zchunk update

[Neal Gompa]

On Mon, Apr 16, 2018 at 12:32 PM, Jonathan Dieter <jdie...@gmail.com> wrote:
> On Mon, 2018-04-16 at 09:00 -0400, Neal Gompa wrote:
>> On Mon, Apr 16, 2018 at 8:47 AM, Jonathan Dieter <jdie...@gmail.com> wrote:
>> > I've also added zchunk support to createrepo_c (see
>> > https://github.com/jdieter/createrepo_c), but I haven't yet created a
>> > pull request because I'm not sure if my current implementation is the
>> > best method.  My current effort only zchunks primary.xml, filelists.xml
>> > and other.xml and doesn't change the sort order.
>> >
>>
>> Fedora COPR, Open Build Service, Mageia, and openSUSE also append
>> AppStream data to repodata to ship AppStream information. Is there a
>> way we can incorporate this into zck rpm-md? There's been an issue for
>> a while to support generating the AppStream metadata as part of the
>> createrepo_c run using the libappstream-builder library[1], which may
>> lend itself to doing this properly.
>
> Is it repomd.xml that actually gets changed or primary.xml /
> filelists.xml / other.xml?
>
> If it's repomd.xml, then it really shouldn't make any difference
> because I'm not currently zchunking it.  As far as I can see, the only
> reason to zchunk it would be to have an embedded GPG signature once
> they're supported in zchunk.
>

repomd.xml is being changed, so it should be fine, then. It'd be nice
to be able to chunk up AppStream data eventually, though.

>> > The one area of zchunk that still needs some API work is the download
>> > and chunk merge API, and I'm planning to clean that up as I add zchunk
>> > support to librepo.
>> >
>> > Some things I'd still like to add to zchunk:
>> >  * A python API
>> >  * GPG signatures in addition to (possibly replacing) overall data
>> >checksum
>>
>> I'd rather not lose checksums, but GPG signatures would definitely be
>> necessary, as openSUSE needs them, and we'd definitely like to have
>> them in Fedora[2], COPR[3], and Mageia[4].
>
> Fair enough.  Would we want zchunk to support multiple GPG signatures
> or is one enough?
>

Historically, we've used only one GPG key because that's what we do
with RPMs, but technically you can specify multiple keys in a .repo
file for Yum, DNF, and Zypper to use for validating packages and
metadata, so it's absolutely possible to have more. I'd probably
suggest if it's not too difficult, supporting multiple signatures.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Zchunk update

[Neal Gompa]

On Tue, Apr 17, 2018 at 3:05 PM, Jonathan Dieter  wrote:
> On Tue, 2018-04-17 at 17:39 +0200, Michal Novotny wrote:
>> On Tue, Apr 17, 2018 at 4:20 PM, Jonathan Dieter  wrote:
>> > On Tue, 2018-04-17 at 09:08 +0200, Michal Novotny wrote:
>> > > Hello Jonathan,
>> > >
>> > > Once it is in createrepo_c, we could try employing it in Fedora COPR.
>> >
>> > Ok, done.  This copr currently has zchunk and createrepo_c in it.  I
>> > did have to disable the python tests for createrepo_c which means I
>> > probably wouldn't use the python bindings with this release.
>> >
>> > https://copr.fedorainfracloud.org/coprs/jdieter/zchunk/
>> >
>> > To enable zchunk creation, run createrepo_c --zck.  I've created
>> > dictionaries that are appropriate for Fedora's metadata at
>> > https://www.jdieter.net/downloads/zchunk-dicts, and they can be used
>> > with --zck-primary-dict, --zck-filelists-dict and --zck-other-dict.
>> >
>> > To make zchunk downloads efficient, the same dictionary must be used
>> > each time metadata is generated.  Dictionaries aren't mandatory, but
>> > they greatly reduce the size of the compressed metadata.
>>
>> Alright, I will deploy it on staging. But we will need to get it into 
>> Fedora's
>> DistGit first to be able to use it on COPR production instance afterwards...
>> Anyway, looking forward to start experimenting with it.
>>
>> Thank you!
>
> I'm assuming that you're referring here to getting zchunk packaged into
> Fedora.  I'd really like to finalize the file format (we're close, but
> I still need a good way of storing signatures in it) and the download
> API before releasing it into Fedora proper.
>

I'm looking forward to this!

> I would recommend using the dicts mentioned above as they give me over
> 40% space savings for both other.xml.zck and primary.xml.zck.  Do
> please let me know if you run into any problems.
>

Are those dictionaries Fedora specific? If so, how can other
distributions generate similar ones? If not, still, how were they
made? :)



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Zchunk update

[Neal Gompa]

On Mon, Apr 16, 2018 at 8:47 AM, Jonathan Dieter  wrote:
> It's been a number of weeks since my last update, so I thought I'd let
> everyone know where things are at.
>
> I've spent most of these last few weeks reworking zchunk's API to make
> it easier to use and more in line with what other compression tools
> use, and I'm mostly happy with it now.  Writing a simple zchunk file
> can be done in a few lines of code, while reading one is also simple.
>
> I've also added zchunk support to createrepo_c (see
> https://github.com/jdieter/createrepo_c), but I haven't yet created a
> pull request because I'm not sure if my current implementation is the
> best method.  My current effort only zchunks primary.xml, filelists.xml
> and other.xml and doesn't change the sort order.
>

Fedora COPR, Open Build Service, Mageia, and openSUSE also append
AppStream data to repodata to ship AppStream information. Is there a
way we can incorporate this into zck rpm-md? There's been an issue for
a while to support generating the AppStream metadata as part of the
createrepo_c run using the libappstream-builder library[1], which may
lend itself to doing this properly.

[1]: https://github.com/rpm-software-management/createrepo_c/issues/75

> The one area of zchunk that still needs some API work is the download
> and chunk merge API, and I'm planning to clean that up as I add zchunk
> support to librepo.
>
> Some things I'd still like to add to zchunk:
>  * A python API
>  * GPG signatures in addition to (possibly replacing) overall data
>checksum

I'd rather not lose checksums, but GPG signatures would definitely be
necessary, as openSUSE needs them, and we'd definitely like to have
them in Fedora[2], COPR[3], and Mageia[4].

[2]: https://pagure.io/releng/issue/133
[3]: https://bugzilla.redhat.com/show_bug.cgi?id=1373331
[4]: https://bugs.mageia.org/show_bug.cgi?id=19432

>  * An expiry field? (I'm obviously thinking about signed repodata here)

Do we need an expiry field if we properly processed the key
revocation/expiration in librepo? My understanding is that current
hiccup with it is that we don't, and that the GPG keyring used in
librepo is independent of the RPM keyring (which it shouldn't be).


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] New RPM-Extras repository

[Neal Gompa]

On Tue, Mar 27, 2018 at 7:02 AM, Florian Festi  wrote:
> Hi!
>
> For quite a while it has become apparent that there are many scripts,
> macro files and other rpm related pieces that all the different
> distributions maintain on their own. We have been trying to get some of
> this merged upstream but there is only so much that can be done there.
> Some things are just not suited for the rpm repository.
>
> Nevertheless having things scattered is not a good solution either. So
> we just created a new repository: rpm-extras [1]
>

Awesome Florian! I know we've talked off and on about it for quite a
while now, so I'm glad to see it finally coming to fruition.

Could I be granted administrative access to the repository to manage
it and help with PRs and such? My GitHub username is Conan-Kudo.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Announcing DNF 3 development

[Neal Gompa]

On Mon, Mar 26, 2018 at 8:26 AM, Martin Sehnoutka <msehn...@redhat.com> wrote:
>
>
> On 03/26/2018 01:38 PM, Neal Gompa wrote:
>> On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl <mc...@cepl.eu> wrote:
>>> On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
>>>> On 03/22/2018 01:40 PM, Daniel Mach wrote:
>>>>> Please read more details on our blog:
>>>>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>>>>
>>>> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should
>>>> use Developer Toolset to compile on Red Hat Enterprise Linux
>>>> 7 if you need C++11 support.  The system compiler, GCC 4.8,
>>>> has limited support only.
>>>
>>> When switching the programming langauge than I would think there
>>> are some better C-successors than C++, namely Rust? Mad rush of
>>> giving up on 46 years old language and switching to one which is
>>> just 33 years old seems a bit bizarre to me.
>>>
>
> Take a look into the code, it is mostly C with few features from C++.
>
> btw what is the motivation to use GOBjects? Is the libdnf api supposed
> to be consumed by dnf frontend via gi repository?
>

It was a thought a while ago with libhif, and as part of the final
rationalization for libdnf, it's being dropped. Because libdnf is
going to be in C++, it's going to use SWIG for bindings generation.

>>
>> I'm okay with not dealing with LLVM for my system package manager,
>> thank you very much. I'd be more open to Rust if Rust also could be
>> built with GCC, and thus supported across literally everything, but no
>> one is investing in that effort.
>>
>
> Well, investment like this will need some justification, not saying that
> dnf should be the one, but you will definitely need a big, important
> project.
>

Considering all the other "big important things" people don't invest
in anyway, I don't think that'd help any.

>> And frankly, Rust is harder to program in than C++, and creating
>> bindings is no walk in the park.
>>
>
> Purely personal opinion. You are probably referring to the learning
> curve, which is known to be steep, but after this period it is well
> worth the effort.
>

Not my personal opinion. That's the opinion of several developers I
know who are working on Rust based projects. Not everyone gets the
benefit of GNOME forcing all the things so that stuff _must_ work.

> Regarding the bindings, if libdnf is meant to be used via gir (see my
> question above), then there is already an effort to make this much
> easier (I'm referring to gnome-class).
>

As I noted earlier in this email, gir is a leftover and is being removed.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Announcing DNF 3 development

[Neal Gompa]

On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl  wrote:
> On 2018-03-26, 10:52 GMT, Florian Weimer wrote:
>> On 03/22/2018 01:40 PM, Daniel Mach wrote:
>>> Please read more details on our blog:
>>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/
>>
>> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should
>> use Developer Toolset to compile on Red Hat Enterprise Linux
>> 7 if you need C++11 support.  The system compiler, GCC 4.8,
>> has limited support only.
>
> When switching the programming langauge than I would think there
> are some better C-successors than C++, namely Rust? Mad rush of
> giving up on 46 years old language and switching to one which is
> just 33 years old seems a bit bizarre to me.
>

I'm okay with not dealing with LLVM for my system package manager,
thank you very much. I'd be more open to Rust if Rust also could be
built with GCC, and thus supported across literally everything, but no
one is investing in that effort.

And frankly, Rust is harder to program in than C++, and creating
bindings is no walk in the park.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] RFC #417 %optional file attribute

[Neal Gompa]

On Mon, Mar 26, 2018 at 1:29 PM, Dmitry V. Levin <l...@altlinux.org> wrote:
> On Mon, Mar 26, 2018 at 01:08:15PM -0400, Neal Gompa wrote:
>> On Mon, Mar 26, 2018 at 6:43 AM, Florian Festi <ffe...@redhat.com> wrote:
>> > Hi!
>> >
>> > We are currently pondering about #417 [1]. For adding a %optional file
>> > attribute that would allow adding file to to %files sections that may
>> > not be built under some circumstances (e.g. some architectures).
>> >
>> > It is already perfectly legal to have files not listed explicitly if
>> > they are within a directory that is included in the %file section. But
>> > some packages (examples wanted) may have trouble using this due to the
>> > way the package files are laid out.
>> >
>> > Otoh %optional would be another spec language key word that packagers
>> > have to deal with and we as RPM upstream developers have a hard time
>> > judging whether the benefit of the attribute really out weights the cost
>> > of bloating the spec language.
>> >
>> > Any input - especially with real world packages that would benefit such
>> > an addition - is welcome.
>> >
>>
>> I personally wouldn't use %optional in most cases, just like I avoid
>> %missingok (the install-time counterpart to this).
>
> %missingok is *not* an install-time counterpart to %optional.
>
> Imagine you have the following use of %missignok:
>
> %ghost %attr(644,root,root) %config(noreplace,missingok) %verify(not md5 
> mtime size) /etc/postfix/*.db
> - this means that some of /etc/postfix/*.db files listed in %files
> *might* be created by the package during its lifetime.
>
> In contrast, %optional means that it's explicitly OK for some of files
> listed in %files to disappear from the package at build time for any reason.
>

That's the missingok modifier. IIRC, there's also a %missingok that exempts
any file for verification because it may be deleted during the
lifetime of its usage, and so it doesn't matter whether it is there or
not, or what happens to it.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] RFC #417 %optional file attribute

[Neal Gompa]

On Mon, Mar 26, 2018 at 6:43 AM, Florian Festi  wrote:
> Hi!
>
> We are currently pondering about #417 [1]. For adding a %optional file
> attribute that would allow adding file to to %files sections that may
> not be built under some circumstances (e.g. some architectures).
>
> It is already perfectly legal to have files not listed explicitly if
> they are within a directory that is included in the %file section. But
> some packages (examples wanted) may have trouble using this due to the
> way the package files are laid out.
>
> Otoh %optional would be another spec language key word that packagers
> have to deal with and we as RPM upstream developers have a hard time
> judging whether the benefit of the attribute really out weights the cost
> of bloating the spec language.
>
> Any input - especially with real world packages that would benefit such
> an addition - is welcome.
>

I personally wouldn't use %optional in most cases, just like I avoid
%missingok (the install-time counterpart to this).

That said, I think it's a perfectly valid strategy for dealing with
some types of content if you're trying to reduce/eliminate large
clusters of ugly conditionals in the %files section (or writing
scripts to generate files list manifests to pass to the files
section). While you can absolutely go without this feature, I think
that's more of a matter for distributions to consider. For example,
SUSE allows for empty file manifests in their packages, while Fedora
does not. And until a few months ago, rich dependencies were banned in
Fedora, while they've been allowed in SUSE for a while.

There are specs that do detection and analysis to fill in these
entries via the manifest, which I consider to be no worse than the
%optional tag for files section. This just reduces the need for
writing potentially error-prone extra scripts for this.

However, I'd like to see a warning thrown whenever a file listed as
%optional is not included because it's not there, just so that
information is captured and obvious when someone looks back through
build logs or something.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] build-id

[Neal Gompa]

On Sun, Mar 18, 2018 at 5:58 PM, Baho Utot <baho-u...@columbus.rr.com> wrote:
>
>
> On 3/18/2018 10:25 AM, Neal Gompa wrote:
>>
>> On Sun, Mar 18, 2018 at 10:20 AM, Baho Utot <baho-u...@columbus.rr.com>
>> wrote:
>>>
>>> I am building a scratch built linux system and I am using rpm as the
>>> package
>>> manager.
>>>
>>> I want to disable build-id in rpm so it discards or doesn't act on the
>>> build-id.
>>>
>>> Can build-id processing be disabled in a macro file?
>>>
>>>If so how does one do that.
>>>
>>> Want to remove the following from the binary rpm package
>>>
>>
>> If you want debuginfo without the build-id files in the main rpm, set
>> the following in your rpm vendor macros file:
>>
>> %_build_id_links alldebug
>>
>> I also highly recommend you set these two in your vendor config, as it
>> makes your debuginfo subpackages more focused:
>>
>> %_debugsource_packages 1
>> %_debuginfo_subpackages 1
>>
>>
>
> I just want to throw away the build-id as I don't have debug packages nor am
> I using sub-packages. This is just a project to build LFS and inserting rpm
> as a package manager as LFS does not have a package manager.  I just need to
> kill off the build-id files in the binary rpm package as the installation
> fails because /usr/lib/buil-id doe not exist.  Having build-id files has no
> use on my scratch built systems.
> That is why I want to just kill them off.
>
> I get the build-id issue when I am using arch linux as the host.
> If I use a scratch built linux for the host build-id does not show up in the
> first build of a package but it does show up if I rebuild the package.
>
>

By default, rpm will not produce debuginfo packages (unless you
install the macros.debug file in the source tree).

But if you want to kill off build-id links, put the following in your
vendor macros file:

%_build_id_links none



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] xml schemas

[Neal Gompa]

On Wed, Mar 20, 2019 at 2:28 PM Pat Riehecky  wrote:
>
>
>
> On 2/18/19 11:14 AM, Neal Gompa wrote:
> > It's on my TODO to collect all the schema documents for rpm-md and
> > host them in one place. My plan is to get that all in one place and
> > set up something that would like you browse the schema for the main
> > stuff and valid extensions.
>
> Any news on this?  I'd love to contribute back the XSD I've got for
> updateinfo
>
> Right now the information on it is scattered a bit.
>

So, it's not much, but this is what I have so far...
https://pagure.io/rpm-metadata

I guess at some point, there should be a project to derive official
schema documents based on the stuff I've collected thus far.



--
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Best practices for replacing directories

[Neal Gompa]

On Thu, Feb 21, 2019 at 2:46 PM Jason L Tibbitts III  wrote:
>
> In the Fedora packaging guidelines we have the following section on
> replacing a directory with a non-directory, and replacing a symlink with
> a directory:
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Directory_Replacement/#_scriptlet_to_replace_a_directory
>
> Recently we received a ticket asking us to revise this slightly:
> https://pagure.io/packaging-committee/issue/859
>
> The suggested revision to the lua code is minor and I suppose makes
> sense, but I'm reluctant to touch anything at all here because the issue
> is subtle.  Furthermore, RPM could potentially change how it deals with
> these cases; for all I know it may even do "the right thing" internally
> now.
>
> Does the Fedora guideline match the current guidance from RPM upstream
> as to how to deal with these situations?
>

Unfortunately, it still does. I don't know if the RPM developers have
a plan to make that obsolete, but it'd be nice if it could go away...

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] Source RPM provides?

[Neal Gompa]

On Tue, Mar 12, 2019 at 10:49 AM Pat Riehecky  wrote:
>
> I've been doing a fair bit of mapping back and forth from binary rpms to
> source rpms.  And I had a question:
>
> Would it make sense for a source rpm to have a generated 'Provides:' for
> any %package defined within the spec?
>
> For example (actual syntax to be determined by folks smarter than myself):
>
> $ rpm -qp --provides man-db-2.8.4-4.fc30.src.rpm
> rpmbuild(man-db)
> rpmbuild(man-db-cron)
> $
>
> I'd expect it to 'provide' all the %packages (even ones masked for your
> specific arch so that we don't deal with executing macros at query and
> whatnot).
>
> Thoughts?
>

It'd probably be better if we had an equivalent to debbuild's
--showpkgs switch that parsed the spec file and identified what
packages would be built from it.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

[Rpm-ecosystem] Using RPM with NDB on macOS

[Neal Gompa]

Hey all,

As I had alluded to in the pull request adding the SQLite rpmdb
backend[1], I have been doing some work every once in a while for the
past few years on rpm for macOS.

A point of frustration has been having a working bdb package, and
maintaining bdb is functionally impossible for me, as I have no
capability to fix bugs there.

I had been experimenting with LMDB since it was introduced.
Unfortunately, it turned out to be more problematic than I expected.
Originally, NDB didn't work on macOS because of its usage of mremap()
(which does not exist on macOS)[2]. Thankfully, Michael Schroeder
recently fixed this in git master[3], so I gave it another try.

After a minor issue[4] and a backport of the ndb fixes to rpm 4.15[5],
I was able to transition to ndb and start trying it.

I've gotten rpm itself working with ndb, and it seems like all the
basic actions work perfectly fine with ndb. I'm working on building up
a package manager stack so that I can start doing some more
interesting tests, but there are issues with libsolv where it makes
the assumption about where the rpmdb path is and what type of rpmdb it
is...

My current plan is to at least get to the point where I can try
working with the solv(1) demo program included in libsolv, and then
proceed from there. At least with the DNF stack, there's still the
problem that it assumes bdb[6] and specific rpmdb paths. I'd like to
get to the point that I could use MicroDNF[7]...

[1]: 
https://github.com/rpm-software-management/rpm/pull/899#issuecomment-543129651
[2]: https://github.com/rpm-software-management/rpm/issues/334
[3]: 
https://github.com/rpm-software-management/rpm/commit/3625ca14c752fa229c79891fcc6374df40b5b588
[4]: https://github.com/rpm-software-management/rpm/pull/907
[5]: 
https://github.com/rpm-software-management/rpm/compare/rpm-4.15.x...Conan-Kudo:rpm-4.15.0-osx-ndb
[6]: https://github.com/rpm-software-management/libdnf/issues/362
[7]: https://github.com/rpm-software-management/microdnf

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] RPMLint 2.0 released!

[Neal Gompa]

On Tue, May 18, 2021 at 8:00 AM Neal Gompa  wrote:
>
> Hey all,
>
> Nearly four years and *754 commits* since rpmlint 1.10, we are
> releasing rpmlint 2.0.0!
>

Oh whoops, I forgot to include the release URL:
https://github.com/rpm-software-management/rpmlint/releases/tag/2.0.0



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

[Rpm-ecosystem] RPMLint 2.0 released!

[Neal Gompa]

Hey all,

Nearly four years and *754 commits* since rpmlint 1.10, we are
releasing rpmlint 2.0.0!

This new release has a _lot_ of new features, but here are the most notable:

* RPMLint now is a "normal" Python application and now supports being
imported like a standard Python module! This means that all the normal
use-cases for RPMLint are still supported, but now you can make it a
part of larger Python-based applications or services.
* RPMLint uses a declarative TOML-based syntax for configuring RPMLint
policy instead of Python code.
* RPMLint now has an override system for the descriptions shown for
various checks, so that distributions who want to give specific policy
information can do so without patching the code.
* RPMLint includes _many more checks_! Nearly all of the generally
useful checks created by the openSUSE community have been merged into
the tree, so distributions can now benefit from a wider offering of
checks to implement policy enforcement.
* RPMLint is Python 3 only and now supports Python 3.6 and newer.
* RPMLint is now built and installed like a standard Python
application using setuptools.

I want to specifically thank Tomáš Chvátal, Martin Liska, Kristyna
Streitova, Dirk Mueller, Miroslav Suchý, Ondřej Súkup, thisisshub, and
Miro Hrončok as top contributors to make this release happen!

Full author list with number of commits:

   309  Tomáš Chvátal
   197  Martin Liska
47  Dirk Mueller
26  Kristyna Streitova
    24  Neal Gompa (ニール・ゴンパ)
24  marxin
    21  Neal Gompa
21  Ondřej Súkup
14  thisisshub
11  Miro Hrončok
 9  Kristýna Streitová
 8  Miroslav Suchý
 6  Markéta Calábková
 5  Ville Skyttä
 4  Ben Greiner
 4  Frank Schreiner
 4  Van de Bugger
 3  David Greaves
 3  Matwey V. Kornilov
 2  Daniel Mach
 2  Matthias Gerstner
 1  Cathy Hu
 1  Ludwig Nussel
 1  MeggyCal
 1  Petr Menšík
 1  Stefan Brüns
 1  Steve Kowalik
 1  Werner Fink
 1  Wolfgang Stöggl
 1  Yanko Kaneti
 1  tpgxyz

--
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] rpm debugedit as a separate project?

[Neal Gompa]

On Fri, Feb 19, 2021 at 3:24 PM Mark Wielaard  wrote:
>
> Hi all,
>
> rpm debugedit has grown from a quick hack that simply listed/replaced
> some files/strings to an almost full blown DWARF reader/writer. It is
> now also used outside rpm(build). Debian packages it separately and
> Flatpak builder has an embedded copy it uses to post-process debuginfo.
>
> It is currently not always easy to update because the people who
> contribute to debugedit are often not regular rpm contributors. And the
> release cycle of rpm doesn't always match up with the release cycle of
> the GNU toolchain. So it sometimes needs a release at a different time
> than rpm gets released.
>
> There have been some requests to have it moved from rpm to elfutils:
> https://sourceware.org/bugzilla/show_bug.cgi?id=27351
> But I think it would be simpler to have it be its own project, so it
> isn't tied to another projects release schedule and processes.
>
> So my proposal is to:
>
> - Create a debugedit project on sourceware, so it is close to binutils,
>   from which it sometimes steals code, elfutils on which it relies for
>   libelf/libdw, and dwz which is a similar, but completely different
>   DWARF processor. Most people currently contributing to rpm debugedit
>   should already have an account there.
>
> - Import tools/debugedit.c tools/hashtab.c tools/hashtab.h and
>   tests/debugedit.at from rpm. Add a minimal build using autoconf and
>   autotest around this. Update the hashtab files from libiberty,
>   check debugedit (and sepdebugcrc checkm see below) for updates
>   which came in from binutils. Note, it also has a popt dependency.
>
> - Setup buildbot using builder.wildebeest.org/buildbot which has
>   support for debian/fedora/centos, armhf, i386, x86_64, aarch64,
>   s390x, ppc64 and ppc64le.
>
> - Provide patches for rpm to have some kind of --with-system-debugedit
>   configure flag so it won't build and install its own debugedit
>   but picks up an installed debugedit on the system.
>
> - Provide patches for flatpak-builder to use debugedit like it already
>   uses eu_elfcompress and eu_strip, instead of calling
>   builder_get_debuginfo_file_references.
>
> - Setup the buildbot so it runs the rpm and flatpak-builder testsuites
>   against debugedit to make sure we keep compatibility.
>
>   This should in theory be easy because both have testsuites that
>   should be zero-fail. But in practice I never got the flatpak-builder
>   tests all green because I don't understand what it is doing with
>   gpg-agent. And I always trip over the usage of fakechroot in the rpm
>   testsuite, on some setups it seems fakechroot is just totally broken?
>
> An open question is for how long rpm and flatpak-builder want to keep
> their internal implementation. And if so, how we keep them in sync. Or
> if we simply decide that once debugedit is ready as separate project
> the next release of rpm and flatpak-builder will simply require
> debugedit as external executable.
>
> Another question is whether the separate debugedit project should also
> adopt some of the other related tools like sepdebugcrcfix, elfdeps and
> maybe scripts/find-debuginfo.sh? sepdebugcrcfix and elfdeps seem easy
> to adopt. find-debuginfo.sh is very tightly linked to how rpm builds
> debuginfo/sources subpackages. But maybe it could be made a little bit
> more generic? But if so, keeping compatibility might be tricky.
>
> I don't think a separate debugedit project needs much maintenance once
> setup. But there are a couple of items to work on. In particular
> support for DWARF5 as emitted by alternative compilers and handling
> Split Dwarf.
>
> Comments and feedback more than welcome.

Splitting out from rpm would make the rpm debuginfo support a bit more
painful to support for non-Linux systems, but I guess most don't care
about that. :(

I don't think it'd be a good idea for it to become part of elfutils,
because the elfutils license makes it unshippable for certain
environments (e.g. automotive and medical, where they get panicky
about that sort of thing).

I'm also not comfortable with the idea of having a part of RPM itself
broken out and transferred to a project with subpar contribution
practices. Most of Sourceware still relies on the email workflow for
contribution of fixes and improvements. While you are the main
contributor the last few years (and you don't use GitHub), you are not
the only one, and you are the only one who uses the email workflow. If
Sourceware projects had something like Pagure overlaid on their Git
repos where they could also take pull requests properly, then I'd be
more comfortable with it.

If we really want to split it out, we could just split it out here in
the rpm-software-management GitHub org. That's where popt lives
too[1]. Also, popt is definitely used by more than just RPM stuff, so
I don't know why you noted it as if it's a problem.

Finally, Debian does not use debugedit for their debuginfo generation.
They've got a much 

Re: [Rpm-ecosystem] rpm debugedit as a separate project?

[Neal Gompa]

On Sun, Feb 21, 2021 at 6:59 PM Mark Wielaard  wrote:
>
> Hi,
>
> rpm-ecosystem@lists.rpm.org wrote:
> > On Fri, Feb 19, 2021 at 09:23:58PM +0100, Mark Wielaard wrote:
> > The main obstacle is that tools/debugedit.c currently depends on rpm:
> >
> > $ git grep -h rpm tools/debugedit.c
> > #include 
> > #include 
> >   if (rpmDigestLength(algorithm) == build_id_size)
> >   ctx = rpmDigestInit(algorithm, 0);
> > rpmDigestUpdate(ctx, build_id_seed, strlen (build_id_seed));
> >   rpmDigestUpdate(ctx, x.d_buf, x.d_size);
> > rpmDigestUpdate(ctx, x.d_buf, x.d_size);
> > rpmDigestUpdate(ctx, d->d_buf, d->d_size);
> >   rpmDigestFinal(ctx, , , 0);
>
> Right. Forgot about needing a hashing algorithm for recreating an unique 
> build-id. This shouldn't be too hard. We can use some libiberty or binutils 
> code for that when we are upgrading the hashtab code. The only trickI part is 
> that the above let's you support arbitrary sized build-ids. But in practice 
> all build-ids have the same size and there are other techniques for chsanging 
> hash sizes.
>

This is not true in practice. Go and Rust build-id's are not the same
size as C/C++ ones, and it'd be a problem if we assumed fixed sizes
for that.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] rpm debugedit as a separate project?

[Neal Gompa]

On Sun, Feb 21, 2021 at 10:19 PM Mark Wielaard  wrote:
>
> Hi Neal,
>
> On Sun, Feb 21, 2021 at 09:13:04PM -0500, Neal Gompa wrote:
> > On Sun, Feb 21, 2021 at 6:59 PM Mark Wielaard  wrote:
> > > rpm-ecosystem@lists.rpm.org wrote:
> > > > On Fri, Feb 19, 2021 at 09:23:58PM +0100, Mark Wielaard wrote:
> > > > The main obstacle is that tools/debugedit.c currently depends on rpm:
> > > >
> > > > $ git grep -h rpm tools/debugedit.c
> > > > #include 
> > > > #include 
> > > >   if (rpmDigestLength(algorithm) == build_id_size)
> > > >   ctx = rpmDigestInit(algorithm, 0);
> > > > rpmDigestUpdate(ctx, build_id_seed, strlen (build_id_seed));
> > > >   rpmDigestUpdate(ctx, x.d_buf, x.d_size);
> > > > rpmDigestUpdate(ctx, x.d_buf, x.d_size);
> > > > rpmDigestUpdate(ctx, d->d_buf, d->d_size);
> > > >   rpmDigestFinal(ctx, , , 0);
> > >
> > > Right. Forgot about needing a hashing algorithm for recreating an
> > > unique build-id. This shouldn't be too hard. We can use some
> > > libiberty or binutils code for that when we are upgrading the
> > > hashtab code. The only trickI part is that the above let's you
> > > support arbitrary sized build-ids. But in practice all build-ids
> > > have the same size and there are other techniques for chsanging
> > > hash sizes.
> >
> > This is not true in practice. Go and Rust build-id's are not the same
> > size as C/C++ ones, and it'd be a problem if we assumed fixed sizes
> > for that.
>
> There are no language specific build-ids. It depends on what you tell
> the linker to use. The binutils ld and gold linkers defaults to
> 160-bits, but also lets you choose 128-bits build-ids, or even none of
> course. In theory a user could provide a fixed size hex string as
> "build-id" but then it isn't really one of course. The lld linker did
> experiment with 8-bit style build-ids. But backed that out when it
> became clear that wasn't enough bits to generate globally unique ids
> (you really need at least 128 bits for that). Depending on how you
> build your Go program it might use a link process that does not
> generate a build-id at all (but it will if you use the system linker
> and then it will use one of the standard sizes).
>
> Since the build-id as seen by debugedit takes up a fixed amount of
> space that cannot be changed anymore (since it is embedded in the
> loadable segment) it just has to deal with the length given.
>
> The code in rpm debugedit only deals with 128, 160, 256, 384 or 512
> bit build-id lengths. See the hashing algorithms it tries. So making
> sure we support at least those lengths will make sure we don't regress
> in support. If you find other lengths in use then I am sure we can
> make the code deal with that.
>

I'm pretty sure I recently saw something that wasn't in those sizes
for Go, but I'll have to look again. The build-id support for these
languages is kind of inconsistent. :(



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] rpm debugedit as a separate project?

[Neal Gompa]

On Sun, Feb 21, 2021 at 1:57 PM Mark Wielaard  wrote:
>
> Hi,
>
> Dmitry V. Levin wrote:
> > On Sun, Feb 21, 2021 at 10:49:03AM -0500, Neal Gompa wrote:
> > [...]
> >> I'm also not comfortable with the idea of having a part of RPM itself
> >> broken out and transferred to a project with subpar contribution
> >> practices. Most of Sourceware still relies on the email workflow for
> >> contribution of fixes and improvements. While you are the main
> >> contributor the last few years (and you don't use GitHub), you are not
> >> the only one, and you are the only one who uses the email workflow. If
> >> Sourceware projects had something like Pagure overlaid on their Git
> >> repos where they could also take pull requests properly, then I'd be
> >> more comfortable with it.
> >
> > On the contrary, I believe all potential debugedit contributors are
> > perfectly fine with email workflow used e.g. in elfutils.
> >
> >> If we really want to split it out, we could just split it out here in
> >> the rpm-software-management GitHub org. That's where popt lives
> >> too[1]. Also, popt is definitely used by more than just RPM stuff, so
> >> I don't know why you noted it as if it's a problem.
> >
> > I don't think the choice of hosting is important at this stage of
> > discussion, but you're well aware that github is out of the consideration
> > for the reason you mentioned above.
>
> I would be happy to try setting up pagure on sourceware even though I believe 
> a simple email based approach encourages more discussion on patches. I am not 
> against forges on principle. It might actually help with the buildbot 
> integration. I don't really like github as a centralized proprietary forge 
> however. It isn't even possible to create an account there without using 
> non-free JavaScript.
>

I would be happy to help with this. :)

Feel free to send emails to the pagure-devel mailing list[1] or join
the IRC channel (#pagure on Freenode) and we can discuss that there.

[1]: https://lists.pagure.io/admin/lists/pagure-devel.lists.pagure.io/



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] why do elf dependency generators cover libraries in paths outside of the library load paths?

[Neal Gompa]

On Fri, Dec 3, 2021 at 3:10 PM Miro Hrončok  wrote:
>
> On 03. 12. 21 20:53, Zbigniew Jędrzejewski-Szmek wrote:
> > We had an incident today [1] that opae-devel has auto-generated provides
> > like libcrypto.so.1.1()(64bit), generated for the bundled copy of openssl
> > (/usr/lib/python3.10/site-packages/pacsign/hsm_managers/openssl/library/libcrypto.so).
> > It was pulled into the buildroot instead of the expected openssl1.1 package.
> > Those deps are generated by /usr/lib/rpm/elfdeps, which is configured in
> > /usr/lib/rpm/fileattrs/elf.attr to act on anything that matches the
> > file magic of '^(setuid,? )?(setgid,? )?(sticky )?ELF (32|64)-bit.*$'.
> >
> > My question: shouldn't we limit the elfdeps generator to files which
> > are in paths that can be loaded automatically by the linker? (This
> > could be implemented as e.g. the paths that are default like
> > /usr/lib64, /usr/local/lib64, …, depending on the architecture, and if
> > the package installs anything in /etc/ld.so/, also the paths listed there.)
> >
> > I always understood those Provides/Requires to be used for library
> > dependency resolution, and it doesn't seem to make sense to generate
> > them for plugins and other "private" objects outside of the linker
> > load paths.
> >
> > I think it's much much more common to have .so libraries outside of
> > the linker paths for which we don't want the automatic provides
> > (python modules, java extensions, various loadable plugins), than to
> > have shared libraries in some custom directory. This should eliminate
> > most of the stupid issues where a "private" dependency leaks and
> > breaks other packages.
> >
> > [1] bugzilla.redhat.com/2028852
>
> As one of the Fedora's Python maintainers, I fully support the idea of only
> generating provides from "normal" library directories.
>
> We still have Python packages that accidentally provide stuff like:
>
> lib.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcalignedsegment.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcalignmentfile.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcbcf.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcbcftools.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcbgzf.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcfaidx.cpython-310-x86_64-linux-gnu.so()(64bit)
> libchtslib.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcsamfile.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcsamtools.cpython-310-x86_64-linux-gnu.so()(64bit)
> libctabix.cpython-310-x86_64-linux-gnu.so()(64bit)
> libctabixproxies.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcutils.cpython-310-x86_64-linux-gnu.so()(64bit)
> libcvcf.cpython-310-x86_64-linux-gnu.so()(64bit)
> libexiv2python.cpython-310-x86_64-linux-gnu.so()(64bit)
> libguestfsmod.cpython-310-x86_64-linux-gnu.so()(64bit)
> libhivexmod.cpython-310-x86_64-linux-gnu.so()(64bit)
> libiscsi.cpython-310-x86_64-linux-gnu.so()(64bit)
> liblo.cpython-310-x86_64-linux-gnu.so()(64bit)
> libpyuhd.cpython-310-x86_64-linux-gnu.so()(64bit)
> libtorrent.cpython-310-x86_64-linux-gnu.so()(64bit)
>
> (This is actually from Fedora Rawhide, as of today.)
>

The issue is self-Requires. Because the RPM dependency generator
doesn't cancel out self-Requires, filtering out Provides leads to
broken Requires in a lot of cases.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] [Rpm-maint] RFC: Relocate RPM and DNF databases to /usr

[Neal Gompa]

On Tue, Dec 7, 2021 at 9:05 AM Panu Matilainen  wrote:
>
> On 12/7/21 06:00, Chris Murphy wrote:
> > Hi RPM and DNF folks,
> >
> > I have a draft change proposal for review and comment, i.e. it's not yet
> > set to be published to Fedora devel@. It's a bit thin, but I expect to
> > fill in more detail following discussion in this thread.
> >
> > https://fedoraproject.org/wiki/Changes/RelocateRPMDNFToUsr
> > 
> >
> >
> > The prior discussions for RPM have happened here:
> > http://lists.rpm.org/pipermail/rpm-maint/2017-October/006681.html
> > 
> > http://lists.rpm.org/pipermail/rpm-maint/2017-October/006723.html
> > 
> >
> > Fedora 36 seems like a good time to do this. What do you think?
>
> rpm-maint is about upstream rpm changes, and for an upstream change, the
> earliest something this drastic could go in would be 4.18 next year,
> presumably aligning with Fedora 37.
>

The more appropriate list would have been rpm-ecosystem@, but many of
the same people are on this list anyway...

> > Is it a given that the relocation should (or must) happen for upgrades?
> > What concerns do you have about relocating both databases during
> > offline-upgrade and ensuring its crash safe? When should the relocation
> > happen; as first or last order of business, or other?
>
> It's not clear to me what exactly you're planning here. Are you talking
> about just following what openSUSE did or something more elaborate?
>
> Technically, pointing rpm to a different database is a matter of
> changing exactly one macro in the configuration. Upgrade business
> generally needs to be left to the distro as rpm will have no way of
> knowing when it's appropriate to migrate and when it's an old chroot
> you're only wanting to peek at. Querying those old chroots and other
> images is something people do quite a lot, and chasing the correct path
> without creating new ones on the way certainly requires more than just a
> macro change.

I think the idea is to pull off the same migration openSUSE did.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

Re: [Rpm-ecosystem] [Rpm-maint] RFC: Relocate RPM and DNF databases to /usr

[Neal Gompa]

(Adding the correct Daniel Mach email address, he moved from Red Hat
to SUSE last month...)

On Thu, Dec 9, 2021 at 10:11 AM Chris Murphy  wrote:
>
> On Thu, Dec 9, 2021 at 5:01 AM Jaroslav Mracek  wrote:
> >
> > Hello Chris,
> >
> > On Tue, Dec 7, 2021 at 5:01 AM Chris Murphy  wrote:
> >>
> >> Hi RPM and DNF folks,
> >>
> >> I have a draft change proposal for review and comment, i.e. it's not yet 
> >> set to be published to Fedora devel@. It's a bit thin, but I expect to 
> >> fill in more detail following discussion in this thread.
> >>
> >> https://fedoraproject.org/wiki/Changes/RelocateRPMDNFToUsr
> >
> >
> > The change is not so simple. It is not only the movement of files from one 
> > location to another one. We store more types of data in that location - 
> > history database (sqlite), module failsafe data (yamls). In future we will 
> > store system state data (toml). Data is not only modified after RPM 
> > transactions but also by module and mark commands. What I want to say is 
> > that the change will be painful but in the proposal there are limited 
> > benefits.
>
> I'm pretty sure I'm generally opposed to the idea of logs being rolled
> back (in a snapshot and rollback regime). Logs should always carry
> forward. Ideally, indicate relevant things like rollbacks in the log,
> because rolling back a log is a form of data loss. The history
> database sounds more like a log than not. So I'm getting squeamish
> about locating that somewhere else. Whereas system state information,
> if it's primarily describing /usr, sounds like it needs to get
> somewhere in /usr.
>

The history database is basically a transaction database like the
rpmdb is. It maps rpmdb transactions to dnf actions. So if you're
rolling back the rpm-managed content, it still makes sense to roll
back the history database, since that's the state of the system. It's
not terribly useful if the matching rpmdb transactions don't exist in
the rpmdb.

That said, the dnfdb is only *important* to move because it's required
for modularity stuff to work properly. Otherwise, it's not a huge deal
one way or another.

> Similar to openSUSE, design efforts around a default snapshot and
> rollback regime in Fedora are complicated when anything in /var
> depends on the state of /usr, and vice versa. As independent and
> interchangeable as they can be, I think the better.
>
> > There is also a question in which location DNF can move data. proposed 
> > `/usr/lib/sysimage/dnf` is maybe not the best one.
>
> What are some alternative suggestions?
>
>
> >> Fedora 36 seems like a good time to do this. What do you think?
> >
> >
> > I don't think it is a good time to perform such a change from a DNF 
> > perspective. We have a plan to introduce a major update to Fedora 38, 
> > therefore it is a better time frame for such a change.
>
> Is it better, worse, or indifferent if the RPM database location were
> changed in Fedora prior to any DNF change?
>

It won't matter. That's how it happened in openSUSE anyway. The rpmdb
path change happened several years earlier, and I changed the
dnfdb path in openSUSE at the top of the year as part of making the
DNF stack work properly on MicroOS.

https://code.opensuse.org/package/libdnf/c/37cdf99c506a7a2189ba26527d43665437c1db86

https://code.opensuse.org/package/dnf/c/24888ad21af0081e7432468d1aadfafd46e23526






--
真実はいつも一つ！/ Always, there's only one truth!
___
Rpm-ecosystem mailing list
Rpm-ecosystem@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-ecosystem