from:"Panu Matilainen"

Re: Debugging fun (wrt C modernization change)

2024-05-19 Thread Panu Matilainen


On 5/17/24 22:42, Owen Taylor wrote:



On Tue, May 14, 2024 at 6:50 AM Panu Matilainen <mailto:pmati...@redhat.com>> wrote:



Patch and source numbers start from zero, that goes for automatically
numbered patches too. So there's an off by one in the application, and
the latter %autopatch which is supposed to apply patches >= 2 simply
has
nothing to do, and falls through silently. That's a bug of course in
itself, filed now:
https://github.com/rpm-software-management/rpm/issues/3093
<https://github.com/rpm-software-management/rpm/issues/3093>

One thing that is quite difficult to do with %autopatch/%autosetup is 
conditionally applied patches, since I've always understood it to be bad 
practice to conditionalize the Patch: lines


The evolution-data-server spec file currently has:


# 0-99: General patches

# 100-199: Flatpak-specific patches
# 
https://gitlab.gnome.org/GNOME/evolution-data-server/-/merge_requests/144 <https://gitlab.gnome.org/GNOME/evolution-data-server/-/merge_requests/144>

Patch100: Make-DBUS_SERVICES_PREFIX-runtime-configurable.patch

[...]

%prep
%autosetup -p1 -S gendiff -N

# General patches
%autopatch -p1 -m 0 -M 99

# Flatpak-specific patches
%if 0%{?flatpak}
%autopatch -p1 -m 100 -M 199
%endif


which will warn if you add a warning about empty %autopatch. Not sure if 
there's a better way of handling that.


Yeah I know. We need a higher level way to handle conditional patches.
Just filed an RFE to look into it: 
https://github.com/rpm-software-management/rpm/issues/3110


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Debugging fun (wrt C modernization change)

2024-05-17 Thread Panu Matilainen


On 5/17/24 15:53, Kevin Kofler via devel wrote:

Michael J Gruber wrote:

%patchlist and %auto* should just go away, or at least banned from Fedora
by a git hook rejecting such specfiles.


We also have unnumbered patch source definition lines, don't we?


IIRC, unnumbered Source: or Patch: just defaults to Source0: resp. Patch0:.
But it should not be used. Use Source0/Patch0 instead.


Unnumbered patch and source lines are automatically numbered nowadays.

- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Debugging fun (wrt C modernization change)

2024-05-17 Thread Panu Matilainen


On 5/17/24 11:32, Michael J Gruber wrote:

Kevin Kofler via devel venit, vidit, dixit 2024-05-16 22:39:00:

Panu Matilainen wrote:

Patch and source numbers start from zero, that goes for automatically
numbered patches too. So there's an off by one in the application, and
the latter %autopatch which is supposed to apply patches >= 2 simply has
nothing to do, and falls through silently. That's a bug of course in
itself, filed now:
https://github.com/rpm-software-management/rpm/issues/3093


And then they say the automagic is a good idea because it prevents people
from accidentally forgetting to apply a patch, LOL.


This would not have happened with autosetup. If you overwrite
automatisms (using invidual patch numbers and flags) you need to know
what you are doing. So this is a very weak argument:


Yes, Kevin missed the point by a wide margin. The problem was exactly 
that the manually numbered patch applications carry these kind of risks. 
It's really, really easy to add "PatchXYZ: foo.patch" and forget to add 
the corresponding %patch XYZ entry, in whatever form. Which is why you 
should let %autosetup do it.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: rich deps result in packages being uninstalled from buildroot

2024-05-16 Thread Panu Matilainen


On 5/16/24 16:10, Vít Ondruch wrote:


Dne 16. 05. 24 v 14:28 Zbigniew Jędrzejewski-Szmek napsal(a):

On Thu, May 16, 2024 at 01:14:16PM +0200, Petr Pisar wrote:

Proper solution is actually minimazing content of the minimal build root

Most of the packages in the buildroot are libraries, pulled in via
dependencies.

@buildsys-build group is:
Mandatory packages   : bash  # basic shell env
  : bzip2 # source extraction
  : coreutils # basic shell env
  : cpio  # source extraction
  : diffutils # source extraction
  : fedora-release-common   # rpm environment
  : findutils # basic shell env
  : gawk  # basic shell env
  : glibc-minimal-langpack  # we want this to 
avoid other langpacks

  : grep  # basic shell env
  : gzip  # source extraction
  : info
  : patch # source extraction
  : redhat-rpm-config  # rpm environment
  : rpm-build  # rpm environment
  : sed   # basic shell env
  : shadow-utils
  : tar   # source extraction
  : unzip # source extraction
  : util-linux    # basic shell env
  : which # basic shell env. (command -v 
is more portable, but meh.)

  : xz    # source extraction



And there is this proposal to reduce the amount of compression libraries:

https://github.com/rpm-software-management/rpm/issues/1396

Actually, since RPM now uses `rpmuncompress` to extract the archives, it 
could also make sense to e.g. extract this utility into separate 
sub-package and move the extraction utilities out from the 
@buildsys-build. That would not reduce the the buildroot size at first, 
but get the dependencies to the right place for the start.




Doesn't work.

rpmuncompress doesn't know how to uncompress ANYTHING AT ALL by itself. 
It only knows which command(s) to call to do that.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Enabling RPM based sysuser handling

2024-05-14 Thread Panu Matilainen


On 5/14/24 13:39, Zbigniew Jędrzejewski-Szmek wrote:

On Mon, May 13, 2024 at 01:37:11PM +0300, Panu Matilainen wrote:

I outlined the migration process last year in 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/NEFOV236FJYS2RED2SEOV5YHDFLDX7DK/#OYCWXKAMIXEZNYPVOM6VQ3YYXQ76M3DG
but failed to follow-up, so I'm glad to see this getting revisited.


I started looking into this, and I think we need to start at the
bottom, i.e. in the setup package.

It currently provides /etc/{passwd,group} with a bunch of ids (23 groups)
and /usr/lib/sysusers.d/20-setup-{users,groups} with a bunch of entries,
but some of the groups listed in sysusers are not listed in the /etc files.
IIUC, once we enable the rpm stuff, rpm will create /etc/{passwd,group}
automatically, and the file provided by setup will be ignored.
(It's specified as %config(noreplace).)

Should be drop the static /etc/{passwd,group} from setup?


The static files aren't harmful as long as they're not duplicated in 
other packages.


I seem to recall seeing systemd-sysusers error out if those files were 
not present, but I might be misremembering and/or it might've changed 
since then. The default mechanism uses useradd/groupadd though, I don't 
know if those support non-existent /etc/{passwd,group}.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Debugging fun (wrt C modernization change)

2024-05-14 Thread Panu Matilainen



Yesterday, we ran into an issue where some of rpm upstream testcases are 
suddenly failing on one system. The difference to everybody elses 
working tests turned out to be Fedora 40, and was easily reproduced 
elsewhere on F40 then.


On a closer look, the failing tests were all complaining about implicit 
function declarations. Okay, something to do with the C modernization 
change. The first of the failure cases was supposed to patch the old 
source against this though, but somehow the patch wasn't getting 
applied. WTH? There was a bit of a panic rising, are we missing patch 
applications somehow? And as a result of the C modernization changes? What?


Looking closer still, the failing test had this:

---
%patchlist
hello-1.0-modernize.patch
hello-1.0-install.patch

%prep
%autosetup -N
%autopatch 1
%autopatch -m 2
---

Spot the error?

Patch and source numbers start from zero, that goes for automatically 
numbered patches too. So there's an off by one in the application, and 
the latter %autopatch which is supposed to apply patches >= 2 simply has 
nothing to do, and falls through silently. That's a bug of course in 
itself, filed now: 
https://github.com/rpm-software-management/rpm/issues/3093


The rest of the failures were simply using a common old source from who 
knows when, relying on implicitly defined printf(). So no hair-rising 
silent patch application failure bugs, thankfully.


We had a pretty good laugh over the whole thing once it unravelled. 
Cheers for the C modernization change work for unearthing these little 
buglets.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Mass Package Change: Turn deprecated %patchN syntax into %patch -PN

2024-05-14 Thread Panu Matilainen


On 5/13/24 17:08, Vít Ondruch wrote:


Dne 13. 05. 24 v 15:22 Panu Matilainen napsal(a):

On 5/13/24 16:09, Vít Ondruch wrote:


Dne 13. 05. 24 v 11:39 Florian Festi napsal(a):
%patch otoh (now) is a regular (though internally implemented) macro 
that is expanded with other macros and though can be used in other 
macros and expressions.



Do I read correctly that we can now use `%patch` in e.g. `%check` 
section? Interesting. Is this documented?


No, while %patch and %setup *could* be made available elsewhere now, 
they are still only available in %prep because that's the only place 
where they make sense.



Working with Ruby, which is interpreted language, there are cases where 
we want to patch tests, while we want to keep them in their original 
form in the package. This might sound weird, but the thing is that for 
running tests, we might be limited by infrastructure. E.g. Koji does not 
have internet access, builders are slow, etc. So we might want to apply 
some patch to workaround such issues.


The rpm model is simply that all source preparation is done in %prep, 
and everything is build around that design. There's no way to enforce 
that, but things like having %setup and %patch only available there are 
pretty strong hints at how it wants to be used.


I would make a copy of the tests for running purposes and patch that, 
all in %prep. That way you can also use %autosetup for it all. There 
will always be special circumstances, but on principle %patch should be 
considered a legacy construct. Packagers shouldn't have to be bothering 
with individual patch applications like that.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Mass Package Change: Turn deprecated %patchN syntax into %patch -PN

2024-05-13 Thread Panu Matilainen


On 5/13/24 16:09, Vít Ondruch wrote:


Dne 13. 05. 24 v 11:39 Florian Festi napsal(a):
%patch otoh (now) is a regular (though internally implemented) macro 
that is expanded with other macros and though can be used in other 
macros and expressions.



Do I read correctly that we can now use `%patch` in e.g. `%check` 
section? Interesting. Is this documented?


No, while %patch and %setup *could* be made available elsewhere now, 
they are still only available in %prep because that's the only place 
where they make sense.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Enabling RPM based sysuser handling

2024-05-13 Thread Panu Matilainen


On 5/13/24 13:24, Daniel P. Berrangé wrote:

On Fri, May 10, 2024 at 01:28:07PM +0200, Florian Festi wrote:

Hi everyone!

RPM 4.19 added automatic sysuser handling [1]. In Fedora 39 this feature
was not enabled right away [2] as it requires some care to properly
transition to it. Also going back to 4.18 was technically still the
fallback option during this change.

I just noticed in an issue in the RPM upstream repository [3] that the
sysuser feature is still not enabled. May be right now might be a good
time to get this going for Fedora 41. I am happy to help with the
technical details but would prefer if this effort was driven from within
Fedora.

Currently users are either done manually by calling useradd in
scriptlets or using the macros in systemd-rpm-macros which is a sub
package of the systemd package. RPM's mechanism is switched off by
rpm-4.18.92-disable-sysusers.patch in the rpm package.

This whole thing probably needs to be a Global Change involving a change
to the Packaging Guidelines [4] and may be an Mass Package Change
(although that might be avoided by changing the macros in
systemd-rpm-macros to NOPs).

Anyone interested in picking this up? I remember quite a few people
being exited about this when it was announced with the rpm-4.19 Change.


IIRC with the current sysusers impl from systemd-rpm-macros, there's an
hard constraint, that the sysusers file had to be explicitly listed
as a separate "SourceNN" file. You could not point %sysusers_create_compat
to a file from the either the build root, or the unpacked source tarball,
as the macro appeared to load its argument at spec parse time.

Does the native RPM implementation avoid this limitation ? It looks like
it probably does...


Yup, no such limitation in the RPM implementation. All you need to do is 
package sysusers file(s) in their native location.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Enabling RPM based sysuser handling

2024-05-13 Thread Panu Matilainen

On 5/13/24 13:07, Florian Festi wrote:

On 5/11/24 12:56, Zbigniew Jędrzejewski-Szmek wrote:

On Fri, May 10, 2024 at 01:28:07PM +0200, Florian Festi wrote:

Anyone interested in picking this up? I remember quite a few people
being exited about this when it was announced with the rpm-4.19 Change.

I would be interested in making this happen.

You mentioned that the transition "requires some care". What are
the problems?

There are Requires created for the users and groups. To make this work
the Provides need to be there first - obviously. So one will probably
need to set %_use_weak_usergroup_deps for a transition period. At least
until the first mass rebuild.

Fedora has already been through at least one mass rebuild under 4.19 so
the provides (and weak requires) are already there.

There are also a large number of packages that are using useradd:
grep useradd *.spec | cut -d: -f1 | sort -u | wc
281 2814090

We need to think what to do with them.

This whole thing probably needs to be a Global Change involving a change
to the Packaging Guidelines [4] and may be an Mass Package Change
(although that might be avoided by changing the macros in
systemd-rpm-macros to NOPs).

The macros are written in a way that if the user/group exist,
no operation is done. Thus, naively, I would think that if rpm
starts to create users and groups on its own, then the existing
scriptlets would become noops. That would mean that we could enable
the feature in rpm without any mass package changes first.

That might work, but I have not looked deep enough into that to do that
blindly.

I'm quite sure there's *some* exception to the rule, but in general even
the useradd era scripts are written in a way to allow pre-existing
users. They kinda have to.

I outlined the migration process last year in
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/NEFOV236FJYS2RED2SEOV5YHDFLDX7DK/#OYCWXKAMIXEZNYPVOM6VQ3YYXQ76M3DG
but failed to follow-up, so I'm glad to see this getting revisited.

- Panu -

--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-02 Thread Panu Matilainen


On 4/1/24 14:46, Neal Gompa wrote:

On Mon, Apr 1, 2024 at 7:38 AM Zbigniew Jędrzejewski-Szmek
 wrote:


On Sun, Mar 31, 2024 at 11:20:17PM -0700, Adam Williamson wrote:

On Sun, 2024-03-31 at 22:13 -0700, Carlos Rodriguez-Fernandez wrote:

Adam,

Is there a way already to achieve test isolation during the rpm build?


Nothing systematic that I'm aware of, no. It would be tricky because
there is no one universal Standard Test System (not even within a
single language ecosystem, let alone across all of them). Currently
you'd have to design something unique, if you wanted to implement this
for your package.


If we wanted to pursue that, I'd suggest the following:
remount $RPM_BUILD_ROOT read-only for the %check phase
(or maybe overmount it with a writable overlayfs that is thrown
away after %check finishes, and warn if any modifications were made.)
%check is executed after %install, so everything should be in place
before %check, and %check may be skipped, so no modifications to
installed files should be done in %check.

Considering possible implemention details, machinectl has 'bind' and
'bind --read-only' that might be useful here. But mock uses
systemd-nspawn in a way that does register the container with machined.
So maybe it'd be more reasonable to just execute a mount command directly
from mock.

This is independent of the test system and does not require splitting
of the test sources.



Another thing to consider is making RPM unshare each build phase like
it can for scriptlets now at install time[1].

Sandboxing and limiting in rpmbuild itself like we can with rpm can
also help with this. I believe moss[2]' boulder tool does this.

[1]: https://github.com/rpm-software-management/rpm/pull/2666
[2]: https://github.com/serpent-os/moss/tree/main/boulder


Sure, this is an area we've been planning to look into even before this 
incident. For example 
https://github.com/rpm-software-management/rpm/issues/2989 and as a 
result of this thread, 
https://github.com/rpm-software-management/rpm/issues/3010. And builds 
with a read-only source: 
https://github.com/rpm-software-management/rpm/issues/2985


None of this will fix a compromised usptream of course.

- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F40 Change Proposal: Optimized Binaries for the AMD64 Architecture (System-Wide)

2024-01-22 Thread Panu Matilainen


On 1/19/24 18:58, Robert Marcano via devel wrote:

On 12/28/23 1:25 PM, Robert Marcano wrote:

On 12/28/23 12:58 PM, Chris Adams wrote:

Once upon a time, Aoife Moloney  said:

Systemd will be modified to insert the additional directories into the
`$PATH` environment variable (affecting all programs on the system)


Anything that depends on PATH entries is IMHO doomed to failure.  There
are way too many things that explicitly set PATH to "known" values (for
good and bad reasons) to be able to depend on extending it.  Heck, it
took a long time to get sudo just to include /usr/local/{bin,sbin}.



Maybe replacing the /usr/bin related entries with a generic wrapper 
that launch the best binary from the per architecture directories.


Note: This may affect a few programs that use argv[0] for something 
meaningful.


The more I think about this, I am more convinced that /usr/bin installed 
binaries must do this redirection too even in the case the PATH is 
modified as this proposal states.


What about programs intentionally use absolute paths? These programs 
will not take advantage of the optimizations of the binary they are 
trying to start.


There many reasons for sometimes using an absolute path is agood idea, 
like:


1. Security: avoid depending on a PATH that can be user manipulated.
2. Configuration files that allow to switch to alternate implementation 
of the tool they call. I found a few on my installation, and their 
defaults include /usr/bin absolute paths.
3. Programs that run inside PATH is reset for security like sudo, 
CGI-BIN, etc.


Notice this PATH based optimization will only work when callers invoke 
the program by name only, relative paths will not get optimized either, 
a rare case but it could happen.


+100 to this.

Using something path-based will also exclude EVERYTHING started with a 
shebang. I guess we didn't want to optimize those 
Python/Perl/Ruby/whatnot scripts here?


A path based approach isn't anywhere near worth the mess it creates. A 
humble opinion.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F40 Change Proposal: F40 Change Proposal: Unify /usr/bin and /usr/sbin (System-Wide)

2024-01-08 Thread Panu Matilainen


On Wed, Dec 20, 2023 at 7:54 PM Aoife Moloney  wrote:


Wiki -> https://fedoraproject.org/wiki/Changes/Unify_bin_and_sbin



I agree unifying the *programs* to a single directory makes sense. But I 
fail to see anything good come out of bringing all those system daemon 
executables into every users path.


This would seem like a good opportunity to move stuff that isn't 
supposed to be directly run as part of normal usage under /usr/libexec. 
I'm sure there are exceptions but as a rule of thumb, if it has a 
systemd service file then it should no longer be in %_sbindir.


- Panu -

--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: rpmbuild core dumps

2024-01-02 Thread Panu Matilainen


On 1/2/24 11:49, Florian Weimer wrote:

* Sam Varshavchik:


Stephen Smoogen writes:


https://github.com/rpm-software-management/rpm/issues/
2826>https://github.com/rpm-software-management/rpm/issues/2826


And thanks for opening a bug. I will watch to see what happens.


I'm genuinely curious. Am I really the only one seeing this? The bug
seems fairly clear cut to me. What the heck.


I suspect most of us package only files from one user, so the cache
never needs evicting?


Indeed.

Technically the "thread-unsafe" bug has been there since rpm 4.15 which 
was the first version to parallelize the package generation. It's just 
that 4.19 eliminates some code that has previously managed to more or 
less mask it it seems. It could've manifested as silent user/group name 
corruption before this AFAICS.


- Panu -
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: DNF5: Checking signatures of packages installed out of a repository?

2023-11-02 Thread Panu Matilainen


On 11/1/23 17:09, Christopher wrote:

On Wed, Nov 1, 2023 at 5:53 AM Paul Howarth  wrote:

Maybe not using dnf, but you can check it using rpm directly:

$ wget mypackage.rpm
$ rpm --checksig mypackage.rpm


Yeah, that's why DNF is more convenient for this... the whole point of
using DNF to install a local file is for consistency of using the same
command as for repo packages, not manually altering the RPM database
outside of YUM/DNF (that results in a warning), and the expectation of


FWIW, dnf doesn't issue any silly warnings about rpm changing its own 
database, that was a yum thing.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Specify koji build machine mem req via. spec file

2023-10-05 Thread Panu Matilainen

On 10/5/23 12:41, Kalev Lember wrote:
On Thu, Oct 5, 2023 at 10:07 AM Miroslav Suchý > wrote:

Dne 04. 10. 23 v 11:43 Martin Stransky napsal(a):
 > Hello guys,
 >
 > Is there's a way how to set requested amount of ram for koji
builders?
 >
 > I'd like to use it as Firefox builds fail recently due low
memory, like
 > https://bugzilla.redhat.com/show_bug.cgi?id=2241690

 >
 > Thanks,
 > Martin
 >
Related - rpm has tunables that allows you to tune the build
according the available resources. However memory is not there:

https://github.com/rpm-software-management/rpm/pull/2418

https://rpm-software-management.github.io/rpm/manual/buildprocess.html 

Oh, nice, I didn't know about that! From a quick look, there are 
actually two memory tunables, %_smp_tasksize_proc and %_smp_tasksize_thread.

Looks like the new thing would replace the constrain_build macro that 
firefox.spec is using:

Instead of current:

# Require 4 GB of RAM per CPU core
%constrain_build -m 4096

It would use:

# Require 4 GB of RAM per CPU core
%global _smp_tasksize_proc 4096

It doesn't look like a huge win from the packager's point of view, but I 
am sure it's better to use the rpm upstream limits instead of external 
macros such as 
constrain_build. 

Yep, memory is certainly considered, it was really the main driver of 
that change.

Compared to %constrain_build the big win is that memory is always 
considered, regardless of whether the packager asked for it or not. It 
would be a useful experiment to drop that tunable from the spec because 
it probably isn't necessary anymore (with rpm >= 4.18 that is).

The other win is that the native version also covers threads, which 
helps on 32bit systems in particular.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Intention to tighten RPM crypto-policy back

2023-09-28 Thread Panu Matilainen


On 9/27/23 20:37, Alexander Sosedkin wrote:

On Tue, Sep 19, 2023 at 11:19 AM Alexander Sosedkin
 wrote:


Hello,

6 months ago, there's been a F38 blocker: https://pagure.io/fesco/issue/2960
Long story short:
RPM has moved to sequoia,
sequoia has started respecting crypto-policies,
Google repos have been signed with a 1024-bit DSA key,
Google Chrome was not installable => F38 blocker.
Back at the time, it's been hastily "resolved"
by relaxing RPM security through crypto-policies
just enough to tolerate that Google signature:
https://bugzilla.redhat.com/show_bug.cgi?id=2170878
https://gitlab.com/redhat-crypto/fedora-crypto-policies/-/merge_requests/129

Since then it has been brought to my attention that
Google has now added a 4096 bit RSA key
https://www.google.com/linuxrepositories/
(EB4C 1BFD 4F04 2F6D DDCC EC91 7721 F63B D38B 4796)

Because of that, I'd like to revert that RPM policy relaxation
https://gitlab.com/redhat-crypto/fedora-crypto-policies/-/commit/a12f7b20638be8f872ad1995c7d2edce41c227b5
in (f39) rawhide and align RPM security with the rest of the policy.

Thoughts / feedback?


OK, I've messed up.

Clemens Lang has kindly pointed me at a flaw in my testing.
Basically, nothing is as rosy as I've previously shown
because of SHA-1 signatures in the keys.
In fact, even Chrome can't be installed with the change properly reverted.
Guess I'll have to shelve the wide discussion for a while, we aren't ready. =(



AIUI the current issue with Chrome is more that they still include the 
old SHA-1 based key in their repo along with the newer one in a way that 
confuses rpm.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: An update on RHEL moving to issues.redhat.com

2023-09-25 Thread Panu Matilainen

On 9/15/23 20:27, Colin Walters wrote:

One thing I find amusing about this list (which like some others is kind of a
long-running soap opera that happens to sometimes produce software as a side
effect) is that many times, I can see just two bits of information:

- The subject of the email
- The name of the person responding

And I basically *know* what they're going to say.

Yeah, I've noticed this as well. It's what happens in a long
relationship, such as marriage. You develop these certain trigger
exchanges that replay over and over again, even as you often are aware
of them. Many are mostly harmless (does ananas belong in pizza?), but
not all.

Married to the community, now there's a thought :D

Maybe one morning I'll be drinking my coffee, reading a thread like this that has
"issues.redhat.com" in the Subject and see e.g. Kevin Kofler reply, open up the email and
he'll say actually something like "JIRA is so awesome! I love the query language!"¹ and
I'll just spew coffee all over my keyboard laughing in surprise. We could all chose to reply to
threads we ordinarily wouldn't, in a different way - just to, you know, spice things up a bit.
Keep the viewers^Hreaders entertained.

Yes, the rarity of those trigger sequences going off rails makes them
absolutely priceless. Like when the person hating olives suddenly
changes their mind after 20 years.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue

Re: An update on RHEL moving to issues.redhat.com

2023-09-15 Thread Panu Matilainen


On 9/14/23 15:50, Fabio Valentini wrote:

On Thu, Sep 14, 2023 at 2:42 PM Colin Walters  wrote:


On Wed, Sep 13, 2023, at 1:44 PM, Matthew Miller wrote:

On Mon, Sep 11, 2023 at 09:20:09AM -0700, Adam Williamson wrote:

IIRC it was a condition of that proposal that we wind up on a hosted
version of the *open source* release of gitlab, which is something we
managed to talk gitlab into doing for us. IMBW, though, it was a while
ago.


Short version is: yes, we did talk them into that with an informal plan, and
then someone higher up at gitlab pulled the plug on the idea.


FWIW I interact with pagure rarely enough that it is somewhat painful to 
context switch each time.  I accept having to deal with both github and gitlab, 
but going from 2 to 3 has a real cost, particularly around things like CI 
systems.


Switch GitLab and Pagure in that statement and I could say the exact same thing.

Personally, I find the Pagure UI (and GitHub) to be much cleaner and
easier to navigate than the UX mess that is GitLab ...
I even find fully FOSS alternatives like Forgejo (Codeberg) *much*
easier to use than GitLab.


Truly.

GitLab used to be quite nice, back in the day. Nowadays I find it 
impossible to navigate. Pagure has its oddities and I never been at home 
there, but it is indeed the (considerably) lesser evil now. And pine for 
Codeberg (which I use for my personal projects).


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: rawhide build errors on i686

2023-07-06 Thread Panu Matilainen


On 7/6/23 01:24, Kevin Fenzi wrote:

On Wed, Jul 05, 2023 at 01:47:05PM +0200, Fabio Valentini wrote:

On Wed, Jul 5, 2023 at 10:01 AM Paul Howarth  wrote:


Whatever this is, it's intermittent and it's still happening. I'm
getting koschei reports about failed builds with this symptom every
day, and yesterday I had a "real" build fail in that way too:


Yeah this is happening pretty frequently and it's making koschei
almost entirely useless.
Right now I need to go through dozens of failed builds every day and
resubmit them just to find out whether it was an intermittent issue or
not ...


I filed https://bugzilla.redhat.com/show_bug.cgi?id=2220650
against rpm to start with

Please provide any helpfull info you all are seeing with this.
Perhaps we can see a pattern?


Also if you can pinpoint the date of the first occurence that will help 
chasing it down. So far the earliest record seems to indicate June 28th, 
if you have seen it before please report.


 - Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Towards enabling rpm sysusers integration

2023-06-29 Thread Panu Matilainen

On 6/28/23 17:15, Lennart Poettering wrote:

On Di, 27.06.23 12:04, Panu Matilainen (pmati...@redhat.com) wrote:

On 6/22/23 19:55, Steve Grubb wrote:

https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format

I would caution against this whole proposal. Not that I'm against it, but
just saying be careful doing it. People often forget about our security
concerns. Currently, shadow-utils has about 400 places which generate audit
events during the managing of system and user accounts. libuser (I saw the
deprecation email) has 55 places where it sends audit events managing
accounts.

There is a 10 year old (or more) standard published here:
https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Account-Lifecycle-Events

If %pre getent, useradd, and groupadd are being replaced by something, that
something needs to conform to the expected security safeguards that currently
exist. It needs to match the kind of events and the format that currently
exists.

Looking at the systemd-sysusers source [1], it seems to do exactly zero
audit logging. So there's a bit of work to do on that front...

last time I looked auditd is started later than
systemd-sysusers. Hence not sure if sysusers would actually generate
audit messages that auditd could pick them up.

For the rpm integration, "started later" is irrelevant as the user/group
creation takes place during rpm transactions.

In general though: people who care about audit need to send us patches
for this, if this matters to them. I don't think anyone in systemd
upstream wil work on this on their own.

Didn't imply any particular party, just that there's work to do.

Both rpm and systemd would like to see systemd-sysusers used for
user/group creation but audit requirement prevents that. Who should do
the work? Guess there's a bit of a Mexican stand-off here :D

The rpm integration doesn't technically require systemd-sysusers, we can
write a script that calls useradd/groupadd instead. So for us it becomes
a choice between writing that script or adding audit support to
systemd-sysusers. Writing a script based on sysusers.generate-pre.sh may
well be less work and would benefit the non-systemd audiences of rpm at
the same time. We'll see.

Re: Is there a chance to phase out `/lib64` directory?

2023-06-27 Thread Panu Matilainen


On 6/27/23 15:04, Neal Gompa wrote:

On Tue, Jun 27, 2023 at 6:37 AM Panu Matilainen  wrote:


On 6/27/23 13:30, Kalev Lember wrote:

On Tue, Jun 27, 2023 at 11:02 AM Vít Ondruch mailto:vondr...@redhat.com>> wrote:

 I don't think that GCC is always the best example to follow.

 Nevertheless, wouldn't it be worth of phasing out the /lib64? I don't
 know what is the history behind, but I don't think this layout is
 conceptual.


I would like to have a layout similar to what Debian is doing, so that
shared libraries would go in /usr/lib/x86_64-redhat-linux/ and
/usr/lib64 would be a legacy symlink pointing to it.


Likewise.


That kind of layout would make it much easier to do cross compilation
because you could just take the whole /usr/lib/another-host-triplet/
directory from another architecture without it interfering with the host
libraries and use it for cross compilation purposes.


Yes, that /lib/ arrangement allows for all manner of things that
just aren't possible with /lib64.

https://wiki.debian.org/Multiarch/TheCaseForMultiarch is a good read.


It would be a lot of work transitioning to a new layout though.


That it is.



I'd rather move toward sysroot multiarch instead, which would expand
that benefit beyond libraries to basically everything.

But to do anything, RPM's handling of architectures needs to change.
Years ago, I had attempted to fix some of this upstream, but it didn't
make it in.

https://github.com/rpm-software-management/rpm/pull/1038


It didn't go in, but not because we disagreed with the concept as such.

Note that support for multi-arch in rpm is now on the roadmap for 4.20 
(https://rpm.org/roadmap)


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Is there a chance to phase out `/lib64` directory?

2023-06-27 Thread Panu Matilainen


On 6/27/23 13:30, Kalev Lember wrote:
On Tue, Jun 27, 2023 at 11:02 AM Vít Ondruch > wrote:


I don't think that GCC is always the best example to follow.

Nevertheless, wouldn't it be worth of phasing out the /lib64? I don't
know what is the history behind, but I don't think this layout is
conceptual.


I would like to have a layout similar to what Debian is doing, so that 
shared libraries would go in /usr/lib/x86_64-redhat-linux/ and 
/usr/lib64 would be a legacy symlink pointing to it.


Likewise.

That kind of layout would make it much easier to do cross compilation 
because you could just take the whole /usr/lib/another-host-triplet/ 
directory from another architecture without it interfering with the host 
libraries and use it for cross compilation purposes.


Yes, that /lib/ arrangement allows for all manner of things that 
just aren't possible with /lib64.


https://wiki.debian.org/Multiarch/TheCaseForMultiarch is a good read.


It would be a lot of work transitioning to a new layout though.


That it is.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Towards enabling rpm sysusers integration

2023-06-27 Thread Panu Matilainen

On 6/22/23 19:55, Steve Grubb wrote:

https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format

There is a 10 year old (or more) standard published here:
https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Account-Lifecycle-Events

Looking at the systemd-sysusers source [1], it seems to do exactly zero
audit logging. So there's a bit of work to do on that front...

- Panu -

[1] https://github.com/systemd/systemd/blob/main/src/sysusers/sysusers.c
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue

Re: Changes to build environment

2023-06-26 Thread Panu Matilainen


On 6/22/23 20:56, Steve Grubb wrote:

Hello,

I have switched to F38 and find a couple items annoying. I have a workflow that
checks things I develop out of github, rolls it up into an rpm, builds it,
and runs the results through annocheck.

If there is a warning I'd like to investigate, I cd into the build directory.
But oops, now its gone (problem #1). OK, I use rpmbuild -bc  to rebuild it. I
cd into the build directory make a change and run make.


This is because rpm >= 4.18 automatically cleans up the build directory 
on a successful binary package build. Use --noclean to override, or add 
something like this to ~/.popt for a more permanent effect:


rpmbuild alias -b --noclean



gcc: fatal error: environment variable 'RPM_ARCH' not defined

(problem #2)


This is a rather unfortunate side-effect of 
https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects 
discussed in (already linked) 
https://bugzilla.redhat.com/show_bug.cgi?id=2043092


- Panu -


How do I fix these things?

-Steve

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Towards enabling rpm sysusers integration

2023-06-26 Thread Panu Matilainen


On 6/23/23 21:20, Michal Domonkos wrote:

On Thu, Jun 22, 2023 at 01:18:27PM +0300, Panu Matilainen wrote:

Now that the initial hurdle of getting rpm 4.19 into rawhide is over, it's
time to start looking towards enabling the sysusers integration:
https://rpm-software-management.github.io/rpm/manual/users_and_groups.html


[...]


3. The various %sysuser_()* macros in systemd-rpm-macros need to be phased
out. As it'll be a long time before the sysusers feature is in all Fedora
versions, it needs a longer term plan. One simple possibility is do what was
done with all those ldconfig from %post back then: change the %sysusers_()
macros to no-ops in rawhide to let rpm handle it, and only actually bother
updating packages once all relevant versions have the sysusers feature.


This proposal would effectively move all existing packages that create users or
groups from useradd/groupadd (called by those %sysuser* macros underneath) to
systemd-sysusers(8).

I wonder if we shouldn't first just move those macros over to systemd-sysusers
to test-drive this utility at a larger scale and catch any potential bugs or
issues before actually proceeding with the remaining steps as outlined in the
email.

That's a lower-risk first step that should be fairly easy to implement right
away, as mentioned in:

 https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format




The big problem with that approach is that it drags all of systemd into 
every user/group creation. Doesn't matter on an already running system 
but is disastrous in the early install stages where even shadow-utils 
easily creates ordering loops.


The rpm integration avoids that entirely by calling the user/group 
creation helper from outside of the chroot (ie systemd-sysusers 
--root=...) Which means one could pretty much drop all of the static 
/etc/{passwd,group} content (now shipped in setup) because the 
user/group creation facilities are available even to the first package 
being installed.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Towards enabling rpm sysusers integration

2023-06-26 Thread Panu Matilainen


On 6/22/23 18:01, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Jun 22, 2023 at 01:18:27PM +0300, Panu Matilainen wrote:

Hey all,

Now that the initial hurdle of getting rpm 4.19 into rawhide is over, it's
time to start looking towards enabling the sysusers integration:
https://rpm-software-management.github.io/rpm/manual/users_and_groups.html


Cool, let's do this!


That's the spirit :D


We (as in rpm-team) are not pushing for doing all this in Fedora 39, this is
more to start discussion and lay down the necessary steps. In the 4.19
builds so far, the sysusers integration has been entirely disabled because
it needs more coordination than just drop it in. Much of it is between
systemd and rpm, but any package with non-root ownerships will be affected
in the end. At least the following, and not necessarily exactly in this
order:

1. systemd has it's own user/group provides generator which directly
conflicts (both on generated content and file level) with the new native
generator in rpm, and the feature will not work with the provides generated
by systemd.


I merged your PR to disable this, so this first step is done.


Thanks! I'll enable the generator in rpm then.


2. systemd provides users and groups that are actually owned by the setup
package. As rpm is now turning non-root file ownership into dependencies,
systemd could end up pulled in where setup is needed (eg early install
stage) which will not end up well. So systemd will need to stop providing
users it does not actually own.


I was hoping we would be make the dependency on setup optional.
It is a fairly heavyweight package (700+ kb) and with lots of
not-that-useful-on-a-typical-modern-installation stuff (mail alias support,
csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it
clutters /etc, which ideally would be empty, and also /etc/services is 700 kb.
setup is currently pulled in by dependencies, but e.g. in the initrd we should
be fine without it. (And the same applies for e.g. minimal container
images without login users or a shell.)

Maybe the non-essential stuff could be split out into a new
subpackage, with setup only providing /etc/{passwd,group,shadow,gshadow}
with the base set of users and groups, and all other files moved to 
setup-clutter.


Oh, "setup" is not special to rpm in any way, I only talk of it because 
that's where /etc/{passwd,group} traditionally reside. Rpm grabs the 
user/group provides from those files on encounter, so no rpm-side 
changes are needed if they get moved to some other package.



3. The various %sysuser_()* macros in systemd-rpm-macros need to be phased
out. As it'll be a long time before the sysusers feature is in all Fedora
versions, it needs a longer term plan. One simple possibility is do what was
done with all those ldconfig from %post back then: change the %sysusers_()
macros to no-ops in rawhide to let rpm handle it, and only actually bother
updating packages once all relevant versions have the sysusers feature.


+1 to this plan.


4. The sysusers "hook" in rpm needs to be enabled (uncomment
%__systemd_sysusers macro in rpm). It wont do anything at all before 1) and
3) are done though.

6. The user/group dependencies for non-root users need to be turned into
hard requires (initially these are just recommends). I would be suprised if
this doesn't cause some disruption somewhere, although the content that is
not root:root owned is pretty scarce these days.

7. Packages creating or using non-root user/group need to be rebuilt.

7. One day a few years from now, replace
https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/
with "supply a sysusers file for your needs" :P
In reality, it'll need adjusting long before that and for that, it'll need
FPC recommendations and all.

8. Remove all user/group addition related macro and script fubar from specs
for good. The first commit in rpm source tree is from 1995, it'd be a nice
30 year celebration... but I don't expect it to happen quite that soon.
Maybe in 2035 new people will start look at old specs in horror, "What do
you mean they had to deal with all this user/group stuff manually! For 30
years!"

I've begun from 1) now:

https://src.fedoraproject.org/rpms/systemd/pull-request/109

This is merged now and the package is built. (I guess it's probably in
gating now.)


https://src.fedoraproject.org/rpms/rpm/pull-request/45

After those have been done, people can start experimenting with the feature.
I don't remember seeing an actual Fedora Change for either file-trigger
enablement or current %sysuser_* macros so I'm not sure it's needed here
either?


https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format


Thanks, I've missed that.

Guess it's best to make a Change out of this too then.

- Panu -

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-l

Re: Towards enabling rpm sysusers integration

2023-06-26 Thread Panu Matilainen


On 6/22/23 19:55, Steve Grubb wrote:

Hello,

On Thursday, June 22, 2023 11:01:28 AM EDT Zbigniew Jędrzejewski-Szmek wrote:

2. systemd provides users and groups that are actually owned by the setup
package. As rpm is now turning non-root file ownership into dependencies,
systemd could end up pulled in where setup is needed (eg early install
stage) which will not end up well. So systemd will need to stop providing
users it does not actually own.


I was hoping we would be make the dependency on setup optional.
It is a fairly heavyweight package (700+ kb) and with lots of
not-that-useful-on-a-typical-modern-installation stuff (mail alias support,
csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it
clutters /etc, which ideally would be empty, and also /etc/services is 700
kb. setup is currently pulled in by dependencies, but e.g. in the initrd
we should be fine without it. (And the same applies for e.g. minimal
container images without login users or a shell.)


There are several trusted databases there that things like getservbyport
consult. I would think csh stuff could be installed by tcsh.


Maybe the non-essential stuff could be split out into a new
subpackage, with setup only providing /etc/{passwd,group,shadow,gshadow}
with the base set of users and groups, and all other files moved to
setup-clutter.


I think a few more files than that are still  needed. But it could use some
pruning.


3. The various %sysuser_()* macros in systemd-rpm-macros need to be
phased
out. As it'll be a long time before the sysusers feature is in all Fedora
versions, it needs a longer term plan. One simple possibility is do what
was done with all those ldconfig from %post back then: change the
%sysusers_() macros to no-ops in rawhide to let rpm handle it, and only
actually bother updating packages once all relevant versions have the
sysusers feature.

+1 to this plan.


4. The sysusers "hook" in rpm needs to be enabled (uncomment
%__systemd_sysusers macro in rpm). It wont do anything at all before 1)
and 3) are done though.

6. The user/group dependencies for non-root users need to be turned into
hard requires (initially these are just recommends). I would be suprised
if this doesn't cause some disruption somewhere, although the content
that is not root:root owned is pretty scarce these days.

7. Packages creating or using non-root user/group need to be rebuilt.

7. One day a few years from now, replace
https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/
with "supply a sysusers file for your needs"
In reality, it'll need adjusting long before that and for that, it'll
need
FPC recommendations and all.

8. Remove all user/group addition related macro and script fubar from
specs for good. The first commit in rpm source tree is from 1995, it'd
be a nice 30 year celebration... but I don't expect it to happen quite
that soon. Maybe in 2035 new people will start look at old specs in
horror, "What do you mean they had to deal with all this user/group
stuff manually! For 30 years!"

I've begun from 1) now:

https://src.fedoraproject.org/rpms/systemd/pull-request/109


This is merged now and the package is built. (I guess it's probably in
gating now.)


https://src.fedoraproject.org/rpms/rpm/pull-request/45

After those have been done, people can start experimenting with the
feature. I don't remember seeing an actual Fedora Change for either
file-trigger enablement or current %sysuser_* macros so I'm not sure
it's needed here either?


https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format


I would caution against this whole proposal. Not that I'm against it, but
just saying be careful doing it. People often forget about our security
concerns. Currently, shadow-utils has about 400 places which generate audit
events during the managing of system and user accounts. libuser (I saw the
deprecation email) has 55 places where it sends audit events managing
accounts.

There is a 10 year old (or more) standard published here:
https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Account-Lifecycle-Events

If %pre getent, useradd, and groupadd  are being replaced by something, that
something needs to conform to the expected security safeguards that currently
exist. It needs to match the kind of events and the format that currently
exists.


Rpm's sysuser integration simply calls out to the configured helper to 
do the heavy lifting. By default that's systemd-sysusers, I have no idea 
what audit events if any it generates, but it's entirely possible to 
replace that with a script that calls out to useradd and groupadd instead.



The system accounts still need to be accessible by the getpw* family of
functions or there could be a lot of breakage.


Uh, yes? Users and groups do need to be created. We're merely talking 
about letting rpm automate the task based on already packaged sysusers.d 
files instead of requiring packagers to fill out tedious boilerplate to 
handle it.

Towards enabling rpm sysusers integration

2023-06-22 Thread Panu Matilainen


Hey all,

Now that the initial hurdle of getting rpm 4.19 into rawhide is over, 
it's time to start looking towards enabling the sysusers integration:

https://rpm-software-management.github.io/rpm/manual/users_and_groups.html

We (as in rpm-team) are not pushing for doing all this in Fedora 39, 
this is more to start discussion and lay down the necessary steps. In 
the 4.19 builds so far, the sysusers integration has been entirely 
disabled because it needs more coordination than just drop it in. Much 
of it is between systemd and rpm, but any package with non-root 
ownerships will be affected in the end. At least the following, and not 
necessarily exactly in this order:


1. systemd has it's own user/group provides generator which directly 
conflicts (both on generated content and file level) with the new native 
generator in rpm, and the feature will not work with the provides 
generated by systemd.


2. systemd provides users and groups that are actually owned by the 
setup package. As rpm is now turning non-root file ownership into 
dependencies, systemd could end up pulled in where setup is needed (eg 
early install stage) which will not end up well. So systemd will need to 
stop providing users it does not actually own.


3. The various %sysuser_()* macros in systemd-rpm-macros need to be 
phased out. As it'll be a long time before the sysusers feature is in 
all Fedora versions, it needs a longer term plan. One simple possibility 
is do what was done with all those ldconfig from %post back then: change 
the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and 
only actually bother updating packages once all relevant versions have 
the sysusers feature.


4. The sysusers "hook" in rpm needs to be enabled (uncomment 
%__systemd_sysusers macro in rpm). It wont do anything at all before 1) 
and 3) are done though.


6. The user/group dependencies for non-root users need to be turned into 
hard requires (initially these are just recommends). I would be suprised 
if this doesn't cause some disruption somewhere, although the content 
that is not root:root owned is pretty scarce these days.


7. Packages creating or using non-root user/group need to be rebuilt.

7. One day a few years from now, replace
https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/ 
with "supply a sysusers file for your needs" :P
In reality, it'll need adjusting long before that and for that, it'll 
need FPC recommendations and all.


8. Remove all user/group addition related macro and script fubar from 
specs for good. The first commit in rpm source tree is from 1995, it'd 
be a nice 30 year celebration... but I don't expect it to happen quite 
that soon. Maybe in 2035 new people will start look at old specs in 
horror, "What do you mean they had to deal with all this user/group 
stuff manually! For 30 years!"


I've begun from 1) now:

https://src.fedoraproject.org/rpms/systemd/pull-request/109
https://src.fedoraproject.org/rpms/rpm/pull-request/45

After those have been done, people can start experimenting with the 
feature. I don't remember seeing an actual Fedora Change for either 
file-trigger enablement or current %sysuser_* macros so I'm not sure 
it's needed here either?


Comments, thoughts etc?

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: What's the error in this build?

2023-06-22 Thread Panu Matilainen


On 6/22/23 11:07, Richard W.M. Jones wrote:


https://koji.fedoraproject.org/koji/taskinfo?taskID=102443065

I don't see what the error is ...

Rich.



Towards the end:

DEBUG util.py:539:  Executing command: ['/usr/bin/systemd-nspawn', '-q', 
'-M', '03701d048e03484c80696e05b2d9f2fe', '-D', 
'/var/lib/mock/f39-build-43645279-5233438-bootstrap/root', '-a', 
'--capability=cap_ipc_lock', 
'--bind=/tmp/mock-resolv.0hjd84i1:/etc/resolv.conf', '--console=pipe', 
'--setenv=TERM=vt100', '--setenv=SHELL=/bin/bash', 
'--setenv=HOME=/var/lib/mock/f39-build-43645279-5233438/root/installation-homedir', 
'--setenv=HOSTNAME=mock', '--setenv=PATH=/usr/bin:/bin:/usr/sbin:/sbin', 
'--setenv=PROMPT_COMMAND=printf "\\033]0;\\007"', 
'--setenv=PS1= \\s-\\v\\$ ', '--setenv=LANG=C.UTF-8', 
'--setenv=LC_MESSAGES=C.UTF-8', '--resolv-conf=off', '/usr/bin/dnf-3', 
'builddep', '--installroot', 
'/var/lib/mock/f39-build-43645279-5233438/root/', 
'--setopt=install_weak_deps=0', '--disableplugin=local', 
'--disableplugin=spacewalk', '--disableplugin=versionlock', 
'/var/lib/mock/f39-build-43645279-5233438/root//builddir/build/SRPMS/libnbd-1.17.1-2.fc39.src.rpm', 
'--setopt=tsflags=nocontexts'] with env {'TERM': 'vt100', 'SHELL': 
'/bin/bash', 'HOME': 
'/var/lib/mock/f39-build-43645279-5233438/root/installation-homedir', 
'HOSTNAME': 'mock', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 
'PROMPT_COMMAND': 'printf "\\033]0;\\007"', 'PS1': 
' \\s-\\v\\$ ', 'LANG': 'C.UTF-8', 'LC_MESSAGES': 
'C.UTF-8', 'SYSTEMD_NSPAWN_TMPFS_TMP': '0', 'SYSTEMD_SECCOMP': '0'} and 
shell False
DEBUG util.py:442:  No matches found for the following disable plugin 
patterns: local, spacewalk, versionlock
DEBUG util.py:444:  Package coreutils-9.3-1.fc39.x86_64 is already 
installed.
DEBUG util.py:444:  Package util-linux-2.39-4.fc39.x86_64 is already 
installed.

DEBUG util.py:595:  Child return code was: 255

...whereas on eg i686, it continues with:

DEBUG util.py:444:  Package coreutils-9.3-1.fc39.i686 is already installed.
DEBUG util.py:444:  Package util-linux-2.39-4.fc39.i686 is already 
installed.
DEBUG util.py:442:  warning: (null): notification message: recovered 669 
frames from WAL file 
/var/lib/mock/f39-build-43645278-5233438/root/var/lib/dnf/history.sqlite-wal

DEBUG util.py:444:  Dependencies resolved.
[...]
DEBUG util.py:444:  Complete!
DEBUG util.py:595:  Child return code was: 0


So that's 'dnf builddep' failing but *why*...

(the "recovered 669 frams from WAL" message in the successful i686 build 
doesn't seem entirely healthy either)


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Fedora Copr builders updated to Fedora 38

2023-06-14 Thread Panu Matilainen

On 6/14/23 11:02, Pavel Raiskup wrote:

On úterý 13. června 2023 16:57:42 CEST Neal H. Walfield wrote:

On Thu, 08 Jun 2023 21:37:09 +0200,
Ondřej Budai wrote:

RPM Sequoia's crypto policies can be configured, so you should be able to
re-enable SHA-1. However, this would
be a global change, not only for EL6... See
https://docs.rs/sequoia-policy-config/latest/sequoia_policy_config/#hash-functions
...
On Thu, Jun 8, 2023 at 5:42 PM Pavel Raiskup wrote:

Hello maintainers!

Copr builders have been updated to Fedora 38 today (some old builders
might still be running F37 ATM, but when they finish the task(s) they
work on, they will be deleted). Our testsuite is passing just fine, so
you _should_ be fine too :-). Please let us know if you have some
troubles.

There was one important change in Fedora 38 - RPM switched to the
Sequoia crypto backend. It refuses SHA-1 in crypto; which basically
disallows Mock to properly check EL6 GPG signatures. To allow further
builds, we switched to gpgcheck=0 for all epel-6 chroots. If you know a
better work-around, let me know.

I find this behavior surprising. The default policy as set by
fedora-crypto-policies is for rpm-sequoia is to accept SHA-1 (and
DSA-1024, ...):

https://gitlab.com/redhat-crypto/fedora-crypto-policies/-/blob/master/policies/FEDORA38.pol#L75

What policy are you using?

The `DEFAULT:SHA1`, but it is weird - I can not reproduce the build
failure now. Is something changing in the backgrounds?

There haven't been any related changes in the last couple of months
(that I'm aware of), but it was different initially yes.

- Panu -

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build JDKs once, repack everywhere (System-Wide Change)

2023-06-01 Thread Panu Matilainen


On 6/1/23 15:43, Robert Marcano via devel wrote:

On 6/1/23 8:33 AM, Richard W.M. Jones wrote:

On Thu, Jun 01, 2023 at 08:28:18AM -0400, Robert Marcano via devel wrote:

On 6/1/23 3:51 AM, Richard W.M. Jones wrote:

On Wed, May 31, 2023 at 05:27:47PM +0200, Jiri Vanek wrote:

This was heavily discussed when we moved to portable build in rpms -
https://fedoraproject.org/wiki/Changes/JdkInTreeLibsAndStdclibStatic
Long story short yes, if yo wish to distribute jdk *binary* it have
to pass java compliance suite.


It sounds like the problem is the software isn't really open source
because it has some field of use restrictions.  The best plan would be
to change this upstream, and if that isn't possible then to remove it

>from Fedora.


Rich.



It is the same discussion about Firefox that is already settled I
think. The JVM code is open source, even free software. The
trademark isn't. Mozilla doesn't allow anyone to call the browser
Firefox without some rules.


Both projects involve heavy-handed enforcement of trademarks, but
don't seem to be the same issue.  (I'll leave this to lawyers to
answer definitely though.)


Red Hat want to run the tests because Java corporate users want
that, so Red Hat does it and at the same time helps to have a more
robust Java ecosystem.


That's nice, and indeed RHEL ships OpenJDK.  The question is about
what we do in Fedora.

Rich.



All this change is about the burden of maintaining so many OpenJDK 
branches as packages in FEdora. Maybe Fedora should stop distributing 
ancient Java versions as one of our missions is to be cutting edge, 
maybe we are still encouraging too many projects to stay running on Java 8.


I am saying this as some that would be affected if Java 8 isn't packaged 
anymore, I still need to develop and test some things with Java 8 
because at work we have some customers still running ancient hardware 
and have been a pain to make them move to modern distributions. So I 
will have to us another universal OpenJDK build (like Temurin)


Maybe Fedora should just package the latest JDK needed for Android 
development and the latest LTS version. An even the Android (11 I think) 
could be skipped because Android Studio includes a build of it. If there 
are packages still requiring old things, help to update them could be 
offered by packagers, or dropped from the distribution.


+1

This sounds much, much more Fedora than the proposal at hand.

Customers running ancient hardware is a problem for the RHEL space, not 
Fedora.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Status of the forge macros?

2023-05-25 Thread Panu Matilainen


On 5/24/23 18:26, Zbigniew Jędrzejewski-Szmek wrote:

On Wed, May 24, 2023 at 11:13:15AM -0400, Ben Beasley wrote:

In your example, the forge macros simplify the spec file only because a 
snapshot is involved; but the forge macros put the snapshot info in the Release 
field, which is still permissible but deprecated[1].

Without the forge macros, the spec file would admittedly be a little more 
complex. I would probably do something like the following:

%global commit   791953030836d39687688a8e7f1a3e708892cfa1
%global snapdate 20230420

Version: 1.2^%{snapdate}git%(echo '%{commit}' | cut -b -7)
Release: 1%{?dist}


Minor comment:
   %(c=%{commit}; echo ${c:0:7})
is a bit nicer because it doesn't require 'cut', it just uses 'echo',
which is a shell builtin.


Or in rpm >= 4.19 (just landed in rawhide), you can just do

%{sub %{commit} 1 8}

There's a bunch of new macro string operations like that, see the 
related group in https://rpm-software-management.github.io/rpm/manual/macros


 :P

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Please test the new version cpio-2.14

2023-05-22 Thread Panu Matilainen


On 5/15/23 16:03, Lukas Javorsky wrote:

Hi,

I've created an MR [1] for the new cpio-2.14 release.

If your package is using cpio, or you use cpio could you please provide 
any feedback on the new build (if it doesn't break your component)?


I CCied the RPM maintainers because that's a very crucial component that 
uses cpio.


Thanks, but actually rpm does not use the cpio utility at all, it has 
it's own implementation both for reading and writing the archive (and in 
larger packages even the format is not cpio)


- Panu -

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: RPM Sequoia: A Sequoia-based backend for the RPM Package Manager

2023-04-28 Thread Panu Matilainen


On 4/27/23 20:55, Robert Relyea wrote:

On 4/27/23 3:51 AM, Neal H. Walfield wrote:

Hi all,

A year and a half ago, I began working with Panu on using Sequoia as
RPM's OpenPGP parser.  I wrote up our journey from the initial
analysis, to adding the code to RPM, and to getting it into Fedora 38
(yay!) in a blog post.  I'm mentioning it here, as I believe it is of
general interest to this community.  If this is considered off topic,
I apologize in advance.

   https://sequoia-pgp.org/blog/2023/04/27/rpm-sequoia/


Thanks Neal.

A good read indeed.

I do wonder about the error message:

||

|because: SHA1 is not considered secure since 1970-01-01T00:00:00Z|

I'm not sure where the date came from, but SHA1 wasn't published until 
1993. 1970-01-01 looks like an epic of some kind. If you must include a 
'not considered secure' date it should be something between 2010 and 
2017 (2010 when peole started worrying about sha1, 2011 and 2013 when 
NIST said 'stop using it' and 2017 when Google (ironically - since they 
are the ones still signing packages with it) actually broke it). 
Probably best to drop the not considered secure if the received date is 
null|.|




Yup, it's not ideal. It's basically an implementation detail leaking 
into view, see https://gitlab.com/sequoia-pgp/sequoia/-/issues/1000


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: It’s time to transform the Fedora devel list into something new

2023-04-24 Thread Panu Matilainen


On 4/21/23 23:38, Ben Cotton wrote:

On Fri, Apr 21, 2023 at 4:05 PM Maxwell G  wrote:


What evidence shows that the group is ever shrinking? I often see Self
Introduction posts and new people interacting with project. I suppose
that whether they continue interacting afterwards is another question.


I'm glad you asked. Earlier this week I decided to avoid doing other
work by putting together some quick charts of devel list
participation. Here's the number of unique participants per month from
2004–2022:
https://bcotton.fedorapeople.org/images/devel-participation-monthly.png

And for a less-noisy version, the median of the monthly numbers per year:
https://bcotton.fedorapeople.org/images/devel-participation-mean.png

There are a lot of questions left unanswered by this quick analysis,
but there's a clear trend in fewer participants over time. In fact,
last month had the second smallest participant count (tied with
October 2022). Of course, these charts don't show _why_, but they do
support the assertion that folks are dropping out of the conversation
faster than others are joining.



There are fewer participants than in the early heydays, but to me the 
overall participation figures look remarkably stable over the last ten 
years or so. Certainly not any "we fell off the cliff" type fall I 
expected to see based on the discussion.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: It’s time to transform the Fedora devel list into something new

2023-04-24 Thread Panu Matilainen


On 4/21/23 22:07, Matthew Miller wrote:

On Fri, Apr 21, 2023 at 11:54:09AM -0400, JT wrote:

So I'm interested by what you bring up here.  Have you run into situations
where someone wanted to contribute to development but was unwilling to use
a mailing list?  With a community as big as Fedora and with a multitude of


I talk to a lot of people about getting involved in Fedora. "Sign up for
this mailing list and introduce yourself" is a big drop-out point.


I would think the latter to be the bigger obstacle of the two.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: It’s time to transform the Fedora devel list into something new

2023-04-21 Thread Panu Matilainen


On 4/21/23 02:21, Simo Sorce wrote:

Hi Matthew, you say: "We're missing people", and I think, "who?".
And who are you going to miss if you move to discourse?

I will be candid, I tried to use forums since the old phpBB times, it
never works for me.
I have no time to go roaming over forums except if a search engine
brings me there.

The mailing list make messages land in my client, on which I am very
efficient, therefore I can check all messages once a day, and respond
if I find a worthy topic.

Unless this discourse has some great mail bridge (it doesn't) or maybe
an rss feed (I do not use those at work, but I guess I could ?) So that
I can skim messages on my terms, I think I (and those like me) will be
the next "missing people".


Ditto.

I actually quite like Discourse - for a forum software - from experience 
related to various freetime activities.


However, Discourse replacing mailing lists WILL be the end of habitually 
skimming through everything that goes on devel (and a whole bunch of 
other lists) to spot issues that might be of my concern. The result will 
be in me being considerably less aware of what goes around in Fedora, 
rpm related or not.


Just FWIW.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Recommended way to disable parallel build

2023-04-18 Thread Panu Matilainen


On 4/18/23 12:24, Florian Weimer wrote:

What's the best way to disable to disable parallel (make) builds?

This?

%undefine _smp_build_ncpus

It seems it will set RPM_BUILD_NCPUS to the empty string, though.


No, don't do that. It depends on where you want to do it. From a spec 
(typically for a package with broken parallel build):


%global _smp_ncpus_max 1

(a %define will do just as well), or "%_smp_ncpus_max 1" in a macros 
file. Or set RPM_BUILD_NCPUS environment variable to 1.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: %patchN deprecated?

2023-04-03 Thread Panu Matilainen


On 3/31/23 18:49, Ron Olson wrote:
One thing to note is that the new format doesn’t work with EPEL 
releases; I had to revert to the %patchN style for them.




The following bit from 
https://rpm-software-management.github.io/rpm/manual/spec.html was 
already quoted in this thread, but:


%patch is used to apply patches on top of the just unpacked
pristine sources. Historically it supported multiple strange
syntaxes and buggy behaviors, which are no longer
maintained. To apply patch number 1, the following are
recognized:

%patch 1 (since rpm >= 4.18)
%patch -P1 (all rpm versions)
%patch1 (deprecated, do not use)

For new packages, the positional argument form 1) is
preferred. For maximal compatibility use 2). Both forms can
be used to apply several patches at once, in the order they
appear on the line. The third form where the number is a
part of the directive is deprecated and should not be used
anymore.

In other words, if you need to care about compatibility, use -PN.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: %patchN deprecated?

2023-03-30 Thread Panu Matilainen


On 3/29/23 18:58, Todd Zullinger wrote:

Florian Festi wrote:

On 3/29/23 10:31, Michael J Gruber wrote:

Has `%patchN` been deprecated in favour of `%patch N`?


Yes, see %patch section on
https://rpm-software-management.github.io/rpm/manual/spec.html


Quoting that:

 %patch is used to apply patches on top of the just unpacked
 pristine sources. Historically it supported multiple strange
 syntaxes and buggy behaviors, which are no longer
 maintained. To apply patch number 1, the following are
 recognized:

 %patch 1 (since rpm >= 4.18)
 %patch -P1 (all rpm versions)
 %patch1 (deprecated, do not use)

 For new packages, the positional argument form 1) is
 preferred. For maximal compatibility use 2). Both forms can
 be used to apply several patches at once, in the order they
 appear on the line. The third form where the number is a
 part of the directive is deprecated and should not be used
 anymore.

Which gets to Michael's question "which releases can take
it?"

Changing `%patch1` to `%patch 1` limits support to Fedora 37
and above, unless this has been backported to older Fedora
and/or RHEL rpm?  Until it's supported by all current Fedora
and RHEL releases, it's not a change I'd want in the
packages I maintain.  I'd have to go with `%patch -P1`
(anywhere that %autosetup / %autopatch wasn't used).


If you need compatibility, use -PN. That's not going away.

The reason for deprecating %patchN is that it's a mind-bogglingly bad 
syntax for what it does, and prevents making %prep like any other 
section in the spec (don't ask, you don't want to know). Positional 
arguments is the natural way to pass data to a command that requires one 
or more pieces of data to do anything, but at least -PN is not *harmful*.


Note that this all only goes for the "%patchN" pseudo-macro, "PatchN" 
*declarations* are not deprecated and wont be going anywhere. There 
seemed to be some confusion about this earlier in this thread.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: crypto-policies

2023-03-27 Thread Panu Matilainen


On 3/27/23 12:40, Fabio Valentini wrote:

On Mon, Mar 27, 2023 at 11:23 AM Kamil Paral  wrote:


On Sat, Mar 25, 2023 at 8:20 AM Neal H. Walfield  wrote:


Panu wrote https://bugzilla.redhat.com/show_bug.cgi?id=2170878#c126 :


To me the key points here are
1) there's a lot of obsolete/broken crypto out there
2) we need better error messages

Properly dealing with 2) needs an API redesign, but we'll try to work out some 
sort of bandaid solution.


Are better diagnostics sufficient from your point of view, or are you
looking for a different solution?



I think my question in https://bugzilla.redhat.com/show_bug.cgi?id=2170878#c125 
wasn't really answered, or at least I don't understand the implications.


Kamil, would've been good to state that in the bug then. I only saw this 
email by sheer luck.



*putting on both my FESCo and rpm-sequoia package maintainer hats*

The issue which was voted on for blocker status by FESCo ("In order to
unblock, RPM must accept SHA-1 hashes and DSA keys for Fedora 38
(...)") has been resolved.
As far as I can tell, the anydesk case is different. It's not a
problem caused by the new crypto policy - the packages don't use a
SHA-1 signature - but happens because the Sequoia PGP implementation
is stricter about checking signatures for sanity / validity.
If I understand correctly, the packages are signed with a key that
fails validation, so I'm inclined to say "this is not our problem"
(and it also looks like this is an issue that's specific to this
third-party package vendor, in contrast to the original SHA-1 / DSA
issue which affected repositories that are officially endorsed by
Fedora Workstation).


Indeed. The RpmSequoia change is not really about phasing out any 
specific algorithms, that's a different topic. The anydesk case is 
actually a fine showcase of Sequoia doing exactly what the change is 
advertising! Only it's getting drowned in this SHA1/DSA noise, and poor 
error messages (which is rpm's, not Sequoias fault).


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Feedback wanted for a proposed improvement to RPM's ELF dependency generator

2023-02-23 Thread Panu Matilainen


On 2/21/23 14:01, Fabio Valentini wrote:

On Tue, Feb 21, 2023 at 9:40 AM Richard W.M. Jones  wrote:


On Mon, Feb 20, 2023 at 10:56:30AM -0800, Gordon Messmer wrote:

On 2023-02-20 10:01, Richard W.M. Jones wrote:

Does it have to be something which looks so much like it might be a
version number?  For example it could be helpful for debugging if the
generated requires was something like:

   Requires: libnghttp2.so.14()(64bit) >= soname.14.24.1



You mean the literal string "soname", right?


Yes.


I don't know a reason
that wouldn't work off the top of my head, but I also can't think of
a reason that it would add information that wasn't inferred in the
current iteration.  One reason we might want to stick with something
that looks like a version is that we might need to extend the system
to allow an epoch in the future (though I really hope that isn't
necessary.)


I mean if I'm looking at an error message, it might help to know that
I should look at the SONAME instead of trying to work out where the
weird version number had come from.


This reminds me, there *is* precedent for stuffing "arbitrary"
versions into RPM Requires / Provides.
Many language ecosystems use virtual Provides / Requires like
"py3dist(foo) = 1.2.3", "crate(foo) = 0.0.11", or even
"golang(github.com/foo/bar/v2) = 2.0.1", where the version is the
*upstream* version (normalized to only contain characters that are
valid in RPM versions), but that doesn't necessarily match version of
the RPM.
The virtual provides that are generated for shared libraries have
always been a bit odd in this regard - maybe we could do something
like "Provides: soname(libnghttp2.so.14()(64bit)) = 14.24.1"?


And that is not "a bit odd"? :D

The ()(64bit) fubar is of course terrible in any form.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Mystery fedpkg srpm failure

2022-12-07 Thread Panu Matilainen


On 12/2/22 11:14, Florian Weimer wrote:

* Aleksei Bavshin:


On 12/1/22 23:18, Aleksei Bavshin wrote:

On 12/1/22 22:28, Florian Weimer wrote:

I don't see what spec file aspect is causing this failure:

$ fedpkg clone -a cups-bjnp
Cloning into 'cups-bjnp'...
remote: Enumerating objects: 278, done.
remote: Counting objects: 100% (278/278), done.
remote: Compressing objects: 100% (222/222), done.
remote: Total 278 (delta 112), reused 96 (delta 47), pack-reused 0
Receiving objects: 100% (278/278), 158.64 KiB | 752.00 KiB/s, done.
Resolving deltas: 100% (112/112), done.
$ cd cups-bjnp
$ fedpkg srpm
Not downloading unused cups-bjnp-2.0.3.tar.gz

setting SOURCE_DATE_EPOCH=1669852800
error: Bad file: /home/fweimer/cups-bjnp/cups-bjnp-2.0.3.tar.gz: No
such file or directory

RPM build errors:
  Bad file: /home/fweimer/cups-bjnp/cups-bjnp-2.0.3.tar.gz: No
such file or directory
Could not execute srpm: Failed to execute command.

fedpkg-simple doesn't have this problem (presumably because it downloads
whatever is in the sources file, whether used or not), so the package
builds fine in Koji.

Any ideas?

Difference in opinions on specfile syntax :)
This is the regex rpkg uses to find the source/patch tags:
      r'^((source[0-9]*|patch[0-9]*)\s*:\s*(?P.*))\s*$'[1]


Actually,

 r'^((source[0-9]*|patch[0-9]*):\s*(?P.*))\s*$',

The line in my message was something I modified to test the hypothesis.


As you can see, it does not expect spaces between Source[0-9]* and
':'.
rpm, however, allows the spaces and there's 6 packages in Fedora
that use this syntax quirk.
[1]: https://pagure.io/rpkg/blob/master/f/pyrpkg/spec.py#_18


Thanks, this certainly explains it.

Should I submitt a pull request for pyrpkg to use your first variant of
the regexp?  Or how should we deal with this?


FWIW, this could just as well be considered an rpm bug. There's no 
particular reason to support whitespace in that place, more likely it's 
just something that accidentally works.


If somebody cares enough, feel free to file upstream.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Help understanding Fedora CI failure wrt RPM Sequoia

2022-11-10 Thread Panu Matilainen


On 11/9/22 11:12, Panu Matilainen wrote:

On 11/9/22 10:08, Panu Matilainen wrote:

On 11/8/22 21:45, Adam Williamson wrote:

On Tue, 2022-11-08 at 08:11 -0800, Adam Williamson wrote:

On Tue, 2022-11-08 at 13:32 +0200, Panu Matilainen wrote:


But I don't know the slightest thing about ansible, beyond a very 
rough

idea of what kind of tool it is. Just understanding what exactly it's
trying to do here would go a long way, I think. But either it's not in
the logs, or I don't know how to read them.


well, to expand on my previous answer, it gets a bit interesting. At
this point in the process the CI system has installed the packages to
be tested and verified that they're installed, which implies RPM does
at least basically work or else that would have failed. you can see
this in these logs:

https://artifacts.dev.testing-farm.io/700d486d-d409-44fe-b7c3-01634243558e/guest-setup-554c73be-c472-4007-9f14-23eea462c69d/artifact-installation-554c73be-c472-4007-9f14-23eea462c69d/

then it hands off to ansible to actually 'run the tests', and that's
when the error happens. I believe it happens here in ansible's code:

https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/dnf.py#L1366

so that may help you figure it out. I'd look further but I have to run
out right now...


well, I'm back now. Looking into it a bit further, I think the codepath
we're on compresses to this, where `self.base` is a DNF base object:

self.base.upgrade_all()
try:
 self.base.resolve(allow_erasing=self.allowerasing)
 [cut internal stuff that couldn't raise a DNF error]
 for package in self.base.transaction.install_set:
 fail = False
 gpgres, gpgerr = self.base._sig_check_pkg(package)
 [cut more stuff that couldn't raise the error]
 self.base.do_transaction()
except dnf.exceptions.Error as e:
 failure_response['msg'] = "Unknown Error occurred: 
{0}".format(to_native(e))

 self.module.fail_json(**failure_response)

I'm pretty sure that's the path we're on, and "An rpm exception
occurred: package not installed" is the text of a dnf.exceptions.Error
exception that's raised by one of those actions that actually involves
the DNF base object (I cut stuff that doesn't involve it, and one
branch where we'd do something different if we hit a
dnf.exceptions.Error). Next step would be to see why dnf is throwing
that error, I guess. It'd be nice if it said *what* package it thinks
is "not installed"...


Indeed. It is an exceptional situation though.

Anyway, Maxwell G's PR showed that it is indeed a weak signature 
someplace (thanks again!), and with clues learned from that I could 
debug it further on my own.


https://artifacts.dev.testing-farm.io/a1c1cc94-0f2b-41f0-a0c2-2456dd50359a/work-tests.ymllY4cuv/ansible-output.txt

and the smoking gun is here:

494 python3-koji-1.28.1-1.fc38.noarch (not an OpenPGP signature)
495 koji-1.28.1-1.fc38.noarch (not an OpenPGP signature)

Despite the fc38 disttag, this is not the koji package from Fedora 
rawhide, the real rawhide version is koji-1.30.1-2.fc38 and *that* is 
signed with RSA/SHA256. What the heck?


Time to file a ticket someplace. Rel-eng, I suppose?


Filed https://pagure.io/fedora-ci/general/issue/371


So, overnight somebody updated the koji package in 
https://kojipkgs.fedoraproject.org/repos/rawhide/ to the current 
unsigned rawhide build, which makes the issue go away.


Thanks to whoever did that.

And now off to build the thing while the lights are green...

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Help understanding Fedora CI failure wrt RPM Sequoia

2022-11-09 Thread Panu Matilainen


On 11/9/22 10:08, Panu Matilainen wrote:

On 11/8/22 21:45, Adam Williamson wrote:

On Tue, 2022-11-08 at 08:11 -0800, Adam Williamson wrote:

On Tue, 2022-11-08 at 13:32 +0200, Panu Matilainen wrote:


But I don't know the slightest thing about ansible, beyond a very rough
idea of what kind of tool it is. Just understanding what exactly it's
trying to do here would go a long way, I think. But either it's not in
the logs, or I don't know how to read them.


well, to expand on my previous answer, it gets a bit interesting. At
this point in the process the CI system has installed the packages to
be tested and verified that they're installed, which implies RPM does
at least basically work or else that would have failed. you can see
this in these logs:

https://artifacts.dev.testing-farm.io/700d486d-d409-44fe-b7c3-01634243558e/guest-setup-554c73be-c472-4007-9f14-23eea462c69d/artifact-installation-554c73be-c472-4007-9f14-23eea462c69d/

then it hands off to ansible to actually 'run the tests', and that's
when the error happens. I believe it happens here in ansible's code:

https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/dnf.py#L1366

so that may help you figure it out. I'd look further but I have to run
out right now...


well, I'm back now. Looking into it a bit further, I think the codepath
we're on compresses to this, where `self.base` is a DNF base object:

self.base.upgrade_all()
try:
 self.base.resolve(allow_erasing=self.allowerasing)
 [cut internal stuff that couldn't raise a DNF error]
 for package in self.base.transaction.install_set:
 fail = False
 gpgres, gpgerr = self.base._sig_check_pkg(package)
 [cut more stuff that couldn't raise the error]
 self.base.do_transaction()
except dnf.exceptions.Error as e:
 failure_response['msg'] = "Unknown Error occurred: 
{0}".format(to_native(e))

 self.module.fail_json(**failure_response)

I'm pretty sure that's the path we're on, and "An rpm exception
occurred: package not installed" is the text of a dnf.exceptions.Error
exception that's raised by one of those actions that actually involves
the DNF base object (I cut stuff that doesn't involve it, and one
branch where we'd do something different if we hit a
dnf.exceptions.Error). Next step would be to see why dnf is throwing
that error, I guess. It'd be nice if it said *what* package it thinks
is "not installed"...


Indeed. It is an exceptional situation though.

Anyway, Maxwell G's PR showed that it is indeed a weak signature 
someplace (thanks again!), and with clues learned from that I could 
debug it further on my own.


https://artifacts.dev.testing-farm.io/a1c1cc94-0f2b-41f0-a0c2-2456dd50359a/work-tests.ymllY4cuv/ansible-output.txt

and the smoking gun is here:

494 python3-koji-1.28.1-1.fc38.noarch (not an OpenPGP signature)
495 koji-1.28.1-1.fc38.noarch (not an OpenPGP signature)

Despite the fc38 disttag, this is not the koji package from Fedora 
rawhide, the real rawhide version is koji-1.30.1-2.fc38 and *that* is 
signed with RSA/SHA256. What the heck?


Time to file a ticket someplace. Rel-eng, I suppose?


Filed https://pagure.io/fedora-ci/general/issue/371

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Help understanding Fedora CI failure wrt RPM Sequoia

2022-11-09 Thread Panu Matilainen


On 11/8/22 21:45, Adam Williamson wrote:

On Tue, 2022-11-08 at 08:11 -0800, Adam Williamson wrote:

On Tue, 2022-11-08 at 13:32 +0200, Panu Matilainen wrote:


But I don't know the slightest thing about ansible, beyond a very rough
idea of what kind of tool it is. Just understanding what exactly it's
trying to do here would go a long way, I think. But either it's not in
the logs, or I don't know how to read them.


well, to expand on my previous answer, it gets a bit interesting. At
this point in the process the CI system has installed the packages to
be tested and verified that they're installed, which implies RPM does
at least basically work or else that would have failed. you can see
this in these logs:

https://artifacts.dev.testing-farm.io/700d486d-d409-44fe-b7c3-01634243558e/guest-setup-554c73be-c472-4007-9f14-23eea462c69d/artifact-installation-554c73be-c472-4007-9f14-23eea462c69d/

then it hands off to ansible to actually 'run the tests', and that's
when the error happens. I believe it happens here in ansible's code:

https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/dnf.py#L1366

so that may help you figure it out. I'd look further but I have to run
out right now...


well, I'm back now. Looking into it a bit further, I think the codepath
we're on compresses to this, where `self.base` is a DNF base object:

self.base.upgrade_all()
try:
 self.base.resolve(allow_erasing=self.allowerasing)
 [cut internal stuff that couldn't raise a DNF error]
 for package in self.base.transaction.install_set:
 fail = False
 gpgres, gpgerr = self.base._sig_check_pkg(package)
 [cut more stuff that couldn't raise the error]
 self.base.do_transaction()
except dnf.exceptions.Error as e:
 failure_response['msg'] = "Unknown Error occurred: 
{0}".format(to_native(e))
 self.module.fail_json(**failure_response)

I'm pretty sure that's the path we're on, and "An rpm exception
occurred: package not installed" is the text of a dnf.exceptions.Error
exception that's raised by one of those actions that actually involves
the DNF base object (I cut stuff that doesn't involve it, and one
branch where we'd do something different if we hit a
dnf.exceptions.Error). Next step would be to see why dnf is throwing
that error, I guess. It'd be nice if it said *what* package it thinks
is "not installed"...


Indeed. It is an exceptional situation though.

Anyway, Maxwell G's PR showed that it is indeed a weak signature 
someplace (thanks again!), and with clues learned from that I could 
debug it further on my own.


https://artifacts.dev.testing-farm.io/a1c1cc94-0f2b-41f0-a0c2-2456dd50359a/work-tests.ymllY4cuv/ansible-output.txt

and the smoking gun is here:

494 python3-koji-1.28.1-1.fc38.noarch (not an OpenPGP signature)
495 koji-1.28.1-1.fc38.noarch (not an OpenPGP signature)

Despite the fc38 disttag, this is not the koji package from Fedora 
rawhide, the real rawhide version is koji-1.30.1-2.fc38 and *that* is 
signed with RSA/SHA256. What the heck?


Time to file a ticket someplace. Rel-eng, I suppose?

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Help understanding Fedora CI failure wrt RPM Sequoia

2022-11-08 Thread Panu Matilainen


On 11/8/22 13:05, Miro Hrončok wrote:

On 08. 11. 22 11:13, Panu Matilainen wrote:

Hey,

Thought I'd try to get on with the times and do the Sequoia change via 
a PR instead of just pushing as we've traditionally done. So far so 
good, but it throws up an error which I have no idea how to debug:


https://artifacts.dev.testing-farm.io/700d486d-d409-44fe-b7c3-01634243558e/

 > Unknown Error occured: An rpm exception occurred: package not 
installed


Okay, something failed, and it appears to be related to this very 
change as the tests pass with a "no-op" PR. And in that case, a good 
thing it got caught. But how on earth to debug this? I don't see 
anything even remotely relevant in the accompanying logs, nor did I 
find any obvious way to run this stuff locally.


Help, Obi-Wan KenoCI.


I am far from being a jeCI master,

but considering the tests there were added by me, for 
pyproject-rpm-macros, I decided to have a look.


The STI tests are at 
https://src.fedoraproject.org/rpms/rpm/blob/rawhide/f/tests/tests.yml


They more or less start with:

   tasks:
     - dnf:
     name: "*"
     state: latest

And the logs you link to say:

   TASK [dnf] 
*

   fatal: [sut]: FAILED! => {
   "changed": false,
   "failures": [],
   "rc": 1,
   "results": []
   }

   MSG:

   Unknown Error occured: An rpm exception occurred: package not installed

This means the problem you hit is that ansible is unable to run the DNF 
task.


Yep, that much I understood: something dnf related is failing. And that 
something looks like a situation where dnf thinks a package should be 
installed but isn't.


Now, such a scenario could occur in this very context if there are 
packages with SHA1 signatures involved: a previously installed package 
would seem uninstalled as the header just became unaccessible due to the 
bad signature. Or other similar scenarios revolving around this. But 
what package is that?


I know how to reproduce the actual tests locally without ansible, but 
that would avoid this problem. I suggest not trying to reproduce the 
tests, but reproducing the ansible failure in isolation. My ansible 
skills are limited, I knwo how to edit the YAML, but I have no idea how 
to actually try this out.




Okay, a good point - this quite obviously is *not* related to the python 
%generate_requires tests, so trying to reproduce that in itself is not 
useful.


But I don't know the slightest thing about ansible, beyond a very rough 
idea of what kind of tool it is. Just understanding what exactly it's 
trying to do here would go a long way, I think. But either it's not in 
the logs, or I don't know how to read them.


Thanks for trying to help.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Help understanding Fedora CI failure wrt RPM Sequoia

2022-11-08 Thread Panu Matilainen


Hey,

Thought I'd try to get on with the times and do the Sequoia change via a 
PR instead of just pushing as we've traditionally done. So far so good, 
but it throws up an error which I have no idea how to debug:


https://artifacts.dev.testing-farm.io/700d486d-d409-44fe-b7c3-01634243558e/

> Unknown Error occured: An rpm exception occurred: package not installed

Okay, something failed, and it appears to be related to this very change 
as the tests pass with a "no-op" PR. And in that case, a good thing it 
got caught. But how on earth to debug this? I don't see anything even 
remotely relevant in the accompanying logs, nor did I find any obvious 
way to run this stuff locally.


Help, Obi-Wan KenoCI.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F38 proposal: RPM Sequoia (System-Wide Change proposal)

2022-10-25 Thread Panu Matilainen

On 10/24/22 23:23, Petr Menšík wrote:

Hi, maybe it was already answered.

Not long ago Thunderbird switched from using installed GPG to its own
implementation inside. I think I have found the library part and it
seems to be in C++, which should be much more easier to integrate than
rust libraries.

I think the project link is:

https://github.com/rnpgp/rnp

Wouldn't it solve the problems cause in more compatible way? Is has
relatively recent release, so it does not seem abandoned. Is there a
specific reason, why is a Rust implementation chosen instead?

Yes it was already answered, see

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/WSKLHCVFABW442MWDHEIBBE4ZJMLACB2/

and

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YOAI3MQYD3DX7KTM3M6ENFJ5ULHYO3I3/

We would've, *of course*, gone for something C-nativeish if that had
been an option at all. As I said in some other post in this thread, I've
been on the lookout for a viable C-native option for 15+ years. Yet here
we are.

And as I've also said elsewhere in this thread, the plan is to keep the
options open for the future. I don't like the shotgun marriage to Rust
any more than the next person out there.

I like Rust language, but its integration into a core system
component does not seem easy.

Except for the matter of bootstrap dependencies (which has also been
discussed here already), I don't know what the difficulty in this case
is supposed to be.

- Panu
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue

Re: F38 proposal: RPM Sequoia (System-Wide Change proposal)

2022-10-20 Thread Panu Matilainen


On 10/20/22 12:03, Miro Hrončok wrote:

On 10. 10. 22 16:32, Ben Cotton wrote:

For the last 20 years or so, RPM has used a home-grown OpenPGP parser
for dealing with keys and signatures. That parser is rather infamous
for its limitations and flaws, and especially in recent years has
proven a significant burden to RPM development. In order to improve
security and free developer resources for dealing with RPM's "core
business" instead, RPM upstream is in the process of deprecating the
internal parser in favor of [https://sequoia-pgp.org/  Sequoia PGP]
based solution written in Rust.
At this point the change is mostly invisible in normal daily use.


Which of the following will happen:

1) rpm will gain ExclusiveArch: %{rust_arches}
2) we will stop requiring the above in Rust packages, as Rust is 100% 
available

3) rpm will %ifarch %{rust_arches} this change
4) something else (what?)

IMHO if we do 1) we could as well do 2) because without rpm, we won't be 
able to build rpms. 3) seems somewhat tedious for no good reason.


I was under the impression Rust was available for all architectures (for 
Fedora anyway), no? There's no Rust code in rpm now either this didn't 
even cross my mind really :D


Technically, I guess the right thing to do is 1) when Sequoia is 
enabled. Ie:


%if %{with sequoia}
%global crypto sequoia
BuildRequires: rpm-sequoia-devel >= 1.0.0
ExclusiveArch: %{rust_archves}
%else
%global crypto openssl
BuildRequires: openssl-devel
%endif

This is already in rawhide, except for the ExclusiveArch thing.

That said, the non-sequoia options should be considered only a bootstrap 
aid, we're not going to provide security support for the internal parser 
for some fringe architectures only.


I'm not sure that answered your questions though.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F38 proposal: RPM Sequoia (System-Wide Change proposal)

2022-10-14 Thread Panu Matilainen

On 10/13/22 15:14, Neal Gompa wrote:

On Thu, Oct 13, 2022 at 4:24 AM Panu Matilainen wrote:

On 10/13/22 10:53, Neal Gompa wrote:

On Thu, Oct 13, 2022 at 3:29 AM Panu Matilainen wrote:

On 10/13/22 07:18, Kevin Kofler via devel wrote:

For the last 20 years or so, RPM has used a home-grown OpenPGP parser
for dealing with keys and signatures. That parser is rather infamous
for its limitations and flaws, and especially in recent years has
proven a significant burden to RPM development. In order to improve
security and free developer resources for dealing with RPM's "core
business" instead, RPM upstream is in the process of deprecating the
internal parser in favor of [https://sequoia-pgp.org/ Sequoia PGP]
based solution written in Rust.

Why are you using a new library written in Rust? Can you not use one of the
existing mature C implementations of OpenPGP? gpgme maybe?

Had there been such an option, we would've switched over years ago.
Gpgme is based around calling and communicating with an external gpg
process, which is a setup you do NOT want in the rpm context where
chroots come and go etc. Also, rpm requires access to the "low-level"
digests-in-progress because it calculates multiple things on a single read.

The real problem is that all other OpenPGP implementations died out
because of GnuPG. And then GnuPG made the choice to force an
inter-process model.

At work, I deal with this on Debian systems, which do indeed use GnuPG
for this. It creates a lot of problems, especially for building images
and dealing with chroots, which is why in the context of RPM PGP
upstream, I never pushed to consider it.

The most serious problems with PackageKit memory leaks and hangs are
actually caused by GnuPG, since DNF uses it for some GPG stuff because
there's no API for using RPM's PGP capabilities. There's no fix unless
the RPM and DNF teams agree on an API and build it out so that libdnf
and librepo no longer need to use GnuPG through gpgme anymore.

This is also the underlying reason why Red Hat has resisted
implementing signed repository metadata and enforcing it by default.
Of course this is a bit of a catch-22 as well, as there's no
motivation to find a solution because neither Fedora nor RHEL offer
signed repository metadata despite repeated calls for it over the past
decade.

Now, don't get me wrong: I'm personally extremely unhappy about having
to depend on the Sequoia stack for RPM PGP. I have a strong distaste
for the Rust community ecosystem these days, and I don't love the idea
of having to have LLVM in the core bootstrap chain (hopefully gcc-rs
will be in place soon enough!). But we literally don't have any other
options. GnuPG/GPGME is out of the question for the reasons Panu and I
stated, and NeoPG died several years ago. There are no C/C++ libraries
for OpenPGP verification.

There's RNP (in C++, used by Thunderbird at least), but alas that
doesn't expose the digest-in-progress either. So at least in it's
current form, it's not an option for rpm. Also, the API appears to have
all manner of quirks and gotchas that aren't welcome in a
security-critical piece.

Huh, I'd forgotten about RNP. It seems it now has an OpenSSL backend
and at least the verification API (which is what RPM would use) seems
to be getting love lately. Insofar as quirks and gotchas, I'm not a
great judge of that at the moment, but I don't think it could be worse
than what we have with GnuPG.

The missing "digest-in-progress" thing is an issue, I guess. Have we
raised the issue with them about it?

No, because beneath the surface it didn't seem all that appetizing
afterall. See
https://sequoia-pgp.org/blog/2021/05/06/202105-thunderbird-rnp-and-the-importance-of-a-good-api/
for an example (even bearing in mind that a blog post from a competitor
may be necessarily a little opionated), but its reputation/security
record doesn't seem that great.

It's something to keep an eye on of course, it would be good to have
alternatives available.

As for bootstrap, there will always (have to) be a way to build rpm
without depending on Rust. Even if that meant no signature verification
support in such a configuration.

Eck. What about the x509 based stuff we were talking about last year?
All the crypto backends RPM supports now support that stuff out of the
box.

Embedded x509 signatures (certs) to replace GPG signatures could work
as an alternative.

x509 seems to be loathed even more than PGP, so it doesn't seem that
appetizing either.

If someone with known crypto-clue would send patches they would be
looked at, *I* have no prejudice about x509 because I also have no clue
about it. Ditto for Signify, which often gets brought up in these
discussions.

And yet, that all is largely irrelevant for the subject at hand: no
matter what, rpm will need OpenPGP support for years to come because
existing content will need to remain usable for a lon

Re: F38 proposal: RPM Sequoia (System-Wide Change proposal)

2022-10-14 Thread Panu Matilainen


On 10/13/22 17:31, Neal Gompa wrote:

On Thu, Oct 13, 2022 at 9:58 AM Kevin Kofler via devel
 wrote:


Neal Gompa wrote:

No, because when you do things like mirror repositories (especially
for private mirrors), that signature is the only way to verify the
integrity. HTTPS is only transport encryption from a particular
connection.


HTTPS protects against a MITM on the connection introducing invalid
repository contents, which I would assume to be the biggest threat here. But
sure, it by design does not guarantee that the data on the remote end is
valid to begin with.


Also, a ton of Fedora mirrors still don't use HTTPS for various reasons.


I would say that those mirrors ought to be kicked out of the mirror list
immediately.

With Let's Encrypt having been available for years, there is really no
excuse for not offering HTTPS. Assuming you own a domain name (which I
assume to already be the case for all mirrors), setting up HTTPS with Let's
Encrypt does not cost you a dime. Even if you are a commercial entity.



I'm not going to get into this too much, but suffice to say, it's not
universally accessible as a CA. And using Let's Encrypt for private
mirrors is sufficiently painful that I wouldn't recommend it.


Well, it might still be worthwhile to split out RPM's OpenPGP
implementation into its own project and allow people to contribute to
it. The worst that can happen is that nothing changes.


If that implementation is really as awfully broken as Panu is saying, I do
not think that that would be of much use, unfortunately.



There have been attempts to fix things, but Panu doesn't feel
qualified to review the changes. That doesn't mean someone else who
would be willing to do so couldn't. But because of... reasons, as long
as it's in the RPM codebase, it's unlikely someone else will be
trusted enough to do those reviews.


This misses the point by a mile.

Parsing OpenPGP is not rpm's "core business", by any stretch of 
imagination. As such, I refuse to let it take up any significant portion 
of rpm development resources.


The parser has been largely dormant in the codebase for like 20 years, 
except for a couple of cleanup rounds when people have pointed out 
random flaws and vulnerabilities. Last year when people started to 
really look into it, it's become obvious that the effort to fix & review 
it all is just nowhere near justifiable.


Consider this the RPM burning platform memo if you like ;)

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F38 proposal: RPM Sequoia (System-Wide Change proposal)

2022-10-14 Thread Panu Matilainen


On 10/13/22 19:35, Demi Marie Obenour wrote:

On 10/13/22 04:23, Panu Matilainen wrote:

On 10/13/22 10:53, Neal Gompa wrote:

On Thu, Oct 13, 2022 at 3:29 AM Panu Matilainen  wrote:


On 10/13/22 07:18, Kevin Kofler via devel wrote:

For the last 20 years or so, RPM has used a home-grown OpenPGP parser
for dealing with keys and signatures. That parser is rather infamous
for its limitations and flaws, and especially in recent years has
proven a significant burden to RPM development. In order to improve
security and free developer resources for dealing with RPM's "core
business" instead, RPM upstream is in the process of deprecating the
internal parser in favor of [https://sequoia-pgp.org/ Sequoia PGP]
based solution written in Rust.


Why are you using a new library written in Rust? Can you not use one of the
existing mature C implementations of OpenPGP? gpgme maybe?


Had there been such an option, we would've switched over years ago.
Gpgme is based around calling and communicating with an external gpg
process, which is a setup you do NOT want in the rpm context where
chroots come and go etc. Also, rpm requires access to the "low-level"
digests-in-progress because it calculates multiple things on a single read.



The real problem is that all other OpenPGP implementations died out
because of GnuPG. And then GnuPG made the choice to force an
inter-process model.

At work, I deal with this on Debian systems, which do indeed use GnuPG
for this. It creates a lot of problems, especially for building images
and dealing with chroots, which is why in the context of RPM PGP
upstream, I never pushed to consider it.

The most serious problems with PackageKit memory leaks and hangs are
actually caused by GnuPG, since DNF uses it for some GPG stuff because
there's no API for using RPM's PGP capabilities. There's no fix unless
the RPM and DNF teams agree on an API and build it out so that libdnf
and librepo no longer need to use GnuPG through gpgme anymore.

This is also the underlying reason why Red Hat has resisted
implementing signed repository metadata and enforcing it by default.
Of course this is a bit of a catch-22 as well, as there's no
motivation to find a solution because neither Fedora nor RHEL offer
signed repository metadata despite repeated calls for it over the past
decade.

Now, don't get me wrong: I'm personally extremely unhappy about having
to depend on the Sequoia stack for RPM PGP. I have a strong distaste
for the Rust community ecosystem these days, and I don't love the idea
of having to have LLVM in the core bootstrap chain (hopefully gcc-rs
will be in place soon enough!). But we literally don't have any other
options. GnuPG/GPGME is out of the question for the reasons Panu and I
stated, and NeoPG died several years ago. There are no C/C++ libraries
for OpenPGP verification.


There's RNP (in C++, used by Thunderbird at least), but alas that
doesn't expose the digest-in-progress either. So at least in it's
current form, it's not an option for rpm. Also, the API appears to have
all manner of quirks and gotchas that aren't welcome in a
security-critical piece.

As for bootstrap, there will always (have to) be a way to build rpm
without depending on Rust. Even if that meant no signature verification
support in such a configuration.


RPM could keep its own internal parser for bootstrap only.  Bootstrap
does not need support for revocation, etc.


I very intentionally left the possibilities open here. Rust-free 
bootstrap is a hard requirement for rpm, how that is achieved is a whole 
other topic.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F38 proposal: RPM Sequoia (System-Wide Change proposal)

2022-10-13 Thread Panu Matilainen


On 10/13/22 10:53, Neal Gompa wrote:

On Thu, Oct 13, 2022 at 3:29 AM Panu Matilainen  wrote:


On 10/13/22 07:18, Kevin Kofler via devel wrote:

For the last 20 years or so, RPM has used a home-grown OpenPGP parser
for dealing with keys and signatures. That parser is rather infamous
for its limitations and flaws, and especially in recent years has
proven a significant burden to RPM development. In order to improve
security and free developer resources for dealing with RPM's "core
business" instead, RPM upstream is in the process of deprecating the
internal parser in favor of [https://sequoia-pgp.org/ Sequoia PGP]
based solution written in Rust.


Why are you using a new library written in Rust? Can you not use one of the
existing mature C implementations of OpenPGP? gpgme maybe?


Had there been such an option, we would've switched over years ago.
Gpgme is based around calling and communicating with an external gpg
process, which is a setup you do NOT want in the rpm context where
chroots come and go etc. Also, rpm requires access to the "low-level"
digests-in-progress because it calculates multiple things on a single read.



The real problem is that all other OpenPGP implementations died out
because of GnuPG. And then GnuPG made the choice to force an
inter-process model.

At work, I deal with this on Debian systems, which do indeed use GnuPG
for this. It creates a lot of problems, especially for building images
and dealing with chroots, which is why in the context of RPM PGP
upstream, I never pushed to consider it.

The most serious problems with PackageKit memory leaks and hangs are
actually caused by GnuPG, since DNF uses it for some GPG stuff because
there's no API for using RPM's PGP capabilities. There's no fix unless
the RPM and DNF teams agree on an API and build it out so that libdnf
and librepo no longer need to use GnuPG through gpgme anymore.

This is also the underlying reason why Red Hat has resisted
implementing signed repository metadata and enforcing it by default.
Of course this is a bit of a catch-22 as well, as there's no
motivation to find a solution because neither Fedora nor RHEL offer
signed repository metadata despite repeated calls for it over the past
decade.

Now, don't get me wrong: I'm personally extremely unhappy about having
to depend on the Sequoia stack for RPM PGP. I have a strong distaste
for the Rust community ecosystem these days, and I don't love the idea
of having to have LLVM in the core bootstrap chain (hopefully gcc-rs
will be in place soon enough!). But we literally don't have any other
options. GnuPG/GPGME is out of the question for the reasons Panu and I
stated, and NeoPG died several years ago. There are no C/C++ libraries
for OpenPGP verification.


There's RNP (in C++, used by Thunderbird at least), but alas that 
doesn't expose the digest-in-progress either. So at least in it's 
current form, it's not an option for rpm. Also, the API appears to have 
all manner of quirks and gotchas that aren't welcome in a 
security-critical piece.


As for bootstrap, there will always (have to) be a way to build rpm 
without depending on Rust. Even if that meant no signature verification 
support in such a configuration.




If there was *any other choice*, we would have taken it. Even
splitting out RPM's internal OpenPGP code into its own project for
someone to rework and upgrade would be an option if someone actually
wanted to. The reality is that nobody wants to work on that.


Yep. In case there's any doubt, I too would've much, MUCH, VERY MUCH 
preferred a C (or even C++) library but in the ~15 years I've been on 
the lookout, no viable and credible candidates have appeared.
Rpm will remain open to alternative backends, should such a thing 
emerge, but I'm not holding my breath.


- Panu -




So here we are, in a subpar situation created by bad tools because
nobody cares enough about security anyway.








--
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 proposal: Replace DNF with DNF5 (System-Wide Change proposal)

2022-10-13 Thread Panu Matilainen


On 10/12/22 17:47, Stephen Smoogen wrote:



On Wed, 12 Oct 2022 at 10:32, Kevin P. Fleming > wrote:


On 10/12/22 08:59, Stephen Smoogen wrote:
 > Maybe call it the Fedora Update Manager 'FUM' ?

Unless we're going to call it RUM when it makes its way into RHEL, that
name may not be the best choice :-)


Well Red Hat shipped the Yellowdog Update Manager for 2 releases so I am 
sure they can go with FUM or RUM (RPM Update Manager).. 


RPM Update Manager, an easily pronouncable and distro agnostic acronym 
which even makes sense. Now that would be a hard sell...


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F38 proposal: RPM Sequoia (System-Wide Change proposal)

2022-10-13 Thread Panu Matilainen


On 10/13/22 07:18, Kevin Kofler via devel wrote:

For the last 20 years or so, RPM has used a home-grown OpenPGP parser
for dealing with keys and signatures. That parser is rather infamous
for its limitations and flaws, and especially in recent years has
proven a significant burden to RPM development. In order to improve
security and free developer resources for dealing with RPM's "core
business" instead, RPM upstream is in the process of deprecating the
internal parser in favor of [https://sequoia-pgp.org/ Sequoia PGP]
based solution written in Rust.


Why are you using a new library written in Rust? Can you not use one of the
existing mature C implementations of OpenPGP? gpgme maybe?


Had there been such an option, we would've switched over years ago. 
Gpgme is based around calling and communicating with an external gpg 
process, which is a setup you do NOT want in the rpm context where 
chroots come and go etc. Also, rpm requires access to the "low-level" 
digests-in-progress because it calculates multiple things on a single read.



At this point the change is mostly invisible in normal daily use.


Not really, because it makes some packages uninstallable.


Not uninstallable. You can use --nosignature, or resign such packages 
with something stronger. Which one is the better course depends on the 
situation of course.



- Some old, insecure (MD5/SHA1 based) signatures are rejected (this is
in line with the stronger crypto settings proposed elsewhere for F38)


Such a hardcoded restriction, without a way for the local administrator to
allow the legacy signatures, is not acceptable.


Mind you, I don't exactly agree with this style of explicit disabling 
either (see 
https://lists.rpm.org/pipermail/rpm-maint/2021-October/018344.html and 
onwards). But. I doubt many people realize just how thin the ice is (and 
has always been) with the existing parser. I consider this step a matter 
of survival, and ultimately some legacy content becoming harder to use 
is an acceptable tradeoff for *that*.


I don't know how deep this all is wired inside Sequoia, but I totally 
agree (as you see in the thread linked above) that this should be based 
on the system crypto policy. As explained in the change, nettle (which 
doesn't support the system crypto policies AIUI) should be seen as a 
temporary stepstone in Fedora, with a plan to switch to openssl (which 
does) in the nearish future.


So technically this is a matter of "Sequoia should honor system crypto 
policy", rpm is just a dumb API user here that sometimes get told "nope" 
by the underlying libraries, whether due to crypto policy, FIPS or whatever.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: DNF5 Blockers

2022-10-11 Thread Panu Matilainen

On 10/11/22 15:17, Stephen Smoogen wrote:

On Tue, 11 Oct 2022 at 07:54, Panu Matilainen <mailto:pmati...@redhat.com>> wrote:

On 10/10/22 10:15, Jaroslav Mracek wrote:
 > Please can you be more specific which kind of functionality is
required for particular command? Why is it important to know what
user case you want to resolve it? Commands has multiple options and
some of them could be unused. Specially repoquery has tons of
options. Knowing critical usercase will help us a lot to not only
provide the same option but to improve DNF5 behavior in comparison
to DNF4.
 >
 > I recommend to create for each user case or command an issue -
https://github.com/rpm-software-management/dnf5/issues
<https://github.com/rpm-software-management/dnf5/issues>. Please
provide as much as possible information to understand the user case
to be able to set a proper priority.

The mission of repoquery is basically to provide scriptable access to
all available repository metadata. That necessarily involves helluva
many switches, added over time to address all manner of use-cases.
Don't
assume anything is there just for giggles.

Maybe it makes sense to put it just as a dedicated command like I 
thought it was until 'recently (*cough* sorry N years ago *cough*)'.

Yup. There were more than one reason for it to be a dedicated utility 
originally.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: DNF5 Blockers

2022-10-11 Thread Panu Matilainen


On 10/10/22 10:15, Jaroslav Mracek wrote:

Please can you be more specific which kind of functionality is required for 
particular command? Why is it important to know what user case you want to 
resolve it? Commands has multiple options and some of them could be unused. 
Specially repoquery has tons of options. Knowing critical usercase will help us 
a lot to not only provide the same option but to improve DNF5 behavior in 
comparison to DNF4.

I recommend to create for each user case or command an issue - 
https://github.com/rpm-software-management/dnf5/issues. Please provide as much 
as possible information to understand the user case to be able to set a proper 
priority.


The mission of repoquery is basically to provide scriptable access to 
all available repository metadata. That necessarily involves helluva 
many switches, added over time to address all manner of use-cases. Don't 
assume anything is there just for giggles.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: When is a file dependency appropriate?

2022-10-06 Thread Panu Matilainen

On 10/6/22 11:55, Vít Ondruch wrote:

Dne 06. 10. 22 v 9:18 Otto Liljalaakso napsal(a):

Miro Hrončok kirjoitti 6.10.2022 klo 2.33:

On 06. 10. 22 1:21, Otto Liljalaakso wrote:
Recently, I have run into some cases where file dependencies like
Requires: /usr/bin/foo are used.

In a recent thread on this mailing list [1],
it is mentioned that such Requires should be avoided,
because resolving file dependencies requires a large amount of memory.

I don't believe that resolving file-Requires from directories listed
at [2] from your email is more memory hungry. Where exactly was that
said?
[1]:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/CRREDQUPPJYWVRMA4DOKYU2KZZLKC4D5/
[2]:
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_file_and_directory_dependencies

Maybe I should have re-read the thread closely before posting,
because the most relevant message is this [3]:

"
The discussion in the bug indicates that this memory growth is related to
loading of the full filepath dataset. We have been discussing splitting
out the non-primary-filepath-data (i.e. paths that are not /etc,
/usr/bin,

/usr/sbin), out into a separate lazilly-loaded file. If we manage to do
that, we'll kill two birds with one stone:

- initial download of repo metadata on every freakin' dnf operation can
go down from 80 to 20 MB
- the peak memory use will go down
"

Which exactly confirms what you say.

I am still wondering a bit if there is something else bad about file
dependencies?

If I am not mistaken, in the YUM days, the file list was split into two
parts. One list contained several high profile directories such as
/usr/bin [1] while the other contained everything. By default YUM have
not loaded the full file list by default, just the short one. The data
were not loaded by default. But AFAIK, DNF always loads everything and
it does not look that this would change. This is one BZ which comes to
my mind for a reference: https://bugzilla.redhat.com/show_bug.cgi?id=968006

IOW for your example, the file dependency was always supported (and IMO
preferable). It probably does not matter these days, unless you really
consider memory consumption (and I don't think we should generally avoid
the file dependencies just on the base of memory consumption).

In the yum days, certain locations of the file list, such as /usr/bin
were embedded in the main data. They're still there, look for
entries in primary.xml.

I don't know if libsolv/dnf look at that data at all, the architecture
is so drastically different compared to yum that the seemingly obvious
lazy loading may well be very very difficult to achieve. I remember that
being the case with apt-rpm (if anybody remembers *that* old beast)
whose operational model was closer to libsolv than yum.

- Panu -

Re: Explicit dependency on systemd-rpm-macros now required?

2022-09-16 Thread Panu Matilainen


On 9/16/22 15:22, Zbigniew Jędrzejewski-Szmek wrote:

On Wed, Sep 14, 2022 at 11:02:22AM -0400, Neal Gompa wrote:

On Wed, Sep 14, 2022 at 10:53 AM Smith, Stewart via devel
 wrote:




On Sep 14, 2022, at 4:17 AM, Tom Hughes via devel 
 wrote:


On 14/09/2022 12:11, Florian Weimer wrote:
I see some new build failures in rawhide related to systemd RPM macros:

Processing files: opencryptoki-3.18.0-4.fc38.s390x
error: File must begin with "/": %{_tmpfilesdir}/opencryptoki.conf
error: File must begin with "/": %{_unitdir}/pkcsslotd.service
[…]
RPM build errors:
 File must begin with "/": %{_tmpfilesdir}/opencryptoki.conf
 File must begin with "/": %{_unitdir}/pkcsslotd.service
Child return code was: 1
EXCEPTION: [Error()]

Is this a package problem (missing dependency on systemd-rpm-macros), or
is this something that should be fixed at the buildroot level?


Guidelines say yes, you do need a BR on that:

https://docs.fedoraproject.org/en-US/packaging-guidelines/Systemd/#packaging


I think there was some change “recently” where it needed to start being 
explicit rather than being brought in by some other dependency (possibly a 
change to systemd?). I hit the same thing in a package in Amazon Linux the 
other day, read the packaging guide and wondered how the package had ever built.


It happened because Zbigniew changed the rich dependency from Requires
to Requires(meta):
https://src.fedoraproject.org/rpms/systemd/c/c971c5b980dff46fb9d7885f9e26b179a5a4749b

I don't think Requires(meta) works when weak dependencies are turned off.


Hmm, but that would be a bug in rpm (or whatever figures out the
dependencies in this case). There is no documentation for the
feature, except for the release notes for rpm 4.16.0:

   Add support for meta dependencies (eg Requires(meta): somepkg) that
   do not affect install/erase ordering (RhBug:1648721)

The addition of "(meta)" should only affect ordering, and not the "strength"
of the dependency.


Yes, meta is NOT a weak dependency at all, if something is treating it 
as such then it's certainly a bug.


- Panu -



Zbyszek


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F38 proposal: Strong crypto settings: phase 3, forewarning 2/2 (System-Wide Change proposal)

2022-09-15 Thread Panu Matilainen


On 9/15/22 00:59, Kevin Kofler via devel wrote:

Alexander Sosedkin wrote:

That's a reason why my initial thread [1] has been named
"Landing a larger-than-release change (distrusting SHA-1 signatures)":
flipping the switch is the easy part, unfortunately.


IMHO, a change that breaks so many things that you expect it to take more
than 6 months to fix the breakage across the entire distribution is just
unacceptable to begin with and should just not be done altogether, ever. At
least not as long as it is expected to break so many things. Maybe in 10 or
20 years, you can even consider dropping SHA-1. The real world does not move
as fast as the progress in cryptanalysis, you just have to accept that.

Maybe it can work to distrust SHA-1 in some particularly security-critical
contexts, e.g., make RPM distrust SHA-1 signatures for packages installed on
the system (but not, e.g., in a mock chroot targeting some older RHEL!) by
default, with an easy way to change that default (I am thinking of something
like "echo 'trust_sha1_sigs 1' >/etc/rpm/macros.trustsha1"). But disallowing
SHA-1 systemwide, with no regards to what the actual application is and what
level of security it provides, is just insane, and will just lead to
applications bundling their own SHA-1 implementation and possibly even their
own PGP signature implementation to work around your deliberate breakage.


Please read the actual proposal: this is about SHA-1 *signatures*. Not 
the hash itself.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: rpm with sequoia pgp

2022-09-07 Thread Panu Matilainen

On 9/6/22 23:10, Simo Sorce wrote:

On Tue, 2022-09-06 at 11:09 +0300, Panu Matilainen wrote:

On 9/2/22 17:31, Neal H. Walfield wrote:

Hi all,

rpm 4.18 is on the horizon and includes a new OpenPGP backend based on
Sequoia PGP.

https://rpm.org/wiki/Releases/4.18.0
https://sequoia-pgp.org/

Thanks to Fabio Valentini (decathorpe) for packaging not only
rpm-sequoia, but all of the Sequoia packages for Fedora.

https://copr.fedorainfracloud.org/coprs/decathorpe/sequoia-test-builds/package/rust-rpm-sequoia/

With this note, I'd firstly like to make the Fedora community more
aware of this project. (I don't think it has been mentioned here
yet.)

Second, although the internal OpenPGP backend is still the default
backend, it will be removed in rpm 4.19:

https://github.com/rpm-software-management/rpm/issues/1935

While that was the initial goal, I suspect we may have to stretch this a
bit. I think we'll first need a release where the upstream default is
something else, and then in the next release we can actually look at
axing it.

It is probably best to start the transition as soon as possible to
work out any kinks.

In that vein, I'd like to offer my help. Making this type of change
needs to be done carefully. Perhaps these are questions or concerns.
I'd like to hear them and respond to them. There is also technical
work that needs to be done. I'm more of a developer than a packager,
but if Fedora decides to use the Sequoia backend, I'd like to offer my
help in any way I can.

Since rpm 4.18 gained the Sequoia support afterall, we can and should
look into swapping over in Fedora 38. That'll help sorting out any rough
edges and make it easier to eventually swap the default in upstream as
well. We probably need to do this with a change process as anything
rpm-related tends to be system/distro wide in a sense (see below)

Once the dust from 4.18 has settled (final is expected in a couple of
weeks) we can start digging into this, although nothing prevents
starting with other "paperwork" etc.

Note: Sequoia currently uses Nettle on Fedora, but there is ongoing
work to port it to Sequoia to OpenSSL:

https://github.com/rpm-software-management/rpm/issues/2041#issuecomment-1219175000

This may well be a blocker on Fedora level, in part to keep container
etc images small but also for distro crypto policies and FIPS (neither
of which nettle supports AIUI).

With my Crypto Team hat on I do not see this as a blocker for Fedora in
the short term, we can start with nettle and then move to OpenSSL
later.

For RHEL we may prefer OpenSSL for various reasons above, but I would
note that although we do not certify nettle directly under FIPS we do
indirectly as part of GnuTLS, so it is certainly tested cryptography.

In fact nettle might be a slightly better choice in some cases for
container images because it is much smaller than OpenSSL.

Yup. My assumption was that since openssl will be pulled in "anyway",
anything else will be seen as unwanted size increase. But, I hadn't
realized nettle is *that* much smaller.

Finally nettle could even be statically built into sequoia (together
with gmp) if we need even smaller footprint or we are concerned about
potential rpm breakage during upgrades.
I am not saying we want to do this, but it is an option that OpenSSL 3
won't provide as easily.

Ack. Highly useful info this all, thanks!

- Panu -

Simo.

Re: rpm with sequoia pgp

2022-09-06 Thread Panu Matilainen

On 9/2/22 17:31, Neal H. Walfield wrote:

Hi all,

rpm 4.18 is on the horizon and includes a new OpenPGP backend based on
Sequoia PGP.

https://rpm.org/wiki/Releases/4.18.0
https://sequoia-pgp.org/

Thanks to Fabio Valentini (decathorpe) for packaging not only
rpm-sequoia, but all of the Sequoia packages for Fedora.

https://copr.fedorainfracloud.org/coprs/decathorpe/sequoia-test-builds/package/rust-rpm-sequoia/

With this note, I'd firstly like to make the Fedora community more
aware of this project. (I don't think it has been mentioned here
yet.)

Second, although the internal OpenPGP backend is still the default
backend, it will be removed in rpm 4.19:

https://github.com/rpm-software-management/rpm/issues/1935

It is probably best to start the transition as soon as possible to
work out any kinks.

Once the dust from 4.18 has settled (final is expected in a couple of
weeks) we can start digging into this, although nothing prevents
starting with other "paperwork" etc.

Note: Sequoia currently uses Nettle on Fedora, but there is ongoing
work to port it to Sequoia to OpenSSL:

https://github.com/rpm-software-management/rpm/issues/2041#issuecomment-1219175000

This may well be a blocker on Fedora level, in part to keep container
etc images small but also for distro crypto policies and FIPS (neither
of which nettle supports AIUI).

Re: fedora-review of m3u8 fails

2022-08-09 Thread Panu Matilainen


On 8/9/22 15:37, Martin Gansser wrote:

Hi,

fedora-review of m3u8 fails with the following error message:

Building target platforms: x86_64
Building for target x86_64
setting SOURCE_DATE_EPOCH=1659916800
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.g2rRRB
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ rm -rf m3u8-3.1.0
+ /usr/lib/rpm/rpmuncompress -x -v/builddir/build/SOURCES/m3u8-3.1.0.tar.gz
rpmuncompress: -v/builddir/build/SOURCES/m3u8-3.1.0.tar.gz: unknown option


That's a bug in rpm 4.18 alpha [1], just update your rpm to 4.18 beta 
which is in rawhide since late June.


[1] https://bugzilla.redhat.com/show_bug.cgi?id=2079127

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F37 Change Proposal: Firefox Langpacks Subpackage (System-Wide Change)

2022-06-30 Thread Panu Matilainen


On 6/29/22 21:26, Vitaly Zaitsev via devel wrote:

On 29/06/2022 19:00, Vipul Siddharth wrote:

Firefox langpacks, which have been bundled in the Fedora firefox base
package until now, will be moved to a firefox-langpacks subpackage.


+1. It might be better to split it even more: firefox-langpack-%{lang} 
and depend on the system-wide language (just like spelling dictionaries).


Users will be able to install only the required locales.



Indeed.

There's some variance in how packages handle this but AIUI the preferred 
scheme is per-language subpackages named -langpack-.


For an example: 
https://src.fedoraproject.org/rpms/tesseract-tessdata/blob/rawhide/f/tesseract-tessdata.spec#_55


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Archive value is out of time_t range

2022-06-06 Thread Panu Matilainen


On 6/6/22 13:29, Petr Pisar wrote:

V Mon, Jun 06, 2022 at 12:07:18PM +0200, Miroslav Lichvar napsal(a):

On Thu, Jun 02, 2022 at 05:46:05PM +0200, Petr Pisar wrote:

$ gcc -m32 -D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64 main.c
$ ./a.out
sizeof(time_t)=8

I recommend you to file a bug against tar in Fedora's Bugzilla. However, this
proposed solution would require rebuilding in the same way all libraries which
tar uses and which pass time_t and similar types in their interface. That
would probably break other packages.


Now that the kernel and glibc have this feature, would it make sense
to change the global CFLAGS to build everything with 64-bit time_t on
all currently supported 32-bit archs? If not now, how close to the
32-bit overflow in 2038?


That would be ideal, but I worry that Fedora will rather drop i686 archicture
than change ABI of all packages.

Most of the uses cases for i686 is a multilib for proprietary software.
Changing ABI would break it. How much of that software will be relevant in 15
years?


The way glibc compatibility works, existing binaries will continue to 
get the ABIs they were built with once upon a time and continue to work 
like nothing ever happened. Except of course that time will wrap around 
eventually.


Or at least that's how the theory goes, IIRC the time_t transition has 
its own peculiarities that other similar transitions haven't had.




We dropped 32-bit ARM because we were unable to build the software (in
a reasonable time). As software becomes hungrier (and less tested on 32 bits), 
we
start observing the same problem in i686.


Yup, but i686 can be easily built on x86_64 which alleviates things 
quite a bit.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 proposal: RPM Macros for Build Flags (System-Wide Change proposal)

2022-06-06 Thread Panu Matilainen


On 6/3/22 13:43, Fabio Valentini wrote:

On Fri, Jun 3, 2022 at 11:25 AM Vít Ondruch  wrote:


BTW isn't the `_flag_` prefix too generic? And also, the initial
underscore implies that this is internal macro which should ideally not
be used. So should it be rather removed or not?


I agree that the "_flag_" prefix might be a little too generic, but
what would be a better alternative?
Maybe something like _optflag_, to match what they are "collected
into" (i.e. %optflags)?

Also, macro names with single leading underscores are *fine* (see also
%_bindir, %_libdir, %_datadir, etc.).
Those with *double* leading underscores are the ones that should be
considered "internal" implementation details.


Once upon a time in past I can still remember, the following rule of 
thumb for macro underscores was set [1]:


Use %__foo to set, %foo to get.

To me it looks like the entire set of suggested flags is basically 
write-only values, and thus should have two leading underscores. So, 
%__build_flag_whatever. Usage should always happen through the 
non-underscored %build_cflags and friends, which can do their own 
internal logic around this stuff, use Y only if X is enabled.


Other misc comments:


%_flag_flto_auto   -flto=auto


Shouldn't this be %_flag_flto instead (or rather, %__build_flag_flto), 
just like %_flag_o does not carry the optimization level in the name?



 %_flag_werror_format_security  -Werror=format-security


...and ditto for this, %__build_flag_werror whose default value is 
-Werror=format-security ... except that unlike -flto, -Werror can appear 
multiple times. Dunno.


What I guess I'm after, having an actual rule for the parameter -> macro 
naming, one that could preferably be automated, would be beneficial. The 
-Wl and -Wp related macro naming would need further consideration wrt that.


%_flag_pipe seems like the odd man out there because it doesn't actually 
relate to code at all, but I guess consistency is the goal there.


[1] https://pagure.io/packaging-committee/issue/907

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: How much free space in /var is required for upgrades?

2022-05-23 Thread Panu Matilainen


On 5/16/22 13:39, Panu Matilainen wrote:

On 5/13/22 21:54, Jason L Tibbitts III wrote:

So I went to do a dnf system-upgrade from F35 to F36 on a test machine,
as part of my usual testing.  In the middle of the process, it appears
that /var filled up and that left the system in an unfortunate state.
Surprisingly (to me) it did boot with a random mix of F35 and F36
packages and even though it's a throwaway test box, I wanted to play
around with fixing it a bit and trying to understand why it ran out of
space instead of just reinstalling.

Turns out that "dnf --releasever 36 --nogpgcheck remove --duplicates"
was able to effectively everything in the system, and while running this
/var filled up again.  When that happened, dnf couldn't even be aborted;
I had to kill -9.  The culprit is the write-ahead log,
/var/lib/rpm/rpmdb.sqlite-wal.  I resized /var and reran, and by the end
of the process had grown to over 9GB:

-rw-r--r--. 1 root root 9124576392 May 13 13:11 rpmdb.sqlite-wal

Of course it immediately went to 0 once the transaction completed,
though rpmdb.sqlite went from:

-rw-r--r--. 1 root root 281739264 May 11 14:24 rpmdb.sqlite

to

-rw-r--r--. 1 root root 730648576 May 13 13:15 rpmdb.sqlite

which seems... odd for what's effectively just reinstalling the existing
package set.

Anyway, obviously the solution is to make sure that /var is "big enough"
before you do a system upgrade.  And we do have warnings about
filesystems being too small, but nothing about needing an extra 10GB for
this.  Certainly my case might be somewhat pathological and it was good
that in the end I was able to get the system back into a useful state
without wiping it.  But in the end I wonder:

1) Is it really expected that the wal file will grow to that size?


No.


2) Is there anything to be done to reduce the size of the log?


Yeah, such as reporting incidents like this.


FWIW, this should be fixed in upstream now and rawhide as of 
rpm-4.18.0-0.alpha2.1 build.


For the future: If you suspect a bug in rpm, just file one. Preferably 
upstream. Messages to fedora-devel are easily lost in the noise and may 
not be seen at all by us rpm maintainers.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: How much free space in /var is required for upgrades?

2022-05-16 Thread Panu Matilainen


On 5/16/22 15:06, Nico Kadel-Garcia wrote:

On Mon, May 16, 2022 at 6:39 AM Panu Matilainen  wrote:


On 5/13/22 21:54, Jason L Tibbitts III wrote:

So I went to do a dnf system-upgrade from F35 to F36 on a test machine,
as part of my usual testing.  In the middle of the process, it appears
that /var filled up and that left the system in an unfortunate state.
Surprisingly (to me) it did boot with a random mix of F35 and F36
packages and even though it's a throwaway test box, I wanted to play
around with fixing it a bit and trying to understand why it ran out of
space instead of just reinstalling.

Turns out that "dnf --releasever 36 --nogpgcheck remove --duplicates"
was able to effectively everything in the system, and while running this
/var filled up again.  When that happened, dnf couldn't even be aborted;
I had to kill -9.  The culprit is the write-ahead log,
/var/lib/rpm/rpmdb.sqlite-wal.  I resized /var and reran, and by the end
of the process had grown to over 9GB:

-rw-r--r--. 1 root root 9124576392 May 13 13:11 rpmdb.sqlite-wal

Of course it immediately went to 0 once the transaction completed,
though rpmdb.sqlite went from:

-rw-r--r--. 1 root root 281739264 May 11 14:24 rpmdb.sqlite

to

-rw-r--r--. 1 root root 730648576 May 13 13:15 rpmdb.sqlite

which seems... odd for what's effectively just reinstalling the existing
package set.

Anyway, obviously the solution is to make sure that /var is "big enough"
before you do a system upgrade.  And we do have warnings about
filesystems being too small, but nothing about needing an extra 10GB for
this.  Certainly my case might be somewhat pathological and it was good
that in the end I was able to get the system back into a useful state
without wiping it.  But in the end I wonder:

1) Is it really expected that the wal file will grow to that size?


No.


2) Is there anything to be done to reduce the size of the log?


Yeah, such as reporting incidents like this.


3) Is there any better way to handle a lack of space in /var during an
RPM transaction?

4) Can we estimate how large the file will grow, and refuse to start a
system upgrade if there is not enough space?  Certainly we already do
this to some degree, but it seems that the estimate of the required
space is a bit too small.


Rpm has had a heuristic on the rpmdb growth for years, but no heuristics
can help against unexpected events eating the space.


An in-place system upgrade is not an "unexpected event". It is a risky
transaction.


I never said an upgrade is unexpected, that'd be absurd. It's a 
long-running (and indeed risky) process that is only transactional from 
the point that it will not start if pre-flight check does not pass.



The big space pig is not /var/lib/rpm: it's /var/cache/dnf, which can
be quite flooded by updated packages  tool suites such as openoffice
or tetex. Another of my favorites for such in-place upgrades is to
take a package list before hand and delete such bulky suites, to
re-install them after the upgrade is complete.


That stuff is already downloaded when the space calculations happen.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: How much free space in /var is required for upgrades?

2022-05-16 Thread Panu Matilainen


On 5/13/22 21:54, Jason L Tibbitts III wrote:

So I went to do a dnf system-upgrade from F35 to F36 on a test machine,
as part of my usual testing.  In the middle of the process, it appears
that /var filled up and that left the system in an unfortunate state.
Surprisingly (to me) it did boot with a random mix of F35 and F36
packages and even though it's a throwaway test box, I wanted to play
around with fixing it a bit and trying to understand why it ran out of
space instead of just reinstalling.

Turns out that "dnf --releasever 36 --nogpgcheck remove --duplicates"
was able to effectively everything in the system, and while running this
/var filled up again.  When that happened, dnf couldn't even be aborted;
I had to kill -9.  The culprit is the write-ahead log,
/var/lib/rpm/rpmdb.sqlite-wal.  I resized /var and reran, and by the end
of the process had grown to over 9GB:

-rw-r--r--. 1 root root 9124576392 May 13 13:11 rpmdb.sqlite-wal

Of course it immediately went to 0 once the transaction completed,
though rpmdb.sqlite went from:

-rw-r--r--. 1 root root 281739264 May 11 14:24 rpmdb.sqlite

to

-rw-r--r--. 1 root root 730648576 May 13 13:15 rpmdb.sqlite

which seems... odd for what's effectively just reinstalling the existing
package set.

Anyway, obviously the solution is to make sure that /var is "big enough"
before you do a system upgrade.  And we do have warnings about
filesystems being too small, but nothing about needing an extra 10GB for
this.  Certainly my case might be somewhat pathological and it was good
that in the end I was able to get the system back into a useful state
without wiping it.  But in the end I wonder:

1) Is it really expected that the wal file will grow to that size?


No.


2) Is there anything to be done to reduce the size of the log?


Yeah, such as reporting incidents like this.


3) Is there any better way to handle a lack of space in /var during an
RPM transaction?

4) Can we estimate how large the file will grow, and refuse to start a
system upgrade if there is not enough space?  Certainly we already do
this to some degree, but it seems that the estimate of the required
space is a bit too small.


Rpm has had a heuristic on the rpmdb growth for years, but no heuristics 
can help against unexpected events eating the space.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: RPM 4.18 (System-Wide Change proposal)

2022-04-28 Thread Panu Matilainen


On 4/27/22 19:09, Miro Hrončok wrote:

On 27. 04. 22 17:53, Stephen Gallagher wrote:

On Wed, Apr 27, 2022 at 5:08 AM Miro Hrončok  wrote:


On 27. 04. 22 10:36, Joe Orton wrote:

On Thu, Apr 07, 2022 at 12:47:25PM -0400, Stephen Gallagher wrote:

On Thu, Apr 7, 2022 at 12:24 PM Ben Cotton  wrote:

== Detailed Description ==

RPM 4.18 contains various improvements over previous versions, but in
particular this release addresses a whole class of symlink handling
related security issues, some with CVE's, from 2021. Other notable
improvements include
* A more intuitive conditional builds macro `%bcond`


I looked this up[1] because it caught my attention. This is an
extremely welcome change and I would like to shower praise upon
everyone who worked on it.


Big +1 from me too, this is so good to see.  Thanks Panu & all.


I like this so much I've opened Fedora 36 and 35 backports:

https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/182
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/183

Will also try to see if this is technically possible in c9s.


It should be possible to put it in epel-rpm-macros at the worst, no?
(Related: can we get a backport to EPEL 8 and EPEL 9?)


What I've meant by "technically possible": Does the RPM version in EL 9 
support the syntax used in the macro?


If it does, it can go to EPEL 9 and even to c9s eventually, if accepted. 
If it doesn't, we might need to use a different syntax which would 
require more work. This also applies to EL 8, but I suspect even more 
problems there.




The %bcond macro relies on a %[] expression which requires rpm >= 4.16, 
so EL9 should be okay, older ones wont work.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: RPM 4.18 (System-Wide Change proposal)

2022-04-27 Thread Panu Matilainen


On 4/26/22 12:53, Fabio Valentini wrote:

On Tue, Apr 26, 2022 at 11:50 AM Panu Matilainen  wrote:


On 4/7/22 19:13, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/RPM-4.18

== Summary ==
Update RPM to the [https://rpm.org/wiki/Releases/4.18.0 4.18] release.



FWIW, this is in rawhide now. Submitted yesterday already but some
bodhi/koji delay caused it to only go live today AFAICS.

As per the usual drill, if you so much as suspect a bug/regression in
rpm, don't dwell on it, just go ahead and file it. A few possible false
positives early on is a cheap price to pay compared to having to do
rebuilds en masse later.


I would have appreciated it if you at least waited with pushing it to
rawhide until the FESCo ticket was approved...


Indeed. I do know how the process goes, been through it many, many 
times, and not sure what made me short-circuit it this time around. 
Perhaps it was the distressed state of mind from our dog getting hit by 
a car on Monday morning.


Apologies for that, and having seen the fesco meeting log - indeed it 
would not hurt to say wait for explicit APPROVED stamp, no matter how 
obvious it seems.


Oh, and the dog appears okay. Seems we were very, very lucky there.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: RPM 4.18 (System-Wide Change proposal)

2022-04-26 Thread Panu Matilainen


On 4/7/22 19:13, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/RPM-4.18

== Summary ==
Update RPM to the [https://rpm.org/wiki/Releases/4.18.0 4.18] release.



FWIW, this is in rawhide now. Submitted yesterday already but some 
bodhi/koji delay caused it to only go live today AFAICS.


As per the usual drill, if you so much as suspect a bug/regression in 
rpm, don't dwell on it, just go ahead and file it. A few possible false 
positives early on is a cheap price to pay compared to having to do 
rebuilds en masse later.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: verifying signature for a package

2022-04-20 Thread Panu Matilainen


On 4/17/22 13:26, Maxwell G via devel wrote:


Apr 16, 2022 8:01:27 PM Globe Trotter via devel :


Source1:    %{source0}.sig

Does this still fail if you use the full path? It looks like `%{source0}` isn't 
getting expanded properly.


Macros are case-sensitive. It's %{SOURCE0}.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: RPM 4.18 (System-Wide Change proposal)

2022-04-12 Thread Panu Matilainen


On 4/8/22 14:17, Neal Gompa wrote:

On Fri, Apr 8, 2022 at 6:34 AM Panu Matilainen  wrote:


On 4/8/22 12:16, Petr Pisar wrote:

V Thu, Apr 07, 2022 at 12:13:42PM -0400, Ben Cotton napsal(a):

https://fedoraproject.org/wiki/Changes/RPM-4.18

== Summary ==
Update RPM to the [https://rpm.org/wiki/Releases/4.18.0 4.18] release.

[...]

* New `%conf` spec section for build configuration


RPM documenation reads:

   In %conf, the unpacked sources are configured for building.

   Different build- and language ecosystems come with their own helper 
macros,
   but rpm has helpers for autotools based builds such as itself which
   typically look like this:

   %conf
   %configure

In context of autotools, sources usually bundle a configure script. To follow
the open source way (and ensure portability to new platform and include
autotools fixes), building from the real sources is desired. Hence I do my
best to call "autoreconf -fi" before %configure.

Where should autoreconf be placed? %pre or %conf?

  %prep   %prep
  %autosetup  %autosetup
  autoreconf -fi

  %conf   %conf
  autoreconf -fi  %configure
  %configure

  %build  %build
  %make_build %make_build

Please bear in mind that %prep usually contains other non-declarative twists
like pruning bundled code, correcting file permissions etc.


My personal opinion is that autoreconf, where used, belongs to %prep
because it can quite literally install bits required by the build system.

It's also something you only run once after unpacking the sources, not
every time you configure. Which suggests that they should be in
different sections. The same logic is applicable to other related steps.



I would disagree and suggest it go into %conf. My reasoning for this
is that autoreconf + configure is combined for most other build
systems and we want those to run in %conf, so consistency beckons that
we do the same here. It also ensures we don't need anything more than
tarball unpacking and patching dependencies for %prep.


I used "personal opinion" very much on purpose, as there would be as 
many opinions on this as there are observers. My opinion is biased 
towards what makes sense from a staged rpmbuild POV, for a --rebuild 
end result all of this is academic.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: RPM 4.18 (System-Wide Change proposal)

2022-04-08 Thread Panu Matilainen


On 4/8/22 12:16, Petr Pisar wrote:

V Thu, Apr 07, 2022 at 12:13:42PM -0400, Ben Cotton napsal(a):

https://fedoraproject.org/wiki/Changes/RPM-4.18

== Summary ==
Update RPM to the [https://rpm.org/wiki/Releases/4.18.0 4.18] release.

[...]

* New `%conf` spec section for build configuration


RPM documenation reads:

In %conf, the unpacked sources are configured for building.

Different build- and language ecosystems come with their own helper 
macros,
but rpm has helpers for autotools based builds such as itself which
typically look like this:

%conf
%configure

In context of autotools, sources usually bundle a configure script. To follow
the open source way (and ensure portability to new platform and include
autotools fixes), building from the real sources is desired. Hence I do my
best to call "autoreconf -fi" before %configure.

Where should autoreconf be placed? %pre or %conf?

 %prep   %prep
 %autosetup  %autosetup
 autoreconf -fi
 
 %conf   %conf

 autoreconf -fi  %configure
 %configure

 %build  %build
 %make_build %make_build

Please bear in mind that %prep usually contains other non-declarative twists
like pruning bundled code, correcting file permissions etc.


My personal opinion is that autoreconf, where used, belongs to %prep 
because it can quite literally install bits required by the build system.


It's also something you only run once after unpacking the sources, not 
every time you configure. Which suggests that they should be in 
different sections. The same logic is applicable to other related steps.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: RPM 4.18 (System-Wide Change proposal)

2022-04-08 Thread Panu Matilainen


On 4/7/22 20:14, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Apr 07, 2022 at 12:13:42PM -0400, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/RPM-4.18



* New interactive shell for working with macros (`rpmspec --shell`)
and embedded Lua (`rpmlua`)


That sounds cool. Can this be used to do standalone interpretation
of  scriptlets? I'm wondering about the issue that we have with
rpm-ostree, which only supports bash scriptlets, so we end up rewriting
 scriptlets and filetriggers to bash for the sake of rpm-ostree.


Well, it's a Lua interpeter, and so you can use it to execute Lua 
scripts from a file using the rpm embedded Lua interpreter so you'll 
have the same extensions loaded. It's not the same as getting executed 
from inside rpm transaction though because all sorts of context is 
missing, and this is not the intended use-case of the thing. That said, 
assuming the caller sets up scriptlet arguments and other state as per 
expectations, I don't see why simple scriptlets would not work. File 
triggers will not.





* New `%conf` spec section for build configuration


Any docs on this?


There's a brief blurb in 
https://rpm-software-management.github.io/rpm/manual/spec.html (not much 
to say really), for rationale see

https://github.com/rpm-software-management/rpm/pull/1932

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: RPM 4.18 (System-Wide Change proposal)

2022-04-08 Thread Panu Matilainen


On 4/7/22 19:47, Stephen Gallagher wrote:

On Thu, Apr 7, 2022 at 12:24 PM Ben Cotton  wrote:


https://fedoraproject.org/wiki/Changes/RPM-4.18

== Summary ==
Update RPM to the [https://rpm.org/wiki/Releases/4.18.0 4.18] release.

== Owner ==

* Name: [[User:pmatilai|Panu Matilainen]]
* Email: pmati...@redhat.com


== Detailed Description ==

RPM 4.18 contains various improvements over previous versions, but in
particular this release addresses a whole class of symlink handling
related security issues, some with CVE's, from 2021. Other notable
improvements include
* A more intuitive conditional builds macro `%bcond`


I looked this up[1] because it caught my attention. This is an
extremely welcome change and I would like to shower praise upon
everyone who worked on it.


One thing I noticed while perusing the new docs: the documentation for
%autosetup is incomplete. The doc reads: "Generally %autosetup accepts
the same arguments as %setup does", but the section on %setup no
longer exists. Only the unique arguments for %autosetup are listed.


I don't think our docs ever covered %setup, that would've been max-rpm 
more likely. Of course, %setup *should* be covered by our docs, and 
ditto for %patch.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 - Errors/Warnings with `dnf update`

2022-04-01 Thread Panu Matilainen


On 3/31/22 14:10, Daniel Walsh wrote:

On 3/31/22 02:35, Carmelo Sarta wrote:

Hello there!
I've never seen this error before
`error: Plugin selinux: hook fsm_file_prepare failed`
but I would try
`dnf reinstall container-selinux`
and maybe
`dnf reinstall podman`
There seems to be an error appening when people are installing 
container-selinux, that I have not pinned down.  It does not happen on 
my system.


There's a bug on this now as well: 
https://bugzilla.redhat.com/show_bug.cgi?id=2070942


Should I reassign to container-selinux?

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: FESCo wants to know what you use i686 packages for

2022-03-18 Thread Panu Matilainen


On 3/17/22 18:14, Carlos "casep" Sepulveda wrote:
On Wed, 16 Mar 2022 at 13:55, David Cantrell > wrote:



If you use i686 packages for something now, please respond to this
thread.


steam / wine


Ditto. Not very often perhaps but that's the primary case.

Occasionally compiling purposes too, ie at least the gcc-glibc bits.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: RPM question: How to get the package VR from a dependency generator

2022-02-28 Thread Panu Matilainen


On 2/28/22 16:32, Richard W.M. Jones wrote:

On Mon, Feb 28, 2022 at 04:26:02PM +0200, Panu Matilainen wrote:

On 2/28/22 16:12, Richard W.M. Jones wrote:

On Mon, Feb 28, 2022 at 01:46:38PM +, Richard W.M. Jones wrote:


I'm writing a simple provides generator.  The documentation is a bit
light on detail:

   
https://rpm-software-management.github.io/rpm/manual/dependency_generators.html

How do I get the version-release of the package currently being built?
At the moment I can only print simple provides like:

   Provides: foo

but I want to include the version of the package being built, eg:

   Provides: foo = 1.2-3.fc36

In theory it seems like the environment variables $RPM_PACKAGE_VERSION
and $RPM_PACKAGE_RELEASE should be set in the dependency generator,
but I just confirmed they are not set.


I dumped out the environment from the dependency generator and the
only RPM-specific environment variable is $RPM_BUILD_ROOT.  I could
get the version from that, but it would be a bit of a hack.


Name, epoch, version and release are available as macros in the
.attr file so you can pass what you need from there, eg to pass
version and release as arguments to the provides generator, you can
do:

%__my_provides /path/to/my/depgen %{version} %{release}

This seems to be missing in the docs, will fix.


Ah nice, I didn't realise you could add parameters there.


Yup, what's in there is entirely up to you, rpm only cares about the 
output. Those lines are macro-expanded at the time of launching the 
script and can have arbitrarily complex macros in them.  In fact you can 
nowadays generate dependencies directly from macro where it makes sense 
("parametric macro generators" in the docs)


And actually the per-package exported macros *are* documented (just 
above parametric generators), it just doesn't explain how to use them. 
Something to fix anyway :)


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: RPM question: How to get the package VR from a dependency generator

2022-02-28 Thread Panu Matilainen


On 2/28/22 16:12, Richard W.M. Jones wrote:

On Mon, Feb 28, 2022 at 01:46:38PM +, Richard W.M. Jones wrote:


I'm writing a simple provides generator.  The documentation is a bit
light on detail:

   
https://rpm-software-management.github.io/rpm/manual/dependency_generators.html

How do I get the version-release of the package currently being built?
At the moment I can only print simple provides like:

   Provides: foo

but I want to include the version of the package being built, eg:

   Provides: foo = 1.2-3.fc36

In theory it seems like the environment variables $RPM_PACKAGE_VERSION
and $RPM_PACKAGE_RELEASE should be set in the dependency generator,
but I just confirmed they are not set.


I dumped out the environment from the dependency generator and the
only RPM-specific environment variable is $RPM_BUILD_ROOT.  I could
get the version from that, but it would be a bit of a hack.


Name, epoch, version and release are available as macros in the .attr 
file so you can pass what you need from there, eg to pass version and 
release as arguments to the provides generator, you can do:


%__my_provides /path/to/my/depgen %{version} %{release}

This seems to be missing in the docs, will fix.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: Curl-minimal as default (System-Wide Change proposal)

2022-02-25 Thread Panu Matilainen


On 2/24/22 16:37, Neal Gompa wrote:

On Thu, Feb 24, 2022 at 8:58 AM Richard W.M. Jones  wrote:


On Thu, Feb 24, 2022 at 02:28:08PM +0100, Kamil Dudka wrote:

On Thursday, February 24, 2022 1:35:38 PM CET Richard W.M. Jones wrote:

Did you discuss modularising curl itself upstream?


It was added to their wish list but I do not remember anybody working on it:

 https://github.com/curl/curl/commit/8204844f


That would be a better idea.


Not necessarily.  Each approach has its pros and cons.


I'm intrigued by what you think the cons would be.  AFAICT if curl was
modular in this way already we wouldn't be discussing this proposal at all,
but a different and better one around packaging splits.



It would also avoid the usability nightmare that comes with trying to
trigger switching implementations. This is a very big hammer that
basically tells people that we're crippling curl by default for users
and it has very large network effects across the entire distribution.
It's quite one thing to use curl-minimal for containers where people
expect tools to be broken in the endless pursuit of smaller base
images, but when real people need to use real systems in complex
configurations, having a reduced functionality curl by default is just
going to lead to support nightmares and complaints about random
breakages in applications on Fedora.


+1

Defaulting to a minimal version for the main distro and then arguing 
whether the loss of functionality is acceptable seems a peculiar and 
troubling idea.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 Change: Authselect: Move State Files to /etc (Self-Contained Change proposal)

2022-01-19 Thread Panu Matilainen


On 1/20/22 06:16, Chris Murphy wrote:

Deleting /var/lib/net-snmp/snmpd.conf means making the agent look like a
new system, with new IDs, a renumbered interface table, and loss of
SNMPv3 users.


You're not going to wipe /var to do a reset casually - the idea would
be, this hardware isn't coming back to this environment. An easy
option with snapshots is you can define your own rollback point, which
includes environment specific configuration rather than the "wipe /var
and /etc" method of factory reset.


Who exactly is the alleged user of the "factory reset" feature?

Nobody in their right mind should think, or be lead to think, that 'rm 
-rf /some /dirs' is sufficient to wipe your possibly confidential data 
to a degree that you can just ship the hardware away.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: /opt [WAS: Re: New top-level dir]

2022-01-12 Thread Panu Matilainen


On 1/12/22 11:53, Zbigniew Jędrzejewski-Szmek wrote:

On Wed, Jan 12, 2022 at 11:24:49AM +0200, Panu Matilainen wrote:

On 1/12/22 11:05, Zbigniew Jędrzejewski-Szmek wrote:

On Mon, Jan 10, 2022 at 02:53:52PM -0700, Chris Murphy wrote:

Should /usr be independently portable? And is that with a version
matched /opt, or can there be mix and match revisions of /usr and
/opt?


We have three similar locations: /usr (system vendor tree),
/usr/local (admin non-packages installations), /opt (external vendor tree).
In other words, both /usr and /opt are for packages, but from different
sources. As an admin, you'd want to treat both package types the same,
and e.g. roll them back together. So having a separate tree for /opt
doesn't make much sense.

[At some point in the future] /opt should be renamed to /usr/opt and
symlinked for backwards compat.


If /usr is to be truly portable and have e.g. 'rpm query, verify,
remove, reinstall' work as expected, you need the metadata (the
database) representing its state to always come along for the ride.
Either the database is already in /usr, or you have to make sure /usr
and /state are inseparable.

If /usr and /state are inseparable, and if rpm can also describe
anything in /etc or /var or /opt, then all or part of those
directories are also inseparable from /state. And thus /usr. So I
think /state doesn't help.


Yeah, /state doesn't help with anything. It'd be just some part of the
whole system state, without a clear separation of why this particular
subset. My preference: /usr/lib/sysimage/rpm > /var/lib/rpm > /state.

To what degree do rpm and dnf intend to touch locations outside of
/usr *and care* about tracking those changes?


Traditionally, packages installed all kinds of files all over the place.
But we're slowly and painfully moving towards the model where:
1. packages are only allowed to install under /usr, /var, and /etc.
(Or under /opt, but I'd want to move that to /usr/opt…)
2. packages must support /var/cache being wiped at any time, and
most packages support anything under /var being wiped at any time.
3. systemd and other projects are trying to only use /etc for local
admin state, and support "factory reset" by wiping /etc and /var.


Oh, right. More hidden agenda behind this thing.


Panu,

I hope you realize that I'm not connected to this change in any shape
or fashion and I'm just commenting on the published proposal like the
other participants in this thread.



Absolutely. I intended to thank you for bringing that behind-the-scenes 
grand plan up but seems it got lost in the edits. So thanks for bringing 
it up!


I actually do know the "factory reset" plan from way way back but had 
completely forgotten about it, and chances are I'm not alone in that. 
And I bet that not everybody in this discussion knew about it at all.


Others have complained about the lack of detail in the "benefit to 
Fedora" section, and I can only concur. At this point it's fairly 
obvious the change is just a tiny tiny part of a MUCH bigger agenda, and 
knowing and understanding the bigger agenda is necessary to make sense 
and valuate the proposal.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: /opt [WAS: Re: New top-level dir]

2022-01-12 Thread Panu Matilainen


On 1/12/22 11:05, Zbigniew Jędrzejewski-Szmek wrote:

On Mon, Jan 10, 2022 at 02:53:52PM -0700, Chris Murphy wrote:

Should /usr be independently portable? And is that with a version
matched /opt, or can there be mix and match revisions of /usr and
/opt?


We have three similar locations: /usr (system vendor tree),
/usr/local (admin non-packages installations), /opt (external vendor tree).
In other words, both /usr and /opt are for packages, but from different
sources. As an admin, you'd want to treat both package types the same,
and e.g. roll them back together. So having a separate tree for /opt
doesn't make much sense.

[At some point in the future] /opt should be renamed to /usr/opt and
symlinked for backwards compat.


If /usr is to be truly portable and have e.g. 'rpm query, verify,
remove, reinstall' work as expected, you need the metadata (the
database) representing its state to always come along for the ride.
Either the database is already in /usr, or you have to make sure /usr
and /state are inseparable.

If /usr and /state are inseparable, and if rpm can also describe
anything in /etc or /var or /opt, then all or part of those
directories are also inseparable from /state. And thus /usr. So I
think /state doesn't help.


Yeah, /state doesn't help with anything. It'd be just some part of the
whole system state, without a clear separation of why this particular
subset. My preference: /usr/lib/sysimage/rpm > /var/lib/rpm > /state.
  

To what degree do rpm and dnf intend to touch locations outside of
/usr *and care* about tracking those changes?


Traditionally, packages installed all kinds of files all over the place.
But we're slowly and painfully moving towards the model where:
1. packages are only allowed to install under /usr, /var, and /etc.
(Or under /opt, but I'd want to move that to /usr/opt…)
2. packages must support /var/cache being wiped at any time, and
most packages support anything under /var being wiped at any time.
3. systemd and other projects are trying to only use /etc for local
admin state, and support "factory reset" by wiping /etc and /var.


Oh, right. More hidden agenda behind this thing. When looking at it with 
these glasses on, it explains quite a few things about the change 
proposal, such as completely ignoring the fact that nearly all packages 
put something in /etc.


I'm not saying these are necessary bad goalsat all, it's just that 
there's a huge disconnect between reality and the above model on which 
this change seems based on, and not a single mention about these goals 
and changes needed to get there. I mean, I totally get that you can't 
change everything at once, but if there's a plan this big behind 
something then maybe it should be brought up front, no?


- Panu -



So although rpm manages files under /var and /etc, it is not to the same
extent as /usr. In this light it makes a lot of sense to hold the
rpmdb state under /usr/ somewhere. The exact subdirectory doesn't really
matter, it's just a matter of convention, so obviously we want to follow
what opensuse and others are already using.


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: New top-level dir: /state [WAS: Re: F36 Change: Relocate RPM database to /usr (System-Wide Change] proposal)

2022-01-12 Thread Panu Matilainen


On 1/12/22 10:45, Chris Murphy wrote:

On Tue, Jan 11, 2022 at 2:00 AM Panu Matilainen  wrote:


The problem with /usr/something is that the rpmdb is not specific to
/usr contents at all, and unlike any other content in there, so putting
it there just *feels so wrong*. That's what /state or /sysimage or, as
we now have, /var supposedly solves.

I thought I'd suggested something at / level back then (in
https://lists.rpm.org/pipermail/rpm-maint/2017-October/006697.html
and/or
https://lists.rpm.org/pipermail/rpm-maint/2017-October/006699.html) but
seems like memory is failing me :) Maybe I thought it would seem too
outrageous to FHS believers to bother.

The point was though, that the rpmdb is not at all the only data of this
kind and so having a dedicated home makes sense.

For many practical purposes it's probably just rearranging the chairs,
but a separate top-level directory describing the *system* state seems
instinctively *much* more correct solution to it than stuffing it
somewhere deep inside a loosely related fs.


If rpm is constrained to /usr, then its database can properly be
somewhere in /usr

If rpm is unconstrained, with transactions touching any of /usr /var
/opt /etc, the resulting paradigm is inherently complicated, and an
additional top-level directory doesn't  help make it less complicated.


WTF is this talk about constraining? There's no such constraint in rpm, 
and never will. And Fedora packages are not constrained there either, a 
vast percentage places stuff in /etc, touches directory tree in /var and 
then there's /boot and the software collections and third party software.



Either top-level directories have their own databases, so rpm knows
what's in them even if one is rolled back but not another; or there's
one database to describe all of these top-level directories which then
need to share a single (sub)volume so they're all snapshot and rolled
back together. Either way, there's additionally a need for carve outs
for things that shouldn't be rolled back.


Erm. Packages don't live conveniently inside a single top level tree, 
they can't be split up that way. Many if not most span at least /etc and 
/usr, and /usr may be further split into separate mounts (eg /usr/local 
or /usr/share). And many things which install to /opt also place things 
into /usr for desktop integration.



We have an example of the latter. (open)SUSE's current layout. There's
a 10 line /etc/fstab, 9 of which are various Btrfs subvolumes to
achieve the necessary carve out. It's sufficiently complex that the
direction they're looking to go in is one in which rpm's only touch
/usr. There's /usr/etc for read-only systemd defaults. And local
customizations go in /etc. Much like rpm-ostree.

Projects within two rpm-based distros independently came to realize
that unconstrained rpm is difficult. Where rpm-ostree decided to do
the hard work of actually becoming aware of the reality there's
multiple system roots.


I don't understand the question. Rpm tracks and cares about all content
it knows about equally, regardless of the path. /usr is NOT special in
any way to rpm, it's just that most of *distro* content ends up in there
but a huge number of packages have content spread across /etc too.





I think rpm can't remain
static for all time. It either needs to become aware of multiple root
trees, and even mix and match top-level directories to create variable
roots. And possibly even manage these things. Or it needs to constrain
its reach to /usr and /opt. Whether /usr and /opt are tied together,
or can be mix and match with their own rpmdb's, I have no strong
opinion on.


Oh, multiple rpmdbs. It's a while since that last turned up. It gets
tossed around as a solution but to me it looks like it brings more
problems than it solves.


Given the choice, I prefer rpm only touches /usr, which includes
/usr/var and /usr/etc.

But if left unconstrained, having databases inside the directories
they describe is more reliable than treating the database as a sidecar
file that can much more easily become disconnected with one or more
top-level directories it ostensibly describes the contents of.


Here seems to be another SMALL undocumented dependency of this change: 
completing the /usrmove thing to cover the whole world including /opt, 
/etc, /var, and presumably /boot as well because packages put stuff in it.


Rpm wont care if you want to move content from /etc to /usr/etc and 
ditto for /var, /opt and /boot which is also used by packages, ie to 
complete to complete the great /usrmove thing started a decade ago. But 
that's quite a hidden agenda in this change if you ask me.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

Re: New top-level dir: /state [WAS: Re: F36 Change: Relocate RPM database to /usr (System-Wide Change] proposal)

2022-01-12 Thread Panu Matilainen


On 1/11/22 17:18, Colin Walters wrote:



On Tue, Jan 11, 2022, at 4:00 AM, Panu Matilainen wrote:


The point was though, that the rpmdb is not at all the only data of this
kind and so having a dedicated home makes sense.


You mentioned dnf/yum/PackageKit data; there's two kinds of that.   One is e.g. 
/var/cache/dnf which does *not* move.  It's just a cache.  (Now there is this whole 
other very interesting thread around "why don't we include a cache of the 
rpm-md inside e.g. ostree or container images?  And the reason why not is we don't 
want to have to respin all images every time a package not in the image changes)

dnf does have its own outside-of-rpm state database in /var/lib/dnf which is closer to 
this. The situation with that is messier.  AIUI this proposal so far is not calling for 
moving this.  Where it lives and how it's versioned has strong implications for the 
people who want to support snapshot/rollback.  But it's not relevant for rpm-ostree, 
which does not use this part of libdnf.  We maintain our own little "state 
file" - for lots more detail on this, see 
https://github.com/coreos/rpm-ostree/issues/2326
(And it's important to note that rpm-ostree's origin file has almost nothing 
*unless* one explicitly engages client-side layering/overrides)


For one, /state (or whatever toplevel directory) allows for the fact
that there are write-operations to rpmdb that do not touch any external
files while maintaining read-only /usr. Such as rpmdb --rebuilddb,


`rpmdb --rebuilddb` is basically about supporting switching from e.g. bdb to 
sqlite, right?
On the rpm-ostree side at least, the default format comes from the base image; 
there's no reason to directly support `rpmdb --rebuilddb` as something any 
normal user or admin would need.


No, it's a database maintenance/repair operation. Converting between 
formats is merely a convenient side-effect.



or rpm --import.


OK yes, this (along with rpms that install into /opt as you mentioned) are I think the 
biggest examples of "rpmdb has stuff not about /usr".

rpm --import actually encapsulates really well all the problems and benefits 
with everything we're trying to do.  I have a big related blurb here
https://github.com/rpm-software-management/libdnf/issues/43

But as I understand it, the creation of /etc/pki/rpm-gpg was at least partially motivated 
by the fact that in the traditional RPM model, the fact that GPG keys are stored in the 
rpmdb meant there was no way to update them "in band" of a default rpm 
operation.  Today Fedora ships these GPG keys as an RPM which hence contain files, and 
when you type `yum upgrade` (or rpm-ostree upgrade) you get updates to those files, the 
same as other files.   Now, as noted in the issue PackageKit (whose code was the 
precursor to libdnf, which has the code that rpm-ostree uses but AFAIK still not current 
dnf which has this logic in Python) auto-imports all of them.  I am not completely sure 
how updating the rpmdb with new e.g. Fedora GPG key works.  It might be part of 
system-upgrade or something?

And then this all relates to a higher level goal we have with "image based updates", 
which is avoiding (or minimizing as much as possible) per-system hysteresis or "unmanaged 
state", particularly opaque (hard to see/diagnose/inspect) state.

This set of trusted GPG keys stored in the rpmdb is both.  The set of keys will 
just keep growing across in-place upgrades, depending on the initial Fedora 
version installed.  And wh And this is security-relevant state, it's the set of 
trusted keys for RPM.  Now, I am sure a number of sysadmins do understand that 
the rpmdb contains GPG keys.  I'd bet a whole lot don't.  I definitely think 
that it's confusing to have both /etc/pki/rpm-gpg *and* keys stored in the 
rpmdb.  Of the two, I think the former - i.e. text files one can edit with 
standard tools - is much easier to understand and work with.  It's also already 
in a place that is designed for users to edit in `/etc`.

So by moving the rpmdb to /usr, it's basically saying that `rpm --import` 
should change.


This doesn't seem to be documented as a dependency of this move...



That said, this design could also clearly use some "systemd style" config 
model.  ostree already supports /usr/share/ostree/trusted.gpg.d which is designed for 
keys shipped by the OS vendor.  But, what's really tricky about this is we want to 
support in-place updates from previous versions (i.e. at least N-1), but hopefully not 
too old versions.   Well, this is leading to a tangent so I'll cut off this sub-thread.

The TL;DR for me is: I think everyone agrees that moving the rpmdb as it is 
today to /usr is not 100% a perfect fit.  But it's a ~90% fit - almost all the 
raw data is just headers which are clearly immutable data generated elsewhere.  
And by making this change, we're basically saying we'll fix the remaining 10% 
of cases.

Note for the

Re: New top-level dir: /state [WAS: Re: F36 Change: Relocate RPM database to /usr (System-Wide Change] proposal)

2022-01-11 Thread Panu Matilainen


On 1/10/22 23:53, Chris Murphy wrote:

On Mon, Jan 10, 2022 at 9:20 AM David Cantrell  wrote:


On Wed, Dec 29, 2021 at 10:01:57AM -0500, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/RelocateRPMToUsr

== Summary ==
Currently, the RPM databases is located in `/var`. Let's move it to
`/usr`. The move is already under way in rpm-ostree-based
installations, and in (open)SUSE.
[snip]


Moving the RPM database to /usr feels incorrect to me, but we should move it
to gain the improvements as noted in the feature proposal.


Going back to the original discussions on moving rpmdb...

Preferred is a new top-level location in /usr, .e.g /usr/sysimage/rpm.
Next is "least worst" in /usr/lib/sysimage/rpm
http://lists.rpm.org/pipermail/rpm-maint/2017-October/006764.html
http://lists.rpm.org/pipermail/rpm-maint/2017-October/006722.html

And the convergence was on /usr/lib/sysimage/rpm
http://lists.rpm.org/pipermail/rpm-maint/2017-October/006785.html

I don't see how /state solves the problem, rather than just
rearranging the chairs.


The problem with /usr/something is that the rpmdb is not specific to 
/usr contents at all, and unlike any other content in there, so putting 
it there just *feels so wrong*. That's what /state or /sysimage or, as 
we now have, /var supposedly solves.


I thought I'd suggested something at / level back then (in 
https://lists.rpm.org/pipermail/rpm-maint/2017-October/006697.html 
and/or 
https://lists.rpm.org/pipermail/rpm-maint/2017-October/006699.html) but 
seems like memory is failing me :) Maybe I thought it would seem too 
outrageous to FHS believers to bother.


The point was though, that the rpmdb is not at all the only data of this 
kind and so having a dedicated home makes sense.


For many practical purposes it's probably just rearranging the chairs, 
but a separate top-level directory describing the *system* state seems 
instinctively *much* more correct solution to it than stuffing it 
somewhere deep inside a loosely related fs.


Just FWIW, I would quit my whining about this right there if it went to 
a new toplevel directory instead because it just *feels* right unlike /usr.



Numerous followups have noted the requirement that /usr contain read-only
content, that it be shareable across hosts, and similar concepts.  While this
may or may not doable now like we could in the past, the bigger thing to me is
around the understanding of what /usr contains.  It is generally understood
that /usr contains [most of] the installed system.  What I think is a bigger
requirement or expection now is that one can tar up /usr and transport it to
another system or virtual machine or container and expect that it will
_probably_ work maybe with a bit of tinkering.  This is a really valuable
thing to have for developers.  Moving the RPM database to this tree adds a
component that is unnecessary and sort of out of place.


Should /usr be independently portable? And is that with a version
matched /opt, or can there be mix and match revisions of /usr and
/opt?

If /usr is to be truly portable and have e.g. 'rpm query, verify,
remove, reinstall' work as expected, you need the metadata (the
database) representing its state to always come along for the ride.
Either the database is already in /usr, or you have to make sure /usr
and /state are inseparable.

If /usr and /state are inseparable, and if rpm can also describe
anything in /etc or /var or /opt, then all or part of those
directories are also inseparable from /state. And thus /usr. So I
think /state doesn't help.


For one, /state (or whatever toplevel directory) allows for the fact 
that there are write-operations to rpmdb that do not touch any external 
files while maintaining read-only /usr. Such as rpmdb --rebuilddb, or 
rpm --import.


And like mentioned in the original discussion and now here, although the 
discussion is on rpmdb, it's not the only data of this kind.




To what degree do rpm and dnf intend to touch locations outside of
/usr *and care* about tracking those changes? 


I don't understand the question. Rpm tracks and cares about all content 
it knows about equally, regardless of the path. /usr is NOT special in 
any way to rpm, it's just that most of *distro* content ends up in there 
but a huge number of packages have content spread across /etc too.



I think rpm can't remain
static for all time. It either needs to become aware of multiple root
trees, and even mix and match top-level directories to create variable
roots. And possibly even manage these things. Or it needs to constrain
its reach to /usr and /opt. Whether /usr and /opt are tied together,
or can be mix and match with their own rpmdb's, I have no strong
opinion on.


Oh, multiple rpmdbs. It's a while since that last turned up. It gets 
tossed around as a solution but to me it looks like it brings more 
problems than it solves.


- Panu -






___
devel mailing list -- devel@lists.fedoraproject.org

Re: F36 Change: DIGLIM (System-Wide Change proposal)

2022-01-04 Thread Panu Matilainen


On 1/4/22 10:41, Roberto Sassu via devel wrote:

Hi everyone

in the FESCo meeting yesterday, Zbigniew asked what is
the relationship between this feature and
https://fedoraproject.org/wiki/Changes/FsVerityRPM.
I try to explain here.

Both features aim at providing reference values, i.e.
values of software fingerprint certified by the software
vendor, in order to enforce an integrity policy.

The main difference between the features is the
granularity at which the vendor certifies the software
fingerprints. DIGLIM adopts the current scheme of RPMs,
and verifies with one signature all the files contained in
the RPM. Since this data format is not suitable for use
by the Linux kernel, for enforcing the integrity policy,
DIGLIM extracts the digests and adds them in a hash
table stored in kernel memory. Enforcement (it would
be better to say security decision) is achieved by doing
a lookup in the hash table.

The main advantage is that DIGLIM can achieve its
objective, providing reference values, without any
change to existing RPMs. It could support fsverity
digests, in addition to file digests, to achieve the same
objective of the FsVerityRPM feature. This would
require introducing a new RPM header section similar
to RPMTAG_FILEDIGESTS.

The FsVerityRPM feature, on the other hand, similarly
to the IMA file signature approach, re-signs all (or the
desired subset of) files, and adds the signatures in a
dedicated section of the RPM header, with increased
size overhead.

The advantage of FsVerityRPM and the IMA file
signatures approach is that the data format is already
suitable for enforcement by fsverity and IMA.


I haven't looked at the details at all, but from a rpm POV birds-eye 
perspective: applauds for the approach!


Besides not bloating up RPMs with seriously expensive per-file data, 
this side-steps the other issues associated with both IMA and fs-verity: 
both require separate signing steps for the file signatures which is a 
non-trivial cost and complexity, and unlike those the file hashes are 
covered (and thus protected) by normal rpm-level signatures too.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 Change: Relocate RPM database to /usr (System-Wide Change proposal)

2022-01-04 Thread Panu Matilainen


On 1/3/22 22:55, Chris Murphy wrote:

Does anyone know what /var/lib/rpm-state/gconf is used for? Owning
package is GConf2-3.2.6-31.fc35.x86_64

On my Fedora 35 Workstation installation, it's empty. So no obvious
conflict with the change proposal, but I'd like to make sure it's not
something that if used is going to get mad if there's an rpmdb state
change that this location isn't privy to.


See 
https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#_saving_state_between_scriptlets, 
the /var/lib/rpm-state/ directory is what Fedora recommends for 
communicating state from one scriptlet to another. Not used or endorsed 
by rpm itself, and not related to the database at all.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 Change: Relocate RPM database to /usr (System-Wide Change proposal)

2022-01-03 Thread Panu Matilainen


On 1/3/22 15:36, Lennart Poettering wrote:

On Mo, 03.01.22 14:15, Florian Weimer (fwei...@redhat.com) wrote:


* Lennart Poettering:


Can you provide an example for such feature requests? i.e. where the
rpmdb should be writable even though /usr is assumed to be immutable?


Maybe if RPM is used to install software under /opt?


I'd argue that conceptually /opt and /usr should be seen as the same
thing for RPM: i.e. during update time both are writable and outside
of updates they are both to be considered read-only from RPM's
PoV. Hence it should be fine too put the rpmdb in /usr/ too in that
case.

Or with other words: if RPM is used to install something in /opt, it
should be fine to require that both /usr and /opt are writable then.


It does seem a bit controversial to require /usr of all things to be 
writable to be install into /opt, /etc, /boot and whatnot.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 Change: Relocate RPM database to /usr (System-Wide Change proposal)

2022-01-03 Thread Panu Matilainen


On 12/29/21 17:01, Ben Cotton wrote:


Upstream RPM accept the change, but institutionally don't like the
loss or weakening of a
[http://lists.rpm.org/pipermail/rpm-ecosystem/2021-December/000781.html
very well known location] for the database, and
[http://lists.rpm.org/pipermail/rpm-ecosystem/2021-December/000781.html
anticipate complaints].


Just noticed this in the change text...

To clarify the upstream position: there are no plans to change the 
upstream default, the level of acceptance is merely that if Fedora wants 
to change it I'm not going to stand in front of the Change truck.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 Change: Relocate RPM database to /usr (System-Wide Change proposal)

2022-01-03 Thread Panu Matilainen


On 1/3/22 14:57, Lennart Poettering wrote:

On Mo, 03.01.22 11:57, Panu Matilainen (pmati...@redhat.com) wrote:


On 12/30/21 09:02, Chris Murphy wrote:

On Wed, Dec 29, 2021 at 8:19 AM Tom Hughes via devel
 wrote:


I don't see how this is FHS compliant, which in turn would make
it non-compliant with Fedora Packaging Guidelines, namely:


https://docs.fedoraproject.org/en-US/packaging-guidelines/#_filesystem_layout

The FHS describes /usr here:

 https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04.html#purpose18

as "/usr is shareable, read-only data" which clearly does not
apply to a database that changes.


In practice it is read-only data, except when software is being
installed or updated. The FHS is a PITA sometimes, but it's not
advocating for systems that can't be updated or changed..



The rpmdb has traditionally been like that, but it doesn't mean it will stay
that way forever more. There are all manner of currently unimplemented
use-cases which would require changing the database outside a direct
install/update/erase context. Many of those use-cases are related to files
and would fall under "but you need writable fs for that anyway" but not all.
Of course it'll always be *mostly* read-only data because of the nature of
the data, compared to a general purpose db in /var.


Can you provide an example for such feature requests? i.e. where the
rpmdb should be writable even though /usr is assumed to be immutable?


There seems to be this strange underlying assumption that all packaged 
content lives in /usr when that's not at all the case. To install a 
kernel, or a config-only package (under etc), or 3rd party software 
putting stuff under /opt, or... you need a writable rpmdb. Ditto for 
'rpm --import'.


But the kind of thing I had in mind when making that comment initially 
is eg ability to update file states in the rpmdb to reflect local 
changes from eg network mounting .. well, the typical example would be 
/usr but in this case that gets a little strange.


In general, "data set by users" is the common case, whether a policy to 
flag a given package as unremovable or non-updatable, or add a "reason" 
(dependency or user-installed), or other annotations.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 Change: Relocate RPM database to /usr (System-Wide Change proposal)

2022-01-03 Thread Panu Matilainen


On 12/30/21 09:02, Chris Murphy wrote:

On Wed, Dec 29, 2021 at 8:19 AM Tom Hughes via devel
 wrote:


I don't see how this is FHS compliant, which in turn would make
it non-compliant with Fedora Packaging Guidelines, namely:


https://docs.fedoraproject.org/en-US/packaging-guidelines/#_filesystem_layout

The FHS describes /usr here:

https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04.html#purpose18

as "/usr is shareable, read-only data" which clearly does not
apply to a database that changes.


In practice it is read-only data, except when software is being
installed or updated. The FHS is a PITA sometimes, but it's not
advocating for systems that can't be updated or changed..



The rpmdb has traditionally been like that, but it doesn't mean it will 
stay that way forever more. There are all manner of currently 
unimplemented use-cases which would require changing the database 
outside a direct install/update/erase context. Many of those use-cases 
are related to files and would fall under "but you need writable fs for 
that anyway" but not all. Of course it'll always be *mostly* read-only 
data because of the nature of the data, compared to a general purpose db 
in /var.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: debug_package when using go_generate_buildrequires

2021-12-20 Thread Panu Matilainen


On 12/20/21 16:02, Bob Mauchin wrote:



On Mon, 20 Dec 2021, 14:09 Panu Matilainen, <mailto:pmati...@redhat.com>> wrote:


On 12/20/21 01:39, Robert-André Mauchin wrote:
 > On 12/6/21 12:42, Mikel Olasagasti wrote:
 >> Hi all,
 >>
 >> I was updating some golang specs I've and switching them to use
 >> go_generate_buildrequires.
 >>
 >> I realized that some specs that are using it fail to build if
`%global
 >> debug_package %{nil}` is not set. Same spec with all BuildRequires
 >> defined works just fine.
 >>
 >> Example:
 >>
 >> - Full spec (go2rpm):
 >>

https://mikel.olasagasti.info/tmp/fedora/golang-github-alecaivazis-survey-2.spec

<https://mikel.olasagasti.info/tmp/fedora/golang-github-alecaivazis-survey-2.spec>

 >>
 >> - go_generate_buildrequires spec:
 >>

https://raw.githubusercontent.com/mikelolasagasti/github-cli/main/golang-github-alecaivazis-survey-2.spec

<https://raw.githubusercontent.com/mikelolasagasti/github-cli/main/golang-github-alecaivazis-survey-2.spec>

 >>
 >>
 >> - Full spec build:
 >> https://koji.fedoraproject.org/koji/taskinfo?taskID=79642572
<https://koji.fedoraproject.org/koji/taskinfo?taskID=79642572>
 >> - go_generate_buildrequires build:
 >> https://koji.fedoraproject.org/koji/taskinfo?taskID=79642147
<https://koji.fedoraproject.org/koji/taskinfo?taskID=79642147>
 >>
 >> Is this expected?
 >> Is adding `%global debug_package %{nil}` correct or good
practice for
 >> this scenario?
 >>
 >> Kind regards,
 >> Mikel Olasagasti (mikelo2)
 >
 > Hello,
 >
 > Yes this is an issue I have noticed but I don't know where/what
causes it.
 >
 > I know that if we have a %build section it will cause the debug
info to
 > be looked for. It does it too with %generate_buildrequires and I
think
 > it shouldn't, but I don't know if this is a behavior triggered by
rpm
 > itself or is it triggered somewhere else during the build process?
 >
 > CC Panu and Devel to weigh in.
 >
 > In the meantime, as long as you don't have a %build section and
thus do
 > not produce a binaries, you can use %global debug_package %{nil}

Having to disable debug packages practically always indicates a bug in
the package - either packaging, upstream or both.

Typical causes for "error: Empty %files file
<...>/debugsourcefiles.list" are missing debug info in the build (eg -g
flag of gcc) and building outside the designated build directory.
Unfortunately diagnosing these from rpm isn't easy.

"%global debug_package %{nil}" is always a workaround and never a fix.

          - Panu -

 >
 > Best regards,
 >
 > Robert-André
 >


Yes I know, but not in this case as go libraries are noarch.
The problem that when we add the %generate_buildrequires section, it 
triggers the detection of the debug info. Without it, the detection of 
debug info is skipped so the package builds normally. I don't know what 
is the expected behavior.


Okay, this wasn't quite so clear in the initial context. 
%generate_buildrequires should not trigger the debuginfo machinery, 
that's a bug of some sort. Please file a ticket at 
https://github.com/rpm-software-management/rpm


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: debug_package when using go_generate_buildrequires

2021-12-20 Thread Panu Matilainen


On 12/20/21 01:39, Robert-André Mauchin wrote:

On 12/6/21 12:42, Mikel Olasagasti wrote:

Hi all,

I was updating some golang specs I've and switching them to use
go_generate_buildrequires.

I realized that some specs that are using it fail to build if `%global
debug_package %{nil}` is not set. Same spec with all BuildRequires
defined works just fine.

Example:

- Full spec (go2rpm):
https://mikel.olasagasti.info/tmp/fedora/golang-github-alecaivazis-survey-2.spec 


- go_generate_buildrequires spec:
https://raw.githubusercontent.com/mikelolasagasti/github-cli/main/golang-github-alecaivazis-survey-2.spec 



- Full spec build: 
https://koji.fedoraproject.org/koji/taskinfo?taskID=79642572

- go_generate_buildrequires build:
https://koji.fedoraproject.org/koji/taskinfo?taskID=79642147

Is this expected?
Is adding `%global debug_package %{nil}` correct or good practice for
this scenario?

Kind regards,
Mikel Olasagasti (mikelo2)


Hello,

Yes this is an issue I have noticed but I don't know where/what causes it.

I know that if we have a %build section it will cause the debug info to 
be looked for. It does it too with %generate_buildrequires and I think 
it shouldn't, but I don't know if this is a behavior triggered by rpm 
itself or is it triggered somewhere else during the build process?


CC Panu and Devel to weigh in.

In the meantime, as long as you don't have a %build section and thus do 
not produce a binaries, you can use %global debug_package %{nil}


Having to disable debug packages practically always indicates a bug in 
the package - either packaging, upstream or both.


Typical causes for "error: Empty %files file 
<...>/debugsourcefiles.list" are missing debug info in the build (eg -g 
flag of gcc) and building outside the designated build directory. 
Unfortunately diagnosing these from rpm isn't easy.


"%global debug_package %{nil}" is always a workaround and never a fix.

 - Panu -



Best regards,

Robert-André


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: rpm bug for multiple README.md or LICENSE.md in EPEL 8 and Fedora

2021-12-08 Thread Panu Matilainen


On 12/7/21 16:18, Nico Kadel-Garcia wrote:

On Tue, Dec 7, 2021 at 5:46 AM Panu Matilainen  wrote:


On 12/5/21 05:07, Nico Kadel-Garcia wrote:

I've been trying to bundle the current ansible-5.0.1 release as an RPM
for Fedora and EPEL use. Leaving aside the peculiar decisions to
replace the pypi.org "ansible" tarball with a tarball of roughly 150
modules from the "ansiblee-collections" repos, and moving the actual
ansible software to a distinct python tarball called "ansible-core"
without changing the source repo or the actual critical installed
python modules, the new "ansible" has more than 300 files called
"README.md" and more than 100 files called "LICENSE.md".

This breaks building RPMs for EPEL 8 or Fedora, because the '%doc' and
'%license' macros strip off the subdirectories of the files and
install them directly at the top of the docdir.

Basicely these only generate one file:

 %doc README.md
 %doc dir1/README.md
 %doc dir2/README.md

 %license LICENSE.md
  %license dir1/LICENSE
  %license dir2/LICENSE.md

When compiled, these would only produce:

/usr/share/doc/package-%{fersion}/README.md
/usr/share/doc/package-%{fersion}/LICENSE.md

RHEL 7 didn't have this problem. I'm not sure if other folks have
noticed this for tools that are built with multiple internal tarballs.
The new "ansible" tarball is fairly unique ints authors insistance on
putting more than one hundred distinct third party packages in the
same master tarball. But for now, this is going to cause a license and
documentation problem in packaging it due to an "optimization" of
stripping out the directory names of document files and licenses.

Does anyone know a decent workaround, or a specfile setting to disable
this filename stripping and restore the RHEL 7 behavior?


There were changes in how %doc is handled around rpm 4.13 (whereas
RHEL-7 has 4.11), maybe there's a regression that nobody noticed (or
complained about) until now. The use-case there seems quite reasonable
to me.

File a bug on rpm (preferably upstream ticket) to have it looked at.

 - Panu -


I've submitted one, at https://bugzilla.redhat.com/show_bug.cgi?id=2029877 .

It seems that the problem existed as well with earlier versions of
rpmbuild, say on RHEL 7, but the warning messages about "files listed
twice" weren't generated. And various slightly differently named
README.md and LICENSE.md files make it not as obvious as one might
expect.


Okay, but since it happens on RHEL 7 too then there's no regression. I 
thought the leading directories were always stripped but given your 
strong statement about RHEL 7 not having this issue I thought maybe I'm 
missing something.


The current behavior is good for other use-cases and we can't go just 
changing it, but this is a totally legit (and common, I think) use-case 
so maybe we can come up with something to optionally not strip leading 
directories.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: rpm bug for multiple README.md or LICENSE.md in EPEL 8 and Fedora

2021-12-07 Thread Panu Matilainen


On 12/5/21 05:07, Nico Kadel-Garcia wrote:

I've been trying to bundle the current ansible-5.0.1 release as an RPM
for Fedora and EPEL use. Leaving aside the peculiar decisions to
replace the pypi.org "ansible" tarball with a tarball of roughly 150
modules from the "ansiblee-collections" repos, and moving the actual
ansible software to a distinct python tarball called "ansible-core"
without changing the source repo or the actual critical installed
python modules, the new "ansible" has more than 300 files called
"README.md" and more than 100 files called "LICENSE.md".

This breaks building RPMs for EPEL 8 or Fedora, because the '%doc' and
'%license' macros strip off the subdirectories of the files and
install them directly at the top of the docdir.

Basicely these only generate one file:

%doc README.md
%doc dir1/README.md
%doc dir2/README.md

%license LICENSE.md
 %license dir1/LICENSE
 %license dir2/LICENSE.md

When compiled, these would only produce:

   /usr/share/doc/package-%{fersion}/README.md
   /usr/share/doc/package-%{fersion}/LICENSE.md

RHEL 7 didn't have this problem. I'm not sure if other folks have
noticed this for tools that are built with multiple internal tarballs.
The new "ansible" tarball is fairly unique ints authors insistance on
putting more than one hundred distinct third party packages in the
same master tarball. But for now, this is going to cause a license and
documentation problem in packaging it due to an "optimization" of
stripping out the directory names of document files and licenses.

Does anyone know a decent workaround, or a specfile setting to disable
this filename stripping and restore the RHEL 7 behavior?


There were changes in how %doc is handled around rpm 4.13 (whereas 
RHEL-7 has 4.11), maybe there's a regression that nobody noticed (or 
complained about) until now. The use-case there seems quite reasonable 
to me.


File a bug on rpm (preferably upstream ticket) to have it looked at.

- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

1 2 3 4 5 6 7 >

1 - 100 of 660 matches

Mail list logo