Re: [gentoo-dev] [PATCH] systemd.eclass: remove SYSROOT from pkg-config output

[Mike Gilbert]

On Mon, Jan 4, 2021 at 6:45 PM Mike Gilbert  wrote:
>
> On Mon, Jan 4, 2021 at 6:18 PM James Le Cuirot  wrote:
> > $ PKG_CONFIG_SYSROOT_DIR=/foo pkg-config --variable=udevdir udev
> > /lib/udev
> >
> > The udevdir variable is not affected by PKG_CONFIG_SYSROOT_DIR at all.
> > And why would it be? The man page says that this variable is only
> > applied to -I and -L flags. I don't know for sure but I suspect that
> > pkg-config just sees this as some arbitrary variable with no special
> > path handling at all. I wonder what led you to think that this fix was
> > necessary?
>
> Interesting!
>
> pkg-config behaves differently on my system:
>
> % PKG_CONFIG_SYSROOT_DIR=/foo pkg-config --variable=udevdir udev
> /foo/lib/udev
>
> This appears to be a difference in behavior between dev-util/pkgconfig
> and dev-util/pkgconf. I am using pkgconf, and I would guess you are
> using pkgconfig.
>
> I guess I will ask pkgconf upstream for help on this; it seems like
> this is probably an unintended behavior.

It seems that the pkgconf behavior is intentional.

https://github.com/pkgconf/pkgconf/issues/69

I opened an issue to see if we can get some kind of opt-out.

https://github.com/pkgconf/pkgconf/issues/205

Re: [gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Patrick McLean]

On Tue, 05 Jan 2021 00:54:58 +0100
Michał Górny  wrote:

> On Mon, 2021-01-04 at 15:50 -0800, Patrick McLean wrote:
> > On Tue, 05 Jan 2021 00:16:49 +0100
> > Michał Górny  wrote:
> > > On Mon, 2021-01-04 at 14:58 -0800, Patrick McLean wrote:  
> > > > On Mon,  4 Jan 2021 18:08:02 +0100
> > > > Michał Górny  wrote:
> > > > > Introduce a few variables to allow easy overrides of common user 
> > > > > account
> > > > > proprerties, that is:
> > > > > 
> > > > > - ACCT_USER__SHELL
> > > > > - ACCT_USER__HOME
> > > > > - ACCT_USER__HOME_OWNER
> > > > > - ACCT_USER__HOME_PERMS
> > > > > - ACCT_USER__GROUPS
> > > > > - ACCT_USER__GROUPS_ADD
> > > > 
> > > > Please also add a way to override the UID/GID for the user/group.
> > > 
> > > Damn it, and I thought I'd avoid that ;-).  But do we really need it? 
> > > The eclass doesn't enforce UID/GID by default if the user exists
> > > already, so it's a bit tangential to the original problem.
> > >   
> > 
> > The user needs to already exist for that to be helpful. When one using
> > automation to build/deploy large numbers of Gentoo systems, it's quite
> > useful to have control over that sort of things. At the moment, the
> > only way is to fork the ebuilds, which of course means they need to be
> > kept in sync.  
> 
> Ok, I'll keep that mind.  However, I suppose you won't mind me
> addressing that separately?  Unlike the patch sent, ID-related logic
> needs to be done twice (due to pkg_pretend).  Ideally, could you report
> a feature request on Bugzilla?

Sure, I don't mind it being addressed separately. I created a feature
request on Bugzilla: https://bugs.gentoo.org/763615

Re: [gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Michał Górny]

On Mon, 2021-01-04 at 15:50 -0800, Patrick McLean wrote:
> On Tue, 05 Jan 2021 00:16:49 +0100
> Michał Górny  wrote:
> 
> > On Mon, 2021-01-04 at 14:58 -0800, Patrick McLean wrote:
> > > On Mon,  4 Jan 2021 18:08:02 +0100
> > > Michał Górny  wrote:
> > >   
> > > 
> > > 
> > > 
> > > > Introduce a few variables to allow easy overrides of common user account
> > > > proprerties, that is:
> > > > 
> > > > - ACCT_USER__SHELL
> > > > - ACCT_USER__HOME
> > > > - ACCT_USER__HOME_OWNER
> > > > - ACCT_USER__HOME_PERMS
> > > > - ACCT_USER__GROUPS
> > > > - ACCT_USER__GROUPS_ADD  
> > > 
> > > Please also add a way to override the UID/GID for the user/group.  
> > 
> > Damn it, and I thought I'd avoid that ;-).  But do we really need it? 
> > The eclass doesn't enforce UID/GID by default if the user exists
> > already, so it's a bit tangential to the original problem.
> > 
> 
> The user needs to already exist for that to be helpful. When one using
> automation to build/deploy large numbers of Gentoo systems, it's quite
> useful to have control over that sort of things. At the moment, the
> only way is to fork the ebuilds, which of course means they need to be
> kept in sync.

Ok, I'll keep that mind.  However, I suppose you won't mind me
addressing that separately?  Unlike the patch sent, ID-related logic
needs to be done twice (due to pkg_pretend).  Ideally, could you report
a feature request on Bugzilla?


-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Patrick McLean]

On Tue, 05 Jan 2021 00:16:49 +0100
Michał Górny  wrote:

> On Mon, 2021-01-04 at 14:58 -0800, Patrick McLean wrote:
> > On Mon,  4 Jan 2021 18:08:02 +0100
> > Michał Górny  wrote:
> >   
> > > Introduce a few variables to allow easy overrides of common user account
> > > proprerties, that is:
> > > 
> > > - ACCT_USER__SHELL
> > > - ACCT_USER__HOME
> > > - ACCT_USER__HOME_OWNER
> > > - ACCT_USER__HOME_PERMS
> > > - ACCT_USER__GROUPS
> > > - ACCT_USER__GROUPS_ADD  
> > 
> > Please also add a way to override the UID/GID for the user/group.  
> 
> Damn it, and I thought I'd avoid that ;-).  But do we really need it? 
> The eclass doesn't enforce UID/GID by default if the user exists
> already, so it's a bit tangential to the original problem.
> 

The user needs to already exist for that to be helpful. When one using
automation to build/deploy large numbers of Gentoo systems, it's quite
useful to have control over that sort of things. At the moment, the
only way is to fork the ebuilds, which of course means they need to be
kept in sync.

Re: [gentoo-dev] [PATCH] systemd.eclass: remove SYSROOT from pkg-config output

[Mike Gilbert]

On Mon, Jan 4, 2021 at 6:18 PM James Le Cuirot  wrote:
> $ PKG_CONFIG_SYSROOT_DIR=/foo pkg-config --variable=udevdir udev
> /lib/udev
>
> The udevdir variable is not affected by PKG_CONFIG_SYSROOT_DIR at all.
> And why would it be? The man page says that this variable is only
> applied to -I and -L flags. I don't know for sure but I suspect that
> pkg-config just sees this as some arbitrary variable with no special
> path handling at all. I wonder what led you to think that this fix was
> necessary?

Interesting!

pkg-config behaves differently on my system:

% PKG_CONFIG_SYSROOT_DIR=/foo pkg-config --variable=udevdir udev
/foo/lib/udev

This appears to be a difference in behavior between dev-util/pkgconfig
and dev-util/pkgconf. I am using pkgconf, and I would guess you are
using pkgconfig.

I guess I will ask pkgconf upstream for help on this; it seems like
this is probably an unintended behavior.

> One last question. Why is this dynamic at all? Shouldn't it just be
> hardcoded to /lib/udev? Sure, a user could patch udev to make it
> something different if they really wanted but there are plenty of other
> paths we just assume. What makes this one special?

sys-apps/systemd has a USE flag called "split-usr". This is meant to
allow users to perform a /usr merge if desired. When split-usr is
disabled, udevdir becomes /usr/lib/udev instead of /lib/udev.

Re: [gentoo-dev] [PATCH] systemd.eclass: remove SYSROOT from pkg-config output

[James Le Cuirot]

On Sun, 3 Jan 2021 10:16:49 -0500
Mike Gilbert  wrote:

> On Sun, Jan 3, 2021 at 7:52 AM James Le Cuirot  wrote:
> >
> > On Sat,  2 Jan 2021 20:09:04 -0500
> > Mike Gilbert  wrote:
> >  
> > > When cross-compiling, users will typically have
> > > PKG_CONFIG_SYSROOT=${SYSROOT} defined via pkg-config wrapper.
> > >
> > > When PKG_CONFIG_SYSROOT is set, all paths included in pkg-config
> > > output get prefixed with this value.
> > >
> > > Signed-off-by: Mike Gilbert 
> > > ---
> > >
> > > This patch has already been pushed, but I figured I would send it for
> > > review in case someone else can think of a failure case, or has a better
> > > solution.
> > >
> > >  eclass/systemd.eclass | 1 +
> > >  1 file changed, 1 insertion(+)
> > >
> > > diff --git a/eclass/systemd.eclass b/eclass/systemd.eclass
> > > index 81065a0af79a..f6d1fa2d92d6 100644
> > > --- a/eclass/systemd.eclass
> > > +++ b/eclass/systemd.eclass
> > > @@ -50,6 +50,7 @@ _systemd_get_dir() {
> > >
> > >   if $(tc-getPKG_CONFIG) --exists systemd; then
> > >   d=$($(tc-getPKG_CONFIG) --variable="${variable}" systemd) 
> > > || die
> > > + d=${d#${SYSROOT}}
> > >   d=${d#${EPREFIX}}
> > >   else
> > >   d=${fallback}  
> 
> > The EPREFIX line is (sometimes) wrong in EAPI 7 though and the same goes
> > for udev.eclass. It took a while to get this agreed and corrected in
> > PMS but if SYSROOT points to / then the effective prefix is BROOT, not
> > EPREFIX; they may not be the same.  
> 
> Ugh, that "corrected" logic in PMS head is nasty.

I wish it could be simpler but that's just the nature of the beast. I
could give an example of why it matters, even in this context, but I
think you already know. There's some good news though...

> > If you just strip ESYSROOT then
> > it will always do the right thing but you'll need this fall back for
> > older EAPIs. I'm not sure why you didn't do it in one line?  
> 
> I was trying to accommodate older EAPIs that do not define ESYSROOT.
> This seemed like the simplest approach. It also allows for the case
> where someone is not using cross-pkg-config, and
> PKG_CONFIG_SYSROOT_DIR is unset. Since SYSROOT and EPREFIX are removed
> separately, if one of them is already missing, it will just be
> skipped.

I saw your new patch. I noticed that it sets the "d" variable but
doesn't make use of it. That aside, I thought perhaps I could come up
with something cleaner. That's when I made an amusing discovery.

$ PKG_CONFIG_SYSROOT_DIR=/foo pkg-config --variable=udevdir udev  
/lib/udev

The udevdir variable is not affected by PKG_CONFIG_SYSROOT_DIR at all.
And why would it be? The man page says that this variable is only
applied to -I and -L flags. I don't know for sure but I suspect that
pkg-config just sees this as some arbitrary variable with no special
path handling at all. I wonder what led you to think that this fix was
necessary?

That said, you'll still need special handling for EAPI 7. Unfortunately
there's no separate variable for the prefix alone but at least it's
simpler now. Something like this?

diff --git a/eclass/udev.eclass b/eclass/udev.eclass
index 2873ae9a92c3..f10ea0de01a2 100644
--- a/eclass/udev.eclass
+++ b/eclass/udev.eclass
@@ -50,12 +50,19 @@ fi
 # @DESCRIPTION:
 # Get unprefixed udevdir.
 _udev_get_udevdir() {
-   if $($(tc-getPKG_CONFIG) --exists udev); then
-   local udevdir="$($(tc-getPKG_CONFIG) --variable=udevdir udev)"
-   echo "${udevdir#${EPREFIX%/}}"
-   else
-   echo /lib/udev
+   local udevdir="/lib/udev"
+
+   if $(tc-getPKG_CONFIG) --exists udev; then
+   udevdir="$($(tc-getPKG_CONFIG) --variable=udevdir udev)" || die
+
+   if [[ ${EAPI:-0} == [0123456] ]]; then
+   udevdir=${udevdir#${EPREFIX}}
+   else
+   udevdir=${udevdir#${ESYSROOT#${SYSROOT}}}
+   fi
fi
+
+   echo "${udevdir}"
 }

One last question. Why is this dynamic at all? Shouldn't it just be
hardcoded to /lib/udev? Sure, a user could patch udev to make it
something different if they really wanted but there are plenty of other
paths we just assume. What makes this one special?

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer


pgpSPRcvfw2Md.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Michał Górny]

On Mon, 2021-01-04 at 14:58 -0800, Patrick McLean wrote:
> On Mon,  4 Jan 2021 18:08:02 +0100
> Michał Górny  wrote:
> 
> > Introduce a few variables to allow easy overrides of common user account
> > proprerties, that is:
> > 
> > - ACCT_USER__SHELL
> > - ACCT_USER__HOME
> > - ACCT_USER__HOME_OWNER
> > - ACCT_USER__HOME_PERMS
> > - ACCT_USER__GROUPS
> > - ACCT_USER__GROUPS_ADD
> 
> Please also add a way to override the UID/GID for the user/group.

Damn it, and I thought I'd avoid that ;-).  But do we really need it? 
The eclass doesn't enforce UID/GID by default if the user exists
already, so it's a bit tangential to the original problem.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Patrick McLean]

On Mon,  4 Jan 2021 18:08:02 +0100
Michał Górny  wrote:

> Introduce a few variables to allow easy overrides of common user account
> proprerties, that is:
> 
> - ACCT_USER__SHELL
> - ACCT_USER__HOME
> - ACCT_USER__HOME_OWNER
> - ACCT_USER__HOME_PERMS
> - ACCT_USER__GROUPS
> - ACCT_USER__GROUPS_ADD

Please also add a way to override the UID/GID for the user/group.

> The first five variables override the respective ACCT_USER_* variables,
> with ACCT_USER_*_GROUPS being a space-separated list.  The *_GROUPS_ADD
> variable appends to groups present in the ebuild, as this seems a common
> necessity.

Re: [gentoo-dev] [News review v3] LibreSSL support discontinued

[Michał Górny]

On Mon, 2021-01-04 at 21:48 +0100, Michał Górny wrote:
> On Mon, 2021-01-04 at 20:59 +0100, Ulrich Mueller wrote:
> > > > > > > On Mon, 04 Jan 2021, Michał Górny wrote:
> > 
> > > Starting 2021-02-01, Gentoo will discontinue supporting
> > > dev-libs/libressl as an alternative to dev-libs/openssl.  While it
> > > will
> > 
> > > [...]
> > 
> > > On 2021-02-01, we will mask the relevant USE flags and packages. 
> > > If
> > > you
> > 
> > > [...]
> > 
> > > necessary to use the user-maintained LibreSSL overlay [1].  As
> > > long-
> > > term
> > 
> > > [...]
> > 
> > > development gained speed and the original reasons for the fork no
> > > longer
> > 
> > > [...]
> > 
> > > problems were related to packages using old/insecure OpenSSL APIs,
> > > today
> > 
> > > [...]
> > 
> > > To the best of our knowledge, the only benefit LibreSSL has over
> > > OpenSSL
> > 
> > This has some strange line breaks now. Please fix.
> 
> It's just my stupid mail client, please disregard that.
> 

Anyway, the correct paste:

---
Title: LibreSSL support discontinued
Author: Michał Górny 
Posted: 202x-xx-xx
Revision: 1
News-Item-Format: 2.0
Display-If-Installed: dev-libs/libressl

Starting 2021-02-01, Gentoo will discontinue supporting
dev-libs/libressl as an alternative to dev-libs/openssl.  While it will
still be possible for expert users to use LibreSSL on their systems,
we are only going to provide support for OpenSSL-based systems.  Most
importantly, we are no longer going to maintain downstream patches for
LibreSSL support -- it will rely on either package upstreams merging
such patches themselves, or LibreSSL upstream finally working towards
better OpenSSL compatibility.

On 2021-02-01, we will mask the relevant USE flags and packages.  If you
wish to continue using LibreSSL, you will be able to undo these masks
for the time being.  However, as packages drop patching for LibreSSL
and the library is eventually removed from ::gentoo, it will become
necessary to use the user-maintained LibreSSL overlay [1].  As long-term
support for LibreSSL is not guaranteed, we recommend switching
to OpenSSL instead.  More information on removal can be found
on the relevant bug [2].

To switch before the aforementioned date, remove 'libressl' from your
USE flags and CURL_SSL targets.  Afterwards, it is recommended to
prefetch all the necessary distfiles before proceeding with the system
upgrade, in case wget(1) becomes broken in the process:

emerge --fetchonly dev-libs/openssl net-misc/wget
emerge --fetchonly --deep --changed-use @world

A --changed-use @world upgrade should automatically cause LibreSSL
to be replaced by OpenSSL, and all affected packages to be rebuilt:

emerge --deselect dev-libs/libressl
emerge --changed-use --deep @world


LibreSSL has been forked off OpenSSL in 2014 to address a number of
problems with the original package.  However, since then OpenSSL
development gained speed and the original reasons for the fork no longer
apply.  Furthermore, LibreSSL started to repeatedly fall behind
and cause growing compatibility problems.  While initially these
problems were related to packages using old/insecure OpenSSL APIs, today
they are mostly related to LibreSSL missing newer OpenSSL APIs
(yet declaring false compatibility with newer OpenSSL versions).

With the little testing it gets, our developers and users had to put
a significant effort into fixing upstream packages.  In some cases
(e.g. Qt), upstream has explicitly refused to support LibreSSL, forcing
us to maintain the patches forever.  This in turn means that
security fixes, regular version bumps or end-user system upgrades are
often delayed because of necessary LibreSSL patching.  What is even
worse, major runtime issues managed to sneak in that broke production
systems running LibreSSL in the past.

To the best of our knowledge, the only benefit LibreSSL has over OpenSSL
right now is the additional libtls library.  For this reason, we have
packaged dev-libs/libretls which is a port of this library that links
to OpenSSL.

All these issues considered, we came to the conclusion that OpenSSL
should remain the only supported production option for Gentoo systems.
While the flexibility of Gentoo should make it possible to keep using
LibreSSL going forward, the effort necessary to provide first-class
official support for LibreSSL has proven to outweigh the benefit.

[1] https://gitweb.gentoo.org/repo/proj/libressl.git/tree/README.md
[2] https://bugs.gentoo.org/762847

---

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [News review v3] LibreSSL support discontinued

[Michał Górny]

On Mon, 2021-01-04 at 20:59 +0100, Ulrich Mueller wrote:
> > > > > > On Mon, 04 Jan 2021, Michał Górny wrote:
> 
> > Starting 2021-02-01, Gentoo will discontinue supporting
> > dev-libs/libressl as an alternative to dev-libs/openssl.  While it
> > will
> 
> > [...]
> 
> > On 2021-02-01, we will mask the relevant USE flags and packages. 
> > If
> > you
> 
> > [...]
> 
> > necessary to use the user-maintained LibreSSL overlay [1].  As
> > long-
> > term
> 
> > [...]
> 
> > development gained speed and the original reasons for the fork no
> > longer
> 
> > [...]
> 
> > problems were related to packages using old/insecure OpenSSL APIs,
> > today
> 
> > [...]
> 
> > To the best of our knowledge, the only benefit LibreSSL has over
> > OpenSSL
> 
> This has some strange line breaks now. Please fix.

It's just my stupid mail client, please disregard that.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [News review v3] LibreSSL support discontinued

[Ulrich Mueller]

> On Mon, 04 Jan 2021, Michał Górny wrote:

> Starting 2021-02-01, Gentoo will discontinue supporting
> dev-libs/libressl as an alternative to dev-libs/openssl.  While it will

> [...]

> On 2021-02-01, we will mask the relevant USE flags and packages.  If
> you

> [...]

> necessary to use the user-maintained LibreSSL overlay [1].  As long-
> term

> [...]

> development gained speed and the original reasons for the fork no
> longer

> [...]

> problems were related to packages using old/insecure OpenSSL APIs,
> today

> [...]

> To the best of our knowledge, the only benefit LibreSSL has over
> OpenSSL

This has some strange line breaks now. Please fix.

Ulrich


signature.asc
Description: PGP signature

Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

[Michael Orlitzky]

On 1/4/21 1:20 PM, Michał Górny wrote:


Most importantly, it doesn't resolve the core issue of 'we need to
update home before merging reverse dependencies'.



Quoth the devmanual, "if your package requires a user, you can no longer 
be sure of that user's home directory or its ownership and permissions."
Any package depending on a new revision of an acct-user ebuild to change 
its home directory is broken.


This was the exact problem I was trying to avoid while you made rude 
comments about how I was wasting everyone's time =P

Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

On 2021-01-04 19:27, Michael Orlitzky wrote:

People like me could just ignore changed users if changes won't go live
until you run said users-update command or make use of INSTALL_MASK.



Changes wouldn't go live until you ran etc-update, and *then* users-update.


This would address my concerns.

But I still wonder if building such a system is worth it... I mean, it 
would be nice to have. Maybe we could build upon such a system to do 
same for (changed) file permissions...



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



OpenPGP_signature
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

[Michael Orlitzky]

On 1/4/21 1:23 PM, Thomas Deutschmann wrote:


People like me could just ignore changed users if changes won't go live
until you run said users-update command or make use of INSTALL_MASK.



Changes wouldn't go live until you ran etc-update, and *then* users-update.

Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

Hi,

On 2021-01-04 19:07, Michael Orlitzky wrote:
We could implement this with something like an /etc/users.d directory 
that would be populated with entries by either the admin or package 
manager with CONFIG_PROTECT enabled. Then the system database would be 
updated by running something like "users-update" (cf. env-update). The 
essential problem that we need to work around is that e.g. /etc/passwd 
is "owned" by multiple system packages.


I think this would accomplish what you and Robin are talking about, but 
it wouldn't solve whissi's problem since it's still a Gentoo-specific 
solution.


If you really want to spend so much time on this, feel free to implement 
something like this. From my point of view this is wasted time. I really 
have no words for anyone believing that there must be a way to deal with 
user config. This is a no go for me and most people in my bubble. Once 
you have created something, it's user data. If you want to make changes, 
tell the user about it but never ever mess with user configs. History is 
full of examples when messing with user configs caused real harm.


For example there is a reason why we don't edit /etc files. Instead have 
CONFIG_PROTECT and are only providing helpers to update config.


Do I really need to explain what can go wrong when you suddenly change 
/home? What will happen to your cron jobs for example?


What will happen when you make changes to groups and reboot?

But as said, if you want to spend so much time on this and create a 
complicated solution which will be adding a lot of complexity which I 
think isn't worth it, *I* could live with it. It's the same like dealing 
with CONFIG_PROTECT already.


People like me could just ignore changed users if changes won't go live 
until you run said users-update command or make use of INSTALL_MASK.



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



OpenPGP_signature
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH gentoolkit] gentoolkit: Remove gentoolkit.test.cmp

[Matt Turner]

Signed-off-by: Matt Turner 
---
Seems to only be used by duplicated unit tests? I guess this might have
been useful when Python 2 was still supported?

 pym/gentoolkit/test/__init__.py  | 23 ---
 pym/gentoolkit/test/test_atom.py | 15 ---
 pym/gentoolkit/test/test_cpv.py  | 15 ---
 3 files changed, 53 deletions(-)

diff --git a/pym/gentoolkit/test/__init__.py b/pym/gentoolkit/test/__init__.py
index 9e41686..e69de29 100644
--- a/pym/gentoolkit/test/__init__.py
+++ b/pym/gentoolkit/test/__init__.py
@@ -1,23 +0,0 @@
-#!/usr/bin/python
-# Copyright 2009 Gentoo Foundation
-#
-# Distributed under the terms of the GNU General Public License v2
-
-__all__ = ['cmp']
-
-# py3k doesn't have cmp emulate it in order to keep testing cmp
-# in python-2.x
-#XXX: not sure if this is the best place for this
-try:
-   cmp = cmp
-except NameError:
-   def cmp(a, b):
-   if a == b:
-   return 0
-   elif a < b:
-   return -1
-   elif a > b:
-   return 1
-   # just to be safe, __lt__/ __gt__ above should have thrown
-   # something like this already
-   raise TypeError("Comparison between unorderable types")
diff --git a/pym/gentoolkit/test/test_atom.py b/pym/gentoolkit/test/test_atom.py
index 664bf40..6df52a9 100644
--- a/pym/gentoolkit/test/test_atom.py
+++ b/pym/gentoolkit/test/test_atom.py
@@ -7,7 +7,6 @@ import unittest
 
 from gentoolkit.atom import Atom
 from gentoolkit.cpv import CPV
-from gentoolkit.test import cmp
 
 """Atom test suite (verbatim) from pkgcore."""
 
@@ -17,26 +16,12 @@ class TestGentoolkitAtom(unittest.TestCase):
# logic bugs hidden behind short circuiting comparisons for 
metadata
# is why we test the comparison *both* ways.
self.assertEqual(o1, o2)
-   c = cmp(o1, o2)
-   self.assertEqual(c, 0,
-   msg="checking cmp for %r, %r, aren't equal: got %i" % 
(o1, o2, c))
self.assertEqual(o2, o1)
-   c = cmp(o2, o1)
-   self.assertEqual(c, 0,
-   msg="checking cmp for %r, %r,aren't equal: got %i" % 
(o2, o1, c))
 
def assertNotEqual2(self, o1, o2):
# is why we test the comparison *both* ways.
self.assertNotEqual(o1, o2)
-   c = cmp(o1, o2)
-   self.assertNotEqual(c, 0,
-   msg="checking cmp for %r, %r, not supposed to be equal, 
got %i"
-   % (o1, o2, c))
self.assertNotEqual(o2, o1)
-   c = cmp(o2, o1)
-   self.assertNotEqual(c, 0,
-   msg="checking cmp for %r, %r, not supposed to be equal, 
got %i"
-   % (o2, o1, c))
 
def test_comparison(self):
self.assertEqual2(Atom('cat/pkg'), Atom('cat/pkg'))
diff --git a/pym/gentoolkit/test/test_cpv.py b/pym/gentoolkit/test/test_cpv.py
index 3817e9f..92ffba5 100644
--- a/pym/gentoolkit/test/test_cpv.py
+++ b/pym/gentoolkit/test/test_cpv.py
@@ -7,7 +7,6 @@
 import unittest
 
 from gentoolkit.cpv import CPV, compare_strs
-from gentoolkit.test import cmp
 
 class TestGentoolkitCPV(unittest.TestCase):
 
@@ -15,26 +14,12 @@ class TestGentoolkitCPV(unittest.TestCase):
# logic bugs hidden behind short circuiting comparisons for 
metadata
# is why we test the comparison *both* ways.
self.assertEqual(o1, o2)
-   c = cmp(o1, o2)
-   self.assertEqual(c, 0,
-   msg="checking cmp for %r, %r, aren't equal: got %i" % 
(o1, o2, c))
self.assertEqual(o2, o1)
-   c = cmp(o2, o1)
-   self.assertEqual(c, 0,
-   msg="checking cmp for %r, %r,aren't equal: got %i" % 
(o2, o1, c))
 
def assertNotEqual2(self, o1, o2):
# is why we test the comparison *both* ways.
self.assertNotEqual(o1, o2)
-   c = cmp(o1, o2)
-   self.assertNotEqual(c, 0,
-   msg="checking cmp for %r, %r, not supposed to be equal, 
got %i"
-   % (o1, o2, c))
self.assertNotEqual(o2, o1)
-   c = cmp(o2, o1)
-   self.assertNotEqual(c, 0,
-   msg="checking cmp for %r, %r, not supposed to be equal, 
got %i"
-   % (o2, o1, c))
 
def test_comparison(self):
self.assertEqual2(CPV('pkg'), CPV('pkg'))
-- 
2.26.2

Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

[Michał Górny]

On Mon, 2021-01-04 at 13:07 -0500, Michael Orlitzky wrote:
> On 1/4/21 11:45 AM, James Cloos wrote:
> > > > > > > "RHJ" == Robin H Johnson  writes:
> > 
> > RHJ> The best I can come up with at the moment, is that any packaging should
> > RHJ> detect if there are user modifications, and provide control to users
> > RHJ> based on that fact.
> > 
> > Exactly.  Akin to etc-update.
> > 
> 
> We could implement this with something like an /etc/users.d directory 
> that would be populated with entries by either the admin or package 
> manager with CONFIG_PROTECT enabled. Then the system database would be 
> updated by running something like "users-update" (cf. env-update). The 
> essential problem that we need to work around is that e.g. /etc/passwd 
> is "owned" by multiple system packages.

Most importantly, it doesn't resolve the core issue of 'we need to
update home before merging reverse dependencies'.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Michał Górny]

On Mon, 2021-01-04 at 18:14 +0100, Thomas Deutschmann wrote:
> On 2021-01-04 18:08, Michał Górny wrote:
> > Introduce a few variables to allow easy overrides of common user account
> > proprerties, that is:
> > 
> > - ACCT_USER__SHELL
> > - ACCT_USER__HOME
> > - ACCT_USER__HOME_OWNER
> > - ACCT_USER__HOME_PERMS
> > - ACCT_USER__GROUPS
> > - ACCT_USER__GROUPS_ADD
> > 
> > The first five variables override the respective ACCT_USER_* variables,
> > with ACCT_USER_*_GROUPS being a space-separated list.  The *_GROUPS_ADD
> > variable appends to groups present in the ebuild, as this seems a common
> > necessity.
> > 
> > We do realize that the original requirement of overriding ebuilds
> > in a local repository was inconvenient.  This new logic should permit
> > easy updates via make.conf.  Additionally, it has the advantage
> > of clearly reporting the changes made in the build logs.
> > 
> > This does not preclude other solutions to the problem.  However, this
> > is probably the best one and it should become the current
> > recommendation.
> 
> This will improve the overlay situation and can be seen as overall 
> improvement but it doesn't address any shared concerns nor is it a 
> replacement for the proposed 'acct-user.eclass: don't modify existing 
> user by default' patch.

If you read the commit message you'd realize I said that it's not
a replacement.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

[Michael Orlitzky]

On 1/4/21 11:45 AM, James Cloos wrote:

"RHJ" == Robin H Johnson  writes:


RHJ> The best I can come up with at the moment, is that any packaging should
RHJ> detect if there are user modifications, and provide control to users
RHJ> based on that fact.

Exactly.  Akin to etc-update.



We could implement this with something like an /etc/users.d directory 
that would be populated with entries by either the admin or package 
manager with CONFIG_PROTECT enabled. Then the system database would be 
updated by running something like "users-update" (cf. env-update). The 
essential problem that we need to work around is that e.g. /etc/passwd 
is "owned" by multiple system packages.


I think this would accomplish what you and Robin are talking about, but 
it wouldn't solve whissi's problem since it's still a Gentoo-specific 
solution.

[gentoo-dev] Last rites: dev-libs/ilbc-rfc3951

[Sam James]

# Jaco Kroon  (2021-01-04)
# media-libs/libilbc is (for every package I manage) a drop-in for
# dev-libs/ilbc-rfc3951. The latter had net-misc/asterisk,
# net-libs/pjproject and net-voip/yate as dependencies.
# (All of which has been bumped to no longer have this dependency.)
# Removal in 30 days, bug #70
dev-libs/ilbc-rfc3951


signature.asc
Description: Message signed with OpenPGP

Re: [gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Alec Warner]

On Mon, Jan 4, 2021 at 9:15 AM Thomas Deutschmann  wrote:
>
> On 2021-01-04 18:08, Michał Górny wrote:
> > Introduce a few variables to allow easy overrides of common user account
> > proprerties, that is:
> >
> > - ACCT_USER__SHELL
> > - ACCT_USER__HOME
> > - ACCT_USER__HOME_OWNER
> > - ACCT_USER__HOME_PERMS
> > - ACCT_USER__GROUPS
> > - ACCT_USER__GROUPS_ADD
> >
> > The first five variables override the respective ACCT_USER_* variables,
> > with ACCT_USER_*_GROUPS being a space-separated list.  The *_GROUPS_ADD
> > variable appends to groups present in the ebuild, as this seems a common
> > necessity.
> >
> > We do realize that the original requirement of overriding ebuilds
> > in a local repository was inconvenient.  This new logic should permit
> > easy updates via make.conf.  Additionally, it has the advantage
> > of clearly reporting the changes made in the build logs.
> >
> > This does not preclude other solutions to the problem.  However, this
> > is probably the best one and it should become the current
> > recommendation.
>
> This will improve the overlay situation and can be seen as overall
> improvement but it doesn't address any shared concerns nor is it a
> replacement for the proposed 'acct-user.eclass: don't modify existing
> user by default' patch.
>

Same response from me, merge it but please also merge the other patch.

-A

>
> --
> Regards,
> Thomas Deutschmann / Gentoo Linux Developer
> fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
>

Re: [gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Thomas Deutschmann]

On 2021-01-04 18:08, Michał Górny wrote:

Introduce a few variables to allow easy overrides of common user account
proprerties, that is:

- ACCT_USER__SHELL
- ACCT_USER__HOME
- ACCT_USER__HOME_OWNER
- ACCT_USER__HOME_PERMS
- ACCT_USER__GROUPS
- ACCT_USER__GROUPS_ADD

The first five variables override the respective ACCT_USER_* variables,
with ACCT_USER_*_GROUPS being a space-separated list.  The *_GROUPS_ADD
variable appends to groups present in the ebuild, as this seems a common
necessity.

We do realize that the original requirement of overriding ebuilds
in a local repository was inconvenient.  This new logic should permit
easy updates via make.conf.  Additionally, it has the advantage
of clearly reporting the changes made in the build logs.

This does not preclude other solutions to the problem.  However, this
is probably the best one and it should become the current
recommendation.


This will improve the overlay situation and can be seen as overall 
improvement but it doesn't address any shared concerns nor is it a 
replacement for the proposed 'acct-user.eclass: don't modify existing 
user by default' patch.



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



OpenPGP_signature
Description: OpenPGP digital signature

[gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

[James Cloos]

> "RHJ" == Robin H Johnson  writes:

RHJ> The best I can come up with at the moment, is that any packaging should
RHJ> detect if there are user modifications, and provide control to users
RHJ> based on that fact.

Exactly.  Akin to etc-update.

-JimC
-- 
James Cloos  OpenPGP: 0x997A9F17ED7DAEA6

[gentoo-dev] [PATCH] acct-user.eclass: Support var overrides for user properties

[Michał Górny]

Introduce a few variables to allow easy overrides of common user account
proprerties, that is:

- ACCT_USER__SHELL
- ACCT_USER__HOME
- ACCT_USER__HOME_OWNER
- ACCT_USER__HOME_PERMS
- ACCT_USER__GROUPS
- ACCT_USER__GROUPS_ADD

The first five variables override the respective ACCT_USER_* variables,
with ACCT_USER_*_GROUPS being a space-separated list.  The *_GROUPS_ADD
variable appends to groups present in the ebuild, as this seems a common
necessity.

We do realize that the original requirement of overriding ebuilds
in a local repository was inconvenient.  This new logic should permit
easy updates via make.conf.  Additionally, it has the advantage
of clearly reporting the changes made in the build logs.

This does not preclude other solutions to the problem.  However, this
is probably the best one and it should become the current
recommendation.

Signed-off-by: Michał Górny 
---
 eclass/acct-user.eclass | 84 +
 1 file changed, 60 insertions(+), 24 deletions(-)

diff --git a/eclass/acct-user.eclass b/eclass/acct-user.eclass
index 22b0038fbff7..5c55b8092c69 100644
--- a/eclass/acct-user.eclass
+++ b/eclass/acct-user.eclass
@@ -82,7 +82,8 @@ readonly ACCT_USER_NAME
 # @ECLASS-VARIABLE: ACCT_USER_SHELL
 # @DESCRIPTION:
 # The shell to use for the user.  If not specified, a 'nologin' variant
-# for the system is used.
+# for the system is used.  This can be overriden in make.conf through
+# ACCT_USER__SHELL variable.
 : ${ACCT_USER_SHELL:=-1}
 
 # @ECLASS-VARIABLE: ACCT_USER_HOME
@@ -90,6 +91,8 @@ readonly ACCT_USER_NAME
 # The home directory for the user.  If not specified, /dev/null is used.
 # The directory will be created with appropriate permissions if it does
 # not exist.  When updating, existing home directory will not be moved.
+# This can be overriden in make.conf through
+# ACCT_USER__HOME variable.
 : ${ACCT_USER_HOME:=/dev/null}
 
 # @ECLASS-VARIABLE: ACCT_USER_HOME_OWNER
@@ -97,11 +100,14 @@ readonly ACCT_USER_NAME
 # @DESCRIPTION:
 # The ownership to use for the home directory, in chown ([user][:group])
 # syntax.  Defaults to the newly created user, and its primary group.
+# This can be overriden in make.conf through
+# ACCT_USER__HOME_OWNER variable.
 
 # @ECLASS-VARIABLE: ACCT_USER_HOME_PERMS
 # @DESCRIPTION:
 # The permissions to use for the home directory, in chmod (octal
-# or verbose) form.
+# or verbose) form.  This can be overriden in make.conf through
+# ACCT_USER__HOME_PERMS variable.
 : ${ACCT_USER_HOME_PERMS:=0755}
 
 # @ECLASS-VARIABLE: ACCT_USER_GROUPS
@@ -110,6 +116,12 @@ readonly ACCT_USER_NAME
 # List of groups the user should belong to.  This must be a bash
 # array.  The first group specified is the user's primary group, while
 # the remaining groups (if any) become supplementary groups.
+#
+# This can be overriden in make.conf through
+# ACCT_USER__GROUPS variable, or appended to
+# via ACCT_USER__GROUPS_ADD.  Please note that
+# due to technical limitations, the override variables are not arrays
+# but space-separated lists.
 
 
 # << Boilerplate ebuild variables >>
@@ -316,23 +328,48 @@ acct-user_pkg_pretend() {
 acct-user_src_install() {
debug-print-function ${FUNCNAME} "${@}"
 
-   if [[ ${ACCT_USER_HOME} != /dev/null ]]; then
+   # serialize for override support
+   local ACCT_USER_GROUPS=${ACCT_USER_GROUPS[*]}
+
+   # support make.conf overrides
+   local override_name=${ACCT_USER_NAME^^}
+   override_name=${override_name//-/_}
+   local var
+   for var in ACCT_USER_{SHELL,HOME{,_OWNER,_PERMS},GROUPS}; do
+   local var_name=ACCT_USER_${override_name}_${var#ACCT_USER_}
+   if [[ -n ${!var_name} ]]; then
+   ewarn "${var_name}=${!var_name} override in effect, 
support will not be provided."
+   else
+   var_name=${var}
+   fi
+   declare -g "_${var}=${!var_name}"
+   done
+   var_name=ACCT_USER_${override_name}_GROUPS_ADD
+   if [[ -n ${!var_name} ]]; then
+   ewarn "${var_name}=${!var_name} override in effect, support 
will not be provided."
+   _ACCT_USER_GROUPS+=" ${!var_name}"
+   fi
+
+   # deserialize into an array
+   local groups=( ${_ACCT_USER_GROUPS} )
+
+   if [[ ${_ACCT_USER_HOME} != /dev/null ]]; then
# note: we can't set permissions here since the user isn't
# created yet
-   keepdir "${ACCT_USER_HOME}"
+   keepdir "${_ACCT_USER_HOME}"
fi
 
insinto /usr/lib/sysusers.d
newins - ${CATEGORY}-${ACCT_USER_NAME}.conf < <(
printf "u\t%q\t%q\t%q\t%q\t%q\n" \
"${ACCT_USER_NAME}" \
-   "${ACCT_USER_ID/#-*/-}:${ACCT_USER_GROUPS[0]}" \
+   "${ACCT_USER_ID/#-*/-}:${groups[0]}" \
"${DESCRIPTION//[:,=]/;}" \
-   "${ACCT_USER_HOME}" \
-   

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Mike Gilbert]

On Mon, Jan 4, 2021 at 11:50 AM Thomas Deutschmann  wrote:
>
> On 2021-01-04 17:38, Michał Górny wrote:
> > You've actually added 'portage' to group 'thomas'.
>
> Yes, I know that.
>
> Well, I understand why this might be confusing for you. Like I was using
> portage as example for the described example when you give another
> service access to a socket like shown in my memcached/redis example.

That's a really weird example, but if that was your intent please
disregard my last message.

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Michał Górny]

On Mon, 2021-01-04 at 17:50 +0100, Thomas Deutschmann wrote:
> On 2021-01-04 17:38, Michał Górny wrote:
> > You've actually added 'portage' to group 'thomas'.
> 
> Yes, I know that.
> 
> Well, I understand why this might be confusing for you. Like I was using 
> portage as example for the described example when you give another 
> service access to a socket like shown in my memcached/redis example.
> 
> 

This is confusing to me (as is probably to everybody else) because your
text is saying the exact opposite.

| Add your user to portage's group [...]

You're doing the exact opposite.

Maybe it's time to admit that you've made a mistake, and you've made
the same mistake the second time in a row, and although it was proven
wrong before you still keep trying to support your arguments with this
mistake.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Mike Gilbert]

On Mon, Jan 4, 2021 at 11:34 AM Thomas Deutschmann  wrote:
>
> On 2021-01-04 17:30, Thomas Deutschmann wrote:
> > On 2021-01-04 17:28, Michał Górny wrote:
> >> It must be a bug in your version of the eclass.  I've just reemerged
> >> acct-group/wheel and to*my great surprise*  I'm still there.  How
> >> unexpected!
> >
> > That's why I wrote
> >
> >  >  (luckily groups like wheel don't have users...)
> >
> > I meant that there is no acct-user/wheel because otherwise this group
> > would get cleaned (reset), too.
>
> Best example is portage. Follow handbook. Add your user to portage's group:
>
>  > usermod -aG  portage

I don't see any mention of usermod in the handbook, so I'm not sure
where this came from.

As mgorny pointed out, you are invoking usermod incorrectly. You want
this instead:

usermod -aG portage 

Don't use "id" to list group members. That lists groups of which a
user is a member. Use getent instead:

getent group portage

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

On 2021-01-04 17:38, Michał Górny wrote:

You've actually added 'portage' to group 'thomas'.


Yes, I know that.

Well, I understand why this might be confusing for you. Like I was using 
portage as example for the described example when you give another 
service access to a socket like shown in my memcached/redis example.



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



OpenPGP_signature
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Michał Górny]

On Mon, 2021-01-04 at 17:34 +0100, Thomas Deutschmann wrote:
> On 2021-01-04 17:30, Thomas Deutschmann wrote:
> > On 2021-01-04 17:28, Michał Górny wrote:
> > > It must be a bug in your version of the eclass.  I've just reemerged
> > > acct-group/wheel and to*my great surprise*  I'm still there.  How
> > > unexpected!
> > 
> > That's why I wrote
> > 
> >  >  (luckily groups like wheel don't have users...)
> > 
> > I meant that there is no acct-user/wheel because otherwise this group 
> > would get cleaned (reset), too.
> 
> Best example is portage. Follow handbook. Add your user to portage's group:
> 
>  > usermod -aG  portage

You've actually added 'portage' to group 'thomas'.


-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

On 2021-01-04 17:30, Thomas Deutschmann wrote:

On 2021-01-04 17:28, Michał Górny wrote:

It must be a bug in your version of the eclass.  I've just reemerged
acct-group/wheel and to*my great surprise*  I'm still there.  How
unexpected!


That's why I wrote

 >  (luckily groups like wheel don't have users...)

I meant that there is no acct-user/wheel because otherwise this group 
would get cleaned (reset), too.


Best example is portage. Follow handbook. Add your user to portage's group:

> usermod -aG  portage

Now re-emerge acct-user/portage...


# usermod -aG thomas portage
# id portage
uid=250(portage) gid=250(portage) groups=250(portage),1000(thomas)
# emerge -a1 acct-user/portage

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R] acct-user/portage-0::gentoo  0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] y

Verifying ebuild manifests
Running pre-merge checks for acct-user/portage-0
Emerging (1 of 1) acct-user/portage-0::gentoo
Installing (1 of 1) acct-user/portage-0::gentoo
Jobs: 1 of 1 complete
Auto-cleaning packages...



No outdated packages were found on your system.


 * GNU info directory index is up-to-date.
# id portage
uid=250(portage) gid=250(portage) groups=250(portage)



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



OpenPGP_signature
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

On 2021-01-04 17:28, Michał Górny wrote:

It must be a bug in your version of the eclass.  I've just reemerged
acct-group/wheel and to*my great surprise*  I'm still there.  How
unexpected!


That's why I wrote

>  (luckily groups like wheel don't have users...)

I meant that there is no acct-user/wheel because otherwise this group 
would get cleaned (reset), too.



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



OpenPGP_signature
Description: OpenPGP digital signature

[gentoo-dev] RFC: dropping support for uclibc-ng

[Anthony G. Basile]

Hi everyone,

I'd like feedback from people about the possibility of dropping support
for uclibc-ng.  If you are unfamiliar, its the successor to uclibc as a
C Standard Library for embedded systems, ie a replacement for glibc
bloat.  However, it is inferior to musl which serves the same purpose
and which has now well supported in Gentoo.

I know people want musl support, but does anyone even care about
uclibc-ng?  If not, I can work towards deprecating it and putting what
little time I have towards musl.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Michał Górny]

On Mon, 2021-01-04 at 17:18 +0100, Thomas Deutschmann wrote:
> On 2021-01-04 16:55, David Seifert wrote:
> > This is what we agree on. We need an escape hatch, and it needs to be
> > off by default. Any sysadmin overriding it gets to keep the pieces, but
> > they need to have that option.
> 
> See Mike's example again.
> 
> In last chapter of Gentoo's handbook (Finalization) we recommend user to 
> call 'usermod' to put themselves into important groups like wheel or 
> portage.
> 
> Now guess what's happening? Whenever acct-user/portage will get 
> remerged, PM will remove that user from portage group (luckily groups 
> like wheel don't have users...).

It must be a bug in your version of the eclass.  I've just reemerged
acct-group/wheel and to *my great surprise* I'm still there.  How
unexpected!


-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

On 2021-01-04 17:14, Michał Górny wrote:

as long as it spews a big fat ewarn that the user loses the right to
support.


Could you please elaborate this a little bit more? I cannot agree with 
the way I understand this at the moment but I might miss your point.



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



OpenPGP_signature
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

On 2021-01-04 16:55, David Seifert wrote:

This is what we agree on. We need an escape hatch, and it needs to be
off by default. Any sysadmin overriding it gets to keep the pieces, but
they need to have that option.


See Mike's example again.

In last chapter of Gentoo's handbook (Finalization) we recommend user to 
call 'usermod' to put themselves into important groups like wheel or 
portage.


Now guess what's happening? Whenever acct-user/portage will get 
remerged, PM will remove that user from portage group (luckily groups 
like wheel don't have users...).


Do you really want to extend handbook and tell everyone, "OK, as last 
step, please create an overlay and fork acct-user/portage...". In case 
the answer will be yes, we now have successfully killed the idea of 
allowing maintainers to fix a user/group if this will ever be necessary 
which will add some kind of slap stick to the whole idea.


That's why I am saying that we don't just need an opt-out option, that's 
why I am argue that all this stuff has to be opt-in by default. It's 
something special and unique in Gentoo.



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



OpenPGP_signature
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Michał Górny]

On Mon, 2021-01-04 at 11:10 -0500, Mike Gilbert wrote:
> On Mon, Jan 4, 2021 at 4:23 AM Michał Górny  wrote:
> > 
> > On Mon, 2021-01-04 at 02:35 +0100, Thomas Deutschmann wrote:
> > > Modifying an existing user is a bad default and makes Gentoo
> > > special because it is common for system administrators to make
> > > modifications to user (i.e. putting an user into another service's
> > > group to allow that user to access service in question) and it
> > > would be unexpected to see these changes reverted during normal
> > > world upgrade (which could break services).
> > 
> > Not modifying an existing user is a horrible default that has already
> > bricked one system (by removing /dev/null).  So, over my dead commit
> > access.
> 
> As the eclass maintainer, would you be willing to merge a similar
> patch that enables user modifications by default, but provides
> sysadmins a way to disable it?

Yes, I don't mind an option, as long as it spews a big fat ewarn that
the user loses the right to support.  However, that's still not
the right solution to the immediate problem, and I'm currently working
on a better patch, so I'd prefer if you waited with that to avoid merge
conflicts.


-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Mike Gilbert]

On Mon, Jan 4, 2021 at 4:23 AM Michał Górny  wrote:
>
> On Mon, 2021-01-04 at 02:35 +0100, Thomas Deutschmann wrote:
> > Modifying an existing user is a bad default and makes Gentoo
> > special because it is common for system administrators to make
> > modifications to user (i.e. putting an user into another service's
> > group to allow that user to access service in question) and it
> > would be unexpected to see these changes reverted during normal
> > world upgrade (which could break services).
>
> Not modifying an existing user is a horrible default that has already
> bricked one system (by removing /dev/null).  So, over my dead commit
> access.

As the eclass maintainer, would you be willing to merge a similar
patch that enables user modifications by default, but provides
sysadmins a way to disable it?

I have a feeling that there will not be a consensus on the default
behavior, and I could see that getting escalated to council. However,
it might be nice to provide people with the option in the meantime.

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[David Seifert]

On Mon, 2021-01-04 at 10:24 -0500, Michael Orlitzky wrote:
> I understand that creating an overlay with acct-user overrides will
> not 
> be for everyone, so I have no problem with adding an escape hatch. I
> do 
> think it should be off by default though, and that missing future 
> ::gentoo changes will not be a problem unless some other error has
> been 
> committed first.

This is what we agree on. We need an escape hatch, and it needs to be
off by default. Any sysadmin overriding it gets to keep the pieces, but
they need to have that option.

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Michael Orlitzky]

On 1/4/21 9:46 AM, Thomas Deutschmann wrote:



So the main problem I see with doing this is that it becomes
impossible to reliably make changes to a user in later ebuild
revisions.


He is obviously looking for a way to allow maintainers to change users
afterwards. But if we tell people, "If you need customization, fork the
user/group ebuild in your overlay" we will disconnect these users from
future changes.


There's pretty much no reason to change a user's settings unless you've 
completely fucked them up the first time around. This is precisely why 
the original GLEP had mailing list reviews.


If a package depends on a user having e.g. a specific home directory so 
that upgrading the package requires a corresponding revision bump of the 
user, then the package is broken. I tried really hard to document this, 
and to enforce it back when we had mailing list reviews. It's all in the 
devmanual now.




...
When you will get LPIC certification one can expect that you have some
basic knowledge in Linux stuff allowing you to do common tasks on all
different Linux systems. Now there comes Gentoo where you aren't allowed
to use standard Linux tools like 'usermod' when deploying another
service if you don't want to risk that your service will go down when
following best practice and do regular world upgrades. Really?


You also can't use the standard linux tools to edit scripts in /usr/bin 
without your changes being overwritten. This is no different... some 
things need to belong to the package manager if you want package 
management to work.




3) More important, the idea of forking acct-* packages whenever you need
to make modifications don't scale. Like I already outlined in February
2020, you cannot create overlays for each different user configuration:

I.e. using memcached/redis: You grant permission to socket via group. So
you put other services belonging to that application you are deploying
into your user running the key value store. Do you really expect that
one would create multiple overlays per application using one of these
services? How would you maintain hundreds of overlays? How would you
keep track that each box will use the correct overlay to get the
specific customized acct-* package? How do you deal with scenarios where
you don't just deploy single instances?


This is literally the example I gave. Our acct-user ebuilds can be added 
to additional groups based on USE flags. Every server uses the same 
overlay/ebuilds, but different machines get different package.use files, 
pushed out by the configuration management tool.


I understand that creating an overlay with acct-user overrides will not 
be for everyone, so I have no problem with adding an escape hatch. I do 
think it should be off by default though, and that missing future 
::gentoo changes will not be a problem unless some other error has been 
committed first.

[gentoo-dev] [News review v3] LibreSSL support discontinued

[Michał Górny]

The third version follows, with requested text changes and '--deep'
option added to rebuilds.

---
Title: LibreSSL support discontinued
Author: Michał Górny 
Posted: 202x-xx-xx
Revision: 1
News-Item-Format: 2.0
Display-If-Installed: dev-libs/libressl

Starting 2021-02-01, Gentoo will discontinue supporting
dev-libs/libressl as an alternative to dev-libs/openssl.  While it will
still be possible for expert users to use LibreSSL on their systems,
we are only going to provide support for OpenSSL-based systems.  Most
importantly, we are no longer going to maintain downstream patches for
LibreSSL support -- it will rely on either package upstreams merging
such patches themselves, or LibreSSL upstream finally working towards
better OpenSSL compatibility.

On 2021-02-01, we will mask the relevant USE flags and packages.  If
you
wish to continue using LibreSSL, you will be able to undo these masks
for the time being.  However, as packages drop patching for LibreSSL
and the library is eventually removed from ::gentoo, it will become
necessary to use the user-maintained LibreSSL overlay [1].  As long-
term
support for LibreSSL is not guaranteed, we recommend switching
to OpenSSL instead.  More information on removal can be found
on the relevant bug [2].

To switch before the aforementioned date, remove 'libressl' from your
USE flags and CURL_SSL targets.  Afterwards, it is recommended to
prefetch all the necessary distfiles before proceeding with the system
upgrade, in case wget(1) becomes broken in the process:

emerge --fetchonly dev-libs/openssl net-misc/wget
emerge --fetchonly --deep --changed-use @world

A --changed-use @world upgrade should automatically cause LibreSSL
to be replaced by OpenSSL, and all affected packages to be rebuilt:

emerge --deselect dev-libs/libressl
emerge --changed-use --deep @world


LibreSSL has been forked off OpenSSL in 2014 to address a number of
problems with the original package.  However, since then OpenSSL
development gained speed and the original reasons for the fork no
longer
apply.  Furthermore, LibreSSL started to repeatedly fall behind
and cause growing compatibility problems.  While initially these
problems were related to packages using old/insecure OpenSSL APIs,
today
they are mostly related to LibreSSL missing newer OpenSSL APIs
(yet declaring false compatibility with newer OpenSSL versions).

With the little testing it gets, our developers and users had to put
a significant effort into fixing upstream packages.  In some cases
(e.g. Qt), upstream has explicitly refused to support LibreSSL, forcing
us to maintain the patches forever.  This in turn means that
security fixes, regular version bumps or end-user system upgrades are
often delayed because of necessary LibreSSL patching.  What is even
worse, major runtime issues managed to sneak in that broke production
systems running LibreSSL in the past.

To the best of our knowledge, the only benefit LibreSSL has over
OpenSSL
right now is the additional libtls library.  For this reason, we have
packaged dev-libs/libretls which is a port of this library that links
to OpenSSL.

All these issues considered, we came to the conclusion that OpenSSL
should remain the only supported production option for Gentoo systems.
While the flexibility of Gentoo should make it possible to keep using
LibreSSL going forward, the effort necessary to provide first-class
official support for LibreSSL has proven to outweigh the benefit.

[1] https://gitweb.gentoo.org/repo/proj/libressl.git/tree/README.md
[2] https://bugs.gentoo.org/762847
---

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [News review] LibreSSL support discontinued

[Michał Górny]

On Mon, 2021-01-04 at 15:40 +0100, Marc Schiffbauer wrote:
> Just a typo...
> 
> * Michał Górny schrieb am 03.01.21 um 21:47 Uhr:
> > All these issued considered, we came to the conclusion that OpenSSL
> 
> s/issued/issues/
> 
> right?
> 

Thanks, done.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

Hi,

On 2021-01-04 04:18, Michael Orlitzky wrote:
It would be nice if this was well-supported by the official way of 
modifying system users/groups; that is, by using an overlay with 
modified user/group ebuilds.


No, this doesn't work:

1) It's conflicting with the goals other have. See Mike's first reply to 
my patch proposal:



So the main problem I see with doing this is that it becomes
impossible to reliably make changes to a user in later ebuild
revisions.


He is obviously looking for a way to allow maintainers to change users 
afterwards. But if we tell people, "If you need customization, fork the 
user/group ebuild in your overlay" we will disconnect these users from 
future changes.



2) Thank you for outlining the process how to make changes to users 
using the new acct-* way. It's a nice 'hack'. But it is an own, new way, 
which makes Gentoo special, unique. And this creates additional problems:


This maybe work for your local system. In environments where everything 
is Gentoo and everyone knows Gentoo. But in today's world you are using 
configuration management tools like Ansible, Puppet, Saltstack, Chef...


People who are not necessarily familiar with every implementation detail 
must be able to write configurations (recipes) and the used tool is 
supposed to take care of the differences. In the end, in the ideal 
world, you are just looking at your code describing a state the system 
should have.


But this doesn't work if you make Gentoo so special that you will break 
most tools, will require adjustments or special Gentoo support just for 
stuff Gentoo is doing differently and like everyone else.


Don't get me wrong here: Yes, these tools already have to implement USE 
flags for example which are unique for Gentoo. But stuff like user 
management isn't or should *not*.


When you will get LPIC certification one can expect that you have some 
basic knowledge in Linux stuff allowing you to do common tasks on all 
different Linux systems. Now there comes Gentoo where you aren't allowed 
to use standard Linux tools like 'usermod' when deploying another 
service if you don't want to risk that your service will go down when 
following best practice and do regular world upgrades. Really?



3) More important, the idea of forking acct-* packages whenever you need 
to make modifications don't scale. Like I already outlined in February 
2020, you cannot create overlays for each different user configuration:


I.e. using memcached/redis: You grant permission to socket via group. So 
you put other services belonging to that application you are deploying 
into your user running the key value store. Do you really expect that 
one would create multiple overlays per application using one of these 
services? How would you maintain hundreds of overlays? How would you 
keep track that each box will use the correct overlay to get the 
specific customized acct-* package? How do you deal with scenarios where 
you don't just deploy single instances?


Again, I understand the acct-* fork idea. But I think we have to admit 
that this will only work for the local system or small environments.


For any professional environment this won't work nor scale.


4) Yet we have to talk about the idea in general that it is really not a 
good idea to automatically make changes to the user if you run the risk 
of overwriting changes explicitly made by the system administrator.


It's one thing that multiple local users will get removed from portage 
group when you remerge acct-user/portage, but if you kill services 
because package maintainers are pushing their vision of how to run the 
package, it's not.



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5

Re: [gentoo-dev] [News review] LibreSSL support discontinued

[Marc Schiffbauer]

Just a typo...

* Michał Górny schrieb am 03.01.21 um 21:47 Uhr:
> All these issued considered, we came to the conclusion that OpenSSL

s/issued/issues/

right?



signature.asc
Description: PGP signature

Re: [gentoo-dev] [News review v2] LibreSSL support discontinued

[Michał Górny]

On Mon, 2021-01-04 at 09:24 -0500, Aaron Bauman wrote:
> On Mon, Jan 04, 2021 at 10:21:58AM +0100, Michał Górny wrote:
> > v2, with additional 'emerge --deselect':
> > ---
> > Title: LibreSSL support discontinued
> > Author: Michał Górny 
> > Posted: 202x-xx-xx
> > Revision: 1
> > News-Item-Format: 2.0
> > Display-If-Installed: dev-libs/libressl
> > 
> > Starting 2021-02-01, Gentoo will no longer actively pursue supporting
> 
> s/no longer actively pursue/discontinue

Applied locally.  I'll resend if there are bigger changes.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [News review v2] LibreSSL support discontinued

[Aaron Bauman]

On Mon, Jan 04, 2021 at 10:21:58AM +0100, Michał Górny wrote:
> v2, with additional 'emerge --deselect':
> ---
> Title: LibreSSL support discontinued
> Author: Michał Górny 
> Posted: 202x-xx-xx
> Revision: 1
> News-Item-Format: 2.0
> Display-If-Installed: dev-libs/libressl
> 
> Starting 2021-02-01, Gentoo will no longer actively pursue supporting

s/no longer actively pursue/discontinue

-Aaron


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Thomas Deutschmann]

On 2021-01-04 10:23, Michał Górny wrote:

Not modifying an existing user is a horrible default that has already
bricked one system (by removing /dev/null).  So, over my dead commit
access.


Have you seen how many user were hit caused by the recent rebuilt on 
2020-12-28 and are already complaining/asking for help through various 
channels?


It's like asking for service auto-restart support in PMS as requested as 
part of current OpenSSH upgrade because if you move from <8.3_p1 to 
>=8.3_p1 and don't restart OpenSSH in time, you can get locked out.


However, an easily looking solution like


Just add something like

if [[ -d /run/systemd/system ]]; then
systemctl try-restart sshd
else
rc-service -q --ifstarted sshd restart
fi

to pkg_postinst


is wrong because even if it works for *some* users it won't work for all 
users but has the potential to cause major problems.


That's why we have elog and newitem system. However, 8.3 is in 
repository for while and multiple people forgot about the newitem and 
didn't pay attention to elog messages. While I agree that it's a problem 
when you lose access to a remote box you don't have physical access to, 
this reached a level where I have to say,


> We cannot rescue/protect everyone.

Back to topic, acct-* stuff:

Like already said in February 2020 when I joined a thread created by a 
user posting same concerns:


There is a reason why *no* distribution on this planet is trying to mess 
with existing data/configurations: Every attempt trying to analyze given 
setup to apply required changes to fix/migrate something automatically 
has been prone to fail the long run.


Please get some experience from real world. Preferable from running 
headless systems not just for yourself and where you are not the only 
person touching the system.


When I worked on bug 605008 long time ago for example, I also ended up 
over-engineering. There is stuff you cannot fix. I am still thinking 
about creating everything the way it should look like in $D and report 
any difference like changed file permissions to user on merge to allow 
them to notice (an improvement, now user only have to pay attention and 
you need to solve the additional problem that the more information you 
present all the time, the more information will be ignored). But 
sometimes users are making changes we wouldn't do, not recommend or just 
don't understand at first. That all doesn't matter: We have to keep in 
mind that these aren't our systems and we have to respect whatever the 
user did on their system.



--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5

Re: [gentoo-dev] [News review v2] LibreSSL support discontinued

[Oliver Smeeton]

OK my mistake I'll send my suggestion to libre...@gentoo.org

On Mon, 4 Jan 2021 at 13:47, Toralf Förster  wrote:

> On 1/4/21 2:39 PM, Oliver Smeeton wrote:
> > You may want to update the Project:LibreSSL
> >  page to reflect the
> > decision to drop support for libressl,
> IMO this is up to the project members only, or ?
>
> --
> Toralf
> PGP 23217DA7 9B888F45
>
>

Re: [gentoo-dev] [News review v2] LibreSSL support discontinued

[Toralf Förster]

On 1/4/21 2:39 PM, Oliver Smeeton wrote:
You may want to update the Project:LibreSSL 
 page to reflect the 
decision to drop support for libressl,

IMO this is up to the project members only, or ?

--
Toralf
PGP 23217DA7 9B888F45



OpenPGP_signature
Description: OpenPGP digital signature

Re: [gentoo-dev] [News review v2] LibreSSL support discontinued

[Oliver Smeeton]

You may want to update the Project:LibreSSL
 page to reflect the
decision to drop support for libressl, also you could add a news item to
the libressl package with instructions or a link to instructions for
migrating back to Openssl.

On Mon, 4 Jan 2021 at 09:22, Michał Górny  wrote:

> v2, with additional 'emerge --deselect':
> ---
> Title: LibreSSL support discontinued
> Author: Michał Górny 
> Posted: 202x-xx-xx
> Revision: 1
> News-Item-Format: 2.0
> Display-If-Installed: dev-libs/libressl
>
> Starting 2021-02-01, Gentoo will no longer actively pursue supporting
> dev-libs/libressl as an alternative to dev-libs/openssl.  While it will
> still be possible for expert users to use LibreSSL on their systems,
> we are only going to provide support for OpenSSL-based systems.  Most
> importantly, we are no longer going to maintain downstream patches for
> LibreSSL support -- it will rely on either package upstreams merging
> such patches themselves, or LibreSSL upstream finally working towards
> better OpenSSL compatibility.
>
> On 2021-02-01, we will mask the relevant USE flags and packages.  If
> you
> wish to continue using LibreSSL, you will be able to undo these masks
> for the time being.  However, as packages drop patching for LibreSSL
> and the library is eventually removed from ::gentoo, it will become
> necessary to use the user-maintained LibreSSL overlay [1].  As long-
> term
> support for LibreSSL is not guaranteed, we recommend switching
> to OpenSSL instead.  More information on removal can be found
> on the relevant bug [2].
>
> To switch before the aforementioned date, remove 'libressl' from your
> USE flags and CURL_SSL targets.  Afterwards, it is recommended to
> prefetch all the necessary distfiles before proceeding with the system
> upgrade, in case wget(1) becomes broken in the process:
>
> emerge --fetchonly dev-libs/openssl net-misc/wget
> emerge --fetchonly --changed-use @world
>
> A --changed-use @world upgrade should automatically cause LibreSSL
> to be replaced by OpenSSL, and all affected packages to be rebuilt:
>
> emerge --deselect dev-libs/libressl
> emerge --changed-use @world
>
>
> LibreSSL has been forked off OpenSSL in 2014 to address a number of
> problems with the original package.  However, since then OpenSSL
> development gained speed and the original reasons for the fork no
> longer
> apply.  Furthermore, LibreSSL started to repeatedly fall behind
> and cause growing compatibility problems.  While initially these
> problems were related to packages using old/insecure OpenSSL APIs,
> today
> they are mostly related to LibreSSL missing newer OpenSSL APIs
> (yet declaring false compatibility with newer OpenSSL versions).
>
> With the little testing it gets, our developers and users had to put
> a significant effort into fixing upstream packages.  In some cases
> (e.g. Qt), upstream has explicitly refused to support LibreSSL, forcing
> us to maintain the patches forever.  This in turn means that
> security fixes, regular version bumps or end-user system upgrades are
> often delayed because of necessary LibreSSL patching.  What is even
> worse, major runtime issues managed to sneak in that broke production
> systems running LibreSSL in the past.
>
> To the best of our knowledge, the only benefit LibreSSL has over
> OpenSSL
> right now is the additional libtls library.  For this reason, we have
> packaged dev-libs/libretls which is a port of this library that links
> to OpenSSL.
>
> All these issued considered, we came to the conclusion that OpenSSL
> should remain the only supported production option for Gentoo systems.
> While the flexibility of Gentoo should make it possible to keep using
> LibreSSL going forward, the effort necessary to provide first-class
> official support for LibreSSL has proven to outweigh the benefit.
>
> [1] https://gitweb.gentoo.org/repo/proj/libressl.git/tree/README.md
> [2] https://bugs.gentoo.org/762847
> ---
>
>
>
>
> --
> Best regards,
> Michał Górny
>
>
>
>

[gentoo-dev] [PATCH] eclass/lua-utils.eclass: Add support for test-runners

[Conrad Kostecki]

During migration of dev-lua/* ebuilds to slotted lua, I noticed, that
many ebuilds use 'dev-lua/busted' for running tests. This change adds
support for running a test-runner, at first only 'busted' for now.
Also a non-color and plaintext output will be used for the test-runner 'busted'.

This is basically a copy of the test-runner section, written by mgorny,
which already exists in 'distutils-r1', but modified and adapted to lua.

In order to use this feature, you can define 'lua_enable_tests busted'
to setup everything needed for tests and run them. By default,
'dev-lua/busted' assumes, that tests are in the 'spec' folder.

If this is not the case, you can add a second argument to specify a
different folder. For example, if the folder is called 'foo', you can
just run 'lua_enable_tests busted foo'.

More test-runners can be added in future, if needed.

Signed-off-by: Conrad Kostecki 
---
 eclass/lua-utils.eclass | 73 +
 1 file changed, 73 insertions(+)

diff --git a/eclass/lua-utils.eclass b/eclass/lua-utils.eclass
index 100be14cb08..b24c4a461f2 100644
--- a/eclass/lua-utils.eclass
+++ b/eclass/lua-utils.eclass
@@ -344,6 +344,79 @@ _lua_export() {
done
 }
 
+# @FUNCTION: lua_enable_tests
+# @USAGE:  
+# @DESCRIPTION:
+# Set up IUSE, RESTRICT, BDEPEND and src_test() for running tests
+# with the specified test runner.  Also copies the current value
+# of RDEPEND to test?-BDEPEND.  The test-runner argument must be one of:
+#
+# - busted: dev-lua/busted
+#
+# Additionally, a second argument can be passed after ,
+# so  will use that directory to search for tests.
+# If not passed, a default directory of  will be used.
+#
+# - busted: spec
+#
+# This function is meant as a helper for common use cases, and it only
+# takes care of basic setup.  You still need to list additional test
+# dependencies manually.  If you have uncommon use case, you should
+# not use it and instead enable tests manually.
+#
+# This function must be called in global scope, after RDEPEND has been
+# declared.  Take care not to overwrite the variables set by it.
+lua_enable_tests() {
+   debug-print-function ${FUNCNAME} "${@}"
+
+   [[ ${#} -ge 1 ]] || die "${FUNCNAME} takes at least one argument: 
test-runner (test-directory)"
+   local test_pkg
+   case ${1} in
+   busted)
+   test_directory="${2:-spec}"
+   test_pkg="dev-lua/busted"
+   if [[ ! ${_LUA_SINGLE_R0} ]]; then
+   lua_src_test() {
+   busted --lua="${ELUA}" 
--output="plainTerminal" "${test_directory}" || die "Tests fail with ${ELUA}"
+   }
+   src_test() {
+   lua_foreach_impl lua_src_test
+   }
+   else
+   src_test() {
+   busted --lua="${ELUA}" 
--output="plainTerminal" "${test_directory}" || die "Tests fail with ${ELUA}"
+   }
+   fi
+   ;;
+   *)
+   die "${FUNCNAME}: unsupported argument: ${1}"
+   esac
+
+   local test_deps=${RDEPEND}
+   if [[ -n ${test_pkg} ]]; then
+   if [[ ! ${_LUA_SINGLE_R0} ]]; then
+   test_deps+=" ${test_pkg}[${LUA_USEDEP}]"
+   else
+   test_deps+=" $(lua_gen_cond_dep "
+   ${test_pkg}[\${LUA_USEDEP}]
+   ")"
+   fi
+   fi
+   if [[ -n ${test_deps} ]]; then
+   IUSE+=" test"
+   RESTRICT+=" !test? ( test )"
+   if [[ ${EAPI} == [56] ]]; then
+   DEPEND+=" test? ( ${test_deps} )"
+   else
+   BDEPEND+=" test? ( ${test_deps} )"
+   fi
+   fi
+
+   # we need to ensure successful return in case we're called last,
+   # otherwise Portage may wrongly assume sourcing failed
+   return 0
+}
+
 # @FUNCTION: lua_get_CFLAGS
 # @USAGE: []
 # @DESCRIPTION:
-- 
2.29.2

[gentoo-dev] [PATCH 5/5] linux-mod.eclass: Add a prebuilt-kernel rebuild flag

[Michał Górny]

Add a 'prebuilt-kernel' USE flag that controls a bound-slot dependency
on virtual/prebuilt-kernel.  If the flag is enabled, it should ensure
that the module will be rebuilt automatically on future kernel upgrades.

TODO: unmask the flag on other architectures

Signed-off-by: Michał Górny 
---
 eclass/linux-mod.eclass | 13 ++---
 profiles/arch/amd64/use.mask|  4 
 profiles/arch/amd64/use.stable.mask |  6 +-
 profiles/arch/base/use.mask |  6 +-
 profiles/use.desc   |  1 +
 5 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/eclass/linux-mod.eclass b/eclass/linux-mod.eclass
index f5a5a4276369..e0444eacc61a 100644
--- a/eclass/linux-mod.eclass
+++ b/eclass/linux-mod.eclass
@@ -1,4 +1,4 @@
-# Copyright 1999-2020 Gentoo Authors
+# Copyright 1999-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: linux-mod.eclass
@@ -144,9 +144,16 @@ esac
0) die "EAPI=${EAPI} is not supported with 
MODULES_OPTIONAL_USE_IUSE_DEFAULT due to lack of IUSE defaults" ;;
 esac
 
-IUSE="kernel_linux 
${MODULES_OPTIONAL_USE:+${_modules_optional_use_iuse_default}}${MODULES_OPTIONAL_USE}"
+IUSE="kernel_linux dist-kernel
+   
${MODULES_OPTIONAL_USE:+${_modules_optional_use_iuse_default}}${MODULES_OPTIONAL_USE}"
 SLOT="0"
-RDEPEND="${MODULES_OPTIONAL_USE}${MODULES_OPTIONAL_USE:+? (} kernel_linux? ( 
sys-apps/kmod[tools] ) ${MODULES_OPTIONAL_USE:+)}"
+RDEPEND="
+   ${MODULES_OPTIONAL_USE}${MODULES_OPTIONAL_USE:+? (}
+   kernel_linux? (
+   sys-apps/kmod[tools]
+   dist-kernel? ( virtual/dist-kernel:= )
+   )
+   ${MODULES_OPTIONAL_USE:+)}"
 DEPEND="${RDEPEND}
 ${MODULES_OPTIONAL_USE}${MODULES_OPTIONAL_USE:+? (}
sys-apps/sed
diff --git a/profiles/arch/amd64/use.mask b/profiles/arch/amd64/use.mask
index 2334ab319381..e4f5aa002053 100644
--- a/profiles/arch/amd64/use.mask
+++ b/profiles/arch/amd64/use.mask
@@ -6,6 +6,10 @@
 
 # SECTION: Unmask
 
+# Michał Górny  (2021-01-03)
+# Prebuilt kernels are supported here.
+-dist-kernel
+
 # Andreas Sturmlechner  (2020-06-06)
 # sci-libs/mkl works on amd64
 -mkl
diff --git a/profiles/arch/amd64/use.stable.mask 
b/profiles/arch/amd64/use.stable.mask
index b83b8507d91c..07bd75f7024c 100644
--- a/profiles/arch/amd64/use.stable.mask
+++ b/profiles/arch/amd64/use.stable.mask
@@ -1,9 +1,13 @@
-# Copyright 1999-2019 Gentoo Authors
+# Copyright 1999-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # This file requires eapi 5 or later. New entries go on top.
 # Please use the same syntax as in use.mask
 
+# Michał Górny  (2021-01-03)
+# Prebuilt kernel rebuilds are supported on stable yet.
+dist-kernel
+
 # Andreas Sturmlechner  (2020-06-06)
 # sci-libs/mkl is not stable, needs online registration to even run pkg_setup
 mkl
diff --git a/profiles/arch/base/use.mask b/profiles/arch/base/use.mask
index ba124ae0487a..704b33512f21 100644
--- a/profiles/arch/base/use.mask
+++ b/profiles/arch/base/use.mask
@@ -1,6 +1,10 @@
-# Copyright 1999-2020 Gentoo Authors
+# Copyright 1999-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
+# Michał Górny  (2021-01-03)
+# Prebuilt kernels are not supported on all architectures.
+dist-kernel
+
 # Andreas Sturmlechner  (2020-06-06)
 # sci-libs/mkl is only supported on specific architectures
 mkl
diff --git a/profiles/use.desc b/profiles/use.desc
index e51db5982020..a1ea2238b3ee 100644
--- a/profiles/use.desc
+++ b/profiles/use.desc
@@ -62,6 +62,7 @@ dbus - Enable dbus support for anything that needs it (gpsd, 
gnomemeeting, etc)
 debug - Enable extra debug codepaths, like asserts and extra output. If you 
want to get meaningful backtraces see 
https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
 dedicated - Add support for dedicated game servers (some packages do not 
provide clients and servers at the same time)
 dga - Add DGA (Direct Graphic Access) support for X
+dist-kernel - Enable subslot rebuilds on Distribution Kernel upgrades
 djvu - Support DjVu, a PDF-like document format esp. suited for scanned 
documents
 doc - Add extra documentation (API, Javadoc, etc). It is recommended to enable 
per package instead of globally
 dri - Enable direct rendering: used for accelerated 3D and some 2D, like DMA
-- 
2.30.0

[gentoo-dev] [PATCH 4/5] sys-kernel/vanilla-kernel: Add a PDEP on virtual/prebuilt-kernel

[Michał Górny]

Signed-off-by: Michał Górny 
---
 sys-kernel/vanilla-kernel/vanilla-kernel-5.10.4.ebuild | 2 ++
 sys-kernel/vanilla-kernel/vanilla-kernel-5.4.86.ebuild | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/sys-kernel/vanilla-kernel/vanilla-kernel-5.10.4.ebuild 
b/sys-kernel/vanilla-kernel/vanilla-kernel-5.10.4.ebuild
index cc953811ce28..7d34d6e93d00 100644
--- a/sys-kernel/vanilla-kernel/vanilla-kernel-5.10.4.ebuild
+++ b/sys-kernel/vanilla-kernel/vanilla-kernel-5.10.4.ebuild
@@ -48,6 +48,8 @@ RDEPEND="
 BDEPEND="
debug? ( dev-util/dwarves )
verify-sig? ( app-crypt/openpgp-keys-kernel )"
+PDEPEND="
+   >=virtual/dist-kernel-${PV}"
 
 VERIFY_SIG_OPENPGP_KEY_PATH=${BROOT}/usr/share/openpgp-keys/kernel.org.asc
 
diff --git a/sys-kernel/vanilla-kernel/vanilla-kernel-5.4.86.ebuild 
b/sys-kernel/vanilla-kernel/vanilla-kernel-5.4.86.ebuild
index 7f4ea0861252..eedfdb40bc65 100644
--- a/sys-kernel/vanilla-kernel/vanilla-kernel-5.4.86.ebuild
+++ b/sys-kernel/vanilla-kernel/vanilla-kernel-5.4.86.ebuild
@@ -46,6 +46,8 @@ RDEPEND="
 BDEPEND="
debug? ( dev-util/dwarves )
verify-sig? ( app-crypt/openpgp-keys-kernel )"
+PDEPEND="
+   >=virtual/dist-kernel-${PV}"
 
 VERIFY_SIG_OPENPGP_KEY_PATH=${BROOT}/usr/share/openpgp-keys/kernel.org.asc
 
-- 
2.30.0

[gentoo-dev] [PATCH 3/5] sys-kernel/gentoo-kernel-bin: Add a PDEP on virtual/prebuilt-kernel

[Michał Górny]

Signed-off-by: Michał Górny 
---
 sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4-r1.ebuild | 2 ++
 sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4.ebuild| 2 ++
 sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86-r1.ebuild | 2 ++
 sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86.ebuild| 2 ++
 4 files changed, 8 insertions(+)

diff --git a/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4-r1.ebuild 
b/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4-r1.ebuild
index 1b430cf56222..7be52665980a 100644
--- a/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4-r1.ebuild
+++ b/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4-r1.ebuild
@@ -22,6 +22,8 @@ RDEPEND="
!sys-kernel/gentoo-kernel:${SLOT}
!sys-kernel/vanilla-kernel:${SLOT}
!sys-kernel/vanilla-kernel-bin:${SLOT}"
+PDEPEND="
+   >=virtual/dist-kernel-${PV}"
 
 QA_PREBUILT='*'
 
diff --git a/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4.ebuild 
b/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4.ebuild
index 68ca59b57351..f663ccda5cc5 100644
--- a/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4.ebuild
+++ b/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.10.4.ebuild
@@ -26,6 +26,8 @@ RDEPEND="
!sys-kernel/gentoo-kernel:${SLOT}
!sys-kernel/vanilla-kernel:${SLOT}
!sys-kernel/vanilla-kernel-bin:${SLOT}"
+PDEPEND="
+   >=virtual/dist-kernel-${PV}"
 
 QA_PREBUILT='*'
 
diff --git a/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86-r1.ebuild 
b/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86-r1.ebuild
index 2378adc3f352..91ed43da6b79 100644
--- a/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86-r1.ebuild
+++ b/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86-r1.ebuild
@@ -30,6 +30,8 @@ RDEPEND="
!sys-kernel/gentoo-kernel:${SLOT}
!sys-kernel/vanilla-kernel:${SLOT}
!sys-kernel/vanilla-kernel-bin:${SLOT}"
+PDEPEND="
+   >=virtual/dist-kernel-${PV}"
 
 QA_PREBUILT='*'
 
diff --git a/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86.ebuild 
b/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86.ebuild
index d6e61fa69369..2f563a9ece0d 100644
--- a/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86.ebuild
+++ b/sys-kernel/gentoo-kernel-bin/gentoo-kernel-bin-5.4.86.ebuild
@@ -26,6 +26,8 @@ RDEPEND="
!sys-kernel/gentoo-kernel:${SLOT}
!sys-kernel/vanilla-kernel:${SLOT}
!sys-kernel/vanilla-kernel-bin:${SLOT}"
+PDEPEND="
+   >=virtual/dist-kernel-${PV}"
 
 QA_PREBUILT='*'
 
-- 
2.30.0

[gentoo-dev] [PATCH 2/5] sys-kernel/gentoo-kernel: Add a PDEP on virtual/prebuilt-kernel

[Michał Górny]

Signed-off-by: Michał Górny 
---
 sys-kernel/gentoo-kernel/gentoo-kernel-5.10.4.ebuild | 2 ++
 sys-kernel/gentoo-kernel/gentoo-kernel-5.4.86.ebuild | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/sys-kernel/gentoo-kernel/gentoo-kernel-5.10.4.ebuild 
b/sys-kernel/gentoo-kernel/gentoo-kernel-5.10.4.ebuild
index 5ec7b92ee625..7bf6ab8f32ed 100644
--- a/sys-kernel/gentoo-kernel/gentoo-kernel-5.10.4.ebuild
+++ b/sys-kernel/gentoo-kernel/gentoo-kernel-5.10.4.ebuild
@@ -47,6 +47,8 @@ RDEPEND="
!sys-kernel/vanilla-kernel-bin:${SLOT}"
 BDEPEND="
debug? ( dev-util/dwarves )"
+PDEPEND="
+   >=virtual/dist-kernel-${PV}"
 
 src_prepare() {
local PATCHES=(
diff --git a/sys-kernel/gentoo-kernel/gentoo-kernel-5.4.86.ebuild 
b/sys-kernel/gentoo-kernel/gentoo-kernel-5.4.86.ebuild
index 0c806189a34d..5878b6bcd6c1 100644
--- a/sys-kernel/gentoo-kernel/gentoo-kernel-5.4.86.ebuild
+++ b/sys-kernel/gentoo-kernel/gentoo-kernel-5.4.86.ebuild
@@ -46,6 +46,8 @@ RDEPEND="
!sys-kernel/vanilla-kernel-bin:${SLOT}"
 BDEPEND="
debug? ( dev-util/dwarves )"
+PDEPEND="
+   >=virtual/dist-kernel-${PV}"
 
 pkg_pretend() {
ewarn "Starting with 5.4.52, Distribution Kernels are switching from 
Arch"
-- 
2.30.0

[gentoo-dev] [PATCH 1/5] virtual/dist-kernel: New virtual

[Michał Górny]

Add a new virtual that can be used to provide subslot rebuilds
for kernel modules.

Signed-off-by: Michał Górny 
---
 profiles/arch/powerpc/ppc64/64le/package.mask |  1 +
 profiles/arch/powerpc/ppc64/package.mask  |  1 +
 virtual/dist-kernel/dist-kernel-5.10.4.ebuild | 19 +++
 virtual/dist-kernel/dist-kernel-5.4.86.ebuild | 19 +++
 virtual/dist-kernel/metadata.xml  |  8 
 5 files changed, 48 insertions(+)
 create mode 100644 virtual/dist-kernel/dist-kernel-5.10.4.ebuild
 create mode 100644 virtual/dist-kernel/dist-kernel-5.4.86.ebuild
 create mode 100644 virtual/dist-kernel/metadata.xml

diff --git a/profiles/arch/powerpc/ppc64/64le/package.mask 
b/profiles/arch/powerpc/ppc64/64le/package.mask
index 784790b1ea86..c42fe27f4868 100644
--- a/profiles/arch/powerpc/ppc64/64le/package.mask
+++ b/profiles/arch/powerpc/ppc64/64le/package.mask
@@ -17,6 +17,7 @@
 # little-endian power8 ppc64 configs are provided
 -sys-kernel/gentoo-kernel
 -sys-kernel/vanilla-kernel
+-virtual/dist-kernel
 
 # Georgy Yakovlev  (2020-01-25)
 # libva unmasked on little-endian profile
diff --git a/profiles/arch/powerpc/ppc64/package.mask 
b/profiles/arch/powerpc/ppc64/package.mask
index f36237c2c003..f78c971cc95f 100644
--- a/profiles/arch/powerpc/ppc64/package.mask
+++ b/profiles/arch/powerpc/ppc64/package.mask
@@ -16,6 +16,7 @@ dev-java/openjfx:11
 # however users can unmask and use savedconfig feature
 sys-kernel/gentoo-kernel
 sys-kernel/vanilla-kernel
+virtual/dist-kernel
 
 # Georgy Yakovlev  (2020-01-21)
 # buggy on big-endian
diff --git a/virtual/dist-kernel/dist-kernel-5.10.4.ebuild 
b/virtual/dist-kernel/dist-kernel-5.10.4.ebuild
new file mode 100644
index ..3df38272de32
--- /dev/null
+++ b/virtual/dist-kernel/dist-kernel-5.10.4.ebuild
@@ -0,0 +1,19 @@
+# Copyright 2021 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+EAPI=7
+
+DESCRIPTION="Virtual to depend on any Distribution Kernel"
+HOMEPAGE=""
+SRC_URI=""
+
+LICENSE=""
+SLOT="0/${PV}"
+KEYWORDS="~amd64 ~arm ~arm64 ~ppc64 ~x86"
+
+RDEPEND="
+   || (
+   ~sys-kernel/gentoo-kernel-${PV}
+   ~sys-kernel/gentoo-kernel-bin-${PV}
+   ~sys-kernel/vanilla-kernel-${PV}
+   )"
diff --git a/virtual/dist-kernel/dist-kernel-5.4.86.ebuild 
b/virtual/dist-kernel/dist-kernel-5.4.86.ebuild
new file mode 100644
index ..11d6573ded7c
--- /dev/null
+++ b/virtual/dist-kernel/dist-kernel-5.4.86.ebuild
@@ -0,0 +1,19 @@
+# Copyright 2021 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+EAPI=7
+
+DESCRIPTION="Virtual to depend on any Distribution Kernel"
+HOMEPAGE=""
+SRC_URI=""
+
+LICENSE=""
+SLOT="0/${PV}"
+KEYWORDS="~amd64 ~arm64 ~ppc64 ~x86"
+
+RDEPEND="
+   || (
+   ~sys-kernel/gentoo-kernel-${PV}
+   ~sys-kernel/gentoo-kernel-bin-${PV}
+   ~sys-kernel/vanilla-kernel-${PV}
+   )"
diff --git a/virtual/dist-kernel/metadata.xml b/virtual/dist-kernel/metadata.xml
new file mode 100644
index ..20e07cdc2be9
--- /dev/null
+++ b/virtual/dist-kernel/metadata.xml
@@ -0,0 +1,8 @@
+
+http://www.gentoo.org/dtd/metadata.dtd;>
+
+   
+   dist-ker...@gentoo.org
+   Distribution Kernel Project
+   
+
-- 
2.30.0

[gentoo-dev] [PATCH 0/5] (linux-mod.eclass) Subslot module rebuilds for dist-kernel

[Michał Górny]

Hello,

Here's a proposal to add support for automatically rebuilding kernel
modules when dist-kernels are upgraded.  The behavior is entirely
opt-in, so the users of other kernels are not affected.

The proposal adds a new global flag 'dist-kernel'.  It is off by default
and has no functional effect for people who do not enable it.  However,
when it is enabled, it causes linux-mod.eclass to inject a := dependency
on virtual/dist-kernel in the packages installing kernel modules.

When the user upgrades one of the dist-kernel packages to a newer
version, an update of virtual/dist-kernel is triggered.  This in turn
causes a change of subslot and provokes rebuilds of all kernel modules.

This represents the next step towards enabling kernel maintenance free
of manual intervention.


Michał Górny (5):
  virtual/dist-kernel: New virtual
  sys-kernel/gentoo-kernel: Add a PDEP on virtual/prebuilt-kernel
  sys-kernel/gentoo-kernel-bin: Add a PDEP on virtual/prebuilt-kernel
  sys-kernel/vanilla-kernel: Add a PDEP on virtual/prebuilt-kernel
  linux-mod.eclass: Add a prebuilt-kernel rebuild flag

 eclass/linux-mod.eclass   | 13 ++---
 profiles/arch/amd64/use.mask  |  4 
 profiles/arch/amd64/use.stable.mask   |  6 +-
 profiles/arch/base/use.mask   |  6 +-
 profiles/arch/powerpc/ppc64/64le/package.mask |  1 +
 profiles/arch/powerpc/ppc64/package.mask  |  1 +
 profiles/use.desc |  1 +
 .../gentoo-kernel-bin-5.10.4-r1.ebuild|  2 ++
 .../gentoo-kernel-bin-5.10.4.ebuild   |  2 ++
 .../gentoo-kernel-bin-5.4.86-r1.ebuild|  2 ++
 .../gentoo-kernel-bin-5.4.86.ebuild   |  2 ++
 .../gentoo-kernel/gentoo-kernel-5.10.4.ebuild |  2 ++
 .../gentoo-kernel/gentoo-kernel-5.4.86.ebuild |  2 ++
 .../vanilla-kernel-5.10.4.ebuild  |  2 ++
 .../vanilla-kernel-5.4.86.ebuild  |  2 ++
 virtual/dist-kernel/dist-kernel-5.10.4.ebuild | 19 +++
 virtual/dist-kernel/dist-kernel-5.4.86.ebuild | 19 +++
 virtual/dist-kernel/metadata.xml  |  8 
 18 files changed, 89 insertions(+), 5 deletions(-)
 create mode 100644 virtual/dist-kernel/dist-kernel-5.10.4.ebuild
 create mode 100644 virtual/dist-kernel/dist-kernel-5.4.86.ebuild
 create mode 100644 virtual/dist-kernel/metadata.xml

-- 
2.30.0

Re: [gentoo-dev] using markup language for eclassdoc tags

[Tim Harder]

On 2021-01-04 Mon 04:18, Michał Górny wrote:
> I'm all for switching to rST in the foreseeable future but let's stick
> with the existing syntax for the transition period, i.e. until new
> pkgcore is stable and deployed on Infra.  I'm not saying you have to
> strictly copy the existing magic, just get a reasonably readable result
> for the existing eclasses.

The existing syntax is supported by the current work that just uses
literal blocks for all multiline text fields. This attempts avoiding all
the semi-formatting issues, the downside being that text flow across
paragraphs is broken since all the formatting is preserved as written in
the eclasses.

Pivoting to an actual markup language will be trivial if/when that
occurs.

Tim

Re: [gentoo-dev] using markup language for eclassdoc tags

[Ulrich Mueller]

> On Mon, 04 Jan 2021, Tim Harder wrote:

> In terms of choice, I'd personally choose reStructuredText since that
> generally plugs into python easier via docutils/sphinx (currently used
> for pkgcore's man/html conversion), but am open to discussion of
> alternatives such as markdown.

About reStructuredText vs Markdown:
- ReST syntax is more complete and better standardised.
- Markdown uses HTML as extension language, which is fine when
  converting to HTML but makes conversion to other formats more
  difficult.
- Trailing whitespace as part of Markdown's syntax is problematic
  (and the current version of app-emacs/ebuild-mode removes it).
- We already use ReST for some of our documentation, like GLEPs.

Ulrich


signature.asc
Description: PGP signature

Re: [gentoo-dev] using markup language for eclassdoc tags

[Michał Górny]

On Mon, 2021-01-04 at 04:03 -0700, Tim Harder wrote:
> Hi,
> 
> I've written nascent support for eclassdoc man page generation (along
> with rST and HTML docs) in pkgcore [1] accessible on the cli via `pmaint
> eclass` that intends to provide an alternative to the current awk
> implementation [2].
> 
> In doing so, I've noticed that the formatting of the docs feels quite
> haphazard due to reinventing some of the syntax for proper markup
> languages via embedded tags (e.g. @CODE for literal code blocks) while
> not handling other basic cases properly (e.g. bulleted lists).
> 
> What does the community think about selecting a specific markup language
> and using that for multiline text for related eclassdoc tags (e.g.
> @DESCRIPTION)? That way, we can toss out the @CODE/@SUBSECTION tag
> workarounds and use what the markup language natively provides directly.
> 
> In terms of choice, I'd personally choose reStructuredText since that
> generally plugs into python easier via docutils/sphinx (currently used
> for pkgcore's man/html conversion), but am open to discussion of
> alternatives such as markdown.

I'm all for switching to rST in the foreseeable future but let's stick
with the existing syntax for the transition period, i.e. until new
pkgcore is stable and deployed on Infra.  I'm not saying you have to
strictly copy the existing magic, just get a reasonably readable result
for the existing eclasses.

-- 
Best regards,
Michał Górny

[gentoo-dev] using markup language for eclassdoc tags

[Tim Harder]

Hi,

I've written nascent support for eclassdoc man page generation (along
with rST and HTML docs) in pkgcore [1] accessible on the cli via `pmaint
eclass` that intends to provide an alternative to the current awk
implementation [2].

In doing so, I've noticed that the formatting of the docs feels quite
haphazard due to reinventing some of the syntax for proper markup
languages via embedded tags (e.g. @CODE for literal code blocks) while
not handling other basic cases properly (e.g. bulleted lists).

What does the community think about selecting a specific markup language
and using that for multiline text for related eclassdoc tags (e.g.
@DESCRIPTION)? That way, we can toss out the @CODE/@SUBSECTION tag
workarounds and use what the markup language natively provides directly.

In terms of choice, I'd personally choose reStructuredText since that
generally plugs into python easier via docutils/sphinx (currently used
for pkgcore's man/html conversion), but am open to discussion of
alternatives such as markdown.

The downside of moving to an actual markup language is that writing docs
for eclasses will get stricter, but conversion issues could be flagged
by pkgcheck aiding verification.

Thanks,
Tim

[1]: https://github.com/pkgcore/pkgcore/commit/b3a6b8da
[2]: 
https://github.com/mgorny/eclass-to-manpage/blob/master/eclass-to-manpage.awk

[gentoo-portage-dev] Re: [PATCH] global_event_loop: return running loop for current thread

[Zac Medico]

On 1/4/21 1:06 AM, Zac Medico wrote:
> Like asyncio.get_event_loop(), return the running loop for the
> current thread if there is one, and otherwise construct a new
> one if needed. This allows the _safe_loop function to become
> synonymous with the global_event_loop function.
> 
> Bug: https://bugs.gentoo.org/763339
> Signed-off-by: Zac Medico 
> ---
>  .../util/_eventloop/global_event_loop.py  | 28 ++-
>  lib/portage/util/futures/_asyncio/__init__.py | 22 ++-
>  2 files changed, 17 insertions(+), 33 deletions(-)

For the case of "loop running in non-main thread" of API consumer, this
change makes portage compatible with PEP 492 coroutines with async and
await syntax. Portage internals can safely begin using async / await
syntax instead of compat_coroutine.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

[Michał Górny]

On Mon, 2021-01-04 at 02:35 +0100, Thomas Deutschmann wrote:
> Modifying an existing user is a bad default and makes Gentoo
> special because it is common for system administrators to make
> modifications to user (i.e. putting an user into another service's
> group to allow that user to access service in question) and it
> would be unexpected to see these changes reverted during normal
> world upgrade (which could break services).

Not modifying an existing user is a horrible default that has already
bricked one system (by removing /dev/null).  So, over my dead commit
access.

-- 
Best regards,
Michał Górny

[gentoo-dev] [News review v2] LibreSSL support discontinued

[Michał Górny]

v2, with additional 'emerge --deselect':
---
Title: LibreSSL support discontinued
Author: Michał Górny 
Posted: 202x-xx-xx
Revision: 1
News-Item-Format: 2.0
Display-If-Installed: dev-libs/libressl

Starting 2021-02-01, Gentoo will no longer actively pursue supporting
dev-libs/libressl as an alternative to dev-libs/openssl.  While it will
still be possible for expert users to use LibreSSL on their systems,
we are only going to provide support for OpenSSL-based systems.  Most
importantly, we are no longer going to maintain downstream patches for
LibreSSL support -- it will rely on either package upstreams merging
such patches themselves, or LibreSSL upstream finally working towards
better OpenSSL compatibility.

On 2021-02-01, we will mask the relevant USE flags and packages.  If
you
wish to continue using LibreSSL, you will be able to undo these masks
for the time being.  However, as packages drop patching for LibreSSL
and the library is eventually removed from ::gentoo, it will become
necessary to use the user-maintained LibreSSL overlay [1].  As long-
term
support for LibreSSL is not guaranteed, we recommend switching
to OpenSSL instead.  More information on removal can be found
on the relevant bug [2].

To switch before the aforementioned date, remove 'libressl' from your
USE flags and CURL_SSL targets.  Afterwards, it is recommended to
prefetch all the necessary distfiles before proceeding with the system
upgrade, in case wget(1) becomes broken in the process:

emerge --fetchonly dev-libs/openssl net-misc/wget
emerge --fetchonly --changed-use @world

A --changed-use @world upgrade should automatically cause LibreSSL
to be replaced by OpenSSL, and all affected packages to be rebuilt:

emerge --deselect dev-libs/libressl
emerge --changed-use @world


LibreSSL has been forked off OpenSSL in 2014 to address a number of
problems with the original package.  However, since then OpenSSL
development gained speed and the original reasons for the fork no
longer
apply.  Furthermore, LibreSSL started to repeatedly fall behind
and cause growing compatibility problems.  While initially these
problems were related to packages using old/insecure OpenSSL APIs,
today
they are mostly related to LibreSSL missing newer OpenSSL APIs
(yet declaring false compatibility with newer OpenSSL versions).

With the little testing it gets, our developers and users had to put
a significant effort into fixing upstream packages.  In some cases
(e.g. Qt), upstream has explicitly refused to support LibreSSL, forcing
us to maintain the patches forever.  This in turn means that
security fixes, regular version bumps or end-user system upgrades are
often delayed because of necessary LibreSSL patching.  What is even
worse, major runtime issues managed to sneak in that broke production
systems running LibreSSL in the past.

To the best of our knowledge, the only benefit LibreSSL has over
OpenSSL
right now is the additional libtls library.  For this reason, we have
packaged dev-libs/libretls which is a port of this library that links
to OpenSSL.

All these issued considered, we came to the conclusion that OpenSSL
should remain the only supported production option for Gentoo systems.
While the flexibility of Gentoo should make it possible to keep using
LibreSSL going forward, the effort necessary to provide first-class
official support for LibreSSL has proven to outweigh the benefit.

[1] https://gitweb.gentoo.org/repo/proj/libressl.git/tree/README.md
[2] https://bugs.gentoo.org/762847
---




-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [News review] LibreSSL support discontinued

[Michał Górny]

On Mon, 2021-01-04 at 09:18 +, Marek Szuba wrote:
> 
> On January 4, 2021 8:25:21 AM UTC, Stefan Strogin  wrote:
> 
> > Doesn't work for me.
> 
> Have you got libressl in your world file, perchance?
> 

Yeah, I guess we need --deselect too.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [News review] LibreSSL support discontinued

[Marek Szuba]

On January 4, 2021 8:25:21 AM UTC, Stefan Strogin  wrote:

>Doesn't work for me.

Have you got libressl in your world file, perchance?

-- 
Marecki

[gentoo-portage-dev] [PATCH] global_event_loop: return running loop for current thread

[Zac Medico]

Like asyncio.get_event_loop(), return the running loop for the
current thread if there is one, and otherwise construct a new
one if needed. This allows the _safe_loop function to become
synonymous with the global_event_loop function.

Bug: https://bugs.gentoo.org/763339
Signed-off-by: Zac Medico 
---
 .../util/_eventloop/global_event_loop.py  | 28 ++-
 lib/portage/util/futures/_asyncio/__init__.py | 22 ++-
 2 files changed, 17 insertions(+), 33 deletions(-)

diff --git a/lib/portage/util/_eventloop/global_event_loop.py 
b/lib/portage/util/_eventloop/global_event_loop.py
index 413011178..bf314dc34 100644
--- a/lib/portage/util/_eventloop/global_event_loop.py
+++ b/lib/portage/util/_eventloop/global_event_loop.py
@@ -1,28 +1,4 @@
-# Copyright 2012-2020 Gentoo Authors
+# Copyright 2012-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
-import portage
-from portage.util._eventloop.asyncio_event_loop import AsyncioEventLoop
-
-_instances = {}
-
-
-def global_event_loop():
-   """
-   Get a global EventLoop (or compatible object) instance which
-   belongs exclusively to the current process.
-   """
-
-   pid = portage.getpid()
-   instance = _instances.get(pid)
-   if instance is not None:
-   return instance
-
-   constructor = AsyncioEventLoop
-
-   # Use the _asyncio_wrapper attribute, so that unit tests can compare
-   # the reference to one retured from _wrap_loop(), since they should
-   # not close the loop if it refers to a global event loop.
-   instance = constructor()._asyncio_wrapper
-   _instances[pid] = instance
-   return instance
+from portage.util.futures._asyncio import _safe_loop as global_event_loop
diff --git a/lib/portage/util/futures/_asyncio/__init__.py 
b/lib/portage/util/futures/_asyncio/__init__.py
index d39f31786..ab1468d43 100644
--- a/lib/portage/util/futures/_asyncio/__init__.py
+++ b/lib/portage/util/futures/_asyncio/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2018-2020 Gentoo Authors
+# Copyright 2018-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 __all__ = (
@@ -37,9 +37,6 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.futures:compat_coroutine@_compat_coroutine',
 )
 from portage.util._eventloop.asyncio_event_loop import AsyncioEventLoop as 
_AsyncioEventLoop
-from portage.util._eventloop.global_event_loop import (
-   global_event_loop as _global_event_loop,
-)
 # pylint: disable=redefined-builtin
 from portage.util.futures.futures import (
CancelledError,
@@ -238,7 +235,7 @@ def _wrap_loop(loop=None):
# The default loop returned by _wrap_loop should be consistent
# with global_event_loop, in order to avoid accidental registration
# of callbacks with a loop that is not intended to run.
-   loop = loop or _global_event_loop()
+   loop = loop or _safe_loop()
return (loop if hasattr(loop, '_asyncio_wrapper')
else _AsyncioEventLoop(loop=loop))
 
@@ -267,8 +264,9 @@ def _safe_loop():
@rtype: asyncio.AbstractEventLoop (or compatible)
@return: event loop instance
"""
-   if portage._internal_caller or threading.current_thread() is 
threading.main_thread():
-   return _global_event_loop()
+   loop = _get_running_loop()
+   if loop is not None:
+   return loop
 
thread_key = threading.get_ident()
with _thread_weakrefs.lock:
@@ -286,6 +284,16 @@ def _safe_loop():
return loop
 
 
+def _get_running_loop():
+   with _thread_weakrefs.lock:
+   if _thread_weakrefs.pid == portage.getpid():
+   try:
+   loop = 
_thread_weakrefs.loops[threading.get_ident()]
+   except KeyError:
+   return None
+   return loop if loop.is_running() else None
+
+
 def _thread_weakrefs_atexit():
with _thread_weakrefs.lock:
if _thread_weakrefs.pid == portage.getpid():
-- 
2.26.2

Re: [gentoo-dev] [News review] LibreSSL support discontinued

[Stefan Strogin]

Hello Michal,

On Sun, Jan 03, 2021 at 09:47:31PM +0100, Michał Górny wrote:
> Hello,
> (...)
> To switch before the aforementioned date, remove 'libressl' from your
> USE flags and CURL_SSL targets.  Afterwards, it is recommended to
> prefetch all the necessary distfiles before proceeding with the system
> upgrade, in case wget(1) becomes broken in the process:
> 
> emerge --fetchonly dev-libs/openssl net-misc/wget
> emerge --fetchonly --changed-use @world
> 
> A --changed-use @world upgrade should automatically cause LibreSSL
> to be replaced by OpenSSL, and all affected packages to be rebuilt:
> 
> emerge --changed-use @world
> 

Doesn't work for me. Emerge prints:

```
[blocks B  ] dev-libs/openssl:0 ("dev-libs/openssl:0" is blocking
dev-libs/libressl-3.3.1)

Total: 37 packages (1 new, 36 reinstalls), Size of downloads: 0 KiB
Conflict: 1 block (1 unsatisfied)
(...)
```

I think you have to remove libressl first, like `emerge -C libressl`,
then install openssl like `emerge -1 openssl`, then rebuild
dependencies. As described here but in opposite way:
https://wiki.gentoo.org/wiki/Project:LibreSSL