Re: [gentoo-user] pm-suspend replacement?

[Alan Mackenzie]

Hello, tastytea.

On Sun, Sep 19, 2021 at 22:50:30 +0200, tastytea wrote:
> On 2021-09-19 20:24+ Alan Mackenzie  wrote:

> > […]
> > > Or maybe I'll try and find pm-suspend again on the web.  Maybe it
> > > had some feature (or bug workaround) which the more modern packages
> > > are lacking.  

> > That's just what I did.  A web search for pm-utils found it easily
> > enough.  pm-suspend works again, and I'm a happy chappy - almost.  Why
> > was pm-utils taken off of portage in the first place?  Was there some
> > sort of security problem, or was it just because it hadn't been
> > updated in a fair while (since 2013, I think)?

> It was removed because upstream abandoned it. It was announced in
> 
> which references .

It was indeed.  I think I missed it because "pm-utils" didn't ring any
alarm bells whereas "pm-suspend" would have done.

> > That's another feature missing from portage - a systematic way of
> > discovering why a package has been removed.

> `eselect news read all | less` is sometimes helpful. 

Thanks!

> -- 
> Get my PGP key with `gpg --locate-keys tasty...@tastytea.de` or at
> .

-- 
Alan Mackenzie (Nuremberg, Germany).

[gentoo-user] Re: python, my nemesis

[Grant Edwards]

On 2021-09-20, Michael  wrote:
> On Monday, 20 September 2021 14:56:46 BST Gerrit Kuehn wrote:
>
>> Well, this was the suggested way to go, see
>> https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html
>> 
>> But also when trying "emerge -1vUD @world" (be it with or without the
>> package.use settings), I get stuck in conflicts (mostly on perl and
>> setuptools). perl issues would probably resolve once I have EAPI8
>> support, i.e. get new portage.
>
> With a year old system you should question if reinstalling your
> system after a back up of configuration and data files would be a
> smarter approach.

In my experience, it will almost definitely be less work to
reinstall --- usually a _lot_ less work.

--
Grant

RE: [gentoo-user] python, my nemesis

[Laurence Perkins]

-Original Message-
From: Michael  
Sent: Monday, September 20, 2021 8:21 AM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] python, my nemesis

On Monday, 20 September 2021 15:52:03 BST Gerrit Kuehn wrote:
>> 
>> Just extracting stage3 over everything that is already there?

>No, I move out of the way the config/data files I want to keep and move them 
>back in after untarring the stage 3.

A less traumatic place to start I find is to unpack the stage3 into /tmp or 
wherever it's out of the way and then chroot into it and use quickpkg to bundle 
up newer versions of blocked core utilities.  Not needing build deps can quite 
often simplify the upgrade path enough to get past certain things.

You can also (with a lot of annoyance) use the new portage that's in the 
tarball to get past eapi restrictions and whatnot and tell it to install the 
stuff it builds into your original system's folders.  I've even used that to 
salvage systems that got moved to new hardware with an incompatible set of 
processor flags.  Not straightforward since you have to manage both the build 
environment in the stage3 and the install environment in the original system, 
but there's very little you can't fix with this approach.

Just don't let your system build binary packages for virtual/* under any 
circumstances.  That never ends well.  I really need to write up a request to 
have portage blacklist those by default when buildpkg is enabled...

LMP

RE: [gentoo-user] python, my nemesis

[Laurence Perkins]

-Original Message-
From: Neil Bothwick  
Sent: Monday, September 20, 2021 8:17 AM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] python, my nemesis

On Mon, 20 Sep 2021 15:56:46 +0200, Gerrit Kuehn wrote:

>> > > ~ # cat /etc/portage/package.use/py
>> > > */* PYTHON_TARGETS: -* python3_9 python3_8
>> > > */* PYTHON_SINGLE_TARGET: -* python3_9
>> > > ---
>> 
>> > You should probably not mess with these variables until after your 
>> > system is 100% updated and consistent. And even then, probably not.
>> > 
>> > With our package manager written in python, you often need old 
>> > python stuff to build the new python stuff, and disabling the old 
>> > python stuff will throw a wrench into that. Even in situations where 
>> > technically some upgrade path exists, the complexity of the python 
>> > dependencies often means that the package manager will give up 
>> > before it finds the solution unless the solution is obvious. By 
>> > tweaking those variables, you make the solution less obvious to it.
>> 
>> Well, this was the suggested way to go, see 
>> https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html
>
>That news item is about going from 3.8 to 3.9, you are on 3.7. I'd try 
>removing the -* items are trying again.
>
>
When upgrading really old systems I find it useful to run:
export PYTHON_TARGETS="$(eix -I | grep -o "python[23]_[0-9]*" | sort -u)"

This will temporarily set the system (for the duration of the current session) 
to build for every version of python that is either available for new builds or 
currently in use.  The dependency resolver seems to have an easier time finding 
a path when it doesn't think it has to throw away all the currently installed 
stuff first.

Note that this will end up rebuilding a lot more than is technically necessary, 
and then you'll need to rebuild again with the python versions you actually 
want and depclean the extras.  Personally though I rather prefer to have the 
computer do more work instead of spending hours using *my* brain to find the 
minimum sized rebuild set to upgrade the system.

LMP

Re: [gentoo-user] python, my nemesis

[Gerrit Kuehn]

On Mon, 20 Sep 2021 16:20:58 +0100
Michael  wrote:

> > Well, yeah, your mileage may vary.  

> Quite, if you can get your existing installation to only run a
> minimal number of rebuilds to arrive at an upgraded toolchain, then
> the benefit of reinstalling wouldn't be there.  This could have been
> the case on an average year, when python deprecations didn't
> accelerate and EAPI didn't change.  If however you end up tying up
> yourself in knots with subsequent python upgrades and difficult to
> resolve conflicts, then the pain Vs gain calculus changes.

I was already wondering why so many things happened during the
(comparatively) short time I didn't watch. Looks like people have been
in lockdown with plenty of time to come around with new things.
Fortunately, the base installation I have to update doesn't contain
many applications (e.g., no X involved). Everything sits on an NFS
server with ZFS below it, so it is quite easy to do snaphots and go
back and forth between them, and we have plenty to CPU power to rebuild
things.
OTOH, the orchestration to have the setup re-done in a fully automated
way is still under development, so I definitely wanted to try updating
before really starting over.


cu
  Gerrit

Re: [gentoo-user] python, my nemesis

[Gerrit Kuehn]

On Mon, 20 Sep 2021 16:17:17 +0100
Neil Bothwick  wrote:

> > Well, this was the suggested way to go, see
> > https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html

> That news item is about going from 3.8 to 3.9, you are on 3.7.

Yes and no. It started with

---
Display-If-Installed: dev-lang/python:3.7
Display-If-Installed: dev-lang/python:3.8
---

So my impression was I might find myself in a similar situation with
3.7.

> I'd try
> removing the -* items are trying again.

Same issue (actually, I tried that first and only found the news item
after this didn't work).
But I think I fixed most issues now (see my previous posts). Thank you
for taking time to look into this.


cu
  Gerrit

Re: [gentoo-user] python, my nemesis

[Michael]

On Monday, 20 September 2021 15:52:03 BST Gerrit Kuehn wrote:
> On Mon, 20 Sep 2021 15:29:30 +0100
> 
> Michael  wrote:
> > Personally, I'd back up /home /etc and world file, plus any databases
> > or websites if stored under /var/, then untar the latest stage 3
> > tarball and update @system and @world.
> 
> Just extracting stage3 over everything that is already there?

No, I move out of the way the config/data files I want to keep and move them 
back in after untarring the stage 3.


> >  Overall it should be a *much*
> > faster approach to allow you to bring your installation up to date.
> 
> Well, yeah, your mileage may vary.

Quite, if you can get your existing installation to only run a minimal number 
of rebuilds to arrive at an upgraded toolchain, then the benefit of 
reinstalling wouldn't be there.  This could have been the case on an average 
year, when python deprecations didn't accelerate and EAPI didn't change.  If 
however you end up tying up yourself in knots with subsequent python upgrades 
and difficult to resolve conflicts, then the pain Vs gain calculus changes.



signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] python, my nemesis

[Neil Bothwick]

On Mon, 20 Sep 2021 15:56:46 +0200, Gerrit Kuehn wrote:

> > > ~ # cat /etc/portage/package.use/py 
> > > */* PYTHON_TARGETS: -* python3_9 python3_8
> > > */* PYTHON_SINGLE_TARGET: -* python3_9
> > > ---
> 
> > You should probably not mess with these variables until after your
> > system is 100% updated and consistent. And even then, probably not.
> > 
> > With our package manager written in python, you often need old python
> > stuff to build the new python stuff, and disabling the old python
> > stuff will throw a wrench into that. Even in situations where
> > technically some upgrade path exists, the complexity of the python
> > dependencies often means that the package manager will give up before
> > it finds the solution unless the solution is obvious. By tweaking
> > those variables, you make the solution less obvious to it.  
> 
> Well, this was the suggested way to go, see
> https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html

That news item is about going from 3.8 to 3.9, you are on 3.7. I'd try
removing the -* items are trying again.


-- 
Neil Bothwick

"We demand rigidly defined areas of doubt and uncertainty!"


pgpd48PCoW9g5.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] python, my nemesis

[Gerrit Kuehn]

On Mon, 20 Sep 2021 10:34:16 -0400
Michael Orlitzky  wrote:

> > Well, this was the suggested way to go, see
> > https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html

> It was only suggested for a period of about three weeks:

Uh, yes, another thing I overlooked. I should get more sleep, I guess.
:) Thanks for pointing this out!


cu
  Gerrit

Re: [gentoo-user] python, my nemesis

[Gerrit Kuehn]

On Mon, 20 Sep 2021 16:41:17 +0200
Arve Barsnes  wrote:

> This is because your gentoolkit installed version has no support for
> python 3.9.

Yes, I must have been overlooking this all the time. That's why I came
here first (more/better eyes on the issue :-).

> You might be able to include gentoolkit in your emerge
> command, or you might need to build portage with python 3.8 support as
> well as 3.9. This type of problem might crop up for several other
> packages as you try to get python updated.

emerge -p1vUD @world

lists about 200 ports to rebuild now, without showing any other issues
ahead. I'll go for it (and keep my fingers crossed ;-). Thank you for
your support.


cu
  Gerrit

Re: [gentoo-user] python, my nemesis

[Gerrit Kuehn]

On Mon, 20 Sep 2021 15:29:30 +0100
Michael  wrote:

> With a year old system you should question if reinstalling your
> system after a back up of configuration and data files would be a
> smarter approach.  If you *must* upgrade your current installation
> for learning or as an experiment, then this is something which has
> been done before.

I know. I'm a happy Gentoo user for more than 20 years now. However,
this is the master installation for a diskless setup providing the
baseline for more than a dozen servers. Of course, I can still reinstall
but this would be more work than just one machine + a few config files.
As one year is not too terribly old (imho) and this looked like
something that might be easily solved, I decided to ask for advice here
first.

> Personally, I'd back up /home /etc and world file, plus any databases
> or websites if stored under /var/, then untar the latest stage 3
> tarball and update @system and @world.

Just extracting stage3 over everything that is already there?

>  Overall it should be a *much*
> faster approach to allow you to bring your installation up to date.

Well, yeah, your mileage may vary.


cu
  Gerrit

Re: [gentoo-user] python, my nemesis

[Arve Barsnes]

On Mon, 20 Sept 2021 at 16:32, Gerrit Kuehn  wrote:
>   (sys-apps/portage-3.0.4-r1-3:0/0::gentoo, installed) USE="(ipc)
>   native-extensions rsync-verify xattr -apidoc -build -doc -gentoo-dev
>   (-selinux) -test" ABI_X86="(64)" PYTHON_TARGETS="python3_7 (-pypy3)
>   -python3_6 -python3_8 -python3_9" pulled in by
>   
> sys-apps/portage[python_targets_python3_7(-),-python_single_target_pypy3(-),-python_single_target_python3_6(-),-python_single_target_python3_7(-),-python_single_target_python3_8(-)]
>   required by (app-portage/gentoolkit-0.5.0:0/0::gentoo, installed)
>   USE="-test" ABI_X86="(64)" PYTHON_TARGETS="python3_7 (-pypy3)
>   -python3_6 -python3_8"
> ---
>
>
> This looks like portage is blocked by itself... so how to solve this
> one?

This is because your gentoolkit installed version has no support for
python 3.9. You might be able to include gentoolkit in your emerge
command, or you might need to build portage with python 3.8 support as
well as 3.9. This type of problem might crop up for several other
packages as you try to get python updated.

Cheers,
Arve

Re: [gentoo-user] python, my nemesis

[Gerrit Kuehn]

On Mon, 20 Sep 2021 16:31:59 +0200
Gerrit Kuehn  wrote:

> This looks like portage is blocked by itself... so how to solve this
> one?

Well, I simply installed the new virtual packages manually and used
"--nodeps" on portage itself afterwards... and here we are. Looks like
new portage will update the rest of my system without any issue now (at
least it says so when checking with -p), so here we go. Still no idea
why this was blocking so hard, though.


cu
  Gerrit

Re: [gentoo-user] python, my nemesis

[Michael Orlitzky]

On Mon, 2021-09-20 at 15:56 +0200, Gerrit Kuehn wrote:
> 
> Well, this was the suggested way to go, see
> https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html

It was only suggested for a period of about three weeks:

  We are planning to switch the default Python target of Gentoo systems
  on 2021-06-01, from Python 3.8 to Python 3.9.  If you have not   
  changed the values of PYTHON_TARGETS or PYTHON_SINGLE_TARGET, the 
  change will have immediate effect on your system and the package 
  manager will try to switch automatically on the next upgrade 
  following the change.

> 
> But also when trying "emerge -1vUD @world" (be it with or without the
> package.use settings), I get stuck in conflicts (mostly on perl and
> setuptools). perl issues would probably resolve once I have EAPI8
> support, i.e. get new portage.
> 
> 

You can try --backtrack=200 (or a larger value) to fix that; it was a
problem for a lot of people. Even though the end result of both
approaches is "it doesn't work," I promise you you're better off not
messing with the python stuff until the system is fully upgraded.
Upgrading a year-old system is hard enough, and the introduction of
EAPI=8 just made it worse.

Re: [gentoo-user] python, my nemesis

[Gerrit Kuehn]

On Mon, 20 Sep 2021 09:18:23 -0400
Michael Orlitzky  wrote:

> With our package manager written in python, you often need old python
> stuff to build the new python stuff, and disabling the old python
> stuff will throw a wrench into that. Even in situations where
> technically some upgrade path exists, the complexity of the python
> dependencies often means that the package manager will give up before
> it finds the solution unless the solution is obvious. By tweaking
> those variables, you make the solution less obvious to it.

I took out the extra settings and solved a few conflicts manually. I'm
down to this now:

---
 ~ # emerge -p --oneshot sys-apps/portage --verbose-conflicts

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N ] acct-group/portage-0 
[ebuild  N ] acct-user/portage-0 
[ebuild U  ] sys-apps/portage-3.0.20-r6 [3.0.4-r1]
PYTHON_TARGETS="python3_9* (-python3_10)" 

!!! Multiple package instances within a single package slot have been
pulled !!! into the dependency graph, resulting in a slot conflict:

sys-apps/portage:0

  (sys-apps/portage-3.0.20-r6:0/0::gentoo, ebuild scheduled for merge)
  USE="(ipc) native-extensions rsync-verify xattr -apidoc -build -doc
  -gentoo-dev (-selinux) -test" ABI_X86="(64)"
  PYTHON_TARGETS="python3_9 (-pypy3) (-python3_10) -python3_8" pulled
  in by sys-apps/portage (Argument)

  (sys-apps/portage-3.0.4-r1-3:0/0::gentoo, installed) USE="(ipc)
  native-extensions rsync-verify xattr -apidoc -build -doc -gentoo-dev
  (-selinux) -test" ABI_X86="(64)" PYTHON_TARGETS="python3_7 (-pypy3)
  -python3_6 -python3_8 -python3_9" pulled in by
  
sys-apps/portage[python_targets_python3_7(-),-python_single_target_pypy3(-),-python_single_target_python3_6(-),-python_single_target_python3_7(-),-python_single_target_python3_8(-)]
  required by (app-portage/gentoolkit-0.5.0:0/0::gentoo, installed)
  USE="-test" ABI_X86="(64)" PYTHON_TARGETS="python3_7 (-pypy3)
  -python3_6 -python3_8" 
---


This looks like portage is blocked by itself... so how to solve this
one?


cu
  Gerrit

Re: [gentoo-user] python, my nemesis

[Michael]

On Monday, 20 September 2021 14:56:46 BST Gerrit Kuehn wrote:

> Well, this was the suggested way to go, see
> https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html
> 
> But also when trying "emerge -1vUD @world" (be it with or without the
> package.use settings), I get stuck in conflicts (mostly on perl and
> setuptools). perl issues would probably resolve once I have EAPI8
> support, i.e. get new portage.
> 
> 
> cu
>   Gerrit

With a year old system you should question if reinstalling your system after a 
back up of configuration and data files would be a smarter approach.  If you 
*must* upgrade your current installation for learning or as an experiment, 
then this is something which has been done before.  You will need to walk 
through all the major upgrades of the last year, by using git to gradually 
catch up with the latest stable versions.  This means a lot of packages will 
have to be updated a number of times.

Personally, I'd back up /home /etc and world file, plus any databases or 
websites if stored under /var/, then untar the latest stage 3 tarball and 
update @system and @world.  Overall it should be a *much* faster approach to 
allow you to bring your installation up to date.

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] python, my nemesis

[Gerrit Kuehn]

On Mon, 20 Sep 2021 09:18:23 -0400
Michael Orlitzky  wrote:

> > ---
> > ~ # cat /etc/portage/package.use/py 
> > */* PYTHON_TARGETS: -* python3_9 python3_8
> > */* PYTHON_SINGLE_TARGET: -* python3_9
> > ---  

> You should probably not mess with these variables until after your
> system is 100% updated and consistent. And even then, probably not.
> 
> With our package manager written in python, you often need old python
> stuff to build the new python stuff, and disabling the old python
> stuff will throw a wrench into that. Even in situations where
> technically some upgrade path exists, the complexity of the python
> dependencies often means that the package manager will give up before
> it finds the solution unless the solution is obvious. By tweaking
> those variables, you make the solution less obvious to it.

Well, this was the suggested way to go, see
https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html

But also when trying "emerge -1vUD @world" (be it with or without the
package.use settings), I get stuck in conflicts (mostly on perl and
setuptools). perl issues would probably resolve once I have EAPI8
support, i.e. get new portage.


cu
  Gerrit

Re: [gentoo-user] python, my nemesis

[Michael Orlitzky]

On Mon, 2021-09-20 at 14:50 +0200, Gerrit Kuehn wrote:
> Hi,
> 
> I'd like to update a system that is about 1 year old. I have python3.7
> and python 3.8 installed, 3.7 is the default. After updating the repo,
> it was suggested to update portage first (probably useful to support
> new EAPI versions).
> I read about updating to python 3.9, so I created
> 
> ---
> ~ # cat /etc/portage/package.use/py 
> */* PYTHON_TARGETS: -* python3_9 python3_8
> */* PYTHON_SINGLE_TARGET: -* python3_9
> ---

You should probably not mess with these variables until after your
system is 100% updated and consistent. And even then, probably not.

With our package manager written in python, you often need old python
stuff to build the new python stuff, and disabling the old python stuff
will throw a wrench into that. Even in situations where technically
some upgrade path exists, the complexity of the python dependencies
often means that the package manager will give up before it finds the
solution unless the solution is obvious. By tweaking those variables,
you make the solution less obvious to it.

[gentoo-user] python, my nemesis

[Gerrit Kuehn]

Hi,

I'd like to update a system that is about 1 year old. I have python3.7
and python 3.8 installed, 3.7 is the default. After updating the repo,
it was suggested to update portage first (probably useful to support
new EAPI versions).
I read about updating to python 3.9, so I created

---
~ # cat /etc/portage/package.use/py 
*/* PYTHON_TARGETS: -* python3_9 python3_8
*/* PYTHON_SINGLE_TARGET: -* python3_9
---

as suggested. However, there are still collisions (and I don't really
understand them):

---
 ~ # emerge -p --oneshot sys-apps/portage

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N ] acct-group/portage-0 
[ebuild  N ] acct-user/portage-0 
[ebuild U  ] sys-devel/automake-1.16.4 [1.16.1-r1] USE="-test%" 
[ebuild  NS] dev-lang/python-3.9.6_p2 [2.7.18-r2, 3.7.8-r2, 3.8.5]
USE="sqlite* -verify-sig%"
[ebuild U  ] dev-python/certifi-10001-r1 [10001]
PYTHON_TARGETS="python3_8* python3_9* (-python3_10)"
[ebuild U  ] dev-python/setuptools-57.4.0-r2 [46.4.0-r3]
PYTHON_TARGETS="python3_8* python3_9* (-python3_10)"
[ebuild  N ] dev-python/toml-0.10.2 USE="-test"
PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_10)"
[ebuild U  ] dev-python/setuptools_scm-6.0.1-r1 [4.1.2-r1]
PYTHON_TARGETS="python3_8* python3_9* (-python3_10)"
[ebuild  N ] dev-python/charset_normalizer-2.0.4  USE="-test"
PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_10)"
[ebuild U ] dev-python/idna-3.2 [2.10-r1] PYTHON_TARGETS="python3_8*
python3_9* (-python3_10)"
[ebuild   R] dev-python/PySocks-1.7.1-r1 PYTHON_TARGETS="python3_8*
python3_9* (-python3_10)"
[ebuild U  ] dev-python/urllib3-1.26.6
[1.25.10-r1] PYTHON_TARGETS="python3_8* python3_9* (-python3_10)"
[ebuild U  ] dev-python/requests-2.26.0 [2.24.0-r1]
PYTHON_TARGETS="python3_8* python3_9* (-python3_10)"
[ebuild U  ] app-portage/gemato-16.2 [15.2]
PYTHON_TARGETS="python3_8* python3_9* (-python3_10)"
[ebuild U  ] sys-apps/portage-3.0.20-r6 [3.0.4-r1]
PYTHON_TARGETS="python3_8* python3_9* (-python3_10)" 

!!! Multiple package instances within a single package slot have been
pulled !!! into the dependency graph, resulting in a slot conflict:

sys-apps/portage:0

  (sys-apps/portage-3.0.20-r6:0/0::gentoo, ebuild scheduled for merge)
  USE="(ipc) native-extensions rsync-verify xattr -apidoc -build -doc
  -gentoo-dev (-selinux) -test" ABI_X86="(64)"
  PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_10)" pulled in
  by sys-apps/portage (Argument)

  (sys-apps/portage-3.0.4-r1-3:0/0::gentoo, installed) USE="(ipc)
  native-extensions rsync-verify xattr -apidoc -build -doc -gentoo-dev
  (-selinux) -test" ABI_X86="(64)" PYTHON_TARGETS="python3_7 (-pypy3)
  -python3_6 -python3_8 -python3_9" pulled in by
  
sys-apps/portage[python_targets_python3_7(-),-python_single_target_pypy3(-),-python_single_target_python3_6(-),-python_single_target_python3_7(-),-python_single_target_python3_8(-)]
  required by (app-portage/gentoolkit-0.5.0:0/0::gentoo, installed)
  USE="-test" ABI_X86="(64)" PYTHON_TARGETS="python3_7 (-pypy3)
  -python3_6 -python3_8" 

[...]
---



Plus many similar ones on packages like setuptools, gemato etc.
I don't see how to get out of this vicious circle, any hints?


cu
  Gerrit