Re: [gentoo-dev] Please migrate your distutils-r1 ebuilds to use PEP517 builds
On 3/9/24 9:32 AM, Michał Górny wrote: > Hi, > > Please consider the "legacy" build mode to be strongly deprecated, both > in distutils-r1 and upstream (to the point that sole presence of > packages installed that way triggers deprecation warnings elsewhere, > sigh). Therefore, if you haven't done that already, please look into > converting your packages to use PEP517 builds (DISTUTILS_USE_PEP517, > installing .dist-info rather than .egg*). We'd like to eventually > remove the legacy code paths from the eclass, as they are not well- > tested at this point, and they certainly are lacking, compared to > the newer code paths. Reminder as well to be *extremely* careful when looking at non-library packages, since projects using setup.py as a "pythonic Makefile" run a very high risk of being broken under PEP 517 (and this is unavoidable, since the python wheel standard doesn't support application packaging, only *.py libraries and executables). As noted in the migration guide link, you MUST check the list of installed files before and after switching and make sure the switch doesn't break anything. > This also applies to overlay maintainers, since overlays will be > affected once we remove old code paths. > > The migration guide is here: > > https://projects.gentoo.org/python/guide/migration.html#migrating-to-pep-517-builds > > While at it, please also look at replacing `distutils_enable_tests > setup.py` with one of the other test runners, as running `setup.py test` > has been deprecated upstream as well. Or running `setup.py` at all, but > the latter is less likely to suddenly stop working. > -- Eli Schwartz OpenPGP_0x84818A6819AF4A9B.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] [PATCH v2 4/7] distutils-r1.eclass: Make vars local before calling filter-lto
On 3/9/24 2:59 PM, Mike Gilbert wrote:
> On Tue, Mar 5, 2024 at 12:16 PM Michał Górny wrote:
>>
>> Make LTO filtering local to the compilation code. This avoids disabling
>> LTO for non-Python parts of an ebuild.
>>
>> Signed-off-by: Michał Górny
>> ---
>> eclass/distutils-r1.eclass | 4
>> 1 file changed, 4 insertions(+)
>>
>> diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
>> index ee1dcef24ff6..134cb39f276a 100644
>> --- a/eclass/distutils-r1.eclass
>> +++ b/eclass/distutils-r1.eclass
>> @@ -1828,6 +1828,10 @@ distutils-r1_run_phase() {
>> # Rust extensions are incompatible with C/C++ LTO compiler
>> # see e.g. https://bugs.gentoo.org/910220
>> if has cargo ${INHERITED}; then
>> + local x
>> + for x in $(all-flag-vars); do
>> + local -x "${x}=${!x}"
>> + done
>> filter-lto
>> fi
>> fi
>
> I had never thought to loop over all-flag-vars like this. Added to my
> mental code repo, thanks!
To be fair, when I suggested this in review of the patch it wasn't my
idea either. I shamelessly stole it from... flag-o-matic.eclass, where
it is used by filter-flags, replace-flags, is-flagq, strip-flags, get-flag.
filter-flags in turn is used by filter-lto, hence all-flag-vars is the
direct API that governs what variables filter-lto will modify. It seemed
fitting to single-source that information.
--
Eli Schwartz
OpenPGP_0x84818A6819AF4A9B.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] Re: RFC: banning "AI"-backed (LLM/GPT/whatever) contributions to Gentoo
On 3/9/24 4:13 PM, Duncan wrote: > I'm not saying don't use gentoo -- I'm a gentooer after all -- I'm saying > gentoo simply isn't in a good position to condemn AI for its energy > inefficiency. In fact, I'd claim that in the Gentoo case there are > demonstrably more energy efficient practical alternatives (can anyone > sanely argue otherwise?, there are binary distros after all), while in the > AI case, for some usage AI is providing practical solutions where there > simply /weren't/ practical solutions /at/ /all/ before. In others, > availability and scale was practically and severely cost-limiting compared > to the situation with AI. At least in those cases despite high energy > usage, AI *is* the most efficient -- arguably including energy efficient > -- practical alternative, being the _only_ practical alternative, at least > at scale. Can Gentoo _ever_ be called the _only_ practical alternative, > at scale or not? > > Over all, I'd suggest that Gentoo is in as bad or worse a situation in > terms of most energy efficient practical alternative than AI, so it simply > can't credibly make the energy efficiency argument against AI. Debian/ > RedHat/etc, perhaps, a case could be reasonably made at least, Gentoo, no, > not credibly. FWIW I am not really convinced of this claim... gentoo is not a monoculture, I could have installed Gentoo in 2012 and was strongly tempted but did not because it didn't have binpkgs, but being an early adopter of https://www.gentoo.org/news/2023/12/29/Gentoo-binary.html is the single reason I have a Gentoo system today. There you go, Gentoo is a binary distro. (If you want it to be one.) You are not required to waste energy in order to use Gentoo. Leaving that aside, I think it's a bit of a red herring to claim that one must be *as energy efficient as possible* in order to have the right to criticize technologies that use orders of magnitude more energy and don't come with an option to avoid spending said energy. You also note that AI is providing practical solutions "where none existed before, for some cases". But I really, really, REALLY don't think this is the case for AI-backed contributions to Gentoo, which plainly do have an exceedingly practical solution that has been in use for a couple decades so far. So we could perhaps agree that LLMs may not be intrinsically an impractical energy waste, but using them to contribute to Gentoo *is*? :) -- Eli Schwartz OpenPGP_0x84818A6819AF4A9B.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
[gentoo-dev] Re: RFC: banning "AI"-backed (LLM/GPT/whatever) contributions to Gentoo
Michał Górny posted on Sat, 09 Mar 2024 16:04:58 +0100 as excerpted: > On Fri, 2024-03-08 at 03:59 +, Duncan wrote: >> Robin H. Johnson posted on Tue, 5 Mar 2024 06:12:06 + as excerpted: >> >> > The energy waste argument is also one that needs to be made >> > carefully: >> >> Indeed. In a Gentoo context, condemning AI for the computative energy >> waste? Maybe someone could argue that effectively. That someone isn't >> Gentoo. Something about people living in glass houses throwing >> stones... > > Could you support that claim with actual numbers? Particularly, > on average energy use specifically due to use of Gentoo on machines vs. > energy use of dedicated data centers purely for training LLMs? I'm not > even talking of all the energy wasted as a result of these LLMs at work. Fair question. Actual numbers? No. But... I'm not saying don't use gentoo -- I'm a gentooer after all -- I'm saying gentoo simply isn't in a good position to condemn AI for its energy inefficiency. In fact, I'd claim that in the Gentoo case there are demonstrably more energy efficient practical alternatives (can anyone sanely argue otherwise?, there are binary distros after all), while in the AI case, for some usage AI is providing practical solutions where there simply /weren't/ practical solutions /at/ /all/ before. In others, availability and scale was practically and severely cost-limiting compared to the situation with AI. At least in those cases despite high energy usage, AI *is* the most efficient -- arguably including energy efficient -- practical alternative, being the _only_ practical alternative, at least at scale. Can Gentoo _ever_ be called the _only_ practical alternative, at scale or not? Over all, I'd suggest that Gentoo is in as bad or worse a situation in terms of most energy efficient practical alternative than AI, so it simply can't credibly make the energy efficiency argument against AI. Debian/ RedHat/etc, perhaps, a case could be reasonably made at least, Gentoo, no, not credibly. That isn't to say that Gentoo can't credibly take an anti-AI position based on the /other/ points discussed in-thread. But energy usage is just not an argument that can be persuasively made by Gentoo, thereby bringing down the credibility of the other arguments made with it that are otherwise viable. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] [PATCH v2 4/7] distutils-r1.eclass: Make vars local before calling filter-lto
On Tue, Mar 5, 2024 at 12:16 PM Michał Górny wrote:
>
> Make LTO filtering local to the compilation code. This avoids disabling
> LTO for non-Python parts of an ebuild.
>
> Signed-off-by: Michał Górny
> ---
> eclass/distutils-r1.eclass | 4
> 1 file changed, 4 insertions(+)
>
> diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
> index ee1dcef24ff6..134cb39f276a 100644
> --- a/eclass/distutils-r1.eclass
> +++ b/eclass/distutils-r1.eclass
> @@ -1828,6 +1828,10 @@ distutils-r1_run_phase() {
> # Rust extensions are incompatible with C/C++ LTO compiler
> # see e.g. https://bugs.gentoo.org/910220
> if has cargo ${INHERITED}; then
> + local x
> + for x in $(all-flag-vars); do
> + local -x "${x}=${!x}"
> + done
> filter-lto
> fi
> fi
I had never thought to loop over all-flag-vars like this. Added to my
mental code repo, thanks!
Re: [gentoo-dev] Re: RFC: banning "AI"-backed (LLM/GPT/whatever) contributions to Gentoo
On Fri, 2024-03-08 at 03:59 +, Duncan wrote: > Robin H. Johnson posted on Tue, 5 Mar 2024 06:12:06 + as excerpted: > > > The energy waste argument is also one that needs to be made carefully: > > Indeed. In a Gentoo context, condemning AI for the computative energy > waste? Maybe someone could argue that effectively. That someone isn't > Gentoo. Something about people living in glass houses throwing stones... Could you support that claim with actual numbers? Particularly, on average energy use specifically due to use of Gentoo on machines vs. energy use of dedicated data centers purely for training LLMs? I'm not even talking of all the energy wasted as a result of these LLMs at work. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] RFC: banning "AI"-backed (LLM/GPT/whatever) contributions to Gentoo
On Fri, 2024-03-01 at 07:06 +, Sam James wrote: > Another person approached me after this RFC and asked whether tooling > restricted to the current repo would be okay. For me, that'd be mostly > acceptable, given it won't make suggestions based on copyrighted code. I think an important question is: how is it restricted? Are we talking about a tool that was clearly trained on specific code, or about a tool that was trained on potentially copyright material, then artificially restricted to the repository (to paper over the concerns)? Can we trust the latter? -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] RFC: banning "AI"-backed (LLM/GPT/whatever) contributions to Gentoo
On Tue, 2024-02-27 at 18:04 +, Sam James wrote: > I'm a bit worried this is slightly performative - which is not a dig at > you at all - given we can't really enforce it, and it requires honesty, > but that's also not a reason to not try ;) I don't think it's really possible or feasible to reliably detect such contributions, and even if it were, I don't think we want to go as far as to actively pursue anything that looks like one. The point of the policy is rather to make a statement that we don't want these, and to kindly ask users not to do that. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
[gentoo-dev] Please migrate your distutils-r1 ebuilds to use PEP517 builds
Hi, Please consider the "legacy" build mode to be strongly deprecated, both in distutils-r1 and upstream (to the point that sole presence of packages installed that way triggers deprecation warnings elsewhere, sigh). Therefore, if you haven't done that already, please look into converting your packages to use PEP517 builds (DISTUTILS_USE_PEP517, installing .dist-info rather than .egg*). We'd like to eventually remove the legacy code paths from the eclass, as they are not well- tested at this point, and they certainly are lacking, compared to the newer code paths. This also applies to overlay maintainers, since overlays will be affected once we remove old code paths. The migration guide is here: https://projects.gentoo.org/python/guide/migration.html#migrating-to-pep-517-builds While at it, please also look at replacing `distutils_enable_tests setup.py` with one of the other test runners, as running `setup.py test` has been deprecated upstream as well. Or running `setup.py` at all, but the latter is less likely to suddenly stop working. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
[gentoo-dev] Last rites: dev-python/anyqt
# Andrew Ammerlaan (2024-03-09) # Requires pyside2/shiboken2, which is unmaintained upstream and # not compatible with llvm-16 and up or python-3.12 and up. # No upstream activity for 2 years. # dev-python/QtPy is an alternative that does support Qt6. # Removal on: 2024-04-09. Bug #926548 dev-python/anyqt
