Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Fri, 2020-03-27 at 17:36 -0500, William Hubbs wrote: > On Fri, Mar 27, 2020 at 06:54:25AM +0100, Michał Górny wrote: > > On Thu, 2020-03-26 at 22:12 +0100, Ulrich Mueller wrote: > > > > > > > > On Thu, 26 Mar 2020, William Hubbs wrote: > > > > If there's a way inside an eclass to check that the ebuild inheriting > > > > it is in ::gentoo, I will use it to die if the ebuild is in ::gentoo > > > > and this variable is set. > > > > > > Oh please, not this again. An ebuild or eclass is supposed to work the > > > same everywhere, independent of the repo it is located in. > > > > > > Why don't you simply copy the eclass to your overlay and do the changes > > > there? > > > > > > > They did, and then they complained that every few months they have to > > update it. See the big slander mail from William. > > Michal, > > I already responded here on the list and apologized for the way I > presented the original issues. > > So, I would appreciate it if you drop the slander accusation. > Apology accepted. I withdraw it. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Fri, Mar 27, 2020 at 06:54:25AM +0100, Michał Górny wrote: > On Thu, 2020-03-26 at 22:12 +0100, Ulrich Mueller wrote: > > > > > > > On Thu, 26 Mar 2020, William Hubbs wrote: > > > If there's a way inside an eclass to check that the ebuild inheriting > > > it is in ::gentoo, I will use it to die if the ebuild is in ::gentoo > > > and this variable is set. > > > > Oh please, not this again. An ebuild or eclass is supposed to work the > > same everywhere, independent of the repo it is located in. > > > > Why don't you simply copy the eclass to your overlay and do the changes > > there? > > > > They did, and then they complained that every few months they have to > update it. See the big slander mail from William. Michal, I already responded here on the list and apologized for the way I presented the original issues. So, I would appreciate it if you drop the slander accusation. Thanks, William signature.asc Description: Digital signature
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Thu, 2020-03-26 at 22:12 +0100, Ulrich Mueller wrote: > > > > > > On Thu, 26 Mar 2020, William Hubbs wrote: > > If there's a way inside an eclass to check that the ebuild inheriting > > it is in ::gentoo, I will use it to die if the ebuild is in ::gentoo > > and this variable is set. > > Oh please, not this again. An ebuild or eclass is supposed to work the > same everywhere, independent of the repo it is located in. > > Why don't you simply copy the eclass to your overlay and do the changes > there? > They did, and then they complained that every few months they have to update it. See the big slander mail from William. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
> On Thu, 26 Mar 2020, William Hubbs wrote: > If there's a way inside an eclass to check that the ebuild inheriting > it is in ::gentoo, I will use it to die if the ebuild is in ::gentoo > and this variable is set. Oh please, not this again. An ebuild or eclass is supposed to work the same everywhere, independent of the repo it is located in. Why don't you simply copy the eclass to your overlay and do the changes there? Ulrich signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Thu, 2020-03-26 at 13:47 -0700, Patrick McLean wrote: > On Thu, 26 Mar 2020 21:11:11 +0100 > Michał Górny wrote: > > > On Thu, 2020-03-26 at 14:13 -0500, William Hubbs wrote: > > > There are situations in which downstream overlays need to have versions > > > of python which Gentoo no longer supports in the tree. > > > > > > Currently, the only way to do this is for the overlay author to fork > > > python-utils-r1.eclass. This is highly undesirable since it creates a > > > very significant maintenance burden for the overlay author. > > > > Is it preferable to create the maintenance burden on Gentoo developers > > instead? Is it fair that paid company developers shift the burden > > on people who work on Gentoo in their free time, and already have their > > plate full? > > We have no intention of shifting maintenance burdens anywhere, we want > to make minor changes to make it easier for us to keep up. It's the > same as a Gentoo developer asking an upstream project to make minor > changes to their build system to support DESTDIR or a sandbox fix. No, that's the exact opposite of it. Supporting DESTDIR is the correct course of action that benefits a lot of people. Adding hacks to make a broken thing to pretend to work is the exact opposite of that. > > > > The simplest way would be to apply the following patch. > > > In this situation, all the overlay author > > > would have to do is adjust the PYTHON_COMPAT_ALLOW_EXTRA_IMPLS variable. > > > > ...which solves exactly zero problems because the eclass still doesn't > > support the implementation in question. The best it could do is sweep > > part of the problem under the carpet. > > We don't care if it *actually* supports it, the ebuilds in question > aren't going to be installed on modern machines. We just want it to not > blow up in the global scope. Which makes literally zero sense. If you don't want them to work, remove them. Don't ask ::gentoo to provide special support to keep ebuilds that are 100% broken. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Thu, 26 Mar 2020 21:11:11 +0100 Michał Górny wrote: > On Thu, 2020-03-26 at 14:13 -0500, William Hubbs wrote: > > There are situations in which downstream overlays need to have versions > > of python which Gentoo no longer supports in the tree. > > > > Currently, the only way to do this is for the overlay author to fork > > python-utils-r1.eclass. This is highly undesirable since it creates a > > very significant maintenance burden for the overlay author. > > Is it preferable to create the maintenance burden on Gentoo developers > instead? Is it fair that paid company developers shift the burden > on people who work on Gentoo in their free time, and already have their > plate full? We have no intention of shifting maintenance burdens anywhere, we want to make minor changes to make it easier for us to keep up. It's the same as a Gentoo developer asking an upstream project to make minor changes to their build system to support DESTDIR or a sandbox fix. > > > The simplest way would be to apply the following patch. > > In this situation, all the overlay author > > would have to do is adjust the PYTHON_COMPAT_ALLOW_EXTRA_IMPLS variable. > > ...which solves exactly zero problems because the eclass still doesn't > support the implementation in question. The best it could do is sweep > part of the problem under the carpet. We don't care if it *actually* supports it, the ebuilds in question aren't going to be installed on modern machines. We just want it to not blow up in the global scope. > Even if it miraculously works right now at all, it will probably fail or > misbehave randomly. Any eclass change might break it. Then one cheap > hack will serve as an excuse to add one more, and another.\ > > > The other option would be to move _PYTHON_ALL_IMPLS and the > > implementation of _python_impl_supported to a separate eclass, e.g. > > python-impls-r1.eclass. This eclass could be forked freely downstream > > without worrying about the other python eclasses. > > Again, this doesn't solve the problem. It just moves the problem > elsewhere. > How does this not solve the problem? Overlays could trivially support different implementations, without maintaining a lot of forks. We are quite happy to do the work to split it out to a separate eclass. > > Thoughts? > > I've already told you that if you want to fork, fork all eclasses. Then > you wouldn't have to worry about internal API going out of sync. > > Or don't autoupdate ::gentoo when eclasses change. >
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Thu, Mar 26, 2020 at 08:37:17PM +0100, David Seifert wrote: *snip* > How do you prevent some extra clever Gentoo developer from doing the following > in ::gentoo > > dev-python/foo/foo-1.ebuild: > > # don't have the time to port this right now, patches welcome > PYTHON_COMPAT_ALLOW_EXTRA_IMPLS=( python3_4 ) > PYTHON_COMPAT=( python3_4 ) > inherit distutils-r1 If there's a way inside an eclass to check that the ebuild inheriting it is in ::gentoo, I will use it to die if the ebuild is in ::gentoo and this variable is set. > Furthermore, defining PYTHON_COMPAT_ALLOW_EXTRA_IMPLS is going to break > metadata > invariance, causing tons of cache mis-hits. How do you prevent that? > > In addition, this will very quickly cause whatever overlay this is being used > in > to become invalid. Imagine I have dev-python/foo::gentoo that supports python > 3.4 and 3.5 and have dev-python/bar::sony supporting 3.4 and 3.5 sitting in a > hypothetical overlay, which depends on dev-python/foo::gentoo. Now we remove > python 3.4 from ::gentoo in python-utils-r1, and one week later we remove > python3.4 from dev-python/foo::gentoo's PYTHON_COMPAT. Now your dev- > python/bar::sony in conjunction with PYTHON_COMPAT_ALLOW_EXTRA_IMPLS has an > invalid depgraph. How do you wish to resolve this? These are both overlay maintenance questions that wouldn't affect ::gentoo, but the answer to the first one is we regenerate the metadata very often so it wouldn't be an issue, and the second one is resolved by grabbing packages and putting them in another overlay until we don't need them. > I feel like keeping up with ::gentoo is ultimately the better strategy than > trying to add backdoors to eclasses because some overlays are somewhat behind. I agree with you. However, sometimes in the real world it doesn't work that way, or it can take longer to move than Gentoo. If we can do something minimal to allow downstream overlays to do what they need to do to catch up, I think we should, especially if someone from downstream is offering to do the work. William signature.asc Description: Digital signature
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Thu, Mar 26, 2020 at 09:11:11PM +0100, Michał Górny wrote: > I've already told you that if you want to fork, fork all eclasses. Then > you wouldn't have to worry about internal API going out of sync. > > Or don't autoupdate ::gentoo when eclasses change. I also suggested something that is a corollary of this: If you effectively freeze ::gentoo to different points for different systems, then you ALSO need for generate frozen releases for your overlay and backport critical changes to those releases. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Thu, 2020-03-26 at 14:13 -0500, William Hubbs wrote: > There are situations in which downstream overlays need to have versions > of python which Gentoo no longer supports in the tree. > > Currently, the only way to do this is for the overlay author to fork > python-utils-r1.eclass. This is highly undesirable since it creates a > very significant maintenance burden for the overlay author. Is it preferable to create the maintenance burden on Gentoo developers instead? Is it fair that paid company developers shift the burden on people who work on Gentoo in their free time, and already have their plate full? > The simplest way would be to apply the following patch. > In this situation, all the overlay author > would have to do is adjust the PYTHON_COMPAT_ALLOW_EXTRA_IMPLS variable. ...which solves exactly zero problems because the eclass still doesn't support the implementation in question. The best it could do is sweep part of the problem under the carpet. Even if it miraculously works right now at all, it will probably fail or misbehave randomly. Any eclass change might break it. Then one cheap hack will serve as an excuse to add one more, and another. > The other option would be to move _PYTHON_ALL_IMPLS and the > implementation of _python_impl_supported to a separate eclass, e.g. > python-impls-r1.eclass. This eclass could be forked freely downstream > without worrying about the other python eclasses. Again, this doesn't solve the problem. It just moves the problem elsewhere. > Thoughts? I've already told you that if you want to fork, fork all eclasses. Then you wouldn't have to worry about internal API going out of sync. Or don't autoupdate ::gentoo when eclasses change. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [PATCH 0/1] allow extra implementations of python
On Thu, 2020-03-26 at 14:13 -0500, William Hubbs wrote: > There are situations in which downstream overlays need to have versions > of python which Gentoo no longer supports in the tree. > > Currently, the only way to do this is for the overlay author to fork > python-utils-r1.eclass. This is highly undesirable since it creates a > very significant maintenance burden for the overlay author. > > There are a couple of things we can do upstream to make this easier, and > I think we should do one of them. > > The simplest way would be to apply the following patch. > In this situation, all the overlay author > would have to do is adjust the PYTHON_COMPAT_ALLOW_EXTRA_IMPLS variable. > > The other option would be to move _PYTHON_ALL_IMPLS and the > implementation of _python_impl_supported to a separate eclass, e.g. > python-impls-r1.eclass. This eclass could be forked freely downstream > without worrying about the other python eclasses. > I will volunteer to do the legwork for this option if we do not like the > first one. > > I would advocate the first option however since no one has to fork > anything. > > Thoughts? > > William > > William Hubbs (1): > python.eclass: add PYTHON_COMPAT_ALLOW_EXTRA_IMPLS > > eclass/python-utils-r1.eclass | 2 ++ > 1 file changed, 2 insertions(+) > How do you prevent some extra clever Gentoo developer from doing the following in ::gentoo dev-python/foo/foo-1.ebuild: # don't have the time to port this right now, patches welcome PYTHON_COMPAT_ALLOW_EXTRA_IMPLS=( python3_4 ) PYTHON_COMPAT=( python3_4 ) inherit distutils-r1 Furthermore, defining PYTHON_COMPAT_ALLOW_EXTRA_IMPLS is going to break metadata invariance, causing tons of cache mis-hits. How do you prevent that? In addition, this will very quickly cause whatever overlay this is being used in to become invalid. Imagine I have dev-python/foo::gentoo that supports python 3.4 and 3.5 and have dev-python/bar::sony supporting 3.4 and 3.5 sitting in a hypothetical overlay, which depends on dev-python/foo::gentoo. Now we remove python 3.4 from ::gentoo in python-utils-r1, and one week later we remove python3.4 from dev-python/foo::gentoo's PYTHON_COMPAT. Now your dev- python/bar::sony in conjunction with PYTHON_COMPAT_ALLOW_EXTRA_IMPLS has an invalid depgraph. How do you wish to resolve this? I feel like keeping up with ::gentoo is ultimately the better strategy than trying to add backdoors to eclasses because some overlays are somewhat behind. Regards David
