Re: [gentoo-dev] [PATCHES] python-r1.eclass: any-of dep API support
When it'll be possible to start to use it? On Sat, May 20, 2017 at 8:30 PM, Michał Górnywrote: > Hi, everyone. > > Here's a set of patches inspired by the recent Sphinx dependency > discussion. They make python-r1 (and therefore distutils-r1) capable > of any-of dependency logic similar to the one used in python-any-r1. > > The basic goal is relatively simple -- to improve handling of pure > build-time dependencies in the eclass. It solves two common problems: > > a. dependencies on packages that support only a subset of PYTHON_COMPAT, > > b. dependencies that need to be implementation-bound between themselves >(e.g. Sphinx plugins). > > The new API improves both of those cases significantly. For the former, > we no longer force user to select additional targets via REQUIRED_USE -- > instead, we just any-of dependencies + python_check_deps() to select > implementation independently of whether it is enabled or not. > > For the latter, we no longer have to force all targets of the package > on all the involved dependencies. Again, using any-of dep > and appropriate python_check_deps() we can enforce a single (any) > target throughout all the packages and use it. > > The first three patches do some code refactoring that makes the change > easier and possibly improves maintainability of the code. The next two > patches add support for python_check_deps() and python_gen_any_dep() > respectively. The last two patches provide examples for both use cases > mentioned. > > Please review. > > -- > Best regards, > Michał Górny > > >
Re: [gentoo-dev] meson.eclass third draft
On Fri, May 05, 2017 at 10:35:43AM -0500, William Hubbs wrote: > All, > > here is the third (and hopefully final) draft of meson.eclass. I am > leaving the code in for the cross support but commented so all I need to > do later is add toolchain-funcs back to inherit and uncomment the code > once I know how to write the cross file, which shouldn't take too long. > I added a link to the documentation for it in the comments. > > Thanks, This has been committed to the tree. https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=844ae0dea3592e0c1a21b4e9dae0662f535955e1 Thanks, William signature.asc Description: Digital signature
[gentoo-dev] [PATCH] ssl-cert.eclass: Set default key length to 4096 bit and allow to specify message digest
--- eclass/ssl-cert.eclass | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/eclass/ssl-cert.eclass b/eclass/ssl-cert.eclass index 6bec347234d..bfe5291314c 100644 --- a/eclass/ssl-cert.eclass +++ b/eclass/ssl-cert.eclass @@ -1,4 +1,4 @@ -# Copyright 1999-2014 Gentoo Foundation +# Copyright 1999-2017 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 # @ECLASS: ssl-cert.eclass @@ -66,7 +66,8 @@ gen_cnf() { # These can be overridden in the ebuild SSL_DAYS="${SSL_DAYS:-730}" - SSL_BITS="${SSL_BITS:-1024}" + SSL_BITS="${SSL_BITS:-4096}" + SSL_MD="${SSL_MD:-sha256}" SSL_COUNTRY="${SSL_COUNTRY:-US}" SSL_STATE="${SSL_STATE:-California}" SSL_LOCALITY="${SSL_LOCALITY:-Santa Barbara}" @@ -166,6 +167,7 @@ gen_crt() { if [ "${1}" ] ; then ebegin "Generating self-signed X.509 Certificate for CA" openssl x509 -extfile "${SSL_CONF}" \ + -${SSL_MD} \ -days ${SSL_DAYS} -req -signkey "${base}.key" \ -in "${base}.csr" -out "${base}.crt" &>/dev/null else @@ -173,7 +175,7 @@ gen_crt() { ebegin "Generating authority-signed X.509 Certificate" openssl x509 -extfile "${SSL_CONF}" \ -days ${SSL_DAYS} -req -CAserial "${SSL_SERIAL}" \ - -CAkey "${ca}.key" -CA "${ca}.crt" \ + -CAkey "${ca}.key" -CA "${ca}.crt" -${SSL_MD} \ -in "${base}.csr" -out "${base}.crt" &>/dev/null fi eend $? -- 2.13.0
Re: [gentoo-dev] Fixing elasticsearch maintainer
> > Tomas, please don't go this road. We all know Patrick does... Oh please cut it. -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Fixing elasticsearch maintainer
On 05/20/2017 11:06 PM, Michał Górny wrote: > On sob, 2017-05-20 at 22:51 +0200, Kristian Fiskerstrand wrote: >> On 05/20/2017 10:46 PM, Michał Górny wrote: >>> Tomas, please don't go this road. We all know Patrick does a shitty job >>> as Gentoo developer, both technically and socially but you do not have >>> to try to match him. >> >> Was this comment really necessary? >> > > Yes, it was. It's enough that Patrick does public lashing here, I don't > want Tomas to do the counter-lashing. Show's over, move along, etc. > I'm more concerned about the tone of the message, we really should strive to be more collegial in our commenting in official Gentoo channels. In this case it seems to be a matter of communication between various maintainers of a package that likely should belong in private before being aired on a public mailing list. Given the number of maintainers, I'm wondering if it shouldn't be a project handling it to begin with, but certain parts of the history looks odd from the outside. -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Fixing elasticsearch maintainer
On sob, 2017-05-20 at 22:51 +0200, Kristian Fiskerstrand wrote: > On 05/20/2017 10:46 PM, Michał Górny wrote: > > Tomas, please don't go this road. We all know Patrick does a shitty job > > as Gentoo developer, both technically and socially but you do not have > > to try to match him. > > Was this comment really necessary? > Yes, it was. It's enough that Patrick does public lashing here, I don't want Tomas to do the counter-lashing. Show's over, move along, etc. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Fixing elasticsearch maintainer
On 05/20/2017 10:46 PM, Michał Górny wrote: > Tomas, please don't go this road. We all know Patrick does a shitty job > as Gentoo developer, both technically and socially but you do not have > to try to match him. Was this comment really necessary? -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Fixing elasticsearch maintainer
On sob, 2017-05-20 at 21:57 +0200, Tomas Mozes wrote: > On Fri, May 19, 2017 at 6:50 PM, Patrick Lauerwrote: > > > I tried removing proxy-maint from metadata after multiple discussions > > failed. Extra happiness towards monsieurp "but the GH PR is over 3 days > > old, I have to commit" and gokturk "Yes I understand. I commit anyway" > > > > This has been an uphill struggle since about October, around New Year I > > stopped actively caring, and since these two commits: > > > > 12c3eacda7c4d23686eaf10eab21d810cc95ea49 > > f42d6679c038c3efcc257d38547267d01823aea9 > > > > I see no way to fix this situation that doesn't involve a review board in > > front of all proxy-maint commits. Because we discussed this in IRC, and > > still ... "but is open bug" > > > > However, as far as I'm aware none of this happened. Note that I might > > > have missed the mail, or it might have been sent before I joined -- > > > correct me if that is the case. > > > > > > > There were multiple discussions in IRC, which the involved people usually > > forgot within about 20 minutes and then resumed doing stuff. > > > > I tried removing proxy-maint from metadata, which was reverted (sooo how > > does one *not* have constant interference?) > > > > As Alec pointed out, it is a normal procedure in Gentoo to remove old > > > versions of software if there is no explicit indication that they need > > > to be kept. Therefore, I don't see anything wrong with the proxied > > > maintainer wishing to clean the old versions up and/or not requesting > > > your explicit permission for that. If you needed the old versions, you > > > should have made that clear. > > > > > > > One could ask, maybe. I guess I can (mis)understand this to mean that I > > can do with packages with you in metadata what I want because ... err... > > shiny! > > > > I should also point out that the steps you've taken (and listed in this > > > mail) are not really relevant. They make you look like a sloppy > > > maintainer, and a bad Gentoo developer at the best -- and I doubt anyone > > > would connect removing proxy-maint team with a necessity of keeping > > > an old version. > > > > > > > The cooperation that I had with ferki was pretty good (mostly because we > > sat next to each other in the office). The contributions from Tomas were on > > average pretty ok, just needed some minor cleanups here and there. > > > > The blind "but PR is open for 3 days" commits from proxy-maint made it > > extremely hard to review what changed in a timely manner, so that I > > basically didn't want to care for this pile of stupid for the last, ahem, 6 > > months or so. Especially since whenever I wanted to review things some > > joker made some new changes which made me go "eh whut how you? banana > > banana!" so I pushed reviewing a week into the future and ... > > > > I have no idea how I could have fixed this without the QA+Comrel banhammer > > combo, which is a totally insane "fix" to a problem that shouldn't even > > exist. But I see no other options how to make people understand that "No > > means no". > > > > Is this the new normal? > > > > > > Everybody makes mistakes, but let's look from another perspective. > Elasticsearch 5.0 got released - a new major version. You did the bump, but > it didn't work (it was clearly pushed to the repo untested as > openrc/systemd version both failed: > https://bugs.gentoo.org/show_bug.cgi?id=598732 > https://bugs.gentoo.org/show_bug.cgi?id=597454 > Why didn't you fix it yourself? > > Same for logstash: > https://bugs.gentoo.org/show_bug.cgi?id=597452 > https://bugs.gentoo.org/show_bug.cgi?id=598422 > Why did you commit a broken ebuild to the repo and never fixed it after > yourself? These bugs were open for weeks and months, not days... Tomas, please don't go this road. We all know Patrick does a shitty job as Gentoo developer, both technically and socially but you do not have to try to match him. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Fixing elasticsearch maintainer
On Fri, May 19, 2017 at 6:50 PM, Patrick Lauerwrote: > I tried removing proxy-maint from metadata after multiple discussions > failed. Extra happiness towards monsieurp "but the GH PR is over 3 days > old, I have to commit" and gokturk "Yes I understand. I commit anyway" > > This has been an uphill struggle since about October, around New Year I > stopped actively caring, and since these two commits: > > 12c3eacda7c4d23686eaf10eab21d810cc95ea49 > f42d6679c038c3efcc257d38547267d01823aea9 > > I see no way to fix this situation that doesn't involve a review board in > front of all proxy-maint commits. Because we discussed this in IRC, and > still ... "but is open bug" > > However, as far as I'm aware none of this happened. Note that I might >> have missed the mail, or it might have been sent before I joined -- >> correct me if that is the case. >> > > There were multiple discussions in IRC, which the involved people usually > forgot within about 20 minutes and then resumed doing stuff. > > I tried removing proxy-maint from metadata, which was reverted (sooo how > does one *not* have constant interference?) > > As Alec pointed out, it is a normal procedure in Gentoo to remove old >> versions of software if there is no explicit indication that they need >> to be kept. Therefore, I don't see anything wrong with the proxied >> maintainer wishing to clean the old versions up and/or not requesting >> your explicit permission for that. If you needed the old versions, you >> should have made that clear. >> > > One could ask, maybe. I guess I can (mis)understand this to mean that I > can do with packages with you in metadata what I want because ... err... > shiny! > > I should also point out that the steps you've taken (and listed in this >> mail) are not really relevant. They make you look like a sloppy >> maintainer, and a bad Gentoo developer at the best -- and I doubt anyone >> would connect removing proxy-maint team with a necessity of keeping >> an old version. >> > > The cooperation that I had with ferki was pretty good (mostly because we > sat next to each other in the office). The contributions from Tomas were on > average pretty ok, just needed some minor cleanups here and there. > > The blind "but PR is open for 3 days" commits from proxy-maint made it > extremely hard to review what changed in a timely manner, so that I > basically didn't want to care for this pile of stupid for the last, ahem, 6 > months or so. Especially since whenever I wanted to review things some > joker made some new changes which made me go "eh whut how you? banana > banana!" so I pushed reviewing a week into the future and ... > > I have no idea how I could have fixed this without the QA+Comrel banhammer > combo, which is a totally insane "fix" to a problem that shouldn't even > exist. But I see no other options how to make people understand that "No > means no". > > Is this the new normal? > > Everybody makes mistakes, but let's look from another perspective. Elasticsearch 5.0 got released - a new major version. You did the bump, but it didn't work (it was clearly pushed to the repo untested as openrc/systemd version both failed: https://bugs.gentoo.org/show_bug.cgi?id=598732 https://bugs.gentoo.org/show_bug.cgi?id=597454 Why didn't you fix it yourself? Same for logstash: https://bugs.gentoo.org/show_bug.cgi?id=597452 https://bugs.gentoo.org/show_bug.cgi?id=598422 Why did you commit a broken ebuild to the repo and never fixed it after yourself? These bugs were open for weeks and months, not days...
Re: [gentoo-dev] Fixing elasticsearch maintainer
On Fri, May 19, 2017 at 6:38 PM, Patrick Lauerwrote: > > I "was" package maintainer and relied on these versions. > > If I as maintainer have no control over such things, why am I maintainer, > and why do I need an overlay? > > > ... that sounds exquisitely confused, I have no idea why this discussion > even exists. > > > The elastic stack is moving pretty fast. As you might have noticed, several security issues were fixed during the development, but still we kept really old versions in portage. I'm fine with keeping older branches if they are maintained, but we kept unmaintained branches (for example we had almost 20 versions of elasticseach).
Re: [gentoo-dev] Fixing elasticsearch maintainer
On Wed, May 17, 2017 at 6:38 PM, Patrick Lauerwrote: > For some strange reason I was listed there as maintainer, but since no one > wanted to listen to my ideas I guess I wasn't. So now last person who > touched it gets stuck with it. > For elasticsearch, you added yourself to the maintainers, so why are you surprised to be there (e6175815b5792f09acd90627af5fe23f616ad245)? You also added yourself to other packages, for example elasticsearch-cutor (bd21ed1ef20cb2d27a87a4dadf780565236a72cd) without asking the maintainer (me at that time). And where exactly have you expressed your ideas? > > Since proxy-maint refuses to be removed from packages (especially since > they were unconditionally added to all packages with a non-gentoo-dev > maintainer in metadata) they are the de facto maintainers, and overrule > everything else. > I've tried multiple strategies including removing them from metadata, but > ... see app-admin/elasticsearch, proxy-maint is like the toe fungus that > always comes back (e.g. commit f0925c10834464e62ce7209f2afa7797b594d350 ) > Why did you add me as the maintainer of elasticsearch without asking and then removing proxy-maint so I cannot make any changes to elasticsearch at all? If you want to make all the changes, then I don't need to be there and I can just open regular bug reports and you merge them. > > Sometimes it's almost absurdly funny, especially when you commit > RESTRICT="test" because tests fail reliably just to have that reverted. > (See dev-python/elasticsearch-py ) > Regarding RESTRICT="test", I was the one to revert your change because I thought it's a mistake as the tests passed for me. As soon as we discussed this via irc that it only happens in chroot, it got back. Now a few days ago @mrueg accidentally removed the restriction but after mailing him he reverted his change so it's as you made it. > > Bonus mention: > bbdc5412061adf598ed935697441a7d6b05f7614 > app-admin/logstash-bin: drop old > > Signed-off-by: Andrew Savchenko > > That removed the versions I was using, so I better maintain the versions I > use in an overlay. Well ok then. > > Logstash is yet another package where you made yourself a maintainer without asking the maintainer (again, it was me). I don't remember you ever wanting to keep the old version of logstash in the repo. Plus, you don't need to update and you can mask the update. If you really want to use and old version (we already had multiple version of 5.x in the tree by that time), you can keep in your overlay.
[gentoo-dev] toolchain meeting agenda for today 19:00 UTC #gentoo-toolchain
So here's the agenda. It's been mostly assembled by tamiko, who may unfortunately not make it tonight. I'll try to stand in... - a short hello :-) - Outstanding CVEs for binutils, elfutils, gcc, and glibc [1] * Who handled CVEs in the past? * Who wants to take care of individual packages? * Stabilizing newer binutils and glibc is urgent. - current state of gcc porting (e.g. troubles with +ssp on alpha) * We have to fix the gcc compilation issues * Open gcc bugs [2] - full multilib compliant toolchain.eclass for gcc-7 - How to tackle all open toolchain bugs? [3] * Volunteers for individual packages? (e.g. binutils, glibc, etc.) - Globally setting -std=c++14 in upcoming 17.0 profiles - What about quick (bi-)monthly meetings (~15 minutes) on IRC to keep ourselves updated? Cheers, Andreas [1] https://bugs.gentoo.org/buglist.cgi?email1=toolchain %40gentoo.org=security %40gentoo.org_to2=1=1=substring=substring_id=3538312_format=advanced=--- [2] https://bugs.gentoo.org/buglist.cgi?email1=toolchain %40gentoo.org_to1=1=substring_id=3538318_format=advanced=--- _desc=gcc_desc_type=allwordssubstr [3] https://bugs.gentoo.org/buglist.cgi?email1=toolchain %40gentoo.org_to1=1=substring_id=3538326_format=advanced=--- -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Integrating Ada into toolchain.eclass, again
Luke A. Guest wrote: > Thoughts? I can't comment on your strategy, but I do agree with and support your goals of being able to use more Ada in Gentoo. Thanks to you and others for your work on this! :) //Peter
[gentoo-dev] [PATCH 6/7] app-portage/gentoopm: Use any-of deps (example)
--- app-portage/gentoopm/gentoopm-.ebuild | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/app-portage/gentoopm/gentoopm-.ebuild b/app-portage/gentoopm/gentoopm-.ebuild index a4529c98bc9b..220247442d2d 100644 --- a/app-portage/gentoopm/gentoopm-.ebuild +++ b/app-portage/gentoopm/gentoopm-.ebuild @@ -21,10 +21,12 @@ RDEPEND=" >=sys-apps/pkgcore-0.9.4[${PYTHON_USEDEP}] >=sys-apps/portage-2.1.10.3[${PYTHON_USEDEP}] >=sys-apps/paludis-3.0.0_pre20170219[python,${PYTHON_USEDEP}] )" -DEPEND="doc? ( dev-python/epydoc[$(python_gen_usedep python2_7)] )" +DEPEND="doc? ( $(python_gen_any_dep 'dev-python/epydoc[${PYTHON_USEDEP}]' python2_7) )" PDEPEND="app-eselect/eselect-package-manager" -REQUIRED_USE="doc? ( $(python_gen_useflags python2_7) )" +python_check_deps() { + has_version "dev-python/epydoc[${PYTHON_USEDEP}]" +} src_configure() { use doc && DISTUTILS_ALL_SUBPHASE_IMPLS=( python2.7 ) -- 2.13.0
[gentoo-dev] [PATCH 2/7] python-r1.eclass: Move PYTHON_COMPAT_OVERRIDE warning into flag check
Move the PYTHON_COMPAT_OVERRIDE warning from _python_obtain_impls() to _python_validate_useflags(). Since the latter function is the only point where the former is called, this is a purely cosmetic change at the moment. However, it makes it possible to reuse the warning in additional places without the necessity of setting MULTIBUILD_VARIANTS. --- eclass/python-r1.eclass | 31 ++- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass index 5eaa802e06b9..ae9e3806e729 100644 --- a/eclass/python-r1.eclass +++ b/eclass/python-r1.eclass @@ -242,10 +242,25 @@ if [[ ! ${_PYTHON_R1} ]]; then # @FUNCTION: _python_validate_useflags # @INTERNAL # @DESCRIPTION: -# Enforce the proper setting of PYTHON_TARGETS. +# Enforce the proper setting of PYTHON_TARGETS, if PYTHON_COMPAT_OVERRIDE +# is not in effect. If it is, just warn that the flags will be ignored. _python_validate_useflags() { debug-print-function ${FUNCNAME} "${@}" + if [[ ${PYTHON_COMPAT_OVERRIDE} ]]; then + if [[ ! ${_PYTHON_COMPAT_OVERRIDE_WARNED} ]]; then + ewarn "WARNING: PYTHON_COMPAT_OVERRIDE in effect. The following Python" + ewarn "implementations will be enabled:" + ewarn + ewarn " ${PYTHON_COMPAT_OVERRIDE}" + ewarn + ewarn "Dependencies won't be satisfied, and PYTHON_TARGETS will be ignored." + _PYTHON_COMPAT_OVERRIDE_WARNED=1 + fi + # we do not use flags with PCO + return + fi + local i for i in "${_PYTHON_SUPPORTED_IMPLS[@]}"; do @@ -490,23 +505,13 @@ python_copy_sources() { # @DESCRIPTION: # Set up the enabled implementation list. _python_obtain_impls() { - if [[ ${PYTHON_COMPAT_OVERRIDE} ]]; then - if [[ ! ${_PYTHON_COMPAT_OVERRIDE_WARNED} ]]; then - ewarn "WARNING: PYTHON_COMPAT_OVERRIDE in effect. The following Python" - ewarn "implementations will be enabled:" - ewarn - ewarn " ${PYTHON_COMPAT_OVERRIDE}" - ewarn - ewarn "Dependencies won't be satisfied, and PYTHON_TARGETS will be ignored." - _PYTHON_COMPAT_OVERRIDE_WARNED=1 - fi + _python_validate_useflags + if [[ ${PYTHON_COMPAT_OVERRIDE} ]]; then MULTIBUILD_VARIANTS=( ${PYTHON_COMPAT_OVERRIDE} ) return fi - _python_validate_useflags - MULTIBUILD_VARIANTS=() local impl -- 2.13.0
[gentoo-dev] [PATCH 7/7] dev-python/backports-unittest-mock: Use any-of API for Sphinx (example)
--- .../backports-unittest-mock/backports-unittest-mock-1.2.1.ebuild | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/dev-python/backports-unittest-mock/backports-unittest-mock-1.2.1.ebuild b/dev-python/backports-unittest-mock/backports-unittest-mock-1.2.1.ebuild index c3c3de101526..5d053726f18b 100644 --- a/dev-python/backports-unittest-mock/backports-unittest-mock-1.2.1.ebuild +++ b/dev-python/backports-unittest-mock/backports-unittest-mock-1.2.1.ebuild @@ -23,10 +23,10 @@ IUSE="doc test" RDEPEND="dev-python/mock[${PYTHON_USEDEP}]" DEPEND="dev-python/setuptools[${PYTHON_USEDEP}] >=dev-python/setuptools_scm-1.15.0[${PYTHON_USEDEP}] - doc? ( + doc? ( $(python_gen_any_dep ' dev-python/sphinx[${PYTHON_USEDEP}] dev-python/rst-linker[${PYTHON_USEDEP}] - ) + ') ) test? ( ${RDEPEND} >=dev-python/pytest-2.8[${PYTHON_USEDEP}] @@ -36,6 +36,11 @@ DEPEND="dev-python/setuptools[${PYTHON_USEDEP}] S="${WORKDIR}/${MY_PN}-${PV}" +python_check_deps() { + has_version "dev-python/sphinx[${PYTHON_USEDEP}]" && + has_version "dev-python/rst-linker[${PYTHON_USEDEP}]" +} + python_compile_all() { if use doc; then cd docs || die -- 2.13.0
[gentoo-dev] [PATCH 5/7] python-r1.eclass: Add python_gen_any_dep, to create any-of deps
Add a python_gen_any_dep() function similar to the one in python-any-r1 to facilitate creating any-of dependencies for the new python_setup syntax. --- eclass/python-r1.eclass | 98 + 1 file changed, 98 insertions(+) diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass index 2992f603cf8a..5ec23d23d8cc 100644 --- a/eclass/python-r1.eclass +++ b/eclass/python-r1.eclass @@ -468,6 +468,86 @@ python_gen_impl_dep() { echo "${matches[@]}" } +# @FUNCTION: python_gen_any_dep +# @USAGE: [...] +# @DESCRIPTION: +# Generate an any-of dependency that enforces a version match between +# the Python interpreter and Python packages. needs +# to list one or more dependencies with verbatim '${PYTHON_USEDEP}' +# references (quoted!) that will get expanded inside the function. +# Optionally, patterns may be specified to restrict the dependency +# to a subset of Python implementations supported by the ebuild. +# +# The patterns can be either fnmatch-style patterns (matched via bash +# == operator against PYTHON_COMPAT values) or '-2' / '-3' to indicate +# appropriately all enabled Python 2/3 implementations (alike +# python_is_python3). Remember to escape or quote the fnmatch patterns +# to prevent accidental shell filename expansion. +# +# This should be used along with an appropriate python_check_deps() +# that checks which of the any-of blocks were matched, and python_setup +# call that enables use of the matched implementation. +# +# Example use: +# @CODE +# DEPEND="$(python_gen_any_dep ' +# dev-python/foo[${PYTHON_USEDEP}] +# || ( dev-python/bar[${PYTHON_USEDEP}] +# dev-python/baz[${PYTHON_USEDEP}] )' -2)" +# +# python_check_deps() { +# has_version "dev-python/foo[${PYTHON_USEDEP}]" \ +# && { has_version "dev-python/bar[${PYTHON_USEDEP}]" \ +# || has_version "dev-python/baz[${PYTHON_USEDEP}]"; } +# } +# +# src_compile() { +# python_foreach_impl usual_code +# +# # some common post-build task that requires Python 2 +# python_setup -2 +# emake frobnicate +# } +# @CODE +# +# Example value: +# @CODE +# || ( +# ( +# dev-lang/python:2.7 +# dev-python/foo[python_targets_python2_7(-)?,python_single_target_python2_7(+)?] +# || ( dev-python/bar[python_targets_python2_7(-)?,python_single_target_python2_7(+)?] +# dev-python/baz[python_targets_python2_7(-)?,python_single_target_python2_7(+)?] ) +# ) +# ( +# dev-lang/python:3.3 +# dev-python/foo[python_targets_python3_3(-)?,python_single_target_python3_3(+)?] +# || ( dev-python/bar[python_targets_python3_3(-)?,python_single_target_python3_3(+)?] +# dev-python/baz[python_targets_python3_3(-)?,python_single_target_python3_3(+)?] ) +# ) +# ) +# @CODE +python_gen_any_dep() { + debug-print-function ${FUNCNAME} "${@}" + + local depstr=${1} + [[ ${depstr} ]] || die "No dependency string provided" + shift + + local i PYTHON_PKG_DEP out= + for i in "${_PYTHON_SUPPORTED_IMPLS[@]}"; do + if _python_impl_matches "${i}" "${@-*}"; then + local PYTHON_USEDEP="python_targets_${i}(-),python_single_target_${i}(+)" + python_export "${i}" PYTHON_PKG_DEP + + local i_depstr=${depstr//\$\{PYTHON_USEDEP\}/${PYTHON_USEDEP}} + # note: need to strip '=' slot operator for || deps + out="( ${PYTHON_PKG_DEP%=} ${i_depstr} ) ${out}" + fi + done + echo "|| ( ${out})" +} + # @ECLASS-VARIABLE: BUILD_DIR # @DESCRIPTION: # The current build directory. In global scope, it is supposed to @@ -608,6 +688,24 @@ python_foreach_impl() { # fi # } # @CODE +# +# Any-of mode example: +# @CODE +# DEPEND="doc? ( +# $(python_gen_any_dep 'dev-python/epydoc[${PYTHON_USEDEP}]' 'python2*') )" +# +# python_check_deps() { +# has_version "dev-python/epydoc[${PYTHON_USEDEP}]" +# } +# +# src_compile() { +# #... +# if use doc; then +# python_setup 'python2*' +# make doc +# fi +# } +# @CODE python_setup() { debug-print-function ${FUNCNAME} "${@}" -- 2.13.0
[gentoo-dev] [PATCH 4/7] python-r1.eclass: Support python_check_deps() in python_setup
Provide an alternate mode for python_setup() that behaves similarly to python-any-r1 eclass. If python_check_deps() function is declared by the ebuild, the python_setup logic switches to accepting any implementation that is in PYTHON_COMPAT, installed and satisfies python_check_deps() independently of USE flags. This new logic makes it possible to replace some of the existing REQUIRED_USE constraints for build-time dependencies with more friendly any-of dependencies. For example, if a package supports both Python 2 & Python 3 but has a purely Python 2 build-time dependency (e.g. for building documentation) we had to force Python 2 being enabled via REQUIRED_USE. Using python_check_deps() with appropriate any-of dependency, we can use Python 2 for this task without actually forcing the user to change USE flags or install the package for Python 2. --- eclass/python-r1.eclass | 48 ++-- 1 file changed, 38 insertions(+), 10 deletions(-) diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass index 9c37a20f7c2e..2992f603cf8a 100644 --- a/eclass/python-r1.eclass +++ b/eclass/python-r1.eclass @@ -231,7 +231,7 @@ _python_set_globals() { PYTHON_DEPS=${deps} PYTHON_REQUIRED_USE=${requse} PYTHON_USEDEP=${usedep} - readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP + readonly PYTHON_DEPS PYTHON_REQUIRED_USE fi } _python_set_globals @@ -563,9 +563,27 @@ python_foreach_impl() { # @FUNCTION: python_setup # @USAGE: [...] # @DESCRIPTION: -# Find the best (most preferred) Python implementation that is enabled -# and matches at least one of the patterns passed (or '*' if no patterns -# passed). Set the Python build environment up for that implementation. +# Find the best (most preferred) Python implementation that is suitable +# for running common Python code. Set the Python build environment up +# for that implementation. This function has two modes of operation: +# pure and any-of dep. +# +# The pure mode is used if python_check_deps() function is not declared. +# In this case, an implementation is considered suitable if it is +# supported (in PYTHON_COMPAT), enabled (via USE flags) and matches +# at least one of the patterns passed (or '*' if no patterns passed). +# +# Implementation restrictions in the pure mode need to be accompanied +# by appropriate REQUIRED_USE constraints. Otherwise, the eclass may +# fail at build time due to unsatisfied dependencies. +# +# The any-of dep mode is used if python_check_deps() is declared. +# In this mode, an implementation is considered suitable if it is +# supported, matches at least one of the patterns and python_check_deps() +# has successful return code. USE flags are not considered. +# +# The python_check_deps() function in the any-of mode needs to be +# accompanied by appropriate any-of dependencies. # # The patterns can be either fnmatch-style patterns (matched via bash # == operator against PYTHON_COMPAT values) or '-2' / '-3' to indicate @@ -577,11 +595,7 @@ python_foreach_impl() { # of python_foreach_impl calls (e.g. for shared processes like doc # building). python_foreach_impl sets up the build environment itself. # -# If the specific commands support only a subset of Python -# implementations, patterns need to be passed to restrict the allowed -# implementations. -# -# Example: +# Pure mode example: # @CODE # DEPEND="doc? ( dev-python/epydoc[$(python_gen_usedep 'python2*')] )" # REQUIRED_USE="doc? ( $(python_gen_useflags 'python2*') )" @@ -603,6 +617,9 @@ python_setup() { pycompat=( ${PYTHON_COMPAT_OVERRIDE} ) fi + local has_check_deps + declare -f python_check_deps >/dev/null && has_check_deps=1 + # (reverse iteration -- newest impl first) local found for (( i = ${#_PYTHON_SUPPORTED_IMPLS[@]} - 1; i >= 0; i-- )); do @@ -612,7 +629,8 @@ python_setup() { has "${impl}" "${pycompat[@]}" || continue # match USE flags only if override is not in effect - if [[ ! ${PYTHON_COMPAT_OVERRIDE} ]]; then + # and python_check_deps() is not defined + if [[ ! ${PYTHON_COMPAT_OVERRIDE} && ! ${has_check_deps} ]]; then use "python_targets_${impl}" || continue fi @@ -620,6 +638,16 @@ python_setup() { _python_impl_matches "${impl}" "${@-*}" || continue python_export "${impl}" EPYTHON PYTHON + + # if python_check_deps() is declared, switch into any-of mode + if [[ ${has_check_deps} ]]; then + # first check if the interpreter is installed + python_is_installed "${impl}" || continue + # then run python_check_deps + local PYTHON_USEDEP="python_targets_${impl}(-),python_single_target_${impl}(+)" +
[gentoo-dev] [PATCH 3/7] python-r1.eclass: Inline implementation loop logic into python_setup
Inline the logic needed to iterate over implementations directly into python_setup instead of using python_foreach_impl. This is mostly NFC, except that we iterate in reverse order now -- that is, we start at the newest implementation and stop at the first one that works for us. Previously we (implicitly) started at the oldest implementation, checked all implementation and used the last one (i.e. the newest) that worked. More importantly, the new code makes it possible to alter the logic more easily and avoid relying on implementation of python_foreach_impl(). --- eclass/python-r1.eclass | 35 ++- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass index ae9e3806e729..9c37a20f7c2e 100644 --- a/eclass/python-r1.eclass +++ b/eclass/python-r1.eclass @@ -597,16 +597,34 @@ python_foreach_impl() { python_setup() { debug-print-function ${FUNCNAME} "${@}" - local best_impl patterns=( "${@-*}" ) - _python_try_impl() { - if _python_impl_matches "${EPYTHON}" "${patterns[@]}"; then - best_impl=${EPYTHON} + _python_validate_useflags + local pycompat=( "${PYTHON_COMPAT[@]}" ) + if [[ ${PYTHON_COMPAT_OVERRIDE} ]]; then + pycompat=( ${PYTHON_COMPAT_OVERRIDE} ) + fi + + # (reverse iteration -- newest impl first) + local found + for (( i = ${#_PYTHON_SUPPORTED_IMPLS[@]} - 1; i >= 0; i-- )); do + local impl=${_PYTHON_SUPPORTED_IMPLS[i]} + + # check PYTHON_COMPAT[_OVERRIDE] + has "${impl}" "${pycompat[@]}" || continue + + # match USE flags only if override is not in effect + if [[ ! ${PYTHON_COMPAT_OVERRIDE} ]]; then + use "python_targets_${impl}" || continue fi - } - python_foreach_impl _python_try_impl - unset -f _python_try_impl - if [[ ! ${best_impl} ]]; then + # check patterns + _python_impl_matches "${impl}" "${@-*}" || continue + + python_export "${impl}" EPYTHON PYTHON + found=1 + break + done + + if [[ ! ${found} ]]; then eerror "${FUNCNAME}: none of the enabled implementation matched the patterns." eerror " patterns: ${@-'(*)'}" eerror "Likely a REQUIRED_USE constraint (possibly USE-conditional) is missing." @@ -615,7 +633,6 @@ python_setup() { die "${FUNCNAME}: no enabled implementation satisfy requirements" fi - python_export "${best_impl}" EPYTHON PYTHON python_wrapper_setup } -- 2.13.0
[gentoo-dev] [PATCHES] python-r1.eclass: any-of dep API support
Hi, everyone. Here's a set of patches inspired by the recent Sphinx dependency discussion. They make python-r1 (and therefore distutils-r1) capable of any-of dependency logic similar to the one used in python-any-r1. The basic goal is relatively simple -- to improve handling of pure build-time dependencies in the eclass. It solves two common problems: a. dependencies on packages that support only a subset of PYTHON_COMPAT, b. dependencies that need to be implementation-bound between themselves (e.g. Sphinx plugins). The new API improves both of those cases significantly. For the former, we no longer force user to select additional targets via REQUIRED_USE -- instead, we just any-of dependencies + python_check_deps() to select implementation independently of whether it is enabled or not. For the latter, we no longer have to force all targets of the package on all the involved dependencies. Again, using any-of dep and appropriate python_check_deps() we can enforce a single (any) target throughout all the packages and use it. The first three patches do some code refactoring that makes the change easier and possibly improves maintainability of the code. The next two patches add support for python_check_deps() and python_gen_any_dep() respectively. The last two patches provide examples for both use cases mentioned. Please review. -- Best regards, Michał Górny
[gentoo-dev] [PATCH 1/7] distutils-r1.eclass: Reuse python_setup for common phases
Rewrite the python_*_all() phase running code to reuse python_setup instead of hacking on top of python_foreach_impl. The resulting code is a bit simpler but most importantly, it avoids duplication of code from python-r1 and ensures that distutils-r1 common phases are directly altered by changes in python_setup. The code still needs to reimplement some of the internals. However, it is mostly limited to code specific to distutils-r1, and should be more maintainable. --- eclass/distutils-r1.eclass | 26 +- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass index e79f86bab12d..167af95eaed6 100644 --- a/eclass/distutils-r1.eclass +++ b/eclass/distutils-r1.eclass @@ -672,20 +672,20 @@ distutils-r1_run_phase() { _distutils-r1_run_common_phase() { local DISTUTILS_ORIG_BUILD_DIR=${BUILD_DIR} - if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then - local best_impl patterns=( "${DISTUTILS_ALL_SUBPHASE_IMPLS[@]-*}" ) - _distutils_try_impl() { - if _python_impl_matches "${EPYTHON}" "${patterns[@]}"; then - best_impl=${MULTIBUILD_VARIANT} - fi - } - python_foreach_impl _distutils_try_impl - unset -f _distutils_try_impl - - local PYTHON_COMPAT=( "${best_impl}" ) + if [[ ${DISTUTILS_SINGLE_IMPL} ]]; then + # reuse the dedicated code branch + _distutils-r1_run_foreach_impl "${@}" + else + local -x EPYTHON PYTHON + local -x PATH=${PATH} PKG_CONFIG_PATH=${PKG_CONFIG_PATH} + python_setup "${DISTUTILS_ALL_SUBPHASE_IMPLS[@]}" + + local MULTIBUILD_VARIANTS=( "${EPYTHON/./_}" ) + # store for restoring after distutils-r1_run_phase. + local _DISTUTILS_INITIAL_CWD=${PWD} + multibuild_foreach_variant \ + distutils-r1_run_phase "${@}" fi - - _distutils-r1_run_foreach_impl "${@}" } # @FUNCTION: _distutils-r1_run_foreach_impl -- 2.13.0
[gentoo-dev] [PATCH 2/2] distutils-r1.eclass: Remove QA-warning for DISTUTILS_NO_PARALLEL_BUILD
The variable was deprecated and the warning put in place in Dec 2014. It is no longer used in any ebuild in ::gentoo. --- eclass/distutils-r1.eclass | 9 - 1 file changed, 9 deletions(-) diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass index 6078fb6d52b7..e79f86bab12d 100644 --- a/eclass/distutils-r1.eclass +++ b/eclass/distutils-r1.eclass @@ -696,15 +696,6 @@ _distutils-r1_run_common_phase() { _distutils-r1_run_foreach_impl() { debug-print-function ${FUNCNAME} "${@}" - if [[ ${DISTUTILS_NO_PARALLEL_BUILD} ]]; then - [[ ${EAPI} == [45] ]] || die "DISTUTILS_NO_PARALLEL_BUILD is banned in EAPI ${EAPI}" - - eqawarn "DISTUTILS_NO_PARALLEL_BUILD is no longer meaningful. Now all builds" - eqawarn "are non-parallel. Please remove it from the ebuild." - - unset DISTUTILS_NO_PARALLEL_BUILD # avoid repeated warnings - fi - # store for restoring after distutils-r1_run_phase. local _DISTUTILS_INITIAL_CWD=${PWD} set -- distutils-r1_run_phase "${@}" -- 2.13.0
[gentoo-dev] [PATCH 1/2] python-r1.eclass: Remove deprecated python_parallel_foreach_impl
The function was (verbosely) deprecated in Dec 2014, and banned since EAPI 6. It is no longer used by any ebuild in ::gentoo. --- eclass/python-r1.eclass | 35 --- 1 file changed, 35 deletions(-) diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass index 809dc5f2b8e5..5eaa802e06b9 100644 --- a/eclass/python-r1.eclass +++ b/eclass/python-r1.eclass @@ -555,41 +555,6 @@ python_foreach_impl() { multibuild_foreach_variant _python_multibuild_wrapper "${@}" } -# @FUNCTION: python_parallel_foreach_impl -# @USAGE: [...] -# @DESCRIPTION: -# Run the given command for each of the enabled Python implementations. -# If additional parameters are passed, they will be passed through -# to the command. -# -# The function will return 0 status if all invocations succeed. -# Otherwise, the return code from first failing invocation will -# be returned. -# -# For each command being run, EPYTHON, PYTHON and BUILD_DIR are set -# locally, and the former two are exported to the command environment. -# -# This command used to be the parallel variant of python_foreach_impl. -# However, the parallel run support has been removed to simplify -# the eclasses and make them more predictable and therefore it is now -# only a deprecated alias to python_foreach_impl. -python_parallel_foreach_impl() { - debug-print-function ${FUNCNAME} "${@}" - - [[ ${EAPI} == [45] ]] || die "${FUNCNAME} is banned in EAPI ${EAPI}" - - if [[ ! ${_PYTHON_PARALLEL_WARNED} ]]; then - eqawarn "python_parallel_foreach_impl() is no longer meaningful. All runs" - eqawarn "are non-parallel now. Please replace the call with python_foreach_impl." - - _PYTHON_PARALLEL_WARNED=1 - fi - - local MULTIBUILD_VARIANTS - _python_obtain_impls - multibuild_foreach_variant _python_multibuild_wrapper "${@}" -} - # @FUNCTION: python_setup # @USAGE: [...] # @DESCRIPTION: -- 2.13.0
[gentoo-dev] [PATCH 2/4] python-r1.eclass: python_setup, add REQUIRED_USE to the example
--- eclass/python-r1.eclass | 1 + 1 file changed, 1 insertion(+) diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass index 8de0a851856c..809dc5f2b8e5 100644 --- a/eclass/python-r1.eclass +++ b/eclass/python-r1.eclass @@ -614,6 +614,7 @@ python_parallel_foreach_impl() { # Example: # @CODE # DEPEND="doc? ( dev-python/epydoc[$(python_gen_usedep 'python2*')] )" +# REQUIRED_USE="doc? ( $(python_gen_useflags 'python2*') )" # # src_compile() { # #... -- 2.13.0
[gentoo-dev] [PATCH 4/4] python-utils-r1.eclass: _python_impl_matches, handle both forms of impl
Make the pattern matching code in _python_impl_matches() more lax, allowing (accidental) mixing of PYTHON_COMPAT-style values with EPYTHON-style values. This is trivial to do, and solves the problem introduced by complexity-by-limitation of other eclasses -- where patterns for dependency strings are using PYTHON_COMPAT syntax, and patterns for python_setup are using EPYTHON syntax. --- eclass/python-utils-r1.eclass | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass index 0bf7e7ec1a3e..68fb9ba2578d 100644 --- a/eclass/python-utils-r1.eclass +++ b/eclass/python-utils-r1.eclass @@ -169,7 +169,8 @@ _python_set_impls() { # Check whether the specified matches at least one # of the patterns following it. Return 0 if it does, 1 otherwise. # -# should be in PYTHON_COMPAT form. The patterns can be either: +# can be in PYTHON_COMPAT or EPYTHON form. The patterns can be +# either: # a) fnmatch-style patterns, e.g. 'python2*', 'pypy'... # b) '-2' to indicate all Python 2 variants (= !python_is_python3) # c) '-3' to indicate all Python 3 variants (= python_is_python3) @@ -186,7 +187,8 @@ _python_impl_matches() { elif [[ ${pattern} == -3 ]]; then python_is_python3 "${impl}" return - elif [[ ${impl} == ${pattern} ]]; then + # unify value style to allow lax matching + elif [[ ${impl/./_} == ${pattern/./_} ]]; then return 0 fi done -- 2.13.0
[gentoo-dev] [PATCH 3/4] distutils-r1.eclass: Use _python_impl_matches()
Update the missed occurence of pattern matching with the new framework. --- eclass/distutils-r1.eclass | 17 ++--- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass index 1376326c9579..6078fb6d52b7 100644 --- a/eclass/distutils-r1.eclass +++ b/eclass/distutils-r1.eclass @@ -1,4 +1,4 @@ -# Copyright 1999-2016 Gentoo Foundation +# Copyright 1999-2017 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 # @ECLASS: distutils-r1.eclass @@ -191,6 +191,12 @@ fi # (allowing any implementation). If multiple values are specified, # implementations matching any of the patterns will be accepted. # +# The patterns can be either fnmatch-style patterns (matched via bash +# == operator against PYTHON_COMPAT values) or '-2' / '-3' to indicate +# appropriately all enabled Python 2/3 implementations (alike +# python_is_python3). Remember to escape or quote the fnmatch patterns +# to prevent accidental shell filename expansion. +# # If the restriction needs to apply conditionally to a USE flag, # the variable should be set conditionally as well (e.g. in an early # phase function or other convenient location). @@ -669,12 +675,9 @@ _distutils-r1_run_common_phase() { if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then local best_impl patterns=( "${DISTUTILS_ALL_SUBPHASE_IMPLS[@]-*}" ) _distutils_try_impl() { - local pattern - for pattern in "${patterns[@]}"; do - if [[ ${EPYTHON} == ${pattern} ]]; then - best_impl=${MULTIBUILD_VARIANT} - fi - done + if _python_impl_matches "${EPYTHON}" "${patterns[@]}"; then + best_impl=${MULTIBUILD_VARIANT} + fi } python_foreach_impl _distutils_try_impl unset -f _distutils_try_impl -- 2.13.0
[gentoo-dev] [PATCH 1/4] python-any-r1.eclass: python_gen_any_dep, add missing 'local i'
--- eclass/python-any-r1.eclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/eclass/python-any-r1.eclass b/eclass/python-any-r1.eclass index 69f7bb736d22..e4d2d46bc706 100644 --- a/eclass/python-any-r1.eclass +++ b/eclass/python-any-r1.eclass @@ -224,7 +224,7 @@ python_gen_any_dep() { local depstr=${1} [[ ${depstr} ]] || die "No dependency string provided" - local PYTHON_PKG_DEP out= + local i PYTHON_PKG_DEP out= for i in "${_PYTHON_SUPPORTED_IMPLS[@]}"; do local PYTHON_USEDEP="python_targets_${i}(-),python_single_target_${i}(+)" python_export "${i}" PYTHON_PKG_DEP -- 2.13.0
[gentoo-dev] [PATCHES] python-r1 suite: minor fixes
Hi, Here's a quick set of minor patches to python-r1 suite. It mostly includes some fixes to issues I've noticed while working on something bigger ;-). The first patch merely fixes missing 'local' for a variable. The second adds REQUIRED_USE to the python_setup() use example in python-r1. The third converts distutils-r1 common impl support to use the new pattern matching function (which is an omission from the original set of patches). The fourth patch is most interesting of all -- it makes the pattern matching work well with mismatched PYTHON_COMPAT/EPYTHON-style impls. This makes the API more lax, and avoids requiring users to be aware of technically implied impl style restrictions, i.e. having to write: REQUIRED_USE="doc? ( || ( $(python_gen_useflags 'python2_*') ) )" src_compile() { use doc && python_setup 'python2.*' } Note that flags used PYTHON_COMPAT form (with '_') while setup functions used EPYTHON form (with '.'). Now both are equally happy with both forms. -- Best regards, Michał Górny