Re: [gentoo-dev] News item for Exim 4.88

[Fabian Groffen]

On 01-03-2017 14:04:07 -0600, Matthias Maier wrote:
> 
> On Wed, Mar  1, 2017, at 13:43 CST, Fabian Groffen  wrote:
> 
> > Hi,
> >
> > I'd like to push out attached news item ASAP.  Please review.
> 
> Looks good. Has a clear and precise structure.

Thanks, I'll adjust the RC capitalisation and push it out.

Fabian


> > Title: =mail-mta/exim-4.88 problem with chunking
> > Author: Fabian Groffen 
> > Content-Type: text/plain
> > Posted: 2017-03-01
> > Revision: 1
> > News-Item-Format: 1.0
> > Display-If-Installed: =mail-mta/exim-4.88
> > 
> > Exim maintainers discovered that version 4.88 has some serious problems
> > with its CHUNKING extension.  To quote:
> > 
> >   There are various known problems which can result in messages stuck in
> >   queues and remote servers dropping connections on larger mails.
> > 
> > In Gentoo, Exim 4.88 is the only stable version available, hence all
> > Exim users are advised to either upgrade to an unstable 4.89 Release
> > Candidate, or patch the configuration as follows:
> 
> Maybe not capitalizing "release candidate".
> 
> > 
> > 1) in the main configuration section, add:
> > 
> >   chunking_advertise_hosts =
> > 
> > 2) for each SMTP transport, add:
> > 
> >   hosts_try_chunking =



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature

[gentoo-dev] Re: new virtual -- virtual/go to fix go build time dependencies

[Jonathan Callen]

On 03/01/2017 07:18 PM, William Hubbs wrote:
> All,
> 
> the dependencies for dev-lang/go need to force a rebuild every time the
> compiler is downgraded or upgraded.
> 
> To avoid abusing slot dependencies for dev-lang/go since it is not
> needed at runtime I need to do the following.
> 
> I need to introduce virtual/go-1.7.5, virtual/go-1.8 and
> virtual/go-. The attachment shows 1.8 but it is the same for all
> versions.
> 
> Also, the attached patch needs to be applied to golang-base.eclass.
> 
> This is based on the following wiki page:
> 
> Ahttps://wiki.gentoo.org/wiki/Project:Quality_Assurance/Subslots#Using_virtuals_to_expose_multiple_ABIs
> 
> Any thoughts?
> 
> William
> 

Unfortunately, this would allow the user to remove all versions of
dev-lang/go before attempting to compile a package using go, as
virtual/go has no RDEPENDs, and only RDEPENDs of DEPENDs are required to
be present (not DEPENDs of DEPENDs).

That is, the following sequence of actions would be perfectly legal,
according to the dependencies:

Merge dev-lang/go
Merge virtual/go
Unmerge dev-lang/go
Merge {package using golang-base.eclass}

While my initial thought was to just add dev-lang/go to the DEPEND in
the eclass (in addition to virtual/go), that doesn't quite work as the
two packages are allowed to get out of sync because virtual/go doesn't
RDEPEND on dev-lang/go at all (so you could install virtual/go for your
current version of dev-lang/go, then upgrade dev-lang/go without
upgrading virtual/go).

-- 
Jonathan Callen



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Mike Gilbert]

On Wed, Mar 1, 2017 at 7:18 PM, William Hubbs  wrote:
> All,
>
> the dependencies for dev-lang/go need to force a rebuild every time the
> compiler is downgraded or upgraded.

This already happens for me. It is working as expected.

> To avoid abusing slot dependencies for dev-lang/go since it is not
> needed at runtime I need to do the following.
>
> I need to introduce virtual/go-1.7.5, virtual/go-1.8 and
> virtual/go-. The attachment shows 1.8 but it is the same for all
> versions.
>
> Also, the attached patch needs to be applied to golang-base.eclass.
>
> This is based on the following wiki page:
>
> Ahttps://wiki.gentoo.org/wiki/Project:Quality_Assurance/Subslots#Using_virtuals_to_expose_multiple_ABIs
>
> Any thoughts?

I don't see how introducing a layer of indirection solves any problem here.

[gentoo-portage-dev] [PATCH] movefile: support in-kernel file copying on Linux (bug 607868)

[Zac Medico]

Perform in-kernel file copying when possible, and also support
reflinks and sparse files. If the optimized implementation
fails at runtime, gracefully fallback to shutil.copyfile.

Compile-time and run-time fallbacks are implemented, so that
any incompatiblities will be handled gracefully. For example,
if the code is compiled on a system that supports the
copy_file_range syscall, but at run-time an older kernel that
does not support this syscall is detected, it will be handled
gracefully.

X-Gentoo-Bug: 607868
X-Gentoo-Bug-Url: https://bugs.gentoo.org/show_bug.cgi?id=607868
---
 pym/portage/tests/util/file_copy/__init__.py  |   0
 pym/portage/tests/util/file_copy/__test__.py  |   0
 pym/portage/tests/util/file_copy/test_copyfile.py |  68 +++
 pym/portage/util/file_copy/__init__.py|  78 
 pym/portage/util/movefile.py  |   4 +-
 setup.py  |   9 +
 src/portage_util_file_copy_reflink_linux.c| 225 ++
 7 files changed, 383 insertions(+), 1 deletion(-)
 create mode 100644 pym/portage/tests/util/file_copy/__init__.py
 create mode 100644 pym/portage/tests/util/file_copy/__test__.py
 create mode 100644 pym/portage/tests/util/file_copy/test_copyfile.py
 create mode 100644 pym/portage/util/file_copy/__init__.py
 create mode 100644 src/portage_util_file_copy_reflink_linux.c

diff --git a/pym/portage/tests/util/file_copy/__init__.py 
b/pym/portage/tests/util/file_copy/__init__.py
new file mode 100644
index 000..e69de29
diff --git a/pym/portage/tests/util/file_copy/__test__.py 
b/pym/portage/tests/util/file_copy/__test__.py
new file mode 100644
index 000..e69de29
diff --git a/pym/portage/tests/util/file_copy/test_copyfile.py 
b/pym/portage/tests/util/file_copy/test_copyfile.py
new file mode 100644
index 000..987a701
--- /dev/null
+++ b/pym/portage/tests/util/file_copy/test_copyfile.py
@@ -0,0 +1,68 @@
+# Copyright 2017 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+import shutil
+import tempfile
+
+from portage import os
+from portage.tests import TestCase
+from portage.checksum import perform_md5
+from portage.util.file_copy import copyfile
+
+
+class CopyFileTestCase(TestCase):
+
+   def testCopyFile(self):
+
+   tempdir = tempfile.mkdtemp()
+   try:
+   src_path = os.path.join(tempdir, 'src')
+   dest_path = os.path.join(tempdir, 'dest')
+   content = b'foo'
+
+   with open(src_path, 'wb') as f:
+   f.write(content)
+
+   copyfile(src_path, dest_path)
+
+   self.assertEqual(perform_md5(src_path), 
perform_md5(dest_path))
+   finally:
+   shutil.rmtree(tempdir)
+
+
+class CopyFileSparseTestCase(TestCase):
+
+   def testCopyFileSparse(self):
+
+   # This test is expected to fail on platforms where we have
+   # not implemented sparse copy, so set the todo flag in order
+   # to tolerate failures.
+   self.todo = True
+
+   tempdir = tempfile.mkdtemp()
+   try:
+   src_path = os.path.join(tempdir, 'src')
+   dest_path = os.path.join(tempdir, 'dest')
+   content = b'foo'
+
+   # Use seek to create some sparse blocks. Don't make 
these
+   # files too big, in case the filesystem doesn't support
+   # sparse files.
+   with open(src_path, 'wb') as f:
+   f.write(content)
+   f.seek(2**18, 1)
+   f.write(content)
+   f.seek(2**19, 1)
+   f.write(content)
+
+   copyfile(src_path, dest_path)
+
+   self.assertEqual(perform_md5(src_path), 
perform_md5(dest_path))
+
+   # If sparse blocks were preserved, then both files 
should
+   # consume the same number of blocks.
+   self.assertEqual(
+   os.stat(src_path).st_blocks,
+   os.stat(dest_path).st_blocks)
+   finally:
+   shutil.rmtree(tempdir)
diff --git a/pym/portage/util/file_copy/__init__.py 
b/pym/portage/util/file_copy/__init__.py
new file mode 100644
index 000..5c7aff1
--- /dev/null
+++ b/pym/portage/util/file_copy/__init__.py
@@ -0,0 +1,78 @@
+# Copyright 2017 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+import os
+import shutil
+import tempfile
+
+try:
+   from portage.util.file_copy.reflink_linux import file_copy as _file_copy
+except ImportError:
+   _file_copy = None
+
+
+_copyfile = None
+
+
+def 

[gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[William Hubbs]

All,

the dependencies for dev-lang/go need to force a rebuild every time the
compiler is downgraded or upgraded.

To avoid abusing slot dependencies for dev-lang/go since it is not
needed at runtime I need to do the following.

I need to introduce virtual/go-1.7.5, virtual/go-1.8 and
virtual/go-. The attachment shows 1.8 but it is the same for all
versions.

Also, the attached patch needs to be applied to golang-base.eclass.

This is based on the following wiki page:

Ahttps://wiki.gentoo.org/wiki/Project:Quality_Assurance/Subslots#Using_virtuals_to_expose_multiple_ABIs

Any thoughts?

William
# Copyright 1999-2017 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

EAPI=6

DESCRIPTION="virtual for the Go compiler"
SLOT="0/${PV}"
KEYWORDS="-* ~amd64 ~arm ~arm64 ~ppc64 ~x86 ~amd64-fbsd 
~x86-fbsd ~x64-macos ~x64-solaris"

DEPEND="~dev-lang/go-${PV}"
diff --git a/eclass/golang-base.eclass b/eclass/golang-base.eclass
index 0cfd07e..e841763 100644
--- a/eclass/golang-base.eclass
+++ b/eclass/golang-base.eclass
@@ -22,7 +22,8 @@ if [[ -z ${_GOLANG_BASE} ]]; then
 
 _GOLANG_BASE=1
 
-DEPEND=">=dev-lang/go-1.7:="
+DEPEND=">=virtual/go-1.7"
+RDEPEND=">=virtual/go-1.7:="
 
 # Do not complain about CFLAGS etc since go projects do not use them.
 QA_FLAGS_IGNORED='.*'


signature.asc
Description: Digital signature

[gentoo-dev] [PATCH 1/1] eclass/kernel-2: Add additional help text when K_SECURITY_UNSUPPORTED is set

[Mike Pagano]

This patch will add some additional text to bring some additional notice
to users
about the security considerations of a specific kernel and direct them
to the
upstream website for further information.  See bug #599454

Signed-off-by: Mike Pagano 
---
 eclass/kernel-2.eclass | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/eclass/kernel-2.eclass b/eclass/kernel-2.eclass
index e95ec07..2aaab58 100644
--- a/eclass/kernel-2.eclass
+++ b/eclass/kernel-2.eclass
@@ -1054,6 +1054,12 @@ postinst_sources() {
#  And now the general message.
if [[ -n ${K_SECURITY_UNSUPPORTED} ]]; then
ewarn "This means that it is likely to be vulnerable to recent
security issues."
+   echo
+   ewarn "Upstream kernel developers recommend always running the 
latest "
+   ewarn "release of any current long term supported Linux kernel 
version."
+   ewarn "To see a list of these versions, their most current 
release and "
+   ewarn "long term support status, please go to 
https://www.kernel.org ."
+   echo
ewarn "For specific information on why this kernel is 
unsupported,
please read:"
ewarn "https://wiki.gentoo.org/wiki/Project:Kernel_Security;
fi
-- 
2.10.2




signature.asc
Description: OpenPGP digital signature

[gentoo-dev] [PATCH] mysql-v2.eclass: Drop dead g3nt8.org mirror

[Thomas Deutschmann]

Removing g3nt8.org mirror which is no longer hosting mysql-extras tarball and
not controlled by any Gentoo dev anymore (i.e. dead domain, up for sale).
---
 eclass/mysql-v2.eclass | 1 -
 1 file changed, 1 deletion(-)

diff --git a/eclass/mysql-v2.eclass b/eclass/mysql-v2.eclass
index 21931cc8e9..b63c306f74 100644
--- a/eclass/mysql-v2.eclass
+++ b/eclass/mysql-v2.eclass
@@ -182,7 +182,6 @@ SRC_URI="${SERVER_URI}"
 if [[ ${MY_EXTRAS_VER} != "live" && ${MY_EXTRAS_VER} != "none" ]]; then
SRC_URI="${SRC_URI}
mirror://gentoo/mysql-extras-${MY_EXTRAS_VER}.tar.bz2
-   http://g3nt8.org/patches/mysql-extras-${MY_EXTRAS_VER}.tar.bz2

https://dev.gentoo.org/~robbat2/distfiles/mysql-extras-${MY_EXTRAS_VER}.tar.bz2

https://dev.gentoo.org/~jmbsvicetto/distfiles/mysql-extras-${MY_EXTRAS_VER}.tar.bz2

https://dev.gentoo.org/~grknight/distfiles/mysql-extras-${MY_EXTRAS_VER}.tar.bz2;
-- 
2.12.0

Re: [gentoo-dev] News item for Exim 4.88

[DarKRaveR]

On a sidenote (from a user perspective), 4.88 dropped some deprecated 
features.


Shouldn't that have been announced aswell?

And why wasn't 4.87 kept as old stable version? Afterall there was a 
4.87.1 release with no feature change but CVE fix applied.


Regards


-Sven


Am 01.03.2017 um 21:04 schrieb Matthias Maier:

On Wed, Mar  1, 2017, at 13:43 CST, Fabian Groffen  wrote:


Hi,

I'd like to push out attached news item ASAP.  Please review.

Looks good. Has a clear and precise structure.



Title: =mail-mta/exim-4.88 problem with chunking
Author: Fabian Groffen 
Content-Type: text/plain
Posted: 2017-03-01
Revision: 1
News-Item-Format: 1.0
Display-If-Installed: =mail-mta/exim-4.88

Exim maintainers discovered that version 4.88 has some serious problems
with its CHUNKING extension.  To quote:

   There are various known problems which can result in messages stuck in
   queues and remote servers dropping connections on larger mails.

In Gentoo, Exim 4.88 is the only stable version available, hence all
Exim users are advised to either upgrade to an unstable 4.89 Release
Candidate, or patch the configuration as follows:

Maybe not capitalizing "release candidate".


1) in the main configuration section, add:

   chunking_advertise_hosts =

2) for each SMTP transport, add:

   hosts_try_chunking =

Re: [gentoo-dev] News item for Exim 4.88

[Matthias Maier]

On Wed, Mar  1, 2017, at 13:43 CST, Fabian Groffen  wrote:

> Hi,
>
> I'd like to push out attached news item ASAP.  Please review.

Looks good. Has a clear and precise structure.


> Title: =mail-mta/exim-4.88 problem with chunking
> Author: Fabian Groffen 
> Content-Type: text/plain
> Posted: 2017-03-01
> Revision: 1
> News-Item-Format: 1.0
> Display-If-Installed: =mail-mta/exim-4.88
> 
> Exim maintainers discovered that version 4.88 has some serious problems
> with its CHUNKING extension.  To quote:
> 
>   There are various known problems which can result in messages stuck in
>   queues and remote servers dropping connections on larger mails.
> 
> In Gentoo, Exim 4.88 is the only stable version available, hence all
> Exim users are advised to either upgrade to an unstable 4.89 Release
> Candidate, or patch the configuration as follows:

Maybe not capitalizing "release candidate".

> 
> 1) in the main configuration section, add:
> 
>   chunking_advertise_hosts =
> 
> 2) for each SMTP transport, add:
> 
>   hosts_try_chunking =


signature.asc
Description: PGP signature

[gentoo-dev] News item for Exim 4.88

[Fabian Groffen]

Hi,

I'd like to push out attached news item ASAP.  Please review.

Thanks in advance,
Fabian

-- 
Fabian Groffen
Gentoo on a different level
Title: =mail-mta/exim-4.88 problem with chunking
Author: Fabian Groffen 
Content-Type: text/plain
Posted: 2017-03-01
Revision: 1
News-Item-Format: 1.0
Display-If-Installed: =mail-mta/exim-4.88

Exim maintainers discovered that version 4.88 has some serious problems
with its CHUNKING extension.  To quote:

  There are various known problems which can result in messages stuck in
  queues and remote servers dropping connections on larger mails.

In Gentoo, Exim 4.88 is the only stable version available, hence all
Exim users are advised to either upgrade to an unstable 4.89 Release
Candidate, or patch the configuration as follows:

1) in the main configuration section, add:

  chunking_advertise_hosts =

2) for each SMTP transport, add:

  hosts_try_chunking =

Please see also the announcement sent to exim-announce:
https://lists.exim.org/lurker/message/20170301.031117.ff024aa8.en.html


signature.asc
Description: Digital signature

[gentoo-dev] [PATCH 05/10] python-r1.eclass: Remove stale EAPI=4 branch in globals

[Michał Górny]

---
 eclass/python-r1.eclass | 8 ++--
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass
index 929ec8fa8f89..549474630353 100644
--- a/eclass/python-r1.eclass
+++ b/eclass/python-r1.eclass
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2017 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: python-r1.eclass
@@ -195,14 +195,10 @@ _python_set_globals() {
# but no point in making this overcomplex, BDEP doesn't hurt anyone
# 2) python-exec should be built with all targets forced anyway
# but if new targets were added, we may need to force a rebuild
-   # 3) use whichever python-exec slot installed in EAPI 5. For EAPI 4,
-   # just fix :2 since := deps are not supported.
if [[ ${_PYTHON_WANT_PYTHON_EXEC2} == 0 ]]; then
die "python-exec:0 is no longer supported, please fix your 
ebuild to work with python-exec:2"
-   elif [[ ${EAPI} != 4 ]]; then
-   PYTHON_DEPS+=">=dev-lang/python-exec-2:=[${PYTHON_USEDEP}]"
else
-   PYTHON_DEPS+="dev-lang/python-exec:2[${PYTHON_USEDEP}]"
+   PYTHON_DEPS+=">=dev-lang/python-exec-2:=[${PYTHON_USEDEP}]"
fi
readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP
 }
-- 
2.12.0

[gentoo-dev] [PATCH 10/10] python-single-r1.eclass: Add integrity checks for globals

[Michał Górny]

---
 eclass/python-single-r1.eclass | 36 ++--
 1 file changed, 30 insertions(+), 6 deletions(-)

diff --git a/eclass/python-single-r1.eclass b/eclass/python-single-r1.eclass
index 85350924b657..fc51c4ec2377 100644
--- a/eclass/python-single-r1.eclass
+++ b/eclass/python-single-r1.eclass
@@ -56,8 +56,6 @@ fi
 
 EXPORT_FUNCTIONS pkg_setup
 
-if [[ ! ${_PYTHON_SINGLE_R1} ]]; then
-
 # @ECLASS-VARIABLE: PYTHON_COMPAT
 # @REQUIRED
 # @DESCRIPTION:
@@ -232,14 +230,40 @@ _python_single_set_globals() {
deps+=">=dev-lang/python-exec-2:=[${usedep}]"
fi
 
-   PYTHON_DEPS=${deps}
-   PYTHON_REQUIRED_USE=${requse}
-   PYTHON_USEDEP=${usedep}
-   readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP
+   if [[ ${PYTHON_DEPS+1} ]]; then
+   if [[ ${PYTHON_DEPS} != "${deps}" ]]; then
+   eerror "PYTHON_DEPS have changed between inherits 
(PYTHON_REQ_USE?)!"
+   eerror "Before: ${PYTHON_DEPS}"
+   eerror "Now   : ${deps}"
+   die "PYTHON_DEPS integrity check failed"
+   fi
+
+   # these two are formality -- they depend on PYTHON_COMPAT only
+   if [[ ${PYTHON_REQUIRED_USE} != ${requse} ]]; then
+   eerror "PYTHON_REQUIRED_USE have changed between 
inherits!"
+   eerror "Before: ${PYTHON_REQUIRED_USE}"
+   eerror "Now   : ${requse}"
+   die "PYTHON_REQUIRED_USE integrity check failed"
+   fi
+
+   if [[ ${PYTHON_USEDEP} != "${usedep}" ]]; then
+   eerror "PYTHON_USEDEP have changed between inherits!"
+   eerror "Before: ${PYTHON_USEDEP}"
+   eerror "Now   : ${usedep}"
+   die "PYTHON_USEDEP integrity check failed"
+   fi
+   else
+   PYTHON_DEPS=${deps}
+   PYTHON_REQUIRED_USE=${requse}
+   PYTHON_USEDEP=${usedep}
+   readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP
+   fi
 }
 _python_single_set_globals
 unset -f _python_single_set_globals
 
+if [[ ! ${_PYTHON_SINGLE_R1} ]]; then
+
 # @FUNCTION: python_gen_usedep
 # @USAGE:  [...]
 # @DESCRIPTION:
-- 
2.12.0

[gentoo-dev] [PATCH 09/10] python-single-r1.eclass: Refactor global setter to use locals, NFC

[Michał Górny]

---
 eclass/python-single-r1.eclass | 20 
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/eclass/python-single-r1.eclass b/eclass/python-single-r1.eclass
index 84d1f067be14..85350924b657 100644
--- a/eclass/python-single-r1.eclass
+++ b/eclass/python-single-r1.eclass
@@ -180,7 +180,6 @@ if [[ ! ${_PYTHON_SINGLE_R1} ]]; then
 _python_single_set_globals() {
_python_set_impls
 
-   PYTHON_DEPS=
local i PYTHON_PKG_DEP
 
local flags_mt=( "${_PYTHON_SUPPORTED_IMPLS[@]/#/python_targets_}" )
@@ -191,12 +190,13 @@ _python_single_set_globals() {
 
IUSE="${flags_mt[*]}"
 
+   local deps requse usedep
if [[ ${#_PYTHON_SUPPORTED_IMPLS[@]} -eq 1 ]]; then
# There is only one supported implementation; set IUSE and other
# variables without PYTHON_SINGLE_TARGET.
-   PYTHON_REQUIRED_USE="${flags_mt[*]}"
+   requse=${flags_mt[*]}
python_export "${_PYTHON_SUPPORTED_IMPLS[0]}" PYTHON_PKG_DEP
-   PYTHON_DEPS="${flags_mt[*]}? ( ${PYTHON_PKG_DEP} ) "
+   deps="${flags_mt[*]}? ( ${PYTHON_PKG_DEP} ) "
# Force on the python_single_target_* flag for this impl, so
# that any dependencies that inherit python-single-r1 and
# happen to have multiple implementations will still need
@@ -205,7 +205,7 @@ _python_single_set_globals() {
else
# Multiple supported implementations; honor 
PYTHON_SINGLE_TARGET.
IUSE+=" ${flags[*]}"
-   PYTHON_REQUIRED_USE="^^ ( ${flags[*]} )"
+   requse="^^ ( ${flags[*]} )"
# Ensure deps honor the same python_single_target_* flag as is 
set
# on this package.
optflags+=,${flags[@]/%/(+)?}
@@ -214,13 +214,13 @@ _python_single_set_globals() {
# The chosen targets need to be in PYTHON_TARGETS as 
well.
# This is in order to enforce correct dependencies on 
packages
# supporting multiple implementations.
-   PYTHON_REQUIRED_USE+=" python_single_target_${i}? ( 
python_targets_${i} )"
+   requse+=" python_single_target_${i}? ( 
python_targets_${i} )"
 
python_export "${i}" PYTHON_PKG_DEP
-   PYTHON_DEPS+="python_single_target_${i}? ( 
${PYTHON_PKG_DEP} ) "
+   deps+="python_single_target_${i}? ( ${PYTHON_PKG_DEP} ) 
"
done
fi
-   PYTHON_USEDEP=${optflags// /,}
+   usedep=${optflags// /,}
 
# 1) well, python-exec would suffice as an RDEP
# but no point in making this overcomplex, BDEP doesn't hurt anyone
@@ -229,8 +229,12 @@ _python_single_set_globals() {
if [[ ${_PYTHON_WANT_PYTHON_EXEC2} == 0 ]]; then
die "python-exec:0 is no longer supported, please fix your 
ebuild to work with python-exec:2"
else
-   PYTHON_DEPS+=">=dev-lang/python-exec-2:=[${PYTHON_USEDEP}]"
+   deps+=">=dev-lang/python-exec-2:=[${usedep}]"
fi
+
+   PYTHON_DEPS=${deps}
+   PYTHON_REQUIRED_USE=${requse}
+   PYTHON_USEDEP=${usedep}
readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP
 }
 _python_single_set_globals
-- 
2.12.0

[gentoo-dev] [PATCH 08/10] python-single-r1.eclass: Remove stale EAPI=4 branch in globals

[Michał Górny]

---
 eclass/python-single-r1.eclass | 8 ++--
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/eclass/python-single-r1.eclass b/eclass/python-single-r1.eclass
index 3b89ef57cfef..84d1f067be14 100644
--- a/eclass/python-single-r1.eclass
+++ b/eclass/python-single-r1.eclass
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2017 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: python-single-r1.eclass
@@ -226,14 +226,10 @@ _python_single_set_globals() {
# but no point in making this overcomplex, BDEP doesn't hurt anyone
# 2) python-exec should be built with all targets forced anyway
# but if new targets were added, we may need to force a rebuild
-   # 3) use whichever python-exec slot installed in EAPI 5. For EAPI 4,
-   # just fix :2 since := deps are not supported.
if [[ ${_PYTHON_WANT_PYTHON_EXEC2} == 0 ]]; then
die "python-exec:0 is no longer supported, please fix your 
ebuild to work with python-exec:2"
-   elif [[ ${EAPI} != 4 ]]; then
-   PYTHON_DEPS+=">=dev-lang/python-exec-2:=[${PYTHON_USEDEP}]"
else
-   PYTHON_DEPS+="dev-lang/python-exec:2[${PYTHON_USEDEP}]"
+   PYTHON_DEPS+=">=dev-lang/python-exec-2:=[${PYTHON_USEDEP}]"
fi
readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP
 }
-- 
2.12.0

[gentoo-dev] [PATCH 07/10] python-r1.eclass: Add integrity checks for globals

[Michał Górny]

---
 eclass/python-r1.eclass | 42 +-
 1 file changed, 37 insertions(+), 5 deletions(-)

diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass
index 1cf65056f637..bacf869ef237 100644
--- a/eclass/python-r1.eclass
+++ b/eclass/python-r1.eclass
@@ -51,6 +51,8 @@ fi
 [[ ${EAPI} == [45] ]] && inherit eutils
 inherit multibuild python-utils-r1
 
+fi
+
 # @ECLASS-VARIABLE: PYTHON_COMPAT
 # @REQUIRED
 # @DESCRIPTION:
@@ -198,15 +200,45 @@ _python_set_globals() {
deps+=">=dev-lang/python-exec-2:=[${usedep}]"
fi
 
-   PYTHON_DEPS=${deps}
-   IUSE=${flags[*]}
-   PYTHON_REQUIRED_USE=${requse}
-   PYTHON_USEDEP=${usedep}
-   readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP
+   if [[ ${PYTHON_DEPS+1} ]]; then
+   # IUSE is magical, so we can't really check it
+   # (but we verify PYTHON_COMPAT already)
+
+   if [[ ${PYTHON_DEPS} != "${deps}" ]]; then
+   eerror "PYTHON_DEPS have changed between inherits 
(PYTHON_REQ_USE?)!"
+   eerror "Before: ${PYTHON_DEPS}"
+   eerror "Now   : ${deps}"
+   die "PYTHON_DEPS integrity check failed"
+   fi
+
+   # these two are formality -- they depend on PYTHON_COMPAT only
+   if [[ ${PYTHON_REQUIRED_USE} != ${requse} ]]; then
+   eerror "PYTHON_REQUIRED_USE have changed between 
inherits!"
+   eerror "Before: ${PYTHON_REQUIRED_USE}"
+   eerror "Now   : ${requse}"
+   die "PYTHON_REQUIRED_USE integrity check failed"
+   fi
+
+   if [[ ${PYTHON_USEDEP} != "${usedep}" ]]; then
+   eerror "PYTHON_USEDEP have changed between inherits!"
+   eerror "Before: ${PYTHON_USEDEP}"
+   eerror "Now   : ${usedep}"
+   die "PYTHON_USEDEP integrity check failed"
+   fi
+   else
+   IUSE=${flags[*]}
+
+   PYTHON_DEPS=${deps}
+   PYTHON_REQUIRED_USE=${requse}
+   PYTHON_USEDEP=${usedep}
+   readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP
+   fi
 }
 _python_set_globals
 unset -f _python_set_globals
 
+if [[ ! ${_PYTHON_R1} ]]; then
+
 # @FUNCTION: _python_validate_useflags
 # @INTERNAL
 # @DESCRIPTION:
-- 
2.12.0

[gentoo-dev] [PATCH 06/10] python-r1.eclass: Refactor globals to use local vars, NFC

[Michał Górny]

---
 eclass/python-r1.eclass | 18 ++
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/eclass/python-r1.eclass b/eclass/python-r1.eclass
index 549474630353..1cf65056f637 100644
--- a/eclass/python-r1.eclass
+++ b/eclass/python-r1.eclass
@@ -165,14 +165,13 @@ inherit multibuild python-utils-r1
 # @CODE
 
 _python_set_globals() {
-   PYTHON_DEPS=
-   local i PYTHON_PKG_DEP
+   local deps i PYTHON_PKG_DEP
 
_python_set_impls
 
for i in "${_PYTHON_SUPPORTED_IMPLS[@]}"; do
python_export "${i}" PYTHON_PKG_DEP
-   PYTHON_DEPS+="python_targets_${i}? ( ${PYTHON_PKG_DEP} ) "
+   deps+="python_targets_${i}? ( ${PYTHON_PKG_DEP} ) "
done
 
local flags=( "${_PYTHON_SUPPORTED_IMPLS[@]/#/python_targets_}" )
@@ -186,10 +185,8 @@ _python_set_globals() {
 
local flags_st=( 
"${_PYTHON_SUPPORTED_IMPLS[@]/#/-python_single_target_}" )
optflags+=,${flags_st[@]/%/(-)}
-
-   IUSE=${flags[*]}
-   PYTHON_REQUIRED_USE="|| ( ${flags[*]} )"
-   PYTHON_USEDEP=${optflags// /,}
+   local requse="|| ( ${flags[*]} )"
+   local usedep=${optflags// /,}
 
# 1) well, python-exec would suffice as an RDEP
# but no point in making this overcomplex, BDEP doesn't hurt anyone
@@ -198,8 +195,13 @@ _python_set_globals() {
if [[ ${_PYTHON_WANT_PYTHON_EXEC2} == 0 ]]; then
die "python-exec:0 is no longer supported, please fix your 
ebuild to work with python-exec:2"
else
-   PYTHON_DEPS+=">=dev-lang/python-exec-2:=[${PYTHON_USEDEP}]"
+   deps+=">=dev-lang/python-exec-2:=[${usedep}]"
fi
+
+   PYTHON_DEPS=${deps}
+   IUSE=${flags[*]}
+   PYTHON_REQUIRED_USE=${requse}
+   PYTHON_USEDEP=${usedep}
readonly PYTHON_DEPS PYTHON_REQUIRED_USE PYTHON_USEDEP
 }
 _python_set_globals
-- 
2.12.0

[gentoo-dev] [PATCH 01/10] python-utils-r1.eclass: _python_set_impls, use local vars

[Michał Górny]

Refactor _python_set_impls to use local variables throughout
the function and assign global values at the end. This prepares it for
double-inherit integrity checks. NFC.
---
 eclass/python-utils-r1.eclass | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass
index 3937d066a095..fd219d95a760 100644
--- a/eclass/python-utils-r1.eclass
+++ b/eclass/python-utils-r1.eclass
@@ -114,21 +114,22 @@ _python_set_impls() {
_python_impl_supported "${i}"
done
 
-   _PYTHON_SUPPORTED_IMPLS=()
-   _PYTHON_UNSUPPORTED_IMPLS=()
+   local supp=() unsupp=()
 
for i in "${_PYTHON_ALL_IMPLS[@]}"; do
if has "${i}" "${PYTHON_COMPAT[@]}"; then
-   _PYTHON_SUPPORTED_IMPLS+=( "${i}" )
+   supp+=( "${i}" )
else
-   _PYTHON_UNSUPPORTED_IMPLS+=( "${i}" )
+   unsupp+=( "${i}" )
fi
done
 
-   if [[ ${#_PYTHON_SUPPORTED_IMPLS[@]} -eq 0 ]]; then
+   if [[ ! ${supp[@]} ]]; then
die "No supported implementation in PYTHON_COMPAT."
fi
 
+   _PYTHON_SUPPORTED_IMPLS=( "${supp[@]}" )
+   _PYTHON_UNSUPPORTED_IMPLS=( "${unsupp[@]}" )
readonly _PYTHON_SUPPORTED_IMPLS _PYTHON_UNSUPPORTED_IMPLS
 }
 
-- 
2.12.0

[gentoo-dev] [PATCH 04/10] python-any-r1.eclass: Add integrity check for globals

[Michał Górny]

---
 eclass/python-any-r1.eclass | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/eclass/python-any-r1.eclass b/eclass/python-any-r1.eclass
index d41cf6a5808d..69f7bb736d22 100644
--- a/eclass/python-any-r1.eclass
+++ b/eclass/python-any-r1.eclass
@@ -57,8 +57,6 @@ fi
 
 EXPORT_FUNCTIONS pkg_setup
 
-if [[ ! ${_PYTHON_ANY_R1} ]]; then
-
 # @ECLASS-VARIABLE: PYTHON_COMPAT
 # @REQUIRED
 # @DESCRIPTION:
@@ -161,12 +159,23 @@ _python_any_set_globals() {
done
deps="|| ( ${deps})"
 
-   PYTHON_DEPS=${deps}
-   readonly PYTHON_DEPS
+   if [[ ${PYTHON_DEPS+1} ]]; then
+   if [[ ${PYTHON_DEPS} != "${deps}" ]]; then
+   eerror "PYTHON_DEPS have changed between inherits 
(PYTHON_REQ_USE?)!"
+   eerror "Before: ${PYTHON_DEPS}"
+   eerror "Now   : ${deps}"
+   die "PYTHON_DEPS integrity check failed"
+   fi
+   else
+   PYTHON_DEPS=${deps}
+   readonly PYTHON_DEPS
+   fi
 }
 _python_any_set_globals
 unset -f _python_any_set_globals
 
+if [[ ! ${_PYTHON_ANY_R1} ]]; then
+
 # @FUNCTION: python_gen_any_dep
 # @USAGE: 
 # @DESCRIPTION:
-- 
2.12.0

[gentoo-dev] [PATCH 03/10] python-any-r1.eclass: Global setter, refactor for more local vars

[Michał Górny]

Refactor _python_any_set_globals() to use local variables while
generating all output, and copy it to final vars at the end. This is
in preparation for integrity checks. NFC.
---
 eclass/python-any-r1.eclass | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/eclass/python-any-r1.eclass b/eclass/python-any-r1.eclass
index a1dd4282c1ef..d41cf6a5808d 100644
--- a/eclass/python-any-r1.eclass
+++ b/eclass/python-any-r1.eclass
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2017 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: python-any-r1.eclass
@@ -148,19 +148,20 @@ if [[ ! ${_PYTHON_ANY_R1} ]]; then
 # @CODE
 
 _python_any_set_globals() {
-   local usestr i PYTHON_PKG_DEP
+   local usestr deps i PYTHON_PKG_DEP
[[ ${PYTHON_REQ_USE} ]] && usestr="[${PYTHON_REQ_USE}]"
 
_python_set_impls
 
-   PYTHON_DEPS=
for i in "${_PYTHON_SUPPORTED_IMPLS[@]}"; do
python_export "${i}" PYTHON_PKG_DEP
 
# note: need to strip '=' slot operator for || deps
-   PYTHON_DEPS="${PYTHON_PKG_DEP%=} ${PYTHON_DEPS}"
+   deps="${PYTHON_PKG_DEP%=} ${deps}"
done
-   PYTHON_DEPS="|| ( ${PYTHON_DEPS})"
+   deps="|| ( ${deps})"
+
+   PYTHON_DEPS=${deps}
readonly PYTHON_DEPS
 }
 _python_any_set_globals
-- 
2.12.0

[gentoo-dev] [PATCH 02/10] python-utils-r1.eclass: _python_set_impls, add integrity check

[Michał Górny]

Add integrity check for multi-inherits, i.e. ensure that PYTHON_COMPAT
has not changed between successive calls to _python_set_impls. If it did
(e.g. because of eclass+ebuild setting different values), then we abort
not to give surprising results to the user.
---
 eclass/python-utils-r1.eclass | 22 +++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass
index fd219d95a760..60129173fc06 100644
--- a/eclass/python-utils-r1.eclass
+++ b/eclass/python-utils-r1.eclass
@@ -128,9 +128,25 @@ _python_set_impls() {
die "No supported implementation in PYTHON_COMPAT."
fi
 
-   _PYTHON_SUPPORTED_IMPLS=( "${supp[@]}" )
-   _PYTHON_UNSUPPORTED_IMPLS=( "${unsupp[@]}" )
-   readonly _PYTHON_SUPPORTED_IMPLS _PYTHON_UNSUPPORTED_IMPLS
+   if [[ ${_PYTHON_SUPPORTED_IMPLS[@]} ]]; then
+   # set once already, verify integrity
+   if [[ ${_PYTHON_SUPPORTED_IMPLS[@]} != ${supp[@]} ]]; then
+   eerror "Supported impls (PYTHON_COMPAT) changed between 
inherits!"
+   eerror "Before: ${_PYTHON_SUPPORTED_IMPLS[*]}"
+   eerror "Now   : ${supp[*]}"
+   die "_PYTHON_SUPPORTED_IMPLS integrity check failed"
+   fi
+   if [[ ${_PYTHON_UNSUPPORTED_IMPLS[@]} != ${unsupp[@]} ]]; then
+   eerror "Unsupported impls changed between inherits!"
+   eerror "Before: ${_PYTHON_UNSUPPORTED_IMPLS[*]}"
+   eerror "Now   : ${unsupp[*]}"
+   die "_PYTHON_UNSUPPORTED_IMPLS integrity check failed"
+   fi
+   else
+   _PYTHON_SUPPORTED_IMPLS=( "${supp[@]}" )
+   _PYTHON_UNSUPPORTED_IMPLS=( "${unsupp[@]}" )
+   readonly _PYTHON_SUPPORTED_IMPLS _PYTHON_UNSUPPORTED_IMPLS
+   fi
 }
 
 # @ECLASS-VARIABLE: PYTHON
-- 
2.12.0

[gentoo-dev] [PATCHES] python-r1, add integrity checks for redefined control vars

[Michał Górny]

Hi,

Here's a patch serie inspired by the recent spidermonkey issue [1]. Long
story short, both mozcoreconf-*.eclass and spidermonkey were setting
different values of PYTHON_REQ_USE and inherit python-any-r1. Thanks to
double include guards, the dependencies were set using the first value
on the first iteration. However, python_setup() used the final value,
being more strict as a result.

This patch series aims to reduce the likeliness of such mistakes,
by comparing the values of output variables between successive eclass
inherits. This both catches some mistakes of overwriting the value,
and any potential integrity issues in eclass itself.

The remaining commits are purely cleanup/refactor.

However, the following technical limitations need to be noted:

1. We can only detect changes of control variables between successive
inherits of eclass, which makes every solution highly dependent
on the inherit order. If you 'inherit mozcoreconf-v4 python-any-r1',
the control variables are overwritten before python-any-r1 can see them.

2. There was a suggestion to make the eclass output variables updated
for the last values of control. However, the output variables are
currently marked readonly, to avoid accidental rewrites. If we were
to update them, we would have to disable that.

3. The same bugs (and also the case of defining after inherit) could
be caught if control variables were made readonly after setting globals.
However, this would cause spurious errors if the ebuild was sourced
more than once by the PM.

Please review.

[1]:https://bugs.gentoo.org/611176

--
Best regards,
Michał Górny

Re: [gentoo-portage-dev] [PATCH 1/2] checksum: Fix overriding fallbacks on broken pycrypto

[Michał Górny]

W dniu 28.02.2017, wto o godzinie 23∶57 -0800, użytkownik Zac Medico
napisał:
> On 02/28/2017 11:34 PM, Michał Górny wrote:
> > The pycrypto override used the same variables as actual hash functions
> > before determining whether its functions are useful. As a result, if
> > pycrypto had a broken module and no hash function was generated,
> > the possible previous implementation was replaced by None.
> > ---
> >  pym/portage/checksum.py | 12 ++--
> >  1 file changed, 6 insertions(+), 6 deletions(-)
> > 
> > diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py
> > index a46b820af..fc38417a7 100644
> > --- a/pym/portage/checksum.py
> > +++ b/pym/portage/checksum.py
> > @@ -105,14 +105,14 @@ except ImportError:
> >  # is broken somehow.
> >  try:
> > from Crypto.Hash import SHA256, RIPEMD
> > -   sha256hash = getattr(SHA256, 'new', None)
> > -   if sha256hash is not None:
> > +   sha256hash_ = getattr(SHA256, 'new', None)
> > +   if sha256hash_ is not None:
> > sha256hash = _generate_hash_function("SHA256",
> > -   sha256hash, origin="pycrypto")
> > -   rmd160hash = getattr(RIPEMD, 'new', None)
> > -   if rmd160hash is not None:
> > +   sha256hash_, origin="pycrypto")
> > +   rmd160hash_ = getattr(RIPEMD, 'new', None)
> > +   if rmd160hash_ is not None:
> > rmd160hash = _generate_hash_function("RMD160",
> > -   rmd160hash, origin="pycrypto")
> > +   rmd160hash_, origin="pycrypto")
> >  except ImportError:
> > pass
> >  
> > 
> 
> Looks good.

Thanks. Merged both patches.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-portage-dev] [PATCHES] Fix md5 refs in vartree

[Michał Górny]

W dniu 28.02.2017, wto o godzinie 11∶03 -0800, użytkownik Zac Medico
napisał:
> On 02/28/2017 02:12 AM, Michał Górny wrote:
> > Hi,
> > 
> > Here's a prequel to my other patch set. It cleans up the use of MD5
> > in vartree.
> > 
> > Currently, MD5 is loaded from three different locations -- most of
> > merging code uses portage.checksum high-level functions, one bit uses
> > implementation detail of portage.checksum and one bit imports hashlib
> > directly.
> > 
> > I've replaced the use of portage.checksum implementation detail with
> > another use of hashlib, and removed the compatibility for Python < 2.5.
> > I think it's a reasonable temporary measure until someone on friend
> > terms with the code reworks it not to use MD5.
> > 
> > --
> > Best regards,
> > Michał Górny
> > 
> > 
> 
> Both series look good to me.

Thanks. Merged both yesterday.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part