[gentoo-commits] proj/portage:master commit in: lib/portage/_emirrordist/

2021-01-21 Thread Zac Medico
commit: 8b5f9678200964c3cbb2eb06b58ebfa138fbac45
Author: Zac Medico  gentoo  org>
AuthorDate: Fri Jan 22 01:00:16 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Fri Jan 22 01:11:35 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=8b5f9678

emirrordist: prevent distfiles_db _pkg_str pickle problems

When storing a cpv as a shelve value, convert _pkg_str to str
in order to prevent pickle problems.

Bug: https://bugs.gentoo.org/766459
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/_emirrordist/FetchTask.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/portage/_emirrordist/FetchTask.py 
b/lib/portage/_emirrordist/FetchTask.py
index 41f96b962..997762082 100644
--- a/lib/portage/_emirrordist/FetchTask.py
+++ b/lib/portage/_emirrordist/FetchTask.py
@@ -44,7 +44,8 @@ class FetchTask(CompositeTask):
if self.config.distfiles_db is not None and \
self.distfile not in self.config.distfiles_db:
self._previously_added = False
-   self.config.distfiles_db[self.distfile] = self.cpv
+   # Convert _pkg_str to str in order to prevent pickle 
problems.
+   self.config.distfiles_db[self.distfile] = str(self.cpv)
 
if not self._have_needed_digests():
msg = "incomplete digests: %s" % " ".join(self.digests)



[gentoo-commits] proj/portage:master commit in: /

2021-01-19 Thread Zac Medico
commit: 04576b58c9c4e74218e3e82cbae6c0515e251d8f
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Jan 19 10:55:44 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Jan 19 11:02:55 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=04576b58

Updates for portage-3.0.14 release

Signed-off-by: Zac Medico  gentoo.org>

 RELEASE-NOTES | 13 +
 setup.py  |  2 +-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index c2a5607cd..430b24a64 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,19 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.14
+==
+* Bug Fixes:
+- Bug 571126 KeyError during package moves "global updates"
+ triggered by FEATURES=binpkg-multi-instance (same
+ root cause as bug 765847)
+- Bug 739004 Add @changed-subslot package set
+- Bug 765847 Spurious package file renames during application of
+ package moves with FEATURES=binpkg-multi-instance
+- Bug 766012 Copy on write when applying a package move to a binary
+ package, so that the old version of a binary package
+ will remain available until eclean-pkg deletes it
+
 portage-3.0.13
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index bd27a5343..5bd899e2c 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.13',
+   version = '3.0.14',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/portage/emaint/modules/move/, lib/portage/dbapi/, lib/portage/tests/update/

2021-01-19 Thread Zac Medico
commit: 79106b8bbce8b7b27db14877ca63c75a1a4a32d3
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Jan 19 09:23:50 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Jan 19 11:02:53 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=79106b8b

binarytree.move_ent: copy on write for package move

Copy on write when applying package moves, and silently
skip package moves when the same move has already been
applied to the same build of the package. Since the old
package instance is preserved, it avoids the problem
of having entries for deleted packages remain in the
package index. We can simply assume that the package
will be deleted by eclean-pkg when its time comes.

Bug: https://bugs.gentoo.org/766012
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dbapi/bintree.py  | 44 +++
 lib/portage/emaint/modules/move/move.py   | 13 +++--
 lib/portage/tests/update/test_move_ent.py |  7 ++---
 3 files changed, 48 insertions(+), 16 deletions(-)

diff --git a/lib/portage/dbapi/bintree.py b/lib/portage/dbapi/bintree.py
index 180e48c3b..ab09b42bc 100644
--- a/lib/portage/dbapi/bintree.py
+++ b/lib/portage/dbapi/bintree.py
@@ -31,6 +31,7 @@ from portage.exception import AlarmSignal, 
InvalidPackageName, \
ParseError, PortageException
 from portage.localization import _
 from portage.package.ebuild.profile_iuse import iter_iuse_vars
+from portage.util.file_copy import copyfile
 from portage.util.futures import asyncio
 from portage.util.futures.compat_coroutine import coroutine
 from portage.util.futures.executor.fork import ForkExecutor
@@ -483,6 +484,17 @@ class binarytree:
myoldpkg = catsplit(mycpv)[1]
mynewpkg = catsplit(mynewcpv)[1]
 
+   # If this update has already been applied to the same
+   # package build then silently continue.
+   applied = False
+   for maybe_applied in 
self.dbapi.match('={}'.format(mynewcpv)):
+   if maybe_applied.build_time == mycpv.build_time:
+   applied = True
+   break
+
+   if applied:
+   continue
+
if (mynewpkg != myoldpkg) and 
self.dbapi.cpv_exists(mynewcpv):
writemsg(_("!!! Cannot update binary: 
Destination exists.\n"),
noiselevel=-1)
@@ -513,24 +525,34 @@ class binarytree:
mydata[_unicode_encode(mynewpkg + 
'.ebuild',

encoding=_encodings['repo.content'])] = ebuild_data
 
-   mytbz2.recompose_mem(portage.xpak.xpak_mem(mydata))
-
-   self.dbapi.cpv_remove(mycpv)
-   del self._pkg_paths[self.dbapi._instance_key(mycpv)]
metadata = self.dbapi._aux_cache_slot_dict()
for k in self.dbapi._aux_cache_keys:
v = mydata.get(_unicode_encode(k))
if v is not None:
v = _unicode_decode(v)
metadata[k] = " ".join(v.split())
+
+   # Create a copy of the old version of the package and
+   # apply the update to it. Leave behind the old version,
+   # assuming that it will be deleted by eclean-pkg when 
its
+   # time comes.
mynewcpv = _pkg_str(mynewcpv, metadata=metadata, 
db=self.dbapi)
-   new_path = self.getname(mynewcpv)
-   self._pkg_paths[
-   self.dbapi._instance_key(mynewcpv)] = 
new_path[len(self.pkgdir)+1:]
-   if new_path != tbz2path:
-   self._ensure_dir(os.path.dirname(new_path))
-   _movefile(tbz2path, new_path, 
mysettings=self.settings)
-   self.inject(mynewcpv)
+   update_path = self.getname(mynewcpv, allocate_new=True) 
+ ".partial"
+   self._ensure_dir(os.path.dirname(update_path))
+   update_path_lock = None
+   try:
+   update_path_lock = lockfile(update_path, 
wantnewlockfile=True)
+   copyfile(tbz2path, update_path)
+   mytbz2 = portage.xpak.tbz2(update_path)
+   
mytbz2.recompose_mem(portage.xpak.xpak_mem(mydata))
+   self.inject(mynewcpv, filename=update_path)
+   finally:
+   if update_path_lock is not None:
+   

[gentoo-commits] proj/portage:master commit in: repoman/lib/repoman/modules/scan/metadata/

2021-01-18 Thread Zac Medico
commit: 030e2fc8ba6cf6c28b582c9ccbb5343ece887776
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Jan 19 07:32:49 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Jan 19 07:33:33 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=030e2fc8

repoman: fix DeprecationWarning: invalid escape sequence \s

Signed-off-by: Zac Medico  gentoo.org>

 repoman/lib/repoman/modules/scan/metadata/pkgmetadata.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/repoman/lib/repoman/modules/scan/metadata/pkgmetadata.py 
b/repoman/lib/repoman/modules/scan/metadata/pkgmetadata.py
index c1ba8e9f4..2e85cc3b9 100644
--- a/repoman/lib/repoman/modules/scan/metadata/pkgmetadata.py
+++ b/repoman/lib/repoman/modules/scan/metadata/pkgmetadata.py
@@ -101,7 +101,7 @@ class PkgMetadata(ScanBase, USEFlagChecks):
 
indentation_chars = Counter()
for l in etree.tostring(_metadata_xml).splitlines():
-   indentation_chars.update(re.match(b"\s*", l).group(0))
+   indentation_chars.update(re.match(rb"\s*", l).group(0))
if len(indentation_chars) > 1:
self.qatracker.add_error("metadata.warning", 
"%s/metadata.xml: %s" %
(xpkg, "inconsistent use of tabs and spaces in 
indentation")



[gentoo-commits] proj/portage:master commit in: lib/portage/, bin/

2021-01-18 Thread Zac Medico
commit: fb40f65ddd7635f98a2e0b14a8914697635896d4
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Jan 19 07:07:57 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Jan 19 07:17:19 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=fb40f65d

bin: misc pylint fixes

Signed-off-by: Zac Medico  gentoo.org>

 bin/clean_locks |  5 +++--
 bin/dispatch-conf   | 17 ++---
 bin/ebuild  |  4 +---
 bin/egencache   |  6 +++---
 bin/emerge  | 13 -
 bin/glsa-check  |  5 +++--
 bin/portageq| 18 +-
 bin/quickpkg| 10 +-
 bin/regenworld  |  6 +++---
 lib/portage/__init__.py |  3 ++-
 10 files changed, 43 insertions(+), 44 deletions(-)

diff --git a/bin/clean_locks b/bin/clean_locks
index d1f296065..e5765fd7e 100755
--- a/bin/clean_locks
+++ b/bin/clean_locks
@@ -1,8 +1,9 @@
 #!/usr/bin/python -b
-# Copyright 1999-2020 Gentoo Authors
+# Copyright 1999-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
-import sys, errno
+import errno
+import sys
 from os import path as osp
 if osp.isfile(osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), 
".portage_not_installed")):
sys.path.insert(0, 
osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), "lib"))

diff --git a/bin/dispatch-conf b/bin/dispatch-conf
index fa047244a..0fdfbaa81 100755
--- a/bin/dispatch-conf
+++ b/bin/dispatch-conf
@@ -1,5 +1,5 @@
 #!/usr/bin/python -b
-# Copyright 1999-2020 Gentoo Authors
+# Copyright 1999-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 #
@@ -16,10 +16,17 @@ import io
 import re
 import subprocess
 import sys
+import termios
+import tty
 
 from stat import ST_GID, ST_MODE, ST_UID
 from random import random
 
+try:
+   import curses
+except ImportError:
+   curses = None
+
 from os import path as osp
 if osp.isfile(osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), 
".portage_not_installed")):
sys.path.insert(0, 
osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), "lib"))
@@ -454,7 +461,6 @@ class dispatch:
 def getch ():
 # from ASPN - Danny Yoo
 #
-import tty, termios
 
 fd = sys.stdin.fileno()
 old_settings = termios.tcgetattr(fd)
@@ -466,8 +472,7 @@ def getch ():
 return ch
 
 def clear_screen():
-try:
-import curses
+if curses is not None:
 try:
 curses.setupterm()
 sys.stdout.write(_unicode_decode(curses.tigetstr("clear")))
@@ -475,8 +480,6 @@ def clear_screen():
 return
 except curses.error:
 pass
-except ImportError:
-pass
 os.system("clear 2>/dev/null")
 
 shell = os.environ.get("SHELL")
@@ -503,7 +506,7 @@ def usage(argv):
 for x in sys.argv:
 if x in ('-h', '--help'):
 usage(sys.argv)
-elif x in ('--version'):
+elif x in ('--version',):
 print("Portage", portage.VERSION)
 sys.exit(os.EX_OK)
 

diff --git a/bin/ebuild b/bin/ebuild
index 09f7f839b..0a2b13a13 100755
--- a/bin/ebuild
+++ b/bin/ebuild
@@ -1,5 +1,5 @@
 #!/usr/bin/python -b
-# Copyright 1999-2020 Gentoo Authors
+# Copyright 1999-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import argparse
@@ -46,7 +46,6 @@ portage._internal_caller = True
 from portage import os
 from portage import _encodings
 from portage import _shell_quote
-from portage import _unicode_decode
 from portage import _unicode_encode
 from portage.const import VDB_PATH
 from portage.exception import PermissionDenied, PortageKeyError, \
@@ -111,7 +110,6 @@ if debug:
 
 # do this _after_ 'import portage' to prevent unnecessary tracing
 if debug and "python-trace" in portage.features:
-   import portage.debug
portage.debug.set_trace(True)
 
 if not opts.color == 'y' and \

diff --git a/bin/egencache b/bin/egencache
index 4ee63edad..9b6df2e7d 100755
--- a/bin/egencache
+++ b/bin/egencache
@@ -1,5 +1,5 @@
 #!/usr/bin/python -b
-# Copyright 2009-2020 Gentoo Authors
+# Copyright 2009-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # unicode_literals for compat with TextIOWrapper in Python 2
@@ -49,7 +49,6 @@ if 
osp.isfile(osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), ".porta
 import portage
 portage._internal_caller = True
 from portage import os, _encodings, _unicode_encode, _unicode_decode
-from _emerge.MetadataRegen import MetadataRegen
 from portage.cache.cache_errors import CacheError, StatCollision
 from portage.cache.index.pkg_desc_index import pkg_desc_index_line_format, 
pkg_desc_index_line_read
 from portage.const import TIMESTAMP_FORMAT
@@ -65,6 +64,7 @@ from portage.util.changelog import ChangeLogTypeSort
 from portage import cpv_getkey
 from portage.dep import Atom, isjustname
 from portage.versions import vercmp
+from 

[gentoo-commits] proj/portage:master commit in: /

2021-01-18 Thread Zac Medico
commit: ed835b66bd86b465893968b7a29b73eaef6cd5e6
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Jan 19 07:09:37 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Jan 19 07:17:19 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=ed835b66

tox.ini: fix pylint command

The previous command did not work correctly due to quirks of
the pylint.utils.expand_modules function.

Signed-off-by: Zac Medico  gentoo.org>

 tox.ini | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tox.ini b/tox.ini
index cebf0e49c..585752686 100644
--- a/tox.ini
+++ b/tox.ini
@@ -20,6 +20,6 @@ deps =
 setenv =
PYTHONPATH={toxinidir}/lib
 commands =
-   bash -c 'rm -rf build && PYTHONPATH=$PWD/lib:$PWD/repoman/lib pylint *'
+   bash -c 'rm -rf build && PYTHONPATH=$PWD/lib:$PWD/repoman/lib pylint 
bin/* lib/* repoman/bin/* repoman/lib/*'
python -b -Wd setup.py test
bash -c 'if python -c "import lxml.etree"; then python -b -Wd 
repoman/setup.py test; else echo "repoman tests skipped due to lxml breakage"; 
fi'



[gentoo-commits] proj/portage:master commit in: repoman/lib/repoman/, repoman/lib/repoman/modules/vcs/svn/, ...

2021-01-18 Thread Zac Medico
can/ebuild/__init__.py repoman/lib/repoman/modules/scan/eclasses/__init__.py 
repoman/lib/repoman/modules/scan/fetch/__init__.py 
repoman/lib/repoman/modules/scan/keywords/__init__.py 
repoman/lib/repoman/modules/scan/manifest/__init__.py 
repoman/lib/repoman/modules/scan/metadata/__init__.py 
repoman/lib/repoman/modules/scan/metadata/ebuild_metadata.py 
repoman/lib/repoman/modules/scan/metadata/pkgmetadata.py 
repoman/lib/repoman/modules/scan/metadata/restrict.py 
repoman/lib/repoman/modules/scan/options/__init__.py 
repoman/lib/repoman/modules/vcs/None/status.py 
repoman/lib/repoman/modules/vcs/__init__.py 
repoman/lib/repoman/modules/vcs/bzr/changes.py 
repoman/lib/repoman/modules/vcs/bzr/status.py 
repoman/lib/repoman/modules/vcs/cvs/status.py 
repoman/lib/repoman/modules/vcs/git/changes.py 
repoman/lib/repoman/modules/vcs/git/status.py 
repoman/lib/repoman/modules/vcs/hg/changes.py 
repoman/lib/repoman/modules/vcs/hg/status.py 
repoman/lib/repoman/modules/vcs/svn/changes.py repoman/lib/repoman/mo
 dules/vcs/svn/status.py repoman/lib/repoman/modules/vcs/vcs.py 
repoman/lib/repoman/repos.py repoman/lib/repoman/tests/commit/__test__.py 
repoman/lib/repoman/tests/runTests.py 
repoman/lib/repoman/tests/simple/__test__.py 
repoman/lib/repoman/tests/simple/test_simple.py repoman/lib/repoman/utilities.py
X-VCS-Directories: repoman/lib/repoman/modules/scan/metadata/ 
repoman/lib/repoman/modules/scan/depend/ 
repoman/lib/repoman/modules/linechecks/quotes/ 
repoman/lib/repoman/modules/scan/fetch/ 
repoman/lib/repoman/modules/linechecks/depend/ 
repoman/lib/repoman/modules/scan/directories/ 
repoman/lib/repoman/modules/scan/eclasses/ 
repoman/lib/repoman/modules/scan/ebuild/ 
repoman/lib/repoman/modules/linechecks/whitespace/ 
repoman/lib/repoman/modules/linechecks/eapi/ 
repoman/lib/repoman/modules/scan/keywords/ 
repoman/lib/repoman/modules/linechecks/useless/ 
repoman/lib/repoman/modules/vcs/cvs/ repoman/lib/repoman/tests/commit/ 
repoman/lib/repoman/modules/linechecks/deprecated/ 
repoman/lib/repoman/modules/linechecks/gentoo_header/ 
repoman/lib/repoman/modules/linechecks/patches/ repoman/lib/repoman/ 
repoman/lib/repoman/tests/simple/ repoman/lib/repoman/modules/vcs/None/ 
repoman/lib/repoman/modules/vcs/svn/ 
repoman/lib/repoman/modules/linechecks/portage/ 
repoman/lib/repoman/modules/vcs/git/ repom
 an/lib/repoman/modules/scan/options/ repoman/lib/repoman/modules/commit/ 
repoman/lib/repoman/modules/linechecks/ 
repoman/lib/repoman/modules/linechecks/do/ 
repoman/lib/repoman/modules/linechecks/use/ repoman/lib/repoman/modules/vcs/ 
repoman/lib/repoman/modules/linechecks/nested/ 
repoman/lib/repoman/modules/linechecks/workaround/ repoman/lib/repoman/tests/ 
repoman/lib/repoman/modules/linechecks/phases/ 
repoman/lib/repoman/modules/linechecks/uri/ 
repoman/lib/repoman/modules/linechecks/helpers/ 
repoman/lib/repoman/modules/scan/eapi/ 
repoman/lib/repoman/modules/scan/manifest/ 
repoman/lib/repoman/modules/linechecks/emake/ 
repoman/lib/repoman/modules/linechecks/assignment/ 
repoman/lib/repoman/modules/vcs/hg/ repoman/lib/repoman/modules/vcs/bzr/
X-VCS-Committer: zmedico
X-VCS-Committer-Name: Zac Medico
X-VCS-Revision: fd216bba8b60335f8ee0119a350dd22ce65331b3
X-VCS-Branch: master
Date: Tue, 19 Jan 2021 07:28:40 + (UTC)

commit: fd216bba8b60335f8ee0119a350dd22ce65331b3
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Jan 19 07:05:46 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Jan 19 07:06:01 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=fd216bba

repoman: misc pylint fixes

Signed-off-by: Zac Medico  gentoo.org>

 repoman/lib/repoman/_subprocess.py |  4 
 repoman/lib/repoman/actions.py |  3 ++-
 repoman/lib/repoman/config.py  |  5 
 repoman/lib/repoman/gpg.py |  2 --
 repoman/lib/repoman/main.py|  3 ++-
 repoman/lib/repoman/metadata.py|  1 -
 repoman/lib/repoman/modules/commit/manifest.py |  1 -
 repoman/lib/repoman/modules/commit/repochecks.py   |  1 -
 repoman/lib/repoman/modules/linechecks/__init__.py |  1 -
 .../modules/linechecks/assignment/__init__.py  |  3 +--
 .../repoman/modules/linechecks/depend/__init__.py  |  3 +--
 .../modules/linechecks/deprecated/__init__.py  |  3 +--
 .../modules/linechecks/deprecated/deprecated.py|  2 +-
 .../lib/repoman/modules/linechecks/do/__init__.py  |  3 +--
 .../repoman/modules/linechecks/eapi/__init__.py|  3 +--
 .../repoman/modules/linechecks/emake/__init__.py   |  3 +--
 .../modules/linechecks/gentoo_header/__init__.py   |  3 +--
 .../repoman/modules/linechecks/helpers/__init__.py |  3 +--
 .../repoman/modules/linechecks/helpers/offset.py   |  2 +-
 .../repoman/modules/linechecks/nested/__init__.py  |  3 +--
 .../repoman/modules/linechecks/patches/__init__.py |  3 +--
 .../repoman/modules/linechecks/patches/patches.py  |  3 +--
 

[gentoo-commits] proj/portage:master commit in: lib/portage/util/

2021-01-18 Thread Zac Medico
commit: 4f77b1ca80a7ae7987c0d3e0ef988b3c06d18744
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Jan 19 00:50:13 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Jan 19 00:50:26 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=4f77b1ca

ProxyManager: remove unused loop parameter

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/socks5.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/util/socks5.py b/lib/portage/util/socks5.py
index ddf6bb4d0..4aa08e1ab 100644
--- a/lib/portage/util/socks5.py
+++ b/lib/portage/util/socks5.py
@@ -73,7 +73,7 @@ class ProxyManager:
"""
return self.socket_path is not None
 
-   async def ready(self, loop=None):
+   async def ready(self):
"""
Wait for the proxy socket to become ready. This method is a 
coroutine.
"""



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/util/, lib/portage/util/

2021-01-18 Thread Zac Medico
commit: 0b11ac9ce296367699e1a191b87be22612b266ff
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Jan 19 00:44:57 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Jan 19 00:45:17 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=0b11ac9c

ProxyManager: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/util/test_socks5.py | 4 ++--
 lib/portage/util/socks5.py| 9 +++--
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/lib/portage/tests/util/test_socks5.py 
b/lib/portage/tests/util/test_socks5.py
index 44d522013..82cf93fe8 100644
--- a/lib/portage/tests/util/test_socks5.py
+++ b/lib/portage/tests/util/test_socks5.py
@@ -1,4 +1,4 @@
-# Copyright 2019 Gentoo Authors
+# Copyright 2019-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import functools
@@ -185,7 +185,7 @@ class Socks5ServerTestCase(TestCase):
}
 
proxy = socks5.get_socks5_proxy(settings)
-   
loop.run_until_complete(socks5.proxy.ready(loop=loop))
+   loop.run_until_complete(socks5.proxy.ready())
 
result = 
loop.run_until_complete(loop.run_in_executor(None,
self._fetch_via_proxy, proxy, host, 
server.server_port, path))

diff --git a/lib/portage/util/socks5.py b/lib/portage/util/socks5.py
index a76d1a741..ddf6bb4d0 100644
--- a/lib/portage/util/socks5.py
+++ b/lib/portage/util/socks5.py
@@ -1,5 +1,5 @@
 # SOCKSv5 proxy manager for network-sandbox
-# Copyright 2015-2020 Gentoo Authors
+# Copyright 2015-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import errno
@@ -11,7 +11,6 @@ import portage.data
 from portage import _python_interpreter
 from portage.data import portage_gid, portage_uid, userpriv_groups
 from portage.process import atexit_register, spawn
-from portage.util.futures.compat_coroutine import coroutine
 from portage.util.futures import asyncio
 
 class ProxyManager:
@@ -74,9 +73,7 @@ class ProxyManager:
"""
return self.socket_path is not None
 
-
-   @coroutine
-   def ready(self, loop=None):
+   async def ready(self, loop=None):
"""
Wait for the proxy socket to become ready. This method is a 
coroutine.
"""
@@ -98,7 +95,7 @@ class ProxyManager:
except EnvironmentError as e:
if e.errno != errno.ENOENT:
raise
-   yield asyncio.sleep(0.2, loop=loop)
+   await asyncio.sleep(0.2)
else:
break
finally:



[gentoo-commits] proj/portage:master commit in: man/

2021-01-18 Thread Ulrich Müller
commit: 9436de9277593b668008c3906c8aca53f544b2ba
Author: Ulrich Müller  gentoo  org>
AuthorDate: Mon Jan 18 18:18:13 2021 +
Commit: Ulrich Müller  gentoo  org>
CommitDate: Mon Jan 18 18:18:13 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=9436de92

man/make.conf.5: Fix typo in markup

Fixes: bcf12c6f26cf ("Add a note that PORTDIR is a repository ...")
Signed-off-by: Ulrich Müller  gentoo.org>

 man/make.conf.5 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/man/make.conf.5 b/man/make.conf.5
index f6eae6f60..10f72d5c0 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -1104,7 +1104,7 @@ the emerge \-\-sync command. The default value of
 \fBPORTAGE_RSYNC_OPTS\fR will protect the default locations of
 \fBDISTDIR\fR and \fBPKGDIR\fR, but users are warned that any other locations
 inside \fBPORTDIR\fR are not necessarily safe for data storage.  You should not
-put other data (such as overlays) in your \fBPORTDIR\fB.  Portage will walk
+put other data (such as overlays) in your \fBPORTDIR\fR.  Portage will walk
 directory structures and may arbitrarily add invalid categories as packages.
 .TP
 \fBPORTDIR_OVERLAY\fR = \fI"[path] [different\-path] [etc...]"\fR



[gentoo-commits] proj/portage:master commit in: lib/portage/util/futures/, lib/portage/repository/storage/

2021-01-18 Thread Zac Medico
commit: 5346ae96d8d4b141454fe802a74e855e3e246593
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 14:37:17 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 15:03:28 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=5346ae96

RepoStorageInterface: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 .../repository/storage/hardlink_quarantine.py  | 45 +++--
 lib/portage/repository/storage/hardlink_rcu.py | 57 +-
 lib/portage/repository/storage/inplace.py  | 27 +++---
 lib/portage/repository/storage/interface.py| 17 +++
 lib/portage/util/futures/_sync_decorator.py|  6 +--
 5 files changed, 56 insertions(+), 96 deletions(-)

diff --git a/lib/portage/repository/storage/hardlink_quarantine.py 
b/lib/portage/repository/storage/hardlink_quarantine.py
index 165ab8324..bab34997b 100644
--- a/lib/portage/repository/storage/hardlink_quarantine.py
+++ b/lib/portage/repository/storage/hardlink_quarantine.py
@@ -1,4 +1,4 @@
-# Copyright 2018 Gentoo Foundation
+# Copyright 2018-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 from portage import os
@@ -7,10 +7,6 @@ from portage.repository.storage.interface import (
RepoStorageInterface,
 )
 from portage.util.futures import asyncio
-from portage.util.futures.compat_coroutine import (
-   coroutine,
-   coroutine_return,
-)
 
 from _emerge.SpawnProcess import SpawnProcess
 
@@ -38,60 +34,55 @@ class HardlinkQuarantineRepoStorage(RepoStorageInterface):
self._spawn_kwargs = spawn_kwargs
self._current_update = None
 
-   @coroutine
-   def _check_call(self, cmd, loop=None):
+   async def _check_call(self, cmd):
"""
Run cmd and raise RepoStorageException on failure.
 
@param cmd: command to executre
@type cmd: list
"""
-   p = SpawnProcess(args=cmd, scheduler=asyncio._wrap_loop(loop), 
**self._spawn_kwargs)
+   p = SpawnProcess(args=cmd, scheduler=asyncio.get_event_loop(), 
**self._spawn_kwargs)
p.start()
-   if (yield p.async_wait()) != os.EX_OK:
+   if await p.async_wait() != os.EX_OK:
raise RepoStorageException('command exited with status 
{}: {}'.\
format(p.returncode, ' '.join(cmd)))
 
-   @coroutine
-   def init_update(self, loop=None):
+   async def init_update(self):
update_location = os.path.join(self._user_location, 
'.tmp-unverified-download-quarantine')
-   yield self._check_call(['rm', '-rf', update_location], 
loop=loop)
+   await self._check_call(['rm', '-rf', update_location])
 
# Use  rsync --link-dest to hardlink a files into 
self._update_location,
# since cp -l is not portable.
-   yield self._check_call(['rsync', '-a', '--link-dest', 
self._user_location,
+   await self._check_call(['rsync', '-a', '--link-dest', 
self._user_location,
'--exclude=/distfiles', '--exclude=/local', 
'--exclude=/lost+found', '--exclude=/packages',
'--exclude', 
'/{}'.format(os.path.basename(update_location)),
-   self._user_location + '/', update_location + '/'], 
loop=loop)
+   self._user_location + '/', update_location + '/'])
 
self._update_location = update_location
 
-   coroutine_return(self._update_location)
+   return self._update_location
 
@property
-   def current_update(self, loop=None):
+   def current_update(self):
if self._update_location is None:
raise RepoStorageException('current update does not 
exist')
return self._update_location
 
-   @coroutine
-   def commit_update(self, loop=None):
+   async def commit_update(self):
update_location = self.current_update
self._update_location = None
-   yield self._check_call(['rsync', '-a', '--delete',
+   await self._check_call(['rsync', '-a', '--delete',
'--exclude=/distfiles', '--exclude=/local', 
'--exclude=/lost+found', '--exclude=/packages',
'--exclude', 
'/{}'.format(os.path.basename(update_location)),
-   update_location + '/', self._user_location + '/'], 
loop=loop)
+   update_location + '/', self._user_location + '/'])
 
-   yield self._check_call(['rm', '-rf', update_location], 
loop=loop)
+   await self._check_call(['rm', '-rf', update_location])
 
-   @coroutine
-   def abort_update(self, loop=None):
+   async def abort_update(self):
if 

[gentoo-commits] proj/portage:master commit in: repoman/lib/repoman/modules/scan/depend/

2021-01-18 Thread Zac Medico
commit: b5918d7416de7ef2e1b8f0ac86cd2d21cae8405e
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 12:39:55 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 13:08:03 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=b5918d74

repoman: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 repoman/lib/repoman/modules/scan/depend/profile.py | 22 +-
 1 file changed, 9 insertions(+), 13 deletions(-)

diff --git a/repoman/lib/repoman/modules/scan/depend/profile.py 
b/repoman/lib/repoman/modules/scan/depend/profile.py
index 468bc55e2..7549a414d 100644
--- a/repoman/lib/repoman/modules/scan/depend/profile.py
+++ b/repoman/lib/repoman/modules/scan/depend/profile.py
@@ -18,7 +18,6 @@ from portage.dep import Atom
 from portage.package.ebuild.profile_iuse import iter_iuse_vars
 from portage.util import getconfig
 from portage.util.futures import asyncio
-from portage.util.futures.compat_coroutine import coroutine, coroutine_return
 from portage.util.futures.executor.fork import ForkExecutor
 from portage.util.futures.iter_completed import async_iter_completed
 
@@ -102,13 +101,12 @@ class ProfileDependsChecks(ScanBase):
for result in results:
self._check_result(task, result)
 
-   loop = asyncio._wrap_loop()
-   loop.run_until_complete(self._async_check(loop=loop, **kwargs))
+   loop = asyncio.get_event_loop()
+   loop.run_until_complete(self._async_check(loop, **kwargs))
 
return False
 
-   @coroutine
-   def _async_check(self, loop=None, **kwargs):
+   async def _async_check(self, loop, **kwargs):
'''Perform async profile dependant dependency checks
 
@param arches:
@@ -118,7 +116,6 @@ class ProfileDependsChecks(ScanBase):
@param unknown_pkgs: set of tuples (type, atom.unevaluated_atom)
@returns: dictionary
'''
-   loop = asyncio._wrap_loop(loop)
ebuild = kwargs.get('ebuild').get()
pkg = kwargs.get('pkg').get()
unknown_pkgs = ebuild.unknown_pkgs
@@ -130,8 +127,8 @@ class ProfileDependsChecks(ScanBase):
 
if self.options.jobs > 1:
for future_done_set in 
async_iter_completed(self._iter_tasks(loop, executor, ebuild, pkg),
-   max_jobs=self.options.jobs, 
max_load=self.options.load_average, loop=loop):
-   for task in (yield future_done_set):
+   max_jobs=self.options.jobs, 
max_load=self.options.load_average):
+   for task in (await future_done_set):
task, results = task.result()
for result in results:
self._check_result(task, result)
@@ -145,10 +142,9 @@ class ProfileDependsChecks(ScanBase):
"dependency.unknown", "%s: %s: %s"
% (ebuild.relative_path, mytype, ", 
".join(sorted(atoms
 
-   @coroutine
-   def _task(self, task, loop=None):
-   yield task.future
-   coroutine_return((task, task.future.result()))
+   async def _task(self, task):
+   await task.future
+   return (task, task.future.result())
 
def _iter_tasks(self, loop, executor, ebuild, pkg):
for keyword, groups, prof in ebuild.relevant_profiles:
@@ -222,7 +218,7 @@ class ProfileDependsChecks(ScanBase):
yield (task, target())
else:
task.future = 
asyncio.ensure_future(loop.run_in_executor(executor, target), loop=loop)
-   yield self._task(task, loop=loop)
+   yield asyncio.ensure_future(self._task(task), 
loop=loop)
 
 
def _task_subprocess(self, task, pkg, dep_settings):



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/util/futures/asyncio/

2021-01-18 Thread Zac Medico
commit: 447ce6fa4a081ff416431a00a43827f67468b015
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 11:56:16 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 11:56:42 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=447ce6fa

ChildWatcherTestCase: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/util/futures/asyncio/test_child_watcher.py | 10 --
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/lib/portage/tests/util/futures/asyncio/test_child_watcher.py 
b/lib/portage/tests/util/futures/asyncio/test_child_watcher.py
index cd547f008..4e7b4fd7f 100644
--- a/lib/portage/tests/util/futures/asyncio/test_child_watcher.py
+++ b/lib/portage/tests/util/futures/asyncio/test_child_watcher.py
@@ -1,4 +1,4 @@
-# Copyright 2018-2019 Gentoo Authors
+# Copyright 2018-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import os
@@ -7,7 +7,6 @@ from portage.process import find_binary, spawn
 from portage.tests import TestCase
 from portage.util._eventloop.global_event_loop import global_event_loop
 from portage.util.futures import asyncio
-from portage.util.futures.compat_coroutine import coroutine
 from portage.util.futures.unix_events import DefaultEventLoopPolicy
 
 
@@ -37,17 +36,16 @@ class ChildWatcherTestCase(TestCase):
def callback(pid, returncode, *args):
future.set_result((pid, returncode, args))
 
-   @coroutine
-   def watch_pid(loop=None):
+   async def watch_pid():
 
with asyncio.get_child_watcher() as watcher:
pids = spawn([true_binary], 
returnpid=True)
watcher.add_child_handler(pids[0], 
callback, *args_tuple)
self.assertEqual(
-   (yield future),
+   (await future),
(pids[0], os.EX_OK, args_tuple))
 
-   loop.run_until_complete(watch_pid(loop=loop))
+   loop.run_until_complete(watch_pid())
finally:
asyncio.set_event_loop_policy(initial_policy)
if loop not in (None, global_event_loop()):



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/process/

2021-01-18 Thread Zac Medico
commit: b7469f2f9777e818e9c0f3fa46e90c3f7f1c904a
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 11:49:28 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 11:50:10 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=b7469f2f

AsyncFunctionTestCase: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/process/test_AsyncFunction.py | 10 --
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/lib/portage/tests/process/test_AsyncFunction.py 
b/lib/portage/tests/process/test_AsyncFunction.py
index b3f80b8ac..ce4b2a93b 100644
--- a/lib/portage/tests/process/test_AsyncFunction.py
+++ b/lib/portage/tests/process/test_AsyncFunction.py
@@ -1,4 +1,4 @@
-# Copyright 2020 Gentoo Authors
+# Copyright 2020-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -9,7 +9,6 @@ from portage.tests import TestCase
 from portage.util._async.AsyncFunction import AsyncFunction
 from portage.util.futures import asyncio
 from portage.util.futures._asyncio.streams import _writer
-from portage.util.futures.compat_coroutine import coroutine
 from portage.util.futures.unix_events import _set_nonblocking
 
 
@@ -20,8 +19,7 @@ class AsyncFunctionTestCase(TestCase):
os.close(pw)
return ''.join(sys.stdin)
 
-   @coroutine
-   def _testAsyncFunctionStdin(self, loop=None):
+   async def _testAsyncFunctionStdin(self, loop):
test_string = '1\n2\n3\n'
pr, pw = os.pipe()
fd_pipes = {0:pr}
@@ -30,8 +28,8 @@ class AsyncFunctionTestCase(TestCase):
os.close(pr)
_set_nonblocking(pw)
with open(pw, mode='wb', buffering=0) as pipe_write:
-   yield _writer(pipe_write, test_string.encode('utf_8'), 
loop=loop)
-   self.assertEqual((yield reader.async_wait()), os.EX_OK)
+   await _writer(pipe_write, test_string.encode('utf_8'))
+   self.assertEqual((await reader.async_wait()), os.EX_OK)
self.assertEqual(reader.result, test_string)
 
def testAsyncFunctionStdin(self):



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/util/futures/asyncio/

2021-01-18 Thread Zac Medico
commit: 9e6914ffe118457afbd29f448d99052d90e5e3dd
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 12:04:02 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 12:04:25 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=9e6914ff

SubprocessExecTestCase: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 .../util/futures/asyncio/test_subprocess_exec.py   | 32 ++
 1 file changed, 14 insertions(+), 18 deletions(-)

diff --git a/lib/portage/tests/util/futures/asyncio/test_subprocess_exec.py 
b/lib/portage/tests/util/futures/asyncio/test_subprocess_exec.py
index 6128a7d06..f9e35f6d4 100644
--- a/lib/portage/tests/util/futures/asyncio/test_subprocess_exec.py
+++ b/lib/portage/tests/util/futures/asyncio/test_subprocess_exec.py
@@ -1,4 +1,4 @@
-# Copyright 2018-2019 Gentoo Authors
+# Copyright 2018-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import os
@@ -9,7 +9,6 @@ from portage.tests import TestCase
 from portage.util._eventloop.global_event_loop import global_event_loop
 from portage.util.futures import asyncio
 from portage.util.futures._asyncio import create_subprocess_exec
-from portage.util.futures.compat_coroutine import coroutine, coroutine_return
 from portage.util.futures.unix_events import DefaultEventLoopPolicy
 
 
@@ -35,41 +34,38 @@ class SubprocessExecTestCase(TestCase):
echo_binary = echo_binary.encode()
 
def test(loop):
-   @coroutine
-   def test_coroutine(loop=None):
 
-   proc = (yield 
create_subprocess_exec(echo_binary, *args_tuple,
-   stdout=subprocess.PIPE, 
stderr=subprocess.STDOUT,
-   loop=loop))
+   async def test_coroutine():
 
-   out, err = (yield proc.communicate())
+   proc = await 
create_subprocess_exec(echo_binary, *args_tuple,
+   stdout=subprocess.PIPE, 
stderr=subprocess.STDOUT)
+
+   out, err = await proc.communicate()
self.assertEqual(tuple(out.split()), args_tuple)
self.assertEqual(proc.returncode, os.EX_OK)
 
-   proc = (yield create_subprocess_exec(
+   proc = await create_subprocess_exec(
'bash', '-c', 'echo foo; echo 
bar 1>&2;',
-   stdout=subprocess.PIPE, 
stderr=subprocess.PIPE,
-   loop=loop))
+   stdout=subprocess.PIPE, 
stderr=subprocess.PIPE)
 
-   out, err = (yield proc.communicate())
+   out, err = await proc.communicate()
self.assertEqual(out, b'foo\n')
self.assertEqual(err, b'bar\n')
self.assertEqual(proc.returncode, os.EX_OK)
 
-   proc = (yield create_subprocess_exec(
+   proc = await create_subprocess_exec(
'bash', '-c', 'echo foo; echo 
bar 1>&2;',
-   stdout=subprocess.PIPE, 
stderr=subprocess.STDOUT,
-   loop=loop))
+   stdout=subprocess.PIPE, 
stderr=subprocess.STDOUT)
 
-   out, err = (yield proc.communicate())
+   out, err = await proc.communicate()
self.assertEqual(out, b'foo\nbar\n')
self.assertEqual(err, None)
self.assertEqual(proc.returncode, os.EX_OK)
 
-   coroutine_return('success')
+   return 'success'
 
self.assertEqual('success',
-   
loop.run_until_complete(test_coroutine(loop=loop)))
+   loop.run_until_complete(test_coroutine()))
 
self._run_test(test)
 



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/process/

2021-01-18 Thread Zac Medico
commit: b3b9acc13c432ccd1610a442d57257dfac763cc4
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 09:43:24 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 11:18:43 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=b3b9acc1

PipeLoggerTestCase: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/process/test_PipeLogger.py | 22 ++
 1 file changed, 10 insertions(+), 12 deletions(-)

diff --git a/lib/portage/tests/process/test_PipeLogger.py 
b/lib/portage/tests/process/test_PipeLogger.py
index acc3b8af9..48e198bc5 100644
--- a/lib/portage/tests/process/test_PipeLogger.py
+++ b/lib/portage/tests/process/test_PipeLogger.py
@@ -1,4 +1,4 @@
-# Copyright 2020 Gentoo Authors
+# Copyright 2020-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 from portage import os
@@ -6,14 +6,12 @@ from portage.tests import TestCase
 from portage.util._async.PipeLogger import PipeLogger
 from portage.util.futures import asyncio
 from portage.util.futures._asyncio.streams import _reader, _writer
-from portage.util.futures.compat_coroutine import coroutine, coroutine_return
 from portage.util.futures.unix_events import _set_nonblocking
 
 
 class PipeLoggerTestCase(TestCase):
 
-   @coroutine
-   def _testPipeLoggerToPipe(self, test_string, loop=None):
+   async def _testPipeLoggerToPipe(self, test_string, loop):
"""
Test PipeLogger writing to a pipe connected to a PipeReader.
This verifies that PipeLogger does not deadlock when writing
@@ -24,7 +22,7 @@ class PipeLoggerTestCase(TestCase):
input_fd, writer_pipe = os.pipe()
_set_nonblocking(writer_pipe)
writer_pipe = os.fdopen(writer_pipe, 'wb', 0)
-   writer = asyncio.ensure_future(_writer(writer_pipe, 
test_string.encode('ascii'), loop=loop), loop=loop)
+   writer = asyncio.ensure_future(_writer(writer_pipe, 
test_string.encode('ascii')))
writer.add_done_callback(lambda writer: writer_pipe.close())
 
pr, pw = os.pipe()
@@ -37,22 +35,22 @@ class PipeLoggerTestCase(TestCase):
 
# Before starting the reader, wait here for a moment, in order
# to exercise PipeLogger's handling of EAGAIN during write.
-   yield asyncio.wait([writer], timeout=0.01, loop=loop)
+   await asyncio.wait([writer], timeout=0.01)
 
-   reader = _reader(pr, loop=loop)
-   yield writer
-   content = yield reader
-   yield consumer.async_wait()
+   reader = _reader(pr)
+   await writer
+   content = await reader
+   await consumer.async_wait()
 
self.assertEqual(consumer.returncode, os.EX_OK)
 
-   coroutine_return(content.decode('ascii', 'replace'))
+   return content.decode('ascii', 'replace')
 
def testPipeLogger(self):
loop = asyncio._wrap_loop()
 
for x in (1, 2, 5, 6, 7, 8, 2**5, 2**10, 2**12, 2**13, 2**14, 
2**17, 2**17 + 1):
test_string = x * "a"
-   output = 
loop.run_until_complete(self._testPipeLoggerToPipe(test_string, loop=loop))
+   output = 
loop.run_until_complete(self._testPipeLoggerToPipe(test_string, loop))
self.assertEqual(test_string, output,
"x = %s, len(output) = %s" % (x, len(output)))



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/util/futures/

2021-01-18 Thread Zac Medico
commit: be0b5d33ee8b355dcc9932b52bb1e025e6bd58d4
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 11:16:53 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 11:31:57 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=be0b5d33

RetryTestCase: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/util/futures/test_retry.py | 28 +---
 1 file changed, 9 insertions(+), 19 deletions(-)

diff --git a/lib/portage/tests/util/futures/test_retry.py 
b/lib/portage/tests/util/futures/test_retry.py
index 6648b1b2c..bce48a693 100644
--- a/lib/portage/tests/util/futures/test_retry.py
+++ b/lib/portage/tests/util/futures/test_retry.py
@@ -1,4 +1,4 @@
-# Copyright 2018-2020 Gentoo Authors
+# Copyright 2018-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 from concurrent.futures import Future, ThreadPoolExecutor
@@ -32,18 +32,11 @@ class SucceedLater:
def __init__(self, duration):
self._succeed_time = time.monotonic() + duration
 
-   def __call__(self):
-   loop = global_event_loop()
-   result = loop.create_future()
+   async def __call__(self):
remaining = self._succeed_time - time.monotonic()
if remaining > 0:
-   loop.call_soon_threadsafe(lambda: None if result.done() 
else
-   result.set_exception(SucceedLaterException(
-   'time until success: {} 
seconds'.format(remaining
-   else:
-   loop.call_soon_threadsafe(lambda: None if result.done() 
else
-   result.set_result('success'))
-   return result
+   await asyncio.sleep(remaining)
+   return 'success'
 
 
 class SucceedNeverException(Exception):
@@ -54,20 +47,17 @@ class SucceedNever:
"""
A callable object that never succeeds.
"""
-   def __call__(self):
-   loop = global_event_loop()
-   result = loop.create_future()
-   loop.call_soon_threadsafe(lambda: None if result.done() else
-   result.set_exception(SucceedNeverException('expected 
failure')))
-   return result
+   async def __call__(self):
+   raise SucceedNeverException('expected failure')
 
 
 class HangForever:
"""
A callable object that sleeps forever.
"""
-   def __call__(self):
-   return asyncio.Future()
+   async def __call__(self):
+   while True:
+   await asyncio.sleep(9)
 
 
 class RetryTestCase(TestCase):



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/emerge/

2021-01-18 Thread Zac Medico
commit: 0e4c062e9dc87ce290e29816df9d8871f25b
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 11:42:30 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 11:43:18 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=0e4c

SimpleEmergeTestCase: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/emerge/test_simple.py | 16 +++-
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/lib/portage/tests/emerge/test_simple.py 
b/lib/portage/tests/emerge/test_simple.py
index 1638fcb66..d26146aa9 100644
--- a/lib/portage/tests/emerge/test_simple.py
+++ b/lib/portage/tests/emerge/test_simple.py
@@ -1,4 +1,4 @@
-# Copyright 2011-2020 Gentoo Authors
+# Copyright 2011-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import subprocess
@@ -14,7 +14,6 @@ from portage.tests.util.test_socks5 import AsyncHTTPServer
 from portage.util import (ensure_dirs, find_updated_config_files,
shlex_split)
 from portage.util.futures import asyncio
-from portage.util.futures.compat_coroutine import coroutine
 
 
 class BinhostContentMap(Mapping):
@@ -225,8 +224,7 @@ call_has_and_best_version() {
loop.run_until_complete(asyncio.ensure_future(
self._async_test_simple(playground, metadata_xml_files, 
loop=loop), loop=loop))
 
-   @coroutine
-   def _async_test_simple(self, playground, metadata_xml_files, loop=None):
+   async def _async_test_simple(self, playground, metadata_xml_files, 
loop):
 
debug = playground.debug
settings = playground.settings
@@ -548,14 +546,14 @@ move dev-util/git dev-vcs/git
else:
local_env = env
 
-   proc = yield 
asyncio.create_subprocess_exec(*args,
-   env=local_env, stderr=None, 
stdout=stdout, loop=loop)
+   proc = await 
asyncio.create_subprocess_exec(*args,
+   env=local_env, stderr=None, 
stdout=stdout)
 
if debug:
-   yield proc.wait()
+   await proc.wait()
else:
-   output, _err = yield proc.communicate()
-   yield proc.wait()
+   output, _err = await proc.communicate()
+   await proc.wait()
if proc.returncode != os.EX_OK:
portage.writemsg(output)
 



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/dbapi/

2021-01-18 Thread Zac Medico
commit: be3613e562203d1d8fd7ad6001aa7db43480b2fe
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 11:31:18 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 11:32:12 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=be3613e5

AuxdbTestCase: Use async and await syntax

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/dbapi/test_auxdb.py | 12 +---
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/lib/portage/tests/dbapi/test_auxdb.py 
b/lib/portage/tests/dbapi/test_auxdb.py
index 1029de70d..c8db65d34 100644
--- a/lib/portage/tests/dbapi/test_auxdb.py
+++ b/lib/portage/tests/dbapi/test_auxdb.py
@@ -1,10 +1,9 @@
-# Copyright 2020 Gentoo Authors
+# Copyright 2020-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 from portage.tests import TestCase
 from portage.tests.resolver.ResolverPlayground import ResolverPlayground
 from portage.util.futures import asyncio
-from portage.util.futures.compat_coroutine import coroutine, coroutine_return
 from portage.util.futures.executor.fork import ForkExecutor
 
 
@@ -65,7 +64,7 @@ class AuxdbTestCase(TestCase):
def test_func():
loop = asyncio._wrap_loop()
return loop.run_until_complete(self._test_mod_async(
-   ebuilds, ebuild_inherited, 
eclass_defined_phases, eclass_depend, portdb, loop=loop))
+   ebuilds, ebuild_inherited, 
eclass_defined_phases, eclass_depend, portdb))
 
self.assertTrue(test_func())
 
@@ -91,14 +90,13 @@ class AuxdbTestCase(TestCase):
 
self.assertEqual(auxdb[cpv]['RESTRICT'], 'test')
 
-   @coroutine
-   def _test_mod_async(self, ebuilds, ebuild_inherited, 
eclass_defined_phases, eclass_depend, portdb, loop=None):
+   async def _test_mod_async(self, ebuilds, ebuild_inherited, 
eclass_defined_phases, eclass_depend, portdb):
 
for cpv, metadata in ebuilds.items():
-   defined_phases, depend, eapi, inherited = yield 
portdb.async_aux_get(cpv, ['DEFINED_PHASES', 'DEPEND', 'EAPI', 'INHERITED'], 
loop=loop)
+   defined_phases, depend, eapi, inherited = await 
portdb.async_aux_get(cpv, ['DEFINED_PHASES', 'DEPEND', 'EAPI', 'INHERITED'])
self.assertEqual(defined_phases, eclass_defined_phases)
self.assertEqual(depend, eclass_depend)
self.assertEqual(eapi, metadata['EAPI'])
self.assertEqual(frozenset(inherited.split()), 
ebuild_inherited)
 
-   coroutine_return(True)
+   return True



[gentoo-commits] proj/portage:master commit in: lib/portage/util/futures/

2021-01-18 Thread Zac Medico
commit: 8d962cb5cc97a5092ff45446c0f8da55b27d2434
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 10:09:05 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 11:18:43 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=8d962cb5

coroutine: do not require loop parameter

The loop parameter is not needed since global_event_loop now
returns the running event loop for the current thread.

Bug: https://bugs.gentoo.org/737698
Bug: https://bugs.gentoo.org/763339
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/futures/compat_coroutine.py | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/lib/portage/util/futures/compat_coroutine.py 
b/lib/portage/util/futures/compat_coroutine.py
index 9a0c5c1c8..3e8dcec02 100644
--- a/lib/portage/util/futures/compat_coroutine.py
+++ b/lib/portage/util/futures/compat_coroutine.py
@@ -1,4 +1,4 @@
-# Copyright 2018 Gentoo Foundation
+# Copyright 2018-2021 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import functools
@@ -68,10 +68,6 @@ def _generator_future(generator_func, *args, **kwargs):
the default event loop.
"""
loop = kwargs.get('loop')
-   if loop is None and portage._internal_caller:
-   # Require an explicit loop parameter, in order to support
-   # local event loops (bug 737698).
-   raise AssertionError("Missing required argument 'loop'")
loop = asyncio._wrap_loop(loop)
result = loop.create_future()
_GeneratorTask(generator_func(*args, **kwargs), result, loop=loop)



[gentoo-commits] proj/portage:master commit in: lib/portage/dbapi/

2021-01-18 Thread Zac Medico
commit: 7007dbcd1013829466498d7e0708c8b84fdd68bf
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 18 08:45:34 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 18 08:46:44 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=7007dbcd

bindbapi.aux_update: fix spurious package file renames

Omit the filename argument for the binarytree.inject method
in aux_update, since an in-place update is desired, and the
filename argument causes the file to be renamed when
binpkg-multi-instance is enabled.

Fixes: 328dd4712f88 ("binpkg-multi-instance 3 of 7")
Bug: https://bugs.gentoo.org/765847
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dbapi/bintree.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/dbapi/bintree.py b/lib/portage/dbapi/bintree.py
index e5ba5893e..180e48c3b 100644
--- a/lib/portage/dbapi/bintree.py
+++ b/lib/portage/dbapi/bintree.py
@@ -214,7 +214,7 @@ class bindbapi(fakedbapi):
del mydata[k]
mytbz2.recompose_mem(portage.xpak.xpak_mem(mydata))
# inject will clear stale caches via cpv_inject.
-   self.bintree.inject(cpv, filename=tbz2path)
+   self.bintree.inject(cpv)
 
 
@coroutine



[gentoo-commits] proj/portage:master commit in: lib/portage/dbapi/

2021-01-17 Thread Zac Medico
commit: 3d39e6891a3ee002fb1aa039c5660e6015c555d2
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Jan 17 13:24:21 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Jan 17 13:27:25 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=3d39e689

bindbapi.move_ent: use cpv_exists instead of getname

The getname method is complicated by binpkg-multi-instance
and soon BINPKG_FORMAT, so it's much simpler to use
cpv_exists.

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dbapi/bintree.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/dbapi/bintree.py b/lib/portage/dbapi/bintree.py
index 528a68979..e5ba5893e 100644
--- a/lib/portage/dbapi/bintree.py
+++ b/lib/portage/dbapi/bintree.py
@@ -483,7 +483,7 @@ class binarytree:
myoldpkg = catsplit(mycpv)[1]
mynewpkg = catsplit(mynewcpv)[1]
 
-   if (mynewpkg != myoldpkg) and 
os.path.exists(self.getname(mynewcpv)):
+   if (mynewpkg != myoldpkg) and 
self.dbapi.cpv_exists(mynewcpv):
writemsg(_("!!! Cannot update binary: 
Destination exists.\n"),
noiselevel=-1)
writemsg("!!! "+mycpv+" -> "+mynewcpv+"\n", 
noiselevel=-1)



[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

2021-01-17 Thread Zac Medico
commit: 0aac11200780225cf14a07d6e4445234f1e0f72a
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Jan 17 12:39:15 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Jan 17 13:02:29 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=0aac1120

doebuild: use NamedTemporaryFile for PORTAGE_BINPKG_TMPFILE

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py 
b/lib/portage/package/ebuild/doebuild.py
index f6cee4518..476689d5e 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1175,11 +1175,14 @@ def doebuild(myebuild, mydo, 
_unused=DeprecationWarning, settings=None, debug=0,
# the current user doesn't have write access to 
$PKGDIR.
if hasattr(portage, 'db'):
bintree = 
portage.db[mysettings['EROOT']]['bintree']
-   mysettings["PORTAGE_BINPKG_TMPFILE"] = \
-   
bintree.getname(mysettings.mycpv) + \
-   ".%s" % (portage.getpid(),)
-   bintree._ensure_dir(os.path.dirname(
-   
mysettings["PORTAGE_BINPKG_TMPFILE"]))
+   binpkg_tmpfile_dir = 
os.path.join(bintree.pkgdir, mysettings["CATEGORY"])
+   bintree._ensure_dir(binpkg_tmpfile_dir)
+   with tempfile.NamedTemporaryFile(
+   prefix=mysettings["PF"],
+   suffix=".tbz2." + 
str(portage.getpid()),
+   dir=binpkg_tmpfile_dir,
+   delete=False) as binpkg_tmpfile:
+   
mysettings["PORTAGE_BINPKG_TMPFILE"] = binpkg_tmpfile.name
else:
parent_dir = 
os.path.join(mysettings["PKGDIR"],
mysettings["CATEGORY"])



[gentoo-commits] proj/portage:master commit in: doc/config/, lib/portage/_sets/, cnf/sets/

2021-01-17 Thread Zac Medico
commit: 6c70596d6d6382160490bb16ef559ccfdd982fae
Author: Matt Turner  gentoo  org>
AuthorDate: Sat Jan 16 02:47:09 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Jan 17 10:35:03 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=6c70596d

Add @changed-subslot package set

This set is the upgradable packages for which the highest visible
version has a different subslot than the currently installed version.

The primary purpose of this feature is for use in catalyst builds. We
update the "seed" stage3 before using it to build a new stage1.

Updating the entire stage is expensive and unnecessary (since we're
going to build the latest packages in stage1 and then rebuild everything
in stage3).

What we definitely do need to update in the original stage3 however, is
any package that would trigger a subslot rebuild.

For example: gcc links with libmpfr.so from dev-libs/mpfr. mpfr's SONAME
changes from libmpfr.so.4 (SLOT="0/4") to libmpfr.so.6 (SLOT="0/6"). If
the seed stage's dev-libs/mpfr is not updated before emerging gcc, gcc
will link with libmpfr.so.4, but the latest version of dev-libs/mpfr
will be built and libmpfr.so.6 included into the stage1. Since the old
libmpfr.so.4 is not included in the stage1, gcc will not work, breaking
subsequent stage builds.

Our current options to update the seed are too large a hammer (e.g.,
"--update --deep --newuse @world" or "--update --deep --newuse
--complete-graph --rebuild-if-new-ver gcc") and spend too much time
updating seed stages for no gain beyond updating only packages for whom
the subslot has changed.

With this set, catalyst will likely use

emerge @changed-subslot --ignore-built-slot-operator-deps y

to update the seed stage.

Thank you to Zac Medico for showing me how to do this.

Bug: https://bugs.gentoo.org/739004
Signed-off-by: Matt Turner  gentoo.org>
Signed-off-by: Zac Medico  gentoo.org>

 cnf/sets/portage.conf  |  5 +
 doc/config/sets.docbook|  8 
 lib/portage/_sets/dbapi.py | 39 +--
 3 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/cnf/sets/portage.conf b/cnf/sets/portage.conf
index 22f0fa3a5..5651a9c53 100644
--- a/cnf/sets/portage.conf
+++ b/cnf/sets/portage.conf
@@ -84,6 +84,11 @@ exclude-files = /usr/bin/Xorg
 [rebuilt-binaries]
 class = portage.sets.dbapi.RebuiltBinaries
 
+# Installed packages for which the subslot of the highest visible ebuild
+# version is different than the currently installed version.
+[changed-subslot]
+class = portage.sets.dbapi.SubslotChangedSet
+
 # Installed packages for which the highest visible ebuild
 # version is lower than the currently installed version.
 [downgrade]

diff --git a/doc/config/sets.docbook b/doc/config/sets.docbook
index 0fe50f252..eba98f468 100644
--- a/doc/config/sets.docbook
+++ b/doc/config/sets.docbook
@@ -574,6 +574,14 @@
to check.


+   
+   portage.sets.dbapi.SubslotChangedSet
+   
+   Package set which contains all packages for which the subslot 
of the highest
+   visible ebuild is different than the currently installed 
version.
+   This class doesn't support any extra options.
+   
+   

portage.sets.dbapi.DowngradeSet


diff --git a/lib/portage/_sets/dbapi.py b/lib/portage/_sets/dbapi.py
index 52367c4a6..7a20553b5 100644
--- a/lib/portage/_sets/dbapi.py
+++ b/lib/portage/_sets/dbapi.py
@@ -1,4 +1,4 @@
-# Copyright 2007-2020 Gentoo Authors
+# Copyright 2007-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import glob
@@ -15,7 +15,7 @@ from portage._sets import SetConfigError, get_boolean
 import portage
 
 __all__ = ["CategorySet", "ChangedDepsSet", "DowngradeSet",
-   "EverythingSet", "OwnerSet", "VariableSet"]
+   "EverythingSet", "OwnerSet", "SubslotChangedSet", "VariableSet"]
 
 class EverythingSet(PackageSet):
_operations = ["merge"]
@@ -167,6 +167,41 @@ class VariableSet(EverythingSet):
 
singleBuilder = classmethod(singleBuilder)
 
+class SubslotChangedSet(PackageSet):
+
+   _operations = ["merge", "unmerge"]
+
+   description = "Package set which contains all packages " + \
+   "for which the subslot of the highest visible ebuild is " + \
+   "different than the currently installed version."
+
+   def __init__(self, portdb=None, vardb=None):
+   super(SubslotChangedSet, self).__init__()
+   self._portdb = portdb
+   self._vardb = vardb
+
+   def load(self):
+   atoms = []
+   xmatch = self._portdb.xmatch
+   xmatch_level = "bestmatch-visible"
+   cp_list = self._vardb.cp_list
+   for cp in self._vardb.cp_all():
+   for pkg in cp_list(cp):
+ 

[gentoo-commits] proj/portage:master commit in: lib/portage/dbapi/

2021-01-17 Thread Zac Medico
commit: 375eb5c02df19833ddeb2c63438aac9faf3d5eae
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Jan 17 08:35:41 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Jan 17 08:46:23 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=375eb5c0

vardbapi.move_ent: remove redundant self._pkg_str calls

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dbapi/vartree.py | 8 ++--
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
index f3d74cf82..2c75be4a1 100644
--- a/lib/portage/dbapi/vartree.py
+++ b/lib/portage/dbapi/vartree.py
@@ -1,4 +1,4 @@
-# Copyright 1998-2020 Gentoo Authors
+# Copyright 1998-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 __all__ = [
@@ -372,11 +372,7 @@ class vardbapi(dbapi):
if not origmatches:
return moves
for mycpv in origmatches:
-   try:
-   mycpv = self._pkg_str(mycpv, None)
-   except (KeyError, InvalidData):
-   continue
-   mycpv_cp = cpv_getkey(mycpv)
+   mycpv_cp = mycpv.cp
if mycpv_cp != origcp:
# Ignore PROVIDE virtual match.
continue



[gentoo-commits] proj/portage:master commit in: lib/portage/dbapi/

2021-01-17 Thread Zac Medico
commit: bdc75b3aa217649f6d835b04eb3cac879459c0a0
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Jan 17 08:47:13 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Jan 17 08:47:26 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=bdc75b3a

bindbapi.move_ent: remove redundant self._pkg_str calls

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dbapi/bintree.py | 10 +++---
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/lib/portage/dbapi/bintree.py b/lib/portage/dbapi/bintree.py
index 7e24589e5..528a68979 100644
--- a/lib/portage/dbapi/bintree.py
+++ b/lib/portage/dbapi/bintree.py
@@ -1,4 +1,4 @@
-# Copyright 1998-2020 Gentoo Authors
+# Copyright 1998-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 __all__ = ["bindbapi", "binarytree"]
@@ -27,7 +27,7 @@ from portage.cache.mappings import slot_dict_class
 from portage.const import BINREPOS_CONF_FILE, CACHE_PATH, 
SUPPORTED_XPAK_EXTENSIONS
 from portage.dbapi.virtual import fakedbapi
 from portage.dep import Atom, use_reduce, paren_enclose
-from portage.exception import AlarmSignal, InvalidData, InvalidPackageName, \
+from portage.exception import AlarmSignal, InvalidPackageName, \
ParseError, PortageException
 from portage.localization import _
 from portage.package.ebuild.profile_iuse import iter_iuse_vars
@@ -466,11 +466,7 @@ class binarytree:
if not origmatches:
return moves
for mycpv in origmatches:
-   try:
-   mycpv = self.dbapi._pkg_str(mycpv, None)
-   except (KeyError, InvalidData):
-   continue
-   mycpv_cp = portage.cpv_getkey(mycpv)
+   mycpv_cp = mycpv.cp
if mycpv_cp != origcp:
# Ignore PROVIDE virtual match.
continue



[gentoo-commits] proj/portage:master commit in: /

2021-01-11 Thread Zac Medico
commit: 9fdddccd4ac0dc5241298decebcf9479d67411ae
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 11 09:01:30 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 11 09:39:51 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=9fdddccd

Updates for portage-3.0.13 release

Signed-off-by: Zac Medico  gentoo.org>

 NEWS  | 6 ++
 RELEASE-NOTES | 7 +++
 setup.py  | 2 +-
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/NEWS b/NEWS
index c9bd5cfb5..685e42b2c 100644
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,11 @@
 News (mainly features/major bug fixes)
 
+portage-3.0.13
+--
+* FETCHCOMMAND now supports a \${DIGESTS} placeholder which expands
+  to a space separated list of digests. Refer to the FETCHCOMMAND
+  documentation in the make.conf(5) man page.
+
 portage-3.0.6
 --
 * emerge --search now detects regular expressions automatically. This

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index 0bbe7ad07..c2a5607cd 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,13 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.13
+==
+* Bug Fixes:
+- Bug 763339 always allow event loops to run in threads
+- Bug 764764 fix virtual/dist-kernel slot operator rebuilds
+- Bug 764905 wrap asyncio child watcher for thread safety
+
 portage-3.0.12
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index be69fffdb..bd27a5343 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.12',
+   version = '3.0.13',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/portage/util/_eventloop/, lib/portage/util/futures/_asyncio/

2021-01-11 Thread Zac Medico
commit: 386178481eb86ac603cd90ef1bb6ac6b68e51c50
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan  4 09:14:36 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 11 09:36:48 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=38617848

global_event_loop: return running loop for current thread

Like asyncio.get_event_loop(), return the running loop for the
current thread if there is one, and otherwise construct a new
one if needed. This allows the _safe_loop function to become
synonymous with the global_event_loop function.

For the case of "loop running in non-main thread" of API
consumer, this change makes portage compatible with PEP
492 coroutines with async and await syntax. Portage
internals can safely begin using async / await syntax instead
of compat_coroutine.

Bug: https://bugs.gentoo.org/763339
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/_eventloop/global_event_loop.py | 28 +++---
 lib/portage/util/futures/_asyncio/__init__.py| 30 +---
 2 files changed, 24 insertions(+), 34 deletions(-)

diff --git a/lib/portage/util/_eventloop/global_event_loop.py 
b/lib/portage/util/_eventloop/global_event_loop.py
index 413011178..cb7a13078 100644
--- a/lib/portage/util/_eventloop/global_event_loop.py
+++ b/lib/portage/util/_eventloop/global_event_loop.py
@@ -1,28 +1,6 @@
-# Copyright 2012-2020 Gentoo Authors
+# Copyright 2012-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
-import portage
-from portage.util._eventloop.asyncio_event_loop import AsyncioEventLoop
+__all__ = ('global_event_loop',)
 
-_instances = {}
-
-
-def global_event_loop():
-   """
-   Get a global EventLoop (or compatible object) instance which
-   belongs exclusively to the current process.
-   """
-
-   pid = portage.getpid()
-   instance = _instances.get(pid)
-   if instance is not None:
-   return instance
-
-   constructor = AsyncioEventLoop
-
-   # Use the _asyncio_wrapper attribute, so that unit tests can compare
-   # the reference to one retured from _wrap_loop(), since they should
-   # not close the loop if it refers to a global event loop.
-   instance = constructor()._asyncio_wrapper
-   _instances[pid] = instance
-   return instance
+from portage.util.futures._asyncio import _safe_loop as global_event_loop

diff --git a/lib/portage/util/futures/_asyncio/__init__.py 
b/lib/portage/util/futures/_asyncio/__init__.py
index d39f31786..5590963f1 100644
--- a/lib/portage/util/futures/_asyncio/__init__.py
+++ b/lib/portage/util/futures/_asyncio/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2018-2020 Gentoo Authors
+# Copyright 2018-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 __all__ = (
@@ -37,9 +37,6 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.futures:compat_coroutine@_compat_coroutine',
 )
 from portage.util._eventloop.asyncio_event_loop import AsyncioEventLoop as 
_AsyncioEventLoop
-from portage.util._eventloop.global_event_loop import (
-   global_event_loop as _global_event_loop,
-)
 # pylint: disable=redefined-builtin
 from portage.util.futures.futures import (
CancelledError,
@@ -238,7 +235,7 @@ def _wrap_loop(loop=None):
# The default loop returned by _wrap_loop should be consistent
# with global_event_loop, in order to avoid accidental registration
# of callbacks with a loop that is not intended to run.
-   loop = loop or _global_event_loop()
+   loop = loop or _safe_loop()
return (loop if hasattr(loop, '_asyncio_wrapper')
else _AsyncioEventLoop(loop=loop))
 
@@ -267,13 +264,15 @@ def _safe_loop():
@rtype: asyncio.AbstractEventLoop (or compatible)
@return: event loop instance
"""
-   if portage._internal_caller or threading.current_thread() is 
threading.main_thread():
-   return _global_event_loop()
+   loop = _get_running_loop()
+   if loop is not None:
+   return loop
 
thread_key = threading.get_ident()
with _thread_weakrefs.lock:
if _thread_weakrefs.pid != portage.getpid():
_thread_weakrefs.pid = portage.getpid()
+   _thread_weakrefs.mainloop = None
_thread_weakrefs.loops = weakref.WeakValueDictionary()
try:
loop = _thread_weakrefs.loops[thread_key]
@@ -283,9 +282,23 @@ def _safe_loop():
except RuntimeError:

_real_asyncio.set_event_loop(_real_asyncio.new_event_loop())
loop = _thread_weakrefs.loops[thread_key] = 
_AsyncioEventLoop()
+
+   if _thread_weakrefs.mainloop is None and threading.current_thread() is 
threading.main_thread():
+   _thread_weakrefs.mainloop = loop
+

[gentoo-commits] proj/portage:master commit in: lib/portage/util/_eventloop/

2021-01-10 Thread Zac Medico
commit: eeeb70bf50994c08ebcddac94474105f0635360c
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 11 06:46:49 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 11 07:28:28 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=eeeb70bf

AsyncioEventLoop: wrap child watcher for thread safety (bug 764905)

Use a child watcher wrapper to deliver the callbacks via the
call_soon_threadsafe method, since documentation for the asycio
AbstractChildWatcher class says that callbacks must be thread
safe.

Bug: https://bugs.gentoo.org/764905
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/_eventloop/asyncio_event_loop.py | 30 ++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/lib/portage/util/_eventloop/asyncio_event_loop.py 
b/lib/portage/util/_eventloop/asyncio_event_loop.py
index 4d7047ae8..b77728088 100644
--- a/lib/portage/util/_eventloop/asyncio_event_loop.py
+++ b/lib/portage/util/_eventloop/asyncio_event_loop.py
@@ -6,6 +6,7 @@ import signal
 
 import asyncio as _real_asyncio
 from asyncio.events import AbstractEventLoop as _AbstractEventLoop
+from asyncio.unix_events import AbstractChildWatcher as _AbstractChildWatcher
 
 import portage
 
@@ -47,6 +48,7 @@ class AsyncioEventLoop(_AbstractEventLoop):
self.set_debug = loop.set_debug
self.get_debug = loop.get_debug
self._wakeup_fd = -1
+   self._child_watcher = None
 
if portage._internal_caller:

loop.set_exception_handler(self._internal_caller_exception_handler)
@@ -87,7 +89,9 @@ class AsyncioEventLoop(_AbstractEventLoop):
@rtype: asyncio.AbstractChildWatcher
@return: the internal event loop's AbstractChildWatcher 
interface
"""
-   return _real_asyncio.get_child_watcher()
+   if self._child_watcher is None:
+   self._child_watcher = 
_ChildWatcherThreadSafetyWrapper(self, _real_asyncio.get_child_watcher())
+   return self._child_watcher
 
@property
def _asyncio_wrapper(self):
@@ -126,3 +130,27 @@ class AsyncioEventLoop(_AbstractEventLoop):
except ValueError:
# This is intended to fail when not called in 
the main thread.
pass
+
+
+class _ChildWatcherThreadSafetyWrapper(_AbstractChildWatcher):
+   def __init__(self, loop, real_watcher):
+   self._loop = loop
+   self._real_watcher = real_watcher
+
+   def close(self):
+   pass
+
+   def __enter__(self):
+   return self
+
+   def __exit__(self, a, b, c):
+   pass
+
+   def _child_exit(self, pid, status, callback, *args):
+   self._loop.call_soon_threadsafe(callback, pid, status, *args)
+
+   def add_child_handler(self, pid, callback, *args):
+   self._real_watcher.add_child_handler(pid, self._child_exit, 
callback, *args)
+
+   def remove_child_handler(self, pid):
+   return self._real_watcher.remove_child_handler(pid)



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/resolver/, lib/_emerge/

2021-01-10 Thread Zac Medico
commit: 4c25c0d7af7ad71fccbfafe1e5019116c691968e
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 11 00:19:06 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 11 03:32:45 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=4c25c0d7

check_reverse_dependencies: dereference virtual expansions (bug 764764)

If an atom is the result of virtual expansion, then derefrence it to
_orig_atom in check_reverse_dependencies so that it will be correctly
handled as a built slot operator dependency when appropriate. This
solves a case triggered in bug 764764 where a virtual expansion from
virtual/dist-kernel:0/5.10.5= to =virtual/dist-kernel-5.10.5 prevented
the atom from being handled as a built slot operator dependency, which
prevented rebuilds from being triggered.

Bug: https://bugs.gentoo.org/764764
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/depgraph.py| 22 ++
 .../resolver/test_slot_operator_reverse_deps.py|  2 +-
 2 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index f3e834a60..2bf04406f 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -2073,6 +2073,12 @@ class depgraph:
for parent, atom in 
self._dynamic_config._parent_atoms.get(existing_pkg, []):
if isinstance(parent, Package):
if parent in 
built_slot_operator_parents:
+   if hasattr(atom, '_orig_atom'):
+   # If atom is the result 
of virtual expansion, then
+   # derefrence it to 
_orig_atom so that it will be correctly
+   # handled as a built 
slot operator dependency when
+   # appropriate (see bug 
764764).
+   atom = atom._orig_atom
# This parent may need to be 
rebuilt, therefore
# discard its soname and built 
slot operator
# dependency components which 
are not necessarily
@@ -2131,6 +2137,22 @@ class depgraph:
allow_repo=True)
if not 
atom_set.findAtomForPackage(candidate_pkg,

modified_use=self._pkg_use_enabled(candidate_pkg)):
+   if debug:
+   parent_atoms = []
+   for other_parent, other_atom in 
self._dynamic_config._parent_atoms.get(existing_pkg, []):
+   if other_parent is 
parent:
+   
parent_atoms.append(other_atom)
+   msg = (
+   "",
+   "",
+   
"check_reverse_dependencies:",
+   "   candidate package 
does not match atom '%s': %s" % (atom, candidate_pkg),
+   "   parent: %s" % 
parent,
+   "   parent atoms: %s" % 
" ".join(parent_atoms),
+   "",
+   )
+   writemsg_level("\n".join(msg),
+   noiselevel=-1, 
level=logging.DEBUG)
return False
return True
 

diff --git a/lib/portage/tests/resolver/test_slot_operator_reverse_deps.py 
b/lib/portage/tests/resolver/test_slot_operator_reverse_deps.py
index 6e7214043..ef884f8ca 100644
--- a/lib/portage/tests/resolver/test_slot_operator_reverse_deps.py
+++ b/lib/portage/tests/resolver/test_slot_operator_reverse_deps.py
@@ -284,7 +284,7 @@ class SlotOperatorReverseDepsVirtualTestCase(TestCase):
["@world"],
options = {"--update": True, "--deep": True},
success = True,
-   mergelist = []
+   mergelist = ['sys-kernel/gentoo-kernel-5.10.6', 
'virtual/dist-kernel-5.10.6', 'app-emulation/virtualbox-modules-6.1.16-r1', 
'x11-drivers/nvidia-drivers-460.32.03']
),
)
 



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/resolver/

2021-01-10 Thread Zac Medico
commit: 9b6417f527aa98742a51d7859051342b6fb1383f
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan 11 01:57:44 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan 11 03:04:28 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=9b6417f5

Add unit test demonstrating bug 764764

Bug: https://bugs.gentoo.org/764764
Signed-off-by: Zac Medico  gentoo.org>

 .../resolver/test_slot_operator_reverse_deps.py| 98 +-
 1 file changed, 97 insertions(+), 1 deletion(-)

diff --git a/lib/portage/tests/resolver/test_slot_operator_reverse_deps.py 
b/lib/portage/tests/resolver/test_slot_operator_reverse_deps.py
index 6641e9987..6e7214043 100644
--- a/lib/portage/tests/resolver/test_slot_operator_reverse_deps.py
+++ b/lib/portage/tests/resolver/test_slot_operator_reverse_deps.py
@@ -1,4 +1,4 @@
-# Copyright 2016-2020 Gentoo Authors
+# Copyright 2016-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 from portage.tests import TestCase
@@ -202,3 +202,99 @@ class SlotOperatorReverseDepsLibGit2TestCase(TestCase):
finally:
playground.debug = False
playground.cleanup()
+
+
+class SlotOperatorReverseDepsVirtualTestCase(TestCase):
+
+   def testSlotOperatorReverseDepsVirtual(self):
+   """
+   Demonstrate bug #764764, where slot operator rebuilds were
+   not triggered for reverse deps of virtual/dist-kernel.
+   """
+
+   ebuilds = {
+
+   "app-emulation/virtualbox-modules-6.1.16-r1": {
+   "EAPI": "7",
+   "DEPEND": "virtual/dist-kernel",
+   "RDEPEND": "virtual/dist-kernel:=",
+   },
+
+   "sys-kernel/gentoo-kernel-5.10.6": {
+   "EAPI": "7",
+   "SLOT": "5.10.6",
+   },
+
+   "sys-kernel/gentoo-kernel-5.10.5": {
+   "EAPI": "7",
+   "SLOT": "5.10.5",
+   },
+
+   "virtual/dist-kernel-5.10.5" : {
+   "EAPI": "7",
+   "SLOT": "0/5.10.5",
+   "RDEPEND": "~sys-kernel/gentoo-kernel-5.10.5",
+   },
+
+   "virtual/dist-kernel-5.10.6" : {
+   "EAPI": "7",
+   "SLOT": "0/5.10.6",
+   "RDEPEND": "~sys-kernel/gentoo-kernel-5.10.6"
+   },
+
+   "x11-drivers/nvidia-drivers-460.32.03" : {
+   "EAPI": "7",
+   "DEPEND": "virtual/dist-kernel",
+   "RDEPEND": "virtual/dist-kernel:=",
+   },
+
+   }
+
+   installed = {
+
+   "app-emulation/virtualbox-modules-6.1.16-r1": {
+   "EAPI": "7",
+   "DEPEND": "virtual/dist-kernel",
+   "RDEPEND": "virtual/dist-kernel:0/5.10.5=",
+   },
+
+   "sys-kernel/gentoo-kernel-5.10.5": {
+   "EAPI": "7",
+   "SLOT": "5.10.5",
+   },
+
+   "virtual/dist-kernel-5.10.5" : {
+   "EAPI": "7",
+   "SLOT": "0/5.10.5",
+   "RDEPEND": "~sys-kernel/gentoo-kernel-5.10.5"
+   },
+
+   "x11-drivers/nvidia-drivers-460.32.03" : {
+   "EAPI": "7",
+   "DEPEND": "virtual/dist-kernel",
+   "RDEPEND": "virtual/dist-kernel:0/5.10.5="
+   },
+
+   }
+
+   world = ["app-emulation/virtualbox-modules", 
"x11-drivers/nvidia-drivers"]
+
+   test_cases = (
+   ResolverPlaygroundTestCase(
+   ["@world"],
+   options = {"--update": True, "--deep": True},
+   success = True,
+   mergelist = []
+   ),
+   )
+
+   playground = ResolverPlayground(ebuilds=ebuilds,
+   installed=installed, world=world, debug=False)
+   try:
+   for test_case in test_cases:
+   playground.run_TestCase(test_case)
+   self.assertEqual(test_case.test_success, True,
+   test_case.fail_msg)
+   finally:
+  

[gentoo-commits] proj/portage:master commit in: man/, lib/portage/package/ebuild/

2021-01-09 Thread Zac Medico
commit: f04dae7728b558ba10b99d1b4979054825fbb0d6
Author: Daniel M. Weeks  danweeks  net>
AuthorDate: Wed Jan  6 18:35:37 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Jan 10 03:22:43 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=f04dae77

Make digests available to fetch command

This makes it possible for the fetch command to utilize a
protocol/service that identifies files by their hash.

Closes: https://github.com/gentoo/portage/pull/655
Signed-off-by: Daniel M. Weeks  danweeks.net>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/package/ebuild/fetch.py |  6 ++
 man/make.conf.5 | 14 +-
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py 
b/lib/portage/package/ebuild/fetch.py
index 7be9d0239..e0fecaf23 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1291,6 +1291,12 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
"FILE":
os.path.basename(download_path)
}
 
+   try:
+   variables['DIGESTS'] = " 
".join(["%s:%s" % (k.lower(), v)
+   for k, v in 
mydigests[myfile].items() if k != 'size'])
+   except KeyError:
+   pass
+
for k in ("DISTDIR", 
"PORTAGE_SSH_OPTS"):
v = mysettings.get(k)
if v is not None:

diff --git a/man/make.conf.5 b/man/make.conf.5
index 494d5a212..f6eae6f60 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -1,4 +1,4 @@
-.TH "MAKE.CONF" "5" "Sep 2020" "Portage VERSION" "Portage"
+.TH "MAKE.CONF" "5" "Jan 2021" "Portage VERSION" "Portage"
 .SH "NAME"
 make.conf \- custom settings for Portage
 .SH "SYNOPSIS"
@@ -746,6 +746,18 @@ the internet.  It must contain the full path to the 
executable as well as the
 place\-holders \\${DISTDIR}, \\${FILE} and \\${URI}.  The command should be
 written to place the fetched file at \\${DISTDIR}/\\${FILE}.
 Also see \fBRESUMECOMMAND\fR.
+.RS
+.TP
+.B Optional FETCHCOMMAND Placeholders
+.TS
+l l l
+___
+l l l.
+PlaceholderMeaning Example
+
+\\${DIGESTS}   Space separated list of file digestsblake2b  
sha512 
+.TE
+.RE
 .TP
 .B FFLAGS FCFLAGS
 Use these variables to set the desired optimization/CPU instruction settings



[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

2021-01-09 Thread Zac Medico
commit: a362242c8a4e9e44c17c3d284dd2dbfc14e59635
Author: Daniel M. Weeks  danweeks  net>
AuthorDate: Wed Nov 11 17:27:15 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Jan 10 03:19:10 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=a362242c

Fix varexpand, splitting in fetch

Performing variable expansion after splitting prevents expanding a
variable to an unquoted form:

Given VAR="a b c": "echo \${VAR}" produces ['echo', 'a b c']

It should be possible for a user to control quoting such that:

"echo \"\${VAR}\"" produces ['echo', 'a b c']
"echo \${VAR}" produces ['echo', 'a', 'b', 'c']

Reordering splitting and variable expansion fixes this and mirrors the
order used in _start_gpg_proc of ManifestTask.

Signed-off-by: Daniel M. Weeks  danweeks.net>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/package/ebuild/fetch.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py 
b/lib/portage/package/ebuild/fetch.py
index ca031f31e..7be9d0239 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1296,8 +1296,9 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
if v is not None:
variables[k] = v
 
-   myfetch = shlex_split(locfetch)
-   myfetch = [varexpand(x, 
mydict=variables) for x in myfetch]
+   myfetch = varexpand(locfetch, 
mydict=variables)
+   myfetch = shlex_split(myfetch)
+
myret = -1
try:
 



[gentoo-commits] proj/portage:master commit in: lib/portage/util/futures/

2021-01-04 Thread Zac Medico
commit: 7efa7ecfe07737239be593b2c32e497cc1d2f154
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan  4 08:54:52 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan  4 08:57:09 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=7efa7ecf

_Retry: Use ensure_future for self._current_task

Use ensure_future for compatibility with PEP 492 coroutines
with async and await syntax.

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/futures/retry.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/util/futures/retry.py 
b/lib/portage/util/futures/retry.py
index 4092f60d6..31cc161da 100644
--- a/lib/portage/util/futures/retry.py
+++ b/lib/portage/util/futures/retry.py
@@ -1,4 +1,4 @@
-# Copyright 2018 Gentoo Foundation
+# Copyright 2018-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 __all__ = (
@@ -113,7 +113,7 @@ class _Retry:
 
def _begin_try(self):
self._tries += 1
-   self._current_task = self._func()
+   self._current_task = asyncio.ensure_future(self._func(), 
loop=self._loop)
self._current_task.add_done_callback(self._try_done)
if self._try_timeout is not None:
self._try_timeout_handle = self._loop.call_later(



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/util/futures/

2021-01-04 Thread Zac Medico
commit: e93549105d5f009e47710db93abea9a7aeb34324
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Jan  4 07:38:55 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Jan  4 08:11:51 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=e9354910

test_retry: use context manager to cancel pending futures

Cancel pending futures in order to avoid this "Task was destroyed but
it is pending!" warning message following migration to PEP 492
coroutines with async and await syntax:

testHangForever 
(portage.tests.util.futures.test_retry.RetryForkExecutorTestCase) ... ok
testHangForever (portage.tests.util.futures.test_retry.RetryTestCase) ... ok
testHangForever 
(portage.tests.util.futures.test_retry.RetryThreadExecutorTestCase) ... ok

--
Ran 3 tests in 0.839s

OK
Task was destroyed but it is pending!
task:  wait_for= 
cb=[RetryForkExecutorTestCase._wrap_coroutine_func..wrapper..done_callback()
 at portage/tests/util/futures/test_retry.py:192]>

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/util/futures/test_retry.py | 181 ---
 1 file changed, 106 insertions(+), 75 deletions(-)

diff --git a/lib/portage/tests/util/futures/test_retry.py 
b/lib/portage/tests/util/futures/test_retry.py
index ce5fb3e11..6648b1b2c 100644
--- a/lib/portage/tests/util/futures/test_retry.py
+++ b/lib/portage/tests/util/futures/test_retry.py
@@ -1,15 +1,18 @@
 # Copyright 2018-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
-from concurrent.futures import ThreadPoolExecutor
+from concurrent.futures import Future, ThreadPoolExecutor
+import contextlib
 
 try:
import threading
 except ImportError:
import dummy_threading as threading
 
+import weakref
 import time
 
+import portage
 from portage.tests import TestCase
 from portage.util._eventloop.global_event_loop import global_event_loop
 from portage.util.backoff import RandomExponentialBackoff
@@ -64,99 +67,100 @@ class HangForever:
A callable object that sleeps forever.
"""
def __call__(self):
-   return global_event_loop().create_future()
+   return asyncio.Future()
 
 
 class RetryTestCase(TestCase):
 
+   @contextlib.contextmanager
def _wrap_coroutine_func(self, coroutine_func):
"""
Derived classes may override this method in order to implement
alternative forms of execution.
"""
-   return coroutine_func
+   yield coroutine_func
 
def testSucceedLater(self):
loop = global_event_loop()
-   func_coroutine = self._wrap_coroutine_func(SucceedLater(1))
-   decorator = retry(try_max=,
-   delay_func=RandomExponentialBackoff(multiplier=0.1, 
base=2))
-   decorated_func = decorator(func_coroutine, loop=loop)
-   result = loop.run_until_complete(decorated_func())
-   self.assertEqual(result, 'success')
+   with self._wrap_coroutine_func(SucceedLater(1)) as 
func_coroutine:
+   decorator = retry(try_max=,
+   
delay_func=RandomExponentialBackoff(multiplier=0.1, base=2))
+   decorated_func = decorator(func_coroutine, loop=loop)
+   result = loop.run_until_complete(decorated_func())
+   self.assertEqual(result, 'success')
 
def testSucceedNever(self):
loop = global_event_loop()
-   func_coroutine = self._wrap_coroutine_func(SucceedNever())
-   decorator = retry(try_max=4, try_timeout=None,
-   delay_func=RandomExponentialBackoff(multiplier=0.1, 
base=2))
-   decorated_func = decorator(func_coroutine, loop=loop)
-   done, pending = 
loop.run_until_complete(asyncio.wait([decorated_func()], loop=loop))
-   self.assertEqual(len(done), 1)
-   self.assertTrue(isinstance(done.pop().exception().__cause__, 
SucceedNeverException))
+   with self._wrap_coroutine_func(SucceedNever()) as 
func_coroutine:
+   decorator = retry(try_max=4, try_timeout=None,
+   
delay_func=RandomExponentialBackoff(multiplier=0.1, base=2))
+   decorated_func = decorator(func_coroutine, loop=loop)
+   done, pending = 
loop.run_until_complete(asyncio.wait([decorated_func()], loop=loop))
+   self.assertEqual(len(done), 1)
+   
self.assertTrue(isinstance(done.pop().exception().__cause__, 
SucceedNeverException))
 
def testSucceedNeverReraise(self):
loop = global_event_loop()
-   func_coroutine = self._wrap_coroutine_func(SucceedNever())
-   decorator = 

[gentoo-commits] proj/portage:master commit in: .github/workflows/, lib/portage/, /

2021-01-02 Thread Zac Medico
commit: cd98dc8ccff848e9750fb729aea3de2b6c34495c
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Jan  2 06:57:30 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Jan  2 08:15:45 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=cd98dc8c

ci: enable repoman tests

Bug: https://bugs.gentoo.org/763045
Signed-off-by: Zac Medico  gentoo.org>

 .github/workflows/ci.yml | 2 +-
 lib/portage/__init__.py  | 3 ++-
 tox.ini  | 2 +-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 09c944ba1..d4b960dc3 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -29,7 +29,7 @@ jobs:
 - name: Install python dependencies
   run: |
 set -xe
-sudo apt-get install -y --no-install-recommends libxslt-dev 
libxml2-dev zstd
+sudo apt-get install -y --no-install-recommends libxslt-dev 
libxml2-dev libxml2-utils zstd
 python -VV
 python -m site
 python -m pip install --upgrade pip

diff --git a/lib/portage/__init__.py b/lib/portage/__init__.py
index 621b1d99f..178d724db 100644
--- a/lib/portage/__init__.py
+++ b/lib/portage/__init__.py
@@ -360,7 +360,8 @@ except (ImportError, OSError) as e:
 # END OF IMPORTS -- END OF IMPORTS -- END OF IMPORTS -- END OF IMPORTS -- END
 # ===
 
-_python_interpreter = os.path.realpath(sys.executable)
+_python_interpreter = (sys.executable if os.environ.get("VIRTUAL_ENV")
+   else os.path.realpath(sys.executable))
 _bin_path = PORTAGE_BIN_PATH
 _pym_path = PORTAGE_PYM_PATH
 _not_installed = os.path.isfile(os.path.join(PORTAGE_BASE_PATH, 
".portage_not_installed"))

diff --git a/tox.ini b/tox.ini
index 27b2206d6..cebf0e49c 100644
--- a/tox.ini
+++ b/tox.ini
@@ -22,4 +22,4 @@ setenv =
 commands =
bash -c 'rm -rf build && PYTHONPATH=$PWD/lib:$PWD/repoman/lib pylint *'
python -b -Wd setup.py test
-   python -b -Wd repoman/setup.py test
+   bash -c 'if python -c "import lxml.etree"; then python -b -Wd 
repoman/setup.py test; else echo "repoman tests skipped due to lxml breakage"; 
fi'



[gentoo-commits] proj/portage:master commit in: .github/workflows/

2021-01-01 Thread Zac Medico
commit: aa9de9f3509700b1b049d2e7d15ecfeb571fc938
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Jan  2 03:17:57 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Jan  2 03:20:09 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=aa9de9f3

ci: upgrade workflow to ubuntu-20.04

This will be the new default soon.

See: https://github.com/actions/virtual-environments/issues/1816
Signed-off-by: Zac Medico  gentoo.org>

 .github/workflows/ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2ec4c57aa..09c944ba1 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -9,7 +9,7 @@ on:
 jobs:
   build:
 
-runs-on: ubuntu-latest
+runs-on: ubuntu-20.04
 strategy:
   matrix:
 python-version:



[gentoo-commits] proj/portage:master commit in: .github/workflows/, /

2021-01-01 Thread Zac Medico
commit: 8d09cb1da12366a0420341be5ab11bed987eb413
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Jan  2 00:43:19 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Jan  2 02:45:30 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=8d09cb1d

.travis.yml: convert to github action

Signed-off-by: Zac Medico  gentoo.org>

 .github/workflows/ci.yml | 43 +++
 .travis.yml  | 26 --
 tox.ini  |  9 +
 3 files changed, 52 insertions(+), 26 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
new file mode 100644
index 0..2ec4c57aa
--- /dev/null
+++ b/.github/workflows/ci.yml
@@ -0,0 +1,43 @@
+name: CI
+
+on:
+  push:
+branches: [ master ]
+  pull_request:
+branches: [ master ]
+
+jobs:
+  build:
+
+runs-on: ubuntu-latest
+strategy:
+  matrix:
+python-version:
+- '3.6'
+- '3.7'
+- '3.8'
+- '3.9'
+- '3.10.0-alpha.3'
+- 'pypy-3.6'
+
+steps:
+- uses: actions/checkout@v2
+- name: Set up Python ${{ matrix.python-version }}
+  uses: actions/setup-python@v2
+  with:
+python-version: ${{ matrix.python-version }}
+- name: Install python dependencies
+  run: |
+set -xe
+sudo apt-get install -y --no-install-recommends libxslt-dev 
libxml2-dev zstd
+python -VV
+python -m site
+python -m pip install --upgrade pip
+python -m pip install tox tox-gh-actions
+- name: Test ./setup.py install --root=/tmp/install-root
+  run: |
+printf "[build_ext]\nportage-ext-modules=true" >> setup.cfg
+./setup.py install --root=/tmp/install-root
+- name: Run tox targets for ${{ matrix.python-version }}
+  run: |
+tox -vv

diff --git a/.travis.yml b/.travis.yml
deleted file mode 100644
index 297286ce0..0
--- a/.travis.yml
+++ /dev/null
@@ -1,26 +0,0 @@
-dist: focal
-language: python
-python:
-- 3.6
-- 3.7
-- 3.8
-- 3.9
-- 3.10-dev
-- pypy3
-
-# command to install dependencies
-before_install:
-# Use "dist: bionic" to get a zstd with --long support.
-- sudo apt-get -y install zstd
-install:
-- pip install tox
-
-script:
-- printf "[build_ext]\nportage-ext-modules=true" >> setup.cfg
-- ./setup.py install --root=/tmp/install-root
-- if [[ ${TRAVIS_PYTHON_VERSION/-dev/} == ?.? ]]; then
-TOX_PYTHON_VERSION=${TRAVIS_PYTHON_VERSION/-dev/};
-tox -e py${TOX_PYTHON_VERSION/./};
-  else
-tox -e ${TRAVIS_PYTHON_VERSION};
-  fi

diff --git a/tox.ini b/tox.ini
index 8aabbd2ce..27b2206d6 100644
--- a/tox.ini
+++ b/tox.ini
@@ -2,6 +2,15 @@
 envlist = py36,py37,py38,py39,py310,pypy3
 skipsdist = True
 
+[gh-actions]
+python =
+3.6: py36
+3.7: py37
+3.8: py38
+3.9: py39
+3.10: py310
+pypy-3.6: pypy3
+
 [testenv]
 deps =
pylint



[gentoo-commits] proj/portage:master commit in: lib/portage/util/

2021-01-01 Thread Zac Medico
commit: 84b555bb33f24b7628bebbaa16fe43325b7823a9
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Jan  2 01:18:30 2021 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Jan  2 01:18:43 2021 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=84b555bb

lib/portage/util/__init__.py: Fix useless-return

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/__init__.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lib/portage/util/__init__.py b/lib/portage/util/__init__.py
index bedcbcfe6..b0922bf09 100644
--- a/lib/portage/util/__init__.py
+++ b/lib/portage/util/__init__.py
@@ -1293,7 +1293,6 @@ class atomic_ofstream(AbstractContextManager, 
ObjectProxy):
self.abort()
else:
self.close()
-   return None
 
def _get_target(self):
return object.__getattribute__(self, '_file')



[gentoo-commits] proj/portage:master commit in: /

2020-12-30 Thread Zac Medico
commit: d5e0f2a452181642ace081de322112822e4171e3
Author: Zac Medico  gentoo  org>
AuthorDate: Thu Dec 31 02:41:26 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Thu Dec 31 02:45:56 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=d5e0f2a4

.travis.yml: remove duplicate call to ./setup.py test

It's called via tox.ini, but the duplication was first introduced
in commit 3e784a1daa3ff9e4cf15181e8c864db1d5f2fff1.

Signed-off-by: Zac Medico  gentoo.org>

 .travis.yml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 3a40617fb..297286ce0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -17,7 +17,6 @@ install:
 
 script:
 - printf "[build_ext]\nportage-ext-modules=true" >> setup.cfg
-- ./setup.py test
 - ./setup.py install --root=/tmp/install-root
 - if [[ ${TRAVIS_PYTHON_VERSION/-dev/} == ?.? ]]; then
 TOX_PYTHON_VERSION=${TRAVIS_PYTHON_VERSION/-dev/};



[gentoo-commits] proj/portage:master commit in: lib/portage/util/

2020-12-30 Thread Zac Medico
commit: 1574ae127b270739c4293271c959d1d981684906
Author: Florian Schmaus  geekplace  eu>
AuthorDate: Fri Dec 18 18:46:39 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Thu Dec 31 01:43:22 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=1574ae12

env_update: use "with statement" on atomic_ofstream

Signed-off-by: Florian Schmaus  geekplace.eu>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/env_update.py | 30 --
 1 file changed, 12 insertions(+), 18 deletions(-)

diff --git a/lib/portage/util/env_update.py b/lib/portage/util/env_update.py
index dec086cf8..5588931a8 100644
--- a/lib/portage/util/env_update.py
+++ b/lib/portage/util/env_update.py
@@ -342,18 +342,17 @@ def _env_update(makelinks, target_root, prev_mtimes, 
contents, env,
 
#create /etc/profile.env for bash support
profile_env_path = os.path.join(eroot, "etc", "profile.env")
-   outfile = atomic_ofstream(profile_env_path)
-   outfile.write(penvnotice)
-
-   env_keys = [x for x in env if x != "LDPATH"]
-   env_keys.sort()
-   for k in env_keys:
-   v = env[k]
-   if v.startswith('$') and not v.startswith('${'):
-   outfile.write("export %s=$'%s'\n" % (k, v[1:]))
-   else:
-   outfile.write("export %s='%s'\n" % (k, v))
-   outfile.close()
+   with atomic_ofstream(profile_env_path) as outfile:
+   outfile.write(penvnotice)
+
+   env_keys = [x for x in env if x != "LDPATH"]
+   env_keys.sort()
+   for k in env_keys:
+   v = env[k]
+   if v.startswith('$') and not v.startswith('${'):
+   outfile.write("export %s=$'%s'\n" % (k, v[1:]))
+   else:
+   outfile.write("export %s='%s'\n" % (k, v))
 
# Create the systemd user environment configuration file
# /etc/environment.d/10-gentoo-env.conf with the
@@ -363,8 +362,7 @@ def _env_update(makelinks, target_root, prev_mtimes, 
contents, env,
 
systemd_gentoo_env_path = os.path.join(systemd_environment_dir,
"10-gentoo-env.conf")
-   systemd_gentoo_env = atomic_ofstream(systemd_gentoo_env_path)
-   try:
+   with atomic_ofstream(systemd_gentoo_env_path) as systemd_gentoo_env:
senvnotice = notice + "\n\n"
systemd_gentoo_env.write(senvnotice)
 
@@ -384,10 +382,6 @@ def _env_update(makelinks, target_root, prev_mtimes, 
contents, env,
line = f"{env_key}={env_key_value}\n"
 
systemd_gentoo_env.write(line)
-   except:
-   systemd_gentoo_env.abort()
-   raise
-   systemd_gentoo_env.close()
 
#create /etc/csh.env for (t)csh support
outfile = atomic_ofstream(os.path.join(eroot, "etc", "csh.env"))



[gentoo-commits] proj/portage:master commit in: lib/portage/util/

2020-12-30 Thread Zac Medico
commit: e93e6d65fa1ca75f676a227f7918f8b6d747425c
Author: Florian Schmaus  geekplace  eu>
AuthorDate: Fri Dec 18 18:46:38 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Thu Dec 31 01:40:42 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=e93e6d65

Make atomic_ofstream a Context Manager

This allows using a "with statement" together with instances of
atomic_ofstream. Allowing for more readable, less error prone and
shorter code.

Signed-off-by: Florian Schmaus  geekplace.eu>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/__init__.py | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/lib/portage/util/__init__.py b/lib/portage/util/__init__.py
index 0412b2b59..bedcbcfe6 100644
--- a/lib/portage/util/__init__.py
+++ b/lib/portage/util/__init__.py
@@ -11,6 +11,7 @@ __all__ = ['apply_permissions', 'apply_recursive_permissions',
'stack_dicts', 'stack_lists', 'unique_array', 'unique_everseen', 
'varexpand',
'write_atomic', 'writedict', 'writemsg', 'writemsg_level', 
'writemsg_stdout']
 
+from contextlib import AbstractContextManager
 from copy import deepcopy
 import errno
 import io
@@ -1246,7 +1247,7 @@ def apply_secpass_permissions(filename, uid=-1, gid=-1, 
mode=-1, mask=-1,
stat_cached=stat_cached, follow_links=follow_links)
return all_applied
 
-class atomic_ofstream(ObjectProxy):
+class atomic_ofstream(AbstractContextManager, ObjectProxy):
"""Write a file atomically via os.rename().  Atomic replacement prevents
interprocess interference and prevents corruption of the target
file when the write is interrupted (for example, when an 'out of space'
@@ -1287,6 +1288,13 @@ class atomic_ofstream(ObjectProxy):
encoding=_encodings['fs'], errors='strict'),
mode=mode, **kargs))
 
+   def __exit__(self, exc_type, exc_val, exc_tb):
+   if exc_type is not None:
+   self.abort()
+   else:
+   self.close()
+   return None
+
def _get_target(self):
return object.__getattribute__(self, '_file')
 



[gentoo-commits] proj/portage:master commit in: /, repoman/

2020-12-30 Thread Zac Medico
commit: 57c13cb8630a7ee78d714a4cc15bceb3d0fd2838
Author: Manuel Rüger  rueg  eu>
AuthorDate: Wed Dec 30 14:40:54 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Thu Dec 31 01:02:45 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=57c13cb8

Test with Python 3.10

Closes: https://github.com/gentoo/portage/pull/649
Signed-off-by: Manuel Rüger  rueg.eu>
Signed-off-by: Zac Medico  gentoo.org>

 .travis.yml  | 5 +++--
 repoman/runtests | 5 +++--
 runtests | 5 +++--
 tox.ini  | 4 ++--
 4 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index d2935fdab..3a40617fb 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,10 +1,11 @@
-dist: bionic
+dist: focal
 language: python
 python:
 - 3.6
 - 3.7
 - 3.8
-- 3.9-dev
+- 3.9
+- 3.10-dev
 - pypy3
 
 # command to install dependencies

diff --git a/repoman/runtests b/repoman/runtests
index 3edaaf0a8..5137b5e6e 100755
--- a/repoman/runtests
+++ b/repoman/runtests
@@ -24,12 +24,13 @@ PYTHON_SUPPORTED_VERSIONS = [
'2.7',
'3.6',
'3.7',
-   '3.8'
+   '3.8',
+   '3.9'
 ]
 # The rest are just "nice to have".
 PYTHON_NICE_VERSIONS = [
'pypy3',
-   '3.9'
+   '3.10'
 ]
 
 EPREFIX = os.environ.get('PORTAGE_OVERRIDE_EPREFIX', '/')

diff --git a/runtests b/runtests
index 685a7d9c7..c062f46cb 100755
--- a/runtests
+++ b/runtests
@@ -23,12 +23,13 @@ import tempfile
 PYTHON_SUPPORTED_VERSIONS = [
'3.6',
'3.7',
-   '3.8'
+   '3.8',
+   '3.9'
 ]
 # The rest are just "nice to have".
 PYTHON_NICE_VERSIONS = [
'pypy3',
-   '3.9'
+   '3.10'
 ]
 
 EPREFIX = os.environ.get('PORTAGE_OVERRIDE_EPREFIX', '/')

diff --git a/tox.ini b/tox.ini
index d6c8cf3b3..8aabbd2ce 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,5 @@
 [tox]
-envlist = py36,py37,py38,py39,pypy3
+envlist = py36,py37,py38,py39,py310,pypy3
 skipsdist = True
 
 [testenv]
@@ -7,7 +7,7 @@ deps =
pylint
pygost
pyyaml
-   py36,py37,py38,py39,pypy3: lxml!=4.2.0
+   py36,py37,py38,py39,py310,pypy3: lxml!=4.2.0
 setenv =
PYTHONPATH={toxinidir}/lib
 commands =



[gentoo-commits] proj/portage:master commit in: src/

2020-12-24 Thread Mike Gilbert
commit: ff72973264827cbde7ba642a22e5d0579eeeb57a
Author: Mike Gilbert  gentoo  org>
AuthorDate: Thu Dec 24 18:11:22 2020 +
Commit: Mike Gilbert  gentoo  org>
CommitDate: Fri Dec 25 00:26:02 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=ff729732

Drop Python 2 compatibility in extension modules

Signed-off-by: Mike Gilbert  gentoo.org>

 src/portage_util_file_copy_reflink_linux.c | 10 +-
 src/portage_util_libc.c| 10 +-
 2 files changed, 2 insertions(+), 18 deletions(-)

diff --git a/src/portage_util_file_copy_reflink_linux.c 
b/src/portage_util_file_copy_reflink_linux.c
index 352342c06..c6affe57a 100644
--- a/src/portage_util_file_copy_reflink_linux.c
+++ b/src/portage_util_file_copy_reflink_linux.c
@@ -1,4 +1,4 @@
-/* Copyright 2017 Gentoo Foundation
+/* Copyright 2017-2020 Gentoo Authors
  * Distributed under the terms of the GNU General Public License v2
  */
 
@@ -25,7 +25,6 @@ static PyMethodDef reflink_linuxMethods[] = {
 {NULL, NULL, 0, NULL}
 };
 
-#if PY_MAJOR_VERSION >= 3
 static struct PyModuleDef moduledef = {
 PyModuleDef_HEAD_INIT,
 "reflink_linux",/* m_name */
@@ -45,13 +44,6 @@ PyInit_reflink_linux(void)
 m = PyModule_Create();
 return m;
 }
-#else
-PyMODINIT_FUNC
-initreflink_linux(void)
-{
-Py_InitModule("reflink_linux", reflink_linuxMethods);
-}
-#endif
 
 
 /**

diff --git a/src/portage_util_libc.c b/src/portage_util_libc.c
index 977b95474..2a3e624dc 100644
--- a/src/portage_util_libc.c
+++ b/src/portage_util_libc.c
@@ -1,4 +1,4 @@
-/* Copyright 2005-2016 Gentoo Foundation
+/* Copyright 2005-2020 Gentoo Authors
  * Distributed under the terms of the GNU General Public License v2
  */
 
@@ -15,7 +15,6 @@ static PyMethodDef LibcMethods[] = {
{NULL, NULL, 0, NULL}
 };
 
-#if PY_MAJOR_VERSION >= 3
 static struct PyModuleDef moduledef = {
PyModuleDef_HEAD_INIT,
"libc", /* 
m_name */
@@ -35,13 +34,6 @@ PyInit_libc(void)
m = PyModule_Create();
return m;
 }
-#else
-PyMODINIT_FUNC
-initlibc(void)
-{
-   Py_InitModule("libc", LibcMethods);
-}
-#endif
 
 
 static PyObject *



[gentoo-commits] proj/portage:master commit in: bin/

2020-12-24 Thread Mike Gilbert
commit: aa20110bde339046fc3d57538ca8ad47227fba99
Author: Mike Gilbert  gentoo  org>
AuthorDate: Thu Dec 24 03:21:12 2020 +
Commit: Mike Gilbert  gentoo  org>
CommitDate: Thu Dec 24 03:30:04 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=aa20110b

Adjust mangling of "arch" value from scanelf output

scanelf may generate output that looks like this:

```
UNKNOWN_TYPE;lib/firmware/ath10k/WCN3990/hw1.0/wlanmdsp.mbn;;  -  ;
EM_ARM;lib/firmware/mediatek/mt8183/scp.img;;  -  ;
...
```

Previously, we removed the first 3 characters of the first field and
stored this as the "arch" in NEEDED.ELF.2. This unintentionally
changes "UNKNOWN_TYPE" to "NOWN_TYPE".

Instead, let's just remove the string "EM_" from the front.

Signed-off-by: Mike Gilbert  gentoo.org>

 bin/misc-functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index c2a16cbe0..d7009d7eb 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -194,7 +194,7 @@ install_qa_check() {
fi
 
echo "${obj} ${needed}" >> 
"${PORTAGE_BUILDDIR}"/build-info/NEEDED
-   echo "${arch:3};${obj};${soname};${rpath};${needed}" >> 
"${PORTAGE_BUILDDIR}"/build-info/NEEDED.ELF.2
+   echo "${arch#EM_};${obj};${soname};${rpath};${needed}" 
>> "${PORTAGE_BUILDDIR}"/build-info/NEEDED.ELF.2
done }
 
[ -n "${QA_SONAME_NO_SYMLINK}" ] && \



[gentoo-commits] proj/portage:master commit in: /

2020-12-07 Thread Zac Medico
commit: 9576de5d1e8538524224d76275d59683fd4871af
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Dec  7 07:57:17 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Dec  7 08:04:05 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=9576de5d

Updates for portage-3.0.12 release

Signed-off-by: Zac Medico  gentoo.org>

 RELEASE-NOTES | 6 ++
 setup.py  | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index 7fff83017..0bbe7ad07 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,12 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.12
+==
+* Bug Fixes:
+- Bug 758740 Use default asyncio event loop in child processes
+- Bug 758755 Use default asyncio event loop in API consumer threads
+
 portage-3.0.11
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index 95a53307d..be69fffdb 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.11',
+   version = '3.0.12',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/portage/util/futures/_asyncio/

2020-12-07 Thread Zac Medico
commit: dcbcac809213537afaa6b4f9822146a2e984f773
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Dec  7 06:05:04 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Dec  7 07:48:10 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=dcbcac80

_safe_loop: instantiate asyncio loop for API consumer thread

In order to maintain compatibility with an API consumer thread which
has not instantiated an asyncio loop for the current thread prior
to calling the portage API, instantiate a loop on its behalf. Since
a ResourceWarning will be triggered if the loop has not been closed
before the process exits, add the loop to a WeakValueDictionary,
and close it if it still exists during exit for the current pid.

Fixes: cecd2f8a259c ("Use default asyncio event loop implementation in API 
consumer threads")
Bug: https://bugs.gentoo.org/758755
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/futures/_asyncio/__init__.py | 49 ++-
 1 file changed, 40 insertions(+), 9 deletions(-)

diff --git a/lib/portage/util/futures/_asyncio/__init__.py 
b/lib/portage/util/futures/_asyncio/__init__.py
index 6f3395a91..d39f31786 100644
--- a/lib/portage/util/futures/_asyncio/__init__.py
+++ b/lib/portage/util/futures/_asyncio/__init__.py
@@ -21,7 +21,8 @@ __all__ = (
 )
 
 import subprocess
-import sys
+import types
+import weakref
 
 import asyncio as _real_asyncio
 
@@ -249,23 +250,53 @@ def _safe_loop():
the main thread, this returns a globally shared event loop instance.
 
For external API consumers calling from a non-main thread, an
-   asyncio loop must be registered for the current thread, or else an
-   error will be raised like this:
+   asyncio loop must be registered for the current thread, or else the
+   asyncio.get_event_loop() function will raise an error like this:
 
  RuntimeError: There is no current event loop in thread 'Thread-1'.
 
-   In order to avoid this RuntimeError, the external API consumer
-   is responsible for setting an event loop and managing its lifecycle.
-   For example, this code will set an event loop for the current thread:
+   In order to avoid this RuntimeError, a loop will be automatically
+   created like this:
 
  asyncio.set_event_loop(asyncio.new_event_loop())
 
-   In order to avoid a ResourceWarning, the caller should also close the
-   corresponding loop before the current thread terminates.
+   In order to avoid a ResourceWarning, automatically created loops
+   are added to a WeakValueDictionary, and closed via an atexit hook
+   if they still exist during exit for the current pid.
 
@rtype: asyncio.AbstractEventLoop (or compatible)
@return: event loop instance
"""
if portage._internal_caller or threading.current_thread() is 
threading.main_thread():
return _global_event_loop()
-   return _AsyncioEventLoop()
+
+   thread_key = threading.get_ident()
+   with _thread_weakrefs.lock:
+   if _thread_weakrefs.pid != portage.getpid():
+   _thread_weakrefs.pid = portage.getpid()
+   _thread_weakrefs.loops = weakref.WeakValueDictionary()
+   try:
+   loop = _thread_weakrefs.loops[thread_key]
+   except KeyError:
+   try:
+   _real_asyncio.get_event_loop()
+   except RuntimeError:
+   
_real_asyncio.set_event_loop(_real_asyncio.new_event_loop())
+   loop = _thread_weakrefs.loops[thread_key] = 
_AsyncioEventLoop()
+   return loop
+
+
+def _thread_weakrefs_atexit():
+   with _thread_weakrefs.lock:
+   if _thread_weakrefs.pid == portage.getpid():
+   while True:
+   try:
+   thread_key, loop = 
_thread_weakrefs.loops.popitem()
+   except KeyError:
+   break
+   else:
+   loop.close()
+
+
+_thread_weakrefs = types.SimpleNamespace(lock=threading.Lock(), loops=None, 
pid=None)
+portage.process.atexit_register(_thread_weakrefs_atexit)



[gentoo-commits] proj/portage:master commit in: lib/portage/dbapi/, lib/portage/package/ebuild/

2020-12-06 Thread Zac Medico
commit: 87ca3d4b09324a2359ea10aa69c1f2875c531486
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Dec  7 02:54:26 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Dec  7 02:54:52 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=87ca3d4b

Remove unused EventLoop lazy imports

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dbapi/vartree.py   | 1 -
 lib/portage/package/ebuild/doebuild.py | 1 -
 2 files changed, 2 deletions(-)

diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
index 1547d2f6d..f3d74cf82 100644
--- a/lib/portage/dbapi/vartree.py
+++ b/lib/portage/dbapi/vartree.py
@@ -41,7 +41,6 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util._dyn_libs.LinkageMapELF:LinkageMapELF@LinkageMap',
'portage.util._dyn_libs.NeededEntry:NeededEntry',
'portage.util._async.SchedulerInterface:SchedulerInterface',
-   'portage.util._eventloop.EventLoop:EventLoop',
'portage.util._eventloop.global_event_loop:global_event_loop',
'portage.versions:best,catpkgsplit,catsplit,cpv_getkey,vercmp,' + \
'_get_slot_re,_pkgsplit@pkgsplit,_pkg_str,_unknown_repo',

diff --git a/lib/portage/package/ebuild/doebuild.py 
b/lib/portage/package/ebuild/doebuild.py
index 3b1991b28..f6cee4518 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -39,7 +39,6 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util._dyn_libs.NeededEntry:NeededEntry',
'portage.util._dyn_libs.soname_deps:SonameDepsProcessor',
'portage.util._async.SchedulerInterface:SchedulerInterface',
-   'portage.util._eventloop.EventLoop:EventLoop',
'portage.util._eventloop.global_event_loop:global_event_loop',
'portage.util.ExtractKernelVersion:ExtractKernelVersion'
 )



[gentoo-commits] proj/portage:master commit in: lib/portage/util/_eventloop/, lib/portage/util/futures/_asyncio/

2020-12-06 Thread Zac Medico
commit: cecd2f8a259cf2991f2324c9a14e26170ba0ddcf
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Dec  6 09:25:17 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Dec  7 02:32:27 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=cecd2f8a

Use default asyncio event loop implementation in API consumer threads

Make the _safe_loop function return an AsyncioEventLoop instance,
so that the default asyncio event loop implementation will be used
in API consumer threads. This is possible because the underlying
asyncio.get_event_loop() function returns a separate  event loop for
each thread. The AsyncioEventLoop _run_until_complete method will
now appropriately handle a ValueError from signal.set_wakeup_fd(-1)
if it is not called in the main thread.

For external API consumers calling from a non-main thread, an
asyncio loop must be registered for the current thread, or else an
error will be raised like this:

  RuntimeError: There is no current event loop in thread 'Thread-1'.

In order to avoid this RuntimeError, the external API consumer
is responsible for setting an event loop and managing its lifecycle.
For example, this code will set an event loop for the current thread:

  asyncio.set_event_loop(asyncio.new_event_loop())

In order to avoid a ResourceWarning, the caller should also close
the corresponding loop before the current thread terminates.

Bug: https://bugs.gentoo.org/758755
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/_eventloop/asyncio_event_loop.py |  6 +-
 lib/portage/util/futures/_asyncio/__init__.py | 26 +--
 2 files changed, 24 insertions(+), 8 deletions(-)

diff --git a/lib/portage/util/_eventloop/asyncio_event_loop.py 
b/lib/portage/util/_eventloop/asyncio_event_loop.py
index 836f1c30a..4d7047ae8 100644
--- a/lib/portage/util/_eventloop/asyncio_event_loop.py
+++ b/lib/portage/util/_eventloop/asyncio_event_loop.py
@@ -121,4 +121,8 @@ class AsyncioEventLoop(_AbstractEventLoop):
try:
return self._loop.run_until_complete(future)
finally:
-   self._wakeup_fd = signal.set_wakeup_fd(-1)
+   try:
+   self._wakeup_fd = signal.set_wakeup_fd(-1)
+   except ValueError:
+   # This is intended to fail when not called in 
the main thread.
+   pass

diff --git a/lib/portage/util/futures/_asyncio/__init__.py 
b/lib/portage/util/futures/_asyncio/__init__.py
index a902ad895..6f3395a91 100644
--- a/lib/portage/util/futures/_asyncio/__init__.py
+++ b/lib/portage/util/futures/_asyncio/__init__.py
@@ -34,7 +34,6 @@ import portage
 portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.futures.unix_events:_PortageEventLoopPolicy',
'portage.util.futures:compat_coroutine@_compat_coroutine',
-   'portage.util._eventloop.EventLoop:EventLoop@_EventLoop',
 )
 from portage.util._eventloop.asyncio_event_loop import AsyncioEventLoop as 
_AsyncioEventLoop
 from portage.util._eventloop.global_event_loop import (
@@ -246,14 +245,27 @@ def _wrap_loop(loop=None):
 def _safe_loop():
"""
Return an event loop that's safe to use within the current context.
-   For portage internal callers, this returns a globally shared event
-   loop instance. For external API consumers, this constructs a
-   temporary event loop instance that's safe to use in a non-main
-   thread (it does not override the global SIGCHLD handler).
+   For portage internal callers or external API consumers calling from
+   the main thread, this returns a globally shared event loop instance.
+
+   For external API consumers calling from a non-main thread, an
+   asyncio loop must be registered for the current thread, or else an
+   error will be raised like this:
+
+ RuntimeError: There is no current event loop in thread 'Thread-1'.
+
+   In order to avoid this RuntimeError, the external API consumer
+   is responsible for setting an event loop and managing its lifecycle.
+   For example, this code will set an event loop for the current thread:
+
+ asyncio.set_event_loop(asyncio.new_event_loop())
+
+   In order to avoid a ResourceWarning, the caller should also close the
+   corresponding loop before the current thread terminates.
 
@rtype: asyncio.AbstractEventLoop (or compatible)
@return: event loop instance
"""
-   if portage._internal_caller:
+   if portage._internal_caller or threading.current_thread() is 
threading.main_thread():
return _global_event_loop()
-   return _EventLoop(main=False)
+   return _AsyncioEventLoop()



[gentoo-commits] proj/portage:master commit in: lib/portage/, lib/portage/util/_eventloop/

2020-12-06 Thread Zac Medico
commit: 58d8e4f6a917e262596108eb7bed1c2ef981e9de
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Dec  6 08:44:20 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Dec  7 02:28:28 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=58d8e4f6

Use default asyncio event loop implementation in child processes

Use the default asyncio event loop implementation in child
processes, instead of portage's internal EventLoop. After
fork, force instantiation of a new event loop policy as
a workaround for https://bugs.python.org/issue22087, which
is necessary for RetryTestCase to succeed.

Bug: https://bugs.gentoo.org/758740
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/__init__.py  | 4 
 lib/portage/util/_eventloop/global_event_loop.py | 7 ---
 2 files changed, 4 insertions(+), 7 deletions(-)

diff --git a/lib/portage/__init__.py b/lib/portage/__init__.py
index 4d4b590a8..621b1d99f 100644
--- a/lib/portage/__init__.py
+++ b/lib/portage/__init__.py
@@ -9,6 +9,7 @@ VERSION = "HEAD"
 # ===
 
 try:
+   import asyncio
import sys
import errno
if not hasattr(errno, 'ESTALE'):
@@ -373,6 +374,9 @@ class _ForkWatcher:
@staticmethod
def hook(_ForkWatcher):
_ForkWatcher.current_pid = _os.getpid()
+   # Force instantiation of a new event loop policy as a workaround
+   # for https://bugs.python.org/issue22087.
+   asyncio.set_event_loop_policy(None)
 
 _ForkWatcher.hook(_ForkWatcher)
 

diff --git a/lib/portage/util/_eventloop/global_event_loop.py 
b/lib/portage/util/_eventloop/global_event_loop.py
index 21a1d1970..413011178 100644
--- a/lib/portage/util/_eventloop/global_event_loop.py
+++ b/lib/portage/util/_eventloop/global_event_loop.py
@@ -2,11 +2,8 @@
 # Distributed under the terms of the GNU General Public License v2
 
 import portage
-from .EventLoop import EventLoop
 from portage.util._eventloop.asyncio_event_loop import AsyncioEventLoop
 
-
-_MAIN_PID = portage.getpid()
 _instances = {}
 
 
@@ -22,10 +19,6 @@ def global_event_loop():
return instance
 
constructor = AsyncioEventLoop
-   # If the default constructor doesn't support multiprocessing,
-   # then multiprocessing constructor is used in subprocesses.
-   if not constructor.supports_multiprocessing and pid != _MAIN_PID:
-   constructor = EventLoop
 
# Use the _asyncio_wrapper attribute, so that unit tests can compare
# the reference to one retured from _wrap_loop(), since they should



[gentoo-commits] proj/portage:master commit in: lib/_emerge/

2020-12-03 Thread Zac Medico
commit: ff270ae58eca457a1e2bbf9507d8438123082ca0
Author: Zac Medico  gentoo  org>
AuthorDate: Thu Dec  3 23:01:30 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Thu Dec  3 23:07:06 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=ff270ae5

depgraph._add_pkg: fix AttributeError when myparent is not a Package

Fixes: b991f23ad915 ("Allow a package to replace its own buildtime dependency")
Bug: https://bugs.gentoo.org/757741
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/depgraph.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index 0450291d4..f3e834a60 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -3107,7 +3107,7 @@ class depgraph:

self._frozen_config.myopts,

modified_use=self._pkg_use_enabled(pkg))),

level=logging.DEBUG, noiselevel=-1)
-   elif (pkg.installed and myparent and
+   elif (pkg.installed and isinstance(myparent, 
Package) and
pkg.root == myparent.root and
pkg.slot_atom == myparent.slot_atom):
# If the parent package is replacing 
the child package then



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/resolver/

2020-12-02 Thread Zac Medico
commit: c1671591dc2b46038c871f9038ff4cb46dccc160
Author: Zac Medico  gentoo  org>
AuthorDate: Wed Dec  2 17:28:45 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Wed Dec  2 17:32:36 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=c1671591

MergeOrderTestCase: fix x11-drivers/xf86-video-fbdev slot operator deps

The real ebuild has the x11-base/xorg-server:= dependency in RDEPEND
rather than DEPEND.

Fixes: 07a604537e74 ("find_smallest_cycle: don't merge satisfied PDEPEND too 
early")
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/resolver/test_merge_order.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/portage/tests/resolver/test_merge_order.py 
b/lib/portage/tests/resolver/test_merge_order.py
index 0510a0636..db063ed9d 100644
--- a/lib/portage/tests/resolver/test_merge_order.py
+++ b/lib/portage/tests/resolver/test_merge_order.py
@@ -231,8 +231,8 @@ class MergeOrderTestCase(TestCase):
},
"x11-drivers/xf86-video-fbdev-0.5.0-r1": {
"EAPI": "7",
-   "DEPEND": "x11-base/xorg-server:=",
-   "RDEPEND": "x11-base/xorg-server",
+   "DEPEND": "x11-base/xorg-server",
+   "RDEPEND": "x11-base/xorg-server:=",
}
}
 
@@ -325,8 +325,8 @@ class MergeOrderTestCase(TestCase):
},
"x11-drivers/xf86-video-fbdev-0.5.0-r1": {
"EAPI": "7",
-   "DEPEND": "x11-base/xorg-server:0/1.14.1=",
-   "RDEPEND": "x11-base/xorg-server",
+   "DEPEND": "x11-base/xorg-server",
+   "RDEPEND": "x11-base/xorg-server:0/1.14.1=",
}
}
 



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/resolver/, lib/_emerge/

2020-12-02 Thread Zac Medico
commit: 07a604537e746814613dc171a5c09072ef0266af
Author: Zac Medico  gentoo  org>
AuthorDate: Wed Dec  2 07:08:04 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Wed Dec  2 08:06:27 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=07a60453

find_smallest_cycle: don't merge satisfied PDEPEND too early

After PDEPENDs have been neglected by the find_smallest_cycle function,
do not try to merge them too early if they are already satisfied by
an installed package. This fixes incorrect merge order for PDEPEND
cycles involving xorg-server and xorg-drivers, which was triggered
by commit 5095c2023595a75e2848f1ad3dbe25b5fb451a44 because it gave
PDEPEND higher priority than satisfied buildtime dependencies.

Fixes: 5095c2023595 ("find_smallest_cycle: enhance search prioritization")
Reported-by: josef64 in #gentoo-portage
Bug: https://bugs.gentoo.org/754903
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/DepPrioritySatisfiedRange.py   |  1 +
 lib/_emerge/depgraph.py|  8 
 lib/portage/tests/resolver/test_merge_order.py | 27 --
 3 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/lib/_emerge/DepPrioritySatisfiedRange.py 
b/lib/_emerge/DepPrioritySatisfiedRange.py
index fb0d7db4e..f546590e0 100644
--- a/lib/_emerge/DepPrioritySatisfiedRange.py
+++ b/lib/_emerge/DepPrioritySatisfiedRange.py
@@ -93,6 +93,7 @@ class DepPrioritySatisfiedRange:
ignore_medium  = _ignore_runtime
ignore_medium_soft = _ignore_satisfied_buildtime_slot_op
ignore_medium_post = _ignore_runtime_post
+   ignore_medium_post_satisifed = _ignore_satisfied_runtime_post
ignore_soft= _ignore_optional
 
 

diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index 1271bda3e..0450291d4 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -8052,18 +8052,18 @@ class depgraph:

(selected_nodes[0],), noiselevel=-1)
 
if selected_nodes and ignore_priority is not None:
-   # Try to merge ignored medium_post deps as soon 
as possible
+   # Try to merge neglected medium_post deps as 
soon as possible
# if they're not satisfied by installed 
packages.
for node in selected_nodes:
children = 
set(mygraph.child_nodes(node))
-   soft = children.difference(
+   medium_post_satisifed = 
children.difference(
mygraph.child_nodes(node,
ignore_priority = \
-   
DepPrioritySatisfiedRange.ignore_soft))
+   
DepPrioritySatisfiedRange.ignore_medium_post_satisifed))
medium_post = children.difference(
mygraph.child_nodes(node,

ignore_priority=DepPrioritySatisfiedRange.ignore_medium_post))
-   medium_post -= soft
+   medium_post -= medium_post_satisifed
for child in medium_post:
if child in selected_nodes:
continue

diff --git a/lib/portage/tests/resolver/test_merge_order.py 
b/lib/portage/tests/resolver/test_merge_order.py
index f81fd2f6f..0510a0636 100644
--- a/lib/portage/tests/resolver/test_merge_order.py
+++ b/lib/portage/tests/resolver/test_merge_order.py
@@ -217,12 +217,23 @@ class MergeOrderTestCase(TestCase):
"IUSE" : "X +encode",
"RDEPEND" : "|| ( 
>=media-video/ffmpeg-0.6.90_rc0-r2[X=,encode=] 
>=media-video/libav-0.6.90_rc[X=,encode=] )",
},
+   "x11-base/xorg-drivers-1.20-r2": {
+   "EAPI": "7",
+   "IUSE": "+video_cards_fbdev",
+   "PDEPEND": "x11-base/xorg-server 
video_cards_fbdev? ( x11-drivers/xf86-video-fbdev )",
+   },
"x11-base/xorg-server-1.14.1" : {
"EAPI" : "5",
"SLOT": "0/1.14.1",
"DEPEND" : "media-libs/mesa",
"RDEPEND" : "media-libs/mesa",
+   "PDEPEND": "x11-base/xorg-drivers",
},
+   "x11-drivers/xf86-video-fbdev-0.5.0-r1": {
+   

[gentoo-commits] proj/portage:master commit in: /

2020-11-28 Thread Zac Medico
commit: 27d44788626684cf9973f6e175ebf3d4946fd8d2
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Nov 28 21:03:29 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Nov 28 21:03:41 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=27d44788

Updates for portage-3.0.11 release

Signed-off-by: Zac Medico  gentoo.org>

 RELEASE-NOTES | 6 ++
 setup.py  | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index 61c3c3d16..7fff83017 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,12 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.11
+==
+* Bug Fixes:
+- Bug 756961 handle dev-lang/rust[system-bootstrap] dependency cycle
+- Bug 757306 backtracking: fix virtual choices for circular deps
+
 portage-3.0.10
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index 443b94422..95a53307d 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.10',
+   version = '3.0.11',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/resolver/, lib/_emerge/, lib/portage/dep/

2020-11-28 Thread Zac Medico
commit: b991f23ad915d1fc37088f11ca855c8c8374f5ae
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Nov 28 01:57:14 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Nov 28 20:50:13 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=b991f23a

Allow a package to replace its own buildtime dependency

If a package has a buildtime dependency on a previous version that
it will replace, then do not treat it as a slot conflict. This
solves inappropriate behavior for dev-lang/rust[system-bootstrap].

This requires adjustments to package selection logic in several
locations, in order to ensure that an installed package instance
will be selected to satisfy a buildtime dependency when
appropriate. Dependencies of the installed package will be
entirely ignored, but that has already been the case when using
installed package to break cycles, as discussed in bug 199856.

Bug: https://bugs.gentoo.org/756793
Bug: https://bugs.gentoo.org/756961
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/depgraph.py| 68 +---
 lib/portage/dep/dep_check.py   | 24 +++---
 .../tests/resolver/test_circular_choices_rust.py   | 94 ++
 3 files changed, 164 insertions(+), 22 deletions(-)

diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index d10474ab3..1271bda3e 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -85,6 +85,8 @@ from _emerge.resolver.output import Display, 
format_unmatched_atom
 
 # Exposes a depgraph interface to dep_check.
 _dep_check_graph_interface = 
collections.namedtuple('_dep_check_graph_interface',(
+   # Checks if parent package will replace child.
+   'will_replace_child',
# Indicates a removal action, like depclean or prune.
'removal_action',
# Checks if update is desirable for a given package.
@@ -507,6 +509,7 @@ class _dynamic_depgraph_config:
# Track missed updates caused by solved conflicts.
self._conflict_missed_update = collections.defaultdict(dict)
dep_check_iface = _dep_check_graph_interface(
+   will_replace_child=depgraph._will_replace_child,
removal_action="remove" in myparams,
want_update_pkg=depgraph._want_update_pkg,
)
@@ -3104,6 +3107,22 @@ class depgraph:

self._frozen_config.myopts,

modified_use=self._pkg_use_enabled(pkg))),

level=logging.DEBUG, noiselevel=-1)
+   elif (pkg.installed and myparent and
+   pkg.root == myparent.root and
+   pkg.slot_atom == myparent.slot_atom):
+   # If the parent package is replacing 
the child package then
+   # there's no slot conflict. Since the 
child will be replaced,
+   # do not add it to the graph. No 
attempt will be made to
+   # satisfy its dependencies, which is 
unsafe if it has any
+   # missing dependencies, as discussed in 
bug 199856.
+   if debug:
+   writemsg_level(
+   "%s%s %s\n" % ("Replace 
Child:".ljust(15),
+   pkg, 
pkg_use_display(pkg,
+   
self._frozen_config.myopts,
+   
modified_use=self._pkg_use_enabled(pkg))),
+   level=logging.DEBUG, 
noiselevel=-1)
+   return 1
 
else:
if debug:
@@ -5877,6 +5896,27 @@ class depgraph:
(arg_atoms or update) and
not self._too_deep(depth))
 
+   def _will_replace_child(self, parent, root, atom):
+   """
+   Check if a given parent package will replace a child package
+   for the given root and atom.
+
+   @param parent: parent package
+   @type parent: Package
+   @param root: child root
+   @type root: str
+   @param atom: child atom
+   @type atom: Atom
+   @rtype: Package
+   @return: child package to replace, or None
+   """
+   if parent.root != root or parent.cp != atom.cp:
+   return None
+   for child in 

[gentoo-commits] proj/portage:master commit in: lib/portage/dep/

2020-11-28 Thread Zac Medico
commit: b59e2545a88dccf70e546d49e08a5a99110dd628
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Nov 28 01:04:07 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Nov 28 03:13:02 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=b59e2545

backtracking: fix virtual choices for circular deps (bug 757306)

Fix virtual choices to be consistent with circular dependency
backtracking choices.

Fixes: f78a91e44e3e ("backtracking: adjust || preference to break dependency 
cycles")
Bug: https://bugs.gentoo.org/757306
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dep/dep_check.py | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/lib/portage/dep/dep_check.py b/lib/portage/dep/dep_check.py
index 60c8ec6d0..b89d5d651 100644
--- a/lib/portage/dep/dep_check.py
+++ b/lib/portage/dep/dep_check.py
@@ -356,6 +356,7 @@ def dep_zapdeps(unreduced, reduced, myroot, use_binaries=0, 
trees=None,
 
# Alias the trees we'll be checking availability against
parent   = trees[myroot].get("parent")
+   virt_parent = trees[myroot].get("virt_parent")
priority = trees[myroot].get("priority")
graph_db = trees[myroot].get("graph_db")
graph= trees[myroot].get("graph")
@@ -596,8 +597,10 @@ def dep_zapdeps(unreduced, reduced, myroot, 
use_binaries=0, trees=None,
if 
match_from_list(atom, cpv_slot_list):
circular_atom = 
atom
break
-   else:
-   for circular_child in 
circular_dependency.get(parent, []):
+   if circular_atom is None and 
circular_dependency is not None:
+   for circular_child in itertools.chain(
+   
circular_dependency.get(parent, []),
+   
circular_dependency.get(virt_parent, [])):
for atom in 
atoms:
if not 
atom.blocker and atom.match(circular_child):

circular_atom = atom



[gentoo-commits] proj/portage:master commit in: lib/portage/util/

2020-11-22 Thread Zac Medico
commit: ae8b18f868c9bd039643f89f28f9d92ce8966c3c
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Nov 22 19:37:05 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov 22 19:38:51 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=ae8b18f8

compare_files: fix missing xattr handling

Fixes: a2e7bf7d1c7d ("compare_files: handle missing xattr support")
Bug: https://bugs.gentoo.org/755950
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/_compare_files.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/util/_compare_files.py 
b/lib/portage/util/_compare_files.py
index 60d43aefa..7692797fc 100644
--- a/lib/portage/util/_compare_files.py
+++ b/lib/portage/util/_compare_files.py
@@ -9,7 +9,7 @@ import stat
 
 from portage import _encodings
 from portage import _unicode_encode
-from portage.util._xattr import xattr
+from portage.util._xattr import XATTRS_WORKS, xattr
 
 def compare_files(file1, file2, skipped_types=()):
"""
@@ -45,7 +45,7 @@ def compare_files(file1, file2, skipped_types=()):
if "device_number" not in skipped_types and file1_stat.st_rdev != 
file2_stat.st_rdev:
differences.append("device_number")
 
-   if (xattr.XATTRS_WORKS and "xattr" not in skipped_types and
+   if (XATTRS_WORKS and "xattr" not in skipped_types and
sorted(xattr.get_all(file1, nofollow=True)) != 
sorted(xattr.get_all(file2, nofollow=True))):
differences.append("xattr")
 



[gentoo-commits] proj/portage:master commit in: /

2020-11-21 Thread Zac Medico
commit: 32889cc8ae09e19d65f9a2b7e1f196dd5a310c07
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Nov 22 06:32:36 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov 22 06:32:46 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=32889cc8

Updates for portage-3.0.10 release

Signed-off-by: Zac Medico  gentoo.org>

 RELEASE-NOTES | 7 +++
 setup.py  | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index 9a5de85dd..61c3c3d16 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,13 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.10
+==
+* Bug Fixes:
+- Bug 753497 profiles should not inherit deprecated messages
+- Bug 754903 minimize use of installed packages to break cycles
+- Bug 755950 compare_files: handle missing xattr support
+
 portage-3.0.9
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index 869cf9992..443b94422 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.9',
+   version = '3.0.10',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/portage/util/

2020-11-21 Thread Zac Medico
commit: a2e7bf7d1c7d518fe8d7cf2c0e6cb30020b8aa94
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Nov 21 21:13:27 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov 22 06:17:29 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=a2e7bf7d

compare_files: handle missing xattr support

Bug: https://bugs.gentoo.org/755950
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/_compare_files.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/portage/util/_compare_files.py 
b/lib/portage/util/_compare_files.py
index de97a9d9d..60d43aefa 100644
--- a/lib/portage/util/_compare_files.py
+++ b/lib/portage/util/_compare_files.py
@@ -45,7 +45,8 @@ def compare_files(file1, file2, skipped_types=()):
if "device_number" not in skipped_types and file1_stat.st_rdev != 
file2_stat.st_rdev:
differences.append("device_number")
 
-   if "xattr" not in skipped_types and sorted(xattr.get_all(file1, 
nofollow=True)) != sorted(xattr.get_all(file2, nofollow=True)):
+   if (xattr.XATTRS_WORKS and "xattr" not in skipped_types and
+   sorted(xattr.get_all(file1, nofollow=True)) != 
sorted(xattr.get_all(file2, nofollow=True))):
differences.append("xattr")
 
if "atime" not in skipped_types and file1_stat.st_atime_ns != 
file2_stat.st_atime_ns:



[gentoo-commits] proj/portage:master commit in: lib/_emerge/, lib/portage/tests/resolver/

2020-11-21 Thread Zac Medico
commit: 5095c2023595a75e2848f1ad3dbe25b5fb451a44
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Nov 16 05:55:54 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov 22 03:19:29 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=5095c202

find_smallest_cycle: enhance search prioritization

Enhance the find_smallest_cycle function to prioritize its
search so that it will minimize the use of installed packages
to break cycles. When installed packages must be used to
break cycles, it will now prefer to do this for runtime
dependencies over buildtime dependencies, since it's
preferable to build against latest versions of buildtime
dependencies whenever possible. This should solve some cases
of bug 199856 which have been triggered by unsafe reliance
on installed packages to break cycles.

The included unit test case demonstrates correct merge order
for a dependency calculation involving 6 independent cycles.
This test case fails with the master branch, due to a buildtime
dependency cycle of 3 packages being merged earlier than cycles
of 2 packages. We can generalize this to say that the master
branch may use an installed package to break an arbitrarily
sized cycle in a somewhat random location, even though that
cycle may be composed of smaller independent cycles which
would be safer to break individually.

Bug: https://bugs.gentoo.org/754903
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/DepPriorityNormalRange.py  |  2 +
 lib/_emerge/DepPrioritySatisfiedRange.py   | 53 --
 lib/_emerge/depgraph.py| 43 -
 lib/portage/tests/resolver/test_merge_order.py | 10 +
 4 files changed, 70 insertions(+), 38 deletions(-)

diff --git a/lib/_emerge/DepPriorityNormalRange.py 
b/lib/_emerge/DepPriorityNormalRange.py
index 5f3f3da70..10f205a3b 100644
--- a/lib/_emerge/DepPriorityNormalRange.py
+++ b/lib/_emerge/DepPriorityNormalRange.py
@@ -14,6 +14,7 @@ class DepPriorityNormalRange:
"""
MEDIUM  = 3
MEDIUM_SOFT = 2
+   MEDIUM_POST = 2
SOFT= 1
NONE= 0
 
@@ -37,6 +38,7 @@ class DepPriorityNormalRange:
 
ignore_medium  = _ignore_runtime
ignore_medium_soft = _ignore_runtime_post
+   ignore_medium_post = _ignore_runtime_post
ignore_soft= _ignore_optional
 
 DepPriorityNormalRange.ignore_priority = (

diff --git a/lib/_emerge/DepPrioritySatisfiedRange.py 
b/lib/_emerge/DepPrioritySatisfiedRange.py
index e056e676f..fb0d7db4e 100644
--- a/lib/_emerge/DepPrioritySatisfiedRange.py
+++ b/lib/_emerge/DepPrioritySatisfiedRange.py
@@ -8,17 +8,18 @@ class DepPrioritySatisfiedRange:
 
not satisfied and buildtimeHARD
not satisfied and runtime  7   MEDIUM
-   not satisfied and runtime_post 6   MEDIUM_SOFT
-   satisfied and buildtime_slot_op5   SOFT
-   satisfied and buildtime4   SOFT
-   satisfied and runtime  3   SOFT
-   satisfied and runtime_post 2   SOFT
+   satisfied and buildtime_slot_op6   MEDIUM_SOFT
+   satisfied and buildtime5   MEDIUM_SOFT
+   satisfied and runtime  4   MEDIUM_SOFT
+   runtime_post   3   MEDIUM_POST
+   satisfied and runtime_post 2   MEDIUM_POST
optional   1   SOFT
(none of the above)0   NONE
"""
MEDIUM  = 7
MEDIUM_SOFT = 6
-   SOFT= 5
+   MEDIUM_POST = 3
+   SOFT= 1
NONE= 0
 
@classmethod
@@ -35,42 +36,51 @@ class DepPrioritySatisfiedRange:
return True
if not priority.satisfied:
return False
+   if priority.buildtime or priority.runtime:
+   return False
return bool(priority.runtime_post)
 
@classmethod
-   def _ignore_satisfied_runtime(cls, priority):
+   def _ignore_runtime_post(cls, priority):
if priority.__class__ is not DepPriority:
return False
if priority.optional:
return True
-   if not priority.satisfied:
+   if priority.buildtime or priority.runtime:
return False
-   return not priority.buildtime
+   return bool(priority.runtime_post)
 
@classmethod
-   def _ignore_satisfied_buildtime(cls, priority):
+   def _ignore_satisfied_runtime(cls, priority):
if priority.__class__ is not DepPriority:
return False
if priority.optional:
return True
-   if priority.buildtime_slot_op:
+ 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/, lib/portage/package/ebuild/_config/

2020-11-21 Thread Zac Medico
commit: 309e28b2da9a7e98039b45e7555833fe6a4bc2f9
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Nov  9 02:41:36 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov 22 00:45:46 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=309e28b2

emerge: Disable profile deprecation warning inheritance (bug 753497)

According to PMS, a deprecated profile warning is not inherited. Since
the current profile node may have been inherited by a user profile
node, the deprecation warning may be relevant even if it is not a
top-level profile node. Therefore, consider the deprecated warning
to be irrelevant when the current profile node belongs to the same
repo as the previous profile node.

Bug: https://bugs.gentoo.org/753497
Signed-off-by: Zac Medico  gentoo.org>

 .../package/ebuild/_config/LocationsManager.py | 30 --
 .../package/ebuild/deprecated_profile_check.py |  9 ---
 2 files changed, 27 insertions(+), 12 deletions(-)

diff --git a/lib/portage/package/ebuild/_config/LocationsManager.py 
b/lib/portage/package/ebuild/_config/LocationsManager.py
index b90b9227c..327400ad6 100644
--- a/lib/portage/package/ebuild/_config/LocationsManager.py
+++ b/lib/portage/package/ebuild/_config/LocationsManager.py
@@ -30,7 +30,9 @@ _PORTAGE1_DIRECTORIES = frozenset([
 
 _profile_node = collections.namedtuple('_profile_node',
('location', 'portage1_directories', 'user_config',
-   'profile_formats', 'eapi', 'allow_build_id'))
+   'profile_formats', 'eapi', 'allow_build_id',
+   'show_deprecated_warning',
+))
 
 _allow_parent_colon = frozenset(
["portage-2"])
@@ -132,7 +134,7 @@ class LocationsManager:
if self.profile_path:
try:

self._addProfile(os.path.realpath(self.profile_path),
-   repositories, known_repos)
+   repositories, known_repos, ())
except ParseError as e:
if not portage._sync_mode:
writemsg(_("!!! Unable to parse 
profile: '%s'\n") % self.profile_path, noiselevel=-1)
@@ -154,7 +156,9 @@ class LocationsManager:
('profile-bashrcs', 'profile-set'),
read_corresponding_eapi_file(
custom_prof + os.sep, default=None),
-   True))
+   True,
+   show_deprecated_warning=False,
+   ))
del custom_prof
 
self.profiles = tuple(self.profiles)
@@ -167,7 +171,7 @@ class LocationsManager:
noiselevel=-1)
raise DirectoryNotFound(var)
 
-   def _addProfile(self, currentPath, repositories, known_repos):
+   def _addProfile(self, currentPath, repositories, known_repos, 
previous_repos):
current_abs_path = os.path.abspath(currentPath)
allow_directories = True
allow_parent_colon = True
@@ -176,8 +180,8 @@ class LocationsManager:
current_formats = ()
eapi = None
 
-   intersecting_repos = [x for x in known_repos
-   if current_abs_path.startswith(x[0])]
+   intersecting_repos = tuple(x for x in known_repos
+   if current_abs_path.startswith(x[0]))
if intersecting_repos:
# Handle nested repositories. The longest path
# will be the correct one.
@@ -214,6 +218,14 @@ class LocationsManager:
for x in layout_data['profile-formats'])
current_formats = tuple(layout_data['profile-formats'])
 
+   # According to PMS, a deprecated profile warning is not 
inherited. Since
+   # the current profile node may have been inherited by a user 
profile
+   # node, the deprecation warning may be relevant even if it is 
not a
+   # top-level profile node. Therefore, consider the deprecated 
warning
+   # to be irrelevant when the current profile node belongs to the 
same
+   # repo as the previous profile node.
+   show_deprecated_warning = \
+   tuple(x[0] for x in previous_repos) != tuple(x[0] for x 
in intersecting_repos)
 
if compat_mode:
offenders = 
_PORTAGE1_DIRECTORIES.intersection(os.listdir(currentPath))
@@ -256,7 +268,7 @@ class LocationsManager:
parentPath = 
os.path.realpath(parentPath)
 
if exists_raise_eaccess(parentPath):
-   

[gentoo-commits] proj/portage:master commit in: man/, lib/_emerge/, lib/portage/util/, lib/portage/package/ebuild/_config/, ...

2020-11-21 Thread Zac Medico
commit: 222adeecbd72f070eaa05e39b0951cabd6ba8026
Author: Petr Šabata  redhat  com>
AuthorDate: Sat Nov  7 11:55:46 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov 22 00:35:49 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=222adeec

Typofix: Use just one definite article

I first noticed this in make.conf(5) but it turned out there were many
more occurences of these; fixed with find & sed.

Closes: https://github.com/gentoo/portage/pull/634
Signed-off-by: Petr Šabata  redhat.com>
Signed-off-by: Zac Medico  gentoo.org>

 doc/package/ebuild/eapi/4.docbook | 2 +-
 lib/_emerge/depgraph.py   | 2 +-
 lib/portage/cache/template.py | 2 +-
 lib/portage/package/ebuild/_config/KeywordsManager.py | 2 +-
 lib/portage/package/ebuild/_config/UseManager.py  | 4 ++--
 lib/portage/package/ebuild/config.py  | 2 +-
 lib/portage/util/netlink.py   | 2 +-
 man/make.conf.5   | 2 +-
 8 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/doc/package/ebuild/eapi/4.docbook 
b/doc/package/ebuild/eapi/4.docbook
index 8dd0f1487..9120c0ff0 100644
--- a/doc/package/ebuild/eapi/4.docbook
+++ b/doc/package/ebuild/eapi/4.docbook
@@ -153,7 +153,7 @@ This new REQUIRED_USE metadata key is used to specify what 
USE flag combinations
 It's a semi common occurrence that an ebuild may need to state that they 
disallow USE flags in specific combinations- either mysql or sqlite for 
example, but not both.
 
 
-Existing solutions rely on checking the the USE configuration in pkg_setup 
which is non-optimal due to pkg_setup being ran potentially hours after the 
initial emerge -p invocation.
+Existing solutions rely on checking the USE configuration in pkg_setup which 
is non-optimal due to pkg_setup being ran potentially hours after the initial 
emerge -p invocation.
 
 
 Current versions of EAPI4 support a phase hook pkg_pretend that is intended to 
move pre-build checks to just after resolution. It has been proposed that 
pkg_pretend should continue the tradition of adhoc shell code validating the 
USE state- this too is non optimal for the following reasons-

diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index 6aaacfe44..a994caea7 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -10061,7 +10061,7 @@ def _resume_depgraph(settings, trees, mtimedb, myopts, 
myparams, spinner):
 
# If this package was pulled in by a parent
# package scheduled for merge, removing this
-   # package may cause the the parent package's
+   # package may cause the parent package's
# dependency to become unsatisfied.
for parent_node, atom in \

mydepgraph._dynamic_config._parent_atoms.get(pkg, []):

diff --git a/lib/portage/cache/template.py b/lib/portage/cache/template.py
index 31a4acc44..55b8dc40c 100644
--- a/lib/portage/cache/template.py
+++ b/lib/portage/cache/template.py
@@ -293,7 +293,7 @@ def serialize_eclasses(eclass_dict, chf_type='mtime', 
paths=True):
"""takes a dict, returns a string representing said dict"""
"""The "new format", which causes older versions of 

[gentoo-commits] proj/portage:master commit in: lib/portage/tests/resolver/

2020-11-14 Thread Zac Medico
commit: e9c0ef7609e8e2068af8c9c7138f1a2dc22fcfb0
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Nov 15 04:25:22 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov 15 05:54:37 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=e9c0ef76

Add slot operator := BDEPEND unit tests for bug 752153

Test these two distinct cases:

1) Regular dev-lang/go upgrade, with rebuild of packages that
have dev-lang/go:= in BDEPEND.

2) Rebuild of packages that have dev-lang/go:= in BDEPEND, after
the built slot operator deps have already been broken by an
earlier dev-lang/go upgrade.

Bug: https://bugs.gentoo.org/752153
Signed-off-by: Zac Medico  gentoo.org>

 .../tests/resolver/test_slot_operator_bdeps.py | 209 +
 1 file changed, 209 insertions(+)

diff --git a/lib/portage/tests/resolver/test_slot_operator_bdeps.py 
b/lib/portage/tests/resolver/test_slot_operator_bdeps.py
new file mode 100644
index 0..f5b1bce1b
--- /dev/null
+++ b/lib/portage/tests/resolver/test_slot_operator_bdeps.py
@@ -0,0 +1,209 @@
+# Copyright 2020 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+from portage.tests import TestCase
+from portage.tests.resolver.ResolverPlayground import (ResolverPlayground,
+   ResolverPlaygroundTestCase)
+
+class SlotOperatorBdependTestCase(TestCase):
+
+   def testSlotOperatorBdepend(self):
+   """
+   Test regular dev-lang/go upgrade, with rebuild of packages
+   that have dev-lang/go:= in BDEPEND.
+   """
+
+   ebuilds = {
+   "app-emulation/buildah-1.16.1":{
+   "EAPI": "7",
+   "BDEPEND": "dev-lang/go:=",
+   },
+
+   "app-emulation/libpod-2.1.0":{
+   "EAPI": "7",
+   "BDEPEND": "dev-lang/go:=",
+   },
+
+   "dev-lang/go-1.15.5":{
+   "EAPI": "7",
+   "SLOT": "0/1.15.5"
+   },
+
+   "dev-lang/go-1.14.12" : {
+   "EAPI": "7",
+   "SLOT": "0/1.14.12"
+   },
+   }
+
+   binpkgs = {
+   "app-emulation/buildah-1.16.1":{
+   "EAPI": "7",
+   "BDEPEND": "dev-lang/go:0/1.14.12=",
+   },
+   "app-emulation/libpod-2.1.0":{
+   "EAPI": "7",
+   "BDEPEND": "dev-lang/go:0/1.14.12=",
+   },
+   "dev-lang/go-1.14.12" : {
+   "EAPI": "7",
+   "SLOT": "0/1.14.12"
+   },
+   }
+
+   installed = {
+   "app-emulation/buildah-1.16.1":{
+   "EAPI": "7",
+   "BDEPEND": "dev-lang/go:0/1.14.12=",
+   },
+   "app-emulation/libpod-2.1.0":{
+   "EAPI": "7",
+   "BDEPEND": "dev-lang/go:0/1.14.12=",
+   },
+   "dev-lang/go-1.14.12" : {
+   "EAPI": "7",
+   "SLOT": "0/1.14.12"
+   },
+   }
+
+   world = ["app-emulation/buildah", "app-emulation/libpod"]
+
+   test_cases = (
+
+   # Test rebuild triggered by slot operator := dependency 
in BDEPEND.
+   ResolverPlaygroundTestCase(
+   ["@world"],
+   options = {
+   "--update": True,
+   "--deep": True,
+   },
+   success = True,
+   mergelist = ["dev-lang/go-1.15.5", 
"app-emulation/buildah-1.16.1", "app-emulation/libpod-2.1.0"]
+   ),
+
+   # Test the above case with --usepkg --with-bdeps=y. It 
should not use the
+   # binary packages because rebuild is needed.
+   ResolverPlaygroundTestCase(
+   ["@world"],
+   options = {
+   "--usepkg": True,
+   "--with-bdeps": "y",
+   "--update": True,
+   "--deep": True,
+   },
+   success = True,
+   mergelist = ["dev-lang/go-1.15.5", 

[gentoo-commits] proj/portage:master commit in: /

2020-11-01 Thread Zac Medico
commit: 1260f8a32a7dff72b05268c7a8c0cc21cd5d0e28
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Nov  2 01:53:29 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Nov  2 01:53:43 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=1260f8a3

Updates for portage-3.0.9 release

Signed-off-by: Zac Medico  gentoo.org>

 RELEASE-NOTES | 9 +
 setup.py  | 2 +-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index efcba5808..9a5de85dd 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,15 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.9
+==
+* Bug Fixes:
+- Bug 199722 Enable QA Notice for deprecated hasq/useq in *rm phases
+- Bug 704498 Fix pid-sandbox to handle Ctrl+Z correctly
+- Bug 752066 Add emerge --quickpkg-direct-root option
+- Bug 752147 Fix make.conf to expand special *ROOT variables
+- Bug 752153 Add @golang-rebuild package set
+
 portage-3.0.8
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index 1efe8c65b..869cf9992 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.8',
+   version = '3.0.9',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

2020-11-01 Thread Zac Medico
commit: e31c0c929673482101a066541d462d692502a6d1
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Nov  1 20:10:18 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Nov  2 01:30:32 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=e31c0c92

make.conf: expand special *ROOT variables (bug 752147)

Bug: https://bugs.gentoo.org/752147
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/package/ebuild/config.py | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/config.py 
b/lib/portage/package/ebuild/config.py
index a09fdbced..a188c700d 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -401,9 +401,14 @@ class config:
expand_map = env_d.copy()
self._expand_map = expand_map
 
-   # Allow make.globals to set default paths relative to 
${EPREFIX}.
+   # Allow make.globals and make.conf to set paths 
relative to vars like ${EPREFIX}.
+   expand_map["BROOT"] = broot
expand_map["EPREFIX"] = eprefix
+   expand_map["EROOT"] = eroot
+   expand_map["ESYSROOT"] = esysroot
expand_map["PORTAGE_CONFIGROOT"] = config_root
+   expand_map["ROOT"] = target_root
+   expand_map["SYSROOT"] = sysroot
 
if portage._not_installed:
make_globals_path = 
os.path.join(PORTAGE_BASE_PATH, "cnf", "make.globals")



[gentoo-commits] proj/portage:master commit in: man/, lib/_emerge/, lib/portage/tests/emerge/

2020-11-01 Thread Zac Medico
commit: 15ac405fecf3e52ffd93d9a34e472bdc18604a4f
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Nov  1 05:19:02 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov  1 22:25:27 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=15ac405f

emerge: add --quickpkg-direct-root option

Specify the root to use as the --quickpkg-direct package source. This
root is assumed to be immutable during the entire emerge operation.
The default is set to "/".

Bug: https://bugs.gentoo.org/752066
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/actions.py  | 19 ---
 lib/_emerge/depgraph.py | 11 +--
 lib/_emerge/main.py |  5 +
 lib/portage/tests/emerge/test_simple.py |  3 ++-
 man/emerge.1| 10 --
 5 files changed, 40 insertions(+), 8 deletions(-)

diff --git a/lib/_emerge/actions.py b/lib/_emerge/actions.py
index 5e8a46957..239bf6f47 100644
--- a/lib/_emerge/actions.py
+++ b/lib/_emerge/actions.py
@@ -49,7 +49,7 @@ from portage.package.ebuild._ipc.QueryCommand import 
QueryCommand
 from portage.package.ebuild.fetch import _hide_url_passwd
 from portage._sets import load_default_config, SETPREFIX
 from portage._sets.base import InternalPackageSet
-from portage.util import cmp_sort_key, writemsg, varexpand, \
+from portage.util import cmp_sort_key, normalize_path, writemsg, varexpand, \
writemsg_level, writemsg_stdout
 from portage.util.digraph import digraph
 from portage.util.SlotObject import SlotObject
@@ -106,13 +106,26 @@ def action_build(emerge_config, trees=DeprecationWarning,
# before we get here, so warn if they're not (bug #267103).
chk_updated_cfg_files(settings['EROOT'], ['/etc/portage'])
 
+   quickpkg_root = normalize_path(os.path.abspath(
+   emerge_config.opts.get('--quickpkg-direct-root',
+   
emerge_config.running_config.settings['ROOT']))).rstrip(os.path.sep) + 
os.path.sep
quickpkg_direct = ("--usepkg" in emerge_config.opts and
emerge_config.opts.get('--quickpkg-direct', 'n') == 'y' and
-   emerge_config.target_config is not emerge_config.running_config)
+   emerge_config.target_config.settings['ROOT'] != quickpkg_root)
if '--getbinpkg' in emerge_config.opts or quickpkg_direct:
kwargs = {}
if quickpkg_direct:
-   kwargs['add_repos'] = 
(emerge_config.running_config.trees['vartree'].dbapi,)
+   if quickpkg_root == 
emerge_config.running_config.settings['ROOT']:
+   quickpkg_vardb = 
emerge_config.running_config.trees['vartree'].dbapi
+   else:
+   quickpkg_settings = portage.config(
+   
config_root=emerge_config.target_config.settings['PORTAGE_CONFIGROOT'],
+   target_root=quickpkg_root,
+   
env=emerge_config.target_config.settings.backupenv.copy(),
+   
sysroot=emerge_config.target_config.settings['SYSROOT'],
+   
eprefix=emerge_config.target_config.settings['EPREFIX'])
+   quickpkg_vardb = 
portage.vartree(settings=quickpkg_settings).dbapi
+   kwargs['add_repos'] = (quickpkg_vardb,)
 
try:
emerge_config.target_config.trees['bintree'].populate(

diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index 0bb0352e7..6aaacfe44 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -41,7 +41,7 @@ from portage._sets import SETPREFIX
 from portage._sets.base import InternalPackageSet
 from portage.util import ConfigProtect, shlex_split, new_protect_filename
 from portage.util import cmp_sort_key, writemsg, writemsg_stdout
-from portage.util import ensure_dirs
+from portage.util import ensure_dirs, normalize_path
 from portage.util import writemsg_level, write_atomic
 from portage.util.digraph import digraph
 from portage.util.futures import asyncio
@@ -4567,8 +4567,15 @@ class depgraph:
self._dynamic_config._skip_restart = True
return False, myfavorites
 
+   # Since --quickpkg-direct assumes that --quickpkg-direct-root is
+   # immutable, assert that there are no merge or unmerge tasks
+   # for --quickpkg-direct-root.
+   quickpkg_root = normalize_path(os.path.abspath(
+   self._frozen_config.myopts.get('--quickpkg-direct-root',
+   
self._frozen_config._running_root.settings['ROOT']))).rstrip(os.path.sep) + 
os.path.sep
if (self._frozen_config.myopts.get('--quickpkg-direct', 'n') == 
'y' and
-   

[gentoo-commits] proj/portage:master commit in: bin/

2020-11-01 Thread Zac Medico
commit: 8b7edb648814cc53774c5841e45d8cc325bcef6e
Author: Zac Medico  gentoo  org>
AuthorDate: Wed Oct 28 08:34:51 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov  1 21:45:01 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=8b7edb64

pid-sandbox: Forward SIGTSTP and SIGCONT (bug 704498)

For correct operation of Ctrl+Z, forward SIGTSTP and SIGCONT
to all sandboxed pids.

Fixes: 37e4dc5ae842 ("pid-sandbox: pid-ns-init setsid support (bug 675870)")
Bug: https://bugs.gentoo.org/704498
Signed-off-by: Zac Medico  gentoo.org>

 bin/pid-ns-init | 24 +++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/bin/pid-ns-init b/bin/pid-ns-init
index 3a218a5df..e410dd028 100644
--- a/bin/pid-ns-init
+++ b/bin/pid-ns-init
@@ -1,5 +1,5 @@
 #!/usr/bin/env python
-# Copyright 2018-2019 Gentoo Authors
+# Copyright 2018-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import errno
@@ -19,6 +19,11 @@ KILL_SIGNALS = (
signal.SIGHUP,
 )
 
+SIGTSTP_SIGCONT = (
+   signal.SIGTSTP,
+   signal.SIGCONT,
+)
+
 
 def forward_kill_signal(pid, signum, frame):
if pid == 0:
@@ -28,6 +33,18 @@ def forward_kill_signal(pid, signum, frame):
os.kill(pid, signum)
 
 
+def forward_sigtstp_sigcont(pid, signum, frame):
+   handler = None
+   if pid == 0:
+   # Temporarily disable the handler in order to prevent it from
+   # being called recursively, since the signal will also be sent
+   # to the current process.
+   handler = signal.signal(signum, signal.SIG_DFL)
+   os.kill(pid, signum)
+   if handler is not None:
+   signal.signal(signum, handler)
+
+
 def preexec_fn(uid, gid, groups, umask):
if gid is not None:
os.setgid(gid)
@@ -97,6 +114,11 @@ def main(argv):
for signum in KILL_SIGNALS:
signal.signal(signum, sig_handler)
 
+   # For correct operation of Ctrl+Z, forward SIGTSTP and SIGCONT.
+   sigtstp_sigcont_handler = functools.partial(forward_sigtstp_sigcont, 0 
if setsid else main_child_pid)
+   for signum in SIGTSTP_SIGCONT:
+   signal.signal(signum, sigtstp_sigcont_handler)
+
# wait for child processes
while True:
try:



[gentoo-commits] proj/portage:master commit in: cnf/sets/

2020-11-01 Thread Zac Medico
commit: 075c1951e1ac84e99a2219ff14be4a366d274f36
Author: Georgy Yakovlev  gentoo  org>
AuthorDate: Fri Oct 16 19:43:54 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Nov  1 21:11:09 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=075c1951

cnf/sets/portage.conf: add new sets for go rebuilding go packages

go-built binaries may contain security
vulnerabilities if a binary built with vulnerable compiler.
go is known to embed vulnerable code to all binaries it builds, if
vulnerability was present in the compiler or one of standard libraries.

This commit adds `golang-rebuild` set, which allows easy
rebuild of most go-compiled system packages.

simple 'emerge @golang-rebuild' should rebuild everything affected.
a prompt to run this command can be added to postinst message in
dev-lang/go ebuild.

Closes: https://github.com/gentoo/portage/pull/630
Bug: https://bugs.gentoo.org/752153
Signed-off-by: Georgy Yakovlev  gentoo.org>
Signed-off-by: Zac Medico  gentoo.org>

 cnf/sets/portage.conf | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/cnf/sets/portage.conf b/cnf/sets/portage.conf
index 0d11d7891..22f0fa3a5 100644
--- a/cnf/sets/portage.conf
+++ b/cnf/sets/portage.conf
@@ -103,3 +103,9 @@ class = portage.sets.dbapi.UnavailableBinaries
 # to the matching portdb entry.
 [changed-deps]
 class = portage.sets.dbapi.ChangedDepsSet
+
+# Installed packages that inherit from known go related eclasses.
+[golang-rebuild]
+class = portage.sets.dbapi.VariableSet
+variable = INHERITED
+includes = golang-base golang-build golang-vcs golang-vcs-snapshot go-module



[gentoo-commits] proj/portage:master commit in: bin/

2020-10-18 Thread Ulrich Müller
commit: c2f413d6a1b820854681847870f94ff08ac35c2a
Author: Ulrich Müller  gentoo  org>
AuthorDate: Sun Oct 18 09:32:26 2020 +
Commit: Ulrich Müller  gentoo  org>
CommitDate: Sun Oct 18 09:32:26 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=c2f413d6

Revert "Skip QA Notice for hasq/useq during *rm phases."

Almost a decade later, the reason for this is no longer valid.
This reverts commit 6620e74643e0baf040488557b1fff7273654a44d.

Bug: https://bugs.gentoo.org/199722#c8
Acked-by: Zac Medico  gentoo.org>
Signed-off-by: Ulrich Müller  gentoo.org>

 bin/isolated-functions.sh | 3 +--
 bin/phase-helpers.sh  | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/bin/isolated-functions.sh b/bin/isolated-functions.sh
index fde684013..ba224b96f 100644
--- a/bin/isolated-functions.sh
+++ b/bin/isolated-functions.sh
@@ -475,8 +475,7 @@ ___parallel_xargs() {
 }
 
 hasq() {
-   has $EBUILD_PHASE prerm postrm || eqawarn \
-   "QA Notice: The 'hasq' function is deprecated (replaced by 
'has')"
+   eqawarn "QA Notice: The 'hasq' function is deprecated (replaced by 
'has')"
has "$@"
 }
 

diff --git a/bin/phase-helpers.sh b/bin/phase-helpers.sh
index 9495465f9..e7b0a4171 100644
--- a/bin/phase-helpers.sh
+++ b/bin/phase-helpers.sh
@@ -190,8 +190,7 @@ dostrip() {
 }
 
 useq() {
-   has $EBUILD_PHASE prerm postrm || eqawarn \
-   "QA Notice: The 'useq' function is deprecated (replaced by 
'use')"
+   eqawarn "QA Notice: The 'useq' function is deprecated (replaced by 
'use')"
use ${1}
 }
 



[gentoo-commits] proj/portage:master commit in: repoman/

2020-10-18 Thread Zac Medico
commit: d8affbd6c2a75c3acd8ecc5dbbad9224ecb43f94
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Oct 18 08:21:36 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Oct 18 08:21:44 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=d8affbd6

repoman: Update for a repoman-3.0.2 release

Signed-off-by: Zac Medico  gentoo.org>

 repoman/RELEASE-NOTES | 5 +
 repoman/setup.py  | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/repoman/RELEASE-NOTES b/repoman/RELEASE-NOTES
index f0b50cef6..a09845b79 100644
--- a/repoman/RELEASE-NOTES
+++ b/repoman/RELEASE-NOTES
@@ -1,6 +1,11 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+repoman-3.0.2
+==
+* Bug Fixes:
+- Bug 748144 Make file.size fatal for files larger than 20 KiB
+
 repoman-3.0.1
 ==
 * Bug Fixes:

diff --git a/repoman/setup.py b/repoman/setup.py
index b2c5fd43f..a405e7816 100755
--- a/repoman/setup.py
+++ b/repoman/setup.py
@@ -448,7 +448,7 @@ def get_manpages():
 
 setup(
name = 'repoman',
-   version = '3.0.1',
+   version = '3.0.2',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: repoman/lib/repoman/modules/scan/fetch/, repoman/cnf/qa_data/, repoman/man/, ...

2020-10-18 Thread Zac Medico
commit: 6258c62c84bf60b8ade8b7fb63ed4475c6195f21
Author: Joonas Niilola  gentoo  org>
AuthorDate: Sun Oct 18 06:20:08 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Oct 18 08:16:12 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=6258c62c

repoman: fix file.size-fatal to be 20 KiB

Closes: https://github.com/gentoo/portage/pull/629
Bug: https://bugs.gentoo.org/748144
Signed-off-by: Joonas Niilola  gentoo.org>
Signed-off-by: Zac Medico  gentoo.org>

 repoman/cnf/qa_data/qa_data.yaml  | 1 -
 repoman/cnf/repository/qa_data.yaml   | 1 -
 repoman/lib/repoman/modules/scan/fetch/fetches.py | 7 +--
 repoman/man/repoman.1 | 2 +-
 4 files changed, 2 insertions(+), 9 deletions(-)

diff --git a/repoman/cnf/qa_data/qa_data.yaml b/repoman/cnf/qa_data/qa_data.yaml
index 9a807aaf3..29a3d6e9f 100644
--- a/repoman/cnf/qa_data/qa_data.yaml
+++ b/repoman/cnf/qa_data/qa_data.yaml
@@ -60,7 +60,6 @@ qahelp:
 file:
 executable: "Ebuilds, digests, metadata.xml, Manifest, and ChangeLog 
do not need the executable bit"
 size: "Files in the files directory must be under 20 KiB"
-size-fatal: "Files in the files directory must be under 60 KiB"
 empty: "Empty file in the files directory"
 name: "File/dir name must be composed of only the following chars: %s "
 UTF8: "File is not UTF8 compliant"

diff --git a/repoman/cnf/repository/qa_data.yaml 
b/repoman/cnf/repository/qa_data.yaml
index 464482056..3fe6b53d5 100644
--- a/repoman/cnf/repository/qa_data.yaml
+++ b/repoman/cnf/repository/qa_data.yaml
@@ -59,7 +59,6 @@ qawarnings:
 - ebuild.badheader
 - ebuild.patches
 - file.empty
-- file.size
 - HOMEPAGE.virtual
 - inherit.unused
 - inherit.deprecated

diff --git a/repoman/lib/repoman/modules/scan/fetch/fetches.py 
b/repoman/lib/repoman/modules/scan/fetch/fetches.py
index 5a958a461..7d4c58908 100644
--- a/repoman/lib/repoman/modules/scan/fetch/fetches.py
+++ b/repoman/lib/repoman/modules/scan/fetch/fetches.py
@@ -120,12 +120,7 @@ class FetchChecks(ScanBase):
continue
filesdirlist.append(y + "/" + z)
# Current policy is no files over 20 KiB, these 
are the checks.
-   # File size between 20 KiB and 60 KiB causes a 
warning,
-   # while file size over 60 KiB causes an error.
-   elif mystat.st_size > 61440:
-   self.qatracker.add_error(
-   "file.size-fatal", "(%d KiB) 
%s/files/%s" % (
-   mystat.st_size // 1024, 
xpkg, y))
+   # File size over 20 KiB causes an error.
elif mystat.st_size > 20480:
self.qatracker.add_error(
"file.size", "(%d KiB) 
%s/files/%s" % (

diff --git a/repoman/man/repoman.1 b/repoman/man/repoman.1
index 6f9a24544..0926e806c 100644
--- a/repoman/man/repoman.1
+++ b/repoman/man/repoman.1
@@ -393,7 +393,7 @@ executable bit
 File/dir name must be composed of only the following chars: a-zA-Z0-9._-+:
 .TP
 .B file.size
-Files in the files directory must be under 20k
+Files in the files directory must be under 20 KiB
 .TP
 .B inherit.missing
 Ebuild uses functions from an eclass but does not inherit it



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/emerge/

2020-10-17 Thread Zac Medico
commit: ac712bb826266719c39a797499c10ec512c5d4f9
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Oct 17 09:19:32 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Oct 17 09:19:43 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=ac712bb8

lib/portage/tests/emerge/test_simple.py: drop unused-import

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/emerge/test_simple.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/lib/portage/tests/emerge/test_simple.py 
b/lib/portage/tests/emerge/test_simple.py
index 25cb54de3..8ba74c609 100644
--- a/lib/portage/tests/emerge/test_simple.py
+++ b/lib/portage/tests/emerge/test_simple.py
@@ -2,11 +2,9 @@
 # Distributed under the terms of the GNU General Public License v2
 
 import subprocess
-import sys
 
 import portage
 from portage import shutil, os
-from portage import _unicode_decode
 from portage.const import (BASH_BINARY, BINREPOS_CONF_FILE, PORTAGE_PYM_PATH, 
USER_CONFIG_PATH)
 from portage.cache.mappings import Mapping
 from portage.process import find_binary



[gentoo-commits] proj/portage:master commit in: cnf/repo.postsync.d/

2020-10-14 Thread Zac Medico
commit: 21e996f9d25ef578b4f4fc4f96c336383136c66f
Author: Zac Medico  gentoo  org>
AuthorDate: Wed Oct 14 22:52:28 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Wed Oct 14 22:59:37 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=21e996f9

cnf/repo.postsync.d/example: Detect metadata/pkg_desc_index in repo

Detect when the repo includes a metadata/pkg_desc_index file and in
that case skip the call to egencache --update-pkg-desc-index.

Signed-off-by: Zac Medico  gentoo.org>

 cnf/repo.postsync.d/example | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/cnf/repo.postsync.d/example b/cnf/repo.postsync.d/example
index f7c6f5092..72d91a60e 100644
--- a/cnf/repo.postsync.d/example
+++ b/cnf/repo.postsync.d/example
@@ -45,7 +45,8 @@ if [ -n "${repository_name}" ]; then
ret=1
fi
fi
-
+fi
+if [ -n "${repository_name}" ] && ! [ -e 
"${repository_path}/metadata/pkg_desc_index" ]; then
# Regenerate the metadata/pkg_desc_index file. This is not
# needed for https://gitweb.gentoo.org/repo/sync/gentoo.git which
# provides a freshly generated copy. The --external-cache-only



[gentoo-commits] proj/portage:master commit in: lib/portage/tests/emerge/

2020-10-12 Thread Zac Medico
commit: 3cd49702014c6641190c858744ce4ea4a047366f
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Oct 12 17:57:12 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Oct 12 18:00:05 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=3cd49702

SimpleEmergeTestCase: fix TypeError: write() argument must be str, not int

Bug: https://bugs.gentoo.org/748012
Fixes: 0552665627bf ("SimpleEmergeTestCase: run tests in coroutine method")
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/tests/emerge/test_simple.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/lib/portage/tests/emerge/test_simple.py 
b/lib/portage/tests/emerge/test_simple.py
index 8635b70e4..25cb54de3 100644
--- a/lib/portage/tests/emerge/test_simple.py
+++ b/lib/portage/tests/emerge/test_simple.py
@@ -558,8 +558,7 @@ move dev-util/git dev-vcs/git
output, _err = yield proc.communicate()
yield proc.wait()
if proc.returncode != os.EX_OK:
-   for line in output:
-   
sys.stderr.write(_unicode_decode(line))
+   portage.writemsg(output)
 
self.assertEqual(os.EX_OK, proc.returncode,
"emerge failed with args %s" % (args,))



[gentoo-commits] proj/portage:master commit in: lib/portage/util/

2020-10-05 Thread Zac Medico
commit: c446394e5b0e71a611ed5cfa0fb923e05802ef5f
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Oct  6 01:21:58 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Oct  6 01:35:09 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=c446394e

movefile: remove dest_tmp_bytes on failure

Reported-by: Boleyn Su  gmail.com>
Bug: https://bugs.gentoo.org/745588
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/movefile.py | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/portage/util/movefile.py b/lib/portage/util/movefile.py
index a251d369d..9c6054209 100644
--- a/lib/portage/util/movefile.py
+++ b/lib/portage/util/movefile.py
@@ -262,6 +262,7 @@ def movefile(src, dest, newmtime=None, sstat=None, 
mysettings=None,
dest_tmp = dest + "#new"
dest_tmp_bytes = _unicode_encode(dest_tmp, 
encoding=encoding,
errors='strict')
+   success = False
try: # For safety copy then move it over.
_copyfile(src_bytes, dest_tmp_bytes)
_apply_stat(sstat, dest_tmp_bytes)
@@ -281,13 +282,18 @@ def movefile(src, dest, newmtime=None, sstat=None, 
mysettings=None,
raise
_rename(dest_tmp_bytes, dest_bytes)
_os.unlink(src_bytes)
-   except SystemExit as e:
-   raise
+   success = True
except Exception as e:
writemsg("!!! %s\n" % _('copy %(src)s -> 
%(dest)s failed.') %
{"src": src, "dest": dest}, 
noiselevel=-1)
writemsg("!!! %s\n" % (e,), noiselevel=-1)
return None
+   finally:
+   if not success:
+   try:
+   _os.unlink(dest_tmp_bytes)
+   except OSError:
+   pass
else:
#we don't yet handle special, so we need to fall back 
to /bin/mv
a = spawn([MOVE_BINARY, '-f', src, dest], 
env=os.environ)



[gentoo-commits] proj/portage:master commit in: /

2020-09-21 Thread Zac Medico
commit: 8c779546ab998932016e669f614e3da199a8db3b
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Sep 20 22:45:54 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep 21 05:21:54 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=8c779546

Updates for portage-3.0.8 release

Signed-off-by: Zac Medico  gentoo.org>

 RELEASE-NOTES | 11 +++
 setup.py  |  2 +-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index a594d7d4f..efcba5808 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,17 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.8
+==
+* Bug Fixes:
+- Bug 404157 emerge: --keep-going support pkg_pretend failures
+- Bug 644246 Add mercurial sync support
+- Bug 710432 emerge: Enable parallel-fetch during pkg_pretend
+- Bug 743115 emerge: Fix slot conflict backtracking to group similar
+  missed updates into a single backtracking try
+- Bug 743631 emerge: Fix slot conflict backtracking to prefer
+  choices that satisfy all dependencies
+
 portage-3.0.7
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index 681fed079..1efe8c65b 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.7',
+   version = '3.0.8',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/_emerge/

2020-09-20 Thread Zac Medico
commit: c7fa3f1eb1ce1ebc0d1219dacba555e1a29d5f22
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Sep 20 00:32:57 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep 21 05:17:24 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=c7fa3f1e

emerge: enable parallel-fetch during pkg_pretend (bug 710432)

Execute pkg_pretend phases in a coroutine while parallel-fetch
is running concurrently. When it's time to execute the pkg_pretend
phase for a remote binary package, use a Scheduler _get_prefetcher
method to get a running prefetcher if available, and otherwise
start a new fetcher.

Since pkg_pretend phases now run inside of the --keep-going retry
loop, --keep-going is now able to recover from pkg_pretend
failures, which fixes bug 404157.

Bug: https://bugs.gentoo.org/404157
Bug: https://bugs.gentoo.org/710432
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/Scheduler.py | 142 +--
 1 file changed, 99 insertions(+), 43 deletions(-)

diff --git a/lib/_emerge/Scheduler.py b/lib/_emerge/Scheduler.py
index a69421288..465f928a0 100644
--- a/lib/_emerge/Scheduler.py
+++ b/lib/_emerge/Scheduler.py
@@ -25,6 +25,7 @@ from portage._sets import SETPREFIX
 from portage._sets.base import InternalPackageSet
 from portage.util import ensure_dirs, writemsg, writemsg_level
 from portage.util.futures import asyncio
+from portage.util.futures.compat_coroutine import coroutine, coroutine_return
 from portage.util.SlotObject import SlotObject
 from portage.util._async.SchedulerInterface import SchedulerInterface
 from portage.package.ebuild.digestcheck import digestcheck
@@ -766,7 +767,8 @@ class Scheduler(PollScheduler):
 
return prefetcher
 
-   def _run_pkg_pretend(self):
+   @coroutine
+   def _run_pkg_pretend(self, loop=None):
"""
Since pkg_pretend output may be important, this method sends all
output directly to stdout (regardless of options like --quiet or
@@ -774,7 +776,7 @@ class Scheduler(PollScheduler):
"""
 
failures = 0
-   sched_iface = self._sched_iface
+   sched_iface = loop = asyncio._wrap_loop(loop or 
self._sched_iface)
 
for x in self._mergelist:
if not isinstance(x, Package):
@@ -789,18 +791,28 @@ class Scheduler(PollScheduler):
if "pretend" not in x.defined_phases:
continue
 
-   out_str =">>> Running pre-merge checks for " + 
colorize("INFORM", x.cpv) + "\n"
-   portage.util.writemsg_stdout(out_str, noiselevel=-1)
+   out_str = "Running pre-merge checks for " + 
colorize("INFORM", x.cpv)
+   self._status_msg(out_str)
 
root_config = x.root_config
-   settings = self.pkgsettings[root_config.root]
+   settings = self._allocate_config(root_config.root)
settings.setcpv(x)
+   if not x.built:
+   # Get required SRC_URI metadata (it's not 
cached in x.metadata
+   # because some packages have an extremely large 
SRC_URI value).
+   portdb = root_config.trees["porttree"].dbapi
+   (settings.configdict["pkg"]["SRC_URI"],) = 
yield portdb.async_aux_get(
+   x.cpv, ["SRC_URI"], myrepo=x.repo, 
loop=loop
+   )
 
# setcpv/package.env allows for per-package 
PORTAGE_TMPDIR so we
# have to validate it for each package
rval = _check_temp_dir(settings)
if rval != os.EX_OK:
-   return rval
+   failures += 1
+   self._record_pkg_failure(x, settings, FAILURE)
+   self._deallocate_config(settings)
+   continue
 
build_dir_path = os.path.join(
os.path.realpath(settings["PORTAGE_TMPDIR"]),
@@ -809,7 +821,7 @@ class Scheduler(PollScheduler):
settings["PORTAGE_BUILDDIR"] = build_dir_path
build_dir = EbuildBuildDir(scheduler=sched_iface,
settings=settings)
-   sched_iface.run_until_complete(build_dir.async_lock())
+   yield build_dir.async_lock()
current_task = None
 
try:
@@ -835,7 +847,7 @@ class Scheduler(PollScheduler):
phase='clean', 
scheduler=sched_iface, settings=settings)
current_task = clean_phase
  

[gentoo-commits] proj/portage:master commit in: lib/_emerge/resolver/, lib/_emerge/, lib/portage/tests/resolver/

2020-09-20 Thread Zac Medico
commit: 9afc5d5a808ca66173b1d803469cd44bc24e99be
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Sep 19 23:18:16 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Sep 20 22:59:06 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=9afc5d5a

_slot_confict_backtrack: group similar missed updates (bug 743115)

When a slot conflict occurs due to a missed update, and some other
similar update(s) are available, add the similar update(s) to the
runtime package mask for the same backtracking choice. This reduces
minimum number of backtrack tries required to solve the test case
for bug 743115 from 7 to 4, where the difference of 3 corresponds
to the number of other similar setuptools updates available.

Bug: https://bugs.gentoo.org/743115
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/depgraph.py| 25 ++
 lib/_emerge/resolver/backtracking.py   |  7 +++---
 .../resolver/test_slot_operator_missed_update.py   |  2 +-
 3 files changed, 26 insertions(+), 8 deletions(-)

diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index 7281d8692..0bb0352e7 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -1795,15 +1795,32 @@ class depgraph:

self._dynamic_config._parent_atoms.get(to_be_masked, set())
conflict_atoms = set(parent_atom for parent_atom in 
all_parents \
if parent_atom not in parent_atoms)
-   backtrack_data.append((to_be_masked, conflict_atoms))
+
+   similar_pkgs = []
+   if conflict_atoms:
+   # If the conflict has been triggered by a 
missed update, then
+   # we can avoid excessive backtracking if we 
detect similar missed
+   # updates and mask them as part of the same 
backtracking choice.
+   for similar_pkg in 
self._iter_similar_available(to_be_masked, slot_atom):
+   if similar_pkg in conflict_pkgs:
+   continue
+   similar_conflict_atoms = []
+   for parent_atom in conflict_atoms:
+   parent, atom = parent_atom
+   if not atom.match(similar_pkg):
+   
similar_conflict_atoms.append(parent_atom)
+   if similar_conflict_atoms:
+   
similar_pkgs.append((similar_pkg, set(similar_conflict_atoms)))
+   similar_pkgs.append((to_be_masked, conflict_atoms))
+   backtrack_data.append(tuple(similar_pkgs))
 
# Prefer choices that minimize conflict atoms. This is intended
# to take precedence over the earlier package version sort. The
# package version sort is still needed or else choices for the
# testOverlapSlotConflict method of 
VirtualMinimizeChildrenTestCase
# become non-deterministic.
-   backtrack_data.sort(key=lambda item: len(item[1]))
-   to_be_masked = backtrack_data[-1][0]
+   backtrack_data.sort(key=lambda similar_pkgs: 
len(similar_pkgs[-1][1]))
+   to_be_masked = [item[0] for item in backtrack_data[-1]]
 
self._dynamic_config._backtrack_infos.setdefault(
"slot conflict", []).append(backtrack_data)
@@ -1814,7 +1831,7 @@ class depgraph:
"",
"backtracking due to slot conflict:",
"   first package:  %s" % existing_node,
-   "  package to mask: %s" % to_be_masked,
+   "  package(s) to mask: %s" % str(to_be_masked),
"  slot: %s" % slot_atom,
"   parents: %s" % ", ".join(
"(%s, '%s')" % (ppkg, atom) for ppkg, 
atom in all_parents

diff --git a/lib/_emerge/resolver/backtracking.py 
b/lib/_emerge/resolver/backtracking.py
index bc3fb3206..ca94623ac 100644
--- a/lib/_emerge/resolver/backtracking.py
+++ b/lib/_emerge/resolver/backtracking.py
@@ -166,13 +166,14 @@ class Backtracker:
self._feedback_slot_conflict(conflicts_data[0])
 
def _feedback_slot_conflict(self, conflict_data):
-   for pkg, parent_atoms in conflict_data:
+   for similar_pkgs in conflict_data:
new_node = copy.deepcopy(self._current_node)
new_node.depth += 1
new_node.mask_steps += 1

[gentoo-commits] proj/portage:master commit in: lib/_emerge/, lib/portage/tests/resolver/

2020-09-20 Thread Zac Medico
commit: f64310749f176f8921b72ce282b4294efe81c3f0
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Sep 19 21:32:41 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Sep 20 22:27:32 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=f6431074

_slot_confict_backtrack: minimize conflict atoms (bug 743631)

Prefer choices that minimize conflict atoms, so that choices
which satisfy all parents are preferred. This reduces the
minimum necessary backtrack tries from 21 to 7 for the unit
test related to bug 743115.

Bug: https://bugs.gentoo.org/743115
Bug: https://bugs.gentoo.org/743631
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/depgraph.py| 6 ++
 lib/portage/tests/resolver/test_slot_operator_missed_update.py | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index 3f864aefc..7281d8692 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -1797,6 +1797,12 @@ class depgraph:
if parent_atom not in parent_atoms)
backtrack_data.append((to_be_masked, conflict_atoms))
 
+   # Prefer choices that minimize conflict atoms. This is intended
+   # to take precedence over the earlier package version sort. The
+   # package version sort is still needed or else choices for the
+   # testOverlapSlotConflict method of 
VirtualMinimizeChildrenTestCase
+   # become non-deterministic.
+   backtrack_data.sort(key=lambda item: len(item[1]))
to_be_masked = backtrack_data[-1][0]
 
self._dynamic_config._backtrack_infos.setdefault(

diff --git a/lib/portage/tests/resolver/test_slot_operator_missed_update.py 
b/lib/portage/tests/resolver/test_slot_operator_missed_update.py
index fce012f62..1ea701003 100644
--- a/lib/portage/tests/resolver/test_slot_operator_missed_update.py
+++ b/lib/portage/tests/resolver/test_slot_operator_missed_update.py
@@ -90,7 +90,7 @@ class BacktrackMissedUpdateTestCase(TestCase):
# Bug 743115: missed updates trigger excessive 
backtracking
ResolverPlaygroundTestCase(
[">=dev-python/pypy3-7.3.2_rc", "@world"],
-   options={"--update": True, "--deep": True, 
"--backtrack": 25},
+   options={"--update": True, "--deep": True, 
"--backtrack": 10},
success=True,
mergelist=[
"dev-python/pypy3-7.3.2_rc2_p37-r1",



[gentoo-commits] proj/portage:master commit in: lib/portage/sync/modules/mercurial/, man/, lib/portage/tests/sync/, lib/portage/

2020-09-20 Thread Zac Medico
commit: 6a7d42b66a51a6438068c685c8df37d0872e8bf2
Author: Brian Evans  gentoo  org>
AuthorDate: Thu Jan 11 17:37:47 2018 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Sep 20 08:18:32 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=6a7d42b6

Add mercurial sync support

Bug: https://bugs.gentoo.org/644246
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/const.py|   1 +
 lib/portage/sync/modules/mercurial/__init__.py  |  39 ++
 lib/portage/sync/modules/mercurial/mercurial.py | 174 
 lib/portage/tests/sync/test_sync_local.py   |  67 -
 man/portage.5   |  24 +++-
 5 files changed, 303 insertions(+), 2 deletions(-)

diff --git a/lib/portage/const.py b/lib/portage/const.py
index b895f0fa9..7effcd85d 100644
--- a/lib/portage/const.py
+++ b/lib/portage/const.py
@@ -82,6 +82,7 @@ LIBC_PACKAGE_ATOM= "virtual/libc"
 OS_HEADERS_PACKAGE_ATOM  = "virtual/os-headers"
 CVS_PACKAGE_ATOM = "dev-vcs/cvs"
 GIT_PACKAGE_ATOM = "dev-vcs/git"
+HG_PACKAGE_ATOM  = "dev-vcs/mercurial"
 RSYNC_PACKAGE_ATOM   = "net-misc/rsync"
 
 INCREMENTALS = (

diff --git a/lib/portage/sync/modules/mercurial/__init__.py 
b/lib/portage/sync/modules/mercurial/__init__.py
new file mode 100644
index 0..c4efbdcf4
--- /dev/null
+++ b/lib/portage/sync/modules/mercurial/__init__.py
@@ -0,0 +1,39 @@
+# Copyright 2018-2020 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+doc = """Mercurial plug-in module for portage.
+Performs a hg pull on repositories."""
+__doc__ = doc[:]
+
+from portage.localization import _
+from portage.sync.config_checks import CheckSyncConfig
+from portage.util import writemsg_level
+
+module_spec = {
+   "name": "mercurial",
+   "description": doc,
+   "provides": {
+   "mercurial-module": {
+   "name": "mercurial",
+   "sourcefile": "mercurial",
+   "class": "MercurialSync",
+   "description": doc,
+   "functions": ["sync", "new", "exists", "retrieve_head"],
+   "func_desc": {
+   "sync": "Performs a hg pull on the repository",
+   "new": "Creates the new repository at the 
specified location",
+   "exists": "Returns a boolean of whether the 
specified dir "
+   + "exists and is a valid Mercurial repository",
+   "retrieve_head": "Returns the head commit hash",
+   },
+   "validate_config": CheckSyncConfig,
+   "module_specific_options": (
+   "sync-mercurial-clone-env",
+   "sync-mercurial-clone-extra-opts",
+   "sync-mercurial-env",
+   "sync-mercurial-pull-env",
+   "sync-mercurial-pull-extra-opts",
+   ),
+   }
+   },
+}

diff --git a/lib/portage/sync/modules/mercurial/mercurial.py 
b/lib/portage/sync/modules/mercurial/mercurial.py
new file mode 100644
index 0..2cc4cd1ea
--- /dev/null
+++ b/lib/portage/sync/modules/mercurial/mercurial.py
@@ -0,0 +1,174 @@
+# Copyright 2018-2020 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+import logging
+import subprocess
+
+import portage
+from portage import os
+from portage.util import writemsg_level, shlex_split
+
+from portage.sync.syncbase import NewBase
+
+
+class MercurialSync(NewBase):
+   """Mercurial sync class"""
+
+   short_desc = "Perform sync operations on mercurial based repositories"
+
+   @staticmethod
+   def name():
+   return "MercurialSync"
+
+   def __init__(self):
+   NewBase.__init__(self, "hg", portage.const.HG_PACKAGE_ATOM)
+
+   def exists(self, **kwargs):
+   """Tests whether the repo actually exists"""
+   return os.path.exists(os.path.join(self.repo.location, ".hg"))
+
+   def new(self, **kwargs):
+   """Do the initial clone of the repository"""
+   if kwargs:
+   self._kwargs(kwargs)
+   try:
+   if not os.path.exists(self.repo.location):
+   os.makedirs(self.repo.location)
+   self.logger(
+   self.xterm_titles, "Created new 
directory %s" % self.repo.location
+   )
+   except IOError:
+   return (1, False)
+
+   sync_uri = self.repo.sync_uri
+   if sync_uri.startswith("file://"):
+   sync_uri = sync_uri[7:]
+
+

[gentoo-commits] proj/portage:master commit in: lib/portage/tests/resolver/

2020-09-19 Thread Zac Medico
commit: 302a93e5857ff56d43ee0b9dabee9f00d907a8ad
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Sep 13 01:42:30 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Sep 19 20:22:41 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=302a93e5

Add unit test demonstrating excessive backtracking for bug 743115

Due to excessive backtracking, this test case currently needs
at least --backtrack=21 to succeed.

Bug: https://bugs.gentoo.org/743115
Signed-off-by: Zac Medico  gentoo.org>

 .../resolver/test_slot_operator_missed_update.py   | 112 +
 1 file changed, 112 insertions(+)

diff --git a/lib/portage/tests/resolver/test_slot_operator_missed_update.py 
b/lib/portage/tests/resolver/test_slot_operator_missed_update.py
new file mode 100644
index 0..fce012f62
--- /dev/null
+++ b/lib/portage/tests/resolver/test_slot_operator_missed_update.py
@@ -0,0 +1,112 @@
+# Copyright 2020 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+from portage.tests import TestCase
+from portage.tests.resolver.ResolverPlayground import (
+   ResolverPlayground,
+   ResolverPlaygroundTestCase,
+)
+
+
+class BacktrackMissedUpdateTestCase(TestCase):
+   def testBacktrackMissedUpdateTestCase(self):
+
+   ebuilds = {
+   "dev-lang/python-2.7.18-r2": {
+   "EAPI": "7",
+   "SLOT": "2.7",
+   },
+   "dev-python/pypy3-7.3.2_rc2_p37-r1": {
+   "EAPI": "7",
+   "SLOT": "0/pypy37-pp73",
+   },
+   "dev-python/pypy3-7.3.1-r3": {
+   "EAPI": "7",
+   "SLOT": "0/pypy36-pp73",
+   },
+   "dev-python/setuptools-50.3.0": {
+   "EAPI": "7",
+   "IUSE": "python_targets_pypy3",
+   "RDEPEND": "python_targets_pypy3? ( 
dev-python/pypy3:= )",
+   },
+   "dev-python/setuptools-50.2.0": {
+   "EAPI": "7",
+   "IUSE": "python_targets_pypy3",
+   "RDEPEND": "python_targets_pypy3? ( 
dev-python/pypy3:= )",
+   },
+   "dev-python/setuptools-50.1.0": {
+   "EAPI": "7",
+   "IUSE": "python_targets_pypy3",
+   "RDEPEND": "python_targets_pypy3? ( 
dev-python/pypy3:= )",
+   },
+   "dev-python/setuptools-49.6.0": {
+   "EAPI": "7",
+   "IUSE": "python_targets_pypy3",
+   "RDEPEND": "python_targets_pypy3? ( 
dev-python/pypy3:= )",
+   },
+   "dev-python/setuptools-46.4.0-r2": {
+   "EAPI": "7",
+   "IUSE": "+python_targets_pypy3 
+python_targets_python2_7",
+   "RDEPEND": "python_targets_pypy3? ( 
dev-python/pypy3:= )",
+   },
+   "dev-lang/python-2.7.18-r2": {
+   "EAPI": "7",
+   "IUSE": "+python_targets_pypy3 
+python_targets_python2_7",
+   "RDEPEND": "python_targets_pypy3? ( 
dev-python/pypy3:= )",
+   },
+   "dev-vcs/mercurial-5.5.1": {
+   "EAPI": "7",
+   "IUSE": "+python_targets_pypy3 
+python_targets_python2_7",
+   "RDEPEND": 
"dev-python/setuptools[python_targets_pypy3?,python_targets_python2_7?] 
python_targets_python2_7? ( dev-lang/python:2.7 ) python_targets_pypy3? ( 
dev-python/pypy3:= )",
+   },
+   }
+
+   installed = {
+   "dev-lang/python-2.7.18-r2": {
+   "EAPI": "7",
+   "SLOT": "2.7",
+   },
+   "dev-python/pypy3-7.3.1-r3": {
+   "EAPI": "7",
+   "SLOT": "0/pypy36-pp73",
+   },
+   "dev-python/setuptools-46.4.0-r2": {
+   "EAPI": "7",
+   "IUSE": "+python_targets_pypy3 
+python_targets_python2_7",
+   "USE": "python_targets_pypy3 
python_targets_python2_7",
+   "RDEPEND": "dev-python/pypy3:0/pypy36-pp73=",
+   },
+   "dev-vcs/mercurial-5.5.1": {
+   "EAPI": "7",
+  

[gentoo-commits] proj/portage:master commit in: /

2020-09-14 Thread Zac Medico
commit: bb8a74aff1e23926438baf2738bb8aae878a801f
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Sep 14 06:10:54 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep 14 06:54:31 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=bb8a74af

Updates for portage-3.0.7 release

Signed-off-by: Zac Medico  gentoo.org>

 RELEASE-NOTES | 7 +++
 setup.py  | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index 40565e92a..a594d7d4f 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,13 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.7
+==
+* Bug Fixes:
+- Bug 737470 egencache: add --external-cache-only option
+- Bug 741474 get_mirror_url: urlquote only for ftp, http, and https
+- Bug 742446 support riscv ilp32 and ilp32d ABIs
+
 portage-3.0.6
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index 81d260fac..681fed079 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.6',
+   version = '3.0.7',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/portage/dep/soname/

2020-09-14 Thread Zac Medico
commit: 4c3b7adb63d3fef8963877291a490600a1a55443
Author: Andreas K. Hüttel  gentoo  org>
AuthorDate: Fri Sep 11 21:42:14 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep 14 06:52:39 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=4c3b7adb

Recognize riscv_{ilp32,ilp32d} ABIs

Bug: https://bugs.gentoo.org/742446
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dep/soname/multilib_category.py | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/portage/dep/soname/multilib_category.py 
b/lib/portage/dep/soname/multilib_category.py
index 301c62a35..fb2839e01 100644
--- a/lib/portage/dep/soname/multilib_category.py
+++ b/lib/portage/dep/soname/multilib_category.py
@@ -14,7 +14,7 @@
 #  m68k_{32,64}
 #  mips_{eabi32,eabi64,n32,n64,o32,o64}
 #  ppc_{32,64}
-#  riscv_{lp64,lp64d}
+#  riscv_{ilp32,ilp32d,lp64,lp64d}
 #  s390_{32,64}
 #  sh_{32,64}
 #  sparc_{32,64}
@@ -99,6 +99,8 @@ def _compute_suffix_riscv(elf_header):
Compute riscv multilib suffix. In order to avoid possible
misidentification, only the following ABIs are recognized:
 
+   * ilp32
+   * ilp32d
* lp64
* lp64d
"""
@@ -111,8 +113,13 @@ def _compute_suffix_riscv(elf_header):
elif elf_header.e_flags == EF_RISCV_RVC | 
EF_RISCV_FLOAT_ABI_DOUBLE:
name = "lp64d"
 
-   return name
+   elif elf_header.ei_class == ELFCLASS32:
+   if elf_header.e_flags == EF_RISCV_RVC:
+   name = "ilp32"
+   elif elf_header.e_flags == EF_RISCV_RVC | 
EF_RISCV_FLOAT_ABI_DOUBLE:
+   name = "ilp32d"
 
+   return name
 
 _specialized_funcs = {
"mips": _compute_suffix_mips,



[gentoo-commits] proj/portage:master commit in: cnf/repo.postsync.d/, bin/, man/

2020-09-14 Thread Zac Medico
commit: c783c57205107bfa75441b90a35f414163baad42
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Sep 12 23:09:56 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep 14 06:23:58 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=c783c572

egencache: add --external-cache-only option (bug 737470)

The --external-cache-only option is useful for client-side
use cases where writing cache files inside the repository itself
may interfere with repository verification. This option is
currently supported for --update and --update-pkg-desc-index
actions, for which consumers of the corresponding cache or
index files are already capable of consuming files from the
external cache directory (/var/cache/edb/dep).

Bug: https://bugs.gentoo.org/737470
Signed-off-by: Zac Medico  gentoo.org>

 bin/egencache   | 44 ++--
 cnf/repo.postsync.d/example | 19 +--
 man/egencache.1 |  9 -
 3 files changed, 43 insertions(+), 29 deletions(-)

diff --git a/bin/egencache b/bin/egencache
index 532e37f20..4ee63edad 100755
--- a/bin/egencache
+++ b/bin/egencache
@@ -107,6 +107,9 @@ def parse_args(args):
common.add_argument("--config-root",
help="location of portage config files",
dest="portage_configroot")
+   common.add_argument("--external-cache-only",
+   action="store_true",
+   help="Output only to the external cache (not the repository 
itself)")
common.add_argument("--gpg-dir",
help="override the PORTAGE_GPG_DIR variable",
dest="gpg_dir")
@@ -246,7 +249,7 @@ def parse_args(args):
 
 class GenCache:
def __init__(self, portdb, cp_iter=None, max_jobs=None, max_load=None,
-   rsync=False):
+   rsync=False, external_cache_only=False):
# The caller must set portdb.porttrees in order to constrain
# findname, cp_list, and cpv_list to the desired tree.
tree = portdb.porttrees[0]
@@ -263,18 +266,21 @@ class GenCache:
else:
self._cp_set = None
self._cp_missing = set()
-   write_auxdb = "metadata-transfer" in portdb.settings.features
+   write_auxdb = external_cache_only or "metadata-transfer" in 
portdb.settings.features
self._regen = MetadataRegen(portdb, cp_iter=cp_iter,
consumer=self._metadata_callback,
max_jobs=max_jobs, max_load=max_load,
write_auxdb=write_auxdb, main=True)
self.returncode = os.EX_OK
conf = portdb.repositories.get_repo_for_location(tree)
-   self._trg_caches = tuple(conf.iter_pregenerated_caches(
-   self._auxdbkeys, force=True, readonly=False))
-   if not self._trg_caches:
-   raise Exception("cache formats '%s' aren't supported" %
-   (" ".join(conf.cache_formats),))
+   if external_cache_only:
+   self._trg_caches = ()
+   else:
+   self._trg_caches = tuple(conf.iter_pregenerated_caches(
+   self._auxdbkeys, force=True, readonly=False))
+   if not self._trg_caches:
+   raise Exception("cache formats '%s' aren't 
supported" %
+   (" ".join(conf.cache_formats),))
 
if rsync:
for trg_cache in self._trg_caches:
@@ -1092,7 +1098,8 @@ def egencache_main(args):
gen_cache = GenCache(portdb, cp_iter=cp_iter,
max_jobs=options.jobs,
max_load=options.load_average,
-   rsync=options.rsync)
+   rsync=options.rsync,
+   external_cache_only=options.external_cache_only)
gen_cache.run()
if options.tolerant:
ret.append(os.EX_OK)
@@ -1100,20 +1107,21 @@ def egencache_main(args):
ret.append(gen_cache.returncode)
 
if options.update_pkg_desc_index:
-   if repo_config.writable:
+   if not options.external_cache_only and repo_config.writable:
writable_location = repo_config.location
else:
writable_location = os.path.join(portdb.depcachedir,
repo_config.location.lstrip(os.sep))
-   msg = [
-   "WARNING: Repository is not writable: %s" % (
-   repo_config.location,),
-   " Using cache directory instead: %s" % (
-   writable_location,)
-  

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

2020-09-14 Thread Zac Medico
commit: 20b6a36fbf791bcc7d42bd429cf8116399a014a7
Author: Zac Medico  gentoo  org>
AuthorDate: Sat Sep 12 21:23:28 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sat Sep 12 22:08:45 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=20b6a36f

get_mirror_url: urlquote only for ftp, http, and https (bug 741474)

It's necessary to use urlquote for correct behavior with ftp, http,
and https, since it's possible for file names to contain percent
encoded characters that need to be protected by an additional layer
of percent encoding. For other protocols such as sftp and rsync,
all characters are interpreted literally, so urlquote must not be
used.

Fixes: c238d5f7ed264179c263f5a2da983c4ee50b4f00
Bug: https://bugs.gentoo.org/719810
Bug: https://bugs.gentoo.org/741474
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/package/ebuild/fetch.py | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py 
b/lib/portage/package/ebuild/fetch.py
index 7c61fe463..ca031f31e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -513,8 +513,12 @@ def get_mirror_url(mirror_url, filename, mysettings, 
cache_path=None):
json.dump(cache, f)
f.close()
 
-   return (mirror_url + "/distfiles/" +
-   
urlquote(mirror_conf.get_best_supported_layout().get_path(filename)))
+   # For some protocols, urlquote is required for correct behavior,
+   # and it must not be used for other protocols like rsync and sftp.
+   path = mirror_conf.get_best_supported_layout().get_path(filename)
+   if urlparse(mirror_url).scheme in ('ftp', 'http', 'https'):
+   path = urlquote(path)
+   return mirror_url + "/distfiles/" + path
 
 
 def fetch(myuris, mysettings, listonly=0, fetchonly=0,



[gentoo-commits] proj/portage:master commit in: lib/portage/

2020-09-11 Thread Zac Medico
commit: a8f0e05d35b0ba2747827ce03dff42682192def8
Author: Zac Medico  gentoo  org>
AuthorDate: Wed Sep  9 05:08:20 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Fri Sep 11 19:00:57 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=a8f0e05d

selinux: encode os path arguments as UTF-8 (bug 741194)

Encode path arguments as UTF-8, like portage.os wrapper.

Fixes: 6137290b2bb8 ("selinux: python3 unicode paths, bug #430488")
Tested-by: Bob Gleitsmann  bellsouth.net>
Bug: https://bugs.gentoo.org/741194
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/_selinux.py | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/lib/portage/_selinux.py b/lib/portage/_selinux.py
index e3e18c0b8..2423ff8d6 100644
--- a/lib/portage/_selinux.py
+++ b/lib/portage/_selinux.py
@@ -14,7 +14,7 @@ except ImportError:
 
 import portage
 from portage import _encodings
-from portage import _native_string
+from portage import _native_string, _unicode_encode
 from portage.localization import _
 
 def copyfile(src, dest):
@@ -41,7 +41,6 @@ def is_selinux_enabled():
return selinux.is_selinux_enabled()
 
 def mkdir(target, refdir):
-   target = _native_string(target, encoding=_encodings['fs'], 
errors='strict')
refdir = _native_string(refdir, encoding=_encodings['fs'], 
errors='strict')
(rc, ctx) = selinux.getfilecon(refdir)
if rc < 0:
@@ -51,20 +50,21 @@ def mkdir(target, refdir):
 
setfscreate(ctx)
try:
-   os.mkdir(target)
+   os.mkdir(_unicode_encode(target, encoding=_encodings['fs'], 
errors='strict'))
finally:
setfscreate()
 
 def rename(src, dest):
src = _native_string(src, encoding=_encodings['fs'], errors='strict')
-   dest = _native_string(dest, encoding=_encodings['fs'], errors='strict')
(rc, ctx) = selinux.lgetfilecon(src)
if rc < 0:
raise OSError(_("rename: Failed getting context of \"%s\".") % 
src)
 
setfscreate(ctx)
try:
-   os.rename(src, dest)
+   os.rename(
+   _unicode_encode(src, encoding=_encodings['fs'], 
errors='strict'),
+   _unicode_encode(dest, encoding=_encodings['fs'], 
errors='strict'))
finally:
setfscreate()
 
@@ -132,8 +132,6 @@ class spawn_wrapper:
return self._spawn_func(*args, **kwargs)
 
 def symlink(target, link, reflnk):
-   target = _native_string(target, encoding=_encodings['fs'], 
errors='strict')
-   link = _native_string(link, encoding=_encodings['fs'], errors='strict')
reflnk = _native_string(reflnk, encoding=_encodings['fs'], 
errors='strict')
(rc, ctx) = selinux.lgetfilecon(reflnk)
if rc < 0:
@@ -143,6 +141,8 @@ def symlink(target, link, reflnk):
 
setfscreate(ctx)
try:
-   os.symlink(target, link)
+   os.symlink(
+   _unicode_encode(target, encoding=_encodings['fs'], 
errors='strict'),
+   _unicode_encode(link, encoding=_encodings['fs'], 
errors='strict'))
finally:
setfscreate()



[gentoo-commits] proj/portage:master commit in: /

2020-09-07 Thread Zac Medico
commit: ca9c60a5b4e3253894756b9860045ed15a3975d1
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Sep  8 03:39:08 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 03:39:20 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=ca9c60a5

Updates for portage-3.0.6 release

Signed-off-by: Zac Medico  gentoo.org>

 NEWS  | 15 +++
 RELEASE-NOTES | 13 +
 setup.py  |  2 +-
 3 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/NEWS b/NEWS
index 1d609bd31..c9bd5cfb5 100644
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,20 @@
 News (mainly features/major bug fixes)
 
+portage-3.0.6
+--
+* emerge --search now detects regular expressions automatically. This
+  behavior is controlled by the --regex-search-auto option.
+* make.conf now treats variable names that begin with two underscores
+  as local variables which are not displayed by emerge --info --verbose.
+* egencache --update-pkg-desc now has a --verbose option which causes
+  it to emulate esync --verbose output.
+* env-update now generates a systemd user-session environment in a file
+  named /etc/environment.d/10-gentoo-env.conf.
+* binrepos.conf is a new configuration file that acts as a substitute
+  for the PORTAGE_BINHOST variable, and allows each repository to have
+  a separate FETCHCOMMAND which can be used to customize authentication.
+  This file is documented in the portage(5) man page.
+
 portage-3.0.0
 --
 * Dropped support for Python 2.x.

diff --git a/RELEASE-NOTES b/RELEASE-NOTES
index 9279b11ef..40565e92a 100644
--- a/RELEASE-NOTES
+++ b/RELEASE-NOTES
@@ -1,6 +1,19 @@
 Release Notes; upgrade information mainly.
 Features/major bugfixes are listed in NEWS
 
+portage-3.0.6
+==
+* Bug Fixes:
+- Bug 668302 binrepos.conf: support fetchcommand customization
+- Bug 668334 Add binrepos.conf to replace PORTAGE_BINHOST
+- Bug 704416 env-update: create systemd user-session environment
+- Bug 737470 egencache --update-pkg-desc: emulate esync --verbose
+- Bug 737480 emerge --search: auto-detect regular expressions
+- Bug 739908 @change-deps: fix erroneous repeated rebuilds
+- Bug 740588 make.conf: Treat __* variables as local
+- Bug 740898 PORTAGE_BINHOST urlopen proxy support
+- Bug 740904 rsync and gemato proxy support
+
 portage-3.0.5
 ==
 * Bug Fixes:

diff --git a/setup.py b/setup.py
index d1e25bd6f..81d260fac 100755
--- a/setup.py
+++ b/setup.py
@@ -655,7 +655,7 @@ class build_ext(_build_ext):
 
 setup(
name = 'portage',
-   version = '3.0.5',
+   version = '3.0.6',
url = 'https://wiki.gentoo.org/wiki/Project:Portage',
author = 'Gentoo Portage Development Team',
author_email = 'dev-port...@gentoo.org',



[gentoo-commits] proj/portage:master commit in: lib/portage/dbapi/

2020-09-07 Thread Zac Medico
commit: 6a3bdcc427c58112075c36cc0481c53215f12db4
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Sep  8 02:43:23 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 02:44:46 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=6a3bdcc4

binarytree.move_ent: fix path comparison prior to _movefile

Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dbapi/bintree.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/dbapi/bintree.py b/lib/portage/dbapi/bintree.py
index e4393e06d..7e24589e5 100644
--- a/lib/portage/dbapi/bintree.py
+++ b/lib/portage/dbapi/bintree.py
@@ -531,7 +531,7 @@ class binarytree:
new_path = self.getname(mynewcpv)
self._pkg_paths[
self.dbapi._instance_key(mynewcpv)] = 
new_path[len(self.pkgdir)+1:]
-   if new_path != mytbz2:
+   if new_path != tbz2path:
self._ensure_dir(os.path.dirname(new_path))
_movefile(tbz2path, new_path, 
mysettings=self.settings)
self.inject(mynewcpv)



[gentoo-commits] proj/portage:master commit in: lib/portage/binrepo/, lib/_emerge/, lib/portage/dbapi/, man/

2020-09-07 Thread Zac Medico
commit: 5ebc8a249b08318da5a2ca89cee2eed604f7e639
Author: Zac Medico  gentoo  org>
AuthorDate: Mon Sep  7 00:13:13 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 01:38:08 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=5ebc8a24

binrepos.conf: support fetchcommand customization (bug 668302)

Support customization of fetchcommand and resumecommand in
binrepos.conf, allowing customized authentication mechanisms for
each repository.

Bug: https://bugs.gentoo.org/668302
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/BinpkgFetcher.py  | 29 +++--
 lib/portage/binrepo/config.py |  2 ++
 lib/portage/dbapi/bintree.py  | 34 +-
 man/portage.5 | 14 ++
 4 files changed, 60 insertions(+), 19 deletions(-)

diff --git a/lib/_emerge/BinpkgFetcher.py b/lib/_emerge/BinpkgFetcher.py
index 218d4d2ab..9a96bde28 100644
--- a/lib/_emerge/BinpkgFetcher.py
+++ b/lib/_emerge/BinpkgFetcher.py
@@ -96,14 +96,17 @@ class _BinpkgFetcherProcess(SpawnProcess):
 
# urljoin doesn't work correctly with
# unrecognized protocols like sftp
+   fetchcommand = None
+   resumecommand = None
if bintree._remote_has_index:
-   instance_key = bintree.dbapi._instance_key(pkg.cpv)
-   rel_uri = bintree._remotepkgs[instance_key].get("PATH")
+   remote_metadata = 
bintree._remotepkgs[bintree.dbapi._instance_key(pkg.cpv)]
+   rel_uri = remote_metadata.get("PATH")
if not rel_uri:
rel_uri = pkg.cpv + ".tbz2"
-   remote_base_uri = bintree._remotepkgs[
-   instance_key]["BASE_URI"]
+   remote_base_uri = remote_metadata["BASE_URI"]
uri = remote_base_uri.rstrip("/") + "/" + 
rel_uri.lstrip("/")
+   fetchcommand = remote_metadata.get('FETCHCOMMAND')
+   resumecommand = remote_metadata.get('RESUMECOMMAND')
else:
uri = settings["PORTAGE_BINHOST"].rstrip("/") + \
"/" + pkg.pf + ".tbz2"
@@ -114,13 +117,19 @@ class _BinpkgFetcherProcess(SpawnProcess):
self._async_wait()
return
 
-   protocol = urllib_parse_urlparse(uri)[0]
-   fcmd_prefix = "FETCHCOMMAND"
+   fcmd = None
if resume:
-   fcmd_prefix = "RESUMECOMMAND"
-   fcmd = settings.get(fcmd_prefix + "_" + protocol.upper())
-   if not fcmd:
-   fcmd = settings.get(fcmd_prefix)
+   fcmd = resumecommand
+   else:
+   fcmd = fetchcommand
+   if fcmd is None:
+   protocol = urllib_parse_urlparse(uri)[0]
+   fcmd_prefix = "FETCHCOMMAND"
+   if resume:
+   fcmd_prefix = "RESUMECOMMAND"
+   fcmd = settings.get(fcmd_prefix + "_" + 
protocol.upper())
+   if not fcmd:
+   fcmd = settings.get(fcmd_prefix)
 
fcmd_vars = {
"DISTDIR" : os.path.dirname(pkg_path),

diff --git a/lib/portage/binrepo/config.py b/lib/portage/binrepo/config.py
index a4bce9073..6ba1a3e9f 100644
--- a/lib/portage/binrepo/config.py
+++ b/lib/portage/binrepo/config.py
@@ -15,7 +15,9 @@ class BinRepoConfig:
__slots__ = (
'name',
'name_fallback',
+   'fetchcommand',
'priority',
+   'resumecommand',
'sync_uri',
)
def __init__(self, opts):

diff --git a/lib/portage/dbapi/bintree.py b/lib/portage/dbapi/bintree.py
index 97018db6e..e4393e06d 100644
--- a/lib/portage/dbapi/bintree.py
+++ b/lib/portage/dbapi/bintree.py
@@ -382,10 +382,10 @@ class binarytree:
self._pkgindex_keys.update(["CPV", "SIZE"])
self._pkgindex_aux_keys = \
["BASE_URI", "BDEPEND", "BUILD_ID", 
"BUILD_TIME", "CHOST",
-   "DEFINED_PHASES", "DEPEND", "DESCRIPTION", 
"EAPI",
+   "DEFINED_PHASES", "DEPEND", "DESCRIPTION", 
"EAPI", "FETCHCOMMAND",
"IUSE", "KEYWORDS", "LICENSE", "PDEPEND",
"PKGINDEX_URI", "PROPERTIES", "PROVIDES",
-   "RDEPEND", "repository", "REQUIRES", "RESTRICT",
+   "RDEPEND", "repository", "REQUIRES", 
"RESTRICT", "RESUMECOMMAND",
"SIZE", "SLOT", "USE"]
self._pkgindex_aux_keys = 

[gentoo-commits] proj/portage:master commit in: lib/portage/, lib/portage/binrepo/, lib/portage/tests/emerge/, lib/_emerge/, ...

2020-09-07 Thread Zac Medico
commit: c36a4ec6694b8b9e22fb63298d1588589acb1ab2
Author: Zac Medico  gentoo  org>
AuthorDate: Sun Sep  6 21:12:32 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 01:37:42 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=c36a4ec6

Add binrepos.conf to replace PORTAGE_BINHOST (bug 668334)

Support /etc/portage/binrepos.conf as a replacement for the
PORTAGE_BINHOST variable. Behavior is similar to repos.conf,
initially supporting just the sync-uri attribute. Both binrepos.conf
and PORTAGE_BINHOST can be used simultaneously, in the same way that
repos.conf and PORTDIR_OVERLAY can be used simultaneously.

The emerge --info output for binrepos.conf looks like this:

Binary Repositories:

example-binhost
sync-uri: https://example.com/binhost

Bug: https://bugs.gentoo.org/668334
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/actions.py  |  13 +++-
 lib/portage/binrepo/__init__.py |   0
 lib/portage/binrepo/config.py   | 131 
 lib/portage/const.py|   1 +
 lib/portage/dbapi/bintree.py|  14 +++-
 lib/portage/tests/emerge/test_simple.py |  14 +++-
 man/make.conf.5 |   3 +-
 man/portage.5   |  38 +
 8 files changed, 206 insertions(+), 8 deletions(-)

diff --git a/lib/_emerge/actions.py b/lib/_emerge/actions.py
index f57269817..5e8a46957 100644
--- a/lib/_emerge/actions.py
+++ b/lib/_emerge/actions.py
@@ -32,7 +32,8 @@ portage.proxy.lazyimport.lazyimport(globals(),
 from portage import os
 from portage import shutil
 from portage import _encodings, _unicode_decode
-from portage.const import _DEPCLEAN_LIB_CHECK_DEFAULT
+from portage.binrepo.config import BinRepoConfigLoader
+from portage.const import BINREPOS_CONF_FILE, _DEPCLEAN_LIB_CHECK_DEFAULT
 from portage.dbapi.dep_expand import dep_expand
 from portage.dbapi._expand_new_virt import expand_new_virt
 from portage.dbapi.IndexedPortdb import IndexedPortdb
@@ -1836,6 +1837,16 @@ def action_info(settings, trees, myopts, myfiles):
for repo in repos:
append(repo.info_string())
 
+   binrepos_conf_path = os.path.join(settings['PORTAGE_CONFIGROOT'], 
BINREPOS_CONF_FILE)
+   binrepos_conf = BinRepoConfigLoader((binrepos_conf_path,), settings)
+   if binrepos_conf and any(repo.name for repo in binrepos_conf.values()):
+   append("Binary Repositories:\n")
+   for repo in reversed(list(binrepos_conf.values())):
+   # Omit repos from the PORTAGE_BINHOST variable, since 
they
+   # do not have a name to label them with.
+   if repo.name:
+   append(repo.info_string())
+
installed_sets = sorted(s for s in
root_config.sets['selected'].getNonAtoms() if 
s.startswith(SETPREFIX))
if installed_sets:

diff --git a/lib/portage/binrepo/__init__.py b/lib/portage/binrepo/__init__.py
new file mode 100644
index 0..e69de29bb

diff --git a/lib/portage/binrepo/config.py b/lib/portage/binrepo/config.py
new file mode 100644
index 0..a4bce9073
--- /dev/null
+++ b/lib/portage/binrepo/config.py
@@ -0,0 +1,131 @@
+# Copyright 2020 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+from collections import OrderedDict
+from collections.abc import Mapping
+from hashlib import md5
+
+from portage.localization import _
+from portage.util import _recursive_file_list, writemsg
+from portage.util.configparser import (SafeConfigParser, ConfigParserError,
+   read_configs)
+
+
+class BinRepoConfig:
+   __slots__ = (
+   'name',
+   'name_fallback',
+   'priority',
+   'sync_uri',
+   )
+   def __init__(self, opts):
+   """
+   Create a BinRepoConfig with options in opts.
+   """
+   for k in self.__slots__:
+   setattr(self, k, opts.get(k.replace('_', '-')))
+
+   def info_string(self):
+   """
+   Returns a formatted string containing informations about the 
repository.
+   Used by emerge --info.
+   """
+   indent = " " * 4
+   repo_msg = []
+   repo_msg.append(self.name or self.name_fallback)
+   if self.priority is not None:
+   repo_msg.append(indent + "priority: " + 
str(self.priority))
+   repo_msg.append(indent + "sync-uri: " + self.sync_uri)
+   repo_msg.append("")
+   return "\n".join(repo_msg)
+
+
+class BinRepoConfigLoader(Mapping):
+   def __init__(self, paths, settings):
+   """Load config from files in paths"""
+
+   # Defaults for value interpolation.
+   parser_defaults = {
+   "EPREFIX" : 

[gentoo-commits] proj/portage:master commit in: man/

2020-09-07 Thread Zac Medico
commit: e108f1ccb4a7d38d27154943d7240fe8d2ff4a7d
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Sep  8 01:16:40 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 01:17:16 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=e108f1cc

man/portage.5: Fix man2html rendering in repos.conf section

Signed-off-by: Zac Medico  gentoo.org>

 man/portage.5 | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/man/portage.5 b/man/portage.5
index db6e9ba76..4cffb194a 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1,4 +1,4 @@
-.TH "PORTAGE" "5" "Jul 2020" "Portage VERSION" "Portage"
+.TH "PORTAGE" "5" "Sep 2020" "Portage VERSION" "Portage"
 .SH NAME
 portage \- the heart of Gentoo
 .SH "DESCRIPTION"
@@ -875,8 +875,8 @@ format as \fBrepos.conf\fR.
 \- attributes are specified in "${attribute} = ${value}" format
 .fi
 
-.I Attributes supported in DEFAULT section:
 .RS
+.I Attributes supported in DEFAULT section:
 .RS
 .TP
 .B main\-repo



[gentoo-commits] proj/portage:master commit in: man/, lib/portage/package/ebuild/

2020-09-07 Thread Zac Medico
commit: 16262dc8ab9ab6cc03565bcefc444726b4dfb32f
Author: Arfrever Frehtes Taifersar Arahesis  Apache  Org>
AuthorDate: Sat Sep  5 22:37:05 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 00:47:14 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=16262dc8

make.conf: Treat __* variables as local and do not propagate them.

Bug: https://bugs.gentoo.org/740588
Signed-off-by: Arfrever Frehtes Taifersar Arahesis  Apache.Org>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/package/ebuild/config.py | 6 ++
 man/make.conf.5  | 4 +++-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/config.py 
b/lib/portage/package/ebuild/config.py
index b62ad3069..a09fdbced 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -370,6 +370,9 @@ class config:
_("Found 2 make.conf files, using both 
'%s' and '%s'") %
tuple(make_conf_paths), noiselevel=-1)
 
+   # __* variables set in make.conf are local and are not 
be propagated.
+   make_conf = {k: v for k, v in make_conf.items() if not 
k.startswith("__")}
+
# Allow ROOT setting to come from make.conf if it's not 
overridden
# by the constructor argument (from the calling 
environment).

locations_manager.set_root_override(make_conf.get("ROOT"))
@@ -621,6 +624,9 @@ class config:
tolerant=tolerant, allow_sourcing=True,
expand=expand_map, recursive=True) or 
{})
 
+   # __* variables set in make.conf are local and are not 
be propagated.
+   mygcfg = {k: v for k, v in mygcfg.items() if not 
k.startswith("__")}
+
# Don't allow the user to override certain variables in 
make.conf
profile_only_variables = 
self.configdict["defaults"].get(
"PROFILE_ONLY_VARIABLES", "").split()

diff --git a/man/make.conf.5 b/man/make.conf.5
index 1b997ad37..8a1ea0603 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -1,4 +1,4 @@
-.TH "MAKE.CONF" "5" "Jun 2020" "Portage VERSION" "Portage"
+.TH "MAKE.CONF" "5" "Sep 2020" "Portage VERSION" "Portage"
 .SH "NAME"
 make.conf \- custom settings for Portage
 .SH "SYNOPSIS"
@@ -36,6 +36,8 @@ make.defaults to make.globals to make.conf to the environment
 settings.  Clearing these variables requires a clear\-all as in:
 export USE="\-*"
 .br
+__* variables set in make.conf are local and are not be propagated.
+.br
 In order to create per\-package environment settings, refer to
 \fBpackage.env\fR in \fBportage\fR(5).
 .SH "VARIABLES"



[gentoo-commits] proj/portage:master commit in: cnf/repo.postsync.d/, man/, bin/

2020-09-07 Thread Zac Medico
commit: 00bb1df12827cfce898e6fce8514d59a4cc844b1
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Sep  1 06:44:57 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 00:35:38 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=00bb1df1

egencache --update-pkg-desc-index: emulate esync --verbose output (bug 737470)

When the --verbose flag is given, make --update-pkg-desc-index emulate
esync --verbose output. Example:

 * Searching for changes

 [ N] acct-group/ultimaker (0):  Group for ultimaker
 [ N] acct-user/ultimaker (0):  User for ultimaker
 [ U] www-client/opera (70.0.3728.144):  A fast and secure web browser
 [MU] www-client/opera-developer (72.0.3798.0):  A fast and secure web browser
 [ U] x11-libs/gtksourceview (4.6.1-r1):  A text widget implementing syntax 
highlighting and other features

Bug: https://bugs.gentoo.org/737470
Signed-off-by: Zac Medico  gentoo.org>

 bin/egencache   | 76 +
 cnf/repo.postsync.d/example |  2 +-
 man/egencache.1 |  3 ++
 3 files changed, 74 insertions(+), 7 deletions(-)

diff --git a/bin/egencache b/bin/egencache
index 264c600fe..532e37f20 100755
--- a/bin/egencache
+++ b/bin/egencache
@@ -35,6 +35,7 @@ else:
 
 signal.signal(debug_signum, debug_signal)
 
+import functools
 import io
 import logging
 import subprocess
@@ -50,8 +51,10 @@ portage._internal_caller = True
 from portage import os, _encodings, _unicode_encode, _unicode_decode
 from _emerge.MetadataRegen import MetadataRegen
 from portage.cache.cache_errors import CacheError, StatCollision
-from portage.cache.index.pkg_desc_index import pkg_desc_index_line_format
+from portage.cache.index.pkg_desc_index import pkg_desc_index_line_format, 
pkg_desc_index_line_read
 from portage.const import TIMESTAMP_FORMAT
+from portage.dep import _repo_separator
+from portage.output import colorize, EOutput
 from portage.package.ebuild._parallel_manifest.ManifestScheduler import 
ManifestScheduler
 from portage.util import cmp_sort_key, writemsg_level
 from portage.util._async.AsyncFunction import AsyncFunction
@@ -131,6 +134,9 @@ def parse_args(args):
common.add_argument("--ignore-default-opts",
action="store_true",
help="do not use the EGENCACHE_DEFAULT_OPTS environment 
variable")
+   common.add_argument("-v", "--verbose",
+   action="count", default=0,
+   help="increase verbosity")
common.add_argument("--write-timestamp",
action="store_true",
help="write metadata/timestamp.chk as required for rsync 
repositories")
@@ -448,13 +454,27 @@ class GenCache:
trg_cache._prune_empty_dirs()
 
 class GenPkgDescIndex:
-   def __init__(self, portdb, output_file):
+   def __init__(self, repo_config, portdb, output_file, verbose=False):
self.returncode = os.EX_OK
+   self._repo_config = repo_config
self._portdb = portdb
self._output_file = output_file
+   self._verbose = verbose
 
def run(self):
 
+   display_updates = self._verbose > 0
+   old = {}
+   new = {}
+   if display_updates:
+   try:
+   with open(self._output_file, 'rt', 
encoding=_encodings["repo.content"]) as f:
+   for line in f:
+   pkg_desc = 
pkg_desc_index_line_read(line)
+   old[pkg_desc.cp] = pkg_desc
+   except FileNotFoundError:
+   pass
+
portage.util.ensure_dirs(os.path.dirname(self._output_file))
f = portage.util.atomic_ofstream(self._output_file,
encoding=_encodings["repo.content"])
@@ -466,10 +486,52 @@ class GenPkgDescIndex:
continue
desc, = portdb.aux_get(pkgs[-1], ["DESCRIPTION"])
 
-   f.write(pkg_desc_index_line_format(cp, pkgs, desc))
+   line = pkg_desc_index_line_format(cp, pkgs, desc)
+   f.write(line)
+   if display_updates:
+   new[cp] = pkg_desc_index_line_read(line)
 
f.close()
 
+   if display_updates:
+   out = EOutput()
+   out.einfo("Searching for changes")
+   print("")
+   items = sorted(new.values(), key=lambda pkg_desc: 
pkg_desc.cp)
+   haspkgs = False
+   for pkg_desc in items:
+   masked = False
+   version = 
self._portdb.xmatch("bestmatch-visible",
+   

[gentoo-commits] proj/portage:master commit in: man/, lib/_emerge/

2020-09-07 Thread Zac Medico
commit: 03ae0d95797f68cf86748ae3da184f3018e8c64c
Author: Zac Medico  gentoo  org>
AuthorDate: Tue Sep  1 02:49:50 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 00:17:54 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=03ae0d95

emerge --search: auto-detect regular expressions (bug 737480)

Automatically detect regular expressions when the search string
contains any of these regular expression characters or character
sequences:

  ^ $ * [ ] { } | ? .+

This simplifies usage, so that users no longer have to remember
to prefix regular expressions with the % character. The new
behavior can be disabled by --regex-search-auto=n, in case the
regular expressions interpretation causes some kind of problem.

Note that fuzzy search and regular expression search are
mutually exclusive, and fuzzy search remains the default for
search strings that do not contain any regular expression
characters.

Bug: https://bugs.gentoo.org/737480
Signed-off-by: Zac Medico  gentoo.org>

 lib/_emerge/actions.py |  1 +
 lib/_emerge/main.py|  6 ++
 lib/_emerge/search.py  | 12 +++-
 man/emerge.1   | 12 +++-
 4 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/lib/_emerge/actions.py b/lib/_emerge/actions.py
index a4ecfe43d..f57269817 100644
--- a/lib/_emerge/actions.py
+++ b/lib/_emerge/actions.py
@@ -2036,6 +2036,7 @@ def action_search(root_config, myopts, myfiles, spinner):
search_index=myopts.get("--search-index", "y") != "n",
search_similarity=myopts.get("--search-similarity"),
fuzzy=myopts.get("--fuzzy-search") != "n",
+   regex_auto=myopts.get("--regex-search-auto") != "n",
)
for mysearch in myfiles:
try:

diff --git a/lib/_emerge/main.py b/lib/_emerge/main.py
index 975738762..5075f7f57 100644
--- a/lib/_emerge/main.py
+++ b/lib/_emerge/main.py
@@ -709,6 +709,12 @@ def parse_opts(tmpcmdline, silent=False):
"action" : "store"
},
 
+   "--regex-search-auto": {
+   "help"   : "Enable or disable automatic regular 
expression detection for search actions",
+   "choices": y_or_n,
+   "default": "y",
+   },
+
"--root": {
 "help"   : "specify the target root filesystem for merging 
packages",
 "action" : "store"

diff --git a/lib/_emerge/search.py b/lib/_emerge/search.py
index a59191c1a..61eed0827 100644
--- a/lib/_emerge/search.py
+++ b/lib/_emerge/search.py
@@ -28,7 +28,7 @@ class search:
#
def __init__(self, root_config, spinner, searchdesc,
verbose, usepkg, usepkgonly, search_index=True,
-   search_similarity=None, fuzzy=True):
+   search_similarity=None, fuzzy=True, regex_auto=False):
"""Searches the available and installed packages for the 
supplied search key.
The list of available and installed packages is created at 
object instantiation.
This makes successive searches faster."""
@@ -42,6 +42,7 @@ class search:
self.spinner = None
self.root_config = root_config
self.setconfig = root_config.setconfig
+   self.regex_auto = regex_auto
self.fuzzy = fuzzy
self.search_similarity = (80 if search_similarity is None
else search_similarity)
@@ -259,6 +260,15 @@ class search:
if '/' in self.searchkey:
match_category = 1
fuzzy = False
+
+   if self.regex_auto and not regexsearch and 
re.search(r'[\^\$\*\[\]\{\}\|\?]|\.\+', self.searchkey) is not None:
+   try:
+   re.compile(self.searchkey, re.I)
+   except Exception:
+   pass
+   else:
+   regexsearch = True
+
if regexsearch:
self.searchre=re.compile(self.searchkey,re.I)
else:

diff --git a/man/emerge.1 b/man/emerge.1
index fe7d05a21..c1bcd0220 100644
--- a/man/emerge.1
+++ b/man/emerge.1
@@ -229,7 +229,9 @@ explicitly discarded by running `emaint \-\-fix 
cleanresume` (see
 .BR \-\-search ", " \-s
 Searches for matches of the supplied string in the ebuild repository.
 By default emerge uses a case-insensitive simple search, but you can
-enable a regular expression search by prefixing the search string with %.
+enable a regular expression search by prefixing the search string with %
+(the % prefix can often be omitted if the
+\fB\-\-regex\-search\-auto\fR option is enabled).
 For example, \fBemerge \-\-search "%^kde"\fR searches for any package whose
 name starts with "kde"; \fBemerge 

[gentoo-commits] proj/portage:master commit in: lib/portage/util/

2020-09-07 Thread Zac Medico
commit: 45a5982fe8076066323e91f6b5fe860f3a429f9f
Author: Florian Schmaus  geekplace  eu>
AuthorDate: Sat Sep  5 07:18:17 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Tue Sep  8 00:01:47 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=45a5982f

env-update: create systemd user-session environment definition

Portage's env-update currently transforms the environment information
from /etc/env.d into /etc/profile.env, which is typically sourced by
every user session, setting up its environment.

However, /etc/profile.env is not sourced by systemd user
services. Instead, for the definition of a systemd user session
environment, the 'environment.d' machinery exists. Unfortunately, up
to now, env-update does not produce a profile.env equivalent for this
machinery, causing issues for systemd user services. For example, an
emacs daemon run as user systemd service does not have a complete
PATH (bug #704412 [1]), because some PATH components are injected by
packages via /etc/env.d. For example, an LLVM ebuild may set
PATH="/usr/lib/llvm/9/bin".

This commit changes env-update so that a systemd user session
environment configuration file named

/etc/environment.d/10-gentoo-env.conf

is created.

Thanks to Michael 'veremitz' Everitt, Arfrever Frehtes Taifersar
Arahesis, Ulrich Müller, Joakim Tjernlund, and Zac Medico for the
useful feedback.

1: https://bugs.gentoo.org/704412

Bug: https://bugs.gentoo.org/704416
Signed-off-by: Florian Schmaus  geekplace.eu>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/util/env_update.py | 42 +++---
 1 file changed, 39 insertions(+), 3 deletions(-)

diff --git a/lib/portage/util/env_update.py b/lib/portage/util/env_update.py
index f130b6f6b..dec086cf8 100644
--- a/lib/portage/util/env_update.py
+++ b/lib/portage/util/env_update.py
@@ -333,14 +333,16 @@ def _env_update(makelinks, target_root, prev_mtimes, 
contents, env,
 
del specials["LDPATH"]
 
-   penvnotice  = "# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.\n"
-   penvnotice += "# DO NOT EDIT THIS FILE. CHANGES TO STARTUP PROFILES\n"
+   notice  = "# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.\n"
+   notice += "# DO NOT EDIT THIS FILE."
+   penvnotice  = notice + " CHANGES TO STARTUP PROFILES\n"
cenvnotice  = penvnotice[:]
penvnotice += "# GO INTO /etc/profile NOT /etc/profile.env\n\n"
cenvnotice += "# GO INTO /etc/csh.cshrc NOT /etc/csh.env\n\n"
 
#create /etc/profile.env for bash support
-   outfile = atomic_ofstream(os.path.join(eroot, "etc", "profile.env"))
+   profile_env_path = os.path.join(eroot, "etc", "profile.env")
+   outfile = atomic_ofstream(profile_env_path)
outfile.write(penvnotice)
 
env_keys = [x for x in env if x != "LDPATH"]
@@ -353,6 +355,40 @@ def _env_update(makelinks, target_root, prev_mtimes, 
contents, env,
outfile.write("export %s='%s'\n" % (k, v))
outfile.close()
 
+   # Create the systemd user environment configuration file
+   # /etc/environment.d/10-gentoo-env.conf with the
+   # environment configuration from /etc/env.d.
+   systemd_environment_dir = os.path.join(eroot, "etc", "environment.d")
+   os.makedirs(systemd_environment_dir, exist_ok=True)
+
+   systemd_gentoo_env_path = os.path.join(systemd_environment_dir,
+   "10-gentoo-env.conf")
+   systemd_gentoo_env = atomic_ofstream(systemd_gentoo_env_path)
+   try:
+   senvnotice = notice + "\n\n"
+   systemd_gentoo_env.write(senvnotice)
+
+   for env_key in env_keys:
+   env_key_value = env[env_key]
+
+   # Skip variables with the empty string
+   # as value. Those sometimes appear in
+   # profile.env (e.g. "export GCC_SPECS=''"),
+   # but are invalid in systemd's syntax.
+   if not env_key_value:
+   continue
+
+   # Transform into systemd environment.d
+   # conf syntax, basically shell variable
+   # assignment (without "export ").
+   line = f"{env_key}={env_key_value}\n"
+
+   systemd_gentoo_env.write(line)
+   except:
+   systemd_gentoo_env.abort()
+   raise
+   systemd_gentoo_env.close()
+
#create /etc/csh.env for (t)csh support
outfile = atomic_ofstream(os.path.join(eroot, "etc", "csh.env"))
outfile.write(cenvnotice)



[gentoo-commits] proj/portage:master commit in: man/

2020-09-07 Thread Zac Medico
commit: f1f9062578936d68040490a8122ccf84c087030a
Author: Michael Everitt  veremit  xyz>
AuthorDate: Sun Sep  6 00:05:59 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep  7 23:40:32 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=f1f90625

man/{portage,make.conf}.5: Files in subdirs are parsed in lexical order

Most files in packages.*, make.conf/*, *.postsync.d/* are parsed not just in
ascending alphabetical order, but actually in lexographical (lexical) order,
which means alpha-numerically + symbols.

Closes: https://github.com/gentoo/portage/pull/616
Signed-off-by: Michael Everitt  veremit.xyz>
Signed-off-by: Zac Medico  gentoo.org>

 man/make.conf.5 |  4 ++--
 man/portage.5   | 15 +++
 2 files changed, 9 insertions(+), 10 deletions(-)

diff --git a/man/make.conf.5 b/man/make.conf.5
index eb812150f..1b997ad37 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -15,8 +15,8 @@ the Simple lexical analysis documentation:
 \fIhttps://docs.python.org/3/library/shlex.html\fR. Note that if you source
 files, they need to be in the same shlex syntax for portage to read them.
 If make.conf is a directory, then all the files in that directory will
-be sorted in ascending alphabetical order by file name and summed
-together as if it were a single file.
+be sorted in lexical order by file name and summed together as if it were
+a single file.
 .br
 Portage will check the currently\-defined environment variables
 first for any settings.  If no environment settings are found,

diff --git a/man/portage.5 b/man/portage.5
index 7472972cc..db6e9ba76 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -229,8 +229,8 @@ will be preferred.
 Any file in this directory, directories of other profiles or top-level
 "profiles" directory that begins with "package." or "use." can be more than
 just a flat file.  If it is a directory, then all the files in that directory
-will be sorted in ascending alphabetical order by file name and summed together
-as if it were a single file. Note that this behavior is only supported since
+will be sorted in lexical order by file name and summed together as if
+it were a single file. Note that this behavior is only supported since
 portage-2.1.6.7, and it is not included in PMS at this time.
 
 .I Example:
@@ -602,8 +602,8 @@ virtual/aspell\-dict   app\-dicts/aspell\-en
 Files in this directory including make.conf, repos.conf, and any file
 with a name that begins with "package." can be
 more than just a flat file.  If it is a directory, then all the files in that
-directory will be sorted in ascending alphabetical order by file name and 
summed
-together as if it were a single file.
+directory will be sorted in lexical order by file name and summed together
+as if it were a single file.
 
 .I Example:
 .nf
@@ -843,15 +843,14 @@ x11\-libs/qt \-mysql
 .TP
 .BR postsync.d/
 This directory is for user supplied postsync hooks to be run once after all
-repositories have been synced.  Each script is called in alphabetic order
+repositories have been synced.  Each script is called in lexical order
 without any arguments.
 .fi
 .TP
 .BR repo.postsync.d/
 This directory is for user supplied postsync hooks to be run once after each
-repository has been synced.  Each script is called in alphabetic order
-with three arguments.
-
+repository has been synced.  Each script is called in lexical order
+with the following three arguments:
 .I args:
 repository name, sync-uri, location
 .fi



[gentoo-commits] proj/portage:master commit in: lib/portage/sync/modules/rsync/, lib/portage/sync/, ...

2020-09-07 Thread Zac Medico
commit: ddbe8bd019552573b3f9c1ef2e5701df2edb4dd6
Author: Frédéric Pierret (fepitre)  qubes-os  
org>
AuthorDate: Tue Aug 18 15:00:16 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep  7 23:31:06 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=ddbe8bd0

syncbase: update with newer Gemato proxy parameter in openpgp env

- Handle global proxy setting which is overrided by the one provided
in Portage configuration (if exists).

Bug: https://bugs.gentoo.org/740904
Closes: https://github.com/gentoo/portage/pull/607
See: 
https://github.com/mgorny/gemato/commit/9980de271de4f8f5e993e2b634d0e8d7753e382f
Signed-off-by: Frédéric Pierret (fepitre)  qubes-os.org>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/sync/modules/git/git.py   | 14 ++
 lib/portage/sync/modules/rsync/rsync.py   |  7 ++-
 lib/portage/sync/modules/webrsync/webrsync.py |  4 ++--
 lib/portage/sync/syncbase.py  | 21 +
 4 files changed, 31 insertions(+), 15 deletions(-)

diff --git a/lib/portage/sync/modules/git/git.py 
b/lib/portage/sync/modules/git/git.py
index d87f1a601..8065fff33 100644
--- a/lib/portage/sync/modules/git/git.py
+++ b/lib/portage/sync/modules/git/git.py
@@ -207,19 +207,17 @@ class GitSync(NewBase):
'sync-git-verify-commit-signature', 
'false').lower() not in ('true', 'yes')):
return True
 
-   if self.repo.sync_openpgp_key_path is not None:
-   if gemato is None:
-   writemsg_level("!!! Verifying against specified 
key requires gemato-11.0+ installed\n",
+   if self.repo.sync_openpgp_key_path is not None and gemato is 
None:
+   writemsg_level("!!! Verifying against specified key 
requires gemato-14.5+ installed\n",
level=logging.ERROR, noiselevel=-1)
-   return False
-   openpgp_env = gemato.openpgp.OpenPGPEnvironment()
-   else:
-   openpgp_env = None
+   return False
+
+   openpgp_env = 
self._get_openpgp_env(self.repo.sync_openpgp_key_path)
 
try:
out = EOutput()
env = None
-   if openpgp_env is not None:
+   if openpgp_env is not None and 
self.repo.sync_openpgp_key_path is not None:
try:
out.einfo('Using keys from %s' % 
(self.repo.sync_openpgp_key_path,))
with 
io.open(self.repo.sync_openpgp_key_path, 'rb') as f:

diff --git a/lib/portage/sync/modules/rsync/rsync.py 
b/lib/portage/sync/modules/rsync/rsync.py
index 35d1461e4..54e285d88 100644
--- a/lib/portage/sync/modules/rsync/rsync.py
+++ b/lib/portage/sync/modules/rsync/rsync.py
@@ -133,10 +133,7 @@ class RsyncSync(NewBase):
if self.verify_metamanifest and gemato is not None:
# Use isolated environment if key is specified,
# system environment otherwise
-   if self.repo.sync_openpgp_key_path is not None:
-   openpgp_env = 
gemato.openpgp.OpenPGPEnvironment()
-   else:
-   openpgp_env = 
gemato.openpgp.OpenPGPSystemEnvironment()
+   openpgp_env = 
self._get_openpgp_env(self.repo.sync_openpgp_key_path)
 
try:
# Load and update the keyring early. If it fails, then 
verification
@@ -361,7 +358,7 @@ class RsyncSync(NewBase):
# if synced successfully, verify now
if exitcode == 0 and self.verify_metamanifest:
if gemato is None:
-   writemsg_level("!!! Unable to verify: 
gemato-11.0+ is required\n",
+   writemsg_level("!!! Unable to verify: 
gemato-14.5+ is required\n",
level=logging.ERROR, 
noiselevel=-1)
exitcode = 127
else:

diff --git a/lib/portage/sync/modules/webrsync/webrsync.py 
b/lib/portage/sync/modules/webrsync/webrsync.py
index 20cc25a2c..cc0dbc4a9 100644
--- a/lib/portage/sync/modules/webrsync/webrsync.py
+++ b/lib/portage/sync/modules/webrsync/webrsync.py
@@ -75,11 +75,11 @@ class WebRsync(SyncBase):
return (1, False)
 
if gemato is None:
-   writemsg_level("!!! Verifying against 
specified key requires gemato-11.0+ installed\n",
+   writemsg_level("!!! Verifying against 
specified key requires gemato-14.5+ 

[gentoo-commits] proj/portage:master commit in: lib/portage/sync/modules/rsync/

2020-09-07 Thread Zac Medico
commit: 051d75acf5e3fbca7c1083cb612e5f07a3a98919
Author: Frédéric Pierret (fepitre)  qubes-os  
org>
AuthorDate: Tue Aug  4 13:12:08 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep  7 22:49:53 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=051d75ac

rsync: ignore addrinfos with proxy settings

Not doing so makes getting addrinfos failing due to
"Temporary failure in name resolution" due to proxy settings.

An alternative to this solution would be to define a socks.setdefaultproxy
and passing socks.socksocket to socket.

Bug: https://bugs.gentoo.org/740904
See: https://github.com/gentoo/portage/pull/607
Signed-off-by: Frédéric Pierret (fepitre)  qubes-os.org>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/sync/modules/rsync/rsync.py | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/lib/portage/sync/modules/rsync/rsync.py 
b/lib/portage/sync/modules/rsync/rsync.py
index 88b36ab79..35d1461e4 100644
--- a/lib/portage/sync/modules/rsync/rsync.py
+++ b/lib/portage/sync/modules/rsync/rsync.py
@@ -230,15 +230,16 @@ class RsyncSync(NewBase):
addrinfos = None
uris = []
 
-   try:
-   addrinfos = getaddrinfo_validate(
-   socket.getaddrinfo(getaddrinfo_host, 
None,
-   family, socket.SOCK_STREAM))
-   except socket.error as e:
-   writemsg_level(
-   "!!! getaddrinfo failed for '%s': %s\n"
-   % (_unicode_decode(hostname), str(e)),
-   noiselevel=-1, level=logging.ERROR)
+   if 'RSYNC_PROXY' not in self.spawn_kwargs['env']:
+   try:
+   addrinfos = getaddrinfo_validate(
+   socket.getaddrinfo(
+   getaddrinfo_host, None, 
family, socket.SOCK_STREAM))
+   except socket.error as e:
+   writemsg_level(
+   "!!! getaddrinfo failed for 
'%s': %s\n"
+   % (_unicode_decode(hostname), 
str(e)),
+   noiselevel=-1, 
level=logging.ERROR)
 
if addrinfos:
 



[gentoo-commits] proj/portage:master commit in: lib/portage/util/, lib/portage/dbapi/

2020-09-07 Thread Zac Medico
commit: 6e6d8e7f522fef3c32a7c71298024167c066a3c5
Author: Frédéric Pierret (fepitre)  qubes-os  
org>
AuthorDate: Thu Aug 20 09:35:23 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Mon Sep  7 22:23:08 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=6e6d8e7f

Use portage proxy settings for fetching BINPKG host

Bug: https://bugs.gentoo.org/740898
See: https://github.com/gentoo/portage/pull/607
Signed-off-by: Frédéric Pierret (fepitre)  qubes-os.org>
Signed-off-by: Zac Medico  gentoo.org>

 lib/portage/dbapi/bintree.py |  9 -
 lib/portage/util/_urlopen.py | 10 +++---
 2 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/lib/portage/dbapi/bintree.py b/lib/portage/dbapi/bintree.py
index 620865a79..ee30542a5 100644
--- a/lib/portage/dbapi/bintree.py
+++ b/lib/portage/dbapi/bintree.py
@@ -964,11 +964,18 @@ class binarytree:
download_timestamp + ttl > 
time.time():
raise 
UseCachedCopyOfRemoteIndex()
 
+   # Set proxy settings for _urlopen -> 
urllib_request
+   proxies = {}
+   for proto in ('http', 'https'):
+   value = self.settings.get(proto + 
'_proxy')
+   if value is not None:
+   proxies[proto] = value
+
# Don't use urlopen for https, unless
# PEP 476 is supported (bug #469888).
if parsed_url.scheme not in ('https',) or 
_have_pep_476():
try:
-   f = _urlopen(url, 
if_modified_since=local_timestamp)
+   f = _urlopen(url, 
if_modified_since=local_timestamp, proxies=proxies)
if hasattr(f, 'headers') and 
f.headers.get('timestamp', ''):
remote_timestamp = 
f.headers.get('timestamp')
except IOError as err:

diff --git a/lib/portage/util/_urlopen.py b/lib/portage/util/_urlopen.py
index b46d1554c..b67d02739 100644
--- a/lib/portage/util/_urlopen.py
+++ b/lib/portage/util/_urlopen.py
@@ -26,7 +26,7 @@ def have_pep_476():
return hasattr(__import__('ssl'), '_create_unverified_context')
 
 
-def urlopen(url, if_modified_since=None):
+def urlopen(url, if_modified_since=None, proxies=None):
parse_result = urllib_parse.urlparse(url)
if parse_result.scheme not in ("http", "https"):
return _urlopen(url)
@@ -40,8 +40,12 @@ def urlopen(url, if_modified_since=None):
request.add_header('If-Modified-Since', 
_timestamp_to_http(if_modified_since))
if parse_result.username is not None:
password_manager.add_password(None, url, parse_result.username, 
parse_result.password)
-   auth_handler = CompressedResponseProcessor(password_manager)
-   opener = urllib_request.build_opener(auth_handler)
+
+   handlers = [CompressedResponseProcessor(password_manager)]
+   if proxies:
+   handlers.append(urllib_request.ProxyHandler(proxies))
+   opener = urllib_request.build_opener(*handlers)
+
hdl = opener.open(request)
if hdl.headers.get('last-modified', ''):
try:



[gentoo-commits] proj/portage:master commit in: man/

2020-09-05 Thread Zac Medico
commit: b851d7f8f3c451d4dd4863580e75461c511b44d0
Author: Michael Everitt  veremit  xyz>
AuthorDate: Sun Sep  6 00:05:04 2020 +
Commit: Zac Medico  gentoo  org>
CommitDate: Sun Sep  6 01:13:48 2020 +
URL:https://gitweb.gentoo.org/proj/portage.git/commit/?id=b851d7f8

man/portage.5: Tidy up some long (>80 char) lines

Closes: https://github.com/gentoo/portage/pull/615
Signed-off-by: Michael Everitt  veremit.xyz>
Signed-off-by: Zac Medico  gentoo.org>

 man/portage.5 | 23 +--
 1 file changed, 13 insertions(+), 10 deletions(-)

diff --git a/man/portage.5 b/man/portage.5
index a7e64cd5f..7472972cc 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -214,8 +214,8 @@ More reading:
 .TP
 \fB/etc/portage/make.profile/\fR or \fB/etc/make.profile/\fR
 This is usually just a symlink to the correct profile in
-\fB/var/db/repos/gentoo/profiles/\fR.  Since it is part of the ebuild 
repository, it
-may easily be updated/regenerated by running `emerge \-\-sync`.  It defines
+\fB/var/db/repos/gentoo/profiles/\fR.  Since it is part of the ebuild 
repository,
+it may easily be updated/regenerated by running `emerge \-\-sync`.  It defines
 what a profile is (usually arch specific stuff).  If you need a custom
 profile, then you should make your own \fBmake.profile\fR
 directory and populate it.  However, if you just wish to override some
@@ -644,8 +644,8 @@ The global custom settings for Portage. See 
\fBmake.conf\fR(5).
 .BR mirrors
 Whenever portage encounters a mirror:// style URI it will look up the actual
 hosts here.  If the mirror set is not found here, it will check the global
-mirrors file at /var/db/repos/gentoo/profiles/thirdpartymirrors.  You may also 
set a
-special mirror type called "local".  This list of mirrors will be checked
+mirrors file at /var/db/repos/gentoo/profiles/thirdpartymirrors.  You may also
+set a special mirror type called "local".  This list of mirrors will be checked
 before GENTOO_MIRRORS and will be used even if the package has
 RESTRICT="mirror" or RESTRICT="fetch".
 
@@ -864,8 +864,9 @@ only.
 .BR repos.conf
 Specifies \fIsite\-specific\fR repository configuration information.
 .br
-Configuration specified in \fBrepos.conf\fR can be overriden by 
\fBPORTAGE_REPOSITORIES\fR
-environmental variable, which has the same format as \fBrepos.conf\fR.
+Configuration specified in \fBrepos.conf\fR can be overriden by
+\fBPORTAGE_REPOSITORIES\fR environmental variable, which has the same
+format as \fBrepos.conf\fR.
 
 .I Format:
 .nf
@@ -918,8 +919,8 @@ since operations performed by these tools are inherently
 .TP
 .B auto\-sync = yes|no|true|false
 This setting determines if the repo will be synced during "\fBemerge 
\-\-sync\fR" or
-"\fBemaint sync \-\-auto\fR" runs.  This allows for repositories to be synced 
only when
-desired via "\fBemaint sync \-\-repo foo\fR".
+"\fBemaint sync \-\-auto\fR" runs.  This allows for repositories to be synced
+only when desired via "\fBemaint sync \-\-repo foo\fR".
 .br
 If unset, the repo will be treated as set
 yes, true.
@@ -1001,7 +1002,8 @@ Extra options to give to git when cloning repository (git 
clone).
 .TP
 .B sync\-git\-env
 Set environment variables for git when cloning or pulling the repository.
-These will be overridden by setting them again in sync-git-clone-env and 
sync-git-pull-env.
+These will be overridden by setting them again in sync\-git\-clone\-env and
+sync\-git\-pull\-env.
 See also example for sync-git-clone-env.
 .TP
 .B sync\-git\-pull\-env
@@ -1390,7 +1392,8 @@ and the newer/faster "md5-dict" format.  Default is to 
detect dirs.
 The EAPI to use for profiles when unspecified. This attribute is
 supported only if profile-default-eapi is included in profile-formats.
 .TP
-.BR profile\-formats " = [pms] [portage-1] [portage-2] [profile-bashrcs] 
[profile-set] [profile-default-eapi] [build-id]"
+.BR profile\-formats " = [pms] [portage-1] [portage-2] [profile-bashrcs] \
+[profile-set] [profile-default-eapi] [build-id]"
 Control functionality available to profiles in this repo such as which files
 may be dirs, or the syntax available in parent files.  Use "portage-2" if 
you're
 unsure.  The default is "portage-1-compat" mode which is meant to be compatible



  1   2   3   4   5   6   7   8   9   10   >