[gentoo-portage-dev] [PATCH 2/2] ManifestScheduler: async fetchlist_dict (bug 653946)

2018-04-24 Thread Zac Medico
In order to avoid event loop recursion, pass fetchlist_dict to ManifestTask as a Future. Bug: https://bugs.gentoo.org/653946 --- .../ebuild/_parallel_manifest/ManifestScheduler.py | 82 ++ .../ebuild/_parallel_manifest/ManifestTask.py | 22 ++ pym/portage/tests/dbapi/

[gentoo-portage-dev] [PATCH 1/2] Add iter_gather function (bug 653946)

2018-04-24 Thread Zac Medico
This is similar to asyncio.gather, but takes an iterator of futures as input, and includes support for max_jobs and max_load parameters. For bug 653946, this will be used to asynchronously gather the results of the portdbapi.async_fetch_map calls that are required to generate a Manifest, while usin

[gentoo-portage-dev] [PATCH 0/2] ManifestScheduler: async fetchlist_dict (bug 653946)

2018-04-24 Thread Zac Medico
In order to avoid event loop recursion, pass fetchlist_dict to ManifestTask as a Future. Zac Medico (2): Add iter_gather function (bug 653946) ManifestScheduler: async fetchlist_dict (bug 653946) .../ebuild/_parallel_manifest/ManifestScheduler.py | 82 ++ .../ebuild

[gentoo-portage-dev] [PATCH v2] ManifestScheduler: async fetchlist_dict (bug 653946)

2018-04-24 Thread Zac Medico
In order to avoid event loop recursion, pass fetchlist_dict to ManifestTask as a Future. Bug: https://bugs.gentoo.org/653946 --- [PATCH v2] fixed _future_fetchlist to limit concurrent async_aux_get calls with async_iter_completed(max_jobs=1) .../ebuild/_parallel_manifest/ManifestSched

[gentoo-portage-dev] [PATCH] ManifestScheduler: async fetchlist_dict (bug 653946)

2018-04-24 Thread Zac Medico
In order to avoid event loop recursion, pass fetchlist_dict to ManifestTask as a Future. Bug: https://bugs.gentoo.org/653946 --- .../ebuild/_parallel_manifest/ManifestScheduler.py | 70 +- .../ebuild/_parallel_manifest/ManifestTask.py | 22 +++ pym/portage/tests/dbapi

[gentoo-portage-dev] [PATCH 2/2] AsynchronousTask: add async_wait() method (bug 653856)

2018-04-23 Thread Zac Medico
Since the AsynchronousTask.wait() method is prone to event loop recursion, deprecate it, and add an async_wait() method method to replace it. Instead of using task.wait() in order to implicitly run the event loop, now loop.run_until_complete(task.async_wait()) will be used to explicitly run the eve

[gentoo-portage-dev] [PATCH 1/2] AsynchronousTask: add scheduler attribute (bug 653856)

2018-04-23 Thread Zac Medico
Add an AsynchronousTask.scheduler attribute, which will hold an event loop instance in order to work with Future instances. Name the attribute "scheduler" because this is the name used by many existing subclasses of AsynchronousTask. Since the AsyncScheduler class already has an _event_loop attribu

[gentoo-portage-dev] [PATCH 0/2] AsynchronousTask: add async_wait() method (bug 653856)

2018-04-23 Thread Zac Medico
event loop. This explicit approach will make it more obvious when code will trigger event loop recursion which would not be compatible with asyncio's default event loop. Bug: https://bugs.gentoo.org/653856 Zac Medico (2): AsynchronousTask: add scheduler attribute (bug 653856) Asynchronou

Re: [gentoo-portage-dev] [PATCH 0/5] EbuildFetcher._get_uri_map(): fix event loop recursion (bug 653810)

2018-04-23 Thread Zac Medico
On 04/23/2018 09:10 AM, Brian Dolbec wrote: > On Sun, 22 Apr 2018 15:30:09 -0700 > Zac Medico wrote: > >> Bug: https://bugs.gentoo.org/653810 >> >> Zac Medico (5): >> portdbapi: add async_fetch_map method (bug 653810) >> EbuildFetcher: inherit Compos

Re: [gentoo-portage-dev] [PATCH] BinpkgFetcher: use async lock (bug 614110)

2018-04-22 Thread Zac Medico
On 04/21/2018 03:07 PM, Brian Dolbec wrote: > On Sat, 21 Apr 2018 12:27:28 -0700 > Zac Medico wrote: > >> In order to avoid event loop recursion, convert the >> _BinpkgFetcherProcess.lock() method to an async_lock >> method for use by BinpkgFetcher. >> >&

[gentoo-portage-dev] [PATCH 3/5] EbuildFetcher: add _async_uri_map method (bug 653810)

2018-04-22 Thread Zac Medico
Add an _async_uri_map method to replace the synchronous _get_uri_map method. This will be used to prevent event loop recursion. Bug: https://bugs.gentoo.org/653810 --- pym/_emerge/EbuildFetcher.py | 27 --- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/pym

[gentoo-portage-dev] [PATCH 4/5] EbuildFetcher: use _async_uri_map in _start (bug 653810)

2018-04-22 Thread Zac Medico
Use _async_uri_map to avoid event loop recursion in _start. Bug: https://bugs.gentoo.org/653810 --- pym/_emerge/EbuildFetcher.py | 34 ++ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/pym/_emerge/EbuildFetcher.py b/pym/_emerge/EbuildFetcher.py ind

[gentoo-portage-dev] [PATCH 2/5] EbuildFetcher: inherit CompositeTask (bug 653810)

2018-04-22 Thread Zac Medico
Make EbuildFetcher inherit CompositeTask, and move remaining code to a _EbuildFetcherProcess class that still inherits ForkProcess. The CompositeTask framework will be used to split EbuildFetcher into subtasks, in order to prevent event loop recursion. Bug: https://bugs.gentoo.org/653810 --- pym/

[gentoo-portage-dev] [PATCH 1/5] portdbapi: add async_fetch_map method (bug 653810)

2018-04-22 Thread Zac Medico
Add a portdbapi.async_fetch_map method which is identical to the existing portdbapi.getFetchMap method, but returns a future. This will be used by EbuildFetcher in order to avoid event loop recursion. Bug: https://bugs.gentoo.org/653810 --- pym/portage/dbapi/porttree.py | 75 +

[gentoo-portage-dev] [PATCH 5/5] EbuildFetcher: add async_already_fetched method (bug 653810)

2018-04-22 Thread Zac Medico
Add an async_already_fetched method to replace the synchronous already_fetched method, and use it to prevent event loop recursion. Bug: https://bugs.gentoo.org/653810 --- pym/_emerge/EbuildBuild.py | 8 +++- pym/_emerge/EbuildFetcher.py | 30 -- 2 files changed,

[gentoo-portage-dev] [PATCH 0/5] EbuildFetcher._get_uri_map(): fix event loop recursion (bug 653810)

2018-04-22 Thread Zac Medico
Bug: https://bugs.gentoo.org/653810 Zac Medico (5): portdbapi: add async_fetch_map method (bug 653810) EbuildFetcher: inherit CompositeTask (bug 653810) EbuildFetcher: add _async_uri_map method (bug 653810) EbuildFetcher: use _async_uri_map in _start (bug 653810) EbuildFetcher: add

[gentoo-portage-dev] [PATCH] BinpkgFetcher: use async lock (bug 614110)

2018-04-21 Thread Zac Medico
In order to avoid event loop recursion, convert the _BinpkgFetcherProcess.lock() method to an async_lock method for use by BinpkgFetcher. Bug: https://bugs.gentoo.org/614110 --- pym/_emerge/BinpkgFetcher.py | 53 +++- 1 file changed, 33 insertions(+), 20 de

[gentoo-portage-dev] [PATCH 5/6] PackageUninstall: use async_lock (bug 614112)

2018-04-21 Thread Zac Medico
Asynchronously lock the build directory, and use AsyncTaskFuture to fit the resulting future into the CompositeTask framework that PackageUninstall uses. Bug: https://bugs.gentoo.org/614112 --- pym/_emerge/PackageUninstall.py | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --gi

[gentoo-portage-dev] [PATCH 6/6] EbuildBuildDir: remove synchronous lock method (bug 614112)

2018-04-21 Thread Zac Medico
The synchronous lock method can trigger event loop recursion if the event loop is already running, which is incompatible with asyncio's default event loop. Therefore, migrate the last consumers to use the async_lock method, and remove the synchronous lock method. Bug: https://bugs.gentoo.org/61411

[gentoo-portage-dev] [PATCH 3/6] Binpkg: use async_lock (bug 614112)

2018-04-21 Thread Zac Medico
Asynchronously lock the build directory, and use AsyncTaskFuture to fit the resulting future into the CompositeTask framework that Binpkg uses. Bug: https://bugs.gentoo.org/614112 --- pym/_emerge/Binpkg.py | 16 +--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/pym/_

[gentoo-portage-dev] [PATCH 4/6] EbuildBuild: use async_lock (bug 614112)

2018-04-21 Thread Zac Medico
Asynchronously lock the build directory, and use AsyncTaskFuture to fit the resulting future into the CompositeTask framework that EbuildBuild uses. Bug: https://bugs.gentoo.org/614112 --- pym/_emerge/EbuildBuild.py | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/pym/_em

[gentoo-portage-dev] [PATCH 1/6] EbuildBuildDir: add async_lock method (bug 614112)

2018-04-21 Thread Zac Medico
Asynchronoulsy handle locking of both the category directory and build directory. Bug: https://bugs.gentoo.org/614112 --- pym/_emerge/EbuildBuildDir.py | 76 +++ 1 file changed, 76 insertions(+) diff --git a/pym/_emerge/EbuildBuildDir.py b/pym/_emerge/Ebui

[gentoo-portage-dev] [PATCH 0/6] EbuildBuildDir: add async_lock method (bug 614112)

2018-04-21 Thread Zac Medico
Asynchronoulsy handle locking of both the category directory and build directory. Bug: https://bugs.gentoo.org/614112 Zac Medico (6): EbuildBuildDir: add async_lock method (bug 614112) AbstractEbuildProcess: use async_lock (bug 614112) Binpkg: use async_lock (bug 614112) EbuildBuild: use

[gentoo-portage-dev] [PATCH 2/6] AbstractEbuildProcess: use async_lock (bug 614112)

2018-04-21 Thread Zac Medico
Asynchronously lock the build directory. The asynchronous lock delays creation of the pid, and it's possible for the _wait method to be called before the pid is available. Therefore, AbstractEbuildProcess._wait() must wait for the pid to become available before it can call the SpawnProcess._wait()

[gentoo-portage-dev] [PATCH] emerge --config: return pkg_config exit status (bug 653638)

2018-04-20 Thread Zac Medico
Fix emerge --config exit status so that it will return 1 if the pkg_config function calls die. Bug: https://bugs.gentoo.org/653638 --- pym/_emerge/actions.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pym/_emerge/actions.py b/pym/_emerge/actions.py index b90aa8cb0..1a28

[gentoo-portage-dev] [PATCH 1/2] EbuildBuildDir: add async_unlock method (bug 614108)

2018-04-19 Thread Zac Medico
This calls the existing AsynchronousLock async_unlock method for the build directory lock, and also handles removal of the category directory (with async lock/unlock). Bug: https://bugs.gentoo.org/614108 --- pym/_emerge/EbuildBuildDir.py | 43 +++ 1 file ch

[gentoo-portage-dev] [PATCH 2/2] EbuildBuild: use async_unlock (bug 614108)

2018-04-19 Thread Zac Medico
This adds an _async_unlock_builddir method which accepts a returncode parameter for cases where it should set the returncode and notify exit listeners. Bug: https://bugs.gentoo.org/614108 --- pym/_emerge/EbuildBuild.py | 52 -- 1 file changed, 32 insert

[gentoo-portage-dev] [PATCH 0/2] EbuildBuildDir: add async_unlock method (bug 614108)

2018-04-19 Thread Zac Medico
Bug: https://bugs.gentoo.org/614108 Zac Medico (2): EbuildBuildDir: add async_unlock method (bug 614108) EbuildBuild: use async_unlock (bug 614108) pym/_emerge/EbuildBuild.py| 52 ++- pym/_emerge/EbuildBuildDir.py | 43

[gentoo-portage-dev] [PATCH] EbuildPhase._ebuild_exit: use async_unlock (bug 614108)

2018-04-19 Thread Zac Medico
Use async_unlock to avoid event loop recursion, and AsyncTaskFuture to fit the resulting future into the CompositeTask framework that EbuildPhase uses. Bug: https://bugs.gentoo.org/614108 --- pym/_emerge/EbuildPhase.py | 22 +- 1 file changed, 17 insertions(+), 5 deletions(-)

[gentoo-portage-dev] [PATCH] emerge --usepkgonly: propagate implicit IUSE and USE_EXPAND (bug 640318)

2018-04-17 Thread Zac Medico
For emerge --usepkgonly mode, it's useful to be able to operate without dependence on a local ebuild repository and profile. Therefore, merge Packages header settings from the binary package database (including binhost(s) if enabled) into the profile configuration, in order to propagate implicit IU

[gentoo-portage-dev] [PATCH] Add async_iter_completed for asyncio migration (bug 591760)

2018-04-17 Thread Zac Medico
This serves as a wrapper around portage's internal TaskScheduler class, allowing TaskScheduler API consumers to be migrated to use asyncio interfaces. Bug: https://bugs.gentoo.org/591760 --- .../tests/util/futures/test_iter_completed.py | 37 - pym/portage/util/futures/iter_compl

[gentoo-portage-dev] [PATCH] config.environ(): filter PORTDIR only when required (bug 653230)

2018-04-15 Thread Zac Medico
The changes related bug 373349 were more aggressive than necessary, causing the PORTDIR variable to be omitted from config.environ() calls when the EAPI is defined. We really only need to filter the PORTDIR variable when the current EAPI requires it. Fixes: 802e7d0bdd96 ("Do not export PORTDIR & E

[gentoo-portage-dev] [PATCH] Implement AbstractEventLoop.connect_write_pipe (bug 649588)

2018-04-15 Thread Zac Medico
In python versions that support asyncio, this allows API consumers to use subprocess.PIPE for asyncio.create_subprocess_exec() stdin parameters. Bug: https://bugs.gentoo.org/649588 --- .../util/futures/asyncio/test_subprocess_exec.py | 34 +++ pym/portage/util/futures/transports.py

[gentoo-portage-dev] [PATCH] Implement AbstractEventLoop.connect_read_pipe (bug 649588)

2018-04-14 Thread Zac Medico
In python versions that support asyncio, this allows API consumers to use subprocess.PIPE for asyncio.create_subprocess_exec() stdout and stderr parameters. Bug: https://bugs.gentoo.org/649588 --- .../util/futures/asyncio/test_subprocess_exec.py | 30 pym/portage/util/futures/unix_events.

[gentoo-portage-dev] [PATCH] rsync: default to sync-rsync-verify-jobs = 1 (bug 650696)

2018-04-13 Thread Zac Medico
Some users have reported that using all processors to verify manifests results in overloading, therefore default to using a single processor. On modern hardware, verification of the gentoo repository completes in less than 20 seconds, so using multiple processors is not really necessary. Also, gema

[gentoo-portage-dev] [PATCH] Implement _PortageEventLoop.subprocess_exec (bug 649588)

2018-04-12 Thread Zac Medico
In python versions that support asyncio, this allows API consumers to use the asyncio.create_subprocess_exec() function with portage's internal event loop. Currently, subprocess.PIPE is not implemented because that would require an implementation of asyncio's private asyncio.unix_events._UnixReadPi

[gentoo-portage-dev] [PATCH] Implement AbstractEventLoopPolicy.get_child_watcher() (bug 649588)

2018-04-11 Thread Zac Medico
Use a _PortageChildWatcher class to wrap portage's internal event loop and implement asyncio's AbstractChildWatcher interface. Bug: https://bugs.gentoo.org/649588 --- .../util/futures/asyncio/test_child_watcher.py | 45 +++ pym/portage/util/_eventloop/EventLoop.py | 3 +- p

[gentoo-portage-dev] [PATCH] _slot_operator.._reinstalls: probe binpkg rebuild (bug 652938)

2018-04-10 Thread Zac Medico
If the parent is not installed, check if it needs to be rebuilt against an installed instance, since otherwise it could trigger downgrade of an installed instance. Bug: https://bugs.gentoo.org/652938 --- pym/_emerge/depgraph.py| 10 - .../tests/resolver/test_slot_o

[gentoo-portage-dev] [PATCH] Add minimal asyncio.AbstractEventLoop implementation (bug 649588)

2018-04-08 Thread Zac Medico
This provides minimal interoperability with existing asyncio code, by adding a portage.util.futures.unix_events.DefaultEventLoopPolicy class that makes asyncio use portage's internal event loop when an instance is passed into asyncio.set_event_loop_policy(). The get_event_loop() method of this poli

[gentoo-portage-dev] [PATCH] _pkg_str: add _db attribute (bug 640318)

2018-04-06 Thread Zac Medico
In order to propagate information from dbapi to Package instance, it's useful for each _pkg_str instance to have a _db attribute which references the dbapi instance that instantiated it. Use a new dbapi._iuse_implicit_cnstr method to delegate implicit IUSE logic to the dbapi instance, in order to m

Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)

2018-04-01 Thread Zac Medico
On 04/01/2018 11:29 AM, Michał Górny wrote: > W dniu nie, 01.04.2018 o godzinie 08∶59 -0700, użytkownik Zac Medico > napisał: >> On 04/01/2018 03:57 AM, Michał Górny wrote: >>> W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico >>> napisał: >

Re: [gentoo-portage-dev] [PATCH] INSTALL_MASK: honor install time config for binary packages (bug 651952)

2018-04-01 Thread Zac Medico
On 04/01/2018 06:54 AM, Michał Górny wrote: > W dniu czw, 29.03.2018 o godzinie 15∶34 -0700, użytkownik Zac Medico > napisał: >> For binary packages, honor the INSTALL_MASK configuration that >> exists at install time, since it might differ from the build time >> setting. &

Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)

2018-04-01 Thread Zac Medico
On 04/01/2018 03:57 AM, Michał Górny wrote: > W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico > napisał: >> Since key refresh is prone to failure, retry using exponential >> backoff with random jitter. This adds the following sync-openpgp-* >>

[gentoo-portage-dev] [PATCH 4/4] rsync: add key refresh retry (bug 649276)

2018-03-31 Thread Zac Medico
Since key refresh is prone to failure, retry using exponential backoff with random jitter. This adds the following sync-openpgp-* configuration settings: sync-openpgp-key-refresh-retry-count = 40 Maximum number of times to retry key refresh if it fails. Between each key refresh attempt, th

[gentoo-portage-dev] [PATCH 3/4] Add retry decorator (API inspired by tenacity)

2018-03-31 Thread Zac Medico
This decorator will be useful for retrying asynchronous operations, such as gpg key refresh (bug 649276). The API is inspired by tenacity, but is simpler. Only asynchronous functions (like @asyncio.coroutine functions) are supported. In order to retry a synchronous function, first convert it to an

[gentoo-portage-dev] [PATCH 2/4] Add ExponentialBackoff and RandomExponentialBackoff

2018-03-31 Thread Zac Medico
This will be useful as parameters for retry decorators. --- pym/portage/util/backoff.py | 48 + 1 file changed, 48 insertions(+) create mode 100644 pym/portage/util/backoff.py diff --git a/pym/portage/util/backoff.py b/pym/portage/util/backoff.py new f

[gentoo-portage-dev] [PATCH 1/4] Add ForkExecutor (bug 649588)

2018-03-31 Thread Zac Medico
This is useful for asynchronous operations that we might need to cancel if they take too long, since (concurrent. futures.ProcessPoolExecutor tasks are not cancellable). This ability to cancel tasks makes this executor useful as an alternative to portage.exception.AlarmSignal. Also add an asyncio-

[gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)

2018-03-31 Thread Zac Medico
limit for all refresh attempts, in units of seconds. Bug: https://bugs.gentoo.org/649276 Zac Medico (4): Add ForkExecutor (bug 649588) Add ExponentialBackoff and RandomExponentialBackoff Add retry decorator (API inspired by tenacity) rsync: add key refresh retry (bug 649276) cnf/repos.conf

[gentoo-portage-dev] [PATCH] revdep-rebuild.sh: use awk instead of gawk

2018-03-31 Thread Zac Medico
From: Hadrien Lacour --- bin/revdep-rebuild.sh | 28 +++- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/bin/revdep-rebuild.sh b/bin/revdep-rebuild.sh index 633701e..5fecf97 100755 --- a/bin/revdep-rebuild.sh +++ b/bin/revdep-rebuild.sh @@ -235,9 +235,19 @

[gentoo-portage-dev] [PATCH] INSTALL_MASK: honor install time config for binary packages (bug 651952)

2018-03-29 Thread Zac Medico
For binary packages, honor the INSTALL_MASK configuration that exists at install time, since it might differ from the build time setting. Fixes: 3416876c0ee7 ("{,PKG_}INSTALL_MASK: python implementation") Bug: https://bugs.gentoo.org/651952 --- bin/misc-functions.sh| 23 ++

[gentoo-portage-dev] [PATCH 3/4] {,PKG_}INSTALL_MASK: python implementation

2018-03-27 Thread Zac Medico
The InstallMask.match code comes from the dblink _is_install_masked method from portage-mgorny: https://github.com/mgorny/portage/commit/f5ac3af3216d209618a2f232b4bf720bc8b520ad --- bin/misc-functions.sh| 73 -- pym/_emerge/PackagePhase.py | 13 - py

[gentoo-portage-dev] [PATCH 4/4] {,PKG_}INSTALL_MASK: Support exclusions (bug 651214)

2018-03-27 Thread Zac Medico
From: Michał Górny Allow INSTALL_MASK patterns to start with '-' to indicate that a specific match is to be excluded from being masked. In this case, the last matching pattern determines whether the file is actually filtered out or kept. Bug: https://bugs.gentoo.org/651214 --- pym/portage/util/

[gentoo-portage-dev] [PATCH 2/4] EbuildPhase: add PackagePhase class for PKG_INSTALL_MASK

2018-03-27 Thread Zac Medico
The PackagePhase class will be a convient place to invoke a python implementation of PKG_INSTALL_MASK. --- bin/misc-functions.sh | 23 - pym/_emerge/EbuildPhase.py | 7 pym/_emerge/PackagePhase.py | 80 + 3 files changed, 93 inse

[gentoo-portage-dev] [PATCH 0/4] {,PKG_}INSTALL_MASK support for exclusions (bug 651214)

2018-03-27 Thread Zac Medico
exclusions (bug 651214) Zac Medico (3): {,PKG_}INSTALL_MASK: record value in vardb EbuildPhase: add PackagePhase class for PKG_INSTALL_MASK {,PKG_}INSTALL_MASK: python implementation bin/misc-functions.sh| 96 + bin/phase-functions.sh | 12

[gentoo-portage-dev] [PATCH 1/4] {,PKG_}INSTALL_MASK: record value in vardb

2018-03-27 Thread Zac Medico
Also, skip preinst_mask phase when INSTALL_MASK is empty. --- bin/misc-functions.sh | 8 bin/phase-functions.sh | 12 +++- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh index 7643af7b5..742ce40d2 100755 --- a/bin/mi

[gentoo-portage-dev] [PATCH] emerge: add --changed-slot [ y | n ] option (bug 631358)

2018-03-26 Thread Zac Medico
This is similar to --changed-deps, but for SLOT metadata. Bug: https://bugs.gentoo.org/631358 --- man/emerge.1 | 7 +++ pym/_emerge/create_depgraph_params.py | 6 ++ pym/_emerge/depgraph.py | 11 +

[gentoo-portage-dev] [PATCH] Disallow package.provided in EAPI 7

2018-03-26 Thread Zac Medico
This is also implemented in portage-mgorny by the following commit: https://github.com/mgorny/portage/commit/386cdb131c99b01541d53c8c894b2ec6534b6dea Bug: https://bugs.gentoo.org/568884 --- pym/portage/eapi.py | 5 + pym/portage/package/ebuild/config.py | 3 ++- 2 files chan

[gentoo-portage-dev] Re: [PATCH 0/7] Add dostrip for EAPI 7

2018-03-26 Thread Zac Medico
On 03/25/2018 01:28 PM, Zac Medico wrote: > The series contains dostrip patches cherry picked from portage-mgorny: > > commit 38273616460fef9b36d3862f366f49f8ba5e7911 > Author: Michał Górny > Date: 2018-03-25 14:37:44 +0200 > > Add dostrip for EAPI 7 &g

[gentoo-portage-dev] [PATCH] Add dostrip for EAPI 7

2018-03-26 Thread Zac Medico
This patch includes the essential parts of the dostrip implementation from portage-mgorny. It also bans the non-standard prepstrip and prepallstrip helpers in EAPI 7, with a die message suggesting to use 'dostrip' instead. All of the prepstrip code has moved to bin/estrip, without any changes excep

[gentoo-portage-dev] [PATCH] emerge --autounmask: prevent unmask due to unsatisfied REQUIRED_USE (bug 622462)

2018-03-26 Thread Zac Medico
Fix autounmask to generate USE changes that violate REQUIRED_USE, in order to prevent it from trying to create inappropriate keywords or package.unmask. This solves the problem reported in bug 622462, where it inappropriately unmasked a newer version of qscintilla. With this fix, it will generate U

[gentoo-portage-dev] [PATCH 4/7] Introduce control variables for estrip

2018-03-25 Thread Zac Medico
From: Michał Górny --- bin/misc-functions.sh | 3 ++- bin/phase-helpers.sh | 2 ++ bin/save-ebuild-env.sh | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh index a7af3ec73..30e911fd6 100755 --- a/bin/misc-functions.sh +++ b/bi

[gentoo-portage-dev] [PATCH 7/7] Add dostrip for EAPI 7

2018-03-25 Thread Zac Medico
From: Michał Górny Bug: https://bugs.gentoo.org/203891 Closes: https://github.com/mgorny/portage/issues/8 --- bin/eapi.sh| 4 bin/ebuild.sh | 2 +- bin/phase-helpers.sh | 26 ++ bin/save-ebuild-env.sh | 2 +- 4 files changed, 32 insertions(+

[gentoo-portage-dev] [PATCH 6/7] Allow individually allowing strip w/ RESTRICT=strip

2018-03-25 Thread Zac Medico
From: Michał Górny Update RESTRICT=strip to control the default strip inclusion list as specified in EAPI 7 rather than disabling stripping entirely. This makes it possible to strip individual files. --- bin/estrip | 4 ++-- bin/phase-helpers.sh | 1 + 2 files changed, 3 insertions(+),

[gentoo-portage-dev] [PATCH 2/7] Disarm prepstrip & prepallstrip

2018-03-25 Thread Zac Medico
From: Michał Górny Move stripping logic to a new estrip helper (that is not exposed to ebuilds) and make prepstrip & prepallstrip do nothing. --- bin/ebuild-helpers/prepallstrip | 11 +- bin/ebuild-helpers/prepstrip| 387 +-- bin/estrip

[gentoo-portage-dev] [PATCH 3/7] estrip: Use queue/dequeue logic

2018-03-25 Thread Zac Medico
From: Michał Górny --- bin/estrip| 70 +++ bin/misc-functions.sh | 3 ++- 2 files changed, 61 insertions(+), 12 deletions(-) diff --git a/bin/estrip b/bin/estrip index 431624498..309a2962e 100755 --- a/bin/estrip +++ b/bin/estrip @@ -

[gentoo-portage-dev] [PATCH 1/7] prepstrip: Disable parallel work

2018-03-25 Thread Zac Medico
From: Michał Górny --- bin/ebuild-helpers/prepstrip | 20 +++- 1 file changed, 3 insertions(+), 17 deletions(-) diff --git a/bin/ebuild-helpers/prepstrip b/bin/ebuild-helpers/prepstrip index 2136e0d4d..74ab52c5b 100755 --- a/bin/ebuild-helpers/prepstrip +++ b/bin/ebuild-helpers/

[gentoo-portage-dev] [PATCH 5/7] estrip: Report pre-stripped files even with RESTRICT=strip

2018-03-25 Thread Zac Medico
From: Michał Górny The purpose of RESTRICT=strip is to prevent files from being stripped, not to silence QA checks about pre-stripped files. --- bin/estrip | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/estrip b/bin/estrip index 309a2962e..24c6a461c 100755 --- a/bin/estr

[gentoo-portage-dev] [PATCH 0/7] Add dostrip for EAPI 7

2018-03-25 Thread Zac Medico
The series contains dostrip patches cherry picked from portage-mgorny: commit 38273616460fef9b36d3862f366f49f8ba5e7911 Author: Michał Górny Date: 2018-03-25 14:37:44 +0200 Add dostrip for EAPI 7 Bug: https://bugs.gentoo.org/203891 Closes: https://github.com/mgorny/portage/issu

[gentoo-portage-dev] [PATCH] portageq repos_config: fix parameter (bug 648062)

2018-03-23 Thread Zac Medico
The parameter is ineffective for commands that query configuration, since the PORTAGE_CONFIGROOT variable controls the location of configuration files. Therefore, for portageq repos_config, implicitly set PORTAGE_CONFIGROOT equal to the value of the parameter. Note that this works correctly for b

Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-22 Thread Zac Medico
On 03/22/2018 05:52 PM, Joakim Tjernlund wrote: > On Mon, 2018-03-19 at 15:59 -0700, Zac Medico wrote: >> On 03/15/2018 12:22 PM, Michał Górny wrote: >>> Hi, >>> >>> Here are three of four INSTALL_MASK updates I've sent long time ago >>> whic

Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-22 Thread Zac Medico
On 03/21/2018 04:30 PM, M. J. Everitt wrote: > On 21/03/18 23:26, M. J. Everitt wrote: >> n 20/03/18 04:49, Manuel Rüger wrote: >>> Hi Zac, >>> >>> alternatively could --exclude be extended to support sets? >>> So users could --exclude @world or @profile. >>> >>> Cheers, >>> Manuel >>> >> The idea

Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-22 Thread Zac Medico
On 03/21/2018 05:59 PM, Manuel Rüger wrote: > On 22.03.2018 01:25, Zac Medico wrote: >> On 03/19/2018 09:49 PM, Manuel Rüger wrote: >>> Hi Zac, >>> >>> alternatively could --exclude be extended to support sets? >>> So users could --exclude @world or @pr

Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Zac Medico
On 03/21/2018 04:53 PM, Joakim Tjernlund wrote: > On Tue, 2018-03-20 at 05:49 +0100, Manuel Rüger wrote: >> Hi Zac, >> >> alternatively could --exclude be extended to support sets? >> So users could --exclude @world or @profile. > > Yes please, I think I have a bug in that direction already(and --

Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Zac Medico
On 03/19/2018 09:49 PM, Manuel Rüger wrote: > Hi Zac, > > alternatively could --exclude be extended to support sets? > So users could --exclude @world or @profile. Your idea doesn't really fit the current meaning of --exclude, since --exclude excludes packages from being merged, but still adds in

[gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Zac Medico
Ignore the @world package set and its dependencies. This may be useful if there is a desire to perform an action even though it might break the dependencies of some installed packages (it might also remove installed packages in order to solve blockers). This also alters the behavior of --complete-g

Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-19 Thread Zac Medico
On 03/15/2018 12:22 PM, Michał Górny wrote: > Hi, > > Here are three of four INSTALL_MASK updates I've sent long time ago > which were not really reviewed. The fourth patch added support > for repo-defined install-mask.conf and I'll do that separately. > > Those patches focus on smaller changes.

Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-18 Thread Zac Medico
On 03/18/2018 02:03 AM, Michał Górny wrote: > W dniu czw, 15.03.2018 o godzinie 22∶10 -0700, użytkownik Zac Medico > napisał: >> A binary package should >> use the value of INSTALL_MASK that existed at build time. >> > > Wait a minute! This doesn't make a

[gentoo-portage-dev] [PATCH] GitSync: abort early for missing git command (bug 650754)

2018-03-17 Thread Zac Medico
Bug: https://bugs.gentoo.org/650754 --- pym/portage/sync/modules/git/git.py | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/pym/portage/sync/modules/git/git.py b/pym/portage/sync/modules/git/git.py index cec760d00..160137a6d 100644 --- a/pym/portage/sync/modules/git/git

Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Zac Medico
On 03/16/2018 02:13 PM, Michał Górny wrote: > W dniu pią, 16.03.2018 o godzinie 10∶07 -0700, użytkownik Zac Medico > napisał: >> On 03/16/2018 03:08 AM, Michał Górny wrote: >>> W dniu czw, 15.03.2018 o godzinie 22∶10 -0700, użytkownik Zac Medico >>> napisał: >&

Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Zac Medico
On 03/16/2018 03:08 AM, Michał Górny wrote: > W dniu czw, 15.03.2018 o godzinie 22∶10 -0700, użytkownik Zac Medico > napisał: >> On 03/15/2018 12:22 PM, Michał Górny wrote: >>> Hi, >>> >>> Here are three of four INSTALL_MASK updates I've sent long time

Re: [gentoo-portage-dev] [PATCH] portage.dbapi.vartree: Remove one more unfounded virtual case

2018-03-16 Thread Zac Medico
On 03/16/2018 03:05 AM, Michał Górny wrote: > --- > pym/portage/dbapi/vartree.py | 7 --- > 1 file changed, 7 deletions(-) > > diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py > index 8b1b77f7d..6406682d6 100644 > --- a/pym/portage/dbapi/vartree.py > +++ b/pym/portage/

[gentoo-portage-dev] [PATCH] EventLoop: implement time method for asyncio compat (bug 591760)

2018-03-16 Thread Zac Medico
Use time.monotonic() which is available in Python 3.3 and later, and otherwise emulate it by using an offset to counteract any backward movements. Bug: https://bugs.gentoo.org/591760 --- pym/portage/util/_eventloop/EventLoop.py | 19 ++- pym/portage/util/monotonic.py|

Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-15 Thread Zac Medico
On 03/15/2018 12:22 PM, Michał Górny wrote: > Hi, > > Here are three of four INSTALL_MASK updates I've sent long time ago > which were not really reviewed. The fourth patch added support > for repo-defined install-mask.conf and I'll do that separately. > > Those patches focus on smaller changes.

Re: [gentoo-portage-dev] keepdir function missing when using qmerge to install binary packages

2018-03-14 Thread Zac Medico
On 03/14/2018 09:21 AM, Joakim Tjernlund wrote: > I got an error msg similar to > keepdir: function missing > using qmerge to install. > > So this function seem missing and I wonder if that one > should be defined/impl. inside the binary packet or if qmerge > should provide it? It's a bug in th

Re: [gentoo-portage-dev] [PATCH] prepstrip: Fix double slash in pre-stripped list

2018-03-14 Thread Zac Medico
On 03/14/2018 03:20 AM, Michał Górny wrote: > --- > bin/ebuild-helpers/prepstrip | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/bin/ebuild-helpers/prepstrip b/bin/ebuild-helpers/prepstrip > index 9b895c60b..2136e0d4d 100755 > --- a/bin/ebuild-helpers/prepstrip > +++ b/bi

Re: [gentoo-portage-dev] [PATCH] ebuild-helpers: Fix overriding 'insinto' directory

2018-03-08 Thread Zac Medico
On 03/08/2018 11:51 AM, Michał Górny wrote: > Closes: https://bugs.gentoo.org/649946 > --- > bin/ebuild-helpers/doconfd | 9 +++-- > bin/ebuild-helpers/doenvd | 9 +++-- > bin/ebuild-helpers/doheader | 6 +- > 3 files changed, 19 insertions(+), 5 deletions(-) > > diff --git a/bin/e

Re: [gentoo-portage-dev] portage perfoemance again ...

2018-03-06 Thread Zac Medico
On 03/06/2018 02:10 PM, Joakim Tjernlund wrote: > .. using portage 2.3.24 on our embedded target(ppc329 It takes forever, > 10 > mins) to just do the > dependencies for emerge -aNDuv world > Here is what strace says, one can see portage is rereading the profile over > and over again. > At this po

[gentoo-portage-dev] [PATCH] gnome2-utils postinst: fix false positive for gtk-update-icon-cache (bug 649464)

2018-03-04 Thread Zac Medico
For the first install of gtk-update-icon-cache, there's no way to initialize state during preinst, so we have to ignore this package in order to avoid false positives. Bug: https://bugs.gentoo.org/649464 --- bin/postinst-qa-check.d/50gnome2-utils | 3 +++ 1 file changed, 3 insertions(+) diff --g

Re: [gentoo-portage-dev] [PATCH 00/10] EAPI 7, part one

2018-03-03 Thread Zac Medico
On 02/26/2018 07:59 AM, Michał Górny wrote: > Hi, everyone. > > Here's the first part of EAPI 7 support patches. All changes included > here were tested and should not break anything. I'd like to merge them > now to avoid having to rebase a very long batch of patches. > > I have added tests for c

Re: [gentoo-portage-dev] [PATCH] test_required_use: Add tests for ?? operator

2018-02-26 Thread Zac Medico
On 02/26/2018 04:40 AM, Michał Górny wrote: > --- > pym/portage/tests/resolver/test_required_use.py | 12 > 1 file changed, 12 insertions(+) > > diff --git a/pym/portage/tests/resolver/test_required_use.py > b/pym/portage/tests/resolver/test_required_use.py > index c8810faef..d40222

Re: [gentoo-portage-dev] [PATCH 1/2] portdbapi: add async_aux_get method (bug 648790)

2018-02-26 Thread Zac Medico
On 02/26/2018 11:58 AM, Michał Górny wrote: > W dniu pon, 26.02.2018 o godzinie 11∶40 -0800, użytkownik Zac Medico > napisał: >> On 02/26/2018 11:23 AM, Michał Górny wrote: >>> W dniu pon, 26.02.2018 o godzinie 10∶09 -0800, użytkownik Zac Medico >>> napisał: >&

Re: [gentoo-portage-dev] [PATCH 1/2] portdbapi: add async_aux_get method (bug 648790)

2018-02-26 Thread Zac Medico
On 02/26/2018 11:23 AM, Michał Górny wrote: > W dniu pon, 26.02.2018 o godzinie 10∶09 -0800, użytkownik Zac Medico > napisał: >> On 02/26/2018 04:29 AM, Michał Górny wrote: >>> W dniu nie, 25.02.2018 o godzinie 17∶50 -0800, użytkownik Zac Medico >>> napisał: >

Re: [gentoo-portage-dev] [PATCH 1/2] portdbapi: add async_aux_get method (bug 648790)

2018-02-26 Thread Zac Medico
On 02/26/2018 10:16 AM, Alec Warner wrote: > > > On Mon, Feb 26, 2018 at 1:09 PM, Zac Medico <mailto:zmed...@gentoo.org>> wrote: > > On 02/26/2018 04:29 AM, Michał Górny wrote: > > W dniu nie, 25.02.2018 o godzinie 17∶50 -0800, użytkownik Zac Medico

Re: [gentoo-portage-dev] [PATCH 1/2] portdbapi: add async_aux_get method (bug 648790)

2018-02-26 Thread Zac Medico
On 02/26/2018 04:29 AM, Michał Górny wrote: > W dniu nie, 25.02.2018 o godzinie 17∶50 -0800, użytkownik Zac Medico > napisał: >> Add async_aux_get method that returns a Future and otherwise >> behaves identically to aux_get. Use async_aux_get to implement >> the sy

Re: [gentoo-portage-dev] [PATCH 2/2] Add iter_completed convenience function (bug 648790)

2018-02-25 Thread Zac Medico
On 02/25/2018 07:17 PM, Alec Warner wrote: > > > On Sun, Feb 25, 2018 at 8:50 PM, Zac Medico <mailto:zmed...@gentoo.org>> wrote: > > The iter_completed function is similar to asyncio.as_completed, but > takes an iterator of futures as input, and includes sup

[gentoo-portage-dev] [PATCH 1/2] portdbapi: add async_aux_get method (bug 648790)

2018-02-25 Thread Zac Medico
Add async_aux_get method that returns a Future and otherwise behaves identically to aux_get. Use async_aux_get to implement the synchronous aux_get method. Bug: https://bugs.gentoo.org/648790 --- pym/portage/dbapi/porttree.py | 91 +-- 1 file changed, 70 in

[gentoo-portage-dev] [PATCH 2/2] Add iter_completed convenience function (bug 648790)

2018-02-25 Thread Zac Medico
The iter_completed function is similar to asyncio.as_completed, but takes an iterator of futures as input, and includes support for max_jobs and max_load parameters. The default values for max_jobs and max_load correspond to multiprocessing.cpu_count(). Example usage for async_aux_get: import p

Re: [gentoo-portage-dev] [PATCH] Deprecate EAPI 6_pre1

2018-02-25 Thread Zac Medico
On 02/25/2018 11:59 AM, Michał Górny wrote: > Deprecated the testing variant of EAPI 6. > --- > pym/portage/__init__.py | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/pym/portage/__init__.py b/pym/portage/__init__.py > index 99f3f98ac..4773738b2 100644 > --- a/pym/

Re: [gentoo-portage-dev] [PATCH v2] FreeBSD: use os.*chflags() instead of calling external tool

2018-02-22 Thread Zac Medico
On 02/22/2018 11:06 AM, Michał Górny wrote: > Use os.chflags() and os.lchflags() built-in functions instead of calling > external 'chflags' tool on FreeBSD. This fixes major performance > problems Portage has on FreeBSD. > > Bug: https://bugs.gentoo.org/648432 > --- > pym/portage/__init__.py | 50

Re: [gentoo-portage-dev] [PATCH] FreeBSD: use os.*chflags() instead of calling external tool

2018-02-22 Thread Zac Medico
On 02/22/2018 07:18 AM, Michał Górny wrote: > - @classmethod > - def lchflags(cls, path, flags): > - return cls.chflags(path, flags, opts='-h') > + @staticmethod > + def chflags(path, flags): > + return os.chfla

<    2   3   4   5   6   7   8   9   10   11   >