Re: [gentoo-portage-dev] [PATCH] emerge: add --onlydeps-with-rdeps= option (bug 294719)
On Sun, May 14, 2017 at 6:30 AM, Brian Dolbec wrote: > On Thu, 11 May 2017 12:01:18 -0700 > Zac Medico wrote: > > > From: William Throwe > > > > Add --onlydeps-with-rdeps=n option in order to omit pure > > run-time dependencies with --onlydeps. The dependencies > > that get pulled in are those that are necessary for the > > equivalent --buildpkgonly command to succeed. The default > > --onlydeps behavior remains unchanged. > > > > X-Gentoo-bug: 294719 > > X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=294719 > > --- > > man/emerge.1 | 5 +++ > > pym/_emerge/depgraph.py| 5 +++ > > pym/_emerge/main.py| 6 +++ > > .../tests/resolver/test_onlydeps_minimal.py| 47 > > ++ 4 files changed, 63 insertions(+) > > create mode 100644 > > pym/portage/tests/resolver/test_onlydeps_minimal.py > > looks fine Thanks, merged: https://gitweb.gentoo.org/proj/portage.git/commit/?id=26a8ea6889190b6a0ec110f76fa49834d53771e9 -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH 2/2] emerge: default --backtrack=10 (bug 540562)
On Sun, May 14, 2017 at 6:37 AM, Brian Dolbec wrote:
> On Tue, 9 May 2017 20:52:35 -0700
> Zac Medico wrote:
>
> > Since the default behavior is now for emerge to terminate early for
> > autounmask changes (unless either --autounmask-backtrack=y or
> > --autounmask-continue is enabled), it is much less likely that time
> > will be wasted by fruitless backtracking. Therefore, raise the default
> > backtrack value from 3 to 10, restoring it to the value it had prior
> > to commit 1891388ea0ae0dd58903a71a3adc779731523601 (see bug 536926).
> > This will allow many users to avoid having to manually raise the
> > --backtrack value.
> >
> > X-Gentoo-bug: 540562
> > X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=540562
> > ---
> > man/emerge.1| 2 +-
> > pym/_emerge/depgraph.py | 2 +-
> > 2 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/man/emerge.1 b/man/emerge.1
> > index 94edc90..240d4de 100644
> > --- a/man/emerge.1
> > +++ b/man/emerge.1
> > @@ -412,7 +412,7 @@ precedence over existing changes. This option is
> > automatically enabled with .BR \-\-backtrack=COUNT
> > Specifies an integer number of times to backtrack if
> > dependency calculation fails due to a conflict or an
> > -unsatisfied dependency (default: \'3\').
> > +unsatisfied dependency (default: \'10\').
> > .TP
> > .BR "\-\-binpkg\-changed\-deps [ y | n ]"
> > Tells emerge to ignore binary packages for which the corresponding
> > diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
> > index 53910dd..cda497b 100644
> > --- a/pym/_emerge/depgraph.py
> > +++ b/pym/_emerge/depgraph.py
> > @@ -9460,7 +9460,7 @@ def _backtrack_depgraph(settings, trees,
> > myopts, myparams, myaction, myfiles, sp
> > debug = "--debug" in myopts
> > mydepgraph = None
> > - max_retries = myopts.get('--backtrack', 3)
> > + max_retries = myopts.get('--backtrack', 10)
> > max_depth = max(1, (max_retries + 1) // 2)
> > allow_backtracking = max_retries > 0
> > backtracker = Backtracker(max_depth)
>
> yeah, is good :)
>
Thanks, merged:
https://gitweb.gentoo.org/proj/portage.git/commit/?id=cbf9f3c0167a68de7901b2a0992c2d0a264b9b22
--
Thanks,
Zac
Re: [gentoo-portage-dev] [PATCH 1/2] emerge: terminate backtracking early for autounmask changes (bug 615680)
On Sun, May 14, 2017 at 6:38 AM, Brian Dolbec wrote: > On Tue, 9 May 2017 20:52:34 -0700 > Zac Medico wrote: > > > Since autounmask changes are a strong indicator that backtracking > > will ultimately fail to produce a solution, terminate early for > > autounmask changes, and add a --autounmask-backtrack= option > > to modify this behavior. The --autounmask-continue option implies > > --autounmask-backtrack=y behavior, for backward compatibility. > > > > When backtracking terminates early, the following warning message > > is displayed after the autounmask change(s): > > > > * In order to avoid wasting time, backtracking has terminated early > > * due to the above autounmask change(s). The --autounmask-backtrack=y > > * option can be used to force further backtracking, but there is no > > * guarantee that it will produce a solution. > > > > With this change, five of the existing cases fail unless > > --autounmask-backtrack=y is added to the options. For each of > > these cases, comments below the test case document how it behaves > > with and without --autounmask-backtrack=y enabled. > > > > X-Gentoo-bug: 615680 > > X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=615680 > > --- > > man/emerge.1 | 10 ++- > > pym/_emerge/depgraph.py| 80 > > ++ > > pym/_emerge/main.py| 6 ++ > > pym/portage/tests/resolver/test_autounmask.py | 57 > > ++- .../tests/resolver/test_autounmask_use_breakage.py | > > 40 +++ .../test_slot_conflict_unsatisfied_deep_deps.py| > > 61 + 6 files changed, 237 insertions(+), 17 > > deletions(-) > > > > Looks good, thank you :) > Thanks, merged: https://gitweb.gentoo.org/proj/portage.git/commit/?id=40505ceeadc769f4f01c66e52a19ce0bf2f59761 -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH] file_copy: fix lseek offset after EINTR (bug 618086)
On Sun, May 14, 2017 at 6:31 AM, Brian Dolbec wrote:
> On Tue, 9 May 2017 23:59:34 -0700
> Zac Medico wrote:
>
> > Fix the lseek offset for the plain read/write loop to account
> > for buffered data that has not been written to to the output
> > file yet (due to previous interruption by EINTR). This code
> > only affects Linux 2.6.32 and earlier (newer kernels use
> > copy_file_range or sendfile).
> >
> > X-Gentoo-bug: 618086
> > X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=618086
> > ---
> > src/portage_util_file_copy_reflink_linux.c | 13 -
> > 1 file changed, 8 insertions(+), 5 deletions(-)
> >
> > diff --git a/src/portage_util_file_copy_reflink_linux.c
> > b/src/portage_util_file_copy_reflink_linux.c index 2fb17a0..4be9e05
> > 100644 --- a/src/portage_util_file_copy_reflink_linux.c
> > +++ b/src/portage_util_file_copy_reflink_linux.c
> > @@ -323,12 +323,14 @@ _reflink_linux_file_copy(PyObject *self,
> > PyObject *args) if (buf == NULL) {
> > error = errno;
> >
> > -/* For the read call, the fd_in file offset must be
> > - * exactly equal to offset_out. Use lseek to ensure
> > - * correct state, in case an EINTR retry caused it to
> > - * get out of sync somewhow.
> > +/* For the read call, the fd_in file offset must be
> > exactly
> > + * equal to offset_out + buf_bytes, where buf_bytes
> > is the
> > + * amount of buffered data that has not been written
> > to
> > + * to the output file yet. Use lseek to ensure
> > correct state,
> > + * in case an EINTR retry caused it to get out of
> > sync
> > + * somewhow.
> > */
> > -} else if (lseek(fd_in, offset_out, SEEK_SET) < 0) {
> > +} else if (lseek(fd_in, offset_out + buf_bytes,
> > SEEK_SET) < 0) { error = errno;
> > } else {
> > while (1) {
> > @@ -345,6 +347,7 @@ _reflink_linux_file_copy(PyObject *self, PyObject
> > *args)
> > } else if (buf_bytes < 0) {
> > error = errno;
> > +buf_bytes = 0;
> > break;
> > }
> > }
>
> looks fine :)
>
Thanks, merged:
https://gitweb.gentoo.org/proj/portage.git/commit/?id=a3c87da25dc1944e6b7720e304318ae5474dff95
--
Thanks,
Zac
Re: [gentoo-portage-dev] [PATCH 1/2] emerge: terminate backtracking early for autounmask changes (bug 615680)
On Tue, 9 May 2017 20:52:34 -0700
Zac Medico wrote:
> Since autounmask changes are a strong indicator that backtracking
> will ultimately fail to produce a solution, terminate early for
> autounmask changes, and add a --autounmask-backtrack= option
> to modify this behavior. The --autounmask-continue option implies
> --autounmask-backtrack=y behavior, for backward compatibility.
>
> When backtracking terminates early, the following warning message
> is displayed after the autounmask change(s):
>
> * In order to avoid wasting time, backtracking has terminated early
> * due to the above autounmask change(s). The --autounmask-backtrack=y
> * option can be used to force further backtracking, but there is no
> * guarantee that it will produce a solution.
>
> With this change, five of the existing cases fail unless
> --autounmask-backtrack=y is added to the options. For each of
> these cases, comments below the test case document how it behaves
> with and without --autounmask-backtrack=y enabled.
>
> X-Gentoo-bug: 615680
> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=615680
> ---
> man/emerge.1 | 10 ++-
> pym/_emerge/depgraph.py| 80
> ++
> pym/_emerge/main.py| 6 ++
> pym/portage/tests/resolver/test_autounmask.py | 57
> ++- .../tests/resolver/test_autounmask_use_breakage.py |
> 40 +++ .../test_slot_conflict_unsatisfied_deep_deps.py|
> 61 + 6 files changed, 237 insertions(+), 17
> deletions(-)
>
> diff --git a/man/emerge.1 b/man/emerge.1
> index f1a9d4f..94edc90 100644
> --- a/man/emerge.1
> +++ b/man/emerge.1
> @@ -363,12 +363,20 @@ the specified configuration file(s), or enable
> the \fBEMERGE_DEFAULT_OPTS\fR variable may be used to
> disable this option by default in \fBmake.conf\fR(5).
> .TP
> +.BR "\-\-autounmask\-backtrack < y | n >"
> +Allow backtracking after autounmask has detected that
> +configuration changes are necessary. This option is not
> +recommended, since it can cause a large amount of time to
> +be wasted by backtracking calculations, even though there
> +is no guarantee that it will produce a solution. This
> +option is disabled by default.
> +.TP
> .BR "\-\-autounmask\-continue [ y | n ]"
> Automatically apply autounmask changes to configuration
> files, and continue to execute the specified command. If
> the dependency calculation is not entirely successful, then
> emerge will simply abort without modifying any configuration
> -files.
> +files. This option implies \fB\-\-autounmask\-backtrack=y\fR.
> \fBWARNING:\fR
> This option is intended to be used only with great caution,
> since it is possible for it to make nonsensical configuration
> diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
> index e1119af..53910dd 100644
> --- a/pym/_emerge/depgraph.py
> +++ b/pym/_emerge/depgraph.py
> @@ -444,6 +444,7 @@ class _dynamic_depgraph_config(object):
> self._autounmask =
> depgraph._frozen_config.myopts.get('--autounmask') != 'n'
> self._displayed_autounmask = False self._success_without_autounmask =
> False
> + self._autounmask_backtrack_disabled = False
> self._required_use_unsatisfied = False
> self._traverse_ignored_deps = False
> self._complete_mode = False
> @@ -1129,7 +1130,8 @@ class depgraph(object):
>
> self._show_merge_list()
>
> - self._dynamic_config._slot_conflict_handler =
> slot_conflict_handler(self)
> + if self._dynamic_config._slot_conflict_handler is
> None:
> + self._dynamic_config._slot_conflict_handler
> = slot_conflict_handler(self) handler =
> self._dynamic_config._slot_conflict_handler
> conflict = handler.get_conflict()
> @@ -4243,17 +4245,7 @@ class depgraph(object):
> # set below is reserved for cases where there are
> *zero* other # problems. For reference, see backtrack_depgraph, where
> it skips the # get_best_run() call when success_without_autounmask is
> True. -
> - digraph_nodes = self._dynamic_config.digraph.nodes
> -
> - if any(x in digraph_nodes for x in
> -
> self._dynamic_config._needed_unstable_keywords) or \
> - any(x in digraph_nodes for x in
> - self._dynamic_config._needed_p_mask_changes)
> or \
> - any(x in digraph_nodes for x in
> -
> self._dynamic_config._needed_use_config_changes) or \
> - any(x in digraph_nodes for x in
> -
> self._dynamic_config._needed_license_changes) :
> + if self._have_autounmask_changes():
> #We failed if the user needs to change the
> configuration self._dynamic_config._success_without_autounmask = True
> if
> (self._frozen_config.myopts.get("--autounmask-continue") is True and
> @@ -8564,6 +8556,17
Re: [gentoo-portage-dev] [PATCH 2/2] emerge: default --backtrack=10 (bug 540562)
On Tue, 9 May 2017 20:52:35 -0700
Zac Medico wrote:
> Since the default behavior is now for emerge to terminate early for
> autounmask changes (unless either --autounmask-backtrack=y or
> --autounmask-continue is enabled), it is much less likely that time
> will be wasted by fruitless backtracking. Therefore, raise the default
> backtrack value from 3 to 10, restoring it to the value it had prior
> to commit 1891388ea0ae0dd58903a71a3adc779731523601 (see bug 536926).
> This will allow many users to avoid having to manually raise the
> --backtrack value.
>
> X-Gentoo-bug: 540562
> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=540562
> ---
> man/emerge.1| 2 +-
> pym/_emerge/depgraph.py | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/man/emerge.1 b/man/emerge.1
> index 94edc90..240d4de 100644
> --- a/man/emerge.1
> +++ b/man/emerge.1
> @@ -412,7 +412,7 @@ precedence over existing changes. This option is
> automatically enabled with .BR \-\-backtrack=COUNT
> Specifies an integer number of times to backtrack if
> dependency calculation fails due to a conflict or an
> -unsatisfied dependency (default: \'3\').
> +unsatisfied dependency (default: \'10\').
> .TP
> .BR "\-\-binpkg\-changed\-deps [ y | n ]"
> Tells emerge to ignore binary packages for which the corresponding
> diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
> index 53910dd..cda497b 100644
> --- a/pym/_emerge/depgraph.py
> +++ b/pym/_emerge/depgraph.py
> @@ -9460,7 +9460,7 @@ def _backtrack_depgraph(settings, trees,
> myopts, myparams, myaction, myfiles, sp
> debug = "--debug" in myopts
> mydepgraph = None
> - max_retries = myopts.get('--backtrack', 3)
> + max_retries = myopts.get('--backtrack', 10)
> max_depth = max(1, (max_retries + 1) // 2)
> allow_backtracking = max_retries > 0
> backtracker = Backtracker(max_depth)
yeah, is good :)
--
Brian Dolbec
Re: [gentoo-portage-dev] [PATCH] file_copy: fix lseek offset after EINTR (bug 618086)
On Tue, 9 May 2017 23:59:34 -0700
Zac Medico wrote:
> Fix the lseek offset for the plain read/write loop to account
> for buffered data that has not been written to to the output
> file yet (due to previous interruption by EINTR). This code
> only affects Linux 2.6.32 and earlier (newer kernels use
> copy_file_range or sendfile).
>
> X-Gentoo-bug: 618086
> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=618086
> ---
> src/portage_util_file_copy_reflink_linux.c | 13 -
> 1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/src/portage_util_file_copy_reflink_linux.c
> b/src/portage_util_file_copy_reflink_linux.c index 2fb17a0..4be9e05
> 100644 --- a/src/portage_util_file_copy_reflink_linux.c
> +++ b/src/portage_util_file_copy_reflink_linux.c
> @@ -323,12 +323,14 @@ _reflink_linux_file_copy(PyObject *self,
> PyObject *args) if (buf == NULL) {
> error = errno;
>
> -/* For the read call, the fd_in file offset must be
> - * exactly equal to offset_out. Use lseek to ensure
> - * correct state, in case an EINTR retry caused it to
> - * get out of sync somewhow.
> +/* For the read call, the fd_in file offset must be
> exactly
> + * equal to offset_out + buf_bytes, where buf_bytes
> is the
> + * amount of buffered data that has not been written
> to
> + * to the output file yet. Use lseek to ensure
> correct state,
> + * in case an EINTR retry caused it to get out of
> sync
> + * somewhow.
> */
> -} else if (lseek(fd_in, offset_out, SEEK_SET) < 0) {
> +} else if (lseek(fd_in, offset_out + buf_bytes,
> SEEK_SET) < 0) { error = errno;
> } else {
> while (1) {
> @@ -345,6 +347,7 @@ _reflink_linux_file_copy(PyObject *self, PyObject
> *args)
> } else if (buf_bytes < 0) {
> error = errno;
> +buf_bytes = 0;
> break;
> }
> }
looks fine :)
--
Brian Dolbec
Re: [gentoo-portage-dev] [PATCH] emerge: add --onlydeps-with-rdeps= option (bug 294719)
On Thu, 11 May 2017 12:01:18 -0700
Zac Medico wrote:
> From: William Throwe
>
> Add --onlydeps-with-rdeps=n option in order to omit pure
> run-time dependencies with --onlydeps. The dependencies
> that get pulled in are those that are necessary for the
> equivalent --buildpkgonly command to succeed. The default
> --onlydeps behavior remains unchanged.
>
> X-Gentoo-bug: 294719
> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=294719
> ---
> man/emerge.1 | 5 +++
> pym/_emerge/depgraph.py| 5 +++
> pym/_emerge/main.py| 6 +++
> .../tests/resolver/test_onlydeps_minimal.py| 47
> ++ 4 files changed, 63 insertions(+)
> create mode 100644
> pym/portage/tests/resolver/test_onlydeps_minimal.py
>
> diff --git a/man/emerge.1 b/man/emerge.1
> index f1a9d4f..c172c5e 100644
> --- a/man/emerge.1
> +++ b/man/emerge.1
> @@ -716,6 +716,11 @@ it possible for \fB\-\-deep\fR to be disabled by
> default. Only merge (or pretend to merge) the dependencies of the
> packages specified, not the packages themselves.
> .TP
> +.BR "\-\-onlydeps\-with\-rdeps < y | n >"
> +Include run time dependencies when \fB\-\-onlydeps\fR is specified.
> +When this is disabled only build time dependencies are included. This
> +option is enabled by default.
> +.TP
> .BR "\-\-package\-moves [ y | n ]"
> Perform package moves when necessary. This option is enabled
> by default. Package moves are typically applied immediately
> diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
> index e1119af..a76a196 100644
> --- a/pym/_emerge/depgraph.py
> +++ b/pym/_emerge/depgraph.py
> @@ -3064,6 +3064,11 @@ class depgraph(object):
> edepend["RDEPEND"] = ""
> edepend["PDEPEND"] = ""
>
> + if pkg.onlydeps and \
> +
> self._frozen_config.myopts.get("--onlydeps-with-rdeps") == 'n':
> + edepend["RDEPEND"] = ""
> + edepend["PDEPEND"] = ""
> +
> ignore_build_time_deps = False
> if pkg.built and not removal_action:
> if
> self._dynamic_config.myparams.get("bdeps") in ("y", "auto"): diff
> --git a/pym/_emerge/main.py b/pym/_emerge/main.py index
> 76e963a..fda0323 100644 --- a/pym/_emerge/main.py
> +++ b/pym/_emerge/main.py
> @@ -147,6 +147,7 @@ def insert_optional_args(args):
> '--jobs' : valid_integers,
> '--keep-going' : y_or_n,
> '--load-average' : valid_floats,
> + '--onlydeps-with-rdeps' : y_or_n,
> '--package-moves': y_or_n,
> '--quiet': y_or_n,
> '--quiet-build' : y_or_n,
> @@ -559,6 +560,11 @@ def parse_opts(tmpcmdline, silent=False):
> "action" : "append",
> },
>
> + "--onlydeps-with-rdeps": {
> + "help": "modify interpretation of
> depedencies",
> + "choices" : true_y_or_n
> + },
> +
> "--rebuild-exclude": {
> "help" :"A space separated list of package
> names or slot atoms. " + \ "Emerge will not rebuild these packages
> due to the " + \ diff --git
> a/pym/portage/tests/resolver/test_onlydeps_minimal.py
> b/pym/portage/tests/resolver/test_onlydeps_minimal.py new file mode
> 100644 index 000..13c79ed --- /dev/null
> +++ b/pym/portage/tests/resolver/test_onlydeps_minimal.py
> @@ -0,0 +1,47 @@
> +# Copyright 2017 Gentoo Foundation
> +# Distributed under the terms of the GNU General Public License v2
> +
> +from portage.tests import TestCase
> +from portage.tests.resolver.ResolverPlayground import
> ResolverPlayground, ResolverPlaygroundTestCase +
> +class OnlydepsMinimalTestCase(TestCase):
> +
> + def testOnlydepsMinimal(self):
> + ebuilds = {
> + "dev-libs/A-1": { "DEPEND": "dev-libs/B",
> + "RDEPEND": "dev-libs/C",
> + "PDEPEND": "dev-libs/D" },
> + "dev-libs/B-1": { },
> + "dev-libs/C-1": { },
> + "dev-libs/D-1": { },
> + }
> + installed = {
> + }
> +
> + test_cases = (
> + ResolverPlaygroundTestCase(
> + ["dev-libs/A"],
> + all_permutations = True,
> + success = True,
> + options = { "--onlydeps": True,
> + "--onlydeps-with-rdeps":
> "y" },
> + mergelist = ["dev-libs/B-1",
> + "dev-libs/C-1",
> + "dev-libs/D-1"]),
> + ResolverPlaygroundTestC
