Re: [gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)
On Wed, Mar 8, 2017 at 10:48 AM, Brian Dolbec wrote: > On Sun, 5 Mar 2017 00:40:25 -0800 > Zac Medico wrote: > >> It's useful to automatically enable --with-bdeps so that @world >> updates will update all packages that are not eligible for removal by >> emerge --depclean. However, many users of binary packages do not want >> unnecessary build time dependencies installed, therefore do not >> auto-enable --with-bdeps for installation actions when the --usepkg >> option is enabled. >> >> A new --with-bdeps-auto= option is provided, making it possible >> to enable or disable the program logic that causes --with-bdeps to be >> automatically enabled. Use --with-bdeps-auto=n to prevent --with-bdeps >> from being automatically enabled for installation actions. This is >> useful for some rare cases in which --with-bdeps triggers unsolvable >> dependency conflicts (and putting --with-bdeps=n in >> EMERGE_DEFAULT_OPTS would cause undesirable --depclean behavior). >> >> X-Gentoo-bug: 598444 >> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=598444 >> --- >> [PATCH v2] adds a --with-bdeps-auto= option which is useful for >> some rare cases in which --with-bdeps triggers unsolvable dependency >> conflicts >> >> man/emerge.1 | 37 +++- >> pym/_emerge/create_depgraph_params.py| 5 + >> pym/_emerge/depgraph.py | 4 +- >> pym/_emerge/main.py | 5 + >> pym/portage/tests/resolver/ResolverPlayground.py | 5 + >> pym/portage/tests/resolver/test_bdeps.py | 215 >> +++ 6 files changed, 266 insertions(+), 5 >> deletions(-) create mode 100644 > > Looks fine, merge please :) Thanks merged: https://gitweb.gentoo.org/proj/portage.git/commit/?id=852c729bdef3d4c2e2d459a43dc21f0a05dfa2ba -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)
On Sun, 5 Mar 2017 00:40:25 -0800 Zac Medico wrote: > It's useful to automatically enable --with-bdeps so that @world > updates will update all packages that are not eligible for removal by > emerge --depclean. However, many users of binary packages do not want > unnecessary build time dependencies installed, therefore do not > auto-enable --with-bdeps for installation actions when the --usepkg > option is enabled. > > A new --with-bdeps-auto= option is provided, making it possible > to enable or disable the program logic that causes --with-bdeps to be > automatically enabled. Use --with-bdeps-auto=n to prevent --with-bdeps > from being automatically enabled for installation actions. This is > useful for some rare cases in which --with-bdeps triggers unsolvable > dependency conflicts (and putting --with-bdeps=n in > EMERGE_DEFAULT_OPTS would cause undesirable --depclean behavior). > > X-Gentoo-bug: 598444 > X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=598444 > --- > [PATCH v2] adds a --with-bdeps-auto= option which is useful for > some rare cases in which --with-bdeps triggers unsolvable dependency > conflicts > > man/emerge.1 | 37 +++- > pym/_emerge/create_depgraph_params.py| 5 + > pym/_emerge/depgraph.py | 4 +- > pym/_emerge/main.py | 5 + > pym/portage/tests/resolver/ResolverPlayground.py | 5 + > pym/portage/tests/resolver/test_bdeps.py | 215 > +++ 6 files changed, 266 insertions(+), 5 > deletions(-) create mode 100644 Looks fine, merge please :) -- Brian Dolbec
Re: [gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)
On 03/05/2017 11:44 AM, Michael Orlitzky wrote: > On 03/05/2017 02:12 PM, Zac Medico wrote: >> >> Incorrect. >> >> ... >> >> Incorrect. >> > > I see my mistakes, but maintain that this is confusing =) It could be worse, and I think it's questionable that we could do any better, given the nature of the problem. >> The --with-bdeps-auto option results in desirable behavior by default, >> and it's also backward compatible with existing --with-bdeps and >> --usepkg usage. It just does the right thing, minimizing the impact to >> existing emerge usage. > > I was hoping that since nothing breaks with --update-bdeps=y and > --clean-bdeps=n (the new defaults), we could just disable --with-bdeps > immediately and emit a warning when it's given. Breaking backward compatibility is too disruptive, unless we allow for a transition period where --with-bdeps is deprecated. >> There some problems with --update-bdeps/--clean-bdeps: >> >> * The program logic will be more complicated, since --with-bdeps will >> have to override both options for backward-compatibility with existing >> --with-bdeps usage. > > Not applicable if --with-bdeps goes away. We don't have a justification to break compatibility without a transition period. >> * The meaning of --clean-bdeps is confusing. Does --clean-bdeps=y mean >> to clean build time deps, or does it mean to pull build time deps into >> the dependency graph for removal operations? > > --clean-bdeps means clean the bdeps. It's confusing to have an option with inverted meaning relative to --with-bdeps. Cleaning the bdeps is a consequence of excluding them from the dependency graph, not an action in itself. --clean-bdeps sounds like an action. > I totally agree that the worst option of all is to keep --with-bdeps AND > introduce the other two. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)
On 03/05/2017 02:12 PM, Zac Medico wrote: > > Incorrect. > > ... > > Incorrect. > I see my mistakes, but maintain that this is confusing =) > > The --with-bdeps-auto option results in desirable behavior by default, > and it's also backward compatible with existing --with-bdeps and > --usepkg usage. It just does the right thing, minimizing the impact to > existing emerge usage. I was hoping that since nothing breaks with --update-bdeps=y and --clean-bdeps=n (the new defaults), we could just disable --with-bdeps immediately and emit a warning when it's given. > > There some problems with --update-bdeps/--clean-bdeps: > > * The program logic will be more complicated, since --with-bdeps will > have to override both options for backward-compatibility with existing > --with-bdeps usage. Not applicable if --with-bdeps goes away. > * The meaning of --clean-bdeps is confusing. Does --clean-bdeps=y mean > to clean build time deps, or does it mean to pull build time deps into > the dependency graph for removal operations? --clean-bdeps means clean the bdeps. I totally agree that the worst option of all is to keep --with-bdeps AND introduce the other two.
Re: [gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)
On 03/05/2017 07:08 AM, Michael Orlitzky wrote: > On 03/05/2017 03:40 AM, Zac Medico wrote: >> >> A new --with-bdeps-auto= option is provided, making it possible to >> enable or disable the program logic that causes --with-bdeps to be >> automatically enabled. Use --with-bdeps-auto=n to prevent --with-bdeps >> from being automatically enabled for installation actions. This is useful >> for some rare cases in which --with-bdeps triggers unsolvable dependency >> conflicts (and putting --with-bdeps=n in EMERGE_DEFAULT_OPTS would cause >> undesirable --depclean behavior). >> > > If I understand correctly, the end result of this is two --flags that > combine in a (rather complicated) way to let the user control the > default bdeps behavior of both "emerge --update" and "emerge --depclean" > separately. I'll try to summarize my understanding: > >I. emerge --update > > I.a. Some people want to --update the build dependencies they have > installed for e.g. security purposes. > > I.b. Others don't want build deps pulled in by "emerge --update", > because they're using binary packages or because it causes > conflicts in the resolver (llvm). > > II. emerge --depclean > > II.a. The default behavior is to not remove build-only dependencies > because, generally, they will just have to rebuilt again. > > II.b. However, some people want to remove the build-only deps that > aren't strictly in use -- particularly if those build-only > deps are not being updated by emerge --update. > > > To choose between those four options... > > i. --with-bdeps=n and --with-bdeps-auto=y gives you I.a. + II.b. Incorrect. These options give you I.b. + II.b. because --with-bdeps=n overrides --with-bdeps-auto settings (the override is backward-compatible with existing --with-bdeps usage). Since I.a. + II.b. is a contradictory combination, there is no combination of EMERGE_DEFAULT_OPTS that will give you I.a. + II.b. Specifying --with-bdeps-auto=y has no effect, since that's the default. > ii. --with-bdeps=n and --with-bdeps-auto=n gives you I.b. + II.b. Correct. Specifying --with-bdeps-auto=n has no effect, since --with-bdeps=n overrides --with-bdeps-auto settings (the override is backward-compatible with existing --with-bdeps usage). > iii. --with-bdeps=y and --with-bdeps-auto=y gives you I.a. + II.a. Correct. Specifying --with-bdeps-auto=y has no effect for 2 reasons: * --with-bdeps-auto=y is the default * --with-bdeps=y overrides --with-bdeps-auto settings (the override is backward-compatible with existing --with-bdeps usage). > iv. --with-bdeps=y and --with-bdeps-auto=n gives you I.a. + II.a. Correct. Specifying --with-bdeps-auto=n has no effect, since --with-bdeps=y overrides --with-bdeps-auto settings (the override is backward-compatible with existing --with-bdeps usage). > That only gets you three out of the four options. You have to read the > fine print to get the other: > > v. passing no flags explicitly gives you I.b. and II.a. Incorrect. Since --with-bdeps-auto=y is the the default setting, you get I.a. + II.a. unless --usepkg is enabled. If --usepkg is enabled, you get I.b. + II.a., which is backward-compatible with existing --usepkg usage. > If there's going to be two flags, can't we do better? Why not just name > the flags after what we want them to do: > > --update-bdeps= (default: y) > > --clean-bdeps= (default: n) The --with-bdeps-auto option results in desirable behavior by default, and it's also backward compatible with existing --with-bdeps and --usepkg usage. It just does the right thing, minimizing the impact to existing emerge usage. There some problems with --update-bdeps/--clean-bdeps: * The program logic will be more complicated, since --with-bdeps will have to override both options for backward-compatibility with existing --with-bdeps usage. * The meaning of --clean-bdeps is confusing. Does --clean-bdeps=y mean to clean build time deps, or does it mean to pull build time deps into the dependency graph for removal operations? > With those two options, it's easy to tell portage exactly what you want. > If I don't like the defaults, I can turn one of them off without > affecting the other. No, there are problems, as described above. > But what about the --usepkg magic? I think the workaround can be left > as-is. When --usepkg is enabled, switch the default for --update-bdeps > to "n" unless it is explicitly set. That would work, but introducing 2 new options is no really appealing, because it will lead to more complicated logic that will be more difficult to explain and for people to comprehend. I really don't want to spend time documenting the interactions between --with-bdeps, --update-bdes, and --clean-bdeps, and then get a ton of frequently asked questions from people having difficulty understanding it. Overall, I think the --with-bdeps-auto approach seems simpler and easi
Re: [gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)
On 03/05/2017 03:40 AM, Zac Medico wrote: > > A new --with-bdeps-auto= option is provided, making it possible to > enable or disable the program logic that causes --with-bdeps to be > automatically enabled. Use --with-bdeps-auto=n to prevent --with-bdeps > from being automatically enabled for installation actions. This is useful > for some rare cases in which --with-bdeps triggers unsolvable dependency > conflicts (and putting --with-bdeps=n in EMERGE_DEFAULT_OPTS would cause > undesirable --depclean behavior). > If I understand correctly, the end result of this is two --flags that combine in a (rather complicated) way to let the user control the default bdeps behavior of both "emerge --update" and "emerge --depclean" separately. I'll try to summarize my understanding: I. emerge --update I.a. Some people want to --update the build dependencies they have installed for e.g. security purposes. I.b. Others don't want build deps pulled in by "emerge --update", because they're using binary packages or because it causes conflicts in the resolver (llvm). II. emerge --depclean II.a. The default behavior is to not remove build-only dependencies because, generally, they will just have to rebuilt again. II.b. However, some people want to remove the build-only deps that aren't strictly in use -- particularly if those build-only deps are not being updated by emerge --update. To choose between those four options... i. --with-bdeps=n and --with-bdeps-auto=y gives you I.a. + II.b. ii. --with-bdeps=n and --with-bdeps-auto=n gives you I.b. + II.b. iii. --with-bdeps=y and --with-bdeps-auto=y gives you I.a. + II.a. iv. --with-bdeps=y and --with-bdeps-auto=n gives you I.a. + II.a. That only gets you three out of the four options. You have to read the fine print to get the other: v. passing no flags explicitly gives you I.b. and II.a. If there's going to be two flags, can't we do better? Why not just name the flags after what we want them to do: --update-bdeps= (default: y) --clean-bdeps= (default: n) With those two options, it's easy to tell portage exactly what you want. If I don't like the defaults, I can turn one of them off without affecting the other. But what about the --usepkg magic? I think the workaround can be left as-is. When --usepkg is enabled, switch the default for --update-bdeps to "n" unless it is explicitly set. Food for thought. Anyway, thanks for working on this.
[gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)
It's useful to automatically enable --with-bdeps so that @world updates will update all packages that are not eligible for removal by emerge --depclean. However, many users of binary packages do not want unnecessary build time dependencies installed, therefore do not auto-enable --with-bdeps for installation actions when the --usepkg option is enabled. A new --with-bdeps-auto= option is provided, making it possible to enable or disable the program logic that causes --with-bdeps to be automatically enabled. Use --with-bdeps-auto=n to prevent --with-bdeps from being automatically enabled for installation actions. This is useful for some rare cases in which --with-bdeps triggers unsolvable dependency conflicts (and putting --with-bdeps=n in EMERGE_DEFAULT_OPTS would cause undesirable --depclean behavior). X-Gentoo-bug: 598444 X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=598444 --- [PATCH v2] adds a --with-bdeps-auto= option which is useful for some rare cases in which --with-bdeps triggers unsolvable dependency conflicts man/emerge.1 | 37 +++- pym/_emerge/create_depgraph_params.py| 5 + pym/_emerge/depgraph.py | 4 +- pym/_emerge/main.py | 5 + pym/portage/tests/resolver/ResolverPlayground.py | 5 + pym/portage/tests/resolver/test_bdeps.py | 215 +++ 6 files changed, 266 insertions(+), 5 deletions(-) create mode 100644 pym/portage/tests/resolver/test_bdeps.py diff --git a/man/emerge.1 b/man/emerge.1 index 5b61220..7db4271 100644 --- a/man/emerge.1 +++ b/man/emerge.1 @@ -986,13 +986,44 @@ The default is set to "y" (on). .TP .BR "\-\-with\-bdeps < y | n >" In dependency calculations, pull in build time dependencies -that are not strictly required. This defaults to \'n\' for -installation actions, meaning they will not be installed, and -\'y\' for the \fB\-\-depclean\fR action, meaning they will not be removed. +that are not strictly required. This option is automatically enabled for +installation actions, meaning they will be installed, and defaults to +\(aqy\(aq for the \fB\-\-depclean\fR action, meaning they will not be +removed. In order to prevent the \fB\-\-with\-bdeps\fR option from being +automatically enabled for installation actions, specify +\fB\-\-with\-bdeps\-auto=n\fR in either the command line or +\fBEMERGE_DEFAULT_OPTS\fR. + +Since many users of binary packages do not want unnecessary build time +dependencies installed, this option is not automatically enabled for +installation actions when the \fB\-\-usepkg\fR option is enabled. In +order to pull in build time dependencies for binary packages with +\fB\-\-usepkg\fR, \fB\-\-with\-bdeps=y\fR must be specified explicitly. +This also applies to options that enable the \fB\-\-usepkg\fR option +implicitly, such as \fB\-\-getbinpkg\fR. + This setting can be added to \fBEMERGE_DEFAULT_OPTS\fR (see make.conf(5)) and later overridden via the command line. .TP +.BR "\-\-with\-bdeps\-auto < y | n >" +This option is used to enable or disable the program logic that causes +\fB\-\-with\-bdeps\fR is to be automatically enabled for installation +actions. This option is enabled by default. Use +\fB\-\-with\-bdeps\-auto=n\fR to prevent \fB\-\-with\-bdeps\fR from +being automatically enabled for installation actions. This setting can +be added to \fBEMERGE_DEFAULT_OPTS\fR (see make.conf(5)) and later +overridden via the command line. + +\fBNOTE:\fR The program logic that causes \fB\-\-with\-bdeps\fR to be +automatically enabled for installation actions does not affect removal +actions such as the \fB\-\-depclean\fR action. Therefore, when +\fB\-\-with\-bdeps\-auto=n\fR is specified in \fBEMERGE_DEFAULT_OPTS\fR, +it does not affect the default \fB\-\-with\-bdeps=y\fR setting that +applies to the \fB\-\-depclean\fR action. The default +\fB\-\-with\-bdeps=y\fR setting that applies to the \fB\-\-depclean\fR +action can be overridden only by specifying \fB\-\-with\-bdeps=n\fR. +.TP .BR "\-\-with\-test\-deps [ y | n ]" For packages matched by arguments, this option will pull in dependencies that are conditional on the "test" USE flag, even if "test" is not diff --git a/pym/_emerge/create_depgraph_params.py b/pym/_emerge/create_depgraph_params.py index 2c64928..cdea029 100644 --- a/pym/_emerge/create_depgraph_params.py +++ b/pym/_emerge/create_depgraph_params.py @@ -13,6 +13,8 @@ def create_depgraph_params(myopts, myaction): # deep: go into the dependencies of already merged packages # empty: pretend nothing is merged # complete: completely account for all known dependencies + # bdeps: satisfy build time dependencies of packages that are + # already built, even though they are not strictly required # remove:build graph for use in removing packages # rebuilt_binaries: replace installed packages with rebuilt binaries # rebuild_i