Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-22 Thread Zac Medico
On 03/21/2018 04:30 PM, M. J. Everitt wrote:
> On 21/03/18 23:26, M. J. Everitt wrote:
>> n 20/03/18 04:49, Manuel Rüger wrote:
>>> Hi Zac,
>>>
>>> alternatively could --exclude be extended to support sets?
>>> So users could --exclude @world or @profile.
>>>
>>> Cheers,
>>> Manuel
>>>
>> The idea is sound enough, but I fear the syntax would be too confusing.
>>
>> Reading a potential command-line as "emerge  > world-file unless --no-replace specified>  puts my head into
>> a spin! I can see it might be clearer for an unmerge perhaps ..
>>
>> Unless I'm missing something fundamental ...
>>
> On a related note, it would be quite handy to enumerate an
> "exclude-from" option like rsync/tar(?) to specify a file with the
> --exclude options in. There could be some way to append @world to this
> option, perhaps?

All that --ignore-world does it remove the implicit @world root the
dependency graph, so --complete-graph options no longer pull @world and
its deep dependencies into the dependency graph. It's still possible to
use @world together with --ignore-world, but in this case --ignore-world
has no effect because @world has been pulled in explicitly (and pulling
@world in implicitly would be redundant in this case).
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-22 Thread Zac Medico
On 03/21/2018 05:59 PM, Manuel Rüger wrote:
> On 22.03.2018 01:25, Zac Medico wrote:
>> On 03/19/2018 09:49 PM, Manuel Rüger wrote:
>>> Hi Zac,
>>>
>>> alternatively could --exclude be extended to support sets?
>>> So users could --exclude @world or @profile.
>>
>> Your idea doesn't really fit the current meaning of --exclude, since
>> --exclude excludes packages from being merged, but still adds installed
>> instances to the dependency graph in order to ensure that their
>> dependencies remain satisfied.
>>
> Thanks for providing the clarification, now I have a better
> understanding what both approaches do and withdraw my suggestion for
> this patch. :-)

Ok, glad to clarify. Thinking some more on the implications of your
question, it seems like you were thinking that packages matched by the
excluded set would somehow be magically eliminated from the dependency
graph? That's not how --ignore-world works at all. Things matched by
@world, @selected, @system, and their deep dependencies can still be
pulled into the dependency graph despite --ignore-world. The only
difference with --ignore-world is that @world is no longer an implicit
member of the dependency graph, so --complete-graph options will not
force @world into the dependency graph, and the packages given as
arguments will be the only root(s) of the dependency graph.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Manuel Rüger
On 22.03.2018 01:25, Zac Medico wrote:
> On 03/19/2018 09:49 PM, Manuel Rüger wrote:
>> Hi Zac,
>>
>> alternatively could --exclude be extended to support sets?
>> So users could --exclude @world or @profile.
> 
> Your idea doesn't really fit the current meaning of --exclude, since
> --exclude excludes packages from being merged, but still adds installed
> instances to the dependency graph in order to ensure that their
> dependencies remain satisfied.
> 
Thanks for providing the clarification, now I have a better
understanding what both approaches do and withdraw my suggestion for
this patch. :-)

> I'd question the usefulness of a finer-grained approach that you're
> suggesting. I don't foresee people wanting to fiddle around with which
> package sets they want to ignore, and I wouldn't encourage them to do so.
> 
> The intention of the --ignore-world option is to say, "I only care about
> the packages that I'm specifying in the emerge arguments, do anything
> necessary to install them." In this sort of situation, I think a person
> generally wants to ignore everything except the given packages and their
> dependencies, because they don't want to do a bunch of fiddling to
> figure out which sets they'd need to exclude in order to avoid
> conflicts. If they want to fiddle with something, they are free to
> adjust their package set configuration, so why wouldn't they?
> 
> Anyway, I'm not necessarily opposed to adding a finer grained
> --ignore-set option. However, it would be more work, it would be more
> complex, and I wouldn't advise anyone to use it.
> 
> If people want to automate something in a disposable system, or they're
> in a position to use --ask and check the result for sanity, then I think
> --ignore-world is a good solution.
> 
> If people want something that's safe to use on a production system, then
> I'll advise them to manually adjust their package set configuration.
> 



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Zac Medico
On 03/21/2018 04:53 PM, Joakim Tjernlund wrote:
> On Tue, 2018-03-20 at 05:49 +0100, Manuel Rüger wrote:
>> Hi Zac,
>>
>> alternatively could --exclude be extended to support sets?
>> So users could --exclude @world or @profile.
> 
> Yes please, I think I have a bug in that direction already(and --exclude 
> during --depclean)

Bug 634092 specified -C which is a synonym for --unmerge, so you need to
decide which one it is. The --unmerge code doesn't deal with
dependencies, so it's an entirely separate feature. You're "emerge -e
@world --exclude @system" suggestion would be another entirely separate
feature, you need to file separate bugs for each one.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Zac Medico
On 03/19/2018 09:49 PM, Manuel Rüger wrote:
> Hi Zac,
> 
> alternatively could --exclude be extended to support sets?
> So users could --exclude @world or @profile.

Your idea doesn't really fit the current meaning of --exclude, since
--exclude excludes packages from being merged, but still adds installed
instances to the dependency graph in order to ensure that their
dependencies remain satisfied.

I'd question the usefulness of a finer-grained approach that you're
suggesting. I don't foresee people wanting to fiddle around with which
package sets they want to ignore, and I wouldn't encourage them to do so.

The intention of the --ignore-world option is to say, "I only care about
the packages that I'm specifying in the emerge arguments, do anything
necessary to install them." In this sort of situation, I think a person
generally wants to ignore everything except the given packages and their
dependencies, because they don't want to do a bunch of fiddling to
figure out which sets they'd need to exclude in order to avoid
conflicts. If they want to fiddle with something, they are free to
adjust their package set configuration, so why wouldn't they?

Anyway, I'm not necessarily opposed to adding a finer grained
--ignore-set option. However, it would be more work, it would be more
complex, and I wouldn't advise anyone to use it.

If people want to automate something in a disposable system, or they're
in a position to use --ask and check the result for sanity, then I think
--ignore-world is a good solution.

If people want something that's safe to use on a production system, then
I'll advise them to manually adjust their package set configuration.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Joakim Tjernlund
On Tue, 2018-03-20 at 05:49 +0100, Manuel Rüger wrote:
> Hi Zac,
> 
> alternatively could --exclude be extended to support sets?
> So users could --exclude @world or @profile.

Yes please, I think I have a bug in that direction already(and --exclude during 
--depclean)


> 
> Cheers,
> Manuel
> 
> On 22.03.2018 00:03, Zac Medico wrote:
> > Ignore the @world package set and its dependencies. This may be useful
> > if there is a desire to perform an action even though it might break
> > the dependencies of some installed packages (it might also remove
> > installed packages in order to solve blockers). This also alters the
> > behavior of --complete-graph options so that only deep dependencies
> > of packages given as arguments are included in the dependency graph.
> > This option may be useful as an alternative to --nodeps in cases where
> > it is desirable to account for dependencies of packages given as
> > arguments.
> > 
> > Bug: https://bugs.gentoo.org/608564
> > ---
> >  man/emerge.1  | 17 +
> >  pym/_emerge/create_depgraph_params.py |  4 
> >  pym/_emerge/depgraph.py   |  8 ++--
> >  pym/_emerge/main.py   |  9 +
> >  pym/portage/tests/resolver/test_complete_graph.py | 18 ++
> >  5 files changed, 54 insertions(+), 2 deletions(-)
> > 
> > diff --git a/man/emerge.1 b/man/emerge.1
> > index a17b65ed2..01ce62e51 100644
> > --- a/man/emerge.1
> > +++ b/man/emerge.1
> > @@ -630,6 +630,23 @@ Therefore, \fB\-\-usepkgonly\fR (or 
> > \fB\-\-getbinpkgonly\fR) must be
> >  used in order to enable soname depedency resolution when installing
> >  packages.
> >  .TP
> > +.BR "\-\-ignore\-world [ y | n ]"
> > +Ignore the @world package set and its dependencies. This may be useful
> > +if there is a desire to perform an action even though it might break
> > +the dependencies of some installed packages (it might also remove
> > +installed packages in order to solve blockers). This also alters the
> > +behavior of \fB\-\-complete\-graph\fR options so that only deep
> > +dependencies of packages given as arguments are included in the
> > +dependency graph. This option may be useful as an alternative to
> > +\fB\-\-nodeps\fR in cases where it is desirable to account for
> > +dependencies of packages given as arguments.
> > +
> > +\fBWARNING:\fR
> > +This option is intended to be used only with great caution, since it is
> > +possible for it to make nonsensical changes which may lead to system
> > +breakage. Therefore, it is advisable to use \fB\-\-ask\fR together with
> > +this option.
> > +.TP
> >  .BR \-j\ [JOBS] ", "  \-\-jobs[=JOBS]
> >  Specifies the number of packages to build simultaneously. If this option is
> >  given without an argument, emerge will not limit the number of jobs that 
> > can
> > diff --git a/pym/_emerge/create_depgraph_params.py 
> > b/pym/_emerge/create_depgraph_params.py
> > index fc7fa60d7..0405011fd 100644
> > --- a/pym/_emerge/create_depgraph_params.py
> > +++ b/pym/_emerge/create_depgraph_params.py
> > @@ -26,6 +26,7 @@ def create_depgraph_params(myopts, myaction):
> >   # ignore_soname_deps: ignore the soname dependencies of built
> >   #   packages, so that they do not trigger dependency resolution
> >   #   failures, or cause packages to be rebuilt or replaced.
> > + # ignore_world: ignore the @world package set and its dependencies
> >   # with_test_deps: pull in test deps for packages matched by arguments
> >   # changed_deps: rebuild installed packages with outdated deps
> >   # changed_deps_report: report installed packages with outdated deps
> > @@ -56,6 +57,9 @@ def create_depgraph_params(myopts, myaction):
> >   myparams["selective"] = True
> >   return myparams
> >  
> > + if myopts.get('--ignore-world') is True:
> > + myparams['ignore_world'] = True
> > +
> >   rebuild_if_new_slot = myopts.get('--rebuild-if-new-slot')
> >   if rebuild_if_new_slot is not None:
> >   myparams['rebuild_if_new_slot'] = rebuild_if_new_slot
> > diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
> > index 6c728684f..f7ea27c37 100644
> > --- a/pym/_emerge/depgraph.py
> > +++ b/pym/_emerge/depgraph.py
> > @@ -163,7 +163,10 @@ class _frozen_depgraph_config(object):
> >   self.trees[myroot]["bintree"] = DummyTree(
> >   
> > DbapiProvidesIndex(trees[myroot]["bintree"].dbapi))
> >  
> > - self._required_set_names = set(["world"])
> > + if params.get("ignore_world", False):
> > + self._required_set_names = set()
> > + else:
> > + self._required_set_names = set(["world"])
> >  
> >   atoms = ' '.join(myopts.get("--exclude", [])).split()
> >   self.excluded_pkgs = _wildcard_set(atoms)
> > @@ -7554,6 

Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread M. J. Everitt
On 21/03/18 23:26, M. J. Everitt wrote:
> n 20/03/18 04:49, Manuel Rüger wrote:
>> Hi Zac,
>>
>> alternatively could --exclude be extended to support sets?
>> So users could --exclude @world or @profile.
>>
>> Cheers,
>> Manuel
>>
> The idea is sound enough, but I fear the syntax would be too confusing.
>
> Reading a potential command-line as "emerge   world-file unless --no-replace specified>  puts my head into
> a spin! I can see it might be clearer for an unmerge perhaps ..
>
> Unless I'm missing something fundamental ...
>
On a related note, it would be quite handy to enumerate an
"exclude-from" option like rsync/tar(?) to specify a file with the
--exclude options in. There could be some way to append @world to this
option, perhaps?

MJE



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread M. J. Everitt
n 20/03/18 04:49, Manuel Rüger wrote:
> Hi Zac,
>
> alternatively could --exclude be extended to support sets?
> So users could --exclude @world or @profile.
>
> Cheers,
> Manuel
>
The idea is sound enough, but I fear the syntax would be too confusing.

Reading a potential command-line as "emergeputs my head into
a spin! I can see it might be clearer for an unmerge perhaps ..

Unless I'm missing something fundamental ...



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Manuel Rüger
Hi Zac,

alternatively could --exclude be extended to support sets?
So users could --exclude @world or @profile.

Cheers,
Manuel

On 22.03.2018 00:03, Zac Medico wrote:
> Ignore the @world package set and its dependencies. This may be useful
> if there is a desire to perform an action even though it might break
> the dependencies of some installed packages (it might also remove
> installed packages in order to solve blockers). This also alters the
> behavior of --complete-graph options so that only deep dependencies
> of packages given as arguments are included in the dependency graph.
> This option may be useful as an alternative to --nodeps in cases where
> it is desirable to account for dependencies of packages given as
> arguments.
> 
> Bug: https://bugs.gentoo.org/608564
> ---
>  man/emerge.1  | 17 +
>  pym/_emerge/create_depgraph_params.py |  4 
>  pym/_emerge/depgraph.py   |  8 ++--
>  pym/_emerge/main.py   |  9 +
>  pym/portage/tests/resolver/test_complete_graph.py | 18 ++
>  5 files changed, 54 insertions(+), 2 deletions(-)
> 
> diff --git a/man/emerge.1 b/man/emerge.1
> index a17b65ed2..01ce62e51 100644
> --- a/man/emerge.1
> +++ b/man/emerge.1
> @@ -630,6 +630,23 @@ Therefore, \fB\-\-usepkgonly\fR (or 
> \fB\-\-getbinpkgonly\fR) must be
>  used in order to enable soname depedency resolution when installing
>  packages.
>  .TP
> +.BR "\-\-ignore\-world [ y | n ]"
> +Ignore the @world package set and its dependencies. This may be useful
> +if there is a desire to perform an action even though it might break
> +the dependencies of some installed packages (it might also remove
> +installed packages in order to solve blockers). This also alters the
> +behavior of \fB\-\-complete\-graph\fR options so that only deep
> +dependencies of packages given as arguments are included in the
> +dependency graph. This option may be useful as an alternative to
> +\fB\-\-nodeps\fR in cases where it is desirable to account for
> +dependencies of packages given as arguments.
> +
> +\fBWARNING:\fR
> +This option is intended to be used only with great caution, since it is
> +possible for it to make nonsensical changes which may lead to system
> +breakage. Therefore, it is advisable to use \fB\-\-ask\fR together with
> +this option.
> +.TP
>  .BR \-j\ [JOBS] ", "  \-\-jobs[=JOBS]
>  Specifies the number of packages to build simultaneously. If this option is
>  given without an argument, emerge will not limit the number of jobs that can
> diff --git a/pym/_emerge/create_depgraph_params.py 
> b/pym/_emerge/create_depgraph_params.py
> index fc7fa60d7..0405011fd 100644
> --- a/pym/_emerge/create_depgraph_params.py
> +++ b/pym/_emerge/create_depgraph_params.py
> @@ -26,6 +26,7 @@ def create_depgraph_params(myopts, myaction):
>   # ignore_soname_deps: ignore the soname dependencies of built
>   #   packages, so that they do not trigger dependency resolution
>   #   failures, or cause packages to be rebuilt or replaced.
> + # ignore_world: ignore the @world package set and its dependencies
>   # with_test_deps: pull in test deps for packages matched by arguments
>   # changed_deps: rebuild installed packages with outdated deps
>   # changed_deps_report: report installed packages with outdated deps
> @@ -56,6 +57,9 @@ def create_depgraph_params(myopts, myaction):
>   myparams["selective"] = True
>   return myparams
>  
> + if myopts.get('--ignore-world') is True:
> + myparams['ignore_world'] = True
> +
>   rebuild_if_new_slot = myopts.get('--rebuild-if-new-slot')
>   if rebuild_if_new_slot is not None:
>   myparams['rebuild_if_new_slot'] = rebuild_if_new_slot
> diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
> index 6c728684f..f7ea27c37 100644
> --- a/pym/_emerge/depgraph.py
> +++ b/pym/_emerge/depgraph.py
> @@ -163,7 +163,10 @@ class _frozen_depgraph_config(object):
>   self.trees[myroot]["bintree"] = DummyTree(
>   
> DbapiProvidesIndex(trees[myroot]["bintree"].dbapi))
>  
> - self._required_set_names = set(["world"])
> + if params.get("ignore_world", False):
> + self._required_set_names = set()
> + else:
> + self._required_set_names = set(["world"])
>  
>   atoms = ' '.join(myopts.get("--exclude", [])).split()
>   self.excluded_pkgs = _wildcard_set(atoms)
> @@ -7554,6 +7557,7 @@ class depgraph(object):
>   ignored_uninstall_tasks = set()
>   have_uninstall_task = False
>   complete = "complete" in self._dynamic_config.myparams
> + ignore_world = 
> self._dynamic_config.myparams.get("ignore_world", False)
>   asap_nodes = []
>  
>   def 

[gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

2018-03-21 Thread Zac Medico
Ignore the @world package set and its dependencies. This may be useful
if there is a desire to perform an action even though it might break
the dependencies of some installed packages (it might also remove
installed packages in order to solve blockers). This also alters the
behavior of --complete-graph options so that only deep dependencies
of packages given as arguments are included in the dependency graph.
This option may be useful as an alternative to --nodeps in cases where
it is desirable to account for dependencies of packages given as
arguments.

Bug: https://bugs.gentoo.org/608564
---
 man/emerge.1  | 17 +
 pym/_emerge/create_depgraph_params.py |  4 
 pym/_emerge/depgraph.py   |  8 ++--
 pym/_emerge/main.py   |  9 +
 pym/portage/tests/resolver/test_complete_graph.py | 18 ++
 5 files changed, 54 insertions(+), 2 deletions(-)

diff --git a/man/emerge.1 b/man/emerge.1
index a17b65ed2..01ce62e51 100644
--- a/man/emerge.1
+++ b/man/emerge.1
@@ -630,6 +630,23 @@ Therefore, \fB\-\-usepkgonly\fR (or 
\fB\-\-getbinpkgonly\fR) must be
 used in order to enable soname depedency resolution when installing
 packages.
 .TP
+.BR "\-\-ignore\-world [ y | n ]"
+Ignore the @world package set and its dependencies. This may be useful
+if there is a desire to perform an action even though it might break
+the dependencies of some installed packages (it might also remove
+installed packages in order to solve blockers). This also alters the
+behavior of \fB\-\-complete\-graph\fR options so that only deep
+dependencies of packages given as arguments are included in the
+dependency graph. This option may be useful as an alternative to
+\fB\-\-nodeps\fR in cases where it is desirable to account for
+dependencies of packages given as arguments.
+
+\fBWARNING:\fR
+This option is intended to be used only with great caution, since it is
+possible for it to make nonsensical changes which may lead to system
+breakage. Therefore, it is advisable to use \fB\-\-ask\fR together with
+this option.
+.TP
 .BR \-j\ [JOBS] ", "  \-\-jobs[=JOBS]
 Specifies the number of packages to build simultaneously. If this option is
 given without an argument, emerge will not limit the number of jobs that can
diff --git a/pym/_emerge/create_depgraph_params.py 
b/pym/_emerge/create_depgraph_params.py
index fc7fa60d7..0405011fd 100644
--- a/pym/_emerge/create_depgraph_params.py
+++ b/pym/_emerge/create_depgraph_params.py
@@ -26,6 +26,7 @@ def create_depgraph_params(myopts, myaction):
# ignore_soname_deps: ignore the soname dependencies of built
#   packages, so that they do not trigger dependency resolution
#   failures, or cause packages to be rebuilt or replaced.
+   # ignore_world: ignore the @world package set and its dependencies
# with_test_deps: pull in test deps for packages matched by arguments
# changed_deps: rebuild installed packages with outdated deps
# changed_deps_report: report installed packages with outdated deps
@@ -56,6 +57,9 @@ def create_depgraph_params(myopts, myaction):
myparams["selective"] = True
return myparams
 
+   if myopts.get('--ignore-world') is True:
+   myparams['ignore_world'] = True
+
rebuild_if_new_slot = myopts.get('--rebuild-if-new-slot')
if rebuild_if_new_slot is not None:
myparams['rebuild_if_new_slot'] = rebuild_if_new_slot
diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
index 6c728684f..f7ea27c37 100644
--- a/pym/_emerge/depgraph.py
+++ b/pym/_emerge/depgraph.py
@@ -163,7 +163,10 @@ class _frozen_depgraph_config(object):
self.trees[myroot]["bintree"] = DummyTree(

DbapiProvidesIndex(trees[myroot]["bintree"].dbapi))
 
-   self._required_set_names = set(["world"])
+   if params.get("ignore_world", False):
+   self._required_set_names = set()
+   else:
+   self._required_set_names = set(["world"])
 
atoms = ' '.join(myopts.get("--exclude", [])).split()
self.excluded_pkgs = _wildcard_set(atoms)
@@ -7554,6 +7557,7 @@ class depgraph(object):
ignored_uninstall_tasks = set()
have_uninstall_task = False
complete = "complete" in self._dynamic_config.myparams
+   ignore_world = 
self._dynamic_config.myparams.get("ignore_world", False)
asap_nodes = []
 
def get_nodes(**kwargs):
@@ -7971,7 +7975,7 @@ class depgraph(object):
# detected as early as possible, which 
makes it possible
# to avoid calling 
self._complete_graph() when it is
# unnecessary