Re: [gentoo-dev] [RFC] Problems and limitations of the current version dependency specs

[Ian Stakenvicius]

On 31/10/16 08:31 PM, Michał Górny wrote:
> Hello, everyone.
> 
> I would like to work on a major version depedencny specification
> improvements as part of the next EAPI. For this reason, I'd like to
> first gather some research on how developers are using the current
> system, what they find efficient and what they find cumbersome.
> 
> Therefore, I would like to ask the following questions:
> 
> 1. How often do you find '~' useful? Do you think there should be
> additional operators that ignore revision part?

The only time I have used this is when I've needed to synchronize the
version of two inter-dependent packages, while wanting to allow the
revision portion to vary.

I have generally found it a lot more useful to use this in my various
/etc/portage/ files than in ebuilds, though, especially
package.accept_keywords


> 
> 2. How often do you find '=...*' wildcard syntax useful? To what
> purpose do you use it? Do you find its behavior confusing [1]?

I find that this syntax is rather useful and often is necessary, due
to it being difficult to properly define both minimum and maximun
versions otherwise at times -- especially when I want to use the := or
:0= slot operator.  I don't find its behavior confusing once I became
educated on how it actually works; I do recognize that it *is not*
intuitive to the layman.


> 3. Do you sometimes find yourself using '<'/'<=' specs that
> accidentally match _pre/_rc/... versions?

No, but that is likely just because I tend not to deal with
dependencies that have _pre and _rc suffixes.


> 
> 4. What are the common tasks that you find unnecessarily complex /
> lengthy with the current version specifications?

It is cumbersome to deal with limiting slots to a specific subset,
when wanting to leverage slot-operator rebuilds right now --
effectively you need to have two atoms, one with :=, and a second set
||()'d with each of the slots that are compatible.


> 5. Do you find any other parts of the current version dependency
> specifications confusing?

The as yet still impossible means of dealing properly with multiple
packages that should either independently or when switching from one
to another trigger a (sub)slot rebuild is quite vexing at times.





signature.asc
Description: OpenPGP digital signature

[gentoo-dev] [RFC] Problems and limitations of the current version dependency specs

[Michał Górny]

Hello, everyone.

I would like to work on a major version depedencny specification
improvements as part of the next EAPI. For this reason, I'd like to
first gather some research on how developers are using the current
system, what they find efficient and what they find cumbersome.

Therefore, I would like to ask the following questions:

1. How often do you find '~' useful? Do you think there should be
additional operators that ignore revision part?

2. How often do you find '=...*' wildcard syntax useful? To what
purpose do you use it? Do you find its behavior confusing [1]?

3. Do you sometimes find yourself using '<'/'<=' specs that
accidentally match _pre/_rc/... versions?

4. What are the common tasks that you find unnecessarily complex /
lengthy with the current version specifications?

5. Do you find any other parts of the current version dependency
specifications confusing?

Please just list the problems and your feeling about the current system
now, not solutions. We will focus on the solutions in a next thread
once we determine what the problems are.

Thanks in advance for your input.


[1]. Few less-known facts about =...*:

a. it does NOT do string pattern matching but allows any version
components following (i.e. 1.1* does not match 1.11),

b. it matches any version components, including _pre/_p/_rc/...
suffixes (i.e. 1.1* matches 1.1_rc1).

-- 
Best regards,
Michał Górny



pgpgV4EmkiK4X.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] OpenPGP verification for gentoo-mirror repos

[Zac Medico]

On 10/31/2016 01:34 AM, Michał Górny wrote:
> The major difference between a developer key and an automated key is
> that the latter is far easier target. I think we can trust Gentoo
> developers to at least have their keys encrypted. I suppose most of
> them don't 'git log -p' the commits their sign but well, it's still
> harder to target a developer PC than a public server that most likely
> keeps its signature key unencrypted (or with cleartext password).

How about if we use subkeys that expire every 3 months or so.
Realistically, won't that provide a reasonable level of security? That
way, whoever is stealing our keys for the purposes of man-in-the-middle
attacks will have to get a new copy of our key every 3 months.
-- 
Thanks,
Zac

Re: [gentoo-dev] rsync.gentoo.org rsync modules: gentoo-repo-changelog added, gentoo-x86-portage & gentoo-sec discontinued.

[Robin H. Johnson]

On Mon, Oct 31, 2016 at 02:12:52PM -0400, Brian Evans wrote:
> We just had a user in #gentoo report that app-dicts/myspell-en had a
> mismatched ChangeLog size in Manifest that blocked him from installing
> software.
> 
> So this does not seem to be treated as a warning for a MISC type.
GLEP60 declared that MISC was supposed to be non-fatal. So we're going
to need that fixed ASAP.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
E-Mail   : robb...@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136


signature.asc
Description: Digital signature

Re: [gentoo-dev] Re: [gentoo-dev-announce] Multiple misc. Python packages for grabs

[Michał Górny]

On Mon, 31 Oct 2016 19:03:22 +0100
Gilles Dartiguelongue  wrote:

> Le samedi 29 octobre 2016 à 23:21 +0200, Michał Górny a écrit :
> > Hello,
> > 
> > The Python team is dropping itself from maintainers of the following
> > packages. As a result, those packages no longer have any maintainer:
> > 
> > app-admin/supervisor  
> 
> I'll take this at least to fix its default configuration which is
> annoying me.
> 
> PS: re-posting as it went to the wrong ml.

Would you be interested in dev-python/superlance as well by any chance?
*wink wink*

-- 
Best regards,
Michał Górny



pgp02dqDO7sft.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] rsync.gentoo.org rsync modules: gentoo-repo-changelog added, gentoo-x86-portage & gentoo-sec discontinued.

[Brian Evans]

On 10/30/2016 4:39 PM, Robin H. Johnson wrote:
> On Sun, Oct 30, 2016 at 09:47:36PM +1300, Kent Fredric wrote:
>> On Sun, 30 Oct 2016 02:54:17 +
>> "Robin H. Johnson"  wrote:
>>
>>> They may or may not still be referenced
>>> by the thick Manifests.
>> Surely they should not continue to be referenced, as being referenced
>> but not present would cause the entire package to be deemed invalid due
>> to failing integrity checks.
> ChangeLogs are supposed to be of Manifest2 type MISC, for which the
> absence or mismatch should be non-fatal (if the file is present on disk,
> and in Manifest, but does not match, display a warning only).
> 

We just had a user in #gentoo report that app-dicts/myspell-en had a
mismatched ChangeLog size in Manifest that blocked him from installing
software.

So this does not seem to be treated as a warning for a MISC type.

Brian



signature.asc
Description: OpenPGP digital signature

[gentoo-dev] Re: [gentoo-dev-announce] Multiple misc. Python packages for grabs

[Gilles Dartiguelongue]

Le samedi 29 octobre 2016 à 23:21 +0200, Michał Górny a écrit :
> Hello,
> 
> The Python team is dropping itself from maintainers of the following
> packages. As a result, those packages no longer have any maintainer:
> 
> app-admin/supervisor

I'll take this at least to fix its default configuration which is
annoying me.

PS: re-posting as it went to the wrong ml.
-- 
Gilles Dartiguelongue 
Gentoo

signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] Multiple misc. Python packages for grabs

[Ilya Tumaykin]

On Saturday 29 October 2016 23:21:57 Michał Górny wrote:
> Hello,
> 
> The Python team is dropping itself from maintainers of the following
> packages. As a result, those packages no longer have any maintainer:
> 
> ...
> x11-misc/menumaker

I'll take this one via proxy-maint if no dev wants it after a week.

-- 
Best regards.
Ilya Tumaykin.

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] OpenPGP verification for gentoo-mirror repos

[Kristian Fiskerstrand]

On 10/31/2016 09:34 AM, Michał Górny wrote:
> The major difference between a developer key and an automated key is
> that the latter is far easier target. I think we can trust Gentoo
> developers to at least have their keys encrypted. I suppose most of
> them don't 'git log -p' the commits their sign but well, it's still
> harder to target a developer PC than a public server that most likely
> keeps its signature key unencrypted (or with cleartext password).

If you go this route it becomes more complex, as you need the private
key stored on a smartcard to avoid leakage when secret key is handled
in-memory (unencrypted properties - so I don't agree with your argument
that developers store secret key encrypted). This is a lot better due to
process separation in gnupg 2.1 as a parsing error in gpg doesn't have
access to keys in gpg-agent as an example, but it is mostly wrong route
to go on discussion.

tl;dr; A signature by a release key is valuable

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] OpenPGP verification for gentoo-mirror repos

[Michał Górny]

On Sun, 30 Oct 2016 15:41:55 -0700
Zac Medico  wrote:

> On 10/30/2016 03:32 PM, Michał Górny wrote:
> > On Sun, 30 Oct 2016 14:58:59 -0700
> > Zac Medico  wrote:
> > 
> >> On 10/30/2016 01:44 PM, Michał Górny wrote:
> >>> Hi, everyone.
> >>>
> >>> Just a quick note: I've prepared a simple tool [1] to verify clones of
> >>> gentoo-mirror repositories. It's still early WiP but can be easily used
> >>> to verify a clone:
> >>>
> >>>   $ ./verify-repo gentoo
> >>>   [/var/db/repos/gentoo]
> >>>   Untrusted signature on 42ccdf48d718287e981c00f25caea2242262906a
> >>>   (you may need to import/trust developer keys)
> >>>   Note: unsigned changes in metadata and/or caches found (it's fine)  
> >>
> >> I don't think it's acceptable to use an unsigned metadata/cache commit.
> >> Can't we use an infrastructure key for this?
> > 
> > How are you going to guarantee that a third-party didn't access
> > the remote server and alter the filesystem just before the commit? Not
> > to mention the pains of keeping the key secure.
> > 
> > It's better not to sign that to provide false security.
> 
> There's no absolute guarantee that the developer's key hasn't been
> compromised either. So we've got varying degrees of trust. An automated
> infrastructure signature may not have as much trust as a developer
> signature, but it's still better than nothing, for the same reason that
> publishing these key fingerprints via https is better than http:
> 
> https://wiki.gentoo.org/wiki/Project:RelEng#Keys

The HTTPS argument is fairly irrelevant. There is a reason why project
publish tarball OpenPGP signatures, rather than relying on official
HTTPS downloads. It's not only about mirrors and subsequent
verification.

The major difference between a developer key and an automated key is
that the latter is far easier target. I think we can trust Gentoo
developers to at least have their keys encrypted. I suppose most of
them don't 'git log -p' the commits their sign but well, it's still
harder to target a developer PC than a public server that most likely
keeps its signature key unencrypted (or with cleartext password).

That said, repo-mirror-ci is running as user processes on a donated
space on a third-party server. We have no way of ensuring that it is
secure. Even if it were on Gentoo Infra, things wouldn't be much
better.

Furthermore, one important point in OpenPGP-signed commits is that
the top signature signs both the commit itself and its parent,
implicitly confirming all past commits. So if I am supposed to sign
the automated commit, I have to verify the signatures on the preceding
commit first. And where are the working tools to do that?

On the other hand, Gentoo developers already sign their commits without
verifying if the commits preceding them are trusted. We only have some
minor server-side verification from Infra, with future possibility of
verifying signatures client-side.

-- 
Best regards,
Michał Górny



pgpH59SKWLSEZ.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] OpenPGP verification for gentoo-mirror repos

[Michał Górny]

On Sun, 30 Oct 2016 15:36:16 -0700
Zac Medico  wrote:

> I'm merging in Michał's reply from the related "[gentoo-portage-dev]
> [PATCH] [sync] Increase the default git sync-depth to 10" thread.
> 
> On 10/30/2016 02:58 PM, Zac Medico wrote:
> > On 10/30/2016 01:44 PM, Michał Górny wrote:
> >> Hi, everyone.
> >>
> >> Just a quick note: I've prepared a simple tool [1] to verify clones of
> >> gentoo-mirror repositories. It's still early WiP but can be easily used
> >> to verify a clone:
> >>
> >>   $ ./verify-repo gentoo
> >>   [/var/db/repos/gentoo]
> >>   Untrusted signature on 42ccdf48d718287e981c00f25caea2242262906a
> >>   (you may need to import/trust developer keys)
> >>   Note: unsigned changes in metadata and/or caches found (it's fine)
> > 
> > I don't think it's acceptable to use an unsigned metadata/cache commit.
> > Can't we use an infrastructure key for this?
> 
> On 10/30/2016 03:03 PM, Michał Górny wrote:
> > I've even written a blog post [1] about that. Long story short,
> > trusting some random key used by automated process running on remote
> > server with no real security is insane. I've made a script that
> > verifies underlying repo commit instead, and diffs for metadata
> > changes.
> >
> >
> [1]:https://blogs.gentoo.org/mgorny/2016/04/15/why-automated-gentoo-mirror-commits-are-not-signed-and-how-to-verify-them-2/
> 
> An automated signature may not have the same degree of trust as a
> manually generated signature, but that does not make it completely
> worthless (is https worthless too?).

I disagree. We don't have any good way of expressing this degree of
trust. Therefore, the user will commonly presume both are of the same
degree of trust.

-- 
Best regards,
Michał Górny



pgpBc2B8cih0g.pgp
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Re: [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

[Michał Górny]

On Mon, 31 Oct 2016 07:16:24 + (UTC)
Martin Vaeth  wrote:

> Michał Górny  wrote:
> > +   if quiet: # -q needs to go first
> > +   update_index_cmd.append('-q')  
> 
> The options -q --unmerged (which are implicitly passed by "git status")
> are not only to suppress verbosity:
> The git-update-index man page has to say for these options:
> 
> : the default behavior is to error out. This option makes
> : git update-index continue anyway.
> 
> and if I understood the git source code correctly, without
> these options git will indeed break some loops early in the
> "error" case, i.e. it will perhaps not do a full update of
> the index in the "error" case.

It's a 'best effort' attempt at making git not error out
in the following merge. If it doesn't work, the merge will print
the error.

-- 
Best regards,
Michał Górny



pgpWdkbJQrA03.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Lastrites: dev-ada/cbind, net-dialup/dtrace, net-nds/lat, app-pda/jpilot-mail, net-dialup/drdsl, dev-util/insight, app-laptop/configure-trackpoint, x11-misc/lxmed, dev-util/ketchup, m

[Michał Górny]

On Sun, 30 Oct 2016 20:31:01 -0700
Matt Turner  wrote:

> On Sat, Oct 29, 2016 at 4:35 AM, David Seifert  wrote:
> > On Sa, 2016-10-08 at 13:57 +0200, Pacho Ramos wrote:  
> >> # Pacho Ramos  (08 Oct 2016)
> >> # Uses get_libdir at global scope (#593382). Dead for ages. Removal
> >> in
> >> a
> >> # month.
> >> app-cdr/nero  
> >
> > I'd like to keep this package, hence fixed the issues, bumped to
> > EAPI=6, took over maintainership.  
> 
> I'm curious why. Does it offer features that other free software CD
> burning applications do not?

Because people love their proprietary software and now that they bought
it, they expect free support for it from Gentoo.

-- 
Best regards,
Michał Górny



pgpkbxmXgVQtX.pgp
Description: OpenPGP digital signature

[gentoo-portage-dev] Re: [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

[Martin Vaeth]

Michał Górny  wrote:
> + if quiet: # -q needs to go first
> + update_index_cmd.append('-q')

The options -q --unmerged (which are implicitly passed by "git status")
are not only to suppress verbosity:
The git-update-index man page has to say for these options:

: the default behavior is to error out. This option makes
: git update-index continue anyway.

and if I understood the git source code correctly, without
these options git will indeed break some loops early in the
"error" case, i.e. it will perhaps not do a full update of
the index in the "error" case.

[gentoo-portage-dev] [PATCH] _expand_new_virtuals: constrain output for dep_zapdeps (bug 597752)

[Zac Medico]

Constrain _expand_new_virtuals output in order to avoid incorrect
re-ordering of || deps in the dep_zapdeps function, as reported in
bug 597752.

The incorrect dep_zapdeps behavior involved a problem in the
construction of the _dep_choice.cp_map dictionary inside the
dep_zapdeps function, with this input:

|| ( dev-java/icedtea-bin:8 =virtual/jdk-1.8.0-r3 >=virtual/jdk-1.5 )
( dev-java/icedtea-bin:7 =virtual/jdk-1.7.0-r2 >=virtual/jdk-1.5 )

The cp_map for virtual/jdk-1.7.0-r2 erroneously contained
virtual/jdk-1.8.0-r3 because that was the highest virtual/jdk matched
by the >=virtual/jdk-1.5 atom. This patch removes the >=virtual/jdk-1.5
atom from the _expand_new_virtuals output, in order to avoid triggering
the erroneous dep_zapdeps behavior. The >=virtual/jdk-1.5 atom is not
completely discarded, since the depgraph is able to access it via the
virt_atom._orig_atom attribute.

In order to demonstrate that this patch solves the problem, run the
command `emerge ant-core` from inside a stage3 chroot, and see that
virtual/jdk:1.8 is favored over virtual/jdk:1.7.

X-Gentoo-Bug: 597752
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=597752
---
 pym/_emerge/depgraph.py  |  6 +-
 pym/portage/dep/dep_check.py | 13 +++--
 2 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
index 26037ad..9161914 100644
--- a/pym/_emerge/depgraph.py
+++ b/pym/_emerge/depgraph.py
@@ -4453,7 +4453,11 @@ class depgraph(object):
# _UNREACHABLE_DEPTH for complete mode.
virt_depth = parent.depth
 
-   chosen_atom_ids = frozenset(id(atom) for atom in 
mycheck[1])
+   chosen_atom_ids = frozenset(chain(
+   (id(atom) for atom in mycheck[1]),
+   (id(atom._orig_atom) for atom in mycheck[1]
+   if hasattr(atom, '_orig_atom')),
+   ))
selected_atoms = OrderedDict()
node_stack = [(parent, None, None)]
traversed_nodes = set()
diff --git a/pym/portage/dep/dep_check.py b/pym/portage/dep/dep_check.py
index 9af4e65..9d2ca4b 100644
--- a/pym/portage/dep/dep_check.py
+++ b/pym/portage/dep/dep_check.py
@@ -188,13 +188,14 @@ def _expand_new_virtuals(mysplit, edebug, mydbapi, 
mysettings, myroot="/",
raise ParseError("%s: %s '%s'" % \
(pkg, mycheck[1], depstring))
 
-   # Pull in virt_atom which refers to the specific version
-   # of the virtual whose deps we're expanding. Also pull
-   # in the original input atom, so that callers can 
reliably
-   # check to see if a given input atom has been selected,
-   # as in depgraph._slot_operator_update_probe.
+   # Replace the original atom "x" with "virt_atom" which 
refers
+   # to the specific version of the virtual whose deps 
we're
+   # expanding. The virt_atom._orig_atom attribute is used
+   # by depgraph to map virt_atom back to the original 
atom.
+   # We specifically exclude the original atom "x" from the
+   # the expanded output here, since otherwise it could 
trigger
+   # incorrect dep_zapdeps behavior (see bug #597752).
mycheck[1].append(virt_atom)
-   mycheck[1].append(x)
a.append(mycheck[1])
if atom_graph is not None:
virt_atom_node = (virt_atom, id(virt_atom))
-- 
2.7.4