[gentoo-portage-dev] Re: Plan for initial integration of gemato with portage

[Duncan]

Michał Górny posted on Wed, 24 Jan 2018 20:58:54 +0100 as excerpted:

> W dniu śro, 24.01.2018 o godzinie 12∶54 -0500, użytkownik Alec Warner
> napisał:
>> 
>> I think its a bit trickier to control the hook's behavior. For
>> instance:
>> 
>> 1) I install portage[rsync-verify]. This installs the hook.
>> 2) I end up not liking the hook, I install portage[-rsync-verify]
>> 3) Does the hook get config-protected here?
> 
> Keeping config-protected files applies only if the file were modified.
> In this case it just gets unmerged. I've just verified that.

That's controlled by FEATURES=config-protect-if-modified .  Granted, 
that's enabled by default, but config-protecting unmodified files as well 
is definitely a user option that should be considered, even if that 
consideration is simply "users disabling the default get to keep the 
pieces".

Meanwhile, if it's "you keep the pieces if you've messed with the 
default", that should at least be mentioned in the news item[1], so users 
can consider whether the risk is worth it if they've had that feature 
specifically disabled previously.

---
[1] New item mention: or the more detailed instructions the news item 
points to if they get too long to be in the news item itself.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-portage-dev] [PATCH v4] rsync: Introduce support for running full-tree gemato verification

[Zac Medico]

On 01/24/2018 01:36 PM, Michał Górny wrote:
> Add two new configuration options to rsync repositories:
> sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
> The first controls whether gemato verification is run for
> the repository (defaults to true for ::gentoo, false otherwise),
> the second makes it possible to override the key path for custom
> repositories.
> ---
>  cnf/repos.conf |  2 ++
>  man/portage.5  |  9 +
>  pym/portage/sync/modules/rsync/__init__.py |  4 +++-
>  pym/portage/sync/modules/rsync/rsync.py| 20 +++-
>  4 files changed, 33 insertions(+), 2 deletions(-)
> 
> v4: also key option to repos.conf

Looks good.

I see that the man page currently does not escape hyphens in
sync-rsync-*, but we can fix those up separately.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH v4] rsync: Introduce support for running full-tree gemato verification

[Michał Górny]

Add two new configuration options to rsync repositories:
sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
The first controls whether gemato verification is run for
the repository (defaults to true for ::gentoo, false otherwise),
the second makes it possible to override the key path for custom
repositories.
---
 cnf/repos.conf |  2 ++
 man/portage.5  |  9 +
 pym/portage/sync/modules/rsync/__init__.py |  4 +++-
 pym/portage/sync/modules/rsync/rsync.py| 20 +++-
 4 files changed, 33 insertions(+), 2 deletions(-)

v4: also key option to repos.conf

diff --git a/cnf/repos.conf b/cnf/repos.conf
index 062fc0d10..0d2b1f4be 100644
--- a/cnf/repos.conf
+++ b/cnf/repos.conf
@@ -6,6 +6,8 @@ location = /usr/portage
 sync-type = rsync
 sync-uri = rsync://rsync.gentoo.org/gentoo-portage
 auto-sync = yes
+sync-rsync-verify-metamanifest = yes
+sync-rsync-openpgp-key-path = 
/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
 
 # for daily squashfs snapshots
 #sync-type = squashdelta
diff --git a/man/portage.5 b/man/portage.5
index e724e1f08..35bf8f03b 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,19 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
+.B sync-rsync-openpgp-key-path
+Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
+if \fBsync-rsync-verify-metamanifest\fR is enabled. If unset,
+the user's keyring is used.
+.TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
+.TP
+.B sync-rsync-verify-metamanifest = true|false
+Require the repository to contain a signed MetaManifest and verify
+it using \fBapp-portage/gemato\fR. Defaults to false.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index c2fdc4188..df9a1995a 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Rsync plug-in module for portage.
@@ -27,7 +27,9 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
+   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-metamanifest',
),
}
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index c80641ba3..47f0e1ea3 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -82,6 +82,16 @@ class RsyncSync(NewBase):
self.extra_rsync_opts.extend(portage.util.shlex_split(

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
+   # Process GLEP74 verification options.
+   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   self.verify_metamanifest = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-verify-metamanifest', 
False))
+   # Default to gentoo-keys keyring.
+   self.openpgp_key_path = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-openpgp-key-path', None))
+
# Real local timestamp file.
self.servertimestampfile = os.path.join(
self.repo.location, "metadata", "timestamp.chk")
@@ -259,6 +269,14 @@ class RsyncSync(NewBase):
exitcode = EXCEEDED_MAX_RETRIES
break
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
+
+   # if synced successfully, verify now
+   if exitcode == 0 and self.verify_metamanifest:
+   command = ['gemato', 'verify', '-s', self.repo.location]
+   if self.openpgp_key_path is not None:
+   command += ['-K', self.openpgp_key_path]
+   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
+
return (exitcode, updatecache_flg)
 
 
-- 

[gentoo-portage-dev] [PATCH v3] rsync: Introduce support for running full-tree gemato verification

[Michał Górny]

Add two new configuration options to rsync repositories:
sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
The first controls whether gemato verification is run for
the repository (defaults to true for ::gentoo, false otherwise),
the second makes it possible to override the key path for custom
repositories.
---
 cnf/repos.conf |  1 +
 man/portage.5  | 10 ++
 pym/portage/sync/modules/rsync/__init__.py |  4 +++-
 pym/portage/sync/modules/rsync/rsync.py| 20 +++-
 4 files changed, 33 insertions(+), 2 deletions(-)

v3: now with manpage fix ;-)

diff --git a/cnf/repos.conf b/cnf/repos.conf
index 062fc0d10..644687515 100644
--- a/cnf/repos.conf
+++ b/cnf/repos.conf
@@ -6,6 +6,7 @@ location = /usr/portage
 sync-type = rsync
 sync-uri = rsync://rsync.gentoo.org/gentoo-portage
 auto-sync = yes
+sync-rsync-verify-metamanifest = yes
 
 # for daily squashfs snapshots
 #sync-type = squashdelta
diff --git a/man/portage.5 b/man/portage.5
index e724e1f08..b658b9f68 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,20 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
+.B sync-rsync-openpgp-key-path
+Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
+if \fBsync-rsync-verify-metamanifest\fR is enabled. Defaults to
+\fB/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg\fR (path
+used by \fBapp-crypt/gentoo-keys\fR).
+.TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
+.TP
+.B sync-rsync-verify-metamanifest = true|false
+Require the repository to contain a signed MetaManifest and verify
+it using \fBapp-portage/gemato\fR. Defaults to false.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index c2fdc4188..df9a1995a 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Rsync plug-in module for portage.
@@ -27,7 +27,9 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
+   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-metamanifest',
),
}
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index c80641ba3..613bedd0c 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -82,6 +82,17 @@ class RsyncSync(NewBase):
self.extra_rsync_opts.extend(portage.util.shlex_split(

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
+   # Process GLEP74 verification options.
+   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   self.verify_metamanifest = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-verify-metamanifest', 
False))
+   # Default to gentoo-keys keyring.
+   self.openpgp_key_path = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-openpgp-key-path',
+   
'/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg'))
+
# Real local timestamp file.
self.servertimestampfile = os.path.join(
self.repo.location, "metadata", "timestamp.chk")
@@ -259,6 +270,13 @@ class RsyncSync(NewBase):
exitcode = EXCEEDED_MAX_RETRIES
break
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
+
+   # if synced successfully, verify now
+   if exitcode == 0 and self.verify_metamanifest:
+   command = ['gemato', 'verify', '-K', 
self.openpgp_key_path,
+   '-s', self.repo.location]
+   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
+
return (exitcode, 

[gentoo-portage-dev] [PATCH v2] rsync: Introduce support for running full-tree gemato verification

[Michał Górny]

Add two new configuration options to rsync repositories:
sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
The first controls whether gemato verification is run for
the repository (defaults to true for ::gentoo, false otherwise),
the second makes it possible to override the key path for custom
repositories.
---
 cnf/repos.conf |  1 +
 man/portage.5  | 11 +++
 pym/portage/sync/modules/rsync/__init__.py |  4 +++-
 pym/portage/sync/modules/rsync/rsync.py| 20 +++-
 4 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/cnf/repos.conf b/cnf/repos.conf
index 062fc0d10..644687515 100644
--- a/cnf/repos.conf
+++ b/cnf/repos.conf
@@ -6,6 +6,7 @@ location = /usr/portage
 sync-type = rsync
 sync-uri = rsync://rsync.gentoo.org/gentoo-portage
 auto-sync = yes
+sync-rsync-verify-metamanifest = yes
 
 # for daily squashfs snapshots
 #sync-type = squashdelta
diff --git a/man/portage.5 b/man/portage.5
index e724e1f08..5e8127778 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,21 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
+.B sync-rsync-openpgp-key-path
+Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
+if \fBsync-rsync-verify-metamanifest\fR is enabled. Defaults to
+\fB/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg\fR (path
+used by \fBapp-crypt/gentoo-keys\fR).
+.TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
+.TP
+.B sync-rsync-verify-metamanifest = true|false
+Require the repository to contain a signed MetaManifest and verify
+it using \fBapp-portage/gemato\fR. Defauls to true for the \fBgentoo\fR
+repository and false otherwise.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index c2fdc4188..df9a1995a 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Rsync plug-in module for portage.
@@ -27,7 +27,9 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
+   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-metamanifest',
),
}
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index c80641ba3..613bedd0c 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -82,6 +82,17 @@ class RsyncSync(NewBase):
self.extra_rsync_opts.extend(portage.util.shlex_split(

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
+   # Process GLEP74 verification options.
+   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   self.verify_metamanifest = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-verify-metamanifest', 
False))
+   # Default to gentoo-keys keyring.
+   self.openpgp_key_path = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-openpgp-key-path',
+   
'/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg'))
+
# Real local timestamp file.
self.servertimestampfile = os.path.join(
self.repo.location, "metadata", "timestamp.chk")
@@ -259,6 +270,13 @@ class RsyncSync(NewBase):
exitcode = EXCEEDED_MAX_RETRIES
break
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
+
+   # if synced successfully, verify now
+   if exitcode == 0 and self.verify_metamanifest:
+   command = ['gemato', 'verify', '-K', 
self.openpgp_key_path,
+   '-s', self.repo.location]
+   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
+
return 

Re: [gentoo-portage-dev] [PATCH] rsync: Introduce support for running full-tree gemato verification

[Zac Medico]

On 01/24/2018 01:03 PM, Michał Górny wrote:
> diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> b/pym/portage/sync/modules/rsync/rsync.py
> index c80641ba3..f3157d093 100644
> --- a/pym/portage/sync/modules/rsync/rsync.py
> +++ b/pym/portage/sync/modules/rsync/rsync.py
> @@ -1,4 +1,4 @@
> -# Copyright 1999-2015 Gentoo Foundation
> +# Copyright 1999-2018 Gentoo Foundation
>  # Distributed under the terms of the GNU General Public License v2
>  
>  import sys
> @@ -82,6 +82,16 @@ class RsyncSync(NewBase):
>   self.extra_rsync_opts.extend(portage.util.shlex_split(
>   
> self.repo.module_specific_options['sync-rsync-extra-opts']))
>  
> + # Process GLEP74 verification options.
> + # Default verification to 'on' for ::gentoo, 'off' otherwise.
> + self.verify_metamanifest = (
> + 
> self.repo.module_specific_options.get('sync-rsync-verify-metamanifest',
> + self.repo.name == 'gentoo'))
> + # Default to gentoo-keys keyring.
> + self.openpgp_key_path = (
> + 
> self.repo.module_specific_options.get('sync-rsync-openpgp-key-path',
> + 
> '/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg'))

Maybe these defaults should go in cnf/repos.conf, and we should have the
ebuild patch the sync-rsync-verify-metamanifest setting depending on the
USE=rsync-verify state?
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH] rsync: Introduce support for running full-tree gemato verification

[Michał Górny]

Add two new configuration options to rsync repositories:
sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
The first controls whether gemato verification is run for
the repository (defaults to true for ::gentoo, false otherwise),
the second makes it possible to override the key path for custom
repositories.
---
 man/portage.5  | 11 +++
 pym/portage/sync/modules/rsync/__init__.py |  4 +++-
 pym/portage/sync/modules/rsync/rsync.py| 19 ++-
 3 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/man/portage.5 b/man/portage.5
index e724e1f08..5e8127778 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,21 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
+.B sync-rsync-openpgp-key-path
+Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
+if \fBsync-rsync-verify-metamanifest\fR is enabled. Defaults to
+\fB/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg\fR (path
+used by \fBapp-crypt/gentoo-keys\fR).
+.TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
+.TP
+.B sync-rsync-verify-metamanifest = true|false
+Require the repository to contain a signed MetaManifest and verify
+it using \fBapp-portage/gemato\fR. Defauls to true for the \fBgentoo\fR
+repository and false otherwise.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index c2fdc4188..df9a1995a 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Rsync plug-in module for portage.
@@ -27,7 +27,9 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
+   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-metamanifest',
),
}
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index c80641ba3..f3157d093 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -82,6 +82,16 @@ class RsyncSync(NewBase):
self.extra_rsync_opts.extend(portage.util.shlex_split(

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
+   # Process GLEP74 verification options.
+   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   self.verify_metamanifest = (
+   
self.repo.module_specific_options.get('sync-rsync-verify-metamanifest',
+   self.repo.name == 'gentoo'))
+   # Default to gentoo-keys keyring.
+   self.openpgp_key_path = (
+   
self.repo.module_specific_options.get('sync-rsync-openpgp-key-path',
+   
'/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg'))
+
# Real local timestamp file.
self.servertimestampfile = os.path.join(
self.repo.location, "metadata", "timestamp.chk")
@@ -259,6 +269,13 @@ class RsyncSync(NewBase):
exitcode = EXCEEDED_MAX_RETRIES
break
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
+
+   # if synced successfully, verify now
+   if exitcode == 0 and self.verify_metamanifest:
+   command = ['gemato', 'verify', '-K', 
self.openpgp_key_path,
+   '-s', self.repo.location]
+   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
+
return (exitcode, updatecache_flg)
 
 
-- 
2.16.1

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Michał Górny]

W dniu śro, 24.01.2018 o godzinie 12∶29 -0800, użytkownik Zac Medico
napisał:
> On 01/24/2018 12:56 AM, Michał Górny wrote:
> > Hi, everyone.
> > 
> > Since the initial review of my patch lost focus, and lacked sufficient 
> > context, here's the plan that I'd like to follow in order to initially 
> > integrate gemato with portage and give our users secure checkouts by 
> > default.
> > 
> > 1. Add postsync hook to Portage git. Eventually, it will be replaced by 
> > direct Portage support.
> > 
> > 2. Add IUSE=+rsync-verify to portage- that controls installing the 
> > hook. This will give users the ability to easily disable it without jumping 
> > through cross package hoops.
> > 
> > 3. Submit a news item for review that will explain how to initially verify 
> > the keys on existing installations.
> > 
> > The news item would be published when the hook hits a release.
> > 
> > What do you think? If you agree, then I'll start writing the news item.
> > 
> 
> For the sake of maintaining stable interfaces for users, I feel like we
> should add the repos.conf sync-rsync-verify setting for this is
> up-front. That way, we won't have to train people to use a new interface
> later. Also, eventually we have to do this anyway if we want portage to
> recognize the nature of the failure and react by quarantining the
> repository.

Do you mean implementing it completely inside sync code without hooks?
I'm all for it. Will submit a patch soon.

-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Zac Medico]

On 01/24/2018 12:56 AM, Michał Górny wrote:
> Hi, everyone.
> 
> Since the initial review of my patch lost focus, and lacked sufficient 
> context, here's the plan that I'd like to follow in order to initially 
> integrate gemato with portage and give our users secure checkouts by default.
> 
> 1. Add postsync hook to Portage git. Eventually, it will be replaced by 
> direct Portage support.
> 
> 2. Add IUSE=+rsync-verify to portage- that controls installing the hook. 
> This will give users the ability to easily disable it without jumping through 
> cross package hoops.
> 
> 3. Submit a news item for review that will explain how to initially verify 
> the keys on existing installations.
> 
> The news item would be published when the hook hits a release.
> 
> What do you think? If you agree, then I'll start writing the news item.
> 

For the sake of maintaining stable interfaces for users, I feel like we
should add the repos.conf sync-rsync-verify setting for this is
up-front. That way, we won't have to train people to use a new interface
later. Also, eventually we have to do this anyway if we want portage to
recognize the nature of the failure and react by quarantining the
repository.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Alec Warner]

On Wed, Jan 24, 2018 at 2:58 PM, Michał Górny  wrote:

> W dniu śro, 24.01.2018 o godzinie 12∶54 -0500, użytkownik Alec Warner
> napisał:
> > On Wed, Jan 24, 2018 at 3:56 AM, Michał Górny  wrote:
> >
> > > Hi, everyone.
> > >
> > > Since the initial review of my patch lost focus, and lacked sufficient
> > > context, here's the plan that I'd like to follow in order to initially
> > > integrate gemato with portage and give our users secure checkouts by
> > > default.
> > >
> > > 1. Add postsync hook to Portage git. Eventually, it will be replaced by
> > > direct Portage support.
> > >
> > > 2. Add IUSE=+rsync-verify to portage- that controls installing the
> > > hook. This will give users the ability to easily disable it without
> jumping
> > > through cross package hoops.
> > >
> >
> > I think it makes sense to avoid installing the hook through this means
> > (e.g. I don't want it, so I set USE=-rsync-verify)
> >
> > I think its a bit trickier to control the hook's behavior. For instance:
> >
> > 1) I install portage[rsync-verify]. This installs the hook.
> > 2) I end up not liking the hook, I install portage[-rsync-verify]
> > 3) Does the hook get config-protected here?
>
> Keeping config-protected files applies only if the file were modified.
> In this case it just gets unmerged. I've just verified that.
>
> > > 3. Submit a news item for review that will explain how to initially
> verify
> > > the keys on existing installations.
> > >
> > > The news item would be published when the hook hits a release.
> > >
> > > What do you think? If you agree, then I'll start writing the news item.
> > >
> >
> > The other part of the user story I don't understand is what actions
> should
> > users take when verification legit fails?
> >
> > 1) file a bug?
> > 2) re-sync their tree?
> > 3) something else?
>
> I'd say one re-sync would be reasonable in case it could have been some
> rsync glitch. Beyond that, it would definitely be worth reporting...
> except the user will probably hit another mirror.
>

This sgtm.

If the number of reports is high we can build more complex reporting
infrastructure (e.g. to locate bad mirrors.)

-A


>
> --
> Best regards,
> Michał Górny
>
>
>

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Michał Górny]

W dniu śro, 24.01.2018 o godzinie 12∶54 -0500, użytkownik Alec Warner
napisał:
> On Wed, Jan 24, 2018 at 3:56 AM, Michał Górny  wrote:
> 
> > Hi, everyone.
> > 
> > Since the initial review of my patch lost focus, and lacked sufficient
> > context, here's the plan that I'd like to follow in order to initially
> > integrate gemato with portage and give our users secure checkouts by
> > default.
> > 
> > 1. Add postsync hook to Portage git. Eventually, it will be replaced by
> > direct Portage support.
> > 
> > 2. Add IUSE=+rsync-verify to portage- that controls installing the
> > hook. This will give users the ability to easily disable it without jumping
> > through cross package hoops.
> > 
> 
> I think it makes sense to avoid installing the hook through this means
> (e.g. I don't want it, so I set USE=-rsync-verify)
> 
> I think its a bit trickier to control the hook's behavior. For instance:
> 
> 1) I install portage[rsync-verify]. This installs the hook.
> 2) I end up not liking the hook, I install portage[-rsync-verify]
> 3) Does the hook get config-protected here?

Keeping config-protected files applies only if the file were modified.
In this case it just gets unmerged. I've just verified that.

> > 3. Submit a news item for review that will explain how to initially verify
> > the keys on existing installations.
> > 
> > The news item would be published when the hook hits a release.
> > 
> > What do you think? If you agree, then I'll start writing the news item.
> > 
> 
> The other part of the user story I don't understand is what actions should
> users take when verification legit fails?
> 
> 1) file a bug?
> 2) re-sync their tree?
> 3) something else?

I'd say one re-sync would be reasonable in case it could have been some
rsync glitch. Beyond that, it would definitely be worth reporting...
except the user will probably hit another mirror.

-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Alec Warner]

On Wed, Jan 24, 2018 at 3:56 AM, Michał Górny  wrote:

> Hi, everyone.
>
> Since the initial review of my patch lost focus, and lacked sufficient
> context, here's the plan that I'd like to follow in order to initially
> integrate gemato with portage and give our users secure checkouts by
> default.
>
> 1. Add postsync hook to Portage git. Eventually, it will be replaced by
> direct Portage support.
>

> 2. Add IUSE=+rsync-verify to portage- that controls installing the
> hook. This will give users the ability to easily disable it without jumping
> through cross package hoops.
>

I think it makes sense to avoid installing the hook through this means
(e.g. I don't want it, so I set USE=-rsync-verify)

I think its a bit trickier to control the hook's behavior. For instance:

1) I install portage[rsync-verify]. This installs the hook.
2) I end up not liking the hook, I install portage[-rsync-verify]
3) Does the hook get config-protected here?
4) I run emerge --depclean # gemato is un-merged here.

Now the hook fails because gemato is gone but the hook remains due to
config-protect?

5) I run etc-update and the hook is deleted; I don't recall etc-update
deleting files...

So basically I'm curious how the hook works w/config-protect; but
admittedly its been a while since I worried about such things so I could be
wrong.

Other considerations include:

1) Hooks don't matter anyway, so we may not care if it breaks.
2) Verification doesn't matter currently (portage will install software
regardless), so we may not care if it doesn't function.


>
> 3. Submit a news item for review that will explain how to initially verify
> the keys on existing installations.
>
> The news item would be published when the hook hits a release.
>
> What do you think? If you agree, then I'll start writing the news item.
>

The other part of the user story I don't understand is what actions should
users take when verification legit fails?

1) file a bug?
2) re-sync their tree?
3) something else?


--
> Best regards,
> Michał Górny (by phone)
>
>

[gentoo-portage-dev] Plan for initial integration of gemato with portage

[Michał Górny]

Hi, everyone.

Since the initial review of my patch lost focus, and lacked sufficient context, 
here's the plan that I'd like to follow in order to initially integrate gemato 
with portage and give our users secure checkouts by default.

1. Add postsync hook to Portage git. Eventually, it will be replaced by direct 
Portage support.

2. Add IUSE=+rsync-verify to portage- that controls installing the hook. 
This will give users the ability to easily disable it without jumping through 
cross package hoops.

3. Submit a news item for review that will explain how to initially verify the 
keys on existing installations.

The news item would be published when the hook hits a release.

What do you think? If you agree, then I'll start writing the news item.
-- 
Best regards,
Michał Górny (by phone)