[gentoo-dev] Last rites: ruby22-only packages and slots

[Hans de Graaff]

# Hans de Graaff  (24 Jan 2018)
# Remove packages that are ruby22-only, and either cannot
# be updated to ruby23, or are in old, obsolete, slots.
# Masked for removal in 30 days.
dev-ruby/amq-protocol:0
dev-ruby/aws-sdk:0
dev-ruby/bson:3
dev-ruby/camping
dev-ruby/execjs:1
dev-ruby/mocha:0.11
dev-ruby/mocha:0.12
dev-ruby/mocha:0.13
dev-ruby/mongo
dev-ruby/net-ssh-multi
dev-ruby/pcaprub:0.11
dev-ruby/postgres_ext:2.4
dev-ruby/rack:1.4
dev-ruby/rack:1.5
dev-ruby/rbtree
dev-ruby/recog:2.0.14
dev-ruby/right_http_connection
dev-ruby/rspec-rails:2
dev-ruby/sass:0
dev-ruby/sass-rails:4.0
dev-ruby/snmplib
dev-ruby/sprockets:2
dev-ruby/test-unit:0
dev-ruby/tilt:0
virtual/ruby-minitest
www-apps/jekyll-gist

signature.asc
Description: This is a digitally signed message part

[gentoo-portage-dev] Re: Plan for initial integration of gemato with portage

[Duncan]

Michał Górny posted on Wed, 24 Jan 2018 20:58:54 +0100 as excerpted:

> W dniu śro, 24.01.2018 o godzinie 12∶54 -0500, użytkownik Alec Warner
> napisał:
>> 
>> I think its a bit trickier to control the hook's behavior. For
>> instance:
>> 
>> 1) I install portage[rsync-verify]. This installs the hook.
>> 2) I end up not liking the hook, I install portage[-rsync-verify]
>> 3) Does the hook get config-protected here?
> 
> Keeping config-protected files applies only if the file were modified.
> In this case it just gets unmerged. I've just verified that.

That's controlled by FEATURES=config-protect-if-modified .  Granted, 
that's enabled by default, but config-protecting unmodified files as well 
is definitely a user option that should be considered, even if that 
consideration is simply "users disabling the default get to keep the 
pieces".

Meanwhile, if it's "you keep the pieces if you've messed with the 
default", that should at least be mentioned in the news item[1], so users 
can consider whether the risk is worth it if they've had that feature 
specifically disabled previously.

---
[1] New item mention: or the more detailed instructions the news item 
points to if they get too long to be in the news item itself.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-portage-dev] [PATCH v4] rsync: Introduce support for running full-tree gemato verification

[Zac Medico]

On 01/24/2018 01:36 PM, Michał Górny wrote:
> Add two new configuration options to rsync repositories:
> sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
> The first controls whether gemato verification is run for
> the repository (defaults to true for ::gentoo, false otherwise),
> the second makes it possible to override the key path for custom
> repositories.
> ---
>  cnf/repos.conf |  2 ++
>  man/portage.5  |  9 +
>  pym/portage/sync/modules/rsync/__init__.py |  4 +++-
>  pym/portage/sync/modules/rsync/rsync.py| 20 +++-
>  4 files changed, 33 insertions(+), 2 deletions(-)
> 
> v4: also key option to repos.conf

Looks good.

I see that the man page currently does not escape hyphens in
sync-rsync-*, but we can fix those up separately.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH v4] rsync: Introduce support for running full-tree gemato verification

[Michał Górny]

Add two new configuration options to rsync repositories:
sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
The first controls whether gemato verification is run for
the repository (defaults to true for ::gentoo, false otherwise),
the second makes it possible to override the key path for custom
repositories.
---
 cnf/repos.conf |  2 ++
 man/portage.5  |  9 +
 pym/portage/sync/modules/rsync/__init__.py |  4 +++-
 pym/portage/sync/modules/rsync/rsync.py| 20 +++-
 4 files changed, 33 insertions(+), 2 deletions(-)

v4: also key option to repos.conf

diff --git a/cnf/repos.conf b/cnf/repos.conf
index 062fc0d10..0d2b1f4be 100644
--- a/cnf/repos.conf
+++ b/cnf/repos.conf
@@ -6,6 +6,8 @@ location = /usr/portage
 sync-type = rsync
 sync-uri = rsync://rsync.gentoo.org/gentoo-portage
 auto-sync = yes
+sync-rsync-verify-metamanifest = yes
+sync-rsync-openpgp-key-path = 
/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
 
 # for daily squashfs snapshots
 #sync-type = squashdelta
diff --git a/man/portage.5 b/man/portage.5
index e724e1f08..35bf8f03b 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,19 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
+.B sync-rsync-openpgp-key-path
+Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
+if \fBsync-rsync-verify-metamanifest\fR is enabled. If unset,
+the user's keyring is used.
+.TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
+.TP
+.B sync-rsync-verify-metamanifest = true|false
+Require the repository to contain a signed MetaManifest and verify
+it using \fBapp-portage/gemato\fR. Defaults to false.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index c2fdc4188..df9a1995a 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Rsync plug-in module for portage.
@@ -27,7 +27,9 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
+   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-metamanifest',
),
}
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index c80641ba3..47f0e1ea3 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -82,6 +82,16 @@ class RsyncSync(NewBase):
self.extra_rsync_opts.extend(portage.util.shlex_split(

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
+   # Process GLEP74 verification options.
+   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   self.verify_metamanifest = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-verify-metamanifest', 
False))
+   # Default to gentoo-keys keyring.
+   self.openpgp_key_path = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-openpgp-key-path', None))
+
# Real local timestamp file.
self.servertimestampfile = os.path.join(
self.repo.location, "metadata", "timestamp.chk")
@@ -259,6 +269,14 @@ class RsyncSync(NewBase):
exitcode = EXCEEDED_MAX_RETRIES
break
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
+
+   # if synced successfully, verify now
+   if exitcode == 0 and self.verify_metamanifest:
+   command = ['gemato', 'verify', '-s', self.repo.location]
+   if self.openpgp_key_path is not None:
+   command += ['-K', self.openpgp_key_path]
+   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
+
return (exitcode, updatecache_flg)
 
 
-- 

[gentoo-portage-dev] [PATCH v3] rsync: Introduce support for running full-tree gemato verification

[Michał Górny]

Add two new configuration options to rsync repositories:
sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
The first controls whether gemato verification is run for
the repository (defaults to true for ::gentoo, false otherwise),
the second makes it possible to override the key path for custom
repositories.
---
 cnf/repos.conf |  1 +
 man/portage.5  | 10 ++
 pym/portage/sync/modules/rsync/__init__.py |  4 +++-
 pym/portage/sync/modules/rsync/rsync.py| 20 +++-
 4 files changed, 33 insertions(+), 2 deletions(-)

v3: now with manpage fix ;-)

diff --git a/cnf/repos.conf b/cnf/repos.conf
index 062fc0d10..644687515 100644
--- a/cnf/repos.conf
+++ b/cnf/repos.conf
@@ -6,6 +6,7 @@ location = /usr/portage
 sync-type = rsync
 sync-uri = rsync://rsync.gentoo.org/gentoo-portage
 auto-sync = yes
+sync-rsync-verify-metamanifest = yes
 
 # for daily squashfs snapshots
 #sync-type = squashdelta
diff --git a/man/portage.5 b/man/portage.5
index e724e1f08..b658b9f68 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,20 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
+.B sync-rsync-openpgp-key-path
+Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
+if \fBsync-rsync-verify-metamanifest\fR is enabled. Defaults to
+\fB/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg\fR (path
+used by \fBapp-crypt/gentoo-keys\fR).
+.TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
+.TP
+.B sync-rsync-verify-metamanifest = true|false
+Require the repository to contain a signed MetaManifest and verify
+it using \fBapp-portage/gemato\fR. Defaults to false.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index c2fdc4188..df9a1995a 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Rsync plug-in module for portage.
@@ -27,7 +27,9 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
+   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-metamanifest',
),
}
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index c80641ba3..613bedd0c 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -82,6 +82,17 @@ class RsyncSync(NewBase):
self.extra_rsync_opts.extend(portage.util.shlex_split(

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
+   # Process GLEP74 verification options.
+   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   self.verify_metamanifest = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-verify-metamanifest', 
False))
+   # Default to gentoo-keys keyring.
+   self.openpgp_key_path = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-openpgp-key-path',
+   
'/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg'))
+
# Real local timestamp file.
self.servertimestampfile = os.path.join(
self.repo.location, "metadata", "timestamp.chk")
@@ -259,6 +270,13 @@ class RsyncSync(NewBase):
exitcode = EXCEEDED_MAX_RETRIES
break
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
+
+   # if synced successfully, verify now
+   if exitcode == 0 and self.verify_metamanifest:
+   command = ['gemato', 'verify', '-K', 
self.openpgp_key_path,
+   '-s', self.repo.location]
+   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
+
return (exitcode, 

[gentoo-portage-dev] [PATCH v2] rsync: Introduce support for running full-tree gemato verification

[Michał Górny]

Add two new configuration options to rsync repositories:
sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
The first controls whether gemato verification is run for
the repository (defaults to true for ::gentoo, false otherwise),
the second makes it possible to override the key path for custom
repositories.
---
 cnf/repos.conf |  1 +
 man/portage.5  | 11 +++
 pym/portage/sync/modules/rsync/__init__.py |  4 +++-
 pym/portage/sync/modules/rsync/rsync.py| 20 +++-
 4 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/cnf/repos.conf b/cnf/repos.conf
index 062fc0d10..644687515 100644
--- a/cnf/repos.conf
+++ b/cnf/repos.conf
@@ -6,6 +6,7 @@ location = /usr/portage
 sync-type = rsync
 sync-uri = rsync://rsync.gentoo.org/gentoo-portage
 auto-sync = yes
+sync-rsync-verify-metamanifest = yes
 
 # for daily squashfs snapshots
 #sync-type = squashdelta
diff --git a/man/portage.5 b/man/portage.5
index e724e1f08..5e8127778 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,21 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
+.B sync-rsync-openpgp-key-path
+Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
+if \fBsync-rsync-verify-metamanifest\fR is enabled. Defaults to
+\fB/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg\fR (path
+used by \fBapp-crypt/gentoo-keys\fR).
+.TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
+.TP
+.B sync-rsync-verify-metamanifest = true|false
+Require the repository to contain a signed MetaManifest and verify
+it using \fBapp-portage/gemato\fR. Defauls to true for the \fBgentoo\fR
+repository and false otherwise.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index c2fdc4188..df9a1995a 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Rsync plug-in module for portage.
@@ -27,7 +27,9 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
+   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-metamanifest',
),
}
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index c80641ba3..613bedd0c 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -82,6 +82,17 @@ class RsyncSync(NewBase):
self.extra_rsync_opts.extend(portage.util.shlex_split(

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
+   # Process GLEP74 verification options.
+   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   self.verify_metamanifest = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-verify-metamanifest', 
False))
+   # Default to gentoo-keys keyring.
+   self.openpgp_key_path = (
+   self.repo.module_specific_options.get(
+   'sync-rsync-openpgp-key-path',
+   
'/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg'))
+
# Real local timestamp file.
self.servertimestampfile = os.path.join(
self.repo.location, "metadata", "timestamp.chk")
@@ -259,6 +270,13 @@ class RsyncSync(NewBase):
exitcode = EXCEEDED_MAX_RETRIES
break
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
+
+   # if synced successfully, verify now
+   if exitcode == 0 and self.verify_metamanifest:
+   command = ['gemato', 'verify', '-K', 
self.openpgp_key_path,
+   '-s', self.repo.location]
+   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
+
return 

Re: [gentoo-portage-dev] [PATCH] rsync: Introduce support for running full-tree gemato verification

[Zac Medico]

On 01/24/2018 01:03 PM, Michał Górny wrote:
> diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> b/pym/portage/sync/modules/rsync/rsync.py
> index c80641ba3..f3157d093 100644
> --- a/pym/portage/sync/modules/rsync/rsync.py
> +++ b/pym/portage/sync/modules/rsync/rsync.py
> @@ -1,4 +1,4 @@
> -# Copyright 1999-2015 Gentoo Foundation
> +# Copyright 1999-2018 Gentoo Foundation
>  # Distributed under the terms of the GNU General Public License v2
>  
>  import sys
> @@ -82,6 +82,16 @@ class RsyncSync(NewBase):
>   self.extra_rsync_opts.extend(portage.util.shlex_split(
>   
> self.repo.module_specific_options['sync-rsync-extra-opts']))
>  
> + # Process GLEP74 verification options.
> + # Default verification to 'on' for ::gentoo, 'off' otherwise.
> + self.verify_metamanifest = (
> + 
> self.repo.module_specific_options.get('sync-rsync-verify-metamanifest',
> + self.repo.name == 'gentoo'))
> + # Default to gentoo-keys keyring.
> + self.openpgp_key_path = (
> + 
> self.repo.module_specific_options.get('sync-rsync-openpgp-key-path',
> + 
> '/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg'))

Maybe these defaults should go in cnf/repos.conf, and we should have the
ebuild patch the sync-rsync-verify-metamanifest setting depending on the
USE=rsync-verify state?
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH] rsync: Introduce support for running full-tree gemato verification

[Michał Górny]

Add two new configuration options to rsync repositories:
sync-rsync-verify-metamanifest and sync-rsync-openpgp-key-path.
The first controls whether gemato verification is run for
the repository (defaults to true for ::gentoo, false otherwise),
the second makes it possible to override the key path for custom
repositories.
---
 man/portage.5  | 11 +++
 pym/portage/sync/modules/rsync/__init__.py |  4 +++-
 pym/portage/sync/modules/rsync/rsync.py| 19 ++-
 3 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/man/portage.5 b/man/portage.5
index e724e1f08..5e8127778 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,21 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
+.B sync-rsync-openpgp-key-path
+Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
+if \fBsync-rsync-verify-metamanifest\fR is enabled. Defaults to
+\fB/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg\fR (path
+used by \fBapp-crypt/gentoo-keys\fR).
+.TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
+.TP
+.B sync-rsync-verify-metamanifest = true|false
+Require the repository to contain a signed MetaManifest and verify
+it using \fBapp-portage/gemato\fR. Defauls to true for the \fBgentoo\fR
+repository and false otherwise.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index c2fdc4188..df9a1995a 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Rsync plug-in module for portage.
@@ -27,7 +27,9 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
+   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-metamanifest',
),
}
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index c80641ba3..f3157d093 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2015 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import sys
@@ -82,6 +82,16 @@ class RsyncSync(NewBase):
self.extra_rsync_opts.extend(portage.util.shlex_split(

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
+   # Process GLEP74 verification options.
+   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   self.verify_metamanifest = (
+   
self.repo.module_specific_options.get('sync-rsync-verify-metamanifest',
+   self.repo.name == 'gentoo'))
+   # Default to gentoo-keys keyring.
+   self.openpgp_key_path = (
+   
self.repo.module_specific_options.get('sync-rsync-openpgp-key-path',
+   
'/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg'))
+
# Real local timestamp file.
self.servertimestampfile = os.path.join(
self.repo.location, "metadata", "timestamp.chk")
@@ -259,6 +269,13 @@ class RsyncSync(NewBase):
exitcode = EXCEEDED_MAX_RETRIES
break
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
+
+   # if synced successfully, verify now
+   if exitcode == 0 and self.verify_metamanifest:
+   command = ['gemato', 'verify', '-K', 
self.openpgp_key_path,
+   '-s', self.repo.location]
+   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
+
return (exitcode, updatecache_flg)
 
 
-- 
2.16.1

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Zac Medico]

On 01/22/2018 02:28 AM, Andreas K. Huettel wrote:
> Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
>>
>> According to Gentoo policy, future ebuild dependency changes need to be
>> accompanied by a revision bump in order to trigger rebuilds for users.
>> Therefore, you should only need to use --changed-deps=y for a single
>> deep @world update. After that, if you encounter installed packages with
>> outdated dependencies in a future deep @world update, then you should
>> report it as a bug.
> 
> Did you come up with a solution how to handle eclass-generated dependency 
> changes then?
> 
> I'd loathe to have to do identical revision bumps for, say, all perl-
> module.eclass consumers...

Would it be possible to isolate the dependency changes in virtual
ebuilds? If the eclass depends on a virtual, the maybe updating to the
latest version of the virtual can serve as the means apply the needed
dependency change?
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Zac Medico]

On 01/22/2018 01:38 AM, Dirkjan Ochtman wrote:
> On Mon, Jan 22, 2018 at 8:01 AM, Zac Medico  > wrote:
> 
> According to Gentoo policy, future ebuild dependency changes need to be
> accompanied by a revision bump in order to trigger rebuilds for users.
> Therefore, you should only need to use --changed-deps=y for a single
> deep @world update. After that, if you encounter installed packages with
> outdated dependencies in a future deep @world update, then you should
> report it as a bug.
> 
> 
> Can we get repoman to warn about changing dependencies without revbumps?

That will require a bit of work since it's going to have to call bash to
generate metadata for the previous state and compare that to the current
state, but it can be done.

That won't help for eclass changes, though, unless we generate metadata
for all consumers and that's expensive.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Zac Medico]

On 01/22/2018 12:24 AM, Alexis Ballier wrote:
> On Sun, 21 Jan 2018 23:01:08 -0800
> Zac Medico  wrote:
> 
>> Please review.
>>
>> Title: Portage Dynamic Deps
>> Author: Zac Medico 
>> Posted: 2018-01-28
>> Revision: 1
>> News-Item-Format: 2.0
>> Display-If-Installed: >
>> Beginning with Portage 2.3.20, the previous default --dynamic-deps=y
>> setting has changed to --dynamic-deps=n. Due to this change, some
>> users may experience emerge dependency calculation failures triggered
>> by installed packages that have outdated dependencies. In order to
>> avoid problems of this nature, use the emerge --changed-deps=y option
>> with your next deep @world update.
> 
> What's the rationale behind this ?
> 
> What I mean is: while '--dynamic-deps=n --changed-deps=n' is the
> technically correct behavior, this just seems like throwing unbearable
> dep calculation failure messages at users' faces while we could default
> to '--dynamic-deps=n --changed-deps=y' and get the already
> policy-mandated behavior of 'force a rebuild when you change deps'.

It depends on how difficult it is to enforce the revision bumps. If it's
really difficult, then it does make sense to enable --changed-deps=y by
default.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Michał Górny]

W dniu śro, 24.01.2018 o godzinie 12∶29 -0800, użytkownik Zac Medico
napisał:
> On 01/24/2018 12:56 AM, Michał Górny wrote:
> > Hi, everyone.
> > 
> > Since the initial review of my patch lost focus, and lacked sufficient 
> > context, here's the plan that I'd like to follow in order to initially 
> > integrate gemato with portage and give our users secure checkouts by 
> > default.
> > 
> > 1. Add postsync hook to Portage git. Eventually, it will be replaced by 
> > direct Portage support.
> > 
> > 2. Add IUSE=+rsync-verify to portage- that controls installing the 
> > hook. This will give users the ability to easily disable it without jumping 
> > through cross package hoops.
> > 
> > 3. Submit a news item for review that will explain how to initially verify 
> > the keys on existing installations.
> > 
> > The news item would be published when the hook hits a release.
> > 
> > What do you think? If you agree, then I'll start writing the news item.
> > 
> 
> For the sake of maintaining stable interfaces for users, I feel like we
> should add the repos.conf sync-rsync-verify setting for this is
> up-front. That way, we won't have to train people to use a new interface
> later. Also, eventually we have to do this anyway if we want portage to
> recognize the nature of the failure and react by quarantining the
> repository.

Do you mean implementing it completely inside sync code without hooks?
I'm all for it. Will submit a patch soon.

-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Zac Medico]

On 01/24/2018 12:56 AM, Michał Górny wrote:
> Hi, everyone.
> 
> Since the initial review of my patch lost focus, and lacked sufficient 
> context, here's the plan that I'd like to follow in order to initially 
> integrate gemato with portage and give our users secure checkouts by default.
> 
> 1. Add postsync hook to Portage git. Eventually, it will be replaced by 
> direct Portage support.
> 
> 2. Add IUSE=+rsync-verify to portage- that controls installing the hook. 
> This will give users the ability to easily disable it without jumping through 
> cross package hoops.
> 
> 3. Submit a news item for review that will explain how to initially verify 
> the keys on existing installations.
> 
> The news item would be published when the hook hits a release.
> 
> What do you think? If you agree, then I'll start writing the news item.
> 

For the sake of maintaining stable interfaces for users, I feel like we
should add the repos.conf sync-rsync-verify setting for this is
up-front. That way, we won't have to train people to use a new interface
later. Also, eventually we have to do this anyway if we want portage to
recognize the nature of the failure and react by quarantining the
repository.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Alec Warner]

On Wed, Jan 24, 2018 at 2:58 PM, Michał Górny  wrote:

> W dniu śro, 24.01.2018 o godzinie 12∶54 -0500, użytkownik Alec Warner
> napisał:
> > On Wed, Jan 24, 2018 at 3:56 AM, Michał Górny  wrote:
> >
> > > Hi, everyone.
> > >
> > > Since the initial review of my patch lost focus, and lacked sufficient
> > > context, here's the plan that I'd like to follow in order to initially
> > > integrate gemato with portage and give our users secure checkouts by
> > > default.
> > >
> > > 1. Add postsync hook to Portage git. Eventually, it will be replaced by
> > > direct Portage support.
> > >
> > > 2. Add IUSE=+rsync-verify to portage- that controls installing the
> > > hook. This will give users the ability to easily disable it without
> jumping
> > > through cross package hoops.
> > >
> >
> > I think it makes sense to avoid installing the hook through this means
> > (e.g. I don't want it, so I set USE=-rsync-verify)
> >
> > I think its a bit trickier to control the hook's behavior. For instance:
> >
> > 1) I install portage[rsync-verify]. This installs the hook.
> > 2) I end up not liking the hook, I install portage[-rsync-verify]
> > 3) Does the hook get config-protected here?
>
> Keeping config-protected files applies only if the file were modified.
> In this case it just gets unmerged. I've just verified that.
>
> > > 3. Submit a news item for review that will explain how to initially
> verify
> > > the keys on existing installations.
> > >
> > > The news item would be published when the hook hits a release.
> > >
> > > What do you think? If you agree, then I'll start writing the news item.
> > >
> >
> > The other part of the user story I don't understand is what actions
> should
> > users take when verification legit fails?
> >
> > 1) file a bug?
> > 2) re-sync their tree?
> > 3) something else?
>
> I'd say one re-sync would be reasonable in case it could have been some
> rsync glitch. Beyond that, it would definitely be worth reporting...
> except the user will probably hit another mirror.
>

This sgtm.

If the number of reports is high we can build more complex reporting
infrastructure (e.g. to locate bad mirrors.)

-A


>
> --
> Best regards,
> Michał Górny
>
>
>

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Michał Górny]

W dniu śro, 24.01.2018 o godzinie 12∶54 -0500, użytkownik Alec Warner
napisał:
> On Wed, Jan 24, 2018 at 3:56 AM, Michał Górny  wrote:
> 
> > Hi, everyone.
> > 
> > Since the initial review of my patch lost focus, and lacked sufficient
> > context, here's the plan that I'd like to follow in order to initially
> > integrate gemato with portage and give our users secure checkouts by
> > default.
> > 
> > 1. Add postsync hook to Portage git. Eventually, it will be replaced by
> > direct Portage support.
> > 
> > 2. Add IUSE=+rsync-verify to portage- that controls installing the
> > hook. This will give users the ability to easily disable it without jumping
> > through cross package hoops.
> > 
> 
> I think it makes sense to avoid installing the hook through this means
> (e.g. I don't want it, so I set USE=-rsync-verify)
> 
> I think its a bit trickier to control the hook's behavior. For instance:
> 
> 1) I install portage[rsync-verify]. This installs the hook.
> 2) I end up not liking the hook, I install portage[-rsync-verify]
> 3) Does the hook get config-protected here?

Keeping config-protected files applies only if the file were modified.
In this case it just gets unmerged. I've just verified that.

> > 3. Submit a news item for review that will explain how to initially verify
> > the keys on existing installations.
> > 
> > The news item would be published when the hook hits a release.
> > 
> > What do you think? If you agree, then I'll start writing the news item.
> > 
> 
> The other part of the user story I don't understand is what actions should
> users take when verification legit fails?
> 
> 1) file a bug?
> 2) re-sync their tree?
> 3) something else?

I'd say one re-sync would be reasonable in case it could have been some
rsync glitch. Beyond that, it would definitely be worth reporting...
except the user will probably hit another mirror.

-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] Plan for initial integration of gemato with portage

[Alec Warner]

On Wed, Jan 24, 2018 at 3:56 AM, Michał Górny  wrote:

> Hi, everyone.
>
> Since the initial review of my patch lost focus, and lacked sufficient
> context, here's the plan that I'd like to follow in order to initially
> integrate gemato with portage and give our users secure checkouts by
> default.
>
> 1. Add postsync hook to Portage git. Eventually, it will be replaced by
> direct Portage support.
>

> 2. Add IUSE=+rsync-verify to portage- that controls installing the
> hook. This will give users the ability to easily disable it without jumping
> through cross package hoops.
>

I think it makes sense to avoid installing the hook through this means
(e.g. I don't want it, so I set USE=-rsync-verify)

I think its a bit trickier to control the hook's behavior. For instance:

1) I install portage[rsync-verify]. This installs the hook.
2) I end up not liking the hook, I install portage[-rsync-verify]
3) Does the hook get config-protected here?
4) I run emerge --depclean # gemato is un-merged here.

Now the hook fails because gemato is gone but the hook remains due to
config-protect?

5) I run etc-update and the hook is deleted; I don't recall etc-update
deleting files...

So basically I'm curious how the hook works w/config-protect; but
admittedly its been a while since I worried about such things so I could be
wrong.

Other considerations include:

1) Hooks don't matter anyway, so we may not care if it breaks.
2) Verification doesn't matter currently (portage will install software
regardless), so we may not care if it doesn't function.


>
> 3. Submit a news item for review that will explain how to initially verify
> the keys on existing installations.
>
> The news item would be published when the hook hits a release.
>
> What do you think? If you agree, then I'll start writing the news item.
>

The other part of the user story I don't understand is what actions should
users take when verification legit fails?

1) file a bug?
2) re-sync their tree?
3) something else?


--
> Best regards,
> Michał Górny (by phone)
>
>

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Ulrich Mueller]

> On Wed, 24 Jan 2018, Martin Vaeth wrote:

> Ulrich Mueller  wrote:
>> "Runtime dependencies (RDEPEND). These must be installed and usable
>> before the results of an ebuild merging are treated as usable."
>> https://projects.gentoo.org/pms/6/pms.html#x1-770008.1
>> 
>> IMHO this implies that the dependencies at merge time are the
>> relevant ones

> IMHO this specifies what is relevant when an emerge merging happens.
> Nothing more, nothing less.

Exactly. RDEPEND is to be evaluated at the time before the package is
merged. For PDEPEND it is even clearer: "These must be installed at
some point before the package manager finishes the batch of installs."

> _If_ one would be willing to interpret it to have a meaning also
> _after_ the emerge, then of course the RDEPEND in PMS can refer to
> the only value which is specified in PMS, i.e. that stored in the
> ebuild

... at the time when the package was merged. You cannot rely on
anything else, because the ebuild may be gone in the meantime.

> (and not in any database which is explicitly not specified by PMS).
> In other words: _If_ one puts any unsaid interpretation into that
> sentence, then this can only be dynamic deps.

No. The only thing that PMS doesn't specify is the special format of
the VDB. However, the spec says that variables must keep their values
between phase functions, which includes pkg_prerm() and pkg_postrm().
By this, *some* sort of storage mechanism must exist.

Ulrich


pgpgC4EFl2XHz.pgp
Description: PGP signature

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Martin Vaeth]

Ulrich Mueller  wrote:
> This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
> --pgp+signed++Zg8D+I6sgRUw0D
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
>
>> On Wed, 24 Jan 2018, Martin Vaeth wrote:
>
>> Rich Freeman  wrote:
>>> 
>>> It would already be broken on any PMS-compliant package manager
>
>> No, it is solely the interpretation of the package manager.
>> PMS does not specify what information is stored in /var/db
>> or how that information is used.
>
> "Runtime dependencies (RDEPEND). These must be installed and usable
> before the results of an ebuild merging are treated as usable."
> https://projects.gentoo.org/pms/6/pms.html#x1-770008.1
>
> IMHO this implies that the dependencies at merge time are the relevant
> ones

IMHO this specifies what is relevant when an emerge merging happens.
Nothing more, nothing less.
_If_ one would be willing to interpret it to have a meaning also _after_
the emerge, then of course the RDEPEND in PMS can refer to the only value
which is specified in PMS, i.e. that stored in the ebuild (and not in
any database which is explicitly not specified by PMS).
In other words: _If_ one puts any unsaid interpretation into that
sentence, then this can only be dynamic deps.

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Ulrich Mueller]

> On Wed, 24 Jan 2018, Martin Vaeth wrote:

> Rich Freeman  wrote:
>> 
>> It would already be broken on any PMS-compliant package manager

> No, it is solely the interpretation of the package manager.
> PMS does not specify what information is stored in /var/db
> or how that information is used.

"Runtime dependencies (RDEPEND). These must be installed and usable
before the results of an ebuild merging are treated as usable."
https://projects.gentoo.org/pms/6/pms.html#x1-770008.1

IMHO this implies that the dependencies at merge time are the relevant
ones, but not any later changes to the ebuild.

Ulrich


pgpBFJdeNNmzj.pgp
Description: PGP signature

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Martin Vaeth]

Rich Freeman  wrote:
>
> It would already be broken on any PMS-compliant package manager

No, it is solely the interpretation of the package manager.
PMS does not specify what information is stored in /var/db
or how that information is used.

[gentoo-portage-dev] Plan for initial integration of gemato with portage

[Michał Górny]

Hi, everyone.

Since the initial review of my patch lost focus, and lacked sufficient context, 
here's the plan that I'd like to follow in order to initially integrate gemato 
with portage and give our users secure checkouts by default.

1. Add postsync hook to Portage git. Eventually, it will be replaced by direct 
Portage support.

2. Add IUSE=+rsync-verify to portage- that controls installing the hook. 
This will give users the ability to easily disable it without jumping through 
cross package hoops.

3. Submit a news item for review that will explain how to initially verify the 
keys on existing installations.

The news item would be published when the hook hits a release.

What do you think? If you agree, then I'll start writing the news item.
-- 
Best regards,
Michał Górny (by phone)