Re: [gentoo-dev] Merge 7 Fedora wallpapers packages to single one with slots?

[Sebastian Pipping]

Hi Alec,


On 27.01.2018 22:58, Alec Warner wrote:
> > I noticed that we have 7 packages on Fedora wallpapers with names that
> > only explain themselves to Fedora insiders:
> 
> So traditionally we follow upstream package naming. If we aim to
> deviate, I'd prefer we have strong reasons for it.

good point.


> > I was thinking that we could merge these packages into a new package
> > "x11-themes/fedora-backgrounds" or so with slots 11 to 16 so that people
> > can still install them in parallel, get slot updates automatically,
> > adding more recent ones does not add more packages, and the package name
> > explains itself.
> 
> Why not just make x11-themes/fedora-backgrounds, a metapackage that
> includes all of the packages?

With one file and use flags for each version or with one ebuild file per
slot?  Fedora 21 was the last release with a release name so if we
package 22+ later, their ebuilds would be non-meta in nature.  I'm not
sure how to blend that into the use-flag version (yet for a meta package
all these files seem overkill too).  Do you have some third option in mind?

Best



Sebastian

Re: [gentoo-portage-dev] [PATCH 5/5] rsync: Issue an explicit warning if Manifest timestamp is >24hr old

[Zac Medico]

On 02/01/2018 04:17 AM, Michał Górny wrote:
> Issue an explicit warning if the Manifest timestamp is 24 hours behind
> the system clock. This is meant to detect attacks based on preventing
> the user from upgrading.
> ---
>  pym/portage/sync/modules/rsync/rsync.py | 6 ++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> b/pym/portage/sync/modules/rsync/rsync.py
> index b7758c2c3..9c352b139 100644
> --- a/pym/portage/sync/modules/rsync/rsync.py
> +++ b/pym/portage/sync/modules/rsync/rsync.py
> @@ -6,6 +6,7 @@ import logging
>  import time
>  import signal
>  import socket
> +import datetime
>  import io
>  import re
>  import random
> @@ -339,6 +340,11 @@ class RsyncSync(NewBase):
>   ts = m.find_timestamp()
>   if ts is None:
>   raise 
> RuntimeError('Timestamp not found in Manifest')
> + if (datetime.datetime.utcnow() 
> - ts.ts).days > 1:
> + out.ewarn('Manifest is 
> over 24 hours old, this is suspicious!')
> + out.ewarn('You may want 
> to try using another mirror and/or reporting this one:')
> + out.ewarn('  %s' % 
> (dosyncuri,))
> + out.ewarn('')
>  
>   out.einfo('Manifest timestamp: 
> %s UTC' % (ts.ts,))
>   out.einfo('Valid OpenPGP 
> signature found:')
> 

This warning is gentoo-centric. We probably want to be able to override
the number of days via layout.conf and repos.conf.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 2/5] rsync: Use gemato routines directly instead of calling the CLI tool

[Michał Górny]

W dniu czw, 01.02.2018 o godzinie 12∶50 -0800, użytkownik Zac Medico
napisał:
> On 02/01/2018 04:17 AM, Michał Górny wrote:
> > +   except Exception as e:
> > +   writemsg_level("!!! Manifest 
> > verification failed:\n%s\n"
> > +   % (e,),
> > +   level=logging.ERROR, 
> > noiselevel=-1)
> > +   exitcode = 1
> 
> If an unexpected exception occurs then this makes it difficult to
> diagnose. Maybe use PortageException instead of RuntimeError, and use
> traceback.print_exc or portage.util.dump_traceback if an unexpected
> exception occurs?

I was wondering how to handle this case. Hmm... maybe I'll also derive
all gemato exceptions from a single one, and catch that. Then we can
just let unexpected exceptions explode normally ;-).

-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] [PATCH 2/5] rsync: Use gemato routines directly instead of calling the CLI tool

[Zac Medico]

On 02/01/2018 04:17 AM, Michał Górny wrote:
> + except Exception as e:
> + writemsg_level("!!! Manifest 
> verification failed:\n%s\n"
> + % (e,),
> + level=logging.ERROR, 
> noiselevel=-1)
> + exitcode = 1

If an unexpected exception occurs then this makes it difficult to
diagnose. Maybe use PortageException instead of RuntimeError, and use
traceback.print_exc or portage.util.dump_traceback if an unexpected
exception occurs?
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 1/2] openpgp_key_path -> sync_openpgp_key_path

[Zac Medico]

On 02/01/2018 12:25 PM, Michał Górny wrote:
> Rename the openpgp_key_path attribute to sync_openpgp_key_path since
> Portage code assumes that attribute name matches config key.
> ---
>  pym/portage/repository/config.py| 6 +++---
>  pym/portage/sync/modules/rsync/rsync.py | 4 ++--
>  2 files changed, 5 insertions(+), 5 deletions(-)

Both patches look good. Please merge.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH 2/2] repository.config: Fix handling sync-openpgp-key-path

[Michał Górny]

Fix sync-openpgp-key-path to be correctly inherited from system
repos.conf and preserved in repo dumps.

Bug: https://bugs.gentoo.org/646314
---
 pym/portage/repository/config.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index a71047d9b..52f44d526 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -522,6 +522,7 @@ class RepoConfigLoader(object):
'clone_depth', 
'eclass_overrides',
'force', 'masters', 
'priority', 'strict_misc_digests',
'sync_depth', 
'sync_hooks_only_on_change',
+   'sync_openpgp_key_path',
'sync_type', 
'sync_umask', 'sync_uri', 'sync_user',

'module_specific_options'):
v = 
getattr(repos_conf_opts, k, None)
@@ -944,7 +945,7 @@ class RepoConfigLoader(object):
def config_string(self):
bool_keys = ("strict_misc_digests",)
str_or_int_keys = ("auto_sync", "clone_depth", "format", 
"location",
-   "main_repo", "priority", "sync_depth",
+   "main_repo", "priority", "sync_depth", 
"sync_openpgp_key_path",
"sync_type", "sync_umask", "sync_uri", 'sync_user')
str_tuple_keys = ("aliases", "eclass_overrides", "force")
repo_config_tuple_keys = ("masters",)
-- 
2.16.1

[gentoo-portage-dev] [PATCH 1/2] openpgp_key_path -> sync_openpgp_key_path

[Michał Górny]

Rename the openpgp_key_path attribute to sync_openpgp_key_path since
Portage code assumes that attribute name matches config key.
---
 pym/portage/repository/config.py| 6 +++---
 pym/portage/sync/modules/rsync/rsync.py | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index d3a622f7c..a71047d9b 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -1,4 +1,4 @@
-# Copyright 2010-2017 Gentoo Foundation
+# Copyright 2010-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 from __future__ import unicode_literals
@@ -86,7 +86,7 @@ class RepoConfig(object):
'sync_type', 'sync_umask', 'sync_uri', 'sync_user', 
'thin_manifest',
'update_changelog', '_eapis_banned', '_eapis_deprecated',
'_masters_orig', 'module_specific_options', 
'manifest_required_hashes',
-   'openpgp_key_path',
+   'sync_openpgp_key_path',
)
 
def __init__(self, name, repo_opts, local_config=True):
@@ -183,7 +183,7 @@ class RepoConfig(object):
self.strict_misc_digests = repo_opts.get(
'strict-misc-digests', 'true').lower() == 'true'
 
-   self.openpgp_key_path = repo_opts.get(
+   self.sync_openpgp_key_path = repo_opts.get(
'sync-openpgp-key-path', None)
 
self.module_specific_options = {}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index e2e6d0658..4471f5bbe 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -277,8 +277,8 @@ class RsyncSync(NewBase):
# if synced successfully, verify now
if exitcode == 0 and not local_state_unchanged and 
self.verify_metamanifest:
command = ['gemato', 'verify', '-s', self.repo.location]
-   if self.repo.openpgp_key_path is not None:
-   command += ['-K', self.repo.openpgp_key_path]
+   if self.repo.sync_openpgp_key_path is not None:
+   command += ['-K', 
self.repo.sync_openpgp_key_path]
if self.verify_jobs is not None:
command += ['-j', self.verify_jobs]
try:
-- 
2.16.1

[gentoo-portage-dev] [PATCH 2/2] git: Support running the verification against sync-openpgp-key-path

[Michał Górny]

---
 pym/portage/sync/modules/git/git.py | 100 +---
 1 file changed, 69 insertions(+), 31 deletions(-)

diff --git a/pym/portage/sync/modules/git/git.py 
b/pym/portage/sync/modules/git/git.py
index 7e5ddf3b5..2b40db0fa 100644
--- a/pym/portage/sync/modules/git/git.py
+++ b/pym/portage/sync/modules/git/git.py
@@ -1,6 +1,7 @@
 # Copyright 2005-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
+import io
 import logging
 import subprocess
 
@@ -13,6 +14,11 @@ bad = create_color_func("BAD")
 warn = create_color_func("WARN")
 from portage.sync.syncbase import NewBase
 
+try:
+   import gemato.openpgp
+except ImportError:
+   gemato = None
+
 
 class GitSync(NewBase):
'''Git sync class'''
@@ -141,39 +147,71 @@ class GitSync(NewBase):
'sync-git-verify-commit-signature', 'false') != 
'true'):
return True
 
-   rev_cmd = [self.bin_command, "log", "--pretty=format:%G?", "-1"]
-   try:
-   status = (portage._unicode_decode(
-   subprocess.check_output(rev_cmd,
-   
cwd=portage._unicode_encode(self.repo.location)))
-   .strip())
-   except subprocess.CalledProcessError:
-   return False
-
-   out = EOutput()
-   if status == 'G':  # good signature is good
-   out.einfo('Trusted signature found on top commit')
-   return True
-   elif status == 'U':  # untrusted
-   out.ewarn('Top commit signature is valid but not 
trusted')
-   return True
+   if self.repo.openpgp_key_path is not None:
+   if gemato is None:
+   writemsg_level("!!! Verifying against specified 
key requires gemato installed",
+   level=logging.ERROR, noiselevel=-1)
+   return False
+   openpgp_env = gemato.openpgp.OpenPGPEnvironment()
else:
-   if status == 'B':
-   expl = 'bad signature'
-   elif status == 'X':
-   expl = 'expired signature'
-   elif status == 'Y':
-   expl = 'expired key'
-   elif status == 'R':
-   expl = 'revoked key'
-   elif status == 'E':
-   expl = 'unable to verify signature (missing 
key?)'
-   elif status == 'N':
-   expl = 'no signature'
+   openpgp_env = None
+
+   try:
+   out = EOutput()
+   env = None
+   if openpgp_env is not None:
+   try:
+   out.einfo('Using keys from %s' % 
(self.repo.openpgp_key_path,))
+   with 
io.open(self.repo.openpgp_key_path, 'rb') as f:
+   openpgp_env.import_key(f)
+   out.ebegin('Refreshing keys from 
keyserver')
+   openpgp_env.refresh_keys()
+   out.eend(0)
+   except Exception as e:
+   writemsg_level("!!! Verification 
impossible due to keyring problem:\n%s\n"
+   % (e,),
+   level=logging.ERROR, 
noiselevel=-1)
+   return (1, False)
+
+   env = os.environ.copy()
+   env['GNUPGHOME'] = openpgp_env.home
+
+   rev_cmd = [self.bin_command, "log", 
"--pretty=format:%G?", "-1"]
+   try:
+   status = (portage._unicode_decode(
+   subprocess.check_output(rev_cmd,
+   
cwd=portage._unicode_encode(self.repo.location),
+   env=env))
+   .strip())
+   except subprocess.CalledProcessError:
+   return False
+
+   if status == 'G':  # good signature is good
+   out.einfo('Trusted signature found on top 
commit')
+   return True
+   elif status == 'U':  # untrusted
+   out.ewarn('Top commit signature is valid but 
not trusted')
+

[gentoo-portage-dev] [PATCH 1/2] git: Support verifying commit signature post-sync

[Michał Górny]

Add a new sync-git-verify-commit-signature option (defaulting to false)
that verifies the top commit signature after syncing. The verification
is currently done using built-in git routines.

The verification passes if the signature is good or untrusted.
In the latter case, a warning is printed. In any other case,
the verification causes sync to fail and an appropriate error is output.
---
 man/portage.5|  4 +++
 pym/portage/sync/modules/git/__init__.py |  3 +-
 pym/portage/sync/modules/git/git.py  | 48 ++--
 3 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/man/portage.5 b/man/portage.5
index d4f755f51..da5a02f5a 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1007,6 +1007,10 @@ See also example for sync-git-clone-env.
 .B sync\-git\-pull\-extra\-opts
 Extra options to give to git when updating repository (git pull).
 .TP
+.B sync\-git\-verify\-commit\-signature = true|false
+Require the top commit in the repository to contain a good OpenPGP
+signature. Defaults to false.
+.TP
 .B sync\-hooks\-only\-on\-change
 If set to true, then sync of a given repository will not trigger postsync
 hooks unless hooks would have executed for a master repository or the
diff --git a/pym/portage/sync/modules/git/__init__.py 
b/pym/portage/sync/modules/git/__init__.py
index 2f1d35226..270d97186 100644
--- a/pym/portage/sync/modules/git/__init__.py
+++ b/pym/portage/sync/modules/git/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 2014-2017 Gentoo Foundation
+# Copyright 2014-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 doc = """Git plug-in module for portage.
@@ -58,6 +58,7 @@ module_spec = {
'sync-git-env',
'sync-git-pull-env',
'sync-git-pull-extra-opts',
+   'sync-git-verify-commit-signature',
),
}
}
diff --git a/pym/portage/sync/modules/git/git.py 
b/pym/portage/sync/modules/git/git.py
index 8b4cab273..7e5ddf3b5 100644
--- a/pym/portage/sync/modules/git/git.py
+++ b/pym/portage/sync/modules/git/git.py
@@ -1,4 +1,4 @@
-# Copyright 2005-2017 Gentoo Foundation
+# Copyright 2005-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 import logging
@@ -7,7 +7,7 @@ import subprocess
 import portage
 from portage import os
 from portage.util import writemsg_level, shlex_split
-from portage.output import create_color_func
+from portage.output import create_color_func, EOutput
 good = create_color_func("GOOD")
 bad = create_color_func("BAD")
 warn = create_color_func("WARN")
@@ -71,6 +71,7 @@ class GitSync(NewBase):
else:
# default
git_cmd_opts += " --depth 1"
+
if 
self.repo.module_specific_options.get('sync-git-clone-extra-opts'):
git_cmd_opts += " %s" % 
self.repo.module_specific_options['sync-git-clone-extra-opts']
git_cmd = "%s clone%s %s ." % (self.bin_command, git_cmd_opts,
@@ -85,6 +86,8 @@ class GitSync(NewBase):
self.logger(self.xterm_titles, msg)
writemsg_level(msg + "\n", level=logging.ERROR, 
noiselevel=-1)
return (exitcode, False)
+   if not self.verify_head():
+   return (1, False)
return (os.EX_OK, True)
 
 
@@ -125,12 +128,53 @@ class GitSync(NewBase):
self.logger(self.xterm_titles, msg)
writemsg_level(msg + "\n", level=logging.ERROR, 
noiselevel=-1)
return (exitcode, False)
+   if not self.verify_head():
+   return (1, False)
 
current_rev = subprocess.check_output(rev_cmd,
cwd=portage._unicode_encode(self.repo.location))
 
return (os.EX_OK, current_rev != previous_rev)
 
+   def verify_head(self):
+   if (self.repo.module_specific_options.get(
+   'sync-git-verify-commit-signature', 'false') != 
'true'):
+   return True
+
+   rev_cmd = [self.bin_command, "log", "--pretty=format:%G?", "-1"]
+   try:
+   status = (portage._unicode_decode(
+   subprocess.check_output(rev_cmd,
+   
cwd=portage._unicode_encode(self.repo.location)))
+   .strip())
+   except subprocess.CalledProcessError:
+   return False
+
+   out = EOutput()
+   if status == 'G':  # good signature is good
+   out.einfo('Trusted signature found on top commit')
+   return True
+   elif status == 'U':  # untrusted
+ 

[gentoo-portage-dev] [PATCH 5/5] rsync: Issue an explicit warning if Manifest timestamp is >24hr old

[Michał Górny]

Issue an explicit warning if the Manifest timestamp is 24 hours behind
the system clock. This is meant to detect attacks based on preventing
the user from upgrading.
---
 pym/portage/sync/modules/rsync/rsync.py | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index b7758c2c3..9c352b139 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -6,6 +6,7 @@ import logging
 import time
 import signal
 import socket
+import datetime
 import io
 import re
 import random
@@ -339,6 +340,11 @@ class RsyncSync(NewBase):
ts = m.find_timestamp()
if ts is None:
raise 
RuntimeError('Timestamp not found in Manifest')
+   if (datetime.datetime.utcnow() 
- ts.ts).days > 1:
+   out.ewarn('Manifest is 
over 24 hours old, this is suspicious!')
+   out.ewarn('You may want 
to try using another mirror and/or reporting this one:')
+   out.ewarn('  %s' % 
(dosyncuri,))
+   out.ewarn('')
 
out.einfo('Manifest timestamp: 
%s UTC' % (ts.ts,))
out.einfo('Valid OpenPGP 
signature found:')
-- 
2.16.1

[gentoo-portage-dev] [PATCH 4/5] rsync: Load and update keys early

[Michał Górny]

Load and update keys early to avoid delaying failures post rsync. Any
failure will prevent verification from happening, and presumably most of
the users will prefer fixing it and trying to sync again. For that case,
it is better to perform the task before actual rsync to avoid
unnecessarily rsyncing twice.
---
 pym/portage/sync/modules/rsync/rsync.py | 416 +---
 1 file changed, 215 insertions(+), 201 deletions(-)

diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index b1faf1ec9..b7758c2c3 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -110,211 +110,222 @@ class RsyncSync(NewBase):
level=logging.WARNING, noiselevel=-1)
self.verify_jobs = None
 
-   # Real local timestamp file.
-   self.servertimestampfile = os.path.join(
-   self.repo.location, "metadata", "timestamp.chk")
-
-   content = portage.util.grabfile(self.servertimestampfile)
-   timestamp = 0
-   if content:
-   try:
-   timestamp = 
time.mktime(time.strptime(content[0],
-   TIMESTAMP_FORMAT))
-   except (OverflowError, ValueError):
-   pass
-   del content
-
-   try:
-   self.rsync_initial_timeout = \
-   
int(self.settings.get("PORTAGE_RSYNC_INITIAL_TIMEOUT", "15"))
-   except ValueError:
-   self.rsync_initial_timeout = 15
+   openpgp_env = None
+   if self.verify_metamanifest and gemato is not None:
+   # Use isolated environment if key is specified,
+   # system environment otherwise
+   if self.repo.openpgp_key_path is not None:
+   openpgp_env = 
gemato.openpgp.OpenPGPEnvironment()
+   else:
+   openpgp_env = 
gemato.openpgp.OpenPGPSystemEnvironment()
 
try:
-   maxretries=int(self.settings["PORTAGE_RSYNC_RETRIES"])
-   except SystemExit as e:
-   raise # Needed else can't exit
-   except:
-   maxretries = -1 #default number of retries
-
-   if syncuri.startswith("file://"):
-   self.proto = "file"
-   dosyncuri = syncuri[7:]
-   unchanged, is_synced, exitcode, updatecache_flg = 
self._do_rsync(
-   dosyncuri, timestamp, opts)
-   self._process_exitcode(exitcode, dosyncuri, out, 1)
-   return (exitcode, updatecache_flg)
+   # Load and update the keyring early. If it fails, then 
verification
+   # will not be performed and the user will have to fix 
it and try again,
+   # so we may as well bail out before actual rsync 
happens.
+   if openpgp_env is not None and 
self.repo.openpgp_key_path is not None:
+   try:
+   out.einfo('Using keys from %s' % 
(self.repo.openpgp_key_path,))
+   with 
io.open(self.repo.openpgp_key_path, 'rb') as f:
+   openpgp_env.import_key(f)
+   out.ebegin('Refreshing keys from 
keyserver')
+   openpgp_env.refresh_keys()
+   out.eend(0)
+   except Exception as e:
+   writemsg_level("!!! Manifest 
verification impossible due to keyring problem:\n%s\n"
+   % (e,),
+   level=logging.ERROR, 
noiselevel=-1)
+   return (1, False)
 
-   retries=0
-   try:
-   self.proto, user_name, hostname, port = re.split(
-   
r"(rsync|ssh)://([^:/]+@)?(\[[:\da-fA-F]*\]|[^:/]*)(:[0-9]+)?",
-   syncuri, maxsplit=4)[1:5]
-   except ValueError:
-   writemsg_level("!!! sync-uri is invalid: %s\n" % 
syncuri,
-   noiselevel=-1, level=logging.ERROR)
-   return (1, False)
+   # Real local timestamp file.
+   self.servertimestampfile = os.path.join(
+   self.repo.location, "metadata", "timestamp.chk")
 
-   self.ssh_opts = self.settings.get("PORTAGE_SSH_OPTS")
+  

[gentoo-portage-dev] [PATCH 2/5] rsync: Use gemato routines directly instead of calling the CLI tool

[Michał Górny]

---
 pym/portage/sync/modules/rsync/rsync.py | 66 -
 1 file changed, 57 insertions(+), 9 deletions(-)

diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index 6b0280032..f6e59e211 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -6,6 +6,7 @@ import logging
 import time
 import signal
 import socket
+import io
 import re
 import random
 import tempfile
@@ -25,6 +26,13 @@ from portage.sync.getaddrinfo_validate import 
getaddrinfo_validate
 from _emerge.UserQuery import UserQuery
 from portage.sync.syncbase import NewBase
 
+try:
+   import gemato.exceptions
+   import gemato.openpgp
+   import gemato.recursiveloader
+except ImportError:
+   gemato = None
+
 if sys.hexversion >= 0x300:
# pylint: disable=W0622
_unicode = str
@@ -285,17 +293,57 @@ class RsyncSync(NewBase):
 
# if synced successfully, verify now
if exitcode == 0 and not local_state_unchanged and 
self.verify_metamanifest:
-   command = ['gemato', 'verify', '-s', self.repo.location]
-   if self.repo.openpgp_key_path is not None:
-   command += ['-K', self.repo.openpgp_key_path]
-   if self.verify_jobs is not None:
-   command += ['-j', str(self.verify_jobs)]
-   try:
-   exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
-   except CommandNotFound as e:
-   writemsg_level("!!! Command not found: %s\n" % 
(command[0],),
+   if gemato is None:
+   writemsg_level("!!! Unable to verify: gemato 
not found\n",
level=logging.ERROR, noiselevel=-1)
exitcode = 127
+   else:
+   # Use isolated environment if key is specified,
+   # system environment otherwise
+   if self.repo.openpgp_key_path is not None:
+   openpgp_env_cls = 
gemato.openpgp.OpenPGPEnvironment
+   else:
+   openpgp_env_cls = 
gemato.openpgp.OpenPGPSystemEnvironment
+
+   try:
+   with openpgp_env_cls() as openpgp_env:
+   if self.repo.openpgp_key_path 
is not None:
+   out.einfo('Using keys 
from %s' % (self.repo.openpgp_key_path,))
+   with 
io.open(self.repo.openpgp_key_path, 'rb') as f:
+   
openpgp_env.import_key(f)
+   out.ebegin('Refreshing 
keys from keyserver')
+   
openpgp_env.refresh_keys()
+   out.eend(0)
+
+   m = 
gemato.recursiveloader.ManifestRecursiveLoader(
+   
os.path.join(self.repo.location, 'Manifest'),
+   
verify_openpgp=True,
+   
openpgp_env=openpgp_env,
+   
max_jobs=self.verify_jobs)
+   if not m.openpgp_signed:
+   raise 
RuntimeError('OpenPGP signature not found on Manifest')
+
+   ts = m.find_timestamp()
+   if ts is None:
+   raise 
RuntimeError('Timestamp not found in Manifest')
+
+   out.einfo('Manifest timestamp: 
%s UTC' % (ts.ts,))
+   out.einfo('Valid OpenPGP 
signature found:')
+   out.einfo('- primary key: %s' % 
(
+   
m.openpgp_signature.primary_key_fingerprint))
+   out.einfo('- subkey: %s' % (
+   
m.openpgp_signature.fingerprint))
+   out.einfo('- timestamp: %s UTC' 
% (
+   
m.openpgp_signature.timestamp))
+
+   out.ebegin('Verifying %s' % 
(self.repo.location,))
+ 

[gentoo-portage-dev] [PATCH 1/5] rsync: Verify the value of sync-rsync-verify-jobs

[Michał Górny]

---
 pym/portage/sync/modules/rsync/rsync.py | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index e2e6d0658..6b0280032 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -92,6 +92,15 @@ class RsyncSync(NewBase):
# Support overriding job count.
self.verify_jobs = self.repo.module_specific_options.get(
'sync-rsync-verify-jobs', None)
+   if self.verify_jobs is not None:
+   try:
+   self.verify_jobs = int(self.verify_jobs)
+   if self.verify_jobs <= 0:
+   raise ValueError(self.verify_jobs)
+   except ValueError:
+   writemsg_level("!!! sync-rsync-verify-jobs not 
a positive integer: %s\n" % (self.verify_jobs,),
+   level=logging.WARNING, noiselevel=-1)
+   self.verify_jobs = None
 
# Real local timestamp file.
self.servertimestampfile = os.path.join(
@@ -280,7 +289,7 @@ class RsyncSync(NewBase):
if self.repo.openpgp_key_path is not None:
command += ['-K', self.repo.openpgp_key_path]
if self.verify_jobs is not None:
-   command += ['-j', self.verify_jobs]
+   command += ['-j', str(self.verify_jobs)]
try:
exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
except CommandNotFound as e:
-- 
2.16.1

[gentoo-portage-dev] [PATCH] rsync: Improve gemato rsync Manifest verification logic

[Michał Górny]

Hi,

Here's a batch of patches meant to be merged after the bugfix release.
They replace the calls to "gemato" executable with Python routine calls,
and further improve the security.

The notable improvements are:

1. New shiny e-style output ;-).

2. Manifest signature is always verified, even if the tree is considered
   unchanged. This allows us to detect recent key revocation even
   without having other changes to verify.

3. OpenPGP keys are loaded and updated before rsync. This allows us
   to bail out early, and let the user fix the situation without having
   to do the whole rsync routine multiple times.

4. Manifest timestamp is compared to the local clock, and a warning
   is issued if the tree received is at least 24 hours old. This allows
   us to detect attacks based on preventing the user from upgrading.

--
Best regards,
Michał Górny

[gentoo-portage-dev] [PATCH 3/5] rsync: Verify the Manifest signature even if tree is unchanged

[Michał Górny]

Always verify the Manifest signature if verification is enabled.
Skipping the deep tree verification for unchanged case is reasonable
but we need to make sure the Manifest signature stays valid to catch
the case of the signing key being revoked.
---
 pym/portage/sync/modules/rsync/rsync.py | 13 +
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index f6e59e211..b1faf1ec9 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -292,7 +292,7 @@ class RsyncSync(NewBase):
self._process_exitcode(exitcode, dosyncuri, out, maxretries)
 
# if synced successfully, verify now
-   if exitcode == 0 and not local_state_unchanged and 
self.verify_metamanifest:
+   if exitcode == 0 and self.verify_metamanifest:
if gemato is None:
writemsg_level("!!! Unable to verify: gemato 
not found\n",
level=logging.ERROR, noiselevel=-1)
@@ -315,6 +315,8 @@ class RsyncSync(NewBase):

openpgp_env.refresh_keys()
out.eend(0)
 
+   # we always verify the Manifest 
signature, in case
+   # we had to deal with key 
revocation case
m = 
gemato.recursiveloader.ManifestRecursiveLoader(

os.path.join(self.repo.location, 'Manifest'),

verify_openpgp=True,
@@ -336,9 +338,12 @@ class RsyncSync(NewBase):
out.einfo('- timestamp: %s UTC' 
% (

m.openpgp_signature.timestamp))
 
-   out.ebegin('Verifying %s' % 
(self.repo.location,))
-   m.assert_directory_verifies()
-   out.eend(0)
+   # if nothing has changed, skip 
the actual Manifest
+   # verification
+   if not local_state_unchanged:
+   out.ebegin('Verifying 
%s' % (self.repo.location,))
+   
m.assert_directory_verifies()
+   out.eend(0)
except Exception as e:
writemsg_level("!!! Manifest 
verification failed:\n%s\n"
% (e,),
-- 
2.16.1

Re: [gentoo-dev] as-needed patch for ltmain.sh

[James Le Cuirot]

On Thu, 1 Feb 2018 11:55:46 +0100
Andreas Fink  wrote:

> Hello,
> I have a question to the patch provided by the package
> app-portage/elt-patches, namely the file as-needed/2.4.3
> 
> Here I can see the following being added to ltmain.sh:
> +  -Wl,--as-needed|-Wl,--no-as-needed)
> +   deplibs="$deplibs $arg"
> +   continue
> +   ;;
> +
> 
> In my understanding the order must be the opposite otherwise it has no
> effect to $deplibs, i.e. the line should be deplibs="$arg $deplibs".

It's been like that in every version of the patch so it's probably like
that for a reason though I don't know what that reason is. This is
interesting as I gather the whole reason for the patch is that an
unpatched libtool inserts the flag in the wrong place.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[gentoo-dev] as-needed patch for ltmain.sh

[Andreas Fink]

Hello,
I have a question to the patch provided by the package
app-portage/elt-patches, namely the file as-needed/2.4.3

Here I can see the following being added to ltmain.sh:
+  -Wl,--as-needed|-Wl,--no-as-needed)
+   deplibs="$deplibs $arg"
+   continue
+   ;;
+

In my understanding the order must be the opposite otherwise it has no
effect to $deplibs, i.e. the line should be deplibs="$arg $deplibs".
From the man page of ld I read:
This option affects ELF DT_NEEDED tags for dynamic libraries mentioned
on the command line after the --as-needed option.

If we add the flag after deplibs it's mostly not in effect at all.

I realized that the flag is uneffective building claws-mail which links
in the end to the library libnsl, although this dependency is
unnecessary, which can be seen with 
ldd -u -r $(which claws-mail)
Unused direct dependencies:
/usr/lib64/libgthread-2.0.so.0
/usr/lib64/libatk-1.0.so.0
/usr/lib64/libpangoft2-1.0.so.0
/usr/lib64/libfontconfig.so.1
/usr/lib64/libfreetype.so.6
/usr/lib64/libnsl.so.2
/usr/lib64/libdbus-1.so.3

The link command used to build claws-mail when it is being installed using 
emerge is:
libtool: link: x86_64-pc-linux-gnu-gcc -mtune=generic -O2 -pipe 
-Wno-unused-function -Wno-pointer-sign -Wall -Wl,-O1 -Wl,--export-dynamic -o 
claws-mail account.o action.o addrcache.o addr_compl.o addressadd.o 
addrgather.o addrharvest.o addritem.o advsearch.o alertpanel.o autofaces.o 
avatars.o codeconv.o compose.o crash.o customheader.o displayheader.o 
edittags.o enriched.o entity.o export.o file_checker.o filtering.o folder.o 
folder_item_prefs.o foldersel.o folderutils.o folderview.o grouplistdialog.o 
headerview.o html.o image_viewer.o imap.o imap_gtk.o import.o inc.o 
localfolder.o main.o mainwindow.o manual.o matcher.o matcher_parser_lex.o 
matcher_parser_parse.o mbox.o message_search.o messageview.o mh.o mh_gtk.o 
mimeview.o msgcache.o news.o news_gtk.o noticeview.o partial_download.o 
password.o password_gtk.o passwordstore.o pop.o prefs_account.o prefs_actions.o 
prefs_common.o prefs_compose_writing.o prefs_customheader.o 
prefs_display_header.o prefs_ext_prog.o prefs_filtering_action.o 
prefs_filtering.o prefs_folder_item.o prefs_fonts.o prefs_gtk.o 
prefs_image_viewer.o prefs_logging.o prefs_matcher.o prefs_message.o 
prefs_migration.o prefs_msg_colors.o prefs_other.o prefs_quote.o 
prefs_receive.o prefs_send.o prefs_spelling.o prefs_summaries.o 
prefs_summary_column.o prefs_summary_open.o prefs_folder_column.o 
prefs_template.o prefs_themes.o prefs_toolbar.o prefs_wrapping.o printing.o 
privacy.o procheader.o procmime.o procmsg.o quote_fmt.o quote_fmt_lex.o 
quote_fmt_parse.o recv.o remotefolder.o send_message.o setup.o sourcewindow.o 
ssl_manager.o statusbar.o stock_pixmap.o summary_search.o summaryview.o 
textview.o toolbar.o undo.o unmime.o uri_opener.o wizard.o addrbook.o 
addrclip.o addrcustomattr.o addressbook.o addressbook_foldersel.o addrindex.o 
addrquery.o addrselect.o addrduplicates.o browseldap.o editaddress.o 
editaddress_other_attributes_ldap.o editbook.o editgroup.o editjpilot.o 
editldap_basedn.o editldap.o editvcard.o exphtmldlg.o expldifdlg.o exporthtml.o 
exportldif.o importldif.o importmutt.o importpine.o jpilot.o ldapctrl.o 
ldaplocate.o ldapquery.o ldapserver.o ldapupdate.o ldaputil.o ldif.o mutt.o 
pine.o vcard.o -Wl,-O1 -pthread -Wl,-O1 -pthread  etpan/.libs/libclawsetpan.a 
-L/usr/lib64 gtk/.libs/libclawsgtk.a -lgmodule-2.0 -lgthread-2.0 -lcrypt 
-lgtk-x11-2.0 -lgdk-x11-2.0 -lpangocairo-1.0 -latk-1.0 -lpangoft2-1.0 
-lpango-1.0 -lfontconfig -lfreetype -lnsl -llber -lldap -lgnutls -lpthread 
-lstartup-notification-1 -ldbus-glib-1 -ldbus-1 -lrsvg-2 -lgio-2.0 
-lgdk_pixbuf-2.0 -lgobject-2.0 -lglib-2.0 -lcairo -lm -letpan -Wl,--as-needed 
-lssl -lcrypto -lz -pthread

As you can see the flag --as-needed is passed rather late on the
commandline pulling in libnsl in the final build. Passing it earlier on
the commandline, removes the dependency on libnsl. I know that it is a
"bug" of claws-mail pulling in libnsl in the first place if it is not
needed at all. But I am wondering about the order of deplibs and arg in
the above mentioned patch, that is applied in the end to a lot of
ebuilds. I assume claws-mail is not the only package suffering from a
useless --as-needed flag, since it's passed way too late.

Cheers
Andreas

[gentoo-dev] Package up for grabs: games-engines/love

[Chí-Thanh Christopher Nguyễn]

Hi all!

Due to lack of time, I have to drop maintainership of games-engines/love.
There is some user interest in this package, and a version bump is 
needed (bug 640802).



Best regards,
Chí-Thanh Christopher Nguyễn