Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 02:59 PM, Zac Medico wrote: > On 07/08/2018 02:50 PM, Aaron W. Swenson wrote: >> On July 8, 2018 5:38:48 PM EDT, Zac Medico wrote: >> >> On 07/08/2018 02:18 PM, Michał Górny wrote: >> >> W dniu nie, 08.07.2018 o godzinie 14∶11 -0700, użytkownik Zac Medico >> napisał: >> >> On 07/08/2018 01:18 PM, Zac Medico wrote: >> >> On 07/08/2018 01:08 PM, Michał Górny wrote: >> >> W dniu nie, 08.07.2018 o godzinie 11∶57 -0700, >> użytkownik Zac Medico >> napisał: >> >> On 07/08/2018 11:42 AM, Michał Górny wrote: >> >> W dniu nie, 08.07.2018 o godzinie 11∶04 >> -0700, użytkownik Zac Medico >> napisał: >> >> On 07/08/2018 06:56 AM, Michał Górny wrote: >> >> W dniu nie, 08.07.2018 o godzinie >> 15∶02 +0200, użytkownik Kristian >> Fiskerstrand napisał: >> >> On 07/08/2018 08:53 AM, Michał >> Górny wrote: >> >> Is safe git syncing >> implemented already? If not, >> maybe finish it first and >> cover both with a single >> news item. Git is going to >> be more efficient here, so >> people may want to learn >> they have an alternative. >> >> >> Why complicate things, and >> increase wait for something that >> benefits >> most users, just to give >> alternatives to a few using >> non-default sync >> mechanism. Securing git >> distribution is a whole >> different ballpark. >> >> >> >> Let me rephrase. Let's say I'm using >> rsync. This new feature is >> something positive but it breaks my >> use case (for one of the listed >> reasons -- overlayfs, inode use, >> small fs cache). After reading this >> news item, I learn that my only >> option is to disable the new feature. >> >> Now, I would appreciate being told >> that there's an alternate sync method >> that handles secure updates without >> having all those drawbacks. >> >> >> The thing is, the normal git tree >> doesn't even provide pre-generated >> metadata, and I see then gentoo-mirror >> repo that provides metadata does >> not have commits signed with an release key: >> >> >> https://github.com/gentoo-mirror/gentoo/commits/stable >> >> So I'm really not comfortable >> recommending git to anyone at this point. >> >> >> Wrong twice. >> >> Firstly, the canonical URL is: >> >> >> https://anongit.gentoo.org/git/repo/sync/gentoo.git >> (https://gitweb.gentoo.org/repo/sync/gentoo.git) >> >> Secondly, the merge commits (i.e. top >> commits that are verified >> by Portage) are signed by dedicated key that >> is part of the infra key >> set. In other words, it works out of the box. >> >> >> Is there any documentation that shows users how >> to migrate to git, and >> what the pros and cons might be? Maybe its >> worthy of its own news item. >> >> >> Maybe. I don't really know, and don't think it's a >> good idea to show 30 >> news item of things users might like
[gentoo-dev] News Item: Portage rsync hardlink support [v2]
In v2 there's a suggestion to use git for better efficiency than rsync, as requested by Michał Górny: Title: Portage rsync hardlink support Author: Zac Medico Posted: 2018-07-11 Revision: 1 News-Item-Format: 2.0 Display-If-Installed: sys-apps/portage For users of the rsync tree, beginning with sys-apps/portage-2.3.42, the default behavior for sync operations will use hardlinks in order to ensure that a repository remains in a valid state if something goes wrong [1]. For example, if signature verification fails during a sync operation, the new hardlink behavior will preserve the previous state of the repository. The new behavior may conflict with configurations that restrict the use of hardlinks, such as overlay filesystems. Therefore, users will have to set "sync-allow-hardlinks = no" in repos.conf if they have a configuration that restricts the use of hardlinks, but this should not be very common: [DEFAULT] sync-allow-hardlinks = no Note that it possible to sync more efficiently using git [2] instead of rsync, though git consumes an increasing amount of disk space over time because shallow pull is currently not supported [3]. [1] https://bugs.gentoo.org/660410 sys-apps/portage: use rsync --link-dest to implement atomic repository updates (and abort if signature verification fails) [2] https://wiki.gentoo.org/wiki/Portage_Security#git-mirror_repo [3] https://bugs.gentoo.org/552814 sync using git pull --depth=1 by default -- Thanks, Zac signature.asc Description: OpenPGP digital signature
[gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2018-07-08 23:59 UTC
The attached list notes all of the packages that were added or removed from the tree, for the week ending 2018-07-08 23:59 UTC. Removals: app-doc/root-docs 20180705-15:31 amadioa2586f2c2bf games-kids/crayon-physics 20180701-16:36 asturme8e0eadd519 media-sound/puddletag 20180708-10:31 billiee474f3fb4f1 Additions: app-crypt/glep63-check20180704-20:57 mgorny9ba7015a40c app-crypt/jitterentropy 20180706-19:24 gokturk 0c858706cfe app-shells/loksh 20180702-07:54 gyakovlev f8df6a8eaba dev-ada/libadalang-tools 20180708-09:12 tupone97976d99803 dev-python/slimit 20180704-10:03 monsieurp 9c8937b34c6 dev-scheme/guile-sqlite3 20180706-21:14 slyfoxd0d1c0816ed kde-plasma/plasma-browser-integration 20180703-10:51 asturma76abf47ec6 net-dialup/openl2tp 20180703-18:49 bircoph 24d104cdffb perl-core/Archive-Tar 20180706-05:45 kentnl1cec7c9d465 sys-boot/mokutil 20180704-15:47 zerochaos 3a15ff8bae4 -- Robin Hugh Johnson Gentoo Linux Developer E-Mail : robb...@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 Removed Packages: media-sound/puddletag,removed,billie,20180708-10:31,e474f3fb4f1 app-doc/root-docs,removed,amadio,20180705-15:31,a2586f2c2bf games-kids/crayon-physics,removed,asturm,20180701-16:36,e8e0eadd519 Added Packages: dev-ada/libadalang-tools,added,tupone,20180708-09:12,97976d99803 dev-scheme/guile-sqlite3,added,slyfox,20180706-21:14,d0d1c0816ed app-crypt/jitterentropy,added,gokturk,20180706-19:24,0c858706cfe perl-core/Archive-Tar,added,kentnl,20180706-05:45,1cec7c9d465 dev-python/slimit,added,monsieurp,20180704-10:03,9c8937b34c6 app-crypt/glep63-check,added,mgorny,20180704-20:57,9ba7015a40c sys-boot/mokutil,added,zerochaos,20180704-15:47,3a15ff8bae4 net-dialup/openl2tp,added,bircoph,20180703-18:49,24d104cdffb kde-plasma/plasma-browser-integration,added,asturm,20180703-10:51,a76abf47ec6 app-shells/loksh,added,gyakovlev,20180702-07:54,f8df6a8eaba Done.
[gentoo-portage-dev] [PATCH 2/2] git: add key refresh retry (bug 660732)
Bug: https://bugs.gentoo.org/660732
---
pym/portage/sync/modules/git/git.py | 7 +++
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/pym/portage/sync/modules/git/git.py
b/pym/portage/sync/modules/git/git.py
index 97c4c1de67..68f8bd1fb9 100644
--- a/pym/portage/sync/modules/git/git.py
+++ b/pym/portage/sync/modules/git/git.py
@@ -8,6 +8,7 @@ import subprocess
import portage
from portage import os
from portage.util import writemsg_level, shlex_split
+from portage.util.futures import asyncio
from portage.output import create_color_func, EOutput
good = create_color_func("GOOD")
bad = create_color_func("BAD")
@@ -197,10 +198,8 @@ class GitSync(NewBase):
out.einfo('Using keys from %s' %
(self.repo.sync_openpgp_key_path,))
with
io.open(self.repo.sync_openpgp_key_path, 'rb') as f:
openpgp_env.import_key(f)
- out.ebegin('Refreshing keys from
keyserver')
- openpgp_env.refresh_keys()
- out.eend(0)
- except GematoException as e:
+ self._refresh_keys(openpgp_env)
+ except (GematoException, asyncio.TimeoutError)
as e:
writemsg_level("!!! Verification
impossible due to keyring problem:\n%s\n"
% (e,),
level=logging.ERROR,
noiselevel=-1)
--
2.13.6
[gentoo-portage-dev] [PATCH 1/2] SyncBase: split out _refresh_keys method (bug 660732)
Split out a _refresh_keys method from the RsyncSync class,
so GitSync can use it for retry support.
Bug: https://bugs.gentoo.org/660732
---
pym/portage/sync/modules/rsync/rsync.py | 33 +-
pym/portage/sync/syncbase.py| 42 +
2 files changed, 43 insertions(+), 32 deletions(-)
diff --git a/pym/portage/sync/modules/rsync/rsync.py
b/pym/portage/sync/modules/rsync/rsync.py
index 382a1eaaef..a715e2818b 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -7,7 +7,6 @@ import time
import signal
import socket
import datetime
-import functools
import io
import re
import random
@@ -23,10 +22,8 @@ good = create_color_func("GOOD")
bad = create_color_func("BAD")
warn = create_color_func("WARN")
from portage.const import VCS_DIRS, TIMESTAMP_FORMAT, RSYNC_PACKAGE_ATOM
-from portage.util._eventloop.global_event_loop import global_event_loop
from portage.util import writemsg, writemsg_stdout
from portage.util.futures import asyncio
-from portage.util.futures.executor.fork import ForkExecutor
from portage.sync.getaddrinfo_validate import getaddrinfo_validate
from _emerge.UserQuery import UserQuery
from portage.sync.syncbase import NewBase
@@ -149,39 +146,11 @@ class RsyncSync(NewBase):
# will not be performed and the user will have to fix
it and try again,
# so we may as well bail out before actual rsync
happens.
if openpgp_env is not None and
self.repo.sync_openpgp_key_path is not None:
-
try:
out.einfo('Using keys from %s' %
(self.repo.sync_openpgp_key_path,))
with
io.open(self.repo.sync_openpgp_key_path, 'rb') as f:
openpgp_env.import_key(f)
- out.ebegin('Refreshing keys from
keyserver')
- retry_decorator =
self._key_refresh_retry_decorator()
- if retry_decorator is None:
- openpgp_env.refresh_keys()
- else:
- def noisy_refresh_keys():
- """
- Since retry does not
help for some types of
- errors, display errors
as soon as they occur.
- """
- try:
-
openpgp_env.refresh_keys()
- except Exception as e:
-
writemsg_level("%s\n" % (e,),
-
level=logging.ERROR, noiselevel=-1)
- raise # retry
-
- # The ThreadPoolExecutor that
asyncio uses by default
- # does not support cancellation
of tasks, therefore
- # use ForkExecutor for task
cancellation support, in
- # order to enforce timeouts.
- loop = global_event_loop()
- with ForkExecutor(loop=loop) as
executor:
- func_coroutine =
functools.partial(loop.run_in_executor,
- executor,
noisy_refresh_keys)
- decorated_func =
retry_decorator(func_coroutine, loop=loop)
-
loop.run_until_complete(decorated_func())
- out.eend(0)
+ self._refresh_keys(openpgp_env)
except (GematoException, asyncio.TimeoutError)
as e:
writemsg_level("!!! Manifest
verification impossible due to keyring problem:\n%s\n"
% (e,),
diff --git a/pym/portage/sync/syncbase.py b/pym/portage/sync/syncbase.py
index 7d4d33272e..fe13f16434 100644
--- a/pym/portage/sync/syncbase.py
+++ b/pym/portage/sync/syncbase.py
@@ -7,13 +7,17 @@ This class contains common initialization code and functions.
'''
from __future__ import unicode_literals
+import functools
+import io
import logging
import os
import portage
from portage.util import
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On Sun, Jul 8, 2018 at 5:50 PM Aaron W. Swenson wrote: > > Does Portage not call attention to critical updates? > > It used to make a special statement for a new stable Portage and strongly > recommended that it be emerged first. It should probably do the same for > openpgp-keys-gentoo-release. If that is only needed for syncing, then I'm not sure there is really a need. If you have the ebuild for the latest keys, then you can install them in any order at any time. If you don't have the ebuild but need it, then portage won't know, and syncing will simply fail unless overridden somehow. -- Rich
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 02:50 PM, Aaron W. Swenson wrote: > On July 8, 2018 5:38:48 PM EDT, Zac Medico wrote: > > On 07/08/2018 02:18 PM, Michał Górny wrote: > > W dniu nie, 08.07.2018 o godzinie 14∶11 -0700, użytkownik Zac Medico > napisał: > > On 07/08/2018 01:18 PM, Zac Medico wrote: > > On 07/08/2018 01:08 PM, Michał Górny wrote: > > W dniu nie, 08.07.2018 o godzinie 11∶57 -0700, > użytkownik Zac Medico > napisał: > > On 07/08/2018 11:42 AM, Michał Górny wrote: > > W dniu nie, 08.07.2018 o godzinie 11∶04 > -0700, użytkownik Zac Medico > napisał: > > On 07/08/2018 06:56 AM, Michał Górny wrote: > > W dniu nie, 08.07.2018 o godzinie > 15∶02 +0200, użytkownik Kristian > Fiskerstrand napisał: > > On 07/08/2018 08:53 AM, Michał > Górny wrote: > > Is safe git syncing > implemented already? If not, > maybe finish it first and > cover both with a single > news item. Git is going to > be more efficient here, so > people may want to learn > they have an alternative. > > > Why complicate things, and > increase wait for something that > benefits > most users, just to give > alternatives to a few using > non-default sync > mechanism. Securing git > distribution is a whole > different ballpark. > > > > Let me rephrase. Let's say I'm using > rsync. This new feature is > something positive but it breaks my > use case (for one of the listed > reasons -- overlayfs, inode use, > small fs cache). After reading this > news item, I learn that my only > option is to disable the new feature. > > Now, I would appreciate being told > that there's an alternate sync method > that handles secure updates without > having all those drawbacks. > > > The thing is, the normal git tree > doesn't even provide pre-generated > metadata, and I see then gentoo-mirror > repo that provides metadata does > not have commits signed with an release key: > > > https://github.com/gentoo-mirror/gentoo/commits/stable > > So I'm really not comfortable > recommending git to anyone at this point. > > > Wrong twice. > > Firstly, the canonical URL is: > > > https://anongit.gentoo.org/git/repo/sync/gentoo.git > (https://gitweb.gentoo.org/repo/sync/gentoo.git) > > Secondly, the merge commits (i.e. top > commits that are verified > by Portage) are signed by dedicated key that > is part of the infra key > set. In other words, it works out of the box. > > > Is there any documentation that shows users how > to migrate to git, and > what the pros and cons might be? Maybe its > worthy of its own news item. > > > Maybe. I don't really know, and don't think it's a > good idea to show 30 > news item of things users might like on every new > Gentoo install. > > > Well if instructions for setting up git sync
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On July 8, 2018 5:38:48 PM EDT, Zac Medico wrote: >On 07/08/2018 02:18 PM, Michał Górny wrote: >> W dniu nie, 08.07.2018 o godzinie 14∶11 -0700, użytkownik Zac Medico >> napisał: >>> On 07/08/2018 01:18 PM, Zac Medico wrote: On 07/08/2018 01:08 PM, Michał Górny wrote: > W dniu nie, 08.07.2018 o godzinie 11∶57 -0700, użytkownik Zac >Medico > napisał: >> On 07/08/2018 11:42 AM, Michał Górny wrote: >>> W dniu nie, 08.07.2018 o godzinie 11∶04 -0700, użytkownik Zac >Medico >>> napisał: On 07/08/2018 06:56 AM, Michał Górny wrote: > W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik >Kristian > Fiskerstrand napisał: >> On 07/08/2018 08:53 AM, Michał Górny wrote: >>> Is safe git syncing implemented already? If not, maybe >finish it first and cover both with a single news item. Git is going to >be more efficient here, so people may want to learn they have an >alternative. >> >> Why complicate things, and increase wait for something that >benefits >> most users, just to give alternatives to a few using >non-default sync >> mechanism. Securing git distribution is a whole different >ballpark. >> > > Let me rephrase. Let's say I'm using rsync. This new feature >is > something positive but it breaks my use case (for one of the >listed > reasons -- overlayfs, inode use, small fs cache). After >reading this > news item, I learn that my only option is to disable the new >feature. > > Now, I would appreciate being told that there's an alternate >sync method > that handles secure updates without having all those >drawbacks. The thing is, the normal git tree doesn't even provide >pre-generated metadata, and I see then gentoo-mirror repo that provides >metadata does not have commits signed with an release key: https://github.com/gentoo-mirror/gentoo/commits/stable So I'm really not comfortable recommending git to anyone at >this point. >>> >>> Wrong twice. >>> >>> Firstly, the canonical URL is: >>> >>> https://anongit.gentoo.org/git/repo/sync/gentoo.git >>> (https://gitweb.gentoo.org/repo/sync/gentoo.git) >>> >>> Secondly, the merge commits (i.e. top commits that are verified >>> by Portage) are signed by dedicated key that is part of the >infra key >>> set. In other words, it works out of the box. >> >> Is there any documentation that shows users how to migrate to >git, and >> what the pros and cons might be? Maybe its worthy of its own news >item. > > Maybe. I don't really know, and don't think it's a good idea to >show 30 > news item of things users might like on every new Gentoo install. Well if instructions for setting up git sync and associated >pros/cons are not documented anywhere then I won't advise anyone to use it. >>> >>> I've attempted to configure it for myself, and this is what it does: >>> >>> * Using keys from /usr/share/openpgp-keys/gentoo-release.asc >>> * Refreshing keys from keyserver ... >>>[ ok ] >>> * No valid signature found: unable to verify signature (missing >key?) >>> >> >> Please report a bug and attach your configuration along with keyring >> version. > >It works after upgrading to openpgp-keys-gentoo-release-20180706 from >openpgp-keys-gentoo-release-20180323. >-- >Thanks, >Zac Does Portage not call attention to critical updates? It used to make a special statement for a new stable Portage and strongly recommended that it be emerged first. It should probably do the same for openpgp-keys-gentoo-release. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. signature.asc Description: PGP signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 02:18 PM, Michał Górny wrote: > W dniu nie, 08.07.2018 o godzinie 14∶11 -0700, użytkownik Zac Medico > napisał: >> On 07/08/2018 01:18 PM, Zac Medico wrote: >>> On 07/08/2018 01:08 PM, Michał Górny wrote: W dniu nie, 08.07.2018 o godzinie 11∶57 -0700, użytkownik Zac Medico napisał: > On 07/08/2018 11:42 AM, Michał Górny wrote: >> W dniu nie, 08.07.2018 o godzinie 11∶04 -0700, użytkownik Zac Medico >> napisał: >>> On 07/08/2018 06:56 AM, Michał Górny wrote: W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian Fiskerstrand napisał: > On 07/08/2018 08:53 AM, Michał Górny wrote: >> Is safe git syncing implemented already? If not, maybe finish it >> first and cover both with a single news item. Git is going to be >> more efficient here, so people may want to learn they have an >> alternative. > > Why complicate things, and increase wait for something that benefits > most users, just to give alternatives to a few using non-default sync > mechanism. Securing git distribution is a whole different ballpark. > Let me rephrase. Let's say I'm using rsync. This new feature is something positive but it breaks my use case (for one of the listed reasons -- overlayfs, inode use, small fs cache). After reading this news item, I learn that my only option is to disable the new feature. Now, I would appreciate being told that there's an alternate sync method that handles secure updates without having all those drawbacks. >>> >>> The thing is, the normal git tree doesn't even provide pre-generated >>> metadata, and I see then gentoo-mirror repo that provides metadata does >>> not have commits signed with an release key: >>> >>> https://github.com/gentoo-mirror/gentoo/commits/stable >>> >>> So I'm really not comfortable recommending git to anyone at this point. >> >> Wrong twice. >> >> Firstly, the canonical URL is: >> >> https://anongit.gentoo.org/git/repo/sync/gentoo.git >> (https://gitweb.gentoo.org/repo/sync/gentoo.git) >> >> Secondly, the merge commits (i.e. top commits that are verified >> by Portage) are signed by dedicated key that is part of the infra key >> set. In other words, it works out of the box. > > Is there any documentation that shows users how to migrate to git, and > what the pros and cons might be? Maybe its worthy of its own news item. Maybe. I don't really know, and don't think it's a good idea to show 30 news item of things users might like on every new Gentoo install. >>> >>> Well if instructions for setting up git sync and associated pros/cons >>> are not documented anywhere then I won't advise anyone to use it. >> >> I've attempted to configure it for myself, and this is what it does: >> >> * Using keys from /usr/share/openpgp-keys/gentoo-release.asc >> * Refreshing keys from keyserver ... >>[ ok ] >> * No valid signature found: unable to verify signature (missing key?) >> > > Please report a bug and attach your configuration along with keyring > version. It works after upgrading to openpgp-keys-gentoo-release-20180706 from openpgp-keys-gentoo-release-20180323. -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
W dniu nie, 08.07.2018 o godzinie 14∶11 -0700, użytkownik Zac Medico napisał: > On 07/08/2018 01:18 PM, Zac Medico wrote: > > On 07/08/2018 01:08 PM, Michał Górny wrote: > > > W dniu nie, 08.07.2018 o godzinie 11∶57 -0700, użytkownik Zac Medico > > > napisał: > > > > On 07/08/2018 11:42 AM, Michał Górny wrote: > > > > > W dniu nie, 08.07.2018 o godzinie 11∶04 -0700, użytkownik Zac Medico > > > > > napisał: > > > > > > On 07/08/2018 06:56 AM, Michał Górny wrote: > > > > > > > W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian > > > > > > > Fiskerstrand napisał: > > > > > > > > On 07/08/2018 08:53 AM, Michał Górny wrote: > > > > > > > > > Is safe git syncing implemented already? If not, maybe finish > > > > > > > > > it first and cover both with a single news item. Git is going > > > > > > > > > to be more efficient here, so people may want to learn they > > > > > > > > > have an alternative. > > > > > > > > > > > > > > > > Why complicate things, and increase wait for something that > > > > > > > > benefits > > > > > > > > most users, just to give alternatives to a few using > > > > > > > > non-default sync > > > > > > > > mechanism. Securing git distribution is a whole different > > > > > > > > ballpark. > > > > > > > > > > > > > > > > > > > > > > Let me rephrase. Let's say I'm using rsync. This new feature is > > > > > > > something positive but it breaks my use case (for one of the > > > > > > > listed > > > > > > > reasons -- overlayfs, inode use, small fs cache). After reading > > > > > > > this > > > > > > > news item, I learn that my only option is to disable the new > > > > > > > feature. > > > > > > > > > > > > > > Now, I would appreciate being told that there's an alternate sync > > > > > > > method > > > > > > > that handles secure updates without having all those drawbacks. > > > > > > > > > > > > The thing is, the normal git tree doesn't even provide pre-generated > > > > > > metadata, and I see then gentoo-mirror repo that provides metadata > > > > > > does > > > > > > not have commits signed with an release key: > > > > > > > > > > > > https://github.com/gentoo-mirror/gentoo/commits/stable > > > > > > > > > > > > So I'm really not comfortable recommending git to anyone at this > > > > > > point. > > > > > > > > > > Wrong twice. > > > > > > > > > > Firstly, the canonical URL is: > > > > > > > > > > https://anongit.gentoo.org/git/repo/sync/gentoo.git > > > > > (https://gitweb.gentoo.org/repo/sync/gentoo.git) > > > > > > > > > > Secondly, the merge commits (i.e. top commits that are verified > > > > > by Portage) are signed by dedicated key that is part of the infra key > > > > > set. In other words, it works out of the box. > > > > > > > > Is there any documentation that shows users how to migrate to git, and > > > > what the pros and cons might be? Maybe its worthy of its own news item. > > > > > > Maybe. I don't really know, and don't think it's a good idea to show 30 > > > news item of things users might like on every new Gentoo install. > > > > Well if instructions for setting up git sync and associated pros/cons > > are not documented anywhere then I won't advise anyone to use it. > > I've attempted to configure it for myself, and this is what it does: > > * Using keys from /usr/share/openpgp-keys/gentoo-release.asc > * Refreshing keys from keyserver ... >[ ok ] > * No valid signature found: unable to verify signature (missing key?) > Please report a bug and attach your configuration along with keyring version. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 01:18 PM, Zac Medico wrote: > On 07/08/2018 01:08 PM, Michał Górny wrote: >> W dniu nie, 08.07.2018 o godzinie 11∶57 -0700, użytkownik Zac Medico >> napisał: >>> On 07/08/2018 11:42 AM, Michał Górny wrote: W dniu nie, 08.07.2018 o godzinie 11∶04 -0700, użytkownik Zac Medico napisał: > On 07/08/2018 06:56 AM, Michał Górny wrote: >> W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian >> Fiskerstrand napisał: >>> On 07/08/2018 08:53 AM, Michał Górny wrote: Is safe git syncing implemented already? If not, maybe finish it first and cover both with a single news item. Git is going to be more efficient here, so people may want to learn they have an alternative. >>> >>> Why complicate things, and increase wait for something that benefits >>> most users, just to give alternatives to a few using non-default sync >>> mechanism. Securing git distribution is a whole different ballpark. >>> >> >> Let me rephrase. Let's say I'm using rsync. This new feature is >> something positive but it breaks my use case (for one of the listed >> reasons -- overlayfs, inode use, small fs cache). After reading this >> news item, I learn that my only option is to disable the new feature. >> >> Now, I would appreciate being told that there's an alternate sync method >> that handles secure updates without having all those drawbacks. > > The thing is, the normal git tree doesn't even provide pre-generated > metadata, and I see then gentoo-mirror repo that provides metadata does > not have commits signed with an release key: > > https://github.com/gentoo-mirror/gentoo/commits/stable > > So I'm really not comfortable recommending git to anyone at this point. Wrong twice. Firstly, the canonical URL is: https://anongit.gentoo.org/git/repo/sync/gentoo.git (https://gitweb.gentoo.org/repo/sync/gentoo.git) Secondly, the merge commits (i.e. top commits that are verified by Portage) are signed by dedicated key that is part of the infra key set. In other words, it works out of the box. >>> >>> Is there any documentation that shows users how to migrate to git, and >>> what the pros and cons might be? Maybe its worthy of its own news item. >> >> Maybe. I don't really know, and don't think it's a good idea to show 30 >> news item of things users might like on every new Gentoo install. > > Well if instructions for setting up git sync and associated pros/cons > are not documented anywhere then I won't advise anyone to use it. I've attempted to configure it for myself, and this is what it does: * Using keys from /usr/share/openpgp-keys/gentoo-release.asc * Refreshing keys from keyserver ... [ ok ] * No valid signature found: unable to verify signature (missing key?) -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 01:08 PM, Michał Górny wrote: > W dniu nie, 08.07.2018 o godzinie 11∶57 -0700, użytkownik Zac Medico > napisał: >> On 07/08/2018 11:42 AM, Michał Górny wrote: >>> W dniu nie, 08.07.2018 o godzinie 11∶04 -0700, użytkownik Zac Medico >>> napisał: On 07/08/2018 06:56 AM, Michał Górny wrote: > W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian > Fiskerstrand napisał: >> On 07/08/2018 08:53 AM, Michał Górny wrote: >>> Is safe git syncing implemented already? If not, maybe finish it first >>> and cover both with a single news item. Git is going to be more >>> efficient here, so people may want to learn they have an alternative. >> >> Why complicate things, and increase wait for something that benefits >> most users, just to give alternatives to a few using non-default sync >> mechanism. Securing git distribution is a whole different ballpark. >> > > Let me rephrase. Let's say I'm using rsync. This new feature is > something positive but it breaks my use case (for one of the listed > reasons -- overlayfs, inode use, small fs cache). After reading this > news item, I learn that my only option is to disable the new feature. > > Now, I would appreciate being told that there's an alternate sync method > that handles secure updates without having all those drawbacks. The thing is, the normal git tree doesn't even provide pre-generated metadata, and I see then gentoo-mirror repo that provides metadata does not have commits signed with an release key: https://github.com/gentoo-mirror/gentoo/commits/stable So I'm really not comfortable recommending git to anyone at this point. >>> >>> Wrong twice. >>> >>> Firstly, the canonical URL is: >>> >>> https://anongit.gentoo.org/git/repo/sync/gentoo.git >>> (https://gitweb.gentoo.org/repo/sync/gentoo.git) >>> >>> Secondly, the merge commits (i.e. top commits that are verified >>> by Portage) are signed by dedicated key that is part of the infra key >>> set. In other words, it works out of the box. >> >> Is there any documentation that shows users how to migrate to git, and >> what the pros and cons might be? Maybe its worthy of its own news item. > > Maybe. I don't really know, and don't think it's a good idea to show 30 > news item of things users might like on every new Gentoo install. Well if instructions for setting up git sync and associated pros/cons are not documented anywhere then I won't advise anyone to use it. -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
W dniu nie, 08.07.2018 o godzinie 11∶57 -0700, użytkownik Zac Medico napisał: > On 07/08/2018 11:42 AM, Michał Górny wrote: > > W dniu nie, 08.07.2018 o godzinie 11∶04 -0700, użytkownik Zac Medico > > napisał: > > > On 07/08/2018 06:56 AM, Michał Górny wrote: > > > > W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian > > > > Fiskerstrand napisał: > > > > > On 07/08/2018 08:53 AM, Michał Górny wrote: > > > > > > Is safe git syncing implemented already? If not, maybe finish it > > > > > > first and cover both with a single news item. Git is going to be > > > > > > more efficient here, so people may want to learn they have an > > > > > > alternative. > > > > > > > > > > Why complicate things, and increase wait for something that benefits > > > > > most users, just to give alternatives to a few using non-default sync > > > > > mechanism. Securing git distribution is a whole different ballpark. > > > > > > > > > > > > > Let me rephrase. Let's say I'm using rsync. This new feature is > > > > something positive but it breaks my use case (for one of the listed > > > > reasons -- overlayfs, inode use, small fs cache). After reading this > > > > news item, I learn that my only option is to disable the new feature. > > > > > > > > Now, I would appreciate being told that there's an alternate sync method > > > > that handles secure updates without having all those drawbacks. > > > > > > The thing is, the normal git tree doesn't even provide pre-generated > > > metadata, and I see then gentoo-mirror repo that provides metadata does > > > not have commits signed with an release key: > > > > > > https://github.com/gentoo-mirror/gentoo/commits/stable > > > > > > So I'm really not comfortable recommending git to anyone at this point. > > > > Wrong twice. > > > > Firstly, the canonical URL is: > > > > https://anongit.gentoo.org/git/repo/sync/gentoo.git > > (https://gitweb.gentoo.org/repo/sync/gentoo.git) > > > > Secondly, the merge commits (i.e. top commits that are verified > > by Portage) are signed by dedicated key that is part of the infra key > > set. In other words, it works out of the box. > > Is there any documentation that shows users how to migrate to git, and > what the pros and cons might be? Maybe its worthy of its own news item. Maybe. I don't really know, and don't think it's a good idea to show 30 news item of things users might like on every new Gentoo install. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On Sun, Jul 8, 2018 at 2:31 PM Kristian Fiskerstrand wrote: > > On 07/08/2018 08:10 PM, Rich Freeman wrote: > > Again, the current portage support for git verification doesn't check > > any developer keys. > > right, so why would it be material for a news item improving the status > quo for those synching using the official rsync method? > I never claimed it should be, though mgorny made a reasonable argument for giving users an alternative if they can't use the hard links. -- Rich
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 11:42 AM, Michał Górny wrote: > W dniu nie, 08.07.2018 o godzinie 11∶04 -0700, użytkownik Zac Medico > napisał: >> On 07/08/2018 06:56 AM, Michał Górny wrote: >>> W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian >>> Fiskerstrand napisał: On 07/08/2018 08:53 AM, Michał Górny wrote: > Is safe git syncing implemented already? If not, maybe finish it first > and cover both with a single news item. Git is going to be more efficient > here, so people may want to learn they have an alternative. Why complicate things, and increase wait for something that benefits most users, just to give alternatives to a few using non-default sync mechanism. Securing git distribution is a whole different ballpark. >>> >>> Let me rephrase. Let's say I'm using rsync. This new feature is >>> something positive but it breaks my use case (for one of the listed >>> reasons -- overlayfs, inode use, small fs cache). After reading this >>> news item, I learn that my only option is to disable the new feature. >>> >>> Now, I would appreciate being told that there's an alternate sync method >>> that handles secure updates without having all those drawbacks. >> >> The thing is, the normal git tree doesn't even provide pre-generated >> metadata, and I see then gentoo-mirror repo that provides metadata does >> not have commits signed with an release key: >> >> https://github.com/gentoo-mirror/gentoo/commits/stable >> >> So I'm really not comfortable recommending git to anyone at this point. > > Wrong twice. > > Firstly, the canonical URL is: > > https://anongit.gentoo.org/git/repo/sync/gentoo.git > (https://gitweb.gentoo.org/repo/sync/gentoo.git) > > Secondly, the merge commits (i.e. top commits that are verified > by Portage) are signed by dedicated key that is part of the infra key > set. In other words, it works out of the box. Is there any documentation that shows users how to migrate to git, and what the pros and cons might be? Maybe its worthy of its own news item. -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
W dniu nie, 08.07.2018 o godzinie 11∶04 -0700, użytkownik Zac Medico napisał: > On 07/08/2018 06:56 AM, Michał Górny wrote: > > W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian > > Fiskerstrand napisał: > > > On 07/08/2018 08:53 AM, Michał Górny wrote: > > > > Is safe git syncing implemented already? If not, maybe finish it first > > > > and cover both with a single news item. Git is going to be more > > > > efficient here, so people may want to learn they have an alternative. > > > > > > Why complicate things, and increase wait for something that benefits > > > most users, just to give alternatives to a few using non-default sync > > > mechanism. Securing git distribution is a whole different ballpark. > > > > > > > Let me rephrase. Let's say I'm using rsync. This new feature is > > something positive but it breaks my use case (for one of the listed > > reasons -- overlayfs, inode use, small fs cache). After reading this > > news item, I learn that my only option is to disable the new feature. > > > > Now, I would appreciate being told that there's an alternate sync method > > that handles secure updates without having all those drawbacks. > > The thing is, the normal git tree doesn't even provide pre-generated > metadata, and I see then gentoo-mirror repo that provides metadata does > not have commits signed with an release key: > > https://github.com/gentoo-mirror/gentoo/commits/stable > > So I'm really not comfortable recommending git to anyone at this point. Wrong twice. Firstly, the canonical URL is: https://anongit.gentoo.org/git/repo/sync/gentoo.git (https://gitweb.gentoo.org/repo/sync/gentoo.git) Secondly, the merge commits (i.e. top commits that are verified by Portage) are signed by dedicated key that is part of the infra key set. In other words, it works out of the box. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
[gentoo-dev] [PATCH v5 16/16] glep-0063: Unify punctuation
Requested-by: Ulrich Müller --- glep-0063.rst | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index ae36d36..c02b89e 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -83,19 +83,19 @@ not be used to commit. at least 256-bit. All subkey self-signatures must use this digest. 2. Signing subkey that is different from the primary key, and does not - have any other capabilities enabled + have any other capabilities enabled. 3. Primary key and the signing subkey are both of type EITHER: - a. RSA, >=2048 bits (OpenPGP v4 key format or later only) + a. RSA, >=2048 bits (OpenPGP v4 key format or later only), - b. ECC curve 25519 + b. ECC curve 25519. 4. Expiration date on key and all subkeys set to no more than 900 days - into the future + into the future. 5. Key expiration date renewed at least 2 weeks before the previous - expiration date + expiration date. 6. Upload your key to the SKS keyserver rotation before usage! @@ -107,9 +107,9 @@ technical reason not to (e.g. hardware limitations, necessity of replacing their primary key). 1. Primary key and the signing subkey are both of type RSA, 2048 bits - (OpenPGP v4 key format or later) + (OpenPGP v4 key format or later). -2. Key expiration renewed annually to a fixed day of the year +2. Key expiration renewed annually to a fixed day of the year. 3. Create a revocation certificate & store it hardcopy offsite securely (it's about ~300 bytes). @@ -142,13 +142,13 @@ External documentation Much of the above was driven by the following: -* NIST SP 800-57 recommendations [#NISTSP800571]_, [#NISTSP800572]_ +* NIST SP 800-57 recommendations [#NISTSP800571]_, [#NISTSP800572]_, -* Debian GPG documentation [#DEBIANGPG]_ +* Debian GPG documentation [#DEBIANGPG]_, -* RiseUp.net OpenPGP best practices [#RISEUP]_ +* RiseUp.net OpenPGP best practices [#RISEUP]_, -* ENISA Algorithms, Key Sizes and Parameters Report 2013 [#ENISA2013]_ +* ENISA Algorithms, Key Sizes and Parameters Report 2013 [#ENISA2013]_. References == -- 2.18.0
[gentoo-dev] [PATCH v5 15/16] glep-0063: Extend SHA-2 requirement to self-signatures on subkeys
--- glep-0063.rst | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/glep-0063.rst b/glep-0063.rst index 84d87d2..ae36d36 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -45,6 +45,9 @@ v2 The ``gpg.conf`` contents have been removed as they were seriously outdated and decreased security over the modern defaults. + The requirement of SHA-2 digest has been extended to apply to self- + signatures made on subkeys. + v1.1 The recommended RSA key size has been changed from 4096 bits to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_. @@ -77,7 +80,7 @@ to commit to Gentoo. Keys that do not conform to those requirements can not be used to commit. 1. SHA-2 series output digest (SHA-1 digests internally permitted), - at least 256-bit. + at least 256-bit. All subkey self-signatures must use this digest. 2. Signing subkey that is different from the primary key, and does not have any other capabilities enabled -- 2.18.0
[gentoo-dev] [PATCH v5 14/16] glep-0063: Remove gpg.conf bits
Remove the gpg.conf bits from recommended and minimal specification. Apparently they are seriously obsolete and worse than the modern defaults. While at it, editorial corrections to 'SHA2' bit. Requested-by: Richard Yao --- glep-0063.rst | 60 --- 1 file changed, 9 insertions(+), 51 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index 37b1f4d..84d87d2 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -42,6 +42,9 @@ v2 The ``gpgfingerprint`` LDAP field has been altered to remove optional whitespace. + The ``gpg.conf`` contents have been removed as they were seriously + outdated and decreased security over the modern defaults. + v1.1 The recommended RSA key size has been changed from 4096 bits to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_. @@ -73,10 +76,8 @@ This section specifies obligatory requirements for all OpenPGP keys used to commit to Gentoo. Keys that do not conform to those requirements can not be used to commit. -1. SHA2-series output digest (SHA1 digests internally permitted), - 256bit or more:: - - personal-digest-preferences SHA256 +1. SHA-2 series output digest (SHA-1 digests internally permitted), + at least 256-bit. 2. Signing subkey that is different from the primary key, and does not have any other capabilities enabled @@ -102,58 +103,15 @@ The developers should follow those practices unless there is a strong technical reason not to (e.g. hardware limitations, necessity of replacing their primary key). -1. Copy ``/usr/share/gnupg/gpg-conf.skel`` to ``~/.gnupg/gpg.conf``, append - the following block:: - - keyserver pool.sks-keyservers.net - - emit-version - - default-recipient-self - - # -- All of the below portion from the RiseUp.net OpenPGP best practices, and - # -- many of them are also in the Debian GPG documentation. - - # when outputting certificates, view user IDs distinctly from keys: - fixed-list-mode - - # long keyids are more collision-resistant than short keyids (it's trivial to make a key - # with any desired short keyid) - # NOTE: this breaks kmail gnupg support! - keyid-format 0xlong - - # when multiple digests are supported by all recipients, choose the strongest one: - personal-digest-preferences SHA512 SHA384 SHA256 SHA224 - - # preferences chosen for new keys should prioritize stronger algorithms: - default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 BZIP2 ZLIB ZIP Uncompressed - - # If you use a graphical environment (and even if you don't) you should be using an agent: - # (similar arguments as https://www.debian-administration.org/users/dkg/weblog/64) - use-agent - - # You should always know at a glance which User IDs gpg thinks are legitimately bound to - # the keys in your keyring: - verify-options show-uid-validity - list-options show-uid-validity - - # include an unambiguous indicator of which key made a signature: - # (see http://thread.gmane.org/gmane.mail.notmuch.general/3721/focus=7234) - # (and http://www.ietf.org/mail-archive/web/openpgp/current/msg00405.html) - sig-notation issuer-...@notations.openpgp.fifthhorseman.net=%g - - # when making an OpenPGP certification, use a stronger digest than the default SHA1: - cert-digest-algo SHA256 - -2. Primary key and the signing subkey are both of type RSA, 2048 bits +1. Primary key and the signing subkey are both of type RSA, 2048 bits (OpenPGP v4 key format or later) -3. Key expiration renewed annually to a fixed day of the year +2. Key expiration renewed annually to a fixed day of the year -4. Create a revocation certificate & store it hardcopy offsite securely +3. Create a revocation certificate & store it hardcopy offsite securely (it's about ~300 bytes). -5. Encrypted backup of your secret keys. +4. Encrypted backup of your secret keys. Gentoo LDAP === -- 2.18.0
[gentoo-dev] [PATCH v5 13/16] glep-0063: Remove whitespace from LDAP field
Requested-by: Robin H. Johnson
---
glep-0063.rst | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/glep-0063.rst b/glep-0063.rst
index 300456b..37b1f4d 100644
--- a/glep-0063.rst
+++ b/glep-0063.rst
@@ -39,6 +39,9 @@ v2
The usage of DSA keys has been disallowed.
+ The ``gpgfingerprint`` LDAP field has been altered to remove optional
+ whitespace.
+
v1.1
The recommended RSA key size has been changed from 4096 bits
to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_.
@@ -157,10 +160,7 @@ Gentoo LDAP
All Gentoo developers must list the complete fingerprint for their primary
keys in the "``gpgfingerprint``" LDAP field. It must be exactly 40 hex digits,
-uppercase, with optional spaces every 8 hex digits. Regular expression for
-validation::
-
-^([[:space:]]*[[:xdigit:]]{8}){5}$
+uppercase, without whitespace.
The prior "``gpgkey``" field will be removed, as it is a subset
of the fingerprint field. In any place that presently displays
--
2.18.0
[gentoo-dev] [PATCH v5 12/16] glep-0063: Disallow using DSA keys
There really is no technical reason to use DSA keys and people who are still using old DSA keys should finally replace them, so remove them from the minimal requirements. --- glep-0063.rst | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index ca834a8..300456b 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -37,6 +37,8 @@ v2 has been added. This is in order to give services and other developers time to refresh the key. + The usage of DSA keys has been disallowed. + v1.1 The recommended RSA key size has been changed from 4096 bits to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_. @@ -78,11 +80,9 @@ not be used to commit. 3. Primary key and the signing subkey are both of type EITHER: - a. DSA, 2048-bit - - b. RSA, >=2048 bits (OpenPGP v4 key format or later only) + a. RSA, >=2048 bits (OpenPGP v4 key format or later only) - c. ECC curve 25519 + b. ECC curve 25519 4. Expiration date on key and all subkeys set to no more than 900 days into the future -- 2.18.0
[gentoo-dev] [PATCH v5 10/16] glep-0063: Update and unify expiration term
Replace the disjoint 'minimum' and 'recommendation' for expiration with a single requirement. Make it 2.5 years with recommended annual renewal to a fixed day of the year (2 years + some grace time for renewal). Also, remove disjoint expiration recommendation for the primary key and subkeys since many developers fail at implementing that anyway. --- glep-0063.rst | 16 +--- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index 7f870bb..9ba778b 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -7,7 +7,7 @@ Author: Robin H. Johnson , Michał Górny Type: Standards Track Status: Final -Version: 1.1 +Version: 2 Created: 2013-02-18 Last-Modified: 2018-07-07 Post-History: 2013-11-10 @@ -28,6 +28,11 @@ OpenPGP key management policies for the Gentoo Linux distribution. Changes === +v2 + The distinct minimal and recommended expirations have been replaced + by a single requirement. The rules have been simplified to use + the same maximum time of 900 days for both the primary key and subkeys. + v1.1 The recommended RSA key size has been changed from 4096 bits to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_. @@ -75,7 +80,8 @@ not be used to commit. c. ECC curve 25519 -4. Key expiry: 5 years maximum +4. Expiration date on key and all subkeys set to no more than 900 days + into the future 5. Upload your key to the SKS keyserver rotation before usage! @@ -132,11 +138,7 @@ their primary key). 2. Primary key and the signing subkey are both of type RSA, 2048 bits (OpenPGP v4 key format or later) -3. Key expiry: - - a. Primary key: 3 years maximum, expiry date renewed annually. - - b. Signing subkey: 1 year maximum, expiry date renewed every 6 months. +3. Key expiration renewed annually to a fixed day of the year 4. Create a revocation certificate & store it hardcopy offsite securely (it's about ~300 bytes). -- 2.18.0
[gentoo-dev] [PATCH v5 11/16] glep-0063: Require renewal 2 weeks before expiration
Add a rule requesting renewal of keys at least two weeks before their expiration date, in order to give services time to refresh. --- glep-0063.rst | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/glep-0063.rst b/glep-0063.rst index 9ba778b..ca834a8 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -33,6 +33,10 @@ v2 by a single requirement. The rules have been simplified to use the same maximum time of 900 days for both the primary key and subkeys. + An additional rule requesting key renewal 2 weeks before expiration + has been added. This is in order to give services and other developers time + to refresh the key. + v1.1 The recommended RSA key size has been changed from 4096 bits to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_. @@ -83,7 +87,10 @@ not be used to commit. 4. Expiration date on key and all subkeys set to no more than 900 days into the future -5. Upload your key to the SKS keyserver rotation before usage! +5. Key expiration date renewed at least 2 weeks before the previous + expiration date + +6. Upload your key to the SKS keyserver rotation before usage! Recommendations --- -- 2.18.0
[gentoo-dev] [PATCH v5 09/16] glep-0063: Stop recommending DSA subkeys
There is really no technical reason to use DSA these days, and we should focus on having a single recommendation. DSA keys are still permitted via 'minimal' requirements. --- glep-0063.rst | 18 -- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index 2402c34..7f870bb 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -36,6 +36,9 @@ v1.1 Minimal specification has been amended to allow for ECC keys. + The option of using DSA subkey has been removed from recommendations. + The section now specifies a single recommendation of using RSA. + Motivation == @@ -126,24 +129,19 @@ their primary key). # when making an OpenPGP certification, use a stronger digest than the default SHA1: cert-digest-algo SHA256 -2. Primary key type RSA, 2048 bits (OpenPGP v4 key format or later) - -3. The signing subkey of EITHER: - - a. DSA 2048 bits exactly. - - b. RSA 2048 bits exactly. +2. Primary key and the signing subkey are both of type RSA, 2048 bits + (OpenPGP v4 key format or later) -4. Key expiry: +3. Key expiry: a. Primary key: 3 years maximum, expiry date renewed annually. b. Signing subkey: 1 year maximum, expiry date renewed every 6 months. -5. Create a revocation certificate & store it hardcopy offsite securely +4. Create a revocation certificate & store it hardcopy offsite securely (it's about ~300 bytes). -6. Encrypted backup of your secret keys. +5. Encrypted backup of your secret keys. Gentoo LDAP === -- 2.18.0
[gentoo-dev] [PATCH v5 08/16] glep-0063: Allow ECC curve 25519 keys
Optionally allow using ECC curve 25519 keys. We already have developers using those keys, and given that they are supported by GnuPG 2.2, there's probably no reason to ban them. However, they're not recommended due to interoperability issues. --- glep-0063.rst | 4 1 file changed, 4 insertions(+) diff --git a/glep-0063.rst b/glep-0063.rst index fb09dd8..2402c34 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -34,6 +34,8 @@ v1.1 The larger recommendation was unjustified and resulted in people unnecessarily replacing their RSA-2048 keys. + Minimal specification has been amended to allow for ECC keys. + Motivation == @@ -68,6 +70,8 @@ not be used to commit. b. RSA, >=2048 bits (OpenPGP v4 key format or later only) + c. ECC curve 25519 + 4. Key expiry: 5 years maximum 5. Upload your key to the SKS keyserver rotation before usage! -- 2.18.0
[gentoo-dev] [PATCH v5 06/16] glep-0063: Explain minimal & recommended sections
--- glep-0063.rst | 8 1 file changed, 8 insertions(+) diff --git a/glep-0063.rst b/glep-0063.rst index 14541d7..f4b49c2 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -41,6 +41,10 @@ Specifications for OpenPGP keys Bare minimum requirements - +This section specifies obligatory requirements for all OpenPGP keys used +to commit to Gentoo. Keys that do not conform to those requirements can +not be used to commit. + 1. SHA2-series output digest (SHA1 digests internally permitted), 256bit or more:: @@ -61,6 +65,10 @@ Bare minimum requirements Recommendations --- +This section specifies the best practices for Gentoo developers. +The developers should follow those practices unless there is a strong +technical reason not to (e.g. hardware limitations, necessity of replacing +their primary key). 1. Copy ``/usr/share/gnupg/gpg-conf.skel`` to ``~/.gnupg/gpg.conf``, append the following block:: -- 2.18.0
[gentoo-dev] [PATCH v5 07/16] glep-0063: Change the recommended RSA key size to 2048 bits
Change the recommended key size recommendation for RSA from 4096 bits to 2048 bits. Use of larger keys is unjustified due to negligible gain in security, and recommending RSA-4096 unnecessarily resulted in developers replacing their RSA-2048 keys for no good reason. --- glep-0063.rst | 20 +++- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index f4b49c2..fb09dd8 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -7,7 +7,7 @@ Author: Robin H. Johnson , Michał Górny Type: Standards Track Status: Final -Version: 1 +Version: 1.1 Created: 2013-02-18 Last-Modified: 2018-07-07 Post-History: 2013-11-10 @@ -25,6 +25,15 @@ Abstract This GLEP provides both a minimum requirement and a recommended set of OpenPGP key management policies for the Gentoo Linux distribution. +Changes +=== + +v1.1 + The recommended RSA key size has been changed from 4096 bits + to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_. + The larger recommendation was unjustified and resulted in people + unnecessarily replacing their RSA-2048 keys. + Motivation == @@ -113,15 +122,13 @@ their primary key). # when making an OpenPGP certification, use a stronger digest than the default SHA1: cert-digest-algo SHA256 -2. Primary key type RSA, 4096 bits (OpenPGP v4 key format or later) - - This may require creating an entirely new key. +2. Primary key type RSA, 2048 bits (OpenPGP v4 key format or later) 3. The signing subkey of EITHER: a. DSA 2048 bits exactly. - b. RSA 4096 bits exactly. + b. RSA 2048 bits exactly. 4. Key expiry: @@ -174,6 +181,9 @@ Much of the above was driven by the following: References == +.. [#GNUPG-FAQ-11-4] GnuPG FAQ: Why doesn’t GnuPG default to using RSA-4096? + (https://www.gnupg.org/faq/gnupg-faq.html#no_default_of_rsa4096) + .. [#DEBIANGPG] Debian GPG documentation (https://wiki.debian.org/Keysigning) -- 2.18.0
[gentoo-dev] [PATCH v5 05/16] glep-0063: Split out the signing subkey into a separate point
Reword the specification to express the requirement for separate signing subkey more verbosely. Replace the ambiguous term 'dedicated' with clear explanation that it needs to be different from the primary key and not used for other purposes. Suggested-by: Kristian Fiskerstrand --- glep-0063.rst | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index 8542031..14541d7 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -46,15 +46,18 @@ Bare minimum requirements personal-digest-preferences SHA256 -2. Primary key and signing subkey of EITHER: +2. Signing subkey that is different from the primary key, and does not + have any other capabilities enabled + +3. Primary key and the signing subkey are both of type EITHER: a. DSA, 2048-bit b. RSA, >=2048 bits (OpenPGP v4 key format or later only) -3. Key expiry: 5 years maximum +4. Key expiry: 5 years maximum -4. Upload your key to the SKS keyserver rotation before usage! +5. Upload your key to the SKS keyserver rotation before usage! Recommendations --- @@ -106,7 +109,7 @@ Recommendations This may require creating an entirely new key. -3. Dedicated signing subkey of EITHER: +3. The signing subkey of EITHER: a. DSA 2048 bits exactly. -- 2.18.0
[gentoo-dev] [PATCH v5 04/16] glep-0063: Root key → primary key
Replace the custom term 'root key' with much more common 'primary key'. This is also the term used in GnuPG output. --- glep-0063.rst | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index f02537d..8542031 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -46,7 +46,7 @@ Bare minimum requirements personal-digest-preferences SHA256 -2. Root key and signing subkey of EITHER: +2. Primary key and signing subkey of EITHER: a. DSA, 2048-bit @@ -102,7 +102,7 @@ Recommendations # when making an OpenPGP certification, use a stronger digest than the default SHA1: cert-digest-algo SHA256 -2. Root key type RSA, 4096 bits (OpenPGP v4 key format or later) +2. Primary key type RSA, 4096 bits (OpenPGP v4 key format or later) This may require creating an entirely new key. @@ -114,7 +114,7 @@ Recommendations 4. Key expiry: - a. Root key: 3 years maximum, expiry date renewed annually. + a. Primary key: 3 years maximum, expiry date renewed annually. b. Signing subkey: 1 year maximum, expiry date renewed every 6 months. @@ -126,7 +126,7 @@ Recommendations Gentoo LDAP === -All Gentoo developers must list the complete fingerprint for their root +All Gentoo developers must list the complete fingerprint for their primary keys in the "``gpgfingerprint``" LDAP field. It must be exactly 40 hex digits, uppercase, with optional spaces every 8 hex digits. Regular expression for validation:: -- 2.18.0
[gentoo-dev] [PATCH v5 03/16] glep-0063: 'Gentoo subkey' → 'Signing subkey'
Replace the 'Gentoo subkey' term that might wrongly suggest that the developers are expected to create an additional, dedicated subkey for Gentoo. Suggested-by: Kristian Fiskerstrand --- glep-0063.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/glep-0063.rst b/glep-0063.rst index 0773e3b..f02537d 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -116,7 +116,7 @@ Recommendations a. Root key: 3 years maximum, expiry date renewed annually. - b. Gentoo subkey: 1 year maximum, expiry date renewed every 6 months. + b. Signing subkey: 1 year maximum, expiry date renewed every 6 months. 5. Create a revocation certificate & store it hardcopy offsite securely (it's about ~300 bytes). -- 2.18.0
[gentoo-dev] [PATCH v5 02/16] glep-0063: RSAv4 -> OpenPGP v4 key format
Replace the 'RSAv4' with 'OpenPGP v4 key format'. The RSA algorithm does not really have versions, and the author most likely meant the v4 of OpenPGP key format as outlined in RFC 4880, section 12.1. This was figured out and explained to me by Kristian Fiskerstrand. --- glep-0063.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index d1dfe3d..0773e3b 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -50,7 +50,7 @@ Bare minimum requirements a. DSA, 2048-bit - b. RSA, >=2048 bits, RSAv4 or later only + b. RSA, >=2048 bits (OpenPGP v4 key format or later only) 3. Key expiry: 5 years maximum @@ -102,7 +102,7 @@ Recommendations # when making an OpenPGP certification, use a stronger digest than the default SHA1: cert-digest-algo SHA256 -2. Root key type RSA, 4096 bits, RSAv4 or later +2. Root key type RSA, 4096 bits (OpenPGP v4 key format or later) This may require creating an entirely new key. -- 2.18.0
[gentoo-dev] [PATCH v5 01/16] glep-0063: Use 'OpenPGP' as appropriate
Replace many of the incorrect uses of GPG/GnuPG [key] with OpenPGP. G[nu]PG has been left where the text clearly refers to the specific implementation of OpenPGP rather than the standard itself. --- glep-0063.rst | 28 +++- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index c59d545..d1dfe3d 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -1,14 +1,15 @@ --- GLEP: 63 -Title: Gentoo GPG key policies +Title: Gentoo OpenPGP policies Author: Robin H. Johnson , Andreas K. Hüttel , -Marissa Fischer +Marissa Fischer , +Michał Górny Type: Standards Track Status: Final Version: 1 Created: 2013-02-18 -Last-Modified: 2015-08-25 +Last-Modified: 2018-07-07 Post-History: 2013-11-10 Content-Type: text/x-rst --- @@ -21,22 +22,22 @@ Many developers and external sources helped in this GLEP. Abstract -This GLEP provides both a minimum requirement and a recommended set of GPG -key management policies for the Gentoo Linux distribution. +This GLEP provides both a minimum requirement and a recommended set of +OpenPGP key management policies for the Gentoo Linux distribution. Motivation == Given the increasing use and importance of cryptographic protocols in internet -transactions of any kind, unified requirements for GnuPG keys used in Gentoo +transactions of any kind, unified requirements for OpenPGP keys used in Gentoo Linux development are sorely needed. This document provides both a set of bare minimum requirements and a set of best practice recommendations for -the use of GnuPG by Gentoo Linux developers. It is intended to provide -a basis for future improvements such as, e.g., consistent ebuild or package -signing and verifying by end users. +the use of GnuPG (or other OpenPGP providers) by Gentoo Linux developers. +It is intended to provide a basis for future improvements such as, e.g., +consistent ebuild or package signing and verifying by end users. -Specifications for GnuPG keys -= +Specifications for OpenPGP keys +=== Bare minimum requirements - @@ -125,7 +126,7 @@ Recommendations Gentoo LDAP === -All Gentoo developers must list the complete GPG fingerprint for their root +All Gentoo developers must list the complete fingerprint for their root keys in the "``gpgfingerprint``" LDAP field. It must be exactly 40 hex digits, uppercase, with optional spaces every 8 hex digits. Regular expression for validation:: @@ -195,7 +196,8 @@ References Copyright = -Copyright (c) 2013 by Robin Hugh Johnson, Andreas K. Hüttel, Marissa Fischer. +Copyright (c) 2013-2018 by Robin Hugh Johnson, Andreas K. Hüttel, +Marissa Fischer, Michał Górny. This work is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License. To view a copy of this license, visit -- 2.18.0
[gentoo-dev] [PATCH v5 00/16] GLEP 63, once again
Hi, Here's another updated version of GLEP 63, taking more feedback into consideration. Changes since v4: - removed all gpg.conf bits (which proved obsolete), - added SHA-2 requirement on subkeys (this is in RISEUP and gnupg default, and we require SHA-2 output anyway, so it makes sense to extend it), - unified punctuation. Full text below. -- Best regards, Michał Górny Michał Górny (16): glep-0063: Use 'OpenPGP' as appropriate glep-0063: RSAv4 -> OpenPGP v4 key format glep-0063: 'Gentoo subkey' → 'Signing subkey' glep-0063: Root key → primary key glep-0063: Split out the signing subkey into a separate point glep-0063: Explain minimal & recommended sections glep-0063: Change the recommended RSA key size to 2048 bits glep-0063: Allow ECC curve 25519 keys glep-0063: Stop recommending DSA subkeys glep-0063: Update and unify expiration term glep-0063: Require renewal 2 weeks before expiration glep-0063: Disallow using DSA keys glep-0063: Remove whitespace from LDAP field glep-0063: Remove gpg.conf bits glep-0063: Extend SHA-2 requirement to self-signatures on subkeys glep-0063: Unify punctuation glep-0063.rst | 167 -- 1 file changed, 81 insertions(+), 86 deletions(-) -- 2.18.0 --- GLEP: 63 Title: Gentoo OpenPGP policies Author: Robin H. Johnson , Andreas K. Hüttel , Marissa Fischer , Michał Górny Type: Standards Track Status: Final Version: 2 Created: 2013-02-18 Last-Modified: 2018-07-07 Post-History: 2013-11-10 Content-Type: text/x-rst --- Credits === Many developers and external sources helped in this GLEP. Abstract This GLEP provides both a minimum requirement and a recommended set of OpenPGP key management policies for the Gentoo Linux distribution. Changes === v2 The distinct minimal and recommended expirations have been replaced by a single requirement. The rules have been simplified to use the same maximum time of 900 days for both the primary key and subkeys. An additional rule requesting key renewal 2 weeks before expiration has been added. This is in order to give services and other developers time to refresh the key. The usage of DSA keys has been disallowed. The ``gpgfingerprint`` LDAP field has been altered to remove optional whitespace. The ``gpg.conf`` contents have been removed as they were seriously outdated and decreased security over the modern defaults. The requirement of SHA-2 digest has been extended to apply to self- signatures made on subkeys. v1.1 The recommended RSA key size has been changed from 4096 bits to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_. The larger recommendation was unjustified and resulted in people unnecessarily replacing their RSA-2048 keys. Minimal specification has been amended to allow for ECC keys. The option of using DSA subkey has been removed from recommendations. The section now specifies a single recommendation of using RSA. Motivation == Given the increasing use and importance of cryptographic protocols in internet transactions of any kind, unified requirements for OpenPGP keys used in Gentoo Linux development are sorely needed. This document provides both a set of bare minimum requirements and a set of best practice recommendations for the use of GnuPG (or other OpenPGP providers) by Gentoo Linux developers. It is intended to provide a basis for future improvements such as, e.g., consistent ebuild or package signing and verifying by end users. Specifications for OpenPGP keys === Bare minimum requirements - This section specifies obligatory requirements for all OpenPGP keys used to commit to Gentoo. Keys that do not conform to those requirements can not be used to commit. 1. SHA-2 series output digest (SHA-1 digests internally permitted), at least 256-bit. All subkey self-signatures must use this digest. 2. Signing subkey that is different from the primary key, and does not have any other capabilities enabled. 3. Primary key and the signing subkey are both of type EITHER: a. RSA, >=2048 bits (OpenPGP v4 key format or later only), b. ECC curve 25519. 4. Expiration date on key and all subkeys set to no more than 900 days into the future. 5. Key expiration date renewed at least 2 weeks before the previous expiration date. 6. Upload your key to the SKS keyserver rotation before usage! Recommendations --- This section specifies the best practices for Gentoo developers. The developers should follow those practices unless there is a strong technical reason not to (e.g. hardware limitations, necessity of replacing their primary key). 1. Primary key and the signing subkey are both of type RSA, 2048 bits (OpenPGP v4 key format or later). 2. Key expiration renewed annually to a fixed day of the year. 3. Create a revocation certificate & store it
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 08:10 PM, Rich Freeman wrote: > Again, the current portage support for git verification doesn't check > any developer keys. right, so why would it be material for a news item improving the status quo for those synching using the official rsync method? -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On Sun, Jul 8, 2018 at 1:50 PM Kristian Fiskerstrand wrote: > > On 07/08/2018 07:34 PM, Rich Freeman wrote: > > The patch is to do the verification before > > checking it out so that if it fails the tree is left in a > > last-known-good state (at least as seen by tools at the filesystem > > level - the fetched bad commits would still be visible to git). > > there is still a very different key management issue discussed. If a > developers credentials are removed from Gentoo LDAP for some reason it > will be stopped from pushing new commits immediately, but the third > party verification can be valid up until that point and after since the > developer might not have published a revocation certificate. > > the git sync method will need a way to distinguish this for end-users, > but the proper rsync synchronization will be able to trust the data at > the point we say it is OK. Again, the current portage support for git verification doesn't check any developer keys. It just checks the keys in /usr/share/openpgp-keys/gentoo-release.asc (after fetching updates). If the HEAD commit verifies, it is good, if not it is bad. It doesn't matter whether any commit in the tree other than the HEAD has a valid signature, and it will reject any signature from a developer on HEAD. The logic is somewhat similar to rsync in that regard. There is a separate proposal out there which is unimplemented to check developer keys, and we aren't talking about that. I agree that it has the challenge of figuring out whether the key was acceptable at the time the signature was made. -- Rich
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 06:56 AM, Michał Górny wrote: > W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian > Fiskerstrand napisał: >> On 07/08/2018 08:53 AM, Michał Górny wrote: >>> Is safe git syncing implemented already? If not, maybe finish it first and >>> cover both with a single news item. Git is going to be more efficient here, >>> so people may want to learn they have an alternative. >> >> Why complicate things, and increase wait for something that benefits >> most users, just to give alternatives to a few using non-default sync >> mechanism. Securing git distribution is a whole different ballpark. >> > > Let me rephrase. Let's say I'm using rsync. This new feature is > something positive but it breaks my use case (for one of the listed > reasons -- overlayfs, inode use, small fs cache). After reading this > news item, I learn that my only option is to disable the new feature. > > Now, I would appreciate being told that there's an alternate sync method > that handles secure updates without having all those drawbacks. The thing is, the normal git tree doesn't even provide pre-generated metadata, and I see then gentoo-mirror repo that provides metadata does not have commits signed with an release key: https://github.com/gentoo-mirror/gentoo/commits/stable So I'm really not comfortable recommending git to anyone at this point. -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 07:34 PM, Rich Freeman wrote: > The patch is to do the verification before > checking it out so that if it fails the tree is left in a > last-known-good state (at least as seen by tools at the filesystem > level - the fetched bad commits would still be visible to git). there is still a very different key management issue discussed. If a developers credentials are removed from Gentoo LDAP for some reason it will be stopped from pushing new commits immediately, but the third party verification can be valid up until that point and after since the developer might not have published a revocation certificate. the git sync method will need a way to distinguish this for end-users, but the proper rsync synchronization will be able to trust the data at the point we say it is OK. -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 08/07/18 18:34, Rich Freeman wrote: > On Sun, Jul 8, 2018 at 9:02 AM Kristian Fiskerstrand wrote: >> On 07/08/2018 08:53 AM, Michał Górny wrote: >>> Is safe git syncing implemented already? If not, maybe finish it first and >>> cover both with a single news item. Git is going to be more efficient here, >>> so people may want to learn they have an alternative. >> Why complicate things, and increase wait for something that benefits >> most users, just to give alternatives to a few using non-default sync >> mechanism. Securing git distribution is a whole different ballpark. >> > I'll agree that it is different, but we're talking about verification > of the HEAD signature by infra, not verification of individual > developer keys, which was the topic of the recent thread. > > Verification is already built-into portage for git syncing (but off by > default). The problem is that portage will still checkout the tree if > it fails verification. The patch is to do the verification before > checking it out so that if it fails the tree is left in a > last-known-good state (at least as seen by tools at the filesystem > level - the fetched bad commits would still be visible to git). > Slightly radical thought here, but hear me out .. Could we use this same functionality to be able to validate the tree integrity with respect to CI testing? I mean, if the tree is 'broken' could we have some kind of warning displayed perhaps? Something that could be toggled (or default Off) would indeed be good, so that users/devs can choose what level or 'standard' of tree state they're prepared to accept. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On Sun, Jul 8, 2018 at 9:02 AM Kristian Fiskerstrand wrote: > > On 07/08/2018 08:53 AM, Michał Górny wrote: > > Is safe git syncing implemented already? If not, maybe finish it first and > > cover both with a single news item. Git is going to be more efficient here, > > so people may want to learn they have an alternative. > > Why complicate things, and increase wait for something that benefits > most users, just to give alternatives to a few using non-default sync > mechanism. Securing git distribution is a whole different ballpark. > I'll agree that it is different, but we're talking about verification of the HEAD signature by infra, not verification of individual developer keys, which was the topic of the recent thread. Verification is already built-into portage for git syncing (but off by default). The problem is that portage will still checkout the tree if it fails verification. The patch is to do the verification before checking it out so that if it fails the tree is left in a last-known-good state (at least as seen by tools at the filesystem level - the fetched bad commits would still be visible to git). -- Rich
Re: [gentoo-dev] [PATCH v4 00/14] GLEP 63 update
W dniu nie, 08.07.2018 o godzinie 15∶06 +0200, użytkownik Kristian Fiskerstrand napisał: > On 07/07/2018 03:11 PM, Michał Górny wrote: > > > > 1. SHA2-series output digest (SHA1 digests internally permitted), > > > >256bit or more:: > > > >personal-digest-preferences SHA256 > > > > > > Is the config line still needed with current GnuPG versions? > > > > I'll let others answer that. In any case, the point itself (requiring > > SHA-2 digest) makes sense. The RiseUp standard requires all self- > > signatures to be SHA-2, and I was planning on verifying that as well. > > > > no, SHA256 in this context is already default, and it doesn't impact > cert-algo that you seem to go on about. > Ok, I've removed this other bit too. I'll wait for more comments before submitting v5. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] News Item: Portage rsync hardlink support
W dniu nie, 08.07.2018 o godzinie 15∶02 +0200, użytkownik Kristian Fiskerstrand napisał: > On 07/08/2018 08:53 AM, Michał Górny wrote: > > Is safe git syncing implemented already? If not, maybe finish it first and > > cover both with a single news item. Git is going to be more efficient here, > > so people may want to learn they have an alternative. > > Why complicate things, and increase wait for something that benefits > most users, just to give alternatives to a few using non-default sync > mechanism. Securing git distribution is a whole different ballpark. > Let me rephrase. Let's say I'm using rsync. This new feature is something positive but it breaks my use case (for one of the listed reasons -- overlayfs, inode use, small fs cache). After reading this news item, I learn that my only option is to disable the new feature. Now, I would appreciate being told that there's an alternate sync method that handles secure updates without having all those drawbacks. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
[gentoo-dev] Last rites: dev-python/hgtools and its reverse dependency dev-python/jaraco-utils
# Louis Sautier (8 July 2018) # Superseded, respectively by dev-python/setuptools_scm # and dev-python/jaraco-* + dev-python/tempora. # Removal in a month. See https://bugs.gentoo.org/582728 dev-python/hgtools dev-python/jaraco-utils signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] [PATCH v4 00/14] GLEP 63 update
On 07/07/2018 03:11 PM, Michał Górny wrote: >>> 1. SHA2-series output digest (SHA1 digests internally permitted), >>>256bit or more:: >>>personal-digest-preferences SHA256 >> Is the config line still needed with current GnuPG versions? > I'll let others answer that. In any case, the point itself (requiring > SHA-2 digest) makes sense. The RiseUp standard requires all self- > signatures to be SHA-2, and I was planning on verifying that as well. > no, SHA256 in this context is already default, and it doesn't impact cert-algo that you seem to go on about. -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 08:53 AM, Michał Górny wrote: > Is safe git syncing implemented already? If not, maybe finish it first and > cover both with a single news item. Git is going to be more efficient here, > so people may want to learn they have an alternative. Why complicate things, and increase wait for something that benefits most users, just to give alternatives to a few using non-default sync mechanism. Securing git distribution is a whole different ballpark. -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 08:08 AM, Zac Medico wrote: > For example, if signature verification fails during a > sync operation, the new hardlink behavior will preserve the previous > state of the repository. This seems like a nice improvement, thank you for implementing it and making it the new default. -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 02:28 AM, Toralf Förster wrote: > On 07/08/2018 08:08 AM, Zac Medico wrote: >> Please review. >> >> Title: Portage rsync hardlink support >> Author: Zac Medico >> Posted: 2018-07-11 >> Revision: 1 >> News-Item-Format: 2.0 >> Display-If-Installed: sys-apps/portage >> IMO there's another heads-up for users having an unsual configuration: > > So we do speak about files under /usr/portage itself and not about that > dir (==changing its inode number), right? > > B/c otherwise there's another heads-up for people bind-mounting > /usr/portage onto chrooted images. This case will continue to work because the download happens in a subdirectory, and after it's verified we call rsync again to apply the changes to the top-level directory. -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 08/07/18 10:21, Zac Medico wrote: > On 07/08/2018 02:15 AM, Michał Górny wrote: >> >> Are you sure about that? That might have been the case so far but this >> hardlink tree may actually tip the balance. > Even if it takes twice a long (which it doesn't), the difference is > negligible for most people because they usually don't sync more than > once per day. It is recommended not to hammer the rsync mirrors with updates, and so this argument holds true for "most users". signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 08:08 AM, Zac Medico wrote: > Please review. > > Title: Portage rsync hardlink support > Author: Zac Medico > Posted: 2018-07-11 > Revision: 1 > News-Item-Format: 2.0 > Display-If-Installed: sys-apps/portage > IMO there's another heads-up for users having an unsual configuration: So we do speak about files under /usr/portage itself and not about that dir (==changing its inode number), right? B/c otherwise there's another heads-up for people bind-mounting /usr/portage onto chrooted images. -- Toralf PGP 23217DA7 9B888F45 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/08/2018 02:15 AM, Michał Górny wrote: > Dnia 8 lipca 2018 09:14:06 CEST, Zac Medico napisał(a): >> On 07/07/2018 11:53 PM, Michał Górny wrote: >>> Dnia 8 lipca 2018 08:08:31 CEST, Zac Medico >> napisał(a): Please review. Title: Portage rsync hardlink support Author: Zac Medico Posted: 2018-07-11 Revision: 1 News-Item-Format: 2.0 Display-If-Installed: sys-apps/portage For users of the rsync tree, beginning with sys-apps/portage-2.3.42, the default behavior for sync operations will use hardlinks in order to ensure that a repository remains in a valid state if something goes wrong [1]. For example, if signature verification fails during >> a sync operation, the new hardlink behavior will preserve the previous state of the repository. The new behavior may conflict with configurations that restrict the use of hardlinks, such as overlay filesystems. Therefore, users will have to set "sync-allow-hardlinks = no" in repos.conf if they have a configuration that restricts the use of hardlinks, but this should not be very common: [DEFAULT] sync-allow-hardlinks = no [1] https://bugs.gentoo.org/660410 sys-apps/portage: use rsync --link-dest to implement atomic repository updates (and abort if signature verification fails) >>> >>> Is safe git syncing implemented already? If not, maybe finish it >> first and cover both with a single news item. Git is going to be more >> efficient here, so people may want to learn they have an alternative. >> >> Yeah there's already a patch for git sync [1] but I'd rather not make >> this news item more complicated than it needs to be. I wouldn't have >> bothered with a news item except that I want to give people some >> warning >> in case they are using overlayfs [2]. I think the efficiency difference >> between rsync and git here are pretty negligible for most people. > > Are you sure about that? That might have been the case so far but this > hardlink tree may actually tip the balance. Even if it takes twice a long (which it doesn't), the difference is negligible for most people because they usually don't sync more than once per day. -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
Dnia 8 lipca 2018 09:14:06 CEST, Zac Medico napisał(a): >On 07/07/2018 11:53 PM, Michał Górny wrote: >> Dnia 8 lipca 2018 08:08:31 CEST, Zac Medico >napisał(a): >>> Please review. >>> >>> Title: Portage rsync hardlink support >>> Author: Zac Medico >>> Posted: 2018-07-11 >>> Revision: 1 >>> News-Item-Format: 2.0 >>> Display-If-Installed: sys-apps/portage >>> >>> For users of the rsync tree, beginning with sys-apps/portage-2.3.42, >>> the default behavior for sync operations will use hardlinks in order >>> to ensure that a repository remains in a valid state if something >>> goes wrong [1]. For example, if signature verification fails during >a >>> sync operation, the new hardlink behavior will preserve the previous >>> state of the repository. >>> >>> The new behavior may conflict with configurations that restrict the >>> use of hardlinks, such as overlay filesystems. Therefore, users will >>> have to set "sync-allow-hardlinks = no" in repos.conf if they have >>> a configuration that restricts the use of hardlinks, but this should >>> not be very common: >>> >>> [DEFAULT] >>> sync-allow-hardlinks = no >>> >>> [1] https://bugs.gentoo.org/660410 sys-apps/portage: use rsync >>>--link-dest to implement atomic repository updates (and abort if >>>signature verification fails) >> >> Is safe git syncing implemented already? If not, maybe finish it >first and cover both with a single news item. Git is going to be more >efficient here, so people may want to learn they have an alternative. > >Yeah there's already a patch for git sync [1] but I'd rather not make >this news item more complicated than it needs to be. I wouldn't have >bothered with a news item except that I want to give people some >warning >in case they are using overlayfs [2]. I think the efficiency difference >between rsync and git here are pretty negligible for most people. Are you sure about that? That might have been the case so far but this hardlink tree may actually tip the balance. > >[1] https://bugs.gentoo.org/660372 >[2] >https://www.brunsware.de/blog/gentoo/portage-tree-squashfs-overlayfs.html -- Best regards, Michał Górny (by phone)
Re: [gentoo-portage-dev] [PATCH v2] rsync: quarantine data prior to verification (bug 660410)
On 07/08/2018 01:48 AM, Fabian Groffen wrote: > Would it be worth having a flag to keep long standing sync behaviour for > situations where either diskspace, or verification results in problems > (like bug #657320)? > > Fabian Yeah, also the hardlinks will not work with overlayfs [1], so I've posted a news item [2] explaining that people can set set "sync-allow-hardlinks = no" in repos.conf to disable the new behavior. [1] https://www.brunsware.de/blog/gentoo/portage-tree-squashfs-overlayfs.html [2] https://archives.gentoo.org/gentoo-dev/message/004d9ca4c918e55638a919b87b5d52dd -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH v2] rsync: quarantine data prior to verification (bug 660410)
Would it be worth having a flag to keep long standing sync behaviour for
situations where either diskspace, or verification results in problems
(like bug #657320)?
Fabian
On 07-07-2018 18:55:52 -0700, Zac Medico wrote:
> Sync into a quarantine subdirectory, using the rsync --link-dest option
> to create hardlinks to identical files in the previous snapshot of the
> repository. If hardlinks are not supported, then show a warning message
> and sync directly to the normal repository location.
>
> If verification succeeds, then the quarantine subdirectory is synced
> to the normal repository location, and the quarantine subdirectory
> is deleted. If verification fails, then the quarantine directory is
> preserved for purposes of analysis.
>
> Even if verification happens to be disabled, the quarantine directory
> is still useful for making the repository update more atomic, so that
> it is less likely that normal repository location will be observed in
> a partially synced state.
>
> The new behavior may conflict with configurations that restrict the
> use of hardlinks, such as overlay filesystems. Therefore, users will
> have to set "sync-allow-hardlinks = no" in repos.conf if they have
> a configuration that prevents the use of hardlinks, but this should
> not be very common.
>
> Bug: https://bugs.gentoo.org/660410
> ---
> [PATCH v2] makes it possible to disable the new behavior by setting
> "sync-allow-hardlinks = no" in repos.conf
>
> cnf/repos.conf | 1 +
> man/portage.5 | 8 +++
> pym/portage/repository/config.py| 8 ++-
> pym/portage/sync/modules/rsync/rsync.py | 89
> ++---
> 4 files changed, 97 insertions(+), 9 deletions(-)
>
> diff --git a/cnf/repos.conf b/cnf/repos.conf
> index 352073cfd5..419f6d1182 100644
> --- a/cnf/repos.conf
> +++ b/cnf/repos.conf
> @@ -6,6 +6,7 @@ location = /usr/portage
> sync-type = rsync
> sync-uri = rsync://rsync.gentoo.org/gentoo-portage
> auto-sync = yes
> +sync-allow-hardlinks = yes
> sync-rsync-verify-jobs = 1
> sync-rsync-verify-metamanifest = yes
> sync-rsync-verify-max-age = 24
> diff --git a/man/portage.5 b/man/portage.5
> index 5adb07d821..acc80791be 100644
> --- a/man/portage.5
> +++ b/man/portage.5
> @@ -973,6 +973,14 @@ files). Defaults to true.
> .br
> Valid values: true, false.
> .TP
> +.B sync\-allow\-hardlinks = yes|no
> +Allow sync plugins to use hardlinks in order to ensure that a repository
> +remains in a valid state if something goes wrong during the sync operation.
> +For example, if signature verification fails during a sync operation,
> +the previous state of the repository will be preserved. This option may
> +conflict with configurations that restrict the use of hardlinks, such as
> +overlay filesystems.
> +.TP
> .B sync\-cvs\-repo
> Specifies CVS repository.
> .TP
> diff --git a/pym/portage/repository/config.py
> b/pym/portage/repository/config.py
> index 1d897bb903..c7440369c2 100644
> --- a/pym/portage/repository/config.py
> +++ b/pym/portage/repository/config.py
> @@ -86,6 +86,7 @@ class RepoConfig(object):
> 'sync_type', 'sync_umask', 'sync_uri', 'sync_user',
> 'thin_manifest',
> 'update_changelog', '_eapis_banned', '_eapis_deprecated',
> '_masters_orig', 'module_specific_options',
> 'manifest_required_hashes',
> + 'sync_allow_hardlinks',
> 'sync_openpgp_key_path',
> 'sync_openpgp_key_refresh_retry_count',
> 'sync_openpgp_key_refresh_retry_delay_max',
> @@ -188,6 +189,9 @@ class RepoConfig(object):
> self.strict_misc_digests = repo_opts.get(
> 'strict-misc-digests', 'true').lower() == 'true'
>
> + self.sync_allow_hardlinks = repo_opts.get(
> + 'sync-allow-hardlinks', 'true').lower() in ('true',
> 'yes')
> +
> self.sync_openpgp_key_path = repo_opts.get(
> 'sync-openpgp-key-path', None)
>
> @@ -534,6 +538,7 @@ class RepoConfigLoader(object):
> 'clone_depth',
> 'eclass_overrides',
> 'force', 'masters',
> 'priority', 'strict_misc_digests',
> 'sync_depth',
> 'sync_hooks_only_on_change',
> + 'sync_allow_hardlinks',
> 'sync_openpgp_key_path',
>
> 'sync_openpgp_key_refresh_retry_count',
>
> 'sync_openpgp_key_refresh_retry_delay_max',
> @@ -962,7 +967,8 @@ class RepoConfigLoader(object):
> def config_string(self):
> bool_keys = ("strict_misc_digests",)
> str_or_int_keys = ("auto_sync", "clone_depth", "format",
> "location",
> -
Re: [gentoo-dev] News Item: Portage rsync hardlink support
On 07/07/2018 11:53 PM, Michał Górny wrote: > Dnia 8 lipca 2018 08:08:31 CEST, Zac Medico napisał(a): >> Please review. >> >> Title: Portage rsync hardlink support >> Author: Zac Medico >> Posted: 2018-07-11 >> Revision: 1 >> News-Item-Format: 2.0 >> Display-If-Installed: sys-apps/portage >> >> For users of the rsync tree, beginning with sys-apps/portage-2.3.42, >> the default behavior for sync operations will use hardlinks in order >> to ensure that a repository remains in a valid state if something >> goes wrong [1]. For example, if signature verification fails during a >> sync operation, the new hardlink behavior will preserve the previous >> state of the repository. >> >> The new behavior may conflict with configurations that restrict the >> use of hardlinks, such as overlay filesystems. Therefore, users will >> have to set "sync-allow-hardlinks = no" in repos.conf if they have >> a configuration that restricts the use of hardlinks, but this should >> not be very common: >> >> [DEFAULT] >> sync-allow-hardlinks = no >> >> [1] https://bugs.gentoo.org/660410 sys-apps/portage: use rsync >>--link-dest to implement atomic repository updates (and abort if >>signature verification fails) > > Is safe git syncing implemented already? If not, maybe finish it first and > cover both with a single news item. Git is going to be more efficient here, > so people may want to learn they have an alternative. Yeah there's already a patch for git sync [1] but I'd rather not make this news item more complicated than it needs to be. I wouldn't have bothered with a news item except that I want to give people some warning in case they are using overlayfs [2]. I think the efficiency difference between rsync and git here are pretty negligible for most people. [1] https://bugs.gentoo.org/660372 [2] https://www.brunsware.de/blog/gentoo/portage-tree-squashfs-overlayfs.html -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News Item: Portage rsync hardlink support
Dnia 8 lipca 2018 08:08:31 CEST, Zac Medico napisał(a): >Please review. > >Title: Portage rsync hardlink support >Author: Zac Medico >Posted: 2018-07-11 >Revision: 1 >News-Item-Format: 2.0 >Display-If-Installed: sys-apps/portage > >For users of the rsync tree, beginning with sys-apps/portage-2.3.42, >the default behavior for sync operations will use hardlinks in order >to ensure that a repository remains in a valid state if something >goes wrong [1]. For example, if signature verification fails during a >sync operation, the new hardlink behavior will preserve the previous >state of the repository. > >The new behavior may conflict with configurations that restrict the >use of hardlinks, such as overlay filesystems. Therefore, users will >have to set "sync-allow-hardlinks = no" in repos.conf if they have >a configuration that restricts the use of hardlinks, but this should >not be very common: > >[DEFAULT] >sync-allow-hardlinks = no > >[1] https://bugs.gentoo.org/660410 sys-apps/portage: use rsync >--link-dest to implement atomic repository updates (and abort if >signature verification fails) Is safe git syncing implemented already? If not, maybe finish it first and cover both with a single news item. Git is going to be more efficient here, so people may want to learn they have an alternative. -- Best regards, Michał Górny (by phone)
[gentoo-dev] News Item: Portage rsync hardlink support
Please review. Title: Portage rsync hardlink support Author: Zac Medico Posted: 2018-07-11 Revision: 1 News-Item-Format: 2.0 Display-If-Installed: sys-apps/portage For users of the rsync tree, beginning with sys-apps/portage-2.3.42, the default behavior for sync operations will use hardlinks in order to ensure that a repository remains in a valid state if something goes wrong [1]. For example, if signature verification fails during a sync operation, the new hardlink behavior will preserve the previous state of the repository. The new behavior may conflict with configurations that restrict the use of hardlinks, such as overlay filesystems. Therefore, users will have to set "sync-allow-hardlinks = no" in repos.conf if they have a configuration that restricts the use of hardlinks, but this should not be very common: [DEFAULT] sync-allow-hardlinks = no [1] https://bugs.gentoo.org/660410 sys-apps/portage: use rsync --link-dest to implement atomic repository updates (and abort if signature verification fails) -- Thanks, Zac
