Re: [gentoo-dev] [News item review] Portage rsync tree verification (v5)
Please may it be the last! --- Title: Portage rsync tree verification Author: Michał GórnyPosted: 2018-01-xx Revision: 1 News-Item-Format: 2.0 Display-If-Installed: sys-apps/portage Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo repository after rsync by default. The new verification is intended for users who are syncing via rsync. Users syncing via git or other methods are not affected, and complete verification for them will be provided in the future. The verification is implemented via app-portage/gemato. Currently, the whole repository is verified after syncing. On systems with slow hard drives, this could take around 2 minutes. If you wish to disable it, you can disable the 'rsync-verify' USE flag on sys-apps/portage or set 'sync-rsync-verify-metamanifest = no' in your repos.conf. Please note that the verification currently does not prevent Portage from using the repository after syncing. If 'emerge --sync' fails, do not install any packages and retry syncing. In case of prolonged or frequent verification failures, please make sure to report a bug including the failing mirror addresses (found in emerge.log). The verification uses information from the binary keyring provided by the app-crypt/gentoo-keys package. The keys are refreshed from the keyserver before every use in order to check for revocation. The post-sync verification ensures that the key package is verified itself. However, manual verification is required before the first use. On Gentoo installations created using installation media that included portage-2.3.22, the keys will already be covered by the installation media signatures. On existing installations, you need to manually compare the primary key fingerprint (reported by gemato on every sync) against the official Gentoo keys [1]. An example gemato output is: INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: 1234567890ABCDEF1234567890ABCDEF12345678 INFO:root:- subkey: FEDCBA0987654321FEDCBA0987654321FEDCBA09 Please note that the above snippet does not include the real key id on purpose. The primary key actually printed by gemato must match the 'Gentoo Portage Snapshot Signing Key' on the website. Please make sure to also check the certificate used for the secure connection to the site! [1]:https://www.gentoo.org/downloads/signatures/ -- Best regards, Michał Górny
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v4)
On Sun, Jan 28, 2018 at 09:30:31PM +0100, Andrew Barchuk wrote: > Hi everyone, > > > three possible solutions for splitting distfiles were listed: > There's another option to use character ranges for each directory > computed in a way to have the files distributed evenly. One way to do > that is to use filename prefix of dynamic length so that each range > holds the same number of files. E.g. we would have Ab/, Ap/, Ar/ but > texlive-module-te/, texlive-module-th/, texlive-module-ti/. A similar > but simpler option is to use file names as range bounds (the same way > dictionaries use words to demarcate page bounds): each directory will > have a name of the first file located inside. This way files will be > distributed evenly and it's still easy to pick a correct directory where > a file will be located manually. This was discussed early on, but thank you for the reminder, as it got dropped from later discussions. > [snip code] > Using the approach above the files will distributed evenly among the > directories keeping the possibility to determine the directory for a > specific file by hand. It's possible if necessary to keep the directory > structure unchanged for very long time and it will likely stay > well-balanced. Picking a directory for a file is very cheap. The only > obvious downside I see is that it's necessary to know list of > directories to pick the correct one (can be mitigated by caching the > list of directories if important). If it's desirable to make directory > names shorter or to look less like file names it's fairly easy to > achieve by keeping only unique prefixes of directories. For example: As for the problem you describe, one of the requirements in the discussion is that given ONLY the file or filename, and NOTHING ELSE, it should be possible to determine where in a hierarchy it should go. No prior knowledge about the hierarchy was permitted. Some parties might answer that you just need an index file then, but that means you have to keep the index file in sync often. It's a superbly readable result (in the general class of perfect hashes based on lots of well-known input). The class of solution suffers another problem in addition the one you noted: if input changes sufficiently, then rebalancing is expensive/hard. As a concrete example, say we add a new category for something something with lots of common prefixes in distfiles. dev-scratch/ as an example, where all distfiles start with 'scratch-'. Unless we know up-front that we're going to add a thousand distfiles here (not unreasonable, dev-python is ~1800 packages), they might start by going into the 'sc' directory, but later we want them to be in 'scratch', as the tree is unweighted otherwise. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: Digital signature
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v4)
Hi everyone, > three possible solutions for splitting distfiles were listed: > > a. using initial portion of filename, > > b. using initial portion of file hash, > > c. using initial portion of filename hash. > > The significant advantage of the filename option was simplicity. With > that solution, the users could easily determine the correct subdirectory > themselves. However, it's significant disadvantage was very uneven > shuffling of data. In particular, the TeΧ Live packages alone count > almost 23500 distfiles and all use a common prefix, making it impossible > to split them further. > > The alternate option of using file hash has the advantage of having > a more balanced split. There's another option to use character ranges for each directory computed in a way to have the files distributed evenly. One way to do that is to use filename prefix of dynamic length so that each range holds the same number of files. E.g. we would have Ab/, Ap/, Ar/ but texlive-module-te/, texlive-module-th/, texlive-module-ti/. A similar but simpler option is to use file names as range bounds (the same way dictionaries use words to demarcate page bounds): each directory will have a name of the first file located inside. This way files will be distributed evenly and it's still easy to pick a correct directory where a file will be located manually. I have implemented a sketch of distfiles splitting that's using file names as bounds in Python to demonstrate the idea (excuse possibly non-idiomatic code, I'm not very versed in Python): $ cat distfile-dirs.py #!/usr/bin/env python3 import sys """ Builds list of dictionary directories to split the list of input files into evenly. Each directory has name of the first file that is located in the directory. Takes number of directories as an argument and reads list of files from stdin. The resulting list or directories is printed to stdout. """ dir_num = int(sys.argv[1]) distfiles = sys.stdin.read().splitlines() distfile_num = len(distfiles) dir_size = distfile_num / dir_num # allows adding files in the beginning without repartitioning dirs = ["0"] next_dir = dir_size while next_dir < distfile_num: dirs.append(distfiles[round(next_dir)]) next_dir += dir_size print("/\n".join(dirs) + "/") $ cat pick-distfiles-dir.py #!/usr/bin/env python3 """ Picks the directory for a given file name. Takes a distfile name as an argument. Reads sorted list of directories from stdin, name of each directory is assumed to be the name of first file that's located inside. """ import sys distfile = sys.argv[1] dirs = sys.stdin.read().splitlines() left = 0 right = len(dirs) - 1 while left < right: pivot = round((left + right) / 2) if (dirs[pivot] <= distfile): left = pivot + 1 else: right = pivot - 1 if distfile < dirs[right]: print(dirs[right-1]) else: print(dirs[right]) $ # distfiles.txt contains all the distfile names $ head -n5 distfiles.txt 0CD9CDDE3F56BB5250D87C54592F04CBC24F03BF-wagon-provider-api-2.10.jar 0CE1EDB914C94EBC388F086C6827E8BDEEC71AC2-commons-lang-2.6.jar 0DCC973606CBD9737541AA5F3E76DED6E3F4D0D0-iri.jar 0ad-0.0.22-alpha-unix-build.tar.xz 0ad-0.0.22-alpha-unix-data.tar.xz $ # calculate 500 directories to split distfiles into evenly $ cat distfiles.txt | ./distfile-dirs.py 500 > dirs.txt $ tail -n5 dirs.txt xrmap-2.29.tar.bz2/ xview-3.2p1.4-18c.tar.gz/ yasat-700.tar.gz/ yubikey-manager-qt-0.4.0.tar.gz/ zimg-2.5.1.tar.gz $ # pick a directory for xvinfo-1.0.1.tar.bz2 $ cat dirs.txt | ./pick-distfiles-dir.py xvinfo-1.0.1.tar.bz2 xview-3.2p1.4-18c.tar.gz/ Using the approach above the files will distributed evenly among the directories keeping the possibility to determine the directory for a specific file by hand. It's possible if necessary to keep the directory structure unchanged for very long time and it will likely stay well-balanced. Picking a directory for a file is very cheap. The only obvious downside I see is that it's necessary to know list of directories to pick the correct one (can be mitigated by caching the list of directories if important). If it's desirable to make directory names shorter or to look less like file names it's fairly easy to achieve by keeping only unique prefixes of directories. For example: xrmap-2.29.tar.bz2/ xview-3.2p1.4-18c.tar.gz/ yasat-700.tar.gz/ yubikey-manager-qt-0.4.0.tar.gz/ zimg-2.5.1.tar.gz/ will become xr/ xv/ ya/ yu/ z/ Thanks for taking time to consider the suggestion. --- Andrew
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v4)
Hopefully the final version. --- Title: Portage rsync tree verification Author: Michał GórnyPosted: 2018-01-xx Revision: 1 News-Item-Format: 2.0 Display-If-Installed: sys-apps/portage Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo repository after rsync by default. The new verification is intended for users who are syncing via rsync. Verification mechanisms for other methods of sync will be provided in the future. This does not affect users syncing using git and other methods. Appropriate verification mechanisms for them will be provided in the future. The verification is implemented via app-portage/gemato. Currently, the whole repository is verified after syncing. On systems with slow hard drives, this could take around 2 minutes. If you wish to disable it, you can disable the 'rsync-verify' USE flag on sys-apps/portage or set 'sync-rsync-verify-metamanifest = no' in your repos.conf. Please note that the verification currently does not prevent Portage from using the repository after syncing. If 'emerge --sync' fails, do not install any packages and retry syncing. In case of prolonged or frequent verification failures, please make sure to report a bug including the failing mirror addresses (found in emerge.log). The verification uses information from the binary keyring provided by the app-crypt/gentoo-keys package. The keys are refreshed from the keyserver before every use in order to check for revocation. The post-sync verification ensures that the key package is verified itself. However, manual verification is required before the first use. On Gentoo installations created using installation media that included portage-2.3.22, the keys will already be covered by the installation media signatures. On existing installations, you need to manually compare the primary key fingerprint (reported by gemato on every sync) against the official Gentoo keys [1]. An example gemato output is: INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: 1234567890ABCDEF1234567890ABCDEF12345678 INFO:root:- subkey: FEDCBA0987654321FEDCBA0987654321FEDCBA09 Please note that the above snippet does not include the real key id on purpose. The primary key actually printed by gemato must match the 'Gentoo Portage Snapshot Signing Key' on the website. Please make sure to also check the certificate used for the secure connection to the site! [1]:https://www.gentoo.org/downloads/signatures/ -- Best regards, Michał Górny
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
On Sat, Jan 27, 2018 at 8:27 AM, Michał Górnywrote: > W dniu czw, 25.01.2018 o godzinie 15∶55 -0600, użytkownik R0b0t1 > napisał: >> On Thu, Jan 25, 2018 at 3:45 PM, Michał Górny wrote: >> > W dniu czw, 25.01.2018 o godzinie 21∶37 +, użytkownik Robin H. >> > Johnson napisał: >> > > On Thu, Jan 25, 2018 at 01:35:17PM +0100, Michał Górny wrote: >> > > > Title: Portage rsync tree verification >> > > > Author: Michał Górny >> > > > Posted: 2018-01-xx >> > > > Revision: 1 >> > > > News-Item-Format: 2.0 >> > > > Display-If-Installed: > > > >> > > Drop Display-If-Installed, they need to always see this until they know >> > > it was bootstrapped. >> > >> > Well, the idea was that if someone starts with stage that has >2.3.21, >> > then he has bootstrapped via verifying the stage signature. >> > >> > > > Starting with sys-apps/portage-2.3.22, Portage enables cryptographic >> > > > verification of the Gentoo rsync repository distributed over rsync >> > > > by default. >> > > >> > > Seems very wordy, suggested cleanup: >> > > > > Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo >> > > > > repository after rsync by default. >> > > > >> > > > This aims to prevent malicious third parties from altering >> > > > the contents of the ebuild repository received by our users. >> > > > >> > > > This does not affect users syncing using git and other methods. >> > > > Appropriate verification mechanisms for them will be provided >> > > > in the future. >> > > >> > > Note that emerge-webrsync has verification via FEATURES=webrsync-gpg? >> > >> > I'm sorry, I have never used that. Does it cover full key maintenance >> > or rely on user to do the gpg work? >> > >> >> It used to be necessary to set up a GnuPG home for portage and pull >> the keys in, but now users can emerge app-crypt/gentoo-keys and set >> PORTAGE_GPG_DIR="/var/lib/gentoo/gkeys/keyrings/gentoo/release". >> > > In that case I'd rather not announce it until it is integrated properly. > What is "properly?" It's referenced in the handbook. Cheers, R0b0t1
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v3)
Am Samstag, 27. Januar 2018, 15:26:44 CET schrieb Michał Górny: > [...] > > The new verification is intended for users who syncing via rsync. > Verification mechanisms for other methods of sync will be provided > in future. s/who syncing/who are syncing/ ("who sync via rsync" would sound a bit odd, but should work aswell.) -- GPG fingerprint: '766B 8122 1342 6912 3401 492A 8B54 D7A3 FF3C DB17' Holgersson signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v3)
On 27/01/18 14:26, Michał Górny wrote [excerpted]: > The verification is implemented via using app-portage/gemato. Currently, > the whole repository is verified after syncing. > I would drop either 'via' or 'using' - they both are the same verb/meaning and one is hence redundant. Just my 2c as a native English speaker :) MJE signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
W dniu czw, 25.01.2018 o godzinie 15∶55 -0600, użytkownik R0b0t1 napisał: > On Thu, Jan 25, 2018 at 3:45 PM, Michał Górnywrote: > > W dniu czw, 25.01.2018 o godzinie 21∶37 +, użytkownik Robin H. > > Johnson napisał: > > > On Thu, Jan 25, 2018 at 01:35:17PM +0100, Michał Górny wrote: > > > > Title: Portage rsync tree verification > > > > Author: Michał Górny > > > > Posted: 2018-01-xx > > > > Revision: 1 > > > > News-Item-Format: 2.0 > > > > Display-If-Installed: > > > > > Drop Display-If-Installed, they need to always see this until they know > > > it was bootstrapped. > > > > Well, the idea was that if someone starts with stage that has >2.3.21, > > then he has bootstrapped via verifying the stage signature. > > > > > > Starting with sys-apps/portage-2.3.22, Portage enables cryptographic > > > > verification of the Gentoo rsync repository distributed over rsync > > > > by default. > > > > > > Seems very wordy, suggested cleanup: > > > > > Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo > > > > > repository after rsync by default. > > > > > > > > This aims to prevent malicious third parties from altering > > > > the contents of the ebuild repository received by our users. > > > > > > > > This does not affect users syncing using git and other methods. > > > > Appropriate verification mechanisms for them will be provided > > > > in the future. > > > > > > Note that emerge-webrsync has verification via FEATURES=webrsync-gpg? > > > > I'm sorry, I have never used that. Does it cover full key maintenance > > or rely on user to do the gpg work? > > > > It used to be necessary to set up a GnuPG home for portage and pull > the keys in, but now users can emerge app-crypt/gentoo-keys and set > PORTAGE_GPG_DIR="/var/lib/gentoo/gkeys/keyrings/gentoo/release". > In that case I'd rather not announce it until it is integrated properly. -- Best regards, Michał Górny
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v3)
Next round: Title: Portage rsync tree verification Author: Michał GórnyPosted: 2018-01-xx Revision: 1 News-Item-Format: 2.0 Display-If-Installed: sys-apps/portage Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo repository after rsync by default. The new verification is intended for users who syncing via rsync. Verification mechanisms for other methods of sync will be provided in future. This does not affect users syncing using git and other methods. Appropriate verification mechanisms for them will be provided in the future. The verification is implemented via using app-portage/gemato. Currently, the whole repository is verified after syncing. On systems with slow hard drives, this could take around 2 minutes. If you wish to disable it, you can disable the 'rsync-verify' USE flag on sys-apps/portage or set 'sync-rsync-verify-metamanifest = no' in your repos.conf. Please note that the verification currently does not prevent Portage from using the repository after syncing. If 'emerge --sync' fails, do not install any packages and retry syncing. In case of prolonged or frequent verification failures, please make sure to report a bug including the failing mirror addresses (found in emerge.log). The verification uses information from the binary keyring provided by the app-crypt/gentoo-keys package. The keys are refreshed from the keyserver before every use in order to check for revocation. The post-sync verification ensures that the key package is verified itself. However, manual verification is required before the first use. On Gentoo installations created using installation media that included portage-2.3.22, the keys will already be covered by the installation media signatures. On existing installations, you need to manually compare the primary key fingerprint (reported by gemato on every sync) against the official Gentoo keys [1]. An example gemato output is: INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: 1234567890ABCDEF1234567890ABCDEF12345678 INFO:root:- subkey: FEDCBA0987654321FEDCBA0987654321FEDCBA09 Please note that the above snippet does not include the real key id on purpose. The primary key actually printed by gemato must match the 'Gentoo Portage Snapshot Signing Key' on the website. Please make sure to also check the certificate used for the secure connection to the site! [1]:https://www.gentoo.org/downloads/signatures/ -- Best regards, Michał Górny
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
On 26 January 2018 at 00:21, Robin H. Johnsonwrote: > On Thu, Jan 25, 2018 at 11:55:58PM +0200, Alon Bar-Lev wrote: >> I did not looked into the detailed implementation, however, please >> make sure integrity check handles the same cases we have applied to >> emerge-webrsync in the past, including: > Gemato is the implementation of GLEP74/MetaManifest, which DOES > explicitly address both of these concerns. Good! Thanks. > >> 1. Fast forward only in time, this is required to avoid hacker to >> redirect into older portage to install vulnerabilities that were >> approved at that time. > Replay attacks per #1 are addressed via TIMESTAMP field in MetaManifest. Interesting, I tried again to understand how it is working without performing rsync to a temporary directory, compare the timestamp and reject if unexpected. Are we doing multiple rsync for the metadata? Long since I used this insecure rsync... For me it seems like webrsync and/or squashfs are much easier/faster to apply integrity into than rsync... :) Regards, Alon
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
On Thu, Jan 25, 2018 at 11:55:58PM +0200, Alon Bar-Lev wrote: > I did not looked into the detailed implementation, however, please > make sure integrity check handles the same cases we have applied to > emerge-webrsync in the past, including: Gemato is the implementation of GLEP74/MetaManifest, which DOES explicitly address both of these concerns. > 1. Fast forward only in time, this is required to avoid hacker to > redirect into older portage to install vulnerabilities that were > approved at that time. Replay attacks per #1 are addressed via TIMESTAMP field in MetaManifest. > 2. Content integrity, especially removal, as far as I understand, the > mechanism will not enable to detect authorized removal of content. I think you meant 'unauthorized' rather than 'authorized' here. It will detect files that are expected to exist but are missing. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: Digital signature
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
Hi, On 25 January 2018 at 14:35, Michał Górnywrote: > > Starting with sys-apps/portage-2.3.22, Portage enables cryptographic > verification of the Gentoo rsync repository distributed over rsync > by default. This aims to prevent malicious third parties from altering > the contents of the ebuild repository received by our users. I did not looked into the detailed implementation, however, please make sure integrity check handles the same cases we have applied to emerge-webrsync in the past, including: 1. Fast forward only in time, this is required to avoid hacker to redirect into older portage to install vulnerabilities that were approved at that time. 2. Content integrity, especially removal, as far as I understand, the mechanism will not enable to detect authorized removal of content. Regards, Alon
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
On Thu, Jan 25, 2018 at 3:45 PM, Michał Górnywrote: > W dniu czw, 25.01.2018 o godzinie 21∶37 +, użytkownik Robin H. > Johnson napisał: >> On Thu, Jan 25, 2018 at 01:35:17PM +0100, Michał Górny wrote: >> > Title: Portage rsync tree verification >> > Author: Michał Górny >> > Posted: 2018-01-xx >> > Revision: 1 >> > News-Item-Format: 2.0 >> > Display-If-Installed: > >> Drop Display-If-Installed, they need to always see this until they know >> it was bootstrapped. > > Well, the idea was that if someone starts with stage that has >2.3.21, > then he has bootstrapped via verifying the stage signature. > >> > Starting with sys-apps/portage-2.3.22, Portage enables cryptographic >> > verification of the Gentoo rsync repository distributed over rsync >> > by default. >> >> Seems very wordy, suggested cleanup: >> > > Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo >> > > repository after rsync by default. >> > This aims to prevent malicious third parties from altering >> > the contents of the ebuild repository received by our users. >> > >> > This does not affect users syncing using git and other methods. >> > Appropriate verification mechanisms for them will be provided >> > in the future. >> >> Note that emerge-webrsync has verification via FEATURES=webrsync-gpg? > > I'm sorry, I have never used that. Does it cover full key maintenance > or rely on user to do the gpg work? > It used to be necessary to set up a GnuPG home for portage and pull the keys in, but now users can emerge app-crypt/gentoo-keys and set PORTAGE_GPG_DIR="/var/lib/gentoo/gkeys/keyrings/gentoo/release". >> >> Rewrite: >> > > The new verification is intended for users who syncing via rsync. >> > > Users who sync by emerge-webrsync should see [linkref]. >> > > Verification mechanisms for other methods of sync will be provided in >> > > future. >> >> >> > On Gentoo installations created using installation media that included >> > portage-2.3.22, the keys will already be covered by the installation >> > media signatures. On existing installations, you need to manually >> > compare the primary key fingerprint (reported by gemato on every sync) >> > against the official Gentoo keys [1]. An example gemato output is: >> > INFO:root:Valid OpenPGP signature found: >> > INFO:root:- primary key: 1234567890ABCDEF1234567890ABCDEF12345678 >> > INFO:root:- subkey: FEDCBA0987654321FEDCBA0987654321FEDCBA09 >> >> Either we should use real key here, or specifically note this is a fake >> key output on purpose. > > Well, I've assumed most people would be able to figure out that it would > be quite a coincidence to see such a key id. I wanted to avoid putting > the real id so that people would actually check that HTTPS site instead > of relying on the security of news item delivery. > > Will send an updated version tomorrow. > > -- > Best regards, > Michał Górny > >
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
W dniu czw, 25.01.2018 o godzinie 21∶37 +, użytkownik Robin H. Johnson napisał: > On Thu, Jan 25, 2018 at 01:35:17PM +0100, Michał Górny wrote: > > Title: Portage rsync tree verification > > Author: Michał Górny> > Posted: 2018-01-xx > > Revision: 1 > > News-Item-Format: 2.0 > > Display-If-Installed: > Drop Display-If-Installed, they need to always see this until they know > it was bootstrapped. Well, the idea was that if someone starts with stage that has >2.3.21, then he has bootstrapped via verifying the stage signature. > > Starting with sys-apps/portage-2.3.22, Portage enables cryptographic > > verification of the Gentoo rsync repository distributed over rsync > > by default. > > Seems very wordy, suggested cleanup: > > > Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo > > > repository after rsync by default. > > This aims to prevent malicious third parties from altering > > the contents of the ebuild repository received by our users. > > > > This does not affect users syncing using git and other methods. > > Appropriate verification mechanisms for them will be provided > > in the future. > > Note that emerge-webrsync has verification via FEATURES=webrsync-gpg? I'm sorry, I have never used that. Does it cover full key maintenance or rely on user to do the gpg work? > > Rewrite: > > > The new verification is intended for users who syncing via rsync. > > > Users who sync by emerge-webrsync should see [linkref]. > > > Verification mechanisms for other methods of sync will be provided in > > > future. > > > > On Gentoo installations created using installation media that included > > portage-2.3.22, the keys will already be covered by the installation > > media signatures. On existing installations, you need to manually > > compare the primary key fingerprint (reported by gemato on every sync) > > against the official Gentoo keys [1]. An example gemato output is: > > INFO:root:Valid OpenPGP signature found: > > INFO:root:- primary key: 1234567890ABCDEF1234567890ABCDEF12345678 > > INFO:root:- subkey: FEDCBA0987654321FEDCBA0987654321FEDCBA09 > > Either we should use real key here, or specifically note this is a fake > key output on purpose. Well, I've assumed most people would be able to figure out that it would be quite a coincidence to see such a key id. I wanted to avoid putting the real id so that people would actually check that HTTPS site instead of relying on the security of news item delivery. Will send an updated version tomorrow. -- Best regards, Michał Górny
Re: [gentoo-dev] [News item review] Portage rsync tree verification
On 25/01/18 11:01, Kristian Fiskerstrand wrote: > On 01/25/2018 11:04 AM, Michał Górny wrote: > >> The verification is implemented using app-portage/gemato. Currently, > ... "implemented in", as opposed to "using"? its implemented using > various cryptographic primitives, but gemato is the implementation > itself of sorts. > > "implemented with gemato" ? signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
On Thu, Jan 25, 2018 at 01:35:17PM +0100, Michał Górny wrote: > Title: Portage rsync tree verification > Author: Michał Górny> Posted: 2018-01-xx > Revision: 1 > News-Item-Format: 2.0 > Display-If-Installed: Starting with sys-apps/portage-2.3.22, Portage enables cryptographic > verification of the Gentoo rsync repository distributed over rsync > by default. Seems very wordy, suggested cleanup: || Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo || repository after rsync by default. > This aims to prevent malicious third parties from altering > the contents of the ebuild repository received by our users. > > This does not affect users syncing using git and other methods. > Appropriate verification mechanisms for them will be provided > in the future. Note that emerge-webrsync has verification via FEATURES=webrsync-gpg? Rewrite: || The new verification is intended for users who syncing via rsync. || Users who sync by emerge-webrsync should see [linkref]. || Verification mechanisms for other methods of sync will be provided in || future. > On Gentoo installations created using installation media that included > portage-2.3.22, the keys will already be covered by the installation > media signatures. On existing installations, you need to manually > compare the primary key fingerprint (reported by gemato on every sync) > against the official Gentoo keys [1]. An example gemato output is: > INFO:root:Valid OpenPGP signature found: > INFO:root:- primary key: 1234567890ABCDEF1234567890ABCDEF12345678 > INFO:root:- subkey: FEDCBA0987654321FEDCBA0987654321FEDCBA09 Either we should use real key here, or specifically note this is a fake key output on purpose. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
> On Thu, 25 Jan 2018, Michał Górny wrote: > Here's the updated version: > --- > Starting with sys-apps/portage-2.3.22, Portage enables cryptographic > verification of the Gentoo rsync repository distributed over rsync > by default. Looks like there's one "rsync" too much in that sentence. pgpCx6DDEeinP.pgp Description: PGP signature
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
On 2018-01-25 13:35, Michał Górny wrote: > Display-If-Installed: =2.3.22 this same information? I know we don’t have expires, yet. How about making it signature.asc Description: Digital signature
Re: [gentoo-dev] [News item review] Portage rsync tree verification (v2)
Here's the updated version: --- Title: Portage rsync tree verification Author: Michał GórnyPosted: 2018-01-xx Revision: 1 News-Item-Format: 2.0 Display-If-Installed: https://www.gentoo.org/downloads/signatures/ --- -- Best regards, Michał Górny
Re: [gentoo-dev] [News item review] Portage rsync tree verification
W dniu czw, 25.01.2018 o godzinie 12∶01 +0100, użytkownik Kristian Fiskerstrand napisał: > On 01/25/2018 11:04 AM, Michał Górny wrote: > > Hi, > > > > Thanks for your work on this! > > > This one would be committed once new sys-apps/portage release is > > wrapped up and hits ~arch. > > > > --- Title: Portage rsync tree verification Author: Michał Górny > >Posted: 2018-01-xx Revision: 1 News-Item-Format: > > 2.0 Display-If-Installed: > > > Starting with sys-apps/portage-2.3.22, Portage enables strong > > cryptographic verification of the Gentoo rsync tree by default. This > > aims to prevent malicious third parties from altering the contents of > > the ebuild repository received by our users. > > Just for sake of it, would remove "strong" here, as it is a description > and not PR document. Should we be consistent with referencing, so e.g > the Gentoo ebuild repository as distributed through rsync, or something? > Atm we seem to be using different terms all of the place, so should try > to harmonize a bit. Done. > > > > > The verification is implemented using app-portage/gemato. Currently, > > ... "implemented in", as opposed to "using"? its implemented using > various cryptographic primitives, but gemato is the implementation > itself of sorts. It was supposed to mean that Portage currently uses gemato to do the verification. 'via using' maybe? > > > the whole repository is verified after syncing. On systems with slow > > hard drives, this could take around 2 minutes. If you wish to > > disable it, you can disable the 'rsync-verify' flag on > > USE flag? Done. > > > sys-apps/portage or set 'sync-rsync-verify-metamanifest = no' in your > > repos.conf. > > > > Please note that the verification currently does not prevent Portage > > from using the repository after syncing. If 'emerge --sync' fails, do > > not install any packages and retry syncing. In case of prolonged or > > frequent verification failures, please make sure to report a bug > > including the failing mirror addresses (found in emerge.log). > > > > The verification uses keys provided by the app-crypt/gentoo-keys > > package. The keys are refreshed from the keyserver before every use > > in order to check for revocation. The post-sync verification ensures > > that the key package is verified itself. However, manua > > verification is required before the first use. > > Maybe some wording around binary keyring? e.g the verification uses > information from the binary keyring provided by app-crypt/gentoo-keys? > In particular the reference to "key package" might be misread (and the > keyring consists of multiple public keyblocks, that includes much more > information than the cryptographic keys per se) Done. > > > > > On new Gentoo installations including portage-2.3.22, the > > stage3s? Nah. I need to think how to word it properly. It's about installations that are created from new stages. > > > verification of the keys will be covered by verifying the > > installation media and repository snapshot signatures. On existing > > installations, you need to manually compare the primary key > > fingerprint (reported by gemato on every sync) against the official > > Gentoo keys [1]. An example gemato output is: > > > > INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: > > 1234567890ABCDEF1234567890ABCDEF12345678 INFO:root:- subkey: > > FEDCBA0987654321FEDCBA0987654321FEDCBA09 > > > > The primary key printed must match 'Gentoo Portage Snapshot Signing > > Key' on the site. Please make sure to also check the certificate > > used for the secure connection to the site! > > > > [1]:https://www.gentoo.org/downloads/signatures/ --- > > > > -- Best regards, Michał Górny
Re: [gentoo-dev] [News item review] Portage rsync tree verification
On 01/25/2018 11:04 AM, Michał Górny wrote: > Hi, > Thanks for your work on this! > This one would be committed once new sys-apps/portage release is > wrapped up and hits ~arch. > > --- Title: Portage rsync tree verification Author: Michał Górny >Posted: 2018-01-xx Revision: 1 News-Item-Format: > 2.0 Display-If-Installed: > Starting with sys-apps/portage-2.3.22, Portage enables strong > cryptographic verification of the Gentoo rsync tree by default. This > aims to prevent malicious third parties from altering the contents of > the ebuild repository received by our users. Just for sake of it, would remove "strong" here, as it is a description and not PR document. Should we be consistent with referencing, so e.g the Gentoo ebuild repository as distributed through rsync, or something? Atm we seem to be using different terms all of the place, so should try to harmonize a bit. > > The verification is implemented using app-portage/gemato. Currently, ... "implemented in", as opposed to "using"? its implemented using various cryptographic primitives, but gemato is the implementation itself of sorts. > the whole repository is verified after syncing. On systems with slow > hard drives, this could take around 2 minutes. If you wish to > disable it, you can disable the 'rsync-verify' flag on USE flag? > sys-apps/portage or set 'sync-rsync-verify-metamanifest = no' in your > repos.conf. > > Please note that the verification currently does not prevent Portage > from using the repository after syncing. If 'emerge --sync' fails, do > not install any packages and retry syncing. In case of prolonged or > frequent verification failures, please make sure to report a bug > including the failing mirror addresses (found in emerge.log). > > The verification uses keys provided by the app-crypt/gentoo-keys > package. The keys are refreshed from the keyserver before every use > in order to check for revocation. The post-sync verification ensures > that the key package is verified itself. However, manua > verification is required before the first use. Maybe some wording around binary keyring? e.g the verification uses information from the binary keyring provided by app-crypt/gentoo-keys? In particular the reference to "key package" might be misread (and the keyring consists of multiple public keyblocks, that includes much more information than the cryptographic keys per se) > > On new Gentoo installations including portage-2.3.22, the stage3s? > verification of the keys will be covered by verifying the > installation media and repository snapshot signatures. On existing > installations, you need to manually compare the primary key > fingerprint (reported by gemato on every sync) against the official > Gentoo keys [1]. An example gemato output is: > > INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: > 1234567890ABCDEF1234567890ABCDEF12345678 INFO:root:- subkey: > FEDCBA0987654321FEDCBA0987654321FEDCBA09 > > The primary key printed must match 'Gentoo Portage Snapshot Signing > Key' on the site. Please make sure to also check the certificate > used for the secure connection to the site! > > [1]:https://www.gentoo.org/downloads/signatures/ --- > -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature