Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, 3 Jul 2018, Matt Turner wrote: On Tue, Jul 3, 2018 at 11:38 AM Rich Freeman wrote: 4. by default git tends to accumulate history, which can eat up disk space. I imagine this could be automatically trimmed if users wanted, though during syncing it would at least need to store all the commits between the last fetched and next-fetched, and that means fetching things that might have been subsequently removed/changed This is why I have not switched to git. I have /usr/portage on a separate 1GB partition (with distfiles and packages stored elsewhere). The ebuild tree is 600MB with rsync and cannot fit on the partition with git. I'd be happy to switch if the space requirements were similar. Same here. One cannot avoid 3 things: death, taxes and insufficient hard-disk space. Andrey
Re: [gentoo-dev] rfc: killing mediawiki
On Tue, Jul 3, 2018 at 1:39 PM William Hubbs wrote: > > All, > > some of us have talked about this on IRC off and on, but I want to bring > it up here as well. > > I don't care that we have a wiki, but can we please look into killing > mediawiki and look at something with a git backend? It would be very > nice to be able to edit wiki pages in markdown or another similar format > and use git to control the changes instead of editing in a browser. I assume that your primary reason for wanting to replace mediawiki is to improve accessibility. I suggest you state that more clearly when making such a proposal. I read from jstein's email that he does not have the same knowledge of the situation that I have, and so his reply is expectedly different from mine.
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018 at 12:36 PM Rich Freeman wrote: > > On Tue, Jul 3, 2018 at 12:22 PM Matt Turner wrote: > > > > On Tue, Jul 3, 2018 at 11:38 AM Rich Freeman wrote: > > > 4. by default git tends to accumulate history, which can eat up disk > > > space. I imagine this could be automatically trimmed if users wanted, > > > though during syncing it would at least need to store all the commits > > > between the last fetched and next-fetched, and that means fetching > > > things that might have been subsequently removed/changed > > > > This is why I have not switched to git. I have /usr/portage on a > > separate 1GB partition (with distfiles and packages stored elsewhere). > > The ebuild tree is 600MB with rsync and cannot fit on the partition > > with git. > > > > git clone https://github.com/gentoo-mirror/gentoo.git . --depth 1 > ... > du -sh . > 662M. > > So, with a shallow clone it seems comparable. > > The issue is getting git to constantly trim, probably along the lines of: > https://stackoverflow.com/a/34829535 Exactly. I'm not sure git can automatically trim out history on git pull and I'm even less sure it would be able to do it without temporarily exceeding 1GB of storage.
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018 at 12:33 PM Matthias Maier wrote: > > > On Tue, Jul 3, 2018, at 11:22 CDT, Matt Turner wrote: > > > I'd be happy to switch if the space requirements were similar. > > $ git clone --depth=1 https://github.com/gentoo-mirror/gentoo > > occupies 662M on my machine (just tested). With full history > (i.e. without --depth=1) I am at 1.1GB. Wait a week and emerge --sync again; it won't fit.
Re: [gentoo-dev] Packages up for grabs: app-emulation/virtualbox-bin
On Tue, Jul 03, 2018 at 09:03:45PM +0200, Jonas Stein wrote: > Dear all, > > The following packages are up for grabs: > > app-emulation/virtualbox-bin > > > after retirement of the proxied maintainer. > > https://packages.gentoo.org/packages/app-emulation/virtualbox-bin > > This famous package has open bugs and > > RepoMan scours the neighborhood... > inherit.deprecated3 >app-emulation/virtualbox-bin/virtualbox-bin-5.1.36.122089.ebuild: > please migrate from 'versionator' to 'eapi7-ver (built-in since EAPI 7)' > on line: 8 >app-emulation/virtualbox-bin/virtualbox-bin-5.1.38.122592.ebuild: > please migrate from 'versionator' to 'eapi7-ver (built-in since EAPI 7)' > on line: 8 >app-emulation/virtualbox-bin/virtualbox-bin-5.2.12.122591.ebuild: > please migrate from 'versionator' to 'eapi7-ver (built-in since EAPI 7)' I'm half inclined to do this work prior to it getting nixed, that way if someone resurrects it it won't have this, at least. Fair? > on line: 8 > repo.eapi-deprecated 3 >app-emulation/virtualbox-bin/virtualbox-bin-5.1.36.122089.ebuild: 5 >app-emulation/virtualbox-bin/virtualbox-bin-5.1.38.122592.ebuild: 5 >app-emulation/virtualbox-bin/virtualbox-bin-5.2.12.122591.ebuild: 5 > > > -- > Best, > Jonas > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [gentoo-dev] rfc: killing mediawiki
On Tue, Jul 03, 2018 at 09:09:16PM +0100, M. J. Everitt wrote: > On 03/07/18 21:01, William Hubbs wrote: > > On Tue, Jul 03, 2018 at 09:20:53PM +0200, Jonas Stein wrote: > >>> I don't care that we have a wiki, but can we please look into killing > >>> mediawiki and look at something with a git backend? > >> I think the wiki is very useful and should remain. > > Like I said, there are wiki packages out there like gollum, ikiwiki, and > > probably others which would allow editing of content via text files and > > use vcs's for version control of the changes, so I'm not advocating for > > shutting down the wiki. I think we should have one that is more > > accessible to users who want to use different interfaces. We shouldn't > > be forcing users to use a full web browser just to contribute to the > > wiki. > > > >>> It would be very nice to be able to edit wiki pages in markdown or > >>> another similar format > >>> and use git to control the changes instead of editing in a browser. > >> I think it is more efficient to convert your yearly contributions to the > >> wiki [1] manually from markdown to mediawiki, instead to convert the > >> existent wiki pages to anything plus setup a new engine and configure > >> user accounts. > > If that is converted from markdown, all you would have to do is use the > > markdown directly if the new wiki supports it. > > > >> Btw: Would a conversion to another wiki mean that we get another long > >> footer on every wikipage "This page was edited by... do not remove..."? > > I have no idea about that, but that alone shouldn't stop this from > > happening. > > > >> For the special case of the Gentoo Manual: > >> I think the Gentoo Manual is better maintained in a git repository, > >> because it was initially written like a book and sometimes it is better > >> to make PRs for the manual. > > I don't really see the manual as a special case. We should use the same > > interface for everything. > > > > William > 1) I think this idea was floated before, and failed before .. That's not a reason for not floating it again. > 2) Existing wiki team are badly understaffed, how would this improve > things? How would new maintainers be registered and managed? It improves things by offering more flexable ways for users to edit the wiki. if you want to use a browser you can, or you can use something like git and edit the content that way. I don't know for sure how maintainers would be registered and managed, but I don't know that on mw either. > 3) Are you volunteering to implement this change yourself (infra are > equally understaffed) and manage the change and transition, in addition > to your existing commitments? I'm not on the infra team, so I would have to be added there to be able to do it I guess, but I would be willing to assist if I could. William signature.asc Description: Digital signature
Re: [gentoo-dev] rfc: killing mediawiki
On 03/07/18 21:01, William Hubbs wrote: > On Tue, Jul 03, 2018 at 09:20:53PM +0200, Jonas Stein wrote: >>> I don't care that we have a wiki, but can we please look into killing >>> mediawiki and look at something with a git backend? >> I think the wiki is very useful and should remain. > Like I said, there are wiki packages out there like gollum, ikiwiki, and > probably others which would allow editing of content via text files and > use vcs's for version control of the changes, so I'm not advocating for > shutting down the wiki. I think we should have one that is more > accessible to users who want to use different interfaces. We shouldn't > be forcing users to use a full web browser just to contribute to the > wiki. > >>> It would be very nice to be able to edit wiki pages in markdown or another >>> similar format >>> and use git to control the changes instead of editing in a browser. >> I think it is more efficient to convert your yearly contributions to the >> wiki [1] manually from markdown to mediawiki, instead to convert the >> existent wiki pages to anything plus setup a new engine and configure >> user accounts. > If that is converted from markdown, all you would have to do is use the > markdown directly if the new wiki supports it. > >> Btw: Would a conversion to another wiki mean that we get another long >> footer on every wikipage "This page was edited by... do not remove..."? > I have no idea about that, but that alone shouldn't stop this from > happening. > >> For the special case of the Gentoo Manual: >> I think the Gentoo Manual is better maintained in a git repository, >> because it was initially written like a book and sometimes it is better >> to make PRs for the manual. > I don't really see the manual as a special case. We should use the same > interface for everything. > > William 1) I think this idea was floated before, and failed before .. 2) Existing wiki team are badly understaffed, how would this improve things? How would new maintainers be registered and managed? 3) Are you volunteering to implement this change yourself (infra are equally understaffed) and manage the change and transition, in addition to your existing commitments? signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: killing mediawiki
On Tue, Jul 03, 2018 at 09:20:53PM +0200, Jonas Stein wrote: > > I don't care that we have a wiki, but can we please look into killing > > mediawiki and look at something with a git backend? > > I think the wiki is very useful and should remain. Like I said, there are wiki packages out there like gollum, ikiwiki, and probably others which would allow editing of content via text files and use vcs's for version control of the changes, so I'm not advocating for shutting down the wiki. I think we should have one that is more accessible to users who want to use different interfaces. We shouldn't be forcing users to use a full web browser just to contribute to the wiki. > > It would be very nice to be able to edit wiki pages in markdown or another > > similar format > > and use git to control the changes instead of editing in a browser. > > I think it is more efficient to convert your yearly contributions to the > wiki [1] manually from markdown to mediawiki, instead to convert the > existent wiki pages to anything plus setup a new engine and configure > user accounts. If that is converted from markdown, all you would have to do is use the markdown directly if the new wiki supports it. > > Btw: Would a conversion to another wiki mean that we get another long > footer on every wikipage "This page was edited by... do not remove..."? I have no idea about that, but that alone shouldn't stop this from happening. > For the special case of the Gentoo Manual: > I think the Gentoo Manual is better maintained in a git repository, > because it was initially written like a book and sometimes it is better > to make PRs for the manual. I don't really see the manual as a special case. We should use the same interface for everything. William signature.asc Description: Digital signature
Re: [gentoo-dev] [PATCH 0/4] GLEP 63: clean up, and reduce key size to RSA-2048
W dniu wto, 03.07.2018 o godzinie 12∶42 -0400, użytkownik Aaron Bauman napisał: > On Tuesday, July 3, 2018 12:40:57 PM EDT Aaron Bauman wrote: > > On Tuesday, July 3, 2018 9:29:53 AM EDT Michał Górny wrote: > > > Hi, everyone. > > > > > > Here's a series of patches for GLEP 63 (key policies). The first three > > > patches are merely editorial changes. The fourth is an actual > > > recommended policy change. > > > > > > The editorial changes are: > > > > > > 1. Using 'OpenPGP' instead of 'GPG' where appropriate. > > > > > > 2. Replacing 'RSAv4' with more correct term. > > > > > > 3. Clarifying the sentence on minimal key requirement to make it clear > > > > > >that dedicated signing subkey is also part of it. > > > > > > The policy change is changing the recommendation from RSA-4096 > > > to RSA-2048. This does not require developers to reroll their RSA-4096 > > > keys but aims to prevent people unnecessarily replacing RSA-2048 with > > > RSA-4096. > > > > > > The new recommendation matches what GnuPG FAQ suggests [1] (see 11.4, > > > 11.5). Long story short, RSA-4096 is only a little stronger than > > > RSA-2048 while it is much slower. If someone really wants to use it, > > > sure; but generally we shouldn't be encouraging people to use it. > > > > > > [1]:https://www.gnupg.org/faq/gnupg-faq.html#no_default_of_rsa4096 > > > > > > -- > > > Best regards, > > > Michał Górny > > > > > > Michał Górny (4): > > > glep-0063: Use 'OpenPGP' as appropriate > > > glep-0063: RSAv4 -> OpenPGP v4 key format > > > glep-0063: Clarify dedicated signing subkey in minimal reqs > > > glep-0063: Change the recommended RSA key size to 2048 bits > > > > > > glep-0063.rst | 44 > > > 1 file changed, 28 insertions(+), 16 deletions(-) > > > > Patches look good to me. I think now would be a good time to address other > > verbage too. e.g. recommendations should be requirements etc > > To clarify. I think this patchset it good as it is. I can create a new > patchset with recommendations for the things I mentioned above. Please do. I tried to keep this to stuff that's not likely to cause much of a bikeshed because I feel like stopping to tell people to do RSA-4096 is somewhat urgent, especially now that people are being asked to update their keys all over the place. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] rfc: killing mediawiki
> I don't care that we have a wiki, but can we please look into killing > mediawiki and look at something with a git backend? I think the wiki is very useful and should remain. > It would be very nice to be able to edit wiki pages in markdown or another > similar format > and use git to control the changes instead of editing in a browser. I think it is more efficient to convert your yearly contributions to the wiki [1] manually from markdown to mediawiki, instead to convert the existent wiki pages to anything plus setup a new engine and configure user accounts. Btw: Would a conversion to another wiki mean that we get another long footer on every wikipage "This page was edited by... do not remove..."? For the special case of the Gentoo Manual: I think the Gentoo Manual is better maintained in a git repository, because it was initially written like a book and sometimes it is better to make PRs for the manual. [1] https://wiki.gentoo.org/wiki/Special:Contributions/WilliamH -- Best, Jonas signature.asc Description: OpenPGP digital signature
[gentoo-dev] Packages up for grabs: app-emulation/virtualbox-bin
Dear all, The following packages are up for grabs: app-emulation/virtualbox-bin after retirement of the proxied maintainer. https://packages.gentoo.org/packages/app-emulation/virtualbox-bin This famous package has open bugs and RepoMan scours the neighborhood... inherit.deprecated3 app-emulation/virtualbox-bin/virtualbox-bin-5.1.36.122089.ebuild: please migrate from 'versionator' to 'eapi7-ver (built-in since EAPI 7)' on line: 8 app-emulation/virtualbox-bin/virtualbox-bin-5.1.38.122592.ebuild: please migrate from 'versionator' to 'eapi7-ver (built-in since EAPI 7)' on line: 8 app-emulation/virtualbox-bin/virtualbox-bin-5.2.12.122591.ebuild: please migrate from 'versionator' to 'eapi7-ver (built-in since EAPI 7)' on line: 8 repo.eapi-deprecated 3 app-emulation/virtualbox-bin/virtualbox-bin-5.1.36.122089.ebuild: 5 app-emulation/virtualbox-bin/virtualbox-bin-5.1.38.122592.ebuild: 5 app-emulation/virtualbox-bin/virtualbox-bin-5.2.12.122591.ebuild: 5 -- Best, Jonas signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: killing mediawiki
On Tue, Jul 03, 2018 at 01:47:19PM -0400, Brian Evans wrote: > On 7/3/2018 1:39 PM, William Hubbs wrote: > > All, > > > > some of us have talked about this on IRC off and on, but I want to bring > > it up here as well. > > > > I don't care that we have a wiki, but can we please look into killing > > mediawiki and look at something with a git backend? It would be very > > nice to be able to edit wiki pages in markdown or another similar format > > and use git to control the changes instead of editing in a browser. > > > > For what purpose? The Handbook? No objections to that as it is limited > access already. We just go back to what we were doing in CVS. We should definitely use git not cvs. :p We don't have to go back to what we were doing in cvs,. There are wikis out there such as gollum, gitit and ikiwiki to name a few, which allow full access to the content via vcs [1]. William [1] https://stackoverflow.com/questions/8255749/wikis-with-vcs-backends signature.asc Description: Digital signature
Re: [gentoo-dev] rfc: killing mediawiki
On 7/3/2018 1:39 PM, William Hubbs wrote: > All, > > some of us have talked about this on IRC off and on, but I want to bring > it up here as well. > > I don't care that we have a wiki, but can we please look into killing > mediawiki and look at something with a git backend? It would be very > nice to be able to edit wiki pages in markdown or another similar format > and use git to control the changes instead of editing in a browser. > For what purpose? The Handbook? No objections to that as it is limited access already. We just go back to what we were doing in CVS. If there is to be a replacement, then it should be equal access to what we have now. Users can create and edit pages which are not protected (currently by namespaces). This should continue IMO. Brian signature.asc Description: OpenPGP digital signature
[gentoo-dev] rfc: killing mediawiki
All, some of us have talked about this on IRC off and on, but I want to bring it up here as well. I don't care that we have a wiki, but can we please look into killing mediawiki and look at something with a git backend? It would be very nice to be able to edit wiki pages in markdown or another similar format and use git to control the changes instead of editing in a browser. Thanks, William signature.asc Description: Digital signature
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018 at 12:41 PM Kristian Fiskerstrand wrote: > > I would expect as much. But my primary argument would be key management > related, it is simply impossible to present a raw copy of our repo to > end-users and have them verify each commit > While related, I think that the question of distribution is still a fair one. We can still check an infra key on the head commit with git distribution. Granted, if we want to go further than that then the implementation will vary between git vs rsync distribution because the signed git metadata is only available easily in git. -- Rich
Re: [gentoo-dev] [PATCH 0/4] GLEP 63: clean up, and reduce key size to RSA-2048
On Tuesday, July 3, 2018 12:40:57 PM EDT Aaron Bauman wrote: > On Tuesday, July 3, 2018 9:29:53 AM EDT Michał Górny wrote: > > Hi, everyone. > > > > Here's a series of patches for GLEP 63 (key policies). The first three > > patches are merely editorial changes. The fourth is an actual > > recommended policy change. > > > > The editorial changes are: > > > > 1. Using 'OpenPGP' instead of 'GPG' where appropriate. > > > > 2. Replacing 'RSAv4' with more correct term. > > > > 3. Clarifying the sentence on minimal key requirement to make it clear > > > >that dedicated signing subkey is also part of it. > > > > The policy change is changing the recommendation from RSA-4096 > > to RSA-2048. This does not require developers to reroll their RSA-4096 > > keys but aims to prevent people unnecessarily replacing RSA-2048 with > > RSA-4096. > > > > The new recommendation matches what GnuPG FAQ suggests [1] (see 11.4, > > 11.5). Long story short, RSA-4096 is only a little stronger than > > RSA-2048 while it is much slower. If someone really wants to use it, > > sure; but generally we shouldn't be encouraging people to use it. > > > > [1]:https://www.gnupg.org/faq/gnupg-faq.html#no_default_of_rsa4096 > > > > -- > > Best regards, > > Michał Górny > > > > Michał Górny (4): > > glep-0063: Use 'OpenPGP' as appropriate > > glep-0063: RSAv4 -> OpenPGP v4 key format > > glep-0063: Clarify dedicated signing subkey in minimal reqs > > glep-0063: Change the recommended RSA key size to 2048 bits > > > > glep-0063.rst | 44 > > 1 file changed, 28 insertions(+), 16 deletions(-) > > Patches look good to me. I think now would be a good time to address other > verbage too. e.g. recommendations should be requirements etc To clarify. I think this patchset it good as it is. I can create a new patchset with recommendations for the things I mentioned above. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] [PATCH 0/4] GLEP 63: clean up, and reduce key size to RSA-2048
On Tuesday, July 3, 2018 9:29:53 AM EDT Michał Górny wrote: > Hi, everyone. > > Here's a series of patches for GLEP 63 (key policies). The first three > patches are merely editorial changes. The fourth is an actual > recommended policy change. > > The editorial changes are: > > 1. Using 'OpenPGP' instead of 'GPG' where appropriate. > > 2. Replacing 'RSAv4' with more correct term. > > 3. Clarifying the sentence on minimal key requirement to make it clear >that dedicated signing subkey is also part of it. > > The policy change is changing the recommendation from RSA-4096 > to RSA-2048. This does not require developers to reroll their RSA-4096 > keys but aims to prevent people unnecessarily replacing RSA-2048 with > RSA-4096. > > The new recommendation matches what GnuPG FAQ suggests [1] (see 11.4, > 11.5). Long story short, RSA-4096 is only a little stronger than > RSA-2048 while it is much slower. If someone really wants to use it, > sure; but generally we shouldn't be encouraging people to use it. > > [1]:https://www.gnupg.org/faq/gnupg-faq.html#no_default_of_rsa4096 > > -- > Best regards, > Michał Górny > > Michał Górny (4): > glep-0063: Use 'OpenPGP' as appropriate > glep-0063: RSAv4 -> OpenPGP v4 key format > glep-0063: Clarify dedicated signing subkey in minimal reqs > glep-0063: Change the recommended RSA key size to 2048 bits > > glep-0063.rst | 44 > 1 file changed, 28 insertions(+), 16 deletions(-) Patches look good to me. I think now would be a good time to address other verbage too. e.g. recommendations should be requirements etc signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018 at 12:34 PM William Hubbs wrote: > > On Tue, Jul 03, 2018 at 11:40:53AM -0400, Rich Freeman wrote: > > On Tue, Jul 3, 2018 at 11:32 AM Brian Dolbec wrote: > > > 2) we have a large infrastructure of rsync mirrors, which we do not for > > > git. > > > > > > > Do we need them. I've yet to see somebody complain about poor syncing > > performance from github. I imagine we could just use that and a few > > other free mirroring services to distribute the tree. > > I don't feel comfortable relying on github as a primary means of > distributing the tree due to our social contract. It is a value-added > kind of service, but we should not rely on it. > Do you know that all our existing mirrors are 100% FOSS? It is a mirror. You upload something. Somebody else downloads the same thing. If we were distributing tarballs via http would we really care if the mirror is running apache vs IIS? Do we even check our existing mirrors for such things? Do we care that they're running on coreboot too, without an IME? Hey, I'm all for having all the mirrors we can, and it isn't like mirroring git is particularly difficult. I just think that there is a double-standard being applied when it comes to get. I completely get the argument when it comes to things like issues/PRs/etc since those aren't distributed, but for git itself you really just need something that supports the protocol and it is trivial to replace. Certainly for anything we host we should use FOSS because it is the cleanest solution anyway. -- Rich
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
I would expect as much. But my primary argument would be key management related, it is simply impossible to present a raw copy of our repo to end-users and have them verify each commit Original message From: William Hubbs Date: 7/3/18 17:39 (GMT+01:00) To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync? On Tue, Jul 03, 2018 at 08:32:55AM -0700, Brian Dolbec wrote: > On Tue, 3 Jul 2018 10:22:35 -0500 > William Hubbs wrote: > > > All, > > > > Mostly because of the recent "trustless infrastructure" thread, I am > > wondering why we are still distributing the portage tree primarily > > via rsync instead of git? > > > > Can someone educate me on that, and is it worth considering moving > > away from rsync distribution? > > > > Thanks, > > > > William > > > > because: > > 1) it is still the most bandwidth economical means of distributing the > tree Even more so than http or https? Thanks, William
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018 at 12:22 PM Matt Turner wrote: > > On Tue, Jul 3, 2018 at 11:38 AM Rich Freeman wrote: > > 4. by default git tends to accumulate history, which can eat up disk > > space. I imagine this could be automatically trimmed if users wanted, > > though during syncing it would at least need to store all the commits > > between the last fetched and next-fetched, and that means fetching > > things that might have been subsequently removed/changed > > This is why I have not switched to git. I have /usr/portage on a > separate 1GB partition (with distfiles and packages stored elsewhere). > The ebuild tree is 600MB with rsync and cannot fit on the partition > with git. > git clone https://github.com/gentoo-mirror/gentoo.git . --depth 1 ... du -sh . 662M. So, with a shallow clone it seems comparable. The issue is getting git to constantly trim, probably along the lines of: https://stackoverflow.com/a/34829535 -- Rich
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 03, 2018 at 11:40:53AM -0400, Rich Freeman wrote: > On Tue, Jul 3, 2018 at 11:32 AM Brian Dolbec wrote: > > 2) we have a large infrastructure of rsync mirrors, which we do not for > > git. > > > > Do we need them. I've yet to see somebody complain about poor syncing > performance from github. I imagine we could just use that and a few > other free mirroring services to distribute the tree. I don't feel comfortable relying on github as a primary means of distributing the tree due to our social contract. It is a value-added kind of service, but we should not rely on it. William signature.asc Description: Digital signature
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018, at 11:22 CDT, Matt Turner wrote: > I'd be happy to switch if the space requirements were similar. $ git clone --depth=1 https://github.com/gentoo-mirror/gentoo occupies 662M on my machine (just tested). With full history (i.e. without --depth=1) I am at 1.1GB. Best, Matthias signature.asc Description: PGP signature
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018 at 11:38 AM Rich Freeman wrote: > 4. by default git tends to accumulate history, which can eat up disk > space. I imagine this could be automatically trimmed if users wanted, > though during syncing it would at least need to store all the commits > between the last fetched and next-fetched, and that means fetching > things that might have been subsequently removed/changed This is why I have not switched to git. I have /usr/portage on a separate 1GB partition (with distfiles and packages stored elsewhere). The ebuild tree is 600MB with rsync and cannot fit on the partition with git. I'd be happy to switch if the space requirements were similar.
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018 at 11:32 AM Brian Dolbec wrote: > > 1) it is still the most bandwidth economical means of distributing the > tree Is this true? If I do two syncs 10min apart, I have to imagine that less data will get transferred for git. Certianly there will be less disk IO. I think the main issue is when does the crossover happen because if I sync a year apart git is going to send every file that was ever added and then removed from the tree in that time. Also, do we care about bandwidth when there are mirrors that offer it for free? > 2) we have a large infrastructure of rsync mirrors, which we do not for > git. > Do we need them. I've yet to see somebody complain about poor syncing performance from github. I imagine we could just use that and a few other free mirroring services to distribute the tree. While I appreciate all the donors giving us mirrors/etc, it seems like we would be much more resilient if we didn't require them in the first place. -- Rich
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 03, 2018 at 08:32:55AM -0700, Brian Dolbec wrote: > On Tue, 3 Jul 2018 10:22:35 -0500 > William Hubbs wrote: > > > All, > > > > Mostly because of the recent "trustless infrastructure" thread, I am > > wondering why we are still distributing the portage tree primarily > > via rsync instead of git? > > > > Can someone educate me on that, and is it worth considering moving > > away from rsync distribution? > > > > Thanks, > > > > William > > > > because: > > 1) it is still the most bandwidth economical means of distributing the > tree Even more so than http or https? Thanks, William signature.asc Description: Digital signature
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, Jul 3, 2018 at 11:22 AM William Hubbs wrote: > > Mostly because of the recent "trustless infrastructure" thread, I am > wondering why we are still distributing the portage tree primarily > via rsync instead of git? > > Can someone educate me on that, and is it worth considering moving away > from rsync distribution? > Here are the pros/cons that I've seen come up in the past: 1. emerge-webrsync is probably more secure at the moment, because emerge --sync with git leaves the tree corrupt if it doesn't verify. That seems like something that could be fixed, and which should be fixed regardless (presumably somebody just has to do the work - I can't imagine the portage team would turn away patches). 2. git seems to be more efficient for frequent syncing, while rsync seems to be more efficient for infrequest syncing. I'd guess the crossover is somewhere around a week or few, but I don't have data to support that. 3. we have more rsync mirrors, though with the possibility of using mirrors like github I don't see why this matters (as long as we actually secure distribution). 4. by default git tends to accumulate history, which can eat up disk space. I imagine this could be automatically trimmed if users wanted, though during syncing it would at least need to store all the commits between the last fetched and next-fetched, and that means fetching things that might have been subsequently removed/changed Personally I stick with git. I want the history anyway, and since I sync frequently it involves WAY less disk IO and seems to be very network-efficient as well. -- Rich
Re: [gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
On Tue, 3 Jul 2018 10:22:35 -0500 William Hubbs wrote: > All, > > Mostly because of the recent "trustless infrastructure" thread, I am > wondering why we are still distributing the portage tree primarily > via rsync instead of git? > > Can someone educate me on that, and is it worth considering moving > away from rsync distribution? > > Thanks, > > William > because: 1) it is still the most bandwidth economical means of distributing the tree 2) we have a large infrastructure of rsync mirrors, which we do not for git. 3) see #1 -- Brian Dolbec pgpjViOjx5GaR.pgp Description: OpenPGP digital signature
[gentoo-dev] rfc: why are we still distributing the portage tree via rsync?
All, Mostly because of the recent "trustless infrastructure" thread, I am wondering why we are still distributing the portage tree primarily via rsync instead of git? Can someone educate me on that, and is it worth considering moving away from rsync distribution? Thanks, William signature.asc Description: Digital signature
[gentoo-dev] [PATCH 4/4] glep-0063: Change the recommended RSA key size to 2048 bits
Change the recommended key size recommendation for RSA from 4096 bits to 2048 bits. Use of larger keys is unjustified due to negligible gain in security, and recommending RSA-4096 unnecessarily resulted in developers replacing their RSA-2048 keys for no good reason. --- glep-0063.rst | 18 +++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index 0082edd..f1512b3 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -6,7 +6,7 @@ Author: Robin H. Johnson , Marissa Fischer Type: Standards Track Status: Final -Version: 1 +Version: 1.1 Created: 2013-02-18 Last-Modified: 2018-07-02 Post-History: 2013-11-10 @@ -24,6 +24,15 @@ Abstract This GLEP provides both a minimum requirement and a recommended set of OpenPGP key management policies for the Gentoo Linux distribution. +Changes +=== + +v1.1 + The recommended RSA key size has been changed from 4096 bits + to 2048 bits to match the GnuPG recommendations [#GNUPG-FAQ-11-4]_. + The larger recommendation was unjustified and resulted in people + unnecessarily replacing their RSA-2048 keys. + Motivation == @@ -101,7 +110,7 @@ Recommendations # when making an OpenPGP certification, use a stronger digest than the default SHA1: cert-digest-algo SHA256 -2. Root key type RSA, 4096 bits (OpenPGP v4 key format or later) +2. Root key type RSA, 2048 bits (OpenPGP v4 key format or later) This may require creating an entirely new key. @@ -109,7 +118,7 @@ Recommendations a. DSA 2048 bits exactly. - b. RSA 4096 bits exactly. + b. RSA 2048 bits exactly. 4. Key expiry: @@ -162,6 +171,9 @@ Much of the above was driven by the following: References == +.. [#GNUPG-FAQ-11-4] GnuPG FAQ: Why doesn’t GnuPG default to using RSA-4096? + (https://www.gnupg.org/faq/gnupg-faq.html#no_default_of_rsa4096) + .. [#DEBIANGPG] Debian GPG documentation (https://wiki.debian.org/Keysigning) -- 2.18.0
[gentoo-dev] [PATCH 3/4] glep-0063: Clarify dedicated signing subkey in minimal reqs
Reword the minimal requirements to clearly indicate that a dedicated signing subkey is required. The current wording may make it unclear whether the 'root key' and 'signing subkey' can be the same key. --- glep-0063.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/glep-0063.rst b/glep-0063.rst index 8e4f0d5..0082edd 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -45,7 +45,7 @@ Bare minimum requirements personal-digest-preferences SHA256 -2. Root key and signing subkey of EITHER: +2. Root key and a dedicated signing subkey, both of EITHER: a. DSA, 2048-bit -- 2.18.0
[gentoo-dev] [PATCH 2/4] glep-0063: RSAv4 -> OpenPGP v4 key format
Replace the 'RSAv4' with 'OpenPGP v4 key format'. The RSA algorithm does not really have versions, and the author most likely meant the v4 of OpenPGP key format as outlined in RFC 4880, section 12.1. This was figured out and explained to me by Kristian Fiskerstrand. --- glep-0063.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index dd61ecc..8e4f0d5 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -49,7 +49,7 @@ Bare minimum requirements a. DSA, 2048-bit - b. RSA, >=2048 bits, RSAv4 or later only + b. RSA, >=2048 bits (OpenPGP v4 key format or later only) 3. Key expiry: 5 years maximum @@ -101,7 +101,7 @@ Recommendations # when making an OpenPGP certification, use a stronger digest than the default SHA1: cert-digest-algo SHA256 -2. Root key type RSA, 4096 bits, RSAv4 or later +2. Root key type RSA, 4096 bits (OpenPGP v4 key format or later) This may require creating an entirely new key. -- 2.18.0
[gentoo-dev] [PATCH 1/4] glep-0063: Use 'OpenPGP' as appropriate
Replace many of the incorrect uses of GPG/GnuPG [key] with OpenPGP. G[nu]PG has been left where the text clearly refers to the specific implementation of OpenPGP rather than the standard itself. --- glep-0063.rst | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/glep-0063.rst b/glep-0063.rst index c59d545..dd61ecc 100644 --- a/glep-0063.rst +++ b/glep-0063.rst @@ -1,6 +1,6 @@ --- GLEP: 63 -Title: Gentoo GPG key policies +Title: Gentoo OpenPGP policies Author: Robin H. Johnson , Andreas K. Hüttel , Marissa Fischer @@ -8,7 +8,7 @@ Type: Standards Track Status: Final Version: 1 Created: 2013-02-18 -Last-Modified: 2015-08-25 +Last-Modified: 2018-07-02 Post-History: 2013-11-10 Content-Type: text/x-rst --- @@ -21,22 +21,22 @@ Many developers and external sources helped in this GLEP. Abstract -This GLEP provides both a minimum requirement and a recommended set of GPG -key management policies for the Gentoo Linux distribution. +This GLEP provides both a minimum requirement and a recommended set of +OpenPGP key management policies for the Gentoo Linux distribution. Motivation == Given the increasing use and importance of cryptographic protocols in internet -transactions of any kind, unified requirements for GnuPG keys used in Gentoo +transactions of any kind, unified requirements for OpenPGP keys used in Gentoo Linux development are sorely needed. This document provides both a set of bare minimum requirements and a set of best practice recommendations for -the use of GnuPG by Gentoo Linux developers. It is intended to provide -a basis for future improvements such as, e.g., consistent ebuild or package -signing and verifying by end users. +the use of GnuPG (or other OpenPGP providers) by Gentoo Linux developers. +It is intended to provide a basis for future improvements such as, e.g., +consistent ebuild or package signing and verifying by end users. -Specifications for GnuPG keys -= +Specifications for OpenPGP keys +=== Bare minimum requirements - @@ -125,7 +125,7 @@ Recommendations Gentoo LDAP === -All Gentoo developers must list the complete GPG fingerprint for their root +All Gentoo developers must list the complete fingerprint for their root keys in the "``gpgfingerprint``" LDAP field. It must be exactly 40 hex digits, uppercase, with optional spaces every 8 hex digits. Regular expression for validation:: -- 2.18.0
[gentoo-dev] [PATCH 0/4] GLEP 63: clean up, and reduce key size to RSA-2048
Hi, everyone. Here's a series of patches for GLEP 63 (key policies). The first three patches are merely editorial changes. The fourth is an actual recommended policy change. The editorial changes are: 1. Using 'OpenPGP' instead of 'GPG' where appropriate. 2. Replacing 'RSAv4' with more correct term. 3. Clarifying the sentence on minimal key requirement to make it clear that dedicated signing subkey is also part of it. The policy change is changing the recommendation from RSA-4096 to RSA-2048. This does not require developers to reroll their RSA-4096 keys but aims to prevent people unnecessarily replacing RSA-2048 with RSA-4096. The new recommendation matches what GnuPG FAQ suggests [1] (see 11.4, 11.5). Long story short, RSA-4096 is only a little stronger than RSA-2048 while it is much slower. If someone really wants to use it, sure; but generally we shouldn't be encouraging people to use it. [1]:https://www.gnupg.org/faq/gnupg-faq.html#no_default_of_rsa4096 -- Best regards, Michał Górny Michał Górny (4): glep-0063: Use 'OpenPGP' as appropriate glep-0063: RSAv4 -> OpenPGP v4 key format glep-0063: Clarify dedicated signing subkey in minimal reqs glep-0063: Change the recommended RSA key size to 2048 bits glep-0063.rst | 44 1 file changed, 28 insertions(+), 16 deletions(-) -- 2.18.0