Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On Sun, Jan 20, 2008 at 07:56:12AM -0500, Thomas Anderson wrote: On Thursday 17 January 2008 16:47:28 Robin H. Johnson wrote: Hi folks, Infra is working on a bunch of things lately, and there are going to be changes or brief outages for the following services (this is pretty much the order they are being worked on). anonvcs.gentoo.org: anoncvs, anonsvn, anongit - Moving between machines - Anonymous SVN is changing from http:// to svn:// [1] Did this plan include disabling of compression for anoncvs? I noticed my compression-enabled cvs up's were spewing out information about gzip-file-contents not being supported. This only started happening within the past few days, so it probably happened with this switch(assuming the switch happened already ;) ). Compression was disabled on the old side as well. Now that we have more CPU however, I may be able to re-enable it. -- Robin Hugh Johnson Gentoo Linux Developer Infra Guy E-Mail : [EMAIL PROTECTED] GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 pgppeN3Ka6Ctn.pgp Description: PGP signature
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On Thursday 17 January 2008 16:47:28 Robin H. Johnson wrote: Hi folks, Infra is working on a bunch of things lately, and there are going to be changes or brief outages for the following services (this is pretty much the order they are being worked on). anonvcs.gentoo.org: anoncvs, anonsvn, anongit - Moving between machines - Anonymous SVN is changing from http:// to svn:// [1] Did this plan include disabling of compression for anoncvs? I noticed my compression-enabled cvs up's were spewing out information about gzip-file-contents not being supported. This only started happening within the past few days, so it probably happened with this switch(assuming the switch happened already ;) ). Regards, Thomas -- 2.6.23-gentoo-r3 signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On 1/19/08, Mike Frysinger [EMAIL PROTECTED] wrote: using https:// to secure your data here is the wrong way to go. if you have a man-in-the-middle attacking you, they can do a lot more than inject crap into your syncs, some of which you wouldnt even notice. for the topic at hand, this topic does not matter i think. The https solves man-in the middle for svn/git sync. There is an option for rsync people (not to use it): http://bugs.gentoo.org/show_bug.cgi?id=130039 Best Regards, Alon Bar-Lev. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On Friday 18 January 2008, Robin H. Johnson wrote: On Sat, Jan 19, 2008 at 12:26:44AM +0200, Alon Bar-Lev wrote: On 1/18/08, Mike Frysinger [EMAIL PROTECTED] wrote: On Thursday 17 January 2008, Robin H. Johnson wrote: anonvcs.gentoo.org: anoncvs, anonsvn, anongit - Anonymous SVN is changing from http:// to svn:// [1] overlays.gentoo.org [3]: - Anonymous SVN is changing from http:// to svn:// i'd point out that http:// syncing is usable from behind firewalls while svn:// is not ... while this does not affect me personally, it's something to keep in mind. -mike Just wanted to note this too... I am one of the affected ones... I think that it is very important to have http, and even https for formal resources. git://, svn://, rsync:// or ssh+X:// are inaccessible for a large group of users. My core concern with the SVN http://, was the crappy performance it provided compared to svn://. The main rsync tree has never been available for iterative syncing via http://, just had tarball snapshots and deltas instead. i'm not suggesting you *not* provide the proper svn:// and git:// ones. i'd always use those myself when possible (as performance is a ton better as ive seen many times). i'm suggesting we provide both and tell people to use svn:// and git://, but if you're behind a stupid firewall, there is also http:// available. Also using none secured protocols, exposes users to man-in-the-middle attacks. The existing http:// had this problem already, it's not a new one. git:// and svn:// do both have patches around adding support for adding TLS. This however just adds overhead, I really need to finish the tree-signing work I was doing, as that protects the content better (MITM is still possible on SSL without it, just a lot harder as an attacker has to deal with the SSL stream first). using https:// to secure your data here is the wrong way to go. if you have a man-in-the-middle attacking you, they can do a lot more than inject crap into your syncs, some of which you wouldnt even notice. for the topic at hand, this topic does not matter i think. -mike signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On 19-01-2008 15:50:09 -0500, Mike Frysinger wrote: i'm not suggesting you *not* provide the proper svn:// and git:// ones. i'd always use those myself when possible (as performance is a ton better as ive seen many times). i'm suggesting we provide both and tell people to use svn:// and git://, but if you're behind a stupid firewall, there is also http:// available. I know of at least two cases where people have to go through a (corporate) firewall, so I fully second this suggestion. -- Fabian Groffen Gentoo on a different level -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
First and foremost: thanks for the work! On 17-01-2008 13:47:28 -0800, Robin H. Johnson wrote: overlays.gentoo.org [3]: - Moving between machines - Git service is already on the new machine - Anonymous SVN is changing from http:// to svn:// - Trac being replaced [2] Footnotes: 1. You do not need to do a new checkout at all, you can use the following command to update your SVN repos: svn switch --relocate \ http://anonsvn.gentoo.org/repositories/$REPO \ svn://anonsvn.gentoo.org/$REPO Would it be possible to have a transition period of 1 or 2 months for the svn repo move? The main problem for me is that Prefix users are on a Portage tree from overlays (in SVN), which means if this switch is done without transition they cannot update (emerge --sync) any more. While this will result in some mail/bug activity, it might also leave other people in the dark, ending up (needlessly) rebootstrapping. If a transition period would be available, I could make Portage alarm users to fix the SYNC variable to reflect the new URL. I also need to add/fix/change Portage's support for this new URL scheme. -- Fabian Groffen Gentoo on a different level -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On Fri, Jan 18, 2008 at 09:37:35AM +0100, Fabian Groffen wrote: First and foremost: thanks for the work! On 17-01-2008 13:47:28 -0800, Robin H. Johnson wrote: overlays.gentoo.org [3]: - Moving between machines - Git service is already on the new machine - Anonymous SVN is changing from http:// to svn:// - Trac being replaced [2] Footnotes: 1. You do not need to do a new checkout at all, you can use the following command to update your SVN repos: svn switch --relocate \ http://anonsvn.gentoo.org/repositories/$REPO \ svn://anonsvn.gentoo.org/$REPO Would it be possible to have a transition period of 1 or 2 months for the svn repo move? The main problem for me is that Prefix users are on a Portage tree from overlays (in SVN), which means if this switch is done without transition they cannot update (emerge --sync) any more. While this will result in some mail/bug activity, it might also leave other people in the dark, ending up (needlessly) rebootstrapping. If a transition period would be available, I could make Portage alarm users to fix the SYNC variable to reflect the new URL. I also need to add/fix/change Portage's support for this new URL scheme. You're syncing directly from overlays SVN-HTTP? I hope not with the sync frequency of some of the other users out there. As you support SVN already, you should just need to change the URL (since you are just handing it to SVN), and run svn switch. But ok, I'll give you 30 days for /repositories/alt/ once the new overlays box has SVN. (Combined with a suitably large warning in the SVN browse view). As a migration help, I have enabled the svn:// protocol on the old overlays box, so you can make a start on getting your users converted. svn://overlays.gentoo.org/proj/alt/ -- Robin Hugh Johnson Gentoo Linux Developer Infra Guy E-Mail : [EMAIL PROTECTED] GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 pgpZ8H3pkWYRn.pgp Description: PGP signature
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On 18-01-2008 01:21:21 -0800, Robin H. Johnson wrote: If a transition period would be available, I could make Portage alarm users to fix the SYNC variable to reflect the new URL. I also need to add/fix/change Portage's support for this new URL scheme. You're syncing directly from overlays SVN-HTTP? I hope not with the sync frequency of some of the other users out there. I have no other option, do I? I requested rsync in some bug a while ago with one of the reasons to reduce overlays' load. As you support SVN already, you should just need to change the URL (since you are just handing it to SVN), and run svn switch. But ok, I'll give you 30 days for /repositories/alt/ once the new overlays box has SVN. (Combined with a suitably large warning in the SVN browse view). The problem is that emerge --sync obfuscates that SVN is being used underneath. Some (most?) users will not have a clue they are using SVN. As a migration help, I have enabled the svn:// protocol on the old overlays box, so you can make a start on getting your users converted. svn://overlays.gentoo.org/proj/alt/ Thanks, I'll priorise on that to get it rolling. Thanks a lot! -- Fabian Groffen Gentoo on a different level -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On 18-01-2008 03:32:36 -0800, Robin H. Johnson wrote: The problem is that emerge --sync obfuscates that SVN is being used underneath. Some (most?) users will not have a clue they are using SVN. How about rolling out a prefix-portage update that just kicks them into updating it? That's exactly what I asked the transition period for. I'll do this asap. I only need the little period for people to catch up and do it. As long as the majority switches, it's all fine. Since you enabled svn:// on the old box, I can already roll a version out this weekend, so people are told to switch. At the same time I can update the bootstrap images/snapshot and scripts to use the new scheme, and install a prefix-portage that uses/supports the new scheme. Thanks. -- Fabian Groffen Gentoo on a different level -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On Thursday 17 January 2008, Robin H. Johnson wrote: anonvcs.gentoo.org: anoncvs, anonsvn, anongit - Anonymous SVN is changing from http:// to svn:// [1] overlays.gentoo.org [3]: - Anonymous SVN is changing from http:// to svn:// i'd point out that http:// syncing is usable from behind firewalls while svn:// is not ... while this does not affect me personally, it's something to keep in mind. -mike signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On Fri, Jan 18, 2008 at 10:46:28AM +0100, Fabian Groffen wrote: On 18-01-2008 01:21:21 -0800, Robin H. Johnson wrote: If a transition period would be available, I could make Portage alarm users to fix the SYNC variable to reflect the new URL. I also need to add/fix/change Portage's support for this new URL scheme. You're syncing directly from overlays SVN-HTTP? I hope not with the sync frequency of some of the other users out there. I have no other option, do I? I requested rsync in some bug a while ago with one of the reasons to reduce overlays' load. Not really doable at the moment (but after some of the other pending infra stuff, it is up for handling). As you support SVN already, you should just need to change the URL (since you are just handing it to SVN), and run svn switch. But ok, I'll give you 30 days for /repositories/alt/ once the new overlays box has SVN. (Combined with a suitably large warning in the SVN browse view). The problem is that emerge --sync obfuscates that SVN is being used underneath. Some (most?) users will not have a clue they are using SVN. How about rolling out a prefix-portage update that just kicks them into updating it? -- Robin Hugh Johnson Gentoo Linux Developer Infra Guy E-Mail : [EMAIL PROTECTED] GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 pgpjSQm3mUmQY.pgp Description: PGP signature
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On 1/19/08, Arfrever Frehtes Taifersar Arahesis [EMAIL PROTECTED] wrote: If I understand correctly, the performance of svn under apache is better than the svnserver The other way round. We are talking about read-only anonymous repository, right? But I will take your word for it :) Thanks! Alon. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On Sat, Jan 19, 2008 at 12:26:44AM +0200, Alon Bar-Lev wrote: On 1/18/08, Mike Frysinger [EMAIL PROTECTED] wrote: On Thursday 17 January 2008, Robin H. Johnson wrote: anonvcs.gentoo.org: anoncvs, anonsvn, anongit - Anonymous SVN is changing from http:// to svn:// [1] overlays.gentoo.org [3]: - Anonymous SVN is changing from http:// to svn:// i'd point out that http:// syncing is usable from behind firewalls while svn:// is not ... while this does not affect me personally, it's something to keep in mind. -mike Just wanted to note this too... I am one of the affected ones... I think that it is very important to have http, and even https for formal resources. git://, svn://, rsync:// or ssh+X:// are inaccessible for a large group of users. My core concern with the SVN http://, was the crappy performance it provided compared to svn://. The main rsync tree has never been available for iterative syncing via http://, just had tarball snapshots and deltas instead. Also using none secured protocols, exposes users to man-in-the-middle attacks. The existing http:// had this problem already, it's not a new one. git:// and svn:// do both have patches around adding support for adding TLS. This however just adds overhead, I really need to finish the tree-signing work I was doing, as that protects the content better (MITM is still possible on SSL without it, just a lot harder as an attacker has to deal with the SSL stream first). -- Robin Hugh Johnson Gentoo Linux Developer Infra Guy E-Mail : [EMAIL PROTECTED] GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 pgp8vJHeAJFgp.pgp Description: PGP signature
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On 1/19/08, Robin H. Johnson [EMAIL PROTECTED] wrote: My core concern with the SVN http://, was the crappy performance it provided compared to svn://. The main rsync tree has never been available for iterative syncing via http://, just had tarball snapshots and deltas instead. If I understand correctly, the performance of svn under apache is better than the svnserver, the same for git... Well... This is only for my experience. In git case, apache is used to transfer files, and it is much better in this than the most available alternatives. In svn case, apache provides the concurrency missing from svnserve. Also using none secured protocols, exposes users to man-in-the-middle attacks. The existing http:// had this problem already, it's not a new one. git:// and svn:// do both have patches around adding support for adding TLS. This however just adds overhead, I really need to finish the tree-signing work I was doing, as that protects the content better (MITM is still possible on SSL without it, just a lot harder as an attacker has to deal with the SSL stream first). Even if tree signing will be available, the developers should work in secured channel... ssh or https... The users will benefit from the signing and not require secured channel. Until signing will be available, I think it is very important for us to provide reliable source. Regards, Alon Bar-Lev. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On 1/18/08, Mike Frysinger [EMAIL PROTECTED] wrote: On Thursday 17 January 2008, Robin H. Johnson wrote: anonvcs.gentoo.org: anoncvs, anonsvn, anongit - Anonymous SVN is changing from http:// to svn:// [1] overlays.gentoo.org [3]: - Anonymous SVN is changing from http:// to svn:// i'd point out that http:// syncing is usable from behind firewalls while svn:// is not ... while this does not affect me personally, it's something to keep in mind. -mike Just wanted to note this too... I am one of the affected ones... I think that it is very important to have http, and even https for formal resources. git://, svn://, rsync:// or ssh+X:// are inaccessible for a large group of users. Also using none secured protocols, exposes users to man-in-the-middle attacks. Best Regards, Alon Bar-Lev. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
On Sat, Jan 19, 2008 at 01:01:04AM +0200, Alon Bar-Lev wrote: On 1/19/08, Robin H. Johnson [EMAIL PROTECTED] wrote: My core concern with the SVN http://, was the crappy performance it provided compared to svn://. The main rsync tree has never been available for iterative syncing via http://, just had tarball snapshots and deltas instead. If I understand correctly, the performance of svn under apache is better than the svnserver, the same for git... Well... This is only for my experience. In git case, apache is used to transfer files, and it is much better in this than the most available alternatives. Umm, I think you've got things a bit reversed here. The core problem with using both SVN and Git over HTTP, is the number of round trips required. Git provides the best example, if the server side isn't already packed, each object needs to get fetched individually. Whereas the git:// protocol effectively sends 'I have rev XYZ, give me everything up to HEAD.' One message in each direction, with a slight wait in the middle while the server prepares the response. In svn case, apache provides the concurrency missing from svnserve. svnserve running under xinetd so it's niced and set to a max of 10 concurrent users. I benched it up with 30 concurrent updates myself, but I want to save room for now. Even if tree signing will be available, the developers should work in secured channel... ssh or https... The users will benefit from the signing and not require secured channel. Until signing will be available, I think it is very important for us to provide reliable source. The git:// and svn:// are for the anonymous side - I did state that clearly in my original post. Git commits are using git+ssh:// (via gitosis), and while I'd like to do the same for SVN, it will probably remain SVN-over-https:// for now. -- Robin Hugh Johnson Gentoo Linux Developer Infra Guy E-Mail : [EMAIL PROTECTED] GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 pgpEzJ3dA06yz.pgp Description: PGP signature
[gentoo-dev] Upcoming Infra maintenance/downtimes: anon{cvs,svn,git}, archives, bouncer, overlays
Hi folks, Infra is working on a bunch of things lately, and there are going to be changes or brief outages for the following services (this is pretty much the order they are being worked on). anonvcs.gentoo.org: anoncvs, anonsvn, anongit - Moving between machines - Anonymous SVN is changing from http:// to svn:// [1] archives.gentoo.org: - Moving between machines - Update of some missing mail bouncer.gentoo.org: - Moving between machines - Maybe upgrade at the same time? overlays.gentoo.org [3]: - Moving between machines - Git service is already on the new machine - Anonymous SVN is changing from http:// to svn:// - Trac being replaced [2] Footnotes: 1. You do not need to do a new checkout at all, you can use the following command to update your SVN repos: svn switch --relocate \ http://anonsvn.gentoo.org/repositories/$REPO \ svn://anonsvn.gentoo.org/$REPO 2. Trac doesn't scale well enough, as users of the existing overlay machine have noted performance problems before. Being replaced with ViewVC and as yet undecided which Wiki application. 3. I'll send another notification closer to the overlays work/switchover. -- Robin Hugh Johnson Gentoo Linux Developer Infra Guy E-Mail : [EMAIL PROTECTED] GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 pgpL9N168F7cC.pgp Description: PGP signature
