Re: portsnap mirror servers
On Fri, 2006-04-21 at 10:19 -0700, Colin Percival wrote: Paul Mather wrote: On Fri, 2006-04-21 at 14:40 +0200, Benjamin Lutz wrote: Hm, but I see a quite noticeable speed difference between portsnap1 and portsnap2. The second one is quite a bit faster. I'll look into this over the summer. I notice that on 4.x portsnap never finds any mirrors because the grep of the output returned by host -t srv ... is not appropriate for 4.x's version of /usr/bin/host, which produces output different to that of 5.x onwards (a BIND8 vs BIND9 issue, I guess). So, maybe because of this, all of the portsnaps running on 4.x machines are hitting the same server each time instead of randomly choosing a mirror, thereby causing that mirror to be a bit more loaded? They are hitting the same server, but that server is portsnap2 (which is also portsnap.daemonology.net, which is the default server for pre-1.0 versions of portsnap from the ports tree). Given that most systems running portsnap are FreeBSD 6.0 or 6.1, this doesn't cause much differential loading. Not sure if this is offtopic but I always had problems with portsnap and it never worked for me, therefore, I dumped it for cvsup. Here is what I get: # portsnap fetch Looking up portsnap.FreeBSD.org mirrors... none found. Fetching snapshot tag... done. Fetching snapshot metadata... done. Updating from Tue Apr 18 03:16:17 AST 2006 to Sat Apr 22 14:30:08 AST 2006. Fetching 4 metadata patches. done. Applying metadata patches... done. Fetching 4 metadata files... /usr/sbin/portsnap: cannot open 22d2106522d8940cbe1385cae5dd831e247ce85793d0423f367656ed0dfda82d.gz: No such file or directory metadata is corrupt. I'm using 6.1-RC built on Apr 18 but that problem was there even on 6.0-RELEASE signature.asc Description: This is a digitally signed message part
Re: portsnap mirror servers
Here is what I get: # portsnap fetch Looking up portsnap.FreeBSD.org mirrors... none found. Fetching snapshot tag... done. Fetching snapshot metadata... done. Updating from Tue Apr 18 03:16:17 AST 2006 to Sat Apr 22 14:30:08 AST 2006. Fetching 4 metadata patches. done. Applying metadata patches... done. Fetching 4 metadata files... /usr/sbin/portsnap: cannot open 22d2106522d8940cbe1385cae5dd831e247ce85793d0423f367656ed0dfda82d.gz: No such file or directory metadata is corrupt. I'm using 6.1-RC built on Apr 18 but that problem was there even on 6.0-RELEASE Are you using an HTTP proxy? The pre-6.1 series portsnap did not play nicely with HTTP proxies. There were some patches committed in March or so to address PR's on the subject. I haven't had an opportunity to test the changes in my work environment (which requires HTTP proxy use) yet, but they look promising. Thanks, Marty ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap mirror servers
I'm using 6.1-RC built on Apr 18 but that problem was there even on 6.0-RELEASE Are you using an HTTP proxy? The pre-6.1 series portsnap did not play nicely with HTTP proxies. There were some patches committed in March or so to address PR's on the subject. I haven't had an opportunity to test the changes in my work environment (which requires HTTP proxy use) yet, but they look promising. Yech, sorry. What I meant to say was that portsnap didn't play nicely with proxies *requiring authentication*. The patches should address that, though I haven't tried them personally. The way to specify auth information should be: HTTP_PROXY=http://proxy.example.com:8080 HTTP_PROXY_AUTH=basic:*:user:pwd (from man 3 fetch). Thanks, Marty ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap mirror servers
On Sat, 2006-04-22 at 07:33 -0500, Martin Jackson wrote: I'm using 6.1-RC built on Apr 18 but that problem was there even on 6.0-RELEASE Are you using an HTTP proxy? The pre-6.1 series portsnap did not play nicely with HTTP proxies. There were some patches committed in March or so to address PR's on the subject. I haven't had an opportunity to test the changes in my work environment (which requires HTTP proxy use) yet, but they look promising. Yech, sorry. What I meant to say was that portsnap didn't play nicely with proxies *requiring authentication*. The patches should address that, though I haven't tried them personally. The way to specify auth information should be: HTTP_PROXY=http://proxy.example.com:8080 HTTP_PROXY_AUTH=basic:*:user:pwd (from man 3 fetch). Thank you for this information but my proxy settings does not require authentication. I just tried changing my proxy settings from http_proxy=http://my.proxy.server:8080; to HTTP_PROXY=http://my.proxy.server:8080 and it worked! :D I will keep on monitoring that and will report back if I have any issues. Thank you very much for your help signature.asc Description: This is a digitally signed message part
Re: portsnap mirror servers
On Wednesday 19 April 2006 00:44, Colin Percival wrote: I have a list of people who have offered mirrors, but so far I haven't seen any need for additional mirrors -- the two which already exist are showing no signs of slowing down. Hm, but I see a quite noticeable speed difference between portsnap1 and portsnap2. The second one is quite a bit faster. Cheers Benjamin pgpAnyrY6oHPL.pgp Description: PGP signature
Re: portsnap mirror servers
On Fri, 2006-04-21 at 14:40 +0200, Benjamin Lutz wrote: On Wednesday 19 April 2006 00:44, Colin Percival wrote: I have a list of people who have offered mirrors, but so far I haven't seen any need for additional mirrors -- the two which already exist are showing no signs of slowing down. Hm, but I see a quite noticeable speed difference between portsnap1 and portsnap2. The second one is quite a bit faster. I notice that on 4.x portsnap never finds any mirrors because the grep of the output returned by host -t srv ... is not appropriate for 4.x's version of /usr/bin/host, which produces output different to that of 5.x onwards (a BIND8 vs BIND9 issue, I guess). So, maybe because of this, all of the portsnaps running on 4.x machines are hitting the same server each time instead of randomly choosing a mirror, thereby causing that mirror to be a bit more loaded? Cheers, Paul. -- e-mail: [EMAIL PROTECTED] Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid. --- Frank Vincent Zappa ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap mirror servers
Paul Mather wrote: On Fri, 2006-04-21 at 14:40 +0200, Benjamin Lutz wrote: Hm, but I see a quite noticeable speed difference between portsnap1 and portsnap2. The second one is quite a bit faster. I'll look into this over the summer. I notice that on 4.x portsnap never finds any mirrors because the grep of the output returned by host -t srv ... is not appropriate for 4.x's version of /usr/bin/host, which produces output different to that of 5.x onwards (a BIND8 vs BIND9 issue, I guess). So, maybe because of this, all of the portsnaps running on 4.x machines are hitting the same server each time instead of randomly choosing a mirror, thereby causing that mirror to be a bit more loaded? They are hitting the same server, but that server is portsnap2 (which is also portsnap.daemonology.net, which is the default server for pre-1.0 versions of portsnap from the ports tree). Given that most systems running portsnap are FreeBSD 6.0 or 6.1, this doesn't cause much differential loading. Colin Percival ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap mirror servers
Hello, On Wed, Apr 19, 2006 at 01:28:56AM +0100, Chris wrote: On 18/04/06, Colin Percival [EMAIL PROTECTED] wrote: Why do you think there should be an .eu mirror? Whilst portsnap is fast, it is a noticeable speed difference when using from eu servers, I also think its a good idea for redundancy. I did not yet check in the sources or with tcpdump, but from the htrougput I see, I'd guess, there's a lot of sequential two-way communication involved. That kind of traffic is massively influenced by latency. While most sites in europe are reachable for me with a latency of 15-30ms, I have between 120 and 200 ms to most sites across the atlantic. Maybe this is old-school and not valid today anymore as it was 10 years ago, but keeping traffic local and not pushing the same data through the big exchanges and long distance lines again and again stil seems reasonable to me for various reasons. I also vote for more geographic distribution and a local mirror in europe. While not having any usage data (current usage, hstroy, perspectives), I cannot decide of course, if the time has already come, but I think time will come in foreseeable future. - Oliver -- | Oliver Brandmueller | Offenbacher Str. 1 | Germany D-14197 Berlin | | Fon +49-172-3130856 | Fax +49-172-3145027 | WWW: http://the.addict.de/ | | Ich bin das Internet. Sowahr ich Gott helfe. | | Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! | pgpny7QCRrb3G.pgp Description: PGP signature
Re: portsnap mirror servers
On 2006.04.19 09:50:31 +0200, Oliver Brandmueller wrote: On Wed, Apr 19, 2006 at 01:28:56AM +0100, Chris wrote: On 18/04/06, Colin Percival [EMAIL PROTECTED] wrote: Why do you think there should be an .eu mirror? Whilst portsnap is fast, it is a noticeable speed difference when using from eu servers, I also think its a good idea for redundancy. I did not yet check in the sources or with tcpdump, but from the htrougput I see, I'd guess, there's a lot of sequential two-way communication involved. That kind of traffic is massively influenced by Hey, Recent portsnap versions (since the ones shipped in 6.0 AFAIR) uses HTTP pipelining (when possible) which means that the latency really doesn't matter since many requests are sent at once without waiting for the reply. I should mention that when pipelining is enabled I don't really see a big difference when using portsnap from Europe compared to systems in the US. More often the local disk limits the speed of portsnap updates for me rather than bandwith/latency. As Colin has said before, there will be more mirrors later, but there really just isn't a need for more right now. -- Simon L. Nielsen pgpEnLZ2E3Q1N.pgp Description: PGP signature
Re: portsnap mirror servers
On Tue, 18 Apr 2006 01:43:52 +0100 Chris [EMAIL PROTECTED] wrote: How many mirrors does portsnap have, it seems to only have around 3 or 4 and they all located in the .us whilst cvs has dozens around the world. Is there a eu pool of mirrors available to use or if not is their a way I can apply to host an eu mirror or even 2 eu mirrors. From man portsnap: If you wish to use portsnap to keep a large number of machines up to date, you may wish to set up a caching HTTP proxy. Since portsnap uses fetch(1) to download updates, setting the HTTP_PROXY environment variable will direct it to fetch updates from the given proxy. This is much more efficient than mirroring the files on the portsnap server, since the vast majority of files are not needed by any particular client. So you could set up a public caching Proxy (maybe just for portsnap.freebsd.org) and tell people to use it. Voila. Your very efficient mirror :) - Marius ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap mirror servers
Chris wrote: How many mirrors does portsnap have, it seems to only have around 3 or 4 and they all located in the .us whilst cvs has dozens around the world. Two mirrors, actually: portsnap1.freebsd.org, and portsnap2.freebsd.org. Is there a eu pool of mirrors available to use or if not is their a way I can apply to host an eu mirror or even 2 eu mirrors. I have a list of people who have offered mirrors, but so far I haven't seen any need for additional mirrors -- the two which already exist are showing no signs of slowing down. Why do you think there should be an .eu mirror? Colin Percival ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap mirror servers
On 18/04/06, Colin Percival [EMAIL PROTECTED] wrote: Chris wrote: How many mirrors does portsnap have, it seems to only have around 3 or 4 and they all located in the .us whilst cvs has dozens around the world. Two mirrors, actually: portsnap1.freebsd.org, and portsnap2.freebsd.org. Is there a eu pool of mirrors available to use or if not is their a way I can apply to host an eu mirror or even 2 eu mirrors. I have a list of people who have offered mirrors, but so far I haven't seen any need for additional mirrors -- the two which already exist are showing no signs of slowing down. Why do you think there should be an .eu mirror? Colin Percival Whilst portsnap is fast, it is a noticeable speed difference when using from eu servers, I also think its a good idea for redundancy. Chris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
portsnap mirror servers
How many mirrors does portsnap have, it seems to only have around 3 or 4 and they all located in the .us whilst cvs has dozens around the world. Is there a eu pool of mirrors available to use or if not is their a way I can apply to host an eu mirror or even 2 eu mirrors. Chris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap mirror servers
On Tue, Apr 18, 2006 at 01:43:52AM +0100, Chris wrote: How many mirrors does portsnap have, it seems to only have around 3 or 4 and they all located in the .us whilst cvs has dozens around the world. Is there a eu pool of mirrors available to use or if not is their a way I can apply to host an eu mirror or even 2 eu mirrors. According to Colin's writeup here: http://people.freebsd.org/~cperciva/funding.html Right now [the mirroring code] uses around one thousand times more bandwidth than an individual client machine; as a result, I've been asking people to use the existing mirrors rather than creating their own, but for a variety of reasons this isn't ideal for everybody. Hopefully that will be fixed this summer and then we'll have lots of mirrors. -- Brooks -- Any statement of the form X is the one, true Y is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 pgp9sLI2bla7I.pgp Description: PGP signature
