Re: Automatic package mirror discovery implementation for pkg_add(1) tool

2010-02-21 Thread frantisek holop
hmm, on Thu, Feb 11, 2010 at 12:35:05AM +0300, Igor Zinovik said that
 Maybe it is not polite to answer to this old thread, but I've integrated
 (somehow) AutoMirrorDiscovery functionality into pkg_add tool, just to
 prove myself that i can do that without crashing pkg_add functionality.  It is
 very clumsy right now, but it works for me (with some bugs of course).

this is a great idea i think.
maybe you would want to rename it MirrorAutoDiscovery though :]

i would also fill the cache file with all the mirrors,
(not just the n fastest) in order from fastest to slowest.

if the list of the mirrors is taken from www.openbsd.org
everytime the list is generated, the list could be
considered authoritative.

-f
-- 
go ahead, jump.  100,000 lemmings can't be wrong.



Re: Automatic package mirror discovery implementation for pkg_add(1) tool

2010-02-10 Thread Igor Zinovik
08.01.2010 15:30, Bob Beck writes:
 2010/1/8 Bob Beck b...@ualberta.ca:
 And what I mean by that Is that I would be willing to put together a similar
 trick for this on ftp.openbsd.org as I did for the installer - if
 someone was willing
 to integrate it into the pkg_add tools.

Maybe it is not polite to answer to this old thread, but I've integrated
(somehow) AutoMirrorDiscovery functionality into pkg_add tool, just to
prove myself that i can do that without crashing pkg_add functionality.  It is
very clumsy right now, but it works for me (with some bugs of course).

This is how it works (currently there is no knob to shut it up):
(/tmp)[213]% sudo pkg_add -d
Pinging anga.funkfeuer.at...OK
Pinging carroll.cac.psu.edu...OK
Pinging filedump.se.rit.edu...OK
Pinging ftp-stud.fht-esslingen.de...OK
Pinging ftp.arcane-networks.fr...OK
Pinging ftp.aso.ee...no response
Pinging ftp.belnet.be...OK
Pinging ftp.bytemine.net...OK
Pinging ftp.ca.openbsd.org...no response
Pinging ftp.cc.uoc.gr...no response
Pinging ftp.chg.ru...OK
Pinging ftp.crans.org...OK
Pinging ftp.cs.pu.edu.tw...no response
Pinging ftp.cse.buffalo.edu...no response
Pinging ftp.das.ufsc.br...OK
Pinging ftp.df.lth.se...OK
Pinging ftp.dkuug.dk...no response
Use of uninitialized value $ret in numeric eq (==) at
/usr/libdata/perl5/OpenBSD/AutoMirrorDiscovery.pm line 58.
Pinging ftp.duth.gr...no response
Pinging ftp.esat.net...OK
Pinging ftp.estpak.ee...OK
Pinging ftp.eu.openbsd.org...OK
Pinging ftp.fmed.uc.pt...no response
Pinging ftp.fr.openbsd.org...OK
Pinging ftp.freebsdchina.org...OK
Pinging ftp.freenet.de...OK
Pinging ftp.fsn.hu...OK
Pinging ftp.gamma.ru...OK
Pinging ftp.heanet.ie...OK
Pinging ftp.iinet.net.au...OK
Pinging ftp.inet.no...OK
Pinging ftp.irisa.fr...OK
Pinging ftp.is.co.za...OK
Pinging ftp.jaist.ac.jp...OK
Pinging ftp.jyu.fi...OK
Pinging ftp.kaist.ac.kr...no response
Pinging ftp.kddlabs.co.jp...no response
Pinging ftp.lambdaserver.com...OK
Pinging ftp.mirrorservice.org...OK
Pinging ftp.netbsd.se...OK
Pinging ftp.nluug.nl...OK
Pinging ftp.obsd.si...OK
Pinging ftp.openbsd.dk...OK
Pinging ftp.openbsd.or.id...OK
Pinging ftp.openbsd.org.ar...no response
Pinging ftp.openbsd.org...OK
Pinging ftp.piotrkosoft.net...no response
Pinging ftp.plig.net...OK
Pinging ftp.rediris.es...OK
Pinging ftp.spline.de...OK
Pinging ftp.task.gda.pl...OK
Pinging ftp.tcc.edu.tw...OK
Pinging ftp.tpnet.pl...OK
Pinging ftp.tw.openbsd.org...OK
Pinging ftp.udc.es...no response
Pinging ftp.ulak.net.tr...OK
Pinging ftp.uninett.no...OK
Pinging ftp.wu-wien.ac.at...OK
Pinging ftp2.fr.openbsd.org...OK
Pinging ftp3.usa.openbsd.org...OK
Pinging ftp5.usa.openbsd.org...OK
Pinging gulus.usherbrooke.ca...no response
Pinging mirror.aarnet.edu.au...OK
Pinging mirror.cdmon.com...OK
Pinging mirror.corbina.net...OK
Pinging mirror.hostfuss.com...OK
Pinging mirror.iawnet.sandia.gov...no response
Pinging mirror.internode.on.net...OK
Pinging mirror.pacific.net.au...OK
Pinging mirror.planetunix.net...OK
Pinging mirror.public-internet.co.uk...no response
Pinging mirror.rit.edu...OK
Pinging mirror.roothell.org...OK
Pinging mirror.switch.ch...OK
Pinging mirrors.24-7-solutions.net...OK
Pinging mirrors.localhost.net.ar...no response
Pinging mirrors.nic.funet.fi...OK
Pinging mirrors.ucr.ac.cr...no response
Pinging obsd.cec.mtu.edu...OK
Pinging openbsd.arcticnetwork.ca...OK
Pinging openbsd.bsdforen.de...OK
Pinging openbsd.ftp.fu-berlin.de...no response
Pinging openbsd.mirror.frontiernet.net...OK
Pinging openbsd.mirrors.pair.com...OK
Pinging openbsd.mirrors.tds.net...OK
Pinging openbsd.noc.jgm.gov.ar...no response
(/tmp)[214]% cat /var/db/ftpmirror.cache
ftp://mirrors.nic.funet.fi/pub/OpenBSD/4.6/packages/i386/
ftp://ftp.gamma.ru/pub/OpenBSD/4.6/packages/i386/
ftp://ftp.eu.openbsd.org/pub/OpenBSD/4.6/packages/i386/
(/tmp)[228]% echo $PKG_PATH

(/tmp)[229]% sudo pkg_add -i fdm
No packages available in the PKG_PATH
fdm-1.6:tdb-1.0.6p0: ok (1 to go)
fdm-1.6: ok


Here is the code:

/usr/src/usr.sbin/pkg_add/OpenBSD/AutoMirrorDiscovery.pm
# ex:ts=8 sw=4:
# $OpenBSD$
#
# Copyright (c) 2009 Igor Zinovik zino...@petrsu.ru
#
# Permission to use, copy, modify, and distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED AS IS AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

# functionality for automatic fastest ftp mirror discovery

use strict;
use warnings;

package OpenBSD::AutoMirrorDiscovery;

use OpenBSD::Paths;
use Net::Ping;
use Net::FTP;

sub 

Re: Automatic package mirror discovery implementation for pkg_add(1) tool

2010-01-08 Thread Bob Beck
2010/1/8 Bob Beck b...@ualberta.ca:
 Apart from this, this is a really tough problem, because of infrastructure
 issues. Basically, our mirrors are not that reliable, and the closest
 one often won't have the packages you need... which is a reason why it's
 mostly some user settings...

 True - although we could do a very similar stunt to what we do with ftplist
 now in the installer - including remembering your most recent choice from your
 IP address, so it just figured out where you liked to get your packages from.


And what I mean by that Is that I would be willing to put together a similar
trick for this on ftp.openbsd.org as I did for the installer - if
someone was willing
to integrate it into the pkg_add tools.



Re: Automatic package mirror discovery implementation for pkg_add(1) tool

2009-12-05 Thread Marc Espie
On Sat, Dec 05, 2009 at 10:47:25PM +0300, Igor Zinovik wrote:
 if (-e /var/db/pkg/ftpmirror.cache) {
   open my $fh, '', /var/db/pkg/ftpmirror.cache or
   die(Permission denied);
   @mirrors = $fh;
   close $fh;
   print $mirrors[0];
   exit;
 }
...
 open $fh, '', /var/db/pkg/ftpmirror.cache or
   die(Cannot create mirror cache file);
 

you can't do this. /var/db/pkg only holds packages. This will confuse
pkg_add when you try to update things.


Apart from this, this is a really tough problem, because of infrastructure
issues. Basically, our mirrors are not that reliable, and the closest
one often won't have the packages you need... which is a reason why it's
mostly some user settings...



Re: Automatic package mirror discovery implementation for pkg_add(1) tool

2009-12-05 Thread Stuart Henderson
On 2009/12/05 22:22, Marc Espie wrote:
 Apart from this, this is a really tough problem, because of infrastructure
 issues. Basically, our mirrors are not that reliable, and the closest
 one often won't have the packages you need... which is a reason why it's
 mostly some user settings...

For snapshots I totally agree, but I think this could work reasonably
well for releases.