Re: [gentoo-dev] Re: Gentoo Janitor scripts
On Wed, Feb 22, 2012 at 7:03 PM, Markos Chandras wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > On 02/22/2012 01:33 PM, Ben wrote: >> On 22 February 2012 20:36, Corentin Chary >> wrote: >>> - "960 http://dev.gentoo.org": that's a lot of package >>> hosted here, is that really right ? >> >> That includes patches >20kb >> > Gentoo devs are supposed to put patches, tarballs and whatever they > want in their space. This is the recommended policy documented in > devmanual as well > > http://devmanual.gentoo.org/general-concepts/mirrors/index.html Yes, you're right, I totally forgot about patches. I'll try to make the script smarter and skip those. -- Corentin Chary http://xf.iksaif.net
Re: [gentoo-dev] Re: Gentoo Janitor scripts
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 02/22/2012 01:33 PM, Ben wrote: > On 22 February 2012 20:36, Corentin Chary > wrote: >> - "960 http://dev.gentoo.org": that's a lot of package >> hosted here, is that really right ? > > That includes patches >20kb > Gentoo devs are supposed to put patches, tarballs and whatever they want in their space. This is the recommended policy documented in devmanual as well http://devmanual.gentoo.org/general-concepts/mirrors/index.html - -- Regards, Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.18 (GNU/Linux) iQIcBAEBCgAGBQJPRS4BAAoJEPqDWhW0r/LCjtsQALww5y0MDZ6zKwx941mPvCgi QFHqegBv5re+JsqPj/E9IyfzYXG7P06RAp/p5PvaaRM9MRXm5RuBeRCOI+opTDEU 8f0PuQ2Ft3I0+EIdI2V0xcJc5dPQZ9vg/fukApwT/3BOU09VcU5W4MMHjFBS58oX akeyPTjnKV/TUwD8gUkw/+nnYgDyYjXagoWgJLcTi6EMDYCwu1ax1iRH2kOUTevs vZHc0Ftejhn8lbemchvfRfqekPR6izhvSE7L1N63SP6cVfY8pykSzELLiqqRSobq tc7T73uk2gLcWHvmO0aK3YmaJCMjUViiYvMi6lzclyeass7zrKrfTIRuT+1Ffas+ +qSeKpHnIT5yOCEdir8HCoQVoqRjr5pGEMrkLiB91F5I2S9t68YsvpPP8KhAuXOe P6aGCE1EpBmcB1LCW3J3MnCJryb0r7fIF+K88ReulDGHn11MJC8dZGqeBJM224Db nlN4GYc+nZTt6wOIkv4yyBwhOFLT8szFFHHwt5vBEl6H2SyXJbFf/nQeiJF/UQkn tWi88Pa+rk0HeXVzy+2fCg844yYny5PRraqODyfP7LNb0tv0ScHtx6aJH5Z27emz Q47OCsdH8v3gAmDDxaYlNu2xVXNoJKJqC9YYVhdI9H08SWq+CYcYl6CC9OrzuqPe DzMIE+spLzZxJj1+joGt =FG1X -END PGP SIGNATURE-
Re: [gentoo-dev] Re: Gentoo Janitor scripts
On Wed, 22 Feb 2012 09:48:52 +0100 Corentin Chary wrote: > From that output we can easilly find out new entries to > thirdpartymirrors, for example: > gentoo-devhttp://dev.gentoo.org Mirrors shouldn't be used as a dumb URL shortener. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Gentoo Janitor scripts
On 22 February 2012 20:36, Corentin Chary wrote: > - "960 http://dev.gentoo.org": that's a lot of package hosted > here, is that really right ? That includes patches >20kb
Re: [gentoo-dev] Re: Gentoo Janitor scripts
On Wed, Feb 22, 2012 at 1:20 PM, Mart Raudsepp wrote: > On K, 2012-02-22 at 09:48 +0100, Corentin Chary wrote: >> I did a quick script to count most used prefixes in SRC_URI yesterday >> (https://github.com/iksaif/portage-janitor/blob/master/mirrors.py) >> >> Here is the (filtered) result: >> >> $ eix --only-names | python mirrors.py --count >> 960 http://dev.gentoo.org >> 372 http://xorg.freedesktop.org >> 372 http://xorg.freedesktop.org/releases >> 372 http://xorg.freedesktop.org/releases/individual >> 306 http://pear.php.net >> 306 http://pear.php.net/get >> 256 http://oss.tresys.com >> 255 http://oss.tresys.com/files >> 255 http://oss.tresys.com/files/refpolicy >> 225 http://hackage.haskell.org/packages >> 225 http://hackage.haskell.org/packages/archive >> 225 http://hackage.haskell.org >> 206 http://ftp.xemacs.org >> 201 https://github.com >> 196 http://ftp.xemacs.org/pub >> 196 http://ftp.xemacs.org/pub/xemacs >> 193 http://ftp.xemacs.org/pub/xemacs/packages >> 181 http://gstreamer.freedesktop.org >> 181 http://gstreamer.freedesktop.org/src >> 175 http://launchpad.net >> 175 http://linuxgazette.net >> 143 http://github.com >> 130 http://pear.horde.org >> 130 http://pear.horde.org/get >> 101 http://savannah.nongnu.org/download >> 101 http://savannah.nongnu.org >> 100 http://get.qt.nokia.com >> 97 ftp://sources.redhat.com/pub >> 97 ftp://sources.redhat.com >> 96 http://get.qt.nokia.com/qt >> 95 http://get.qt.nokia.com/qt/source >> 90 http://download.gna.org >> 75 http://pecl.php.net >> 75 http://pecl.php.net/get >> 72 http://components.ez.no/get >> 72 http://components.ez.no >> 69 https://fedorahosted.org >> 67 http://www.phrack.org/archives >> 67 http://www.phrack.org/archives/tgz >> 67 http://www.phrack.org >> >> >> From that output we can easilly find out new entries to >> thirdpartymirrors, for example: >> gentoo-dev http://dev.gentoo.org >> xorg http://xorg.freedesktop.org >> gna http://download.gna.org >> pecl http://pecl.php.net >> pear http://pear.php.net >> github https://github.com http://github.com >> xemacs http://ftp.xemacs.org/pub/ ftp://ftp.sa.xemacs.org/pub/ >> launchpad http://launchpad.net >> redhat ftp://sources.redhat.com/pub/ (and probably others !) >> etc... >> >> The good part is that once you've modified thirdpartymirrors with new >> mirrors, running mirrors.py --all will generate a big patch for all >> your ebuilds to use those new mirrors ! > > If you want this, then you should better figure out actual upstream > mirroring systems and their list of mirrors they would want us to use. > Until such, this seems to be just for shortening SRC_URI addresses when > an upstream tarball domain name or path repeats, and that's definitely > not what thirdpartymirrors is for. Yes, of course, that was just a quick example, not something definitive. But lets some examples: - http://xorg.freedesktop.org, it's easy to find mirror for that one, http://ftp.x.org/pub/ for example - github: packages seems to use http and https, this script can help to standardize the url used - "960 http://dev.gentoo.org": that's a lot of package hosted here, is that really right ? And still, thirdpartymirrors have some entries with only one "mirror" and I believe factorizing SRC_URIs is a good thing (if something changes, you just patch thirdpartymirros, not hundreds of ebuilds). -- Corentin Chary http://xf.iksaif.net
Re: [gentoo-dev] Re: Gentoo Janitor scripts
On K, 2012-02-22 at 09:48 +0100, Corentin Chary wrote: > I did a quick script to count most used prefixes in SRC_URI yesterday > (https://github.com/iksaif/portage-janitor/blob/master/mirrors.py) > > Here is the (filtered) result: > > $ eix --only-names | python mirrors.py --count > 960 http://dev.gentoo.org > 372 http://xorg.freedesktop.org > 372 http://xorg.freedesktop.org/releases > 372 http://xorg.freedesktop.org/releases/individual > 306 http://pear.php.net > 306 http://pear.php.net/get > 256 http://oss.tresys.com > 255 http://oss.tresys.com/files > 255 http://oss.tresys.com/files/refpolicy > 225 http://hackage.haskell.org/packages > 225 http://hackage.haskell.org/packages/archive > 225 http://hackage.haskell.org > 206 http://ftp.xemacs.org > 201 https://github.com > 196 http://ftp.xemacs.org/pub > 196 http://ftp.xemacs.org/pub/xemacs > 193 http://ftp.xemacs.org/pub/xemacs/packages > 181 http://gstreamer.freedesktop.org > 181 http://gstreamer.freedesktop.org/src > 175 http://launchpad.net > 175 http://linuxgazette.net > 143 http://github.com > 130 http://pear.horde.org > 130 http://pear.horde.org/get > 101 http://savannah.nongnu.org/download > 101 http://savannah.nongnu.org > 100 http://get.qt.nokia.com > 97 ftp://sources.redhat.com/pub > 97 ftp://sources.redhat.com > 96 http://get.qt.nokia.com/qt > 95 http://get.qt.nokia.com/qt/source > 90 http://download.gna.org > 75 http://pecl.php.net > 75 http://pecl.php.net/get > 72 http://components.ez.no/get > 72 http://components.ez.no > 69 https://fedorahosted.org > 67 http://www.phrack.org/archives > 67 http://www.phrack.org/archives/tgz > 67 http://www.phrack.org > > > From that output we can easilly find out new entries to > thirdpartymirrors, for example: > gentoo-devhttp://dev.gentoo.org > xorg http://xorg.freedesktop.org > gna http://download.gna.org > pecl http://pecl.php.net > pear http://pear.php.net > github https://github.com http://github.com > xemacs http://ftp.xemacs.org/pub/ ftp://ftp.sa.xemacs.org/pub/ > launchpadhttp://launchpad.net > redhat ftp://sources.redhat.com/pub/ (and probably others !) > etc... > > The good part is that once you've modified thirdpartymirrors with new > mirrors, running mirrors.py --all will generate a big patch for all > your ebuilds to use those new mirrors ! If you want this, then you should better figure out actual upstream mirroring systems and their list of mirrors they would want us to use. Until such, this seems to be just for shortening SRC_URI addresses when an upstream tarball domain name or path repeats, and that's definitely not what thirdpartymirrors is for. Best, Mart Raudsepp
Re: [gentoo-dev] Re: Gentoo Janitor scripts
On Wed, Feb 22, 2012 at 12:48 AM, Corentin Chary wrote: > I did a quick script to count most used prefixes in SRC_URI yesterday > (https://github.com/iksaif/portage-janitor/blob/master/mirrors.py) > > Here is the (filtered) result: > > $ eix --only-names | python mirrors.py --count > 960 http://dev.gentoo.org > 372 http://xorg.freedesktop.org > 372 http://xorg.freedesktop.org/releases > 372 http://xorg.freedesktop.org/releases/individual > 306 http://pear.php.net > 306 http://pear.php.net/get > 256 http://oss.tresys.com > 255 http://oss.tresys.com/files > 255 http://oss.tresys.com/files/refpolicy > 225 http://hackage.haskell.org/packages > 225 http://hackage.haskell.org/packages/archive > 225 http://hackage.haskell.org > 206 http://ftp.xemacs.org > 201 https://github.com > 196 http://ftp.xemacs.org/pub > 196 http://ftp.xemacs.org/pub/xemacs > 193 http://ftp.xemacs.org/pub/xemacs/packages > 181 http://gstreamer.freedesktop.org > 181 http://gstreamer.freedesktop.org/src > 175 http://launchpad.net > 175 http://linuxgazette.net > 143 http://github.com > 130 http://pear.horde.org > 130 http://pear.horde.org/get > 101 http://savannah.nongnu.org/download > 101 http://savannah.nongnu.org > 100 http://get.qt.nokia.com > 97 ftp://sources.redhat.com/pub > 97 ftp://sources.redhat.com > 96 http://get.qt.nokia.com/qt > 95 http://get.qt.nokia.com/qt/source > 90 http://download.gna.org > 75 http://pecl.php.net > 75 http://pecl.php.net/get > 72 http://components.ez.no/get > 72 http://components.ez.no > 69 https://fedorahosted.org > 67 http://www.phrack.org/archives > 67 http://www.phrack.org/archives/tgz > 67 http://www.phrack.org > > > From that output we can easilly find out new entries to > thirdpartymirrors, for example: > gentoo-dev http://dev.gentoo.org Don't encourage people to do bad things by making it a mirror target ;) -A > xorg http://xorg.freedesktop.org > gna http://download.gna.org > pecl http://pecl.php.net > pear http://pear.php.net > github https://github.com http://github.com > xemacs http://ftp.xemacs.org/pub/ ftp://ftp.sa.xemacs.org/pub/ > launchpad http://launchpad.net > redhat ftp://sources.redhat.com/pub/ (and probably others !) > etc... > > The good part is that once you've modified thirdpartymirrors with new > mirrors, running mirrors.py --all will generate a big patch for all > your ebuilds to use those new mirrors ! > > -- > Corentin Chary > http://xf.iksaif.net >
[gentoo-dev] Re: Gentoo Janitor scripts
I did a quick script to count most used prefixes in SRC_URI yesterday (https://github.com/iksaif/portage-janitor/blob/master/mirrors.py) Here is the (filtered) result: $ eix --only-names | python mirrors.py --count 960 http://dev.gentoo.org 372 http://xorg.freedesktop.org 372 http://xorg.freedesktop.org/releases 372 http://xorg.freedesktop.org/releases/individual 306 http://pear.php.net 306 http://pear.php.net/get 256 http://oss.tresys.com 255 http://oss.tresys.com/files 255 http://oss.tresys.com/files/refpolicy 225 http://hackage.haskell.org/packages 225 http://hackage.haskell.org/packages/archive 225 http://hackage.haskell.org 206 http://ftp.xemacs.org 201 https://github.com 196 http://ftp.xemacs.org/pub 196 http://ftp.xemacs.org/pub/xemacs 193 http://ftp.xemacs.org/pub/xemacs/packages 181 http://gstreamer.freedesktop.org 181 http://gstreamer.freedesktop.org/src 175 http://launchpad.net 175 http://linuxgazette.net 143 http://github.com 130 http://pear.horde.org 130 http://pear.horde.org/get 101 http://savannah.nongnu.org/download 101 http://savannah.nongnu.org 100 http://get.qt.nokia.com 97 ftp://sources.redhat.com/pub 97 ftp://sources.redhat.com 96 http://get.qt.nokia.com/qt 95 http://get.qt.nokia.com/qt/source 90 http://download.gna.org 75 http://pecl.php.net 75 http://pecl.php.net/get 72 http://components.ez.no/get 72 http://components.ez.no 69 https://fedorahosted.org 67 http://www.phrack.org/archives 67 http://www.phrack.org/archives/tgz 67 http://www.phrack.org >From that output we can easilly find out new entries to thirdpartymirrors, for example: gentoo-devhttp://dev.gentoo.org xorg http://xorg.freedesktop.org gna http://download.gna.org pecl http://pecl.php.net pear http://pear.php.net github https://github.com http://github.com xemacs http://ftp.xemacs.org/pub/ ftp://ftp.sa.xemacs.org/pub/ launchpadhttp://launchpad.net redhat ftp://sources.redhat.com/pub/ (and probably others !) etc... The good part is that once you've modified thirdpartymirrors with new mirrors, running mirrors.py --all will generate a big patch for all your ebuilds to use those new mirrors ! -- Corentin Chary http://xf.iksaif.net
