Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-27 Thread Corentin Chary
On Thu, Apr 26, 2012 at 8:41 PM, Michał Górny mgo...@gentoo.org wrote:
 On Thu, 26 Apr 2012 10:21:36 +0200
 Corentin Chary corentin.ch...@gmail.com wrote:

 Second solution:
 github http://cloud.github.com/downloads
 github-bad-uris -http://github.com/downloads/
 -https://github.com/downloads/

 The good thing with the first one is that it would allow repoman to
 outputs something like you should use 'mirror://github'.

 Well, we could decide on something common and special like:

 github:bad-uris http://.

 And then let repoman suggest using mirror with ':bad-uris' stripped.

Works for me. What would be the next step to push this ?


-- 
Corentin Chary
http://xf.iksaif.net



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-27 Thread Michał Górny
On Fri, 27 Apr 2012 10:06:38 +0200
Corentin Chary corentin.ch...@gmail.com wrote:

 On Thu, Apr 26, 2012 at 8:41 PM, Michał Górny mgo...@gentoo.org
 wrote:
  On Thu, 26 Apr 2012 10:21:36 +0200
  Corentin Chary corentin.ch...@gmail.com wrote:
 
  Second solution:
  github http://cloud.github.com/downloads
  github-bad-uris -http://github.com/downloads/
  -https://github.com/downloads/
 
  The good thing with the first one is that it would allow repoman to
  outputs something like you should use 'mirror://github'.
 
  Well, we could decide on something common and special like:
 
  github:bad-uris http://.
 
  And then let repoman suggest using mirror with ':bad-uris' stripped.
 
 Works for me. What would be the next step to push this ?

Next step would be to find a good name :P. I suggest going the Perl way
and prepending '~' ;P.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-26 Thread Corentin Chary
On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny mgo...@gentoo.org wrote:
 On Wed, 25 Apr 2012 09:16:05 +0200
 Corentin Chary corentin.ch...@gmail.com wrote:

 On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny mgo...@gentoo.org
 wrote:
  On Tue, 24 Apr 2012 16:19:11 +
  Robin H. Johnson robb...@gentoo.org wrote:
 
  On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
$ ./mirrors.py --all --count
297 ?? ?? http://pear.php.net
297 ?? ?? http://pear.php.net/get
88 ?? ?? ??http://pecl.php.net
88 ?? ?? ??http://pecl.php.net/get
These are already mirror bouncers. If you visit the above,
you'll get the closest mirror for downloading.
   And since there is already ~10 mirrors with only one actual
   backend, should they go to thirdpartymirrors or not ? If not,
   what about this pseudo-mirrors already present in
   thirdpartymirrors ?
  I think we should add the pseudo-mirrors, but explicitly mark them
  as such in the file, so that they don't get duplicate entries
  added (eg adding us.pear, de.pear and the pear bouncer is bad.
  Should have just the bouncer).
 
  It'd be great if we could add some kind of additional mirror
  entries, which would be used by repoman to signal missing mirror://
  entries but won't be used for downloads.

 Yep, we could put that in it too:
 github                http://github.com/downloads/
 https://github.com/downloads/

 Per spec, portage can choose a random mirror of the list. If we put
 entries like that, these two will be equally possible as the preferred
 cloud. URL -- while they redirect one to another.

 We might decide on some common syntax like preceding all extra entries
 with '-' but I don't want to be the one deciding here.

I checked, and current portage code already handle entries starting
with a - gracefully thanks to stack_dictlist (removing them from the
list of mirrors).

-- 
Corentin Chary
http://xf.iksaif.net



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-26 Thread Zac Medico
On 04/26/2012 12:30 AM, Corentin Chary wrote:
 On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny mgo...@gentoo.org wrote:
 On Wed, 25 Apr 2012 09:16:05 +0200
 Corentin Chary corentin.ch...@gmail.com wrote:

 On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny mgo...@gentoo.org
 wrote:
 On Tue, 24 Apr 2012 16:19:11 +
 Robin H. Johnson robb...@gentoo.org wrote:

 On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
 $ ./mirrors.py --all --count
 297 ?? ?? http://pear.php.net
 297 ?? ?? http://pear.php.net/get
 88 ?? ?? ??http://pecl.php.net
 88 ?? ?? ??http://pecl.php.net/get
 These are already mirror bouncers. If you visit the above,
 you'll get the closest mirror for downloading.
 And since there is already ~10 mirrors with only one actual
 backend, should they go to thirdpartymirrors or not ? If not,
 what about this pseudo-mirrors already present in
 thirdpartymirrors ?
 I think we should add the pseudo-mirrors, but explicitly mark them
 as such in the file, so that they don't get duplicate entries
 added (eg adding us.pear, de.pear and the pear bouncer is bad.
 Should have just the bouncer).

 It'd be great if we could add some kind of additional mirror
 entries, which would be used by repoman to signal missing mirror://
 entries but won't be used for downloads.

 Yep, we could put that in it too:
 githubhttp://github.com/downloads/
 https://github.com/downloads/

 Per spec, portage can choose a random mirror of the list. If we put
 entries like that, these two will be equally possible as the preferred
 cloud. URL -- while they redirect one to another.

 We might decide on some common syntax like preceding all extra entries
 with '-' but I don't want to be the one deciding here.
 
 I checked, and current portage code already handle entries starting
 with a - gracefully thanks to stack_dictlist (removing them from the
 list of mirrors).

That means repoman will ignore them too. If you want existing versions
of repoman to check for those paths in SRC_URI, you can add a line like
this to thirdpartymirrors:

github-bad-urls http://github.com/downloads/ https://github.com/downloads/
-- 
Thanks,
Zac



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-26 Thread Corentin Chary
On Thu, Apr 26, 2012 at 9:57 AM, Zac Medico zmed...@gentoo.org wrote:
 On 04/26/2012 12:30 AM, Corentin Chary wrote:
 On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny mgo...@gentoo.org wrote:
 On Wed, 25 Apr 2012 09:16:05 +0200
 Corentin Chary corentin.ch...@gmail.com wrote:

 On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny mgo...@gentoo.org
 wrote:
 On Tue, 24 Apr 2012 16:19:11 +
 Robin H. Johnson robb...@gentoo.org wrote:

 On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
 $ ./mirrors.py --all --count
 297 ?? ?? http://pear.php.net
 297 ?? ?? http://pear.php.net/get
 88 ?? ?? ??http://pecl.php.net
 88 ?? ?? ??http://pecl.php.net/get
 These are already mirror bouncers. If you visit the above,
 you'll get the closest mirror for downloading.
 And since there is already ~10 mirrors with only one actual
 backend, should they go to thirdpartymirrors or not ? If not,
 what about this pseudo-mirrors already present in
 thirdpartymirrors ?
 I think we should add the pseudo-mirrors, but explicitly mark them
 as such in the file, so that they don't get duplicate entries
 added (eg adding us.pear, de.pear and the pear bouncer is bad.
 Should have just the bouncer).

 It'd be great if we could add some kind of additional mirror
 entries, which would be used by repoman to signal missing mirror://
 entries but won't be used for downloads.

 Yep, we could put that in it too:
 github                http://github.com/downloads/
 https://github.com/downloads/

 Per spec, portage can choose a random mirror of the list. If we put
 entries like that, these two will be equally possible as the preferred
 cloud. URL -- while they redirect one to another.

 We might decide on some common syntax like preceding all extra entries
 with '-' but I don't want to be the one deciding here.

 I checked, and current portage code already handle entries starting
 with a - gracefully thanks to stack_dictlist (removing them from the
 list of mirrors).

 That means repoman will ignore them too. If you want existing versions
 of repoman to check for those paths in SRC_URI, you can add a line like
 this to thirdpartymirrors:

 github-bad-urls http://github.com/downloads/ https://github.com/downloads/

Hum, I checked repoman source code, and I didn't find where it checks
if SRC_URI matches something in thirdpartymirror. Any hint ?


-- 
Corentin Chary
http://xf.iksaif.net



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-26 Thread Zac Medico
On 04/26/2012 01:03 AM, Corentin Chary wrote:
 On Thu, Apr 26, 2012 at 9:57 AM, Zac Medico zmed...@gentoo.org wrote:
 On 04/26/2012 12:30 AM, Corentin Chary wrote:
 On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny mgo...@gentoo.org wrote:
 On Wed, 25 Apr 2012 09:16:05 +0200
 Corentin Chary corentin.ch...@gmail.com wrote:

 On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny mgo...@gentoo.org
 wrote:
 On Tue, 24 Apr 2012 16:19:11 +
 Robin H. Johnson robb...@gentoo.org wrote:

 On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
 $ ./mirrors.py --all --count
 297 ?? ?? http://pear.php.net
 297 ?? ?? http://pear.php.net/get
 88 ?? ?? ??http://pecl.php.net
 88 ?? ?? ??http://pecl.php.net/get
 These are already mirror bouncers. If you visit the above,
 you'll get the closest mirror for downloading.
 And since there is already ~10 mirrors with only one actual
 backend, should they go to thirdpartymirrors or not ? If not,
 what about this pseudo-mirrors already present in
 thirdpartymirrors ?
 I think we should add the pseudo-mirrors, but explicitly mark them
 as such in the file, so that they don't get duplicate entries
 added (eg adding us.pear, de.pear and the pear bouncer is bad.
 Should have just the bouncer).

 It'd be great if we could add some kind of additional mirror
 entries, which would be used by repoman to signal missing mirror://
 entries but won't be used for downloads.

 Yep, we could put that in it too:
 githubhttp://github.com/downloads/
 https://github.com/downloads/

 Per spec, portage can choose a random mirror of the list. If we put
 entries like that, these two will be equally possible as the preferred
 cloud. URL -- while they redirect one to another.

 We might decide on some common syntax like preceding all extra entries
 with '-' but I don't want to be the one deciding here.

 I checked, and current portage code already handle entries starting
 with a - gracefully thanks to stack_dictlist (removing them from the
 list of mirrors).

 That means repoman will ignore them too. If you want existing versions
 of repoman to check for those paths in SRC_URI, you can add a line like
 this to thirdpartymirrors:

 github-bad-urls http://github.com/downloads/ https://github.com/downloads/
 
 Hum, I checked repoman source code, and I didn't find where it checks
 if SRC_URI matches something in thirdpartymirror. Any hint ?

Search for SRC_URI.mirror in /usr/bin/repoman.
-- 
Thanks,
Zac



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-26 Thread Corentin Chary
On Thu, Apr 26, 2012 at 10:07 AM, Zac Medico zmed...@gentoo.org wrote:
 On 04/26/2012 01:03 AM, Corentin Chary wrote:
 On Thu, Apr 26, 2012 at 9:57 AM, Zac Medico zmed...@gentoo.org wrote:
 On 04/26/2012 12:30 AM, Corentin Chary wrote:
 On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny mgo...@gentoo.org wrote:
 On Wed, 25 Apr 2012 09:16:05 +0200
 Corentin Chary corentin.ch...@gmail.com wrote:

 On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny mgo...@gentoo.org
 wrote:
 On Tue, 24 Apr 2012 16:19:11 +
 Robin H. Johnson robb...@gentoo.org wrote:

 On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
 $ ./mirrors.py --all --count
 297 ?? ?? http://pear.php.net
 297 ?? ?? http://pear.php.net/get
 88 ?? ?? ??http://pecl.php.net
 88 ?? ?? ??http://pecl.php.net/get
 These are already mirror bouncers. If you visit the above,
 you'll get the closest mirror for downloading.
 And since there is already ~10 mirrors with only one actual
 backend, should they go to thirdpartymirrors or not ? If not,
 what about this pseudo-mirrors already present in
 thirdpartymirrors ?
 I think we should add the pseudo-mirrors, but explicitly mark them
 as such in the file, so that they don't get duplicate entries
 added (eg adding us.pear, de.pear and the pear bouncer is bad.
 Should have just the bouncer).

 It'd be great if we could add some kind of additional mirror
 entries, which would be used by repoman to signal missing mirror://
 entries but won't be used for downloads.

 Yep, we could put that in it too:
 github                http://github.com/downloads/
 https://github.com/downloads/

 Per spec, portage can choose a random mirror of the list. If we put
 entries like that, these two will be equally possible as the preferred
 cloud. URL -- while they redirect one to another.

 We might decide on some common syntax like preceding all extra entries
 with '-' but I don't want to be the one deciding here.

 I checked, and current portage code already handle entries starting
 with a - gracefully thanks to stack_dictlist (removing them from the
 list of mirrors).

 That means repoman will ignore them too. If you want existing versions
 of repoman to check for those paths in SRC_URI, you can add a line like
 this to thirdpartymirrors:

 github-bad-urls http://github.com/downloads/ https://github.com/downloads/

 Hum, I checked repoman source code, and I didn't find where it checks
 if SRC_URI matches something in thirdpartymirror. Any hint ?

 Search for SRC_URI.mirror in /usr/bin/repoman.

Arg.. ok, I only looked in pym/repoman/.

So two solutions here:

First one:
github http://cloud.github.com/downloads -http://github.com/downloads/
-https://github.com/downloads/
+ a small patch that would allow repoman to do something like
settings.thirdpartymirrors(keep_bad_uris=True) in order to keep uris
starting with a - in the list.

Second solution:
github http://cloud.github.com/downloads
github-bad-uris -http://github.com/downloads/ -https://github.com/downloads/

The good thing with the first one is that it would allow repoman to
outputs something like you should use 'mirror://github'.


-- 
Corentin Chary
http://xf.iksaif.net



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-26 Thread Michał Górny
On Thu, 26 Apr 2012 10:21:36 +0200
Corentin Chary corentin.ch...@gmail.com wrote:

 Second solution:
 github http://cloud.github.com/downloads
 github-bad-uris -http://github.com/downloads/
-https://github.com/downloads/
 
 The good thing with the first one is that it would allow repoman to
 outputs something like you should use 'mirror://github'.

Well, we could decide on something common and special like:

github:bad-uris http://.

And then let repoman suggest using mirror with ':bad-uris' stripped.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-25 Thread Corentin Chary
On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny mgo...@gentoo.org wrote:
 On Tue, 24 Apr 2012 16:19:11 +
 Robin H. Johnson robb...@gentoo.org wrote:

 On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
   $ ./mirrors.py --all --count
   297 ?? ?? http://pear.php.net
   297 ?? ?? http://pear.php.net/get
   88 ?? ?? ??http://pecl.php.net
   88 ?? ?? ??http://pecl.php.net/get
   These are already mirror bouncers. If you visit the above, you'll
   get the closest mirror for downloading.
  And since there is already ~10 mirrors with only one actual
  backend, should they go to thirdpartymirrors or not ? If not, what
  about this pseudo-mirrors already present in thirdpartymirrors ?
 I think we should add the pseudo-mirrors, but explicitly mark them as
 such in the file, so that they don't get duplicate entries added (eg
 adding us.pear, de.pear and the pear bouncer is bad. Should have just
 the bouncer).

 It'd be great if we could add some kind of additional mirror entries,
 which would be used by repoman to signal missing mirror:// entries but
 won't be used for downloads.

Yep, we could put that in it too:
github  http://github.com/downloads/ https://github.com/downloads/
nongnu  http://download.savannah.nongnu.org/releases/

(I'm not really sure for nongnu).


-- 
Corentin Chary
http://xf.iksaif.net



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-25 Thread Michał Górny
On Wed, 25 Apr 2012 09:16:05 +0200
Corentin Chary corentin.ch...@gmail.com wrote:

 On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny mgo...@gentoo.org
 wrote:
  On Tue, 24 Apr 2012 16:19:11 +
  Robin H. Johnson robb...@gentoo.org wrote:
 
  On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
$ ./mirrors.py --all --count
297 ?? ?? http://pear.php.net
297 ?? ?? http://pear.php.net/get
88 ?? ?? ??http://pecl.php.net
88 ?? ?? ??http://pecl.php.net/get
These are already mirror bouncers. If you visit the above,
you'll get the closest mirror for downloading.
   And since there is already ~10 mirrors with only one actual
   backend, should they go to thirdpartymirrors or not ? If not,
   what about this pseudo-mirrors already present in
   thirdpartymirrors ?
  I think we should add the pseudo-mirrors, but explicitly mark them
  as such in the file, so that they don't get duplicate entries
  added (eg adding us.pear, de.pear and the pear bouncer is bad.
  Should have just the bouncer).
 
  It'd be great if we could add some kind of additional mirror
  entries, which would be used by repoman to signal missing mirror://
  entries but won't be used for downloads.
 
 Yep, we could put that in it too:
 githubhttp://github.com/downloads/
 https://github.com/downloads/

Per spec, portage can choose a random mirror of the list. If we put
entries like that, these two will be equally possible as the preferred
cloud. URL -- while they redirect one to another.

We might decide on some common syntax like preceding all extra entries
with '-' but I don't want to be the one deciding here.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


[gentoo-dev] [RFC] New third party mirrors

2012-04-24 Thread Corentin Chary
Hello,

As part of my portage-janitor scripts 
(https://github.com/iksaif/portage-janitor)
I created a tool that counts URIs prefixes and sort them.
Here is the result of this script on the gentoo-x86 tree:

$ ./mirrors.py --all --count
1262http://dev.gentoo.org
483 http://oss.tresys.com
482 http://oss.tresys.com/files
482 http://oss.tresys.com/files/refpolicy
473 http://xorg.freedesktop.org
473 http://xorg.freedesktop.org/releases
473 http://xorg.freedesktop.org/releases/individual
313 http://dev.gentoo.org/~swift
313 http://dev.gentoo.org/~swift/patches
309 http://dev.gentoo.org/~swift/patches/selinux-base-policy
297 http://pear.php.net
297 http://pear.php.net/get
296 http://hackage.haskell.org/packages
296 http://hackage.haskell.org/packages/archive
296 http://hackage.haskell.org
209 http://launchpad.net
206 http://ftp.xemacs.org
197 https://github.com
196 http://ftp.xemacs.org/pub
196 http://ftp.xemacs.org/pub/xemacs
193 http://ftp.xemacs.org/pub/xemacs/packages
179 http://gstreamer.freedesktop.org
179 http://gstreamer.freedesktop.org/src
176 http://github.com
175 http://linuxgazette.net/ftpfiles
175 http://linuxgazette.net
139 http://xorg.freedesktop.org/releases/individual/app
130 http://pear.horde.org/get
130 http://pear.horde.org
126 http://xorg.freedesktop.org/releases/individual/driver
115 http://dev.gentoo.org/~vapier
112 http://dev.gentoo.org/~vapier/dist
96  ftp://sources.redhat.com/pub
96  ftp://sources.redhat.com
94  http://savannah.nongnu.org/download
94  http://savannah.nongnu.org
92  http://xorg.freedesktop.org/releases/individual/lib
88  http://pecl.php.net
88  http://download.gna.org
88  http://pecl.php.net/get
84  http://dev.gentoo.org/~caster/distfiles
84  http://dev.gentoo.org/~caster
80  https://fedorahosted.org
74  http://get.qt.nokia.com
72  http://components.ez.no/get
72  http://components.ez.no
71  http://get.qt.nokia.com/qt
70  http://get.qt.nokia.com/qt/source
67  http://www.phrack.org/archives
67  http://www.phrack.org/archives/tgz

And here are some mirrors that can be deduced from that (except freebsd-jp
which I found reading the linphone ebuild):

freebsd-jp  ftp://ports.jp.FreeBSD.org/pub/FreeBSD-jp/ 
ftp://ftp.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp1.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp2.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp3.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp4.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp5.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp6.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp7.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp8.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp9.jp.freebsd.org/pub/FreeBSD-jp/ 
ftp://ftp.ics.es.osaka-u.ac.jp/pub/FreeBSD-jp/ 
ftp://ftp.ics.es.osaka-u.ac.jp/pub/mirrors/FreeBSD-jp/ 
ftp://ftp.allbsd.org/pub/FreeBSD-jp/ ftp://ftp.nagaokaut.ac.jp/pub/FreeBSD-jp/ 
ftp://ftp.nara.wide.ad.jp/pub/FreeBSD-jp/ ftp://ftp.jaist.ac.jp/pub/FreeBSD-jp/
pearhttp://pear.php.net/ http://de.pear.php.net/ 
http://us.pear.php.net/
redhat  ftp://sources.redhat.com/pub/ 
ftp://mirrors.kernel.org/sources.redhat.com/ 
ftp://mirrors.kernel.org/sources.redhat.com/ 
http://mirrors.kernel.org/sources.redhat.com/ 
http://sources-redhat.mirrors.airband.net/ 
ftp://gd.tuwien.ac.at/gnu/sourceware/ http://gd.tuwien.ac.at/gnu/sourceware/ 
ftp://ftp.gwdg.de/pub/linux/sources.redhat.com/ 
http://ftp.gwdg.de/pub/linux/sources.redhat.com/ 
ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/sourceware.org/ 
http://ftp-stud.fht-esslingen.de/pub/Mirrors/sourceware.org/ 
http://bo.mirror.garr.it/mirrors/sourceware.org/ 
ftp://bo.mirror.garr.it/mirrors/sourceware.org/ 
ftp://ftp.mirrorservice.org/sites/sources.redhat.com/pub/ 
http://www.mirrorservice.org/sites/sources.redhat.com/pub/
xemacs  http://ftp.xemacs.org/pub/ ftp://ftp.sa.xemacs.org/pub/
xorghttp://xorg.freedesktop.org/pub/ http://ftp.x.org/pub/

That would affect ~1000 URIs.
Unfortunatly I didn't found any mirror for:
- pecl.php.net
- hackage.haskell.org
- get.qt.nokia.com
- launchpad

But maybe we should still add them to thirdpartymirrors ? Look at 3dgamers, 
beyondunreal, bitbucket, github, gnome, kernel, xfce, vdrfiles and 
vdr-developerorg: they all only have one mirror !

For those afraid to add these mirrors and patch all affected ebuilds: no worry, 
mirrors.py can do it for you, and I already generated splitted patches 
generated from various thirdpartymirrors here: http://xf.iksaif.net/dev/gentoo/

Note that as part of https://bugs.gentoo.org/show_bug.cgi?id=405533 mgorny 
already commited a massive amount of patches adding mirror:// when the URI 
prefix was already present in thirdpartymirrors.

Thanks,

-- 
Corentin Chary
http://xf.iksaif.net/


pgp2V9f9HGBZb.pgp
Description: PGP signature


Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-24 Thread Robin H. Johnson
On Tue, Apr 24, 2012 at 03:10:00PM +0200, Corentin Chary wrote:
 Hello,
 
 As part of my portage-janitor scripts 
 (https://github.com/iksaif/portage-janitor)
 I created a tool that counts URIs prefixes and sort them.
 Here is the result of this script on the gentoo-x86 tree:
 
 $ ./mirrors.py --all --count
 297 http://pear.php.net
 297 http://pear.php.net/get
 88  http://pecl.php.net
 88  http://pecl.php.net/get
These are already mirror bouncers. If you visit the above, you'll get
the closest mirror for downloading.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-24 Thread Corentin Chary
On Tue, Apr 24, 2012 at 4:35 PM, Robin H. Johnson robb...@gentoo.org wrote:
 On Tue, Apr 24, 2012 at 03:10:00PM +0200, Corentin Chary wrote:
 Hello,

 As part of my portage-janitor scripts 
 (https://github.com/iksaif/portage-janitor)
 I created a tool that counts URIs prefixes and sort them.
 Here is the result of this script on the gentoo-x86 tree:

 $ ./mirrors.py --all --count
 297     http://pear.php.net
 297     http://pear.php.net/get
 88      http://pecl.php.net
 88      http://pecl.php.net/get
 These are already mirror bouncers. If you visit the above, you'll get
 the closest mirror for downloading.

And since there is already ~10 mirrors with only one actual backend,
should they go to thirdpartymirrors or not ? If not, what about this
pseudo-mirrors already present in thirdpartymirrors ?

-- 
Corentin Chary
http://xf.iksaif.net



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-24 Thread Michał Górny
On Tue, 24 Apr 2012 16:19:11 +
Robin H. Johnson robb...@gentoo.org wrote:

 On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
   $ ./mirrors.py --all --count
   297 ?? ?? http://pear.php.net
   297 ?? ?? http://pear.php.net/get
   88 ?? ?? ??http://pecl.php.net
   88 ?? ?? ??http://pecl.php.net/get
   These are already mirror bouncers. If you visit the above, you'll
   get the closest mirror for downloading.
  And since there is already ~10 mirrors with only one actual
  backend, should they go to thirdpartymirrors or not ? If not, what
  about this pseudo-mirrors already present in thirdpartymirrors ?
 I think we should add the pseudo-mirrors, but explicitly mark them as
 such in the file, so that they don't get duplicate entries added (eg
 adding us.pear, de.pear and the pear bouncer is bad. Should have just
 the bouncer).

It'd be great if we could add some kind of additional mirror entries,
which would be used by repoman to signal missing mirror:// entries but
won't be used for downloads.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-24 Thread Zac Medico

On 04/24/2012 09:38 AM, Michał Górny wrote:

On Tue, 24 Apr 2012 16:19:11 +
Robin H. Johnsonrobb...@gentoo.org  wrote:


On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:

$ ./mirrors.py --all --count
297 ?? ?? http://pear.php.net
297 ?? ?? http://pear.php.net/get
88 ?? ?? ??http://pecl.php.net
88 ?? ?? ??http://pecl.php.net/get

These are already mirror bouncers. If you visit the above, you'll
get the closest mirror for downloading.

And since there is already ~10 mirrors with only one actual
backend, should they go to thirdpartymirrors or not ? If not, what
about this pseudo-mirrors already present in thirdpartymirrors ?

I think we should add the pseudo-mirrors, but explicitly mark them as
such in the file, so that they don't get duplicate entries added (eg
adding us.pear, de.pear and the pear bouncer is bad. Should have just
the bouncer).


It'd be great if we could add some kind of additional mirror entries,
which would be used by repoman to signal missing mirror:// entries but
won't be used for downloads.


We could just add them in thirpartymirrors under fake names that nobody 
will use.

--
Thanks,
Zac



Re: [gentoo-dev] [RFC] New third party mirrors

2012-04-24 Thread Hans de Graaff
On Tue, 2012-04-24 at 15:10 +0200, Corentin Chary wrote:

 And here are some mirrors that can be deduced from that (except freebsd-jp
 which I found reading the linphone ebuild):
 
 xemacs  http://ftp.xemacs.org/pub/ ftp://ftp.sa.xemacs.org/pub/

I only consider the first site to be a reliable source for the packages,
and this is what the eclass provides (only 4 packages actually provide a
SRC_URI, all others are handled through the eclass).

Kind regards,

Hans


signature.asc
Description: This is a digitally signed message part