Why packages sources are NOT mirrored?

[Marcin Juszkiewicz]

My traceroute  [v0.85]
puchatek (0.0.0.0)Thu Sep 17 14:42:22 2015
Keys:  Help   Display mode   Restart statistics   Order of fields quit
 Packets   Pings
 Host   Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. krzys.lan0.0%160.3   0.3   0.3   0.3   0.0
 2. 192.168.0.1  0.0%160.6   0.6   0.5   0.8   0.0
 3. ???
 4. 89-75-11-97.infra.chell  0.0%168.5   9.7   7.3  16.4   2.1
 5. 84.116.252.162   0.0%16  224.2 220.9 218.1 227.2   2.4
 6. 84.116.252.174   0.0%16  219.6 220.3 218.3 226.8   1.9
 7. 84.116.252.9 0.0%15  223.8 221.4 219.9 225.0   1.5
 8. 84.116.252.410.0%15  230.5 221.8 219.6 230.5   2.7
 9. 84.116.136.141  71.4%15   44.0  42.3  40.7  44.0   1.3
10. 84-116-130-130.aorta.ne  0.0%15  222.2 220.3 218.2 227.9   2.3
11. ch-zrh02a-ra1-xe-2-1-0.  0.0%15  224.3 229.6 219.6 251.9  11.2
12. 84-116-130-125.aorta.ne 85.7%15   42.3  41.5  40.8  42.3   1.0
13. 84-116-130-122.aorta.ne  0.0%15  148.2 149.9 145.4 177.0   8.2
14. 84.116.133.158   0.0%15  222.7 220.7 219.5 222.7   0.7
15. mpr1.sjc7.us 0.0%15  222.5 223.3 219.4 253.3   8.5
16. ae9.cr1.sjc2.us.zip.zay  0.0%15  249.4 225.0 218.8 249.4   9.7
17. v11.ae29.cr1.lax112.us.  0.0%15  232.1 231.2 228.1 240.9   3.1
18. ae7.mpr3.phx2.us.zip.za  0.0%15  242.7 239.8 236.5 249.9   3.5
19. 208.184.108.38.IPYX-072  0.0%15  223.8 222.8 221.5 226.2   1.1
20. border1.po1-bbnet1.phx0  0.0%15  221.7 222.2 220.7 226.0   1.3
21. redhat-2.border1.phx004  0.0%15  226.7 224.8 222.0 238.2   3.9
22. tx-160-181-132-209.redh  0.0%15  222.4 222.7 221.4 227.2   1.4
23. pkgs.fedoraproject.org   0.0%15  232.3 225.3 222.6 232.3   2.7

This is my connection to pkgs.fedoraproject.org machine. Fetching 48MB 
archive reached 36% in 2 (two) hours...


We ship binaries through mirrors all around the world. yum/dnf knows how 
to use them to get packages in a quick way. Knows how to handle lack of 
connection etc.


But sources used to build packages came from one place. ONE...

Why? Why we are not have them mirrored?

Or if we have then which magic option I did not managed to find yet?
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Why packages sources are NOT mirrored?

[Kevin Fenzi]

On Thu, 17 Sep 2015 14:46:33 +0200
Marcin Juszkiewicz  wrote:

...snip...

> This is my connection to pkgs.fedoraproject.org machine. Fetching
> 48MB archive reached 36% in 2 (two) hours...
> 
> We ship binaries through mirrors all around the world. yum/dnf knows
> how to use them to get packages in a quick way. Knows how to handle
> lack of connection etc.
> 
> But sources used to build packages came from one place. ONE...
> 
> Why? Why we are not have them mirrored?

Very few people download sources directly from our lookaside cache, and
I am not sure mirrors would see much advantage in mirroring that for
little use. 

> Or if we have then which magic option I did not managed to find yet?

You can get the source from the src.rpm which is mirrored?

kevin


pgpRrsRlyt0vR.pgp
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Why packages sources are NOT mirrored?

[Stephen John Smoogen]

On Sep 17, 2015 06:46, "Marcin Juszkiewicz"  wrote:
>
> My traceroute  [v0.85]
 This is my connection to pkgs.fedoraproject.org machine. Fetching 48MB
archive reached 36% in 2 (two) hours...
>
> We ship binaries through mirrors all around the world. yum/dnf knows how
to use them to get packages in a quick way. Knows how to handle lack of
connection etc.
>
> But sources used to build packages came from one place. ONE...
>
> Why? Why we are not have them mirrored?
>

I would talk with the mirrors in your area and find out why they don't
mirror it. Mirroring is a volunteer activity and mirrors do what they think
will be useful for users near them.

And to pre-answer something that might be related, there is no budget to
put this into a CDN. Those are extremely expensive operations and do not
volunteer space.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Why packages sources are NOT mirrored?

[Kevin Fenzi]

On Thu, 17 Sep 2015 16:36:52 -0400
DJ Delorie  wrote:

> 
> Stephen John Smoogen  writes:
> > I would talk with the mirrors in your area and find out why they
> > don't mirror it.
> 
> Do mirror sites have the option to not include sources? [*]

The pkgs.fedoraproject.org lookaside cache? 
(Which is what this thread was about)
No, because we don't offer it for mirroring currently.

If you mean SRPMS, then yes, they can choose to mirror some or none of
those.

> If so, does letting them have this choice have GPL implications?  I
> mean, if they're distributing binaries without making the sources
> available via the same site...  (and one of the arguments for that
> clause of the GPL was "the user may not be able to access the part of
> the internet where you put the sources" - which seems to be the case
> here)

IANAL, but no. 

kevin


pgpwp5hxsBeb6.pgp
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Why packages sources are NOT mirrored?

[DJ Delorie]

Stephen John Smoogen  writes:
> I would talk with the mirrors in your area and find out why they don't
> mirror it.

Do mirror sites have the option to not include sources? [*]

If so, does letting them have this choice have GPL implications?  I
mean, if they're distributing binaries without making the sources
available via the same site...  (and one of the arguments for that
clause of the GPL was "the user may not be able to access the part of
the internet where you put the sources" - which seems to be the case
here)


[*] granted, they can mirror whatever they want, but I mean, in order to
be listed as an official Fedora mirror and participate in the yum
update network...
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Why packages sources are NOT mirrored?

[Marcin Juszkiewicz]

W dniu 17.09.2015 o 22:42, Kevin Fenzi pisze:

> >I would talk with the mirrors in your area and find out why they
> >don't mirror it.

>
>Do mirror sites have the option to not include sources? [*]



The pkgs.fedoraproject.org lookaside cache?
(Which is what this thread was about)
No, because we don't offer it for mirroring currently.


Ok, for next time when I will have that issue I will forget about 
'fedpkg sources' and will fetch srpm, extract it and then build local 
srpm for hacking package for local build.


It would be nice to have an option/fallback in fedpkg to do it 
automatically but it is not so often needed.

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Why packages sources are NOT mirrored?

[Florian Weimer]

On 09/17/2015 07:26 PM, Stephen John Smoogen wrote:

> And to pre-answer something that might be related, there is no budget to
> put this into a CDN. Those are extremely expensive operations and do not
> volunteer space.

Some of them do, for Debian at least.  But I don't know what has
happened to early attempts in that direction, lack of IPv6 support was a
bit of a dealbreaker back then.

The problem I see is that CDNs are only work well if you use popular
resources because unlikely mirrors, local nodes are not preseeded with
content almost no one ever downloads.  So getting a SRPM or source
tarball from a CDN will likely trigger a local cache miss and a sluggish
CDN-internal download.

For my needs and my network location, the Fedora (and CentOS and Debian)
mirror infrastructure offers significantly better performance than
cdn.redhat.com.

-- 
Florian Weimer / Red Hat Product Security
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct