> I must echo something that Ed touched on:
> 
> >> * Doesn't the Foundation plan to enable a transparent mirroring
> >> system very soon that would make all this p2.mirrorsUrl useless?
> > No!!!  With mirror URLs, the mirrors are directly accessed with no
> > further access to download.eclipse.org.  With transparent mirroring,
> > the download server remains a bottleneck because it must be consulted
> > in order to redirect "transparently" to some other site.
> 
> Agreed -- transparent mirroring is nice for zip files and such, where a
> short URL can be convenient. But for p2 repos which are accessed by
> machines, NO!!!!  :)  Let the client gather bits from the local source
> directly.

This goes waaaay off-topic from the original discussion, but:

Depending on how the transparent mirroring works, I could actually see it help 
a lot for P2 repos as well - at least with a little support from P2.

A) The situation right now is, for some fictional composite release update site 
on eclipse.org, and a very simplified P2 install:

1. Get http://download.eclipse.org/foo/updates/latest/p2.index
2. Get http://download.eclipse.org/foo/updates/latest/compositeContent.jar
3. Read child URL "../../drops/1.2.3", resolve it to 
http://download.eclipse.org/foo/drops/1.2.3
4. Get http://download.eclipse.org/foo/drops/1.2.3/p2.index
5. Get http://download.eclipse.org/foo/drops/1.2.3/content.jar
6. Resolve the P2 install plan
7. Get http://download.eclipse.org/foo/drops/1.2.3/artifacts.jar
8. Read p2.mirrorsUrl from artifacts.jar
9. Get mirror list from p2.mirrorsUrl (e.g. 
http://www.eclipse.org/downloads/download.php?file=/foo/drops/1.2.3&format=xml)
10. Select a mirror from the list and download individual artifacts from the 
mirror

While step 10 is typically the one that transfers the most actual bits over the 
pipes, that's actually six requests with non-trivial content that hit 
eclipse.org before we ever reach the mirror. In my experience, this already 
adds up to some significant delay if download.eclipse.org has a slow day or 
your pipe to it is thin in general. 

Also, having to mess with the artifacts.xml, e.g. to provide this repository 
completely from your own mirror, is quite messy. And having your Eclipse still 
contact download.eclipse.org due to the p2.mirrorsUrl when you've put up a 
mirror e.g. on a local server can be quite a surprise.

B) Now imagine a transparent mirroring solution where step 1 immediately 
redirects us to a suitable mirror, and P2 cooperates:

1. Request http://download.eclipse.org/foo/updates/latest/p2.index
a. Receive HTTP 307 to http://mirror.example.org/foo/updates/latest/p2.index 
b. Update P2 repo object so further repository loading uses the redirected URL
c. Follow redirect and get 
http://mirror.example.org/foo/updates/latest/p2.index 
2. Get http://mirror.example.org/foo/updates/latest/compositeContent.jar
3. Read child URL "../../drops/1.2.3", resolve it to 
http://mirror.example.org/foo/drops/1.2.3
...

That's a single request to eclipse.org (maybe two or three if there is no 
p2.index and P2 has to guess) and we're off to a faster mirror. And all that 
without messing with p2.mirrorsUrl. Also, it's much closer to how HTTP 
redirects are handled on the web and feels a lot more natural, to me at least.

Right now, this doesn't work because P2 transparently fetches the redirected 
content for each request, one at a time, but does not interpret it as a 
location change for the whole repo (which means you still get the first 
sequence of requests, just with a redirect after each step). I never quite 
understood the rationale behind this behavior. Was this a concious decision, 
and for what use-case? Or is it just coincidentally the ECF transport's 
redirect handling "just doing its job"?

WDYT?
Carsten
_______________________________________________
equinox-dev mailing list
equinox-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from 
this list, visit
https://dev.eclipse.org/mailman/listinfo/equinox-dev

Reply via email to