Hi all,
I fully agree with what Carsten said, plus 2 more observations:
1. If "everybody should use p2.mirrorsUrl", does the CBI support adding
this with Maven/Tycho? Are there any official docs how to persuade Tycho
to add that slot? The only doc that I could find was a blog from Andreas
at Codetrails, but that doesn't look like "the Eclipse.org official
story for something every project should do"?
2. If I prefer downloaded Zips of p2 repos for local archiving and
competely local install (since I'm behind a company proxy that only my
Web Browser can bypass), I found the p2.mirrorsUrl inside the ZIPs to be
not very helpful. Is it possible adding the p2.mirrorsUrl slot to a
composite, instead of the content.jar of individual children? That would
make more sense to me (and also alleviate part of Carsten's concerns).
Thanks,
Martin
--
Martin Oberhuber <http://wiki.eclipse.org/Martin_Oberhuber> | Software
Architect, Project Lead & Consultant | Austria
Am 2017-06-27 um 18:14 schrieb Carsten Reckord:
I must echo something that Ed touched on:
* Doesn't the Foundation plan to enable a transparent mirroring
system very soon that would make all this p2.mirrorsUrl useless?
No!!! With mirror URLs, the mirrors are directly accessed with no
further access to download.eclipse.org. With transparent mirroring,
the download server remains a bottleneck because it must be consulted
in order to redirect "transparently" to some other site.
Agreed -- transparent mirroring is nice for zip files and such, where a
short URL can be convenient. But for p2 repos which are accessed by
machines, NO!!!! :) Let the client gather bits from the local source
directly.
This goes waaaay off-topic from the original discussion, but:
Depending on how the transparent mirroring works, I could actually see it help
a lot for P2 repos as well - at least with a little support from P2.
A) The situation right now is, for some fictional composite release update site
on eclipse.org, and a very simplified P2 install:
1. Gethttp://download.eclipse.org/foo/updates/latest/p2.index
2. Gethttp://download.eclipse.org/foo/updates/latest/compositeContent.jar
3. Read child URL "../../drops/1.2.3", resolve it
tohttp://download.eclipse.org/foo/drops/1.2.3
4. Gethttp://download.eclipse.org/foo/drops/1.2.3/p2.index
5. Gethttp://download.eclipse.org/foo/drops/1.2.3/content.jar
6. Resolve the P2 install plan
7. Gethttp://download.eclipse.org/foo/drops/1.2.3/artifacts.jar
8. Read p2.mirrorsUrl from artifacts.jar
9. Get mirror list from p2.mirrorsUrl
(e.g.http://www.eclipse.org/downloads/download.php?file=/foo/drops/1.2.3&format=xml)
10. Select a mirror from the list and download individual artifacts from the
mirror
While step 10 is typically the one that transfers the most actual bits over the
pipes, that's actually six requests with non-trivial content that hit
eclipse.org before we ever reach the mirror. In my experience, this already
adds up to some significant delay if download.eclipse.org has a slow day or
your pipe to it is thin in general.
Also, having to mess with the artifacts.xml, e.g. to provide this repository
completely from your own mirror, is quite messy. And having your Eclipse still
contact download.eclipse.org due to the p2.mirrorsUrl when you've put up a
mirror e.g. on a local server can be quite a surprise.
B) Now imagine a transparent mirroring solution where step 1 immediately
redirects us to a suitable mirror, and P2 cooperates:
1. Requesthttp://download.eclipse.org/foo/updates/latest/p2.index
a. Receive HTTP 307 tohttp://mirror.example.org/foo/updates/latest/p2.index
b. Update P2 repo object so further repository loading uses the redirected URL
c. Follow redirect and gethttp://mirror.example.org/foo/updates/latest/p2.index
2. Gethttp://mirror.example.org/foo/updates/latest/compositeContent.jar
3. Read child URL "../../drops/1.2.3", resolve it
tohttp://mirror.example.org/foo/drops/1.2.3
...
That's a single request to eclipse.org (maybe two or three if there is no
p2.index and P2 has to guess) and we're off to a faster mirror. And all that
without messing with p2.mirrorsUrl. Also, it's much closer to how HTTP
redirects are handled on the web and feels a lot more natural, to me at least.
Right now, this doesn't work because P2 transparently fetches the redirected content for
each request, one at a time, but does not interpret it as a location change for the whole
repo (which means you still get the first sequence of requests, just with a redirect
after each step). I never quite understood the rationale behind this behavior. Was this a
concious decision, and for what use-case? Or is it just coincidentally the ECF
transport's redirect handling "just doing its job"?
WDYT?
Carsten
_______________________________________________
cross-project-issues-dev mailing list
[email protected]
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
_______________________________________________
cross-project-issues-dev mailing list
[email protected]
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
