Hi John! I'll try to get through it while on the train to $dayjob. Maybe we can dump this into a Wiki page as 'under construction'?
txs and LieGrue, strub --- On Tue, 7/19/11, John Casey <[email protected]> wrote: > From: John Casey <[email protected]> > Subject: Re: [DISCUSS] Repository routing for Maven 3 > To: "Maven Developers List" <[email protected]> > Date: Tuesday, July 19, 2011, 3:37 AM > heh, hit send to fast... > > On 7/18/11 11:11 PM, John Casey wrote: > > This has been a long time coming, and I apologize for > the delay. I've > > been trying to clear enough time to work on some last > touches to the > > implementation before writing this, to make sure the > description agrees > > with the code. > > > > The code for this discussion is in the > mirror-group-routing branch of > > maven3: > > > > https://svn.apache.org/repos/asf/maven/maven-3/branches/mirror-group-routing > > > > > > The goal of this branch is to create a mechanism by > which the Maven > > repository can be fragmented, allowing artifact > resolution to be routed > > to appropriate repository URLs based on groupId and > canonical repository > > URL. > > > > The cornerstone of this mapping is the routes file, > which will have a > > default copy hosted on Maven project infrastructure. > The file format > > (for now) is JSON. Users will be free to create their > own routes file > > and point Maven to that instead if they prefer. > Alternatively, the > > routes file can be generated by a repository manager > or really any > > application capable of hosting or generating the > required JSON file. > > > > So, what's in this routes file? It has two sections: > groupId -> > > canonical-URL mappings, and canonical-URL -> > mirror-URL mappings. These > > sections are separate to allow generators to > pass-through one set of > > mappings while augmenting or replacing the other...for > example, a > > generator may not want to alter the groupId -> > canonical-URL mappings, > > but will probably want to generate a custom > canonical-URL -> mirror-URL* > > map. > > > > To begin with, each groupId can have a canonical > repository URL attached > > to it in the routing table. This is meant to be the > main Maven > > repository that hosts artifacts for that groupId. Many > open-source > > projects will probably list something like: > > > > http://oss.sonatype.org/content/groups/public > > > > as the canonical URL for their groupId, since many > open-source projects > > use Sonatype OSS to deploy their artifacts. > > > > Given a canonical repository URL for an artifact > groupId, the routing > > table then matches this canonical URL up to one or > more mirror URLs. In > > the example above, the default routing table JSON > would probably specify > > something like (using very rough pseudo-JSON): > > > > "http://oss.sonatype.org/content/groups/public": [ > > { "id": "central", "url": "http://repo1.maven.org/maven2/"}, > > { "id": "ibiblio", "url": > > "http://mirrors.ibiblio.org/pub/mirrors/maven2/"; } > > ] > > > > Ideally, I'd like to see the ASF Maven infrastructure > host an > > application that allowed any OSS project to register > their groupId and > > canonical URL, and allow mirror operators to register > which canonical > > URLs they mirror. This would allow us to delegate > administration to > > those who have a vested interest in ensuring the > correctness of that > > metadata. This application could be something very > lightweight, like a > > CouchDB instance that hosts an HTML- and JS-based > application...or, it > > could be something more in line with the interests of > our developers, if > > needed. The point is it's just the routes file we need > to host, and it > > doesn't really matter how. > > > > To support the demands of groupId and mirror routing, > I've had to hack > > into the aether-connector-wagon and wrap the > WagonRepositoryConnector in > > a class that can query the router for the correct URL > before delegating > > to the main WagonRepositoryConnector. Additionally, > I've introduced > > modifications and new logic to the MirrorSelector > classes used by Maven > > (yes, there are two: one in the old legacy code, and > another in Aether) > > in order to make them routing-aware. > > > > I'm also including a DNS TXT-record-based discovery > strategy as well, to > > demonstrate the auto-discovery logic. DNS TXT is > something I've been > > playing around with for my home network, to > automatically select my > > local Archiva instance when I'm at home, and use the > unmirrored > > locations outside the house. > > > > I think the implementation is getting pretty close. > I'm currently having > > trouble with some sort of race condition that's > showing up in the ITs. > > I'm still going through it, but it's likely to be slow > going for awhile > > to pin it down, especially since concurrency > programming isn't something > > I'm expert at (yet). If anyone has an inclination to > take a look and see > > if they can figure out why ITs are failing > intermittently, I'd welcome > > the help. > > > > If there's anything in here that doesn't make sense > > ...just let me know and I can fill in details. > > > > > -- > John Casey > Developer, PMC Member - Apache Maven (http://maven.apache.org) > Blog: http://www.johnofalltrades.name/ > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
