Re: Maven and repository@apache.org
On Tuesday 11 January 2005 12:18, Russell Gold wrote: Is there currently a policy for how long files are kept in the repository? It seems to me that old versions are not kept very long at ibiblio, if there are newer versions available. I am not sure if you are talking about the Transit client or central repositories in general. So let's cover both. Transit's CacheHandler is currently non-clearing, meaning all artifacts are held in the local file system cache until explicitly deleted, and AFAIK that is the case in Maven as well. We (the DPML folks) are planning to change that to a LRU algorithm at some point. Central repositories should *never* remove artifacts, as this may have detrimental effects on applications. The general policy at ibiblio.org is the same, and I think ibiblio.org itself doesn't interfere. It is the people who have write access to ibiblio.org/maven that can possibly remove older versions, but I have personally not seen this happening. Old versions lacking is more often than not a matter of, only the recent ones have been added. We (the DPML folks) are planning another ResourceHost type, which will request the location(s) of the artifacts from its main host, and then download directly from the source, such as SourceForge, tigris.org and others. We think this is a more proper way of resolving artifact resources. Cheers Niclas -- +-//---+ | http://www.dpml.net| | http://niclas.hedhman.org | +--//--+
Re: Maven and repository@apache.org
Hi gang, On 05-01-2005 09:39, Nicola Ken Barozzi [EMAIL PROTECTED] wrote: What relationship does the repository list have with Maven? What relationship does Maven have with the repository list? Ass Brett Porter has written, and I agree: 1) Maven PMC takes ownership for getting the repository right IIRC [EMAIL PROTECTED] is actually a President's committee sort-of hanging off infra@ that's been tasked with figuring all this stuff out. The reason its not just the Maven PMC in the first place is that there's some other people that have something to say as well (i.e. precisely the people listed below). So I don't get why there's an ownership issue to reconsider. The maven PMC peeps to work on this (Brett, Mark, anyone else who volunteers) just get on the repository mailing list and JFDI. 2) Henk, myself (Maven PMC), Mark Diggory (if available), representative from interested Apache projects PMC (most likely someone from Ant) get together to sort out exactly what we think needs doing (we can use the repository list if appropriate) Yep, sounds like a good idea. 3) suggest small steps to fix each problem rather than coming up with a killer solution that will never get implemented Sounds like a good idea too. I'm guessing this is why I bailed out of that mailing list in the first place; we were solving too big a problem at once. 4) can use the repository wiki for information based on what has already been discussed And again, sounds like a good idea. I'd add that to use a common repository, there needs an implementation for all projects to use. The [EMAIL PROTECTED] project was not tasked to deal with that. I think this conflicts with #3. Maven has support, ant will have support, magic (from the DPML guys) has support, several ant-based scripts are around that provide support. Any tool that can HTTP GET can trivially be made to get files from the repository. For example I think I've done the basics in python, ruby, and bash+wget+grep every now and then. Cheers, - Leo
Re: Maven and repository@apache.org
On Thursday 06 January 2005 09:21, Dain Sundstrom wrote: How do you get around the problem that it is very difficult to get a url handler installed? Does this library work only when it is installed into the system class path? That is corrrect. The issue is in the java.net.URL handling, where the internal class (I think it was called URLClassPath) first tries to load the Handler with the callers Classloader (which is the Bootstrap CL since java.net.URL is the caller), and since that will fail for all custom ones, it then tries ClassLoader.getSystemClassLoader(); Unfortunately, so far Sun claims that the security issues are so unclear, that they do not intend to fix this. I fully agree that this is less than an ideal situation, but messing with the default URLStreamHandlerFactory was not any better. Internally we are working on a two layered approach, where the Handler is loaded by the System CL, but the entire workload is delegated down to reloadable parts. That is not ready yet. Cheers Niclas --- Hard work pays off in the future, laziness pays off now. - Steven Wright +-//---+ | http://www.dpml.net| | http://niclas.hedhman.org | +--//--+
Maven and repository@apache.org
Now that Depot has closed doors, the need for Apache-wide artifact handling has remained, and Maven is the only actor here that is willing and able to pursue this road. What relationship does the repository list have with Maven? What relationship does Maven have with the repository list? Ass Brett Porter has written, and I agree: 1) Maven PMC takes ownership for getting the repository right 2) Henk, myself (Maven PMC), Mark Diggory (if available), representative from interested Apache projects PMC (most likely someone from Ant) get together to sort out exactly what we think needs doing (we can use the repository list if appropriate) 3) suggest small steps to fix each problem rather than coming up with a killer solution that will never get implemented 4) can use the repository wiki for information based on what has already been discussed I'd add that to use a common repository, there needs an implementation for all projects to use. It's easy, correct and resonable to think that it should be under Maven. Is Wagon still active? The Depot project SVN is still there, ready to be used if/when needed by the Maven project. Thus I would add that the [EMAIL PROTECTED] list and wikis should support and point users to the Maven repository handling code. Does this sound reasonable? -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) -
Re: Maven and repository@apache.org
Ant is adding a task so that providers can plug into it, and Wagon can continue to be used in Ant. Absolutely - all I meant was that Wagon maintaining its own Ant task was probably not necessary given that. Wagon can plugin in to that. If we splinter implementations, the repository effort will always be half-baked, and acceptance will suffer. I agree. What interests me WRT repository is that it's possible for Ant users to install an antlib that makes Wagon usable in Ant, and for Eclipse and Netbeans users to be able to integrate it nicely in their artifact stuff. Depot had an Eclipse pluin BTW. Can you elaborate on how it was integrated into Eclipse and Netbeans? I assume this would be when you add a dependency to a project - if so, that's definitely something that has been talked about (the MevenIDE project may have already done this - I'm not sure), and could use what was in depot to do so. If this means that we also need a maven.jar in the classpath, I have no problem with it, as long as users are not forced to run maven itself to download the artifacts. Right. It wouldn't even be maven.jar - just a smaller JAR that does the repository and snapshot handling. It's definitely separate. Out of interest, this is the proposed future repository layout for Maven: http://docs.codehaus.org/pages/viewpage.action?pageId=8003 Any recommendation on how to move on from there, keeping in mind that there are already two wikis with different specs and two different lists ([EMAIL PROTECTED] and this one)? I don't think discussing the repository format is helpful at this point - it is a big undertaking to change and I think there are other priorities (security, mirroring and cleanup). That seems to be where this list got bogged down last time, though what is there now looks pretty much final. When it's not so late, I'll look at what is actually different and whether this is even an issue. The repository code can have different repository layouts, so if necessary we can support different ones. Obviously, its preferable to agree and be consistent. Eventually it will make sense to cutover to a new layout - but we'll want to ensure everything is in place first. The extra features needed are Apache mirroring support and use of the Sourceforge system without needing to recreate a repository, that Depot somewhat supported. That's interesting - being involved with a sourceforge project they prefered to release to a maven repository and not use sourceforge :) That's a good feature, though probably at the bottom of the list so far. I'm fine with the proposed layout, or any other reasonable alternative, as it does not interfere with the above, being still http and file based. Great. Cheers, Brett
Re: Maven and repository@apache.org
Take the commons-jelly/ directories on ibiblio and place them in a similar structure as an example. org/apache/commons/jelly/commons-jelly/1.0/commons-jelly-1.0.jar[.md5|.asc], etc org/apache/commons/jelly/commons-jelly-tags-ant/1.0/commons-jelly-1.0.jar org/apache/commons/jelly/commons-jelly-tags-util/1.1/commons-jelly-1.1.jar I guess there's a risk (especially if a project splits), that there be a conflict between artifactId and groupId[n]. I've noted it in confluence, and I'll look at how this differs to ASFRepository when I'm not in need of sleep :) Does this look ok? I think that document will need some more work. Looking at it now, I agree it needs more examples, though I'm still happy with the ideas. - Brett
Re: Maven and repository@apache.org
putting repository@ back on the list - gmail tends to drop lists unless you reply to all (hope this is what you intended). Seems a reasonable first step. However it seems a little fragile if a project changes artifact names or splits an existing artifact. e.g. if commons-jelly splits into jelly-api and jelly-impl, at least with the existing structure all artifacts of a given type are together and can be easily seen. Once the directory structure changes in the new layout, browsing will be difficult. I'm not sure what you mean: the existing one has commons-jelly-* JARS, then commons-jelly-api-* and commons-jelly-impl-*. The new one would have a commons-jelly subdirectory, a commons-jelly-api and commons-jelly-impl subdirectory. They are all still side by side in o/a/c/jelly/ ? Still, an archival policy is a good idea so that commons-jelly can be removed from here so people know what the current artifacts are (while still being able to get the old ones). I think there's merit to having 3 repositories: snapshot, archive and current (current is not just the latest release, but all supported releases). Equivalent to cvs.apache, archive.apache and www.apache now. Each can have different policies, but a common access technique. - Brett
Re: Maven and repository@apache.org
On Wed, 05 Jan 2005 09:39:28 +0100, Nicola Ken Barozzi [EMAIL PROTECTED] wrote: 2) Henk, myself (Maven PMC), Mark Diggory (if available), representative from interested Apache projects PMC (most likely someone from Ant) get together to sort out exactly what we think needs doing (we can use the repository list if appropriate) I'll be the Ant rep. I am co-author of the (still stabilising) Ant libraries task; it'd be nice if we can get the immediate improvements into the repository when there is time to get the changes into ant before 1.7 ships (which is not in the immediate future) 3) suggest small steps to fix each problem rather than coming up with a killer solution that will never get implemented Agreed. I'm very interested in 1. security. this could be with MD5 checksums, or it could be with signed JARs. JAR signing needs retrofitting to existing files, but has the advantage that JVMs integrate with it and you can do other tricks (like put http://ibiblio.org.../artifact.jar on the classpath with security turned on) 2. licenses. not just auto-download of .LICENSE files, but ideally some way to do click-through that even Sun are happy with. I was discussing the latter informally with someone at Sun recently -how it'd be nice to be able to retrieve files from some Sun server with the appropriate licenses appearing and needing accepting before the jar files are decrypted. This would be similar to the licensing stuff you get in SuSE Linux's package updater, where you have to accept the licenses for some downloads. (2) is clearly more complex, lower priority and really needs active involvement from Sun. -steve
RE: Maven and repository@apache.org
What about having an XML description of the contents of a repository? Such a description could serve multiple purposes, it could be used to enumerate known mirrors, it could be used to segment the namespace - say we reach some agreement with Sun and all sun artifacts fall into a namespace of sun-* and must be redirected to a Sun server, etc. This XML file could be used by tools that want to provide a list of every possible artifact. What if every Maven repository knew of every other Maven repository because they all shared a common resolution.xml file (think /etc/hosts before the existence of DNS - hackish but it worked). -Original Message- From: Brett Porter [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 05, 2005 6:43 AM To: [EMAIL PROTECTED]; Steve Loughran Subject: Re: Maven and [EMAIL PROTECTED] I'll be the Ant rep. Great, thanks. I am co-author of the (still stabilising) Ant libraries task; it'd yeah, I've got to 50 mail threads sitting flagged in gmail to read one day, as this is about the extent of what I know about it :) (after you introduced it to repository@ last year) 1. security. this could be with MD5 checksums, or it could be with signed JARs. MD5's aren't going to do much for security - they're mainly for download integrity. checking and publishing ASC files is a definite want I have, and that can be ramped up to the level of security you need (there are obviously varying levels of trust of the files and the KEYS themselves). JAR signing needs retrofitting to existing files, but has the advantage that JVMs integrate with it and you can do other tricks (like put http://ibiblio.org.../artifact.jar on the classpath with security turned on) That I haven't looked into, but would also be a good, but optional feature. I think this is more of a build feature than a repository feature? In fact, I'm sure we already do this for JNLP. 2. licenses. not just auto-download of .LICENSE files, but ideally some way to do click-through that even Sun are happy with. Yeah, there's a low hundreds JIRA entry for that (ie OLD :) I think even that wouldn't fly with Sun IIRC but it doesn't hurt to ask. Should be easy to add hooks and allow a user to say never ask again for this license to always accept ASL or something, but still report the license on download. Good ideas and reminders - keep them coming, and I'll put all this together on the wiki tomorrow-ish. Thanks, Brett
Re: Maven and repository@apache.org
Brett Porter wrote: ... What interests me WRT repository is that it's possible for Ant users to install an antlib that makes Wagon usable in Ant, and for Eclipse and Netbeans users to be able to integrate it nicely in their artifact stuff. Depot had an Eclipse pluin BTW. Can you elaborate on how it was integrated into Eclipse and Netbeans? I assume this would be when you add a dependency to a project - if so, that's definitely something that has been talked about (the MevenIDE project may have already done this - I'm not sure), and could use what was in depot to do so. http://krysalis.org/ruper/eclipse/ Note that it supports multiple repositories. ... Out of interest, this is the proposed future repository layout for Maven: http://docs.codehaus.org/pages/viewpage.action?pageId=8003 Any recommendation on how to move on from there, keeping in mind that there are already two wikis with different specs and two different lists ([EMAIL PROTECTED] and this one)? I don't think discussing the repository format is helpful at this point - it is a big undertaking to change and I think there are other priorities (security, mirroring and cleanup). That seems to be where this list got bogged down last time, though what is there now looks pretty much final. When it's not so late, I'll look at what is actually different and whether this is even an issue. The repository code can have different repository layouts, so if necessary we can support different ones. Obviously, its preferable to agree and be consistent. Eventually it will make sense to cutover to a new layout - but we'll want to ensure everything is in place first. Big +1 to this. Write it in stone :-) The extra features needed are Apache mirroring support and use of the Sourceforge system without needing to recreate a repository, that Depot somewhat supported. That's interesting - being involved with a sourceforge project they prefered to release to a maven repository and not use sourceforge :) That's a good feature, though probably at the bottom of the list so far. It permits project artifacts not to be put inside ibiblio, with the security and access issues that it brings. I'm fine with the proposed layout, or any other reasonable alternative, as it does not interfere with the above, being still http and file based. Great. -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) -
Re: Maven and repository@apache.org
Tim O'Brien wrote: From: Nicola Ken Barozzi [mailto:[EMAIL PROTECTED] ... This is something that is already present in the JJar[1] project. Wouldn't it suffice to just tell the system in what repository to look for to get a certain artifact? Sure, you could do this, but then you get into the problem of management. The early internet became unwieldy because every /etc/hosts was explicit, then came DNS. I kinda disagree. Eclipse 'forces' you to enter the http of the repository from which you can get a plugin, and AFAIK it has not become (yet?) unwieldy. Consider ibiblio.org, if every artifact was associated with an organization (apache.org - http://http://www.apache.org/dist/java-repository/), then you could (optionally) configure your client to only download an artifact from an authoritative server for that particular artifact. You could use some file like resolution.xml to just provide pointers to ASF managed resources - say a version of the ASF maven repository at one of the mirrors. This way there is never a need to create one centralized conglomeration of artifacts. You could have many maven repositories all containing only a portion of the entire artifact space. I agree with this. I was just thinking of another way to do it. You see, directing users to repositories with diversely licensed jars, is not something that is particularly nice, as you put yourself in a position of being liable. If instead the user has to enter the URL of the repository that contains an artifact, this doesn't happen. Also, if the central DNS goes down the system would halt It would also still force anybody that wants to have something published to have an entry in that DNS. But still, this is becoming OT for the purpose of this discussion. -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) -
RE: Maven and repository@apache.org
IMO we should [make] the Maven artifact handling better and used by all Apache and non-Apache projects. Is Maven willing to provide suitable support for Ant to use it? I just want to make sure that this is not the Maven repository, but is THE repository. Tool-agnostic. --- Noel
Re: Maven and repository@apache.org
On Wed, 5 Jan 2005 14:24:13 -0500, Noel J. Bergman [EMAIL PROTECTED] wrote: Is Maven willing to provide suitable support for Ant to use it? I just want to make sure that this is not the Maven repository, but is THE repository. Tool-agnostic. Yes - I've already discussed this and Steve and Russell seem prepared to integrate it into their libraries task(s). There shouldn't need to be anything special required for Ant to use it - its a plain Java API. Cheers, Brett
