Re: cool uris don't change, don't they ?
With the amount of versioning going on, eventually a release falls into a state of non-usage, I suspect there should be room for such a mechanism, otherwise mirrors will become bloated with unused, outdated, antiquated and obsolete content. I suspect some sort of "redirect" mechanism would be sufficient in cases where an unmirrored archive is used. Something that most web servers support (for instance Apache Httpd and .htaccess files) Existing Example: archives.apache.org represents content from www.apache.org/dist that has been "decommissioned" from the mirroring process, of course mirrors may maintain copies of these files by not deleting contents. Ideally, such a mechanism could even be automated based on historical download information on the resource. I.E. if the resource hasn't been downloaded in 5 years, move it into an archive and provide a redirect or notice. -Mark Nick Chalko wrote: Patrick Chanezon wrote: Did you specify a lifecycle for artifacts, with some durations, and a process to decommision them ? Good question. This may be something to put to the board. My general thought are. "Released" version should live forever, unless a security or other fatal flaw is found in a release. As a minimum I think the latest version of each point release should be kept ie 1.2.x. R, Nick -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: duplicate data
Th issue is... the jars/distributables are placed into the java-repository using maven. so, currently, if you look in something like the commons project.properties you'll see that they are pointing to the central repository for the location of where to "publish" files. ## # Apache Central Repository ## maven.repo.central=www.apache.org maven.repo.central.directory=/www/www.apache.org/dist/java-repository maven.remote.group=apcvs To publish using Maven, the only logical way I can see to resolve this discrepancy is to symlink from the projects dir itself, something like .../java-repository/ ---> /dist//**/... The "convergence issues" we currently have for the repository: 1.) We want single copies of files on the mirrors. 2.) we want the repository to reflect the hierarchical structure of our projects/subprojects. 3.) Many projects already have groupId/projectIds in maven that do not match the hierarchical nature of the the projects at apache. Its very difficult to actually move these at this time due to dependency issues. for instance /java-repository/commons-collections I can't identify if this is jakarta, xml or apache commons...(we all know from experience its jakarta commons collections. So the issue is, which direction is the convergence going to go. I'd personally like to see the day when /www/www.apache.org/dist is the repository location and the maven project ids are something like: jakarta/commons/collections///... xml/commons/resolver///... commons/...///... avalon/...///... which would match fairly well the directory structure of the dist directory, the major changes would then be /source /binary would be replaced with ///-. or whatever we can agree upon finally for the repository url structure (One thats not "theoretical", but actually "used" by tools)... So the point is, yes, we want to resolve the replication issues. My best conclusion is A.) Currently just keep "jars" in the java-repository, do not keep them in your /dist// directory. Remove all distributions form the java-repository. Currently, I haven't seen much use in actually releasing tarballs/zips into the Maven repository, others may have better opinions. B.) If a project really wants to publish both jars and tar/zip distributions to the same location. Then have them have them symlnk the appropriate java-repository dir into their appropriate "dist" directory. C.) Discussion about how to finalize the directory structure such that "Repository", "Dist/Mirror" and "Maven" has to move forward. Refactoring steps and interim solutions have to be discussed and thought about. This will probably make for a great weekend discussion, -Mark Noel J. Bergman wrote: Mark, An issue was raised earlier today that should be addressed. The impression is that java-repository is publishing copies of jars that are also under dist/TLP/..., which puts more of a load on the mirrors. It might be best if the jars/ directories contained symlinks and not copies of artifacts published elsewhere. This doesn't address the fact that at some point an artifact may exist only in the archives, but that would require meta-data aware clients. --- Noel -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: duplicate data
Noel J. Bergman wrote: The issue is... the jars/distributables are placed into the java-repository using maven. so, currently, if you look in something like the commons project.properties you'll see that they are pointing to the central repository for the location of where to "publish" files. The "convergence issues" we currently have for the repository: 1.) We want single copies of files on the mirrors. +1 My best conclusion is keep "jars" in the java-repository, do not keep them in your /dist// directory. Remove all [jar/zip/tar files] from the java-repository. symlnk the appropriate java-repository dir into their appropriate "dist" directory. That sounds OK to me, but folks like Sander and others more involved in mirroring should be put in the loop. Everything we put under dist/ effects 100s of mirrors. Yes, I learned that the hard way when we created the contents of java-repository... that was not a happy weekend. I don't make any "rash" changes to dist any more...Only well thought out moves. But we are in a state of cleanup now as well, we have to consider what we are going to do next. Discussion about how to finalize the directory structure such that "Repository", "Dist/Mirror" and "Maven" has to move forward. That would be good. In our last discussion, I think one of the conclusions that was arrived at as well, was the idea of breaking the java-repository up into two different locations. www/cvs.apache.org/dist/java-repository --> nightly builds www/www.apache.org/dist/java-repository --> official releases. the idea was that nightly/weekly builds are not things we want to see on mirrors but to be available for developers. And that official release of jars are things we want to see mirrored. When it comes to things like the ibiblio maven repository, it would only maintain full version releases of apache projects. If your an apache project and need to be on the bleeding edge for a component, then you can simply add www/cvs.apache.org/dist/java-repository as your first repository location and get your apache jars straight off the nightly builds... The big question is how to facilitate this a build process, I think the last decision on the Jakarta Commons/General/Maven lists was that we would automate the build process for releasing the nightly jars into www/cvs.apache.org/dist/java-repository And the only publishing of jars by actual humans (Release Managers) would be the full releases onto www/www.apache.org/dist/java-repository I believe this sort of approach would be inline with the policies of the dist/mirrors. -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: duplicate data
I'll try to expand on the functionalities of Maven below. Sander Striker wrote: On Sat, 2004-02-21 at 01:01, Mark R. Diggory wrote: Noel J. Bergman wrote: The issue is... the jars/distributables are placed into the java-repository using maven. Can you explain this a bit? I thought Maven was used to fetch projects and dependencies. Ofcourse I can read up on Maven, but a quick summary of the technicalities would be appreciated. Maven is used to both fetch jars from the repository and to publish the jars to the repository. In regards to the latter, it does this basically through ssh sessions where it completes a number of commands (scp, md5, chmod, chgrp). Because its encapsulated within maven the user can rely on Mavens deployement mechanism to setup the jar/signature in the repository for their project, since its scripted, it is done the same way every time. This takes a great deal of the effort invovled with publishing jars to the repository out of the users hands. Maven is really doing nothing more than acting as an ssh client for the user and automating the deployment process for them using their apache account. This benefits Maven because it can rely on the repository being maintained in a structure it can predict and locate dependencies within. so, currently, if you look in something like the commons project.properties you'll see that they are pointing to the central repository for the location of where to "publish" files. The "convergence issues" we currently have for the repository: 1.) We want single copies of files on the mirrors. +1 This is the core point. Yes, we all agree on this one... My best conclusion is keep "jars" in the java-repository, do not keep them in your /dist// directory. Remove all [jar/zip/tar files] from the java-repository. symlnk the appropriate java-repository dir into their appropriate "dist" directory. That would mean that this entire area would have to be rw to all groups producing releases that are to be in there. This kindof means apcvs group ownership, which I don't really fancy doing. The other way around, control and access of each projects dist/ area seperated, and symlinking to that from java-repository, seems a bit sa[fn]er to me. Ultimately we are seeking a convergence here between what the repository folks want to see, the maven users want to see and the infrastructure folks want to see. 1.) For the repository (and Maven) folks, we want to see the contents of dist become standardized according to the Repository URI specification. This means "all" distributables (java or not) are organized according to this specification. 2.) For Maven users, no matter what happens, we need to maintain a functionally working repository the works with the existing version of Maven. 3.) For Infrastructure, all this needs to be properly secured and maintained according to Apache standards. The java-repository structure is broken down into .../java-repository/// this would mean each project would need to maintain a separate set of symlinks for "jars", "distributables", "...". That sounds OK to me, but folks like Sander and others more involved in mirroring should be put in the loop. Everything we put under dist/ effects 100s of mirrors. Not me specifically, but Infrastructure. Others are more actively maintaining the mirrors list and monitoring the mirrors. The mirrors are a precious resource and we want to be careful not to 'scare' any mirrors away with actions on our end. Yes, I learned that the hard way when we created the contents of java-repository... that was not a happy weekend. I don't make any "rash" changes to dist any more...Only well thought out moves. But we are in a state of cleanup now as well, we have to consider what we are going to do next. If you are making large changes to the directory structure and the majority of the files is already on the mirrors, send a mail to mirrors@, attach a shell script that moves everything around locally, and give them a heads up on when this shuffle is happening. This save a _lot_ of bandwidth. Also, when adding a lot, make sure to inform the mirrors, so they are prepared. Discussion about how to finalize the directory structure such that "Repository", "Dist/Mirror" and "Maven" has to move forward. I don't parse this, but since Noel can read it, I am probably missing context/background. Just that these groups are all focused on different aspects of the distributables in the dist directory: The Repository projects Url structure is important in standardizing and improving the dist contents into a more formal structure. The Maven project represents a working example of a tool that implements itself upon this structure. Between the dist directory maintainers and the the mirrors out there represent a "control
Re: duplicate data
Wow, I'll try to be concise! Sander Striker wrote: I understand. But it has to be fairly mature before one can deploy and recommend using it to the PMCs. Also, you can't force all the projects to use it. For this you need some way of handmaintaining 'shadow' packages in java-repository, correct? [...] Maven exists and is used by many apache projects, I am, in no way, promoting any sort of "requirement" that projects be forced to use it. I am primarily concerned with maintaining the content properly for the projects that do use it and in assuring that the Apache Repository structure be convergent and that future versions of Maven be able to support it. I approach this because I am and will be a user of both, not as a developer of one or the other. Currently the duplication is minimal because projects that distribute actual jars are already using maven specifically, otherwise they package into archives (zip/tar). That would mean that this entire area would have to be rw to all groups producing releases that are to be in there. This kindof means apcvs group ownership, which I don't really fancy doing. The other way around, control and access of each projects dist/ area seperated, and symlinking to that from java-repository, seems a bit sa[fn]er to me. Ultimately we are seeking a convergence here between what the repository folks want to see, the maven users want to see and the infrastructure folks want to see. 1.) For the repository (and Maven) folks, we want to see the contents of dist become standardized according to the Repository URI specification. This means "all" distributables (java or not) are organized according to this specification. But this is fairly utopic at this point, no? Is the Repository URI spec stable? Is the tool mature? I may only be able to speak only for myself at this point. The spec is actually very "independent" of the clients/tools that may make use of it. Using Maven as an example, the tool is becoming mature, not in that it has officially gone 1.0, but that the usage (especially for java projects at apache) is becoming very prevalent. It represents a popular base of users that need accessibility to the repository. It may be utopic in that I/We are working to unify disparate groups and existing resources into a standardized directory structure. But ultimately such a goal can only benefit Apache as a whole. 2.) For Maven users, no matter what happens, we need to maintain a functionally working repository the works with the existing version of Maven. Isn't the repository format versioned? Can Maven advise the user to upgrade? Yes, the user can upgrade, Maven can release new versions. But overall, on the developer side, I and others may represent a "lobbying" force in the direction that the various repository formats Maven may/will be able to support. But I digress, the point of this statement is that there is a currently working Maven Repository with jars that people do use and which I am working to keep functioning while we evolve "java-repository" through its various refactorings. This is obvious to me because, for example, in my placing the bad (non-rsyncable) symlinks in java-repository, reports started poping up on the Maven IRC channel that access attempts on the symlinks were throwing 404's on ibiblio. SO there is a flow of content here. 3.) For Infrastructure, all this needs to be properly secured and maintained according to Apache standards. Yes. I need allot of help here, if I'm not doing something up to par, I need to hear about it. The help you've provided in this area is greatly appreciated. The java-repository structure is broken down into .../java-repository/// this would mean each project would need to maintain a separate set of symlinks for "jars", "distributables", "...". I'm assuming you are stating this as fact, correct? Yes, its just the way its implemented at this point. Not necessarily the way it will be in the future. Given a 'regular' release in .../dist//, would Maven be able to automate the creation of the symlinks? It already automates allot of symlinks. Symlinks provide a way to "resolve" what is called a SNAPSHOT to a existing versioned release. Only when someone releases a build that is not intrinsically different from the last, ie only differing in file name, do we currently get duplicates. This is something that would/should be automated to test the directory contents and symlink properly if the md5's are identical. I've been working on simple ant scripts to do this. ... Between the dist directory maintainers and the the mirrors out there represent a "control" on the whole situation, if it doesn't work for them, then its not realistic as a strategy. I'm assuming that you mean that Maven is the tool implementing the "control"? No, I mean "control" as in "test group". the whole "dist/mirrors" efforts represent the group that is a "control" in that if we create a directory archite
Re: adding jaxme jars
I don't think we're referring to it as the official "Apache Repository". It is a Maven repository located on www.apache.org /www/www.apache.org/dist/java-repository is the location to get things into maven (it is rsynced with the maven repository at ibiblio. -Mark robert burrell donkin wrote: i have a jelly tag library that uses jaxme (the apache JAXB implementation currently under incubation) that i'd like to commit into jakarta-commons. AIUI the right way to make ASF jars available for use in mavenized builds is to upload them into the apache repository. is this correct and is there any documentation about the right way to do this? - robert -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
I can setup this if you point me at the jars, however, here are the steps if you want to explore it yourself right now: --- You need to establish what your groups id will be. The current naming trend for Apache "xml" projects has been a bit haphazard. xercies xml-security xml-resolver xalan Some standardization here would be good. But that will not effect maven. There is currently a problem with Maven publishing to a BSD system and md5 signature generation, if you do not want to use maven to do the publishing then these are the tasks you need to complete: 1.) The easiest way right now is for you to place the jars into java-repository/xml-jaxme/jars/... 2.) Generate an md5 signature for each one in the same directory. For instance as an examples "md5 xml-jaxme-1.0.jar > xml-jaxme-1.0.jar.md5" will produce: java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar.md5 3.) Generate the following symlinks the above files from java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar.md5 4.) and create a file java-repository/xml-jaxme/jars/xml-jaxme-snapshot-version with the contents "1.0" 5.) use the following "POM" in the java-repository/xml-jaxme/poms/project/pom 3 xml-jaxme xml-jaxme 1.0 Apache Software Foundation www.apache.org ASL 2.0 ??? ??? -Mark robert burrell donkin wrote: (i don't think that the raw jars are distributed at the moment so symlinking is out of the question.) since it's a maven repository, are there any guidelines about the right way to put stuff into there? what about the artifact id/group id stuff? - robert On 6 Mar 2004, at 23:20, Sander Striker wrote: On Sat, 2004-03-06 at 23:43, Mark R. Diggory wrote: I don't think we're referring to it as the official "Apache Repository". It is a Maven repository located on www.apache.org /www/www.apache.org/dist/java-repository is the location to get things into maven (it is rsynced with the maven repository at ibiblio. And, reminder, all the mirrors listed here: http://www.apache.org/mirrors/ If you have jaxme jars elsewhere in dist/, please consider symlinking. Sander -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: duplicate data
Sorry for the later response, currently, I think the major issues are in managing the content of java-repository in responsible manner. Key issues I can see needing to be addressed are the following. 1.) Get projects to be as "responsible" for their content in java-repository as they are for the content of their "project" directories in dist. 2.) Resolve the only duplication left which has to do with the fact that avalon runs their own "Avalon Repository" in /dist/avalon. So its contents are currently duplicated with java-repository/ibiblio. 3.) Maintaining proper permissions on all the directory contents of the repository (group write, group ownership of files and new directories should be the users primary group) 4.) Find a way around the current "shortsightedness" in Maven where the command executed on the serverside are all "gnu linux" and do not map to BSD, md5 = md5sum. So using the "repository" goals in maven fails to produce proper md5 checksums. Nicola Ken Barozzi wrote: Some action items: 1- how can we make mirroring work for both of us? (IIRC Ruper already showed it's easy to do, but I need help from Adam that already tried it) 2- How does Mark think we can proceed in not making it compulsory to phisically have jars in a defined location? These two are a "double-whammy", but I think I have a possible solution to the whole subject. Currently we think of the "Repository" as just that "A Repository", a physical location for the jars. But what if we defined the URL's for a "Repository" to simple be pointers or addresses that when resolved by a client, point to the proper location of that resource. This in essance, makes the repository into a resolver or a naming service. In my line of work (Digital Libraries) we already have a service that accomplishes this task (actually we have 2 competing/complimentary naming systems) PURLS (Persistent Uniform Resource Identifier) http://www.purl.org http://www.oclc.org/research/projects/purl/download.htm Handles (analogous to publicId's, ISBN's, Dewey Decimal System,, etc...) http://www.handle.net/ http://www.handle.net/download.html If your wondering how PURLs and Handles stack up, heres some comparison documentation. http://www.nclis.gov/govt/assess/handles.html http://memory.loc.gov/ammem/award/docs/PURL-handle.html http://web.mit.edu/handle/www/purl-eval.html So what we are talking about really is a naming system that provides for the "resolution" of registered names to physical locations. Interesting ly, I think the lack of this sort of separation of "Resolution" from "Storage" is exactly the issue that is causing "friction" in our community. I think its quite possible, that one could completely and transparently replace the underlying URL based repository syntax in both Maven and other tools with a resolving layer. to clarify this, heres a few examples. 1.) an example using PURL's. http://www.ibiblio.org/maven/xerces/jars/xerces.jar this is currently a URL pointing to physical resource on ibiblio. if this were not a physical resource but a PURL in the PURL naming system, then it could (redirect using currently existing PURL server software) the client to the appropriate resource (mirrored or not). http://repository.apache.org/maven/xerces/jars/xerces.jar would actually resolve (through redirection to) http://www.apache.org/dist/xml/xerces-j/jars/xerces.jar and http://repository.apache.org/xml.apache.org/xerces/2.0/jars/xerces.jar could also point to http://www.apache.org/dist/xml/xerces-j/jars/xerces.jar This provides a layer of flexiblity, its solves issues with both the projects needing to place their content in a specific structure/location and it also solves issues of name changes over time, 2.) So if we decide that we want to have different "groupID's" in maven for a specific project, the naming system maintains the old naming structure pointing to the jars as a means for dependent projects to still be able to resolve to the resource. http://www.ibiblio.org/maven/commons-collections/jars/commons-collections-1.0.jar we are currently planning to adopt a more hierarchical naming approach http://www.ibiblio.org/maven/jakarta.apache.org/commons-collections/jars/commons-collections-1.0.jar We could (at little cost in both maintenance and diskspace) , maintain the old naming resolution and the new one. In fact, this is the very foundation of the PURL system, the old uri's stay persistent over time. 3.) With such a level of "redirection", we can also maintain archival and production releases of the content without the actual location specifier changing. So when Apache retires commons-collection-1.0.jar from production and removes it from the mirrors, instead placing it onto archives.apache.org, then that resolver entry in the PURL database can be adjusted to point at the new location> http://repository.apache.org/maven/commons-collections/jars/commons-collections-1.0.jar now points to the following
Re: adding jaxme jars
robert burrell donkin wrote: hi mark i've given it a go myself but i have a few questions and i'd appreciate it if you'd check my work so far. On 7 Mar 2004, at 16:18, Mark R. Diggory wrote: You need to establish what your groups id will be. The current naming trend for Apache "xml" projects has been a bit haphazard. xercies xml-security xml-resolver xalan Some standardization here would be good. But that will not effect maven. i've had a chat on IRC and gone for apache-jaxme There is currently a problem with Maven publishing to a BSD system and md5 signature generation, if you do not want to use maven to do the publishing then these are the tasks you need to complete: 1.) The easiest way right now is for you to place the jars into java-repository/xml-jaxme/jars/... gone (but apache-jaxme rather than xml-jaxme) 2.) Generate an md5 signature for each one in the same directory. For instance as an examples "md5 xml-jaxme-1.0.jar > xml-jaxme-1.0.jar.md5" will produce: java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar.md5 done 3.) Generate the following symlinks the above files from java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar.md5 done 4.) and create a file java-repository/xml-jaxme/jars/xml-jaxme-snapshot-version with the contents "1.0" these releases are versioned 0.2: should this file contain 0.2 (since it refers to the version of the jars) or 1.0 (since it's a mavenesque version number) It would be wise, so yes, apache-jaxme-0.2.jar 5.) use the following "POM" in the java-repository/xml-jaxme/poms/project/pom 3 xml-jaxme xml-jaxme 1.0 Apache Software Foundation www.apache.org ASL 2.0 ??? ??? there are a number of jar in the release. i'd been working on the theory that each needs a separate artifact id. what should the contents of the file be in this case? - robert Yes, it should look like: /java-repository/apache-jaxme/jars/-.jar /java-repository/apache-jaxme/poms/project/pom 3 apache-jaxme Apache Jaxme 0.2 Apache Software Foundation www.apache.org ASL 2.0 http://www.apache.org/licenses/LICENSE-2.0 sound good, I'll take a look in the directory and verify all looks well. -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
More likely the mirroring process is still being problematic. I'll grill Jason for a few minutes and get things updated. -Mark robert burrell donkin wrote: hi mark i can see the jaxme jars on apache but not in the maven repository on ibiblio. is there anything else i need to do to push the content over there? (or have i just make some mistake in the upload...) - robert On 8 Mar 2004, at 22:20, Mark R. Diggory wrote: robert burrell donkin wrote: hi mark i've given it a go myself but i have a few questions and i'd appreciate it if you'd check my work so far. On 7 Mar 2004, at 16:18, Mark R. Diggory wrote: You need to establish what your groups id will be. The current naming trend for Apache "xml" projects has been a bit haphazard. xercies xml-security xml-resolver xalan Some standardization here would be good. But that will not effect maven. i've had a chat on IRC and gone for apache-jaxme There is currently a problem with Maven publishing to a BSD system and md5 signature generation, if you do not want to use maven to do the publishing then these are the tasks you need to complete: 1.) The easiest way right now is for you to place the jars into java-repository/xml-jaxme/jars/... gone (but apache-jaxme rather than xml-jaxme) 2.) Generate an md5 signature for each one in the same directory. For instance as an examples "md5 xml-jaxme-1.0.jar > xml-jaxme-1.0.jar.md5" will produce: java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar.md5 done 3.) Generate the following symlinks the above files from java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar.md5 done 4.) and create a file java-repository/xml-jaxme/jars/xml-jaxme-snapshot-version with the contents "1.0" these releases are versioned 0.2: should this file contain 0.2 (since it refers to the version of the jars) or 1.0 (since it's a mavenesque version number) It would be wise, so yes, apache-jaxme-0.2.jar 5.) use the following "POM" in the java-repository/xml-jaxme/poms/project/pom 3 xml-jaxme xml-jaxme 1.0 Apache Software Foundation www.apache.org ASL 2.0 ??? ??? there are a number of jar in the release. i'd been working on the theory that each needs a separate artifact id. what should the contents of the file be in this case? - robert Yes, it should look like: /java-repository/apache-jaxme/jars/-.jar /java-repository/apache-jaxme/poms/project/pom 3 apache-jaxme Apache Jaxme 0.2 Apache Software Foundation www.apache.org ASL 2.0 http://www.apache.org/licenses/LICENSE-2.0 sound good, I'll take a look in the directory and verify all looks well. -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
Its ok, half the projects in Maven repository never deployed pom's so its usage is somewhat "speculative" in the first place. -Mark Erik Abele wrote: On 08.03.2004, at 23:20, Mark R. Diggory wrote: 3 apache-jaxme Apache Jaxme 0.2 Apache Software Foundation The official name is 'The Apache Software Foundation', as incorporated within the State of Delaware. See http://apache.org/foundation/records/certificate.html. www.apache.org ASL 2.0 It's AL 2.0 not ASL 2.0 (Apache License not Apache Software License). http://www.apache.org/licenses/LICENSE-2.0 Perhaps a bit pedantic but... :-) Cheers, Erik -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
Robert, the jars are there, we should probably do a little renaming. My fault for not looking yesterday. http://www.ibiblio.org/maven/apache-jaxme/jars/ !! http://www.ibiblio.org/maven/apache-jaxme/jars/jaxme-incubation-0.2.jar we should probably have it more simply be http://www.ibiblio.org/maven/jaxme/jars/jaxme-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-rt-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-rt-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-api-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-api-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-js-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-js-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-pm-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-pm-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-xs-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-xs-SNAPSHOT.jar -Mark robert burrell donkin wrote: hi mark i can see the jaxme jars on apache but not in the maven repository on ibiblio. is there anything else i need to do to push the content over there? (or have i just make some mistake in the upload...) - robert On 8 Mar 2004, at 22:20, Mark R. Diggory wrote: robert burrell donkin wrote: hi mark i've given it a go myself but i have a few questions and i'd appreciate it if you'd check my work so far. On 7 Mar 2004, at 16:18, Mark R. Diggory wrote: You need to establish what your groups id will be. The current naming trend for Apache "xml" projects has been a bit haphazard. xercies xml-security xml-resolver xalan Some standardization here would be good. But that will not effect maven. i've had a chat on IRC and gone for apache-jaxme There is currently a problem with Maven publishing to a BSD system and md5 signature generation, if you do not want to use maven to do the publishing then these are the tasks you need to complete: 1.) The easiest way right now is for you to place the jars into java-repository/xml-jaxme/jars/... gone (but apache-jaxme rather than xml-jaxme) 2.) Generate an md5 signature for each one in the same directory. For instance as an examples "md5 xml-jaxme-1.0.jar > xml-jaxme-1.0.jar.md5" will produce: java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar.md5 done 3.) Generate the following symlinks the above files from java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar.md5 done 4.) and create a file java-repository/xml-jaxme/jars/xml-jaxme-snapshot-version with the contents "1.0" these releases are versioned 0.2: should this file contain 0.2 (since it refers to the version of the jars) or 1.0 (since it's a mavenesque version number) It would be wise, so yes, apache-jaxme-0.2.jar 5.) use the following "POM" in the java-repository/xml-jaxme/poms/project/pom 3 xml-jaxme xml-jaxme 1.0 Apache Software Foundation www.apache.org ASL 2.0 ??? ??? there are a number of jar in the release. i'd been working on the theory that each needs a separate artifact id. what should the contents of the file be in this case? - robert Yes, it should look like: /java-repository/apache-jaxme/jars/-.jar /java-repository/apache-jaxme/poms/project/pom 3 apache-jaxme Apache Jaxme 0.2 Apache Software Foundation www.apache.org ASL 2.0 http://www.apache.org/licenses/LICENSE-2.0 sound good, I'll take a look in the directory and verify all looks well. -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
I've started this process http://www.apache.org/dist/java-repository/jaxme/jars/ Mark R. Diggory wrote: Robert, the jars are there, we should probably do a little renaming. My fault for not looking yesterday. http://www.ibiblio.org/maven/apache-jaxme/jars/ !! http://www.ibiblio.org/maven/apache-jaxme/jars/jaxme-incubation-0.2.jar we should probably have it more simply be http://www.ibiblio.org/maven/jaxme/jars/jaxme-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-rt-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-rt-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-api-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-api-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-js-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-js-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-pm-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-pm-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-xs-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-xs-SNAPSHOT.jar -Mark robert burrell donkin wrote: hi mark i can see the jaxme jars on apache but not in the maven repository on ibiblio. is there anything else i need to do to push the content over there? (or have i just make some mistake in the upload...) - robert On 8 Mar 2004, at 22:20, Mark R. Diggory wrote: robert burrell donkin wrote: hi mark i've given it a go myself but i have a few questions and i'd appreciate it if you'd check my work so far. On 7 Mar 2004, at 16:18, Mark R. Diggory wrote: You need to establish what your groups id will be. The current naming trend for Apache "xml" projects has been a bit haphazard. xercies xml-security xml-resolver xalan Some standardization here would be good. But that will not effect maven. i've had a chat on IRC and gone for apache-jaxme There is currently a problem with Maven publishing to a BSD system and md5 signature generation, if you do not want to use maven to do the publishing then these are the tasks you need to complete: 1.) The easiest way right now is for you to place the jars into java-repository/xml-jaxme/jars/... gone (but apache-jaxme rather than xml-jaxme) 2.) Generate an md5 signature for each one in the same directory. For instance as an examples "md5 xml-jaxme-1.0.jar > xml-jaxme-1.0.jar.md5" will produce: java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar.md5 done 3.) Generate the following symlinks the above files from java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar.md5 done 4.) and create a file java-repository/xml-jaxme/jars/xml-jaxme-snapshot-version with the contents "1.0" these releases are versioned 0.2: should this file contain 0.2 (since it refers to the version of the jars) or 1.0 (since it's a mavenesque version number) It would be wise, so yes, apache-jaxme-0.2.jar 5.) use the following "POM" in the java-repository/xml-jaxme/poms/project/pom 3 xml-jaxme xml-jaxme 1.0 Apache Software Foundation www.apache.org ASL 2.0 ??? ??? there are a number of jar in the release. i'd been working on the theory that each needs a separate artifact id. what should the contents of the file be in this case? - robert Yes, it should look like: /java-repository/apache-jaxme/jars/-.jar /java-repository/apache-jaxme/poms/project/pom 3 apache-jaxme Apache Jaxme 0.2 Apache Software Foundation www.apache.org ASL 2.0 http://www.apache.org/licenses/LICENSE-2.0 sound good, I'll take a look in the directory and verify all looks well. -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
Robert, this process is finished in java repository http://www.apache.org/dist/java-repository/jaxme/jars/ I didn't bother maintain the pom, as it is apparent that the other projects are not using them. We should have this sync'ed shortly. -Mark Mark R. Diggory wrote: I've started this process http://www.apache.org/dist/java-repository/jaxme/jars/ Mark R. Diggory wrote: Robert, the jars are there, we should probably do a little renaming. My fault for not looking yesterday. http://www.ibiblio.org/maven/apache-jaxme/jars/ !! http://www.ibiblio.org/maven/apache-jaxme/jars/jaxme-incubation-0.2.jar we should probably have it more simply be http://www.ibiblio.org/maven/jaxme/jars/jaxme-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-rt-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-rt-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-api-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-api-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-js-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-js-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-pm-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-pm-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-xs-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-xs-SNAPSHOT.jar -Mark robert burrell donkin wrote: hi mark i can see the jaxme jars on apache but not in the maven repository on ibiblio. is there anything else i need to do to push the content over there? (or have i just make some mistake in the upload...) - robert On 8 Mar 2004, at 22:20, Mark R. Diggory wrote: robert burrell donkin wrote: hi mark i've given it a go myself but i have a few questions and i'd appreciate it if you'd check my work so far. On 7 Mar 2004, at 16:18, Mark R. Diggory wrote: You need to establish what your groups id will be. The current naming trend for Apache "xml" projects has been a bit haphazard. xercies xml-security xml-resolver xalan Some standardization here would be good. But that will not effect maven. i've had a chat on IRC and gone for apache-jaxme There is currently a problem with Maven publishing to a BSD system and md5 signature generation, if you do not want to use maven to do the publishing then these are the tasks you need to complete: 1.) The easiest way right now is for you to place the jars into java-repository/xml-jaxme/jars/... gone (but apache-jaxme rather than xml-jaxme) 2.) Generate an md5 signature for each one in the same directory. For instance as an examples "md5 xml-jaxme-1.0.jar > xml-jaxme-1.0.jar.md5" will produce: java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar java-repository/xml-jaxme/jars/xml-jaxme-1.0.jar.md5 done 3.) Generate the following symlinks the above files from java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar java-repository/xml-jaxme/jars/xml-jaxme-SNAPSHOT.jar.md5 done 4.) and create a file java-repository/xml-jaxme/jars/xml-jaxme-snapshot-version with the contents "1.0" these releases are versioned 0.2: should this file contain 0.2 (since it refers to the version of the jars) or 1.0 (since it's a mavenesque version number) It would be wise, so yes, apache-jaxme-0.2.jar 5.) use the following "POM" in the java-repository/xml-jaxme/poms/project/pom 3 xml-jaxme xml-jaxme 1.0 Apache Software Foundation www.apache.org ASL 2.0 ??? ??? there are a number of jar in the release. i'd been working on the theory that each needs a separate artifact id. what should the contents of the file be in this case? - robert Yes, it should look like: /java-repository/apache-jaxme/jars/-.jar /java-repository/apache-jaxme/poms/project/pom 3 apache-jaxme Apache Jaxme 0.2 Apache Software Foundation www.apache.org ASL 2.0 http://www.apache.org/licenses/LICENSE-2.0 sound good, I'll take a look in the directory and verify all looks well. -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
Basically, in Maven the difference between "version releases" and "build releases" is designated in the version, not in the artifact id. /jaxme/jars/jaxme-.jar and /jaxme/jars/jaxme-.jar It is important to have your artifactId's match across all versions of an artifact, otherwise they aren't the same "artifact", so /jaxme/jars/jaxme-incubator-x.x.jar does not share the same artifact id as /jaxme/jars/jaxme-SNAPSHOT.jar to have them match it would have to be /jaxme/jars/jaxme-incubator-SNAPSHOT.jar So, if your going to be distributing a "build release" I recommend the naming strategy where all artifactID's match. If the jaxme project is planning to leave the incubator anytime in the future, I would highly recommend you not use the word "incubator" in the artifact id. My personal opinion added to this is that its the same way we don't use "jakarta" in in projects like the Jakarta Commons, because some aspects Jakarta is a Foundry. Incubator is also a Foundry. My future planned strategy to support "build-releases" is to have build releases (daily, weekly builds etc) not be present on: /www/www.apache.org/dist/java-repository but to actually be present on: /www/cvs.apache.org/builds/java-repository Then, to actually use this approach with Maven you'd have to add the cvs.apache.org repository to your Maven client, this makes project dependencies hairy in the Current release of Maven because the contents wouldn't be present at ibiblio. We still have much to decide to do and infrastructure to setup to have this build location. My current strategy in this case is to have the build-releases in the same project directory as the version releases, then they are all rsynced with ibiblio properly. This is simply because build releases already existed in the repository when we moved the apache contents of ibiblio back to Apache. What is the status of jaxme in the incubator? When do they anticipate an official release? Maybe this is a reason enough to push forward the development of the daily/weekly/... build repository. Allowing any more build-releasing in the dist/java-repository is a sensitive issue when it comes to mirroring. It would be wise for us to stop doing this asap. This is probably a good location where I could use assistance from repository folks in working out these details. -Mark robert burrell donkin wrote: hi mark the release in question was made whilst jaxme was in the incubator. AIUI it's an important requirement that it's clear that these are not official ASF releases but incubation releases. the most direct way to do this seemed (to me) to be continue to use the naming convention used by incubation releases. - robert On 11 Mar 2004, at 00:18, Mark R. Diggory wrote: Robert, the jars are there, we should probably do a little renaming. My fault for not looking yesterday. http://www.ibiblio.org/maven/apache-jaxme/jars/ !! http://www.ibiblio.org/maven/apache-jaxme/jars/jaxme-incubation-0.2.jar we should probably have it more simply be http://www.ibiblio.org/maven/jaxme/jars/jaxme-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-rt-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-rt-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-api-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-api-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-js-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-js-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-pm-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-pm-SNAPSHOT.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-xs-0.2.jar http://www.ibiblio.org/maven/jaxme/jars/jaxme-xs-SNAPSHOT.jar -Mark robert burrell donkin wrote: hi mark i can see the jaxme jars on apache but not in the maven repository on ibiblio. is there anything else i need to do to push the content over there? (or have i just make some mistake in the upload...) - robert On 8 Mar 2004, at 22:20, Mark R. Diggory wrote: robert burrell donkin wrote: hi mark i've given it a go myself but i have a few questions and i'd appreciate it if you'd check my work so far. On 7 Mar 2004, at 16:18, Mark R. Diggory wrote: You need to establish what your groups id will be. The current naming trend for Apache "xml" projects has been a bit haphazard. xercies xml-security xml-resolver xalan Some standardization here would be good. But that will not effect maven. i've had a chat on IRC and gone for apache-jaxme There is currently a problem with Maven publishing to a BSD system and md5 signature generation, if you do not want to use maven to do the publishing then these are the tasks you need to complete: 1.) The easiest way right now is for you to place the jars i
Re: adding jaxme jars
Sander Striker wrote: From: Mark R. Diggory [mailto:[EMAIL PROTECTED] Sent: Thursday, March 11, 2004 7:52 PM Basically, in Maven the difference between "version releases" and "build releases" is designated in the version, not in the artifact id. /jaxme/jars/jaxme-.jar and /jaxme/jars/jaxme-.jar It is important to have your artifactId's match across all versions of an artifact, otherwise they aren't the same "artifact", so /jaxme/jars/jaxme-incubator-x.x.jar does not share the same artifact id as /jaxme/jars/jaxme-SNAPSHOT.jar to have them match it would have to be /jaxme/jars/jaxme-incubator-SNAPSHOT.jar Hmm, couldn't the filename and artifact id be decoupled? I'm not suggesting rightness or wrongness of the approach in the following comments. Simply just what Maven currently does: This naming structure is important only in terms of the existing Maven Repository Design (probably less likely in future Maven2 designs). The repository structure for Maven is currently rather strict in this regard ///-. One question: How would you then "lookup" that file by only "group", "artifact" and "version"? You'd need to know how to generate its "filename" (ie in this case:-.). When ever I think about this, I always come full circle that the only "flexible" solution to this sort of issue is not in the URL "syntax" but in the "resolution" mechanism. What we are essentially doing is trying to establish an "Identifier" for a resource that can be "resolved" to that resource. I'm really getting more and more of the opinion that what Apache/Maven/Depot etc. needs more than a "Repository" structure is a "resolving service". A service very much similar in structure to that which "Public Id's" in XML/SGML or DOI's/Handles/Purl's in the Library community. Then a Project is simply assigned a Public Id Namespace in which to "identify" its resources, Then resolver/proxy services can be used to map that "id" to an actual URL location of the file. A url location that can be "anywhere" and of "any format". Then the "structure" of these directories become loose and simply just for managing a projects released files again. http://www.handle.net http://www.purl.org http://www.doi.org http://www.dspace.org Think about it, This gives us a layer of "flexibility" in resolution which is "decoupled" from the layer of storage. As such files can move around and files can be located in multiple locations (mirrors). The resolving mechanism handles the persistence of the id. This way, if (for instance) a jar file is "demoted" from public distribution (aka www.apache.org/dist) on the mirrors, its id simply is pointed at the archived version instead of the mirrored version. And, if a file is mirrored, the id can be resolved using a mechanism similar to "/dyn/closer.cgi" to resolve to a distribution mirror for that resource instead of the Apache server. So, if your going to be distributing a "build release" I recommend the naming strategy where all artifactID's match. If the jaxme project is planning to leave the incubator anytime in the future, I would highly recommend you not use the word "incubator" in the artifact id. My personal opinion added to this is that its the same way we don't use "jakarta" in in projects like the Jakarta Commons, because some aspects Jakarta is a Foundry. Incubator is also a Foundry. Incubator is a bit of a different beast than Jakarta. Projects in the Incubator are not 'full fledged ASF projects' yet, they will be as soon as they leave in Incubator. The incubation designation has come up on the [EMAIL PROTECTED] list. If this is problematic for any reason I suggest you bring that up there. Sounds similar to the Jakarta Commons Sandbox, Actually, projects there are not allowed to release jars until they graduate to the Jakarta Commons Proper. Generally speaking, jars produce by such projects are not distributed anywhere but by the author on their local system. Yet, I believe that Jakarta does complete a daily build process for these projects as well, so even though they are not officially released, there are packages which can be gotten at on cvs.apache.org. My future planned strategy to support "build-releases" is to have build releases (daily, weekly builds etc) not be present on: /www/www.apache.org/dist/java-repository but to actually be present on: /www/cvs.apache.org/builds/java-repository Which is where they belong. Snapshots don't ever belong on www.apache.org/dist/. Absolutely Correct. Then, to actually use this approach with Maven you'd have to add the cvs.apache.org repository to your Mav
Re: adding jaxme jars
Is there just an issue with where its going to be located then? (I.E. site, cvs/svn location and distribution directories)? I'm very unfamiliar with the Incubator and its policies so excuse my naivety. Once a project has graduated via Vote, is there a process for its "migration"? I think this was possibly Roberts point. -Mark Noel J. Bergman wrote: jaxme would have been out of incubation by now but there are some major hassles with procedure. No, there was a vote taken. It was then redone a day later in a public forum. Hardly a major hassle, nor a significant delay. --- Noel -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
I have to get back to Jason, he was getting me an Ibilio account today. I'll clean it up. -Mark Ben Walding wrote: One side effect of changing groupIds around after they have been mirrored is that ibiblio's maven repo now has jaxme stuff slotted into both /jaxme and /apache-jaxme. Once someone has decided which way it should be, we'll need to remove the old cruft. Mark, I leave this chore in your hands! Cheers, Ben robert burrell donkin wrote: i have a jelly tag library that uses jaxme (the apache JAXB implementation currently under incubation) that i'd like to commit into jakarta-commons. AIUI the right way to make ASF jars available for use in mavenized builds is to upload them into the apache repository. is this correct and is there any documentation about the right way to do this? - robert -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: adding jaxme jars
Noel J. Bergman wrote: sander's post early on this thread (quite reasonably) suggested that it'd be a good idea for the repository team to think about how to handle incubating projects in general (before any more come along). i think that sounds like a very reasonable plan. Agreed. And it would be helpful if the Repository team was aware, as the rest of infrastructure is, that releases must be voted on by a PMC. If there is a request for publication, it should be cc'd to the PMC and made sure that the PMC doesn't raise a flag. That's the same as we do when creating new accounts, lists, etc. Yes, I probably have been a little ambitious in placing content in the repository. I will yield 100% to whatever policies we invoke concerning publishing of jar's in accordance with the PMC's. 1.) I think we should really have an effort now set though the next level of cleanup --> moving the build-versions that are in the repository off into cvs.apache.org 2.) It's next to impossible to enforce a directory structure/naming policy with the contents of the directories writable (let alone the fact that its an actual directory structure). Even those intimate with the project and effort end up doing things they consider minor which in turn effect the entire community adversely. One factor, unique to the Incubator, is that we have a concept known as a PPMC, which consists of the PMC and others. For a release, we should make sure that the both the PPMC and PMC mailing lists are cc'd. Yes, once a system is established. I think this should be a process which the Release Manager for a project and their PMC complete. We should eventually setup a policy whereby no one from the Repository group actually "Publishes Releases" and instead plays a more "advisory" and "maintenance" role in terms of the repository. it also seems to me (at least) that it's really the incubator pmc who are the right people to ask about the management of releases created whilst under incubation (including any special measures that might be required for jars released whilst in incubation). Absolutely. However, in this specific case, JaxMe has been voted to the WS PMC, who will be responsible for it. --- Noel I've contacted the WS group more formally to discuss this on the general@ws.apache.org mailing list. -Mark -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
cvs.apache.org SNAPSHOT repository (was: Re: [vote] Nominate Ibiblio Content Manager)
I recommend we use the following location for SNAPSHOT/NIGHTLY/WEEKLY builds of jars for Apache Projects. http://cvs.apache.org/repository/ -Mark Mark R. Diggory wrote: I don't believe I'm on the directory developers list, sorry. There is absolutely no such "Requirement", You can configure your client to point at any "repository location" for downloading artifacts (ibiblio or not). However, there is no "distributed repository" lookup mechanism in Maven whatsoever, the default configuration points only at ibiblio. So, if you want to publicize your jars for use by the world, then the greatest exposure is via ibiblio at the moment because it is the default. Noel J. Bergman wrote: Are you saying that in order for someone to use Maven to build the code, they need access to project artifacts published on ibiblio? If you are saying that Maven forces Incubator projects to make public publication of code, that raises some concern. Mark, is it not possible for us to host artifacts for maven on cvs.apache.org somewhere so that they are not put out in the public perception as "released" code? Yes, this is something we are in the process of setting up. Can you work with us so that Incubator projects can use Maven without having to push the boundaries? Does something like "cvs.apache.org/incubator-repository" make sense? Please advise. :-) Sort of, we need a more generic location for all apache projects cvs.apache.org/builds/repository/ Ideally, in Jakarta-Commons for instance, there are ambitions to see a nightly/weekly automated build process actually generate such content for this repository. Since projects don't stay in the incubator forever, I suggest you don't use it in the groupId for a incubator project. This way it can be migrated more easily to www.apache.dist/java-repository without modification of the groupId. cvs.apache.org/builds/repository/jaxme/jars/... cvs.apache.org/builds/repository/directory/jars/... ... -Mark --- Noel -Original Message- From: Alex Karasulu [mailto:[EMAIL PROTECTED] Sent: Saturday, March 13, 2004 12:55 To: 'Apache Directory Developers List' Subject: RE: [vote] Nominate Ibiblio Content Manager This is not for any release but for snapshots to allow the build to happen. -Original Message- From: Noel J. Bergman [mailto:[EMAIL PROTECTED] Sent: Saturday, March 13, 2004 12:32 PM To: Apache Directory Developers List Subject: RE: [vote] Nominate Ibiblio Content Manager I'd like to nominate myself as the Ibiblio content manager if that's ok with the group. If stuff like snapshots need to go up to Ibiblio I'll push it up on behalf of the team. If others would like the privilege instead let me know. Just remember that we should not be doing any releases while in the Incubator. Exceptions can be voted on, with suitable tagging, but as a normative thing, projects in the incubator don't put out releases. --- Noel -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: cvs.apache.org SNAPSHOT repository
Noel J. Bergman wrote: Mark R. Diggory wrote: If your using your own workstation to build using Maven, then you can set your central-repository property to point to maven.repo.central=minotaur.apache.org maven.repo.central.directory=/www/cvs.apache.org/repository/ Incubator can gladly be the first to start producing builds here. This is not mirrored, there is no concern about making changes if you screw up a deploy here. I think that last point is important for infrastructure to know. And please correct me if I am wrong, but my understanding is that the clients (maven in this case) will check that location, but only download if they need something that hasn't been cached. Therefore this shouldn't be a big bandwidth issue. Not only that, but it is also the case that only Apache developers should really be using this (with some bleed). If you look on cvs.apache.org, you'll also notice that other nightly/snapshot builds are getting dumped there and are web accessible. http://cvs.apache.org/dist/ http://cvs.apache.org/builds/ http://cvs.apache.org/snapshots/ So we want to highly disapprove of external projects using this location at all for getting hold of jars from apache projects. Someday, maybe we'll have smarter repository tools that can make use of http auth mechanisms, then this can really be enforced. -Mark -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Thoughts on CJAN
When we talk about distributed Maven repositories, ASF repositories etc that are related to "Java". I think we should consider that CPAN/CRAN etc are models of distributing such content that have been successful. Given that, I started to research the existence of a CJAN project, and of course, immediately found: http://www.cjan.org/ Ultimately, I wonder how much "larger" than Apache re such efforts as this? We should seriously consider establishing a "standards" community to establish supporting efforts such as this. CJAN been hard at work with alot of "documentation effort". What are individuals thoughts on this project and its model? http://sourceforge.net/docman/?group_id=41872 I think its important to maintain that its not impossible to attract multiple groups/organizations to such an effort. -- Mark R. Diggory Software Developer - VDC Project Harvard MIT Data Center http://www.hmdc.harvard.edu
RE: Thoughts on CJAN
On Tue, 2004-03-16 at 15:02, Alex Karasulu wrote: > > So really, who cares about CJAN. The spec that goes defacto is the one > > manifest in the mechanism used by the most people. Unless of course SUN > > steps in at some point and tries to propose a JSR which wouldn't > > surprise me at all. > > It's such an undertaking that it might be good to have a JSR around it but > that could take so long :(. > > Alex Yes, an by that time, I doubt a JSR would be of value because there would already be tools and specs in place and used by the community... I suspect a JSR would be valuable if the network existed already and we were talking about native support for resolving and downloading dependencies in the J2RE/SDK. -- Mark R. Diggory Software Developer - VDC Project Harvard MIT Data Center http://www.hmdc.harvard.edu
RE: Thoughts on CJAN
On Tue, 2004-03-16 at 14:41, Nicola Ken Barozzi wrote: > Mark R. Diggory wrote: > ... > > We should seriously consider establishing a "standards" community > > to establish supporting efforts such as this. > > This list is our "standards" community. While trying to maintain a tone of fairness I have to state that, the history of flame wars on this list really injured the productivity of the project and the involvement of "all" the parties. I don't think its possible to have a standard with only one party present at the table. All the Apache projects interested in this spec have to stay at the table, everyone has to be willing to make compromises to maintain the groups cohesion. If this list is going to be a "standards community" everyone needs to stay involved. > > CJAN been hard at work with alot of "documentation effort". What are > > individuals thoughts on this project and its model? > > That it has nothing to show, only ideas and musings. We already have > working systems, and also a possible 2.0 spec that has come out of > discussions here. > > We still have to finish to implement the 1.0 spec at Apache, and IMHO > this is what we should focus ATM. Yes, a spec is pretty much useless without actual tools that are capable of applying it, and spec isn't a "standard" unless all tools agree to use maintain the contracts defined within it. If any dominant tool goes renegade or egotistical and begins altering the contract, then...so much for a standard. On Tue, 2004-03-16 at 13:58, Jason van Zyl wrote: > On Tue, 2004-03-16 at 13:20, Alex Karasulu wrote: > > > I looked through the documentation and the proposals really are not > > much. They're one pager brain dumps of nice to haves and standards that > > are to be followed. There really is not much to cjan.org at this point > > besides the web façade and the sf.net project. > I was unsure what had actually been accomplished by CJAN, if an actual network existed or not. Unfortunately, without a substantial content base, user-ship of anything thy implement will probably not be very high anyways. > They've been around longer than a year. I contacted them a little over > two years ago to see what was up and they got rather huffy when I > suggested what Maven was trying to do. They had even less around at that > point but look at what they have now: 20 documents describing what they > want to do and they've been around for 2 years and is there anything > that incorporates their ideas in a working piece of software? No. > I guess all I can add is that its too bad that the CJAN domain got taken up by such a slow moving project... It would be an ideally suited name for the network of repositories established via the efforts of the repository project. :-( > So really, who cares about CJAN. The spec that goes defacto is the one > manifest in the mechanism used by the most people. However, by following a similar line of reasoning you could take for an analogy that IE is the predominate browser on the market, so what ever it defines as a DOM model or CSS implementation would be the standard, but yet, we all know this isn't true. The w3c standards in these areas (standards Microsoft is supposedly a member of the consortia defining) are far from being met by IE. > Unless of course SUN > steps in at some point and tries to propose a JSR which wouldn't > surprise me at all. Well, I would suspect (or at least hope) that Apache would be a driving force behind such an endeavor. -Mark -- Mark R. Diggory Software Developer - VDC Project Harvard MIT Data Center http://www.hmdc.harvard.edu
RE: Thoughts on CJAN
On Tue, 2004-03-16 at 17:20, Jason van Zyl wrote: > On Tue, 2004-03-16 at 16:12, Mark R. Diggory wrote: > > > However, by following a similar line of reasoning you could take for an > > analogy that IE is the predominate browser on the market, so what ever > > it defines as a DOM model or CSS implementation would be the standard, > > but yet, we all know this isn't true. > > I don't think that's a valid analogy as there is a standard for DOM/CSS > to which MS doesn't conform. In the case of Java repository tools there > is no standard. Prior to any spec being used the tool used most is > basically the spec. > historically I'll speculate IE (and its DOM) probably existed before the w3c spec was established. > > Well, I would suspect (or at least hope) that Apache would be a driving > > force behind such an endeavor. > > I would imagine so. > > > -Mark -- Mark R. Diggory Software Developer - VDC Project Harvard MIT Data Center http://www.hmdc.harvard.edu
Avalon currently is not a paradise!
A significant divergence in the Directory structures of Maven Repositories and the "dist" mirrors I've encountered involves the usage of "binaries", "sources" and "distributions" naming for artifact type directories. While this isn't much of an issue in terms of the "URISyntax" spec, which I believe supports any of this sort of naming convention, the issue lies more in resolving content duplication issues in projects like Avalon which maintains both structures in their directories. There is no simple "symlink" that can be approached to resolve these duplication issues because the directory structures are not the same. IE avalon/phoenix/jars/phoenix-client-4.0.4.jar java-repository/avalon-phoenix/jars/avalon-phoenix-client-4.0.4.jar avalon/logkit/v1.2/LogKit-1.2-bin.zip java-repository/logkit/distributions/LogKit-1.2-bin.zip avalon/framework/source/avalon-framework-4.1.5.src.zip java-repository/avalon-framework/distributions/avalon-framework-4.1.5.src.zip As you can see there are several different naming strategies being used in the Avalon repository contents. See Henk's dist management report's "fyi : some doubles" section: http://www.apache.org/~henkp/md5/ We need a solution to this issue of content duplication between the ASF Repository and the separate Repositories that Avalon is maintaining. -Mark -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: [Fwd: Avalon currently is not a paradise!]
I do like the download pages for Avalon however! These are very nice and provides some very solid navigation of "distributed download" capabilities. I'd really like to adopt this strategy for a global ASF Repository navigation. http://avalon.apache.org/download.cgi Should we consider putting together a small Maven based documentation project to establish a "ASF Repository" Site using this strategy? At least for documentation for this type of content? Then we can actually establish a canonical location for any documentation concerning the ASF Repository and its usage. We can also work out a strategy to provide templates of this xdoc for other projects who wish maintain links into their repository contents. -Mark J Aaron Farr wrote: Just a heads up. We need to look into these duplications in the repositories. jaaron - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Avalon currently is not a paradise!
While that is a possible solution, it creates alot of maintence issues and isn't very scalable. I think there could be a cleaner solution if both parties got involved with making adjustments. -Mark Noel J. Bergman wrote: There is no simple "symlink" that can be approached to resolve these duplication issues because the directory structures are not the same. Correct me if I am wrong, but it appears that although the directory structure needs to be duplicated, at the leaf a symlink can remove duplication of the file data. Either the leaf in java-repository points to the avalon copy, or the avalon copy points to the jar in java-repository. I don't particularly care which. I don't recall if you had requirement. --- Noel -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Avalon currently is not a paradise!
Thnx, I would like to see where discussion goes first. The duplication has been there for some time, resolving it immediately is not high on my list. Making any significant changes has to be well thought out in directories that are mirrored. Establishing a path of convergence for the contents is at the top of my list concerning this issue. -M. Noel J. Bergman wrote: While that is a possible solution, it creates alot of maintence issues and isn't very scalable. I think there could be a cleaner solution if both parties got involved with making adjustments. Understood. Just trying to propose something that can be done immediately, while the adjustments are discussed. --- Noel -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Geronimo files in Java-repository?
Nightly Builds Are considered non-releases and shouldn't be distributed to the mirrors. To resolve this issue we are working on the following: Full releases (mirrored): /www/www.apache.org/dist/java-repository Be very careful about placing content here, you should use a staging area outside the mirror and then move everything in when you have the permissions and md5 signatures correctly setup. Nightly/interm builds /www/cvs.apache.org/repository This for nightly builds. The opinion here is that we are not distributing anything that is not an "official versioned release" to third parties such as Ibiblio. -Mark Diggory Noel J. Bergman wrote: As I understand it, these should have gone to cvs.apache.org/repository/incubator-geronimo. Please talk with Mark Diggory about the ASF Repository layout, policies, and other details. --- Noel -Original Message- From: Cron Daemon [mailto:[EMAIL PROTECTED] Sent: Thursday, April 29, 2004 3:45 To: [EMAIL PROTECTED] Subject: Cron <[EMAIL PROTECTED]> /usr/local/bin/rsync -av www.apache.org::apache-dist /www/archive.apache.org/dist/ receiving file list ... done ./ java-repository/ java-repository/geronimo/ java-repository/geronimo-spec/jars/ java-repository/geronimo/jars/ java-repository/geronimo/plugins/ java-repository/geronimo/wars/ DATE java-repository/geronimo-spec/jars/geronimo-spec-ejb-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-ejb-1.0-M1.jar.md5 java-repository/geronimo-spec/jars/geronimo-spec-j2ee-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-j2ee-1.0-M1.jar.md5 java-repository/geronimo-spec/jars/geronimo-spec-j2ee-connector-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-j2ee-connector-1.0-M1.jar.m d5 java-repository/geronimo-spec/jars/geronimo-spec-j2ee-deployment-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-j2ee-deployment-1.0-M1.jar. md5 java-repository/geronimo-spec/jars/geronimo-spec-j2ee-jacc-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-j2ee-jacc-1.0-M1.jar.md5 java-repository/geronimo-spec/jars/geronimo-spec-j2ee-management-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-j2ee-management-1.0-M1.jar. md5 java-repository/geronimo-spec/jars/geronimo-spec-j2eeschema-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-j2eeschema-1.0-M1.jar.md5 java-repository/geronimo-spec/jars/geronimo-spec-jms-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-jms-1.0-M1.jar.md5 java-repository/geronimo-spec/jars/geronimo-spec-jsp-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-jsp-1.0-M1.jar.md5 java-repository/geronimo-spec/jars/geronimo-spec-jta-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-jta-1.0-M1.jar.md5 java-repository/geronimo-spec/jars/geronimo-spec-servlet-1.0-M1.jar java-repository/geronimo-spec/jars/geronimo-spec-servlet-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-clustering-1.0-M1.jar java-repository/geronimo/jars/geronimo-clustering-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-common-1.0-M1.jar java-repository/geronimo/jars/geronimo-common-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-connector-1.0-M1.jar java-repository/geronimo/jars/geronimo-connector-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-core-1.0-M1.jar java-repository/geronimo/jars/geronimo-core-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-deployment-1.0-M1.jar java-repository/geronimo/jars/geronimo-deployment-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-j2ee-1.0-M1.jar java-repository/geronimo/jars/geronimo-j2ee-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-jetty-1.0-M1.jar java-repository/geronimo/jars/geronimo-jetty-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-kernel-1.0-M1.jar java-repository/geronimo/jars/geronimo-kernel-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-naming-1.0-M1.jar java-repository/geronimo/jars/geronimo-naming-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-network-1.0-M1.jar java-repository/geronimo/jars/geronimo-network-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-remoting-1.0-M1.jar java-repository/geronimo/jars/geronimo-remoting-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-security-1.0-M1.jar java-repository/geronimo/jars/geronimo-security-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-system-1.0-M1.jar java-repository/geronimo/jars/geronimo-system-1.0-M1.jar.md5 java-repository/geronimo/jars/geronimo-transaction-1.0-M1.jar java-repository/geronimo/jars/geronimo-transaction-1.0-M1.jar.md5 java-repository/geronimo/plugins/geronimo-deployment-plugin-1.0-M1.jar java-repository/geronimo/plugins/geronimo-deployment-plugin-1.0-M1.jar.md5 java-repository/geronimo/plugins/geronimo-xmlbeans-plugin-1.0-M1.jar java-repository/geronimo/plugins/geronimo-xmlbeans-plugin-1.0-M1.jar.md5 java-repository/geronimo/wars/geronimo-console-web-1.0-M1.war java-repository/geronimo/wars/geronimo-console-web-1.0-M1.war.md5 java-repository/geroni
Re: betwixt 0.5 release jars
Yes, pretty much the case. The big thing is to make sure you set group write permissions and make sure the group is apcvs. I can look into setting the deployment dir to be /www/www.apache.org/dist/java-repository/ but I remember requests from the mirror folks to make sure that deployment happened in a working/scratch directory and the copied into the dist this way the md5 checksum scipts that test the contents of dist wouldn't bark during the upload process. -Mark Brett Porter wrote: > i'm a little sensitive to the needs for supervision at the moment > (after being strongly reminded on this list before by members > who know > about these issues). the jar's i've posted have been approved for > release. i don't really want to have to run a second vote > (again) for a > release of different jars to the repo. If you have a controlled environment and build from a clean checkout of a tag... There shouldn't be a problem? > i'm sure that once the process is better understood and documented > we'll be able to move toward automation without too many people > screaming. until then, i'd feel better if some member of the repo > committee moved the stuff into place. I think anyone who can commit to CVS can put a jar into /www/www.apache.org/dist/java-repository/commons-betwixt/jars, so you should be good to go. I'm just here out of interest - I'm not sure I form part of the "repo committee"? :) Cheers, Brett begin:vcard fn:Mark Diggory n:Diggory;Mark org:Harvard University;Harvard MIT Data Center adr:Harvard University;;G-6 Littauer Center (North Yard);Cambridge;Ma;02138-2901;United States email;internet:[EMAIL PROTECTED] title:Software Engineer tel;work:617 496 7246 tel;fax:617 495 0438 tel;home:617 718 2033 tel;cell:617 285 4106 url:http://www.hmdc.harvard.edu version:2.1 end:vcard
ASF Repository, closer.cgi and Depot
Sorry for the cross post but this seems relevant to both these groups. I was thinking about the subject of mirroring and redirection for the ASF Repository. Currently, there was some discussion on the Depot list concerning this. I feel we could address this subject again for both groups interest. www.apache.org/dyn/closer cgi provides a simple resolution strategy to attempt to determine the closest mirror available to the client browser. It then generates an html page via a template that lists the selected mirror as well as other available mirrors. With Depot, we have a customized download client that could be extended to manage downloading from a list of mirrors as well. Here are my thoughts on this subject: A.) This script is really not that big (90% of it is just parsing the mirrors file), and the database (a flat text file called mirrors.list) as well is not very big. While closer.cgi is a neat service for browsers. Its not exactly helpful for automated clients. Yet, mirrors.list is an excellent example of metadata that is exposed in a effective manner such that automated clients can access it. http://www.apache.org/mirrors/mirrors.list I'm somewhat convinced that a it would be simple to create a client implementation which accomplished the same functionality as closer.cgi programatically so that it could be used in terms of resolving a location to download from when mirrors are available. This would be beneficial to the Apache Bandwidth issue in that if a client such as Depot/DownloadManager managed the same capability as closer.cgi then: 1.) to determine if the list file has been updated, all one needs to do is a head request on the file and review the lastModified date, downloading it if it is newer than the client local copy. 2.) Apache server cpu time is spent parsing this file for each "closer.cgi" request on the server side, instead the client spends the cpu time doing this calculation. After the intial head request to check when the mirror list was last updated, no other requests occur to www.apache.org in the download process. B.) Downfalls? 1.) If such a service were server-side, we do get a centralized way of managing it. But its difficult to control http client behavior from the server outside of the most simplistic of "http redirects", the cost of downloading a file becomes much greater in that each download request has to be redirected through closer.cgi. 2.) Statistics: I guess the benefit that I do see is that one could log requests through closer.cgi to track download statistics. But these again would only be "partial stats" because any browser can simply bookmark a mirror and go to it directly. It seems more appropriate that a "download stats" tool would operate more behind the scenes of all the mirrors and be aggrigated across all the mirrors to gain more accuracy in such statistics. Cheers, -Mark begin:vcard fn:Mark Diggory n:Diggory;Mark org:Harvard University;Harvard MIT Data Center adr:Harvard University;;G-6 Littauer Center (North Yard);Cambridge;Ma;02138-2901;United States email;internet:[EMAIL PROTECTED] title:Software Engineer tel;work:617 496 7246 tel;fax:617 495 0438 tel;home:617 718 2033 tel;cell:617 285 4106 url:http://www.hmdc.harvard.edu version:2.1 end:vcard
Re: ASF Repository, closer.cgi and Depot
Erik Abele wrote: I suspect their views would include what you suggest, that distribution might save some nomimal (c.f. artifact sizes) bandwidth savings & give some CPU saving, but it'd be at significant loss of 'control' (of well behaved clients). Central control over this seems the most appealing. Agreed. Since I doubt the CPU cycles are worth saving (or the script would've been optimised), could we not just change the script to check for some header from the client, and return XML or some structured text, for non-human browsers. [BTW: viewcvs seems to do this nicely, returning the file if non-human and the presentation is human (as browser identifies). This sounds promising. You have central control, you get the geoip-mapping stuff for free and the CPU cycles as well as the bandwidth for (XML-ized) responses are a no-brainer in this case. But then this becomes a project spanning both the Repository group and the various clients out there "Depot/Maven/etc". And agreement on the GEO_IP request protocol and xml format etc becomes a touchy subject don't they? -Mark begin:vcard fn:Mark Diggory n:Diggory;Mark org:Harvard University;Harvard MIT Data Center adr:Harvard University;;G-6 Littauer Center (North Yard);Cambridge;Ma;02138-2901;United States email;internet:[EMAIL PROTECTED] title:Software Engineer tel;work:617 496 7246 tel;fax:617 495 0438 tel;home:617 718 2033 tel;cell:617 285 4106 url:http://www.hmdc.harvard.edu version:2.1 end:vcard
Re: any ideas why beanutils 1.7.0 jars can't be downloaded by maven?
I don't see anything wrong with the permissions, when did you add the file to java-repository? I also notice the SNAPSHOT symlinks are pretty ancient, you might want to update those links to point to the new release? -Mark Brett Porter wrote: I don't see it on ibiblio (beanutils-core and beanutils-collections are there, but beanutils itself is not). Do you have the correct permissions on java-repository? - Brett On Tue, 3 Aug 2004 22:04:51 +0100, robert burrell donkin <[EMAIL PROTECTED]> wrote: i uploaded the jars from the beanutils 1.7.0 release to the java repository. they have been mirrored to ibiblio. but maven does not seem able to automatically download them. this is a bit of a PITA since i'm currently trying to prepare a mavenized release whose dependencies need to be updated to the latest beanutils so any help would be really appreciated. - robert -- Mark R. Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: any ideas why beanutils 1.7.0 jars can't be downloaded by maven?
Looks like the beanutils jar went through on the last scheduled rsync at 8pm EST. receiving file list ... done commons-beanutils/jars/commons-beanutils-1.7.0.jar commons-beanutils/jars/commons-beanutils-1.7.0.jar.asc commons-beanutils/jars/commons-beanutils-1.7.0.jar.md5 wrote 151 bytes read 300289 bytes 8011.73 bytes/sec total size is 284096595 speedup is 945.60 Mark R. Diggory wrote: I don't see anything wrong with the permissions, when did you add the file to java-repository? I also notice the SNAPSHOT symlinks are pretty ancient, you might want to update those links to point to the new release? -Mark Brett Porter wrote: I don't see it on ibiblio (beanutils-core and beanutils-collections are there, but beanutils itself is not). Do you have the correct permissions on java-repository? - Brett On Tue, 3 Aug 2004 22:04:51 +0100, robert burrell donkin <[EMAIL PROTECTED]> wrote: i uploaded the jars from the beanutils 1.7.0 release to the java repository. they have been mirrored to ibiblio. but maven does not seem able to automatically download them. this is a bit of a PITA since i'm currently trying to prepare a mavenized release whose dependencies need to be updated to the latest beanutils so any help would be really appreciated. - robert -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Rsync Emails
I'm curious, would others be interested in seeing the rsync emails from ibiblio for the rsync that runs every 4 hours on the login.ibibilio.org server? I could direct them to this list or possibly the maven devel list if its more appropriate. -Mark -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Rsync Emails
Depends on how much publishing activity, for instance, heres the last one: receiving file list ... done commons-beanutils/jars/commons-beanutils-1.7.0.jar commons-beanutils/jars/commons-beanutils-1.7.0.jar.asc commons-beanutils/jars/commons-beanutils-1.7.0.jar.md5 wrote 151 bytes read 300289 bytes 8011.73 bytes/sec total size is 284096595 speedup is 945.60 otherwise, if no new content is found its just 3-4 lines: receiving file list ... done wrote 0 bytes read 300289 bytes 8011.73 bytes/sec total size is 0 speedup is 945.60 I could possibly tweek it to not send an email if there are no new files being moved. -Mark Adam R. B. Jack wrote: On Tue, 3 Aug 2004, Mark R. Diggory wrote: I'm curious, would others be interested in seeing the rsync emails from ibiblio for the rsync that runs every 4 hours on the login.ibibilio.org server? I could direct them to this list or possibly the maven devel list if its more appropriate. Are they verbose, or only in errors? This list seems a good place, unless overly verbose. regards Adam -- Mark R. Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Cron /export/sunsite/users/mdiggory/bin/sync-apache
Just doublechecking, did everyone on the repository list recieve this? -Mark Cron Daemon wrote: receiving file list ... done commons-configuration/poms/commons-configuration-1.0-rc1.pom turbine/jars/turbine-2.4-M1.jar wrote 135 bytes read 760909 bytes 16021.98 bytes/sec total size is 317115246 speedup is 416.68 -- Mark R. Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Cron /export/sunsite/users/mdiggory/bin/sync-apache
Yes, when the ibiblio rsync of the java-repository occurs, it now posts and email if any files are transfered. Earlier we discussed having these come to the Repository list. This is the first case of transfer since I set it up. The only issue I can see is that the From address is [EMAIL PROTECTED], I wish I could change this somehow so that it came from my account specifically, But I don't see how with the crontab at this point without writing a custom script. -Mark Noel J. Bergman wrote: Just doublechecking, did everyone on the repository list recieve this? Yes. Was this by accident, or something you wanted coming to the list now? --- Noel -- Mark R. Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Cron /export/sunsite/users/mdiggory/bin/sync-apache
I've added my own sendmail scripting to have the rsync announce from my
email address directly when files are transfered. So you will not need
to add [EMAIL PROTECTED]
cheers,
Mark
Noel J. Bergman wrote:
The only issue I can see is that the From address is [EMAIL PROTECTED]
I will leave it to you to figure out. When you've done, please let me know.
Right now I have to moderate these through. I'll add it to allow list after
you're done fixing the address, although it it ends up being yours, I won't
have to worry about it.
--- Noel
--
Mark R. Diggory
Software Developer
Harvard MIT Data Center
http://www.hmdc.harvard.edu
Re: Cron /export/sunsite/users/mdiggory/bin/sync-apache
plugin...hibernate is already in the ibiblio repository. Nicola Ken Barozzi wrote: Cron Daemon wrote: receiving file list ... done maven/plugins/maven-cruisecontrol-plugin-1.5.jar maven/plugins/maven-cruisecontrol-plugin-1.5.jar.md5 maven/plugins/maven-eclipse-plugin-1.8.jar maven/plugins/maven-eclipse-plugin-1.8.jar.md5 maven/plugins/maven-hibernate-plugin-1.2.jar maven/plugins/maven-hibernate-plugin-1.2.jar.md5 maven/poms/maven-cruisecontrol-plugin-1.5.pom maven/poms/maven-cruisecontrol-plugin-1.5.pom.md5 maven/poms/maven-eclipse-plugin-1.8.pom maven/poms/maven-eclipse-plugin-1.8.pom.md5 maven/poms/maven-hibernate-plugin-1.2.pom maven/poms/maven-hibernate-plugin-1.2.pom.md5 wrote 295 bytes read 181387 bytes 854.97 bytes/sec total size is 317178373 speedup is 1745.79 Isn't Hibernate LGPL, which the ASF does not want to distribute? Is this plugin also LGPL? -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: [rsync] Repository rsync to ibiblio
Sigh, it was working last night, now what... Noel J. Bergman wrote: Mark, You need to fix these. The format is wrong now (HTML with an attachment) and the sending address is [EMAIL PROTECTED] --- Noel -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: [rsync] Repository rsync to ibiblio
Ok, I should have the html/attachment issue fixed. I'm trying to send it with the from being [EMAIL PROTECTED] If this is not working then we'll need to workout it coming from my ibiblio account instead. -Mark Mark R. Diggory wrote: Sigh, it was working last night, now what... Noel J. Bergman wrote: Mark, You need to fix these. The format is wrong now (HTML with an attachment) and the sending address is [EMAIL PROTECTED] --- Noel -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: [rsync] Repository rsync to ibiblio
Noel, Did this work out ok this time, the From address looks ok, but I'm unsure about the "wrapper" itself. -Mark [EMAIL PROTECTED] wrote: receiving file list ... done ojb/jars/db-ojb-1.0.0-tools.jar ojb/jars/db-ojb-1.0.0-tools.jar.md5 ojb/jars/xdoclet-ojb-module-1.2.1.jar ojb/jars/xdoclet-ojb-module-1.2.1.jar.md5 wrote 167 bytes read 426078 bytes 2991.19 bytes/sec total size is 317971252 speedup is 745.98 -- Mark R. Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: [rsync] Repository rsync to ibiblio
Ok, can we add that address so the list can accept it (without list emails being sent to the address). I don't believe I can alter it any further, as it happens within the crontab and somehow ibiblio restricts altering the wrapper, which makes sense because then it'd be a spoof. -Mark Noel J. Bergman wrote: I had to moderate it. The envelope sender is still [EMAIL PROTECTED] instead of @latte.harvard.edu. --- Noel -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: [rsync] Repository rsync to ibiblio
Good point, these should be in separate src/bin artifact directories. These were deployed by jmitchell. I've forwarded this message to him as well. -Mark matthew.hawthorne wrote: Why are there tar.gz files in the 'jars' directory? That doesn't seem right... [EMAIL PROTECTED] wrote: receiving file list ... done commons-validator/jars/commons-validator-1.1.3-src.tar.gz commons-validator/jars/commons-validator-1.1.3-src.tar.gz.MD5 commons-validator/jars/commons-validator-1.1.3-src.zip commons-validator/jars/commons-validator-1.1.3-src.zip.MD5 commons-validator/jars/commons-validator-1.1.3.jar commons-validator/jars/commons-validator-1.1.3.tar.gz commons-validator/jars/commons-validator-1.1.3.tar.gz.MD5 commons-validator/jars/commons-validator-1.1.3.zip commons-validator/jars/commons-validator-1.1.3.zip.MD5 commons-validator/jars/struts-help.html lucene/jars/lucene-1.4.1.jar lucene/jars/lucene-1.4.1.jar.md5 wrote 295 bytes read 2379035 bytes 8763.65 bytes/sec total size is 326237504 speedup is 137.11 -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: [rsync] Repository rsync to ibiblio
I was able to move them to commons-validator/distributions. -Mark James Mitchell wrote: For some reason, I was able to copy those there, but I cannot remove them. -- James Mitchell Software Engineer / Open Source Evangelist EdgeTech, Inc. 678.910.8017 AIM: jmitchtx - Original Message - From: "Mark R. Diggory" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> Sent: Friday, August 27, 2004 11:24 PM Subject: Re: [rsync] Repository rsync to ibiblio Good point, these should be in separate src/bin artifact directories. These were deployed by jmitchell. I've forwarded this message to him as well. -Mark matthew.hawthorne wrote: Why are there tar.gz files in the 'jars' directory? That doesn't seem right... [EMAIL PROTECTED] wrote: receiving file list ... done commons-validator/jars/commons-validator-1.1.3-src.tar.gz commons-validator/jars/commons-validator-1.1.3-src.tar.gz.MD5 commons-validator/jars/commons-validator-1.1.3-src.zip commons-validator/jars/commons-validator-1.1.3-src.zip.MD5 commons-validator/jars/commons-validator-1.1.3.jar commons-validator/jars/commons-validator-1.1.3.tar.gz commons-validator/jars/commons-validator-1.1.3.tar.gz.MD5 commons-validator/jars/commons-validator-1.1.3.zip commons-validator/jars/commons-validator-1.1.3.zip.MD5 commons-validator/jars/struts-help.html lucene/jars/lucene-1.4.1.jar lucene/jars/lucene-1.4.1.jar.md5 wrote 295 bytes read 2379035 bytes 8763.65 bytes/sec total size is 326237504 speedup is 137.11 -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: No jars at Ibiblio
We appear to have a permissions issue happening mesg: ttyname: Operation not supported receiving file list ... done mkstemp hivemind/jars/.hivemind-1.0-rc-2.jar.yxE71a failed: Permission denied mkstemp hivemind/jars/.hivemind-1.0-rc-2.jar.md5.sV2j93 failed: Permission denied mkstemp hivemind/jars/.hivemind-lib-1.0-rc-2.jar.PviAgX failed: Permission denied mkstemp hivemind/jars/.hivemind-lib-1.0-rc-2.jar.md5.04VRrQ failed: Permission denied wrote 80 bytes read 463709 bytes 16865.05 bytes/sec total size is 352725880 speedup is 760.53 rsync error: some files could not be transferred (code 23) at main.c(1046) -Mark Howard Lewis Ship wrote: I pasted the wrong URL. It was failing, is now ok ... but the hivemind rc2 jars are NOT showing up at ibiblio, even though they are in the apache java-repository properly. On Tue, 21 Sep 2004 12:37:54 -0400, Mark R. Diggory <[EMAIL PROTECTED]> wrote: Did you look under http://www.ibiblio.org/maven/hivemind/jars/ -Mark Howard Lewis Ship wrote: Something is up at ibiblio. Try pointing a browser at http://www.ibiblio.org/maven/hivemind/ The files are in place with the right permissions at /x1/www/www.apache.org/dist/java-repository/hivemind/jars and should have been mirrored out to ibiblio. On Tue, 21 Sep 2004 09:55:31 -0400, Howard Lewis Ship <[EMAIL PROTECTED]> wrote: I'll get this put together shortly. - Original Message - From: James Carman <[EMAIL PROTECTED]> Date: Fri, 17 Sep 2004 15:02:06 -0400 Subject: No jars at Ibiblio To: hivemind-dev@jakarta.apache.org The build file for my article automatically downloads the necessary jars from the maven repository at www.ibiblio.org. However, the rc2 jars are not there. Can we get them published up there? -- Howard M. Lewis Ship Independent J2EE / Open-Source Java Consultant Creator, Jakarta Tapestry Creator, Jakarta HiveMind http://howardlewisship.com -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: No jars at Ibiblio
Did you look under http://www.ibiblio.org/maven/hivemind/jars/ -Mark Howard Lewis Ship wrote: Something is up at ibiblio. Try pointing a browser at http://www.ibiblio.org/maven/hivemind/ The files are in place with the right permissions at /x1/www/www.apache.org/dist/java-repository/hivemind/jars and should have been mirrored out to ibiblio. On Tue, 21 Sep 2004 09:55:31 -0400, Howard Lewis Ship <[EMAIL PROTECTED]> wrote: I'll get this put together shortly. - Original Message - From: James Carman <[EMAIL PROTECTED]> Date: Fri, 17 Sep 2004 15:02:06 -0400 Subject: No jars at Ibiblio To: hivemind-dev@jakarta.apache.org The build file for my article automatically downloads the necessary jars from the maven repository at www.ibiblio.org. However, the rc2 jars are not there. Can we get them published up there? -- Howard M. Lewis Ship Independent J2EE / Open-Source Java Consultant Creator, Jakarta Tapestry Creator, Jakarta HiveMind http://howardlewisship.com -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: No jars at Ibiblio
Believe I corrected this on the ibiblio side. We will see during the next pass. -Mark Howard Lewis Ship wrote: I pasted the wrong URL. It was failing, is now ok ... but the hivemind rc2 jars are NOT showing up at ibiblio, even though they are in the apache java-repository properly. On Tue, 21 Sep 2004 12:37:54 -0400, Mark R. Diggory <[EMAIL PROTECTED]> wrote: Did you look under http://www.ibiblio.org/maven/hivemind/jars/ -Mark Howard Lewis Ship wrote: Something is up at ibiblio. Try pointing a browser at http://www.ibiblio.org/maven/hivemind/ The files are in place with the right permissions at /x1/www/www.apache.org/dist/java-repository/hivemind/jars and should have been mirrored out to ibiblio. On Tue, 21 Sep 2004 09:55:31 -0400, Howard Lewis Ship <[EMAIL PROTECTED]> wrote: I'll get this put together shortly. - Original Message - From: James Carman <[EMAIL PROTECTED]> Date: Fri, 17 Sep 2004 15:02:06 -0400 Subject: No jars at Ibiblio To: hivemind-dev@jakarta.apache.org The build file for my article automatically downloads the necessary jars from the maven repository at www.ibiblio.org. However, the rc2 jars are not there. Can we get them published up there? -- Howard M. Lewis Ship Independent J2EE / Open-Source Java Consultant Creator, Jakarta Tapestry Creator, Jakarta HiveMind http://howardlewisship.com -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: No jars at Ibiblio
Yes, the permissions issue was corrected. mesg: ttyname: Operation not supported receiving file list ... done hivemind/jars/hivemind-1.0-rc-2.jar hivemind/jars/hivemind-1.0-rc-2.jar.md5 hivemind/jars/hivemind-lib-1.0-rc-2.jar hivemind/jars/hivemind-lib-1.0-rc-2.jar.md5 wrote 80 bytes read 463983 bytes 48848.74 bytes/sec total size is 353256473 speedup is 761.23 -Mark Howard Lewis Ship wrote: I pasted the wrong URL. It was failing, is now ok ... but the hivemind rc2 jars are NOT showing up at ibiblio, even though they are in the apache java-repository properly. On Tue, 21 Sep 2004 12:37:54 -0400, Mark R. Diggory <[EMAIL PROTECTED]> wrote: Did you look under http://www.ibiblio.org/maven/hivemind/jars/ -Mark Howard Lewis Ship wrote: Something is up at ibiblio. Try pointing a browser at http://www.ibiblio.org/maven/hivemind/ The files are in place with the right permissions at /x1/www/www.apache.org/dist/java-repository/hivemind/jars and should have been mirrored out to ibiblio. On Tue, 21 Sep 2004 09:55:31 -0400, Howard Lewis Ship <[EMAIL PROTECTED]> wrote: I'll get this put together shortly. - Original Message - From: James Carman <[EMAIL PROTECTED]> Date: Fri, 17 Sep 2004 15:02:06 -0400 Subject: No jars at Ibiblio To: hivemind-dev@jakarta.apache.org The build file for my article automatically downloads the necessary jars from the maven repository at www.ibiblio.org. However, the rc2 jars are not there. Can we get them published up there? -- Howard M. Lewis Ship Independent J2EE / Open-Source Java Consultant Creator, Jakarta Tapestry Creator, Jakarta HiveMind http://howardlewisship.com -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Rsync From Address and Source changing.
Jason has taken over the management of the ibiblio rsync emails (he's standardized the rsync process for all organizations rsyncing content to ibiblio. Currently rsync logs are sent to the [EMAIL PROTECTED] list, but I I suspect the moderation is rejecting them? Can we have this adjusted again? thanks, -Mark -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Rsync From Address and Source changing.
Here are the email headers for the new rsync reporting address: Return-Path: <[EMAIL PROTECTED]> Received: from tribal.metalab.unc.edu (IDENT:[EMAIL PROTECTED] [152.2.210.122]) by hogshead.codehaus.org (8.11.6/8.11.6) with ESMTP id i8LKXN331667 for <[EMAIL PROTECTED]>; Tue, 21 Sep 2004 15:33:24 -0500 Date: Tue, 21 Sep 2004 15:33:24 -0500 From: [EMAIL PROTECTED] Message-Id: <[EMAIL PROTECTED]> Subject: Synchronization report for Apache Software Foundation -Mark Mark R. Diggory wrote: Jason has taken over the management of the ibiblio rsync emails (he's standardized the rsync process for all organizations rsyncing content to ibiblio. Currently rsync logs are sent to the [EMAIL PROTECTED] list, but I I suspect the moderation is rejecting them? Can we have this adjusted again? thanks, -Mark -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: old JAR files changed in java-repository
Brett, I suspect this is the same issue encountered earlier in the month where Dion Gillard had updated/published some jars using maven. At the same point in time some modifications occurred int he directory which we could not identify the origin of. Too alleviate some of the issues with the fact that anyone in group apcvs could alter any files in the repository. We've begun a slow process of privatizing the files so that only members of a specific group (jakarta,xml etc) have rights to alter the content of the original artifact/md5 pair. The best I personally can do to get this started is to restrict the rights to files that at least I own at the moment (if only because I transfered the content originally from ibiblio to apache) and then transfer the rights to those files to the appropriate group/user on a case by case basis. Brett, I'll open these files specifically with write access to the jakarta group, you then should be able to update them. I also recommend that you take ownership of them by moving them to temp files and back to change the ownership to your userid. Then restrict the group ownership so only you (or other Commons Jelly Developers) have rights to write them. -Mark Brett Porter wrote: I tried to build maven 1.0 clean today and found out that on Sep 2, commons-jelly and a few of its tag libs had changed on ibiblio. These were already released jars that changed size. I checked around and discovered the source of the change seems to be java-repository at Apache. Any ideas how this happened? One particular offending file: ls -l /www/www.apache.org/dist/java-repository/commons-jelly/jars/ | grep 20030902 -rw-r--r-- 1 mdiggory jakarta 161479 Sep 2 00:08 commons-jelly-20030902.160215.jar md5 d1e3117b90f697e6503e4ddf76bc0402 The original: [EMAIL PROTECTED] repaired]$ md5sum commons-jelly/jars/commons-jelly-20030902.160215.jar b171e535366191e437cff6d64df33561 commons-jelly/jars/commons-jelly-20030902.160215.jar [EMAIL PROTECTED] repaired]$ ls -l commons-jelly/jars/commons-jelly-20030902.160215.jar -rwxr-xr-x1 mavenusers 154418 Sep 28 09:00 commons-jelly/jars/commons-jelly-20030902.160215.jar (this is identical to the version distributed with Maven 1.0 back in July) The full list from the Maven 1.0 build that differ: commons-jelly/jars/commons-jelly-20030902.160215.jar commons-jelly/jars/commons-jelly-tags-antlr-20030211.143720.jar commons-jelly/jars/commons-jelly-tags-define-20030211.142932.jar commons-jelly/jars/commons-jelly-tags-html-20030317.100924.jar commons-jelly/jars/commons-jelly-tags-interaction-20030211.143817.jar commons-jelly/jars/commons-jelly-tags-jsl-20030211.143151.jar commons-jelly/jars/commons-jelly-tags-log-20030211.142821.jar commons-jelly/jars/commons-jelly-tags-swing-20030211.143925.jar commons-jelly/jars/commons-jelly-tags-util-20030211.141939.jar commons-jelly/jars/commons-jelly-tags-velocity-20030303.205659.jar commons-jelly/jars/commons-jelly-tags-xml-20030211.142705.jar I have corrected the versions on ibiblio (being newer, they should stay that way), but left Apache alone to find out what happened. I'd really like to know if we can resolve this to avoid it in the future, and to see if anything else was possibly affected. Thanks, Brett -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Subversive Repository
Here's a thought that has been propagating through my neurons. What if the ASF Repository content was maintained in Subversion and exported into dist/java-repository for mirror propagation using some automation? Given Subversions binary diffing, this would be manageable. 1.) This would allow us to control permissions at the same level as the cvs/subversion source control. As such: Jakarta Commons Jelly and Jakarta Commons Collections would have enough fine grained permissions to control to separate who gets the right to add/remove/modify content at that level instead of what we have right now, which are only the groups in the shell (ie xml, jakarta etc). 2.) This would give us a layer (automated update of dist/java-repository) to catch issues such as: a.) md5 and signature errors. b.) dated-build vs full version releases. The downfall is that Repository Clients (Like Maven) would need to be capable of doing svn commits etc to publish artifacts instead of ssh/scp requests. -Mark -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
[jelly] Re: md5 errors in java-repository
I'm forwarding this onto the "Repository" email list. Dion, the reason I was suggesting it was your action is because files also owned by yourself were added to the commons-jelly/jars directory. I assume it was your upload of those files which altered them so that the md5's no longer matched. -rw-rw-r-- 1 dion apcvs 12377 Sep 2 00:39 commons-jelly-tags-validate-20040902.073836.jar -rw-rw-r-- 1 dion apcvs 32 Sep 2 00:39 commons-jelly-tags-validate-20040902.073836.jar.md5 ... -rw-rw-r-- 1 dion apcvs 10343 Sep 2 00:39 commons-jelly-tags-velocity-20040902.073917.jar -rw-rw-r-- 1 dion apcvs 32 Sep 2 00:39 commons-jelly-tags-velocity-20040902.073917.jar.md5 ... -rw-rw-r-- 1 dion apcvs 161479 Sep 2 00:08 commons-jelly-20040902.070806.jar -rw-rw-r-- 1 dion apcvs 32 Sep 2 00:09 commons-jelly-20040902.070806.jar.md5 ... -rw-rw-r-- 1 dion apcvs 35076 Sep 2 00:21 commons-jelly-tags-xml-20040902.072037.jar -rw-rw-r-- 1 dion apcvs 32 Sep 2 00:21 commons-jelly-tags-xml-20040902.072037.jar.md5 ... -rw-rw-r-- 1 mdiggory apcvs 11489 Sep 2 00:28 commons-jelly-tags-xmlunit-20030211.144251.jar -rw-rw-r-- 1 mdiggory apcvs 33 Jan 19 2004 commons-jelly-tags-xmlunit-20030211.144251.jar.md5 So I was suspecting it was Dion because the published jars under his name like those represented above got transferred last night to ibiblio: http://www.ibiblio.org/maven/reports/apache/2004/09/02/apache-20040902-050103.txt All I know is that there was a process running around midnight which walked through this directory and updated jar files within it, breaking some of the md5 signatures on files which were owned by me. Henk P. Penning wrote: On Fri, 3 Sep 2004, Dion Gillard wrote: Date: Fri, 3 Sep 2004 09:11:25 +1000 From: Dion Gillard <[EMAIL PROTECTED]> To: Henk P. Penning <[EMAIL PROTECTED]>, [EMAIL PROTECTED] Subject: Re: md5 errors in java-repository Sorry guys, I haven't updated those files? .. then some of the 909 other members of group apcvs did it. true, true. We need to get the permissions locked down. IMHO, these files shouldn't be group writable ; only the directories should be; now file ownership means nothing. Well, I'm unsure how different individuals are publishing, I assume most are using maven, which will set files permission according to how it coded in the Maven client. If the Maven client were configurable, then maybe this could be managed. I'm still very curious how java-repository is maintained -- what stuff is added, and how Really, its only supposed to be getting full releases published within it. -- what stuff is deleted and when Currently, release which are removed from the public should be removed. Whouldn't it be easier if there was a config like this-jar FROM that-dist-tgz GOES there so that -- 'this-jar' can be removed if 'that-dist-tgz' is gone That would be managed by the project owners, like it is now. -- installation in java-repository can be automated Again, Maven, as a client is used to publish a release jar into the repository based on the project that is being worked with on the client side, the structure is fairly "strict". HPP _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_ Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/ http://www.cs.uu.nl/staff/henkp.html M [EMAIL PROTECTED] \_/ -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: md5's
Henk P. Penning wrote: On Tue, 5 Oct 2004, Mark R. Diggory wrote: Date: Tue, 05 Oct 2004 14:11:03 -0400 From: Mark R. Diggory <[EMAIL PROTECTED]> To: Henk P. Penning <[EMAIL PROTECTED]> Cc: [EMAIL PROTECTED], dion <[EMAIL PROTECTED]>, [EMAIL PROTECTED] Subject: Re: md5's In the opposite direction, individual ownership without group write capabilities blocks individuals from "removing" releases when it is time for them to be excised. I don't think it does. If you have write permission on a directory, you can remove any file in it, including the files you don't own and can't write. On Solaris 'rm' asks first, but you can remove. I can't check the situation on minotaur. I assume the same thing goes. HPP Your right, that makes sense, I was being shortsighted. So your saying that if the directory is group writable and the files are not, then any member of the group can still delete the file and replace it with a new one, in which case I don't need to give write permissions on the files for others to take ownership by copy/moving them individually. Ok, so I'll put the permissions back on those files, if others want to take ownership of the files great. So, I agree files shouldn't be group writable in this case. Directories, yes, but files, no... But this still doesn't solve the fact that at the project level members of separate projects can still manipulate the contents of other projects within that unix group. Maybe back when the unix groups were established the the number of projects under a groups wasn't an issue, now however, this isn't scalable, with so many projects in jakarta, there's little control across the project permissions, in terms of distributions. -Mark -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: md5's
Dion, I really think this is more systemic in terms of the upload behavior of Maven. The delema is that the general policy in promotion for /dist/ right now is that group "writable" on files are very bad. However, as Henk pointed out, this doesn't mean you can't take ownership of the jars, it just means that Maven needs to be a little more savvy about publishing files. For instance the artifact publishing plugin could catch this issue: if the file isn't group writable and the directory is, then the file should just be moved out of the way or deleted and the new file uploaded. It will then be owned by the individual who uploaded it. This way we are not left guessing who in the group last altered the file which we have no way of recording right now when maven uploads a file over an existing group writable file. Ultimately it would, as well, be of great benefit if the artifact publishing worked initially in a staging area as not to confuse existing cron jobs attempting to validate md5's on the server. This would probably be best served in the systems default tmp directory. -Mark Dion Gillard wrote: On Tue, 05 Oct 2004 14:11:03 -0400, Mark R. Diggory <[EMAIL PROTECTED]> wrote: Sigh, yes I did, but it was to allow Dion to take ownership of them, now I see he has not. Um, if you want me to change the ownership of some files, just let me know. At the moment, group writability is enough from what I understand. [snip] -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: md5's
The ball is in play. http://jira.codehaus.org/browse/MPARTIFACT-39 -Mark Henk P. Penning wrote: On Wed, 6 Oct 2004, Mark R. Diggory wrote: Date: Wed, 06 Oct 2004 14:08:38 -0400 From: Mark R. Diggory <[EMAIL PROTECTED]> To: Dion Gillard <[EMAIL PROTECTED]> Cc: Henk P. Penning <[EMAIL PROTECTED]>, [EMAIL PROTECTED], [EMAIL PROTECTED], Maven Developers List Subject: Re: md5's Ultimately it would, as well, be of great benefit if the artifact publishing worked initially in a staging area as not to confuse existing cron jobs attempting to validate md5's on the server. This would probably be best served in the systems default tmp directory. That would be great. I think, the best way for adding/replace stuff is -- write a 'temp' -- rename 'temp' to 'file' because a rename is truly atomic if 'temp' and 'file' are in the same file system. If you can implement the 'temp' for 'file' to be, for instance, '.tmp.file', I can easily teach the checkers to ignore '.tmp.*' files. I think rsync does something like that (even better .tmp.$$.file). Just a thought. -Mark HPP _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_ Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/ http://www.cs.uu.nl/staff/henkp.html M [EMAIL PROTECTED] \_/ -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: md5's
Sigh, yes I did, but it was to allow Dion to take ownership of them, now
I see he has not.
But, I'm not wholly convinced having what I was suggesting about groups
taking ownership of their files is now tractable.
But I'm slowly becoming convinced that this cannot be maintained based
on the current unix groups and historical policies behind the dist
directory. The unix groups are not fine-grained enough to support
adequate restriction at that project level. Further restriction, for
instance "jakarta-commons" or "jakarta-tomcat" is required the group
"jakarta" can't adequately protect project level ownership/modification
of these files. In the opposite direction, individual ownership without
group write capabilities blocks individuals from "removing" releases
when it is time for them to be excised.
I just do not have the proper permissions to process all these files and
set them to be owned by the appropriate groups. This is why I started to
suggest that svn may be a more appropriate location for the
java-repository, because at least then we can have much greater control
of these characteristics as well as historical logging of changes if any
do occur.
-Mark
Henk P. Penning wrote:
On Tue, 5 Oct 2004, Mark R. Diggory wrote:
Date: Tue, 05 Oct 2004 13:04:40 -0400
From: Mark R. Diggory <[EMAIL PROTECTED]>
To: Henk P. Penning <[EMAIL PROTECTED]>
Cc: [EMAIL PROTECTED], dion <[EMAIL PROTECTED]>
Subject: Re: md5's
I just wanted to verify if this was corrected, I saw the jelly files
that were not matching yesterday, I went back today to look into fixing
these and I believe that Dion made some corrections? Or was it someone else?
Not getting a reaction, I waved the problems through, this morning.
I mean, the 'problem' was that the files changed, together with the
md'5 (The checker notices, because it remembers the original md5's).
In my database, I set the 'orig md5' to the 'current md5'.
All I was looking for was a message saying: yes, I changed the files.
If you didn't change those files, there might be a (security) problem.
I should have been clearer.
Note that the changed files are (again) group writable, so,
some 400 users can change them (everyone in group jakarta).
Since our last exchange, I've seen the recomandation,
to set the umask to '002' ;
http://cvs.apache.org/~bodewig/mirror.html
This makes files group writable by default.
It seems to be common apache practice.
Just curious:
-- Did you change the files ?
-- Is your umask 002 ?
Apmirror,
Isn't is time to change this 'umask 002' practice ?
Even a cronjob like
find dist -type d -exec chmod g+w {} \;
is to be preferred, I think.
As things are now, file ownership means nothing.
Regards.
Henk Penning
-Mark
Henk P. Penning wrote:
Hi,
... and another batch. See
http://www.apache.org/~henkp/md5/
HPP
_
Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_
Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \
Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/
http://www.cs.uu.nl/staff/henkp.html M [EMAIL PROTECTED] \_/
-- Forwarded message --
Date: 1 Oct 2004 06:23:18 -
From: Cron Daemon <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Subject: Cron <[EMAIL PROTECTED]> ( cd /home/henkp/md5 ; /usr/bin/make
-s cron )
*** md5 changed
java-repository/commons-jelly/jars/commons-jelly-20030902.160215.jar
hist=d1e3117b90f697e6503e4ddf76bc0402 curr=b171e535366191e437cff6d64df33561
*** md5 changed
java-repository/commons-jelly/jars/commons-jelly-tags-antlr-20030211.143720.jar
hist=94cc61cbdcdfd3b75139d0ce2725138d curr=fe1ae9e40f3fd66031c781e9030b03b9
*** md5 changed
java-repository/commons-jelly/jars/commons-jelly-tags-define-20030211.142932.jar
hist=8ce6559775be62cfae8df109b2457a9c curr=5cc2cf3c1937887c1573ef1582fb3591
*** md5 changed
java-repository/commons-jelly/jars/commons-jelly-tags-html-20030317.100924.jar
hist=481b3ef3a7787ba232c4e1c43c32fe90 curr=040346a692601e498f4ea246d073f624
*** md5 changed
java-repository/commons-jelly/jars/commons-jelly-tags-interaction-20030211.143817.jar
hist=5e4fdc5465c3219b76aea54d7f2d47f3 curr=5450333754b59865bab146caf84c80df
*** md5 changed
java-repository/commons-jelly/jars/commons-jelly-tags-jsl-20030211.143151.jar
hist=f9d5c4302f9159217456e360217dc8b6 curr=24ea6cfe760c82d0707608fc785bc446
*** md5 changed
java-repository/commons-jelly/jars/commons-jelly-tags-log-20030211.142821.jar
hist=a8caadca9a8b82e0739e22742533592f curr=ba37d770969889069ad4fdddaf79209a
*** md5 changed
java-repository/commons-jelly/jars/commons-jelly-tags-swing-20030211.143925.jar
hist=55b0117e87a2e5b022ba9ed81d4008f8 curr=5bfe39
Re: md5 format decision
This should cause no problems with ASF boxes. We would prefer to see this format used. As well, I'd like to make a recommendation that we make sure that when any files are created in the repository, that they are not group writable. More specifically: Directories: group writable (drwxrwxr-x) Files: not group readable (lrw-r--r--) -Mark Brett Porter wrote: Hi, Was there a final decision made to change the format of md5's to: d1dcb0fbee884bb855bb327b8190af36 commons-collections-3.1.jar ie, md5 hash, 1 space, filename If so, I'd like to drop the change into Maven 1.1. Will this cause any problems for any scripts running on ASF boxes? Cheers, Brett -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: Ant and repositories
Hi Steve, Steve Loughran wrote: Hello, I'm Steve Loughran of the Ant project; Nicolaken said I should get on this mail list 1. I have just added to Ant CVS_HEAD a task to get libraries from a repository; built in support is for maven layouts, though others are possible. This is a great idea. 2. I worry about the security aspects. I dont think it is enough to verify the MD5 signatures, because they are served up on the same (http) server. What should I be doing for verifying remote downloads are the intended ones, or what changes are planned in the near future that our task should ready itself for? Note that the task is focused on JAR/WAR/Ear archives only, so we can do full jar signature checking if that is felt the best solution. And we can ship with the public key of an Apache/Maven/Gump CA to verify signatures. Indeed, the fact that nothing has shipped at all yet (and wont till 1.7 alpha) means that we have time to get things right here -Steve This subject is going to be dependent on the overall capabilities of Maven itself. I think, as Maven moves forward your going to see more requirements for signatures. I think that in your case, all the Ant task would probably maintain is some "warning" or interactive y/n/a/na concerning the signature being missing or bad. This is because no matter what policies we put in place for the ASF Repository, they are but a subset of possible outcomes in Maven. Ultimately, users of the task should be using http://www.ibiblio.org/maven an Apache mirror or another local Maven repository as the target for downloading dependencies and not ever the /dist/java-repository on minotaur directly. In theory. All pgp signatures on files in the repository should have public keys stored somewhere under "KEYS" like other contents of /dist/ but I don't currently think this a well maintained or organized practice in the ASF Repository. It should be better maintained and we've had discussions about improving it. -Mark -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: Synchronization report for Apache Software Foundation
Jason, What is the status of the rsync reporting via email? This is the first time we've actually received an email at [EMAIL PROTECTED] It appears to be caused by a file IO error involved with creating a directory that already exists? -Mark wrote: mesg: ttyname: Operation not supported receiving file list ... done rsync: mkdir /public/html/maven: No such file or directory rsync error: error in file IO (code 11) at main.c(319) You can view the syncronization reports for today here: http://www.ibiblio.org/maven/reports/apache/2004/11/21 -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: [ASF Repository Project Wiki] Updated: Participants
Please drop by the newly migrated project wiki and update your status if your still interested in working in the group. http://wiki.apache.org/ASFRepository/Participants cheers, Mark [EMAIL PROTECTED] wrote: Date: 2004-12-06T15:49:08 Editor: MarkDiggory <[EMAIL PROTECTED]> Wiki: ASF Repository Project Wiki Page: Participants URL: http://wiki.apache.org/ASFRepository/Participants Update Membership Roles. Change Log: -- @@ -1,5 +1,9 @@ People involved in formalizing the ASFRepository + * MarkDiggory + +Previous List of Members (Move back up if your still involved): + * AdamJack * AnouMana * MichalMaczka -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: [java-repository] licenses update + unreleased components]
I'm moving this discussion to [EMAIL PROTECTED] Phil and others, if you could join this list, then we can have a more focused discussion about how to approach this. http://wiki.apache.org/ASFRepository Phil Steitz wrote: I agree with all of your points above. The issue is oversight. I have nothing against public RCs, betas, milestones or nightlies -- on the contrary I like and use all of these things. I just think that other than nightlies, these need to be endorsed in some way by the community if we are going to distribute them. Right now we have some jars out there that, to my knowledge, were never voted on and / or have no community support. I think that these should be removed. I think that we have always agreed that they need to be removed. Its just been an issue of having the time to do it properly. I would take issue with pushing nightlies or alphas to ibiblio.org. We do need to put some pressure on people to release. So, you are OK with milestone builds, but not nightlies? Do we agree that the milestones should have some "status" before they get published and they should also be published with release notes, etc. to the main distribution site? Phil Note, theres already been allot of discussion concerning these issues across the [EMAIL PROTECTED] and [EMAIL PROTECTED] lists. The MOST important issue with nightlies etc has also to do with the mirroring of www.apache.org/dist. Its important to be conservative concerning bandwidth, mirrors and rsync. So theres external pressure there not to be pumping auto build nightlies over rsync. Mainly because theres no need or want to have these things on the mirrors. So to clarify, yes we need to get all interim builds out of java-repository. I don't think we need to work hard at this, 1.) just delete any interim builds (they are already archived on archive.apache.org) 2.) delete and any snapshot symlinks referring to them. 3.) delete and sanpshot-version files referring to an interim build. -Mark -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu
Re: Where to publish Xalan code on http://www.apache.org/dist (fwd)
Mixed terminology again. if the "SNAPSHOT" refers to a fully sanctioned release not an interim or daily build, then the usage is fine. Remember in this case SNAPSHOT is no different than "LATEST" or "CURRENT". I wish we could have Maven folks explore usage of a more accurate terminology for these. -Mark Henk P. Penning wrote: Hi, [ I cut this from another message ; I hope it is correct ] --On Wednesday, January 5, 2005 6:43 AM +1100 Brett Porter <[EMAIL PROTECTED]> wrote: This is for Maven users (and possible future tools, eg I believe Ant 1.7 has repository support) to be able to get your releases easily (releases only - development snapshots to http://cvs.apache.org/repository). This was put in 'java-repository/maven/poms/' just now : Jan 11 11:43 maven-site-plugin-1.6-SNAPSHOT.pom Jan 11 11:43 maven-site-plugin-1.6-SNAPSHOT.pom.md5 Just to make sure : isn't this a 'development snapshot' ? Shouldn't this have gone to 'http://cvs.apache.org/repository' ? HPP _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_ Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/ http://www.cs.uu.nl/staff/henkp.html M [EMAIL PROTECTED] \_/ -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: Where to publish Xalan code on http://www.apache.org/dist (fwd)
Dion, Maybe I formed my ideas earlier in Mavens history when SNAPSHOT was simply a symlink that could be moved to point at any release. Some places in the repository actually have SNAPSHOT symlinks that point to a fully versioned releases... These are not interim builds... lrwxrwxr-x 1 jvanzylapcvs 26 Jan 17 2004 excalibur-logger-SNAPSHOT.jar -> excalibur-logger-1.0.1.jar lrwxrwxr-x 1 rdonkin apcvs 16 Jul 19 15:18 jaxme-rt-SNAPSHOT.jar -> jaxme-rt-0.3.jar lrwxrwxr-x 1 rdonkin apcvs 16 Jul 19 15:17 jaxme-xs-SNAPSHOT.jar -> jaxme-xs-0.3.jar Clearly there is vague definition here if this symlink can point at a release like this. Some are concerned that removing these could cause some folks distros to break. Though, no one in thier right mind should release against a SNAPSHOT. -Mark Dion Gillard wrote: Huh? SNAPSHOT versions are never a 'fully sanctioned release' and are simply a point in time build. It is VERY different from CURRENT in the apache sense which means latest released. AFAIK, there's no confusion over this. -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: Where to publish Xalan code on http://www.apache.org/dist (fwd)
Henk P. Penning wrote: Hm, it would seem the latest sanctioned 'maven' is in /www/www.apache.org/dist/maven/binaries/maven-1.0.2.tar.gz .. unpacking it shows me maven-1.0.2/plugins/maven-site-plugin-1.5.2.jar so the latest sanctioned maven-site-plugin appears to be '1.5.2'. Or isn't it ? Possibly, depends on which version maven decided to package, it could be an older version in the repo. I'm not sure this adiquates portrays the issue. Also, if SNAPSHOT is no different than "LATEST" or "CURRENT", what does "LATEST" or "CURRENT" stand for ?? ( 1, 2, 3 ..) My point exactly! I look at where its used and I see mixed meanings. 1.) It could be a link to a interim build 2.) It could be a copy of a interim build. 3.) It could be a link to a full release. 4.) It could be a copy to a full release. Its used by Maven to set dependency resolution to be dynamic, so that if a dependecy deploys a new version (interim or release) to the repository and moves the SNAPSHOT link/copy to point at it, then the project dependency resolves to use it. I don't know how many projects take advantage of it, we've avoided it over time in the Jakarta Commons because its hard to cut a release when its dependent on an interim build that is very different than the previous release. -Mark -Mark HPP _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_ Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/ http://www.cs.uu.nl/staff/henkp.html M [EMAIL PROTECTED] \_/ -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: Where to publish Xalan code on http://www.apache.org/dist (fwd)
Henk P. Penning wrote: My bad. I copied it to the wrong location. I fixed the JARs, forgot the POMs and will remove them next time I have SSH access. Ok ; fine ; I was afraid I missed something. To generalise a little further, are all *SNAPSHOT* thingies in the repository 'bad' (as in, don't belong there) ? Given the problems we have defining them! I'm going to say YES. How about stuff like this (a *SNAPSHOT* symlink to a date-like *20040825.214656* thing) ? Very BAD ./turbine/poms/maven-turbine-plugin-SNAPSHOT.pom -> maven-turbine-plugin-20040825.214656.pom We have 474 '*SNAPSHOT*' files and 174 symlinks. [EMAIL PROTECTED]: find . -name '*SNAPSHOT*' -type f -ls | wc 4745214 75820 [EMAIL PROTECTED]: find . -name '*SNAPSHOT*' -type l -ls | wc 1742262 32093 Remove ? I've been planning to, I'm just concerned of the impact, I'm trying to be thorough and verify it will not cause others major problems when I remove them. HPP _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_ Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/ http://www.cs.uu.nl/staff/henkp.html M [EMAIL PROTECTED] \_/ -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: Where to publish Xalan code on http://www.apache.org/dist (fwd)
As I was discussing in the jakarta-general list earlier. There are a number of projects with mavan project properties setup with the following parameter. maven.repo.remote=http://www.apache.org/dist/java-repository/,http://www.ibiblio.org Bretts correct that in this case the requests should fall over to ibibilio. It would be wise that Projects probably shouldn't be hardcoding the java-repository here. They should leave assigning this param upto the release manager for the project and use the default for all their development. There is another case: maven.repo.central.directory=/www/www.apache.org/dist/java-repository this is actually bad as well, because the developers cutting interim builds will deploy them accidentally into the java-repository. I feel that this should be avoided as well by everyone but the release manager. -Mark Henk P. Penning wrote: On Tue, 11 Jan 2005, Mark R. Diggory wrote: Date: Tue, 11 Jan 2005 19:31:22 -0500 From: Mark R. Diggory <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Subject: Re: Where to publish Xalan code on http://www.apache.org/dist (fwd) [ I wrote : ] Remove ? [ Mark : ] I've been planning to, I'm just concerned of the impact, I'm trying to be thorough and verify it will not cause others major problems when I remove them. [ but Brett wrote somewhere else : ] Nobody is building against java-repository, they are using ibiblio, so there is no harm in cleaning these up AFAICT. Mark, what kind of problems could result in removing things ? According to Brett we could remove the entire repository, and nobody's build would fail (not that I'm suggesting that :-). Are there other worries ? HPP _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_ Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/ http://www.cs.uu.nl/staff/henkp.html M [EMAIL PROTECTED] \_/ -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: Where to publish Xalan code on http://www.apache.org/dist (fwd)
Brett Porter wrote: There is another case: maven.repo.central.directory=/www/www.apache.org/dist/java-repository By Maven 1.0 this was deprecated. You can now specify multiple deployment targets, so I have: maven.repo.apache=scp://www.apache.org maven.repo.apache.directory=/www/www.apache.org/dist/java-repository and maven.repo.apachecvs=scp://cvs.apache.org maven.repo.apachecvs.directory=/www/cvs.apache.org/repository with maven.repo.list=apachecvs by default. The release manager uses -Dmaven.repo.list=apache to publish to the other location. This can be enforced by permissions if we really have a problem (though once people get harassed about snapshots when they turn up, I think it'll quickly stop). Bingo! Very cool. It's definitly not very clear in the Maven docs that one can do this. I think this should be in the project.properties of all Maven based projects. -Mark -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Re: repository thoughts
Hey guys, I was quiet for a awhile due to the birth of my new daughter a couple weeks ago. To catch back up here, I'll just note that we did some work in the ant checksum task to get it to output in various formats BSD/GNU etc. There was an excellent thread last year that really resolved this subject in the Ant community http://www.ant-tasks.com/msg/17054.html -Mark Henk P. Penning wrote: On Sun, 23 Jan 2005, Brett Porter wrote: Date: Sun, 23 Jan 2005 16:08:22 +1100 From: Brett Porter <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Subject: repository thoughts Hi folks, I've put up basically a TODO list from recent discussions. Sorry it took so long. http://wiki.apache.org/ASFRepository/BrettPorter Did I miss anything? Brett, This is a very good start ; thanks. -- MD5 checksum format : in my experience the format doesn't matter much ; it is always easy to grab a 32 byte hex number from any format. Counter examples are easy to find of course (filenames like '41eab7db2910a04500751bbc2ed7f5bf') but in general, different formats aren't a problem. Perhaps it is wize to state that md5 checksums should be in a '.md5' (lowercase) file because some people put the md5 checksum in a '.MD5' file. -- For cleanup purposes it would be nice if one could link something (X) in the repository, to something (Y) in '/dist/' : we can flag X for removal if Y is gone. Brett Henk Penning _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_ Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/ http://www.cs.uu.nl/staff/henkp.html M [EMAIL PROTECTED] \_/ -- Mark Diggory Open Source Software Developer Apache Jakarta Project http://jakarta.apache.org
Removing interim dated builds from /www/www.apache.org/dist/java-repository
Please excuse the cross post. I'm planning to run some commands on the
java-repository to remove interim builds and SNAPSHOTS. Specifically,
I'll be running:
#!/bin/sh
> LOCATION=/www/www.apache.org/dist/java-repository
>
find ${LOCATION} -name '*200[0-4]*' | while read j;
do
rm -f $j;
done
find ${LOCATION} -name '*SNAPSHOT*' | while read j;
do
rm -f $j;
done
find ${LOCATION} -name '*snapshot-version*' | while read j;
do
rm -f $j;
done
I want to announce this so that folks have an opportunity to object or
make other recommendations.
For anyone wondering, all these files are currently available in
/www/archive.apache.org/dist/java-repository as well. So if anything
needs recovery it can be recovered from there.
-Mark
--
Mark Diggory
Open Source Software Developer
Apache Jakarta Project
http://jakarta.apache.org
Re: Synchronization report for Apache Software Foundation
Yes, I'm hoping though, that Jason would set this to only report when theres actually a transfer. The original script I wrote, which ran on ibiblio behaved so. -Mark Carlos Sanchez wrote: Hi, This is the report of the apache to ibiblio repo sync. I know it can be a bit annoying but isn't this the right place? Cheers Carlos Sanchez -Original Message- From: Niclas Hedhman <[EMAIL PROTECTED]> Date: Sun, 6 Feb 2005 22:39:09 +0800 Subject: Re: Synchronization report for Apache Software Foundation To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] On Sunday 06 February 2005 20:52, [EMAIL PROTECTED] wrote: mesg: ttyname: Operation not supported receiving file list ... done wrote 16 bytes read 128048 bytes 2640.49 bytes/sec total size is 428414805 speedup is 3345.32 You can view the syncronization reports for today here: http://repo1.maven.org/reports/apache/2005/02/06 Can someone inform us what this is all about?? Cheers Niclas
Re: Proposal for a centralized Eclipse update manager site for Apache projects/software
Brett Porter wrote: I echo David's sentiments on this. I'd be interested to know what is in the eclipse metadata you want to attach to the various artifacts in the repository, and whether any of it coincides with existing metadata formats in use here, such as the following for parts of jakarta commons: http://www.apache.org/dist/java-repository/commons-logging/poms/commons-logging-1.0.4.pom http://cvs.apache.org/viewcvs.cgi/gump/project/jakarta-commons.xml Its going to be similar, but specific to Eclipse and wrapped up in a Jar file. More specifically, Eclipse plugins are now based in OSGI standards: http://www.osgi.org/ To get even more specific plugins require a Manifest file and either of the following (plugin.xml, fragment.xml, or feature.xml). When Eclipse becomes fully OSGI standard, all plugins will be distributed as actual jars whose contents will be accessed using standard java resources, until then these are not real "jars" and get expanded into eclipses plugins directory, thus they contain a heterogeneous mix of static resources, classes and jar archives. The closest thing that I can compare the entire thing to is a Webapplications WAR file. I don't think this is a question of a new repository, but rather what we can do to the existing one to better support what you are trying to achieve. Exactly, we'd probably place "WAR"s into our repository, why not OSGI standard plugins. Someday, in a perfect world, you might actually be talking about the same exact thing when you refer to a a Jar on the classpath and a plugin in Maven, Ant, Eclipse, NetBeans ...) then this becomes ultimately powerful. -Mark Diggory
Re: Proposal for a centralized Eclipse update manager site for Apache projects/software
Jeffrey, Yes as they pointed out I'm moving our responses to the thread to the repository group. Each project is responsible for their repository contents. So, from this standpoint, if you took Axis for example, the Axis group would be responsible for managing the release of any Axis Eclipse plugin artifacts into the repository. Now, the benefit (or burden) of this is that Axis gets canonical control of managing the distribution of such a plugin and can maintain authority of its release cycle. So ultimately, you should hop onto the Axis list and start a discussion there concerning if they are interested in maintaining the plugin. Some important facts about the repository. 1.) The repository under www.apache.org/dist/java-repository is only for full releases of apache projects (not nightly/integration builds). 2.) Its is not advised that you point users directly at the above url, but go through any of the many Apache mirrors. 3.) All released artifacts should be signed by author using both PGP and MD5 signatures. In regards to (2), I'd be interested in seeing how the new mirrors feature of the Eclipse update manager can handle this gracefully. Jeffrey Liu wrote: Hi Mark, Thanks for the clarification. If I understand it correctly, this means potentially I can put plugin distributions into this repository, host a site.xml file somewhere and point to these plugin distributions? Also, this is probably a dumb question, but I'll ask anyways. Who is responsible for putting these release distributions into this repository? Individual projects? Is this done automatically (each build/release)? Thanks, Jeffrey Liu IBM Rational Software - Performance Analyst IBM Toronto Lab. 8200 Warden Ave. Markham, Ontario, L6G 1C7 Internal mail: D3/R8V/8200/MKM (D3-268) T/L: 969 3531 Tel: (905) 413 3531 Fax: (905) 413 4920 [EMAIL PROTECTED] Mark Diggory <[EMAIL PROTECTED]> 05/03/2005 03:58 PM Please respond to general To general@incubator.apache.org cc Subject Re: Proposal for a centralized Eclipse update manager site for Apache projects/software Slow down, to clarify, the repository is not just for "Jar's", it is for any release distributions. An Eclipse plugin for, lets say, Ant, would be a distributable, just like any other binary or src distribution in the repository. In Eclipse, a plugin archive is "poorly" prefixed with the extension "jar", it is really a zip archive containing everything for that plugin (from what I understand, this will change in the future and it will truly be a jar, but that is an aside). The repository can hold these files and referencing them using a site.xml could happen from just about anywhere else (such as the project website). -Mark Diggory Jeffrey Liu wrote: Hi Dion, Thanks for your response. The existing repository won't work. Unlike a Java application, putting JARs on the classpath is not sufficient to make Eclipse aware of these libraries. Eclipse needs to work with something known as an Eclipse plug-in, which is really the JARs themselves plus a couple XML configuration files. So, in order to make use of the Eclipse update manager, we need to bundle the JARs, the XML configuration files and other files such as license/readme/etc together in the form of an Eclipse plug-in. Thanks, Jeffrey Liu IBM Rational Software - Performance Analyst IBM Toronto Lab. 8200 Warden Ave. Markham, Ontario, L6G 1C7 Internal mail: D3/R8V/8200/MKM (D3-268) T/L: 969 3531 Tel: (905) 413 3531 Fax: (905) 413 4920 [EMAIL PROTECTED] Dion Gillard <[EMAIL PROTECTED]> 05/03/2005 10:33 AM Please respond to general To general@incubator.apache.org cc Subject Re: Proposal for a centralized Eclipse update manager site for Apache projects/software Is there a reason we can't reuse the existing repository at http://www.apache.org/dist/java-repository/ ? In your example, Eclipse could be configured to check http://www.apache.org/dist/java-repository/axis/jars/ for an update? On 5/3/05, Jeffrey Liu <[EMAIL PROTECTED]> wrote: Hi, I want to propose a centralized Eclipse update manager site for Apache projects/software. This may not be the correct place to ask this, but I can find a better place to do it, so I decided to start here. If this is not the right place, can somebody point me to the correct location? Thanks! Reason I propose an Eclipse update manager site for Apache projects/software is because Eclipse projects such as the Web Tools Platform (WTP) project often reuses software that are provided by Apache, for example, Axis, Tomcat, Derby, etc... Often times, these Apache software are not redistributed by the Eclipse projects, but instead, they are listed as prerequisites. This means, end users must first download the Eclipse project and all the Apache software prereq by the project, and configure these software in the Eclipse project before they can get started. This is error prone and hampers the out-of-the-box experience. Imagine th
Re: Proposal for a centralized Eclipse update manager site for Apache projects/software
Niclas Hedhman wrote: On Wednesday 04 May 2005 04:12, Jeffrey Liu wrote: I want to propose a centralized Eclipse update manager site for Apache projects/software. Reason I propose an Eclipse update manager site for Apache projects/software IIUYC, and perhaps others have not expressed this distinctly enough; It should be a lot less effort to ask Eclipse to support Maven styled repositories as well as its own system. By doing so, you get what I think you are fishing for, without the need for every Apache and SF.net project to publish their respective artifacts in Eclipse specific format. Thats a little pretentious, so Eclipse should support Maven, but not the other way around? This is not about "every" Apache project having to release Eclipse plugin format, just those that want to. Its trivial for a project to organize an artifact directory for Eclipse plugins/features. And doing so promotes more interoperability between the two. A simple directory structure for eclipse artifacts: /maven/axis/site.xml /maven/axis/plugins/.jar /maven/axis/features/.jar or /maven/axis/eclipse/site.xml /maven/axis/eclipse/plugins/.jar /maven/axis/eclipse/features/.jar On top of that, people over at the DPML (www.dpml.net) are working on providing Maven repository support for Eclipse, by leveraging the Transit sub-project which is capable of referencing artifacts (i.e. content) on Maven-styled organized (and others) repositories. Example; URL url = new URL( "artifact:jar:tomcat/catalina#5.5.9" ); InputStream in = url.openStream(); and the catalina-5.5.9.jar will be searched on the configured repositories. The Eclipse support effort is basically a matter of registration of the artifact: protocol to the OSGi network subsystem, and overcoming Eclipse's inability to use URLs instead of almost always requiring files :o( , so it will take quite a long time before those guys reach all the way we want. In any event, when the external Jar reference is created, the artifact: URI could be used, and with proper support in Eclipse, I think this is a non-issue for ASF projects. Cheers Niclas Yes, thats very elegant, I hope that they are talking with Eclipse folks and OSGi folk, or at least reviewing the specification to establish if common strategies already exist there. But, thats a long way down the road to interoperability. What's hoped for here is a place to store plugin and feature artifacts for an apache project "now". It is not think it much to ask...This IS what Maven was designed to be capable of doing. If one wants to publish their own custom distributable, they can configure Maven to do so. -Mark
Re: Proposal for a centralized Eclipse update manager site for Apache projects/software
Niclas Hedhman wrote: On Wednesday 04 May 2005 20:06, Mark Diggory wrote: Thats a little pretentious, so Eclipse should support Maven, but not the other way around? This is not about "every" Apache project having to release Eclipse plugin format, just those that want to. Its trivial for a project to organize an artifact directory for Eclipse plugins/features. And doing so promotes more interoperability between the two. My assertion is mostly based on the fact that almost 600 top level group names exist on http://www.ibiblio.org/maven, and go around asking the same question to a larger group of those seems to me to be very ineffective, especially since the benefit isn't obvious to those who have to do the work. Sorry if my response sounded abrasive, I'm not trying to be. He was simply asking advice on who to consult on publishing Eclipse artifacts. The ASF Repository list is about discussing this kind of subject. The benefit is in a greater, more useful "ASF Repository". Initially, they thought they would need a separate "repository" to maintain these distributables. They brought the subject up on the Incubator list because of this, the moderator suggested it was a topic for the repository list (which it is). The question has now turned into how to physically structure a eclipse plugin/feature distribution in the ASF repository so that it can be accessed using the Eclipse update mechanism. I don't think the goal was to require all TLP's to begin packaging everything to work in the Eclipse updater. Nor, that they have a responsibility to do so, only that if they wish to do so, that they may. If it ultimately is determined that ASF repository isn't the place for this (which would be sad indeed), then I'd recommend that the plugins just be distributed out of the projects "/www/www.apache.org/dist/" directory at Apache (which ultimately defeats our efforts to have a generic repository architecture). If Maven can generate it without any additional effort, then you have solved it for a chunk of those projects, that's great. I still maintain as a general rule that those who benefit, should also carry the majority of the burden, in this case Eclipse users/developers. Cheers Niclas I think many of us wear multiple hats, working on many different OS projects. The burden become less about "who is responsible" and more about "wheres the best place to maintain code/releases". -Mark
Re: Proposal for a centralized Eclipse update manager site for Apache projects/software
Brett Porter wrote: On 5/5/05, Dion Gillard <[EMAIL PROTECTED]> wrote: Where do you see the Eclipse version number (not the artifact version) fitting in? I was under the impression this would be captured within the archive itself. But should it need to be part of the path, it is probably sensible to make it part of the artifactId/versioning of the file: mylib-1.0-eclipse.jar mylib-1.0-eclipse2.jar Very rough, obviously. I wasn't really thinking of the specifics at this point, however I think it is reasonable to assume that we can or should map it to the repository in some way to leverage that. If it needs to be able to do more, then we can look at that too. I believe this naming scheme is critical for Eclipse, so attempting to redefine like you have above it is a problem. We might even find that the "plugins" directory is unchangeable. So, I'm suggesting that to publish a site in the repository, its going to have to look like this: When Eclipse packages the archive, its not the "Eclipse" version thats Identified, but the plugin version. so for instance, an Eclipse update site for Axis would contain the following: .../plugins/org.apache.axis_1.0.0.jar .../plugins/org.apache.axis_2.0.0.jar .../site.xml The site.xml contains references to the above files. When the jar is installed using the updater it gets expanded under the path in Eclipse: eclipse/plugins/org.apache.axis_2.0.0/... /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_1.0.0.jar /java-repository/axis/eclipse-distributions/plugins/site.xml I'm going to also suggest that The updater will be responsible for determining dependencies, this means that the closest packaging gets to a dependency on a specif c "release" of Eclipse is really about the plugin having different dependencies on the core eclipse runtime. This means that say we had several versions of axis out where the differences the dependencies are on a specific runtime plugin. /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_1.0.0.jar >= org.eclipse.runtime_2.1 /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_1.0.1.jar >= org.eclipse.runtime_2.1 /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_1.0.2.jar >= org.eclipse.runtime_2.1 /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_2.0.0.jar >= org.eclipse.runtime_3.0 /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_2.0.1.jar >= org.eclipse.runtime_3.0 /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_2.0.2.jar >= org.eclipse.runtime_3.0 /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_3.0.0.jar >= org.eclipse.runtime_3.1 /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_3.0.1.jar >= org.eclipse.runtime_3.1 /java-repository/axis/eclipse-distributions/plugins/org.apache.axis_3.0.2.jar >= org.eclipse.runtime_3.1 The dependencies would be determined by the Eclipse update manager via the site.xml, not by the structure of the repository. By reading the plugins dependencies, Eclipse determines which is the best fit for the install. -Mark And what of eclipse features which are a bundling of related plugins and resources? Again, I thought they were all described in the same way and its internal structure decided that. Cheers, Brett
Re: Proposal for a centralized Eclipse update manager site for Apache projects/software
Good stuff, maybe it'd be good to start a separate thread from the Eclipse plugin artifact topic. -Mark Steve Loughran wrote: On 5/4/05, Niclas Hedhman <[EMAIL PROTECTED]> wrote: On Wednesday 04 May 2005 19:34, Steve Loughran wrote: maven repositories are fun because every JAR is on a URL; you can pass them to a URL classloader as is if you want. The .pom also declares dependencies for transitive work; a bit like debian apt-get. I have missed this progress. Can you point me to the implementation of this? nothing in ant yet; maybe maven2. But the idea of a java version of apt-get with less 'one version per system' is a vision that should drive us all. -a gui client that could work with the existing Apache repositories -a client build on AWT for broadest cross platform execution. -some way of pushing out packages to multiple systems in a cluster -support per/user, per-system packages -good auditing to determine package set (with JMX integration) That is, a Java successor to all the rpm management tools. Do you have any pointer to more elaborate ideas in this area?? not yet. Like I said, I was at a configuration workshop last week. It was mostly about XML representations of configuration/state, but we drifted into the Open Source area a few times -Carwen covered JPackage and Config4Gnu jpackage is interesting, even though I'm personally unhappy with it. Its strength is it tries to integrate java package redist with the rpm framework. Its weaknesses are 1. the core project teams dont think about it, need to be coerced into patching scripts and other things to work with jpackage 2. RPM is very superuser-centric. I shouldnt have to be su to install stuff on my account. 3. RPM is only good on redhat/suse/mandrake, and is somewhat brittle there. Partly that's to do with binaries, but also to do with locations of things. For the curious, I was talking about Smartfrog, and demoed a live download and run of Axis tcpmon from the m2 repository. The biggest issue i have there is signatures; we cant sign the JARs without side effects; I can put the SHA1 or MD5 checksums in the deployment descriptor, but that makes it too brittle to change. Incidentally, one pressing problem with the maven repositories is there is no easy way to fetch Sun stuff. That is, if I want to use Axis with mail.jar and activation,jar, you cant get those from the Apache site for legal reasons. Do you propose any solution to the clickthrough licensing problem? Milos, didn't Netbeans itself devise some funky system for the non-distributables and click-thru license approvals?? It publishes stuff encrypted; you need to click through or have a secret decryption key for automated builds. I am getting very close to mounting a maven2 repository of sun stuff at work, just to avoid their clickthrough junk.
Re: Proposal for a centralized Eclipse update manager site for Apache projects/software
Niclas Hedhman wrote: On Wednesday 04 May 2005 21:26, Mark R. Diggory wrote: Sorry if my response sounded abrasive, I'm not trying to be. No problem. And ditto sorry. I think I belong to a group who is not totally Eclipse savvy/friendly, and somewhat negative to an explicit support of Eclipse and not for other IDEs. Niclas I think that as long as we maintain that an Eclipse plugin is just a distributable artifact, then we do not provide any more support for Eclipse than we would for other IDE's. cheers, Mark
