Jason van Zyl wrote:
On Thu, 2003-11-06 at 12:49, Leo Simons wrote:
Jason van Zyl wrote:
What I would simply like to do is have a directory structure where
projects can place there artifacts and have them rsync'd over to ibiblio
on a regular basis.
<snip/>
yes it does; IIRC the repository project is currently a "president's committee" thatIt might also be a good idea to subscribe to [EMAIL PROTECTED] and talk to
the people over there about this.
Yup, I did a week ago when told about its existence. But all I simply want is the existing structure to be used for now and I will participate as time permits. Why is there a separate mailing list? Doesn't the repository fall under the purview of infrastructure?
was charged with figuring out how to integrate jar files into the distribution layout.
Note that the avalon project already has its artifacts published in a
maven-compatible manner (while still following the mirroring guidelines). One
way to extract the jar files only is (I think):
rsync -rtlzv \ --exclude='*' \ --include='*.jar' \ www.apache.org::apache-dist/avalon/ \ /local/path/to/repo|
This is nice, because it allows the use of all ASF mirrors as maven repositories:
That sounds like the most sensible plan whereby the infrastructure requirements are satisfied and the Maven structure can be satisfied as well. Do you think you would have time to help me get this setup?
can't remember the last time I had time for anything....but I'll JFDI :D
It is all rather trivial. AIUI, there's only two things to do:
1) set up a cronjob on ibiblio that runs the appropriate rsync commands
2) get project artifacts in place on the asf dist site in maven-compatible layout
IMHO, projects that want their artifacts published should take care of that
themselves. But we can make this terribly easy. Take a look at the attached
bash script; it will automagically retrieve all jar files of all top-level projects
and attempt to place them in a sensible location. All you need to do (I think)
is set up a crontab on the ibiblio machine that runs this script.
At the moment, this will only actually work for avalon, but its about as easy
as we can make things for other projects to manage their maven repository.
Projects that do not wish to include their jars in their main distribution layout
still have the option of placing them at the appropriate location in
java-repository.
We could, of course, run this script locally on minotaur to autofill the java-repository but I'm a bit weary of hacking the distribution directory like that.
http://avalon.apache.org/download.cgi#Maven%20Repository
Ah cool, I think this would be easy enough to setup for any project with a few scripts.
indeed. Its trivial. Here's download.cgi:
[EMAIL PROTECTED] ~]$ cat /www/avalon.apache.org/download.cgi #!/bin/sh cd /www/www.apache.org/dyn/mirrors /www/www.apache.org/dyn/mirrors/mirrors.cgi $*
the relevant snippet of download.html:
<p><code> maven.repo.remote = [preferred]/avalon,http://www.ibiblio.org/maven </code></p>
finally, a line like this is required in mirrors.conf:
[avalon.apache.org] download.cgi = /www/avalon.apache.org/download.html
----
lemme know if you need anything else.
cheers!
- Leo
#!/bin/bash
# # A trivial script which walks the apache distribution site # using rsync, retrieving only the jar files and license files # that correspond to the maven repository layout. # When done, it merges all of the TLP repositories into the # common java-repository. # # In other words, we match files like # # /www/www.apache.org/dist/avalon/excalibur-i18n/jars/excalibur-18n-1.0.jar # /www/www.apache.org/dist/avalon/excalibur-i18n/LICENSE.txt # /www/www.apache.org/dist/java-repository/some-component/jars/somefile.jar # # but nothing else. # # You could use this script to keep your local maven repository # up to date, or you could use it to manage a remote repository. # It should be easy enough to adapt it to be more flexible. It should # also be possible to migrate this kind of functionality to an ant # or jelly script. Have fun! # # # - Leo Simons # contact at [EMAIL PROTECTED] # the location of your actual local maven repository REPODIR=~/.maven/repository # the location of your ASF-only repository ASF_REPODIR=~/.maven/asf-repository # the file we will redirect all output to LOG=$REPODIR/asf-repository-auto-sync.log # end of config ################################################ # ensure directories exist mkdir -p $REPODIR mkdir -p $ASF_REPODIR echo Updating ASF artifacts... > $LOG echo Start date: `date` >> $LOG echo Working directory: $ASF_REPODIR >> $LOG echo Destination directory: $REPODIR >> $LOG echo ------------------------------------------------------ >> $LOG echo Running rsync against the ASF distribution site... >> $LOG echo ------------------------------------------------------ >> $LOG # run rsync rsync -rtlzv \ --delete \ --include=*/ \ --include=*/*/ \ --include=*/*/jars/ \ --include=*/*/jars/*.jar \ --include=*/*/LICENSE* \ --include=*/*/jars/LICENSE* \ --exclude=** \ www.apache.org::apache-dist $ASF_REPODIR 2>&1 >> $LOG echo ------------------------------------------------------ >> $LOG echo Removing all empty directories again... >> $LOG echo ------------------------------------------------------ >> $LOG # remove all empty directories for i in `find $ASF_REPODIR -type d 2>>$LOG`; do contents=`find $i -type f ! -name LICENSE* 2>>$LOG` if [[ -z $contents ]]; then rm -Rf $i fi done echo ------------------------------------------------------ >> $LOG echo Merging the ASF artifacts into the main repository... >> $LOG echo ------------------------------------------------------ >> $LOG # merge all the asf top-level proejct repositories into the main repository for i in `find $ASF_REPODIR -type d -maxdepth 1 -mindepth 1 2>>$LOG`; do rsync -rtlzv $i/ $REPODIR 2>&1 >> $LOG done echo echo ------------------------------------------------------ >> $LOG echo 'All done! Time is now:' `date` >> $LOG echo ------------------------------------------------------ >> $LOG
