Jason van Zyl wrote:

On Thu, 2003-11-06 at 12:49, Leo Simons wrote:


Jason van Zyl wrote:

What I would simply like to do is have a directory structure where
projects can place there artifacts and have them rsync'd over to ibiblio
on a regular basis.


<snip/>

It might also be a good idea to subscribe to [EMAIL PROTECTED] and talk to
the people over there about this.


Yup, I did a week ago when told about its existence. But all I simply
want is the existing structure to be used for now and I will participate
as time permits. Why is there a separate mailing list? Doesn't the
repository fall under the purview of infrastructure?

yes it does; IIRC the repository project is currently a "president's committee" that
was charged with figuring out how to integrate jar files into the distribution layout.


Note that the avalon project already has its artifacts published in a
maven-compatible manner (while still following the mirroring guidelines). One
way to extract the jar files only is (I think):


rsync -rtlzv \
 --exclude='*' \
 --include='*.jar' \
 www.apache.org::apache-dist/avalon/ \
 /local/path/to/repo|

This is nice, because it allows the use of all ASF mirrors as maven repositories:


That sounds like the most sensible plan whereby the infrastructure
requirements are satisfied and the Maven structure can be satisfied as
well. Do you think you would have time to help me get this setup?

can't remember the last time I had time for anything....but I'll JFDI :D

It is all rather trivial. AIUI, there's only two things to do:

1) set up a cronjob on ibiblio that runs the appropriate rsync commands
2) get project artifacts in place on the asf dist site in maven-compatible layout


IMHO, projects that want their artifacts published should take care of that
themselves. But we can make this terribly easy. Take a look at the attached
bash script; it will automagically retrieve all jar files of all top-level projects
and attempt to place them in a sensible location. All you need to do (I think)
is set up a crontab on the ibiblio machine that runs this script.


At the moment, this will only actually work for avalon, but its about as easy
as we can make things for other projects to manage their maven repository.
Projects that do not wish to include their jars in their main distribution layout
still have the option of placing them at the appropriate location in
java-repository.


We could, of course, run this script locally on minotaur to autofill the
java-repository but I'm a bit weary of hacking the distribution directory
like that.

http://avalon.apache.org/download.cgi#Maven%20Repository


Ah cool, I think this would be easy enough to setup for any project with
a few scripts.

indeed. Its trivial. Here's download.cgi:

[EMAIL PROTECTED] ~]$ cat /www/avalon.apache.org/download.cgi
#!/bin/sh
cd /www/www.apache.org/dyn/mirrors
/www/www.apache.org/dyn/mirrors/mirrors.cgi $*

the relevant snippet of download.html:

<p><code>
maven.repo.remote = [preferred]/avalon,http://www.ibiblio.org/maven
</code></p>

finally, a line like this is required in mirrors.conf:

[avalon.apache.org]
download.cgi = /www/avalon.apache.org/download.html

----

lemme know if you need anything else.

cheers!

- Leo


#!/bin/bash

#
# A trivial script which walks the apache distribution site
# using rsync, retrieving only the jar files and license files
# that correspond to the maven repository layout.
# When done, it merges all of the TLP repositories into the
# common java-repository.
#
# In other words, we match files like
#
# /www/www.apache.org/dist/avalon/excalibur-i18n/jars/excalibur-18n-1.0.jar
# /www/www.apache.org/dist/avalon/excalibur-i18n/LICENSE.txt
# /www/www.apache.org/dist/java-repository/some-component/jars/somefile.jar
#
# but nothing else.
#
# You could use this script to keep your local maven repository
# up to date, or you could use it to manage a remote repository.
# It should be easy enough to adapt it to be more flexible. It should
# also be possible to migrate this kind of functionality to an ant
# or jelly script. Have fun!
#
#
# - Leo Simons
# contact at [EMAIL PROTECTED]

# the location of your actual local maven repository
REPODIR=~/.maven/repository

# the location of your ASF-only repository
ASF_REPODIR=~/.maven/asf-repository

# the file we will redirect all output to
LOG=$REPODIR/asf-repository-auto-sync.log

# end of config
################################################

# ensure directories exist
mkdir -p $REPODIR
mkdir -p $ASF_REPODIR

echo Updating ASF artifacts... > $LOG
echo Start date: `date` >> $LOG
echo Working directory: $ASF_REPODIR >> $LOG
echo Destination directory: $REPODIR >> $LOG
echo ------------------------------------------------------ >> $LOG
echo Running rsync against the ASF distribution site... >> $LOG 
echo ------------------------------------------------------ >> $LOG

# run rsync
rsync -rtlzv \
 --delete \
 --include=*/ \
 --include=*/*/ \
 --include=*/*/jars/ \
 --include=*/*/jars/*.jar \
 --include=*/*/LICENSE* \
 --include=*/*/jars/LICENSE* \
 --exclude=** \
 www.apache.org::apache-dist $ASF_REPODIR 2>&1 >> $LOG

echo ------------------------------------------------------ >> $LOG
echo Removing all empty directories again... >> $LOG
echo ------------------------------------------------------ >> $LOG

# remove all empty directories
for i in `find $ASF_REPODIR -type d 2>>$LOG`; do
  contents=`find $i -type f ! -name LICENSE* 2>>$LOG`
  if [[ -z $contents ]]; then
   rm -Rf $i
  fi
done

echo ------------------------------------------------------ >> $LOG
echo Merging the ASF artifacts into the main repository... >> $LOG
echo ------------------------------------------------------ >> $LOG

# merge all the asf top-level proejct repositories into the main repository
for i in `find $ASF_REPODIR -type d -maxdepth 1 -mindepth 1 2>>$LOG`; do
  rsync -rtlzv $i/ $REPODIR 2>&1 >> $LOG
done

echo
echo ------------------------------------------------------ >> $LOG
echo 'All done! Time is now:' `date` >> $LOG
echo ------------------------------------------------------ >> $LOG


Reply via email to