I've basically done what you are talking about with a combination of CruiseControl and IvyRoundup (http://code.google.com/p/ivyroundup/). (I'm cross-posting to IvyRoundup in case anyone there finds this useful or might be able to provide more input.) I have my CruiseControl machine checkout a read-only version of IvyRoundup. It does periodic updates to capture any changes made to the IvyRoundup repository. I then took portions of the build script that comes with IvyRoundup, and created my own build script that stages their source packages, overlays the packages I've added, resolves and retrieves the packages that we use, and publishes them via a filesystem publisher to a directory on our build box that is served up by apache.
Roundup uses a packager resolver so instead of ivy.xml files and jars, it provides ivy.xml and packager.xml files. The jars remain at their original websites or in the Maven repository (that is until they have been resolved). I put my additions in our SVN repository as the directory IvyRepo. I used SVN external and mapped http://ivyroundup.googlecode.com/svn/trunk/src to the directory reposrc, so after a svn co of IvyRepo the directory looks like this: build.xml (shown below) filter.properties (shown below) reposrc (SVN external) local-modules (my own additions) |_ javax.javaee |_ org.w3c.css |_ ... You'll notice that in my ivysettings, I don't ever just go out on the Internet to resolve stuff. This way, we can make whatever is available in IvyRoundup available, or we can weed out packages that we don't want to have internally, we can also add our own 3rd party packages with ivy.xml and packager.xml files. Here are some files to show what I'm talking about. ==================================== My build.xml executed by CruiseControl ==================================== <?xml version="1.0" encoding="UTF-8"?> <!-- ! Builds the local Ivy repository using the IvyRoundup project as the seed ! --> <project name="IvyRepo" default="update" basedir="." xmlns:ivy="urn:org.apache.ivy.ant"> <!-- This is from where apache serves up our repository --> <property name="pub.repo" location="/var/www/html/repository" /> <property name="src.d" value="reposrc" /> <property name="src.ad" location="${src.d}" /> <property name="build.d" value="build" /> <property name="build.ad" location="${build.d}" /> <property name="stage.d" value="stage" /> <property name="stage.ad" location="stage" /> <property name="modules.file" location="${pub.repo}/modules.xml" /> <property name="resolve.d" value="resolve" /> <target name="update" description="Updates the local Ivy Repo." depends="clean, prep, repo, resolve" /> <target name="clean"> <delete dir="${stage.d}" /> <delete dir="cache" /> <delete dir="${resolve.d}" /> <delete dir="${build.d}" /> <delete dir="${pub.repo}" includeEmptyDirs="true"> <exclude name="modules/com.mycompany*/**" /> <!-- This is where CruiseControl is publishing our own build artifacts for team sharing so we don't want to delete our own artifacts. --> <include name="css/**" /> <include name="images/**" /> <include name="modules.xml" /> <include name="xsd/**" /> <include name="xsl/**" /> <include name="**/ivy.xml*" /> <include name="**/packager.xml*" /> </delete> </target> <target name="prep" depends="prep:init, prep:mkdirs" /> <target name="prep:init"> <property name="rev.cmd" value="( svn info ${src.d} 2>/dev/null | grep -E '^Last Changed Rev:' || echo '0' ) | sed 's/^Last Changed Rev: //g'" /> <exec outputproperty="svn.revision" executable="/bin/bash" logError="true" failonerror="true"> <arg value="-c" /> <arg value="${rev.cmd}" /> </exec> <property file="filter.properties" /> </target> <target name="prep:mkdirs"> <mkdir dir="${stage.d}" /> <mkdir dir="${build.d}" /> <mkdir dir="${resolve.d}" /> </target> <target name="repo" depends="repo:copy, repo:modules, repo:xform" /> <target name="repo:copy"> <copy todir="${stage.d}"> <!-- Fist fileset is to get all the supporting files --> <fileset dir="${src.d}"> <include name="images/**" /> <include name="xsl/ivy-doc.xsl" /> <include name="xsl/modules.xsl" /> <include name="css/ivy-style.css" /> <include name="xsd/packager-1.0.xsd" /> <include name="xsd/ivy.xsd" /> </fileset> <!-- Second fileset is to get all the downloaded modules --> <fileset dir="${src.d}"> <include name="modules/**/*" /> </fileset> </copy> <copy todir="${stage.d}/modules" overwrite="true"> <!-- Third fileset is to get the local additions --> <fileset dir="local-modules"> <include name="**/*" /> </fileset> </copy> <copy todir="${pub.repo}"> <fileset dir="${stage.d}"> <exclude name="ivy.xml" /> <exclude name="packager.xml" /> </fileset> </copy> </target> <target name="repo:modules"> <property name="gen.mod.cmd" value="TIMESTAMP=`date -u +'%Y-%m-%d %H:%M:%S %Z'` ; export LC_ALL=C; find ${stage.d}/modules -name ivy.xml -print | sort | awk -F/ -v SVNREV=${svn.revision} -v TIMESTAMP="$TIMESTAMP" -f ${src.ad}/awk/xmlify.awk > ${modules.file}" /> <exec executable="/bin/bash" logError="true" failonerror="true"> <arg value="-c" /> <arg value="${gen.mod.cmd}" /> </exec> </target> <target name="repo:xform" depends="xform:svnsub, xform:ivy, xform:packager"> <!-- Validate all generated XML files --> <xmlvalidate failonerror="true" warn="yes" lenient="no"> <attribute name="http://xml.org/sax/features/namespaces"; value="true"/> <attribute name="http://apache.org/xml/features/validation/schema"; value="true"/> <attribute name="http://apache.org/xml/features/validation/schema-full-checking " value="true"/> <fileset dir="${pub.repo}/modules"> <include name="**/ivy.xml"/> <include name="**/packager.xml"/> </fileset> </xmlvalidate> </target> <target name="xform:svnsub"> <copy file="${src.d}/xsl/ivy-repo.xsl" todir="${build.d}/xsl"> <filterset> <filter token="SVNROOTPATH" value="${ivyroundup.svn.url}" /> </filterset> </copy> </target> <target name="xform:ivy"> <!-- Generate ivy.xml files --> <xslt style="${src.d}/xsl/apply-xsl.xsl" in="${modules.file}" out="/dev/null" force="true"> <param name="filename" expression="ivy.xml"/> <param name="stylesheet" expression="${build.ad}/xsl/ivy-repo.xsl"/> <param name="target.repo" expression="${pub.repo}" /> <param name="source.base" expression="${stage.ad}" /> </xslt> </target> <target name="xform:packager"> <!-- Generate packager.xml files --> <xslt style="${src.d}/xsl/apply-xsl.xsl" in="${modules.file}" out="/dev/null" force="true"> <param name="filename" expression="packager.xml"/> <param name="stylesheet" expression="packager-repo.xsl"/> <param name="target.repo" expression="${pub.repo}" /> <param name="source.base" expression="${stage.ad}" /> </xslt> </target> <target name="resolve" depends="resolve:mkbuild, resolve:do" /> <target name="resolve:mkbuild"> <xslt style="${src.d}/xsl/resolve.xsl" in="${modules.file}/" out="${resolve.d}/build.xml"> <param name="retrieve.pattern" expression="${pub.repo}/modules/[organisation]/[module]/[revision]/[type]s/[artifact]-[revision](-[classifier]).[ext]" /> <param name="resolve.org" expression="${resolve.org}" /> <param name="resolve.mod" expression="${resolve.mod}" /> <param name="resolve.rev" expression="${resolve.rev}" /> </xslt> <copy file="${src.d}/ivy/resolve-settings.xml" todir="${resolve.d}" /> </target> <target name="resolve:do"> <ant dir="${resolve.d}" inheritall="false"> <property name="target.repo" value="${pub.repo}" /> </ant> </target> </project> ==================================== filter.properties (comma separated lists will determine which packages get retrieved, all empty will resolve everything) ==================================== resolve.org=org.apache.commons,org.apache.ant resolve.mod= resolve.rev= ==================================== ivysettings.xml ==================================== <?xml version="1.0" encoding="UTF-8"?> <ivysettings> <settings defaultResolver="default" /> <property name="ivy.local.default.root" value="${ivy.default.ivy.user.dir}/local" override="false" /> <property name="ivy.local.default.ivy.pattern" value="[organisation]/[module]/[revision]/ivy.xml" override="false" /> <property name="ivy.local.default.artifact.pattern" value="[organisation]/[module]/[revision]/[type]s/[module].[ext]" override="false" /> <property name="module.pat" value="modules/[organisation]/[module]/[revision]" /> <property name="ivy.pat" value="ivy.xml" /> <property name="packager.pat" value="packager.xml" /> <property name="artifact.pat" value="[type]s/[artifact]-[revision](-[classifier]).[ext]" /> <property name="repo.ivy.pat" value="${module.pat}/${ivy.pat}" /> <property name="repo.packager.pat" value="${module.pat}/${packager.pat}" /> <property name="repo.artifact.pat" value="${module.pat}/${artifact.pat}" /> <property name="repo.host" value="http://myserver"; /> <property name="repo.root" value="${repo.host}/repository" /> <resolvers> <filesystem name="local" checkmodified="true" changingMatcher="regexp" changingPattern=".*_SNAPSHOT"> <ivy pattern="${ivy.local.default.root}/${repo.ivy.pat}" /> <artifact pattern="${ivy.local.default.root}/${repo.artifact.pat}" /> </filesystem> <url name="shared"> <ivy pattern="${repo.root}/${repo.ivy.pat}" /> <artifact pattern="${repo.root}/${repo.artifact.pat}" /> </url> <filesystem name="smg-repository" local="true"> <ivy pattern="${ivy.publish.d}/${repo.ivy.pat}" /> <artifact pattern="${ivy.publish.d}/${repo.artifact.pat}" /> </filesystem> <packager name="public" buildRoot="${ivy.default.ivy.user.dir}/packager/build" resourceCache="${ivy.default.ivy.user.dir}/packager/cache"> <ivy pattern="${repo.root}/${repo.ivy.pat}" /> <artifact pattern="${repo.root}/${repo.packager.pat}" /> </packager> </resolvers> <!-- <include url="${ivy.default.settings.dir}/ivysettings-public.xml" /> --> <include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml" /> <include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml" /> </ivysettings> On Fri, May 8, 2009 at 9:08 AM, Garima Bathla <[email protected]>wrote: > OK; Seems like a scenario which should be very common? Most of the teams > will not want to depend on public repository all the time - what if a > developer wants to work offline with no internet access without messing > around with settings? > > I am no IVY expert - but below is a sure shot solution that my colleague > has > implemented for our team to work in similar model - go to public repository > only when it is needed. > > So we have 3 types of repositories: > > 1. Thirdparty repositories - where we publish third-party jars that do not > exist on any public repository - we convert them to an ivy module and > publish. > 2. Internal repository - where we publish all our teams specific modules . > 3. public repository - used to resolve the rest of the modules ( we depend > on spring source repo). > > So in your case first of all if you can build company repository > successfully then I assume you will want Ivy to resolve in below order > > 1. Try Company repository ( with ibiblio modules and thirdparty modules?) > 2. Try ibiblio repository > > So in your ivysettings you can define new resolvers i.e. > > Shared resolver using company repository to resolve modules > Public resolver using ibiblio to resolve modules > > and then chain them > > <chain name="chain-resolver" returnFirst="true" > <resolver ref="shared-resolver" > <resolver ref="public-resolver"> > </chain> > > So you will have to modify ivysettings.xml and not ivy.xml. I hope you do > have ivysettings.xml? > > HTH, > > Garima. > > > > > On Fri, May 8, 2009 at 4:25 AM, Kent Larsson <[email protected]> > wrote: > > > Hi, > > > > I have an ivy.xml which looks something like: (for proper formatting: > > http://pastebin.com/f21948262 ) > > > > <?xml version="1.0" encoding="UTF-8"?> > > <ivy-module version="2.0" > > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > > xsi:noNamespaceSchemaLocation=" > http://ant.apache.org/ivy/schemas/ivy.xsd > > "> > > <info > > organisation="net.mycompany" > > module="webcollab" > > status="integration"/> > > <configurations> > > <conf name="runtime" description="" /> > > <conf name="compile" extends="runtime" description="" /> > > </configurations> > > <dependencies> > > <dependency org="org.apache.wicket" name="wicket" > > rev="1.3.5" > > conf="runtime->default;foride->sources,javadoc"> > > <exclude org="org.apache.portals.bridges" > > module="portals-bridges-common"/> > > <exclude org="javax.portlet" > module="portlet-api"/> > > </dependency> > > <dependency org="junit" name="junit" rev="4.5" > > conf="runtime->default;foride->sources,javadoc" /> > > <dependency org="org.apache.wicket" > > name="wicket-spring-annot" > > rev="1.3.5" conf="runtime->default;foride->sources,javadoc"/> > > <dependency org="org.slf4j" name="slf4j-log4j12" > rev="1.5.6" > > conf="runtime->default;foride->sources,javadoc"/> > > <dependency org="org.slf4j" name="jcl-over-slf4j" > > rev="1.5.6" > > conf="runtime->default;foride->sources,javadoc"/> > > <!-- Some dependencies removed --> > > </dependencies> > > </ivy-module> > > > > My current ivy.xml uses the default repository ibiblio in Maven2-mode. > > > > What I would like is: > > > > 1. Have a company repository with copies of everything I usually take > > from ibiblio. > > > > 2. Also have some JAR's in that repository which are not in ibiblio, > > like JavaMail and Activation, both from Sun. > > > > 3. In the Ivy resolution on the client machines, try the company > > repository first, then try ibiblio. > > > > Motivation for 3 is that if I add something to my ivy.xml which is > > available in ibiblio I could get it directly, while my company > > repository might be synchronized with ibiblio once each day. Also if > > my company repository is down it might be nice to fall back on ibiblio > > for non licensed files (and hope that JavaMail etc is in the local > > cache already). > > > > Is there any information out there for this scenario? It seems like it > > would be a common scenario. I can't find any, and I don't know where > > to start. If I just can get some more information I'll try to create a > > blog post about it or something, in case there is a lack of > > information on the net. > > > > Thank you for reading! I hope you can help me out. Have a nice weekend! > > > > Best regards, Kent > > >
