Le 12 oct. 2011 à 18:06, Richard Mauri a écrit : > Please comment on a couple enhancements relating to performing an > optimized multi-module incremental build. > > I am driving an effort at Ariba, Inc. to use these approaches to > optimize their large product build system. The work is not yet complete, > and I've been given a green light to contribute this work to Apache > community if it makes sense. > > > 1. Given a set of identified leaf modules that were known to have > changed, perform an optimal transitive rebuild only if a performing a > binary compatibility check of the leaf's artifacts are incompatible with > respect to prior published artifacts. > > This implies that the build/analysis would be performed while traversing > the dependency DAG - not pre-computing a list as is done with > ivy:buildlist. > > The analysis code should be pluggable, and certainly the binary > compatibility checker plug-in would be available. > I have a BCEL based dependency checker that implements the JLS binary > compatibility specification (Ch 13) > > 2. Another plug-in could be a metadata validator, which would compare > the declared ivy.xml dependencies against the external dependencies as > discovered by byte code analysis of just product artifacts. The point of > the validator is to detect that correct first-order dependencies are > declared. This helps ensure that first-order dependencies are declared - > for this to work, an index that maps Java package name to defining > modules is consulted. If there is an ivy dependency that is not in the > discovered set of available defining modules there is an option to fail > the build. The validator also leverages BCEL to get the external > dependencies. > > 3. Another plug-in is an indexer tool that incrementally updates the > java to owning modules index mentioned above. > This also leverages BCEL to report what packages the artifact publishes. > > > So to recap: > > While actually doing the DFS in the dependency DAG, identify leaf > components and call out to a module that: > a) performs the module build > b) calls out the analysis modules that: > > b1) checks binary compatibility of latest artifact vs. prior artifact > and sets a flag that influences whether the next module in the DAG > traversal needs to be transitively rebuilt. > > b2) update the Java pkg to owning module index again by leveraging > BCEL to keep > > b3) validates the artifact's external first-order dependencies from > byte code analysis match what's declared in ivy.xml - optionally fail > the build. > > > I think that captures the gist of it. > > I'm having a hard time explaining to the VP's why nobody else has > thought or implemented this.
probably some have thought of this, but the time to actually implement is often a hard line to cross. > Any helpful feedback on that is > appreciated. I have no particular feedback to give apart that this looks very interesting ! Your presentation of the integration into Ivy makes totally sense. No deep patching in Ivy but a new feature: a DAG dependency traverser based on a set of ModuleDescriptors. Just that it would be fine for me to integrate it into Ivy. If you also provide some plugin/hooks like the ones you need, it would be awesome. Nicolas --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
