On 05/16/2010 06:18 PM, Adam Murdoch wrote:
On 13/05/10 6:56 AM, Kurt Harriger wrote:
I'm having a slight problem with the execution order.
I like how goals in maven are attached to a lifecycle. I thought this
would be trivial to replicate in gradle using tasks and dependsOn but
things aren't working quite the way I had hoped.
Here is an example:
-------------- build.gradle --------------
// default lifecycle (based on maven lifecycle)
task validate
task initialize( dependsOn: validate )
task generateSources( dependsOn: initialize )
task processSources( dependsOn: generateSources )
task generateResources( dependsOn: processSources )
task processResources( dependsOn: generateResources )
task compile( dependsOn: processResources )
task processClasses( dependsOn: compile )
task generateTestSources( dependsOn: processClasses )
task processTestSources( dependsOn: generateTestSources )
task generateTestResources( dependsOn: processTestSources )
task processTestResources( dependsOn: generateTestResources )
task testCompile(dependsOn: processTestResources )
task processTestClasses(dependsOn: testCompile )
task test(dependsOn: processTestClasses )
task preparePackage(dependsOn: test )
task createPackage( dependsOn: preparePackage )
task preIntegrationTest( dependsOn: createPackage )
task integrationTest( dependsOn: preIntegrationTest )
task postIntegrationTest( dependsOn: integrationTest )
task verify( dependsOn: postIntegrationTest )
task install( dependsOn: verify )
task deploy( dependsOn: install )
task beforeMsBuild << {
println "Executing beforeMsBuild"
}
task msbuild(dependsOn: beforeMsBuild) << {
println "Executing MSBuild"
}
task generateSolutionVersionInfo << {
println "Creating solution version info"
}
generateSources.dependsOn generateSolutionVersionInfo
compile.dependsOn msbuild
-------------- output --------------
:msbuild
Executing MSBuild
:generateSolutionVersionInfo
Creating solution version info
:validate UP-TO-DATE
:initialize UP-TO-DATE
:generateSources
:processSources
:generateResources
:processResources
:compile
BUILD SUCCESSFUL
----
My intention here was to execute these tasks as part of that lifecycle
phase, and I expected that the task dependencies would be executed
in-order defined. Howerver, in this case, msbuild is being executed
before generateSolutionVersionInfo.
Technically, speaking msbuild does not depend on any tasks, so from a
DAG perspective this one correct solution to the stated dependencies,
but not the one I'm looking for.
I could change msbuild to dependsOn the processResources task, but I
wanted the reason I did not do this is because I wanted to keep the
lifecycle and implementation separate as maven does. In some cases I
may want to run "gradle msbuild" directly without executing any other
tasks, but adding a dependsOn processResources would force execution
of the lifecycle up to processResources which is not what I want.
Another way to approach this is perhaps to use execute() instead of
dependsOn.
generateSources << { generateSolutionVersionInfo.execute() }
compile << { msbuild.execute() }
However if I use execute the dependsOn is not processed at all and the
tasks are never added to the taskGraph and beforeMsBuild is not
executed which prevents me from creating mini-lifecycles.
------- output -------
W:\build-sandbox\test>gradle compile
:validate UP-TO-DATE
:initialize UP-TO-DATE
:generateSources
Creating solution version info
:processSources
:generateResources
:processResources
:compile
Executing msbuild
BUILD SUCCESSFUL
Is there another way to go about this?
Yes, you might think about not trying to replicate the phases and goals
model. It's a weak model for describing a build. For example, it does
not describe inter-phase dependencies. Integration tests almost never
actually depend on unit tests. Sometimes they do, but they usually
don't. From the model, there's no way for the build tool to determine
one way or the other. And so, it must be conservative and always run the
unit tests to completion before it starts the integration tests. Which
is almost always unnecessary and almost always undesired. Even when
there is some relationship between the unit and integration tests, a
weak model doesn't give the build tool any opportunity to extract
parallel work, such as running the unit and integration tests
concurrently. Similarly, install almost never actually depends on unit
or integration tests, but the weakness of the model means that the build
tool cannot determine whether or not it can perform an install without
running the unit or integration tests, and so it must assume that they
must be run. And that's almost never what the user actually wants.
I like the phases and goals model because it provides a separation
between what and how. Unit testing depends on the system being in the
test phase, but I do not need to specify which tasks are required to
reach that phase. This gives me the flexibility to change the
implementation of prior phases without modifying the unit testing phase.
My testing goals need only know about testing things they don't need
to know about compilation things and I think that is a good thing.
Integration testing is performed after unit testing because unit tests
run are designed to fail fast. I don't want waste any more time then
necessary automated or manual resetting an integration test environment
if a unit test could have caught the bug before the tests were even
started. Likewise you don't usually want to install broken artifacts
into your repository, so the tests should be run before they are
installed.
So, instead, you should think about going the Gradle way, and describing
only the actual dependencies of each task, and let Gradle do the rest.
You might add in some high-level lifecycle tasks, similar to how the
Java plugin adds in 'check', 'assemble' and 'build'. Using this approach
means Gradle can execute the fewest set of tasks to perform the work you
actually want to have happen. It also means that Gradle will (in later
releases) be able to extract the maximum concurrency, and have the most
freedom to distribute the build across multiple machines.
There are places where the maven lifecycle seems a bit unnecessarily
ordered, but the problem itself is ordered. Unit tests can't be run
until compilation has been completed so that dependency will exist
either as dependency on an abstraction (a phase) or as a dependency on a
specific implementation (a task).
If you really want to go with the phases-goals model, you should be able
to code it up. I'd use the dependsOn approach applied by some general
purpose code. Something like this:
// Add a task for each phase
phases = ['phase1', 'phase2']
phases.each {
task "$it"
}
// Add the goals and attach to phases
task someGoal
phase1.dependsOn someGoal
// Wire up the dependencies
afterEvaluate { project ->
for (int i = 1; i < phases.size(); i++ ) {
def phaseTask = project.tasks[phases[i]]
def previousPhase = project.tasks[phases[i-1]
// All the dependencies of current phase (ie goals) depend on the
previous phase
phaseTasks.dependsOn*.previousPhase
}
}
Of course, this drags in too many dependencies. You're pretty much stuck
with that for now. There are a couple of new features which we might add
after the 0.9 release, which would make this work better.
Cool, this is exactly what I was trying to do.
One is to introduce the concept of a weak, or optional, dependency. A
weak dependency, such as task A weakly dependsOn task B, only applies
when both task A and B are already in the DAG and simply applies an
ordering, so that task A must execute before task B. Using this approach
to implement the phases model, you could have the goals weakly depend on
the previous phase, and each phase strongly depend on the previous phase.
A simple in-order tree traversal where nodes are visited only once would
work for me.
Another option is to introduce the concepts of builds (for want of a
better term). A build is basically an ordered sequence of tasks or
builds. When the build is executed, the tasks are executed to completion
in the order specified. Using this approach to implement the phases
model, you could declare each phase as a build and have it sequence the
previous phase first.
I probably don't care about to much about the order of tasks within a
phase and these probably can be parallelized, so build != phase. But I
think lifecycle = build, phase = task could work.
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
