On Sep 9, 2009, at 8:27 AM, Hans Dockter wrote:
On Sep 9, 2009, at 1:25 AM, Adam Murdoch wrote:
Hans Dockter wrote:
On Sep 2, 2009, at 1:44 AM, Adam Murdoch wrote:
Hi,
I've just committed fixes for some performance regressions which
have snuck in since the 0.7 release:
- Ported AnnotationProcessingTaskFactory from Groovy to Java, and
added some caching.
- Use ASM instead of Groovy to generate task subclasses which mix
in convention mapping and dynamic properties.
- Cache all scripts. In particular, cache empty/missing init
scripts and the default buildSrc script. More on this below.
Using our performance test multi-project build (which has 25
projects), gradle -t executes in:
Gradle 0.8-20090829161512+1000: 8.41 seconds (this is the
version we're using for gradlew)
Gradle 0.7: 3.98 seconds
Gradle head: 3.58 seconds
So, head is a now a little faster than the 0.7 release.
Also, the developerBuild is down from ~30 min to ~20 min on my
not-particularly-good laptop.
One implication of caching every script is that we don't always
have a directory in which to put the .gradle cache. So, I've
changed the script compilation to cache all scripts under
~/.gradle/scriptCache. An advantage of this is we no longer end
up with .gradle directories scattered all through the source tree
(unless you end up using an internal repository). The downside is
we will need some way to garbage collect this cache. This isn't
actually a new problem, because we needed to solve this any way -
It's just more important now.
There are other areas where we also need a place to store
metadata. In those cases we don't have the situation of possibly
not having a directory.
- Changedetection (at the moment stored in <checkedDir>/.gradle)
Should we move this also to ~/.gradle ?
I think we should cache everything in the same place. But that's
really just my personal preference.
- task history (build/.gradle/task-execution-history)
I thought it makes sense to put the task history in the build dir.
If the output is deleted you might as well delete the history
(which purpose is to control which output needs to be reproduced).
We avoid any GC/stale problems that way.
Except it's not quite right.
I know. We talked about it a little in our last dev conference with
Steve and Tom.
It assumes all tasks which generate output do so into the build
directory. This means that after a clean, we'll incorrectly re-
execute tasks which generate their output somewhere else,
Yes
or which do not generate any output.
Task with no output don't have a history in the current
implementation as the history is part of the output handling. The
history returns if the output was created successfully and the last
time the output was modified. That makes no sense for let's say the
clean task. This behavior is also owned to the fact that we use the
build directory for storing the history (which collides with a clean
history). But the major reason is that right now our only history
use case if for output generation optimization.
The rule for skipping a task is: skip the task if 1) its input
files have changed since last time it successfully executed, and 2)
its output files exist.
That would be the most reliable approach and what we should do that
in the future. Stuff would work out of the box. But for 0.8 we will
go with custom onlyIf rules I think.
They would look like:
test.onlyIf { !test.output.wasSuccessfullyCreated ||
test.output.lastModified < testCompile.output.lastModified ... }
That means if someone manually deletes build output the build would
behave in an erroneous way.
Of course we could check already for output in such custom onlyIf's.
What would be the check for example for the test task? Check if some
test-results exists, check if all necessary test-results exists? I can
imagine that the output checks could become a bit time consuming. So
relying only on the input and that the last execution was successful
is a possible performance optimization but not as reliable.
- Hans
--
Hans Dockter
Gradle Project Manager
http://www.gradle.org
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
