On 12/12/09 2:17 AM, Steve Appling wrote:
On 12/9/2009 10:44 PM, Adam Murdoch wrote:
On 10/12/09 12:36 AM, Steve Appling wrote:
Can someone explain how the up to date checking logic is supposed to
work. What exactly does it do with the task.inputs and task.output
settings?
Currently, it works like this:
1. if the task does not declare both inputs and outputs (possibly
empty), then assume the task is out-of-date.
2. if any of the output files were not built by the task, then the task
is out-of-date.
3. if any input file has been removed or added to the set of input files
since the task built the output files, then the task is out-of-date.
4. if any of the input files have changed since the task built the
output files, then the task is out-of-date.
5. the task is up-to-date.
We're missing a step between 4. and 5, which I haven't implementing yet
(but will soon):
4a. if any of the task inputs have changed since the task built the
output files, then the task is out-of-date.
Thanks for the description. It also explains why I couldn't get the
out of date checking to work for a task where I was only specifying
the input files. Unfortunately these tasks were writing to an output
directory that was a target for multiple tasks in different projects.
I think that step 2 will keep this from being useful for this task.
The checking should deal with multiple tasks which have overlapping
output directories. I'll add some more int test cases when I rework the
output file tracking.
This doesn't seem to work for the current Copy task. I don't
understand how the up to date code can know which input files are
important (match the CopySpec)
The task tells it. In the case of the Copy task, the getSource() method
is marked with @InputFiles.
and what files in the output a particular input file maps to.
The up-to-date code doesn't care about this. It's just interested in
whether any output file is out-of-date wrt any input file. The mapping
is the task's business.
This seems like knowledge that only the Copy task has. Because of
this, I'm not convinced that the Copy task should use the up to date
checking.
I think it's an excellent candidate for getting the up-to-date checking
right: there's a complex relationship between inputs and outputs, and
it's possible to do an incremental copy. We should try to get the
incremental build API to the point where it's a natural and comfortable
fit for the Copy task.
This will take a few steps. We're just at the first step, where we do
some coarse-grained checking to decide whether to skip or execute. For
this step, we just need to get Copy working correctly, and on average
faster than in 0.8 (which may mean slower for some executions and faster
for others). We can improve performance again, and the API, in later
steps.
I must be missing something here, because it still doesn't make sense
to do this to me. Is this right: The up to date check walks the
filesystem (applying Pattern rules) checking dates and sizes of all
input files that match the patterns and all output files against
cached values to see if anything has changed. If anything has
changed, then it triggers the Copy task which walks the filesystem
again (using Pattern rules) to pick inputs and again checks the dates
of the inputs compared to the outputs before copying. I still don't
see how doing this twice is a good idea.
It's not. But it's easy, and this is just the first cut. There's nothing
special about the Copy task in this respect, the problem applies to all
tasks which have to scan for input files. The problem also exists for
tasks which share the same set of input files, such as doing a
compileJava, javadoc, and checkstyleMain in the same build. So we're
going to have to do some caching at some point.
From my profiling, the scanning is actually quite fast, and there are
other things that have a larger impact on build time. We should tackle
issues starting with the things that have the biggest impact. At some
point, that will be the redundant scans.
Even if you update the the up-to-date checking to make it more
complicated, I don't see how you can improve on walking the file
system once knowing exactly which source file goes with which target
file and comparing them to see if the target is out of date.
I don't think it's necessarily about making it more complicated. I think
we want to chop up what the up-to-date checking does, so the task can
get more involved in parts of it, without having to reimplement everything.
--
Adam Murdoch
Gradle Developer
http://www.gradle.org
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
