Mark Watson wrote:
I should say that I'm running on WindowsXP. On WindowsXP I found pretty much the opposite, minimizings calls to File.exist? provided a tiny improvement, but the change to File.mtime provided a large speedup :p
Well, both changes improve, so why not apply both?
What OS are you running on?
Linux (Ubuntu)
Also it may depend on the project, i.e. the total number of headers and how the dependencies are distributed among the source files. 2008/7/23 Ittay Dror <[EMAIL PROTECTED]>:More dicoveries: It looks like 'File.exists' is slow. I did two changes: - comment the first line in FileTask#needed? - change FileTask#timestamp to: begin File.stat(name).mtime rescue Rake::EARLY end this shaved 0.5 second (20% from the whole run time, or 30% from the time added after adding the header dependencies). ittay Ittay Dror wrote: I applied your change and it sped up the execution by 0.1s :-(. I then just added a 'return Rake::EARLY' at the start of #timestamp. This also didn't improve time. So it must be something with the rake code that calls the timestamp method. Ittay Mark Watson wrote: I ran into a similar issue, the way i got around it was by caching the timestamp the first time File.mtime is called. This brings about its own problems because you don't necessarily know that the file won't be modified outside the file task. If you have a task that generates several files the cached timestamps won't be updated. I tried using Windows API directly instead of File.mtime but this wasn't much faster. I didn't get a chance to look into what make is doing, but i suspect it is caching the timestamps too. Make does allow you to specify several files as the output of one task, possibly as a way around the problems with caching the timestamp. # File task with timestamp speedup. The regular file task will query # the File.mtime for every task that depends on this file. Typically # when compiling c/c++ some header files are included very often, and # if a a regular file task is used to represent header files then the # File.mtime function will be called each time a source file depends # on the header. By caching the timestamp we insure that this # operation occurs only once for each header file. On typical c/c++ # projects where most files depend on a core set of headers, a ~3x # speedup on rake startup time can be expected with this patch. # # WARNING: Caching the timestamp may cause problems if the file is # generated/updated by another task as a side effect. # class Rake::FastFileTask < Rake::FileTask # Time stamp for file task. def timestamp # Cache the timestamp since it is accessed often if instance_variable_defined?(:@timestamp) @timestamp else # exist? and mtime can be done together with one call to File.stat, but # for some reason this takes longer on Windows. Ruby is probably making # several Win32 calls. Opimization would be to call GetFileAttributesEx() # directly. Using Win32API module this is slightly faster, making the # call via a c extension would probably be faster again. if File.exist?(name) @timestamp = File.mtime(name.to_s) # Update the time stamp when the task is executed enhance do @timestamp = Time.now end @timestamp else # Build me! Rake::EARLY end end end end 2008/7/22 Ittay Dror <[EMAIL PROTECTED]>: Hi, I'm using Rake (actually Buildr) to compile C++ files. As part of that I'm loading dependencies on header files. The total amount of dependencies is 869 on 142 obj files. The issue I'm seeing is that it takes a lot of time for Rake to timestamp these files to reach a conclusion that it has nothing to do. Compared to make that takes 0.7 seconds to run, Rake takes 3 seconds. If I remove the creation of the dependencies (but still leave the loading and parsing of the files) rake takes 1 second. I understand that 2 seconds seem like not a long time, but the project I'm working on has over 1000 of source files, so I expect the time to be much larger. Do you think anything can be done to make Rake closer to make? note that without the timestamp checking, the rest of the work doesn't amount to a lot of overhead, so I am hoping it is not the dynamic nature of Ruby that is the cause of the problem (but maybe poor implementation of File(file).mtime?) Thank you, Ittay -- -- Ittay Dror <[EMAIL PROTECTED]> _______________________________________________ Rake-devel mailing list [email protected] http://rubyforge.org/mailman/listinfo/rake-devel _______________________________________________ Rake-devel mailing list [email protected] http://rubyforge.org/mailman/listinfo/rake-devel -- -- Ittay Dror <[EMAIL PROTECTED]> -- -- Ittay Dror <[EMAIL PROTECTED]> _______________________________________________ Rake-devel mailing list [email protected] http://rubyforge.org/mailman/listinfo/rake-devel_______________________________________________ Rake-devel mailing list [email protected] http://rubyforge.org/mailman/listinfo/rake-devel
-- -- Ittay Dror <[EMAIL PROTECTED]>
_______________________________________________ Rake-devel mailing list [email protected] http://rubyforge.org/mailman/listinfo/rake-devel
