Hi Bill, I decided to have the daemon copy the source tree because I wanted
to make sure that once a background build starts, it can complete or fail
without additional source file changes.
About copying the directory: I guess that if the scons scan can correctly
locate every source file and dependency, then only the source and
SConscripts/SConstruct files need to be copied to the temp directory.
I did assume that the SConstruct file was in the top level of each source tree.
It could be changed that the SConscript files are used to indicate the top
level instead. I didn't consider that a usage case.
About memory usage: It looks like the memory usage roughly doubles because the
parent process retains the data from the scan and then the child process
retains the memory during it's build.
My memory stats:
Num source_files=107, num target_files=21, num SConscripts=728.772MB res for
parent process24.5MB res for child process
I will try to add some tests.Where do you suggest putting them?
Can you give me a stub in the Wiki to add documentation?
Sincerely,Ben
On Thursday, January 14, 2016 1:55 PM, Bill Deegan
<[email protected]> wrote:
A few thoughts..
Seems like you are assuming that the SConstruct is in the top directory of a
source tree.
This is not a safe assumption. I've seen builds where the SConscripts are
scattered over several filesystems (building a project from several shared
source repositories).
Can you add some tests?
And documentation?
Any comments on the memory footprint of running this on a larger source tree?
(where SCons is using a fair amount of memory already)
Is it really necessary to make a full copy of the source tree to a temp dir?
How would this impact the use of VariantDir?
-Bill
On Thu, Jan 14, 2016 at 11:56 AM, Schleimer, Ben via Scons-dev
<[email protected]> wrote:
Hi,
I finished up the watch mode as best as I could and posted a pull request
for it here https://bitbucket.org/scons/scons/pull-requests/289/watch-mode/diff
I am hoping that people will try it out for their incremental builds and see it
if is working for them or not and if it helps their iteration time.
I tried to minimize changes in the core scons code. The only major change I had
to make was to add
SCons.Script._SConscript.global_list_of_processed_sconscipt_files
This is used to track where the SConscript files are so that a commonprefix can
be established for all of the SConstruct files.
Sincerely
Ben
On Tuesday, December 29, 2015 8:23 PM, "Schleimer, Ben via Scons-dev"
<[email protected]> wrote:
>
>
>Hi William,
>
> The non-interactive timings are not affected by the patch.
>
>
>Besides, I'm giving up on that patch and writing a completely separate Watch
>mode. It'll be started by "scons --watch" and it'll be a background
>compilation triggered by file changes.
>
>
>
>Once I get something simple working, I'll post the pull request so people can
>criticize it and hopefully improve it so that it works in a general way.
>
>
>Sincerely
>Ben
>
>
>
>
>On Tuesday, December 29, 2015 7:25 PM, William Blevins <[email protected]>
>wrote:
>
>
>>
>>
>>No, I just wanted the profile from before the patch and after the patch on
>>the same code. Unless I misunderstood your output from last time, you only
>>supplied the information regarding the patched version correct?
>>
>>
>>V/R,
>>William
>>
>>
>>On Tue, Dec 29, 2015 at 11:25 PM, Schleimer, Ben via Scons-dev
>><[email protected]> wrote:
>>
>>Hi,
>>>
>>>
>>>
>>>> Try running with --debug=time
>>>
>>>
>>>Did you want to see all of the build output?
>>>
>>>
>>>Ben
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>On Tuesday, December 29, 2015 2:57 PM, Bill Deegan
>>><[email protected]> wrote:
>>>
>>>
>>>>
>>>>
>>>>Try running with --debug=time
>>>>
>>>>
>>>>--debug=time
>>>>Prints various time profiling information:
the time spent executing each individual build command;
the total build time (time SCons ran from beginning to end);
the total time spent reading and executing SConscript files;
the total time spent SCons itself spend running
(that is, not counting reading and executing SConscript files);
and both the total time spent executing all build commands
and the elapsed wall-clock time spent executing those build commands.
(When scons is executed without the -j option,
the elapsed wall-clock time will typically
be slightly longer than the total time spent
executing all the build commands,
due to the SCons processing that takes place
in between executing each command.
When scons is executed with the -j option,
and your build configuration allows good parallelization,
the elapsed wall-clock time should
be significantly smaller than the
total time spent executing all the build commands,
since multiple build commands and
intervening SCons processing
should take place in parallel.)
>>>>-Bill
>>>>
>>>>
>>>>
>>>>
>>>>On Tue, Dec 29, 2015 at 2:47 PM, William Blevins <[email protected]>
>>>>wrote:
>>>>
>>>>I guess my point here was that I wanted to know where the time savings was
>>>>happening since SCons does split out some process timings.
>>>>>
>>>>>
>>>>>I just wanted to see a regular build timing with and without your patch.
>>>>>
>>>>>
>>>>>V/R,
>>>>>William
>>>>>
>>>>>
>>>>>On Tue, Dec 29, 2015 at 7:07 PM, Schleimer, Ben via Scons-dev
>>>>><[email protected]> wrote:
>>>>>
>>>>>Hi William,
>>>>>>
>>>>>>
>>>>>>>Can you give the time saving using --debug=time for -j1 and -j4?
>>>>>>>
>>>>>>
>>>>>> Sure,
>>>>>>
>>>>>>regular scons build (scons -jX --debug=time)
>>>>>>a clean build with -j1 is:
>>>>>>Total build time: 20.901390 seconds
>>>>>>Total SConscript file execution time: 0.149543 seconds
>>>>>>Total SCons execution time: 0.444850 seconds
>>>>>>Total command execution time: 20.306997 seconds
>>>>>>
>>>>>>
>>>>>>an incremental build with -j1 is:
>>>>>>Total build time: 2.439754 seconds
>>>>>>Total SConscript file execution time: 0.151794 seconds
>>>>>>Total SCons execution time: 0.337350 seconds
>>>>>>Total command execution time: 1.950610 seconds
>>>>>>
>>>>>>
>>>>>>a clean build with -j4 is:
>>>>>>Total build time: 7.158301 seconds
>>>>>>Total SConscript file execution time: 0.153021 seconds
>>>>>>Total SCons execution time: 0.107556 seconds
>>>>>>Total command execution time: 6.897724 seconds
>>>>>>
>>>>>>
>>>>>>an incremental build with -j4 is:
>>>>>>Total build time: 2.368260 seconds
>>>>>>Total SConscript file execution time: 0.151281 seconds
>>>>>>Total SCons execution time: 0.183930 seconds
>>>>>>Total command execution time: 2.033049 seconds
>>>>>>
>>>>>>
>>>>>>interactive build (scons --interactive -jX --debug=time)
>>>>>>a clean build with -j1:
>>>>>>time to do a build = 19.983217001 sec
>>>>>>
>>>>>>an incremental build with -j1:
>>>>>>time to do a build = 2.2111852169 sec
>>>>>>
>>>>>>a clean build with -j4:
>>>>>>time to do a build = 6.93614792824 sec
>>>>>>
>>>>>>an incremental build with -j4:
>>>>>>time to do a build = 2.17438697815 sec
>>>>>>
>>>>>>
>>>>>>It's consistently 0.2 seconds faster with the interactive build.
>>>>>>Not a huge amount but I'm not using that many SConscript files.
>>>>>>(5 SConscript files and 2 SConstruct files)
>>>>>>
>>>>>>
>>>>>>
>>>>>>Cheers
>>>>>>
>>>>>>
>>>>>>Ben
>>>>>>_______________________________________________
>>>>>>Scons-dev mailing list
>>>>>>[email protected]
>>>>>>https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>Scons-dev mailing list
>>>>>[email protected]
>>>>>https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>_______________________________________________
>>>Scons-dev mailing list
>>>[email protected]
>>>https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>
>>>
>>
>>
>>
>
>_______________________________________________
>Scons-dev mailing list
>[email protected]
>https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
>
_______________________________________________
Scons-dev mailing list
[email protected]
https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
[email protected]
https://pairlist2.pair.net/mailman/listinfo/scons-dev
