One answer is to use env.SideEffect with a dummy target.
For example, it's best to run spinning-platter-media linking serialized. We
have this in our SConscript (enabled only for magnetic disks, not for SSDs,
mind you):
noParallel = []
def NoParallelLink(env, disable = True):
global noParallel
if disable:
env.SideEffect("#src/dummy_no_parallel", noParallel)
noParallel = []
and then in builders that want something serialized, we add to the noParallel
list.
dll = env.SharedLibrary(target = dest, source = source + obj)
noParallel.append(dll)
and then finally, at the end of the SConscript, we call the NoParallelLink
function (which is attached to our environment for historical reasons, probably
because the original author thought that was the best way to do it).
if GetOption('serialld'):
env.BnetNoParallelLink()
Basically, the SideEffect function says "the commands in this list can't be run
at the same time". We could have actually implemented it as serial calls to
SideEffect, because it's cumulative, it's just the way we chose to do it.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Kenny, Jason L
Sent: Wednesday, November 20, 2013 3:34 PM
To: [email protected]
Subject: Re: [Scons-dev] SCons with shared resource
Technically all tasks in SCons run in parallel, thanks to a GIL. What run in
parallel are the subprocess calls to a command line ( as these turn into waits
from the python point of view). The easy way to do what you suggest its to add
locks around this shared data to force the exeution of one task at a time.
Jason
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of [email protected]
Sent: Wednesday, November 20, 2013 5:03 PM
To: [email protected]
Subject: [Scons-dev] SCons with shared resource
Hi,
I am successfully using scons for batch processing of simulation work.
Unfortunately some of the tasks share resources that prevent them being
executed in parallel.
Other tasks are can be run in parallel.
This means that I need to run scons in single processor mode, meaning some
tasks are unnecessarily serialised.
I am considering modifying the scons code to allow specification of a shared
resource/ mutex for certain tasks.
Does anyone have any advise as to how this is best implemented ?
Thanks,
Alex
Alex Burton
Research Engineer - Solar Thermal - Energy Technology - CSIRO
+61 2 4960 6110
+61 416 594 752
_______________________________________________
Scons-dev mailing list
[email protected]
http://two.pairlist.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
[email protected]
http://two.pairlist.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
[email protected]
http://two.pairlist.net/mailman/listinfo/scons-dev
