On 02.04.2014 23:38, Gary Oberbrunner wrote:
On Wed, Apr 2, 2014 at 4:51 PM, Dirk Bächle <tshor...@gmx.de <mailto:tshor...@gmx.de>> wrote:This idea may be feasible, but I'd rather try to get the actual shell spawning to be as fast as possible. We have some valid approaches for this, so let's try them out...maybe one of them is fast enough, such that we don't have to care about the "extra" work mentioned above anymore. Speeding up the spawn/fork stuff would be more transparent to the user than trying to "detect" which commands need a full shell and which don't.They're orthogonal. Both useful, but either can be pursued independently of the other. Avoiding shells will be most valuable in builds with lots of tiny commands (could halve the build time). Avoiding fork is most valuable when SCons is using lots of memory (which it often is). My sense, based largely on Dirk's research, is fixing spawn has a bigger payoff right now.
Since we now know that the "fork" problem is something to care about, I'd really like to use the current momentum. I don't want to push anybody, I want us to have a clear vision about how to take steps in the right direction.
Is someone working on this right now? If not, who volunteers? Let's just communicate...and be on the same page about knowing *who* does *what*, and *when*.
What's a little behind this is, that I think this point has some strategic importance. It's a single issue that keeps a lot of users from switching to SCons. By getting it out of the way, we can present (and kind of "sell") our project much better in any media out there. I have submitted a proposal for the poster session at the EuroPython 2014 in Berlin. It's title is "How spawning many processes sequentially can kill performance", and if I could not only talk about our problem, but also present a solution that has some benefit for the Python community, that definitely carries some potential in my opinion.
So let's really get this crackin'! ;) Best regards, Dirk
_______________________________________________ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
