I will get it pushed... life getting in the way. From: [email protected] [mailto:[email protected]] On Behalf Of Eugene Leskinen Sent: Tuesday, February 18, 2014 7:03 AM To: SCons developer list Subject: Re: [Scons-dev] Fix async subprocess execution with proper handling of std* streams
Hi Anatoly, The code supporting I/O redirection in Parts is located in parts/part_logger.py file. Look at http://parts.tigris.org/source/browse/parts/trunk/parts/parts/part_logger.py?revision=486&view=markup The code uses Python threading module to create threads for each stream and then runs a loop in every thready like your code does: line = ' ' try: while line: line = self.pipein.readline() if line: self.output.WriteStream(self.taskId, self.streamId, line) except: # There was an error... that shouldn't happen, but still it did. So we report it # to the caller and close our pipe end so that spawned program won't block self.error = traceback.format_exc() self.pipein.close() self.pipein = None Please note that public repo is a bit out-of-date. I hope Jason will find time to pull up-to-date version from Intel internal repository. 2014-02-18 16:30 GMT+04:00 anatoly techtonik <[email protected]<mailto:[email protected]>>: On Wed, Feb 12, 2014 at 6:25 PM, Kenny, Jason L <[email protected]<mailto:[email protected]>> wrote: >> Fix async subprocess execution with proper handling of std* streams. > > So I fixed the issue with output in Parts. I not saying it could not be > clean up some more. However it solves the output issues, and adds coloring ( > which seems to need some fixing in certain cases.. ie mac mostly) and > logging support. It might not be too hard for someone to move the code over > to SCons and integrated its usage into SCons. Do you have, by any chance a description of output issues? It seems that I've lost my list during the last Windows partition crash. Also, can you pinpoint the parts in Parts that are relevant to the issues? I may have time to port them over to SCons. My approach https://bitbucket.org/techtonik/absub to make subprocess asynchronous by moving processing of input/output streams into separate threads. I don't remember why, but the work stalled. I remember there was some kind of error that I was not able to debug due to distributed nature of the process. I needed to write more tests for that, but it is either I run out of time, multithreaded tests were complicated or I just had an annual Windows filesystem crash. Now I am thinking about replacing concept of input/output streams with the concept of channel, which is a queue of strings with limited size. The queue can be rotated to become unlimited. Can be synchronous to wait (block) until messages are read if the queue is full. This may help to clear confusion around all that input output streams handling, pipes and descriptor inheritances. -- anatoly t. _______________________________________________ Scons-dev mailing list [email protected]<mailto:[email protected]> http://two.pairlist.net/mailman/listinfo/scons-dev -- Regards, Eugene Leskinen
_______________________________________________ Scons-dev mailing list [email protected] http://two.pairlist.net/mailman/listinfo/scons-dev
