Thanks. That would be great. ;) On Tue, Feb 18, 2014 at 5:21 PM, Kenny, Jason L <[email protected]> wrote: > I will get it pushed… life getting in the way. > > > > From: [email protected] [mailto:[email protected]] On > Behalf Of Eugene Leskinen > Sent: Tuesday, February 18, 2014 7:03 AM > To: SCons developer list > Subject: Re: [Scons-dev] Fix async subprocess execution with proper handling > of std* streams > > > > Hi Anatoly, > > > > The code supporting I/O redirection in Parts is located in > parts/part_logger.py file. Look at > http://parts.tigris.org/source/browse/parts/trunk/parts/parts/part_logger.py?revision=486&view=markup > > > > The code uses Python threading module to create threads for each stream and > then runs a loop in every thready like your code does: > > > > line = ' ' > > try: > > while line: > > line = self.pipein.readline() > > if line: > > self.output.WriteStream(self.taskId, self.streamId, > line) > > except: > > # There was an error... that shouldn't happen, but still it did. > So we report it > > # to the caller and close our pipe end so that spawned program > won't block > > self.error = traceback.format_exc() > > self.pipein.close() > > self.pipein = None > > > > Please note that public repo is a bit out-of-date. I hope Jason will find > time to pull up-to-date version from Intel internal repository. > > > > 2014-02-18 16:30 GMT+04:00 anatoly techtonik <[email protected]>: > > On Wed, Feb 12, 2014 at 6:25 PM, Kenny, Jason L <[email protected]> > wrote: >>> Fix async subprocess execution with proper handling of std* streams. >> >> So I fixed the issue with output in Parts. I not saying it could not be >> clean up some more. However it solves the output issues, and adds coloring >> ( >> which seems to need some fixing in certain cases.. ie mac mostly) and >> logging support. It might not be too hard for someone to move the code >> over >> to SCons and integrated its usage into SCons. > > Do you have, by any chance a description of output issues? > It seems that I've lost my list during the last Windows partition crash. > > Also, can you pinpoint the parts in Parts that are relevant to the issues? > I may have time to port them over to SCons. > > My approach > https://bitbucket.org/techtonik/absub > to make subprocess asynchronous by moving processing > of input/output streams into separate threads. I don't remember why, but the > work stalled. I remember there was some kind of error that I was not able to > debug due to distributed nature of the process. I needed to write more tests > for that, but it is either I run out of time, multithreaded tests were > complicated > or I just had an annual Windows filesystem crash. > > Now I am thinking about replacing concept of input/output streams with the > concept of channel, which is a queue of strings with limited size. The queue > can be rotated to become unlimited. Can be synchronous to wait (block) > until messages are read if the queue is full. This may help to clear > confusion > around all that input output streams handling, pipes and descriptor > inheritances. > > -- > anatoly t. > _______________________________________________ > Scons-dev mailing list > [email protected] > http://two.pairlist.net/mailman/listinfo/scons-dev > > > > > > -- > Regards, > Eugene Leskinen > > > _______________________________________________ > Scons-dev mailing list > [email protected] > http://two.pairlist.net/mailman/listinfo/scons-dev >
-- anatoly t. _______________________________________________ Scons-dev mailing list [email protected] http://two.pairlist.net/mailman/listinfo/scons-dev
