Re: patch to support output synchronization under -j

2011-04-14 Thread Tim Murphy
Right, of course :-) That is the obvious reason for using files, thanks! I was biased against writing things to disc which is probably silly, because it's probably not slow when compared to the size of the job that's running. Stdout and stderr will now all be stdout, BTW (no matter what method o

Re: patch to support output synchronization under -j

2011-04-14 Thread David Boyce
On Thu, Apr 14, 2011 at 3:08 PM, David Boyce wrote: > On Thu, Apr 14, 2011 at 2:10 PM, Paul Smith wrote: >> I'm not sure I like the idea of having to define a lockfile to enable >> this feature though.  It feels a little too much like exposing the >> implementation details to the user. > > Agree.

Re: patch to support output synchronization under -j

2011-04-14 Thread Paul Smith
On Thu, 2011-04-14 at 12:24 -0700, Howard Chu wrote: > > However, you could do both with some heuristics. Hm. Maybe not. I was > > going to say you could merge them if stdout and stderr were going to the > > same tty or file, but I don't think there's any good way in UNIX to know > > whether two

Re: patch to support output synchronization under -j

2011-04-14 Thread Paul Smith
On Thu, 2011-04-14 at 20:59 +0100, Tim Murphy wrote: > To prevent any kind of deadlock you sort of want to empty everything > the stderr and stdout pipe buffers may contain. > > It is conceivable that the stdout pipe might contain two lines of text > and stderr pipe might contain 1 by the time you

Re: patch to support output synchronization under -j

2011-04-14 Thread Tim Murphy
Ok, To prevent any kind of deadlock you sort of want to empty everything the stderr and stdout pipe buffers may contain. It is conceivable that the stdout pipe might contain two lines of text and stderr pipe might contain 1 by the time your select statement (or waitformultipleobjects) has woken u

Re: patch to support output synchronization under -j

2011-04-14 Thread Tim Murphy
That was a confused email - apologies. I will rethink it and return. :-) On 14 April 2011 20:43, Tim Murphy wrote: > -- Forwarded message -- > From: Tim Murphy > Date: 14 April 2011 20:43 > Subject: Re: patch to support output synchronization under -j > To: psm...@gnu.org > > >

Fwd: patch to support output synchronization under -j

2011-04-14 Thread Tim Murphy
-- Forwarded message -- From: Tim Murphy Date: 14 April 2011 20:43 Subject: Re: patch to support output synchronization under -j To: psm...@gnu.org The reason for splitting stderr and stdout is to do with deadlock and reading pipes. IIRC.  e.g. blocking on a read to stderr which

Re: patch to support output synchronization under -j

2011-04-14 Thread Howard Chu
Paul Smith wrote: On Thu, 2011-04-14 at 14:08 -0400, David Boyce wrote: Of course, either way some context is lost. If you put both into one temp file you lose track of which was which; if you keep them in separate files you lose ordering instead. So it becomes a matter of taste, or perhaps an o

Re: patch to support output synchronization under -j

2011-04-14 Thread Paul Smith
On Thu, 2011-04-14 at 14:08 -0400, David Boyce wrote: > On Thu, Apr 14, 2011 at 1:29 PM, Paul Smith wrote: > > One example: I think saving stdout and stderr to different files and > > then printing them separately is problematic; consider if your recipe > > prints lots of information lines, with e

Re: patch to support output synchronization under -j

2011-04-14 Thread David Boyce
On Thu, Apr 14, 2011 at 3:00 PM, Eli Zaretskii wrote: > Yes, but a few words about how is this semaphore supposed to get job > done, and in fact what kind of "synchronization" will this bring to > Make, would be appreciated.  I don't think you described the feature > too much in your original post

Re: patch to support output synchronization under -j

2011-04-14 Thread David Boyce
On Thu, Apr 14, 2011 at 2:10 PM, Paul Smith wrote: > There is no specific region of the file that's locked: the lockfile is > basically a file-based, system-wide semaphore. Yes, it's conceptually a semaphore. In fact a Windows port might prefer to use real semaphores. The reason I stayed away fro

Re: patch to support output synchronization under -j

2011-04-14 Thread J.T. Conklin
Paul Smith writes: > One example: I think saving stdout and stderr to different files and > then printing them separately is problematic; consider if your recipe > prints lots of information lines, with errors (to stdout) interspersed. > If you throw all the errors to the end you lose a lot of con

Re: patch to support output synchronization under -j

2011-04-14 Thread Eli Zaretskii
> Date: Thu, 14 Apr 2011 14:12:16 -0400 > From: David Boyce > Cc: psm...@gnu.org, bug-make@gnu.org > > On Thu, Apr 14, 2011 at 1:48 PM, Eli Zaretskii wrote: > > David, can you explain why you needed to lock the files?  Also, what > > region(s) of the file you are locking?  fcntl with F_WRLCK won

Re: patch to support output synchronization under -j

2011-04-14 Thread Tim Murphy
File locking would be the exclusion mechanism with a shell implementation, right? With Talon we used a system semaphore with timeout-wait semantics . I would assume that one doesn't need it if the code is in make since make can choose when to start and stop reading from any particular file handle

Re: patch to support output synchronization under -j

2011-04-14 Thread David Boyce
On Thu, Apr 14, 2011 at 1:48 PM, Eli Zaretskii wrote: > David, can you explain why you needed to lock the files?  Also, what > region(s) of the file you are locking?  fcntl with F_WRLCK won't work > on Windows, so the question is how to emulate it. I was about to write this up but I see Paul beat

Re: patch to support output synchronization under -j

2011-04-14 Thread Paul Smith
On Thu, 2011-04-14 at 20:48 +0300, Eli Zaretskii wrote: > > From: Paul Smith > > Date: Thu, 14 Apr 2011 13:29:09 -0400 > > > > On Thu, 2011-04-14 at 11:01 -0400, David Boyce wrote: > > > On Tue, Apr 12, 2011 at 1:46 PM, David Boyce > > > wrote: > > > > So I've made a proof-of-concept patch > >

Re: patch to support output synchronization under -j

2011-04-14 Thread David Boyce
On Thu, Apr 14, 2011 at 1:29 PM, Paul Smith wrote: > I've looked at it and as a concept I don't have too many issues with it Thanks. > One example: I think saving stdout and stderr to different files and > then printing them separately is problematic; consider if your recipe > prints lots of inf

Re: patch to support output synchronization under -j

2011-04-14 Thread Eli Zaretskii
> From: Paul Smith > Date: Thu, 14 Apr 2011 13:29:09 -0400 > > On Thu, 2011-04-14 at 11:01 -0400, David Boyce wrote: > > On Tue, Apr 12, 2011 at 1:46 PM, David Boyce > > wrote: > > > So I've made a proof-of-concept patch > > > against 3.82.90 which seems to work without that overhead and my > >

Re: patch to support output synchronization under -j

2011-04-14 Thread Paul Smith
On Thu, 2011-04-14 at 11:01 -0400, David Boyce wrote: > On Tue, Apr 12, 2011 at 1:46 PM, David Boyce wrote: > > So I've made a proof-of-concept patch > > against 3.82.90 which seems to work without that overhead and my > > question is, would this be of interest towards 3.83? > > Ping? > > The or

Re: patch to support output synchronization under -j

2011-04-14 Thread David Boyce
On Tue, Apr 12, 2011 at 1:46 PM, David Boyce wrote: > So I've made a proof-of-concept patch > against 3.82.90 which seems to work without that overhead and my > question is, would this be of interest towards 3.83? Ping? The original patch attachment was made by hand using "diff -u" and I had som