Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-10-05 Thread David Boyce
On Mon, Sep 30, 2013 at 3:12 PM, Frank Heckenbach f.heckenb...@fh-soft.dewrote: I tested the new version and found no new issues, so as far as I'm concerned this feature can now be considered finished. Thanks for the initial patch, David, and the integration into GNU make, Paul! And special

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-09-30 Thread Frank Heckenbach
I tested the new version and found no new issues, so as far as I'm concerned this feature can now be considered finished. Thanks for the initial patch, David, and the integration into GNU make, Paul! ___ Bug-make mailing list Bug-make@gnu.org

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-09-29 Thread Paul Smith
As always thanks for your thorough testing Frank. On Tue, 2013-09-24 at 20:41 +0200, Frank Heckenbach wrote: Paul Smith wrote: On Thu, 2013-09-19 at 14:47 +0200, Frank Heckenbach wrote: Paul Smith wrote: I didn't fix it this way. Instead I used the existing MAKE_RESTART

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-09-21 Thread Paul Smith
On Sat, 2013-09-21 at 07:28 +, Edward Welbourne wrote: I've never understood why someone would use $(shell ...) in a recipe... I mean, the recipe will be run in the shell!! I remember we once had a library where the command-line to the archiver was too long (about a quarter megabyte,

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-09-21 Thread Edward Welbourne
No, that wouldn't work. It's not the individual command (between simicolons) that's too long, the problem is that make can't invoke the shell itself because the command line + environment is too large. The only way to work around this limitation is to avoid invoking a single command that's

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-09-20 Thread Frank Heckenbach
Paul Smith wrote: On Thu, 2013-09-19 at 14:47 +0200, Frank Heckenbach wrote: Hm. This is pretty contrived. I have a hard time imagining a real makefile wanting to do this for a good reason. However, it does seem that the solution may be simple enough. I also doubt someone

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-09-19 Thread Frank Heckenbach
Paul Smith wrote: *sigh* If it weren't for the enter/leave messaging, the output-sync feature would have been quite straightforward! :-/ :-). I'm afraid so. But I think we're almost there now. On Mon, 2013-09-16 at 12:18 +0200, Frank Heckenbach wrote: 1. Twice Entering, once

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-09-13 Thread Paul Smith
On Thu, 2013-09-12 at 04:19 -0400, Paul Smith wrote: I think there may still be some change needed for directory tracking for the -Orecurse mode. If we're collecting output for an entire recursive make invocation we don't need enter/leave notifications around each individual target or line,

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-05-05 Thread Paul Smith
On Sun, 2013-05-05 at 04:37 +0200, Frank Heckenbach wrote: COMMANDS_RECURSE _does_ mean to recurse. The reason for the '+' prerequisite is to tell make that this line, even though it may not look like it, will run a recursive make. OK, let me just say that the meaning of recursive may

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-05-04 Thread Frank Heckenbach
: I've tested your changes and so far -O job works for my use cases. I shouldn't have written that. :-( Shortly afterwards, I found a bug or perhaps two: foo: @echo foo +@echo bar (a) % make -Ojob foo bar foo (b) % make -Otarget bar foo As you see, (a) -Ojob writes foo twice

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-05-04 Thread Eli Zaretskii
Date: Sat, 04 May 2013 04:42:32 +0200 Cc: e...@gnu.org, david.s.bo...@gmail.com, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de : /* This is needed to avoid the label at end of compound statement : diagnostics on Posix platforms. */ I don't think that's a POSIX

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-05-04 Thread Paul Smith
On Sat, 2013-05-04 at 08:57 +0200, Frank Heckenbach wrote: I shouldn't have written that. :-( Shortly afterwards, I found a bug or perhaps two: foo: @echo foo +@echo bar (a) % make -Ojob foo bar foo (b) % make -Otarget bar foo As you see, (a) -Ojob writes foo

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-05-04 Thread Frank Heckenbach
Paul Smith wrote: The first one I've seen but hadn't had time to debug. I'll look at your patch. I left the truncate where it was rather than doing it after the sync_output() because I was hoping to avoid truncating a file that we'll never use again anyway, but I guess it isn't a big deal.

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Tim Murphy
Let me add my voice as a user. If you are one of the lucky people whose builds consist mostly of 1 line of output per rule then you will rarely have any trouble in a good build but try interpreting error messages from compiler/tool X when they're 10 lines from the file that they refer to and

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Eli Zaretskii
Date: Mon, 29 Apr 2013 09:58:50 +0100 From: Tim Murphy tnmur...@gmail.com Cc: bug-make@gnu.org bug-make@gnu.org try interpreting error messages from compiler/tool X when they're 10 lines from the file that they refer to and don't include the filename in the error message. That's

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Tim Murphy
On 29 April 2013 16:19, Eli Zaretskii e...@gnu.org wrote: Date: Mon, 29 Apr 2013 09:58:50 +0100 From: Tim Murphy tnmur...@gmail.com Cc: bug-make@gnu.org bug-make@gnu.org try interpreting error messages from compiler/tool X when they're 10 lines from the file that they refer to and

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Eli Zaretskii
Date: Mon, 29 Apr 2013 16:40:03 +0100 From: Tim Murphy tnmur...@gmail.com Cc: bug-make@gnu.org bug-make@gnu.org cc fred.c -c -o fred.o cc bob.c -c -o bob.o error on line 20 -X Which one? Make will actually tell you which one, something like: makefile:342: recipe for target

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Philip Guenther
On Mon, Apr 29, 2013 at 10:30 AM, Tim Murphy tnmur...@gmail.com wrote: cc fred.c -c -o fred.o cc bob.c -c -o bob.o error on line 20 -X error on line 30 - error on line 330 - makefile:342: recipe for target 'fred.o' failed makefile:350: recipe for target 'bob.o' failed ?

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Tim Murphy
Come now - the broken excuse is an excuse. There's plenty of crap free software out there and some poor bastard trying to build it who can't change the source because the people who own it think it should be make's problem. :-) Cheers, Tim On 29 April 2013 19:00, Philip Guenther

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Eli Zaretskii
Date: Mon, 29 Apr 2013 18:30:37 +0100 From: Tim Murphy tnmur...@gmail.com Cc: bug-make@gnu.org bug-make@gnu.org cc fred.c -c -o fred.o cc bob.c -c -o bob.o error on line 20 -X error on line 30 - error on line 330 - makefile:342: recipe for target 'fred.o' failed

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Eli Zaretskii
Date: Mon, 29 Apr 2013 19:33:10 +0100 From: Tim Murphy tnmur...@gmail.com Cc: Eli Zaretskii e...@gnu.org, bug-make@gnu.org bug-make@gnu.org Come now - the broken excuse is an excuse. There's plenty of crap free software out there and some poor bastard trying to build it who can't change

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Tim Murphy
One doesn't have to suffer the problems and learn the option exists afterwards. In the end I can understand why a new feature might not be default to start with - until a lot of people have used it and are sure that it works everywhere. Cheers, Tim On 29 April 2013 20:21, Eli Zaretskii

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-29 Thread Edward Welbourne
Eli: cc fred.c -c -o fred.o cc bob.c -c -o bob.o error on line 20 -X error on line 30 - error on line 330 - makefile:342: recipe for target 'fred.o' failed makefile:350: recipe for target 'bob.o' failed You need to look in both anyway. That is true of the very specific

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-28 Thread David Boyce
On Sun, Apr 28, 2013 at 1:34 AM, Paul Smith psm...@gnu.org wrote: I'm not excited about that term (job); it's kind of accurate, but in the documentation for example we're really mushy about exactly what a job is, vs. a recipe or a command line etc. I'd like to pick some terms for this, define

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-28 Thread Eli Zaretskii
From: Paul Smith psm...@gnu.org Cc: e...@gnu.org, david.s.bo...@gmail.com, bug-make@gnu.org Date: Sun, 28 Apr 2013 01:34:44 -0400 On Thu, 2013-04-18 at 22:36 +0200, Frank Heckenbach wrote: This is useful (to me) because at any time, I know what's running. ([Start] messages minus

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-28 Thread Eli Zaretskii
Date: Sun, 28 Apr 2013 10:22:40 -0400 From: David Boyce david.s.bo...@gmail.com Cc: Frank Heckenbach f.heckenb...@fh-soft.de, Eli Zaretskii e...@gnu.org, bug-make bug-make@gnu.org So I'd argue for: -O line (new) -O job (current -O target) -O make Agree about line (assuming I

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-28 Thread Paul Smith
On Sun, 2013-04-28 at 20:00 +0300, Eli Zaretskii wrote: I've pushed a change to add a new argument to the -O/--output-sync option, job, to write output after each line of the recipe. What is its purpose? To avoid mixing in the same screen line characters from several parallel sub-makes?

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-28 Thread Paul Smith
On Thu, 2013-04-18 at 22:36 +0200, Frank Heckenbach wrote: % make -Omake # same with -Otarget m:2: recipe for target 'foo' failed make: *** [foo] Error 1 foo:error This seems at least strange to me: The conclusion recipe failed is printed before the reason (the messages from the job).

Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-28 Thread Paul Smith
Now that we seem to have a workable solution for output synchronization for both POSIX and Windows systems, I wonder if we shouldn't consider enabling it as the default mode when parallel builds are running. I understand that this will be a change that could be visible (beyond the collection of

Re: Default output-sync setting (was: Re: [bug #33138] .PARLLELSYNC enhancement with patch)

2013-04-28 Thread Eli Zaretskii
From: Paul Smith psm...@gnu.org Cc: bug-make@gnu.org Date: Sun, 28 Apr 2013 22:03:39 -0400 Now that we seem to have a workable solution for output synchronization for both POSIX and Windows systems, I wonder if we shouldn't consider enabling it as the default mode when parallel builds are

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-27 Thread Eli Zaretskii
From: Paul Smith psm...@gnu.org Cc: Frank Heckenbach f.heckenb...@fh-soft.de, bug-make@gnu.org Date: Sun, 21 Apr 2013 19:30:05 -0400 On Fri, 2013-04-19 at 14:09 +0300, Eli Zaretskii wrote: Date: Fri, 19 Apr 2013 11:54:05 +0200 Cc: bo...@kolpackov.net, bug-make@gnu.org From: Frank

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-27 Thread Eli Zaretskii
Date: Sat, 27 Apr 2013 13:09:02 +0300 From: Eli Zaretskii e...@gnu.org Cc: f.heckenb...@fh-soft.de, bug-make@gnu.org The basic feature can be tested trivially like this: all: one two one two: @echo start $@ @sleep 1 @echo stop $@ Now if

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-27 Thread Paul Smith
On Sat, 2013-04-27 at 13:09 +0300, Eli Zaretskii wrote: Is this intended behavior that these two messages: mkfsync:6: recipe for target 'two' failed gnumake[1]: [two] Error 1 (ignored) are separated and wrapped by separate Entering...Leaving blocks, instead of being produced together?

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-27 Thread Paul Smith
On Sat, 2013-04-27 at 14:39 +0300, Eli Zaretskii wrote: The changes needed to make -O work on MS-Windows are now committed to the master repository, see commits da7df54 and 049f8e8. Please review and comment. Thanks Eli!! I'll take a look over the next few days.

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-27 Thread Paul Smith
On Thu, 2013-04-18 at 22:36 +0200, Frank Heckenbach wrote: This is useful (to me) because at any time, I know what's running. ([Start] messages minus [End] messages.) Thanks, this is the reason I was looking for; that use-case wasn't clear to me based on the previous email. OK, so

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-26 Thread Eli Zaretskii
Date: Wed, 24 Apr 2013 21:19:45 +0300 From: Eli Zaretskii e...@gnu.org Cc: bug-make@gnu.org Date: Wed, 24 Apr 2013 19:14:01 +0200 Cc: bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de Testing clearly shows it doesn't: GetFileInformationByHandle simply fails for

Re: Fwd: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-26 Thread Tim Murphy
On 25 April 2013 20:06, Eli Zaretskii e...@gnu.org wrote: Date: Thu, 25 Apr 2013 19:36:28 +0100 From: Tim Murphy tnmur...@gmail.com Cc: bug-make@gnu.org bug-make@gnu.org 1) sem_timedwait() in posix lets you timeout so in a big build when something crashes or just sits around, there

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-26 Thread Eli Zaretskii
Date: Thu, 25 Apr 2013 02:16:33 +0200 Cc: e...@gnu.org, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de On Windows, you said fstat was very expensive, didn't you? Or is lseek even worse? I think anything that potentially moves the file pointer can be sometimes

Re: Fwd: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-26 Thread Eli Zaretskii
Date: Fri, 26 Apr 2013 09:05:18 +0100 From: Tim Murphy tnmur...@gmail.com Cc: Paul D. Smith psm...@gnu.org, bug-make@gnu.org bug-make@gnu.org How much would you use for the timeout, though? A sub-Make could legitimately run for a very long time, depending on what's in the Makefile.

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-26 Thread Frank Heckenbach
Eli Zaretskii wrote: You underestimate me ;-) What I have is actually this: [...] and I wrote 'fcntl' emulation for Windows that uses a mutex. Indeed, I had not expected this. :-) That said, I'm not wedded to the above approach, and if people like a completely disjoint code for

Fwd: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-25 Thread Tim Murphy
-- Forwarded message -- From: Tim Murphy tnmur...@gmail.com Date: 25 April 2013 07:13 Subject: Re: [bug #33138] .PARLLELSYNC enhancement with patch To: Paul D. Smith psm...@gnu.org To be honest, I have done all this before with named semaphores including the file that gets left

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-25 Thread Eli Zaretskii
Date: Thu, 25 Apr 2013 05:14:41 +0200 Cc: psm...@gnu.org, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de My suggestion was under the assumption that we use different declarations because of different types anyway -- which I still recommend, though Paul might have to decide

Re: Fwd: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-25 Thread Paul Smith
On Thu, 2013-04-25 at 07:14 +0100, Tim Murphy wrote: To be honest, I have done all this before with named semaphores including the file that gets left over problem and it's all solvable quite nicely. You pass the build id in the environment which is, after all, what it's for. Sure. Given

Re: Fwd: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-25 Thread Tim Murphy
Ok, I don't want to be too much of a pain - I can see the way things have gone and my attitude is oh well but I thought I'd put the case anyhow: 1) sem_timedwait() in posix lets you timeout so in a big build when something crashes or just sits around, there is at least the option of printing an

Re: Fwd: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-25 Thread Eli Zaretskii
Date: Thu, 25 Apr 2013 19:36:28 +0100 From: Tim Murphy tnmur...@gmail.com Cc: bug-make@gnu.org bug-make@gnu.org 1) sem_timedwait() in posix lets you timeout so in a big build when something crashes or just sits around, there is at least the option of printing an error message or giving up

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Frank Heckenbach
Eli Zaretskii wrote: That's one way. Another one is to discuss the design more thoroughly before the patches are accepted. I think it was discussed quite extensively. Also in retrospect, I don't see how the design could have been much different (see below). I tried to turn the discussion

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
Date: Wed, 24 Apr 2013 05:56:49 +0300 From: Eli Zaretskii e...@gnu.org Cc: f.heckenb...@fh-soft.de, bug-make@gnu.org I will see if locking works on console handles; if not, I will have to introduce a command-line argument for passing the name or the handle of a mutex to a sub-Make. As I

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
Date: Thu, 18 Apr 2013 21:57:56 +0300 From: Eli Zaretskii e...@gnu.org Cc: bug-make@gnu.org, bo...@kolpackov.net Date: Thu, 18 Apr 2013 19:09:06 +0200 From: Frank Heckenbach f.heckenb...@fh-soft.de . calculation of combined_output in start_job_command will need to be

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Frank Heckenbach
Eli Zaretskii wrote: I know about this method, but I'm not sure it does what we want here. The number used by this method is not guaranteed to be unique on some versions of Windows. More importantly, it only works for disk files, I don't know whether it reports a meaningful value for the

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Tim Murphy
Some time ago when solving the same problem in a different way we used semaphores on Windows and Linux. Compatibility might make it less interesting but I would suggest pretending that one has semaphores first with some nice little abstraction and then implementing them in the best way the

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread David Boyce
On Wed, Apr 24, 2013 at 10:26 AM, Tim Murphy tnmur...@gmail.com wrote: I would suggest pretending that one has semaphores first with some nice little abstraction and then implementing them in the best way the platform has to offer. How about we introduce functions called acquire_semaphore()

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
Date: Wed, 24 Apr 2013 18:50:15 +0200 Cc: bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de That's one way. Another one is to discuss the design more thoroughly before the patches are accepted. I think it was discussed quite extensively. Also in retrospect, I don't see

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
Date: Wed, 24 Apr 2013 19:14:01 +0200 Cc: bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de Testing clearly shows it doesn't: GetFileInformationByHandle simply fails for handles open on console devices and the null device. And we will be comparing handles for console

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
Date: Wed, 24 Apr 2013 10:34:20 -0700 From: David Boyce david.s.bo...@gmail.com Cc: Frank Heckenbach f.heckenb...@fh-soft.de, bug-make@gnu.org bug-make@gnu.org On Wed, Apr 24, 2013 at 10:26 AM, Tim Murphy tnmur...@gmail.com wrote: I would suggest pretending that one has

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Paul Smith
On Wed, 2013-04-24 at 21:17 +0300, Eli Zaretskii wrote: There's one issue that perhaps needs discussing. A mutex is identified by a handle, which on Windows is actually a pointer to an opaque object (maintained by the kernel). As such, using just 'int' for sync_handle is not wide enough,

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Frank Heckenbach
Eli Zaretskii wrote: : Btw, there will be other side effects, at least on non-Posix : platforms, due to the fact that stuff that was supposed to go to the : screen is redirected to a file instead. Some programs sense that and : behave differently, e.g. with binary non-printable

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Paul Smith
I'm fully prepared to accept the blame for not doing the best job getting buy-in etc. on this effort. Can we leave the discussion on the process behind? I'd prefer that, unless there are real constructive comments on how to do better next time rather than rehashing what was done wrong. I think

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
From: Paul Smith psm...@gnu.org Cc: e...@gnu.org, bug-make@gnu.org Date: Wed, 24 Apr 2013 15:07:21 -0400 I'm not so sure fstat() is that cheap. struct stat contains a lot of information. Although I guess since we are only ever talking about temp files, not NFS files or something, it's

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
Date: Wed, 24 Apr 2013 20:46:14 +0200 Cc: p...@mad-scientist.net, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de I don't see why it would terminate prematurely It was long ago, but I presume I thought about the ^Z character that some programs write or interpret to

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Paul Smith
On Wed, 2013-04-24 at 22:25 +0300, Eli Zaretskii wrote: From: Paul Smith psm...@gnu.org Cc: e...@gnu.org, bug-make@gnu.org Date: Wed, 24 Apr 2013 15:07:21 -0400 I'm not so sure fstat() is that cheap. struct stat contains a lot of information. Although I guess since we are only ever

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Paul Smith
On Wed, 2013-04-24 at 22:39 +0300, Eli Zaretskii wrote: Nothing is actually read by lseek (and even if it were, it would only need to look at the first and last part of the file, not read all the content, if that was the worry). Are you sure? How can lseek jump to the last byte of the

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Tim Murphy
why not use a named semaphore wherever possible (windows and linux) and lock a file where not instead of trying to pass kernel object handles around (seems a bit nasty to me)? On 24 April 2013 21:19, Paul Smith psm...@gnu.org wrote: On Wed, 2013-04-24 at 22:39 +0300, Eli Zaretskii wrote:

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Paul Smith
On Wed, 2013-04-24 at 22:55 +0100, Tim Murphy wrote: why not use a named semaphore wherever possible (windows and linux) and lock a file where not instead of trying to pass kernel object handles around (seems a bit nasty to me)? Hi Tim; I think you're late to the party :-). Let me summarize a

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Frank Heckenbach
Paul Smith wrote: On Wed, 2013-04-24 at 20:46 +0200, Frank Heckenbach wrote: That's true about SEEK_CUR which was there originally. I actually changed it to SEEK_END, which does move the position to the end. Oh right. My head cold is keeping me foggy. What was the reason to change to

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
From: Paul Smith p...@mad-scientist.net Cc: Frank Heckenbach f.heckenb...@fh-soft.de, bug-make@gnu.org Date: Wed, 24 Apr 2013 16:03:56 -0400 Or maybe we should abandon this optimization and take the lock regardless. How bad can that be? Well, we want to know if the file has any

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Frank Heckenbach
Eli Zaretskii wrote: From: Paul Smith p...@mad-scientist.net Cc: Frank Heckenbach f.heckenb...@fh-soft.de, bug-make@gnu.org Date: Wed, 24 Apr 2013 16:03:56 -0400 Or maybe we should abandon this optimization and take the lock regardless. How bad can that be? Well, we want to

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
Date: Wed, 24 Apr 2013 22:55:59 +0100 From: Tim Murphy tnmur...@gmail.com Cc: bug-make@gnu.org bug-make@gnu.org trying to pass kernel object handles around (seems a bit nasty to me)? Why is that nasty? The handle is returned by a documented interface, and communicating it to another

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Eli Zaretskii
Date: Thu, 25 Apr 2013 02:16:33 +0200 Cc: e...@gnu.org, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de I can't follow you here. On POSIX, we don't need to pass a fd because it's always stdout/stderr. Or do mean something else? I meant the file descriptor passed to

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-24 Thread Frank Heckenbach
Eli Zaretskii wrote: Date: Thu, 25 Apr 2013 02:16:33 +0200 Cc: e...@gnu.org, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de I can't follow you here. On POSIX, we don't need to pass a fd because it's always stdout/stderr. Or do mean something else? I meant

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Eli Zaretskii
Date: Fri, 19 Apr 2013 11:54:05 +0200 Cc: bo...@kolpackov.net, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de Eli Zaretskii wrote: Initial investigation indicates that tmpfile should do the job just fine: the file is deleted only when the last descriptor for it is

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread David Boyce
Since you asked basic questions I'm going to start this at a basic level. Apologies if it covers some stuff you already know or if I misinterpreted the questions. Note that I haven't actually looked at the patch that went in so this is generally wrt the original. The first thing is get the word

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Eli Zaretskii
Date: Tue, 23 Apr 2013 11:29:35 -0700 From: David Boyce david.s.bo...@gmail.com Cc: Frank Heckenbach f.heckenb...@fh-soft.de, bug-make bug-make@gnu.org Since you asked basic questions I'm going to start this at a basic level. Apologies if it covers some stuff you already know or if I

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Paul Smith
On Tue, 2013-04-23 at 21:40 +0300, Eli Zaretskii wrote: Date: Tue, 23 Apr 2013 11:29:35 -0700 From: David Boyce david.s.bo...@gmail.com Cc: Frank Heckenbach f.heckenb...@fh-soft.de, bug-make bug-make@gnu.org Since you asked basic questions I'm going to start this at a basic level.

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Eli Zaretskii
From: Paul Smith psm...@gnu.org Cc: David Boyce david.s.bo...@gmail.com, f.heckenb...@fh-soft.de, bug-make@gnu.org Date: Tue, 23 Apr 2013 15:04:39 -0400 When thinking about this, remember that it's not enough to consider how a single make invocation will work. If you run with a

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread David Boyce
On Tue, Apr 23, 2013 at 12:04 PM, Paul Smith psm...@gnu.org wrote: I'm not sure if the lock locks the FD (so that if you dup'd the FD but it still pointed to the same output, you could take exclusive locks on both), or if it locks the underlying resource. I'm pretty sure it's the underlying

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Frank Heckenbach
(TL;DR: Probably irrelevant.) Paul Smith wrote: I'm not sure if the lock locks the FD (so that if you dup'd the FD but it still pointed to the same output, you could take exclusive locks on both), or if it locks the underlying resource. If the former I guess it's possible to break the

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Frank Heckenbach
David Boyce wrote: The first thing is get the word lock out of your mind because we aren't really locking anything. Yes, that API is in use but it's only to create a semaphore or baton. Nobody is ever prevented from doing anything. It just happens that on Unix the most portable (i.e. oldest)

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Eli Zaretskii
Date: Tue, 23 Apr 2013 21:41:22 +0200 From: Frank Heckenbach f.heckenb...@fh-soft.de Yes, as I wrote in another mail, even a completely global semaphore should do. Not healthy, IMHO. Some snafu could inadvertently and completely silently stop an unrelated build somewhere on the same

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Paul Smith
On Tue, 2013-04-23 at 23:16 +0300, Eli Zaretskii wrote: All it requires is inheriting the redirected stdout/stderr to child processes. This was already possible under Dos (with the exception that since there was no fork, you had to redirect in the parent process, call the child, then

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Frank Heckenbach
Eli Zaretskii wrote: Date: Tue, 23 Apr 2013 21:41:22 +0200 From: Frank Heckenbach f.heckenb...@fh-soft.de Sure, it's excluding much more than necessary, but since the critical section is very short, this shouldn't hurt much. (In other words, if make jobs produce such huge output that

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-23 Thread Eli Zaretskii
From: Paul Smith p...@mad-scientist.net Cc: Frank Heckenbach f.heckenb...@fh-soft.de, bug-make@gnu.org Date: Tue, 23 Apr 2013 16:54:33 -0400 Without knowing what kind of resource Windows can take locks on, we can't really know how to help with that. The only resources that don't need their

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-21 Thread Paul Smith
On Fri, 2013-04-19 at 14:09 +0300, Eli Zaretskii wrote: Date: Fri, 19 Apr 2013 11:54:05 +0200 Cc: bo...@kolpackov.net, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de Is there a simple enough Makefile somewhere that could be used to test this feature, once

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-21 Thread Paul Smith
On Fri, 2013-04-19 at 12:36 +0300, Eli Zaretskii wrote: Also, where is the best place to put the emulated Posix functions? Some new file in w32/compat/? I'd like to see it there. I'm thinking I want to move the new stuff out of job.c even for POSIX systems. The ifdefs are really getting to

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-19 Thread Eli Zaretskii
Date: Fri, 19 Apr 2013 00:06:52 +0300 From: Eli Zaretskii e...@gnu.org Cc: bo...@kolpackov.net, bug-make@gnu.org Indeed, as you suggested earlier, it might be useful to use the main part of open_tmpfile() (i.e. without the fdopen()), though we'd have to manually remove the file

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-19 Thread Frank Heckenbach
Eli Zaretskii wrote: Initial investigation indicates that tmpfile should do the job just fine: the file is deleted only when the last descriptor for it is closed. That includes any duplicated descriptors. Great. As for fcntl, F_SETLKW, and F_GETFD, they will need to be emulated. In

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-19 Thread Eli Zaretskii
Date: Fri, 19 Apr 2013 11:54:05 +0200 Cc: bo...@kolpackov.net, bug-make@gnu.org From: Frank Heckenbach f.heckenb...@fh-soft.de Is there a simple enough Makefile somewhere that could be used to test this feature, once implemented? We have a test in the test suite (output-sync). Can you

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Frank Heckenbach
Paul Smith wrote: I've applied the patch from Frank. Thanks. I did some tests and so far everything works in my setup. Since I was away for a day, I couldn't follow the discussion earlier, so allow me to respond to several mails at once ... I changed the behavior so that the entire recipe

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Paul Smith
On Thu, 2013-04-18 at 19:09 +0200, Frank Heckenbach wrote: This mechanism was unaffected by my output-sync patch, and I expected your change broke it. I was reading your email with interest, waiting for the punch-line, but then after all that description you just said that the change broke it,

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread David Boyce
On Thu, Apr 18, 2013 at 10:09 AM, Frank Heckenbach f.heckenb...@fh-soft.dewrote: FILE is an opaque structure which should never be allocated by user code, so its layout can be implementation specific. Of course it's an opaque structure. The problem is that the implementation can't change its

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Frank Heckenbach
Paul Smith wrote: On Thu, 2013-04-18 at 19:09 +0200, Frank Heckenbach wrote: This mechanism was unaffected by my output-sync patch, and I expected your change broke it. I was reading your email with interest, waiting for the punch-line, but then after all that description you just said

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Frank Heckenbach
David Boyce wrote: On Thu, Apr 18, 2013 at 10:09 AM, Frank Heckenbach f.heckenb...@fh-soft.dewrote: FILE is an opaque structure which should never be allocated by user code, so its layout can be implementation specific. Of course it's an opaque structure. The problem is that the

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Paul Smith
On Thu, 2013-04-18 at 20:36 +0200, Frank Heckenbach wrote: And with my progress mechanism, that's exactly what I want. In my case it'd look like this: [Start] Compiling foo.c [Start] Compiling bar.c # time passes foo.c: some error # time passes bar.c: some error # time passes [End]

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Eli Zaretskii
Date: Thu, 18 Apr 2013 19:09:06 +0200 From: Frank Heckenbach f.heckenb...@fh-soft.de . calculation of combined_output in start_job_command will need to be reimplemented for Windows, since the reliance on st_dev and st_ino makes assumptions that are false on Windows. What we

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Eli Zaretskii
Date: Thu, 18 Apr 2013 20:39:44 +0200 Cc: psm...@gnu.org, e...@gnu.org, bo...@kolpackov.net From: Frank Heckenbach f.heckenb...@fh-soft.de Indeed, as you suggested earlier, it might be useful to use the main part of open_tmpfile() (i.e. without the fdopen()), though we'd have to manually

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread David Boyce
On Thu, Apr 18, 2013 at 2:39 PM, Frank Heckenbach f.heckenb...@fh-soft.dewrote This might be right (I was just objecting to your claim that it was necessarily so on any 32-bit Unix), and I'd prefer to use fd's anyway. Well ... Linux is not Unix, as partisans will proudly tell you, so

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Frank Heckenbach
Eli Zaretskii wrote: Date: Thu, 18 Apr 2013 20:39:44 +0200 Cc: psm...@gnu.org, e...@gnu.org, bo...@kolpackov.net From: Frank Heckenbach f.heckenb...@fh-soft.de Indeed, as you suggested earlier, it might be useful to use the main part of open_tmpfile() (i.e. without the fdopen()),

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Frank Heckenbach
Paul Smith wrote: On Thu, 2013-04-18 at 20:36 +0200, Frank Heckenbach wrote: And with my progress mechanism, that's exactly what I want. In my case it'd look like this: [Start] Compiling foo.c [Start] Compiling bar.c # time passes foo.c: some error # time passes bar.c: some

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-18 Thread Eli Zaretskii
Date: Thu, 18 Apr 2013 22:05:33 +0200 Cc: bug-make@gnu.org, david.s.bo...@gmail.com, psm...@gnu.org, bo...@kolpackov.net From: Frank Heckenbach f.heckenb...@fh-soft.de Eli Zaretskii wrote: Date: Thu, 18 Apr 2013 20:39:44 +0200 Cc: psm...@gnu.org, e...@gnu.org,

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-17 Thread Eli Zaretskii
From: Paul Smith psm...@gnu.org Cc: bo...@kolpackov.net, bug-make@gnu.org, f.heckenb...@fh-soft.de Date: Tue, 16 Apr 2013 12:56:21 -0400 On Tue, 2013-04-16 at 19:20 +0300, Eli Zaretskii wrote: From: Paul Smith psm...@gnu.org Cc: bo...@kolpackov.net, bug-make@gnu.org,

Re: [bug #33138] .PARLLELSYNC enhancement with patch

2013-04-17 Thread Paul Smith
On Wed, 2013-04-17 at 19:10 +0300, Eli Zaretskii wrote: That could be a misunderstanding on my part: I didn't realize that by handle you mean a FILE object. I thought you meant Windows specific HANDLE objects (which underly every open file). I'm not very familiar with Windows terminology. Is

  1   2   >