Re: [bug #22923] option to prevent "interspersed" output in parallel builds

[Howard Chu]

Brian Kustel wrote:


Follow-up Comment #3, bug #22923 (project make):

Thanks ipjrest and psmith for both replies. They are very appreciated.

Responding to ipjrest's comments:
=
I agree some commands are long-running and that if "watching" progress is
important, the proposed behavior may not be desireable. This is one reason I
suggested the proposed new behavior be optional.

1) Line-buffering: While it would help in some cases, not intermixing lines
from different processes is only partially helpful. The fact that a "single
line" from process A might be interrupted by a line from process B is my
concern here. For example, instead of seeing
   Process A Fragment 1 was emitted and then fragment 2 was emitted.
   Process B Message
one might see
   A: Process A Fragment 1 was emitted
   B: Process B Message
   A: and then fragment 2 was emitted.
if the line for process A is emitted in 2 separate write operations.
2) The VS line prefixing would be OK. In fact I like this idea. But it still
has the problem that single lines can be broken if the tool being run
sometimes emits lines "in pieces". I have noticed that gcc has this problem in
some cases (at least the gcc version I am currently using, 4.1.0).

Responding to psmith's comments:

Thanks for pointing out that make is not multithreaded. I guess I was
assuming it was. I agree this complicates matters somewhat. Also, I see your
point about my original stdout/stderr "marking" idea. In both cases I assumed
make would have either an auxilliary thread or its main thread doing these
tasks but I now realize that this is not how make currently works.

But I think something along the lines of your "special variable" idea could
be employed in a generic fashion. Perhaps this is what you intended your
second paragraph below to imply? Specifically, the proposed option could cause
all "commands" to have their output piped to a standard tool which stores
(its) stdin to a temp file and then writes it synchronously to stdout after
stdin is closed. Also, I think it could be reasonable to allow for an option
to either combine stdout/stderr here or to pipe them to separate processes (so
that stdout/stderr could be atomically emitted to stdout/stderr
respectively).

While I have nothing against the "special variable" idea, I'd very much like
to see a way to accomplish this via a make command-line option and the
variable idea sounded as if it might require a makefile modification.


Just my 2 cents, the Alliant parallel make (which inspired my design of the 
make -j jobserver) also redirected all of the spawned processes' stdio to a 
set of pipes, so that all of their output was prefixed with |jobnum|. It was 
quite nice, but I never felt it was important enough to duplicate that 
behavior for gmake.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #22923] option to prevent "interspersed" output in parallel builds

[Brian Kustel]

Follow-up Comment #3, bug #22923 (project make):

Thanks ipjrest and psmith for both replies. They are very appreciated. 

Responding to ipjrest's comments:
=
I agree some commands are long-running and that if "watching" progress is
important, the proposed behavior may not be desireable. This is one reason I
suggested the proposed new behavior be optional.

1) Line-buffering: While it would help in some cases, not intermixing lines
from different processes is only partially helpful. The fact that a "single
line" from process A might be interrupted by a line from process B is my
concern here. For example, instead of seeing
  Process A Fragment 1 was emitted and then fragment 2 was emitted.
  Process B Message
one might see 
  A: Process A Fragment 1 was emitted 
  B: Process B Message
  A: and then fragment 2 was emitted.
if the line for process A is emitted in 2 separate write operations. 
2) The VS line prefixing would be OK. In fact I like this idea. But it still
has the problem that single lines can be broken if the tool being run
sometimes emits lines "in pieces". I have noticed that gcc has this problem in
some cases (at least the gcc version I am currently using, 4.1.0).

Responding to psmith's comments:

Thanks for pointing out that make is not multithreaded. I guess I was
assuming it was. I agree this complicates matters somewhat. Also, I see your
point about my original stdout/stderr "marking" idea. In both cases I assumed
make would have either an auxilliary thread or its main thread doing these
tasks but I now realize that this is not how make currently works. 

But I think something along the lines of your "special variable" idea could
be employed in a generic fashion. Perhaps this is what you intended your
second paragraph below to imply? Specifically, the proposed option could cause
all "commands" to have their output piped to a standard tool which stores
(its) stdin to a temp file and then writes it synchronously to stdout after
stdin is closed. Also, I think it could be reasonable to allow for an option
to either combine stdout/stderr here or to pipe them to separate processes (so
that stdout/stderr could be atomically emitted to stdout/stderr
respectively).

While I have nothing against the "special variable" idea, I'd very much like
to see a way to accomplish this via a make command-line option and the
variable idea sounded as if it might require a makefile modification.


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #26001] Evaluating eval expressions does not work for % type names

[Olexiy Buyanskyy]

Follow-up Comment #1, bug #26001 (project make):

I think you have to use two dollars.
try next

file%: $$(call reverseLookup,file%)



___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make