Re: Possible bug with grep/sed/tail?

2008-11-30 Thread Pádraig Brady
Jim Meyering wrote: Brian Dessent [EMAIL PROTECTED] wrote: Jim Meyering wrote: local setvbuf='__attribute__ ((constructor)) void f () { setvbuf (stdout, NULL, _IOLBF, 0); }' echo $setvbuf | gcc -s -include stdio.h -x c - -fPIC -shared \ -o $line_buffer_so

Re: Possible bug with grep/sed/tail?

2008-11-29 Thread Jim Meyering
Brian Dessent [EMAIL PROTECTED] wrote: Jim Meyering wrote: local setvbuf='__attribute__ ((constructor)) void f () { setvbuf (stdout, NULL, _IOLBF, 0); }' echo $setvbuf | gcc -s -include stdio.h -x c - -fPIC -shared \ -o $line_buffer_so Note that I named the

Re: Possible bug with grep/sed/tail?

2008-11-25 Thread Pádraig Brady
Jim Meyering wrote: Pádraig Brady [EMAIL PROTECTED] wrote: Jim Meyering wrote: What about adding buffer control to all coretuils filters. Is that still a desired feature do you think? I don't relish the idea of adding an option or feature to each and every filter in coreutils. Especially

Re: Possible bug with grep/sed/tail?

2008-11-25 Thread Jim Meyering
Paolo Bonzini [EMAIL PROTECTED] wrote: So -ol (that's an el) would mean line-buffered stdout? That has to be equivalent to -o -l, and unless you consider ordering and multiple -l options (e.g., -i -l -o -l is ugly), then it doesn't let you line-buffer more than one of the three streams. It

Re: Possible bug with grep/sed/tail?

2008-11-25 Thread Paolo Bonzini
So -ol (that's an el) would mean line-buffered stdout? That has to be equivalent to -o -l, and unless you consider ordering and multiple -l options (e.g., -i -l -o -l is ugly), then it doesn't let you line-buffer more than one of the three streams. It would be +i::o::e:: in getopt parlance;

Re: Possible bug with grep/sed/tail?

2008-11-25 Thread Pádraig Brady
Jim Meyering wrote: Pádraig Brady [EMAIL PROTECTED] wrote: I'm now thinking of 3 options: stdbuf -i -o -e The usual use case is: stdbuf -ol But you could also do: stdbuf -i4096 -o8192 We would warn about redundant combos like: stdbuf -il So -ol (that's an el) would mean line-buffered

Re: Possible bug with grep/sed/tail?

2008-11-25 Thread Paolo Bonzini
Jim Meyering wrote: Paolo Bonzini [EMAIL PROTECTED] wrote: So -ol (that's an el) would mean line-buffered stdout? That has to be equivalent to -o -l, and unless you consider ordering and multiple -l options (e.g., -i -l -o -l is ugly), then it doesn't let you line-buffer more than one of the

Re: Possible bug with grep/sed/tail?

2008-11-25 Thread Jim Meyering
Pádraig Brady [EMAIL PROTECTED] wrote: ... tail -f access.log | stdbuf --fd=1 --size=1 cut -d' ' -f1 | uniq size=0 = unbuffered size=1 = line buffered size1 - specific buffer size Also we could have aliases for stdin stdout, linebuffered, ... We still have the requirement on LD_PRELOAD,

Re: Possible bug with grep/sed/tail?

2008-11-24 Thread Jim Meyering
Pádraig Brady [EMAIL PROTECTED] wrote: Jim Meyering wrote: What about adding buffer control to all coretuils filters. Is that still a desired feature do you think? I don't relish the idea of adding an option or feature to each and every filter in coreutils. Especially considering that this

Re: Possible bug with grep/sed/tail?

2008-11-24 Thread Pádraig Brady
Jim Meyering wrote: What about adding buffer control to all coretuils filters. Is that still a desired feature do you think? I don't relish the idea of adding an option or feature to each and every filter in coreutils. Especially considering that this approach solves the problem -- albeit

Re: Possible bug with grep/sed/tail?

2008-11-22 Thread Jim Meyering
Jim Meyering [EMAIL PROTECTED] wrote: Andreas Schwab [EMAIL PROTECTED] wrote: Paolo Bonzini [EMAIL PROTECTED] writes: alias line-buffer='preload /t/linebuf.so' If you end the alias expansion with a space then the next word is also expanded as an alias. Ah, yes. That was it. Thanks ;-)

Re: Possible bug with grep/sed/tail?

2008-11-22 Thread Pádraig Brady
Jim Meyering wrote: Jim Meyering [EMAIL PROTECTED] wrote: Andreas Schwab [EMAIL PROTECTED] wrote: Paolo Bonzini [EMAIL PROTECTED] writes: alias line-buffer='preload /t/linebuf.so' If you end the alias expansion with a space then the next word is also expanded as an alias. Ah, yes. That

Re: Possible bug with grep/sed/tail?

2008-11-22 Thread Brian Dessent
Jim Meyering wrote: local setvbuf='__attribute__ ((constructor)) void f () { setvbuf (stdout, NULL, _IOLBF, 0); }' echo $setvbuf | gcc -s -include stdio.h -x c - -fPIC -shared \ -o $line_buffer_so Note that I named the function 'f()' for brevity in the

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Jim Meyering
Brian Dessent [EMAIL PROTECTED] wrote: It has been said before in previous threads but I want to re-state that this is possible in a very simple and general way with the LD_PRELOAD facility: $ echo '__attribute__((constructor)) void f() { setvbuf (stdout, NULL, _IOLBF, 0); }' | \ gcc

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Reuben Thomas
On Fri, 21 Nov 2008, Jim Meyering wrote: But now, when I see it's so easy to roll your own, I wonder if it's worth adding a C program to do that for you. Opinions? The most important thing is that this apparently long-standing problem be fixed in the way that places the least burden on the

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Andreas Schwab
Paolo Bonzini [EMAIL PROTECTED] writes: alias line-buffer='preload /t/linebuf.so' If you end the alias expansion with a space then the next word is also expanded as an alias. Andreas. -- Andreas Schwab, SuSE Labs, [EMAIL PROTECTED] SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg,

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Jim Meyering
Andreas Schwab [EMAIL PROTECTED] wrote: Paolo Bonzini [EMAIL PROTECTED] writes: alias line-buffer='preload /t/linebuf.so' If you end the alias expansion with a space then the next word is also expanded as an alias. Ah, yes. That was it. Thanks ;-)

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Paolo Bonzini
A) # tail trace.txt | grep com - WORKS: produces output B) # tail trace.txt | grep com | cat - WORKS: produces output C) # tail -f trace.txt | grep com - WORKS: produces output, then waits and reports new lines D) # tail -f trace.txt | grep com | cat - FAILS: no output from existing lines,

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Paolo Bonzini
Is there currently a way to do that? This problem has certainly caught me out in the past (admittedly about ten years ago). sed has -u (unbuffered). I guess you can then emulate grep with sed -un /foo/p. Paolo

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Brian Dessent
Pádraig Brady wrote: It's probably useful to add this functionality to all coreutil filters. I still need to be convinced that glibc is not the correct place for this. If it was there then any users of stdio could be controlled. It has been said before in previous threads but I want to

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Paolo Bonzini
Karl Berry wrote: It's probably useful to add this functionality to all coreutil filters. Wouldn't it suffice to add buffer control options to cat, rather than every program? Unfortunately no -- it's the other program that needs to be taught about the buffering policy to adopt. tail -f

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Paolo Bonzini
But if the shared library is installed, you can instead use your own line-buffer function: line-buffer() { LD_PRELOAD=/t/linebuf.so $@ } but that doesn't expand the command name when it's an alias or function. It actually works for functions. To support aliases, you can do

Re: Possible bug with grep/sed/tail?

2008-11-21 Thread Paolo Bonzini
Andreas Schwab wrote: Paolo Bonzini [EMAIL PROTECTED] writes: alias line-buffer='preload /t/linebuf.so' If you end the alias expansion with a space then the next word is also expanded as an alias. Indeed, this does it alias line-buffer='LD_PRELOAD=/t/linebuf.so ' Cool, thanks!

Re: Possible bug with grep/sed/tail?

2008-11-20 Thread Matthew Wakeling
On Thu, 20 Nov 2008, David Corlette wrote: Hello there, Wow, that's quite a cross-post you have there. Thoughts? Is there some limitation of which I am not aware that prevents this from working? Where does the data go in case D? It's all to do with buffering. The data is being held in a

Re: Possible bug with grep/sed/tail?

2008-11-20 Thread Pádraig Brady
Paolo Bonzini wrote: A) # tail trace.txt | grep com - WORKS: produces output B) # tail trace.txt | grep com | cat - WORKS: produces output C) # tail -f trace.txt | grep com - WORKS: produces output, then waits and reports new lines D) # tail -f trace.txt | grep com | cat - FAILS: no output

Re: Possible bug with grep/sed/tail?

2008-11-20 Thread Karl Berry
It's probably useful to add this functionality to all coreutil filters. Wouldn't it suffice to add buffer control options to cat, rather than every program? I still need to be convinced that glibc is not the correct place for this. If it was there then any users of stdio could be