Re: tail does not exit
On Dec 20, 2007, at 1:58 AM, Mikhail Teterin wrote:
On середа 19 грудень 2007, Chuck Swiger wrote:
= A quick test suggests that "tail -f" will close when it gets a
SIGPIPE.
SIGPIPE? How is that relevant? Does tail get a SIGPIPE, when awk
disappears
in my example? If it does not, why do you bring it up?
tail should get a SIGPIPE when it tries to write to a pipeline where
the other end has closed.
And if it does get SIGPIPE, then you are wrong, because the posted
"quick test" shows the exact opposite behavior -- tail does NOT go
away.
Please, clarify... Thanks.
Worked for me. I opened two SSH sessions to a FreeBSD 5.5 system, and
did this in one:
% touch /tmp/logfile
% echo "line 1" >> /tmp/logfile
...and this in the other:
% tail -f /tmp/logfile | awk '{print "Line: " $1 ; exit(0)}END{print
"Bye"}'
...when I then did a:
% echo "line 2" >> /tmp/logfile
...in the first, the tail -f process terminated in the second.
--
-Chuck
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
четвер 20 грудень 2007 11:58 до, Erik Osterholm Ви написали: > Ah, I see. With very, very long lines, tail doesn't send the output > all at once. The cutoff seems to be 65536 bytes on my system. They don't even have to be very very long -- unless in an artificial example, such as the one I posted. Normal-width text files can also trigger inconsistent behavior in some real-life scenario, where awk actually does some real processing of its input for a while. The awk script may decide to quit after processing the first 1000 (normal-length) lines, for example... The behavior of the program will then be different depending on whether the average line-length is above, at, or below 65.536 characters. Maybe, it is awk's fault -- it should not be read-ing more than one line at a time, because the script may cause it to ignore some of the read data. Using line-buffering or some such? -mi ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
On Thu, Dec 20, 2007 at 11:02:59AM -0500, Mikhail Teterin wrote: > On ?? 20 ??? 2007, Erik Osterholm wrote: > = The same behavior happens if I use a larger file. I see no > = inconsistent behavior, nor any bugs. > > The inconsistency is in the fact, that the behavior depends on the size of > the > buffer and length of the lines (not the size of the file). > > If the 10 lines, which tail tries to output initially, exceed the size of the > buffer, tail learns about awk going away immediately. If the lines are not > long enough, it does not. > > Also, I would expect a program to be notified (by SIGPIPE?) /immediately/, > when any of its output pipes are closed -- instead of waiting for it to try > to write into the pipe. But this issue is not, it seems, FreeBSD-specific... > > -mi Ah, I see. With very, very long lines, tail doesn't send the output all at once. The cutoff seems to be 65536 bytes on my system. If tail has to write more than this amount, it breaks it up into mutliple writes of a maximum of 65536 characters each. The problem is that after the first 65536 characters, awk has exited, causing the next 65536 characters which tail attempts to write to cause a SIGPIPE. It seems to be working as intended, though. When piping, you have to be aware of these issues, but I do not think that it is a bug. There must be some boundary where tail splits the output into multiple writes. If, after the first write, a \n hasn't been encountered yet, awk will consider at least some portion of the next write (up until the first \n) to be the same line, at least until it hits its own limit. I have not checked to see what this limit might be. As for SIGPIPE, that's just how the POSIX standard works. The signal is sent to the writing process when it attempts to write to a broken or closed pipe, not when the pipe has closed. If you think that this behavior is bad, you might want to contact IEEE. Erik ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
On Thu, Dec 20, 2007 at 05:40:11AM -0500, Mikhail Teterin wrote:
> On ?? 20 ??? 2007, Max N. Boyarov wrote: = | MT> Is not that
> a bug in itself? = = | Tail write buffer at all, i.e. all 10 lines
> writes to pipe.
>
> So, the behavior depends on the size of the buffer -- and thus the
> size of the input lines.
>
> A bug indeed...
I don't understand.
aleph:~$ cat test
blah1
blah2
blah3
aleph:~$ tail -f test | awk '{print $1; exit 0}'
blah1
(hangs)
This is expected. Awk printed one time and exited, per the given
script. The output from tail/input from awk went all at once, awk
printed the first line, exited, and the rest of the input disappeared.
'tail' sent "blah1\nblah2\nblah3\n" to awk, awk printed until the
first newline and exited.
If I now write to test from another terminal:
aleph:~$ echo "blah4" >> test
Tail tries to write to the pipe, which it finds closed. It receives a
SIGPIPE (tried to write to a pope with no reader--see man signal), and
it terminates.
The same behavior happens if I use a larger file. I see no
inconsistent behavior, nor any bugs.
Erik
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
On четвер 20 грудень 2007, Erik Osterholm wrote: = The same behavior happens if I use a larger file. I see no = inconsistent behavior, nor any bugs. The inconsistency is in the fact, that the behavior depends on the size of the buffer and length of the lines (not the size of the file). If the 10 lines, which tail tries to output initially, exceed the size of the buffer, tail learns about awk going away immediately. If the lines are not long enough, it does not. Also, I would expect a program to be notified (by SIGPIPE?) /immediately/, when any of its output pipes are closed -- instead of waiting for it to try to write into the pipe. But this issue is not, it seems, FreeBSD-specific... -mi ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
On четвер 20 грудень 2007, Max N. Boyarov wrote: = MT> Is not that a bug in itself? = = Tail write buffer at all, i.e. all 10 lines writes to pipe. So, the behavior depends on the size of the buffer -- and thus the size of the input lines. A bug indeed... -mi ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
> "MT" == Mikhail Teterin writes:
MT> On четвер 20 грудень 2007, Max N. Boyarov wrote:
MT> = after something writeln to /var/log/messages tail get SIGPIPE
MT> But why is that needed for tail to notice? It is trying to output 10 lines.
MT> After it outputs the very first one of them, awk exits, and the 9
subsequent
MT> lines go into thin air /without tail noticing/.
MT> Is not that a bug in itself?
Tail write buffer at all, i.e. all 10 lines writes to pipe.
$ cat test
line1
line2
line3
line4
line5
line6
line7
line8
line9
line10
line11
tail -f test | awk '{print "Exiting" $1; exit 0}'
open("test", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=68, ...}) = 0
syscall_477(0, 0x44, 0x1, 0x1, 0x3, 0, 0) = 0x28175000
write(1, "line2\nline3\nline4\nline5\nline6\nli"..., 62Exitingline2
) = 62
fstat(3, {st_mode=S_ISUID|S_ISVTX|070, st_size=0, ...}) = 0
syscall_478(0x3, 0, 0, 0x1) = 0
syscall_478(0x3, 0, 0, 0) = 0
read(3, "line1\nline2\nline3\nline4\nline5\nli"..., 4096) = 68
^^ write buff
munmap(0x28175000, 68) = 0
read(3, "", 4096) = 0
kqueue(0x28172d40) = 4
syscall_397(0x3, 0xbfbfe44c)= 0
kevent(0x4, 0x28205040, 0x1, 0, 0, 0xbfbfe624) = 0
kevent(0x4, 0, 0, 0x28205040, 0x1, 0) = 1
read(3, "line++\n", 4096) = 7
^^ new line added ^
fstat(1, {st_mode=031545, st_size=7596457873570623081, ...}) = 0
read(3, "", 4096) = 0
write(1, "line++\n", 7) = -1 EPIPE (Broken pipe)
^ try write ^^
--- SIGPIPE (Broken pipe: 13) ---
--- SIGPIPE (Broken pipe: 13) ---
exit ^
--
Max N. Boyarov
pgpQlaBQK0jet.pgp
Description: PGP signature
Re: tail does not exit
On четвер 20 грудень 2007, Max N. Boyarov wrote: = after something writeln to /var/log/messages tail get SIGPIPE But why is that needed for tail to notice? It is trying to output 10 lines. After it outputs the very first one of them, awk exits, and the 9 subsequent lines go into thin air /without tail noticing/. Is not that a bug in itself? -mi ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
> "MT" == Mikhail Teterin writes:
[...]
MT> I'm sorry, this does not make sense to me. Starting with an empty
MT> file, as you do in 1), /may/ make tail not notice, that awk went
MT> away, because tail has nothing to write to stdout.
MT> But /var/log/messages is not empty, and awk -- in my example -- would
MT> exit upon seeing the very first line of its input (tail's output).
MT> Yet tail fails to notice, that its subsequent output (starting with the
MT> second line) is written to nowhere...
MT> Why?
Because nothing writeln to /var/log/messages
$ sh -x /tmp/x.sh
+ + tail -f /var/log/messages
awk {print "Exiting"; exit 0}
Exiting
+ echo Exited
Exited
+ exit 0
after something writeln to /var/log/messages tail get SIGPIPE
open("/var/log/messages",O_RDONLY,0666) = 3 (0x3)
fstat(3,{mode=-rw-r--r-- ,inode=141327,size=4997,blksize=4096}) = 0 (0x0)
mmap(0x0,4997,PROT_READ,MAP_SHARED,3,0x0)= 672616448 (0x28175000)
Exiting
write(1,"Dec 20 11:59:04 solar kernel: da"...,1090) = 1090 (0x442)
fstat(3,{mode=-rw-r--r-- ,inode=141327,size=4997,blksize=4096}) = 0 (0x0)
lseek(3,0x0,SEEK_CUR)= 0 (0x0)
lseek(3,0x1000,SEEK_SET) = 4096 (0x1000)
read(3,"kernel: da1: 1.000MB/s transfers"...,4096) = 901 (0x385)
munmap(0x28175000,4997) = 0 (0x0)
read(3,0x28204000,4096) = 0 (0x0)
kqueue(0x28172d40,0x4,0xa,0x0,0x28201088,0x1)= 4 (0x4)
fstatfs(0x3,0xbfbfe44c,0x2815fe98,0x28172d40,0x0,0x2806dee4) = 0 (0x0)
kevent(4,{0x3,EVFILT_READ,EV_ADD|EV_ENABLE|EV_CLEAR,0,0x0,0x0},1,0x0,0,{0.0})
= 0 (0x0)
kevent(4,0x0,0,{0x3,EVFILT_READ,EV_CLEAR,0,0x3e,0x0},1,0x0) = 1 (0x1)
read(3,"Dec 20 12:06:27 solar su: BAD SU"...,4096) = 62 (0x3e)
fstat(1,{mode=p- ,inode=0,size=0,blksize=4096}) = 0 (0x0)
read(3,0x28204000,4096) = 0 (0x0)
write(1,"Dec 20 12:06:27 solar su: BAD SU"...,62) ERR#32 'Broken pipe'
SIGNAL 13 (SIGPIPE)
--
Max N. Boyarov
pgp6Vf07VlDFE.pgp
Description: PGP signature
Re: tail does not exit
On середа 19 грудень 2007, Chuck Swiger wrote:
= A quick test suggests that "tail -f" will close when it gets a SIGPIPE.
SIGPIPE? How is that relevant? Does tail get a SIGPIPE, when awk disappears
in my example? If it does not, why do you bring it up?
And if it does get SIGPIPE, then you are wrong, because the posted
"quick test" shows the exact opposite behavior -- tail does NOT go
away.
Please, clarify... Thanks.
On середа 19 грудень 2007, Max N. Boyarov wrote:
= try to test your script with anoter file and add somthing to it
=
= 1) cons1$ touch /tmp/test
= 2) cons1$ tail -f /tmp/test | awk '{print "Line: " $1 ; exit(0)}END{print
"Bye"}'
= 2a) Line: Line1
= 2b) Bye
I'm sorry, this does not make sense to me. Starting with an empty
file, as you do in 1), /may/ make tail not notice, that awk went
away, because tail has nothing to write to stdout.
But /var/log/messages is not empty, and awk -- in my example -- would
exit upon seeing the very first line of its input (tail's output).
Yet tail fails to notice, that its subsequent output (starting with the
second line) is written to nowhere...
Why?
-mi
P.S. Here is the example again:
#!/bin/sh
if tail -f /var/log/messages | awk '{print "Exiting"; exit 0}'
then
echo Exited
else
echo Failed
fi
exit 0
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
On Dec 19, 2007, at 4:06 PM, Mikhail Teterin wrote: Josh Tolbert: Cause the -f option to tail doesn't work that way. -f always waits for more input. I know very well about -f waiting for more *input*. What puzzles me, is that tail does not quit, when its *output* is closed. James Harrison: Is there a reason you want the -f flag? Yes, I want awk to be processing the lines, which are appended to the file, until it finds, what it is looking for, and exits. I expect tail to go away, when its stdout is closed, but it does not. tail -f holds on for dear life until a ctrl-c happens. IT HAS A DEATH GRIP! This seems like a bug to me... It should hold on to its input file(s), but exit peacefully, when its stdout closes. No? A quick test suggests that "tail -f" will close when it gets a SIGPIPE. -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
> "MT" == Mikhail Teterin writes:
MT> Max N. Boyarov:
>> -f The -f option causes tail to not stop when end of file is
>> reached, but rather to wait for additional data to be appended
>> to the input. The -f option is ignored if the standard input is a pipe,
>> but not if it is a FIFO.
MT> Josh Tolbert:
>> Cause the -f option to tail doesn't work that way. -f always waits for more
>> input.
MT> I know very well about -f waiting for more *input*. What puzzles me, is
that
MT> tail does not quit, when its *output* is closed.
MT> James Harrison:
>> Is there a reason you want the -f flag?
MT> Yes, I want awk to be processing the lines, which are appended to the
file,
MT> until it finds, what it is looking for, and exits. I expect tail to go
away,
MT> when its stdout is closed, but it does not.
>> tail -f holds on for dear life until a ctrl-c happens. IT HAS A DEATH
>> GRIP!
MT> This seems like a bug to me... It should hold on to its input file(s), but
MT> exit peacefully, when its stdout closes. No?
tail -f /tmp/foo | awk '{print "$1"}
tail write to pipe (reader wait data) (2)
reader get data (2a) and exit (2b). action (3) from another console
tail write data to pipe again. action (4) from another console
tail get SIGPIPE
tail exit
try to test your script with anoter file and add somthing to it
1) cons1$ touch /tmp/test
2) cons1$ tail -f /tmp/test | awk '{print "Line: " $1 ; exit(0)}END{print
"Bye"}'
2a) Line: Line1
2b) Bye
3) cons2$ echo Line1 >> /tmp/test
4) cons2$ echo Line2 >> /tmp/test
--
Max N. Boyarov
pgpgAFND1AZND.pgp
Description: PGP signature
Re: tail does not exit
Max N. Boyarov: > -f The -f option causes tail to not stop when end of file is > reached, but rather to wait for additional data to be appended > to the input. The -f option is ignored if the standard input is a pipe, > but not if it is a FIFO. Josh Tolbert: > Cause the -f option to tail doesn't work that way. -f always waits for more > input. I know very well about -f waiting for more *input*. What puzzles me, is that tail does not quit, when its *output* is closed. James Harrison: > Is there a reason you want the -f flag? Yes, I want awk to be processing the lines, which are appended to the file, until it finds, what it is looking for, and exits. I expect tail to go away, when its stdout is closed, but it does not. > tail -f holds on for dear life until a ctrl-c happens. IT HAS A DEATH > GRIP! This seems like a bug to me... It should hold on to its input file(s), but exit peacefully, when its stdout closes. No? -mi ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
> "MT" == Mikhail T. writes:
MT> Why does not the script below actually ever exit?
MT> #!/bin/sh
MT> if tail -f /var/log/messages | awk '{print "Exiting"; exit 0}'
MT> then
MT> echo Exited
MT> else
MT> echo Failed
MT> fi
MT> exit 0
MT> Awk exits as advertised, but tail stays around -- even though its
MT> stdout is closed... Why?
---start quote from man 1 tail---
-f The -f option causes tail to not stop when end of file is
reached, but rather to wait for additional data to be appended to
the input. The -f option is ignored if the standard input is a
pipe, but not if it is a FIFO.
---end quote---
try without "-f" option
--
Max N. Boyarov
pgp85bt53VpkK.pgp
Description: PGP signature
Re: tail does not exit
James Harrison writes: > tail -f holds on for dear life until a ctrl-c happens. IT HAS A > DEATH GRIP! Agreed. Robert Huff ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: tail does not exit
On Wed, 2007-12-19 at 18:22 -0500, Mikhail T. wrote:
> #!/bin/sh
>
> if tail -f /var/log/messages | awk '{print "Exiting"; exit 0}'
> then
> echo Exited
> else
> echo Failed
> fi
>
> exit 0
I assume it has something to do with tail -f being used, rather than
tail. Is there a reason you want the -f flag?
tail -f holds on for dear life until a ctrl-c happens. IT HAS A DEATH
GRIP!
James
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
tail does not exit
Why does not the script below actually ever exit?
#!/bin/sh
if tail -f /var/log/messages | awk '{print "Exiting"; exit 0}'
then
echo Exited
else
echo Failed
fi
exit 0
Awk exits as advertised, but tail stays around -- even though its
stdout is closed... Why?
Thanks!
-mi
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
