bug#33946: tail -f stops abruptly in AIX when piped.
"Pádraig Brady" wrote on 01/20/2019 02:16:38 PM: > From: "Pádraig Brady" > To: Ayappan P2 > Cc: 33...@debbugs.gnu.org, Bernhard Voelker voelker.de>, Bug-coreutils > Date: 01/20/2019 02:16 PM > Subject: Re: bug#33946: tail -f stops abruptly in AIX when piped. > > On 15/01/19 19:11, Pádraig Brady wrote: > > On 15/01/19 04:42, Ayappan P2 wrote: > >> > >> The patch/commit is not proper. The select() call will still be invoked in > >> AIX. > > > > Drats I pushed the debugging version of my patch :/ > > I'll fix up (preferably with Bernhard's test issue also addressed) > > Pushed the fix up in your name: > https://urldefense.proofpoint.com/v2/url? > u=https-3A__git.sv.gnu.org_gitweb_-3Fp-3Dcoreutils.git-3Ba-3Dcommitdiff-3Bh-3D17983b2=DwID- > g=jf_iaSHvJObTbx-siA1ZOg=SRx7SyASbvCxu7GP- > Qbph4o5MPmrwcLUo4BhenbwbOs=mPZV60JE_yQsMdKRyI8gKwP3fVfRN1ITodgfpwR6zMQ=M7DG1CswoQMrPvdaWpHfDQ2jJCvnVySL8cKhuM657do= > > Bernhard's test issue will be addressed separately. > > cheers, > Pádraig > > Great. Thank You. Regards Ayappan P
bug#33946: tail -f stops abruptly in AIX when piped.
On 1/20/19 10:00 AM, Pádraig Brady wrote: > Right. So the broken pipe is detected fine which is the main thing. > It's just that the osc system has SIGPIPE ignored > (python2 based systems do this by default, which may be related). Bingo. I confirmed that 'osc' is getting into the SIGPIPE handling by directly chroot'ing into the build directory: the (previous version of the) test passed. > I was looking are setting normal handling with `trap - SIGPIPE` in the test, > but that's only effective if ignored in the same shell. > If the parent/login shell has ignored SIGPIPE, > then resetting it is ineffective. > However... > > tail should be exiting irrespective of the handling of SIGPIPE. > In fact it goes into an infinite loop in the edge case of: > inotify + ignored sigpipe + early exit filters. > > The attached ensures the tail process exits, > which handles the infinite loop and the test failure. LGTM: the test passes under 'osc' now, too. Thanks & have a nice day, Berny
bug#33946: tail -f stops abruptly in AIX when piped.
On 17/01/19 06:25, Bernhard Voelker wrote:
> On 1/16/19 4:09 AM, Pádraig Brady wrote:
>> On 14/01/19 23:54, Bernhard Voelker wrote:
>>> On 1/13/19 4:31 AM, Pádraig Brady wrote:
Thanks for testing. Pushed at:
https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=d5ab4cb
>>>
-timeout 10 tail -f $mode $fastpoll out | sleep .1 || fail=1
+(returns_ 124 timeout 10 tail -n2 -f $mode $fastpoll out && touch
timed_out) |
+ sed 2q > out2
+test -e timed_out && fail=1
+compare exp out2 || fail=1
>>>
>>> I see the 'timed_out' file when running the test on openSUSE's build service
>>> for Linux x86_64, and can reproduce when running that in the local 'osc'
>>> build
>>> environment (chroot-based).
>>>
>>> I'm not sure what's the problem though, but could this be related to
>>> how we fixed 'tests/misc/seq-epipe.sh' a while ago in v8.25-42-g383e4b2ce?
>>
>> I can't see the problem offhand.
>
> I also still don't see the problem. In the log, it's just:
>
> + returns_ 124 timeout 10 tail -n2 -f ---disable-inotify -s.1
> --max-unchanged-stats=1 out
> + sed 2q
> + touch timed_out
> + test -e timed_out
> + fail=1
>
> Well, under strace:
>
> In the good case, i.e., without chroot, the process terminates upon the first
> SIGPIPE received:
>
> ...
> inotify_init() = 4
> write(1, "==> standard input <==\nar\n", 26) = 26
> inotify_add_watch(4, "out", IN_MODIFY) = 1
> stat("out", {st_dev=makedev(0x8, 0x20), st_ino=298091,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=717, st_gid=1000, ...}) = 0
> fstat(3, {st_dev=makedev(0x8, 0x20), st_ino=298091, st_mode=S_IFREG|0644,
> st_nlink=1, st_uid=717, st_gid=1000, ...}) = 0
> select(5, [1 4], NULL, NULL, NULL) = 1 (in [1])
> rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
> getpid()= 29422
> gettid()= 29422
> tgkill(29422, 29422, SIGPIPE) = 0
> rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29422, si_uid=717}
> ---
> +++ killed by SIGPIPE +++
>
> In the bad case, i.e., in the chroot'ed "osc build" environment or on
> 'build.opensuse.org',
> I see:
>
> ...
> inotify_init() = 4
> write(1, "==> standard input <==\nar\n", 26) = 26
> inotify_add_watch(4, "out", IN_MODIFY) = 1
> stat("out", {st_dev=makedev(0x8, 0x1), st_ino=192286, st_mode=S_IFREG|0644,
> st_nlink=1, st_uid=399, st_gid=399, ...}) = 0
> fstat(3, {st_dev=makedev(0x8, 0x1), st_ino=192286, st_mode=S_IFREG|0644,
> st_nlink=1, st_uid=399, st_gid=399, ...}) = 0
> select(5, [1 4], NULL, NULL, NULL) = 1 (in [1])
> rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
> getpid()= 29191
> gettid()= 29191
> tgkill(29191, 29191, SIGPIPE) = 0
> rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29191, si_uid=399}
> ---
> select(5, [1 4], NULL, NULL, NULL) = 1 (in [1])
> rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
> getpid()= 29191
> gettid()= 29191
> tgkill(29191, 29191, SIGPIPE) = 0
> rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29191, si_uid=399}
> ---
> select(5, [1 4], NULL, NULL, NULL) = 1 (in [1])
> rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
> getpid()= 29191
> gettid()= 29191
> tgkill(29191, 29191, SIGPIPE) = 0
> rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29191, si_uid=399}
> ---
> [... a.s.o ...]
>
> and finally gets killed by 'timeout 10':
>
> ...
> --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29191, si_uid=399}
> ---
> --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=29187, si_uid=399}
> ---
> +++ killed by SIGTERM +++
>
> Any idea?
Right. So the broken pipe is detected fine which is the main thing.
It's just that the osc system has SIGPIPE ignored
(python2 based systems do this by default, which may be related).
I was looking are setting normal handling with `trap - SIGPIPE` in the test,
but that's only effective if ignored in the same shell.
If the parent/login shell has ignored SIGPIPE,
then resetting it is ineffective.
However...
tail should be exiting irrespective of the handling of SIGPIPE.
In fact it goes into an infinite loop in the edge case of:
inotify + ignored sigpipe + early exit filters.
The attached ensures the tail process exits,
which handles the infinite loop and the test failure.
cheers,
Pádraig.
From cf36c2983a150d5e9a9bf631832a91d4a31259d0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?=
Date: Sun, 20 Jan 2019
bug#33946: tail -f stops abruptly in AIX when piped.
On 15/01/19 19:11, Pádraig Brady wrote: > On 15/01/19 04:42, Ayappan P2 wrote: >> >> The patch/commit is not proper. The select() call will still be invoked in >> AIX. > > Drats I pushed the debugging version of my patch :/ > I'll fix up (preferably with Bernhard's test issue also addressed) Pushed the fix up in your name: https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=17983b2 Bernhard's test issue will be addressed separately. cheers, Pádraig
bug#33946: tail -f stops abruptly in AIX when piped.
On 1/16/19 4:09 AM, Pádraig Brady wrote:
> On 14/01/19 23:54, Bernhard Voelker wrote:
>> On 1/13/19 4:31 AM, Pádraig Brady wrote:
>>> Thanks for testing. Pushed at:
>>> https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=d5ab4cb
>>
>>> -timeout 10 tail -f $mode $fastpoll out | sleep .1 || fail=1
>>> +(returns_ 124 timeout 10 tail -n2 -f $mode $fastpoll out && touch
>>> timed_out) |
>>> + sed 2q > out2
>>> +test -e timed_out && fail=1
>>> +compare exp out2 || fail=1
>>
>> I see the 'timed_out' file when running the test on openSUSE's build service
>> for Linux x86_64, and can reproduce when running that in the local 'osc'
>> build
>> environment (chroot-based).
>>
>> I'm not sure what's the problem though, but could this be related to
>> how we fixed 'tests/misc/seq-epipe.sh' a while ago in v8.25-42-g383e4b2ce?
>
> I can't see the problem offhand.
I also still don't see the problem. In the log, it's just:
+ returns_ 124 timeout 10 tail -n2 -f ---disable-inotify -s.1
--max-unchanged-stats=1 out
+ sed 2q
+ touch timed_out
+ test -e timed_out
+ fail=1
Well, under strace:
In the good case, i.e., without chroot, the process terminates upon the first
SIGPIPE received:
...
inotify_init() = 4
write(1, "==> standard input <==\nar\n", 26) = 26
inotify_add_watch(4, "out", IN_MODIFY) = 1
stat("out", {st_dev=makedev(0x8, 0x20), st_ino=298091, st_mode=S_IFREG|0644,
st_nlink=1, st_uid=717, st_gid=1000, ...}) = 0
fstat(3, {st_dev=makedev(0x8, 0x20), st_ino=298091, st_mode=S_IFREG|0644,
st_nlink=1, st_uid=717, st_gid=1000, ...}) = 0
select(5, [1 4], NULL, NULL, NULL) = 1 (in [1])
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
getpid()= 29422
gettid()= 29422
tgkill(29422, 29422, SIGPIPE) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29422, si_uid=717} ---
+++ killed by SIGPIPE +++
In the bad case, i.e., in the chroot'ed "osc build" environment or on
'build.opensuse.org',
I see:
...
inotify_init() = 4
write(1, "==> standard input <==\nar\n", 26) = 26
inotify_add_watch(4, "out", IN_MODIFY) = 1
stat("out", {st_dev=makedev(0x8, 0x1), st_ino=192286, st_mode=S_IFREG|0644,
st_nlink=1, st_uid=399, st_gid=399, ...}) = 0
fstat(3, {st_dev=makedev(0x8, 0x1), st_ino=192286, st_mode=S_IFREG|0644,
st_nlink=1, st_uid=399, st_gid=399, ...}) = 0
select(5, [1 4], NULL, NULL, NULL) = 1 (in [1])
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
getpid()= 29191
gettid()= 29191
tgkill(29191, 29191, SIGPIPE) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29191, si_uid=399} ---
select(5, [1 4], NULL, NULL, NULL) = 1 (in [1])
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
getpid()= 29191
gettid()= 29191
tgkill(29191, 29191, SIGPIPE) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29191, si_uid=399} ---
select(5, [1 4], NULL, NULL, NULL) = 1 (in [1])
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
getpid()= 29191
gettid()= 29191
tgkill(29191, 29191, SIGPIPE) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29191, si_uid=399} ---
[... a.s.o ...]
and finally gets killed by 'timeout 10':
...
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_TKILL, si_pid=29191, si_uid=399} ---
--- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=29187, si_uid=399} ---
+++ killed by SIGTERM +++
Any idea?
Have a nice day,
Berny
bug#33946: tail -f stops abruptly in AIX when piped.
On 14/01/19 23:54, Bernhard Voelker wrote: > On 1/13/19 4:31 AM, Pádraig Brady wrote: >> Thanks for testing. Pushed at: >> https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=d5ab4cb > >> -timeout 10 tail -f $mode $fastpoll out | sleep .1 || fail=1 >> +(returns_ 124 timeout 10 tail -n2 -f $mode $fastpoll out && touch >> timed_out) | >> + sed 2q > out2 >> +test -e timed_out && fail=1 >> +compare exp out2 || fail=1 > > I see the 'timed_out' file when running the test on openSUSE's build service > for Linux x86_64, and can reproduce when running that in the local 'osc' build > environment (chroot-based). > > I'm not sure what's the problem though, but could this be related to > how we fixed 'tests/misc/seq-epipe.sh' a while ago in v8.25-42-g383e4b2ce? I can't see the problem offhand. > BTW: in the 2nd iteration, the test doesn't delete 'timed_out', > so will always set fail=1 if the 1st one failed. Yes we should `rm -f timed_out` beforehand to be cleaner. cheers, Pádraig
bug#33946: tail -f stops abruptly in AIX when piped.
On 15/01/19 04:42, Ayappan P2 wrote: > > The patch/commit is not proper. The select() call will still be invoked in > AIX. Drats I pushed the debugging version of my patch :/ I'll fix up (preferably with Bernhard's test issue also addressed)
bug#33946: tail -f stops abruptly in AIX when piped.
The patch/commit is not proper. The select() call will still be invoked in AIX. It should be like this. # diff -u src/tail.c_orig src/tail.c --- src/tail.c_orig 2019-01-01 19:39:32 + +++ src/tail.c 2019-01-15 17:58:23 + @@ -30,6 +30,9 @@ #include #include #include +#ifdef _AIX +# include +#endif #include "system.h" #include "argmatch.h" @@ -338,6 +341,15 @@ if (! monitor_output) return; +#ifdef _AIX + /* select on AIX was seen to give a readable event immediately. */ + struct pollfd pfd; + pfd.fd = STDOUT_FILENO; + pfd.events = POLLERR; + + if (poll (, 1, 0) >= 0 && (pfd.revents & POLLERR)) +raise (SIGPIPE); +#else struct timeval delay; delay.tv_sec = delay.tv_usec = 0; @@ -349,6 +361,7 @@ and implies an error condition on output like broken pipe. */ if (select (STDOUT_FILENO + 1, , NULL, NULL, ) == 1) raise (SIGPIPE); +#endif } Thanks Ayappan P From: Bernhard Voelker To: 33...@debbugs.gnu.org, p...@draigbrady.com, ayapp...@in.ibm.com Date: 01/15/2019 01:25 PM Subject: bug#33946: tail -f stops abruptly in AIX when piped. Sent by:"Bug-coreutils" On 1/13/19 4:31 AM, Pádraig Brady wrote: > Thanks for testing. Pushed at: > https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=d5ab4cb > -timeout 10 tail -f $mode $fastpoll out | sleep .1 || fail=1 > +(returns_ 124 timeout 10 tail -n2 -f $mode $fastpoll out && touch timed_out) | > + sed 2q > out2 > +test -e timed_out && fail=1 > +compare exp out2 || fail=1 I see the 'timed_out' file when running the test on openSUSE's build service for Linux x86_64, and can reproduce when running that in the local 'osc' build environment (chroot-based). I'm not sure what's the problem though, but could this be related to how we fixed 'tests/misc/seq-epipe.sh' a while ago in v8.25-42-g383e4b2ce? BTW: in the 2nd iteration, the test doesn't delete 'timed_out', so will always set fail=1 if the 1st one failed. Have a nice day, Berny
bug#33946: tail -f stops abruptly in AIX when piped.
On 1/13/19 4:31 AM, Pádraig Brady wrote: > Thanks for testing. Pushed at: > https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=d5ab4cb > -timeout 10 tail -f $mode $fastpoll out | sleep .1 || fail=1 > +(returns_ 124 timeout 10 tail -n2 -f $mode $fastpoll out && touch timed_out) > | > + sed 2q > out2 > +test -e timed_out && fail=1 > +compare exp out2 || fail=1 I see the 'timed_out' file when running the test on openSUSE's build service for Linux x86_64, and can reproduce when running that in the local 'osc' build environment (chroot-based). I'm not sure what's the problem though, but could this be related to how we fixed 'tests/misc/seq-epipe.sh' a while ago in v8.25-42-g383e4b2ce? BTW: in the 2nd iteration, the test doesn't delete 'timed_out', so will always set fail=1 if the 1st one failed. Have a nice day, Berny
bug#33946: tail -f stops abruptly in AIX when piped.
On 07/01/19 01:04, Ayappan P2 wrote: > The poll() solution is working in AIX. Great. Thanks for testing. Pushed at: https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=d5ab4cb cheers, Pádraig
bug#33946: tail -f stops abruptly in AIX when piped.
Any updates on this guys ? Thanks Ayappan P > On 08-Jan-2019, at 12:03 PM, Ayappan P2 wrote: > > > Now the poll() solution is working in AIX, will there be any official patch > for this ? > > It will be great to have this integrated into the next release. > > Thanks > Ayappan P > > > > > From:"Ayappan P2" > To:"Pádraig Brady" > Cc:33...@debbugs.gnu.org, Bernhard Voelker >, Bug-coreutils > > Date: 01/07/2019 02:37 PM > Subject:bug#33946: tail -f stops abruptly in AIX when piped. > Sent by:"Bug-coreutils" +ayappap2=in.ibm@gnu.org> > > > > > The poll() solution is working in AIX. Great. > > Thanks > Ayappan P > > > > From: "Pádraig Brady" > To: Ayappan P2 , Bernhard Voelker > > Cc: 33...@debbugs.gnu.org > Date: 01/04/2019 11:37 PM > Subject: bug#33946: tail -f stops abruptly in AIX when piped. > Sent by: "Bug-coreutils" +ayappap2=in.ibm@gnu.org> > > > >> On 03/01/19 23:01, Ayappan P2 wrote: >> >> The problem happens only when we pipe the output of "tail -f" . >> >> I am not sure how one can take the truss of "/tail -f test_file | grep >> 123" . >> >> I did little debugging on the tail code. This function > "check_output_alive" >> introduced by the commit (mentioned earlier in the thread) sents SIGPIPE >> after doing a select () call in AIX. >> And that makes it exit immediately. >> >> fd_set rfd; >> FD_ZERO (); >> FD_SET (STDOUT_FILENO, ); >> >>/* readable event on STDOUT is equivalent to POLLERR, >> and implies an error condition on output like broken pipe. */ >> if (select (STDOUT_FILENO + 1, , NULL, NULL, ) == 1) >> raise (SIGPIPE); >> } >> >> I didn't understand the real reason behind this commit. >> >> Thanks >> Ayappan P >> >> >> >> From: Bernhard Voelker >> To: Ayappan P2 , > 33...@debbugs.gnu.org >> Date: 01/03/2019 11:53 PM >> Subject: bug#33946: tail -f stops abruptly in AIX > when piped. >> Sent by: "Bug-coreutils" >+ayappap2=in.ibm@gnu.org> >> >> >> >> On 1/3/19 6:39 PM, Ayappan P2 wrote: >>>> On 01-Jan-2019, at 10:36 PM, Ayappan P2 wrote: >>>> Hi, >>>> >>>> I am running coreutils 8.30 in AIX machine and it seems like "tail -f" >> is >>>> not working as it used to be when the output is piped. >>>> >>>> # ./tail -f test_file | grep 123 >>>> >>>> (1) root @ aixoss-automation-3: 6.1.0.0: / >>>> >>>> It stops immediately and it seems like this commit >>>> >>>> >> > https://github.com/coreutils/coreutils/commit/ce0415fda108b7ec35181118fd7a2c9ee70331ee > > >> >>>> >>>> has introduce this behavior. >>>> >>>> I checked in Linux with coreutils 8.30 where it works as like earlier >>>> versions. >>>> >>>> Thanks >>>> Ayappan P >> >>> Anyone has any idea on this issue ? >>> >>> Thanks >>> Ayappan P >> >> Thanks for reporting. >> It's hard (at least for me) to get hold on to an AIX system, >> so would you post a trace file (from 'truss'), please? >> >> Second, is this specific to a certain AIX version? >> >> BTW: our tests should have caught this before the release. >> Do you also get an error during 'make check'? > > Our tests were incorrect :/ > > Note the need for this extra check was discussed at: > https://lists.gnu.org/archive/html/coreutils/2017-06/msg00010.html > > > > Note the initial implementation there was with poll() > rather than select(). That may work better on AIX. > Could you try the poll() solution at the above link, > on your system? > > Attached is a fixup for the test and an avoidance > of the issue on AIX. > > cheers, > Pádraig. > [attachment "tail-aix.patch" deleted by Ayappan P2/India/IBM] > > [attachment "graycol.gif" deleted by Ayappan P2/India/IBM] > >
bug#33946: tail -f stops abruptly in AIX when piped.
Now the poll() solution is working in AIX, will there be any official patch for this ? It will be great to have this integrated into the next release. Thanks Ayappan P From: "Ayappan P2" To: "Pádraig Brady" Cc: 33...@debbugs.gnu.org, Bernhard Voelker , Bug-coreutils Date: 01/07/2019 02:37 PM Subject: bug#33946: tail -f stops abruptly in AIX when piped. Sent by:"Bug-coreutils" The poll() solution is working in AIX. Great. Thanks Ayappan P From:"Pádraig Brady" To: Ayappan P2 , Bernhard Voelker Cc: 33...@debbugs.gnu.org Date:01/04/2019 11:37 PM Subject: bug#33946: tail -f stops abruptly in AIX when piped. Sent by: "Bug-coreutils" On 03/01/19 23:01, Ayappan P2 wrote: > > The problem happens only when we pipe the output of "tail -f" . > > I am not sure how one can take the truss of "/tail -f test_file | grep > 123" . > > I did little debugging on the tail code. This function "check_output_alive" > introduced by the commit (mentioned earlier in the thread) sents SIGPIPE > after doing a select () call in AIX. > And that makes it exit immediately. > >fd_set rfd; >FD_ZERO (); >FD_SET (STDOUT_FILENO, ); > > /* readable event on STDOUT is equivalent to POLLERR, > and implies an error condition on output like broken pipe. */ >if (select (STDOUT_FILENO + 1, , NULL, NULL, ) == 1) > raise (SIGPIPE); > } > > I didn't understand the real reason behind this commit. > > Thanks > Ayappan P > > > > From: Bernhard Voelker > To: Ayappan P2 , 33...@debbugs.gnu.org > Date: 01/03/2019 11:53 PM > Subject:bug#33946: tail -f stops abruptly in > AIX when piped. > Sent by:"Bug-coreutils" +ayappap2=in.ibm@gnu.org> > > > > On 1/3/19 6:39 PM, Ayappan P2 wrote: >>> On 01-Jan-2019, at 10:36 PM, Ayappan P2 wrote: >>> Hi, >>> >>> I am running coreutils 8.30 in AIX machine and it seems like "tail -f" > is >>> not working as it used to be when the output is piped. >>> >>> # ./tail -f test_file | grep 123 >>> >>> (1) root @ aixoss-automation-3: 6.1.0.0: / >>> >>> It stops immediately and it seems like this commit >>> >>> > https://github.com/coreutils/coreutils/commit/ce0415fda108b7ec35181118fd7a2c9ee70331ee > >>> >>> has introduce this behavior. >>> >>> I checked in Linux with coreutils 8.30 where it works as like earlier >>> versions. >>> >>> Thanks >>> Ayappan P > >> Anyone has any idea on this issue ? >> >> Thanks >> Ayappan P > > Thanks for reporting. > It's hard (at least for me) to get hold on to an AIX system, > so would you post a trace file (from 'truss'), please? > > Second, is this specific to a certain AIX version? > > BTW: our tests should have caught this before the release. > Do you also get an error during 'make check'? Our tests were incorrect :/ Note the need for this extra check was discussed at: https://lists.gnu.org/archive/html/coreutils/2017-06/msg00010.html Note the initial implementation there was with poll() rather than select(). That may work better on AIX. Could you try the poll() solution at the above link, on your system? Attached is a fixup for the test and an avoidance of the issue on AIX. cheers, Pádraig. [attachment "tail-aix.patch" deleted by Ayappan P2/India/IBM] [attachment "graycol.gif" deleted by Ayappan P2/India/IBM]
bug#33946: tail -f stops abruptly in AIX when piped.
The poll() solution is working in AIX. Great. Thanks Ayappan P From: "Pádraig Brady" To: Ayappan P2 , Bernhard Voelker Cc: 33...@debbugs.gnu.org Date: 01/04/2019 11:37 PM Subject: bug#33946: tail -f stops abruptly in AIX when piped. Sent by:"Bug-coreutils" On 03/01/19 23:01, Ayappan P2 wrote: > > The problem happens only when we pipe the output of "tail -f" . > > I am not sure how one can take the truss of "/tail -f test_file | grep > 123" . > > I did little debugging on the tail code. This function "check_output_alive" > introduced by the commit (mentioned earlier in the thread) sents SIGPIPE > after doing a select () call in AIX. > And that makes it exit immediately. > >fd_set rfd; >FD_ZERO (); >FD_SET (STDOUT_FILENO, ); > > /* readable event on STDOUT is equivalent to POLLERR, > and implies an error condition on output like broken pipe. */ >if (select (STDOUT_FILENO + 1, , NULL, NULL, ) == 1) > raise (SIGPIPE); > } > > I didn't understand the real reason behind this commit. > > Thanks > Ayappan P > > > > From: Bernhard Voelker > To: Ayappan P2 , 33...@debbugs.gnu.org > Date: 01/03/2019 11:53 PM > Subject: bug#33946: tail -f stops abruptly in AIX when piped. > Sent by: "Bug-coreutils" +ayappap2=in.ibm@gnu.org> > > > > On 1/3/19 6:39 PM, Ayappan P2 wrote: >>> On 01-Jan-2019, at 10:36 PM, Ayappan P2 wrote: >>> Hi, >>> >>> I am running coreutils 8.30 in AIX machine and it seems like "tail -f" > is >>> not working as it used to be when the output is piped. >>> >>> # ./tail -f test_file | grep 123 >>> >>> (1) root @ aixoss-automation-3: 6.1.0.0: / >>> >>> It stops immediately and it seems like this commit >>> >>> > https://github.com/coreutils/coreutils/commit/ce0415fda108b7ec35181118fd7a2c9ee70331ee > >>> >>> has introduce this behavior. >>> >>> I checked in Linux with coreutils 8.30 where it works as like earlier >>> versions. >>> >>> Thanks >>> Ayappan P > >> Anyone has any idea on this issue ? >> >> Thanks >> Ayappan P > > Thanks for reporting. > It's hard (at least for me) to get hold on to an AIX system, > so would you post a trace file (from 'truss'), please? > > Second, is this specific to a certain AIX version? > > BTW: our tests should have caught this before the release. > Do you also get an error during 'make check'? Our tests were incorrect :/ Note the need for this extra check was discussed at: https://lists.gnu.org/archive/html/coreutils/2017-06/msg00010.html Note the initial implementation there was with poll() rather than select(). That may work better on AIX. Could you try the poll() solution at the above link, on your system? Attached is a fixup for the test and an avoidance of the issue on AIX. cheers, Pádraig. [attachment "tail-aix.patch" deleted by Ayappan P2/India/IBM]
bug#33946: tail -f stops abruptly in AIX when piped.
On 03/01/19 23:01, Ayappan P2 wrote:
>
> The problem happens only when we pipe the output of "tail -f" .
>
> I am not sure how one can take the truss of "/tail -f test_file | grep
> 123" .
>
> I did little debugging on the tail code. This function "check_output_alive"
> introduced by the commit (mentioned earlier in the thread) sents SIGPIPE
> after doing a select () call in AIX.
> And that makes it exit immediately.
>
>fd_set rfd;
>FD_ZERO ();
>FD_SET (STDOUT_FILENO, );
>
> /* readable event on STDOUT is equivalent to POLLERR,
> and implies an error condition on output like broken pipe. */
>if (select (STDOUT_FILENO + 1, , NULL, NULL, ) == 1)
> raise (SIGPIPE);
> }
>
> I didn't understand the real reason behind this commit.
>
> Thanks
> Ayappan P
>
>
>
> From: Bernhard Voelker
> To: Ayappan P2 , 33...@debbugs.gnu.org
> Date: 01/03/2019 11:53 PM
> Subject: bug#33946: tail -f stops abruptly in AIX when piped.
> Sent by: "Bug-coreutils" +ayappap2=in.ibm@gnu.org>
>
>
>
> On 1/3/19 6:39 PM, Ayappan P2 wrote:
>>> On 01-Jan-2019, at 10:36 PM, Ayappan P2 wrote:
>>> Hi,
>>>
>>> I am running coreutils 8.30 in AIX machine and it seems like "tail -f"
> is
>>> not working as it used to be when the output is piped.
>>>
>>> # ./tail -f test_file | grep 123
>>>
>>> (1) root @ aixoss-automation-3: 6.1.0.0: /
>>>
>>> It stops immediately and it seems like this commit
>>>
>>>
> https://github.com/coreutils/coreutils/commit/ce0415fda108b7ec35181118fd7a2c9ee70331ee
>
>>>
>>> has introduce this behavior.
>>>
>>> I checked in Linux with coreutils 8.30 where it works as like earlier
>>> versions.
>>>
>>> Thanks
>>> Ayappan P
>
>> Anyone has any idea on this issue ?
>>
>> Thanks
>> Ayappan P
>
> Thanks for reporting.
> It's hard (at least for me) to get hold on to an AIX system,
> so would you post a trace file (from 'truss'), please?
>
> Second, is this specific to a certain AIX version?
>
> BTW: our tests should have caught this before the release.
> Do you also get an error during 'make check'?
Our tests were incorrect :/
Note the need for this extra check was discussed at:
https://lists.gnu.org/archive/html/coreutils/2017-06/msg00010.html
Note the initial implementation there was with poll()
rather than select(). That may work better on AIX.
Could you try the poll() solution at the above link,
on your system?
Attached is a fixup for the test and an avoidance
of the issue on AIX.
cheers,
Pádraig.
From 69f4b0db2af91b2b67608ae5205d54b7fff2a53c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?=
Date: Fri, 4 Jan 2019 09:29:13 -0800
Subject: [PATCH] tail: don't exit immediately with filters on AIX
* src/tail.c: Avoid the check_output_available check on AIX.
* tests/tail-2/pipe-f.sh: Fix the test which always passed
due to only the exit code of sleep being checked.
* NEWS: Mention the bug fix and rearrange alphabetically.
Addresses http://bugs.gnu.org/33946
---
NEWS | 8 +---
src/tail.c | 5 +
tests/tail-2/pipe-f.sh | 5 -
3 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/NEWS b/NEWS
index 4a57d22..6a9c0bc 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,9 @@ GNU coreutils NEWS-*- outline -*-
** Bug fixes
+ 'base64 a b' now correctly diagnoses 'b' as the extra operand, not 'a'.
+ [bug introduced in coreutils-5.3.0]
+
When B already exists, 'cp -il A B' no longer immediately fails
after asking the user whether to proceed.
[This bug was present in "the beginning".]
@@ -21,9 +24,8 @@ GNU coreutils NEWS-*- outline -*-
sync no longer fails for write-only file arguments.
[bug introduced with argument support to sync in coreutils-8.24]
- In 'base64 a b', and likewise for base32, the tool now correctly
- diagnoses 'b' as the extra operand, not 'a'.
- [bug introduced in coreutils-5.3.0]
+ 'tail -f file | filter' no longer exits immediately on AIX.
+ [bug introduced in coreutils-8.28]
** Changes in behavior
diff --git a/src/tail.c b/src/tail.c
index 0270cbe..85de88d 100644
--- a/src/tail.c
+++ b/src/tail.c
@@ -336,6 +336,11 @@ named file in a way that accommodates renaming, removal and creation.\n\
static void
check_output_alive (void)
{
+#ifdef _AIX
+ /* TODO: AIX was seen to give a readable event immediately. */
+ return;
+#endif
+
if (! monitor_output)
return;
diff --git a/tests/tail-2/pipe-f.sh b/tests/tail-2/pipe-f.sh
index 9231cac..4a5b444 100
bug#33946: tail -f stops abruptly in AIX when piped.
The problem happens only when we pipe the output of "tail -f" . I am not sure how one can take the truss of "/tail -f test_file | grep 123" . I did little debugging on the tail code. This function "check_output_alive" introduced by the commit (mentioned earlier in the thread) sents SIGPIPE after doing a select () call in AIX. And that makes it exit immediately. fd_set rfd; FD_ZERO (); FD_SET (STDOUT_FILENO, ); /* readable event on STDOUT is equivalent to POLLERR, and implies an error condition on output like broken pipe. */ if (select (STDOUT_FILENO + 1, , NULL, NULL, ) == 1) raise (SIGPIPE); } I didn't understand the real reason behind this commit. Thanks Ayappan P From: Bernhard Voelker To: Ayappan P2 , 33...@debbugs.gnu.org Date: 01/03/2019 11:53 PM Subject: bug#33946: tail -f stops abruptly in AIX when piped. Sent by:"Bug-coreutils" On 1/3/19 6:39 PM, Ayappan P2 wrote: >> On 01-Jan-2019, at 10:36 PM, Ayappan P2 wrote: >> Hi, >> >> I am running coreutils 8.30 in AIX machine and it seems like "tail -f" is >> not working as it used to be when the output is piped. >> >> # ./tail -f test_file | grep 123 >> >> (1) root @ aixoss-automation-3: 6.1.0.0: / >> >> It stops immediately and it seems like this commit >> >> https://github.com/coreutils/coreutils/commit/ce0415fda108b7ec35181118fd7a2c9ee70331ee >> >> has introduce this behavior. >> >> I checked in Linux with coreutils 8.30 where it works as like earlier >> versions. >> >> Thanks >> Ayappan P > Anyone has any idea on this issue ? > > Thanks > Ayappan P Thanks for reporting. It's hard (at least for me) to get hold on to an AIX system, so would you post a trace file (from 'truss'), please? Second, is this specific to a certain AIX version? BTW: our tests should have caught this before the release. Do you also get an error during 'make check'? Have a nice day, Berny
bug#33946: tail -f stops abruptly in AIX when piped.
On 1/3/19 6:39 PM, Ayappan P2 wrote: >> On 01-Jan-2019, at 10:36 PM, Ayappan P2 wrote: >> Hi, >> >> I am running coreutils 8.30 in AIX machine and it seems like "tail -f" is >> not working as it used to be when the output is piped. >> >> # ./tail -f test_file | grep 123 >> >> (1) root @ aixoss-automation-3: 6.1.0.0: / >> >> It stops immediately and it seems like this commit >> >> https://github.com/coreutils/coreutils/commit/ce0415fda108b7ec35181118fd7a2c9ee70331ee >> >> has introduce this behavior. >> >> I checked in Linux with coreutils 8.30 where it works as like earlier >> versions. >> >> Thanks >> Ayappan P > Anyone has any idea on this issue ? > > Thanks > Ayappan P Thanks for reporting. It's hard (at least for me) to get hold on to an AIX system, so would you post a trace file (from 'truss'), please? Second, is this specific to a certain AIX version? BTW: our tests should have caught this before the release. Do you also get an error during 'make check'? Have a nice day, Berny
bug#33946: tail -f stops abruptly in AIX when piped.
Anyone has any idea on this issue ? Thanks Ayappan P > On 01-Jan-2019, at 10:36 PM, Ayappan P2 wrote: > > > > Hi, > > I am running coreutils 8.30 in AIX machine and it seems like " tail -f " is > not working as it used to be when the output is piped. > > # ./tail -f test_file | grep 123 > > (1) root @ aixoss-automation-3: 6.1.0.0: / > > It stops immediately and it seems like this commit > https://github.com/coreutils/coreutils/commit/ce0415fda108b7ec35181118fd7a2c9ee70331ee > has introduce this behavior. > > I checked in Linux with coreutils 8.30 where it works as like earlier > versions. > > Thanks > Ayappan P >
bug#33946: tail -f stops abruptly in AIX when piped.
Hi, I am running coreutils 8.30 in AIX machine and it seems like " tail -f " is not working as it used to be when the output is piped. # ./tail -f test_file | grep 123 (1) root @ aixoss-automation-3: 6.1.0.0: / It stops immediately and it seems like this commit https://github.com/coreutils/coreutils/commit/ce0415fda108b7ec35181118fd7a2c9ee70331ee has introduce this behavior. I checked in Linux with coreutils 8.30 where it works as like earlier versions. Thanks Ayappan P
