Re: implicit redirection of background within pipeline
On Wed, 13 Jan 2016, Chet Ramey wrote:
> On 1/13/16 4:08 AM, Andreas Schwab wrote:
> > Martin Kealey writes:
> >
> >> What do other shells do? Ksh? Dash? Zsh?
> >
> > $ ksh -c 'printf "foo1\nfoo2\n" | { (read x; echo 1: $x) & (read x; echo 2:
> > $x) & }'
> > 1: foo1
> > 2: foo2
>
> I changed bash to behave like this, which I think is the most reasonable.
Agreed, thank you.
-Martin
Re: implicit redirection of background within pipeline
On 1/13/16 4:08 AM, Andreas Schwab wrote:
> Martin Kealey writes:
>
>> What do other shells do? Ksh? Dash? Zsh?
>
> $ ksh -c 'printf "foo1\nfoo2\n" | { (read x; echo 1: $x) & (read x; echo 2:
> $x) & }'
> 1: foo1
> 2: foo2
I changed bash to behave like this, which I think is the most reasonable.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: implicit redirection of background within pipeline
On 1/12/16 7:20 PM, Martin Kealey wrote: > > > On Mon, 11 Jan 2016, Chet Ramey wrote: >> "The standard input for an asynchronous list, before any explicit >> redirections are performed, shall be considered to be assigned to a file >> that has the same properties as /dev/null." > > I thought the (only) point of that to avoid a background job reading the tty > in an environment that lacked job control. That's the effect now, yes. The original Bourne shell, since it didn't have job control, made no distinction, and the standard doesn't either. > The standard is actually a bit vague; under "Compound Commands" it mentions > that "Each redirection will apply to all the commands within the compound > command that do not explicitly override that redirection" but it fails to > mention that this applies to pipes as well; so I submit that it is a > reasonable interpretation that a pipe on an outer block counts as an > (explicit) redirection for the purpose of reading the "Asynchronous Lists" > sub section. Agreed, and, as I said, the existing code attempts to do that. The problem is propagating the input pipe status into commands executed as part of a compound command like a () subshell. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: implicit redirection of background within pipeline
On Wed, 13 Jan 2016, Andreas Schwab wrote: > What does "take them as read" mean? Taking something as read means that the author thought we would understand it without them actually writing it down. Or more prosaically, it means to pretend that they're written there, even though they're not, because otherwise what's actually written is nonsense. In this case, they mention that redirections applied to a compound statement take effect on all its parts, but they do NOT mention that a pipe connecting a compound statement to some other statement takes effect on its parts. But we have to assume it does, otherwise all manner of stuff is broken, even though they haven't written it. -Martin
Re: implicit redirection of background within pipeline
Martin Kealey writes: > On 13 Jan 2016 8:14 p.m., "Andreas Schwab" wrote: > >> I don't think so. >> > http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07 >> doesn't mention pipelines. > > That was rather my point. Given that they aren't mentioned, one has to take > them as read, What does "take them as read" mean? Andreas. -- Andreas Schwab, SUSE Labs, sch...@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."
Re: implicit redirection of background within pipeline
On 13 Jan 2016 8:14 p.m., "Andreas Schwab" wrote: > I don't think so. > http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07 > doesn't mention pipelines. That was rather my point. Given that they aren't mentioned, one has to take them as read, otherwise even ( ( foo 2>&1 ) | bar ) won't work as advertised.
Re: implicit redirection of background within pipeline
Martin Kealey writes: > The standard is actually a bit vague; under "Compound Commands" it mentions > that "Each redirection will apply to all the commands within the compound > command that do not explicitly override that redirection" but it fails to > mention that this applies to pipes as well; so I submit that it is a > reasonable interpretation that a pipe on an outer block counts as an > (explicit) redirection for the purpose of reading the "Asynchronous Lists" > sub section. I don't think so. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07 doesn't mention pipelines. Andreas. -- Andreas Schwab, SUSE Labs, sch...@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."
Re: implicit redirection of background within pipeline
Martin Kealey writes:
> What do other shells do? Ksh? Dash? Zsh?
$ ksh -c 'printf "foo1\nfoo2\n" | { (read x; echo 1: $x) & (read x; echo 2: $x)
& }'
1: foo1
2: foo2
$ dash -c 'printf "foo1\nfoo2\n" | { (read x; echo 1: $x) & (read x; echo 2:
$x) & }'
1:
2:
$ zsh -c 'printf "foo1\nfoo2\n" | { (read x; echo 1: $x) & (read x; echo 2: $x)
& }'
1: foo1
2: foo2
$ bash -c 'printf "foo1\nfoo2\n" | { (read x; echo 1: $x) & (read x; echo 2:
$x) & }'
1:
2: foo1
Andreas.
--
Andreas Schwab, SUSE Labs, sch...@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
Re: implicit redirection of background within pipeline
On Mon, 11 Jan 2016, Chet Ramey wrote: > "The standard input for an asynchronous list, before any explicit > redirections are performed, shall be considered to be assigned to a file > that has the same properties as /dev/null." I thought the (only) point of that to avoid a background job reading the tty in an environment that lacked job control. The standard is actually a bit vague; under "Compound Commands" it mentions that "Each redirection will apply to all the commands within the compound command that do not explicitly override that redirection" but it fails to mention that this applies to pipes as well; so I submit that it is a reasonable interpretation that a pipe on an outer block counts as an (explicit) redirection for the purpose of reading the "Asynchronous Lists" sub section. What do other shells do? Ksh? Dash? Zsh? -Martin
Re: implicit redirection of background within pipeline
On 1/10/16 2:23 AM, Martin D Kealey wrote: > Bash Version: 4.3 > Patch Level: 11 > Release Status: release > > Description: > The first backgrounded element within a pipeline component has its > stdin redirected to /dev/null; this does not apply to the second and > subsequent background elements. > > On a Linux system this can be observed with: > echo x | ( ls -ld /proc/self/fd/0 & > ls -ld /proc/self/fd/0 & > wait ) > > On other systems it is apparent simply with: > echo x | ( cat & wait ) Thanks for the report. I fixed this family of problems and the fix will be in the next bash release, as well as the next devel snapshot. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: implicit redirection of background within pipeline
Surely a pipe counts as an explicit redirection? And even if it didn't, the inconsistency between the first and subsequent asynchronous elements is confusing. On 12 Jan 2016 02:41, "Chet Ramey" wrote: > On 1/10/16 12:54 PM, Piotr Grzybowski wrote: > > hey, > > > > I am quite sure it happens here (devel branch, at > > 6f82653c5ef09aeeeba4376a1c65ce86c3605c00): > > > > execute_cmd.c +5115: > >if ((cmdflags & CMD_STDIN_REDIR) && > > pipe_in == NO_PIPE && > > (stdin_redirects (redirects) == 0)) > > async_redirect_stdin (); > > > > but after reading the comments some 3.5k lines earlier I am not sure > > how to fix it; of course commenting out the if in > > execute_cmd.c:5112-5115 solves it, but who can tell what it breaks? > > Posix requires this behavior: > > "The standard input for an asynchronous list, before any explicit > redirections are performed, shall be considered to be assigned to a file > that has the same properties as /dev/null." > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, ITS, CWRUc...@case.edu > http://cnswww.cns.cwru.edu/~chet/ >
Re: implicit redirection of background within pipeline
On 1/11/16 3:37 PM, Chet Ramey wrote: > On 1/11/16 3:29 PM, Martin Kealey wrote: >> Surely a pipe counts as an explicit redirection? > > It doesn't. I should be clear that this is according to what the standard says. The existing code attempts to inhibit this implicit redirection in the presence of an input pipe. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: implicit redirection of background within pipeline
On 1/11/16 3:29 PM, Martin Kealey wrote: > Surely a pipe counts as an explicit redirection? It doesn't. > And even if it didn't, the inconsistency between the first and subsequent > asynchronous elements is confusing. Agreed. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: implicit redirection of background within pipeline
On 1/11/16 11:37 AM, Andreas Schwab wrote: > Chet Ramey writes: > >> On 1/10/16 12:54 PM, Piotr Grzybowski wrote: >>> hey, >>> >>> I am quite sure it happens here (devel branch, at >>> 6f82653c5ef09aeeeba4376a1c65ce86c3605c00): >>> >>> execute_cmd.c +5115: >>>if ((cmdflags & CMD_STDIN_REDIR) && >>> pipe_in == NO_PIPE && >>> (stdin_redirects (redirects) == 0)) >>> async_redirect_stdin (); >>> >>> but after reading the comments some 3.5k lines earlier I am not sure >>> how to fix it; of course commenting out the if in >>> execute_cmd.c:5112-5115 solves it, but who can tell what it breaks? >> >> Posix requires this behavior: >> >> "The standard input for an asynchronous list, before any explicit >> redirections are performed, shall be considered to be assigned to a file >> that has the same properties as /dev/null." > > But isn't it a bug that subsequent asynchronous commands in the list are > not redirected from /dev/null? Yes, clearly the existing behavior is inconsistent. My response answered Piotr's question about what it breaks. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: implicit redirection of background within pipeline
Chet Ramey writes: > On 1/10/16 12:54 PM, Piotr Grzybowski wrote: >> hey, >> >> I am quite sure it happens here (devel branch, at >> 6f82653c5ef09aeeeba4376a1c65ce86c3605c00): >> >> execute_cmd.c +5115: >>if ((cmdflags & CMD_STDIN_REDIR) && >> pipe_in == NO_PIPE && >> (stdin_redirects (redirects) == 0)) >> async_redirect_stdin (); >> >> but after reading the comments some 3.5k lines earlier I am not sure >> how to fix it; of course commenting out the if in >> execute_cmd.c:5112-5115 solves it, but who can tell what it breaks? > > Posix requires this behavior: > > "The standard input for an asynchronous list, before any explicit > redirections are performed, shall be considered to be assigned to a file > that has the same properties as /dev/null." But isn't it a bug that subsequent asynchronous commands in the list are not redirected from /dev/null? Andreas. -- Andreas Schwab, SUSE Labs, sch...@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."
Re: implicit redirection of background within pipeline
On 1/10/16 12:54 PM, Piotr Grzybowski wrote: > hey, > > I am quite sure it happens here (devel branch, at > 6f82653c5ef09aeeeba4376a1c65ce86c3605c00): > > execute_cmd.c +5115: >if ((cmdflags & CMD_STDIN_REDIR) && > pipe_in == NO_PIPE && > (stdin_redirects (redirects) == 0)) > async_redirect_stdin (); > > but after reading the comments some 3.5k lines earlier I am not sure > how to fix it; of course commenting out the if in > execute_cmd.c:5112-5115 solves it, but who can tell what it breaks? Posix requires this behavior: "The standard input for an asynchronous list, before any explicit redirections are performed, shall be considered to be assigned to a file that has the same properties as /dev/null." -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: implicit redirection of background within pipeline
hey, I am quite sure it happens here (devel branch, at 6f82653c5ef09aeeeba4376a1c65ce86c3605c00): execute_cmd.c +5115: if ((cmdflags & CMD_STDIN_REDIR) && pipe_in == NO_PIPE && (stdin_redirects (redirects) == 0)) async_redirect_stdin (); but after reading the comments some 3.5k lines earlier I am not sure how to fix it; of course commenting out the if in execute_cmd.c:5112-5115 solves it, but who can tell what it breaks? cheers, pg On Sun, Jan 10, 2016 at 8:23 AM, Martin D Kealey wrote: > > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' > -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' > -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL > -DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib -D_FORTIFY_SOURCE=2 > -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat > -Werror=format-security -Wall > uname output: Linux treehug.home.kurahaupo.gen.nz 3.13.0-55-generic > #94-Ubuntu SMP Thu Jun 18 00:27:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux > Machine Type: x86_64-pc-linux-gnu > > Bash Version: 4.3 > Patch Level: 11 > Release Status: release > > Description: > The first backgrounded element within a pipeline component has its > stdin redirected to /dev/null; this does not apply to the second and > subsequent background elements. > > On a Linux system this can be observed with: > echo x | ( ls -ld /proc/self/fd/0 & >ls -ld /proc/self/fd/0 & >wait ) > > On other systems it is apparent simply with: > echo x | ( cat & wait ) > # no output is produced > > Repeat-By: > # this command produces no output: > echo x | ( cat & wait ) > > Fix: > # this is only a work-around: > echo x ( :& cat & wait ) > >
implicit redirection of background within pipeline
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall uname output: Linux treehug.home.kurahaupo.gen.nz 3.13.0-55-generic #94-Ubuntu SMP Thu Jun 18 00:27:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 4.3 Patch Level: 11 Release Status: release Description: The first backgrounded element within a pipeline component has its stdin redirected to /dev/null; this does not apply to the second and subsequent background elements. On a Linux system this can be observed with: echo x | ( ls -ld /proc/self/fd/0 & ls -ld /proc/self/fd/0 & wait ) On other systems it is apparent simply with: echo x | ( cat & wait ) # no output is produced Repeat-By: # this command produces no output: echo x | ( cat & wait ) Fix: # this is only a work-around: echo x ( :& cat & wait )
