helo!
before Ryan kills me :) I am replying to all, no matter what
meaningless rubbish I post. tell Ryan I got his message :)
Yuri: I have verified that under linux 3.2.0, bash-4.2.25, nothing of
the sort takes place. tee gets /dev/fd/${somefd}, why it is not
supported in bsd kernel i have no idea, but I thought it was. I've
even built mplayer and run your script (btw. probably there is an
easier way of doing what you want), I also performed some tests with
command that actually outputs what grep looks for. Also please note
that `command substitution`, or $(command substitution) is slightly
different from >(process substitution).
sincerely,
pg
On Wed, Dec 11, 2013 at 10:36 PM, Yuri <y...@rawbw.com> wrote:
> On 12/10/2013 23:29, Yuri wrote:
>>
>> Some of my scripts use command substitution, and now I see that there are
>> lots of files like these in /tmp:
>> prw------- 1 yuri wheel 0 Dec 10 13:32 sh-np-1386738492
>> -rw-r--r-- 1 yuri wheel 3278909 Dec 10 14:54 sh-np-1386721176
>>
>> Besides the obvious question why they aren't deleted in the end, two
>> questions:
>> Shouldn't bash delete the pipe files once the child process opened them?
>> On most platforms the deleted pipe will function the same with the benefit
>> that it will get deleted automatically.
>> Why some of he files /tmp/sh-np-NNN are regular files, not pipes? When I
>> look through code I see mkfifo call creates them. Shouldn't it always create
>> fifos and not files?
>
>
> Ok, I figured the details out.
> The actual script that I run is this radio playing command:
> stdbuf -i 0 -o 0 -e 0 mplayer
> http://kqed-ice.streamguys.org:80/kqedradio-ch-e1 | \
> stdbuf -i 0 -o 0 -e 0 tee >(grep "^XXX XXX" | prepend-time >> test.log)
>
> prepend-time is a script:
>
> gawk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; }'
>
> The problem only occurs with stdbuf(1) commands inserted there, and not
> without them.
> 1: bash creates a FIFO /tmp/sh-np-NNNNN
> 2: bash unlinks this FIFO
> 3: tee is passed /tmp/sh-np-NNNNN as an argument, and it creates a regular
> file with the same name
>
> This is the race condition.
> Obviously, bash doesn't ensure that unlink is issued after the child command
> opons it, and not before that. Order of operations 2 and 3 should be
> reversed.
> And just waiting for a while isn't enough! bash should only unlink the fifo
> after it has been open by the child command.
>
> Please also note, that this is on FreeBSD, which behavior might be
> different. Linux or other OSes might be getting lucky here with timing, not
> sure. FreeBSD also has its own version of stdbuf(1).
> This bug needs to be fixed.
>
> The original intention of stdbuf there was to make it all unbufferer.
> Whether this works or not isn't clear, but it doesn't matter for the purpose
> of this bug report.
>
> Suggested solution:
> On FreeBSD, for example, there is the kqueue(2) facility, which is able to
> wait on the event of "link-change": EVFILT_VNODE operation (takes file
> descriptor) allows sub-operation NOTE_LINK (The link count on the file
> changed)
> So the sequence will be like this:
> 1. bash creates FIFO
> 2. bash opens FIFO
> 3. bash waits with kqueue(2) with EVFILT_VNODE/NOTE_LINK
> 4. bash launches the child command
> 5. child command opens the FIFO as stdio
> 6. bash catches the link-change event
> 7. bash unlinks FIFO
>
> Linux and other OSes must have an equivalent facility.
> Sounds like a fun project to do. -)
> Absolute must-have if bash needs to support command substitution.
>
> Yuri
>
> CCing to the maintainer of bash on FreeBSD
