Re: program gets faster when it outputs more on stderr
On Sat, Mar 10, 2018 at 04:50:22PM -0500, Chet Ramey wrote: > On 3/10/18 9:46 AM, bash...@jonkmans.nl wrote: > > > Bash Version: 4.4 > > Patch Level: 12 > > Release Status: release > > > > Description: > > When a function is ran in a subshell environment (via backticks), > > the program runs faster when that function also writes to stderr. > > I don't get these results. I ran it a few times out of curiosity, and the > `fast' version ran 2-3 times slower than the `slow' one. Even running it > against a profiling version doesn't show any significant difference in > function calls. > > Your results could be due to many factors: stdio buffering, a scheduler > that biases perceived interactive processes, I/O optimizations. Nothing > to do with bash, though. > When i run the script with realtime prio (as root) via : chrt -r 99
Re: program gets faster when it outputs more on stderr
Date:Sat, 10 Mar 2018 15:46:41 +0100 (CET) From:bash...@jonkmans.nl Message-ID: <20180310144641.62962cc0...@sint.jonkmans.nl> First: | Note that when the fast version is ran with 2>/dev/null, | it also performs slowly. This is not unexpected, and probably unrelated - written that way the program needs to look up /dev/null and open it, and then close it again, on every iteration, which neither of the other variants require. To make it obvious whether there is a difference writing to /dev/null or stderr, you should do it like exec 3>/dev/null (in the main program, before anything starts) and then use >&3 in the "sub". I can't comments much on the real point of your message, as I don't know the bash code, but ... | When a function is ran in a subshell environment (via backticks), | the program runs faster when that function also writes to stderr. | Expected that, the version writing to stderr would be slower. You ought also measure and print the CPU time consumed, I expect you'd find that actually grows (by a small amount) in the "fast" case. As to the issue - I'd expect some scheduelling/signalling issue, where the output on stderr causes the parent process (running sub requires a subshell environment because of the ``) to awaken sooner than it otherwise would, at that point it detects that the child is done, again sooner than it otherwise would, and continues with the next iteration. Why that happens (assuming that is what is happening) someone else (probably Chet) would need to explain. Assuming I'm right about why things speed up, then the version using /dev/null via fd 3 is also likely to not gain from adding the echo - but it will take an experiment to discover if that is true or not. kre ps: the numbers in the output are probably thousandths of a second in a run, rather than thousands of seconds, 443000 seconds is something like 5 days...
Re: program gets faster when it outputs more on stderr
On 3/10/18 9:46 AM, bash...@jonkmans.nl wrote: > Bash Version: 4.4 > Patch Level: 12 > Release Status: release > > Description: > When a function is ran in a subshell environment (via backticks), > the program runs faster when that function also writes to stderr. I don't get these results. I ran it a few times out of curiosity, and the `fast' version ran 2-3 times slower than the `slow' one. Even running it against a profiling version doesn't show any significant difference in function calls. Your results could be due to many factors: stdio buffering, a scheduler that biases perceived interactive processes, I/O optimizations. Nothing to do with bash, though. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
program gets faster when it outputs more on stderr
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
-DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib -Wdate-time
-D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/bash-IrsGKQ/bash-4.4=.
-fstack-protector-strong -Wformat -Werror=format-security -Wall -no-pie
-Wno-parentheses -Wno-format-security
uname output: Linux sint 4.13.0-36-generic #40-Ubuntu SMP Fri Feb 16 20:07:48
UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu
Bash Version: 4.4
Patch Level: 12
Release Status: release
Description:
When a function is ran in a subshell environment (via backticks),
the program runs faster when that function also writes to stderr.
This effect does not occur when stderr is redirected to /dev/null
Expected that, the version writing to stderr would be slower.
I am not sure where the problem lies.
It could be bash, terminal, X, kernel or hardware.
Tested this on XWayland, gnome-terminal (with and without tmux),
and xterm.
Hope you can help.
Regards, Mike
Repeat-By:
An example program that exhibits the described behaviour:
### cut here ###
#!/bin/bash
sub () {
if [ "$SPEED" = fast ]; then
echo -n fast >&2
fi
echo sub
}
prog () {
declare x line
while read line; do
x=`sub`
done < <(cat /etc/services /etc/services)
}
main () {
local -g SPEED="$1"
local -i i=${RUNS:-10}
local start
echo $SPEED
while [ $((i--)) -gt 0 ]; do
start=$(date +%s%N)
prog
echo $(( ($(date +%s%N) - $start)/100 ))
done
}
main "$@"
### cut here ###
When saved as 'bug.bash' and ran as :
./bug.bash slow < <( cat /etc/services /etc/services )
slow
1644
1652
1663
1663
1661
1658
1656
1653
1646
1641
On the other hand, when ran as:
./bug.bash fast < <( cat /etc/services /etc/services )
fast
443
424
436
422
439
421
433
428
425
435
(numbers are thousands of seconds in a single run)
The second, fast-version, does more, but takes less time.
Note that when the fast version is ran with 2>/dev/null,
it also performs slowly.
I have looked at the strace's of both variants, but saw
nothing realy different.
