While tracking down the details of the performance question I reported a couple of days ago (thanks, Elliot and Brad!), I have uncovered surprising behavior on my system. I'm not sure if this is a bug in my code, a Chapel bug (probably library, maybe compiler?), an install problem on my system, or just something even stranger.
The problem is this:
- the system seems to busy-wait during my call to the "sleep" library
function
- when I create an array, all cores do the busy-wait instead of one
- when I create an array, n-1 cores busy-wait during a pause for input
The simple program I wrote to demonstrate this is:
davew@liskov:~/Teaching/Theses/btrush/chapel$ cat
WisconsinTourismFederation.chpl
use Time;
config const nMax : int = 6000;
config const seconds : int = 4;
config const buildit : bool = false;
if (buildit) {
var MatrixD: domain(2) = {1..nMax, 1..nMax};
var m : [MatrixD] int;
}
writeln("start by calling sleep");
sleep(seconds);
writeln("somebody say, 'one, two, three, go!' ...");
var sundance=stdin.read(string);
writeln("heard ", sundance);
When I pause 0 seconds for 'sleep' and 0 seconds for 'stdin.read(string)',
it runs fast, regardless of the value of buildit:
davew@liskov:~/Teaching/Theses/btrush/chapel$ make -f Makefile-cray.mk WTF
chpl --fast -O WisconsinTourismFederation.chpl -o WTF
davew@liskov:~/Teaching/Theses/btrush/chapel$ (sleep 0; echo
'one,_two,_three,_go!') | time ./WTF --buildit=true --seconds=0
start by calling sleep
somebody say, 'one, two, three, go!' ...
heard one,_two,_three,_go!
0.04user 0.31system 0:00.11elapsed 306%CPU (0avgtext+0avgdata
283600maxresident)k
0inputs+0outputs (0major+69446minor)pagefaults 0swaps
davew@liskov:~/Teaching/Theses/btrush/chapel$
However, when I change the amounts of the pauses, it takes a lot of time,
especially with the array creation turned on:
davew@liskov:~/Teaching/Theses/btrush/chapel$ (sleep 0; echo
'one,_two,_three,_go!') | time ./WTF --buildit=false --seconds=0
start by calling sleep
somebody say, 'one, two, three, go!' ...
heard one,_two,_three,_go!
0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata
3116maxresident)k
0inputs+0outputs (0major+151minor)pagefaults 0swaps
davew@liskov:~/Teaching/Theses/btrush/chapel$ (sleep 0; echo
'one,_two,_three,_go!') | time ./WTF --buildit=false --seconds=20
start by calling sleep
somebody say, 'one, two, three, go!' ...
heard one,_two,_three,_go!
6.50user 13.49system 0:20.00elapsed 99%CPU (0avgtext+0avgdata
3120maxresident)k
0inputs+0outputs (0major+151minor)pagefaults 0swaps
davew@liskov:~/Teaching/Theses/btrush/chapel$ (sleep 20; echo
'one,_two,_three,_go!') | time ./WTF --buildit=false --seconds=0
start by calling sleep
somebody say, 'one, two, three, go!' ...
heard one,_two,_three,_go!
0.00user 0.00system 0:20.00elapsed 0%CPU (0avgtext+0avgdata
3064maxresident)k
0inputs+0outputs (0major+145minor)pagefaults 0swaps
davew@liskov:~/Teaching/Theses/btrush/chapel$ (sleep 0; echo
'one,_two,_three,_go!') | time ./WTF --buildit=true --seconds=20
start by calling sleep
somebody say, 'one, two, three, go!' ...
heard one,_two,_three,_go!
21.84user 56.74system 0:20.11elapsed 390%CPU (0avgtext+0avgdata
283640maxresident)k
0inputs+0outputs (0major+66378minor)pagefaults 0swaps
davew@liskov:~/Teaching/Theses/btrush/chapel$ (sleep 20; echo
'one,_two,_three,_go!') | time ./WTF --buildit=true --seconds=0
start by calling sleep
somebody say, 'one, two, three, go!' ...
heard one,_two,_three,_go!
16.34user 43.61system 0:20.00elapsed 299%CPU (0avgtext+0avgdata
283844maxresident)k
0inputs+0outputs (0major+66375minor)pagefaults 0swaps
davew@liskov:~/Teaching/Theses/btrush/chapel$
When I look at the output of "htop" while running this, it shows the
relevant cores 100% busy, but only a part (a quarter to a third) of the bar
graph for each processor is green, with the rest being colored red, I think
(I'm partly color-blind). I'm not entirely sure what that means from htop,
though.
I have attached the WisconsinTourismFederation.chpl example and the
makefile used above. I also have a (39MB) video file
<https://drive.google.com/file/d/0B9RUvs7FnqoJWEFPRXoyLUl4TjA/view> showing
the results of a run with 4 seconds each of sleep and input waiting, in
case you want to see it.
For reference, "sleep 20" in the shell, which presumably calls C's 'sleep'
function, does not show anything interesting on "htop" or "time":
$ time sleep 20
0.00user 0.00system 0:20.00elapsed 0%CPU (0avgtext+0avgdata
1936maxresident)k
0inputs+0outputs (0major+77minor)pagefaults 0swaps
I've CC'd my sysadmin, Divesh Otwani, in case there are questions/thoughts
about the installation.
Thanks for any guidance you can provide,
Dave W
WisconsinTourismFederation.chpl
Description: Binary data
CHPL=chpl
CHOFLAGS=--fast -O
CHDFLAGS=-g
simple-speed-test: nussinov-simple nussinov-simple-experiment
@echo " " ; echo "# The following executes on one core; unsurprisingly, this is fast (6.5s):"
(echo '*' ; sleep 0; echo 'one,_two,_three,_go!') | time ./nussinov-simple-experiment --nMax=3000
@echo " " ; echo "# SURPRISINGLY TO ME, The following executes on all cores; unsurprisingly, this is slow (12.4s):"
(echo '*' ; sleep 60; echo 'one,_two,_three,_go!') | time ./nussinov-simple --nMax=3000
WTF: WisconsinTourismFederation.chpl Makefile-cray.mk
${CHPL} ${CHOFLAGS} $< -o $@
(sleep 8; echo 'one,_two,_three,_go!') | time ./WTF --buildit=false
(sleep 8; echo 'one,_two,_three,_go!') | time ./WTF --buildit=true
nussinov-simple: nussinov-simple-pause.chpl
${CHPL} ${CHOFLAGS} $< -o $@
nussinov-simple-experiment: nussinov-simple-pause.chpl
${CHPL} ${CHOFLAGS} -sparallelInitElts=false $< -o $@
clean:
rm ./nussinov-simple ./nussinov-simple-experiment
------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev
_______________________________________________ Chapel-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-users
