On 19/03/15 23:30, Bernhard Voelker wrote: > On 03/19/2015 11:18 AM, Pádraig Brady wrote: >> Yes 30 was a bit aggressive. >> I had already changed it to 15 (150) locally here after more testing. > > Here I'm getting best results when using 100 as the limit, i.e., with > an average of 9 characters in the first 10 lines; 100 because of: > (9 chars + 1 '\n') * 10 = 100 > > This is the script I was using (openSUSE-13.2, i5-4570 CPU, 20GB RAM): > > #!/bin/sh > file='file.txt' > # Test with various line lengths. > for len in 0 1 2 5 6 7 8 9 10 11 15 20 50 100 1000; do > printf "\n====== %s ======\n" "$len" > # Prepare a 500M file with LEN characters per line. > yes "$(seq -s '' 1000 | head -c $len)" \ > | src/dd iflag=fullblock status=none bs=1M count=500 of="$file" > # Do 5 test runs with the old/new 'wc'. > for i in $(seq 3); do > # dummy run > wc -l "$file" > /dev/null > for wc in src/wc-before src/wc; do > printf "\n= %s =\n" "$wc" > time "$wc" -l "$file" > done > done > done
Thanks for this. I'm still getting 150 for the limit with this on my i3-2310M I guess either the function call overhead is slightly lower or the memchr code is slightly more efficient on your newer processor. > You are of course encouraged to play with/reduce the list for 'len' > in that script. > > Other than that, the patch looks great per se. I couldn't find a way > to make it faster than that. > > One thing: do you think that the compiler will get better with > memchr() for short lines some day? Then we should maybe add a > "FIXME: re-evaluate magic number (100 or 150) in 2017". I'll add the comment. > Finally: can we get some performance test results from some other > platforms (ARM, PPC, 32 vs 64 bit)? I mean, this patch helps on > Linux/x86_64, but if it makes things worse on other platforms, then > I don't think complicating the code is worth it. Cut-off is around 15 also on PPC (IBM,8231-E2B) cheers, Pádraig.
