RE: is it a bug?

2010-03-02 Thread Voelker, Bernhard
On 3/1/2010 5:58 PM, Eric Blake wrote: According to Igor Zakharoff on 3/1/2010 9:49 AM: $ echo -e ru.unix /h\nru.unix.ftn /h\nru.unix.prog /h | sort I somehow get ru.unix.ftn /h ru.unix /h ru.unix.prog /h Is it a bug? Nope. I reproduced your results with the en_US locale, but not

Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

2010-03-02 Thread Pádraig Brady
On 02/03/10 06:32, Joey Degges wrote: 2010/3/1 Joey Deggesjdeg...@gmail.com Thanks for testing, these results are interesting. I will run some tests on various systems tonight and report back. Maybe my earlier results were caused by peculiar hardware. Results differ once again, this time

Taking advantage of L1 and L2 cache in sort

2010-03-02 Thread Pádraig Brady
Currently when sorting we take advantage of the RAM vs disk speed bump by using a large mem buffer dependent on the size of RAM. However we don't take advantage of the cache layer in the memory hierarchy which has an increasing importance in modern systems given the disparity between CPU and RAM

Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

2010-03-02 Thread Jim Meyering
Joey Degges wrote: ... Here is pastebin of the code I used to spin. Its some math in a large loop, should run for ~5 minutes (enough time to do most tests): http://codepad.org/tCtjLw7H If I want to burn some CPU time, starting a few of these one-liners does the job: perl -e '$t=1; while (1)

Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

2010-03-02 Thread Erik Auerswald
Hi, On Mon, Mar 01, 2010 at 05:33:38PM -0800, Joey Degges wrote: Were you sure to remount your devices to clear the cache before running these tests? While testing this patch early on the cache caused me many incorrect readings. Another approach I took to clear the cache was to fill up all of

Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

2010-03-02 Thread Pádraig Brady
On 02/03/10 11:42, Erik Auerswald wrote: Hi, On Mon, Mar 01, 2010 at 05:33:38PM -0800, Joey Degges wrote: Were you sure to remount your devices to clear the cache before running these tests? While testing this patch early on the cache caused me many incorrect readings. Another approach I took

RE: is it a bug?

2010-03-02 Thread Voelker, Bernhard
Eric Blake wrote: According to Voelker, Bernhard on 3/2/2010 1:34 AM: I understand that the sort order depends on the locale, i.e. LC_ALL, but this doesn't explain the differences I get on Solaris 5.10, SLES 10.1, and Cygwin (given that sort didn't change about this point in the past). The

Re: [PATCH] md5sum: handle carriage returns at the eol in input sum files

2010-03-02 Thread Eric Blake
According to Ondřej Vašík on 3/2/2010 8:03 AM: Hello, as reported in rhbz#569550 , md5sum (and all others sum utilities) fails to work with DOS md5 files. This issue was previously reported on this list as http://lists.gnu.org/archive/html/bug-coreutils/2004-11/msg00037.html ,

Re: [PATCH] Solaris ACL handling

2010-03-02 Thread Eric Blake
According to Ben Walton on 2/12/2010 6:32 AM: Hi Ben, I'm building coreutils for OpenCSW (http://opencsw.org) and was seeing the test suite fail several tests with 'Operation not supported on tranport endpoint' messages. After digging around a bit, I found that errno was EOPNOTSUPP in

Re: Taking advantage of L1 and L2 cache in sort

2010-03-02 Thread Chen Guo
This is exactly what that guy Shaun Jackman was talking about earlier. I'm actually really surprised this is faster, if I can dig up his e-mail I'll forward him this, I remember him saying something about experimenting with exactly this. Can you profile the difference in the number of I/O system

Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

2010-03-02 Thread Joey Degges
2010/3/2 Pádraig Brady p...@draigbrady.com On 02/03/10 06:32, Joey Degges wrote: 2010/3/1 Joey Deggesjdeg...@gmail.com Thanks for testing, these results are interesting. I will run some tests on various systems tonight and report back. Maybe my earlier results were caused by peculiar

Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

2010-03-02 Thread Joey Degges
2010/3/2 Pádraig Brady p...@draigbrady.com On 02/03/10 11:42, Erik Auerswald wrote: Hi, On Mon, Mar 01, 2010 at 05:33:38PM -0800, Joey Degges wrote: Were you sure to remount your devices to clear the cache before running these tests? While testing this patch early on the cache caused me

Re: Taking advantage of L1 and L2 cache in sort

2010-03-02 Thread Joey Degges
2010/3/2 Pádraig Brady p...@draigbrady.com Currently when sorting we take advantage of the RAM vs disk speed bump by using a large mem buffer dependent on the size of RAM. However we don't take advantage of the cache layer in the memory hierarchy which has an increasing importance in modern

Re: Taking advantage of L1 and L2 cache in sort

2010-03-02 Thread Chen Guo
Forgot to CC the list: I did a quick time -v, and found that sorting a 96M file, with -S500M there were 36358 page faults, and only 5380 page faults with -S10M. Wow. So system time goes up, but user time goes down. It seems odd that user time would go down, but I believe it's in the

Re: Taking advantage of L1 and L2 cache in sort

2010-03-02 Thread Shaun Jackman
Hi Chen, I haven't done any testing yet, but I was planning on doing pretty much the same sort of tests as Pádraig reported. I tested on a single-processor machine with 1 MB cache, and I see similar results (see below). The next step would be to perform the merging in parallel on a multiprocessor

Re: side-effect implementing the mv command

2010-03-02 Thread Eric Blake
According to Derick Centeno on 1/15/2010 2:58 PM: Hello Derick, Apologies for a long delay with no response. I'm weeding through my inbox, and saw this thread. Recently I executed the following: $ sudo mv ./libjavaplugin_oji.so /usr/lib/mozilla/plugins Explanation: I was within the

Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

2010-03-02 Thread Pádraig Brady
On 02/03/10 18:35, Joey Degges wrote: 2010/3/2 Pádraig Bradyp...@draigbrady.com On 02/03/10 06:32, Joey Degges wrote: 2010/3/1 Joey Deggesjdeg...@gmail.com Thanks for testing, these results are interesting. I will run some tests on various systems tonight and report back. Maybe my

Re: Taking advantage of L1 and L2 cache in sort

2010-03-02 Thread Pádraig Brady
On 02/03/10 18:20, Chen Guo wrote: This is exactly what that guy Shaun Jackman was talking about earlier. I'm actually really surprised this is faster, if I can dig up his e-mail I'll forward him this, I remember him saying something about experimenting with exactly this. I missed that thread

Re: [PATCH] Solaris ACL handling

2010-03-02 Thread Jim Meyering
Eric Blake wrote: According to Ben Walton on 2/12/2010 6:32 AM: Hi Ben, I'm building coreutils for OpenCSW (http://opencsw.org) and was seeing the test suite fail several tests with 'Operation not supported on tranport endpoint' messages. After digging around a bit, I found that errno was