2010/3/2 Pádraig Brady <[email protected]> > On 02/03/10 06:32, Joey Degges wrote: > >> 2010/3/1 Joey Degges<[email protected]> >> >> Thanks for testing, these results are interesting. I will run some tests >>> on >>> various systems tonight and report back. Maybe my earlier results were >>> caused by peculiar hardware. >>> >>> >>> Results differ once again, this time I've tested on a different system. >> >> Here is pastebin of the code I used to spin. Its some math in a large >> loop, >> should run for ~5 minutes (enough time to do most tests): >> http://codepad.org/tCtjLw7H >> >> CPU: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz >> RAM: 2G >> kernel: 2.6.31-19-generic #56-Ubuntu SMP x86_64 >> input file: 97M, gensort -a 1000000 file >> cmd: umount /mnt&& mount /dev/foo /mnt&& /usr/bin/time -v sort< file> >> /dev/null >> >> mechanical disk @ 80MB/s, ext3 - 4x spinning processes >> mode : real, %CPU, voluntary/involuntary context switches >> NORMAL : 14.73s, 10%, 408/34 >> SEQUENTIAL: 10.95s, 13%, 220/52 >> WILLNEED : 05.22s, 35%, 026/57 >> >> flash key @ 20MB/s, ext4 - 4x spinning processes >> NORMAL : 88.49s, 01%, 1521/42 >> SEQUENTIAL: 69.76s, 02%, 1131/57 >> WILLNEED : 50.29s, 03%, 0174/31 >> >> flash key @ 11MB/s, FAT32 - 4x spinning processes >> NORMAL : 106.5s, 01%, 1334/32 >> SEQUENTIAL: 85.63s, 01%, 0765/53 >> WILLNEED : 68.70s, 02%, 0193/59 >> >> mechanical disk @ 80MB/s, ext3 >> NORMAL : 02.70s, 64%, 748/9 >> SEQUENTIAL: 02.71s, 61%, 374/8 >> WILLNEED : 02.79s, 57%, 026/5 >> >> flash key @ 20MB/s, ext4 >> NORMAL : 06.25s, 25%, 1505/8 >> SEQUENTIAL: 06.18s, 26%, 1136/11 >> WILLNEED : 06.10s, 26%, 0156/8 >> >> flash key @ 11MB/s, FAT32 >> NORMAL : 10.23s, 15%, 1320/7 >> SEQUENTIAL: 10.12s, 16%, 0762/7 >> WILLNEED : 10.14s, 16%, 0193/4 >> >> >> Note the large difference in the number of voluntary context switches >> between the different modes. This number appears to only be dependent on >> the >> device being read from and the advice mode, not the system load. When the >> system is heavily loaded the process that context switches the least will >> be >> able to consume more resources. From these results you can see that >> SEQUENTIAL reduces the voluntary context switching from 25-46% and >> WILLNEED >> reduces the voluntary context switching from 85-96%. This also explains >> why >> you won't notice the improvements due to SEQUENTIAL or WILLNEED very much >> in >> unburdened systems. >> > > Thanks for the thorough testing. > That confirms the context switch is done when the read() > needs data to be retrieved from disk, and that it has a > large affect on your sort on multicore, whereas it has > no affect on my single core. > > I'm still unsure as to whether we should enable WILLNEED > as I've got a counter point in another email I'll send in a while. > Also I'm worried in the low RAM case that WILLNEED will thrash > other caches, perhaps useful to sort itself if it's processing > multiple files or running in merge mode. I'll do some more > testing with a view to enabling WILLNEED, but in any case > SEQUENTIAL should also be enabled so that readahead is done > on any portion of the file that it's not possible to > prepopulate the cache with. > > Agreed, you have convinced me :]
In the future when/if the WILLNEED implementation becomes asynchronous this might be worth investigating once again. Thanks, Joey
