On Wed, 12 Feb 2003, [ISO-8859-1] Hans-J$B|(Brgen Sch$Bv(Bnig wrote:
(B> Be careful with sort_mem - this might lead to VERY unexpected results. I
(B> did some testing on my good old Athlon 500 with a brand new IBM 120 Gigs
(B> HDD. Reducing the sort_mem gave me significantly faster results when
(B> sorting/indexing 20.000.000 randon rows.
(BActually, the results are completely expected once you know what's
(Bexactly is going on. I found it weird that my sorts were also slowing
(Bdown with more sort memory until Tom or Bruce or someone pointed out to
(Bme that my stats said my sorts were swapping.
(BIf I'm understanding this correctly, this basically meant that my sort
(Bresults would start hitting disk becuase they were being paged out to
(Bswap space, but then once the block was sorted, it would be read in
(Bagain from disk, and then written out to disk again (in a different
(Bplace), creating a lot more I/O than was really necessary.
(BThis strikes me, too, as another area where mmap might allow the system
(Bto do a better job with less tuning. Basically, the sort is getting
(Bsplit into a bunch of smaller chunks, each of which is individually
(Bsorted, and then you merge at the end, right? So if all those individual
(Bchunks were mmaped, the system could deal with paging them out if and
(Bwhen necessary, and for the sorts you do before the merge, you could
(Bmlock() the area that you're currently sorting to make sure that it
(BIf the VM system accepts hints, you might also get some further
(Boptimizations because you can tell it (using madvise()) when you're
(Bdoing random versus sequential access on a chunk of memory.
(BCurt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org
(B Don't you know, in this new Dark Age, we're all light. --XTC
(B---------------------------(end of broadcast)---------------------------
(BTIP 4: Don't 'kill -9' the postmaster