On Tue, 2005-10-04 at 09:52 -0700, Doug Cutting wrote: > Rod Taylor wrote: > >>Tell me how it behaves during the sort phase. > > > > I ran 8 jobs simultaneously. Very high await time (1200) and it was > > doing about 22MB/sec data writes. Nearly 0 reads from disk (everything > > would be cached in memory). > > This is during the sort part? This first writes a big file, then reads > it, then sorts it. With 20M records I think the file is around 2.5GB, > so eight of these would be 20GB. Do you have 20GB of RAM?
I split the 20M record chunk into 8 jobs of 2.5M records each (better replicates what I had earlier -- a 10M page segment with up to 8 tasks on the machine). The boxes were using have 4GB ram and the NAS has 2GB ram. Yes, If I tried dealing with 160M records at a time I most certainly would run out of IO being as there is only a single SCSI drive. -- Rod Taylor <[EMAIL PROTECTED]>
