On Mon, 2005-10-03 at 15:33 -0700, Doug Cutting wrote: > Rod Taylor wrote: > > Virtually no IO reported at all. Averages about 200kB/sec read and > > writes are usually 0, but burst to 120MB/sec for under 1 second once > > every 30 seconds or so. > > That's strange. I wonder what it's doing. Can you use 'kill -QUIT' to > get a thread dump? Try a few of these to sample the stack and see where > it seems to be spending time.
The stacktrace is running FileOutputStream.writeBytes in the LocalFileSystem.write function, so it is definitely IO bound. I had the admins change a number of settings on a couple of machines (ext3 fs to journal=writeback, noatime, etc.) and those machines have completed their work in a much more reasonable timeframe. I guess we can chalk it up to a lot of head movement I guess bumping access times and things for all of the .out files had a large impact. Thanks for your patience. Is there a "best practises" page in the Wiki that I could contribute this insight to? -- Rod Taylor <[EMAIL PROTECTED]>
