Rod Taylor wrote:
Tell me how it behaves during the sort phase.
I ran 8 jobs simultaneously. Very high await time (1200) and it was
doing about 22MB/sec data writes. Nearly 0 reads from disk (everything
would be cached in memory).
This is during the sort part? This first writes a big file, then reads
it, then sorts it. With 20M records I think the file is around 2.5GB,
so eight of these would be 20GB. Do you have 20GB of RAM?
Doug
-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
