On Tue, 2005-12-13 at 21:43 +0100, Andrzej Bialecki wrote: > > Most of the time we deal with very large files, with sequential > access. > Only in few places we deal with a lot of small files (e.g. indexing). > So, I think the best would be an FS optimized for efficient > sequential > write/read of large files.
But beware what happens if you run more than one task per machine. Each individual task might be sequential but several in parallel will generate plenty of disk head movement that approximates parallel IO -- especially on a filesystem that uses small blocks and a driver with poor read-ahead support. -- Rod Taylor <[EMAIL PROTECTED]> ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
