Probably you will be able to, but disk access will always take a big chunk of the processing time, no matter whether you use NIO or standard java streams (which, in fact, delegate to NIO anyway). Disk operations are just quite time consuming by nature -- you'd need a distributed file system to save on drive's physical head seeks to avoid it.
Not even remotely a hardware specialist, Dawid
Byron Miller wrote:
Thanks Dawid,
I joined his list and have been brainstorming with him, so i'll see what he says.
I plan on converting some of the calls to NIO in nutch, just wondered if anyone hadn't done it for a specific reason or what.
With 36million pages, analyze DB took 192 minutes total time, 118 minutes "user time" and only 37 minutes "cpu time"
Going to see if i can beat that.
-byron
--- Dawid Weiss <[EMAIL PROTECTED]> wrote:
Leo Galambos of the Egothor project has done it I
believe -- if you ask him directly I'm sure he'll provide you with the
difference it made for Egothor. Nutch and Egothor are obviously two
different things, but the results might give you a hint about the general
performance gain from converting to NIO.
Cheers, Dawid
Byron Miller wrote:
Has anyone done any testing of converting
read/writes
& io to NIO or do you think that wouldn't offer
much gain?
-------------------------------------------------------
This SF.Net email is sponsored by The 2004
JavaOne(SM) Conference
Learn from the experts at JavaOne(SM), Sun's
Worldwide Java Developer
Conference, June 28 - July 1 at the Moscone Center
in San Francisco, CA
REGISTER AND SAVE! http://java.sun.com/javaone/sf
Priority Code NWMGYKND
_______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
-------------------------------------------------------
This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
------------------------------------------------------- This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
------------------------------------------------------- This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
