IMHO the number of disk reads is simply a function of how much memory there is, how much can be used to cache the node.db4o by the operating system. We can do more caching at the JVM level, this will reduce CPU usage but not actual IOs hitting the disk, and will cost more heap memory.
But we can reduce the number of disk writes further for downloads: Turn off scheduling by retry count. Schedule solely by priority then round-robin over the different request clients and requests then randomly over the individual SendableGet's: as we do now, but without the retry count layer. Only update the retry count for an individual block if MaxRetries != -1. Keep the cooldown queue entirely in RAM. Two structures so that we can select requests quickly: 1) A structure tracking all the keys fetched in the last half hour. For each key, the 3 times at which it was last fetched. When selecting requests, we would not send any request for a key in this list - at least not for persistent requests. 2) A structure tracking all the BaseSendableGet's sent in the last half hour. For each, record the start and finish times for the last 3 times it was fetched. When selecting requests, we can shortcut having to ask the first structure for each block by excluding a whole BaseSendableGet through this structure. Memory cost? Well, we do actually keep the first structure already, in the FailureTable ... some changes would be needed perhaps. The second structure would be new, and somewhat problematic, as it would keep these objects in memory even if they are deactivated. We could introduce a unique identifier or even use db4o's object IDs to avoid this (probably the best solution). A simple hashset of the objects themselves probably would not work well for activation reasons. Network performance cost? Well, what would be the impact of not scheduling by retry count any more? I dunno, any opinions? CPU cost? We might spend more time trying to find requests to run ... this might even result in more disk reads ... ? But on the whole it ought to be positive ... Code changes? Well, we could keep SplitFileFetcherSubSegment, and be backward compatible, it'd be a bit awkward though... eventually we'd want a new class merging SFFS and SFFSS... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20090506/b8160441/attachment.pgp>
