This problem has been widely reported in the past, the developers HOPE that nextgen routing will fix this. Right now your options are 1) put up with it, 2) run an experimental version, 3) shut your node down or set it to transient (if you do go transient, you'll have to wipe out your node file to change your node's identity because the network now knows about your node). The problem seems to be caused too many freenet.node.states.data.DataStateInitiator threads. A suggestion to the developers; if there are too many freenet.node.states.data.DataStateInitiator running, just kill some, starting with the threads that have been idle the longest. At least this way the node will be able to fullfill some requests instead of doing nothing for long periods.
----- Original Message ----- From: "Gordan" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Monday, September 01, 2003 8:38 AM Subject: [freenet-dev] Threads / Performance / File System Start Time > I have noticed something that seems rather unusual. > > The default setting for threads and connections seem to be 128 and 512 > respectively (or thereabout). At those settings, my node is constantly > running a load of 110%+ (on number of threads), and serving about 5-10 times > the average number of requests (e.g. 40,000-80,000 against the network > average of about 8,000). At that rate, the throttled bandwidth doesn't appear > to be used up (set for 40,000 bytes/sec). The consumption is barely around > 2/3 of that (peak), and it frequently drops through the floor, presumably > because my node is under such a heavy load and is rejecting connections. > > OK, I thought, if it is running out of threads, I'll increase the number of > threads. So, I doubled it to 256. The performance degraded horribly (only the > threads setting was changed). It was still overloaded (110%+), but it was > managing to serve fewer requests and the bandwidth use reduced to less than > half of what it was with 128 threads. > > Is there an explanation for such behaviour? It is almost as if the throttling > code starts to be the bottleneck when having to balance so many transfers > simultaneously. > > I am running 6163 (latest unstable with NG routing), and the machine is most > definitely not underpowered (Freenet is not using 100% of CPU time across all > CPUs at any one time, running at nice -n 15). > > Before I try it (it will require re-starting the node and that takes forever), > would it help if I just set the throttle to 0 (i.e. disable it) and use other > external means of shaping traffic? > > Another issue I wanted to bring up is the file system startup time. If my > understanding is correct from browsing through the documentation and some of > the code, the entire file system "map" is kept in memory, and has to be > pre-loaded before the node starts. When the data store reaches several tens > of GB in size and hundreds of thousands of keys, it takes quite a long time > to start up the file systems. Would it not be better to make the file system > startup asynchronous? > > By this, I mean get the networking up first and start accepting connections. > Then either forget about keeping a list of keys in memory all together and > just take the performance hit from a file system look-up for the key > (possibly a better option for very large data stores), or add the key to the > map, if it exists, when the key is requested. Meanwhile, the map can be > constructed in the background. > > Also, I've seen a post recently about possibly working on ways to use pthreads > in a reasonably stable way. Is this any closer to fruition? The performance > gain is considerable before the node just stops responding all together after > a few hours. > > Gordan > _______________________________________________ > Devl mailing list > [EMAIL PROTECTED] > http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl > > __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com _______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
