I also didn't want to change to many settings. I've always been going on the assumption that freenet is just trying to work my computers to hard. I setup a test machine with a default config with a 2gig datastore and let it fill. After it filled I only changed 2 settings. Maximumconnections = 256 and maxthreads=360. The results are at the bottom. Can you tell when I changed over?? =) So people can get requests into me now but everyone is still rejecting between 40-80% of requests I make.
Here's what I think is happening which is causing the constant node overloads. All nodes default to 512 connections regardless of the amount of bandwidth/CPU available. High powered machines can handle the 512 connections but all nodes default to 120 threads. Slow speed nodes are opening up way to many transfers killing their bandwidth which makes the transfers take forever to finish. This ties up tons of threads(120 doesn't go very far it seems) on other nodes causing them to excessivly QR(more threads gone). This is happening across freenet causing even more resource use on slow machines/connections and the inability of highspeed machines/connections to process the 100's of transfers/10's of 1000's of queries they should be able to. My test node is currently receiving from 100 sources. That number slowly goes up and never goes down because most of those sources are taking forever to transmit the data to me on their over utilized connections. I've got tons of bandwidth(100Mbps to the net) available that is being wasted by connections that will takes hours to finish causing my node to overload. The above in one line: Over utilized connections complicated by bad thread/connection configurations is causing serious problems. Something like how WinMX handles this might fix this problem. You select your speed from a drop down box when you install. If you select 56k modem you can connect to x amount of nodes. If you select OC-192 we'll assume your lieing and set you to something higher but not insane. Processor Power also needs to somehow be taken into consideration since a Pentium 133 connected on a OC-192 set for 512 threads will probably spontanously combust. Its late and I hope this doesn't sound like mindless babble. I've only been using freenet for about 6 months and this is what I believe is currently causing the most trouble. localquerytraffic 11/3/03 1:00:00 AM EST 58777 14753 0.2509995406366436 (25%) 11/3/03 2:00:00 AM EST 45497 11421 0.25102754027738094 (25%) 11/3/03 3:00:00 AM EST 45248 9163 0.20250618811881188 (20%) 11/3/03 4:00:00 AM EST 40810 10125 0.24810095564812545 (24%) 11/3/03 5:00:00 AM EST 56339 12458 0.22112568558192372 (16%) 11/3/03 6:00:00 AM EST 42341 6826 0.1612148980893224 (16%) 11/3/03 7:00:00 AM EST 28979 8669 0.29914765864936677 (29%) 11/3/03 8:00:00 AM EST 26677 7590 0.284514750534168 (28%) 11/3/03 9:00:00 AM EST 26624 3639 0.13668118990384615 (13%) 11/3/03 10:00:00 AM EST 19028 4257 0.22372293462266135 (22%) 11/3/03 11:00:00 AM EST 15450 3692 0.2389644012944984 (23%) 11/3/03 12:00:00 PM EST 16044 3455 0.21534530042383446 (21%) 11/3/03 1:00:00 PM EST 35039 8963 0.2558006792431291 (25%) 11/3/03 2:00:00 PM EST 35136 7973 0.22691826047358835 (22%) 11/3/03 3:00:00 PM EST 8505 6949 0.8170487948265726 (81%) 11/3/03 4:00:00 PM EST 8609 8286 0.9624811244046928 (96%) 11/3/03 5:00:00 PM EST 6151 5788 0.9409852056576167 (94%) 11/3/03 6:00:00 PM EST 21072 18057 0.8569191343963554 (85%) 11/3/03 7:00:00 PM EST 16912 12303 0.7274716177861873 (72%) 11/3/03 8:00:00 PM EST 18174 14568 0.8015846814130075 (80%) 11/3/03 9:00:00 PM EST 11693 9541 0.8159582656290089 (81%) 11/3/03 10:00:00 PM EST 13807 9329 0.6756717607010937 (67%) 11/3/03 11:00:00 PM EST 20022 12525 0.6255618819298772 (62%) 11/4/03 12:00:00 AM EST 16514 11922 0.7219329054135885 (72%) 11/4/03 1:00:00 AM EST 17500 10861 0.6206285714285714 (62%) Dave ----- Original Message ----- From: "Ken Corson" <[EMAIL PROTECTED]> To: "Discussion of development issues" <[EMAIL PROTECTED]> Sent: Tuesday, November 04, 2003 12:54 AM Subject: Re: [freenet-dev] Interesting statistics and anecdotes > Daves Lists wrote: > > I just replied to the Usability improvement ideas thread on this before I > > noticed this thread. All 4 of my nodes reject for the same reason. They > > don't like the default config. I had to reduce the max connections down to > > 256 and raise the threads to around 300. Thats the only way I can stop it > > rejecting everything. I need about 50 more threads than connections or the > > node starts getting behind. I haven't tried to many different values yet but > > 256 connections is working pretty well with only about 20% rejections. It > > was around 80% before. > > > > Dave > > I hate to admit this, but I have deviated from using all the default > values (which was my intention). One thing I did was to increase > the store to 1G. The other was setting max connections=300 some > number of days ago (before I started all this complaining). I see > bad rejection rates (localQueryTraffic) with max connections = 300 : > > 11/3/03 9:00:00 PM EST 25757 1933 0.07504755988663275 (93%) > 11/3/03 10:00:00 PM EST 11729 1244 0.10606189786000511 (90%) > 11/3/03 11:00:00 PM EST 26867 1216 0.04525998436743961 (96%) > 11/4/03 12:00:00 AM EST 29795 3752 0.12592716898808526 (88%) > > I have not touched any thread settings like factory choice or count. > I don't want to vary a bunch of settings, until I completely > understand the ramifications. > > > _______________________________________________ > Devl mailing list > [EMAIL PROTECTED] > http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl > _______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
