[freenet-support] Re: Stable build 5087
Klaus Brüssel wrote: Am Dienstag, 27. Juli 2004 23:26 schrieb Toad: Stable build 5087 is now available. The snapshots have been updated. Please upgrade. You can do this on Windows by using the update option on Well...upgraded with update.sh on my linux box anf then: Same here, just after upgrade, and I can't connect to the main interface (but there are a bunch of java processes running). pause for commercials Solved it changing the -Xmx128m to -Xmx256m. I suppose this can be a general problem, I'm running stable without any tweaks since a month or so. In the good side, two minutes running and I can see all the activelinks, where the previous build failed after many hours :) - [EMAIL PROTECTED]:~/freenet ./start-freenet.sh Detected freenet-ext.jar Detected freenet.jar Sun java detected. Starting Freenet now: Command line: java -Xmx128m -XX:MaxDirectMemorySize=128m freenet.node.Main Done [EMAIL PROTECTED]:~/freenet INFO: Native CPUID library jcpuid loaded from resource INFO: Optimized native BigInteger library 'libjbigi-linux-athlon.so' loaded from resource Caught java.lang.OutOfMemoryError running or seeding node java.lang.OutOfMemoryError Caught, in Main: java.lang.OutOfMemoryError java.lang.OutOfMemoryError - [EMAIL PROTECTED]:~/freenet cat freenet.log 28.07.2004 01:29:23 (freenet.node.rt.NGRoutingTable, main, ERROR): Caught java.io.IOException: Value out of range: 450.0 deserializing a NodeEstimator for DataObjectRoutingMemory:tcp/220.147.157.155:32367, sessions=1, presentations=3, ID=DSA(6bdd 26bd 0a5f f693 8fe4 9809 0c53 24f9 af1f cc36), version=Fred,0.5,STABLE-1.50,5084:6bdd26bd0a5ff6938fe498090c5324f9af1fcc36 java.io.IOException: Value out of range: 450.0 at freenet.node.rt.BootstrappingDecayingRunningAverage.init(BootstrappingDecayingRunningAverage.java:163) at freenet.node.rt.BootstrappingDecayingRunningAverageFactory.create(BootstrappingDecayingRunningAverageFactory.java:36) at freenet.node.rt.SlidingBucketsKeyspaceEstimator.init(SlidingBucketsKeyspaceEstimator.java:158) at freenet.node.rt.OptimizingSlidingBucketsKeyspaceEstimator.init(OptimizingSlidingBucketsKeyspaceEstimator.java:27) at freenet.node.rt.OptimizingSlidingBucketsKeyspaceEstimatorFactory.createTime(OptimizingSlidingBucketsKeyspaceEstimatorFactory.java:82) at freenet.node.rt.StandardNodeEstimator.init(StandardNodeEstimator.java:563) at freenet.node.rt.StandardNodeEstimatorFactory.create(StandardNodeEstimatorFactory.java:95) at freenet.node.rt.NGRoutingTable.loadEstimators(NGRoutingTable.java:370) at freenet.node.rt.NGRoutingTable.init(NGRoutingTable.java:187) at freenet.node.Main.main(Main.java:858) 28.07.2004 01:31:12 (freenet.node.Main, main, ERROR): Caught java.lang.OutOfMemoryError running or seeding node java.lang.OutOfMemoryError - linux kernel 2.6.7 sun jvm Java HotSpot(TM) Client VM (build 1.4.2_04-b05, mixed mode) questions: 1) builds 5085 and 5086 often crashed with signal 11, can this corrupt the routing table files (those *_a *_b) ? 2) is something wrong with the seednodes.ref ? 3) startup times are much too long lately - only conducted by the big datastore ? How much memory (RAM) is recommended if used with 20+ GB datastore ? P.S. I deleted those *_a and *_b files and replaced seednodes.ref with a version that is much smaller (2 nodes) and my node starts up ! ciao --klaus ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED] ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] Re: Stable build 5087
Okay, I think I know what's causing the increased memory usage. Will be fixed in 5088. On Wed, Jul 28, 2004 at 12:06:54PM +0200, Jano wrote: Klaus Br?ssel wrote: Am Dienstag, 27. Juli 2004 23:26 schrieb Toad: Stable build 5087 is now available. The snapshots have been updated. Please upgrade. You can do this on Windows by using the update option on Well...upgraded with update.sh on my linux box anf then: Same here, just after upgrade, and I can't connect to the main interface (but there are a bunch of java processes running). pause for commercials Solved it changing the -Xmx128m to -Xmx256m. I suppose this can be a general problem, I'm running stable without any tweaks since a month or so. In the good side, two minutes running and I can see all the activelinks, where the previous build failed after many hours :) - [EMAIL PROTECTED]:~/freenet ./start-freenet.sh Detected freenet-ext.jar Detected freenet.jar Sun java detected. Starting Freenet now: Command line: java -Xmx128m -XX:MaxDirectMemorySize=128m freenet.node.Main Done [EMAIL PROTECTED]:~/freenet INFO: Native CPUID library jcpuid loaded from resource INFO: Optimized native BigInteger library 'libjbigi-linux-athlon.so' loaded from resource Caught java.lang.OutOfMemoryError running or seeding node java.lang.OutOfMemoryError Caught, in Main: java.lang.OutOfMemoryError java.lang.OutOfMemoryError - [EMAIL PROTECTED]:~/freenet cat freenet.log 28.07.2004 01:29:23 (freenet.node.rt.NGRoutingTable, main, ERROR): Caught java.io.IOException: Value out of range: 450.0 deserializing a NodeEstimator for DataObjectRoutingMemory:tcp/220.147.157.155:32367, sessions=1, presentations=3, ID=DSA(6bdd 26bd 0a5f f693 8fe4 9809 0c53 24f9 af1f cc36), version=Fred,0.5,STABLE-1.50,5084:6bdd26bd0a5ff6938fe498090c5324f9af1fcc36 java.io.IOException: Value out of range: 450.0 at freenet.node.rt.BootstrappingDecayingRunningAverage.init(BootstrappingDecayingRunningAverage.java:163) at freenet.node.rt.BootstrappingDecayingRunningAverageFactory.create(BootstrappingDecayingRunningAverageFactory.java:36) at freenet.node.rt.SlidingBucketsKeyspaceEstimator.init(SlidingBucketsKeyspaceEstimator.java:158) at freenet.node.rt.OptimizingSlidingBucketsKeyspaceEstimator.init(OptimizingSlidingBucketsKeyspaceEstimator.java:27) at freenet.node.rt.OptimizingSlidingBucketsKeyspaceEstimatorFactory.createTime(OptimizingSlidingBucketsKeyspaceEstimatorFactory.java:82) at freenet.node.rt.StandardNodeEstimator.init(StandardNodeEstimator.java:563) at freenet.node.rt.StandardNodeEstimatorFactory.create(StandardNodeEstimatorFactory.java:95) at freenet.node.rt.NGRoutingTable.loadEstimators(NGRoutingTable.java:370) at freenet.node.rt.NGRoutingTable.init(NGRoutingTable.java:187) at freenet.node.Main.main(Main.java:858) 28.07.2004 01:31:12 (freenet.node.Main, main, ERROR): Caught java.lang.OutOfMemoryError running or seeding node java.lang.OutOfMemoryError - linux kernel 2.6.7 sun jvm Java HotSpot(TM) Client VM (build 1.4.2_04-b05, mixed mode) questions: 1) builds 5085 and 5086 often crashed with signal 11, can this corrupt the routing table files (those *_a *_b) ? 2) is something wrong with the seednodes.ref ? 3) startup times are much too long lately - only conducted by the big datastore ? How much memory (RAM) is recommended if used with 20+ GB datastore ? P.S. I deleted those *_a and *_b files and replaced seednodes.ref with a version that is much smaller (2 nodes) and my node starts up ! ciao --klaus ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED] ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED] -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] Re: Stable build 5087
On Wed, 2004-07-28 at 06:06, Jano wrote: Solved it changing the -Xmx128m to -Xmx256m. I suppose this can be a general problem, I'm running stable without any tweaks since a month or so. In the good side, two minutes running and I can see all the activelinks, where the previous build failed after many hours :) I upgraded promptly to 5085, 5086, and 5087. All used more CPU time, but eventually would finish. Stackdump showed lots of biginteger stuff while compute-bound. Inserts were better than 5084, but retrieval much worse. RNF occurred in around 10 to 50% of all requests. (Each request was for a unique key). Requesting peers-mode ocm seemed to trigger a lot of CPU usage. Switching back to 5084, I get much better performance on requests, with very few RNF's. Even fewer than I used to get, probably due to load factors. It appears that many stable nodes are switching back to 5084, since I seem to recall that a day or two ago there were relatively few 5084 nodes. (I didn't save the histogram then). # Histogram of node versions in fred's Routing table # Jul 28, 2004 8:25:46 AM # nodes: 1556 Fred,0.5,STABLE-1.49,5063 1 Fred,0.5,STABLE-1.50,5070 1 Fred,0.5,STABLE-1.50,5083 1 Fred,0.5,STABLE-1.50,5084 692 Fred,0.5,STABLE-1.50,5085 164 Fred,0.5,STABLE-1.50,5086 615 Fred,0.5,STABLE-1.50,5087 81 unknown 1 signature.asc Description: This is a digitally signed message part ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] Re: Stable build 5087
Obviously my efforts are worthless. I should go get a job stacking shelves. On Wed, Jul 28, 2004 at 08:32:12AM -0400, Edward J. Huff wrote: On Wed, 2004-07-28 at 06:06, Jano wrote: Solved it changing the -Xmx128m to -Xmx256m. I suppose this can be a general problem, I'm running stable without any tweaks since a month or so. In the good side, two minutes running and I can see all the activelinks, where the previous build failed after many hours :) I upgraded promptly to 5085, 5086, and 5087. All used more CPU time, but eventually would finish. Stackdump showed lots of biginteger stuff while compute-bound. Inserts were better than 5084, but retrieval much worse. RNF occurred in around 10 to 50% of all requests. (Each request was for a unique key). Requesting peers-mode ocm seemed to trigger a lot of CPU usage. Switching back to 5084, I get much better performance on requests, with very few RNF's. Even fewer than I used to get, probably due to load factors. It appears that many stable nodes are switching back to 5084, since I seem to recall that a day or two ago there were relatively few 5084 nodes. (I didn't save the histogram then). # Histogram of node versions in fred's Routing table # Jul 28, 2004 8:25:46 AM # nodes: 1556 Fred,0.5,STABLE-1.49,5063 1 Fred,0.5,STABLE-1.50,5070 1 Fred,0.5,STABLE-1.50,5083 1 Fred,0.5,STABLE-1.50,5084 692 Fred,0.5,STABLE-1.50,5085 164 Fred,0.5,STABLE-1.50,5086 615 Fred,0.5,STABLE-1.50,5087 81 unknown 1 ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED] -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] Re: Stable build 5087
On Wed, 2004-07-28 at 08:42, Toad wrote: Obviously my efforts are worthless. I should go get a job stacking shelves. Not at all. I'm just pointing out the facts. It's a lot harder to retrieve data from 5087 than from 5084, and the number of nodes running 5084 seems to be increasing. Just now the number of 5087 also increased, or my node connected to more of them. Later the numbers decreased, because some nodes were dropped from the RT. 5085 and later have much more accurate upstream bandwidth limiting. Also freenet.client.cli.Main seems to work better in 5085 and later. Upon falling back, my local success ratio went from 0.3% to 10%. This is requesting splitfile blocks in random order from a large list of blocks (which have been reported inserted or successfully retrieved by others) with random htl ranging 0 to 20, with no immediate retries. My 75 GiB datastore is not yet full... There is something going on which causes lots of quick RNF's. See attached logs of my request script (which uses freenet.client.cli.Main to issue requests). The 5087 performance data came before the 5084 data. Build 5087 was getting large numbers of RNF's, even with only one request pending at a time. 5084 got a bunch of them just after restart, when there were only a few connections open, but after the node got connected, there have been almost none, and the number of pending requests maxed out at 40. localRequestSuccessRatio: hour1090994400 12623 0.002377179080824089 hour1090998000 10066 0.005964214711729622 hour1091001600 11831 8.453085376162299E-4 hour1091005200 11563 0.0025951557093425604 fall back to 5084 -- fewer requests, more successes. hour1091008800 471 20 0.04246284501061571 hour1091012400 579 17 0.02936096718480138 hour1091016000 480 25 0.052086 hour1091019600 784 79 0.10076530612244898 requestSuccessRatio: hour1090998000 194420 0.0102880658436214 hour1091001600 24198 0.0033071517155849523 hour1091005200 239416 0.006683375104427736 hour1091008800 897 38 0.042363433667781496 hour1091012400 832 34 0.040865384615384616 hour1091016000 772 29 0.03756476683937824 hour1091019600 1786114 0.06382978723404255 hour1091023200 1684210 0.12470308788598575 # Histogram of node versions in fred's Routing table # Jul 28, 2004 9:55:04 AM # nodes: 1599 Fred,0.5,STABLE-1.49,5063 1 Fred,0.5,STABLE-1.50,5070 1 Fred,0.5,STABLE-1.50,5083 1 Fred,0.5,STABLE-1.50,5084 708 Fred,0.5,STABLE-1.50,5085 165 Fred,0.5,STABLE-1.50,5086 607 Fred,0.5,STABLE-1.50,5087 115 unknown 1 # Histogram of node versions in fred's Routing table # Jul 28, 2004 10:44:39 AM # nodes: 1537 Fred,0.5,STABLE-1.50,5083 2 Fred,0.5,STABLE-1.50,5084 686 Fred,0.5,STABLE-1.50,5085 156 Fred,0.5,STABLE-1.50,5086 576 Fred,0.5,STABLE-1.50,5087 117 All inbound requests ever received, from 201 unique hosts: Receive Accept Acc/Rcv Succeed Suc/Acc Host AddressVersion Data by host version 119911991.000 43 0.036 5087 117311731.000 18 0.015 5084 472 472 1.000 16 0.034 5086 119 119 1.000 1 0.008 5085 Connections open (Inbound/Outbound/Limit) 164 (111/53/800) Transfers active (Transmitting/Receiving) 89 (45/44) Amount of data transmitted/received over currently open connections 162 MiB/118 MiB Total amount of data transmitted/received 231 MiB/149 MiB Uptime 5 hours 43 minutes Current upstream bandwidth usage 13865 bytes/second (138.6%) On Wed, Jul 28, 2004 at 08:32:12AM -0400, Edward J. Huff wrote: On Wed, 2004-07-28 at 06:06, Jano wrote: Solved it changing the -Xmx128m to -Xmx256m. I suppose this can be a general problem, I'm running stable without any tweaks since a month or so. In the good side, two minutes running and I can see all the activelinks, where the previous build failed after many hours :) I upgraded promptly to 5085, 5086, and 5087. All used more CPU time, but eventually would finish. Stackdump showed lots of biginteger stuff while compute-bound. Inserts were better than 5084, but retrieval much worse. RNF occurred in around 10 to 50% of all requests. (Each request was for a unique key). Requesting peers-mode ocm seemed to trigger a lot of CPU usage. Switching back to 5084, I get much better performance on requests, with very few RNF's. Even fewer than I used to get, probably due to load factors. It appears that many stable nodes are switching back to 5084, since I seem to recall that a day or two ago there were relatively few 5084 nodes. (I didn't save the histogram then). # Histogram of node versions in fred's
Re: [freenet-support] Re: Stable build 5087
Maybe you are right. I did see some suspicious RNFs earlier... On Wed, Jul 28, 2004 at 11:15:40AM -0400, Edward J. Huff wrote: On Wed, 2004-07-28 at 08:42, Toad wrote: Obviously my efforts are worthless. I should go get a job stacking shelves. Not at all. I'm just pointing out the facts. It's a lot harder to retrieve data from 5087 than from 5084, and the number of nodes running 5084 seems to be increasing. Just now the number of 5087 also increased, or my node connected to more of them. Later the numbers decreased, because some nodes were dropped from the RT. 5085 and later have much more accurate upstream bandwidth limiting. Also freenet.client.cli.Main seems to work better in 5085 and later. Upon falling back, my local success ratio went from 0.3% to 10%. This is requesting splitfile blocks in random order from a large list of blocks (which have been reported inserted or successfully retrieved by others) with random htl ranging 0 to 20, with no immediate retries. My 75 GiB datastore is not yet full... There is something going on which causes lots of quick RNF's. See attached logs of my request script (which uses freenet.client.cli.Main to issue requests). The 5087 performance data came before the 5084 data. Build 5087 was getting large numbers of RNF's, even with only one request pending at a time. 5084 got a bunch of them just after restart, when there were only a few connections open, but after the node got connected, there have been almost none, and the number of pending requests maxed out at 40. localRequestSuccessRatio: hour 1090994400 12623 0.002377179080824089 hour 1090998000 10066 0.005964214711729622 hour 1091001600 11831 8.453085376162299E-4 hour 1091005200 11563 0.0025951557093425604 fall back to 5084 -- fewer requests, more successes. hour 1091008800 471 20 0.04246284501061571 hour 1091012400 579 17 0.02936096718480138 hour 1091016000 480 25 0.052086 hour 1091019600 784 79 0.10076530612244898 requestSuccessRatio: hour 1090998000 194420 0.0102880658436214 hour 1091001600 24198 0.0033071517155849523 hour 1091005200 239416 0.006683375104427736 hour 1091008800 897 38 0.042363433667781496 hour 1091012400 832 34 0.040865384615384616 hour 1091016000 772 29 0.03756476683937824 hour 1091019600 1786114 0.06382978723404255 hour 1091023200 1684210 0.12470308788598575 # Histogram of node versions in fred's Routing table # Jul 28, 2004 9:55:04 AM # nodes: 1599 Fred,0.5,STABLE-1.49,5063 1 Fred,0.5,STABLE-1.50,5070 1 Fred,0.5,STABLE-1.50,5083 1 Fred,0.5,STABLE-1.50,5084 708 Fred,0.5,STABLE-1.50,5085 165 Fred,0.5,STABLE-1.50,5086 607 Fred,0.5,STABLE-1.50,5087 115 unknown 1 # Histogram of node versions in fred's Routing table # Jul 28, 2004 10:44:39 AM # nodes: 1537 Fred,0.5,STABLE-1.50,5083 2 Fred,0.5,STABLE-1.50,5084 686 Fred,0.5,STABLE-1.50,5085 156 Fred,0.5,STABLE-1.50,5086 576 Fred,0.5,STABLE-1.50,5087 117 All inbound requests ever received, from 201 unique hosts: Receive Accept Acc/Rcv Succeed Suc/Acc Host AddressVersion Data by host version 1199 11991.000 43 0.036 5087 1173 11731.000 18 0.015 5084 472 472 1.000 16 0.034 5086 119 119 1.000 1 0.008 5085 Connections open (Inbound/Outbound/Limit) 164 (111/53/800) Transfers active (Transmitting/Receiving) 89 (45/44) Amount of data transmitted/received over currently open connections 162 MiB/118 MiB Total amount of data transmitted/received 231 MiB/149 MiB Uptime 5 hours 43 minutes Current upstream bandwidth usage 13865 bytes/second (138.6%) On Wed, Jul 28, 2004 at 08:32:12AM -0400, Edward J. Huff wrote: On Wed, 2004-07-28 at 06:06, Jano wrote: Solved it changing the -Xmx128m to -Xmx256m. I suppose this can be a general problem, I'm running stable without any tweaks since a month or so. In the good side, two minutes running and I can see all the activelinks, where the previous build failed after many hours :) I upgraded promptly to 5085, 5086, and 5087. All used more CPU time, but eventually would finish. Stackdump showed lots of biginteger stuff while compute-bound. Inserts were better than 5084, but retrieval much worse. RNF occurred in around 10 to 50% of all requests. (Each request was for a unique key). Requesting peers-mode ocm seemed to trigger a lot of CPU usage. Switching back to 5084, I get much better performance on requests, with very few RNF's. Even fewer than I used to get, probably due to load factors. It appears that many stable nodes are switching
[freenet-support] Re: Stable build 5087
Klaus Brüssel schrieb: P.S. I deleted those *_a and *_b files and replaced seednodes.ref with a version that is much smaller (2 nodes) and my node starts up ! ciao --klaus Reading the big seednode files in takes a whole lot of memory. Using an 8MB seednode.ref I had an initial memory usage of more the 200MB when fred starts, which dropped to around 70MB after some minutes. As the current ref has 23MB I suspect the initial memory usage to be gigantic. ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
