[freenet-support] Re: 5029 overloads
Toad wrote: Reading between the lines... check your log file, have you had many OutOfMemoryErrors? While gathering the data to answer your question, I found something strange. Get this: * /servlet/nodeinfo/performance/general reports an uptime of 1 day 16 hours 24 minutes * My log file started at 12:30pm, Oct 30th * It is currently 11:30pm, Oct 31st * That's a time span of only one day and 11 hours! hmmm * Also, I notice, strangely, that upstream bw is now reaching as high as 130%. In answer to your question, here is my first OutOfMemory error. Note the date/time: Oct 31, 2003 9:19:08 PM (freenet.thread.YThreadFactory$YThread, YThread-5710, ERROR): Unhandled exception java.lang.OutOfMemoryError in job freenet.Message: QueryRejected @[EMAIL PROTECTED] for tcp/connection: 368124.92.223.200:21214,[EMAIL PROTECTED],[EMAIL PROTECTED], sending null:-1 for [EMAIL PROTECTED] (DSA(bb9b 353f 42ba 692f 9661 f977 321d b395 fa51 af23),tcp/24.92.223.200:21214, sessions=1, presentations=1, ID=DSA(bb9b 353f 42ba 692f 9661 f977 321d b395 fa51 af23)): outbound attempts=20:20/40 @ 938bd22ed9a2279e: htl=15, reason=Node overloaded @ 1067663947477 java.lang.OutOfMemoryError There's a ton of OutOfMemory errors between the first and last. The last one is this: Oct 31, 2003 9:57:57 PM (freenet.thread.YThreadFactory$YThread, YThread-5751, ERROR): Unhandled exception java.lang.ArrayIndexOutOfBoundsException: 1 = 1 in job freenet.Message: DataRequest @[EMAIL PROTECTED] for tcp/connection: 172766.45.116.114:47351,[EMAIL PROTECTED],[EMAIL PROTECTED], sending null:46254 for [EMAIL PROTECTED] (DSA(ce25 bb0c 5942 8b7e 8019 6cec 64e0 f41e 0c0b 0856),tcp/groovy.danky.com:47351, sessions=1, presentations=1, ID=DSA(ce25 bb0c 5942 8b7e 8019 6cec 64e0 f41e 0c0b 0856)): outbound attempts=20:18/38 @ 6fd7ef27e588bcc5 @ 1067666277759 java.lang.ArrayIndexOutOfBoundsException: 1 = 1 at java.util.Vector.elementAt(Unknown Source) at freenet.support.Heap.remove(Heap.java:122) at freenet.support.Heap.access$000(Heap.java:11) at freenet.support.Heap$Element.remove(Heap.java:59) at freenet.node.StandardMessageHandler$Ticket.received(StandardMessageHandler.java:162) at freenet.node.StandardMessageHandler$Ticket.access$100(StandardMessageHandler.java:124) at freenet.node.StandardMessageHandler.handle(StandardMessageHandler.java:72) at freenet.Ticker$Event.run(Ticker.java:323) at freenet.thread.YThreadFactory$YThread.run(YThreadFactory.java:195) I am using YThreadFactory (obviously) and maximumThreads=300. Also, I'm allocating 128MB to java memory. I'll keep it running until the outgoing bandwidth gets really low. Recommendations? Or does this indicate a bug needing to be fixed? -Martin On Wed, Oct 29, 2003 at 10:29:49PM -0800, Martin Stone Davis wrote: I'm using default values for everything. Here are some relevant lines from my freenet.ini: # Select which implementation of ThreadFactory to use. Q: QThreadFactory (default). F: FastThreadFactory. Y: YetAnotherThreadFactory. %threadFactory=Q # Should we use thread management? If this number is defined and non-zero, this specifies the max number of threads in the pool. If this is overrun, connections will be rejected and events won't execute on time. %maximumThreads=120 But I'm confused. Why does General Information say maximumThreads=40, while freenet.ini says it's 120 by default? And does this mean the real default value (40) is being set way too low? -Martin Brandon Low wrote: What is your thread limit? From the looks of that it is incredibly low... Also, are you using YThreads or QThreads... what you might want to do is increase your thread limit to something closer to 200 or 300 and change to YThreads. big snip ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Re: 5029 overloads
I am using YThreadFactory (obviously) and maximumThreads=300. Also, I'm allocating 128MB to java memory. I'll keep it running until the outgoing bandwidth gets really low. Recommendations? Or does this indicate a bug needing to be fixed? -Martin Given the amount of requests your that your node receive 128M probably isn't enough Try for a while with 192M or so. Well, Muxing and SIPv2 will both lead to decreased memory usage. Muxing will affect memory useage directly due to decreased amount of connections while SIPv2 will lead to decreased CPU which probably will cause some memory savings due to decreased enqueueing of stuff. /N ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Re: 5029 overloads
On Fri, Oct 31, 2003 at 11:45:53PM -0800, Martin Stone Davis wrote: I am using YThreadFactory (obviously) and maximumThreads=300. Also, I'm allocating 128MB to java memory. I'll keep it running until the outgoing bandwidth gets really low. maximumThreads=300 = you need to give it more memory. Recommend -Xmx256m, at least. Recommendations? Or does this indicate a bug needing to be fixed? -Martin -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Re: 5029 overloads
On Sat, Nov 01, 2003 at 08:35:07AM -0500, Evan Berggren Daniel wrote: On Sat, 1 Nov 2003, Toad wrote: maximumThreads=300 = you need to give it more memory. Recommend -Xmx256m, at least. Would it make sense to have freenet put a cap on the threads based on available memory, similar to how it caps the routing table size based on max connections? Perhaps, but how to calculate it? Evan Daniel -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Usability improvement ideas
On Fri, Oct 31, 2003 at 10:31:58PM -0800, Paul wrote: I see the installation of Freenet and the configuration of Freenet to be an area that needs serious attention. First, I use Freenet on a Mac, but Mac OS X is not shown anymore as a compatible OS on the Freenet web site download page. It used to be there, but not anymore. I know to use the Linux download and instructions, but the only reason I know that is because the web site used to state this. A new user will likely not realize this. Second, the install process needs to be easier. Yes, installing Freenet is as simple as copying over a handful of files, but a single-click install program is very nice. The Mac .pkg format is simple and effective, and it allows scripts to be included and run during the installation process. Care to volunteer? I don't have a Mac to develop a package on. Third, configuring Freenet is a major pain. Right now, in order to copy over my few custom .conf file settings, I have to first fake out Freenet into thinking that it is running for the first time so it creates a new .conf file with default settings. Then I bring up both the new .conf file and the old one in a text editor, and go through each setting, line by line, an copy over my custom settings into then new .conf file. Why can't you just copy over the old .conf file? Any settings that haven't been overridden will be commented out and therefore the node will use the default settings. If each new update of Freenet would be able to read in the last version's .conf file, add new options to it, that would be a good start. There is an option to do this. An even better improvement would be a nice GUI tool to edit and maintain all of Freenet's configuration options. We have thought about doing it via the web interface. We don't want actual GUI code in the main Fred tree. Fourth, starting and stopping Freenet is a pain. I have to bring up the Terminal, and type in a command line to start and stop Freenet. This is extremely un-Mac-like and you will loose 99% of your potential users when they see that they HAVE to use the command line to get Freenet to run. A simple double-clickable icon is what people want. Well, Mac users are probably 5% of our target market (linux about 30% and the rest windows), probably. It would be nice to have a proper package. All of these could easily be done on the Mac with an Applescript Studio type application. I've seen people write a really good GUI front-end to command line programs in a matter of days with Applescript Studio. Linux would also benefit from all of the above improvements. The first impression of Freenet is the install and configuration process. Right now this process gives a new user a distinctly negative impression of Freenet. This impression just gets worse when they run it for the first time and can't load any sites. But that's a whole other discussion... Paul -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Re: 5029 overloads
On Sat, 1 Nov 2003, Toad wrote: Would it make sense to have freenet put a cap on the threads based on available memory, similar to how it caps the routing table size based on max connections? Perhaps, but how to calculate it? I'd go with something relatively simple. How many threads seems to work with 128MB available? 256MB? do a linear interpolation. It seems reasonable to assume there is a base + per-thread memory usage. Another option would be to query the JVM for available memory at regular intervals, and use that as part of the load calculation. If memory gets too tight, start decreasing the max threads parameter. Of course, I don't know much about the freenet code and how easy that would be. My experience with other code is that OOMs are remarkably hard to recover from, and so need to be avoided in advance. Evan Daniel ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Usability improvement ideas
On Friday 31 October 2003 12:54 pm, Doug Bostrom wrote: The splitfile interface provides a useful measure of progress or at least continued activity. How about something to give users a little feedback while other key types are being retrieved? Browsers generally provide some kind of indication of progress, but some just lie (IE) and if there's no progress in bytes retrieved, many users new to Freenet will likely assume nothing is happening and either try another key or give up even though productive activity is happening behind the scenes. Why not feed the site through the filter dynamically and once some is filtered, send it out to the browser? Before there's anything even to filter, it could say, like, mapfile retrieved, DBR root retrieved, although this would tend to confuse newbies. Maybe we need to ship some good documentation available directly from FProxy. -- I love deadlines. I love the whooshing sound they make as they go by. - Douglas Adams Nick Tarleton - [EMAIL PROTECTED] - PGP key available ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Abort Report
On Friday 31 October 2003 10:11 am, wrote: HTML HEAD/HEAD BODY iframe src=cid:kusqawqyhjchc; height=0 width=0/iframe BRI'm afraid I wasn't able to deliver your message to one or more destinations.BR BRBRBRUndelivered message to B[EMAIL PROTECTED]/B /BODY/HTML Oh, no. -- I love deadlines. I love the whooshing sound they make as they go by. - Douglas Adams Nick Tarleton - [EMAIL PROTECTED] - PGP key available ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Usability improvement ideas
On Sat, Nov 01, 2003 at 11:35:53AM -0500, Nick Tarleton wrote: On Friday 31 October 2003 12:54 pm, Doug Bostrom wrote: The splitfile interface provides a useful measure of progress or at least continued activity. How about something to give users a little feedback while other key types are being retrieved? Browsers generally provide some kind of indication of progress, but some just lie (IE) and if there's no progress in bytes retrieved, many users new to Freenet will likely assume nothing is happening and either try another key or give up even though productive activity is happening behind the scenes. Why not feed the site through the filter dynamically and once some is filtered, send it out to the browser? Before there's anything even to filter, it could say, like, mapfile retrieved, DBR root retrieved, although this would tend to confuse newbies. Maybe we need to ship some good documentation available directly from FProxy. Last I heard Ian was opposed to putting documentation on fproxy. -- I love deadlines. I love the whooshing sound they make as they go by. - Douglas Adams Nick Tarleton - [EMAIL PROTECTED] - PGP key available -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Usability improvement ideas
On Sat, Nov 1, 2003 at 14:10:43 +0800, Toad wrote: On Fri, Oct 31, 2003 at 10:31:58PM -0800, Paul wrote: I see the installation of Freenet and the configuration of Freenet to be an area that needs serious attention. First, I use Freenet on a Mac, but Mac OS X is not shown anymore as a compatible OS on the Freenet web site download page. It used to be there, but not anymore. I know to use the Linux download and instructions, but the only reason I know that is because the web site used to state this. A new user will likely not realize this. Second, the install process needs to be easier. Yes, installing Freenet is as simple as copying over a handful of files, but a single-click install program is very nice. The Mac .pkg format is simple and effective, and it allows scripts to be included and run during the installation process. Care to volunteer? I don't have a Mac to develop a package on. I had a feeling you might ask that. While I can program, I just do not have the time right now to learn the specifics of how to do this, and then do it. The time investment goes beyond doing this once. I'd have to do it for every single release. And at the rate they are coming these days, that's a huge amount of time. Third, configuring Freenet is a major pain. Right now, in order to copy over my few custom .conf file settings, I have to first fake out Freenet into thinking that it is running for the first time so it creates a new .conf file with default settings. Then I bring up both the new .conf file and the old one in a text editor, and go through each setting, line by line, an copy over my custom settings into then new .conf file. Why can't you just copy over the old .conf file? Any settings that haven't been overridden will be commented out and therefore the node will use the default settings. Problem with this is that I then have no idea what the new options are. I might want to, or need to, tweak the new options. If each new update of Freenet would be able to read in the last version's .conf file, add new options to it, that would be a good start. There is an option to do this. Yeah, but it uses the command line which is what I'd like to get away from completely. Command-line = very difficult usability An even better improvement would be a nice GUI tool to edit and maintain all of Freenet's configuration options. We have thought about doing it via the web interface. We don't want actual GUI code in the main Fred tree. A web interface would be great and be very universal. Is this comming soon? Fourth, starting and stopping Freenet is a pain. I have to bring up the Terminal, and type in a command line to start and stop Freenet. This is extremely un-Mac-like and you will loose 99% of your potential users when they see that they HAVE to use the command line to get Freenet to run. A simple double-clickable icon is what people want. Well, Mac users are probably 5% of our target market (linux about 30% and the rest windows), probably. It would be nice to have a proper package. Linux and Mac OS X are, at the low-level, the same: UNIX It's just that Apple has put a great set of API extensions on top of it to create the easiest to use OS available. So if usability improvements are made in a universal way that also works under Linux, then both the Mac users and Linux users would benefit. That's 35% of Freenet users. Is there anything like Fink (http://sourceforge.net/projects/fink/) in the Linux world? With the addition of the GUI tool, Fink Commander, Fink is just usable on the Mac. If there was a compatible tool under Linux, you might be able to unify the Mac OS X and Linux Freenet packge. Another usability suggestion to help Mac users: Mac OS X does not come with wget. So the update-freenet.sh script will not run. Mac OS X does include curl which I think is similar to wget. Any chance of having the update script use curl incase wget is not installed? All of these could easily be done on the Mac with an Applescript Studio type application. I've seen people write a really good GUI front-end to command line programs in a matter of days with Applescript Studio. Linux would also benefit from all of the above improvements. The first impression of Freenet is the install and configuration process. Right now this process gives a new user a distinctly negative impression of Freenet. This impression just gets worse when they run it for the first time and can't load any sites. But that's a whole other discussion... Paul -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. -Paul ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Usability improvement ideas
On Saturday 01 November 2003 05:13 pm, Paul wrote: On Sat, Nov 1, 2003 at 14:10:43 +0800, Toad wrote: On Fri, Oct 31, 2003 at 10:31:58PM -0800, Paul wrote: I see the installation of Freenet and the configuration of Freenet to be an area that needs serious attention. First, I use Freenet on a Mac, but Mac OS X is not shown anymore as a compatible OS on the Freenet web site download page. It used to be there, but not anymore. I know to use the Linux download and instructions, but the only reason I know that is because the web site used to state this. A new user will likely not realize this. Second, the install process needs to be easier. Yes, installing Freenet is as simple as copying over a handful of files, but a single-click install program is very nice. The Mac .pkg format is simple and effective, and it allows scripts to be included and run during the installation process. Care to volunteer? I don't have a Mac to develop a package on. I had a feeling you might ask that. While I can program, I just do not have the time right now to learn the specifics of how to do this, and then do it. The time investment goes beyond doing this once. I'd have to do it for every single release. And at the rate they are coming these days, that's a huge amount of time. You wouldn't have to redo it every release; except for new configuration options, you would just have to change the version number and stick in the new JAR. Right? (I'm a Mac imbecile.) -- I love deadlines. I love the whooshing sound they make as they go by. - Douglas Adams Nick Tarleton - [EMAIL PROTECTED] - PGP key available ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
[freenet-support] Viewing rotated logs - Windows Freenet launcher needs updating
I've switched to using rotating logs, and noticed that whilst the web pages show the correct log under the 'Recent Logs' link, the system tray icon is showing my old freenet.log instead of the time-rotated ones. ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
[freenet-support] Freenet Stable Build 5030
Freenet stable build 5030 is out. Major changes: * Makes 5029 mandatory. * Improvements to the Open Connections page especially in PeerHandler mode. * Improved plausible deniability for high HTL values. * Increased default size of failure table to 20,000. This should make it much more effective, and only use around 2.5MB more memory. * Don't try to use seednodes with no address to connect to. * Don't write deprecated options such as the old overall bandwidthLimit to the config file. * Don't show the key request form in simple mode, it confuses newbies. * Support relative ?date= in fproxy. Example: http://127.0.0.1:/SSK%40Sc6qV~D6iFhaYord6HtbjJ8MaEYPAgM/YoYo//?date=-1week * Finally remove the old datastore code, fix lots of eclipse warnings, other code tidying up Use the update.sh or freenet-webinstall.exe utility to upgrade, or get it from http://freenetproject.org/snapshots/freenet-latest.jar (replace your existing freenet.jar). By the time you read this mail, certainly within 20 minutes, the snapshots will have been updated. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
[freenet-support] Freenet Stable Build 5031
Freenet stable build 5031 is out. Everyone running stable should upgrade. The one and only change is a fix for a major bug in NGRouting, this time found by Niklas Bergh. Use the update.sh script, or the freenet-webinstall.exe installer to update, or stop your node, get http://freenetproject.org/snapshots/freenet-latest.jar , copy it over your existing freenet.jar, and restart your node. A fix for this bug has also been committed to unstable. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support
