[freenet-support] Re: OSX builds of freenet
Matthew Toseland writes: > > On Fri, Sep 30, 2005 at 08:04:42PM +, Squished Squirrel wrote: > > Matthew Toseland ...> writes: > > > > > > > > Hmm, cool. Can you send me a patch to detect OS/X and automatically fix > > > it? I'm not sure how we would tell the node about it... > > > > > > > Sure. Let me get this machine home and running a few days to make sure > > it all works first. There is a good possibility I am talking out my ass about > > all this > > > > Yeh, I don't know how you could verify that the fd limit had been upped > > within a java program. I think the safest thing to do is to continue to default > > OSX boxes to 128 maxNodeConnections, and in the start script and README, > > put comments explaining that if you uncomment and up the node count, you > > MUST use the start-freenet.sh shell script to start freenet so it can increase > > the fd limit. > > But then it will have to edit the config file before configuring, right? > Does freenet.jar accept command line options? In other words, can I pass maxNodeConnections as a command line option? If so, then the script could be self contained... both increasing the fd count and increasing the Node count without actually touching the .conf file. I'm afraid if the script edits the .conf file, and the end user switches to launching freenet with his own script or by double clicking it, the increased fd limit won't be in effect, and freenet may crash.
[freenet-support] Re: OSX builds of freenet
Matthew Toseland writes: > > Hmm, cool. Can you send me a patch to detect OS/X and automatically fix > it? Well, the OSX native jbigi library went through the weekend without issue. I did a fair amount of surfing, using frost, downloading, etc. and not a hiccup. The increased fd limit is definately in effect. I was running around 242 connections. The store passed 4GB this morning. No kernel panics, no freezes. Squirrel
[freenet-support] Re: OSX builds of freenet
Matthew Toseland [EMAIL PROTECTED] writes: Hmm, cool. Can you send me a patch to detect OS/X and automatically fix it? Well, the OSX native jbigi library went through the weekend without issue. I did a fair amount of surfing, using frost, downloading, etc. and not a hiccup. The increased fd limit is definately in effect. I was running around 242 connections. The store passed 4GB this morning. No kernel panics, no freezes. Squirrel ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
[freenet-support] Re: OSX builds of freenet
Matthew Toseland writes: > > Hmm, cool. Can you send me a patch to detect OS/X and automatically fix > it? I'm not sure how we would tell the node about it... > Sure. Let me get this machine home and running a few days to make sure it all works first. There is a good possibility I am talking out my ass about all this ;-) Yeh, I don't know how you could verify that the fd limit had been upped within a java program. I think the safest thing to do is to continue to default OSX boxes to 128 maxNodeConnections, and in the start script and README, put comments explaining that if you uncomment and up the node count, you MUST use the start-freenet.sh shell script to start freenet so it can increase the fd limit.
[freenet-support] Re: OSX builds of freenet
Matthew Toseland writes: > > Umm, you can raise the descriptor limit as a regular user on OS/X?! > Please check what value it was before and after... > Ohh, to clarify, you are correct in that you can't change the HARD limit on file descriptors, but that isn't a problem because the hard limit is set to unlimited. The hard limit is only limited by the system limit for all processes, which is over 12000. The soft limit is the only one that needs to be upped, since it is what is capping us at 256. So, trying to do: ulimit -n 1024 might fail because it is trying to change both the soft and hard limits, whereas: ulimit -S -n 1024 Is just changing the soft limit (the hard limit is unlimited...)
[freenet-support] Re: OSX builds of freenet
Matthew Toseland writes: > > Umm, you can raise the descriptor limit as a regular user on OS/X?! > Please check what value it was before and after... > Yes, you can change the limit as a standard or admin user on OSX. The page that describes it pretty well is: http://www.amug.org/~glguerin/howto/More-open-files.html Here is the reult: sh-2.05b$ ulimit -a core file size(blocks, -c) 0 data seg size (kbytes, -d) 6144 file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files(-n) 256 pipe size (512 bytes, -p) 1 stack size(kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes(-u) 100 virtual memory(kbytes, -v) unlimited sh-2.05b$ ulimit -S -n 1024 sh-2.05b$ ulimit -a core file size(blocks, -c) 0 data seg size (kbytes, -d) 6144 file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files(-n) 1024 pipe size (512 bytes, -p) 1 stack size(kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes(-u) 100 virtual memory(kbytes, -v) unlimited
[freenet-support] Re: OSX builds of freenet
Paul writes: > > Can we get this Mac OS X native jbigi library incorporated in to > future Freenet releases? > > I've been running Freenet on Mac OS X for about two years now. It > usually runs for 2 to 7 days before it grinds to a halt with either > an out of memory error or some kind of error in the "abstractor loop". > > Having better performance while Freenet is up would be great. Fixing > the above errors would also be nice. > > Thanks, > Paul > I'm about to test it over the weekend and see what happens. I know Java 1.4.2 (at least the latest version installed with 10.3.9) will crater with Kernel Panics. It sounds a lot like the problems that Azureus suffered from on Dual Processor Macs. When too many connections were opened simultaniously, Azureus would kernel panic. Apparently they found a work- around though. For my test this weekend, I'm gonna go for broke and use the latest JVM available for Tiger (1.5.0_04-39) and see how it goes. (If anyone else installs it, keep in mind your computer will continue to use the 1.4.X version unless you change a specific directory link, or use a hard coded path to the java 1.5 binary.) I did see some things in the start-freenet.sh script that might need to be changed to improve performance/clarity. First, it doesn't set the java argument -xx:MaxDirectMemorySize if you are running on OSX with JVM 1.4.2 or later. That is a legit flag for the OSX Sun JVM, so I don't know why it skips that argument only on OSX (Darwin). I'm just commenting out the "if" and "fi" statements above and below the line that sets that argument. Second, when launched on a machine running a JVM later than 1.4.2, it still says you are using 1.4.2. It should say: "Sun Java 1.4.2 or later detected." Finally, as the readme suggests, I'm adding these lines to the start script just before the "Starting freenet now" line: if test `uname` == "Darwin"; then ulimit -S -n 1024 fi That ups the file descriptor limit to the same as the default on other platforms. After that, you need to up the "maxNodeConnections" parameter to 512 in the config file. I'm also upping the memory limit to -Xmx256m just because the machine has a decent amount of ram. With these changes, OSX should be running with the same parameters as linux or windows.
[freenet-support] Re: OSX builds of freenet
Paul [EMAIL PROTECTED] writes: Can we get this Mac OS X native jbigi library incorporated in to future Freenet releases? I've been running Freenet on Mac OS X for about two years now. It usually runs for 2 to 7 days before it grinds to a halt with either an out of memory error or some kind of error in the abstractor loop. Having better performance while Freenet is up would be great. Fixing the above errors would also be nice. Thanks, Paul I'm about to test it over the weekend and see what happens. I know Java 1.4.2 (at least the latest version installed with 10.3.9) will crater with Kernel Panics. It sounds a lot like the problems that Azureus suffered from on Dual Processor Macs. When too many connections were opened simultaniously, Azureus would kernel panic. Apparently they found a work- around though. For my test this weekend, I'm gonna go for broke and use the latest JVM available for Tiger (1.5.0_04-39) and see how it goes. (If anyone else installs it, keep in mind your computer will continue to use the 1.4.X version unless you change a specific directory link, or use a hard coded path to the java 1.5 binary.) I did see some things in the start-freenet.sh script that might need to be changed to improve performance/clarity. First, it doesn't set the java argument -xx:MaxDirectMemorySize if you are running on OSX with JVM 1.4.2 or later. That is a legit flag for the OSX Sun JVM, so I don't know why it skips that argument only on OSX (Darwin). I'm just commenting out the if and fi statements above and below the line that sets that argument. Second, when launched on a machine running a JVM later than 1.4.2, it still says you are using 1.4.2. It should say: Sun Java 1.4.2 or later detected. Finally, as the readme suggests, I'm adding these lines to the start script just before the Starting freenet now line: if test `uname` == Darwin; then ulimit -S -n 1024 fi That ups the file descriptor limit to the same as the default on other platforms. After that, you need to up the maxNodeConnections parameter to 512 in the config file. I'm also upping the memory limit to -Xmx256m just because the machine has a decent amount of ram. With these changes, OSX should be running with the same parameters as linux or windows. ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
[freenet-support] Re: OSX builds of freenet
Matthew Toseland [EMAIL PROTECTED] writes: Umm, you can raise the descriptor limit as a regular user on OS/X?! Please check what value it was before and after... Yes, you can change the limit as a standard or admin user on OSX. The page that describes it pretty well is: http://www.amug.org/~glguerin/howto/More-open-files.html Here is the reult: sh-2.05b$ ulimit -a core file size(blocks, -c) 0 data seg size (kbytes, -d) 6144 file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files(-n) 256 pipe size (512 bytes, -p) 1 stack size(kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes(-u) 100 virtual memory(kbytes, -v) unlimited sh-2.05b$ ulimit -S -n 1024 sh-2.05b$ ulimit -a core file size(blocks, -c) 0 data seg size (kbytes, -d) 6144 file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files(-n) 1024 pipe size (512 bytes, -p) 1 stack size(kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes(-u) 100 virtual memory(kbytes, -v) unlimited ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
[freenet-support] Re: OSX builds of freenet
Matthew Toseland [EMAIL PROTECTED] writes: Umm, you can raise the descriptor limit as a regular user on OS/X?! Please check what value it was before and after... Ohh, to clarify, you are correct in that you can't change the HARD limit on file descriptors, but that isn't a problem because the hard limit is set to unlimited. The hard limit is only limited by the system limit for all processes, which is over 12000. The soft limit is the only one that needs to be upped, since it is what is capping us at 256. So, trying to do: ulimit -n 1024 might fail because it is trying to change both the soft and hard limits, whereas: ulimit -S -n 1024 Is just changing the soft limit (the hard limit is unlimited...) ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
[freenet-support] Re: OSX builds of freenet
Matthew Toseland [EMAIL PROTECTED] writes: Hmm, cool. Can you send me a patch to detect OS/X and automatically fix it? I'm not sure how we would tell the node about it... Sure. Let me get this machine home and running a few days to make sure it all works first. There is a good possibility I am talking out my ass about all this ;-) Yeh, I don't know how you could verify that the fd limit had been upped within a java program. I think the safest thing to do is to continue to default OSX boxes to 128 maxNodeConnections, and in the start script and README, put comments explaining that if you uncomment and up the node count, you MUST use the start-freenet.sh shell script to start freenet so it can increase the fd limit. ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
[freenet-support] Re: OSX builds of freenet
Matthew Toseland writes: > > Cool. What code changes? > I added a couple case statements to NativeBigInteger.java, similar to those suggested by Josh Watzman earlier this month. I think his diff also added support for loading an external jbigi library without having to rebuild the freenet-ext.jar file. My changes only work if the library is put into the jar. I'm *assuming* that his code changes result in a requested library name of "libjbigi-osx-none.jnilib". I don't know for sure, since I found his diff after I had already made my changes. I used the jbigi source from i2p since the build scripts for it had already been modified to support OSX. The resulting file was libjbigi.jnilib, which just needed to be renamed to "libjbigi-osx-none.jnilib" and inserted into freenet-ext.jar. The i2p_0_6_0_6 source builds the jbigi library static, so I'm hoping that it will run fine on a Mac that doesn't have gmp installed. Someone else said they had managed to build the library on the Mac, but their library was dynamic and would only run on a Mac with gmp already installed (or something to that effect.) After the osx jbigi library is inserted in to the freenet-ext.jar, I dropped that into the lib folder of the current stable source and built the project using ant. I won't discuss how many wrong turns I made on the way ;-) And the result on starting freenet... INFO: Non-optimized native BigInteger library 'net/i2p/util/libjbigi-osx-none.jnilib' loaded from resource
[freenet-support] Re: OSX builds of freenet
Matthew Toseland [EMAIL PROTECTED] writes: Cool. What code changes? I added a couple case statements to NativeBigInteger.java, similar to those suggested by Josh Watzman earlier this month. I think his diff also added support for loading an external jbigi library without having to rebuild the freenet-ext.jar file. My changes only work if the library is put into the jar. I'm *assuming* that his code changes result in a requested library name of libjbigi-osx-none.jnilib. I don't know for sure, since I found his diff after I had already made my changes. I used the jbigi source from i2p since the build scripts for it had already been modified to support OSX. The resulting file was libjbigi.jnilib, which just needed to be renamed to libjbigi-osx-none.jnilib and inserted into freenet-ext.jar. The i2p_0_6_0_6 source builds the jbigi library static, so I'm hoping that it will run fine on a Mac that doesn't have gmp installed. Someone else said they had managed to build the library on the Mac, but their library was dynamic and would only run on a Mac with gmp already installed (or something to that effect.) After the osx jbigi library is inserted in to the freenet-ext.jar, I dropped that into the lib folder of the current stable source and built the project using ant. I won't discuss how many wrong turns I made on the way ;-) And the result on starting freenet... INFO: Non-optimized native BigInteger library 'net/i2p/util/libjbigi-osx-none.jnilib' loaded from resource ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
[freenet-support] OSX builds of freenet
Thanks to those who tried to help me get a pathetic old Celeron 400 optimized to run freenet acceptably well. I caved and spent the time to build freenet for OSX (With native jbigi libaries!!!) using the current stable source. I knew just enough programming to make the few code changes to support loading the jbigi library for OSX and managed to compile jbigi.jnilib using the I2P source and insert it into the freenet.ext.jar file. Then I stuck that into the freenet source directory so that I could then compile freenet using ant. That may be nothing to the the developers that hang about here, but I'm damn pleased ;-) ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
[freenet-support] Re: Disk Thrashing issues
S [EMAIL PROTECTED] writes: I would venture to say that increasing VM is more likely to increase disk access, not decrease it. -Xmx does seem to be what you want, though; it will set a ceiling on the amount of RAM that Java will allocate. Try disabling Virtual Memory in Windows altogether, and see if that helps any with the disk thrashing. You should be able to run Windows plus Freenet reasonably well in 640 megs, especially if you kill off any tray utilities and unnecessary services. Check the task manager to see how much RAM Windows wants for itself, then use -Xmx (or FLaunch.ini) to give Java most of whatever's left. s In my haste to post, I said VM to 192... I meant JavaMem. Looking at the task manager, I'm not seeing any paging per se, so I don't think that is it. It seems to be a periodic task that freenet itself is performing. It might go away if I dropped a drive in that had a decent cache. This is a 5 year old 30GB maxtor drive, and I doubt it has much of a cache on the drive itself. I think I really need to dig up another box with a wee bit more modern CPU and drive. OK, I'm really beginning to hate all the limitations on the web version of the gmane support page. Time to get on the list. Pathetic thing can't even linewrap. Squirrel ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
[freenet-support] Disk Thrashing issues
I've decided to dedicate an old PC as a freenet node, and I'd like to get it optimized the best I can, given its resources. It is a Celeron 400 with 640MB RAM running XP. The problem I see is a lot of disk access. About every second, the drive thrashes. I've read the posts on upping the VM to 192..256..512 or so using -Xmx or by changing the JavaMem line in flaunch.ini, and it doesn't really make any difference. The only reason I am concerned is I'd like to keep wear and tear on that drive to a minimum so it doesn't fail prematurely (the loss of the cache and all that entails.) My connection has a potential of 1800KB/sec in, 230KB/sec out (fiber 15000kbps/2000kbps), however, I've limited the bandwidth to 50KB/sec in and out to try and keep the CPU at about 90% rather than pegged at 100% at all times. Am I simply asking this old machine to do too much? I'd run this on a dual processor OSX box, but I don't have the time to figure out how to compile native encryption libraries. If anyone has a link to precompiled binaries, I would appreciate it! Squirrel
[freenet-support] Disk Thrashing issues
I've decided to dedicate an old PC as a freenet node, and I'd like to get it optimized the best I can, given its resources. It is a Celeron 400 with 640MB RAM running XP. The problem I see is a lot of disk access. About every second, the drive thrashes. I've read the posts on upping the VM to 192..256..512 or so using -Xmx or by changing the JavaMem line in flaunch.ini, and it doesn't really make any difference. The only reason I am concerned is I'd like to keep wear and tear on that drive to a minimum so it doesn't fail prematurely (the loss of the cache and all that entails.) My connection has a potential of 1800KB/sec in, 230KB/sec out (fiber 15000kbps/2000kbps), however, I've limited the bandwidth to 50KB/sec in and out to try and keep the CPU at about 90% rather than pegged at 100% at all times. Am I simply asking this old machine to do too much? I'd run this on a dual processor OSX box, but I don't have the time to figure out how to compile native encryption libraries. If anyone has a link to precompiled binaries, I would appreciate it! Squirrel ___ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
