Dalibor Topic ([EMAIL PROTECTED]) wrote: > I assume that the Freenet developers lurking on this > mailing have some experience with respect to > scalability of kaffe on their platforms, as they use > kaffe with > a) tons of threads > b) to run crypto code > c) lots of network activity
> Matthew, Greg, could you provide some > anectodes/numbers? Matthew is a Freenet developer; I'm a user, documentation writer, and unofficial tech support helper. I'm using Kaffe to run Freenet on OpenBSD because, as far as I know, there is no other usable Java virtual machine for OpenBSD. The Linux x86 Java binaries from Sun will run, to a certain extent, but they did not work for me when running Freenet. (The Linux emulation layer is obviously not 100% complete/bug-free. The major symptom was that reads from a socket that didn't already have data waiting didn't "block" as they were supposed to; they returned without any data. Since Freenet is supposed to provide network services, this made it unusable.) I'm configuring Kaffe with "./configure --with-includes=/usr/local/include --with-libraries=/usr/local/lib --with-engine=jit3". I also have to edit the libtool file by hand after the configuration, to change "need_version" from "no" to "yes". Apart from this, it compiles and installs cleanly. I have two problems with Freenet under Kaffe: crashing and memory usage. I've posted gdb stack traces of Kaffe core dumps to this mailing list already. Since the FD_ISSET assertion (jthread.c:1750) was removed from CVS recently, I no longer get that particular one; but I'm still getting this one occasionally: assertion "!INTS_DISABLED()" failed: file "exception.c", line 398 However, I'm getting dramatically *fewer* crashes with CVS Kaffe than I did when I started using Kaffe (1.0.7). (It's also worth mentioning that Kaffe 1.0.6 and earlier will not work with Freenet at all. The OpenBSD Kaffe port is still 1.0.6, and therefore less than useful for me.) In terms of memory usage, a long-running Freenet node creeps continually upward. I've got the ulimits set so that Freenet cannot exceed 256 MB of memory, and I'm running the java command with "-mx 224M". After a few days, it will hit this limit (if it doesn't crash first). It's not supposed to do that; Linux users report that Freenet's memory usage under Sun's Java VM remains steady at something like 30 MB (depending on which build of Freenet is being used, which version of the Java VM, etc.). Apart from these two issues, Freenet runs quite well under Kaffe for me. Performance is extremely difficult to measure with Freenet because the amount of work being done by a live node is unpredictable. Data requests from other nodes could occur at any time. The single most intensive operation the node performs, at least from the end user's perspective, is encoding/decoding large files with FEC (Forward Error Correction). Large files in Freenet are broken into "segments" with a maximum size of 128 MB. Each segment consists of 128 or fewer data blocks, and a number of check blocks (currently 50% of the data blocks, so a maximum of 64). Retrieving any 128 of these 192 blocks is sufficient to reconstitute the whole segment, but decoding the 128 MB segment can take several minutes. On my 1300 MHz Duron, it's a bit over 2 minutes, wall-clock time, but this can be deceptive because the node is not idle during this operation; it's still processing other requests. (It's also not entirely CPU-bound; there is a considerable amount of disk activity associated with this, writing the blocks to temporary files during the fetching, reading them back in during the decoding, and writing the reconstituted segments out to the final destination.) I'm using the Java FEC library that's supplied with Freenet. There is also supposed to be a native x86 FEC library, but I have not tried it. I'm not even sure what the compatibility requirements are. Since I can't run Freenet under Linux on this machine to give numbers which would be useful for comparison purposes, all I can say is that I'm pleased with Kaffe's performance under heavy loads. > On a side note, is there a FreenetMark ? There is (or was) a simulator which fired up several Freenet nodes on a single machine in order to test the networking protocols, routing, and so on. I don't think it was designed to measure performance; but rather, correctness and scalability of the Freenet code and heuristics. The Freenet developers might have more information about it; it was mentioned recently on the Freenet development list. (The thread started here: <http://hawk.freenetproject.org:8080/pipermail/devl/2003-February/004408.html> and continued here: <http://hawk.freenetproject.org:8080/pipermail/devl/2003-March/004415.html> It seems that the mailing list archives can't handle threads that span an end-of-month boundary.) -- Greg Wooledge | "Truth belongs to everybody." [EMAIL PROTECTED] | - The Red Hot Chili Peppers http://wooledge.org/~greg/ |
pgp00000.pgp
Description: PGP signature