-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, May 6, 2009 at 12:50 PM, Juiceman wrote: > On Wed, May 6, 2009 at 11:37 AM, Matthew Toseland > wrote: >> On Wednesday 06 May 2009 14:43:59 Victor Denisov wrote: >>> Matthew Toseland wrote: >>> > This is the downside of db4o. If it is a widespread problem, we're gonna >> have >>> > to revert it. Which means throwing away more than 6 months work largely >>> > funded by Google's $18K. >>> >>> I think that using a database is a good idea (although I personally >>> would've opted for a relational database such as Derby). So I'd prefer >>> to try and understand and fix the issue rather than hiding from it :-). >>> >>> > My database queue is usually pretty empty, even with queued downloads, but >> I >>> > have 8G and fast mirrored disks... >>> >>> The problem's that Freenet *doesn't* even use the amount of memory I >>> provide it with (I'm yet to see it use more than 120 megs out of 320 I >>> allow for the heap). I'd be willing to dedicate as much memory as >>> required if only it'd help. >>> >>> My hard drives are nothing special - 250Gb 7200 RPM Seagate ones, 16 Mb >>> cache, SATA2, no NCQ - though definitely not the slowest out there. I >>> see ~35 Mb/s read speed and ~28 Mb/s write speed for medium-sized files >>> and ~5 Mb/s to 8 Mb/s for small files in the tests I'd done. I'll >>> probably have to test the same from inside Java to make absolutely sure >>> that it's not some weird JVM issue on my platform, though. >>> >>> > 2650 handles is strange, on unix we are generally limited to 1024 and >>> > generally we don't exceed that. Both of your problems may be caused by >> flaky >>> > hardware, but frankly we do need to run on flaky real world hardware. :| >>> >>> I don't have Freenet running right now, will check it later. But I2P is >>> using 2670 handles right now, and Azureus uses 1450 - so 2600 for >>> Freenet is definitely nothing out of the ordinary on Windows. Oh, and >>> the highest handle user on my machine is MySQL, which uses ~69000 >>> handles and works absolutely fine :-). >>> >>> >> Same here. Enormous disk queues. I've also compared i/o counts with i/o >>> >> bytes read/written - that's how I know that i/o operations are small. In >>> >> the statistics screen, I routinely see 100+ outstanding database jobs. >>> >> It can't be good. >>> > >>> > This just confirms that disk I/O is the problem ... and almost certainly >>> > caused by db4o as it goes away if nothing is queued. >>> >>> My thinking exactly. Would providing you with a snapshot of CPU/memory >>> performance under YourKit Profiler (I have academic licenses for both >>> 7.5 and 8.0, IIRC) or VisualVM (which is now a part of the JDK >>> distributive) on my machine help? Any logging I can turn on to help? >>> BTW, I have logging set to ERROR for now, as with NORMAL level it logs >>> ~2Mb per minute, adding noticeably to overall disk contention. >>> >>> Regards, >>> Victor Denisov. >> >> One other thing, for both you and Juiceman: >> How's the CPU usage? Given how much RAM you have I would expect node.db4o to >> be cached in memory (how big is it?). But doing a read through the OS to the >> OS disk cache may cost a lot of CPU (context switch etc) ... Is there a lot >> of CPU usage for the freenet process? To the point that it might be the cause >> of the poor overall system performance? And how much CPU usage is system? >> > > Freenet CPU usage fluctuates between 2 and 27% of a quad core system. > The rest of the machine rarely uses more than 15% unless I am gaming, > then it still only hits 50%. CPU usage is quite acceptable for now. > I have 3GB of RAM, 512 allocated to Freenet. Node.db4o was 375 MB. No uploads, 1 GB of queued downloads. How often is this file written to? Anyway to queue writes in a RAM buffer and write to disk periodically? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Use GnuPG with Firefox : http://getfiregpg.org (Version: 0.7.5) iEYEARECAAYFAkoCFKUACgkQ4esu1mlKOs/ocgCfdm8v9JstR1RrHMg3SM1/NnUK kvkAnj/fg5e0JCFwsJpPL+y+sEtC2/4V =4EOX -----END PGP SIGNATURE----- _______________________________________________ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:support-requ...@freenetproject.org?subject=unsubscribe
