Sounds great! With this, Freetalk, the new Windows installer, and other things - 0.8 should be a great release.
Ian. On Fri, Jan 30, 2009 at 8:01 PM, Matthew Toseland <[email protected]> wrote: > The db4o branch is rapidly nearing completion, with most of the serious bugs > and performance issues worked out. Instant download resuming, and support for > a nearly unlimited volume of downloads while using very little RAM, are the > main features. > > However, there are memory usage spikes at certain points during inserts and > requests. In particular, if you attempt a 4GB ISO insert on a node with a > 128MB memory limit, it will go into 100% CPU for some time (Full GC's), due > to running out of memory and having to do lots of garbage collection, and > probably eventually fail with an OutOfMemoryException. So we will probably > want to keep the default memory usage limit of 192MB. > > However, with the exception of *really* big inserts (well over 4GB), 192MB > should be enough for almost all users with db4o, so there is no reason to ask > the end-user about it any more IMHO. In fact, it will allow for multiple 4GB > inserts, since any one of them will only cause a spike once. 25GB single file > inserts would obviously produce a larger spike - somewhere in the region of > 1GB of RAM would be needed. > > It would be possible to remove this limit completely, but it would require > more work and complicate the API, and reduce performance starting big inserts > slightly on nodes with lots of RAM; it wouldn't necessarily take more than a > day, but since IMHO it could be done post-merge without too back > compatibility issues, it can be safely postponed. > > Note that with the exception of the memory spikes while starting an insert (or > a request), db4o will need very little memory. Almost everything is kept in > the database and is only cached in RAM, with the exception of the request > bloom filters, which are approximately 3.7MB per 10GB of queued downloads. > > One problem remaining is that inserts are quite resource intensive (lots of > database access / disk I/O) on the db4o branch, more so than on the current > code. This largely results from the current inefficient architecture of > splitfile inserts; I could fix this before merging, and it would avoid > complications with back compatibility later, but it would be somewhat > significant work, and the back compatibility issues for doing it later will > be surmountable, so I probably won't deal with inserts before merging db4o. > > Obviously the db4o branch does more disk i/o than the current code. IMHO this > is an acceptable tradeoff, especially for downloads. > > Remaining work: > - Finish the unused structure deletion code. > - Consider temp file garbage collection. > - Recently-succeeded preference in request selection. > - Consider removing use of obsolete structures. > - More debugging. > > _______________________________________________ > Devl mailing list > [email protected] > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > -- Ian Clarke CEO, Uprizer Labs Email: [email protected] Ph: +1 512 422 3588 Fax: +1 512 276 6674 _______________________________________________ Devl mailing list [email protected] http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
