On Friday 27 February 2009 05:02:51 Daniel Cheng wrote: > On Fri, Feb 27, 2009 at 5:19 AM, Matthew Toseland > <toad at amphibian.dyndns.org> wrote: > > PROGRESS REPORT > > =========== > > > > FINANCIAL STATUS: > > We have $3566.87, that's enough for around another 47 days after my current > > (approximately monthly) contract completes. > > > > ZERO POINT EIGHT: > > Which brings us on to 0.8. The current plan is to get the db4o branch merged, > > and get an alpha or beta of 0.8.0 out in the next month, by around April 1st. > > There are various features we want to have in 0.8 which are not yet ready: > > - the db4o branch > > - Freetalk, and any necessary plugin changes > > - The new windows installer > > - Progress screen etc > > - Possibly some UI improvements > > - Librarian improvements > > > > It is unlikely that these will all be ready before April 1st, so we will > > release an alpha rather than a feature-complete beta. > > > > DB4O BRANCH: > > The db4o branch is a temporary fork of the main codebase, which uses db4o, an > > open source object database, to persist queued downloads and inserts. The > > main objectives are to greatly reduce the memory needed especially with large > > numbers of queued downloads or uploads, and to instantly resume requests on > > restarting the node. A common cause of high CPU usage is having more > > downloads or inserts than you have enough memory for, causing the node to do > > constant garbage collection trying to find the last byte to re-use, and > > eventually crashing. And on a typical trunk node with lots of downloads > > queued, it can take hours after startup before the node stops thrashing the > > disk finding the blocks it has already downloaded. However, the background > > level of disk I/O will be higher with db4o than the background level of disk > > I/O on the current code (when the current code isn't resuming downloads). > > > > Both goals have been largely achieved. Right now, requests work, inserts work, > > and freesite inserts work. After db4o has been finished, and merged, I will > > set the default memory limit to 192MB, and not ask the user about it in the > > wizard. Really big inserts (over 4GB) may require a memory limit over 192MB, > > that is something we can solve later if necessary. > > > > Unfortunately, most recent work has been debugging and finishing architectural > > changes to inserts, so the todo list is not that far from what it was last > > time. However, there really isn't that much left now, I hope to merge the > > db4o branch in the coming weeks: > > - Fix database object leaks in inserts. > > - Prefer recently successful requests in the scheduler. > > - More debugging. > > - Think about garbage collection for persistent temporary files. This will > > only be needed to recover space left over due to bugs, so it's not clear > > whether we need it ... > > - Think about turning off the persistent blob tempfile, or implementing defrag > > for it if necessary. This puts all the 32KB temp files into one big file; it > > can get fragmented, causing the file to be larger than it needs to be; > > implementing defrag would be a little messy but not difficult, but without > > this big inserts are problematic on small memory nodes. > > - Delete one or two structures using deprecated db4o collections. > > - Backup the database automatically on a schedule, in case of severe problems. > > - More debugging, merge it. > > > > FREETALK: > > p0s has been building the Freetalk plugin (and the WoT plugin it is based on) > > for some time. This is a web-of-trust based chat system similar to FMS. It > > may or may not be compatible with FMS, debates over specifications are > > ongoing. It will have a web interface, and be integrated into the node by > > default, showing up on the menu. p0s has 2 months vacation which he is hoping > > to use to finish Freetalk. Messaging is working, but trust list management is > > missing at the moment. Because Freetalk depends on the WoT plugin, it will be > > necessary to implement plugin versioning as well as auto-update of plugins > > before we release 0.8. > > > > WINDOWS INSTALLER: > > Our current installer does not work well on Windows. It has more screens than > > are necessary and therefore confuses and annoys users. It doesn't work at all > > on Vista for most users. And it doesn't work well with multiple installs. > > Zero3 has been working on a new installer using AutoHotKey which solves all > > of these problems. More testing would be a good idea; he has posted a URI to > > the installer on several occasions. One caveat is that it does use some > > closed-source freeware, however IIRC our current installer does too on > > Windows. Also, I haven't used AHK, so reviewing the code is problematic. But > > it promises to be a great improvement. It is currently designed as a bundle > > installer, that is, it will need to be rebuilt manually on every release. I'm > > not sure what the remaining issues are...? > > > > PROGRESS SCREEN ETC: > > Because we got rid of the firefox profile, and for general usability reasons, > > we need some sort of screen displayed when Freenet is fetching a freesite. > > This will be implemented before 0.8. It should reduce the "it's gone off into > > limbo" factor considerably for new nodes. Initially this will just be a 5 > > second auto-refresh, but ideally it would be dynamically updated with > > javascript, for faster response times and a better looking interface. We > > might be able to lift the size limits on data sent to the browser at the same > > time, which would be especially impressive if we have time to implement some > > simple multimedia filters (mp3, ogg theora etc). Also, for the same reasons, > > if a user is loading a page from freenet in his browser, with lots of inline > > images, the browser will currently limit it to 8 connections and so it will > > load very slowly. Ideally we would replace each image with a loading graphic, > > and then replace it with the actual content when it is available. I have been > > sent some javascript for this; if you have any ability in javascript, your > > skills may be helpful. We will also need a loading screen for the search box. > > > > UI IMPROVEMENTS: > > Ideally we would improve the user interface before 0.8. There are lots of > > relatively small changes that could improve usability considerably, for > > example moving geeky stuff to submenus. Whether we will have time remains to > > be seen. > > > > LIBRARIAN IMPROVEMENTS: > > The wAnnA? index, which is used by the search box on the homepage (aka > > XMLLibrarian), was having some problems for a while due to the inability to > > insert very large freesites (a librarian index is more or less a freesite > > from Freenet's point of view). As far as I can tell, this has been fixed in > > db4o, and hopefully also in stable build 1205. sdiz has been doing work on > > internationalisation, code cleanups, and so on, on the XMLLibrarian plugin, > > ON internationalisation: > I am planning to use ICU4J, because of Thai language support. > It use some data file from unicode.org (unicode data license[1]) and icu[2]. > both license look BSD-like to me, but IANAL. > > The whole library is ~5.4M. If I remove the stuff I don't need, it is ~800k. > It is needed for both the spider and the librarian. > > [1] The Exhibit 1: http://unicode.org/copyright.html#Exhibit1 > [2] http://icu-project.org/repos/icu/icu/trunk/license.html
Sounds fine to me. You might want to look it up, maybe even ask fsf, but it should be a known issue... > > > as well as his earlier major improvements to XMLSpider, which now uses the > > Perst database to store its progress and minimise memory usage. IMHO we > > should improve the UI of the XMLLibrarian before releasing 0.8. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20090302/743a9bcd/attachment.pgp>
