On Fri, Feb 27, 2009 at 5:19 AM, Matthew Toseland <toad at amphibian.dyndns.org> wrote: > PROGRESS REPORT > =========== > > FINANCIAL STATUS: > We have $3566.87, that's enough for around another 47 days after my current > (approximately monthly) contract completes. > > ZERO POINT EIGHT: > Which brings us on to 0.8. The current plan is to get the db4o branch merged, > and get an alpha or beta of 0.8.0 out in the next month, by around April 1st. > There are various features we want to have in 0.8 which are not yet ready: > - the db4o branch > - Freetalk, and any necessary plugin changes > - The new windows installer > - Progress screen etc > - Possibly some UI improvements > - Librarian improvements > > It is unlikely that these will all be ready before April 1st, so we will > release an alpha rather than a feature-complete beta. > > DB4O BRANCH: > The db4o branch is a temporary fork of the main codebase, which uses db4o, an > open source object database, to persist queued downloads and inserts. The > main objectives are to greatly reduce the memory needed especially with large > numbers of queued downloads or uploads, and to instantly resume requests on > restarting the node. A common cause of high CPU usage is having more > downloads or inserts than you have enough memory for, causing the node to do > constant garbage collection trying to find the last byte to re-use, and > eventually crashing. And on a typical trunk node with lots of downloads > queued, it can take hours after startup before the node stops thrashing the > disk finding the blocks it has already downloaded. However, the background > level of disk I/O will be higher with db4o than the background level of disk > I/O on the current code (when the current code isn't resuming downloads). > > Both goals have been largely achieved. Right now, requests work, inserts work, > and freesite inserts work. After db4o has been finished, and merged, I will > set the default memory limit to 192MB, and not ask the user about it in the > wizard. Really big inserts (over 4GB) may require a memory limit over 192MB, > that is something we can solve later if necessary. > > Unfortunately, most recent work has been debugging and finishing architectural > changes to inserts, so the todo list is not that far from what it was last > time. However, there really isn't that much left now, I hope to merge the > db4o branch in the coming weeks: > - Fix database object leaks in inserts. > - Prefer recently successful requests in the scheduler. > - More debugging. > - Think about garbage collection for persistent temporary files. This will > only be needed to recover space left over due to bugs, so it's not clear > whether we need it ... > - Think about turning off the persistent blob tempfile, or implementing defrag > for it if necessary. This puts all the 32KB temp files into one big file; it > can get fragmented, causing the file to be larger than it needs to be; > implementing defrag would be a little messy but not difficult, but without > this big inserts are problematic on small memory nodes. > - Delete one or two structures using deprecated db4o collections. > - Backup the database automatically on a schedule, in case of severe problems. > - More debugging, merge it. > > FREETALK: > p0s has been building the Freetalk plugin (and the WoT plugin it is based on) > for some time. This is a web-of-trust based chat system similar to FMS. It > may or may not be compatible with FMS, debates over specifications are > ongoing. It will have a web interface, and be integrated into the node by > default, showing up on the menu. p0s has 2 months vacation which he is hoping > to use to finish Freetalk. Messaging is working, but trust list management is > missing at the moment. Because Freetalk depends on the WoT plugin, it will be > necessary to implement plugin versioning as well as auto-update of plugins > before we release 0.8. > > WINDOWS INSTALLER: > Our current installer does not work well on Windows. It has more screens than > are necessary and therefore confuses and annoys users. It doesn't work at all > on Vista for most users. And it doesn't work well with multiple installs. > Zero3 has been working on a new installer using AutoHotKey which solves all > of these problems. More testing would be a good idea; he has posted a URI to > the installer on several occasions. One caveat is that it does use some > closed-source freeware, however IIRC our current installer does too on > Windows. Also, I haven't used AHK, so reviewing the code is problematic. But > it promises to be a great improvement. It is currently designed as a bundle > installer, that is, it will need to be rebuilt manually on every release. I'm > not sure what the remaining issues are...? > > PROGRESS SCREEN ETC: > Because we got rid of the firefox profile, and for general usability reasons, > we need some sort of screen displayed when Freenet is fetching a freesite. > This will be implemented before 0.8. It should reduce the "it's gone off into > limbo" factor considerably for new nodes. Initially this will just be a 5 > second auto-refresh, but ideally it would be dynamically updated with > javascript, for faster response times and a better looking interface. We > might be able to lift the size limits on data sent to the browser at the same > time, which would be especially impressive if we have time to implement some > simple multimedia filters (mp3, ogg theora etc). Also, for the same reasons, > if a user is loading a page from freenet in his browser, with lots of inline > images, the browser will currently limit it to 8 connections and so it will > load very slowly. Ideally we would replace each image with a loading graphic, > and then replace it with the actual content when it is available. I have been > sent some javascript for this; if you have any ability in javascript, your > skills may be helpful. We will also need a loading screen for the search box. > > UI IMPROVEMENTS: > Ideally we would improve the user interface before 0.8. There are lots of > relatively small changes that could improve usability considerably, for > example moving geeky stuff to submenus. Whether we will have time remains to > be seen. > > LIBRARIAN IMPROVEMENTS: > The wAnnA? index, which is used by the search box on the homepage (aka > XMLLibrarian), was having some problems for a while due to the inability to > insert very large freesites (a librarian index is more or less a freesite > from Freenet's point of view). As far as I can tell, this has been fixed in > db4o, and hopefully also in stable build 1205. sdiz has been doing work on > internationalisation, code cleanups, and so on, on the XMLLibrarian plugin,
ON internationalisation: I am planning to use ICU4J, because of Thai language support. It use some data file from unicode.org (unicode data license[1]) and icu[2]. both license look BSD-like to me, but IANAL. The whole library is ~5.4M. If I remove the stuff I don't need, it is ~800k. It is needed for both the spider and the librarian. [1] The Exhibit 1: http://unicode.org/copyright.html#Exhibit1 [2] http://icu-project.org/repos/icu/icu/trunk/license.html > as well as his earlier major improvements to XMLSpider, which now uses the > Perst database to store its progress and minimise memory usage. IMHO we > should improve the UI of the XMLLibrarian before releasing 0.8. > > ZERO POINT NINE: > Various changes are tentatively planned for 0.9: > - A bulk vs quick flag. This would be set per request: bulk requests would be > optimised for throughput, quick requests optimised for latency. > - More work on the user interface. Much can be done to improve on the current > situation! > - Possibly increase the number of nodes faster nodes can connect to. > - Encrypted splitfiles: Encrypting each splitfile with a random key, unless > told not to, should greatly improve security against a distant but mobile > attacker trying to trace the author of specific content, at a negligible (in > most cases) performance cost. > - Encrypted tunnels: Based on rendezvous of several requests, which are either > random routed or partially random routed, these would greatly reduce the > impact of most of the major known attacks on Freenet, particularly those > related to tracing the author of content. They are superior to the older > premix routing proposal. However they would cost some hops. > - Bloom filter sharing: We will share our datastore bloom filter with our > peers, depending on our security level and so on. Given the previous item, > sharing our datastore filter with our direct peers is not a great risk. This > should reduce the average number of hops to find a block significantly. > > After 0.9, things that I'd like to see: > - Passive requests/long term requests. > - Better darknet support. Everything from easier ways to connect to your > friends to a more resilient swapping algorithm (we have ideas for both of > those). > - Transport plugins. Freenet over TCP, Freenet over HTTP etc. > - A more efficient transport layer. > - Better load management. > > SUMMER OF CODE: > Freenet will participate in Google Summer of Code 2009 if possible. > Organisations can apply from the 9th of March. Students can apply from the > 23rd of March. > > TOP FIVE USERVOICE SUGGESTIONS: > 1. Release the 20 nodes barrier. > This is marked as "under review", it may happen in 0.9. It requires some > alchemy/tweaking. :| > 2. One GUI for all. > This is partly a plea for better UI, partly a request for it to be possible to > do everything from within the web interface, partly advocacy for a dedicated > Freenet browser. We are not going to spend any resources on the latter for > the foreseeable future, although saces may come up with one. However, 0.8 > will have Freetalk, and already had the Thaw index browser. Hopefully in the > future it will have Freemail, and better filesharing support. > 3. Add a 'pause' feature. > This is planned, eventually. It would be most useful if we had a system tray > icon from which to turn it on and off when gaming. > 4. Show a progress screen when loading a page. > This is planned for 0.8, see above. > 5. Add an anonymous code repository via freenethg > This is a good idea, and should be more or less feasible at this point. > Whether it will work in practice, and whether anyone will use it, remains to > be seen. Also there are legal issues with accepting anonymous patches - if we > take a patch containing stolen code, and we can't trace the poster, we may be > held liable for it. Having said that, in the long term clearly we want to > encourage an anonymous development community for Freenet... > > OTHER STUFF: > Some idiot recently hired a botnet and spammed a bunch of IRC networks with > advertisements for us. saces is making some progress with wxFCP, which may > eventually yield a dedicated Freenet browser application. Vive is working on > a set of simulations for a paper about Freenet 0.7, and has committed the > simulator to SVN. > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >
