Matthew Toseland wrote:

Everything sounds great, and I really think that freetalk may make 0.8 a 
real hit.

Thanks for your hard work.

> BUILD 1240
> Our last stable build, 1239, was in November. We have just released a new
> one, 1240. This has many changes (opennet stuff, optimisations, all sorts
> of stuff), which I list in the mail about it. One of the most important is
> that there are several new seednodes, and many dead ones have been
> removed. I have tested it 3 times today and it's bootstrapped fast each
> time, although yesterday it bootstrapped very slowly one time.
> Evan Daniel has been doing some useful work analysing the network. Amongst
> other things, he has discovered that: - The Guardian article, in December,
> which was reprinted around the world, has more than doubled the size of
> our network, although there is a slight downward trend now. This may be
> due to seednodes issues and not having had a build since November. - We
> have around 4500-7000 nodes online at any given time. - Over 5 days, we
> have around 14000 non-transient nodes. - For nodes online at any one time,
> roughly 37% are 24x7 nodes (96% uptime average), 33% are regular users
> (56% average uptime), and 30% are occasional or newbie nodes (16% average
> uptime).
> We have finally gotten rid of emu! Our faithful and powerful dedicated
> server supplied at a discount by Bytemark is no more. We now have a
> virtual machine called Osprey, which does most of the same job, for a much
> lower cost, and has a much simplified setup so should be easier to
> maintain. We have tried to outsource services, for example we use Google
> Code for our downloads, but some things will have to stay under our direct
> control for some time to come e.g. mailing lists and the bug tracker.
> You may have some difficulty with the update scripts, if you use
> / update.cmd. If it doesn't work, try updating the script manually from
> (or
> Xor (also known as p0s) continues to work on the Web of Trust and Freetalk
> plugins. These are approaching the point where we can make them loadable
> from the plugins page, and then bundle them, enabled by default.
> WoT is the backend system which implements a pseudonymous web of trust,
> which functions in a similar way to that in FMS. You can create
> identities, assign trust to other identities, announce your identity via
> CAPTCHAs and so on. This is the Community menu, from which you can see
> your identities and other people's, and the trust relationships between
> them. WoT is used by Freetalk, FlogHelper, and probably soon by
> distributed searching, real time chat and other things.
> Freetalk is a spam-resistant chat system based on WoT. This is similar to
> FMS, but it will eventually be bundled with Freenet, and will be a part of
> it by default. You will be able to embed a Freetalk board on your
> freesite. FlogHelper is a WoT-based plugin for writing a flog (freenet
> blog), which is very easy to use, but uses WoT to manage identities. I
> would have bundled FlogHelper months ago, but WoT isn't ready yet and
> FlogHelper needs it.
> WoT should be ready soon. Recently a major issue has been discovered with
> the trust calculation algorithm, after that is fixed and some minor
> issues, WoT will become a semi-official plugin, which will sadly require
> flushing the existing "testing" web of trust, so sadly all old messages
> and identities will go away. Freetalk needs more work, about 50% of the
> bugs marked for 0.1 on the roadmap are fixed at the moment.
> In build 1240, we pull in a new version of Library. This is a great
> improvement over the old version, it is faster, it supports embedding a
> search on a freesite, and has many bugs fixed. However searching for
> common terms can still cause out of memory crashes.
> There is another issue with Library: infinity0 spent last summer creating
> a scalable index format for Library, which should make it a lot easier to
> insert and maintain big indexes. We will soon change the spider to use
> this new format, and in the process we expect to greatly improve
> performance for writing indexes, so it doesn't take a week any more and is
> done incrementally. I realise this has been promised before, but it is
> important, so it will happen sooner or later, hopefully sooner.
> Full Web of Trust-based distributed searching, with a focus on
> filesharing, is on the distant horizon at the moment. infinity0 might be
> able to do some work on it as part of his studies, we'll see. It won't be
> in 0.8.0.
> We would like to get 0.8 out soon, or at least a beta of 0.8. Several
> major issues: - The windows installer needs to be fixed on 64-bit. This is
> being worked on. - Freetalk must be ready.
> - Auto-configuration of memory limits in the installers, and asking the
> user about memory usage (at least in some cases) is relatively easy and
> important, but not vital. - Substantial improvements to opennet,
> particularly making nodes announce onto the network and get where they
> should be as quickly as possible. - Substantial improvements to data
> persistence. We have done much here already but there is more to do. -
> Library must work well and fast out of the box. This means amongst other
> things the new spider mentioned above. - MANY BUG FIXES! The first beta
> does not need to be perfect, but there are some critical issues that need
> dealing with, such as the fact that nodes often don't resume properly
> after being suspended for a while.
> Please test Freenet, and report any bugs and usability issues you find on
> the bug tracker ( ) or via Freetalk board
> en.freenet (note that this will be wiped soon so if after a new Freetalk
> release it is wiped you may need to resend).
> We have many ideas on how to improve opennet bootstrapping (make nodes
> assimilate into the network more quickly), and to improve opennet
> generally. Some of these are implemented in 1240, including many bugfixes.
> More will be put out over time so we can see their impact. Improving
> opennet should improve performance for the majority of users who don't run
> 24x7 and it should improve performance for everyone else too, as those
> nodes will get connected and start doing useful work more quickly.
> We have many ideas on how to improve data persistence. There is a lot of
> capacity on the network, yet data seems to become inaccessible quite
> quickly (stats below). I am convinced that improving data persistence will
> improve Freenet's usability and perceived performance immensely. The
> continued popularity of insert on demand on uservoice demonstrates this as
> much as anything: People want a system that works! IMHO we can greatly
> improve things without resorting to insert on demand, although filesharing
> clients based on distributed searching may eventually offer it (but there
> are serious security issues with insert on demand).
> Evan is convinced that mostly poor data persistence is not due to data
> falling out of stores, but due to the small number of nodes that stored
> the data (as opposed to caching it) going offline or becoming unreachable.
> We have increased the number of nodes that store data, we have made the
> node use the store for caching if there is free space, we have done
> various things aimed at improving data persistence, and there is much more
> we can do. An immediate question is whether the security improvements
> gained last year by not caching at high HTL have broken many inserts by
> making them not get cached on the right nodes; we will test this in 1241.
> A related question is why inserting the same key 3 times gives such a huge
> performance gain relative to inserting it once; we will investigate this
> soon after. We will probably triple-insert the top blocks of splitfiles
> soonish, but the bigger prize is to achieve the 90%+ success after a week
> that we see with triple-insertion of a single block, and this may well be
> possible with some changes to how inserts work...
> Finally, the redundancy in the client layer could be a lot smarter: We
> divide files up into groups of 128 blocks, called segments, and then add
> another 128 "check blocks" for redundancy. Unfortunately this means that
> sometimes the last segment only has 1 block and 1 check block, and so is
> much less reliable than the rest of the splitfile. We will fix this.
> We have been collecting statistics on data retrievability over time. The
> below are "worst case" in that they relate to single CHK blocks, with no
> retries. Real life, with many retries (at least 2 for a direct fetch and
> more if the file is queued), and with large, redundant splitfiles, should
> be substantially better than these numbers. Every day we insert 32 blocks
> and fetch a bunch of 32 blocks from 1 day ago, 3 days ago, 7 days ago,
> etc. There are two of these running to get more data, so I am just showing
> both results here. The percentages are the proportion of the original
> insert that is still retrievable:
> 1 day 76% / 77%
> 3 days        66% / 70%
> 7 days        60% / 61%
> 15 days       48% / 48%
> 31 days       36% / 33%
> 63 days       21% / 19%
> Now, here's an interesting one. In each case we insert a 64KB CHK
> splitfile - that is, one block at the top and four underneath it. We
> insert one three times, and we insert three different ones once each. We
> then pull them after a week. We can therefore compare success rates for a
> single block inserted once, a single block inserted 3 times, and a
> simulated MHK, that is, a block which has been re-encoded into 3 blocks so
> that we fetch all of them and if any of them succeeds we can regenerate
> the others.
> Total attempts where insert succeeded and fetch executed: 63
> Single keys succeeded: 61
> MHKs succeeded: 58
> Single key individual fetches: 189
> Single key individual fetches succeeded: 141
> Success rate for individual keys (from MHK inserts): 0.746031746031746
> Success rate for the single key triple inserted: 0.9682539682539683
> Success rate for the MHK (success = any of the 3 different keys worked):
> 0.9206349206349206
> Ian's friend pupok is working on a new AJAXy user interface mockup for
> Freenet. sashee's web-pushing branch, which makes the user interface a lot
> more dynamic without making it look much difference, should be merged
> soon, but turned off by default, since it has some nasty bugs. When it is
> turned on, it solves the age-old parallel connections bug, showing
> individual progress for each image without hogging your browser's limited
> number of connections (6 or 8 on modern browsers). Both of these may miss
> 0.8.
> More broadly on usability, usability testing is always welcome: Persuade a
> friend to install Freenet, watch them do it, don't help them unless they
> get really stuck, report any problems they have or any comments they make
> about how it could be better.

Support mailing list
Unsubscribe at

Reply via email to