There is a list below of various features that may or may not be necessary for 0.8.0. Most are relatively easy. This is copied from my flog. There are broader questions too...
The most serious question not covered below is load management. Every time we
have something go wrong (a mandatory build, accidentally changing insert
healing to do a lot more inserts as happened recently, etc) the network goes
bananas for a week or more. It also means it doesn't cope well with lots of
newbies (although this has improved a lot) or with darknet, and reduces
security slightly. Also it seems very likely that poor data persistence is
closely related to poor load management. I am fairly sure I know what needs to
be done to implement a new, much better load management system, but it would be
a relatively large task (probably weeks) and might cause serious disruption for
a while.
In the near future a period of stabilisation is probably necessary. I have
tried to fix all the remaining issues with the cooldown queue in 1273. The
network was disrupted severely by the healing foulup but is IMHO improving
rapidly.
And of course the broader question is whether we want to prioritise getting
0.8.0 out very soon, whether we need to avoid big changes, etc. Some would
argue for a feature freeze right now. IMHO some of the below items are
essential e.g. Freetalk, new wininstaller, auto-backup. Sorting out disk
encryption is a prerequisite for auto-backups. Some (evanbd, filesharers) would
like us to include partial reinsert support, and some would argue that we
should sort out data persistence before 0.8 - after all, if we can improve data
half-lives by a factor of 3-5, and given that right now they are quite poor,
isn't that rather important? There is an argument for sorting out load
management - it clearly sucks right now, although if it is given time to settle
it is adequate for the average use case. If we sorted out the Pitch Black
attack we could release a paper at the same time as 0.8 - but that depends on
Oskar or Vive getting some credible attack simulations done. And some would
even argue for full blown WoT-based filesharing as a prerequisite for release -
"Write a killer file-sharing application" is the top suggestion on Uservoice by
a fair margin, and a minimal implementation would seem to be feasible in weeks.
So we should probably have a discussion about timing and priorities. Predicated
on the assumption that the Google money comes through (which it surely will, an
approved purchase order is about as final as things get). So there is no
immediate financial pressure - but it's been a long time since 0.7.5 let alone
0.7.
* Sort out disk encryption: Currently we encrypt the client database
(node.db4o) if the physical security level is set to NORMAL, HIGH or MAXIMUM.
We should make it much clearer that you should install Truecrypt and encrypt
your full system (especially swap), but at the same time, it is not acceptable
that casual, lazy or computer illiterate users have no protection, especially
as if they are only browsing HTML or forums we actually protect them reasonably
well apart from the swap issue. Anyway, the problem is node.db4o is encrypted
using CTR mode, which isn't good with journaling and block remapping, but fails
spectacularly when we implement the next item; I plan to port the GF128 code
from the kernel and use XEX/XTS.
* Related physical security stuff: There are various minor things -
encrypting plugin databases (easy), keeping the bookmarks, the list of recently
completed downloads, various plugin data in the database (all easy but there
are several items). None of this is needed if we pull all the crypto code out
but I don't think that makes sense.
* Auto-backups and related stuff: Sooner or later node.db4o gets corrupted.
This is a fact of life, especially on flaky commodity hardware with unclean
shutdown etc etc. A solution to this is to regularly make copies of the
node.db4o; db4o provides a mechanism for this, and although it isn't quite
"online", it's close enough. A serious difficulty is that the node.db4o backup
may refer to temporary files or temporary buckets in the persistent-blob file,
so we need to ensure that we don't delete/reuse them until after the next
backup. However, once we have this, aggressively caching node.db4o to limit the
number of disk writes, including possibly not writing it at all for some time
and then writing it all at once every few minutes (provided it is relatively
small relative to available memory) become both very attractive and easy to
implement.
* Freetalk: Not something I can give a whole lot of help on, although there
are some things such as encrypting the database that I need to deal with. In
any case, p0s is getting very close to releasing it as the first quasi-official
version, available from the plugins list.
* New Windows installer: Zero3 has a new alpha of the wininstaller, which
hopefully will fix the service errors. This version installs to the user
installing and starts on login via a normal startup link, rather than
installing as a service. It appears that the permissions problems that cause
the service not started errors are related to running as a service. Anyway,
when we have had a few more successful test reports, and Zero3 gets around to
writing an update script, this will become the new official wininstaller.
* Localisation in plugins: This is important, and fairly easy.
* GSoC: sajack's work on filters: Spencer (sajack) has written us a range
of infrastructure improvements for filtering, some of which was merged a while
back, more will be merged in 1272, possibly including support for filtering
inserts (e.g. stripping out EXIF in JPEGs, detecting charsets on insert). He
has also written filters for ogg vorbis, ogg theora, and mp3, and the
beginnings of embedded HTML5 video/audio tags support (dependant on
web-pushing). Hopefully we will merge the infrastructure and the new filters;
web-pushing won't be on by default for some time unless somebody decides to fix
it, it's not a high priority as far as I'm concerned... Another thing we could
do on filters is merge the ATOM and SVG filters from last year.
* GSoC: zidel's new packet format: This should improve efficiency
(especially payload %) significantly, allow Freenet to work on lower MTUs, and
pave the way for transport plugins. It may or may not go in before 0.8.0. The
code is generally of good quality and has many unit tests at various levels.
* Data persistence work: I still haven't written the new block insert
tester, which compares data persistence with various insert tweaks recently
coded. This will hopefully confirm our theories as to why a block inserted once
is 75% retrievable after a week but the same block inserted 3 times in rapid
succession is 95% retrievable, even though it is apparently routed the same
each time. Which hopefully will result in dramatic improvements to data
persistence.
* Find a new host for the uninstall survey: It was on Google Docs but has
been hopelessly broken for a year. Maybe we can fix it, or find some other
host...
* Consider hierarchical USKs: We have the insert-time support; maybe it's
worth thinking about the request-time support.
* Think seriously about Library: We currently search both the old and new
wanna. The old one takes quite a while because it has to transfer a lot of
data; the new one is faster but fails relatively often, and is surprisingly
slow. We will probably also search gogo, but it may be necessary to make some
format improvements (e.g. putting the metadata for each layer into the layer
above).
* Finish the Mac installer changes, and possibly Linux system tray applet:
We still get reports of Freenet not working after restart on Mac's, and the
system tray applet and memory autodetection don't work. If possible it would be
really good to fix this before release, as e.g. a lot of journalists use Mac's!
* Memory autodetection tweaks: This is trivial but needs to be done.
* Auto-updater key change support: Murphy's Law dictates that sooner or
later we will need to change the update keys. It would be best to have a smooth
upgrade path sorted out in advance. This is likely a couple days work.
* Basic RSKs: Still worth considering, would allow an official project
freesite and useful for hostile environments.
* Native FEC: We need to decide what to do about this. It might be best to
leave it out given the problems we've had, but if somebody trustworthy and
JNI-aware reviews the code carefully, maybe we should restore it for better
performance especially on slow CPUs.
* Easy full reinserts: This is probably a good idea, although it is no
safer than inserting as a CHK...
* Various work on usability and UI: For example, we need the queue page to
have checkboxes for easy bulk removal. We probably want to move the bookmarks
to a menu, and use the minimalist theme. Etc.
* Lots of bugfixes!
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Devl mailing list [email protected] http://freenetproject.org/cgi-bin/mailman/listinfo/devl
