On Sunday 02 Sep 2012 17:51:49 xor wrote:
> On Thursday 30 August 2012 00:40:13 Matthew Toseland wrote:
> > Sadly Freetalk/WoT do use rollback so has to
> > commit EVERY TIME.
> I oppose to the "sadly". Transaction-based programming has proven to be a
> valid approach to solve many issues of traditional "manually undo everything
> upon error"-programming.
Lets get one thing clear to begin with: I am not advocating that Freetalk/WoT
aggregate transactions by simply not committing. The basic design flaw in
Fred's use of the database layer was to assume that object databases work as
advertised and you can simply take a big complex in-memory structure and
persist it more or less transparently, and then add a load of (de)activation
code and reduce memory usage.
Freetalk and WoT may be better designed on that level. However they produce
more disk I/O. And the reason for this is, mainstream database practice and
theory are designed for two cases:
1. Lots of data (absolute amount and transactions/sec) on fast, reliable
hardware with professional sysadmins.
2. Tiny amounts of data on commodity hardware.
If you have lots of data on commodity disks, it falls down. And note that
mainstream use of databases in the second case frequently fails. For example, I
have a huge set of bookmarks in a tree on one of my browsers. Every time I
create a new category, another one gets renamed.
> The approach of fred to not use it is just wrong from a computer-science
> The fact that it does not perform so well is an implementation issue of the
> database, not of the client code which uses the database.
No it is not. The load you put on it requires many seeks.
Of course, it might require fewer seeks if it was using a well-designed SQL
schema rather than trying to store objects. And I'm only talking about writes
here; obviously if everything doesn't fit in memory you need to do seeks on
read as well, and they can be very involved given db4o's lack of two-column
indexes. However, because of the constant fsync's, we still needs loads of
seeks *even if the whole thing fits in the OS disk cache*!
> Also, Freetalk/WoT themselves are CPU/IO hogs due to them using the wrong
> algorithms, so we are not even at the point where we can tell whether their
> ACID-usage is a problem. There is too much noise generated from the highly
> inefficient algorithms which are being used by them, we cannot measure the
> performance impact of commit/rollback.
However, the bottom line is if you have to commit every few seconds, you have
to fsync every few seconds. IMHO avoiding that problem, for instance by turning
off fsync and making periodic backups, would dramatically improve performance.
As regards Freenet I am leaning towards a hand-coded on-disk structure. We
don't use queries anyway, and mostly we don't need them; most of the data could
be handled as a series of flat-files, and it would be far more robust than
db4o, and likely faster too. But the first thing to do is upgrade the database
and see whether it 1) breaks even worse or 2) fixes the problems. Past testing
suggests #1, but past testing was based on later versions of 7.4, not on the
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 198 bytes
Desc: This is a digitally signed message part.