On Sunday 02 Sep 2012 17:51:49 xor wrote:
> On Thursday 30 August 2012 00:40:13 Matthew Toseland wrote:
> > Sadly Freetalk/WoT do use rollback so has to
> > commit EVERY TIME. 
> I oppose to the "sadly". Transaction-based programming has proven to be a 
> valid approach to solve many issues of traditional "manually undo everything 
> upon error"-programming.

Lets get one thing clear to begin with: I am not advocating that Freetalk/WoT 
aggregate transactions by simply not committing. The basic design flaw in 
Fred's use of the database layer was to assume that object databases work as 
advertised and you can simply take a big complex in-memory structure and 
persist it more or less transparently, and then add a load of (de)activation 
code and reduce memory usage.

Freetalk and WoT may be better designed on that level. However they produce 
more disk I/O. And the reason for this is, mainstream database practice and 
theory are designed for two cases:
1. Lots of data (absolute amount and transactions/sec) on fast, reliable 
hardware with professional sysadmins.
2. Tiny amounts of data on commodity hardware.

If you have lots of data on commodity disks, it falls down. And note that 
mainstream use of databases in the second case frequently fails. For example, I 
have a huge set of bookmarks in a tree on one of my browsers. Every time I 
create a new category, another one gets renamed.

> The approach of fred to not use it is just wrong from a computer-science 
> perspective.
> The fact that it does not perform so well is an implementation issue of the 
> database, not of the client code which uses the database.

No it is not. The load you put on it requires many seeks.

Of course, it might require fewer seeks if it was using a well-designed SQL 
schema rather than trying to store objects. And I'm only talking about writes 
here; obviously if everything doesn't fit in memory you need to do seeks on 
read as well, and they can be very involved given db4o's lack of two-column 
indexes. However, because of the constant fsync's, we still needs loads of 
seeks *even if the whole thing fits in the OS disk cache*!
> Also, Freetalk/WoT themselves are CPU/IO hogs due to them using the wrong 
> algorithms, so we are not even at the point where we can tell whether their 
> ACID-usage is a problem. There is too much noise generated from the highly 
> inefficient algorithms which are being used by them, we cannot measure the 
> performance impact of commit/rollback.

That's possible.

However, the bottom line is if you have to commit every few seconds, you have 
to fsync every few seconds. IMHO avoiding that problem, for instance by turning 
off fsync and making periodic backups, would dramatically improve performance.

As regards Freenet I am leaning towards a hand-coded on-disk structure. We 
don't use queries anyway, and mostly we don't need them; most of the data could 
be handled as a series of flat-files, and it would be far more robust than 
db4o, and likely faster too. But the first thing to do is upgrade the database 
and see whether it 1) breaks even worse or 2) fixes the problems. Past testing 
suggests #1, but past testing was based on later versions of 7.4, not on the 
latest 7.12.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.

Reply via email to