Ian Eslick wrote: > >> I'm a bit surprised about this. Most relational databases use >> journalling and / or rollback files to ensure that they can commit or >> rollback large transactions as one atomic unit. The procedure is quite >> neatly described in http://www.sqlite.org/atomiccommit.html and >> http://www.sqlite.org/tempfiles.html, but works AFAIK more or less the >> same on other RDMSs. > > Actually, as I understand it, SQlite makes all the edits in user > memory rather than in a shared memory region so it's just easier to > allocate the memory in that case. The tradeoff as you point out is > serializing side-effecting transactions at the whole-DB level. > Actually if you had shared locks you could probably still pull of the > same rough model... > > BDB is tuned for highly concurrent, but independent, write > transactions across multiple threads/processes. The shared memory > region is used for sharing the fine-grained locks as well as for a > shared cache so each process doesn't have to separately allocate > memory and that you can have processes with various levels of read > isolation. This unfortunately leads to the requirement to > pre-allocate that shared memory region and your current dilemma. > that sounds like the key for this specific mismatch of the BDB locking philosophy – which for the scenarios you describe is perfect – and our quite different needs. We can (well, have to) live with these restrictions for now. However, if there ever were a new lisp backend, I don't think it'd necessarily have to have precisely the same constraints. >> I've just tried it out with sqlite which quite happily handles a >> transaction of 10 million inserts as an atomic unit both for commit and >> rollback – to be sure, in this case at the price of an extended read / >> write lock on the DB and, of course, not in one physical write >> operation. Main memory hardly enters the equation there. > > Yep, this is definitely a sweet spot for SQLite. > > We actually might be able to configure BDB to behave similarly (tune > it for single-writer, multiple-readers) as an optional setup to > Elephant; but we'd have to look into this a bit as I'm not entirely > sure BDB can meet your constraints here. I'm a little stale on all > this these days... > that'd be great, of course.
Best regards, Marc _______________________________________________ elephant-devel site list elephant-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel
