I've more or less decided to implement robustness externally. What this means is writing the access times (and for SSKs, the keys) to the store/cache file along with the data and headers, so that we can do a 100% reconstruction when necessary.
On Thursday 17 May 2007 00:31, Florent Daigni?re wrote: > * Chris Carlin <carlin at jlab.org> [2007-05-16 13:42:33]: > > > So our options are: > > > 1) Open the block numbers database with sorted duplicates enabled. Then > > > scan through it for dupes, keep the correct one in each case, close the > > > database, and re-open it again with sorted duplicates disabled. > > > 2) Keep the block numbers index open with sorted duplicates enabled. > > > When we need to look up a block number, deal with the fact that there > > > may be multiple keys referring to it, and delete as appropriate. Deal > > > with the fact that this may cause keys in the main database that don't > > > exist in the store. 3) Improve the data stored on disk: Store the LRU > > > access time, and the key, on disk. Probably we would need migration > > > code. > > > 4) Use a completely different database for the index. > > > 5) Use a completely different database for the whole store, and trust > > > it not to lose the data. (from our experience with BDB, it is likely > > > that it will!) 6) Roll our own database code. > > > > It seems to me that BDB has been rejected as the proper solution > > multiple times on this mailing list. Maybe I misunderstood, but in > > particular I remember when I was going to follow up with Oracle about > > memory usage issues but stopped when consensus seemed to find that BDB > > wasn't worth pursuing for various other reasons. > > > > ~Chris > > We weren't using it properly, hence the high memory usage we noticed... > Now toad is willing to switch to something else because of the > "robustness" criteria he has defined (ability for the database to > auto-repair itself and perform catastrophic recovery proceedure in an > automated way without involving the user). I do think that we are > looking for a rara avis here :/ > > NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20070518/8b35dcf8/attachment.pgp>
