Jens,
This sounds amazing. I can't wait to play with the next build. Any comments on the size of the distribution or increase in build complexity? . On Thu, Apr 24, 2014 at 4:16 PM, Jens Alfke <[email protected]> wrote: > On Apr 24, 2014, at 11:01 AM, Matt Quinn <[email protected]> wrote: >> Is there any more information you're ready to share publicly about ForestDB? > I just checked with the team; they want to keep the source private until > they've implemented the remaining features (probably June) but it's OK to > talk about it. > ForestDB is a persistent key-value storage library, roughly similar in > features to Tokyo Cabinet or Berkeley DB. It's a key-to-value map where the > keys and values are binary blobs. You can get and set values and enumerate > the keys in (lexicographic) order. It also has a parallel by-sequence index > that records the order in which values were updated. > The data format is, at a high level, somewhat like CouchDB's (or Couchbase > Server's.) It's strictly append-only: new values are written at the end of > the file, and any modified b-tree nodes are also written to the end, and then > finally a header [sic] is added to mark a commit. This type of file is less > space-efficient than a traditional database but extremely robust and supports > very fast write speeds. It also has the nice property that writing to the > file doesn't interrupt readers; in fact a reader can keep using its own > snapshot of the database as long as it wants to. > (If you want to look at this sort of thing in more detail today, the closest > equivalent is the CouchStore library that Couchbase Server uses. It's a > re-implementation of CouchDB's storage manager, ported from Erlang to C for > speed and then modified further from there.) > What's new in ForestDB is a clever data structure that's a combination of a > b-tree and a trie. This makes the tree nodes a lot more compact and speeds up > lookups. It's also careful about how it aligns its data with the filesystem's > page size, which supposedly makes it more efficient on SSDs, and it's got an > efficient in-memory page cache. > I've been working part-time for a month or so on moving Couchbase Lite/iOS on > top of ForestDB. I've got it running and passing the unit tests. Some basic > benchmarks show it from 50% to 300% faster on various tasks (the biggest > speedup is in view queries.) I'm working on adding more concurrency using GCD > which should speed it up even more. While I'm doing this I'm also changing > the way documents are stored; instead of every revision being a separate > database row, the entire document is stored together, with the revisions > encoded inside it. That gives better locality-of-reference and simplifies the > code too. (Also, the new document/revision storage is written in C so it can > be reused on Android.) > This is still an investigation; so no promises and no timelines. But it's > looking very good so far. > --Jens > -- > You received this message because you are subscribed to the Google Groups > "Couchbase Mobile" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/mobile-couchbase/8C271242-B4E7-46D8-9EAB-1C1717D27321%40couchbase.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Couchbase Mobile" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/1398395056310.9db71fe6%40Nodemailer. For more options, visit https://groups.google.com/d/optout.
