Very interesting! So how does it behave during conflict situations? Raft selects a winning WAL write and any others in flight are aborted?
And when not enough nodes are available, writes are hung until consensus? I won't be able to use it due to Go but it's great to know that this is on the horizon of possibilities… Very nice! On Sat, Aug 19, 2017 at 10:47 AM Free Ekanayaka <f...@ekanayaka.io> wrote: > Hi, > > first of all let me thank Richard Hipp and the rest of the SQLite team > and community for such a great piece of software. I've been recently > working with SQLite's code base and found it an absolute pleasure to > read (and have learned from it too). > > In this mail I'd like to: > > 1) Present dqlite, a library replicating your application's SQLite > database across N nodes and safely surviving any minority of them > dying or disconnecting (only for Go applications for now, see below). > > 2) Submit a patch to SQLite that introduces a minimal replication API, > and get feedback about its possible inclusion upstream. > > = dqlite = > > It's a Go package that uses the Raft algorithm to replicate SQLite WAL > frames across a cluster of nodes. This roughly means that you can open a > SQLite connection using the "database/sql" standard lib API and have > anything you transactionally replicated. No external process needed. > > Ideally this library should have been written in C or Rust, to support > binding to any language. However, due to the use case and timeline of > the first project that will use it (LXD [0]), and due to the lack of > mature Raft implementations in C/Rust, Go was chosen instead. It should > hopefully at least serve as reference to anyone needing a C/Rust > version. > > The work has been funded by Canonical, the company behind Ubuntu. Please > see the dqlite's home page [1] for more details. > > = SQLite replication API patch = > > This is the SQLite patch that dqlite depends on. It essentially adds a > few key hooks in the pager and write-ahead log to let external libraries > like dqlite implement WAL-based database replication. > > As you'll quickly see, it's by no means ready for upstream inclusion, in > particular it lacks unit tests and more comprehensive documentation > comments (note however that virtually every code path introduced by the > patch is already exercised indirectly by dqlite's own unit tests). > > If the SQLite team thinks there is room for upstream inclusion, I'll be > more than glad to do the necessary work to make the patch adhere to > SQLite's standards and go through a review process. > > The patch has currently 703 additions and 22 deletions, and is published > on GitHub [2]. > > Cheers, > > Free > > [0] https://linuxcontainers.org/ > [1] https://github.com/CanonicalLtd/dqlite > [2] > https://github.com/CanonicalLtd/sqlite/commit/2a9aa8b056f37ae05f38835182a2856ffc95aee4 > _______________________________________________ > sqlite-users mailing list > sqlite-users@mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
