On Fri, May 22, 2009 at 1:44 PM, Chris Anderson <[email protected]> wrote: > On Fri, May 22, 2009 at 1:26 PM, Damien Katz <[email protected]> wrote: >> Chris Anderson has done some work here with the batch feature for similar >> efficiency reasons, except that his feature is to buffer the docs into >> memory and return success to the client immediately, it's for things where >> losing some messages is acceptable (like logging), so long as the >> transactions are fast and efficient as possible. You instead want the client >> to wait and get the success or fail. it seems the batch work is where to >> build this into. >> > > Exactly. It'd be a "trivial" change to the batch=ok feature to add > wait=true so that the client hangs open on the connection and receives > a 201 Created when the batch is committed. It'd actually be sort of a > challenging patch, but I can perhaps add it before 1.0. > > I've added it to Jira here: https://issues.apache.org/jira/browse/COUCHDB-356 >
Forgot to mention that you can almost get this now by using batch=ok and then when the client wants to be sure that the batch is committed, send a POST to /db/_ensure_full_commit You'll have a little more http overhead this way but the semantics are mostly the same. > Chris > >> -Damien >> >> >> On May 22, 2009, at 4:08 AM, Brian Candler wrote: >> >>> On Wed, May 20, 2009 at 11:48:10AM -0400, Damien Katz wrote: >>>>> >>>>> As for timing of fsync: ideally what I would like is for each write >>>>> operation to return some sort of checkpoint tag (which could just be >>>>> the >>>>> current file size). Then have a new HTTP operation "wait for sync >>>>> <tag>" >>>>> which blocks until the file has been fsync'd at or after that >>>>> position. >>>> >>>> Just use the x-couch-full-commit=true http header. >>> >>> That's not what I want (unless I've misunderstood what this header does). >>> >>> Suppose I have 100 concurrent client connections, each client is >>> committing >>> one document at roughly the same time. I don't one 100 fsync's; I want >>> each >>> client to be able to write a doc, and then be notified when the next >>> convenient fsync takes place, so they know their doc is on persistent >>> storage. >>> >>> Setting the default commit interval down to (say) 200ms would then give >>> clients a reasonably low latency for getting their docs committed to disk, >>> without killing performance by fsyncing every write. >>> >>> Otherwise, the only solution I can see is a batching proxy: it takes PUT >>> and >>> POST requests, builds them into a single _bulk_docs, and submits that >>> periodically with x-couch-full-commit:true before responding. But that >>> gets >>> very awkward when dealing with multiple databases, and/or multiple users >>> with different credentials. >>> >>> Regards, >>> >>> Brian. >> >> > > > > -- > Chris Anderson > http://jchrisa.net > http://couch.io > -- Chris Anderson http://jchrisa.net http://couch.io
