On 22 May 2009, at 10:08, Brian Candler wrote:
On Wed, May 20, 2009 at 11:48:10AM -0400, Damien Katz wrote:
As for timing of fsync: ideally what I would like is for each write
operation to return some sort of checkpoint tag (which could just be
the
current file size). Then have a new HTTP operation "wait for sync
<tag>"
which blocks until the file has been fsync'd at or after that
position.
Just use the x-couch-full-commit=true http header.
That's not what I want (unless I've misunderstood what this header
does).
Suppose I have 100 concurrent client connections, each client is
committing
one document at roughly the same time. I don't one 100 fsync's; I
want each
client to be able to write a doc, and then be notified when the next
convenient fsync takes place, so they know their doc is on persistent
storage.
Setting the default commit interval down to (say) 200ms would then
give
clients a reasonably low latency for getting their docs committed to
disk,
without killing performance by fsyncing every write.
Otherwise, the only solution I can see is a batching proxy: it takes
PUT and
POST requests, builds them into a single _bulk_docs, and submits that
periodically with x-couch-full-commit:true before responding. But
that gets
very awkward when dealing with multiple databases, and/or multiple
users
with different credentials.
I think we could do a DbUpdateNotification that sends you a
`{"db":"dbname",
"type":"fsync"}` notification. Of course, this is already legacy API,
we could
add this to the upcoming `/_changes` API as well. Say we add a
`"last_fsync":<timestamp>` to the `/dbname` call (the db info), and
let you
subscribe to changes to that property. Details are blurry, because I
haven't
looked at `_changes` yet, but I think we can add that.
Cheers
Jan
--
