[
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151314#comment-13151314
]
Paul Joseph Davis commented on COUCHDB-1342:
--------------------------------------------
This patch looks like its a mishmash of a couple patches that are applied
slightly differently. Not sure if that's a branch thing or what. But for
instance, there's a lot of id_btree(Db) -> Db#db.by_docid_btree type changes.
There's also a switch between couch_file:sync(Filepath) and
couch_file:sync(Fd). I'm not sure if that's on purpose or not because we
switched to Filepath on purpose. I see it as being possible but I don't see a
reference to it.
I'm really not a fan of exposing the inner workings of couch_file to every
client as can be seen by the massive number of flush calls that this creates. I
find it quite likely that we'll eventually end up missing one of these flush
calls and causing a nasty bug because the error would look like data hadn't
been written. I'd want to see things rewritten so that the usage of couch_file
doesn't change. Easiest solution I see would be to automatically flush buffers
if a read request comes in for something that's not on disk.
I'm pretty sure we talked about this at one point, but can someone describe the
differences between this and Erlang's delayed_write mode for files?
couch_file becomes much more complicated with this patch. I'd prefer to see it
simplified if at all possible. I'm not entirely certain how it'd look but I
might start by making a couch_file_reader and couch_file_writer gen_servers
instead of having bare processes in couch_file.
I have to say that this patch scares me a bit. If we make the switch to
something like this then we're opening up a whole new type of failure scenario
where file "writes" appear to succeed but then end up failing after the caller
has moved on. While there are certainly cases where I can see this being a fine
tradeoff (view updaters, compactors, etc) it worries me a bit for the main
database file. The fact is that I can't (yet) reasonably consider all of the
possible new failure modes and convince myself that things are correct. I've
already seen some super weird reactions to things like running out of disk
space, it seems like not knowing about such an error until you've "written" a
megabyte of data seems awkward.
> Asynchronous file writes
> ------------------------
>
> Key: COUCHDB-1342
> URL: https://issues.apache.org/jira/browse/COUCHDB-1342
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Reporter: Jan Lehnardt
> Fix For: 1.3
>
> Attachments: COUCHDB-1342.patch
>
>
> This change updates the file module so that it can do
> asynchronous writes. Basically it replies immediately
> to process asking to write something to the file, with
> the position where the chunks will be written to the
> file, while a dedicated child process keeps collecting
> chunks and write them to the file (and batching them
> when possible). After issuing a series of write request
> to the file module, the caller can call its 'flush'
> function which will block the caller until all the
> chunks it requested to write are effectively written
> to the file.
> This maximizes the IO subsystem, as for example, while
> the updater is traversing and modifying the btrees and
> doing CPU bound tasks, the writes are happening in
> parallel.
> Originally described at http://s.apache.org/TVu
> Github Commit:
> https://github.com/fdmanana/couchdb/commit/e82a673f119b82dddf674ac2e6233cd78c123554
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
