On Mar 4, 2009, at 8:34 PM, Adam Kocoloski wrote:
I was working through some replication tickets this week and
thinking more and more that the replicator could benefit from being
restructured along OTP principles. So, I went ahead and did it.
Here's the structure I've worked out so far:
http://github.com/kocolosk/couchdb/tree/otpify-replication
* couch_primary_services gets a new child
(couch_replication_supervisor) which is itself a supervisor using a
one_for_one strategy.
* Replications are modeled as gen_servers and added as transient
children of couch_rep_sup. couch_rep_sup ensures that identical
replications do not run simultaneously.
* The gen_server spawn_links a separate process that enumerates the
docs on the source and makes synchronous calls to the gen_server
telling it to copy the documents that don't already exist on the
target.
* Opening and saving of documents and attachments is handled
internally by the gen_server. I'm hoping that concentrating all
this activity in the server will help us do a better job of pipeline
and memory management.
This sounds good.
Sorry for not discussing a change this big ahead of time.
We are discussing it now :)
Damien, I saw that you had some recent checkins related to
replication security; I'd be happy to merge those in. The
replication API itself has not changed at all.
Let me know if you need any help merging.
This branch is still work-in-progress, but in my opinion it's almost
as fast as and more robust than the code in trunk right now. If it
meets with peoples' approval and we can get some feedback from the
various reporters in JIRA, it might make sense to tag it for 0.9.
Cheers,
This work will also be useful for setting up continuos replications.
Soon we'll have Comet events to notify a waiting replicator of changes
to replicate, so change replicate as soon as they happen, not just on
a schedule.
-Damien
