nickva opened a new pull request #1012: Fix replicator cluster stability race condition URL: https://github.com/apache/couchdb/pull/1012 Replicator clustering module is in charge of keeping track of when the cluster is stable or unstable. A cluster is said to be "stable" if there weren't any new nodes added or removed for some period of time. Replicator document processor uses cluster state to determine what to do with document updates as they come in. If a document update comes during a time when cluster is unstable, the update is skipped. The idea is that when cluster becomes stable again, all the documents will be rescanned anyway. In order for document updates to not be dropped, there is an implicit constraint when cluster becomes stable again (couch_replicator_notifier gen_event broadcasts `{cluster, stable}` event), any subsequent calls to `couch_replicator_clustering:is_stable()` would return `true`. If that's not the case then this sequence of events is possible: 1. mem3_cluster process calls the cluster_stable event 2. couch_replicator_notifier broadcasts {cluster, stable} event 3. couch_replicator_doc_processor start processing documents 4. On first document update couch_replicator_clustering:is_stable() is false, because that gen_server wasn't notifier yet. 5. Document update is dropped. 6. There won't be any rescans until cluster membership is changed again. To fix this, switch to setting stable state first via a gen_server call. This way after the `{cluster, stable}` has been called, is_stable() is guaranteed to return true. Note: This issue is mostly theoretical. It was noticed by examinig the code related to another bug. The chance of the clustering process going to sleep immediately after gen_event broadcast, then not handling the cast long enough for a document to be processed by doc processor is pretty low. #### Checklist - [x] Code is written and works correctly; - [x] Changes are covered by tests;
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
