Avaq opened a new issue #1081: Replicator infinite failure loop URL: https://github.com/apache/couchdb/issues/1081 ## Expected Behavior No matter the circumstances, the `_changes` feed should accept incoming connections in under 30 seconds. ## Current Behavior When supplying a `filter` argument to the `/db/_changes` end-point (filtered replication), the connection is not accepted until a document in the database passes the filter. This causes connections to drop (due to `replicator.connection_timeout`) on large datasets with only few documents matching the filter. ## Possible Solution It is trivial to ask: so why don't I just increase the value for `replicator.connection_timeout`? This is how I've solves the issue thus far. But it's not a complete solution because of several reasons: * The replication targets are managed by third-parties. It's not easy to get all of them to increase the timeout consistently. When one party fails to do so, they ruin it for all other parties (See Context). * Increasing the replication timeout will only work for an arbitrary amount of documents. As the amount of documents in my database increases, the chance that the new timeout is not long enough also increases. A permanent solution seems very simple to me; CouchDB should already start sending heartbeats *while* it's filtering changes, preventing the connection from timing out while it's working on a response. Connections should only time out if the actual connection to the server could not be established. ## Steps to Reproduce 1. Prepare a database ```shell curl -X PUT localhost:5984/replication-source curl -X PUT localhost:5984/replication-source/_design/test -d '{"filters":{"test":"function(){var future = Date.now() + 2000; while(Date.now() < future){}; return false}"}}' for i in {1..20}; do curl -X POST -H 'Content-Type: application/json' localhost:5984/replication-source -d '{"foo":"bar"}'; done ``` 2. Test the behavior ```shell curl 'localhost:5984/replication-source/_changes?since=0&filter=test/test&heartbeat=2000' ``` 3. On CouchDB 1.6, this behaves correctly. The server immediately responds with `{"results":[`, followed by a newline every two seconds. On CouchDB 2.x, the request times out without any response. ## Context In my case, connections drop *consistently*, which means the replication *target* never receives any documents, as the replication is re-initiated every thirty seconds using the same sequence number. But worse, the replication *source* gets overloaded because it's continuously filtering the same large set of data. This causes other clients to also start losing connections due to the increased load on the server, which in turn increases the load, et cetera. ## Your Environment * Version used: `2.1` * Browser Name and version: `curl 7.57.0 (x86_64-pc-linux-gnu) libcurl/7.57.0 OpenSSL/1.0.2n zlib/1.2.11 libssh2/1.8.0 nghttp2/1.24.0` * Operating System and version: `Linux 4.9.68 #1-NixOS SMP Sat Dec 9 21:01:57 UTC 2017 x86_64 GNU/Linux` * Secondary operating System and version: `Darwin 17.2.0 Darwin Kernel Version 17.2.0: Fri Sep 29 18:27:05 PDT 2017; root:xnu-4570.20.62~3/RELEASE_X86_64 x86_64` ## Personal plea This bug is causing an otherwise beautiful key business application that synchronises data over many low-bandwidth off-shore nodes to fail. It's of great personal interest to me to get it fixed. If somebody could point me to the relevant piece of source code, I wouldn't mind having a go at resolving the issue via a pull-request.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
