Nick Vatamaniuc created COUCHDB-2833:
----------------------------------------
Summary: Replicator client doesn't handle un-expectedly closed
pipelined connections well
Key: COUCHDB-2833
URL: https://issues.apache.org/jira/browse/COUCHDB-2833
Project: CouchDB
Issue Type: Bug
Security Level: public (Regular issues)
Components: Database Core
Reporter: Nick Vatamaniuc
This was found investigating the failure of replication tests. Specifically
couch_replicator_large_atts_tests, the {local, remote} sub-case.
The test sets up push replications from local to remote.
Replication workers have more than 1 document larger than
MAX_BULK_ATT_SIZE=64K. They start pushing them to the target, using a
keep-alive connection (default for HTTP 1.1), the first few pipelined requests
will go through using the same connection, then server will accept the first
PUT to …/docid?edits=false, then return Connection:close and close the
connection after the 201 Created result. Workers don't expect that, and try to
do another PUT on same connection. And then crash on ibrowser's
connection_closing error, which they don't handle. That causes the whole async
replication process to exit.
Potentially there are 2 issues.
couch_replicator_http layer needs to handle this case better. On closing error,
shut down the socket quickly and then retry. (Not shutting it down and retrying
means retrying for at least 5 or so seconds until something cleans up that
connection state).
Adding this clause to couch_replicator_httpc.erl seems to do the trick:
{code}
process_response({error,connection_closing}, Worker, HttpDb, Params, _Cb)->
couch_log:notice("Connection closed by server. Closing the socket and
trying again",[]),
ibrowse_http_client:stop(Worker),
throw({retry, HttpDb, Params});
{code}
Another issue is to make the server not close the connection after first PUT to
.../db/docid/new_edits=false when using pipeline connections.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
