GitHub user tonysun83 opened a pull request:
https://github.com/apache/couchdb-couch-replicator/pull/42
Address Connection: Closed In Headers
This was encountered when testing replication with a proxy server that
injected Connection:closed headers in a response. Two issues arise in our
replicator when we see this in the header:
1) An older version of ibrowse would throw a {error, {'EXIT', normal}},
when a connection:closed header was received. In the newer version of ibrowse,
it throws {error, req_timedout} instead. This leads to a maybe_retry function
call because we do not have a clause that handles this error. Which inevitably
leads to the replication process dying once it exhausts the retry limit. So we
change
the process_response clause to address this bug. However, this also means
we could end up trying forever for real timeouts. If there is a way to
distinguish between a real timeout vs a closed:connection, I'm open for
sugesstions.
2) We call ibrowse:stop on the worker and release it from the worker pool.
But our clean_mailbox tries to clean the mailbox of this worker when it's
already dead, leading to a timeout that crashes the replicator. So we check to
ensure that the Worker is still alive before we call ibrowse:stream_next.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloudant/couchdb-couch-replicator
69053-address-closed-connection
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/couchdb-couch-replicator/pull/42.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #42
----
commit 84429564fc26fdfdf521ceaa383e8eda6b2a3a5d
Author: Tony Sun <[email protected]>
Date: 2016-06-28T01:43:57Z
Change process_response clause
An older version of ibrowse would throw a {error, {'EXIT', Reason}},
when a connection:closed header was received. In the newer version
of ibrowse, it throws {error, req_timedout} instead. This leads
to a maybe_retry function call because we do not have a clause
that handles this error. Which inevitably leads to the replication
process dying once it exhausts the retry limit. So we change
the process_response clause to address this bug. However, this also
means we could end up trying forever for real timeouts.
BugzId:69053
commit b7eb63dcdbbf482cc37b6d2cd7b94c5692680e29
Author: Tony Sun <[email protected]>
Date: 2016-06-28T01:54:43Z
Check if worker is alive for clean_mailbox
When a connection:closed header is sent from the server, we handle it
by calling ibrowse:stop on the worker and release it from the worker
pool. But our clean_mailbox tries to clean the mailbox of this worker
when it's already dead, leading to a timeout that crashes the
changes_reader process and subsequently the replicator process. So
we check to ensure that the Worker is still alive before we call
ibrowse:stream_next.
BugzId:69053
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---