[
https://issues.apache.org/jira/browse/COUCHDB-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597006#comment-14597006
]
Adam Kocoloski commented on COUCHDB-2724:
-----------------------------------------
I started work on this enhancement here:
https://github.com/apache/couchdb-chttpd/compare/2724-chunked-buffering
The diff looks more intimidating than it really is; the first two commits on
that branch do some cleanup and refactoring to make it easier to evolve the
accumulator that we're using in the callback. An earlier, uglier version of
this patch was showing throughput improvements of over 30%; I haven't tested
this specific patch.
Extending this patch to live / continuous / event source feeds still looks a
bit tricky, as the last thing we want to do is add latency because we're
sitting around waiting for a heartbeat event to trigger a flush. We might need
to create a new event to signal the end of one internal traversal of the
_changes feed and force a flush then.
> Batch rows in streaming responses to improve throughput
> -------------------------------------------------------
>
> Key: COUCHDB-2724
> URL: https://issues.apache.org/jira/browse/COUCHDB-2724
> Project: CouchDB
> Issue Type: Improvement
> Security Level: public(Regular issues)
> Components: Database Core, HTTP Interface
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
>
> [~tonysun83] showed me some profiling of the {{_changes}} feed which
> indicated that the coordinator process was spending about 1/3 of its time
> executing inside {{send_delayed_chunk}}. We can reduce the number of
> invocations of this function by buffering individual rows until we reach a
> (configurable) threshold for sending the data out the wire.
> We'll of course want to be careful about continuous feeds; if we're in the
> "slow drip" portion of the feed we'll obviously want to emit right away
> instead of adding latency unnecessarily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
