[
https://issues.apache.org/jira/browse/COUCHDB-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14624988#comment-14624988
]
Paul Joseph Davis commented on COUCHDB-2732:
--------------------------------------------
When we saw this in testing my recollection was that we'd probably missed it
due to the concurrency issue.
For reference, the tests that we use to illustrate the performance difference
is to set up a view on a clustered database with q=128 and then ask for a set
of 10 rows from that view with a large number of clients bypassing HAProxy. The
end result is that we end up having to call couch_ejson_compare when streaming
the view response an extremely large number of times in lots of different
request handling processes. This was enough to demonstrate that the mutex
locking was a global bottleneck. On single node couch the number of collations
is significantly smaller because it doesn't have to merge the responses from
all 128 shards before returning them to the user.
> Use thread local storage for couch_ejson_compare NIF
> ----------------------------------------------------
>
> Key: COUCHDB-2732
> URL: https://issues.apache.org/jira/browse/COUCHDB-2732
> Project: CouchDB
> Issue Type: Improvement
> Security Level: public(Regular issues)
> Reporter: Adam Kocoloski
>
> Some folks inside IBM have demonstrated conclusively that the NIF we use for
> JSON sorting is a significant bottleneck with more than a few concurrent
> users hitting us. The VM ends up spending all of its time dealing with lock
> contention. We'd be better off sticking with the pure Erlang code, but we
> have an even better alternative, which is to use thread local storage to pin
> an allocator to each OS thread and eliminate the locks.
> Patch forthcoming, but I wanted to make sure this got in the tracker. The
> improvement looks really signficant. Interestingly, there was some discussion
> about a performance regression after this was introduced back in COUCHDB-1186
> ... maybe the missing element in that discussion was the client concurrency?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
