couch_query_servers pool deadlock when running "os_process_limit" indexers
--------------------------------------------------------------------------
Key: COUCHDB-1375
URL: https://issues.apache.org/jira/browse/COUCHDB-1375
Project: CouchDB
Issue Type: Bug
Components: View Server Support
Affects Versions: 1.1
Reporter: Filipe Manana
When using external view servers, such as couchjs for example, if we trigger
"os_process_limit" (or more) index updates simultaneously, we can reach a
deadlock case.
The issue is that each index updater will acquire a couchjs (os_proc record)
process from couch_query_servers to apply the map function against each
document. After the indexer finishes, it will release (return to
couch_query_servers) the couchjs process used for the maps.
However, while the indexer is writing to the btrees, if the reduce function is
a JavaScript function (or any other language other than Erlang or a builtin
reduce function), the function is called often to reduce key-values - this
results in the view updater process to ask couch_query_servers for a another
couchjs process (this is done on every reduce function call) - however
"os_process_limit" couchjs processes are already allocated to
"os_process_limit" indexers for the mapping.
The solution here would be to have the index updater to preallocate a couchjs
process for the reduces and release it when it finishes (like it's done for the
maps).
This only happens if the number of changes to index is greater than 500 (the
work queue sizes).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
