Devs, My design doc query server branch is now ready for commit. It maintains backwards compatibility, with some small improvements to query handling, across all the APIs except for the query server protocol. It does not effect the map-reduce part of the query server protocol.
The current branch can be found here: http://github.com/jchris/couchdb/tree/ddoc-qs I'm also attaching an (almost current) snapshot of the diff to: https://issues.apache.org/jira/browse/COUCHDB-589 I've got all the tests passing, and moved the native query servers to a gen_server (although I didn't completely eliminate use of the process dictionary, this will not be hard now). There are other goodies in here, like better query server error handling. I've also refactored the JavaScript code to avoid global variables, although we could do more work here on the sandboxing. I intend to commit this patch soon. It's a lot of lines, so I want to get it in the next couple of days. I don't think it breaks anything, but it's still better to let it sit in trunk for a while before the release. The only people this change should effect are those using alternate query servers for more than just map-reduce. If you are one of these people, please inspect the branch and let me know how you feel about it. The gen_server around the native query server could use some review, too. It all seems to work just fine but there's nothing really in there about when the server should be shut down. I think that's fine as long as it goes down on config change, which it seems to. The way design docs are cached inside the server simplifies function dispatch and fixes the one os-process per filtered _changes feed problem. It would be good to profile list / show / update / validation now. They should be faster as they involve less data sent between Couch and the query server. One concern is that on systems with a large number of distinct design documents (or very large design documents) the memory utilization could go up. Currently attachments are not loaded into the design document for this reason. Only the latest version of each design document is stored in the query server, so if you have 2 dbs with the same ddoc at different revs, it's possible that some flapping could occur. The alternative is storing ddocs by rev and using a JavaScript LRU cache. That'd be a worthy improvement but it can come later. The patch opens the door for more flexibility within the JavaScript query server. For instance, CouchDB will be able to support APIs like the require() statement from CommonJS: http://commonjs.org/specs/modules/1.0.html Chris On Thu, Dec 17, 2009 at 6:39 PM, Chris Anderson <[email protected]> wrote: > I've been working on a medium-sized branch for a while and I'm finally > getting something worth showing... almost. > > I'm developing on http://github.com/jchris/couchdb/tree/ddoc-qs > > The essential difference is in the way the code is handled by the > query server. The query server keeps a cache of design documents > around. This allows us to simplify the programming model for show, > list, update, etc, and also I think will make it easier to add new > commands in the future. > > My aim was to cut a lot of code, but stay close to a pure refactoring. > So I haven't substantially changed the Futon test suite. But I did > make some changes to query_server_spec.rb. > > If anyone would like to pull the Erlang query server into line with > these changes, I'd appreciate the help. > > The branch is still deeply messy in some places. When it comes time to > push to CouchDB trunk, I'll hand-edit the patch for log levels and > inane comments. > > Chris > > -- > Chris Anderson > http://jchrisa.net > http://couch.io > -- Chris Anderson http://jchrisa.net http://couch.io
