On Jul 14, 2010, at 5:42 PM, Mark Anderson wrote:
> We're having a bit of a problem with couchdb views 'disappearing', and I've
> not been able to make headway diagnosing the problem. I'd love to have
> suggestions on how to isolate this.
>
> After some time running, we start seeing errors in the log:
> 2010-07-12_17:59:04.82351 [info] [<0.8900.246>] 10.192.210.79 - - 'GET'
> /authorization/_design/objects/_view/by_type?key=%2245c4ef2de7981991a5aaf23cd7fb0bbf%22
> 404
>
> Running curl
> % curl 'localhost:5984//authorization/_design/objects/_view/by_type?limit=0'
are the double slashes after 5984 intentional? does removing them help?
> {"error":"not_found","reason":"missing_named_view"}
> This would normally succeed.
>
> The design documents are present and can be fetched.
> % curl 'localhost:5984//authorization/_design/objects'
> {"_id":"_design/objects","_rev":"1-3598677772","views":{"by_type":{"map":"function(doc)
> {emit(doc._id,doc.type)}"}}}
>
> Couchdb never recovers. Restarting couchdb fixes the problem. This problem
> repeats, in the sense that it happens pretty consistently, but we've had
> trouble reproducing the problem neatly; synthetic couchdb workloads do not
> seem to trigger this.
>
> The only patterns we've been able to spot is that they seem to happen right
> after a series of rapid updates to an document indexed by the view in
> question. We see a bunch of 'PUT' entries, and 'checkpointing view update at
> seq XXX for authorization'. This is happening across all of our databases,
> but seems connected with load.
>
this could be an issue with being able to open another file-descriptor?
> * Using native erlang views does not seem to prevent the problem, just defer
> it a bit.
> * There is plenty of disk space; 30GB used in a 100GB partition. None of the
> databases are larger than 1GB, but some of the views get very large (12G or
> more)
>
> We're running CouchDB 0.11.0; Ubuntu 10.04. I've not yet been able to repro
> the problem in 1.0.0, and will try 0.11.1 as soon as I give up on breaking
> 1.0.0.
>
> I have a ec2 machine in this state, so if anyone has suggestions of
> diagnostics to run or the like, I'd be glad to poke at it a bit.
>
>
>
