Hi Conor, Great to hear! How did you find out? Maybe you can post this for others also...
Cheers Sebastian Von meinem iPhone gesendet > Am 16.02.2015 um 17:04 schrieb Conor Mac Aoidh <[email protected]>: > > Hi Sebastian, > > Thanks for the help in debugging. Turns out this issue was caused by a recent > update to our 'validate_doc_update' function. This caused an error in the > validation of 'deleted' documents, which triggered lots of retries and ended > up causing the server to hang. I'm pretty sure this was the problem anyway, > since I've fixed the validation function, the issue has not recurred. > > Conor > >> On 13/02/15 21:21, Sebastian Rothbucher wrote: >> Hi Conor, >> >> I'd look at the bottom of the file for two reasons >> - you know the ID of the last doc from the in trace - possibly this is the >> one causing the problem (and you can analyze or download+delete it) >> - you'd know when in or out trace violate the one-line-in-and-one-line-out >> policy of couchjs, i.e. lines are (for whichever reason) not one-liners >> with valid JSON >> >> The index is build upon requesting the view. So you have to do some change >> (or at least re-submit a doc via curl) and then redo the index to cause any >> map/reduce work to actually happen. I'd try to have just one indexing >> process going on at a time when debugging. >> >> Still, it's hard to tell from remote what actually this might be. A hanging >> communication with couchjs might just be one reason anyway >> >> But good luck anyways >> Sebastian >> >> On Thu, Feb 12, 2015 at 2:47 PM, Conor Mac Aoidh <[email protected]> >> wrote: >> >>> Hi Sebastian, >>> >>> It actually just re-occurred on one of our servers. I've applied your >>> logging solution and I'm getting /a lot/ of output. Any idea what Is should >>> be looking for in particular? >>> >>> Also, when should I look? I assume the index is not built at the same time >>> a request is made.. maybe adding a document would trigger re-indexing? >>> >>> Thanks >>> >>> Conor >>> >>> >>>> On 12/02/15 13:20, Conor Mac Aoidh wrote: >>>> >>>> Hi Sebastian, >>>> >>>> Thanks for the detailed reply! Unfortunately the problem is intermittent >>>> and can't be triggered by the likes of stress tests, so I have to wait >>>> until it occurs again to try your logging solution and debug further. >>>> >>>> Thanks >>>> >>>> Conor >>>> >>>>> On 11/02/15 21:05, Sebastian Rothbucher wrote: >>>>> >>>>> Hi Conor, >>>>> >>>>> it's hard to tell really. Maybe a solution is intercepting the calls to >>>>> couchjs. I just posted one possible solution here: >>>>> https://gist.github.com/sebastianrothbucher/01afe929095a55ab233e >>>>> >>>>> Basically, it's using the tee command to write input and output to a file >>>>> to analyze. >>>>> >>>>> I don't really have a striking idea, but maybe looking at the outputs in >>>>> your case helps. >>>>> >>>>> Good luck >>>>> Sebastian >>>>> >>>>> On Tue, Feb 10, 2015 at 5:00 PM, Conor Mac Aoidh < >>>>> [email protected]> >>>>> wrote: >>>>> >>>>> Hi All, >>>>>> Been experiencing a strange issue where a couchdb view hangs repeatedly >>>>>> and does not stop hanging until the couch service is restarted. It >>>>>> seems to >>>>>> be happening on all databases (with the same view/design function) >>>>>> simultaneously, ie. once its triggered, it happens everywhere where that >>>>>> design doc is used. >>>>>> >>>>>> Stranger still, in some of the databases that are hanging, there are >>>>>> only >>>>>> 1 or 2 documents to index, so it does not appear to be a load issue. I >>>>>> have >>>>>> debug mode enabled and all I get in the logs for each request is: >>>>>> / >>>>>> //[Tue, 10 Feb 2015 15:33:41 GMT] [debug] [<0.1620.0>] 'GET' >>>>>> /ie_conormacaoidh_gmail_com/_design/proj//ects/_view/projects/all {1,// >>>>>> //1} from// >>>>>> // "193.1.208.99"// >>>>>> //Headers: [{'Accept',"*/*"},// >>>>>> // {'Authorization',"Basic YWRtaW46UVlIOHdFcGZLQXZzUFBu"},// >>>>>> // {'Host',"mysite.com:5984"},// >>>>>> // {'User-Agent',"curl/7.32.0"}]// >>>>>> //[Tue, 10 Feb 2015 15:33:41 GMT] [debug] [<0.1620.0>] OAuth Params: []/ >>>>>> >>>>>> >>>>>> And the view function: >>>>>> / >>>>>> //function (doc) {// >>>>>> // if(doc.type === 'project' && doc.status !== 'deleted') {// >>>>>> // emit(doc.created, doc);// >>>>>> // }// >>>>>> //}/ >>>>>> >>>>>> Anyone have any ideas what is causing this or how to debug it further? >>>>>> >>>>>> P.S. I'm using couchdb 1.6.1 >>>>>> >>>>>> Thanks >>>>>> >>>>>> Conor >
