I will give this ago. I think I've isolated all my problems down to a reduce function that couchbase does not like -- which is making indexing on the bucket hang/execute indefinitely. This was causing all sorts of problems with failover and client view requests.
On Saturday, December 13, 2014 10:26:36 AM UTC-8, Aliaksey Kandratsenka wrote: > > > > On Sat, Dec 13, 2014 at 12:20 AM, Michael Liu <[email protected] > <javascript:>> wrote: >> >> I have a cluster of 2 nodes and one node was going out of memory >> performing compaction. So, I did a failover, and now the remaining server >> is unresponsive to my nodejs client. I have access to all my views and >> documents through http rest, so I'm guessing something is wrong with the >> way smart clients connect and make requests. >> >> In the couchdb log is full of these timeouts: >> >> [couchdb:error,2014-12-13T8:10:50.001,[email protected]:<0.468.0>: >> couch_log:error:44]dcp client (loop, mapreduce_view: loop >> _design/sync_housekeeping >> (prod/main)): Obtaining mutation from server timed out after 60.0 >> seconds [RequestId 123, PartId 532, StartSeq 23, EndSeq 25]. Waiting... >> >> In the web console I see a popup that just says Indexing all my design >> docs/views, with the progress bar at 0. >> >> I was thinking to perform a backup/restore, but cbbbackup won't complete >> either: >> couchbase@union-1:~$ bin/cbbackup -v http://localhost:8091 backups/ >> 2014-12-13 08:04:49,305: mt cbbackup... >> 2014-12-13 08:04:49,305: mt source : http://localhost:8091 >> 2014-12-13 08:04:49,305: mt sink : backups/ >> 2014-12-13 08:04:49,305: mt opts : {'username': None, 'verbose': 1, >> 'dry_run': False, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'data_only' >> : 0.0, 'uncompress': 0.0, 'nmv_retry': 1.0, 'conflict_resolve': 1.0, >> 'cbb_max_mb': 100000.0, 'mcd_compatible': 1.0, 'try_xwm': 1.0, >> 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, >> 'seqno': 0.0, 'batch_max_size': 1000.0, 'report': 5.0, 'design_doc_only': >> 0.0, 'recv_min_bytes': 4096.0}, 'single_node': False, 'vbucket_list': >> None, 'threads': 4, 'mode': 'full', 'key': None, 'password': None, 'id': >> None, 'bucket_source': None} >> 2014-12-13 08:04:49,320: mt bucket: loop >> >> >> >> ^Cinterrupted.###### ] 83.2% (6474/estimated 7780 msgs) >> >> >> Any help appreciated, at a loss here... >> > > This looks like some bug within memcached. File bug at > http://www.couchbase.com/issues/secure/Dashboard.jspa. Attach > cbcollectinfos and in this case it might be useful to grab backtraces of > all threads in memcached. E.g. > > # gdb -p <pid of your memcached> > > > thread app all bt > > > quit > > >> -- You received this message because you are subscribed to the Google Groups "Couchbase" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
