I did set the LCB_LOGLEVEL to 5 and change the libcouchbase to 2.4.0 again, but unfortunately uwsgi hide the debug output of the liccouchbase. And I got a lot "*[err] evmap.c:401: Assertion ctx failed in evmap_io_active* " this cycle. I wonder if there are some other methods to get the libcouchbase debug output just like a log file or others.
2014-07-18 0:40 GMT+08:00 Mark Nunberg <[email protected]>: > Unfortunately I am not very well versed in uwsgi, however from from what I > see, you are instantiating the Connection object in global scope. > Presumably the Category.on_post method is called inside the workers which > are already spawned once the application has been loaded (inheriting the > Connection object). This may lead to unwanted behavior, ranging from data > corruption to random crashes. > > The crashes you are seeing seem to be coming from libevent which is > obviously not happy about being forked (in the 2.4.0 client). The 2.3.x > client crashes because it is receiving garbage data from the socket (since > you have multiple processes reading from the same socket). > > Please set LCB_LOGLEVEL=5 in the environment before spawning your program. > This will show a lot of debug output about the client library (continue > using 2.4 for this); the output will tell us if you are actually trying to > use the same client from multiple processes. > If indeed this is the case you will need to figure out a way to ensure > that the client object is never inherited from a parent to a child process. > There is the Python `atfork` (https://pypi.python.org/pypi/atfork/0.1.2) > module which you can use to help with this. > > Regards, > Mark > > On Jul 17, 2014, at 9:31 AM, 孙如镇 <[email protected]> wrote: > > 33 keys were used by get_multi, my uwsgi command is "*sudo uwsgi -s > /tmp/uwsgi.sock --wsgi-file api.py --callable app --master --process 4 -R > 10000", *I didn't use threads. > > > 2014-07-18 0:24 GMT+08:00 Mark Nunberg <[email protected]>: > >> Just how many keys are you passing to get_multi/set_multi? >> >> Mentioning uwsgi, what form of thread/process configuration are you >> using? Keep in mind that in the case of forking, you must ensure the >> Connection object is create in the child process, not the parent process. >> >> On Jul 17, 2014, at 9:19 AM, 孙如镇 <[email protected]> wrote: >> >> Hi Nunberg, >> >> My libcouchbase-dev is 2.3.2. I have tried to use your >> libcouchbase-dev 2.4.0, but situation even be much worsen.A lot " >> *timeout*" were throw out,but no *"src/server.c:666: >> lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes' failed."* >> errors. Now I am sure that the error " *src/server.c:666: >> lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes' failed.*" >> makes my python application container(uwsgi) down before. By the way some >> time another error *"evmap.c:401: Assertion ctx failed in >> evmap_io_active*", So really need your help. >> >> Best regards. >> >> >> 2014-07-17 22:52 GMT+08:00 孙如镇 <[email protected]>: >> >>> Hi Nunberg, >>> >>> Thanks for your replay. I will follow you advices and do some >>> experiments. >>> >>> Best regards. >>> >>> >>> 2014-07-17 22:42 GMT+08:00 M. Nunberg <[email protected]>: >>> >>> Hi >>>> >>>> Such errors would fall under the domain of the underlying libcouchbase. >>>> What version of libcouchbase are you using? Version 2.3.1 and earlier has a >>>> bug which may manifest itself in the symptom you are describing, so please >>>> try 2.3.2 (http://www.couchbase.com/communities/c-client-library) >>>> >>>> I would also suggest you try to run your tests with the new 2.4.0-beta >>>> version to see how that goes. Get it here >>>> http://packages.couchbase.com/clients/c/index.html >>>> >>>> >>>> On 07/17/2014 04:23 AM, Chen Sun wrote: >>>> >>>>> I am a new guy to couchbase.I try to use couchbase and couchbase's >>>>> python client in my python-base web APIs.But I find some errors when I >>>>> use ab(a tool to test the web api performence) to test the api with >>>>> some >>>>> concurrecny http requests. The errors of the python clinet throwed >>>>> bellow here.I want know why these errors occoured,and how to solve this >>>>> problem. >>>>> >>>>> The errors >>>>> ------------------------------------------------------------ >>>>> --------------------------------------------------------------- >>>>> >>>>> uwsgi: src/server.c:666: lcb_server_purge_implicit_responses: >>>>> Assertion `c->cmd_log.nbytes' failed. >>>>> python-couchbase: self->nremaining == 0 at src/oputil.c:77. >>>>> AbortTraceback (most recent call last): >>>>> File "api.py", line 111, in falcon.api.API.__call__ >>>>> (falcon/api.c:1389) >>>>> # startkey=start, >>>>> endkey=end, >>>>> File "api.py", line 96, in falcon.api.API.__call__ >>>>> (falcon/api.c:1210) >>>>> # end = '/' + adid + '/' + end >>>>> File "api.py", line 174, in on_post >>>>> query_results = couchbase_category.get_multi(keys, quiet=True) >>>>> File "/usr/local/lib/python2.7/dist-packages/couchbase/connection.py", >>>>> line 1053, in get_multi >>>>> return _Base.get_multi(self, keys, ttl=ttl, quiet=quiet, >>>>> replica=replica, no_format=no_format) >>>>> couchbase.exceptions._TimeoutError_0x17 (generated, catch >>>>> TimeoutError): <Key=u'com.htc.fm', RC=0x17[Client-Side timeout >>>>> exceeded for operation. Inspect network conditions or increase the >>>>> timeout], Operational Error, Results=34, C Source=(src/multiresult.c,286) >>>>> > >>>>> >>>>> ------------------------------------------------------------ >>>>> ------------------------------------------------------------ >>>>> ------------ >>>>> >>>>> My python code >>>>> >>>>> couchbase_category= Couchbase.connect( >>>>> >>>>> bucket='Category', host='127.0.0.1', timeout=5) >>>>> class Category: >>>>> >>>>> def on_post(self, req, resp): >>>>> >>>>> t1= datetime.datetime.now() >>>>> content= {} >>>>> # try: >>>>> params= json.load(req.stream, 'utf-8') >>>>> keys= [] >>>>> for appin params['apps']: >>>>> >>>>> keys.append(app['bundle_id']) >>>>> >>>>> unknown_apps= [] >>>>> result= {} >>>>> result['known_bundle_ids'] = [] >>>>> result['unknown_bundle_ids'] = [] >>>>> result['apps'] = [] >>>>> result['categorys'] = [] >>>>> >>>>> query_results= couchbase_category.get_multi(keys, >>>>> quiet=True) >>>>> for keyin query_results: >>>>> >>>>> if query_results[key].success: >>>>> result['apps'].append(query_results[key].value) >>>>> result['known_bundle_ids'].append(key) >>>>> result['categorys'].append( >>>>> query_results[key].value['categoryName']) >>>>> else: >>>>> result['unknown_bundle_ids'].append(key) >>>>> result['categorys'] = list(set(result['categorys'])) >>>>> for itemin params['apps']: >>>>> >>>>> if item['bundle_id'] in result['unknown_bundle_ids']: >>>>> unknown_apps.append(item) >>>>> timestamp= time.time() >>>>> pipe= r.pipeline() >>>>> for appin unknown_apps: >>>>> >>>>> pipe.zadd('uncached_apps', timestamp, app) >>>>> pipe.execute() >>>>> content['is_ok'] = True >>>>> content['result'] = result >>>>> t2= datetime.datetime.now() >>>>> >>>>> content['total_time'] = (t2 - t1).total_seconds() >>>>> # except Exception, err: >>>>> # content['is_ok'] = False >>>>> # finally: >>>>> resp.status = falcon.HTTP_201 >>>>> resp.body = json.dumps(content) >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Couchbase" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected] >>>>> <mailto:[email protected]>. >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Couchbase" group. >>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>> topic/couchbase/mhiJsqPkk7g/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Couchbase" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> >> For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Couchbase" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > You received this message because you are subscribed to the Google Groups > "Couchbase" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Couchbase" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Couchbase" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
