I solved my problem by using a uwsgi param (--lazy) to set the uwsgi loading code after forking new worker-processes , that reason of the problem is that uwsgi loads python code first ,and than fork new work-processes .
2014-07-18 1:32 GMT+08:00 孙如镇 <[email protected]>: > I did set the LCB_LOGLEVEL to 5 and change the libcouchbase to 2.4.0 > again, but unfortunately uwsgi hide the debug output of the liccouchbase. > And I got a lot "*[err] evmap.c:401: Assertion ctx failed in > evmap_io_active* " this cycle. I wonder if there are some other methods > to get the libcouchbase debug output just like a log file or others. > > > 2014-07-18 0:40 GMT+08:00 Mark Nunberg <[email protected]>: > > Unfortunately I am not very well versed in uwsgi, however from from what I >> see, you are instantiating the Connection object in global scope. >> Presumably the Category.on_post method is called inside the workers which >> are already spawned once the application has been loaded (inheriting the >> Connection object). This may lead to unwanted behavior, ranging from data >> corruption to random crashes. >> >> The crashes you are seeing seem to be coming from libevent which is >> obviously not happy about being forked (in the 2.4.0 client). The 2.3.x >> client crashes because it is receiving garbage data from the socket (since >> you have multiple processes reading from the same socket). >> >> Please set LCB_LOGLEVEL=5 in the environment before spawning your >> program. This will show a lot of debug output about the client library >> (continue using 2.4 for this); the output will tell us if you are actually >> trying to use the same client from multiple processes. >> If indeed this is the case you will need to figure out a way to ensure >> that the client object is never inherited from a parent to a child process. >> There is the Python `atfork` (https://pypi.python.org/pypi/atfork/0.1.2) >> module which you can use to help with this. >> >> Regards, >> Mark >> >> On Jul 17, 2014, at 9:31 AM, 孙如镇 <[email protected]> wrote: >> >> 33 keys were used by get_multi, my uwsgi command is "*sudo uwsgi -s >> /tmp/uwsgi.sock --wsgi-file api.py --callable app --master --process 4 -R >> 10000", *I didn't use threads. >> >> >> 2014-07-18 0:24 GMT+08:00 Mark Nunberg <[email protected]>: >> >>> Just how many keys are you passing to get_multi/set_multi? >>> >>> Mentioning uwsgi, what form of thread/process configuration are you >>> using? Keep in mind that in the case of forking, you must ensure the >>> Connection object is create in the child process, not the parent process. >>> >>> On Jul 17, 2014, at 9:19 AM, 孙如镇 <[email protected]> wrote: >>> >>> Hi Nunberg, >>> >>> My libcouchbase-dev is 2.3.2. I have tried to use your >>> libcouchbase-dev 2.4.0, but situation even be much worsen.A lot " >>> *timeout*" were throw out,but no *"src/server.c:666: >>> lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes' failed."* >>> errors. Now I am sure that the error " *src/server.c:666: >>> lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes' failed.*" >>> makes my python application container(uwsgi) down before. By the way some >>> time another error *"evmap.c:401: Assertion ctx failed in >>> evmap_io_active*", So really need your help. >>> >>> Best regards. >>> >>> >>> 2014-07-17 22:52 GMT+08:00 孙如镇 <[email protected]>: >>> >>>> Hi Nunberg, >>>> >>>> Thanks for your replay. I will follow you advices and do some >>>> experiments. >>>> >>>> Best regards. >>>> >>>> >>>> 2014-07-17 22:42 GMT+08:00 M. Nunberg <[email protected]>: >>>> >>>> Hi >>>>> >>>>> Such errors would fall under the domain of the underlying >>>>> libcouchbase. What version of libcouchbase are you using? Version 2.3.1 >>>>> and >>>>> earlier has a bug which may manifest itself in the symptom you are >>>>> describing, so please try 2.3.2 (http://www.couchbase.com/ >>>>> communities/c-client-library) >>>>> >>>>> I would also suggest you try to run your tests with the new 2.4.0-beta >>>>> version to see how that goes. Get it here >>>>> http://packages.couchbase.com/clients/c/index.html >>>>> >>>>> >>>>> On 07/17/2014 04:23 AM, Chen Sun wrote: >>>>> >>>>>> I am a new guy to couchbase.I try to use couchbase and couchbase's >>>>>> python client in my python-base web APIs.But I find some errors when I >>>>>> use ab(a tool to test the web api performence) to test the api with >>>>>> some >>>>>> concurrecny http requests. The errors of the python clinet throwed >>>>>> bellow here.I want know why these errors occoured,and how to solve >>>>>> this >>>>>> problem. >>>>>> >>>>>> The errors >>>>>> ------------------------------------------------------------ >>>>>> --------------------------------------------------------------- >>>>>> >>>>>> uwsgi: src/server.c:666: lcb_server_purge_implicit_responses: >>>>>> Assertion `c->cmd_log.nbytes' failed. >>>>>> python-couchbase: self->nremaining == 0 at src/oputil.c:77. >>>>>> AbortTraceback (most recent call last): >>>>>> File "api.py", line 111, in falcon.api.API.__call__ >>>>>> (falcon/api.c:1389) >>>>>> # startkey=start, >>>>>> endkey=end, >>>>>> File "api.py", line 96, in falcon.api.API.__call__ >>>>>> (falcon/api.c:1210) >>>>>> # end = '/' + adid + '/' + end >>>>>> File "api.py", line 174, in on_post >>>>>> query_results = couchbase_category.get_multi(keys, quiet=True) >>>>>> File "/usr/local/lib/python2.7/dist-packages/couchbase/connection.py", >>>>>> line 1053, in get_multi >>>>>> return _Base.get_multi(self, keys, ttl=ttl, quiet=quiet, >>>>>> replica=replica, no_format=no_format) >>>>>> couchbase.exceptions._TimeoutError_0x17 (generated, catch >>>>>> TimeoutError): <Key=u'com.htc.fm', RC=0x17[Client-Side timeout >>>>>> exceeded for operation. Inspect network conditions or increase the >>>>>> timeout], Operational Error, Results=34, C Source=(src/multiresult.c,286) >>>>>> > >>>>>> >>>>>> ------------------------------------------------------------ >>>>>> ------------------------------------------------------------ >>>>>> ------------ >>>>>> >>>>>> My python code >>>>>> >>>>>> couchbase_category= Couchbase.connect( >>>>>> >>>>>> bucket='Category', host='127.0.0.1', timeout=5) >>>>>> class Category: >>>>>> >>>>>> def on_post(self, req, resp): >>>>>> >>>>>> t1= datetime.datetime.now() >>>>>> content= {} >>>>>> # try: >>>>>> params= json.load(req.stream, 'utf-8') >>>>>> keys= [] >>>>>> for appin params['apps']: >>>>>> >>>>>> keys.append(app['bundle_id']) >>>>>> >>>>>> unknown_apps= [] >>>>>> result= {} >>>>>> result['known_bundle_ids'] = [] >>>>>> result['unknown_bundle_ids'] = [] >>>>>> result['apps'] = [] >>>>>> result['categorys'] = [] >>>>>> >>>>>> query_results= couchbase_category.get_multi(keys, >>>>>> quiet=True) >>>>>> for keyin query_results: >>>>>> >>>>>> if query_results[key].success: >>>>>> result['apps'].append(query_results[key].value) >>>>>> result['known_bundle_ids'].append(key) >>>>>> result['categorys'].append( >>>>>> query_results[key].value['categoryName']) >>>>>> else: >>>>>> result['unknown_bundle_ids'].append(key) >>>>>> result['categorys'] = list(set(result['categorys'])) >>>>>> for itemin params['apps']: >>>>>> >>>>>> if item['bundle_id'] in result['unknown_bundle_ids']: >>>>>> unknown_apps.append(item) >>>>>> timestamp= time.time() >>>>>> pipe= r.pipeline() >>>>>> for appin unknown_apps: >>>>>> >>>>>> pipe.zadd('uncached_apps', timestamp, app) >>>>>> pipe.execute() >>>>>> content['is_ok'] = True >>>>>> content['result'] = result >>>>>> t2= datetime.datetime.now() >>>>>> >>>>>> content['total_time'] = (t2 - t1).total_seconds() >>>>>> # except Exception, err: >>>>>> # content['is_ok'] = False >>>>>> # finally: >>>>>> resp.status = falcon.HTTP_201 >>>>>> resp.body = json.dumps(content) >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Couchbase" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to [email protected] >>>>>> <mailto:[email protected]>. >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "Couchbase" group. >>>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>>> topic/couchbase/mhiJsqPkk7g/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Couchbase" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Couchbase" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Couchbase" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Couchbase" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Couchbase" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
