Thanks for your help again. Your are so conscientious in your work which makes me a deep impression. Thank you again. Wish you a happy work.
2014-07-18 1:45 GMT+08:00 孙如镇 <[email protected]>: > I solved my problem by using a uwsgi param (--lazy) to set the uwsgi > loading code after forking new worker-processes , that reason of the > problem is that uwsgi loads python code first ,and than fork new > work-processes . > > > 2014-07-18 1:32 GMT+08:00 孙如镇 <[email protected]>: > > I did set the LCB_LOGLEVEL to 5 and change the libcouchbase to 2.4.0 >> again, but unfortunately uwsgi hide the debug output of the liccouchbase. >> And I got a lot "*[err] evmap.c:401: Assertion ctx failed in >> evmap_io_active* " this cycle. I wonder if there are some other methods >> to get the libcouchbase debug output just like a log file or others. >> >> >> 2014-07-18 0:40 GMT+08:00 Mark Nunberg <[email protected]>: >> >> Unfortunately I am not very well versed in uwsgi, however from from what >>> I see, you are instantiating the Connection object in global scope. >>> Presumably the Category.on_post method is called inside the workers which >>> are already spawned once the application has been loaded (inheriting the >>> Connection object). This may lead to unwanted behavior, ranging from data >>> corruption to random crashes. >>> >>> The crashes you are seeing seem to be coming from libevent which is >>> obviously not happy about being forked (in the 2.4.0 client). The 2.3.x >>> client crashes because it is receiving garbage data from the socket (since >>> you have multiple processes reading from the same socket). >>> >>> Please set LCB_LOGLEVEL=5 in the environment before spawning your >>> program. This will show a lot of debug output about the client library >>> (continue using 2.4 for this); the output will tell us if you are actually >>> trying to use the same client from multiple processes. >>> If indeed this is the case you will need to figure out a way to ensure >>> that the client object is never inherited from a parent to a child process. >>> There is the Python `atfork` (https://pypi.python.org/pypi/atfork/0.1.2) >>> module which you can use to help with this. >>> >>> Regards, >>> Mark >>> >>> On Jul 17, 2014, at 9:31 AM, 孙如镇 <[email protected]> wrote: >>> >>> 33 keys were used by get_multi, my uwsgi command is "*sudo uwsgi -s >>> /tmp/uwsgi.sock --wsgi-file api.py --callable app --master --process 4 -R >>> 10000", *I didn't use threads. >>> >>> >>> 2014-07-18 0:24 GMT+08:00 Mark Nunberg <[email protected]>: >>> >>>> Just how many keys are you passing to get_multi/set_multi? >>>> >>>> Mentioning uwsgi, what form of thread/process configuration are you >>>> using? Keep in mind that in the case of forking, you must ensure the >>>> Connection object is create in the child process, not the parent process. >>>> >>>> On Jul 17, 2014, at 9:19 AM, 孙如镇 <[email protected]> wrote: >>>> >>>> Hi Nunberg, >>>> >>>> My libcouchbase-dev is 2.3.2. I have tried to use your >>>> libcouchbase-dev 2.4.0, but situation even be much worsen.A lot " >>>> *timeout*" were throw out,but no *"src/server.c:666: >>>> lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes' >>>> failed."* >>>> errors. Now I am sure that the error " *src/server.c:666: >>>> lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes' >>>> failed.*" >>>> makes my python application container(uwsgi) down before. By the way some >>>> time another error *"evmap.c:401: Assertion ctx failed in >>>> evmap_io_active*", So really need your help. >>>> >>>> Best regards. >>>> >>>> >>>> 2014-07-17 22:52 GMT+08:00 孙如镇 <[email protected]>: >>>> >>>>> Hi Nunberg, >>>>> >>>>> Thanks for your replay. I will follow you advices and do some >>>>> experiments. >>>>> >>>>> Best regards. >>>>> >>>>> >>>>> 2014-07-17 22:42 GMT+08:00 M. Nunberg <[email protected]>: >>>>> >>>>> Hi >>>>>> >>>>>> Such errors would fall under the domain of the underlying >>>>>> libcouchbase. What version of libcouchbase are you using? Version 2.3.1 >>>>>> and >>>>>> earlier has a bug which may manifest itself in the symptom you are >>>>>> describing, so please try 2.3.2 (http://www.couchbase.com/ >>>>>> communities/c-client-library) >>>>>> >>>>>> I would also suggest you try to run your tests with the new >>>>>> 2.4.0-beta version to see how that goes. Get it here >>>>>> http://packages.couchbase.com/clients/c/index.html >>>>>> >>>>>> >>>>>> On 07/17/2014 04:23 AM, Chen Sun wrote: >>>>>> >>>>>>> I am a new guy to couchbase.I try to use couchbase and couchbase's >>>>>>> python client in my python-base web APIs.But I find some errors when >>>>>>> I >>>>>>> use ab(a tool to test the web api performence) to test the api with >>>>>>> some >>>>>>> concurrecny http requests. The errors of the python clinet throwed >>>>>>> bellow here.I want know why these errors occoured,and how to solve >>>>>>> this >>>>>>> problem. >>>>>>> >>>>>>> The errors >>>>>>> ------------------------------------------------------------ >>>>>>> --------------------------------------------------------------- >>>>>>> >>>>>>> uwsgi: src/server.c:666: lcb_server_purge_implicit_responses: >>>>>>> Assertion `c->cmd_log.nbytes' failed. >>>>>>> python-couchbase: self->nremaining == 0 at src/oputil.c:77. >>>>>>> AbortTraceback (most recent call last): >>>>>>> File "api.py", line 111, in falcon.api.API.__call__ >>>>>>> (falcon/api.c:1389) >>>>>>> # startkey=start, >>>>>>> endkey=end, >>>>>>> File "api.py", line 96, in falcon.api.API.__call__ >>>>>>> (falcon/api.c:1210) >>>>>>> # end = '/' + adid + '/' + end >>>>>>> File "api.py", line 174, in on_post >>>>>>> query_results = couchbase_category.get_multi(keys, quiet=True) >>>>>>> File >>>>>>> "/usr/local/lib/python2.7/dist-packages/couchbase/connection.py", >>>>>>> line 1053, in get_multi >>>>>>> return _Base.get_multi(self, keys, ttl=ttl, quiet=quiet, >>>>>>> replica=replica, no_format=no_format) >>>>>>> couchbase.exceptions._TimeoutError_0x17 (generated, catch >>>>>>> TimeoutError): <Key=u'com.htc.fm', RC=0x17[Client-Side timeout >>>>>>> exceeded for operation. Inspect network conditions or increase the >>>>>>> timeout], Operational Error, Results=34, C >>>>>>> Source=(src/multiresult.c,286) >>>>>>> > >>>>>>> >>>>>>> ------------------------------------------------------------ >>>>>>> ------------------------------------------------------------ >>>>>>> ------------ >>>>>>> >>>>>>> My python code >>>>>>> >>>>>>> couchbase_category= Couchbase.connect( >>>>>>> >>>>>>> bucket='Category', host='127.0.0.1', timeout=5) >>>>>>> class Category: >>>>>>> >>>>>>> def on_post(self, req, resp): >>>>>>> >>>>>>> t1= datetime.datetime.now() >>>>>>> content= {} >>>>>>> # try: >>>>>>> params= json.load(req.stream, 'utf-8') >>>>>>> keys= [] >>>>>>> for appin params['apps']: >>>>>>> >>>>>>> keys.append(app['bundle_id']) >>>>>>> >>>>>>> unknown_apps= [] >>>>>>> result= {} >>>>>>> result['known_bundle_ids'] = [] >>>>>>> result['unknown_bundle_ids'] = [] >>>>>>> result['apps'] = [] >>>>>>> result['categorys'] = [] >>>>>>> >>>>>>> query_results= couchbase_category.get_multi(keys, >>>>>>> quiet=True) >>>>>>> for keyin query_results: >>>>>>> >>>>>>> if query_results[key].success: >>>>>>> result['apps'].append(query_results[key].value) >>>>>>> result['known_bundle_ids'].append(key) >>>>>>> result['categorys'].append( >>>>>>> query_results[key].value['categoryName']) >>>>>>> else: >>>>>>> result['unknown_bundle_ids'].append(key) >>>>>>> result['categorys'] = list(set(result['categorys'])) >>>>>>> for itemin params['apps']: >>>>>>> >>>>>>> if item['bundle_id'] in result['unknown_bundle_ids']: >>>>>>> unknown_apps.append(item) >>>>>>> timestamp= time.time() >>>>>>> pipe= r.pipeline() >>>>>>> for appin unknown_apps: >>>>>>> >>>>>>> pipe.zadd('uncached_apps', timestamp, app) >>>>>>> pipe.execute() >>>>>>> content['is_ok'] = True >>>>>>> content['result'] = result >>>>>>> t2= datetime.datetime.now() >>>>>>> >>>>>>> content['total_time'] = (t2 - t1).total_seconds() >>>>>>> # except Exception, err: >>>>>>> # content['is_ok'] = False >>>>>>> # finally: >>>>>>> resp.status = falcon.HTTP_201 >>>>>>> resp.body = json.dumps(content) >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Couchbase" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send >>>>>>> an email to [email protected] >>>>>>> <mailto:[email protected]>. >>>>>>> >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "Couchbase" group. >>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>>>> topic/couchbase/mhiJsqPkk7g/unsubscribe. >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Couchbase" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Couchbase" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Couchbase" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Couchbase" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- You received this message because you are subscribed to the Google Groups "Couchbase" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
