Currently it only logs to standard error. I would guess it would log elsewhere
though - perhaps a file or similar. Still the more I think about your issue the
more I am inclined to assume that you are passing the client object to the
child processes.
My advice would be to place the initialization (e.g. cb = Couchbase.connect())
inside the function that is first called when your worker process is invoked,
so e.g:
import os
from warnings import warn
from couchbase import Couchbase
cbclients = {}
def get_cb_client(**cbkw):
curpid = os.getpid()
# You might want to modify this if dealing with multiple buckets
if cbclients:
# Purge the dictionary from older clients
for pid in cbclients.keys():
if pid != curpid:
warn("Found leaked client object from PID {0}. We are
{1}".format(pid, curpid))
del cbclients[pid]
return cbclients.setdefault(curpid, Couchbase.connect(**cbkw))
class Category:
def on_post(self, req, resp):
bucket = get_cb_client(bucket='Category', host='127.0.0.1', timeout=5)
# Rest of the code goes here..
Note that I haven't tested this example, nor would I recommend it as a
replacement for proper design of applications that are to be forked, but it
should at least help debug/fix some of the issues you are having.
On Jul 17, 2014, at 10:32 AM, 孙如镇 <[email protected]> wrote:
> I did set the LCB_LOGLEVEL to 5 and change the libcouchbase to 2.4.0 again,
> but unfortunately uwsgi hide the debug output of the liccouchbase. And I got
> a lot "[err] evmap.c:401: Assertion ctx failed in evmap_io_active " this
> cycle. I wonder if there are some other methods to get the libcouchbase debug
> output just like a log file or others.
>
>
> 2014-07-18 0:40 GMT+08:00 Mark Nunberg <[email protected]>:
> Unfortunately I am not very well versed in uwsgi, however from from what I
> see, you are instantiating the Connection object in global scope. Presumably
> the Category.on_post method is called inside the workers which are already
> spawned once the application has been loaded (inheriting the Connection
> object). This may lead to unwanted behavior, ranging from data corruption to
> random crashes.
>
> The crashes you are seeing seem to be coming from libevent which is obviously
> not happy about being forked (in the 2.4.0 client). The 2.3.x client crashes
> because it is receiving garbage data from the socket (since you have multiple
> processes reading from the same socket).
>
> Please set LCB_LOGLEVEL=5 in the environment before spawning your program.
> This will show a lot of debug output about the client library (continue using
> 2.4 for this); the output will tell us if you are actually trying to use the
> same client from multiple processes.
> If indeed this is the case you will need to figure out a way to ensure that
> the client object is never inherited from a parent to a child process. There
> is the Python `atfork` (https://pypi.python.org/pypi/atfork/0.1.2) module
> which you can use to help with this.
>
> Regards,
> Mark
>
> On Jul 17, 2014, at 9:31 AM, 孙如镇 <[email protected]> wrote:
>
>> 33 keys were used by get_multi, my uwsgi command is "sudo uwsgi -s
>> /tmp/uwsgi.sock --wsgi-file api.py --callable app --master --process 4 -R
>> 10000", I didn't use threads.
>>
>>
>> 2014-07-18 0:24 GMT+08:00 Mark Nunberg <[email protected]>:
>> Just how many keys are you passing to get_multi/set_multi?
>>
>> Mentioning uwsgi, what form of thread/process configuration are you using?
>> Keep in mind that in the case of forking, you must ensure the Connection
>> object is create in the child process, not the parent process.
>>
>> On Jul 17, 2014, at 9:19 AM, 孙如镇 <[email protected]> wrote:
>>
>>> Hi Nunberg,
>>>
>>> My libcouchbase-dev is 2.3.2. I have tried to use your
>>> libcouchbase-dev 2.4.0, but situation even be much worsen.A lot "timeout"
>>> were throw out,but no "src/server.c:666:
>>> lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes' failed."
>>> errors. Now I am sure that the error " src/server.c:666:
>>> lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes' failed."
>>> makes my python application container(uwsgi) down before. By the way some
>>> time another error "evmap.c:401: Assertion ctx failed in evmap_io_active",
>>> So really need your help.
>>>
>>> Best regards.
>>>
>>>
>>> 2014-07-17 22:52 GMT+08:00 孙如镇 <[email protected]>:
>>> Hi Nunberg,
>>>
>>> Thanks for your replay. I will follow you advices and do some
>>> experiments.
>>>
>>> Best regards.
>>>
>>>
>>> 2014-07-17 22:42 GMT+08:00 M. Nunberg <[email protected]>:
>>>
>>> Hi
>>>
>>> Such errors would fall under the domain of the underlying libcouchbase.
>>> What version of libcouchbase are you using? Version 2.3.1 and earlier has a
>>> bug which may manifest itself in the symptom you are describing, so please
>>> try 2.3.2 (http://www.couchbase.com/communities/c-client-library)
>>>
>>> I would also suggest you try to run your tests with the new 2.4.0-beta
>>> version to see how that goes. Get it here
>>> http://packages.couchbase.com/clients/c/index.html
>>>
>>>
>>> On 07/17/2014 04:23 AM, Chen Sun wrote:
>>> I am a new guy to couchbase.I try to use couchbase and couchbase's
>>> python client in my python-base web APIs.But I find some errors when I
>>> use ab(a tool to test the web api performence) to test the api with some
>>> concurrecny http requests. The errors of the python clinet throwed
>>> bellow here.I want know why these errors occoured,and how to solve this
>>> problem.
>>>
>>> The errors
>>> ---------------------------------------------------------------------------------------------------------------------------
>>>
>>> uwsgi: src/server.c:666: lcb_server_purge_implicit_responses: Assertion
>>> `c->cmd_log.nbytes' failed.
>>> python-couchbase: self->nremaining == 0 at src/oputil.c:77. AbortTraceback
>>> (most recent call last):
>>> File "api.py", line 111, in falcon.api.API.__call__ (falcon/api.c:1389)
>>> # startkey=start,
>>> endkey=end,
>>> File "api.py", line 96, in falcon.api.API.__call__ (falcon/api.c:1210)
>>> # end = '/' + adid + '/' + end
>>> File "api.py", line 174, in on_post
>>> query_results = couchbase_category.get_multi(keys, quiet=True)
>>> File "/usr/local/lib/python2.7/dist-packages/couchbase/connection.py",
>>> line 1053, in get_multi
>>> return _Base.get_multi(self, keys, ttl=ttl, quiet=quiet,
>>> replica=replica, no_format=no_format)
>>> couchbase.exceptions._TimeoutError_0x17 (generated, catch TimeoutError):
>>> <Key=u'com.htc.fm', RC=0x17[Client-Side timeout exceeded for operation.
>>> Inspect network conditions or increase the timeout], Operational Error,
>>> Results=34, C Source=(src/multiresult.c,286)>
>>>
>>> ------------------------------------------------------------------------------------------------------------------------------------
>>>
>>> My python code
>>>
>>> couchbase_category= Couchbase.connect(
>>>
>>> bucket='Category', host='127.0.0.1', timeout=5)
>>> class Category:
>>>
>>> def on_post(self, req, resp):
>>>
>>> t1= datetime.datetime.now()
>>> content= {}
>>> # try:
>>> params= json.load(req.stream, 'utf-8')
>>> keys= []
>>> for appin params['apps']:
>>>
>>> keys.append(app['bundle_id'])
>>>
>>> unknown_apps= []
>>> result= {}
>>> result['known_bundle_ids'] = []
>>> result['unknown_bundle_ids'] = []
>>> result['apps'] = []
>>> result['categorys'] = []
>>>
>>> query_results= couchbase_category.get_multi(keys, quiet=True)
>>> for keyin query_results:
>>>
>>> if query_results[key].success:
>>> result['apps'].append(query_results[key].value)
>>> result['known_bundle_ids'].append(key)
>>> result['categorys'].append(
>>> query_results[key].value['categoryName'])
>>> else:
>>> result['unknown_bundle_ids'].append(key)
>>> result['categorys'] = list(set(result['categorys']))
>>> for itemin params['apps']:
>>>
>>> if item['bundle_id'] in result['unknown_bundle_ids']:
>>> unknown_apps.append(item)
>>> timestamp= time.time()
>>> pipe= r.pipeline()
>>> for appin unknown_apps:
>>>
>>> pipe.zadd('uncached_apps', timestamp, app)
>>> pipe.execute()
>>> content['is_ok'] = True
>>> content['result'] = result
>>> t2= datetime.datetime.now()
>>>
>>> content['total_time'] = (t2 - t1).total_seconds()
>>> # except Exception, err:
>>> # content['is_ok'] = False
>>> # finally:
>>> resp.status = falcon.HTTP_201
>>> resp.body = json.dumps(content)
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Couchbase" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected]
>>> <mailto:[email protected]>.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Couchbase" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Couchbase" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to [email protected].
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Couchbase" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Couchbase" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to a topic in the Google
> Groups "Couchbase" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/couchbase/mhiJsqPkk7g/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Couchbase" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Couchbase" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
