> On 02/24/2013 01:02 AM, Roberto De Ioris wrote: >>> Hello, >>> >>> I am using the uWSGI cache feature to reduce the number of DB queries. >>> The >>> flow looks like this (code is written in Python): >>> >>> request comes in: >>> >>> value = cache_get(key) >>> if value: >>> return value >>> else: >>> value = db_query(...) >>> cache_set(key, pickle.dumps(value), 60) >>> >>> It works well for few minutes. After that, it calls cache_set for some >>> keys more than once every 60 second (once a second or more frequently). >>> I >>> would expect cache_set to not be called more than once every 60 seconds >>> because I pass expires=60 to cache_set. >>> >>> cache_get doesn't return anything even if cache_set succeeds (return >>> value >>> is True). This is what worries me. >>> >>> Let me know if you need more details. >>> >>> uWSGI config: >>> >>> [uwsgi] >>> socket=/var/run/uwsgi.sock >>> master=true >>> enable-threads=true >>> listen=8192 >>> socket-timeout=60 >>> buffer-size=8192 >>> pidfile=/var/run/uwsgi.pid >>> module=app.start >>> pythonpath=blabla >>> processes=8 >>> threads=0 >>> daemonize=/var/log/uwsgi.log >>> disable-logging=false >>> logto=/var/log/uwsgi-access.log >>> logdate=true >>> no-orphans=true >>> vacuum=true >>> uid=user >>> gid=group >>> cache=10000 >>> cache-blocksize=4096 >>> cache-report-freed-items=true_______________________________________________ >>> uWSGI mailing list >>> [email protected] >>> http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi >>> >> >> Just two things: >> >> - be sure to run a version >= 1.4.4 as older version had a corner-case >> bug >> in which collisioned keys could be swallowed (if you have lot of >> collisions you could have hit that) >> >> - remember that cache_set does not overwrite already-existent keys. Use >> cache_update if you want auto-overwrite. >> > I am using 1.4.5. I used to encounter the deadlock issue with 0.9.9.2 so > I moved to the latest version recently. > > What I am seeing right now happens rarely but it fills up the cache > (10000 elements) even if I am only trying to store about 80 keys. When > that issue happens, I see cache_set being called over and over for the > same key and cache_get keeps returning None. It quickly fills up the > cache even if cache_set is called for the same key. > > cache is empty. > cache_set: key1, expires 60. > cache_set: key1, expires 60. > cache_get: key1 -> None. > cache_set: key1, expires 60. > cache_get: key1 -> None. > ... > cache is reported as full in the log. > > cache_set is called for the same key from multiple workers but with the > uWSGI locking mechanism, it should not be an issue and cache_get should > return something after the first cache_set succeeds. > > I am puzzled right now. > _______________________________________________ > uWSGI mailing list > [email protected] > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi >
Have you noted if the problem starts happening after more than an hour or earlier ? the only thing popping in my mind is some bug in the auto-expiration system (80 keys with 60 seconds expiration should start giving problems pretty late on a 10k store) -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
