Once PR1158 <https://github.com/web2py/web2py/pull/1158>gets merged web2py
will have a preferred way to use redis. It's through the new
gluon.contrib.redis_utils "RConn".
Some redis libraries are coming along and redis-py, while maintaining a
solid "#1 position" among them, could be one day in #2.
Moreover, connection parameters passed to redis have been added and it's
been almost impossible to keep web2py's modules that use redis in sync.
It'd have meant to lock a particular release of redis-py to a particular
release of web2py's contrib redis_* modules.
Seen that and issue#958 <https://github.com/web2py/web2py/issues/958>, a
refactor was needed. Better late than ever, the refactor is here.
This bring BREAKING changes for the next version of web2py. Anyone using
gluon.contrib.redis_cache and/or gluon.contrib.redis_session will need to
alter app's code to match the new behaviour.
Basically now every module in web2py that uses redis needs (and should, for
new modules you may want to contrib) to have a redis_conn parameter that
takes a redis instance. That redis instance can be provided out of the box
- tuned for the unique environment of web2py - with
from gluon.contrib.redis_utils import RConn
rconn = RConn()
now, RConn takes ALL parameters and passes them unchanged down the pipe
directory to redis.StrictRedis .
This means that any parameter that was used to connect to redis for those
modules isn't there anymore.
Specifically, if you had a redis instance available at 10.0.0.10 on port
5555, your instantiation of RedisCache was similar to
from gluon.contrib.redis_cache import RedisCache
cache.redis = RedisCache('10.0.0.10:5555',debug=True)
from here on, it'll be instead
from gluon.contrib.redis_utils import RConn
from gluon.contrib.redis_cache import RedisCache
rconn = RConn(host='10.0.0.10', port=5555)
cache.redis = RedisCache(redis_conn=rconn,debug=True)
want to connect to redis through ssl (previously unavailable)? pass the
relevant ssl, ssl_keyfile, etc etc etc to RConn.
You can also reuse the rconn object as if it was a redis.StrictRedis
instance in your app's code. Everything will work fine. Just use a sane key
prefix (all modules use "w2p") in order to avoid conflicts. redis_cache,
redis_session and redis_scheduler (and your app) can use a single RConn
instance without issues.
Was this only a refactor ? Nope!
There is also a new (not so shiny, 'cause - guilty as charged - I'm using
it for 10 months in production now) redis-backed scheduler.
Of course it's experimental (as everything in contrib) but it's not
untested (all w2p_scheduler_tests
<http://github.com/niphlod/w2p_scheduler_tests> passed).
This is one of the first modules completely coded on Windows (sigh). Linux
has been tested too through w2p_scheduler_tests (still waiting for someone
to come up with unittests that will run on our CI) but the production
environment kicking for 10 months is WS2012R2 with redis 3.0 .
RScheduler is a slip-in of the factory scheduler, with the - usual -
redis_conn parameter.
from gluon.contrib.redis_utils import RConn
from gluon.contrib.redis_scheduler import RScheduler
rconn = RConn()
mysched = RScheduler(db, ..., redis_conn=rconn)
This first release basically moves everything happening in the
scheduler_worker table to redis, to alleviate locking and contention where
it happens the most.
It'll probably enable usage patterns for who had problems in production
with a high number of workers. It's also a lot snappier because with redis
tasks gets from QUEUED to RUNNING without passing from ASSIGNED. Use it,
test it, break it, fix it (or report bugs). Read the source code, improve
it, move ideas around.... just crunch tasks...do whatever you want with it.
Hope you'll find it as useful as I am.
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
