This is really interesting, specially the scheduler and the possibility of sharing the redis connection. Is there any ETA to have it in stable?
Thanks Niphlod 2016-01-03 0:44 GMT+01:00 Niphlod <[email protected]>: > Once PR1158 <https://github.com/web2py/web2py/pull/1158>gets merged > web2py will have a preferred way to use redis. It's through the new > gluon.contrib.redis_utils "RConn". > Some redis libraries are coming along and redis-py, while maintaining a > solid "#1 position" among them, could be one day in #2. > Moreover, connection parameters passed to redis have been added and it's > been almost impossible to keep web2py's modules that use redis in sync. > It'd have meant to lock a particular release of redis-py to a particular > release of web2py's contrib redis_* modules. > Seen that and issue#958 <https://github.com/web2py/web2py/issues/958>, a > refactor was needed. Better late than ever, the refactor is here. > > This bring BREAKING changes for the next version of web2py. Anyone using > gluon.contrib.redis_cache and/or gluon.contrib.redis_session will need to > alter app's code to match the new behaviour. > Basically now every module in web2py that uses redis needs (and should, > for new modules you may want to contrib) to have a redis_conn parameter > that takes a redis instance. That redis instance can be provided out of the > box - tuned for the unique environment of web2py - with > > from gluon.contrib.redis_utils import RConn > rconn = RConn() > > now, RConn takes ALL parameters and passes them unchanged down the pipe > directory to redis.StrictRedis . > > This means that any parameter that was used to connect to redis for those > modules isn't there anymore. > Specifically, if you had a redis instance available at 10.0.0.10 on port > 5555, your instantiation of RedisCache was similar to > > from gluon.contrib.redis_cache import RedisCache > cache.redis = RedisCache('10.0.0.10:5555',debug=True) > > from here on, it'll be instead > > from gluon.contrib.redis_utils import RConn > from gluon.contrib.redis_cache import RedisCache > rconn = RConn(host='10.0.0.10', port=5555) > cache.redis = RedisCache(redis_conn=rconn,debug=True) > > want to connect to redis through ssl (previously unavailable)? pass the > relevant ssl, ssl_keyfile, etc etc etc to RConn. > > You can also reuse the rconn object as if it was a redis.StrictRedis > instance in your app's code. Everything will work fine. Just use a sane key > prefix (all modules use "w2p") in order to avoid conflicts. redis_cache, > redis_session and redis_scheduler (and your app) can use a single RConn > instance without issues. > > Was this only a refactor ? Nope! > There is also a new (not so shiny, 'cause - guilty as charged - I'm using > it for 10 months in production now) redis-backed scheduler. > Of course it's experimental (as everything in contrib) but it's not > untested (all w2p_scheduler_tests > <http://github.com/niphlod/w2p_scheduler_tests> passed). > This is one of the first modules completely coded on Windows (sigh). Linux > has been tested too through w2p_scheduler_tests (still waiting for someone > to come up with unittests that will run on our CI) but the production > environment kicking for 10 months is WS2012R2 with redis 3.0 . > RScheduler is a slip-in of the factory scheduler, with the - usual - > redis_conn parameter. > > from gluon.contrib.redis_utils import RConn > from gluon.contrib.redis_scheduler import RScheduler > rconn = RConn() > mysched = RScheduler(db, ..., redis_conn=rconn) > > This first release basically moves everything happening in the > scheduler_worker table to redis, to alleviate locking and contention where > it happens the most. > It'll probably enable usage patterns for who had problems in production > with a high number of workers. It's also a lot snappier because with redis > tasks gets from QUEUED to RUNNING without passing from ASSIGNED. Use it, > test it, break it, fix it (or report bugs). Read the source code, improve > it, move ideas around.... just crunch tasks...do whatever you want with it. > Hope you'll find it as useful as I am. > > -- > Resources: > - http://web2py.com > - http://web2py.com/book (Documentation) > - http://github.com/web2py/web2py (Source code) > - https://code.google.com/p/web2py/issues/list (Report Issues) > --- > You received this message because you are subscribed to the Google Groups > "web2py-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
