Dmitriy, Gunicorn + gevent + pecan play nicely together, and they’re a combination I’ve used to good success in the past. Pecan even comes with some helpers for integrating with gunicorn:
$ gunicorn_pecan pecan_config.py -k gevent -w4 http://pecan.readthedocs.org/en/latest/deployment.html?highlight=gunicorn#gunicorn --- Ryan Petrello Senior Developer, DreamHost [email protected] On Mar 18, 2014, at 2:51 PM, Dmitriy Ukhlov <[email protected]> wrote: > Hello openstackers, > > We are working on MagnetoDB project and trying our best to follow OpenStack > standards. > > So, MagnetoDB is aimed to be high performance scalable OpenStack based WSGI > application which provide interface to high available distributed reliable > key-value storage. We investigated best practices and separated the next > points: > • to avoid problems with GIL our application should be executed in > single thread mode with non-blocking IO (using greenlets or another python > specific approaches to rich this) > • to make MagnetoDB scalable it is necessary to make MagnetoDB > stateless. It allows us run a lot of independent MagnetoDB processes and > switch all requests flow between them: > • at single node to load all CPU’s cores > • at the different nodes for horizontal scalability > • use Cassandra as most reliable and mature distributed key-value > storage > • use datastax python-driver as most modern cassandra python client > which supports newest CQL3 and Cassandra native binary protocol features set > > So, considering this points The next technologies was chosen: > • gevent as one of the fastest non-blocking single-thread WSGI server. > It is based on greenlet library and supports monkey patching of standard > threading library. It is necessary because of datastax python driver uses > threading library and it’s backlog has task to add gevent backlog. (We > patched python-driver ourselves to enable this feature as temporary solution > and waiting for new python-driver releases). It makes gevent more interesting > to use than other analogs (like eventlet for example) > • gunicorn as WSGI server which is able to run a few worker processes > and master process for workers managing and routing request between them. > Also it has integration with gevent and can run gevent based > workers. We also analyzed analogues, such as uWSGI. It looks like more faster > but unfortunately we didn’t manage to work uWSGI in multi process mode with > MagnetoDB application. > > Also I want to add that currently oslo wsgi framework is used for organizing > request routing. I know that current OpenStack trend is to migrate WSGI > services to Pecan wsgi framework. Maybe is it reasonable for MagnetoDB too. > > We would like to hear your opinions about the libraries and approaches we > have chosen and would appreciate you help and support in order to find the > best balance between performance, developer friendness and OpenStack > standards. > -- > Best regards, > Dmitriy Ukhlov > Mirantis Inc. > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
