Doug and Ryan, Thank you for your opinion! It is very important for us to gather experience as more as we can for passing OpenStack incubation painlessly. Also I would be glad to see you and everyone who is interested in MagnetoDB project on MagnetoDB design session "MagnetoDB, key-value storage. OpenStack usecases" in scope of OpenStack Juno Design Summit.
On Wed, Mar 19, 2014 at 3:49 PM, Ryan Petrello <[email protected]>wrote: > Dmitriy, > > Gunicorn + gevent + pecan play nicely together, and they're a combination > I've used to good success in the past. Pecan even comes with some helpers > for integrating with gunicorn: > > $ gunicorn_pecan pecan_config.py -k gevent -w4 > > http://pecan.readthedocs.org/en/latest/deployment.html?highlight=gunicorn#gunicorn > > --- > Ryan Petrello > Senior Developer, DreamHost > [email protected] > > On Mar 18, 2014, at 2:51 PM, Dmitriy Ukhlov <[email protected]> wrote: > > > Hello openstackers, > > > > We are working on MagnetoDB project and trying our best to follow > OpenStack standards. > > > > So, MagnetoDB is aimed to be high performance scalable OpenStack based > WSGI application which provide interface to high available distributed > reliable key-value storage. We investigated best practices and separated > the next points: > > * to avoid problems with GIL our application should be executed in > single thread mode with non-blocking IO (using greenlets or another python > specific approaches to rich this) > > * to make MagnetoDB scalable it is necessary to make MagnetoDB > stateless. It allows us run a lot of independent MagnetoDB processes and > switch all requests flow between them: > > * at single node to load all CPU's cores > > * at the different nodes for horizontal scalability > > * use Cassandra as most reliable and mature distributed key-value > storage > > * use datastax python-driver as most modern cassandra python > client which supports newest CQL3 and Cassandra native binary protocol > features set > > > > So, considering this points The next technologies was chosen: > > * gevent as one of the fastest non-blocking single-thread WSGI > server. It is based on greenlet library and supports monkey patching of > standard threading library. It is necessary because of datastax python > driver uses threading library and it's backlog has task to add gevent > backlog. (We patched python-driver ourselves to enable this feature as > temporary solution and waiting for new python-driver releases). It makes > gevent more interesting to use than other analogs (like eventlet for > example) > > * gunicorn as WSGI server which is able to run a few worker > processes and master process for workers managing and routing request > between them. Also it has integration with gevent and can run > gevent based workers. We also analyzed analogues, such as uWSGI. It looks > like more faster but unfortunately we didn't manage to work uWSGI in multi > process mode with MagnetoDB application. > > > > Also I want to add that currently oslo wsgi framework is used for > organizing request routing. I know that current OpenStack trend is to > migrate WSGI services to Pecan wsgi framework. Maybe is it reasonable for > MagnetoDB too. > > > > We would like to hear your opinions about the libraries and approaches > we have chosen and would appreciate you help and support in order to find > the best balance between performance, developer friendness and OpenStack > standards. > > -- > > Best regards, > > Dmitriy Ukhlov > > Mirantis Inc. > > > > _______________________________________________ > > OpenStack-dev mailing list > > [email protected] > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Best regards, Dmitriy Ukhlov Mirantis Inc.
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
