> 2011/9/18 Roberto De Ioris <[email protected]>: >> > > I will use uWSGI to host a single high load app. (with alot of > incoming requests.. maybe some of them will be tough and take more > time.. so am thinking about async.) > I don`t understand what you wrote. First you saying that it is only > possible to host multiple apps using multiple interpreters.
I mean that multiple interpreters are used ONLY to host multiple apps, but they are not required for this purpose (you can host all of your apps in the same interpreter, obviously they will share modules and objects). So, multiple interpreters are only a form of app-isolation then in > the last sentence - that i must disable multiple interpreters. > Anyway - if i would use global module level vars like for example: > > ------ my_module.py ---- > > some_global_state_var = 1 > > --- my_app_.py ---- > import my_module > def application(...): > my_module.some_global_state_var = value > .... > # code using some_global_state_var here > ---------------------------- > I think this will break with concurrently incoming connections in a > single interpreter. > I am just asking of how to write my application. some_global_state_var will be shared by all of the cores. Albeit technically possibile, you would not want to use an interpreter per-core, as the context-switch is very heavy (you have to swap the python thread-state) >>> if my app use module level variables with async mode will it be messed >>> by context switch? yes, each core will overwrite the same object >> > hmm... i can understand you design of passing through env dict > env['uwsgi.core'] and env['.. timeout'] but construction > request[env['uwsgi.core']].something > does`t look elegant. I think this is mutch like a threadlocals. yes, very similar in the result. It could be inelegant, but you have not much choice for storing "local-values". Maybe you can use some design-pattern to abstract it (check the "request" object in the flask framework, this is an amazing example of a fake-global object that is really a local object) > The thing that bothers me - i will use lxml and maybe 1-2 other 3rd > parity libs (zlib for example) and how it will work out with async > mode inside signle interpreter.. hmm.. Can i use separate interpreters > for each async thread ? (i understand that this is memory waste but.. > this is much more safe) i see no problems if you pay attention to work only on core-specific objects > > Also a few days ago i saw a real specification at > http://wsgi.org/wsgi/Specifications/fdevent > and now a page redirect or gives 404. uWSGI is probably the only one server implementing it, but this is sub-optimal and i think no-one is currently using it. Its biggest limit is the inhability to wait for multiple fds. > > Well stackless is not a option obviously. > Eventlet - well it is on top of libevent and thus have some limitations. > Gevent - is moving to libev and think old code (on libevent) is gonna be > leaved. yes, the gevent uWSGI plugin supports only the new branch (libev-based) > Here http://blog.gevent.org/ Denis write about moving to libev and > bugs in libevent. > Also the stability of both projects seems unclear for me as i cant > find any big production companies using it. i do not know for eventlet (maybe ep.io uses it ?), but gevent is used by spotify and it looks like heroku has choosen it for its python platform. Honestly i give trust to both, but i would never run them without a process controller like uWSGI :) > (i also checked a bit gevent alpha and it did showed really strage and > slow results on gethostbyname example 6 seconds vs 0.2 of stable > gevent. Denis said something about c-ares but on other tasks 1.0 alpha > was also slower.. it just show me that it is too raw for production. ) libev has no dns facility, and this limit beat new gevent in the head. c-ares is a beautiful library, and probably it is the right choice (i am even thinking about including an async dns-resolver in uWSGI itself based on c-ares). But yes, currently gevent 1.0 dns support is a bit flaky. > > I think my best choice is uGreen :) (it is ok that it is low level) > but i cant find a good examples of using it. only a tiny iobound_async.py > although uwsgi.wait_fd_write(c, 2) looks a litle bit odd after > standard poll stuff. it is build on top of new kernel facilities (epoll/kqueue/devpoll/ports) so you do not have a list of fds but simply add them to the poll list: uwsgi.wait_fd_read(1, 2) uwsgi.wait_fd_read(2, 2) uwsgi.wait_fd_read(3, 2) uwsgi.wait_fd_read(4, 2) uwsgi.suspend() will register fd 1,2,3 and 4 in the poll system (with a timeout of 2 secs) and will suspend execution until one of them is ready. > btw: if i registered 10 descriptors for write and 10 for read.. and then > uwsgi > says yay there is event on fd5 .. should i again reregister fd? or > registration is persistent? you have to re-register events after each context-switch (this is needed to be compliant with fdevent) > timeout parameter - individual for each fd? yes > i think uwsgi.async_connect,uwsgi.is_connected, uwsgi.recv - undocumented. as uwsgidecorators.py i would like to write a uwsgiasync.py abstraction. It will be useful as code base but more as a doc base. > how does uwsgi.send behaves? switches context if would block until all > data is written? that's the idea :) > should i intercept ordinary socket errors on uwsgi.send? an exception would rise if write() fails for some reason > > Maybe you can provide a little bit more complex example using more > than 1 socket and registering its with wait_fd_read/write in a loop? i will try to find the time for uwsgiasync.py > > And also - is it possible to create a new async thread for every > incoming connection instead of statically > setting max number at start? or it is a limitation of posix > makecontext family calls? it is only for performance reason, when you start uWSGI in async mode you will get the amount of memory allocated for cores. You will see that a single core uses less than 2 pages. -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
