On Sun, Sep 18, 2011 at 12:40 AM, Roberto De Ioris <[email protected]> wrote: > > Hi, pretty long stuff, i will answer point by point below :)
yeah, sorry about that, much appreciated though ... i've had this message flagged 4+ months ... time to reply. >> `--worker-exec` describes "controlling a pool of fast-cgi processes >> [php]" ... i could get the [php] processes running, and execute python >> from the master process *once*, and afterwards the children/workers >> would receive requests directly ... this is all fine/makes sense ... >> but how would you suggest to execute python again? the spooler runs >> in a separate process (yes?) and thus it could be used as a way to >> control the "fcgi pool" via uwsgi module/etc. i never tried signals >> but that seems plausible as well ... in short i want to use >> python/uwsgi to manage the php i _have_ to deal with; any ideas? make >> a dumb php pool and proxy to it [see next paragraph]? how do i >> properly (non-locking?) control this python pack leader? i'm >> especially interested in combining this with zerg mode and scaling >> (via an "intelligent python framework") vs. php's own process manager >> ... > > when you set a binary with --worker-exec, those workers will lose all of > the old process image. No way to came back to python. Only the master and > the spooler continue to maintain the old image. An intersting thing is > that you can send signals (or tasks) via php to the spooler (simply a one > byte write() ) as the signal file descriptor is inherited. A solution for > your problem will be allowing worker-exec to be mapped only to a list of > workers: > > --processes 8 --worker-exec /usr/bin/php-cgi:1,2,3,4 > > will run php only on the first 4 nodes, the others will continue normally. > > By the way, Zerg mode works without problems in --worker-exec mode ok, i knew i couldn't return to python from php, i mainly wanted to know which uWSGI children would remain in a python image for reception of signals/etc ... seems spooler/mules/master are the main ones. i'll have to think about this a bit more ... since PHP SAPI support was added recently i may no longer need worker-exec. the wiki specifically said "controlling a pool of fast-cgi PHP processes", and i wanted to know *how* :-) >> ... once the php is running, how do i actually serve a request to it >> via uwsgi alone? do i need 2 instances of uwsgi? i tried >> `--http`/`--http-socket` but the children expected fastcgi protocol >> and thus failed when given HTTP. i tried various combinations of >> `--fastcgi-socket` and anything that seemed to make sense ... but what >> is the "proper" way :-)? i want to use uwsgi alone (multiple is fine >> if needed) to deliver an HTTP request to the PHP processes ... so, how >> do i make uwsgi speak both ends of the fastcgi proto? > > currently the uwsgi http router can route only to uwsgi servers. Adding > fastcgi support would be easy as all of the functions are already > available > for the uwsgi.fcgi() function: > > http://projects.unbit.it/uwsgi/wiki/CustomRouting > > (check part 8, you could even use it to route from uwsgi to php right now) this [part 8] does work, but has anything changed here since 2011-18-09? i'd still like to make use of the php-fpm fastcgi server, even if i also use some via uWSGI SAPI, and i'd rather not proxy thru a python handler ... would --protocol fastcgi do the trick? any way to speak to a fastcgi process directly? >> i'm loving the embedded stuff. in the spirit of ROCK SOLID >> mode/build, how about a SEALED or FROZEN mode where the only config >> allowed are internal `sym://config_ini` (or `sym://*`, as long as it's >> embedded [think includes])? this way the "authoritative" config is >> locked away internal to the process and not easily changeable -- no >> command line args would be accepted ... > > i suppose that a whitelist/blacklist option subsystem would be much more > useful, expecially for cloud/paas service providers that want to leave a > bit of control to their customer instances. > > a 'blacklist = all' will create a "frozen" binary. this is something i'd really like ... the ability to *disable* runtime argument processing *completely* (or partially as you suggest) ... uWSGI becomes a transparent container for my app, any arguments passed to uWSGI are simply passed to my master process normally via `sys.argv`. ... basically i'd love a way to: a) embed options at compile time (already possible via embedded_config, IIRC) b) disable *all* forms of runtime argument processing c) populate sys.argv with CLI args, if any, passed at runtime ... or equivalent. this makes uWSGI behave like the normal python interpreter (albeit 100x awesom-er); does all that makes sense? wrap my application in a hard super-shell, and pretend to be a little more than an embedded interpreter. >> ... could even be a slick way to write a uwsgi-based control app for >> uwsgi itself ... fire up uwsgi (`--no-server`?), and send some >> commands to wherever, with it's own CLI :-) > > this could be already accomplished with --pyshell (and --rbshell for ruby > [currently broken]) i tried this and couldn't get it to work right, or i misunderstood. additional arguments following --pyshell are still interpreted as [bogus] uWSGI options. per my previous paragraphs, i'd like all command-line arguments to be treated as app arguments -- passed thru as sys.argv -- and uWSGI arguments predefined internally (eg, via embedded_config). ... additionally, in general, it might be useful to support the `--` "end argument processing" idiom. in this case, all args following a `--` populate sys.argv ultimately, i want to use uWSGI as a completely transparent, self-contained, distributable wrapper for my apps, where end-user can run the "uWSGI-enhanced application binary" directly, without affecting the internal configuration, and supporting it's own app-specific arguments (ie, via sys.argv). >> [... snip me rambling about how to accomplish 100% self-containment ...] > > it is all wonderful, but sadly no unix-variant out-there has support for > exec/dlopen of stream of data (they must be real files). > > For example i am embedding psycopg2 in my uWSGI binary compiling it as a > static library (.a) and linking to uWSGi itself (that will call > initpsycopg2() on startup). Really hard job, but it is currently working. > The problem is that automating this is pratically impossibile :( > > Embedding python is possible (simply link uWSGI with the static libpython > variant, this is the default on some distro that do not compile python > with --enable-shared), but you will need its core-modules. You can put the > whole /usr/lib/pythonX.X in zip and it works out of the box, but you will > always require an additional file (the .zip one) interesting, thanks ... requiring a single zip file is perfectly fine; would it be possible to include that zip file (ie. the entire python stdlib) as a data://N object? i'll play around with this a bit later ... my C skills are sub-n00b at best, but have to start somewhere :-) >> how about `(sym|data)://` loading a pre-compiled python module? right >> now it pukes expecting a python source file. this could possibly be >> extended generically to support multiple python impls via the py3k >> __pycache__ lookup system ... is there any benefit to this? besides >> maybe speed, my other thought was obscuring the python module's source >> from simple eyes. > > it is doable, maybe it is only a two-line fix. not sure if this was implemented off-hand; is this a reasonable request? i'll open a feature request if so. >> ... and similar for `sym://config_ini` -- possibility to compress/etc >> so it's not plain text in the binary? im also thinking about total >> binary size -- plain text files get excellent compression. >> >> i see interesting uses like encrypted/verified boot of sorts sprouting >> from this and previous notes. > > compression/encryption functions will require new dependancies and a lot > of code. This code should go to plugins (no need to increase the already > big core). i will investigate on this after 1.0 sounds good :-) just brainstorming. >> i was unable to embedded a config with path components, eg. >> `sym://path_to_config_ini` failed with "unknown symbols/first use", >> etc ... reproduce by setting: >> >> embed_config = path/to/config.ini >> >> ... in your build conf. lmk if i need to expand this one, probably an >> easy fix. > > the ld embedding subsystem is a bit "strange". It will rename your full path > to something like _path_to_config_ini. > > use objdump -t <binary_name> to get the assigned symbol name do i need to make sure embedded files are always in the working directory then? ... thanks again for replying to all these ... in the future i'll not let questions accumulate so long, and also try to curb my natural verbosity ... -- C Anthony _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
