Re: [web2py] uwsgi question
Hello Peter. I've got also a problem with scheduler/sqlite, OperationalError: database is locked. Have you found a solution? Thanks. You can't generally solve it. Sqlite has a pretty weak concurrent support, so if a process try to do something while another has initiated a still-uncommitted transaction you get that error. You should find various posts about it (for the various languages), there is some trick, but generally if you need strong concurrency sqlite is not a good solution. -- Roberto De Ioris http://unbit.com -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [web2py] nginx and uwsgi: upstream sent too big header while reading response header from upstream
This error is giving met a lot of trouble. I have read about solutions using something like fastcgi_buffers 8 256k; fastcgi_buffer_size 128k; where nginx is using fastcgi, but I am working with uwsgi and this does not help me. Am I the only web2py user with this problem? I will be glad to hear about solutions from other web2py-users. Regards Johann nginx buffers can be set for all of the supported protocols: http://wiki.nginx.org/HttpUwsgiModule#uwsgi_buffer_size -- Roberto De Ioris http://unbit.it -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [web2py] web2py uwsgi
Hi, I'm on os x 10.6.8 machine and I'm trying to run web2py with uwsgi. I'm following the tutorial on http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html. Downloaded and compiled uwsgi. when I run uwsgi with the ini file I'm not able to connect to the admin interface. In the browser I give https://localhost:9090/admin/default/index If I run ./uwsgi --http :9090 --wsgi-file /Users/xaver/web2py/web2py/src/wsgihandler.py --master --process 4 --threads 2 --stats 127.0.0.1:9191 it generate an error ticket which last lines are: S'Traceback (most recent call last):\n File /Users/xaver/web2py/web2py/src/gluon/restricted.py, line 217, in restricted\nexec ccode in environment\n File /Users/saverioscavelli/web2py/web2py/src/applications/admin/models/access.py, line 29, in module\nport = int(request.env.server_port or 0)\nValueError: invalid literal for int() with base 10: \'9090,foobar.crt,foobar.key\'\n' Another problem I'm having is with the virtualenv. In the above cited tutorial it says that is enough to simple make an entry in the iniffile eg. virtualenv=/path/to/the/virtualenv and uwsgi will find all the needed stuff, but that doesn't work for me: uWSGI http bound on :9090,foobar.crt,foobar.key fd 4 uwsgi socket 0 bound to TCP address 127.0.0.1:56782 (port auto-assigned) fd 3 Python version: 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) [GCC 4.2.1 (Apple Inc. build 5646)] Set PythonHome to /Users/xaver/web2py/web2py/ 'import site' failed; use -v for traceback this is the output after running make: pcre = True kernel = Darwin malloc = libc execinfo = False ifaddrs = True ssl = True zlib = True locking = osx_spinlock plugin_dir = . timer = kqueue yaml = embedded json = False filemonitor = kqueue routing = True debug = False capabilities = False xml = libxml2 event = kqueue How you can see ssl is compiled with. and this is my uwsgi ini file: [uwsgi] http = :9090,foobar.crt,foobar.key it is https not http. certificates on plain http have no meaning -- Roberto De Ioris http://unbit.it -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] web2py uwsgi
On Sunday, March 2, 2014 1:58:31 PM UTC+1, Roberto De Ioris wrote: Hi, I'm on os x 10.6.8 machine and I'm trying to run web2py with uwsgi. I'm following the tutorial on http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html. Downloaded and compiled uwsgi. when I run uwsgi with the ini file I'm not able to connect to the admin interface. In the browser I give https://localhost:9090/admin/default/index If I run ./uwsgi --http :9090 --wsgi-file /Users/xaver/web2py/web2py/src/wsgihandler.py --master --process 4 --threads 2 --stats 127.0.0.1:9191 it generate an error ticket which last lines are: S'Traceback (most recent call last):\n File /Users/xaver/web2py/web2py/src/gluon/restricted.py, line 217, in restricted\nexec ccode in environment\n File /Users/saverioscavelli/web2py/web2py/src/applications/admin/models/access.py, line 29, in module\nport = int(request.env.server_port or 0)\nValueError: invalid literal for int() with base 10: \'9090,foobar.crt,foobar.key\'\n' Another problem I'm having is with the virtualenv. In the above cited tutorial it says that is enough to simple make an entry in the iniffile eg. virtualenv=/path/to/the/virtualenv and uwsgi will find all the needed stuff, but that doesn't work for me: uWSGI http bound on :9090,foobar.crt,foobar.key fd 4 uwsgi socket 0 bound to TCP address 127.0.0.1:56782 (port auto-assigned) fd 3 Python version: 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) [GCC 4.2.1 (Apple Inc. build 5646)] Set PythonHome to /Users/xaver/web2py/web2py/ 'import site' failed; use -v for traceback this is the output after running make: pcre = True kernel = Darwin malloc = libc execinfo = False ifaddrs = True ssl = True zlib = True locking = osx_spinlock plugin_dir = . timer = kqueue yaml = embedded json = False filemonitor = kqueue routing = True debug = False capabilities = False xml = libxml2 event = kqueue How you can see ssl is compiled with. and this is my uwsgi ini file: [uwsgi] http = :9090,foobar.crt,foobar.key it is https not http. certificates on plain http have no meaning Hi, thks. for the answer. But please be a bit more verbose. Not matter how I call it: over http or https I can't acces the admin interface of web2py mybe the problem lays to the web2py security restriction. Tkhs. I was referring to the uWSGI option: NOT http = :9090,foobar.crt,foobar.key BUT https = :9090,foobar.crt,foobar.key http = does not take certificates as parameter so your SERVER_PORT will be mapped to the string 9090,foobar.crt,foobar.key that is the cause of the first reported exception -- Roberto De Ioris http://unbit.it -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: uwsgi respawning
OK. My real doubt is more about how uwsgi works. It scanned through many files and probably loaded quite a bit of those in memory. When a new process is spawned, will this (buffer) be passed to that? If so, it is better to increase the limit. On the other than hand, if it is some bootstrap process that is taking memory and other processes that come along won't take a lot of memory, I would rather let this die and spawn a new one. Unless you enable caching or queueing, or other shared memory features, the memory usage of uWSGI itself is between 800k and 1.8 megs. All of the other memory is related to the language vm (python) and your app. What you need to check if the memory usage grows after each request. In such a case you a have a leak to spot in your app. -- Roberto De Ioris http://unbit.it -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: uwsgi respawning
In my case, the rss is pretty OK (a 4 GB machine, 12 workers). But address
space is high -
{address space usage: 284536832 bytes/271MB} {rss usage: 22073344
bytes/21MB}
And why you are worried ? It is virtual memory, i could allocate 2GB of
address space without wasting more than few k of real memory with a simple
mmap() call.
There is nothing wrong in high virtual memory usage, you should only care
about RSS. If you want to limit memory usage on Linux use cgroups (they
are way better and works on real memory and swap)
If you are asking who is mapping that virtual memory, run cat
/proc/pid/maps
and you will get the whole memory mapping.
--
Roberto De Ioris
http://unbit.it
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] UWSGI memory error
I am testing out nginx/uwsgi on our production server over an ssh tunnel. The server is working hard (load average of 9-14 constantly) and I am running long running queries (some more than 40 minutes). I have seen this in the logs now and am a bit concerned about it and would like to know how can I find out what is causing it: Thu Nov 21 13:58:18 2013 - Memory Error detected !!! Check if you have limit-as option or some other memory limit in place -- Roberto De Ioris http://unbit.it -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] gunicorn vs uwsgi
Hello, I'm testing both gunicorn and uwsgi with web2py on my (production) server with nginx acting as front-end/reverse-proxy. Deploying of gunicorn is pure simplicity which I like (it reminds me on webpy) and I can access web2py's admin via https, while having problem to do the same with uwsgi. │ My sites are and, after converting to web2py, will be small/low-traffic and can raise to medium/medium-traffic in the future, while on my hosting (ala webfaction) I have limit on the total memory used by apps to 512MB and number of procs is 50. Any idea which WSGI server behave better in such constrained environment? uWSGI certainly looks as very powerful, but maybe it is an overkill for our needs and brings more trouble with configuration comparing it with simple deployment via gunicorn? While the complexity of uWSGI in latest times is questionable (documentation improved a lot and the quickstart introduce to basically everything you need), i always suggest newcomers to go pure-python (rocket, gunicorn, flup...) as compilation can be a problem sometimes. The problem is that once you start placing apps in constrained environments, you will soon start to face low-level problems (dealing with rss, address space, monitoring), something uWSGI is optimized for (for example you can tell it to reboot after a memory limit is reached and so on) In addition to this uWSGI uses lot less memory and supports threading that when you have few resources is a good approach for increasing concurrency. The overkill definition is really funny (you are not the first person saying it) becase generally uWSGI has hundreds more features than competitors but it is the one consuming less resources :) And just as a note (from the web2py directory, after having copied wsgihandler.py from the handlers directory): gunicorn --bind :9090 wsgihandler uwsgi --http-socket :9090 --wsgi wsgihandler is it so different ;) ? -- Roberto De Ioris http://unbit.it -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: gunicorn vs uwsgi
Well, I was mostly thinking if/whether uWSGI's power/advantage is becoming more obvious when serving big web sites with *LOT* of traffic where one wants to squueze last drop of performance. The problem is that when you are just learning, the first objective is getting things done, you are generally not interested in problems you do not even immagine will arise (soon or later) This is why projects like uWSGI, Passenger or the big J2EE application servers seems overpowered to beginners (or to super-lucky people that never had problems with webapps :) It is not (only) a matter of performance, from a sysadmin point of view things are different, that is why i removed developer-friendly from the uWSGI front-page :P python anyserver.py -p localport -s gunicorn wait wait, you are talking about anyserver.py, it is a cool script included in web2py for simplified deployments, you cannot really say you have tried gunicorn and its true power :) regarding the https problem, well, if the password file is correct it should works transparently (uWSGI honour the X-Forwarded-SSL header sent by the webfaction proxy). -- Roberto De Ioris http://unbit.it -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: gunicorn vs uwsgi
regarding the https problem, well, if the password file is correct it should works transparently (uWSGI honour the X-Forwarded-SSL header sent by the webfaction proxy). First, I'm not on webfaction, but on something similar to it and it does not work with uWSGI, while it works with gunicorn. And don't you think it would be better to understand why ? Maybe it is an easy thing, maybe it is something isp-specific that would be great to address. I am not forcing you to use uWSGI (sorry if i gave you such idea) but as you are a developer i suspect when a customer simply says does not work without context you became a fury :) In addition to this, this is open source, collaboration is what makes it great (and better) -- Roberto De Ioris http://unbit.it -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] websockets
Hello. Some questions about websockets: 1) Can websockets be implemented within web2py without any in-socket server side component? 2) If yes, how is the constructor used on a local machine in the constructor, say, as: var ws = new WebSocket(ws://localhost/echo); Thank you! - uWSGI exposes a very fast websockets api you can easily use in web2py too. The problem is that once you complete the websocket handshake a whole thread will remains busy until the connection is not closed. You can reduce the problem spawning multiple threads, but you need a good amount of memory. This is an example echo server: https://github.com/unbit/uwsgi/blob/master/tests/websockets_echo.py basically you call uwsgi.websocket_handshake to ack the connection and then you can use uwsgi.websocket_recv and uwsgi.websocket_send -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Excessive memory usage with uwsgi/nginx
My web2py application is running on Webfaction's servers over uwsgi/nginx. When the servers are started, four instances of nginx and four instances of uwsgi are spawned, with each nginx instance taking up ~18M of RAM, and each nginx instance taking up a couple of M. Accessing any part of the page (i.e. clicking a menu item) adds about 30M to one of the uwsgi instances. Clicking a menu item again can add between 2 and 30 M to any of the uwsgi instances (it will be 30 M if nothing has yet been added to the RAM usage of that particular instance). This happens until my memory limit (256 M) is exceeded. I think I've mitigated the behavior somewhat by moving some things from models to modules, and by serving static content separately, but I think my memory usage is still quite excessive. Is this normal behavior? 30-40 MB per worker is pretty normal. To gain memory you can move to multithreading: master = true processes = 2 threads = 2 thread-stacksize = 512 should be good enough Monitoring memory is a good thing, use --reload-on-rss 80 to avoid your app growing up -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Need help : Errors when running web2py app using uWSGI on Webfaction
Hi, I'm trying to setup web2py to use uWSGI using the instructions from https://groups.google.com/d/msg/web2py/DOKMzOsppFk/91BZtU2cCQkJ The only difference being - I used pip install uwsgi rather than compile it, as mentioned in the above link. uWSGI version that got installed is 1.9.10 web2py version is Version 2.4.6-stable+timestamp.2013.04.06.17.37.38 Here are the problems : First I tried the command as follows : (Same as mentioned in the link above) uwsgi --http 127.0.0.1:PORTNUM --pythonpath /home/username/webapps/myapp/web2py --module wsgihandler -d /home/username/tmp/uwsgi.log -t 20 --async 16 --ugreen --limit-as 64 -r --no-orphans -M -p 1 --touch-reload /home/username/tmp/ uwsgireload.txt --reload-on-rss 50 You get memory errors, just remove limits (64 MB address space are not enough for sure). Even async and ugreen are useless for web2py. Generally if you care memory, you'd better to let your app runs for a couple of days so you have enough metrics to choose a good limit value. -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: SOLVED : Re: [web2py] Need help : Errors when running web2py app using uWSGI on Webfaction
You get memory errors, just remove limits (64 MB address space are not enough for sure). As I mentioned later in my question - removing --limit-as 64 did not help either. If anything - it may have made things worse - I got SegFault (See my original post) You said you reduced to 8 and 16 (i suppose you thought it worked the opposite ;) This is the reason for segfaults Webfaction recently raised memory limits to 512 MB so generally you do not need special enforcements. -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Pythonanywhere down today?
i still can access it both of web2py.com and my own All fine from both Italy and Germany -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: [OT] uWSGI quickstart for Web2Py (works on windows too)
so. Build runs fine now. Something is missing or my cygwin installation (fresh one) is missing some bits https://www.dropbox.com/s/9u8xz0yhfjsed0e/uwsgi.txt you need to rebaseall the cygwin installation: http://cygwin.wikia.com/wiki/Rebaseall (smashed my head over that kind of errors for days, but after rebasing no more problems) -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: [OT] uWSGI quickstart for Web2Py (works on windows too)
I don't think that the binary that came out is runnable outside the cygwin env ... and I really don't know what are all the dll involved. Anyway, it's https://www.dropbox.com/s/gbrojey9ncj1e8i/uwsgi.exe Basically it should work in every system with the cygwin libraries used to compile this .exe. Another approach: You can put all of the required dll (about 10) + the uwsgi.exe + the python environment (site-packages...) + web2py dir in the same directory and you can move to systems without cygwin and python. I suppose you can even make an msi installer. -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] [OT] uWSGI quickstart for Web2Py (works on windows too)
Hi, everyone, sorry for the OT but i would like to point you to the new uWSGI quickstart that has a section for Web2Py (and eventually HTTPS mode): http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html#deploying-web2py In addition to this the 1.9 release (available from march 17) can be compiled on windows (using cygwin), and the example in the quickstart works on windows too (included the https part) Bye -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: [OT] uWSGI quickstart for Web2Py (works on windows too)
I was missing the windows build option totally If speeds between unix and Windows are at least on the same scale it can solve the mother-of-all-issues of windows deployments of python apps without using apache or iis. I was not able to make some serious benchmark as all of the windows machines i have are virtualized systems. From what i can measure the cygwin translation has a 10-15% impact in some area (mainly in startup phases so it is nothing relevant), but we plan to rewrite some critical part using the native windows api (we have already done that for the locking part that instead of pthreads uses windows mutexes). Another limit i have not investigated is windows select() implementation (on the others posix system we use more modern event facilities). But i suppose until you do not use it with hundreds of concurrent file descriptors you should not experiment problems. -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: [OT] uWSGI quickstart for Web2Py (works on windows too)
me neither. My attempts to build 1.9 rc2 under cygwin fail miserably in a uWSGI compiling server core *** [gcc] core/utils.o In file included from core/utils.c:1:0: ./uwsgi.h:297:23: fatal error: sys/event.h: No such file or directory compilation terminated.* error I guess I'll wait for a binary too :) You have to use the version from github, windows support has been added after -rc2 -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] hosting on dreamhost and passenger
Hi, Anyone has experience deploying web2py on dreamhost using passenger ? any instructions on that would be welcome. Thanks in advance. Khalil I strongly suggest you to use uWSGI instead of passenger (less resources needed, more features and obviously better performance): https://github.com/unbit/uwsgi-docs/blob/master/tutorials/dreamhost.rst -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh
Hello, I have setup nginx uwsgi on lubuntu with the script and it works great (best server ever!), except for a part of my application that requires multi threads. I have changed the /etc/uwsgi/web2py.xml to enable threads, tried to set each worker with 2 threads, set the thread memory stack to 512, it doesn't work: I quickly get a 'resource not available (error 11)' message. Anyone has a tip on what I could work on to make this work? Because otherwise the application works fine with rocket / gevent for example. Thanks, Andre If your app is multithreads based you cannot spawn additional processes (they do not share memory). Just set processes to 1 and threads to a decent value -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Passenger WSGI Session
Unfortunately according to web2py on Dreamhosthttp://wiki.dreamhost.com/Web2py, only FastCGI and Passenger WSGI seem to work. And that Passenger works significantly faster than FastCGI. I had heard the stability of Passenger WSGI was also better on Dreamhost. But I can give FastCGI a chance, but is there an Apache file to configure then(to the access.wsgi) for sessions with service calls? I have voted for Dreamhost to setup mod_wsgi, but who knows when that will happen. Thank you. Kitu. I am pretty sure i can make uWSGI to work pretty easily on dreamhost (via fastcgi). Would you mind asking them if they want to contact me privately to make some test ? -- Roberto De Ioris http://unbit.it -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: can't install v 2.3
Back in business. There was detritus from previous installs interfering with new installs. So, the culprit is not uwsgi: unjustly accused. Apologies. No problem, i have just read the thread so i lost momentum :) By the way, the funny thing about uWSGI is that you can go-easy, like uwsgi --http :8080 --chdir web2py --module wsgihandler (remember you can even have https or spdy in latest versions) and you will cover 80% of the users need (without configuring webservers and whatever you want). Or go complex and have config files with dozens of lines and optimizations. The sad thing currently is that the debian package is ages back from the official project, and completely different in the config style from the official blessed ways (they choose to apply an apache-style debianism to uWSGI). So i strongly suggest to continue installing it via pip or sources (at least until there will be official debian packages constantly synced) -- Roberto De Ioris http://unbit.it --
Re: [web2py] uwsgi and post-buffering
Hi all, this is my first post here so allow me to give my compliments to Mr. Di Pierro and all the people that works on this wonderful project. I do not have a big experience in web programming, i came from offline :) programming and triyng to add value to the products of my software house with online capabilities so i'm in learning mode (about 1 month now) with python uwsgi nginx and all the web stuff in general i have implemented a nginx+uwsgi_emperor+web2py multisite solution on a new server that will serve (in future) different sites and services for our company and customers. (it works great for now :) i'm a little worried about a sentence in this page: http://projects.unbit.it/uwsgi/wiki/ThingsToKnow I do not understand this paragraph: If an http request has a body (like a POST request generated by a form) you have to read it in the application. [...] If you do not do it, the communication socket with your webserver could be clobbered. i want to implement a lot of restful calls in many parts of my sites and i'm ok reading the body of a legit request but i don't know how to handle NON legit requests... do i need to read the body in any case?? The important thing is always reading the body of legit requests to avoid nginx closing the socket abruptuly sending back error to the client (even if the response was correctly generated). For non-legit request i think (hope ?) it is not a problem being rejected by nginx. -- Roberto De Ioris http://unbit.it --
Re: [web2py] Re: nginx+uwsgi and ubuntu 12.10
bump 2012/10/21 Marco Túlio Cícero de M. Porto mtcpo...@gmail.com Was there any changes on newest release of Ubuntu that would explain uwsgi stop working ? (using latest web2py and nginx+uwsgi script - 2.2.1 Stable) Just wondering... -- []'s Marco Tulio I strongly suggest you to use official uWSGI distribution (until i manage to start releasing official PPAs). You can use pip to install it if you feel more confortable. Debian uWSGI packages use a really complex (read:overcomplex) system to manage startup, while uWSGI has its emperor mode requiring a single line to startup... ...but debian team refused to use it as it is non-debian friendly... steps (as root/sudo): 1# pip install uwsgi 2# mkdir /etc/uwsgi 3# edit /etc/init/uwsgi-emperor description uWSGI Emperor start on runlevel [2345] stop on runlevel [!2345] exec uwsgi --emperor /etc/uwsgi --logto /tmp/uwsgi.log 4# start uwsgi-emperor ...now just start dropping uWSGI config files in /etc/uwsgi and your instances will start automatically (check /tmp/uwsgi.log in case of problems) -- Roberto De Ioris http://unbit.it --
Re: [web2py] Re: nginx+uwsgi and ubuntu 12.10
Sorry... meant: didn't know about that. Just trusted Ubuntu team. But as Bruce stated before, this is not the first time something goes wrong with uwsgi and ubuntu. Happened before. And that's sad, because if I can't use uwsgi, nginx is not an option for me. Another thing is that Apache2+wsgi works fine, update after update. So, I'm startting to think on keep using Apache2+wsgi, instead of Nginx+uwsgi, because of that... it's really complicated having issues everytime you upgrade something. But again, still have hope that either Apache2 learns something from Nginx, or that Nginx+uwsgi becomes a bit more stable in updates to come (on Ubuntu). Bruce, if you happen to know what happened this time, would you care to share ? :) Seems you know a bit about this particular subject. Thanks for helping! Cheers, Marco Tulio Bruce meant that lot of things goes wrong during updates (and i find that normal), he was not referring to some specific package. And i can add this is not specific to Ubuntu. Every time i did some kind of upgrade something screwed up. You cannot really take apache as an example as it is integrated in debian from ages and you will find users blaming at it after an upgrade even in that time. By the way, i cannot help you with debian-based packages as i really do not get why they use such a complex system. If you want to follow the 'standard' way i will be happy to help you. -- Roberto De Ioris http://unbit.it --
Re: [web2py] allowing admin on UWSGI deployment , HTTP Mode
Il giorno 05/ott/2012, alle ore 16:19, Phyo Arkar phyo.arkarl...@gmail.com ha
scritto:
Hello web2py
i need to allow admin on web2py development machine , which we testing with
uwsgi enabled. It is needed to test over the internet coz our client is from
different country.
We tried enabling by disabling following code inside
if request.env.http_x_forwarded_for or request.is_https:
session.secure()
elif not request.is_local and not DEMO_MODE:
raise HTTP(200, T('Admin is disabled because insecure channel'))
but it ends up to the login page .. WITHOUT Password box , Damn..
What to do?
From 1.3 (latest stable) you can use https in uWSGI too:
http://projects.unbit.it/uwsgi/wiki/HTTPserver#HTTPSsupportfrom1.3-dev
--
Roberto De Ioris
http://unbit.it
JID: robe...@jabber.unbit.it
--
Re: [web2py] User accessed site without logging in
Il giorno 25/lug/2012, alle ore 18:54, Neil ha scritto: I got to the point where I could reproduce this locally using incognito mode. Looks like it is a known uwsgi bug that was just patched 3 days ago: http://stackoverflow.com/questions/11598935/uwsgi-resends-headers-in-async-mode You can get the updated stable-branch from here: http://projects.unbit.it/hg/uwsgi-1.2 (the maintainance release will be tomorrow) but generally you will get not advantages in using plain async mode with web2py (probably that's why the bug popped up so late) -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it --
Re: [web2py] Re: nginx+uWSGI+Web2Py for Ubuntu Precise 12.04
Il giorno 29/mag/2012, alle ore 15:34, orsomannaro ha scritto: Il 28/05/2012 18:00, orsomannaro ha scritto: how must I modify nginx and uwsgi configuration file if I what web2py respond in https://my ip/w2p/ ? I have a sever with nginx+uwsgi+Web2Py and I would like also to configure hgweb(Mercurial), but I'm not able to configure nginx/uwsgi... no one?? Do you want web2py under /w2p and mercurial under something like /hg for the same domain ? -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: nginx+uWSGI+Web2Py for Ubuntu Precise 12.04
Il giorno 29/mag/2012, alle ore 17:37, orsomannaro ha scritto:
Il 29/05/2012 15:43, Roberto De Ioris ha scritto:
Do you want web2py under /w2p and mercurial under something like /hg for the
same domain ?
Yes, perfect!!
Create a WSGI module for mercurial:
# call it hg_wsgi.py
from mercurial.hgweb.hgwebdir_mod import hgwebdir
from mercurial.hgweb.request import wsgiapplication
application = hgwebdir(hgwebdir.config)
Create the hgwebdir.config file:
[web]
style = coal
[paths]
/foo = /bar
/foo2 = /bar2
Each option in the [paths] section is a repository to mount under a specific
path.
For example if you have the repository in /var/repos/foobar you can use
/foo = /var/repos/foobar
Finally configure uWSGI:
[uwsgi]
; your communication socket
socket = XXX
; enable the master
master = true
; set the number of processes
processes = 4
; mount mercurial under /hg
mount = /hg=path_to/hg_wsgi.py
; mount web2py under /w2p
mount = /w2p=path_to_web2py/wsgihandler.py
; make uWSGI smart about SCRIPT_NAME
manage-script-name = true
You do not need to configure nginx for both apps, simply map / to the uwsgi
socket
location / {
include uwsgi_params;
uwsgi_pass XXX;
}
--
Roberto De Ioris
http://unbit.it
JID: robe...@jabber.unbit.it
Re: [web2py] nginx uwsgi error 502 bad gate way
Il giorno 18/mag/2012, alle ore 12:05, Lewis ha scritto: That's the news from Lake Woebegone. On the surface, nginx looks much easier to set up than apache. But, uwsgi seems much more difficult than mod_wsgi. Thanks for everyone's help. From your latest config i have to assume you have finally followed my advice on reading the Quickstart. Finally your config does not look a copypaste :) I would make it even simpler, but for now is enough. By the way, do not you think ps aux has helped you showing the problem ? /usr/local/bin/uwsgi -x /etc/uwsgi/uwsgi-conf.xml -p /var/run/uwsgi.pid why that -p ? from the --help: -p|--processes n spawn n uwsgi worker processes -p|--workers n spawn n uwsgi worker processes So: /usr/local/bin/uwsgi -x /etc/uwsgi/uwsgi-conf.xml should do the trick. After that (i suppose it will start working). Re-read your config, check official docs, and tell me if you really think all of this stuff does not make sense or is over-hard ;) You simply pointed your eyes to the wrong places (and that is a common problem, do not worry) and you have been beaten by some funny distro choice. -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: nginx uwsgi error 502 bad gate way
Il giorno 18/mag/2012, alle ore 12:42, Lewis ha scritto: Ran that command line. Doesn't work. Instead of command line arg for processes, I put it in xml. does it matter? You mean running ps aux now shows /usr/local/bin/uwsgi -x /etc/uwsgi/uwsgi-conf.xml instead of the (wrong) /usr/local/bin/uwsgi -x /etc/uwsgi/uwsgi-conf.xml -p /var/run/uwsgi.pid ? (the second is wrong because -p takes a number, and in addition to that you have already specified the number of processes inthe xml file, so no need to double-use it) By the way try to follow that steps (run a ps aux after each step to be sue all is fine): 1) stop all (nginx, uwsgi) 2) run only nginx 3) from your terminal run (this is the minimal config for web2py) sudo /usr/local/bin/uwsgi --chdir /var/web2py/ --module wsgihandler --socket 127.0.0.1:9001 (it means: move to /var/web2py/, load wsgihandler module and bind to socket 127.0.0.1:9001) Now visit your website with your browser. Does it work ? If it does not work double check if /var/web2py is the correct path of your web2py installation. -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: nginx uwsgi error 502 bad gate way
Bruce: do you only need two lines of uwsgi config? Where do you put the rest of the directives? command line? Does anyone know how to have one uwsgi config file load another (as we do in apache)? I was trying to see an example or find it in uwsgi docs to no avail. http://projects.unbit.it/uwsgi/wiki/ParsingOrder -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: nginx uwsgi error 502 bad gate way
Well, I revised my init.d script to work. Config files do not work at all in any way, shape or form. Just broken utterly. It turns out even loading simple startup variables to the init script does not work. Period. It would appear that for some reason the only way uwsgi works is with command line arguments, many of which aren't documented. For example, everyone uses -M. I did a search for it in the wiki. Many examples include it. The global listing of (not) all arguments does not show it. What is it? I am on a linode vps running ubuntu 11.10. The only person who logs on is me to do config horrors. I log on as root. So, the commmand line to start uwsgi is run as root. As a command line, it works. When it references any external config file it does not work. Should I file a ticket, Roberto? You'll say you can't reproduce. But, I can't reproduce it ever working at all. So, I have wasted 20 hours attempting to debug config files that are inherently unusable. Run uwsgi --help for the full list of options (and their shortcuts). I suggest you to not use shortcuts, as you will forget what they mean soon. Use long format. -M -- --master Regarding config files, read here how they works: http://projects.unbit.it/uwsgi/wiki/Doc (you do not need to read it all, just the first part). Try that (call it foo.ini) [uwsgi] http = :8080 master = true Then uwsgi --ini foo.ini Go to your browser to port 8080, it will work for sure. I suspect you are trying with xml files, but without xml support. If you really love xml format, be sure to have installed libxml2-dev as described here: http://projects.unbit.it/uwsgi/wiki/Install and rebuild uWSGI. Finally, you are on Ubuntu, throw away those ancient init scripts and use what Ubuntu wants: upstart scripts http://projects.unbit.it/uwsgi/wiki/Upstart Obviously the previous link does not show you how upstart works, but you can find more complete docs on ubuntu site (it is a lot easier than init scripts and Ubuntu will not move to SystemD, so you'd better to invest a bit of time in it). Another important link is that one: http://projects.unbit.it/uwsgi/wiki/ThingsToKnow I think you have all the pieces. -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: web2py uWSGI cherokee -- I give up
Thanks. The versions I got were in the apt-get repositories for that version of Ubuntu. I hadn't realized Cherokee had been abandoned. not abandoned, it is on hold (or slowed down in development) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: web2py uWSGI cherokee -- I give up
I hadn't realized Cherokee had been abandoned. I am not in love with anything. I can go back to Apache. It is certainly well-documented and well-supported. Is Nginx the cat's meow? My site is not at all performance bound. I was just looking for an easier option. uwsgi has a built-in webserver which is quite performant, you can find the solution I provided in in the group list I have seen of lot of users/customers moving in that direction. Expecially if you host assets on a dedicated cdn, having a simple http proxy for your app seems to be a reasonable choice. Maybe it is time to add https support... -- Roberto De Ioris http://unbit.it
Re: [web2py] nginx uwsgi error 502 bad gate way
Guys, I have followed the cookbook, the posts here, and a posting at stackoverflow. I am getting error 502 bad gateway. My stuff looks like everyone else's but there are deeper layers of indirection in uwsgi itself and nginx. I have several uwsgi sym links in /etc/init.dIt is not clear which is the real one and how to invoke it. Here is the error from the nginx log. I guess I interpret this to mean that nginx can't find uwsgi: 2012/05/16 22:53:29 [error] 2460#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 76.104.195.84, server: www.lewcl.com, request: GET / HTTP/1.1, upstream: uwsgi://127.0.0.1:9001, host: www.lewcl.com I have no clue what to do. Have you followed my previous advice of starting with uWSGI quickstart ? http://projects.unbit.it/uwsgi/wiki/Quickstart If you are able to follow it, then adding a webserver should be a pretty easy task (and yesterday, uWSGI got https support so you do not even need a full webserver for simple deployments) automagic-script are not the best way to learn. And i really hope you want to learn, because if you blindly cutpaste you will be in serious trouble when something will go wrong (and things always go wrong soon or later). By the way, i am happy you offered help in improving docs, a lot of users tend to only rant forgetting they are using something for free, and helping in something annoying like writing docs is always appreciated. -- Roberto De Ioris http://unbit.it
Re: [web2py] Updated slice: nginx and SCGI | Recipe | Web2py Slices - Recipes, Tutorials, Code, Questions
Changes: - keep-alive support from nginx. Hi Michele, does wsgitools SCGI server really supports keepalive connections ? That would be a kick-ass feature i need to copy for sure :) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: nginx uwsgi error 502 bad gate way
On Thu, 17 May 2012 06:22:20 +0200 Roberto De Ioris robe...@unbit.it wrote: If you are able to follow it, then adding a webserver should be a pretty easy task (and yesterday, uWSGI got https support so you do not even need a full webserver for simple deployments) What is considered 'simple deployment'? In my mind, less than 20 concurrent connections :) For others it is often a matter of money, if i get 10$ for configuring a webserver would you prefer to setup lighttpd/nginx/apache/cherokee/mongrel2 + uwsgi/gunicorn/mod_wsgi or simply do: uwsgi --https :443,cert.pem,cert.key --wsgi-file web2py/wsgihandler.py --master --processes 4 obviously if you start needing features, like client certificate management, ssl caching and so on, the uWSGI https implementation is no more than a toy. -- Roberto De Ioris http://unbit.it
Re: [web2py] web2py uWSGI cherokee -- I give up
(note that it is sudo kill... which only did the job once...) I am really at my wit's end here after hours of research and reconfiguring. These things are so fragile and poorly documented (except for the cookbook, which is entirely clear as far as it goes--but like any cookbook, it doesn't provide much in the way of diagnostics). Such a waste of time when we could be writing nice Python code for web2py sites... I suggest you to hire a sysadmin if you do not want to waste time on another job (yes system administration and programming are completely different jobs, from skills to approaches) ;) By the way, you should take in account a couple of things: - You are using outdated and unmantained packages, at leats for uWSGI you are using a really ancient release with (maybe) hundreds of bugs. - Cherokee is currently starved in development and has been removed from debian packages. Are you sure you want to (currently) use it or invest on it ? Do not get me wrong, i like it and i have worked a lot on it, but would you use a (currently) unmaintained software ? We are talking about a webserver... - kill is for sending signals, not for killing processes (i know, the name is controversial :P). In uWSGI you can use signals to govern it. http://projects.unbit.it/uwsgi/wiki/uWSGISignals has the list of signals. Send them to the master. If you do not like the signal mapping you can change it (just run uwsgi --help on a stable release) Having said that, if you want to be a better sysadmin, start with tiny/slow steps whenever you start learning something. When i need to introduce someone to uWSGI and generally on WSGI deployment, i point him/her to that page: http://projects.unbit.it/uwsgi/wiki/Quickstart After having delployed the first hello world, simply move to the web2py directory and call wsgihandler.py instead of hello.py. After that, you could leave the things in that way (the --http options spawns an http proxy, bind it to the port 80 [remember to drop privileges!!!] and you are ready) or choose a full webserver (you have a lot of choice). But this is a second step... -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: post attack
Thanks, I know how to use the apache and nginx one but how to I set limit-post in uwsgi? --limit-post 4096 to accept a maximum of 4k body On Sun, Apr 22, 2012 at 11:48 AM, Niphlod niph...@gmail.com wrote: apache has LimitRequestBody directive, nginx has client_max_body_size, uwsgi has limit-post. Il giorno domenica 22 aprile 2012 17:09:17 UTC+2, Bruce Wade ha scritto: chunk_size : 10 dest : open file 'fdopen', mode 'w+b' size : 179 src : mod_wsgi.Input object On Sun, Apr 22, 2012 at 8:03 AM, Bruce Wade bruce.w...@gmail.com wrote: Hi, It looks like someone on my site is trying to use a post submission to take down my server. Unfortunately for them we have multiple servers. How can I detect this kind of attack and just ignore it? Some how they are submitting a chunk size of 10 which is killing my I/O on the server. -- -- Regards, Bruce Wade http://ca.linkedin.com/in/**brucelwadehttp://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.**com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- -- Regards, Bruce Wade http://ca.linkedin.com/in/**brucelwadehttp://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.**com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- Roberto De Ioris http://unbit.it
Re: [web2py] CPU 210%
Thanks this is very helpful to know. I am building a system with nginx and uwsgi to get the system running on that. I have been avoiding using uwsgi for anything serious because I have been seeing a lot of threads reporting problems with it. ...and you will continue to see that :) Threads are hard to domate (in-fact most of the time you will go multiprocess). The current (1.2) uWSGI threading implementation is pretty solid (while 0.9.x are completely broken), but being an ultra-complex part i am pretty sure more bugs will pop-up in the future. (and i have to thanks Graham Dumpleton [author of mod_wsgi] for pointing me to better directions) By the way, it is pretty strange you found 'threading' support a uwsgi-blocker as very few (maybe 3-4, uWSGI included) production-grade WSGI servers seriously supports them (with mod_wsgi having the better implementation for sure). In addition to this, a little part of web-related popular libraries are not thread-safe, that is why multithreading is not a too much popular paradigm in python-hosting. -- Roberto De Ioris http://unbit.it
Re: [web2py] memory 210% (was Re: CPU 210%)
Il giorno 18/apr/2012, alle ore 11:11, Gour ha scritto: On Wed, 18 Apr 2012 08:54:29 +0200 Roberto De Ioris robe...@unbit.it wrote: By the way, it is pretty strange you found 'threading' support a uwsgi-blocker as very few (maybe 3-4, uWSGI included) production-grade WSGI servers seriously supports them (with mod_wsgi having the better implementation for sure). What method/server would you recommend for running few smaller low-traffic web2py-powered sites on a host which allows me to use 100MB 50 processes, iow. memory is the bottleneck? I'm looking into uwsgi, but wonder if there is something better memory-wise? I am the one you are already talking about it on irc :) By the way uWSGI is for sure the one consuming less memory (in relation to features you get) in the group of preforking/multithreaded servers. You can even rebuild it removing all of the unneeded features. It is built expecially for shared hosting with limited processes/threads and memory so you will find options to limit its memory or to trigger reload on specific conditions (or exceptions). Obviously it has a very high learning curve, so be prepared for some headache (but it will payback for sure). Another popular solution (in pure python this time) is gunicorn, it is in the spirit do only one thing, do it well, so from a programmer point of view it is a lot more appealing than uWSGI (that is sysadmins-oriented). It has no threads support, so you have to rely on processes and some external tool if you want to monitor/manage your app. -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] CPU 210%
Ok got it running, restarted uwsgi not sure what the issue was. However following the guides I my static content isn't being loaded: http://50.116.6.124 The error you got earlier, happens when you do not read the body of the request (for example when you have POST data, but you do not read them). This is how nginx works. You can avoid that (if you do not want to read post data) with --post-buffering n in uWSGI. It is a trick to force uWSGI to blindly read the http body independently by your app. Regarding static files, you have to fight with nginx (uWSGI should only manage the python part) Normally working on the location directives and root/alias options is enough On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade bruce.w...@gmail.com wrote: uwsgi is sure not as easy to get working correctly as apache is. Keep getting the following: 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely closed connection while reading response header from upstream, client: 173.180.212.86, server: youadstage, request: GET / HTTP/1.1, upstream: uwsgi://127.0.0.1:9001, host: 50.116.6.124 On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha rochacbr...@gmail.comwrote: There are directions for this on book http://web2py.com/books/default/chapter/29/4#session Store session on db or set separate=True On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha rochacbr...@gmail.comwrote: I recommend you to store sessions on db or cache like memcached, redis... If you are going to run in a load balance, sessions in disk will be a problem. On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade bruce.w...@gmail.comwrote: Currently FS On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha rochacbr...@gmail.comwrote: Are you storing sessions in db or FS? http://zerp.ly/rochacbruno Em 18/04/2012 02:18, Bruce Wade bruce.w...@gmail.com escreveu: Yeah, I am not sure how well web2py can handle load but I am going to find out pretty fast. I finished setting up a load balancer today, going to have 3 servers under it running nginx + uwsgi + web2py a single dedicated DB server for now, but figure I am going to need some slave read only servers. The speed at which web2py allows you to develop applications will surpass any load issues, well unless it could only handle 50,000 users/per second then that would be a big issue :D My one concern is how good the sticky sessions on linode's nodebalancer works. Regarding uwsgi I have ready time out issues when using it with web2py. -- Regards, Bruce On Tue, Apr 17, 2012 at 9:55 PM, pbreit pbreitenb...@gmail.comwrote: I've never read anything but good things about uwsgi. Django and Flask both promote it. -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- Bruno Rocha [http://rochacbruno.com.br] -- Bruno Rocha [http://rochacbruno.com.br] -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- Roberto De Ioris http://unbit.it
Re: [web2py] CPU 210%
Any ideas what would cause the system to stop working without errors when using database sessions? I suggest you to start with a uWSGI config resembling mod_wsgi: --lazy --enable-threads Should be enough, and start with a single process (and a single thread). If it works (and it should), remove --lazy (in that way you will abuse fork() cow). And then remove --enable-threads (even if web2py uses threads in a couple of areas, so i would not remove it) -- Roberto De Ioris http://unbit.it
[web2py] [ANNOUNCE] uWSGI 1.1
I am pleased to announce the 1.1 release of the uWSGI project. This release focuses on a new option parser subsystem, improved perl/psgi and ruby/rack support and a new (stable) php plugin. A lot of optimizations have been introduced for the fastrouter and the various threading modes. This is the first release officially deprecating some old-style configuration (see notes). Changelog [20120317] - new options parser subsystem - improved php support http://projects.unbit.it/uwsgi/wiki/PHP - multiple spooler with multiprocessing simply add multiple --spooler options (on different directories) to have multiple spooler. Add --spooler-processes to fork() them - segmentation fault and floating point exceptions manager you will get a handy backtrace whenever SIGSEGV and SIGFPE are triggered - never-swap option enabling this option will force uWSGI to lock all memory areas on physical memory to avoid on-disk swapping. - zergpool plugin another step towards down-free reloads http://projects.unbit.it/uwsgi/wiki/ZergMode - perl/psgi multiple interpreters support you can now mount multiple perl-apps in the same process - deadlock detector on some specific conditions, uwsgi locking subsystem can be damaged, the master process got a new feature, allowing it to detect such conditions - pluggable cheaper algos by default the old-fashioned apache-style spare algorithm is used for adaptive process spawning. A new one, based on socket listen queue can be used, or you can implement a completely new approach: http://projects.unbit.it/uwsgi/browser/plugins/cheaper_backlog2/cheaper_backlog2.c - new internal routing susbsystem http://projects.unbit.it/uwsgi/wiki/InternalRouting - preliminary support for config logic http://projects.unbit.it/uwsgi/wiki/ConfigLogic - log rotation infrastructure you can now trigger log-rotation simply touching files - support for paste loggers - threading improvements - plugins can now mapped to specific modifiers - app tag is deprecated (use mount option) - report startup time for apps - reintroduced expat support by default libxml2 is searched, if it is not available libexpat will be used. If both are not available, xml support will not be compiled in. - reintroduced remote spooler plugin - various fastrouter optimizations - added ruby rvm support add --gemset name option to set a specific rvm gemset - a lot of fixes all over the place Notes: - app tag has been deprecated, some simple old-style-config still works, but you should move to the new mount option. In uWSGI 1.2 the app tag will map 1:1 to the mount one. - xml support is now automatically detected (and eventually not compiled in) Special thanks for this release go to Anthon van der Neut (mongrel2 fixes and setuptools packaging) Łukasz Mierzwa (bug-hunter in-chief) Raffaele Colace (for the upython contrib script) Cal Leeming Ultrabug (ultrabug.fr, sorry i do not know your real name) Riccardo Magliocchetti C Anthony Risinger Evgeny Turnaev Marcin Deranek Mike Kuznetsov (sorry if i have missed someone) You can download uWSGI 1.1 from http://projects.unbit.it/downloads/uwsgi-1.1.tar.gz -- Roberto De Ioris http://unbit.it
[web2py] Re: [ANNOUNCE] uWSGI 1.1
hi, great news, thanks is it possible now to use uwsgi with pypy? how stable is that? any steps to do that if it is possible? thank you It requires a couple of modifications in pypy (when compiled as a shared library), i will send them a real patch soon or later. Currently it is only a (slow) work in progress, as (at least for me) having full knowledge of pypy implementation is not easy as the cpython one -- Roberto De Ioris http://unbit.it
Re: [web2py] share data across uwsgi processes
Hi, My production environment is: latest web2py trunk, ubuntu 10.04, postgresql 8.4, nginx, uwsgi. I need to have data shared (not cached for certain time) across the uwsgi processes. Currently I'm accessing such data via db select, but I'm wondering if there's a faster ram method (which auto-clears when restarting uwsgi server), with no need to access the database. Is cache.ram or memcache or redis what I need?, preferably something simple to setup. Note that I do not need to cache anything for certain time, but instead share data (set/get common data) across my uwsgi processes. dict-based: http://projects.unbit.it/uwsgi/wiki/CachingFramework queue-based: http://projects.unbit.it/uwsgi/wiki/QueueFramework raw-memory: http://projects.unbit.it/uwsgi/wiki/SharedArea -- Roberto De Ioris http://unbit.it
Re: [web2py] Web2py + uWSGI and timeouts!!
Hello I am testing web2py with uWSGI and some request have random timeout (after IO Heavy operation is started , but it is spawned using subprocess.Popen()) And then progress is selected from DB Every 5 Seconds (Progress Percent Insert,Commit to db is only done at each 5% intervals) At the IO Heavy times , the progress status is Timedout! even tho I set timeouts to really huge 6000 secs , it still timeout. If you generate a process during a request it will inherit the opened file descriptors. Add --close-on-exec to uWSGI to let it automatically close opened sockets before execing a new process. Also it seems that all uWSGI worker processes are running at the same core as extraction process is running . how can i make it run in different cores (to make use of multple cores?) I think you are referring to 'uwsgi cores' in the logs. They are referred to threads and async modes. They do not reflect the cpu core in which it is running. -- Roberto De Ioris http://unbit.it
Re: [web2py] Web2py + uWSGI and timeouts!!
When i check i got this error : 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Traceback (most recent call last): File /home/v3ss/WorkSpace/FastTract/web2py/gluon/main.py, line 447, in wsgibase parse_get_post_vars(request, environ) File /home/v3ss/WorkSpace/FastTract/web2py/gluon/main.py, line 275, in parse_get_post_vars request.body = copystream_progress(request) ### stores request body File /home/v3ss/WorkSpace/FastTract/web2py/gluon/main.py, line 143, in copystream_progress copystream(source, dest, size, chunk_size) File /home/v3ss/WorkSpace/FastTract/web2py/gluon/fileutils.py, line 376, in copystream data = src.read(chunk_size) IOError: error waiting for wsgi.input data If you are on chrome + https on cherokee, it will not works. it is a cherokee bug, you can try with latest svn snapshot (iirc it has been fixed there) -- Roberto De Ioris http://unbit.it
Re: [web2py] uWSGI Question
I would like to know about uWSGI . i haven't tried it yet. 1 - Is that a gateway to interface between other web servers ? (Apache/Cherokee/nginx etc) it depends, if you need a specific webserver front-end you can use it and connect to uWSGI via one of the supported protocols (oviously the uwsgi protocol is the suggested one). Normally, putting an application directly connected to the public network is not a good idea. So you will very probably ends always putting uWSGI behind some sort of filter (read: webserver/webrouter/loadbalancer) 2 - It seems that can be use as a webserver alone but how it works. Is that reliable /Fast? again, it depends on your setup. If you put static assets on another host, you only need a filter before your app, and you can use the included http-router. It can manages thousand of HTTP requests (in non blocking way) and forward them to uWSGI nodes. This area is fastly evolving (like persistent connections support, sctp and automatic clustering management) and could be interesting from a research poin of view. Serving static files via uWSGI could be really fast if you have few concurrent connections, but it cannot reach nginx/cherokee performance on request storming (by default uWSGI concurrency is limited by the number of spawned threads/processes) 3 - It was mentioned that The download/upload corruption do not happen in uWSGI , how you use it? Stanalone or as an gateway/interface for other webserver/reverse proxy? If you want to follow the solid path, use some rock-solid webserver (like nginx, cherokee or apache) forwarding requests to one (or more) uWSGI nodes. If you plan to have dozens of nodes with ultra scalability with no effort, try to follow the http-router development (even if you could need something like nginx/haproxy for managing https connections). -- Roberto De Ioris http://unbit.it
Re: [web2py] uWSGI Question
Thank you very much! i am going to give uWSGI standalone mode a try! Forgot an important point: --http :port will spawn the http router/filter/proxy --http-socket :port will let uWSGI speaking the http protocol (instead of the uwsgi one). You need the first form if you plan to put uWSGI alone in the network. -- Roberto De Ioris http://unbit.it
Re: [web2py] nginx/uwsgi setup on Debian
I am trying to replace an apache setup with nginx/uwsgi on my home network as a pilot project for a more formal setup on a server where I am deploying an app. With the help of people on this list I could change my Webfaction configuration to nginx/uwsgi but on a plain Debian computer I am making no progress. I am runing Debian unstable on my home computer and Debian stable on the server. On both I could get the ubuntu-setup script that comes with web2py to set up apache/web2py successfully. However, the script setup-web2py-nginx-ubuntu, modified to use Debian packages and paths and not ubuntu ones, only produces an nginx running and answering with *502 Bad Gateway*. I then purged all the uwsgi and nginx packages from Debian and tried the script at http://stackoverflow.com/questions/5335153/how-to-deply-web2py-using-nginx. That also ended with *502 Bad Gateway*.:( I have read somewhere that nginx is more simple to set up than Apache, but in this case i have spent several hours reading and trying several setup scripts and configuration and I got nowhere. Are there anybody running a nginx/uwsgi/web2py setup on Debian? If so, are you willing to share some knowledge that can help me? Regards Johann Having both nginx and uwsgi on webfaction is now useless. This is a quick sequence you can run on debian too: wget http://projects.unbit.it/downloads/uwsgi-1.0.4.tar.gz tar zxvf uwsgi-1.0.4.tar.gz cd uwsgi-1.0.4 make if all goes well run uWSGI ./uwsgi --http-socket :8080 --chdir path_to_webp2y --module wsgihandler point your browser to port 8080 on localhost and you should see your app For webfaction substitute :8080 with the assigned port -- Roberto De Ioris http://unbit.it
Re: [web2py] nginx/uwsgi setup on Debian
On 24 February 2012 15:46, Roberto De Ioris robe...@unbit.it wrote: This is a quick sequence you can run on debian too: wget http://projects.unbit.it/downloads/uwsgi-1.0.4.tar.gz tar zxvf uwsgi-1.0.4.tar.gz cd uwsgi-1.0.4 make if all goes well run uWSGI ./uwsgi --http-socket :8080 --chdir path_to_webp2y --module wsgihandler point your browser to port 8080 on localhost and you should see your app For webfaction substitute :8080 with the assigned port Thanks for your reply. Webfaction is not my problem as you saw in my original message. Your advice worked. But as soon as I tried a tcp-socket like '-s 192.168.0.1:8080' I get an error message: The connection was reset and on the console: $ uwsgi -x web2py.xml [uWSGI] parsing config file web2py.xml *** Starting uWSGI 1.0.4 (64bit) on [Fri Feb 24 16:24:10 2012] *** compiled with version: 4.6.2 on 24 February 2012 16:11:08 current working directory: /etc/uwsgi/apps-enabled detected binary path: /usr/local/bin/uwsgi *** WARNING: you are running uWSGI without its master process manager *** your memory page size is 4096 bytes uwsgi socket 0 bound to TCP address 192.168.0.1:8080 fd 5 Python version: 2.7.2+ (default, Nov 30 2011, 22:33:30) [GCC 4.6.2] Python main interpreter initialized at 0x1cf11b0 your server socket listen backlog is limited to 100 connections *** Operational MODE: single process *** added /home/www-data/web2py/ to pythonpath. WSGI application 0 (mountpoint='/') ready on interpreter 0x1cf11b0 pid: 16706 (default app) *** uWSGI is running in multiple interpreter mode *** spawned uWSGI worker 1 (and the only) (pid: 16706, cores: 1) invalid request block size: 21573 (max 4096)...skip Fri Feb 24 16:24:26 2012 - error parsing request I suspect that nginx would be play the role of the 'master process manager'. -s means --socket that set the usage of the uwsgi protocol instead of the HTTP one In this case you need to point nginx to this socket (as your browser does not speak uwsgi obviously) via the uwsgi_pass directive -- Roberto De Ioris http://unbit.it
Re: [web2py] uwsgi configuration: why many lines in /root/var/log/uwsgi-python/web2py log : routing 0 routes 0 ?
strange, really :D I don't mind if that turns out in the logging, but I have the same behaviour niphlod@platypus:~$ uwsgi-python --version uWSGI 0.9.6.8 I use uwsgi-python in production /usr/bin/uwsgi-python . If I recall correctly that is a symbolic link to `/etc/alternatives/uwsgi-python', and that one is afile symbolic link to `/usr/bin/uwsgi-python2.6' I then have a /usr/bin/uwsgi that is the result of pip-installing uwsgi package: actually I'm experimenting on dropping the uwsgi-python package (some real mess to understand clearly, at least for me, especially with its /etc/init.d scripts) . With the emperor mode it's really easier to manage apps Maybe some shared libraries between versions ? The original uwsgi debian packages started when the emperor was still a prototype. The author (Leonid Borisenko) did his best to simulate what the Emperor is today simply using init script and various form of configuration inheritance. For me, Emperor is the simplest (and versatile) way to go, as it relies on config files only: you create one to start a new instance, you remove it to remove an instance, you touch it to reload an instance. No script, nor funny permission issues (you can choose the directory in which config files are looked up). In addition to this, 0.9.6 is near to end-of-maintainance (june 2012), so upgrading to 1.x would be in general a better choice. -- Roberto De Ioris http://unbit.it
Re: [web2py] uwsgi configuration: why many lines in /root/var/log/uwsgi-python/web2py log : routing 0 routes 0 ?
ok,
this line is dne by uwsgi, and I don't know how to turn it off.
Thu Feb 9 17:58:12 2012 - routing 0 routes 0
if you have memory report on, there will be also a line like
{address space usage: 46809088 bytes/44MB} {rss usage: 34369536
bytes/32MB}
[pid: 17838|app: 0|req: 23/90] 188.10.176.36 () {44 vars in 901 bytes}
[Thu
Feb 9 17:58:12 2012] GET /app/default/index = generated 12313 bytes in
420 msecs (HTTP/1.1 200) 6 headers in 308 bytes (0 async switches on async
core 0)
let's wait for someone better than me or roberto himself to know if also
the routing 0 routes 0 can be toggled by some directive :D
Those lines are printed when UWSGI_ROUTING c constant is defined and the
only way to do that is passing -DUWSGI_ROUTING to the makefile. The funny
thing is that uWSGI routing has been only really added in 1.1 (still
unreleased) so i really do not know why you have that flag turned on :)
--
Roberto De Ioris
http://unbit.it
Re: [web2py] Re: Nginx + uwsgi file download problem
Hard to say.
I don't know if this helps but here's how appadmin does it for a csv file:
def csv():
import gluon.contenttype
response.headers['Content-Type'] = \
gluon.contenttype.contenttype('.csv')
db = get_database(request)
query = get_query(request)
if not query:
return None
response.headers['Content-disposition'] = 'attachment;
filename=%s_%s.csv'\
% tuple(request.vars.query.split('.')[:2])
return str(db(query).select())
Try to set Content-Length header to avoid nginx losing data
--
Roberto De Ioris
http://unbit.it
Re: [web2py] Re: High memory usage in webfaction
can I rotate log (uwsgi.log) with this command? I don't want the log file to go on increasing in size log-maxsize = n where n is the maximum size in bytes -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: web2py with Plesk Control Panel ?
Il giorno 23/gen/2012, alle ore 10:09, peter ha scritto: The problem I found was that Plesk requires python 2.4 and Web2py requires at least python 2.5. So one has to install python 2.5 or higher (I use 2.6), and use this in parallel with the python 2.4. I could get web2py working with the rocket server. I never did get it working with Apache so I will be interested in your results. I currently use uwsgi on its own. This means the static directory is served through web2py but is otherwise fast. A better route would be nginx and uwsgi, which will work but on my system there are still issues to resolve. Recent uWSGI releases (1.0.1) contains highly optimized file serving via the options: --check-static dir --map-static [mountpoint=]dir file will be served without passing control to web2py Good luck, keep us informed of your progress Thanks Peter On Jan 22, 5:06 am, Bruno Rocha rochacbr...@gmail.com wrote: -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: Rocket vs mod_wsgi
Compiled uwsgi 0.9.9.3 (the 0.9.8.1 did not now about pythonpath) uwsgi --pythonpath /opt/web-apps/web2py --module wsgihandler --http :80 -s /tmp/we2py.sock uwsgi.log 21 1 CPU: 17.83 [#/sec] (better than rocket) 2 CPUs: 17.98 [#/sec] uwsgi --pythonpath /opt/web-apps/web2py --module wsgihandler --http :80 -s /tmp/we2py.sock -M -p2 uwsgi.log 21 2 CPUs: 31.30 [#/sec] I guess with the -p 2 enabled it was not as fast as nginx for static content. because you are comparing a preforking (apache-style) approach with a non-blocking one (nginx). For determistic areas (like serving static files) there is no competition. This is why having a non blocking-server on front of the application server is a common setup. Having a front-end will obviously slow-down things, but the impact (as you have already noted) is non-existant. (by the way, --http is still a proxied-setup as the uwsgi http server will run in another process, if you want native http you have to use --http-socket) In addition to this, you have to think about security: nginx, apache, cherokee... are all developed for being front-line servers, and this requires a higher level of security in mind. Anyhow, is this a recommended setup? Doesn't it show the same behavior as gunicorn ( Without this (nginx) buffering Gunicorn will be easily susceptible to denial-of-service attacks. ) as i have already said having a front line server (like --http does) is always a good choice, independently by the front-line server. Nginx has obviously more tolerance in high-concurrency in serving static file (have you tried the --static-map and --check-static option in uWSGI 1.0.x ?), but if you are developing the next amazon, you should really start thinking about using a CDN for your static assets instead of your own webserver. regarding gunicorn, it does only one thing and does it well (TM), its philosophy is towards simplicity, writing logic to check the good-behaviour of your app (like uWSGI morbidly does) is at the opposite of that philosophy. I am perfectly agree with this approach (programmers should fix their apps), but i work in a ISP where customers expect their apps being available as most as possibile and tend to blame the ISP for their fault (yes, it is a ugly world :). That's why uWSGI exists and that's why you cannot compare it with gunicorn (different philosophy and different target) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Webfaction: Memory usage way over your limit ; again.
and also uwsgi --version doesn't give the version but tells that there is an error This could only be generated by a wrong compiled-binary (maybe a make without a make clean after some config changes). --version triggers only a fprintf() on a fixed binary blob, so if it fails something really bad is in place :) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Webfaction: Memory usage way over your limit ; again.
I was migrated to a newer webfaction server and now uwsgi compiles. I first tried limiting to 64 MB of memory and 1 process: uwsgi --http=127.0.0.1:port --pythonpath /user/web2py --module wsgihandler -d /user/uwsgi.log -t 20 --async 24 --ugreen --limit-as 64 -r --no-orphans -M -p 1 --touch-reload /user/uwsgireload.txt --reload-on-rss 50 But this lead to MemoryError's for my more database intensive pages. Then increased to 80 MB and the MemoryError's have stopped. Should this much memory really be needed, or is there likely a problem with web2py / my app? Richard You could try moving to a 1.0 release (rc7 is just out there) and change your command line to: uwsgi --http-socket 127.0.0.1:port --pythonpath /user/web2py --module wsgihandler -d /user/uwsgi.log -t 20 -r --no-orphans -M -p 1 --touch-reload /user/uwsgireload.txt --evil-reload-on-rss 64 you will gain one process and async monitoring of the --evil-reload-on-rss option. You should make a run of the server with the -m option to get memory usage after each request in your logs, to eventually catch some leak of your app. -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Webfaction: Memory usage way over your limit ; again.
Il giorno 22/nov/2011, alle ore 14:19, Plumo ha scritto: very useful information - thanks Robert. I got compile errors for nginx - tried multiple versions and only 0.9.6.5 built, which doesn't support --http-socket. I am on web9 which I understand is one of the older servers: http://community.webfaction.com/questions/3042/uwsgi-installation Which server did you manage to compile nginx on? In the source distribution there is the directory buildconf/ that contains the various build profiles. Open default.ini and change locking = auto to locking = flock and run make this should work even in so ancient systems :) -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Webfaction: Memory usage way over your limit ; again.
Again at Webfaction my memory usage is way over my limit: Below is the list of processes that you're running with the memory that they use (the command used to list these processes is ps -u fitwise -o rss,etime,pid,command): User - Memory - Elapsed Time - Pid - Command: fitwise - 1MB - 24 days, 11:45:32 - 30464 - /home/fitwise/webapps/w2p/ apache2/bin/httpd.worker -f /home/fitwise/webapps/w2p/apache2/conf/ httpd.conf -k start fitwise - 27MB - 24 days, 11:45:32 - 30465 - /home/fitwise/webapps/w2p/ apache2/bin/httpd.worker -f /home/fitwise/webapps/w2p/apache2/conf/ httpd.conf -k start fitwise - 25MB - 24 days, 11:45:32 - 30466 - /home/fitwise/webapps/w2p/ apache2/bin/httpd.worker -f /home/fitwise/webapps/w2p/apache2/conf/ httpd.conf -k start fitwise - 25MB - 24 days, 11:45:32 - 30467 - /home/fitwise/webapps/w2p/ apache2/bin/httpd.worker -f /home/fitwise/webapps/w2p/apache2/conf/ httpd.conf -k start fitwise - 26MB - 24 days, 11:45:32 - 30468 - /home/fitwise/webapps/w2p/ apache2/bin/httpd.worker -f /home/fitwise/webapps/w2p/apache2/conf/ httpd.conf -k start fitwise - 25MB - 24 days, 11:45:32 - 30469 - /home/fitwise/webapps/w2p/ apache2/bin/httpd.worker -f /home/fitwise/webapps/w2p/apache2/conf/ httpd.conf -k start fitwise - 2MB - 24 days, 11:45:32 - 30470 - /home/fitwise/webapps/w2p/ apache2/bin/httpd.worker -f /home/fitwise/webapps/w2p/apache2/conf/ httpd.conf -k start I did this: Serve static media with a Static-only application. I did not yet set up Nginx+uwsgi because at Webfaction they said: Using Nginx+uwsgi may reduce your memory usage, but it will involve some experimentation with your application. and I don't have much time to experiment at the moment. There is nothing to experiment to with webfaction + uWSGI. After having installed uWSGI 1.0 (from mercurial and with python2.6) simply run it with uwsgi --wsgi-file web2py/wsgihandler.py -m --http-socket :PORT where PORT is the port assigned by webfaction. The -m flag report the memory usage after each request, and you can obviously add flags to automatically respawn instances based on memory usage. PAY ATTENTION: only the 1.0 branch of uWSGI is fully usable in webfaction without nginx or the --http option, and has support for their HTTPS proxy out of the box (thanks to the report of a web2py user in this list). Webfaction guys should really invest some time in uWSGI analysys, as it is built with their kind of services in mind. QUICK AND DIRT WAY (from webfaction ssh home) # install mercurial easy_install mercurial hg clone http://projects.unbit.it/hg/uwsgi cd uwsgi # compile uwsgi make -f Makefile.Py26 cd .. # download web2py wget http://www.web2py.com/examples/static/web2py_src.zip unzip web2py_src.zip cd web2py # run web2py rocket server to setup password python2.6 web2py.py # set your password then kill the server cp parameters_8080.py parameters_PORT.py # where PORT is your webfaction port cd .. # start uwsgi ./uwsgi/uwsgi --wsgi-file web2py/wsgihandler.py -m --http-socket :PORT -- Roberto De Ioris http://unbit.it
Re: [web2py] Debugging with uWSGI
I want to get dubugging setup, and the main options I see are using winpdb, pydev, or wingide. But all the examples I see are to debug the web2py.py app when it is run and the debugger grabs the process. But I am using the cherokee webserver with uWSGI, so how would U use it in this way? I want it to grab an existing process and monitor the port (wingide calls it passive listening, I just don't want to have to buy wing if I have other good options). I am leaning either towards using winpdb with my current IDE, or using the pydev plugin in AptanaStudio. So can anyone recommend how to setup debugging when im not running web2py.py to start the process? Thanks. winpdb is maybe the only one that can be embedded in every scenario without headache. (even in extremely thread-based modes) Simply choose when (and where) to run the debug-server then follow this guide: http://winpdb.org/docs/embedded-debugging/ -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: IMPORTANT
Il giorno 13/ott/2011, alle ore 07:20, guruyaya ha scritto: I could be wrong, but I don't think I remember any discussions that go out of line, in this forum. I don't remember any politics discussions, any religious or scientific, that went out of line. Is this, really a such a big problem that requires it's own policy? I do not know, and (as you) i do not remember such kind of discussion. But please, read this thread (i get my own responsibility for whoever get offended by this link): https://bugzilla.redhat.com/show_bug.cgi?id=681339 I suppose most of you will start rotfl'ing reading this, but the guy opening the thread is very serious. Does he deserve respect for that kind of request ? I do not know, but generally, i tend to ignore people taking theirself (or their culture/religion/belief) too much seriously at such rate to disturb other people. There are limits in every joke, i suppose we only have to be smart and conscious about this. Now i came back eating spaghetti and playing my mandolino. -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: webfaction clean install unable to save any files in admin edit mode
Ok Roberto, I managed to build and test. At first view It seems to work. I can modify files and save them. So, there is great progress, thanks a lot. Is there anything else I could do to help you or the community in anyway? Don't hesitate to ask or suggest. Michel Thanks a lot, i would like to know it this new feature works out of the box on webfaction: http://projects.unbit.it/uwsgi/wiki/StatsServer (using uwsgitop) We are ready to backport it to the stable branch, so knowing if it works in very resource-limited environment could be very useful. -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: webfaction clean install unable to save any files in admin edit mode
Bad news... I left my site well alone after the success yesterday. This morning, unfortunately, it was as if nothing had be done. I killed uwsgi and re-started after overwriting the copy... All to no avail... I get the double log in (wth two web2py logos in top left corner), and request to re-log in, nearly every page, and a communication error on any attempt to save a file in the admin panel. That's all folks! I am getting suspicious that things are achanging from under my feet at webfaction. Is that paranoia? Maybe there are different uWSGI versions in your account, check the startup logs to see which one is running -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: webfaction clean install unable to save any files in admin edit mode
But where do I download the code for that version? In the directory where you cloned the sources yesterday, run hg pull hg update make -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: webfaction clean install unable to save any files in admin edit mode
Il giorno 03/ott/2011, alle ore 07:51, MidGe ha scritto: Thank you for your interest and quick reply. I was meaning disabling it in: applications/welcome/controllers/appadmin.py Ah! Yes, I could do that, But I also am able to log in and edit when using apache as a server. i suspect the webfaction proxy is not passing the correct headers (or is using some other technic) to let uWSGI knows the scheme is https. Yes, something like that is the cause. This can be easy fixed with a WSGI middleware, but i would like to be sure about it. Fine I can duplicate an re-duplicate here. I can insert any code to debug/test and am quite prepared to help in anyway. I just need some guidance as I am in an area where I have little knowledge (uwsgi). Thanks again Michel I have just committed support for X-FORWARDED-SSL header (it looks like it is what webfaction is using). It works only in --http-socket mode (not --http one). If you can install the mercurial version on webfaction and test it, i will backport the patch to the stable tree. Thanks -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: webfaction clean install unable to save any files in admin edit mode
OK, will give it a shot and let you know. What is the mercurial command for the version you committed? Michel hg clone http://projects.unbit.it/hg/uwsgi cd uwsgi make -- Roberto De Ioris http://unbit.it
Re: [web2py] webfaction clean install unable to save any files in admin edit mode
Il giorno 02/ott/2011, alle ore 10:17, MidGe ha scritto: I suspect the problem must be with uwsgi. The reason I say that, is that if I use the apache as per install script, it works. If I stop apache and setup uwsgi, everything exactly the same, I have the problem. I can access the site. login even, but I get in a loop showing me back to the login screen when trying to access admin area. Killing uwsgi and restarting apache.. everything hunky dory! I suspect that using uwsgi, and specially the built-in server option sort of messes up the users. Any ideas? Have you tried disabling https-requirement for the web2py admin interface ? If it works the problem is clear and bypassing it to make webfaction happy would be easy -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: webfaction clean install unable to save any files in admin edit mode
Hi there, If I disable the https requirement for the admin, I will not be able to get into the admin area either. I was meaning disabling it in: applications/welcome/controllers/appadmin.py The strange thing is that I can log in. The system recognizes my password as valid (when it is, and not when it is not) but it does get me back to the login screen, in an endless loop. i suspect the webfaction proxy is not passing the correct headers (or is using some other technic) to let uWSGI knows the scheme is https. This can be easy fixed with a WSGI middleware, but i would like to be sure about it. -- Roberto De Ioris http://unbit.it
Re: [web2py] uwsgi question
Il giorno 29/set/2011, alle ore 15:50, elffikk ha scritto: hi, sqlite 3.7+ supports better concurrency (see http://www.sqlite.org/draft/wal.html), on webfaction, there are multiple python versions installed, but python2.7 is built against 3.3.6 so I built from sources sqlite, python 2.7 and now on /home/my_user/ bin/python2.7 I can use latest sqlite import sqlite3 sqlite3.sqlite_version shows 3.7+ if I run /home/my_user/bin/python2.7 web2py.py sqlite3.sqlite_version is 3.7 but if I run uwsgi the way I described in other posts it seems to see the standard one (3.3.6) also I put in uwsgihandler.py first line to be #!/home/my_user/bin/ python2.7 and my question is: how I can enforce to use sqlite 3.7 (is it because it uses the default installed python 2.7) uWSGI is not a python app, so you have to tell it where your alternative libpython.so is located LD_LIBRARY_PATH=/home/my_user/lib uwsgi ….. should be enough but if you want to avoid messing with library search path you can rebuild uwsgi for the new python: (in the uwsgi sources dir) /home/my_user/bin/python2.7 uwsgiconfig.py --build another solution is installing setuptools/pip in the new environment and install uWSGI from it: /home/my_user/bin/pip install uwsgi -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] uwsgi question
Il giorno 29/set/2011, alle ore 16:36, Vasile Ermicioi ha scritto: /home/my_user/bin/python2.7 uwsgiconfig.py --build but I did so will that work export LD_LIBRARY_PATH=/home/my_user/lib/python2.7 then running the uwsgi command? This is the directory where python core modules are, not the directory containing libpython.so Are you sure you have added --enable-shared in your ./configure script ? -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] criticism of web2py
Hello! I'm starting with python web2py and yesterday visited Hello, I'm starting with Python Web2Py and yesterday visited one python-related project channel where I mentioned web2py and as the consequence I was quickly advised to abandon it due to 'non-Pythonic' etc. This is not the first time I hear people criticise web2py due to, amongst other things, breaking Explicit is better than implicit. principle alothough Massimo nicely explains it in the introduction of web2py book. I played for sometime with the Haskell, but eventuall abandon it due to not being pragmatic enough for daily use (e.g. all GUI bindings are maintained by few people only). The same situation was/is with D. Now, I see Python as very pragmatic language, with lot of 'batteries included' and very suitable for all kinds of tasks, so I wonder waht is the real reason of web2py criticism: a) stubborness of (python) people to see that Explicit is better than implicit. is not engraved in stone and can be violated to keep DRY, b) envy of web2py's success c) something else? As far as I'm concerned, I do not care much 'cause I'm accustomed to do things which are not mainstream, but just curios what do you think? If you want an adivse, do not trust those kind of people. Whoever claim itself a technology-guy and has fear of diversity, or 'non-ortodox' approaches should probably start looking in other areas... If someone blaim a project because it does not work, it has horrible documentation or the author likes killing kitten, this is good. But blaming a project for not being loyal to some concept is silly. It looks like more a religious approach than a techy one. Web2py works (and most of the time it simply works thanks to its design), it is well documented and AFAIK Massimo does not like killing kittens. But yes, if your priest.py does not approve it, do not use it :P -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: uWSGI 0.9.9
Il giorno 14/set/2011, alle ore 17:08, elffikk ha scritto: seems fixed in 0.9.9.2 thank you and some questions: is it possible to run uwsgi with jython ? i really do not know, no plan currently to push the jvm/jni plugin when do you think it will be possible to run uwsgi with pypy ? it is already possible, but lack half of the cpython features (most pf the pypy c-adapter is incomplete) UWSGI_PROFILE=pypy pypy uwsgiconfig.py --build this will build a libuwsgi.so library that you can use with pypyuwsgi.py (included in the source tree) pypyuwsgi.py works in the same way of uwsgi: pypyuwsgi.py --socket :3031 --module welcome thank you again for your awesome project -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] [SLICE] nginx + SCGI
it is a bit harder to setup are you joking ? search this group and u will see my comments using uwsgi on webfaction nginx -socket- uwsgi+python u can use builtin webserver and u will not have layers at all and SCGIServer is written in python which makes it slower it scales smoothly on multicores uwsgi has master and slave processes, also has uGreen module (~ greenlet) - u see: master-child processes-ugreenlets (if I can say so) - that makes it able to handle much more requests than scgiserver can and not very least feature: it is STABLE also it is able to reload child processes, can limit the memory for child processes and many many features, and still FASTER and a very, very nice feature is touch reload when testing routes I respect ur contribution, but u didn't convinced me that scgi is better than uwsgi Come on, no need to fight :) I do not think Michele is saying that SCGI (that remember is only a communication protocol) is better than something else. Even Cherokee (albeit being a uWSGI supporter) uses SCGI pure-python for its admin interface. Implementing SCGI in python is very easy (no more than a dozen of lines) so if you only need a gateway (and not a deployment platform) it could be a very good choice. The Michele analysys about performance is a bit wrong as he should say: (for scgi) webserver - socket - python scgi parser - web2py (for uWSGI with uwsgi protocol) webserver - socket - C uwsgi parser - web2py But generally, if you need to attach an app to a webserver, implementing SCGI is the easiest choice (both uwsgi and fastcgi are binary protocol, and writing good http parser is not the easiest task in the world) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: uWSGI 0.9.9
the one that works with 0.9.8.5 uwsgi --protocol http --socket 127.0.0.1: --pythonpath /home/myusername/web2py --module wsgihandler -d /home/myusername/tmp/uwsgi.log -t 20 --async 24 --ugreen --limit-as 64 -r --no-orphans -M -p 1 --touch-reload /home/myusername/tmp/uwsgireload.txt --reload-on-rss 50 the one that works with 0.9.9.1 but exits on touch (to reload) and dispatches 2 child processes instead of one uwsgi --http 127.0.0.1: --pythonpath /home/myusername/web2py --module wsgihandler -d /home/myusername/tmp/uwsgi.log -t 20 --async 12 --ugreen --limit-as 36 -r --no-orphans -M -p 1 --touch-reload /home/myusername/tmp/uwsgireload.txt --reload-on-rss 50 Oh ok, you are pratically using --http addr instead of --http-socket addr The first one spawn a dedicated http server, the second one is a shortcut for --protocol http --socket addr -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: uWSGI 0.9.9
hi Roberto, This version seems to be backward incompatible: --protocol http --socket 127.0.0.1: doesn't seems to work for me and using uwsgi --http 127.0.0.1: with --touch-reload makes uwsgi to exit completely (no uwsgi processes after touching the file ) Hi, can you post the whole command line ? -- Roberto De Ioris http://unbit.it
Re: [web2py] upload/post of semi-large files
Hi, My production environment is ubuntu, postgresql, uwsgi and nginx. The issue I'm having is when posting a form with a semi-large file (~2 mb), where it keeps 'resetting' the % uploaded every 30 seconds up to 2 to 3 times, and finally failing (with no specific error that I could see). I thought this could be fixed by setting a big number for uwsgi's harakiri option (which defaults to 60 secs), but that doesn't seem to help. What should I do to fix this and allow posting forms with semi-large files that take about a minute or so to upload?. Thanks!, Carlos Check your nginx error log, you are probably receiving a 413 error (entity too large). By default the limit is 1 mb http://wiki.nginx.org/HttpCoreModule#client_max_body_size -- Roberto De Ioris http://unbit.it
[web2py] [ANNOUNCE] uWSGI 0.9.9
Hi everyone, i am pleased to announce a new major release of the uWSGI project. The 0.9.9 version focuses on optimizations in various area, an improved spooler and native gevent support. A new python-specific feature is the 'pyshell': adding --pyshell to your options will run a python shell in the uwsgi environment (useful for testing the uwsgi api). The rack plugin has been improved and the old-sharedarea has been revamped to support 64bit. Pump (an alternative to WSGI) support has been added, it is only a bunch of lines, so do not flame about it :) A new Emperor mode, called Tyrant, is available for highly-untrusted environments. This is the first release linkable as a shared library (will be the base for pypy support). Look at skel.c in source distribution for an example. * Changelog for 0.9.9 [20110829] - various spooler improvements - upload progress optimizations and fixes - new emperor mode: Tyrant http://projects.unbit.it/uwsgi/wiki/Emperor#tyrant - allows multiple --ini options - better binary path detection - gevent loop engine support http://projects.unbit.it/uwsgi/wiki/Gevent - resolve domain names in socket logging - snmp fixes and reintroduction of the snmp-api - fixed grunt mode - you can specify build profile via env var UWSGI_PROFILE - reintroduction of string-based multiapp - new option --emperor-throttle - improved queue and sharedarea subsystems http://projects.unbit.it/uwsgi/wiki/QueueFramework http://projects.unbit.it/uwsgi/wiki/SharedArea - uWSGI can now be built as a shared library - new api attribute: uwsgi.magic_table - improved apache2 module - new option --pyshell - allow threads in gateways - fastrouter improvements http://projects.unbit.it/uwsgi/wiki/FastRouter - the application list is now shared and exported to the api - allows setting the thread stack size - UWSGI_APPID for better multiapp hosting http://projects.unbit.it/uwsgi/wiki/uWSGIVars - better reloading in lazy mode - native pump support http://projects.unbit.it/uwsgi/wiki/Pump - various rack fixes - support for attaching binary data in the executable - new option --set - extreme-optimized http protocol parser - WSGI handler optimizations You can download it from: http://projects.unbit.it/downloads/uwsgi-0.9.9.tar.gz or pip install uwsgi Many thanks to all the people helping in the project. Remember to ping me for adding you in the CONTRIBUTORS file. -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: Tiny example on uwsgidecorators+web2py
Il giorno 22/lug/2011, alle ore 11:09, Tiago Moutinho ha scritto: thks Roberto, tiago@tiago:~$ cd /opt/web2py/ tiago@tiago:/opt/web2py$ mkdir myspool tiago@tiago:/opt/web2py$ uwsgi --socket :3031 --spooler myspool -- master --processes 4 --import mytasks --module web2py.wsgihandler uwsgi: unrecognized option '--import' Are you using the latest stable release ? --import was added in 0.9.8.2 (there are other ways to implement this in 0.9.8.1 but --import is so handy that the upgrade is worthy :) ) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Tiny example on uwsgidecorators+web2py
I using 0.9.8.1. How can i update to 0.9.8.2 thks in advance pip install uwsgi (or pip install -U uwsgi) Or download the sources from the official site and recompile them. BTW we are getting a bit Off-topic i suppose we can move the discussion to the uWSGI list (or the irc channel) and come back here when you have a working setup :) On Jul 22, 10:15 am, Roberto De Ioris robe...@unbit.it wrote: Il giorno 22/lug/2011, alle ore 11:09, Tiago Moutinho ha scritto: thks Roberto, tiago@tiago:~$ cd /opt/web2py/ tiago@tiago:/opt/web2py$ mkdir myspool tiago@tiago:/opt/web2py$ uwsgi --socket :3031 --spooler myspool -- master --processes 4 --import mytasks --module web2py.wsgihandler uwsgi: unrecognized option '--import' Are you using the latest stable release ? --import was added in 0.9.8.2 (there are other ways to implement this in 0.9.8.1 but --import is so handy that the upgrade is worthy :) ) -- Roberto De Iorishttp://unbit.it -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Tiny example on uwsgidecorators+web2py
Hello Roberto, I've tried to start web2py has you demonstrated: uwsgi --socket :3031 --spooler myspool --master --processes 4 -- import mytasks --module web2py.wsgihandler but I got this error: [spooler directory] access(): No such file or directory [uwsgi.c line 2994] What is wrong? the spooler directory must exists (it is a security measure) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Tiny example on uwsgidecorators+web2py
How can i create the spoller directory? mkdir myspool It is a normal, empty directory thks in advance On Jul 21, 6:06 pm, Roberto De Ioris robe...@unbit.it wrote: Hello Roberto, I've tried to start web2py has you demonstrated: uwsgi --socket :3031 --spooler myspool --master --processes 4 -- import mytasks --module web2py.wsgihandler but I got this error: [spooler directory] access(): No such file or directory [uwsgi.c line 2994] What is wrong? the spooler directory must exists (it is a security measure) -- Roberto De Iorishttp://unbit.it -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Application Scope?
Il giorno 11/lug/2011, alle ore 17.27, David J. ha scritto: I am not using celery; The more I think about it I think celery would be easier to work with than what I had asked; Celery is a process that runs in the background its uses AMPQ so that service is running; so if there were a client you could easily integrate the two; what I am asking would be a bit different; because I want when the application starts to start a background process; which you can do as Anthony pointed out in the docs; but the next question is that if you use uwsgi; then you would need some way to perform that same startup scenario in wsgi. Thanks. I do not want to look non-modest, but the current uWSGI (0.9.8.2-dev) version makes celery completely useless (do not forget that celery requires a daemon for itself, an amqp server and some form of integration). It is still not very clear to me what you are trying to accomplish, but looks like the spooler is what you are searching for. Try reading this http://projects.unbit.it/uwsgi/wiki/Decorators and then http://projects.unbit.it/uwsgi/wiki/Spooler Integrating with web2py is not a problem, as they all works with normal python functions, so simply define them and call asynchronously from web2py views. Again, i am not sure to understand what you are trying do to, but i bet some of the uWSGI features will help you without istalling additional stacks. -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Application Scope?
Il giorno 11/lug/2011, alle ore 17.48, David J. ha scritto: Roberto; Thanks so much; this is almost the information I am looking for; Spooler. I have to see how I would use this in web2py. Thanks. I have just realized that there is no doc about uwsgi api + web2py integration. I will write something in the next few hours -- Roberto De Ioris http://unbit.it
[web2py] Tiny example on uwsgidecorators+web2py
Just a tiny example, but should be enough for spooler jobs and timers http://projects.unbit.it/uwsgi/wiki/Decorators#Example:web2pyspoolertimer Obviously you can call web2py module/functions in the mytasks.py file (just import them as a normal python app) -- Roberto De Ioris http://unbit.it
Re: [web2py] Tiny example on uwsgidecorators+web2py
Roberto; Do you need the latest version of uwsgi or will http://projects.unbit.it/downloads/uwsgi-0.9.8.1.tar.gz This work? I see the docs say 0.9.8.2? not 0.9.8.1. You have to take the version from mercurial repository, it has tons of fixes for the signal framework. (the official release should happen this week) On 7/11/11 1:34 PM, Roberto De Ioris wrote: Just a tiny example, but should be enough for spooler jobs and timers http://projects.unbit.it/uwsgi/wiki/Decorators#Example:web2pyspoolertimer Obviously you can call web2py module/functions in the mytasks.py file (just import them as a normal python app) -- Roberto De Ioris http://unbit.it -- Roberto De Ioris http://unbit.it
Re: [web2py] Tiny example on uwsgidecorators+web2py
Is it possible to include these options in the init script? uwsgi socket127.0.0.1:9006/socket pythonpath/home/www-data/myapp//pythonpath app mountpoint=/ scriptwsgihandler/script /app /uwsgi Yes, all of the options can be moved in config files. uwsgi socket127.0.0.1:9006/socket pythonpath/home/www-data/myapp//pythonpath app mountpoint=/ scriptwsgihandler/script /app spoolerdirectory/spooler importmytasks/import master/ ... /uwsgi -- Roberto De Ioris http://unbit.it
Re: [web2py] Tiny example on uwsgidecorators+web2py
Robeto;
I am not exactly sure how I would start my application in the background
using this method;
The spooler is started (and managed) by uWSGI, you do not need to
daemonize it. If you want to daemonize all of your stack, simply add
--daemonize logfile to uWSGI options.
I think I need to run my app in the background and then access it to
add_tasks etc;
Thanks.
David.
If i have understand correctly, you want some kind of daemon receiving
messages from your workers. You cannot use threads (or their queue) as you
will lose communication between processes (a thread created in process 1
is obviously not available on process 2).
This example:
http://projects.unbit.it/uwsgi/wiki/Example#threadqueue
can be of interest for you, but you can use it only in multithreaded
environment (no multiple processes) otherwise you will need to create a
independent threadpool for each process (as the example does)
The Spooler is a process receiving messages, you can configure it to work
in the way you need without using decorators:
# create the spooler code server
def receive_request(dictionary_message):
print(i have received a message)
...do something...
# attach the function to the spooler
uwsgi.spooler = receive_request
Then from your workers/threads send messages (dictionary of strings) to it:
uwsgi.spool({'name':'foo','surname':'bar'})
If i have not understand correctly probably it is better for you to send a
real example of what you want to do.
--
Roberto De Ioris
http://unbit.it
Re: [web2py] Re: Call functions periodically from WEB2PY at short time basis (like 0.05 ... 5 seconds)
Il giorno 08/lug/2011, alle ore 18.14, Francisco Costa ha scritto: Hardo to say. 0.9.7 branch is now obsolete, and a lot of fix has been added to signal framework in 0.9.8. If you want to manage timer reliably you should use the latest tip (it is really the 0.9.8.2 release, i am only waiting for a last patch before release) -- Roberto De Iorishttp://unbit.it I now have installed uwsgi-0.9.8.1 from source How do you install uwsgi decorators in Ubuntu 11.04 I found that there is a python-uwsgidecorators package but its only available for Ubuntu 11.10 You can download the file from here http://projects.unbit.it/uwsgi/browser/uwsgidecorators.py and put it in the web2py directory (or whatever dir you have in the pythonpath) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Call functions periodically from WEB2PY at short time basis (like 0.05 ... 5 seconds)
This loops every 5 seconds regardless of how long the remainder of the loop takes (as long as it is somewhat less than 5 seconds) from time import time, sleep next_time = time()+5 while True: sleep(next_time-time()) next_time+=5 # ... I do not know if it is pertinent with the topic, but uWSGI has tons of (cheap, very cheap compared with solutions like celery) facilities to allow this sort of tasks: http://projects.unbit.it/uwsgi/wiki/Decorators -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Call functions periodically from WEB2PY at short time basis (like 0.05 ... 5 seconds)
I do not know if it is pertinent with the topic, but uWSGI has tons of (cheap, very cheap compared with solutions like celery) facilities to allow this sort of tasks: http://projects.unbit.it/uwsgi/wiki/Decorators I have uwsgi-0.9.7.1 installed is it compatible with uwsgi decorators? Hardo to say. 0.9.7 branch is now obsolete, and a lot of fix has been added to signal framework in 0.9.8. If you want to manage timer reliably you should use the latest tip (it is really the 0.9.8.2 release, i am only waiting for a last patch before release) -- Roberto De Ioris http://unbit.it
Re: [web2py] vps - memory problems
Hi, I'm testing my production environment (vps with ubuntu 10.4 lts lucid, nginx, uwsgi, postgresql, latest web2py), and I detected some memory problems, please check: root@vps:~# htop 1 [|| 1.0%] Tasks: 104 total, 1 running 2 [ 0.0%] Load average: 0.57 0.17 0.04 Mem[|||342/358MB] Uptime: 5 days, 05:55:08 Swp[ 297/1023MB] PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command 25948 www-data 20 0 124M 932 412 S 0.0 0.3 0:01.31 `- /usr/bin/uwsgi-python --master --no-orphans --processes 2 --logdate --chmod-socket=660 --uid www-data --gid www-data --daemonize /var/log/uwsgi-python/web2py.log --pidfile 25963 www-data 20 0 393M 88796 2124 D 0.0 24.2 0:12.39 | `- /usr/bin/uwsgi-python --master --no-orphans --processes 2 --logdate --chmod-socket=660 --uid www-data --gid www-data --daemonize /var/log/uwsgi-python/web2py.log --pid 25962 www-data 20 0 364M 189M 2116 S 0.0 52.9 0:11.34 | `- /usr/bin/uwsgi-python --master --no-orphans --processes 2 --logdate --chmod-socket=660 --uid www-data --gid www-data --daemonize /var/log/uwsgi-python/web2py.log --pid Add --memory-report to your uWSGI command line. You will get in /var/log/uwsgi-python/web2py.log the memory consumed after each request. If there is a leak you should find it preatty easy. -- Roberto De Ioris http://unbit.it
Re: [web2py] vps - memory problems
Hi, My models are relatively big in file size ( 200 kb in total). Is ALL this code being loaded into memory for each web request over and over again, and without reusing the same files (that do not change)?. I'm noticing a consistent increase in memory % in each of the uwsgi-python processes/workers for simple web requests. This makes me think that web2py (or uwsgi-python) is loading the code for all models into memory for each web request without any caching / reusing, and without releasing any memory from previous requests, but only consuming more and more memory each time. This means that I run out of memory after a very few requests ! Am I not correct above?, am I doing something wrong?, is this actually expected?, does this occur regardless of the web server or wsgi interface?. On a related note, should I move everything to modules (instead of models)?, will this fix the memory issues?. How can I solve this show-stopper problem?. Honestly, (without knowing your app) i do no think this is a problem related to web2py (and for sure it is not a uWSGI as its policy is: NO LEAK). Probably something in your app is leaking memory, but in the mean time you can patch the problem with something like --reload-on-rss n where n is the number of megs after which a worker is restarted (freeing its memory). I think 200 (megs) should be enough to maintain your webserver healthy. -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: how to upgrade Web2py in Webfaction?
http://projects.unbit.it/uwsgi/ http://projects.unbit.it/uwsgi/http://projects.unbit.it/uwsgi/wiki/Doc I use uwsgi without any other server and works fine so far you can limit memory usage setup instructions: you need to create a custom application wget http://projects.unbit.it/downloads/uwsgi-0.9.7.2.tar.gz tar xvzf uwsgi-0.9.7.2.tar.gz cd uwsgi-0.9.7.2 make -f Makefile.Py27 mv uwsgi /home/yourusername/bin uwsgi --http 127.0.0.1:your_custom_app_port --pythonpath /home/yourusername/web2py --module wsgihandler -d /home/yourusername/tmp/uwsgi.log -t 20 --async 51 --ugreen --limit-as 64 -r --no-orphans remark --limit-as 64 In 0.9.8 you can setup uwsgi to natively speaks http (so you will use less memory and you will get better performance as you do not need an additional proxy level): uwsgi --protocol=http --socket 127.0.0.1:your_custom_app_port --pythonpath /home/yourusername/web2py --module wsgihandler -d /home/yourusername/tmp/uwsgi.log -t 20 --async 51 --ugreen --limit-as 64 -r --no-orphans If i remember correctly, webfaction supports fastcgi too. 0.9.8 even talks fastcgi so you can set --protocol=fastcgi to gain a little more performance -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Web2py - the one-man-band of all python frameworks :)
It is still not clear to me if they mean a framework that does a little bit of everything (compliment) or a framework that does a little bit of everything (not compliment) or a framework with one developer (which I do not think is the case because I trust they know better) Anyway, perhaps you should ask them. Massimo It is a compliment, you can be sure. You have to read it in the rock/heavy metal way, where Web2Py is the talented player that can manage every intrument, but can make amazing concert with the best 'sessions musicians' out there (as he has the skills to choose them). So, as another user said in this thread, it can plays both ways. Sorry batteries included and on steroids are so over-used... ;) -- Roberto De Ioris http://unbit.it
