Re: [web2py] Re: Web2py - the one-man-band of all python frameworks :)
On Thu, May 12, 2011 at 2:00 AM, Roberto De Ioris robe...@unbit.it wrote: You have to read it in the rock/heavy metal way, where Web2Py is the talented player that can manage every intrument, http://en.wikipedia.org/wiki/Ayreon Exactly :) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Cherokee server locking while upload
Thx, but what is the best solution for now to deploy web2py with more than 3k users online, which can upload and download files? No using a streaming upload webserver will solve your problem Move to nginx, this will solve your specific problem, and use X-Accel-Redirect (this is the equivalent of X-Sendfile in nginx) for all the downloads. On 27 ÜÐà, 11:46, Roberto De Ioris robe...@unbit.it wrote: Thx, processes4/processes helps. But so if 4 users go to page with image, or upload some file, all will be crashed again. Is it a good to have processes1000/processes? Sounds bad This is a normal deploy problem (not related to uWSGI). This is why async/evented is so loved this days :) If your machine can tolerate 1000 processes (but sorry i do not think so) use this, but every process is a full copy of your app stack so memory will be devoured. If you have so many users doing uploads, you can try threads (they are cheaper than processes) but the best solution would be Cherokee disabiling streaming uploads to pass the request to uWSGI only when the upload is done. I will talk to the Cherokee lead developer to reintroduce this feature (before 1 point releases Cherokee worked that way) -- Roberto De Iorishttp://unbit.it -- Roberto De Ioris http://unbit.it
Re: [web2py] Cherokee server locking while upload
I have web2py + uWSGI + Cherokee When I upload file, server is locking for all users. Does somebody khow how to fix it? You have to increase the number of workers (or threads). Cherokee is a streaming server so it will pass POST data to uWSGI as soon as they arrive leaving it blocked for all the upload cycle. Add -p n (ncpu*2 is a good number to start with) or if you are using a 0.9.7.x release you can use threads with --threads n -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: Cherokee server locking while upload
Thx, processes4/processes helps. But so if 4 users go to page with image, or upload some file, all will be crashed again. Is it a good to have processes1000/processes? Sounds bad This is a normal deploy problem (not related to uWSGI). This is why async/evented is so loved this days :) If your machine can tolerate 1000 processes (but sorry i do not think so) use this, but every process is a full copy of your app stack so memory will be devoured. If you have so many users doing uploads, you can try threads (they are cheaper than processes) but the best solution would be Cherokee disabiling streaming uploads to pass the request to uWSGI only when the upload is done. I will talk to the Cherokee lead developer to reintroduce this feature (before 1 point releases Cherokee worked that way) -- Roberto De Ioris http://unbit.it
Re: [web2py] web2py unavailable during file upload cherokee+ssl+uWSGI+web2py
Il giorno 04/gen/2011, alle ore 00.49, PhE ha scritto: Following Massimo's advice on my previous post on file upload with rocket/ssl, I decided to give another try to cherokee/web2py. Now I've got reliable ssl file uploads : 4 successful uploads of a 5 MB file (3 to 4 minutes transfer each). This is a great forward step ;-) But I still have a big issue : during the transfer the web2py service is unavailable. An attempt to access the index page of web2py makes Cherokee to respond : 503 Service Unavailable Cherokee web server 1.0.15 (Ubuntu), Port 443 It looks to me that you have no concurrency in your setup. Cherokee is a streamed-upload server so the uploads is entirely managed by your app. If you spawn only one uwsgi process this will be be busy during uploads and cannot accept new requests. Simply add -p n to your command line or processesn/processes to your xml config file. Where n is the number of processes you want to spawn. -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: edit the book, get a free book
Il giorno 19/nov/2010, alle ore 13.59, John Kim ha scritto: Oh, also I was just reading about Web2Py support on Cherokee via uWSGI: - http://web2pyslices.com/main/slices/take_slice/64 - http://groups.google.com/group/web2py/browse_thread/thread/5213ca17c3352c2e/221cebcfa01609b1?#221cebcfa01609b1 It's not mentioned in the official book, yet. Is uWSGI officially supported? I was wondering how it compares to FastCGI: - uWSGI seems faster - however some people ran into issues using uWSGI; were those resolved? John-Kim The only issue with Cherokee+uWSGI is on the Cherokee-wizard that does not support most of the new uWSGI features. (but this will be obviously fixed soon) If you configure manually the whole stack (as you would do with other webservers) there are no known issue. Here there is a list of (updated) example for web2py http://projects.unbit.it/uwsgi/wiki/Example#Web2Py2uWSGI (the links you posted are a bit outdated and over-complex) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: uWSGI examples page updated for web2py
I read the example with great interest, but I don't understand why 3 Web2Py instances, one XML configuration is better than multiple configuration files. I'm not saying it's not better, I just really don't understand it. It would seem that either way, one still has 3 uwsgi instances and 3 web2py instances. Could you please explain why this is useful? Imagine a sysadmin of a web2py-only company that has to deploy hundreds of apps. Every time its the same task. Having one config file that inherit common options could be a huge commodity (he add only 2 lines instead of defining a new file). It is matter of taste, for example i prefer to have a file for every app :) -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: remote admin access errors
Thank you for the quick reply. I'm not sure how to check whether uwsgi is setting the wsgi_url_scheme. A quick search of this group and the uwsgi wiki provided no direction. How can I check this? There are two (very old) threads on the official uWSGI list about https and scheme. uWSGI supports setting the scheme in two way, via the (non-standard) UWSGI_SCHEME var or via the standard HTTPS cgi variable. I suggest you to add the UWSGI_SCHEME var to uwsgi_params configuration: UWSGI_SCHEME $scheme; -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: nginx with bultin uwsgi module
I'm not sure I understand what the advantage of wsgi would be, in any case if somebody finds it useful here is the part of my nginx.conf with which I run web2py with https access on a remote server; if somebody could post instructions on running web2py through wsgi on nginx I would like to test the difference. It is uwsgi not wsgi. It is a network protocol for fast transfer of information (mainly string dictionary). It is binary based and very easy to parse (for a machine not for a human ;) ): http://projects.unbit.it/uwsgi/wiki/uwsgiProtocol You can think about it as simplified fastcgi/scgi, even if uwsgi is not (only) related to cgi/http messages. Then there is the uWSGI project (that uses the uwsgi protocol) that is a web application container (it was born as a WSGI only server, but over time it has evolved a lot, supporting other languages). It includes all you can expect from a deploy platform (failover, automatic respawning, process monitoring, inter-cluster communications, resource limiting, logging...). You can find web2py configuration example here: http://projects.unbit.it/uwsgi/wiki/Example#Web2Py2uWSGI Handlers for the uwsgi protocol are available even for Cherokee (included in the official distribution), Apache2 (as external module) and lighttpd (external, not very well tested, module) I hope it can be useful -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: cherokee problem with fastcgi and uWSGI
Il giorno 10/set/2010, alle ore 10.36, salbefe ha scritto: Roberto, thank you for all your responses by I can't make this thing working. As you told me I did the following: In a new terminal and as root I did :uwsgi --pythonpath /var/web2py -- module wsgihandler -s /tmp/ we2bpy.sock --socket :9092 and this one too: uwsgi --pythonpath /var/web2py --module wsgihandler --socket :9092 Then in cherokee-admin I created a new source as follows: Type: remote host. Nick: web2py Connection : 127.0.0.1:9092 And then I have Virtual Sever - default and rule - Directory / with all this config when I point to localhos I get: uWSGI Error, wsgi application not found. That is a step forward :) You should fix the SCRIPT_NAME, but it is an annoying task. Download the current mercurial tip (it is stable as it will be the 0.9.6.1) and it will automagically manage the SCRIPT_NAME issue. In the mean time i have written an updated cherokee wizard that supports xml,python,wsgi and ini files. -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: cherokee problem with fastcgi and uWSGI
Il giorno 10/set/2010, alle ore 11.17, salbefe ha scritto:
After downloading the current mercurial tip and after trying to
compile I get the following error:
[r...@tango uwsgi]# make -f Makefile.Py26
python2.6 uwsgiconfig.py --build
Traceback (most recent call last):
File uwsgiconfig.py, line 83, in module
gcc_version = str(spcall2(%s -v % GCC)).split('\n')[-1].split()
[2]
File uwsgiconfig.py, line 79, in spcall2
return p.stderr.read().rstrip().decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
787: ordinal not in range(128)
make: *** [all] Error 1
:((
Can you post the output of
gcc -v
?
--
Roberto De Ioris
http://unbit.it
JID: robe...@jabber.unbit.it
Re: [web2py] Re: cherokee problem with fastcgi and uWSGI
Il giorno 10/set/2010, alle ore 12.07, salbefe ha scritto: gcc -v Usando especificaciones internas. Objetivo: i686-redhat-linux Configurado con: ../configure --prefix=/usr --mandir=/usr/share/man -- infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/ bugzilla --enable-bootstrap --enable-shared --enable-threads=posix -- enable-checking=release --with-system-zlib --enable-__cxa_atexit -- disable-libunwind-exceptions --enable-gnu-unique-object --enable- languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk -- disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj- jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with- ppl --with-cloog --with-tune=generic --with-arch=i686 --build=i686- redhat-linux Modelo de hilos: posix gcc versión 4.4.4 20100630 (Red Hat 4.4.4-10) (GCC) Ok, retry with the latest tip (it was a problem with python3.k compatibility) -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: cherokee problem with fastcgi and uWSGI
Yes !! It's working now ! Thank you very much :) Only one question more: because I'm running uWSGI independently from cherokee, how can I do to start uWSGI at system startup? You can put the commandline in /etc/rc.local or if your system uses upstart you can use this script: http://projects.unbit.it/uwsgi/browser/contrib/upstart/uwsgi.conf you have to customize it for your needs but ALWAYS remember to pass the --uid and --gid options (otherwise uWSGI will run as root) -- Roberto De Ioris http://unbit.it
Re: [web2py] cherokee problem with fastcgi and uWSGI
Il giorno 09/set/2010, alle ore 12.42, salbefe ha scritto: Hello, I'm trying to deploy web2py with cherokee on a fedora core 13 distro but I have several problems on it. I have read the slice web2py with Cherokee via uWSGI: a simple, easy guide that it could be found in web2pyslices but this guide is outdated because it seems to be done for an older cherokee version than 1.X.X. Current cherokee version is 1.0.8 Some things are not clear to me: 1.- For example where should be web2py: on /var/web2py or /var/www/ web2py. After reading the slice it seems that it should be on /var/ web2py. 2.- www-data user does not exist on fedora distro so where it says sudo chown -hR www-data\: /var/web2py I did sudo chown -hR root\: / var/web2py 3.- Its impossible to follow steps E throught G because options described there does not exists anymore in cherokee admin interface and I could not find other places where they could be. After trying to setup following more or less these steps when I point to http://localhost I always get the cherokee test page. The same problem with the steps that are on the online book to setup cherokee with fastcgi interface. The steps described there should be now a bit different to setup web2py with cherokee. I need some help, please. Thanks in advance! Hi, first of all read the web2py related examples here: http://projects.unbit.it/uwsgi/wiki/Example choose the one you like and add --http :9090 (or substitute :9090 with whatever port you want) now go to http://localhost:9090 with your webbrowser. If all works well remove the --http option and restart the uwsgi server. Now go to the cherokee admin and map your resource (probably the default one) to the address of your uwsgi server. I suggest you to use TCP sockets instead of UNIX to avoid permission problems. -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: cherokee problem with fastcgi and uWSGI
Hello, This works fine: uwsgi --pythonpath /var/web2py --module wsgihandler -s /tmp/ we2py.sock --http :9090 If I point to localhost:9090 works ok and I see the welcome application. After that: If I make a config.xml file as is described on http://projects.unbit.it/uwsgi/wiki/Example and I try with the virtualserver wizard to creat a uWSGI server I get the following error: /var/web2py/config.xml seems not to be a valid uWSGI xml configuration file. To get these step ok my config.xml file has to be the same as this: uwsgi pythonpath/var/web2py//pythonpath app mountpoint=/ scriptwsgihandler/script /app /uwsgi I fear that Cherokee has an outdated wizard, you should be able to load every valid uWSGI xml file in it. I will try to send a patch to the Cherokee team ASAP. In the mean time do not use its wizard but simply add a remote source specifying the address of the uWSGI server -- Roberto De Ioris http://unbit.it
Re: [web2py] Re: cherokee problem with fastcgi and uWSGI
looking at /var/log/cherokee/error_log that is what I get:
[uWSGI] parsing config file /var/web2py/config.xml
*** Starting uWSGI 0.9.6 (32bit) on [Thu Sep 9 23:37:45 2010] ***
compiled with version: 4.4.4 20100630 (Red Hat 4.4.4-10)
Python version: 2.6.4 (r264:75706, Jun 4 2010, 18:20:16)
[GCC 4.4.4 20100503 (Red Hat 4.4.4-2)]
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag)
***
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow
upload could be rejected on post-unbuffered webservers ***
allocated 404 bytes (0 KB) for 1 request's buffer.
Setting PythonHome to /var...
'import site' failed; use -v for traceback
binding on TCP port: 36986
your server socket listen backlog is limited to 64 connections
initializing hooks...done.
'import site' failed; use -v for traceback
added /var/web2py/ to pythonpath.
interpreter for app 0 initialized.
Traceback (most recent call last):
File /var/web2py/wsgihandler.py, line 20, in module
import os
ImportError: No module named os
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 3360)
spawned uWSGI worker 1 (pid: 3361)
'import site' failed; use -v for traceback
added /var/web2py/ to pythonpath.
interpreter for app 0 initialized.
ImportError: No module named �O�^H^P^G�^H^P
[pid: 3361|app: -1|req: -1/1] 127.0.0.1 () {54 vars in 879 bytes} [Thu
Sep 9 23:37:46 2010] GET / = generated 46 bytes in 2 msecs (HTTP/1.1
500) 2 headers in 63 bytes (0 async switches on async core 0)
Please run uWSGI independently from cherokee (its wizard is broken) with
one of the configuration you find on the official uWSGI site.
Then simply add a 'remote source' that point to the socket you have
choosen and map it to the directory /
All of your pythonpath and virtualenv are wrong.
In one of the previous post you managed to run flawlessly with the
embedded http server. Simply substitute --http with --socket
--
Roberto De Ioris
http://unbit.it
Re: [web2py] Re: uWSGI examples page updated for web2py
Hi Roberto, I can use this setting in version 0.9.5? José Hi, yesterday the 0.9.6 version has been released -- Roberto De Ioris http://unbit.it
[web2py] uWSGI examples page updated for web2py
Hi all, we have added a section in our examples wiki page for web2py: http://projects.unbit.it/uwsgi/wiki/Example I hope this can be useful -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it
Re: [web2py] Re: uWSGI examples page updated for web2py
thanks. Some users have experience dropped requests with uWSGI on heavy traffic. Can you reproduce the problem? Never had specific report about this (socket misconfiguration apart), they should post relevant data to the uWSGI mailing-list. uWSGI will never take down your machine if the load goes wild, so probably they are reaching some (configurable) limit. If they experienced that on benchmarking there is a (old) procedure to follow: http://lists.unbit.it/pipermail/uwsgi/2009-December/40.html -- Roberto De Ioris http://unbit.it
