Michele, I'm using the admin feature to export as csv. Niphlod, records are not big. With rocket it works nice.
On Mon, Oct 7, 2013 at 6:17 PM, Michele Comitini <[email protected]> wrote: > There is not enough information to say much. > if possible try to stream directly rather than buffering data first in a > variable, this should make things faster and lighter. > > > 2013/10/7 Niphlod <[email protected]> >> >> did you try upping the memory limits ? >> limit-as, reload-on-as, reload-on-rss >> >> >> PS: dumping 50k records in a csv is definitely not something for a >> webserver (timeout, anyone ?) >> >> On Monday, October 7, 2013 10:46:34 PM UTC+2, viniciusban wrote: >>> >>> Hi guys. >>> >>> I'm experiencing a problem I think is not web2py fault, but uwsgi or >>> nginx configuration. >>> >>> I'm running web2py inside a Vagrant box (Virtualbox) with nginx. >>> >>> I have a table with more than 50k lines and when I try to export as >>> csv in admin, the generated file doesn't have all records. Each time >>> it generates a >>> different number of rows. >>> >>> >>> Reading logs, I found something: >>> >>> in /var/log/nginx/error.log: >>> 2013/10/07 20:25:15 [crit] 955#0: *291 open() >>> "/var/lib/nginx/uwsgi/9/00/0000000009" failed (13: Permission denied) >>> while reading upstream, client: 10.11.12.1, server: wsl, request: "GET >>> /wsl/appadmin/csv/db?query=db.publicacao.id%3E0 HTTP/1.1", upstream: >>> "uwsgi://unix:///tmp/web2py.socket:", host: "10.11.12.13", referrer: >>> "https://10.11.12.13/wsl/appadmin/select/db?query=db.publicacao.id%3E0"; >>> >>> >>> in /var/log/uwsgi/uwsgi.log: >>> SIGPIPE: writing to a closed pipe/socket/fd (probably the client >>> disconnected) on request >>> /wsl/appadmin/csv/db?query=db.publicacao.id%3E0 (ip 10.11.12.1) !!! >>> uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 296] >>> IOError: write error >>> [pid: 2168|app: 0|req: 53/153] 10.11.12.1 () {46 vars in 1015 bytes} >>> [Mon Oct 7 20:25:06 2013] GET >>> /wsl/appadmin/csv/db?query=db.publicacao.id%3E0 => generated 0 bytes >>> in 9064 msecs (HTTP/1.1 200) 7 headers in 371 bytes (4 switches on >>> core 0) >>> ...The work of process 2168 is done. Seeya! >>> worker 4 killed successfully (pid: 2168) >>> Respawned uWSGI worker 4 (new pid: 2326) >>> mapping worker 4 to CPUs: 1 >>> >>> >>> >>> >>> What is happening? It seems somehow the socket connection is being >>> lost. Do you have an idea to fix that? >>> >>> BTW, I'm using webp2y v2.4.6 on Ubuntu 12.04 server and it was configured >>> by >>> setup-web2py-nginx-uwsgi-ubuntu.sh distributed along with web2py. >>> >>> My /etc/uwsgi/web2py.ini file is: >>> [uwsgi] >>> socket = /tmp/web2py.socket >>> pythonpath = /home/vagrant/web2py/ >>> mount = /=wsgihandler:application >>> processes = 4 >>> master = true >>> harakiri = 60 >>> reload-mercy = 8 >>> cpu-affinity = 1 >>> stats = /tmp/stats.socket >>> max-requests = 2000 >>> limit-as = 512 >>> reload-on-as = 256 >>> reload-on-rss = 192 >>> uid = vagrant >>> gid = vagrant >>> cron = 0 0 -1 -1 -1 python /home/vagrant/web2py/web2py.py -Q -S >>> welcome -M -R scripts/sessions2trash.py -A -o >>> no-orphans = true >>> >>> >>> >>> Any help? >> >> -- >> Resources: >> - http://web2py.com >> - http://web2py.com/book (Documentation) >> - http://github.com/web2py/web2py (Source code) >> - https://code.google.com/p/web2py/issues/list (Report Issues) >> --- >> You received this message because you are subscribed to the Google Groups >> "web2py-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. > > > -- > Resources: > - http://web2py.com > - http://web2py.com/book (Documentation) > - http://github.com/web2py/web2py (Source code) > - https://code.google.com/p/web2py/issues/list (Report Issues) > --- > You received this message because you are subscribed to the Google Groups > "web2py-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
