Re: [uWSGI] Help
Em sáb, 2 de mar de 2019 às 10:58, Tamer Higazi escreveu: > > Dear Leo, > > The main problem is that the hardware already is with very poor specs. > You have a dual core CPU (perhaps 32 bit even) and then with 4GB RAM. Main problem!? Really!? He's on AWS of couse it's 64bits, 2 CPU cores is a LOT of power for more than 20 users, I serve 50 concurent user on AWS with a SINGLE core, and 1GB of RAM, that doesn't scratch 1% of CPU usage and it is writting to Postgres DB all the time, surely I don't use python/flask but 4GB is more than enough for that. > > you want to provide services and even like to earn money with it, and > start saving money on hardware. > Why this nonsense ? Nobody said that you have to buy HIGH-END Hardware, > but get the requirements 1st before doing anything. Sorry non-sense is caring about hardware for just 20 users... > > Have you turned on logging and see what had been written inside that > causes the crash? > What is written in the logs ? > Have you opened a shell and executed "top" to see what ressources are > consumed ? > > Serving long term connections is also no problem. > I have deald with websockets connections with written Python Stack and > nginx as backend without any problems at all. > > And why using flask web framework doing rest calls ? > Take Twisted. for example, or something totally small: > http://docs.python-eve.org/en/latest/ > > > best, Tamer Now for the log error message that is a bit unclear what happens, trying to mimic the problem with 'wrk' and perhaps a simple app to reproduce would help better. > > > PS: best is to answer to the list and not taking individual addresses in > CC like everybody else. > > > On 02.03.19 14:40, Léo El Amri wrote: > > On 02/03/2019 14:04, Tamer Higazi wrote: > >> 2. And with your comment "So please help to scale the application for > >> concurrent users." > >> > >> is very unpolite. > > I think it was just badly written english. I don't think they meant to > > be unpolite. > ___ > uWSGI mailing list > uWSGI@lists.unbit.it > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] post uploads for large files
IIRC if the value of post-buffering is lower than the file size the data is stored to a file, now about two temp files it's probably nginx doing the same thing. 2017-07-18 20:43 GMT-03:00 Marco Falcioni <mfalci...@fabricgenomics.com>: > Hello > we have a python app running on uwsgi behind nginx. I would like to > understand how post requests are cached to file before they are handed to > the underlying app. > For very large file uploads, we see that the same data is stored to a temp > file at least twice. > What controls this behavior? > Thanks > Marco > > > ___ > uWSGI mailing list > uWSGI@lists.unbit.it > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > > -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Switching from Nginx to uWSGI + OpenBSD's httpd for Ruby on Rails + Puma application
Also notice (from your gist) your app won't be forced to run chrooted, because httpd (chrooted) would talk to uWSGI which doesn't need to be, you can place the FastCGI socket under /var/www or serve it under 127.0.0.1. 2017-01-12 5:43 GMT-02:00 Roberto De Ioris <robe...@unbit.it>: > >> Hi, >> >> My Ruby on Rails app currently runs Nginx and Puma, however I'd like to >> replace Nginx with OpenBSD's native httpd which would involve using uWSGI >> as well. But, to be frank, I am super confused as to how I'm supposed to >> go >> about doing that as there is little to no docs on the subject apart from >> uWSGI's own docs which I find a wee bit overwhelming. > > > Hi, unfortunately OpenBSD httpd is relatively new so i am not surprised to > find really few docs. On the other side i have lot of customer using uWSGI > with Ruby On Rails, the support is really strong, but never became > something of 'mass use' like with python or perl. So you are pretty on > hard-paths :) My suggestion is starting with deploying your rails app with > uWSGI alone. The advantage is that uWSGI supports really old ruby/rails > combos so generally it is a good thing (in the long term) to invest on it. > > > -- > Roberto De Ioris > http://unbit.com > ___ > uWSGI mailing list > uWSGI@lists.unbit.it > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Fork memory usage
> Let me say it again, if you want to help with that fire up valgrind and see > where the memory is used and eventually leaked. I don't think it's leaked nor have the time atm to inspect it, I just started the thread to see if there was something different done before fork, if that's not the case maybe someone that know the code well could rethink something. Redesigning some bits of Cutelyst to better use --async might scale better on the long go.. Thanks ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Fork memory usage
2016-04-14 9:09 GMT-03:00 Riccardo Magliocchetti <riccardo.magliocche...@gmail.com>: > Il 14/04/2016 13:38, Daniel Nicoletti ha scritto: >> >> 2016-04-14 8:17 GMT-03:00 Riccardo Magliocchetti >>> >>> Il 14/04/2016 13:04, Daniel Nicoletti ha scritto: >>>> >>>> yesterday I was doing an experiment to see how fast would it be to uwsgi >>>> spawn 1000 workers with my app loaded, it took ~2s but 1.5GB of RAM, >>>> then instead of 1000 process I started 1 process and 1000 threads, >>>> namely >>>> my plugin will instantiate 1000 QThreads, and memory usage was 300MB >>>> now I got surprised of this, so I wrote an application that would load >>>> my >>>> app like uwsgi and fork 1000, granted this app doesn't have any >>>> protocols >>>> handling, and forking 1000 used 300MB as using threads did. >>>> >>>> uwsgi process here is of 800KB of size and my test app was of 100KB, >>>> which is aproximately the difference. >>> >>> >>> >>> Is less than a MB per process really that much? >> >> Sorry that was a typo, the uwsgi *file* has 800KB, each process takes >> ~1.5MB, >> so comparing 300MB to 1.5GB is around 1.2GB of memory that could >> be free to other stuff. Yes it's unlikely that I'll spawn 1000 workers >> for a single >> app but 100 apps with 10 workers would result in the same. > > > Aren't QThreads threads though? If so you are comparing processes with > threads which is unfair. Yes, I just said it's a QThread because it also carry the class overhead, It's not unfair since threads are basically process to linux kernel and from my simple test it showed that mem usage could be reduced. > >>>> Now I wonder why is uwsgi so big? Can it's protocols be split in plugins >>>> loaded at runtime to reduce this? It even seemed that share libraries >>> >>> >>> >>> Unless you embed plugins that's they way things currently are >>> >>>> didn't add up to free due shared nature, maybe it would be useful if >>>> uwsgi had a shared library, to possibily share among process. Or even >>>> is there something different about how uwsgi forks()? maybe explicity >>>> sharing less if that's even possible? >>> >>> >>> >>> As in any software of course things could be improved :) >> >> sure, I'm trying to understand what could be improved in this case > > > Well, then just fire up valgrind and see where the memory is used :) > Some low hanging fruit would be to reorder fields in structs but > unfortunately we can't because of ABI compatibility. That would probably reduce just a little and more likely only in runtime. what I just looked is that the uwsgi process seems to have a lot of text: daniel@bart:~/code/hello-cutelyst/build$ size -t /usr/bin/uwsgi-core /usr/bin/cutelyst textdata bss dec hex filename 762919 43080 13496 819495 c8127 /usr/bin/uwsgi-core 655161824 40 67380 10734 /usr/bin/cutelyst 19417161888 40 1943644 1da85c /home/daniel/code/cutelyst/build/cmd/cutelyst but even if I bloat my test app with even more text it still adds ~300MB to used ram, all I can think is that if uwsgi doesn't do anything special when forking I'd guess that it probably create say 1000 worker structs that are expensive and possibly could be freed on the worker process. >>>> Note that I dunno which build options Debian used to uwsgi, maybe it's >>>> possible to compile with reduced size or maybe it would just be better >>>> if >>>> it had it's code split into plugins. >>> >>> >>> >>> Most of the stuff is already splitted in plugins :) >> >> Even protocols are loaded at runtime? > > > I don't think so. > > > -- > Riccardo Magliocchetti > @rmistaken > > http://menodizero.it > ___ > uWSGI mailing list > uWSGI@lists.unbit.it > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Fork memory usage
2016-04-14 8:17 GMT-03:00 Riccardo Magliocchetti <riccardo.magliocche...@gmail.com>: > Hello, > > Il 14/04/2016 13:04, Daniel Nicoletti ha scritto: >> >> Hi, >> >> yesterday I was doing an experiment to see how fast would it be to uwsgi >> spawn 1000 workers with my app loaded, it took ~2s but 1.5GB of RAM, >> then instead of 1000 process I started 1 process and 1000 threads, namely >> my plugin will instantiate 1000 QThreads, and memory usage was 300MB >> now I got surprised of this, so I wrote an application that would load my >> app like uwsgi and fork 1000, granted this app doesn't have any protocols >> handling, and forking 1000 used 300MB as using threads did. >> >> uwsgi process here is of 800KB of size and my test app was of 100KB, >> which is aproximately the difference. > > > Is less than a MB per process really that much? Sorry that was a typo, the uwsgi *file* has 800KB, each process takes ~1.5MB, so comparing 300MB to 1.5GB is around 1.2GB of memory that could be free to other stuff. Yes it's unlikely that I'll spawn 1000 workers for a single app but 100 apps with 10 workers would result in the same. > >> Now I wonder why is uwsgi so big? Can it's protocols be split in plugins >> loaded at runtime to reduce this? It even seemed that share libraries > > > Unless you embed plugins that's they way things currently are > >> didn't add up to free due shared nature, maybe it would be useful if >> uwsgi had a shared library, to possibily share among process. Or even >> is there something different about how uwsgi forks()? maybe explicity >> sharing less if that's even possible? > > > As in any software of course things could be improved :) sure, I'm trying to understand what could be improved in this case >> Note that I dunno which build options Debian used to uwsgi, maybe it's >> possible to compile with reduced size or maybe it would just be better if >> it had it's code split into plugins. > > > Most of the stuff is already splitted in plugins :) Even protocols are loaded at runtime? > > > -- > Riccardo Magliocchetti > @rmistaken > > http://menodizero.it > ___ > uWSGI mailing list > uWSGI@lists.unbit.it > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] Fork memory usage
Hi, yesterday I was doing an experiment to see how fast would it be to uwsgi spawn 1000 workers with my app loaded, it took ~2s but 1.5GB of RAM, then instead of 1000 process I started 1 process and 1000 threads, namely my plugin will instantiate 1000 QThreads, and memory usage was 300MB now I got surprised of this, so I wrote an application that would load my app like uwsgi and fork 1000, granted this app doesn't have any protocols handling, and forking 1000 used 300MB as using threads did. uwsgi process here is of 800KB of size and my test app was of 100KB, which is aproximately the difference. Now I wonder why is uwsgi so big? Can it's protocols be split in plugins loaded at runtime to reduce this? It even seemed that share libraries didn't add up to free due shared nature, maybe it would be useful if uwsgi had a shared library, to possibily share among process. Or even is there something different about how uwsgi forks()? maybe explicity sharing less if that's even possible? Note that I dunno which build options Debian used to uwsgi, maybe it's possible to compile with reduced size or maybe it would just be better if it had it's code split into plugins. Best, -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] Data in post-buffering mode
Hi, I'm getting a crash on post buffering mode, this used to work, so I'm not sure it's related to some version o uwsgi, or if I am doing something wrong. basically my code calls uwsgi_request_body_read() to do the multipart parsing, after I know the body part offsets I call uwsgi_request_body_seek(0) after that the user will likely want to save one or all the parts to a file so another seek is set to point to the beggining of the part and uwsgi_request_body_read() is called. Now the function returns fine, as it has read data, but accessing the buffer for debugging or for calling memcpy crashes as it had already freed that buffer. I know this is the behavior of when not using post-buffering so I just fill a buffer to enable the same behavior. What could I be doing wrong? the code: https://github.com/cutelyst/cutelyst/blob/master/uwsgiEngine/bodyuwsgi.cpp#L58 thanks -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Data in post-buffering mode
Ok, I've looked at the code and concluded one cannot have uwsgi_request_body_read and uwsgi_request_body_readline mixed as readline has it's own pos and seek does not reset it, so I was doing this after seek: m_request->post_readline_pos = m_request->post_pos; and this caused the crash... I'll just remove the calls to readline so QIODevice emulates that for me... best 2016-02-01 11:03 GMT-02:00 Daniel Nicoletti <dantt...@gmail.com>: > Hi, > > I'm getting a crash on post buffering mode, this used to work, so I'm > not sure it's > related to some version o uwsgi, or if I am doing something wrong. > > basically my code calls uwsgi_request_body_read() > to do the multipart parsing, after I know the body part offsets I call > uwsgi_request_body_seek(0) > after that the user will likely want to save one or all the parts to a file > so another seek is set to point to the beggining of the > part and > uwsgi_request_body_read() is called. > Now the function returns fine, as it has read data, but > accessing the buffer for debugging or for calling > memcpy crashes as it had already freed that buffer. > > I know this is the behavior of when not using post-buffering > so I just fill a buffer to enable the same behavior. > > What could I be doing wrong? > > the code: > https://github.com/cutelyst/cutelyst/blob/master/uwsgiEngine/bodyuwsgi.cpp#L58 > > thanks > > -- > Daniel Nicoletti > > KDE Developer - http://dantti.wordpress.com -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] documentation of uwsgi.h file
2015-10-24 5:26 GMT-02:00 Bo Lorentsen <b...@lue.dk>: > On 10/24/2015 08:30 AM, Roberto De Ioris wrote: >> Hi, the cutelyst project is a qt webframework built upon uWSGI: >> >> http://cutelyst.org/ >> >> you can check their work or (as already suggested) look at the cplusplus >> included plugin as a starting point. > Ok, thanks ... I am not sure i would have found this project myself. > > They have made some done some wrapping work and use cmake for build, > thats a good start. > > Are there any were where the plugin hooks are defined, or is the best > action to just browse different plugins ? In cutelyst the plugin part is in uwsgiEngine directory https://github.com/cutelyst/cutelyst/tree/master/uwsgiEngine There in plugin.cpp https://github.com/cutelyst/cutelyst/blob/master/uwsgiEngine/plugin.cpp#L339 you have the hooks, which uWSIG will call. Depending on your usage you won't want a custom event loop (which is needed so that Qt classes work properly, Qt can integrate with any event loop but that would be more work), so not all hooks are needed. On the C++ plugin the hooks where defined with named syntax .name = "cplusplus", this isn't valid C++ code so you either have to put into a .c file file or use the way I did in plugin.cpp, which is better if you happened to share a config structure and not want to get unaligned warnings on 64bits. Best, -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Can I get improvement via gevent
If all your calls are sync then nothing will likely change, you need to check if the API you call is able to do it async and even if it can it needs to integrate with your event loop so it can get to work on another request until the the async call finishes, you might want to read: http://uwsgi-docs.readthedocs.org/en/latest/Async.html 2015-08-18 4:39 GMT-03:00 Jerry OELoo oylje...@gmail.com: Hi: I am using uWSGI + Flask to provide a web API, Currently, I am using processes + threads working mode. When get a query request from client, The Web API will call C so library API via ctypes, the C API will do some network I/O operation, the total time is around 0.1s. Maybe I will have 1k+ client query server at same time. Recently I know gevent which can support cocurrent well, So is processes + gevent suitable than processes + threads for my scenario? Thanks~ -- Rejoice,I Desire! ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] connection reset by peer
I have just found out that weighttp keep alive option was set but I didn't tell uwsgi to keep alive so it was closing the connections after request, thus making error: read() failed: Connection reset by peer (104) pop up and failing half of the requests adding: uwsgi_response_add_header(wsgi_req, (char *)Connection, 10, (char *)HTTP/1.1, 8); makes the http plugin to can_keepalive, however weighttp expects a keep-alive string to mark succeed++, which is what servers send, anyway to change this? Thanks, 2014-07-17 17:43 GMT-03:00 Daniel Nicoletti dantt...@gmail.com: hi, I'm running weighttp to measure my app performance and so far I get lots of: error: read() failed: Connection reset by peer (104) The application only writes a reply with hello world at first I thought it was some OS limit but the same doesn't happen to nginx. Any tips on what could be happening? Thanks -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] connection reset by peer
hi, I'm running weighttp to measure my app performance and so far I get lots of: error: read() failed: Connection reset by peer (104) The application only writes a reply with hello world at first I thought it was some OS limit but the same doesn't happen to nginx. Any tips on what could be happening? Thanks -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Qt event loop
2014-07-12 7:32 GMT-03:00 Roberto De Ioris robe...@unbit.it: So I did: uwsgi.wait_read_hook = uwsgi_simple_wait_read_hook; and it also works, however implementing the Qt loop async is a must, as the programer can easily create a local loop with QEvenLoop and have the socket notifier to get another request which fails to process here, what should be done? Many thanks, Some day ago you showed me an implementation of read/write hook using qt. I think it is the best approach to follow (using QSocketNotifier interface). That implementation is very inefficient as it creates a new QAbstractSocket on each call, creating and deleting takes time for no visible gain imo. Feel free to make a pull request for the uwsgi-qtloop project. I will do that once I have the engine in Cutelyst working well. By the way, i fear you did somethign wrong as uwsgi_simple_wait_read_hook is automatically initialized on startup (in the init.c source) Well I fear that too, but I couldn't find what I might have done wrong. (besides that I manually set my own loop engine in Cutelyst plugin, if that is a problem anyway). Would you mind bringing me some clarification to async mode? When using QCoreApplication being able to deal with requests in async mode is a must, basically what might happen is 1 New request #2 2 New request #1 3 Processing request #1 4 while processing the app creates a local loop which returns to main loop (QCoreApp) #1 Here what should happen is: 4 Processing request #2 I could create a new wsgi_request and call wsgi_req_setup(); however it might make sense to use the --async feature, but the core number that comes with it makes an almost infinite loop while a the async queue is freed, so I wonder what would you suggest. Besides this, can I assume for every uwsgi.thread will always be a uwsgi.sock? Thanks -- Roberto De Ioris http://unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] option typo
Hi,
I was trying to figure out if the
relation threads - uwsgi_socket was 1:1
and found this sacket typo:
core/uwsgi.c: {shared-socket, required_argument, 0, create a
shared sacket for advanced jailing or ipc,
uwsgi_opt_add_shared_socket, NULL, 0},
core/uwsgi.c: {undeferred-shared-socket, required_argument, 0,
create a shared sacket for advanced jailing or ipc (undeferred
mode), uwsgi_opt_add_shared_socket, NULL, 0},
--
Daniel Nicoletti
KDE Developer - http://dantti.wordpress.com
___
uWSGI mailing list
uWSGI@lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Qt event loop
So I did:
uwsgi.wait_read_hook = uwsgi_simple_wait_read_hook;
and it also works, however implementing the Qt loop async
is a must, as the programer can easily create a local loop
with QEvenLoop and have the socket notifier to get another
request which fails to process here, what should be done?
Many thanks,
2014-07-11 1:18 GMT-03:00 Roberto De Ioris robe...@unbit.it:
So I thought the wait_read_hook would have a default
implementation if I didn't overwrite it's function pointer,
so far I have added this:
static int cutelyst_wait_fd_read(int fd, int timeout) {
qDebug() Q_FUNC_INFO;
QAbstractSocket *sock = new
QAbstractSocket(QAbstractSocket::TcpSocket, qApp);
sock-setSocketDescriptor(fd);
sock-waitForReadyRead(timeout)
qDebug() Q_FUNC_INFO sock-bytesAvailable();
return sock-bytesAvailable();
}
and it works, however this is far from optimal.
Is the fd always an TcpSocket?
Since the fd is always the same (at least here),
wouldn't be better to use the socket notifier
on the req-fd?
Thanks
Your implementation should be added for sure in the qtloop plugin (feel
free to make a pull request even for the write and sleep hooks), but as
you said, it should fallback to the default implementation (the poll-based
one)
The default hooks are initialized in init.c:
uwsgi.wait_read_hook = uwsgi_simple_wait_read_hook;
uwsgi.wait_write_hook = uwsgi_simple_wait_write_hook;
uwsgi.wait_milliseconds_hook = uwsgi_simple_wait_milliseconds_hook;
uwsgi.wait_read2_hook = uwsgi_simple_wait_read2_hook;
--
Roberto De Ioris
http://unbit.it
___
uWSGI mailing list
uWSGI@lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
--
Daniel Nicoletti
KDE Developer - http://dantti.wordpress.com
___
uWSGI mailing list
uWSGI@lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] Qt event loop
Hi, I'm trying to include the Qt event loop you wrote, but I get a crash as soon as: int ret = uwsgi.wait_read_hook(wsgi_req-fd, uwsgi.socket_timeout); is called. When I attach gdb to it the process get stuck and I can't get a decent backtrace, do you have an idea of what can be wrong or how to get a proper bt? Best, -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Qt event loop
So I thought the wait_read_hook would have a default
implementation if I didn't overwrite it's function pointer,
so far I have added this:
static int cutelyst_wait_fd_read(int fd, int timeout) {
qDebug() Q_FUNC_INFO;
QAbstractSocket *sock = new
QAbstractSocket(QAbstractSocket::TcpSocket, qApp);
sock-setSocketDescriptor(fd);
sock-waitForReadyRead(timeout)
qDebug() Q_FUNC_INFO sock-bytesAvailable();
return sock-bytesAvailable();
}
and it works, however this is far from optimal.
Is the fd always an TcpSocket?
Since the fd is always the same (at least here),
wouldn't be better to use the socket notifier
on the req-fd?
Thanks
2014-07-10 13:34 GMT-03:00 Daniel Nicoletti dantt...@gmail.com:
Hi,
I'm trying to include the Qt event loop you wrote,
but I get a crash as soon as:
int ret = uwsgi.wait_read_hook(wsgi_req-fd, uwsgi.socket_timeout);
is called.
When I attach gdb to it the process get stuck and
I can't get a decent backtrace, do you have an idea
of what can be wrong or how to get a proper bt?
Best,
--
Daniel Nicoletti
KDE Developer - http://dantti.wordpress.com
--
Daniel Nicoletti
KDE Developer - http://dantti.wordpress.com
___
uWSGI mailing list
uWSGI@lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] file uploads
hmm well if you say it's a bad idea I can just remove readLine implementation and QIODevice will emulate it calling read and finding the end of line. Should I remove it then? This is how I'm using it: https://gitorious.org/cutelyst/cutelyst/source/961abeab6419b66385a255967f7e517fa8cfbd5c:uwsgiEngine/bodyuwsgi.cpp Thanks 2014-07-01 2:13 GMT-03:00 Roberto De Ioris robe...@unbit.it: Finally figured out what I was doing wrong with my IO device and now everything works but when someone class seek() on my device I have to uwsgi_request_body_seek(new_pos); request-post_readline_pos = request-post_pos; Any reason why readline pos isn't changed in body_seek function? well mixing readline based with chunk based reads is really messy, even when you call POSIX fread() after seek you will mess with buffering. Can you report how you are using it ? -- Roberto De Ioris http://unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] file uploads
Finally figured out what I was doing wrong with my IO device and now everything works but when someone class seek() on my device I have to uwsgi_request_body_seek(new_pos); request-post_readline_pos = request-post_pos; Any reason why readline pos isn't changed in body_seek function? ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] file uploads
So now I have a parser in place and an abstraction to read the body as a QIODevice which in turn calls uwsgi_(read|readline|seek) However the seek method fails if there is no request-post_file, looking at the source uwsgi_request_body_seek I see the function only does something if post-buffering is in place, but in the second if where the buffer limit didn't reach to create a file I can't re-read the buffer still: ie seek(0) and try to read() Should I read all the buffer and keep a second buffer instead? 2014-06-24 2:20 GMT-03:00 Roberto De Ioris robe...@unbit.it: 2014-06-23 6:00 GMT-03:00 Roberto De Ioris robe...@unbit.it: hi, I've been trying to get the uploaded files on a form, but I don't find where in the wsgi_request struct they are, I tried probably most of the params and none seemed to have it, I also tried to grep the code but nothing. What am missing? file uploads must be managed at the application level, you read the body and the parse it to obtain the file (or files) chunks. Thanks, now I wonder if there is some option where uwsgi stores the body in a file? if you enable post-buffering, the body of the request will completely read and stored in memory or a file. But i strongly suggest you to avoid this trick to read the body (pay attention, as i am talking about reading the body, interpreting it as a file upload is part of your app layer). It is something the user may want to control. If so can I still use uwsgi_request_body_read to read it independently on how/where is it stored? uwsgi_request_body_read uses always the same chunk of memory, you call it to get chunks and then you parse them to build a file. Generally file uploads are encoded as multipart/form-data, described here: http://www.w3.org/TR/html401/interact/forms.html basically every time you encounter the boundary string you start storing a new file. So, there are no fields in uWSGI for file uploads, as they are managed at higher (the application) level. -- Roberto De Ioris http://unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] file uploads
Nevermind, actually I'm doing something wrong when I qstrncpy the read char, after this uwsgi_body_ready() return empty strings so something is freeing it tho I explicity made a copy. 2014-06-27 12:30 GMT-03:00 Roberto De Ioris robe...@unbit.it: So now I have a parser in place and an abstraction to read the body as a QIODevice which in turn calls uwsgi_(read|readline|seek) However the seek method fails if there is no request-post_file, looking at the source uwsgi_request_body_seek I see the function only does something if post-buffering is in place, but in the second if where the buffer limit didn't reach to create a file I can't re-read the buffer still: ie seek(0) and try to read() Should I read all the buffer and keep a second buffer instead? it depends by your framework. I would follow a cheap approach: bodies bigger than N (8megs ?) are stored on disk, the others in memory. Technically this is what happens in post buffering mode. Maybe you can simply force it (like we do in the rack plugin, as the rack specs enforce seekable streams) 2014-06-24 2:20 GMT-03:00 Roberto De Ioris robe...@unbit.it: 2014-06-23 6:00 GMT-03:00 Roberto De Ioris robe...@unbit.it: hi, I've been trying to get the uploaded files on a form, but I don't find where in the wsgi_request struct they are, I tried probably most of the params and none seemed to have it, I also tried to grep the code but nothing. What am missing? file uploads must be managed at the application level, you read the body and the parse it to obtain the file (or files) chunks. Thanks, now I wonder if there is some option where uwsgi stores the body in a file? if you enable post-buffering, the body of the request will completely read and stored in memory or a file. But i strongly suggest you to avoid this trick to read the body (pay attention, as i am talking about reading the body, interpreting it as a file upload is part of your app layer). It is something the user may want to control. If so can I still use uwsgi_request_body_read to read it independently on how/where is it stored? uwsgi_request_body_read uses always the same chunk of memory, you call it to get chunks and then you parse them to build a file. Generally file uploads are encoded as multipart/form-data, described here: http://www.w3.org/TR/html401/interact/forms.html basically every time you encounter the boundary string you start storing a new file. So, there are no fields in uWSGI for file uploads, as they are managed at higher (the application) level. -- Roberto De Ioris http://unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Roberto De Ioris http://unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] file uploads
2014-06-23 6:00 GMT-03:00 Roberto De Ioris robe...@unbit.it: hi, I've been trying to get the uploaded files on a form, but I don't find where in the wsgi_request struct they are, I tried probably most of the params and none seemed to have it, I also tried to grep the code but nothing. What am missing? file uploads must be managed at the application level, you read the body and the parse it to obtain the file (or files) chunks. Thanks, now I wonder if there is some option where uwsgi stores the body in a file? If so can I still use uwsgi_request_body_read to read it independently on how/where is it stored? btw what is req-file, sendfile_fd and post_file? would be great if those parameter had a single line of commend :D BTW I've mentioned on IRC that master is producing an invalid .h file on --dot-h due to an odd char at the end. which char ? on my system it ends with two \n I git pulled today and maybe it was some corrupted file or something lying around as it is working again.. Best ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] file uploads
hi, I've been trying to get the uploaded files on a form, but I don't find where in the wsgi_request struct they are, I tried probably most of the params and none seemed to have it, I also tried to grep the code but nothing. What am missing? BTW I've mentioned on IRC that master is producing an invalid .h file on --dot-h due to an odd char at the end. Thanks, -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Virtual hosts
Thanks, I'll take a look then :) 2014-03-14 7:54 GMT-03:00 Guido Berhoerster guido+uw...@berhoerster.name: * Daniel Nicoletti dantti12-re5jqeeqqe8avxtiumw...@public.gmane.org [2014-03-14 00:45]: I have googled a bit, asked on irc and looked up on the docs, still I have not found if it's possible or not to setup uswgi to deal with virtual host setups. I ask this because if I run uswgi without ngnix siege report much more requests per seconds plus when there are more than 150 simultaneous connections nginx fail them. So is it possible to get rid of it? Yes, it is possible. The most flexible approach is to use the routing system (in particular route-host) in order to handle/manipulate requests (e.g. set the docroot, direct to a plugin) based on the hostname. -- Guido Berhoerster ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] Virtual hosts
Hi, I have googled a bit, asked on irc and looked up on the docs, still I have not found if it's possible or not to setup uswgi to deal with virtual host setups. I ask this because if I run uswgi without ngnix siege report much more requests per seconds plus when there are more than 150 simultaneous connections nginx fail them. So is it possible to get rid of it? Thanks ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] uWSGI and Cutelyst
Hi, I have chatted with some of you on IRC, but today's DDoS on FreeNode made me resort to the list :P So basically I'm writting a C++/Qt web framework, and because uWSGI allows me to get all uWSGI goodies without worrying to create parsers I created a plugin. It's almost done thanks to some people on IRC but I still have two issues, how do I iterate over wsgi_request-headers (trying to access any struct member crashes the app), and how should I handle loading application on init_apps(), by this I unsure because this method doesn't have arguments and so from where could I get the path to the apps to load. Besides that it's working great already, if you want to take a look at the code it is in: https://gitorious.org/cutelyst there is a uwsgiEngine dir containing the plugin. Thanks, -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
