2016-04-14 9:09 GMT-03:00 Riccardo Magliocchetti <[email protected]>: > Il 14/04/2016 13:38, Daniel Nicoletti ha scritto: >> >> 2016-04-14 8:17 GMT-03:00 Riccardo Magliocchetti >>> >>> Il 14/04/2016 13:04, Daniel Nicoletti ha scritto: >>>> >>>> yesterday I was doing an experiment to see how fast would it be to uwsgi >>>> spawn 1000 workers with my app loaded, it took ~2s but 1.5GB of RAM, >>>> then instead of 1000 process I started 1 process and 1000 threads, >>>> namely >>>> my plugin will instantiate 1000 QThreads, and memory usage was 300MB >>>> now I got surprised of this, so I wrote an application that would load >>>> my >>>> app like uwsgi and fork 1000, granted this app doesn't have any >>>> protocols >>>> handling, and forking 1000 used 300MB as using threads did. >>>> >>>> uwsgi process here is of 800KB of size and my test app was of 100KB, >>>> which is aproximately the difference. >>> >>> >>> >>> Is less than a MB per process really that much? >> >> Sorry that was a typo, the uwsgi *file* has 800KB, each process takes >> ~1.5MB, >> so comparing 300MB to 1.5GB is around 1.2GB of memory that could >> be free to other stuff. Yes it's unlikely that I'll spawn 1000 workers >> for a single >> app but 100 apps with 10 workers would result in the same. > > > Aren't QThreads threads though? If so you are comparing processes with > threads which is unfair. Yes, I just said it's a QThread because it also carry the class overhead, It's not unfair since threads are basically process to linux kernel and from my simple test it showed that mem usage could be reduced.
> >>>> Now I wonder why is uwsgi so big? Can it's protocols be split in plugins >>>> loaded at runtime to reduce this? It even seemed that share libraries >>> >>> >>> >>> Unless you embed plugins that's they way things currently are >>> >>>> didn't add up to free due shared nature, maybe it would be useful if >>>> uwsgi had a shared library, to possibily share among process. Or even >>>> is there something different about how uwsgi forks()? maybe explicity >>>> sharing less if that's even possible? >>> >>> >>> >>> As in any software of course things could be improved :) >> >> sure, I'm trying to understand what could be improved in this case > > > Well, then just fire up valgrind and see where the memory is used :) > Some low hanging fruit would be to reorder fields in structs but > unfortunately we can't because of ABI compatibility. That would probably reduce just a little and more likely only in runtime. what I just looked is that the uwsgi process seems to have a lot of text: daniel@bart:~/code/hello-cutelyst/build$ size -t /usr/bin/uwsgi-core /usr/bin/cutelyst text data bss dec hex filename 762919 43080 13496 819495 c8127 /usr/bin/uwsgi-core 65516 1824 40 67380 10734 /usr/bin/cutelyst 1941716 1888 40 1943644 1da85c /home/daniel/code/cutelyst/build/cmd/cutelyst but even if I bloat my test app with even more text it still adds ~300MB to used ram, all I can think is that if uwsgi doesn't do anything special when forking I'd guess that it probably create say 1000 worker structs that are expensive and possibly could be freed on the worker process. >>>> Note that I dunno which build options Debian used to uwsgi, maybe it's >>>> possible to compile with reduced size or maybe it would just be better >>>> if >>>> it had it's code split into plugins. >>> >>> >>> >>> Most of the stuff is already splitted in plugins :) >> >> Even protocols are loaded at runtime? > > > I don't think so. > > > -- > Riccardo Magliocchetti > @rmistaken > > http://menodizero.it > _______________________________________________ > uWSGI mailing list > [email protected] > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
