> Hello again! Today’s topic is memory limits. > > We were hoping to use limit-as to limit the total addressable memory and > reload-on-rss (or evil-reload-on-rss) to catch gradual memory leaks. > > We’re in our usual setup: an emperor with chrooted vassals + workers. > > We found two problems – problems for us, they may not be problems for > uWSGI: > > 1) limit-as works, but it leaves the worker in a crashed state, where it > just hits a MemoryError for every request. Unfortunately, it does not die > and free up memory. So, the feature request would be, some kind of > monitor that kills workers when they’ve hit a memory limit. The next > problem is related however: > > 2) reload-on-rss does not work for us. We think it’s because our chroot > environment does not expose /proc, so the master process cannot tell how > much memory the worker is using. By way of evidence, we tried using the > memory-report flag to add memory info to the logs, and they would always > show 0virt and 0 rss used. The only solution we can think of is to give > the job of killing misbehaving workers to the emperor, instead of to the > master. > >
I suppose you do not want to move from chroot to namespaces, but the unshare() syscall can be the silver bullet for you: [uwsgi] chroot = /root001 ; detach mountpoints from the main namespace unshare = fs ; the following commands are executed in the chroot exec-as-root = mkdir /proc exec-as-root = mount -t proc none /proc ; drop normally... uid = foo gid = bar unshare will move your process to a new namespace for the mountpoints, so when you run "mount -t proc none /proc" you do not touch the main filesystem namespace. Even better (as you run untrusted code) will be: unshare = fs,ipc in this way even shared memory, semaphores and queuews will be "specific" for a vassal Some notes: in my hosting company we started limiting address space (now we moved to cgroups). I have to say it was not a winning choice as "virtual memory" is not a so clear concept for the average developer. Maybe the best approach is having it with a high limit and have a lower evil-reload-on-rss policy. Cgroups are lot more user-friendly and recent uwsgi releases (check the changelogs for some example) allows interfacing with cgroups event system (so users can be notified of memory errors or directly trigger events on it). Obviously if you prefer the Emperor to manage memory monitoring it is something we can talk about (i suppose we can simply spawn a thread scanning /proc for all the children of each vassal and make the sum) -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
