Just discovered htop tree view. With processes=5 threads=2 I wouldn't expect to see 5 threads under each process. Memory for one of these processes has ballooned to 283M. Eventually when all 5 of them get there, that means I'll be using 1.5G of ram for mod_wsgi.
20179 www-data 20 0 147M 76408 5680 S 28.0 0.9 10:17.85 │ ├─ mod-wsgi -k start 20240 www-data 20 0 147M 76408 5680 S 1.0 0.9 0:14.99 │ │ ├─ mod-wsgi -k start 20215 www-data 20 0 147M 76408 5680 S 12.0 0.9 5:05.16 │ │ ├─ mod-wsgi -k start 20214 www-data 20 0 147M 76408 5680 S 14.0 0.9 4:53.99 │ │ ├─ mod-wsgi -k start 20213 www-data 20 0 147M 76408 5680 S 0.0 0.9 0:00.63 │ │ ├─ mod-wsgi -k start 20212 www-data 20 0 147M 76408 5680 S 0.0 0.9 0:00.00 │ │ └─ mod-wsgi -k start 20178 www-data 20 0 138M 67680 5212 S 52.0 0.8 11:01.62 │ ├─ mod-wsgi -k start 20241 www-data 20 0 138M 67680 5212 S 0.0 0.8 0:15.45 │ │ ├─ mod-wsgi -k start 20230 www-data 20 0 138M 67680 5212 S 15.0 0.8 5:17.81 │ │ ├─ mod-wsgi -k start 20229 www-data 20 0 138M 67680 5212 S 35.0 0.8 5:24.63 │ │ ├─ mod-wsgi -k start 20228 www-data 20 0 138M 67680 5212 S 0.0 0.8 0:00.71 │ │ ├─ mod-wsgi -k start 20227 www-data 20 0 138M 67680 5212 S 0.0 0.8 0:00.00 │ │ └─ mod-wsgi -k start 20177 www-data 20 0 137M 67764 5428 S 7.0 0.8 10:47.27 │ ├─ mod-wsgi -k start 20207 www-data 20 0 137M 67764 5428 S 0.0 0.8 0:15.18 │ │ ├─ mod-wsgi -k start 20206 www-data 20 0 137M 67764 5428 S 7.0 0.8 5:16.82 │ │ ├─ mod-wsgi -k start 20205 www-data 20 0 137M 67764 5428 S 0.0 0.8 5:11.55 │ │ ├─ mod-wsgi -k start 20204 www-data 20 0 137M 67764 5428 S 0.0 0.8 0:00.69 │ │ ├─ mod-wsgi -k start 20203 www-data 20 0 137M 67764 5428 S 0.0 0.8 0:00.00 │ │ └─ mod-wsgi -k start 20176 www-data 20 0 138M 68144 5004 S 48.0 0.8 10:47.64 │ ├─ mod-wsgi -k start 20192 www-data 20 0 138M 68144 5004 S 1.0 0.8 0:15.62 │ │ ├─ mod-wsgi -k start 20191 www-data 20 0 138M 68144 5004 S 15.0 0.8 5:25.78 │ │ ├─ mod-wsgi -k start 20190 www-data 20 0 138M 68144 5004 S 31.0 0.8 5:02.00 │ │ ├─ mod-wsgi -k start 20189 www-data 20 0 138M 68144 5004 S 0.0 0.8 0:00.60 │ │ ├─ mod-wsgi -k start 20188 www-data 20 0 138M 68144 5004 S 0.0 0.8 0:00.00 │ │ └─ mod-wsgi -k start 20175 www-data 20 0 283M 197M 5540 S 72.0 2.4 12:24.34 │ └─ mod-wsgi -k start 20200 www-data 20 0 283M 197M 5540 S 0.0 2.4 0:14.62 │ ├─ mod-wsgi -k start 20199 www-data 20 0 283M 197M 5540 S 22.0 2.4 5:25.50 │ ├─ mod-wsgi -k start 20198 www-data 20 0 283M 197M 5540 S 50.0 2.4 6:40.60 │ ├─ mod-wsgi -k start 20197 www-data 20 0 283M 197M 5540 S 0.0 2.4 0:00.69 │ ├─ mod-wsgi -k start 20196 www-data 20 0 283M 197M 5540 S 0.0 2.4 0:00.00 │ └─ mod-wsgi -k start On Wednesday, February 19, 2014 12:05:56 PM UTC-5, scoopseven wrote: > > This question started on SO: > http://serverfault.com/questions/576527/apache-processes-in-top-more-than-maxclients/576600 > > I've updated my Apache config and mod_wsgi settings, but am still > experiencing memory creep. Here's my site conf and my apache2.conf: > > WSGIDaemonProcess mywsgi user=www-data group=www-data processes=5 > threads=5 display-name=mod-wsgi > python-path=/home/admin/.virtualenvs/django/lib/python2.7/site-packages > WSGIPythonHome /home/admin/.virtualenvs/django > WSGIRestrictEmbedded On > WSGILazyInitialization On > > <VirtualHost 127.0.0.1:8080> > ServerName www.mysite.com > DocumentRoot /srv/mysite > > SetEnvIf X-Forwarded-Protocol https HTTPS=1 > WSGIScriptAlias / /srv/mysite/system/apache/django.wsgi > process-group= mywsgi application-group=%{GLOBAL} > RequestHeader add X-Queue-Start "%t" > </VirtualHost> > > <IfModule mpm_worker_module> > StartServers 1 > ThreadsPerChild 5 > MinSpareThreads 5 > MaxSpareThreads 10 > MaxClients 25 > ServerLimit 5 > MaxRequestsPerChild 0 > MaxMemFree 1024 > </IfModule> > > I'm watching apache and mod_wsgi via htop and apache seems to be playing > by the rules, never loading more than 25 threads. It usually stays around > 10-15 threads. We average around 5-6 requests/second monitored by > /server-status/. The thing that's bothering me is that I'm counting 44 > mod_wsgi threads in htop. I assumed that since I had processes=5 threads=5 > I would only see a maximum of 30 threads below (5 processes + 25 threads). > > Partial htop dump: > > 2249 www-data 20 0 159M 65544 4676 S 26.0 0.8 2:09.93 mod-wsgi > -k start > 2248 www-data 20 0 164M 69040 5560 S 148. 0.8 2:10.72 mod-wsgi > -k start > 2274 www-data 20 0 159M 65544 4676 S 0.0 0.8 0:12.58 mod-wsgi > -k start > 2250 www-data 20 0 157M 62212 5168 S 10.0 0.7 1:50.35 mod-wsgi > -k start > 2291 www-data 20 0 164M 69040 5560 S 41.0 0.8 0:17.07 mod-wsgi > -k start > 2251 www-data 20 0 165M 69320 4676 S 0.0 0.8 1:59.48 mod-wsgi > -k start > 2272 www-data 20 0 159M 65544 4676 S 0.0 0.8 0:28.67 mod-wsgi > -k start > 2282 www-data 20 0 165M 69320 4676 S 0.0 0.8 0:33.85 mod-wsgi > -k start > 2292 www-data 20 0 164M 69040 5560 S 28.0 0.8 0:28.08 mod-wsgi > -k start > 2298 www-data 20 0 157M 62212 5168 S 0.0 0.7 0:14.93 mod-wsgi > -k start > 2299 www-data 20 0 157M 62212 5168 S 1.0 0.7 0:23.71 mod-wsgi > -k start > 2358 www-data 20 0 164M 69040 5560 S 1.0 0.8 0:02.62 mod-wsgi > -k start > 2252 www-data 20 0 165M 70468 4660 S 41.0 0.8 1:55.85 mod-wsgi > -k start > 2273 www-data 20 0 159M 65544 4676 S 10.0 0.8 0:29.03 mod-wsgi > -k start > 2278 www-data 20 0 159M 65544 4676 S 1.0 0.8 0:02.79 mod-wsgi > -k start > 2264 www-data 20 0 165M 70468 4660 S 0.0 0.8 0:07.50 mod-wsgi > -k start > 2266 www-data 20 0 165M 70468 4660 S 25.0 0.8 0:39.49 mod-wsgi > -k start > 2300 www-data 20 0 157M 62212 5168 S 6.0 0.7 0:28.78 mod-wsgi > -k start > 2265 www-data 20 0 165M 70468 4660 S 15.0 0.8 0:31.44 mod-wsgi > -k start > 2294 www-data 20 0 164M 69040 5560 R 54.0 0.8 0:34.82 mod-wsgi > -k start > 2279 www-data 20 0 165M 69320 4676 S 0.0 0.8 0:32.63 mod-wsgi > -k start > 2297 www-data 20 0 157M 62212 5168 S 3.0 0.7 0:09.68 mod-wsgi > -k start > 2302 www-data 20 0 157M 62212 5168 S 0.0 0.7 0:27.62 mod-wsgi > -k start > 2323 www-data 20 0 157M 62212 5168 S 0.0 0.7 0:02.56 mod-wsgi > -k start > 2280 www-data 20 0 165M 69320 4676 S 0.0 0.8 0:13.00 mod-wsgi > -k start > 2263 www-data 20 0 165M 70468 4660 S 0.0 0.8 0:19.35 mod-wsgi > -k start > 2322 www-data 20 0 165M 69320 4676 S 0.0 0.8 0:03.05 mod-wsgi > -k start > 2275 www-data 20 0 165M 70468 4660 S 0.0 0.8 0:02.72 mod-wsgi > -k start > 2285 www-data 20 0 164M 69040 5560 S 0.0 0.8 0:00.00 mod-wsgi > -k start > 2288 www-data 20 0 164M 69040 5560 S 0.0 0.8 0:00.11 mod-wsgi > -k start > 2290 www-data 20 0 164M 69040 5560 S 4.0 0.8 0:15.66 mod-wsgi > -k start > 2293 www-data 20 0 164M 69040 5560 S 20.0 0.8 0:29.01 mod-wsgi > -k start > 2268 www-data 20 0 159M 65544 4676 S 0.0 0.8 0:00.00 mod-wsgi > -k start > 2269 www-data 20 0 159M 65544 4676 S 0.0 0.8 0:00.11 mod-wsgi > -k start > 2270 www-data 20 0 159M 65544 4676 S 15.0 0.8 0:26.62 mod-wsgi > -k start > 2271 www-data 20 0 159M 65544 4676 S 0.0 0.8 0:26.55 mod-wsgi > -k start > > Last night I had processes=3 threads=3 and my NR capacity report reported > 100% usage ( > https://rpm.newrelic.com/accounts/67402/applications/1132078/optimize/capacity_analysis), > > so I upped it to processes=5 threads=5 and now I have 44 threads going. > Despite the instance count reported by NR staying relatively stable, memory > consumption continues to increase ( > https://rpm.newrelic.com/accounts/67402/servers/1130000/processes#id=152494639). > > I realize that nobody except for Graham can see those NR reports, sorry. > > Has anyone dealt with this situation before? > > Mark > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscr...@googlegroups.com. To post to this group, send email to modwsgi@googlegroups.com. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/groups/opt_out.
