Yeah, I'm with Bruce on that. We switched from mod_wsgi to nginx/gunicorn because it was a pain and took more resources.
Alex On Thu, Nov 11, 2010 at 3:52 PM, Bruce Kroeze <bkro...@gmail.com> wrote: > Bleah. > I'm going to jump in here and say - as I always do in this type of situation > - that Apache is a terrible solution for Django/Satchmo. Djangoproject's > recommendation is simply wrong for most use cases. > This is yet another reason why. You can't determine where the leak is. Is > it Apache? Is it mod_wsgi misconfigured? Who knows? It isn't something > you can get to the bottom of without a ton of testing, and in the meantime > your site is crashing. > Who cares? Stop using Apache. Mod_wsgi stinks, almost as much as > mod_python. > My preferred solution these days has changed, and I'm preparing a detailed > article about it, but it isn't very far from this solution. > http://brandonkonkle.com/blog/2010/jun/25/provisioning-new-ubuntu-server-django/ > Differences in my most-preferred-solution: I use buildout instead of pip, > and Lighttpd instead of Nginx. Neither change would affect the superiority > of this solution. > Here's the big deal - this is why you should do this in a nutshell: > 1) You will explicitly know which process is eating memory, since you will > have separate django daemon threads. > 2) Green Unicorn will allow you to kill and recycle worker threads after > timeouts. If it really is Satchmo eating memory, then that will release the > memory on a continual basis. > 2.5) Green Unicorn is almost unbelievably nice. Really. I love it. > 3) Your site will almost certainly be faster. In any case, much faster > compared to a crashed server. > 4) Figuring out what is wrong will be much much easier. > #4 is the biggest issue, IMHO. 90% of the cost of a store is maintenance. > Anything that makes the store easier to maintain and debug is worth it, > even if #3 is not true. Worth a little speed slowdown (doubtful in any > case) in exchange for testability and clarity. > Good Luck, > Bruce Kroeze > On Thu, Nov 11, 2010 at 1:27 PM, Alex Robbins > <alexander.j.robb...@gmail.com> wrote: >> >> Sorry, at this point its hard for me to say exactly what is going on. >> I'd say if you can't see any processes with the name you assigned in >> the config directive, then it seems like there probably aren't daemon >> processes. This is what happened to me earlier. I had to tweak and >> mess with the daemon process settings to get it to work. If I remember >> my situation correctly, the process group wasn't setup right, but >> yours looks ok to me. >> >> As for how is it running at all? If it isn't in daemon mode, then it >> is running in embedded mode. There is a python interpreter in every >> httpd process, which would take a lot of ram if you spawned a bunch of >> httpd worker processes. >> >> Alex >> >> On Thu, Nov 11, 2010 at 3:13 PM, Josh <josh...@gmail.com> wrote: >> > Ok so the conf now looks like this: >> > >> > WSGIDaemonProcess hatikva.com user=hatikva group=hatikva python-path=/ >> > usr/local/lib/python2.6/site-packages display-name=%{GROUP} >> > WSGIProcessGroup hatikva.com >> > WSGIScriptAlias / /home/hatikva/store/apache/store.wsgi >> > >> > but when I ps -A I don't see anything like wsgi:hatikva. Does this >> > potentially mean it's not really running in daemon mode? If so any >> > ideas on how it is running at all? >> > >> > >> > On Nov 11, 12:53 pm, Alex Robbins <alexander.j.robb...@gmail.com> >> > wrote: >> >> Yeah, looks like you are right. I think I normally used that >> >> 'display-name option' that Graham mentioned. Sorry about the confusion >> >> there. If you use that, then do they show up in ps? >> >> >> >> Alex >> >> >> >> On Thu, Nov 11, 2010 at 2:47 PM, Josh <josh...@gmail.com> wrote: >> >> > I also noticed that down near the bottom of this thread >> >> >> >> > >http://groups.google.com/group/modwsgi/browse_thread/thread/9d0e72b2c... >> >> > Graham Dumpleton said that "Under 'top' or 'ps', the mod_wsgi daemon >> >> > process will still show as a apache/httpd process." So I think it >> >> > all >> >> > gets lumped together. >> >> >> >> > On Nov 11, 12:38 pm, Josh <josh...@gmail.com> wrote: >> >> >> Hmm, i don't think I'm actually seeing the daemons. What do they >> >> >> show >> >> >> up as under COMMAND (I dont see anything with wsgi). But there is >> >> >> another small dev site running on the same server (they are both >> >> >> theoretically set up under daemon mode as I showed above). I think >> >> >> that if it wasn't in daemon mode they wouldn't both work although I >> >> >> could be wrong about that. Looking at the apache conf I posted >> >> >> earlier does it seem that I have properly configured daemon mode? >> >> >> Thanks. >> >> >> >> >> On Nov 11, 12:22 pm, Alex Robbins <alexander.j.robb...@gmail.com> >> >> >> wrote: >> >> >> >> >> > If you are really running mod_wsgi in daemon mode, then django and >> >> >> > satchmo won't be able to affect the size of the httpd processes. >> >> >> > The >> >> >> > python interpreter should live in the mod_wsgi daemon, which is a >> >> >> > completely separate process. I have had troubles before with >> >> >> > mod_wsgi >> >> >> > running in embedded mode, even though it is supposed to be daemon. >> >> >> > Can >> >> >> > you see any processes for mod_wsgi? If you don't get the config >> >> >> > exactly right, it won't use daemon mode. (This might not even be >> >> >> > the >> >> >> > problem, but if there aren't daemon processes something is >> >> >> > misconfigured.) >> >> >> >> >> > If you can see the daemon processes, and httpd is separate and >> >> >> > huge, >> >> >> > then I'm not sure what is going on. It wouldn't be python making >> >> >> > it >> >> >> > big. Maybe there are some other modules installed? (mod_php, >> >> >> > mod_rails >> >> >> > or something like that?) >> >> >> >> >> > On Thu, Nov 11, 2010 at 2:07 PM, Josh <josh...@gmail.com> wrote: >> >> >> > > Ok so the server is a VPS and it uses virtualmin and centos. My >> >> >> > > understanding is that each domain has its own httpd process (two >> >> >> > > if >> >> >> > > you have ssl enabled). I have been watching the output of top >> >> >> > > for >> >> >> > > awhile and when I say memory usage I am talking about the virt >> >> >> > > of a >> >> >> > > specific httpd process. I am fairly certain that this is the >> >> >> > > httpd >> >> >> > > process which the site runs from because it is the correct user >> >> >> > > and as >> >> >> > > I opened multiple connections the memory usage began going up. >> >> >> > > This >> >> >> > > single httpd process now is up to 867m virt and 651m res. There >> >> >> > > are >> >> >> > > other things on the system using memory but they have remained >> >> >> > > constant and I am concerned that a single process has gone up so >> >> >> > > much >> >> >> > > when it seems that nothing is going on with the site. Thanks >> >> >> > > for the >> >> >> > > help. >> >> >> >> >> > > On Nov 11, 11:53 am, Alex Robbins >> >> >> > > <alexander.j.robb...@gmail.com> >> >> >> > > wrote: >> >> >> > >> First off, mod_wsgi in daemon mode with apache should be a >> >> >> > >> decent >> >> >> > >> deployment method for ram consumption. I don't think your >> >> >> > >> problem is >> >> >> > >> there. >> >> >> >> >> > >> When you say the memory usage is 650mb, what is actually using >> >> >> > >> all >> >> >> > >> that memory? Is it httpd processes? The mod_wsgi processes? >> >> >> >> >> > >> On Thu, Nov 11, 2010 at 1:49 PM, Josh <josh...@gmail.com> >> >> >> > >> wrote: >> >> >> > >> > Oh yeah my urls looks like this: >> >> >> >> >> > >> > from django.conf.urls.defaults import * >> >> >> > >> > from store.urls import urlpatterns >> >> >> >> >> > >> > urlpatterns += patterns('', >> >> >> > >> > ('^pages/', include('django.contrib.flatpages.urls')), >> >> >> > >> > (r'^product_info\.php', >> >> >> > >> > 'store.localsite.views.old_redirect'), >> >> >> > >> > (r'^searchRedirect/', >> >> >> > >> > 'store.localsite.views.redirect_search'), >> >> >> > >> > (r'^reports/', 'store.localsite.views.reports.view'), >> >> >> > >> > ) >> >> >> >> >> > >> > and I have local_dev and debug set to false. (searchRedirect >> >> >> > >> > and the >> >> >> > >> > product_info\.php above were set up to redirect because the >> >> >> > >> > site used >> >> >> > >> > to be osCommerce based). >> >> >> >> >> > >> > On Nov 11, 11:46 am, Josh <josh...@gmail.com> wrote: >> >> >> > >> >> It is centos but we are using apache and mod_wsgi in daemon >> >> >> > >> >> mode, here >> >> >> > >> >> is the relevant part of my apache conf: >> >> >> > >> >> ------- >> >> >> >> >> > >> >> Alias /static/ /home/hatikva/store/static/ >> >> >> >> >> > >> >> <Directory /home/hatikva/store/static> >> >> >> > >> >> Order deny,allow >> >> >> > >> >> Allow from all >> >> >> > >> >> </Directory> >> >> >> >> >> > >> >> Alias /media/ >> >> >> > >> >> /usr/local/lib/python2.6/site-packages/django/contrib/ >> >> >> > >> >> admin/media/ >> >> >> >> >> > >> >> <Directory >> >> >> > >> >> /usr/local/lib/python2.6/site-packages/django/contrib/admin/ >> >> >> > >> >> media> >> >> >> > >> >> Order deny,allow >> >> >> > >> >> Allow from all >> >> >> > >> >> </Directory> >> >> >> >> >> > >> >> WSGIDaemonProcess hatikva.com user=hatikva group=hatikva >> >> >> > >> >> python-path=/ >> >> >> > >> >> usr/local/lib/python2.6/site-packages >> >> >> > >> >> WSGIProcessGroup hatikva.com >> >> >> > >> >> WSGIScriptAlias / /home/hatikva/store/apache/store.wsgi >> >> >> >> >> > >> >> ------- >> >> >> >> >> > >> >> My understanding is that the way the static directory is set >> >> >> > >> >> up above >> >> >> > >> >> means that apache and not django serves media. >> >> >> >> >> > >> >> As I have watched the memory usage has continued to go up, >> >> >> > >> >> its now at >> >> >> > >> >> ~650m, up from ~220m (I have stopped refreshing and this has >> >> >> > >> >> happened >> >> >> > >> >> in the past 20 minutes or so). >> >> >> >> >> > >> >> I am open to recommendations as to a better setup (which I >> >> >> > >> >> may or may >> >> >> > >> >> not be able to do depending on the person I have made the >> >> >> > >> >> site for). >> >> >> > >> >> Thanks for the quick response! >> >> >> >> >> > >> >> -Josh >> >> >> >> >> > >> >> On Nov 11, 11:39 am, Laszlo Antal <lzan...@gmail.com> wrote: >> >> >> >> >> > >> >> > Hi, >> >> >> >> >> > >> >> > Could you check to make sure django does not serve static >> >> >> > >> >> > media? >> >> >> > >> >> > I had a very similar issue (4000+ products) and I left by >> >> >> > >> >> > accident the static_serve in urls.py >> >> >> > >> >> > Just a thought >> >> >> >> >> > >> >> > lzantal >> >> >> >> >> > >> >> > On Nov 11, 2010, at 11:30, Josh <josh...@gmail.com> wrote: >> >> >> >> >> > >> >> > > I have been working on a satchmo site with ~3000 >> >> >> > >> >> > > products that has >> >> >> > >> >> > > repeatedly over the past week or so crashed the server >> >> >> > >> >> > > it is running. >> >> >> > >> >> > > It is on a VPS with 2G dedicated ram. It seems that >> >> >> > >> >> > > once the httpd >> >> >> > >> >> > > process allocates memory it never releases it, >> >> >> > >> >> > > eventually taking all >> >> >> > >> >> > > available memory and crashing the server. I tried >> >> >> > >> >> > > opening about ten >> >> >> > >> >> > > pages from the site and repeatedly hard refreshed them >> >> >> > >> >> > > and watched the >> >> >> > >> >> > > memory usage (via top) shoot up more than 150m in about >> >> >> > >> >> > > 10 minutes, >> >> >> > >> >> > > its still going up as I write this. >> >> >> >> >> > >> >> > > Are there any known memory leaks in satchmo? Why would >> >> >> > >> >> > > memory usage >> >> >> > >> >> > > continue to go up after I have stopped hard refreshing? >> >> >> > >> >> > > (I guess it >> >> >> > >> >> > > is possible that other people are visiting the site but >> >> >> > >> >> > > every two >> >> >> > >> >> > > second or so it seems to go up about 1m, which if it >> >> >> > >> >> > > continues the >> >> >> > >> >> > > server will crash again). Thanks in advance for any >> >> >> > >> >> > > help. >> >> >> >> >> > >> >> > > -Josh >> >> >> >> >> > >> >> > > -- >> >> >> > >> >> > > You received this message because you are subscribed to >> >> >> > >> >> > > the Google Groups "Satchmo users" group. >> >> >> > >> >> > > To post to this group, send email to >> >> >> > >> >> > > satchmo-us...@googlegroups.com. >> >> >> > >> >> > > To unsubscribe from this group, send email to >> >> >> > >> >> > > satchmo-users+unsubscr...@googlegroups.com. >> >> >> > >> >> > > For more options, visit this group >> >> >> > >> >> > > athttp://groups.google.com/group/satchmo-users?hl=en. >> >> >> >> >> > >> > -- >> >> >> > >> > You received this message because you are subscribed to the >> >> >> > >> > Google Groups "Satchmo users" group. >> >> >> > >> > To post to this group, send email to >> >> >> > >> > satchmo-us...@googlegroups.com. >> >> >> > >> > To unsubscribe from this group, send email to >> >> >> > >> > satchmo-users+unsubscr...@googlegroups.com. >> >> >> > >> > For more options, visit this group >> >> >> > >> > athttp://groups.google.com/group/satchmo-users?hl=en. >> >> >> >> >> > > -- >> >> >> > > You received this message because you are subscribed to the >> >> >> > > Google Groups "Satchmo users" group. >> >> >> > > To post to this group, send email to >> >> >> > > satchmo-us...@googlegroups.com. >> >> >> > > To unsubscribe from this group, send email to >> >> >> > > satchmo-users+unsubscr...@googlegroups.com. >> >> >> > > For more options, visit this group >> >> >> > > athttp://groups.google.com/group/satchmo-users?hl=en. >> >> >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups "Satchmo users" group. >> >> > To post to this group, send email to satchmo-us...@googlegroups.com. >> >> > To unsubscribe from this group, send email to >> >> > satchmo-users+unsubscr...@googlegroups.com. >> >> > For more options, visit this group >> >> > athttp://groups.google.com/group/satchmo-users?hl=en. >> >> >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Satchmo users" group. >> > To post to this group, send email to satchmo-us...@googlegroups.com. >> > To unsubscribe from this group, send email to >> > satchmo-users+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/satchmo-users?hl=en. >> > >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Satchmo users" group. >> To post to this group, send email to satchmo-us...@googlegroups.com. >> To unsubscribe from this group, send email to >> satchmo-users+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/satchmo-users?hl=en. >> > > > > -- > Bruce Kroeze > http://www.ecomsmith.com > It's time to hammer your site into shape. > > -- > You received this message because you are subscribed to the Google Groups > "Satchmo users" group. > To post to this group, send email to satchmo-us...@googlegroups.com. > To unsubscribe from this group, send email to > satchmo-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/satchmo-users?hl=en. > -- You received this message because you are subscribed to the Google Groups "Satchmo users" group. To post to this group, send email to satchmo-us...@googlegroups.com. To unsubscribe from this group, send email to satchmo-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/satchmo-users?hl=en.