On Aug 20, 2:29 am, Jens Hoffrichter <[email protected]> wrote: > Hi everyone, > > As this mailing list has proved very proficient in the past in helping > with my problems, I thought you could help me now by brain storming > for more optimizations on our production site here. I have done all > the things I can think of by now, but maybe I'm missing something > fundamental. > > Recently, one of our pylons projects has gathered quite a bit of > attention, and the requests to the site have gone up massively. > > It is still a Pylons 0.9.6.2 project (yes, yes, I know how ancient > that is ;) ). It is running on a Xen Vserver with 4 GB of memory, 4 > cores from the dual i7 975 processor in that machine, and a chunk from > the raid configured in the logical volume manager. The operating > system for the machine is Debian Lenny. > > The site is running using daemon mode of mod_wsgi in Apache, most of > the static files are served directly by Apache (the ones which don't > need more access control). A tomcat is running on that machine, too, > for the Solr search engine which powers large amounts of the site. > > The database is a postgres for that site, mainly configured with the > defaults for a Debian installation. > > We are doing around 3k visits a day, with around 10-15k pageviews on the site. > > The problem I am seeing now is mostly load on the server, which is now > constantly near or over 1. WIth 4 cores, I know that I can go safely > up to a load of around 4, but I am still a bit concerned by the > scaling of the site. Another project (a drupal php site) is doing 20k > pageviews without taxing a very similar configured server at all. > > The "problems" in here might be related to a photo gallery we put live > a month or so ago, which hosts quite a bit of pictures, and offers a > drupal imagecache like function. I have looked very carefully that the > PIL functions are only used if no cached version of the resized photo > exists, to not tax the server memory too much. But I need to route all > requests for images through the controller, as we have strict access > control in place, so I can't just let the apache serve the cache > images. I'm using the FileApp approach I have asked about 2 months > back, which works without (visible) problem. > > I'm not seeing much "waiting" cpu load on the server, which would > indicate an IO bottleneck somewhere, but really user cpu load (in top) > > I just want to explore more options before throwing more hardware at > the project, if I am missing something fundamental here. > > This is just a rough sketch about the project, if you have an idea and > need more information about that, just ask, and I will try to provide > the information.
1. Have you minimized your static files as much as possible, especially your CSS and JS? This probably won't help much with CPU load, but it could make the app a bit snappier. 2. Are you running multiple instances of your app server? I've had good results with multiple Paste server instance and a simple Apache proxy cluster. 3. Just thinking out loud here: can you use Pylons to do auth but then still serve images directly from Apache, or can you implement your auth scheme in Apache? On a busy site, I definitely would try to avoid serving up all those images from Pylons/FileApp. -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
