On Fri, May 30, 2008 at 5:12 AM, Alex Marandon <[EMAIL PROTECTED]> wrote: > > 2008/5/30 Graham Dumpleton <[EMAIL PROTECTED]>: >>> class FooController(BaseController): >>> >>> def bla(self): >>> return 'Hello World! %s' % time.time() >>> >>> def slow_bla(self): >>> time.sleep(10) >>> return 'Hello slow World!' >>> >>> With something like that, manual testing works just fine (ie: I'm able >>> to hit the action bla as many times as I want while the action >>> slow_bla is still serving). >>> >>> When testing with ab though, I do get failed requests, but I actually >>> get even more failed requests with Apache/mod_wsgi. >> >> What options are you using to 'ab'? > > -n 1000 -c 3 > >> The 'ab' program can give a perception of errors, normally manifesting >> as lots of exceptions on server side in log file, if you use >> concurrent requests. This occurs because it only stops when it >> receives responses for the number of requests you tell it to make. >> With a request handler that takes a long time, because of a sleep() >> call, the 'ab' program can actually issue more requests than you tell >> it to. When it does receive responses for the number you told it, it >> will simply shutdown the connections for all the additional requests >> it made which it hasn't yet received a response. The result of this is >> that on the server side it will see a bunch of closed connections >> causing failures to be able to write the responses for the additional >> requests. > > Note that I didn't test against the action that does a sleep. I'm a > bit confused by your explanation. Does it mean that the errors should > show up in the logs on the server side? I was only looking at ab's > output.
I have a few comments: * First of all, within the last few weeks, there was a massive thread about the best way to deploy Pylons applications. That's a must read. Personally, I opted for Nginx with Paster, but YMMV. My situation is a bit strange anyway. * There were a couple good points made above about whether to run one or multiple copies of Paster. It's true that this is one place where Pylons/Python is very different from Rails/Ruby, as mentioned above. If you're mostly blocking on IO, then multiple copies won't help much since multithreaded programs are fine in Python if they're IO constrained. If you're blocking on CPU, then multiple copies of Paster running are a really good idea, otherwise the GIL will get in the way of your making use of multiple cores. * Socket connections have a setting for how many requests are allowed to wait in the queue before being accepted. I don't know what the setting for Paster is or how to change it. I do know that I've used ab before, and it hit this limit. This is the likely cause of "Connection Denied" errors. * ab is a bit simple minded to begin with. I've told it to use 100 concurrent requests (which I saw that you didn't do), and it saw all sorts of failed connections. Of course, getting 100 *concurrent* requests isn't all that common for most production servers. If you were able to handle 100 concurrent requests, and process each in 0.1 seconds, that would lead to 86,400,000 requests a day! Yeah, I wish ;) JMeter is nice in that it allows you to script how a real user would behave. I've also heard that HTTP Perf is nice, but I haven't tried it. * I saw a great talk on Google App Engine yesterday where one of the speakers was criticizing naive load testing. He said that if you slam a Google App Engine app, it doesn't have enough time to warm up a bunch of Python interpreters to serve your app. It'll perform poorly. However, if you test a more sustained, realistic, and organic growth rate, the infrastructure organically grows to scale. It was interesting. Anyway, happy hacking, and best of luck! -jj -- I, for one, welcome our new Facebook overlords! http://jjinux.blogspot.com/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
