The 500 requests per second number relies on the probably-unreasonable assumption that each request can complete in ~75ms. Deliberately making your requests take a whole 3 seconds each is, obviously, not going to work. You can only have 10 instances active at a time by default; if the pages you're serving actually take 3 seconds to complete you'll need to optimize things a whole lot or be stuck with a 3.33 request/sec maximum.
On Mar 1, 11:33 pm, Gary Orser <[email protected]> wrote: > Hi Nick, > > Hmm, I was running tests on a billing enabled appspot today. 100 > requests/test. > > 10 threads getting a URL with a 3 second sleep (to emulate > computation) on appspot, was the most I could get without getting 500 > errors. > If I raised the thread pool beyond 10, I started getting errors?? > > That doesn't reconcile very well with this statement from the > appengine website. > "Requests > The total number of requests to the app. The per-minute quotas for > application with billing enabled allow for up to 500 requests per > second--more than one billion requests per month. If your application > requires even higher quotas than the "billing-enabled" values listed > below, you can request an increase in these limits here. > " > > Is there some billing setting that affects this? > > Cheers, Gary > > PS. dead simple request handler. > > import time > from django import http > def sit(req): > time.sleep(3) > return http.HttpResponse('foo') > > errors are: > > 03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe) > 153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] "GET /sit/91 HTTP/1.1" > 500 0 - "gzip(gfe)" ".appspot.com" > W 03-01 04:15PM 58.197 > Request was aborted after waiting too long to attempt to service your > request. Most likely, this indicates that you have reached your > simultaneous dynamic request limit. This is almost always due to > excessively high latency in your app. Please > seehttp://code.google.com/appengine/docs/quotas.htmlfor more details. > > On Mar 1, 2:36 pm, Michael Wesner <[email protected]> wrote: > > > Correction/addition to my last email. > > > It turns out that our requests for this EC2 pull thing are actually much > > faster now. Gary and our other devs have reworked it. I need updated > > numbers, but they don't take 10s, probably more like 2s. We still have > > some heavy ~5s services though, so the same issue exists with the simul-req > > stuff, just to less extent. We don't actually hit this limit much now with > > the current beta that is in production, but it is low traffic at the > > moment. We are just getting ready to ramp up heavily. > > > I asked Nick what we should do, well just today after my last email, I have > > made contact with a Developer Advocate and whatnot, which is fantastic. It > > looks like we, as a business, will be able to have better contact with the > > GAE team. We would very much like to continue working with you to figure > > out what actions we can take and what provisioning we can do to make our > > product successful and scale it as we grow in the near future. Gary Orser > > will be replying to this thread soon with more findings from both our real > > app code and a little test app we are using and which he will share with > > you. > > > We plan on having a presence at Google I/O this year as we did at PyCon. > > Hopefully we can even get setup in the demonstration area at I/O. > > > Thanks Nick for your help. Could we possibly setup a quick skype conf call > > at some point? > > > -Mike Wesner > > > On Mar 1, 2010, at 1:13 PM, Michael Wesner wrote: > > > > Nick, > > > > If we (I work with Gary) require fairly heavy requests which run for > > > multiple seconds then it is not possible to get anywhere near 400 QPS. > > > The math used on the docs page only applies to 75ms requests. > > > > (1000 ms/second / 75 ms/request) * 30 = 400 requests/second > > > > so lets say each request takes 10 seconds (and ours, pulling data to EC2 > > > for a heavy operation that we can't do on GAE could take that much since > > > we have to process and update some XML before sending it) > > > > (1000 ms/second / 10000 ms/request) * 30 = 3 requests/second > > > > And that does not even take into account all the other traffic to our > > > application, nor the fact that many users could be doing this same heavy > > > operation at the same time. Our application will see spikes in this type > > > of activity also. The docs also mention that CPU heavy apps incur > > > penalties, which is vague and scary. > > > > Great effort is put into doing things in the most efficient way possible, > > > but not everyones apps can do everything in 75ms. Most all of our service > > > calls are under 250ms. We do have a little overhead from our framework > > > which we are constantly working on improving. Our application is AMF > > > service/object based which is inherently heavy compared to simple web > > > requests. It limits the amount of memcache work we can do also, but we > > > are also working on improving our use of that. > > > > We easily hit these boundaries during testing so I think we really need > > > much higher simultaneous dynamic request limits for not only our > > > production instance but our dev/qa instances so we can test and load them > > > to some degree. Our QA team could easily bust this limit 20 times over. > > > > So, Nick Johnson... I ask your advice. We are running a company/product > > > on GAE. We are more than happy to pay for quota/service/extra assistance > > > in these matters. What do you suggest we do? > > > > I should also mention that I spoke with Brett Slatkin at PyCon and he is > > > now at least semi-familiar with the scope of product we have developed. > > > I have exchanged contact info with him but have not heard anything back > > > from him yet. We would really appreciate contact or even a brief meeting > > > at some point (in person or otherwise). > > > > Thanks, > > > > -Mike Wesner > > > > On Mar 1, 2010, at 7:40 AM, Nick Johnson (Google) wrote: > > > >> Hi Gary, > > > >> Practically speaking, for an app that hasn't been given elevated > > >> permissions, you should be able to have at least 30 concurrent requests > > >> - equating to around 400 QPS if your app is fairly efficient. What > > >> problems are you running into that lead you to conclude you're hitting a > > >> limit at 4 QPS, and that the problem is at App Engine's end? > > > >> -Nick Johnson > > > >> On Sat, Feb 27, 2010 at 8:23 PM, Gary Orser <[email protected]> wrote: > > >> Hi all, > > > >> We were trying to create programmatic parallel access to our appengine > > >> application. > > > >> From EC2, we were attempting (with threads) to run parallel access > > >> (url gets/posts) to > > >> our appid. There are some long running processes that we need to run > > >> on EC2, for which > > >> we would like to get a bunch of information (entities + processing on > > >> appspot) quickly. > > > >> We seem to be running into a limit on the number of accesses that are > > >> allowed. > > >> (4 threads seems to be the effective limit) > > > >> Is there some sort of denial of service limit imposed on multiple > > >> accesses from a single IP? > > > >> Cheers, Gary > > > >> -- > > >> You received this message because you are subscribed to the Google > > >> Groups "Google App Engine" group. > > >> To post to this group, send email to [email protected]. > > >> To unsubscribe from this group, send email to > > >> [email protected]. > > >> For more options, visit this group > > >> athttp://groups.google.com/group/google-appengine?hl=en. > > > >> -- > > >> Nick Johnson, Developer Programs Engineer, App Engine > > >> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > > >> Number: 368047 > > > >> -- > > >> You received this message because you are subscribed to the Google > > >> Groups "Google App Engine" group. > > >> To post to this group, send email to [email protected]. > > >> To unsubscribe from this group, send email to > > >> [email protected]. > > >> For more options, visit this group > > >> athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
