Steve, Mainly, I'm trying to get down to the core issue you are trying to avoid to see if going the Task Queue route pays you more than it can cost you.
Have you actually run into dynamic request limit issues or throttling of your app (when you were just doing vanilla puts)? Or did you more generally get one of those log warnings saying something to the effect of "This process used a high amount of CPU and may be throttled (or may soon exceed its quota)."? If you do see that error.. what is the exact one? It would seem odd that Google would throttle your app across the board just because the occasional put took more than 1 second, and that seems wrong of them to do. Though, I am very familiar with hitting the dynamic request limit due to thousands of near simultaneous calls to the app that each take more than 1 second to run. Most of my app is all back end processing using chained and fanned out tasks so I'm definitely not here to tell you to not use Task Queue. But, the put process you describe has several moving parts, and each part is a point of failure. So, I'm trying to determine if you had to go this path due to actual throttling, or if you went down this path because you thought you had to in order to avoid the potential of being throttled. (This is no direct judgement of what I perceive your abilities or foresight to be.. I ask questions like this since I am frequently rewriting complex processes that I initially came up with.. only to later realize that the reasons I had for creating the complex process no longer exist. So, I am perpetually churning out a bunch of code to get something done.. and then post-optimizing it back down to a more manageable size after the fact.) On Thu, Nov 11, 2010 at 2:35 PM, stevep <[email protected]> wrote: > On Nov 10, 11:28 am, Eli Jones <[email protected]> wrote: > > > How big is the average entity for this Model that you are putting to? > (Are > > you just putting one entity at a time?) > > The data will vary based on user input. For the model in question, the > size is generally 3k or less. However, we have a large number of > custom indices to support user searches on multiple combinations of > properties. Pretty sure the index updates are the real resource cost > for the new record put(). > > > > > If you create a separate Model with the same properties but no indexes, > how > > long and how much CPU does it use up on a put (in comparison to your > fully > > indexed model)? Also, what do you mean by "puts run too close to the > 1,000 > > ms limit"? Do you just mean that your app uses up 1,000 CPU MS or 1,000 > > API_CPU MS? > > If I look at the appstats detail for my POST function the api-cost > cost for the most recent put is approximately 585 ms. There are some > other DB accesses in the full function call, so I segmented out this > data. This 585 ms is the bulk of the total api-cpu cost for the full > function. I am sorry, but right now I do not have anything set up to > test a write a similar model entity without any indices. > > To be honest, I am not sure what the "throttling due to 1,000 cpu ms > response" really means relative to CPU or API_CPU ms, whether it > involves any smoothing across multiple handler functions. > > Just looking at the 585 ms api put() resource indicates we would > likely have throttling issues should the infrastructure slow down due > to overall load. So, we're moving the put()s to the task queue. This > is in accordance to what I believe Google engineers' comments to this > forum about the 1,000 ms throttling affect (some posts perhaps > suggesting 800 ms is a more practical limit vs. the 1,000 ms > "theoretical limit" -- the latter being my term). > > The TQ approach also allows us to break up some of the overhead in the > current single function. Overall, we'll be much better off re: > throttling risk, and task separations, but it does impose the overhead > of not being able to respond to the client that the everything related > to the function was complete in response to a client's single POST > request. > > > > > Why are you generating a custom integer id instead of using the one that > the > > datastore would create (I am not saying you should not do this, but I am > > wondering what the requirement is that makes you need to do it.)? > > In retrospect, using our own generated integer key value may not have > been needed when the put() is done by the handler function called by > the client POST call. However, as we look to use the task queue for > executing the post, it will prove beneficial because it allows us to > quickly stage the task queue update, and provide information to the > client about what key value to request when it seeks to confirm the > task queue function using GETs subsequent to the initial POST. > > > > > Also, you mention that you are not very write intensive and new records > > occur infrequently.. so what is the main reasoning for this complicated > put > > process (does the processing leading up to the put place you near the 30 > > second limit)? > > The current process is pretty simple. If we did not face the issue > with our handler being throttled because we have a transaction or two > going through over 1,000 ms, then I'd leave it as is. > > However (again according to my reading of their comments) Google's > engineers advocate moving logic that can take over 1,000 ms to the > task queue. I'm ok with that quite honestly. > > As I think through the TQ step we will need to use, I am not sure how > a client can with a single POST to the on-line handler can be assured > that the new record TQ put() is completed. So that leads me to the > complicated process of the initial client POST being followed by a GET > to ensure completion. > > If I am missing something, and the client can simply send the POST > data to the Task Queue and assume 100% of the TQ tasks sent will be > completed, then please advise. Just having the client send the current > single POST call is so much easier. > > We are no where near 30 seconds for anything, even with a cold start. > No client URL calls currently take more that few seconds when combined > with a cold start. Prior to the Nov 6th update, all the new record > puts for this model were between 1-2 seconds, but are now coming in > lower than 1 sec. > > Again, this is all to avoid being throttled due to a the new record > put()s should: 1) current gains from Nov 6th maintenance not hold, or > 2) variability in infrastructure load even with the Nov 6th gains push > our 585 ms api-cpu over 1,000. (Getting throttled when the > infrastructure overall is running slow is a double whammy for which > users might not wait!) > > > > > Depending on what your restrictions are.. there are different > > recommendations that can/could be made. > > I'll happily try to clarify the restrictions, but am not sure what is > being asked here. > > Again, many thanks for your help. > stevep > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
