Hi Nikita, On Thu, Aug 12, 2010 at 3:02 PM, Nikita Srivastava <[email protected]>wrote:
> Thanks for the suggestions. I had one more question - Although we're using > sparse entity distribution, each of our entry is indexed on two primary > properties, so we don't build any composite indexes. But the entities are > retrieved through the basic index at a really fast rate, no writes, only > reads, but they are really fast. Should I be worried about contention > issues? Some entities might face 1000+ reads/sec retrieved though a query > done on a property. But we retrieve only upto 30-50 entities per query. No, this won't be a problem. Bigtable tabletservers cache frequently read data in memory, and split tablets across more servers if they're more popular - so it would take a truly astounding read rate on a single row to even begin to cause an issue, at which point you should be using Memcache anyway! -Nick Johnson > Thanks again! > > > On Thu, Aug 12, 2010 at 1:32 AM, Ikai L (Google) <[email protected]>wrote: > >> Measure your QPS, use AppStats, and be sure your requests that are user >> facing finish quickly. If the majority of your requests finish in under >> 800ms and return to the user, we'll spin up new instances for you. 400ms is >> best, however. That's 800ms of perceived time for the user - not total time >> (datastore calls are parallelized, so you could consume 2000ms of CPU time, >> but if your request returns to the user in 300ms you are okay). Do any work >> that will take longer in Task Queues or Cron when possible. >> >> On Wed, Aug 11, 2010 at 11:11 AM, Nikita Srivastava <[email protected] >> > wrote: >> >>> Makes sense. Also, another user here pointed me to the 1.3.6 specs which >>> says that custom error pages would be soon available. So the Quota Error be >>> less of an issue, because we do get errors and we do show them, but the main >>> issue was fear of sudden 'app engine' error when no user knows what app >>> engine is. I hope 1.3.6 resolves this. >>> >>> As for the reason you mentioned, I think what we should do is set daily >>> budget twice of what we paid for CPU previously, which was not cheaper I >>> think ($0.08 per 1.2Ghz CPU hour). So I am planning to set this to >>> $2000-2500 range as daily limit to avoid the Quota errors to legit users. >>> Any other things that you can suggest that I should do? >>> >>> Thanks! >>> >>> >>> On Wed, Aug 11, 2010 at 8:13 PM, Nick Johnson (Google) < >>> [email protected]> wrote: >>> >>>> Hi Nikita, >>>> >>>> On Wed, Aug 11, 2010 at 7:01 AM, Nikita <[email protected]> wrote: >>>> >>>>> Hello, >>>>> >>>>> We're trying to deploy a large project on app engine, and by this >>>>> time, we have an app engine version in internal testing. However, we >>>>> have a few concerns: >>>>> >>>>> First, the information this gentleman expresses: >>>>> http://aralbalkan.com/1504 >>>>> >>>>> Here is a quote - "You build an awesome new app on Google App Engine. >>>>> You tell your friends. They tell 1,000 of their friends on Twitter who >>>>> tell 1,000 of their friends and then, suddenly, you have all these >>>>> developers hitting Google App Engine for the first time to see your >>>>> app. Paradoxically, by doing that, they trigger the "intelligent >>>>> throttling" "feature" in Google App Engine which freaks out and shuts >>>>> down your app with an "Over Quota" error -- effectively making the >>>>> "Over Quota" message the first impression most of your audience has of >>>>> Google App Engine." >>>>> >>>>> To what extent is this true? As paying customers with a sizable budget >>>>> (We're paying over $35,000 a month on our current host, bandwidth >>>>> exclusive), this is completely unacceptable. >>>>> >>>> >>>> There's no "intelligent throttling". Every app has quotas, determined by >>>> the amount of budget you allocate to each resource (or a fixed amount, in >>>> the case of free apps), and the quotas are accounted for both on a daily >>>> basis and in smaller bins. In other words, your traffic can vary, but we >>>> won't let your app consume its entire budget before we start serving over >>>> quota errors. I hope you can see why this is: we assume you don't want a >>>> single spike of traffic, for example because your app is being abused, to >>>> be >>>> able to consume your entire quota for the day, leaving your app serving >>>> 'out >>>> of quota' errors for the rest of the day! >>>> >>>> >>>>> I assumed that this might be true only for free accounts, but then one >>>>> of our engineers found this -> >>>>> >>>>> http://groups.google.com/group/google-appengine/browse_thread/thread/32876e345d075878/e2ec2e4fdd7ca3e0 >>>>> >>>>> This is a paying customer begging for your mercy to scale. He claims >>>>> he gets a Out of Quota error in 1 of 6 requests. This shows he isn't >>>>> really out o his daily Quota, but there is something else that's going >>>>> wrong. >>>>> >>>> >>>> As you can see from the thread, the user was only experiencing issues >>>> because some of their requests were taking too long to complete. We impose >>>> limits on how many instances of an app we will schedule if the app takes >>>> too >>>> long to respond, because slow apps don't scale very well and would take >>>> disproportionate amounts of resources. >>>> >>>> >>>> -Nick Johnson >>>> >>>> >>>>> It would be great i somebody from Google can comment on the validity o >>>>> the above concerns. We don't want to shift our entire business to app >>>>> engine and have our customers watch the "Out of Quota - App engine >>>>> errors". Frankly, this would be like BSOD of the web 2.0 world. :) >>>> >>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To post to this group, send email to [email protected] >>>>> . >>>>> To unsubscribe from this group, send email to >>>>> [email protected]<google-appengine%[email protected]> >>>>> . >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>> >>>>> >>>> >>>> >>>> -- >>>> Nick Johnson, Developer Programs Engineer, App Engine Google Ireland >>>> Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 >>>> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration >>>> Number: 368047 >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]<google-appengine%[email protected]> >>>> . >>>> For more options, visit this group at >>>> http://groups.google.com/group/google-appengine?hl=en. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<google-appengine%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> >> >> -- >> Ikai Lan >> Developer Programs Engineer, Google App Engine >> Blog: http://googleappengine.blogspot.com >> Twitter: http://twitter.com/app_engine >> Reddit: http://www.reddit.com/r/appengine >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<google-appengine%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
