Ikai, Do you have a definitive answer on whether or not task/cron requests count towards the 1000ms threshold? There seems to be some confusion and counter-evidence here.
Including our cron/task requests, we run at 1500-2000ms / request. This is largely because we have LOTS of taskqueue items and we tend to do a fair amount of work in them. Further, when we do large spike jobs (e.g., mapreduce), we see lots of deadline-related errors. What is the best way to know if we're above or below this threshold? (appid: steprep) j On Sep 16, 7:41 pm, "Jan Z/ Hapara" <[email protected]> wrote: > Hi Ikai - the behavior we are seeing suggests the "offline" tasks are > subject to the same 1000msec rule as external requests. > > Queuing up a number of tasks reliably results in the "Request was > aborted after waiting too long to attempt to service your request" > error - which is actually fine, BUT, the appengine kicks in the back- > off algorithm. > > This results in tasks that cycle for 20+ generations, with mean time > between run attempts of 19hr+. > > How do we know the 1000 msec rule is in effect? > > The situation improves drastically if we introduce a large number of > "no-op" tasks that complete in ~40 msec and skew the averages. > > J > > On Sep 17, 2:05 am, "Ikai Lan (Google)" <[email protected]> > wrote: > > > > > Jason, I think your situation is fine. Offline tasks have the property that, > > unlike user-facing tasks, do not require instant execution. If you schedule > > an offline task for "now", that actually means "when there's capacity" and > > App Engine can allocate idle capacity to process your request. Thus, the > > need to spin up additional instances is unnecessary in most cases. Are you > > seeing that your tasks are backed up? > > > On Thu, Sep 16, 2010 at 12:56 PM, bFlood <[email protected]> wrote: > > > "which in turn affects the capacity available for running offline > > > tasks" - so, if you have a low volume site, you won't get that many > > > instances for your tasks? likewise, if you have some user facing > > > requests that go longer then 1000ms (by design or otherwise), the > > > instances available for your tasks are impacted? or am I confused? > > > > On Sep 16, 8:44 am, "Nick Johnson (Google)" <[email protected]> > > > wrote: > > > > Hi Jason, > > > > > The same appservers are used to serve user-facing and offline traffic. > > > The > > > > volume of user-facing traffic (that is below the latency threshold) you > > > > serve determines how many appservers we provision for your application, > > > > which in turn affects the capacity available for running offline (task > > > queue > > > > and cron) tasks. > > > > > -Nick Johnson > > > > > On Thu, Sep 16, 2010 at 1:41 PM, Jason C <[email protected]> > > > wrote: > > > > > The number of instances that App Engine makes available to your > > > > > application depends on if you keep your average request time under > > > > > 1000ms for user-facing requests. > > > > > > Ikai Lan (I believe) said that taskqueue and cron job requests do not > > > > > count against this boundary. Ikai also said that this boundary was in > > > > > place because longer requests were bad for the ecosystem. > > > > > > Since taskqueue and cron job requests do not count against this > > > > > boundary, in order for them to not be bad for the ecosystem, I'm > > > > > guessing that they are served from a different set of servers than > > > > > user-facing requests are. > > > > > > We (appid: steprep) have a number of external machines that also hit > > > > > our urls. While we make every effort to keep user-facing requests > > > > > quick and responsive, we often use many seconds serving the requests > > > > > that are built for external machines (by design). > > > > > > It has only just struck me this morning that this could be having a > > > > > bad (perhaps dramatic) impact on our overall scaleability. > > > > > > First off, is it true that cron and taskqueue items are served on a > > > > > different set of servers? If so, is there any way to designate that a > > > > > particular url is being requested by a machine and can be routed to > > > > > this alternate set (of presumably slower) servers (e.g., a request > > > > > header)? > > > > > > If I'm way off on all of this, and if taskqueue and cron jobs are > > > > > served from the same set of servers, I'm not sure how the "bad for the > > > > > ecosystem" argument holds, and perhaps Google should revisit this > > > > > 1000ms boundary condition altogether. > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > Groups > > > > > "Google App Engine" group. > > > > > To post to this group, send email to [email protected] > > > . > > > > > To unsubscribe from this group, send email to > > > > > [email protected]<google-appengine%2Bunsubscrib > > > > > [email protected]><google-appengine%2Bunsubscrib > > > [email protected]> > > > > > . > > > > > For more options, visit this group at > > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland > > > > Ltd. > > > :: > > > > Registered in Dublin, Ireland, Registration Number: 368047 > > > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > > > Number: > > > > 368047 > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Google App Engine" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > > [email protected]<google-appengine%2Bunsubscrib > > > [email protected]> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
