Hi Ikai - the behavior we are seeing suggests the "offline" tasks are subject to the same 1000msec rule as external requests.
Queuing up a number of tasks reliably results in the "Request was aborted after waiting too long to attempt to service your request" error - which is actually fine, BUT, the appengine kicks in the back- off algorithm. This results in tasks that cycle for 20+ generations, with mean time between run attempts of 19hr+. How do we know the 1000 msec rule is in effect? The situation improves drastically if we introduce a large number of "no-op" tasks that complete in ~40 msec and skew the averages. J On Sep 17, 2:05 am, "Ikai Lan (Google)" <[email protected]> wrote: > Jason, I think your situation is fine. Offline tasks have the property that, > unlike user-facing tasks, do not require instant execution. If you schedule > an offline task for "now", that actually means "when there's capacity" and > App Engine can allocate idle capacity to process your request. Thus, the > need to spin up additional instances is unnecessary in most cases. Are you > seeing that your tasks are backed up? > > On Thu, Sep 16, 2010 at 12:56 PM, bFlood <[email protected]> wrote: > > "which in turn affects the capacity available for running offline > > tasks" - so, if you have a low volume site, you won't get that many > > instances for your tasks? likewise, if you have some user facing > > requests that go longer then 1000ms (by design or otherwise), the > > instances available for your tasks are impacted? or am I confused? > > > On Sep 16, 8:44 am, "Nick Johnson (Google)" <[email protected]> > > wrote: > > > Hi Jason, > > > > The same appservers are used to serve user-facing and offline traffic. > > The > > > volume of user-facing traffic (that is below the latency threshold) you > > > serve determines how many appservers we provision for your application, > > > which in turn affects the capacity available for running offline (task > > queue > > > and cron) tasks. > > > > -Nick Johnson > > > > On Thu, Sep 16, 2010 at 1:41 PM, Jason C <[email protected]> > > wrote: > > > > The number of instances that App Engine makes available to your > > > > application depends on if you keep your average request time under > > > > 1000ms for user-facing requests. > > > > > Ikai Lan (I believe) said that taskqueue and cron job requests do not > > > > count against this boundary. Ikai also said that this boundary was in > > > > place because longer requests were bad for the ecosystem. > > > > > Since taskqueue and cron job requests do not count against this > > > > boundary, in order for them to not be bad for the ecosystem, I'm > > > > guessing that they are served from a different set of servers than > > > > user-facing requests are. > > > > > We (appid: steprep) have a number of external machines that also hit > > > > our urls. While we make every effort to keep user-facing requests > > > > quick and responsive, we often use many seconds serving the requests > > > > that are built for external machines (by design). > > > > > It has only just struck me this morning that this could be having a > > > > bad (perhaps dramatic) impact on our overall scaleability. > > > > > First off, is it true that cron and taskqueue items are served on a > > > > different set of servers? If so, is there any way to designate that a > > > > particular url is being requested by a machine and can be routed to > > > > this alternate set (of presumably slower) servers (e.g., a request > > > > header)? > > > > > If I'm way off on all of this, and if taskqueue and cron jobs are > > > > served from the same set of servers, I'm not sure how the "bad for the > > > > ecosystem" argument holds, and perhaps Google should revisit this > > > > 1000ms boundary condition altogether. > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to [email protected] > > . > > > > To unsubscribe from this group, send email to > > > > [email protected]<google-appengine%[email protected]><google-appengine%2Bunsubscrib > > [email protected]> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. > > :: > > > Registered in Dublin, Ireland, Registration Number: 368047 > > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > > Number: > > > 368047 > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
