+1 However please include sub-second tasks Just today I was looking at my logs/appstats. A client "new recod" write function I have that consists of three separate new kinds being put. It seems to run consistently at 250-300ms per HR put(). These occur serially: first one in my on-line handler, second in a high-rate/ high-token task queue, third in a low-rate/low-token queue. It is fine if the second and third puts occur minutes after the first. Seems much better than a 750 ms on-line handler function.
Looking at my logs, nearly every write I do for indexed kinds is in this ballpark for latency. Only one on-line handler task is up around 500 ms because I have to do two puts in it. Everything else is 300 ms or less. So I am very happy with this setup. The recent thread where Brandon/John analyzed high instance rates shows what might happen if average latency viewed by the scheduler is skewed by a few very high latency functions. (Fortunately for my read/query/write client needs, I can avoid big OLH functions, but it is a serious design challenge.) However, the downside right now is that I do not know how the Task Queue scheduler interacts with the Instance Scheduler. My imagined ideal would be for developers to eventually be able to specify separate TQ instances (I believe Robert K. asked for this when he suggested TQ calls could be made to a separate version.) The Scheduler for these separate TQ instances would need to analyze cumulative pending queue tasks (I think the current TQ Scheduler does some of this), and only spawns new instances when the cumulative total exceeded a developer set value -- which would allow minute values rather than seconds. thanks, stevep On Sep 10, 6:03 am, John <[email protected]> wrote: > I'd like to know what is the impact of tasks on the scheduler. > > Obviously tasks have very high latency (up to 10 minutes, but not using much > cpu - mostly I/O). What is their impact on the scheduler if any ? > Would be nice to have some sample use cases on how the scheduler is supposed > to react. For example if I have 1 task which takes 1 minute, spawn every 1s, > vs every 10s, vs 1 min ? > > Since the tasks use very low cpu, technically an instance could easily run > 60 of them concurrently so 1 qps with 1-min tasks could take only one > instance. But I doubt the scheduler would spawn only one instance. > > App Engine team, any insights ? > > Thanks -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
