How many is "a bunch"? Also, You say "they are all very much parallelized" but then you say that you've scheduled them 10 minutes apart and they don't overlap.. those two statements are contradictory, please explain more clearly your cron-taskqueu setup and how it works and what exactly it is doing.
When you say that the cron jobs "spawn tasks that write to one entity each".. what do you mean? The cron job is there to fire of the initial task.. and that task runs once, putting one entity and that's it? If so, why are you having these tasks only put one entity at a time.. instead of creating multiple entities and putting them in batches? Does each task put() new entities? or are they sometimes putting an entity that may already exist? More info is more better for help. On Fri, Feb 26, 2010 at 3:35 PM, Marc Provost <[email protected]> wrote: > Ok, here's my situation: > > * I use the java implementation and my app id is poolfana. > * I have a bunch of cron jobs scheduled at night (Eastern Time) > * They are all very much parallelized. I am being very strict: they > spawn tasks that only write to one entity each. Each tasks will > execute in a few hundred ms. > * A given cron job and its spawned tasks will terminate in a few > minutes at most. > * I have scheduled each cron job at least 10 minutes apart, so they do > not overlap. > * In my dashboard, my max request per second is 3. The max limit is > supposed to be 30. > * My problem? The cron jobs fail sporadically (marked as "failed" in > the dashboard) with this error: > > "Request was aborted after waiting too long to attempt to service your > request. Most likely, this indicates that you have reached your > simultaneous dynamic request limit. This is almost always due to > excessively high latency in your app. Please see > http://code.google.com/appengine/docs/quotas.html for more details." > > There is an issue for this problem: > http://code.google.com/p/googleappengine/issues/detail?id=2396 > > It was starred 50+ times, but it was not acknowledged yet by the > google team. I'm writing this post to discuss potential workarounds, > potential misuses of the API with the google team or other people that > might have solved this problem. What else can I do? Is it a problem on > the google side or I'm I doing something wrong? Right now, I need to > re-execute the cron jobs manually everyday... > > Thank you! > Marc > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
