> We use that some, but that doesn't help with things that take a long time.
You just probably need to shard it in a different way, so that each chunk requires less time to get processed, i.e. within the limits. Some other times, for simpler tasks, we employ a little trick by enqueueing a simple task which would process just the right amount if data and then re-enqueue itself with a pointer of some kind (often a datastore cursor) advanced to the next chunk. > And there are weeks when backends will work as advertised. Then they won't. You'd have to back that up with more than just words and links to SO/issue tracker with missing config params or "more info needed" label :) On Fri, Nov 23, 2012 at 10:04 AM, Brandon Wirtz <[email protected]> wrote: >>There's also Mapreduce/Pipeline - you might wanna check that if backend > isn't solving your problem. > We use that some, but that doesn't help with things that take a long time. > If I manage them rather than choosing dynamic we can spin up and down > backends, but if they are user initiated that is a lot harder. > And there are weeks when backends will work as advertised. Then they won't. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
