I have spent a bunch of time (with support) trying to get to the bottom of this sort of thing.
There are 2 major contributors that I've discovered: (1) *Many namespaces.* If you have many namespaces, then Datastore Statistics are no longer computed. You can tell by going to Datastore Statistics. If it says "Last updated: [long time ago]", then you are in this situation. When this happens, the Map-Reduce framework uses only 32 shards, which can greatly extend the time to backup. (2) *General Files API failures.* If the Files API is having difficulty, then the worker shard can fail. If it fails 10 times (I believe), then it simply dies. The M-R jobs never completes and the controllers just keep on retrying. This situation can also occur if you hit the daily limit for Files API transfer (100 GB or so, I think). This is a pain and has caused me to continually manually clean queues, and of course, there are Kinds that we simply can't get backed up. Hopefully this all gets better under the new GCS client.... j -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
