On Wed, Aug 1, 2012 at 2:36 AM, Richard Watson <[email protected]> wrote: > > You could also scale the amount in the list depending on how many > requests per second you're getting. If 1000, batch like mad. If < > 20, submit every request. The risk with this is that you get 1000's, > then instantly zero. But you then likely have another problem. > Hopefully workable?
Anything that involves batching in a frontend risks orphaning data in the frontend... there's just no efective way to ensure that batching happens and that the queue is purged when "done". > I thought GAE can also do 1000 puts per second if you have a lot of > instances? I'm kinda assuming Richard's trying to push lots through > relatively few instances so one pause in the list affects many values > being processed. Richard, could you clarify? That's the solution that Richard is using right now. The problem is that the reaping process is a simple query; because of eventual consistency, you can't guarantee that the collator will get all the scores. A burp in the datastore and you get nothing. > One other question, since I've never used a backend. We're able to > address them directly, yes? Can't he just do an HTTP POST to the > backend directly rather than a put()? See my thread about performance of backends: Horrible. They top out at 80qps doing no-ops, and even minor amounts of work cut that to 20. Plus there's a mysterious 200ms (!!) added for every frontend-to-backend call. I'm having a genuinely hard time finding any application for which backends are actually suitable. I really don't see any problem with running this service in another cloud provider. It's slightly more complicated but it gives you "the right tool for the job". Use another PaaS provider like Heroku to keep the fuss minimal. Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
