Hey Steve, First, "[t]hink of your client's put() as your customer's head, and the 503 as the brick wall" is great, it is humorous and accurate analogy. I can relate. And, I also deal with low-write-rate but extremely high-value writes in some apps. It is tricky and makes small failures / blips a much more significant issue.
In any client-server app, there will occasionally be communication / server-side issues. Have you thought about storing data locally on submit (or only if there is a server-side error) then implementing some type of start-up recovery logic that will retry the save? Could be particularly helpful if you have some way of making the write idempotent (ie using a key_name). I know it is not perfect but it could help out a bit. There are some little tools that might help with this, like lawnchair (http://westcoastlogic.com/lawnchair/). If you couple that with saving via an AJAX call, even in an error situation the user experience would be at least a little better. Another question, how far over the 10K limit are you? Have you tried using bzip to compress your data? Perhaps if you compress the payload it would be possible to enqueue it in a task? Robert On Sat, Feb 12, 2011 at 10:40, stevep <[email protected]> wrote: > Thanks Calvin and Robert (as always). > > Calvin: I initially did my puts() in the handler. Easy and clean as > you suggest until your put that normally takes 200ms is hit by a > series of 503s because of infrastructure issues Google controls (not > you). > > Think of your client's put() as your customer's head, and the 503 as > the brick wall. You can beat the head against the wall handling all > the various scenarios via dialogs. That's an option yes, but my > customers are not technically inclined, and will be unhappy. Also, you > **cannot** assume this write will happen. Very, very likely the > frustrated customer closes her browser window. Then you're SOL. > > My option using the task queue was to simply send the put there, and > give the client the means to verify its occurrence. If the > infrastructure is thrashing away with 503s, then the client can easily > determine that the customer needs a nice dialog saying she will be > apprised via email when the record is written. > > I'm happy: record is not lost. Customer is happy: she has an > assurance, and is free to get on with her life. > > Conclusion: it is becoming apparent to me that GAE cannot provide the > means for a developer to ensure a record is written. To minimize risk, > I am now setting up a high replication handler writing to db class > that has the least possible number of indices. AND, I get to set up > all those client dialogs. > > So, so much easier if there would be one task queue were I could send > the transaction data from my client's post, and take advantage of all > the benefits of the TQ. > > Sorry for the rant-like tone -- too little sleep last few days. > stevep > > Robert: >> Instead could catch timeout, contention, or similar exceptions and >> retry the put. You might also wan to set an limit on the RPC timeout >> so that you'll have time to retry the write at least once within the >> 30 second request limit. If you still get an error after some number >> of retries, notify the client. > > > Calvin: >> What I'm not understanding from the original question is why can't you do >> the put in the request handler? That way if there is a failure (and there >> will always be occasional failures) you can catch the exception and return >> an error to the client. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
