On Wednesday, September 11, 2013 3:48:12 PM UTC+1, Vinny P wrote: > > You were able to write a single character without a problem, but 1 MB of > characters was too much for it. So I were you, I'd try to find that > inflection point and then save to Cloud Storage in chunks less than that. > For example, try to write to Cloud Storage in chunks of 250 KB, if that > works, try writing in chunks of 400 KB, and so on. Of course this is only a > bandaid solution - something is clearly wrong with the underlying urlfetch > implementation. > >
I'm not too convinced - I wouldn't expect it to be so binary. Writes of 1MB aren't *always* failing so if there is some size-dependant problem, I would expect smaller writes to continue to fail but with a lower probability (this might be fine if a failure didn't cause the instance to hang indefinitely, as you could then retry!) On Wednesday, September 11, 2013 3:48:12 PM UTC+1, Vinny P wrote: > > But if I had to make a recommendation, I'd say skip the whole backend > writing thing altogether. It's a waste of money to load up backends for the > purposes of writing (unless you're Bill Gates or similar!). You said > frontend instances seem to be much better in terms of writing - try using > task queues to spawn off frontend requests which are solely dedicated to > writing to Cloud Storage. If you need a temporary place to store data, > write it to the datastore or Cloud SQL until the tasks can get to the GCS > writing. > Well yes, clearly my actual requirement isn't just to store some number of 'f's in GCS, the backends are required to do some amount of processing before writing results. Thanks for the suggestion though - that's definitely something I can explore if there is no actual fix for this issue. Cheers, Ben -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
