On Mon, Nov 18, 2013 at 11:48 AM, Ronoaldo Pereira <[email protected]> wrote:
> While working on a migration tool from Blobstore to Google Cloud Storage, > I was surprised by a series of consistent, frequent InternalError while > fetching from Blobstore. The errors happen after a few attempts to read > from particular byte ranges, but I'm not sure if they are instead a limit > imposed internally to avoid your app to consume resources too fast. I was > unable to find anything from any documentation page describing any limits > in that regard. > > The code fetching blobs is the following, failing at some particular byte > ranges, repeatedly (when task retry happen; eventually it works, but > sometimes it don't): > Length here is about 512 * 1024, to stay lower than the max download size > from blobstore. Strange fact is: if I download the same blob, reupload to > another appid, then the code works without any issues. > > Anyone had similar erros in the past? Could this be a problem with the > blobstore of the production app? > > The Blobstore has always been a bit "weird" when it comes to anything other than uploading/downloading files (i.e. reading and writing from the App Engine application). Instead of using the *fetch_data *command, can you try setting up a urlfetch that uses Blobstore's standard blob byte range serving? See https://developers.google.com/appengine/docs/python/blobstore/#Python_Serving_a_bloband scroll down to "*Blob byte ranges.*" You'd basically be downloading the blob from yourself, but it's a better (and more tested!) way to access your blobstore. ----------------- -Vinny P Technology & Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
