It was a lot of big entities. The exception said it was the size, not the quantity.
On Oct 30, 2010, at 9:51 AM, Jeff Schwartz wrote: > How many entities were there when the batch put failed? > > Was it the size of the entities or the number of entities that caused the > batch put to fail? > > Jeff > > On Sat, Oct 30, 2010 at 8:39 AM, Stephen <[email protected]> wrote: > > > On Oct 29, 6:24 pm, Joshua Smith <[email protected]> wrote: > > I'm running into a too-large exception when I bulk put a bunch of entities. > > So obviously, I need to break up my puts into batches. I want to do > > something like this pseudo code: > > > > size = 0 > > for o in objects: > > if size + o.size() > 1MB: > > db.put(list) > > size = 0 > > list = [] > > list.append(o) > > > > Any idea what I could use for the "o.size()" method? I could crawl through > > all the fields and build up an estimate, but it seems likely to me that > > there is a way to get the API-size of an entity more elegantly. > > > How about something like: > > > from google.appengine.api import datastore > from google.appengine.runtime import apiproxy_errors > > def put_all(entities, **kw): > try: > return datastore.Put(entities, **kw) > except apiproxy_errors.RequestTooLargeError: > n = len(entities) / 2 > a, b = entities[:n], entities[n:] > return put_all(a, **kw).extend(put_all(b, **kw)) > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
