Hi repr won't be very accurate. In many cases you will just get <MyModel hashvalue> which is no where near the size of the entities.
An expensive will be to serialize the entity with proto buffer as thats what will happen to them when they are put into the datastore, this approach is also redundant as it means the system will be doing it twice. I suggest a better strategy might be to have a look at the statistics in your control and and set a value based on the average entity size. This might need reviewing on a regular basis, but it will be a lot quicker. Then put batch sizes in a config file for each entity type. Rgds Tim On Oct 30, 1:53 am, Rafael Sierra <[email protected]> wrote: > On Fri, Oct 29, 2010 at 3:24 PM, Joshua Smith <[email protected]> > wrote: > > I'm running into a too-large exception when I bulk put a bunch of entities. > > So obviously, I need to break up my puts into batches. I want to do > > something like this pseudo code: > > > size = 0 > > for o in objects: > > if size + o.size() > 1MB: > > db.put(list) > > size = 0 > > list = [] > > list.append(o) > > If you are talking about db entities, you can try repr(o), it's used > to send objects to cache, I saw somewhere in the documentation: > > `obj = eval(repr(obj))` > > So, you can do "if size + len(repr(o)) > 1MB:" (pseudo code) > > > > > Any idea what I could use for the "o.size()" method? I could crawl through > > all the fields and build up an estimate, but it seems likely to me that > > there is a way to get the API-size of an entity more elegantly. > > > Thoughts? > > > -Joshua > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > Rafael Sierrahttp://blog.rafaelsdm.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
