Thanks Andrin! If anyone is wondering why this is, it's because the List property is serialized and deserialized using protocol buffers. Protocol buffers are very useful for things like validation and cross language compatibility, but you can sometimes gain some performance benefits by serializing and deserializing using native methods. The protocol buffer implementation for Python is noticeably slow, even via the native extension (if you can improve this - Google wants to hire you), so I'm not surprised techniques like this work.
-- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Tue, Aug 2, 2011 at 9:18 AM, Andrin von Rechenberg <[email protected]>wrote: > So you want to store a "very long list of numbers" in your AppEngine Model > which doesn't need to be indexed. > > Well, you could just do this: > > *class Foo(db.Model): > numbers = db.ListProperty(int, indexed=False) > * > However, you will quickly notice that the performance and the memory > consumption of this sucks if you add thousands of integers to this list. > > Instead you should use the native python "array.array" type as the list and > store it in a BlobProperty. It's up to 30x faster! > > I created a library that does exactly this (free, apache 2.0 license): > > http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html > > Once you have the library, all you need to do is this to get a huge > performance boost: > > *class Foo(db.Model): > numbers = db.ArrayProperty()* > > The blogpost also has more details and an appstat comparison. > > Cheers & hope you like it > -Andrin > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
