I opted for cpickle. My Slowness the first pass was that I was testing with 250k binaries and Pickle escapes them... So they were very large and didn't actually fit (I didn't check they matched going in and coming out)
Once I applied ZIP to the Pickle it appears to be fast. Which brings me to the next question... I use datastore like many people would use Memcache and store with date. Does anyone want to place bets on how much faster a 500 byte file reads than a 100k file? I'm thinking about storing the last modified date and checking it, before I read the actual data. But I need to bench the difference in speed an calculate for the cache miss ratio. But it would appear that with ASync Writes, Async reads for Initialization, and serialization my average latency is going to go from 600 MS to 185ms and knock another 50-60% off of my hosting bill. That will put me to less than the price I was at in beta. (in terms of price per traffic served) (and google bot is still the most expensive part of that) -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Andreas Sent: Wednesday, January 18, 2012 6:36 PM To: [email protected] Subject: Re: [google-appengine] Re: Do you Serialize your Data? i started pre-serializing my objects with an objectproperty which writes to the blobstore on put . while serializing 50 entities with around 10 properties each on request took around 5-6 seconds as an average, now i cut the time in half at least. On Jan 18, 2012, at 8:19 PM, Mahron wrote: > I use my own serialization to communicate with a flash app and also > for the datastore. > > It is binary and gziped and allows for nested items and binary > properties. > > The compression, decompression, serialize and deserialize time is > negligible compared to the datastore request time and sending back the > data. > > Btw, you seem to be the smart guy around here, > > What happens under the hood when list properties are modified, index > and composite index wise ? > Does it modify added and removed values only ? > > -- > You received this message because you are subscribed to the Google Groups "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to [email protected]. > For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
