Have you weighed Compression vs uncompressed in terms of speed, and cost? I'll end up testing to see, but I like to know the answers beforehand :-)
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of bFlood Sent: Tuesday, January 17, 2012 4:54 AM To: Google App Engine Subject: [google-appengine] Re: Do you Serialize your Data? hi brandon I do this with a bunch of models, keep a few properties for search and then dump the bulk of data into a compressed, json text property. Obviously the json part might be overkill but its a lot more flexible going forward if (err, when) you need to add more data (especially nested data) reads and writes are faster, YMMV YMMV b On Jan 17, 4:44 am, "Brandon Wirtz" <[email protected]> wrote: > We are looking at restructuring some of our calls, optimizing again > for speed, we are thinking that rather than having a datastore call > for a single cell, we'd serialize things. > > This makes searching suck, and we have had a few issues where > serialized and unserialized data didn't quite match when we were done > (because of weird encoded characters from sites) > > Serializing also locks you in to a single data structure for the most part. > > We are also just considering: > data as a zipped, delimited array > uncompressed, delimited > Double storing, so that we pay twice for writes, and can run > analytics, but so that reads are against the serialized data, and faster and cost less. > > What are anyone's thoughts? Have you had these discussions with your > dev team? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
