Hi Guys, My app has taken off recently, and it's been very exciting to see so much traffic. I am trying now to deal with a very high volume of infrequently accessed /archival data store entries and making sure i'm not going to hit a performance / cost bottleneck.
Let's say I have a store of entities without any relationships that grows by 100,000 a day. The entries are accessed by a query for three properties (a foreign key and timestamps that are between two provided dates), usually returning several thousand values at a time. Any advice for the best way to store these values? I have been considering splitting the data i'm storing into tenants, using a multitenant architecture, using a background task to compress chunks of data into blobs and putting them in the store, and so on, but i'm wondering if the values are indexed is it smarter to just leave them alone and let the store grow to billions of entries. Do queries slow down by N as the datastore / index grows? Thoughts? Is there a performance benefit with a multi-tenant architecture? Ben My app is an online data historian called Nimbits www.nimbits.com - it's free and open source on google code and allows developers to feed sensor data into data points to do calculations, alerts, relay etc - Internet of things sort of thing. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
