"So, I am considering serialising each dictionary entry and storing them as Blobs (or Texts)"
Going to go out on a limb here ... what about storing the entire dictionary as a single Blob? Or at least the index as a single Blob? You read the index into local instance memory and lazily load the extra data as needed. This makes each lookup take: 1a. (Best case scenario) Local memory index traversal - cost is time in instance hours (very fast), log(8000) 1b. (Worst case scenario) Load the entire index from a single datastore Blob into local instance memory and then traverse (cost of 1a + building in-memory data structure and cost of reading a single entity) 2. Read the necessary entities from the datastore With only 8000 entries, you should be able to keep most indexes in memory, even within the 128mb limit (I could be wrong). Can you describe some of the searches you need to do in a bit more detail? -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com On Fri, Jan 27, 2012 at 9:28 AM, Magnus Gunnarsson < [email protected]> wrote: > I have been experimenting a bit with the Datastore and it seems to be a > good thing with many possibilities. However, I am uncertain about how it is > intended to be used, what the typical design patterns should look like. I > am used to relational databases and know their strengths and weaknesses, > and I design my applications accordingliy. I suspect that is somewhat > inadequate when using GAE. > > I have a sign language dictionary (8000 entries) where I fetch data from > an external server, store it in GAE and present it to the user (in a > hopefully appealing way) using GWT. My current design (which is not yet > fully implemented) is to store each dictionary entry as an Entity in the > Datastore, with most of the information pieces for the entry as unindexed > properties of the Entity. The searches in the dictionary are sometimes > quite complex, and the search capabilities in Datastore are not > sophisticated enough. Thus I create a small number of indexes and put these > in MemCache. > > Now, it seems that the application hit the quota limitations quite easily. > Just loading the dictionary from the external server seems to be impossible > with out hitting the Datastore write limit. (For the free version, of > course. I have no money.) > > So, I am considering serialising each dictionary entry and storing them as > Blobs (or Texts), individually. Another option is to serialise the entire > dictionary and store it in a Blob. (I do that, in practice, in the old > version of the dictionary application, written in PHP.) > > Any advice? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/KujFnRst_0cJ. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
