> The datastore is > designed to maintain its performance characteristics over very large data > sets and heavy traffic, so performance with small data sets and low traffic > may not always compare to a single-server SQL database under similar > conditions. in the same sense that it won't compare to a RAM cache of the > same data.
I not expecting it to work the same as sql. But I do expect to build a list of a couple hundred records. I'm not crazy this is a totally reasonable thing for an app to do. Sorry guys the serial return speed of these entity loaded queries is an issue. It's not me it's you. And it's just a couple changes away from working. Can't change it myself so I'm at your mercy google. Can we allow your libraries to spawn threads? SimpleDB solves this by allowing you to bang it with many threads at once. Even though the fetch is slow for the entities since you can do so many in parallel it works out. That way you could: Keys=QueryObject.getKeys() NeededEntities=[] for Key in Keys: #if key is not in memcache add to NeededEntities DictonaryOfReturnedEntities=Model.fetchMassEntities (NeededEntities,Threads=30) Now I want .fetchMassEntities(NeededEntities,Threads=30) to kick 30 threads off and get my entities. Which I will cache. This should be totally workable for lists up to 1,000 I would think. It's not a sql vs BigTable think it's a architecture issue with the current setup. > Part of scaling data is > determining how quickly updates need to be available, so you can > pre-calculate and cache results to optimize queries, i.e. make data that > doesn't need to be instantly dynamic less dynamic. If I make a degenerate table model just the info I want to query and the key stored is it not just the same number of entity fetches right? I am I getting punished by the number or returned entities? or the number of properties returned? > Incidentally, returning just keys for a query has been on our radar for a > while, we just haven't gotten to it. I can't promise anything, but it's on > our list. Feel free to file a feature request in the issue tracker to > promote it. Will do. Thanks for your time Dan. :) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
