Thanks Stephen and Kevin, My original plan was to do what Stephen suggests, above, but the serialization costs are non-trivial. There's a lot more about that in Brett Slatkin's Building Scalable, Complex app... talk from google i/o 2009. In the examples that he uses, there is a 10x improvement when only keys are fetched, rather that the whole list.
The many uses for key_names that Kevin suggests are really intriguing. I haven't used key_names much, so far, but they look like a really promising avenue for optimization. Many thanks for all your help. - Shailen Tuli On Oct 9, 8:38 am, Kevin Pierce <[email protected]> wrote: > That is perfectly valid too.The tradeoff is that when ever a url entity is > returned the list is serialized. > If you know you want the tag list most of the time then place it with the > url. > However if you are only interested in using the list for lookups, then you > can place it as an IndexModel in the Url entity group. > > Even more food for thought.... > If your url is not very long... > Then you can use it as the key_name for the Url/Tag entity. > Then you can do a __keys_only__ query > > A) > class URL(db.Model): > url = db.StringProperty() > tags = db.StringListProperty() > # psuedo code > key_name = url > > B) > class URL(db.Model): > url = db.StringProperty() > # psuedo code ... > key_name = url > > class Tags(db.Model): > tags = db.StringListProperty() > # psuedo code ... > key_name = url > > a query against A for __keys_only__ will return faster than returning the > entity. and your fetch density will be higher. > urlkey.name() will give you the url without making datastore retrieve the > entity. > > B wouldn't require Tags to be in the URL entity group, unless you really > want transaction capabilities on that. > They can simply have the same key_name. thus creating a 1 to 1 relationship > between URL and Tags. > > if URL has a bunch more properties this would make more sense. however if > the 2 fields you have mentioned are the only ones and your URLs are not > super long.. I would use the key_name trick. This also means you can do > db.get(key) for retrieving or testing url existence. This is faster then a > query for 2 reasons. > > 1. Query for an entity is 2 datastore calls under the hood (1st to setup the > query, 2nd to fetch results) and requires 2 operations inside datastore (1st > index scan for keys, 2nd locate and retrieve entities) > > 2. db.get for an entity is 1 datastore call and no index lookups. > > So if you want your datastore access to fly think about leveraging the > key_name as a useful pkey. > > Cheers, > Kevin > > > > On Fri, Oct 9, 2009 at 8:51 AM, Stephen <[email protected]> wrote: > > > On Oct 8, 9:00 pm, Shailen <[email protected]> wrote: > > > > Option 2, use list properties: > > > > class URL(db.Model): > > > url = db.StringProperty() > > > > class Tags(db.Model): > > > tags = db.StringListProperty() > > > > Store a list of tags with a URL key as parent. > > > Why not simply add the tags to the URL entity? > > > class URL(db.Model): > > url = db.StringProprety() > > tags = db.StringListProperty() > > > �...@classmethod > > def fetch_by_tag(cls, *tags): > > query = URL.all() > > for t in tags: > > query.filter('tags =', t) > > return query.fetch() > > > urls = URL.fetch_by_tag('foo', 'bar', 'baz') > > -- > Kevin Pierce > Software Architect > VendAsta Technologies Inc. > [email protected] > (306)955.5512 ext 103www.vendasta.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
