Thanks guys for all the responses.
I checked out Brett's presentation and he talks about this exact issue
on how to optimize using list properties
So following the Brett's presentation, create 2 entitiy kinds.
Page and PageIndex where Page is the parent of PageIndex (specify
parent in PageIndex constructor)
class Page(db.Model):
pagekey= db.StringProperty(required=True)
class PageIndex(db.Model):
usernames = db.StringListProperty(required=True)
indexes = db.GqlQuery("SELECT __key__ FROM MessageIndex "
"WHERE usernames = :1", username)
keys = [k.parent() for k in indexes]
pages = db.get(keys)
Since we are querying by key, it is 10x faster, and no unnecessary
serialization.
We can fan out by adding multiple PageIndex, if reaches 5000 max.
Is this about right?
Thanks
Jon
On Aug 31, 9:34 am, Jeff Schwartz <[email protected]> wrote:
> A list property's size is limited to 5000.
>
> If you only want to know if a user has visited a page, you can use the first
> model & query it returning only keys to avoid list serialization issues. In
> addition, key only queries are very fast.
>
> Additionally, if your Page model's key had a string name that was the
> pagecode then you could even use the key to identify the page. In other
> words, you'd be materializing the view in the key of the model. This would
> eliminate the need to serialize the entity entirely when it is used for
> lookup.
>
> Writing out the entity after having updated its list property is another
> story all together as writes are subject to fail due to contention. To
> reduce the odds of contention you can shard your model by a factor of 10 or
> 20 to reduce but not eliminate the possibility of contention.
>
> There are a number of Google IO videos on YouTube that you can watch which
> cover these techniques.
>
> Jeff
>
>
>
>
>
> On Tue, Aug 31, 2010 at 5:22 AM, ogterran <[email protected]> wrote:
> > Hi,
>
> > I have a choice to store the data two ways. Which way is more
> > efficient when querying in BigTable?
>
> > First way:
> > class Page(db.Model):
> > pagekey= db.StringProperty(required=True)
> > usernames = db.StringListProperty(required=True)
>
> > So all the users who visits the page, will be added to the username
> > list
>
> > Second way:
>
> > class Page(db.Model):
> > pagekey= db.StringProperty(required=True)
> > username = db.StringProperty(required=True)
> > All the users who visits the page will have its own row
>
> > The query will be with both username and pagekey.
> > There can be a lot of users.
> > Is there a limit on the StringList size?
> > what are the advantages of each methods of storing data?
>
> > Thanks
> > Jon
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to [email protected].
> > To unsubscribe from this group, send email to
> > [email protected]<google-appengine%2Bunsubscrib
> > [email protected]>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> --
> Jeff
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
