Hi Steve, I was thinking along the same lines. Having the root comment or thread hold a cache of the whole tree, which would be updated whenever an entity is updated, added or removed seems like the most efficient way for large hierarchies.
You could have your cached tree represented using nested dictionary objects on your root object using a PickleProperty (see: http://groups.google.com/group/google-appengine/msg/8433525107a8bb92) to be converted to XML as required. The comment class could have a path property (e.g. "id1/id2/id3") which could be quickly traversed to retrieve or update child comments. Since it doesn't need to be indexed, it can be a TextProperty() which isn't limited to 500 bytes. CRUD operations could be overridden in your Model class: class Comment(db.Model): path = TextProperty() def put(self): # update root comment super(Comment, self).save() def delete(self): # update root comment super(Comment, self).delete() Pros: - Reads only need to fetch 1 (root tree) or 2 entities (look-up child path & return sub-tree from root) - Not limited by depth Cons: - Writes update 2 entities. - Pickling/Unpickling could be resource intensive On Oct 31, 2:11 pm, "Steve Schwarz" <[EMAIL PROTECTED]> wrote: > On Fri, Oct 31, 2008 at 3:18 PM, Calvin Spealman <[EMAIL PROTECTED]>wrote: > > > It seems to me that the question at hand isn't hierarchial entities, per > > se. We have quite a few options for representing them and any will do, > > truthfully. > > > It is the operations on them which pose issues. Notably, how do we make our > > queries for decendants cost-effective? > > > I say use whatever representation feels comfortablee to you. Other issues > > you may deal with are a matter of indexing and other operations both > > important and separate from the exact representation. If the need is "give > > me all of this node's decendants" then we may create simple index entries > > mapping a node to each of its decendants and mmay be in a group without > > impacting the nodes themselves or other nodes' decendant indexes. > > Calvin, > I very new to GAE but that does seem to be the issue Chris was posing. While > the "natural" representation is a tree structure, the cost of getting each > comment might exceed the compute/selects allowed by GAE w/in a request. > Imagine if these were the comments for some popular Slashdot article. At > what point do you optimize for read speed? I'd guess the use case for > comments is "read many write few". It could be that the best model is to > actually update a single comment object with new comments (it would store > the hierarchy) so that the read case involves querying for a single large > object. > > An alternative is to store each comment in a hierarchy as described in > Chris' email and have an external periodic job hit the server and update a > single comment instance with the hierarchy when it detects that new comments > have been added (a kind of external mapreduce). Or a combination that > updates the "master" comment with the new comment when it is added. If you > want to do moderation the moderation step could be the one that appends the > new comment to the master comment. > > I'm thrashing on a similar problem. I want to serve a relatively large XML > file containing a couple thousand elements. It will have very few updates > and many reads. I'd like to be able to perform CRUD operations on the > elements and then have the XML file be updated when any of those elements > change. I don't want to prematurely optimize, but then again don't want to > have a "non-starter" either. > > From reading this list and the docs it seems there are times when > traditional DB backed designs have to be modified to work well in the GAE > environment. I'm also looking for direction/patterns/best practices. > > Best Regards, > Steve --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
