On Mon, Jul 27, 2009 at 7:39 PM, Jai <[email protected]> wrote: > > Hi Nick, > > Thanks for replying, that made the things a bit more clear. But I > still don't understand the reasoning behind it. > > You said the following example > > Foo.all().filter('tags =', foo).filter('tags =', bar).order('bar') > > will result in an exploding index. > > But why so?
Because as soon as you apply an inequality filter or sort order to your query, it requires a composite index. In this case, the composite index has to include two list properties (actually the same list property twice), which leads to an exploding index. Naturally, if your number of tags is limited, this may not be a big problem. > > > I believe above statement is equivalent to: > > select * from Foo where tags='foo' AND tags = 'bar' order by 'bar1' > > I changed 'bar' to 'bar1' just to avoid confusion. > > Now since the order by is applied to 'bar1' and not to 'tags'(can that > be even done??) why will two separate indexes be required. Not two separate indexes - a single composite index that includes all the fields being queried on (the number of times they're used). > I > understand they can be required if order by is applied on tags > somehow, but for this case an index with default ordering for > listproperty class could have been used. With the exception of merge join queries (those with only equality filters and no sort orders), queries only ever use a single index. > > > The non exploding index that can be used here is: > > - kind: Foo > property: > - tags > - bar1 > direction:desc > > Or will this not work for the query in question? It won't, because the tags property has to appear as many times as it's used in a query. -Nick Johnson > > > Thanks for bearing with me. > > > On Jul 27, 12:56 pm, "Nick Johnson (Google)" <[email protected]> > wrote: > > Hi Jai, > > > > Both your example and my original example are cases of exploding indexes. > In > > my example, indexing on the same listproperty twice is required for > certain > > queries (for example, the Foo.all().filter('tags =', foo).filter('tags > =', > > bar).order('bar') example), but likewise results in an exploding index. > > > > -Nick Johnson > > > > > > > > On Mon, Jul 27, 2009 at 4:30 PM, Jai <[email protected]> wrote: > > > > > Hi Kyle, > > > > > I think what Nick meant was this: > > > > > - kind: Foo > > > property: tags1 > > > property: tags2 > > > > > that is having two different properties of type ListProperty because > > > then the index will have rows for each combination of values for each > > > of the property and number of rows will be multiplication of number of > > > distinct values in the indexes. > > > > > For your case it is ok because, there will be a single index 'tags' > > > with an entity being referenced by the DIFFERENT tag values in the > > > SAME index. So the references might be multiple, number of rows will > > > still be linear. > > > > > Hope it helps. > > > > > Regards, > > > Jai > > > On Jul 27, 11:16 am, Kyle Jensen <[email protected]> wrote: > > > > Hi Nick, > > > > > > I noticed that the dev server will automatically add indexes that > > > > appear to be exploding indexes when I run queries like the example > > > > above. Ie, if I do Foo.get_by_tags(['tag1', 'tag2']) the dev server > > > > will create an index in index.yaml that is identical to the one you > > > > posted above: > > > > > > - kind: Foo > > > > property: tags > > > > property: tags > > > > > > ** Is that index not required for the snippet > Foo.get_by_tags(['tag1', > > > > 'tag2'])? > > > > ** What about Foo.get_by_tags(['tag1', 'tag2']).order('bar') where > bar > > > > is an IntegerProperty > > > > > > Thanks for your help and sorry for my ignorance!! > > > > Kyle > > > > > > On Jul 27, 8:07 am, "Nick Johnson (Google)" <[email protected] > > > > > > wrote: > > > > > > > Hi Kyle, > > > > > > > Exploding indexes only come into play when you're _defining_ > indexes - > > > for > > > > > example, an index like this would be an exploding one: > > > > > > > - kind: Foo > > > > > property: tags > > > > > property: tags > > > > > > > As long as you can satisfy your queries using the built in merge > join > > > > > support (which is the case in your example, where you don't specify > > > > > inequality filters or sort orders), you'll be fine. > > > > > > > -Nick Johnson > > > > > > > On Mon, Jul 27, 2009 at 3:50 PM, Kyle Jensen <[email protected]> > > > wrote: > > > > > > > > Hi, I'd like to know if multiple equality filters on a > ListProperty > > > > > > leads to exploding indexes. > > > > > > > > I have a model something like the following (python): > > > > > > > > class Foo(db.model): > > > > > > tags = db.ListProperty(db.Category) > > > > > > > > @classmethod > > > > > > def get_by_tags(cls, tags): > > > > > > query = cls.all() > > > > > > for tag in tags: > > > > > > query.filter('tags =', tag) > > > > > > return query > > > > > > > > I'd like to know if I can filter by an arbitrary number of tags > > > > > > without encountering the exploding index problem (its not clear > to me > > > > > > from the docs). > > > > > > > > E.g., can I do > > > > > > query = Foo.get_by_tags(['tag0', 'tag1', 'tag2', 'tag3' .... > 'tagN']) > > > > > > > > etc etc. > > > > > > > > Thanks! Kyle > > > > > > > -- > > > > > Nick Johnson, Developer Programs Engineer, App Engine > > > > -- > > Nick Johnson, Developer Programs Engineer, App Engine > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
