The issue here is zig zag merge join. It's a bit more complex subject than I can easily explain in email, but to sum things up, what's happening is that we take multiple indexes and zig-zag between them to generate your result set. When this takes too long, that causes that exception to be thrown. Your query is using the searchables index multiple times. The way an AND query works is by making multiple queries and merging the results by zig-zagging between them. The query engine is doing this:
- Go to index pointing to an entity where searchables = hidden:::false - Now go to index pointing to an entity where searchables = sex - Now go to index pointing to an entity where searchables = repeat ... - Is there an entity that matches all these properties? No. Repeat the process. After traversing a large amount of zig/zags, the query engine will determine that you've attempted a query that is too complex and needs composite indices. This is much more likely to happen if the combination of the two facts is true: 1. You have a large amount of entities that do NOT match 2. You have a large amount of properties to match The reason is that most of your index traversals are going to result in nothing being returned. If this sounds complex, it *is*, but some visuals may help. You'll want to look at the links below: http://code.google.com/appengine/articles/datastore/overview.html http://www.youtube.com/watch?v=AgaL6NGpkB8 A solution to your problem may be simply to whittle down the result set to as small as you can reasonably get it, then traverse the entities in memory. We're currently working on a solution that would at least return a subset of the results so this exception isn't thrown, and you'll be able to resume the query from the returned cursor, but this won't be available for a few releases. On Thu, Sep 16, 2010 at 10:17 AM, Nickolay Tzvetinov <[email protected] > wrote: > An exact example is "SELECT __key__ FROM SearchIndex WHERE searchables > = ^_hidden:::false_^ AND searchables = sex AND searchables = repeat > AND searchables = mate AND searchables = feed AND searchables = sugar > AND searchables = rating:1". > > Executing this query I get the exception when the entities in which I > search are about 5000... > > Best regarts, > Nickolay Georgiev (Meddle) > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine-java%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
