There isn't a solid time line for this, but it's on my short list ;-) (which actually doesn't seem to be that short as of late).
Something to keep in mind though: Removing these limits won't make your query run faster. So it's a trade off between write vs read latency and cost (although adding more indexes should only affect write cost) and most web applications are read heavy. So you may still want to have these indexes once the limit is removed. When the limit is removed the only real difference will be the lack of an exception to force your hand (instead you would have to rely on appstats to figure out which queries to optimize). Once the limit is removed a really pathological query could have your users waiting up to 30 seconds for results (which we think is better than no results at all). On Tue, Jun 15, 2010 at 1:06 AM, Ravi <[email protected]> wrote: > Hi Alfred Fuller, > > Thanks for your information, videos gave me lot of information. > As per your suggestion I have added few composite index to avoid this > exception still index are getting built. > As we get exception on some queries, will add those to composite > index. But this will eat my lot of CPU quota. > Could you please tell me when will this restriction on query will be > removed because we will plan our product enhancement accordingly. > > Thanks, > Ravi > > On Jun 15, 4:57 am, Alfred Fuller > <[email protected]<arfuller%[email protected]> > > > wrote: > > Yes, you can create composite indexes that contain columns that are often > > queried together. The merge join algorithm will use these indexes to make > > queries that include those specific columns more efficient. Or if you > see > > this error more often on some queries than others, create composite > indexes > > for those queries (or the common columns between those queries) > > > > http://www.youtube.com/watch?v=ofhEyDBpngMat time 30:30 gives an example > of > > this. > > > > > > > > On Mon, Jun 14, 2010 at 7:58 AM, Ravi <[email protected]> wrote: > > > Hi Geoffrey Spear, > > > Thanks for your clarification. > > > We suspected the same reason. Last few days back our data was less > > > around 20,000 rows there was no problem. > > > Only when more data is added we are facing this problem. > > > > > Our table is too large which is around 50 fields/columns and around > > > 50,000 rows. Query filter is also dynamically generated (user selects > > > few fields and submits). > > > Our servlet will query database depending on the user selected fields. > > > In this scenario we dont know which fields will be selected by user > > > and certainly I cant have composite index for 50 combinations. > > > > > Is there any work around for this problem?? > > > > > Thanks, > > > Ravi > > > > > On Jun 14, 5:59 pm, Geoffrey Spear <[email protected]> wrote: > > > > It's not a problem with indexing, it's a problem with your data. If > > > > you don't add composite indexes, the system will attempt to serve > > > > queries that contain only equality filters using a merge-join > > > > strategy, which is less efficient than a simple index lookup (with > the > > > > benefit of not needing to add the indexes, which use space and write- > > > > time CPU). If your data becomes such that the query can not be done > > > > efficiently with the merge-join anymore, you'll start to see that > > > > exception and you'll need to add a composite index. > > > > > > On Jun 12, 2:00 am, Ravi <[email protected]> wrote: > > > > > > > Hi Google Team, > > > > > > > All these days our application was working fine and we have not > > > > > changed any datastore index or codebase of our application. > > > > > Everyday we upload new set of data to our application. Today also > we > > > > > have uploaded new set of data but today after uploading the data we > > > > > are getting very strange exception as below: > > > > > > > com.google.appengine.api.datastore.DatastoreNeedIndexException: The > > > > > built-in indices are not efficient enough for this query and your > > > > > data. Please add a composite index for this query.. An index is > > > > > missing but we are unable to tell you which one due to a bug in the > > > > > App Engine SDK. If your query only contains equality filters you > most > > > > > likely need a composite index on all the properties referenced in > > > > > those filters. > > > > > > > We are worried after seeing the above exception, we dont have any > > > > > complex queries, all the queries contain equals to filter (==). > > > > > We tested the same set of data and query in our development > > > > > environment, it is working fine. > > > > > For testing purpose, we uploaded the same set of data to our other > > > > > testing appengine applcation, it is working fine there also. > > > > > > > Only in our production system, we are getting > > > > > DatastoreNeedIndexException. > > > > > > > Could you please confirm is there any problem with the datastore > > > > > indexing. > > > > > Data stored in our production datastore is more than 0.5 GB, will > > > > > thee be any problem in indexing if there is high amount of data. > > > > > > > Application ID: bigpaisa2.appspot.com > > > > > Latest application version: > > >http://061102.latest.bigpaisa2.appspot.com > > > > > - In this version DatastoreNeedIndexException is occuring. > > > > > > > Please help us as soon as possible. > > > > > > > Thanks, > > > > > Ravi > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to [email protected] > . > > > To unsubscribe from this group, send email to > > > [email protected]<google-appengine%[email protected]><google-appengine%2Bunsubscrib > [email protected]> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
