Hi Ikai, Thanks for your clarification, for time being we will create index.
Thanks, Ravi On Jun 15, 10:13 pm, "Ikai L (Google)" <[email protected]> wrote: > Ravi, > > There's no ETA for the index traversal optimizations Alfred is talking > about. We'll try to get it out as quickly as possible. Create the indexes > for the time being. You can remove them later using the vacuum_indexes tool. > > > > > > On Tue, Jun 15, 2010 at 1:06 AM, Ravi <[email protected]> wrote: > > Hi Alfred Fuller, > > > Thanks for your information, videos gave me lot of information. > > As per your suggestion I have added few composite index to avoid this > > exception still index are getting built. > > As we get exception on some queries, will add those to composite > > index. But this will eat my lot of CPU quota. > > Could you please tell me when will this restriction on query will be > > removed because we will plan our product enhancement accordingly. > > > Thanks, > > Ravi > > > On Jun 15, 4:57 am, Alfred Fuller > > <[email protected]<arfuller%[email protected]> > > > wrote: > > > Yes, you can create composite indexes that contain columns that are often > > > queried together. The merge join algorithm will use these indexes to make > > > queries that include those specific columns more efficient. Or if you > > see > > > this error more often on some queries than others, create composite > > indexes > > > for those queries (or the common columns between those queries) > > > >http://www.youtube.com/watch?v=ofhEyDBpngMattime 30:30 gives an example > > of > > > this. > > > > On Mon, Jun 14, 2010 at 7:58 AM, Ravi <[email protected]> wrote: > > > > Hi Geoffrey Spear, > > > > Thanks for your clarification. > > > > We suspected the same reason. Last few days back our data was less > > > > around 20,000 rows there was no problem. > > > > Only when more data is added we are facing this problem. > > > > > Our table is too large which is around 50 fields/columns and around > > > > 50,000 rows. Query filter is also dynamically generated (user selects > > > > few fields and submits). > > > > Our servlet will query database depending on the user selected fields. > > > > In this scenario we dont know which fields will be selected by user > > > > and certainly I cant have composite index for 50 combinations. > > > > > Is there any work around for this problem?? > > > > > Thanks, > > > > Ravi > > > > > On Jun 14, 5:59 pm, Geoffrey Spear <[email protected]> wrote: > > > > > It's not a problem with indexing, it's a problem with your data. If > > > > > you don't add composite indexes, the system will attempt to serve > > > > > queries that contain only equality filters using a merge-join > > > > > strategy, which is less efficient than a simple index lookup (with > > the > > > > > benefit of not needing to add the indexes, which use space and write- > > > > > time CPU). If your data becomes such that the query can not be done > > > > > efficiently with the merge-join anymore, you'll start to see that > > > > > exception and you'll need to add a composite index. > > > > > > On Jun 12, 2:00 am, Ravi <[email protected]> wrote: > > > > > > > Hi Google Team, > > > > > > > All these days our application was working fine and we have not > > > > > > changed any datastore index or codebase of our application. > > > > > > Everyday we upload new set of data to our application. Today also > > we > > > > > > have uploaded new set of data but today after uploading the data we > > > > > > are getting very strange exception as below: > > > > > > > com.google.appengine.api.datastore.DatastoreNeedIndexException: The > > > > > > built-in indices are not efficient enough for this query and your > > > > > > data. Please add a composite index for this query.. An index is > > > > > > missing but we are unable to tell you which one due to a bug in the > > > > > > App Engine SDK. If your query only contains equality filters you > > most > > > > > > likely need a composite index on all the properties referenced in > > > > > > those filters. > > > > > > > We are worried after seeing the above exception, we dont have any > > > > > > complex queries, all the queries contain equals to filter (==). > > > > > > We tested the same set of data and query in our development > > > > > > environment, it is working fine. > > > > > > For testing purpose, we uploaded the same set of data to our other > > > > > > testing appengine applcation, it is working fine there also. > > > > > > > Only in our production system, we are getting > > > > > > DatastoreNeedIndexException. > > > > > > > Could you please confirm is there any problem with the datastore > > > > > > indexing. > > > > > > Data stored in our production datastore is more than 0.5 GB, will > > > > > > thee be any problem in indexing if there is high amount of data. > > > > > > > Application ID: bigpaisa2.appspot.com > > > > > > Latest application version: > > > >http://061102.latest.bigpaisa2.appspot.com > > > > > > - In this version DatastoreNeedIndexException is occuring. > > > > > > > Please help us as soon as possible. > > > > > > > Thanks, > > > > > > Ravi > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to [email protected] > > . > > > > To unsubscribe from this group, send email to > > > > [email protected]<google-appengine%2Bunsubscrib > > > > [email protected]><google-appengine%2Bunsubscrib > > [email protected]> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%2Bunsubscrib > > [email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog:http://googleappengine.blogspot.com > Twitter:http://twitter.com/app_engine > Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
