When I created new index it was building it hole day. I thought it added all data to index as when I create index for small number of entities then index is created much faster.
May be there is some bug when index is built and google may fix it. Thanks John you helped a lot! On Jan 19, 11:36 pm, John Patterson <[email protected]> wrote: > You shouldn't need to do it again unless you change a property from > unindexed to indexed. Doing so will not update the implicit index > (i.e. single property index) and so any custom indexes you define will > also not include it. But I remember one of the Google engineers > looking into a possible bug regarding indexes and missing entities a > few weeks ago. Not sure if there was ever a real issue. > > On 20 Jan 2010, at 04:06, Max wrote: > > > > > Thanks John. > > > It worked. > > > Is there other way to re-create index then re-insert 5 million > > entities each time we detect problem? > > > We may add more indexes later. Should we re-insert everything again? > > > Here is BeanShell script that re-inserts 500 record (5 second) per > > request > > > import java.util.Iterator; > > import java.util.List; > > import com.google.appengine.api.datastore.DatastoreService; > > import com.google.appengine.api.datastore.DatastoreServiceFactory; > > import com.google.appengine.api.datastore.Entity; > > import com.google.appengine.api.datastore.FetchOptions; > > import com.google.appengine.api.datastore.Key; > > import com.google.appengine.api.datastore.KeyFactory; > > import com.google.appengine.api.datastore.PreparedQuery; > > import com.google.appengine.api.datastore.Query; > > > DatastoreService ds = DatastoreServiceFactory.getDatastoreService(); > > Key key = KeyFactory.createKey("Table1", "92417102:2345"); > > Entity entity1 = ds.get(key); > > Query query = new Query("Table1"); > > query.addFilter("cid", Query.FilterOperator.EQUAL, 2345l); > > query.addFilter("__key__", Query.FilterOperator.GREATER_THAN_OR_EQUAL, > > key ); > > FetchOptions options = FetchOptions.Builder.withLimit(500); > > PreparedQuery p1 = ds.prepare(query); > > Iterable it = p1.asIterable(options); > > ds.put(it); > > > On Jan 19, 3:35 pm, John Patterson <[email protected]> wrote: > >> Is it possible that some of your entities in Table1 were stored when > >> cid was declared unindexed? If so you will need to re-store them so > >> they get indexed in the implicit cid index and also in the custom > >> index on cid-ctime. > > >> Whether or not this was the cause of some entities not being included > >> in the custom index, you should try to re-store those entities that > >> are missing from your query. > > >> On 19 Jan 2010, at 20:01, Max wrote: > > >>> How do we to contact support? > >>> If we pay money then there must be at least email address where I > >>> can > >>> send email? > > >>> Basic query does not return every second line that it should return. > > >>> May be something wrong with index. > > >>> Please, help > > >>> On Jan 17, 3:59 pm, Max <[email protected]> wrote: > >>>> I can't find way to report serious problem. > >>>> We have uploaded real customer data and there is bug in datastore > >>>> results. > > >>>> But I can't find way to send screenshots and queries that will > >>>> not be > >>>> seem by everyone. > >>>> I can't show customer emails or IDs. > > >>>> In short problem is that when I do following query (I have changed > >>>> IDs > >>>> and table names) > > >>>> SELECT * FROM Table1 WHERE ctime >= DATETIME('2009-12-01 00:00:00') > >>>> ORDER BY ctime > > >>>> key - is key > >>>> cid - is indexed long > >>>> ctime - is indexed date > > >>>> first 2 lines are > >>>> cid:1234 cid:2009-12-01 00:00:37 > >>>> cid:2345 ctime:2009-12-01 00:01:11 > > >>>> when I filter it to show cis=2345 only > > >>>> SELECT * FROM Table1 WHERE cid= 2345 and ctime >= > >>>> DATETIME('2009-12-01 > >>>> 00:00:00') ORDER BY ctime > > >>>> first 2 lines are > >>>> cid:2345 ctime:2009-12-01 00:03:26 > >>>> cid:2345 ctime:2009-12-01 00:15:09 > > >>>> ---- > > >>>> I expected to see "cid:2345 ctime:2009-12-01 00:01:11 " as > >>>> first line > >>>> in second query. > >>>> But it is missing. > > >>>> We have uploaded 5,426,307 entities (9 GBytes) data to datastore > >>>> and > >>>> our report for December 2009 till now showed that > >>>> countMissing = 28253 > >>>> countExists = 17555 > > >>>> ---- > > >>>> How can I contact you privately? This data is stored on paid > >>>> account. > > >>>> Thanks, Max > >>> -- > >>> You received this message because you are subscribed to the Google > >>> Groups "Google App Engine for Java" group. > >>> To post to this group, send email to > >>> [email protected] > >>> . > >>> To unsubscribe from this group, send email to > >>> [email protected] > >>> . > >>> For more options, visit this group > >>> athttp://groups.google.com/group/google-appengine-java?hl=en > >>> . > > > -- > > You received this message because you are subscribed to the Google > > Groups "Google App Engine for Java" group. > > To post to this group, send email to [email protected] > > . > > To unsubscribe from this group, send email to > > [email protected] > > . > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine-java?hl=en > > .
-- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
