Eric, Sorry my previous answer was wrong. The query time increases with the number of entities fetched faster than I thought. >From a small benchmark in Python average time for your query is 150-200ms. This is for fetching 130 entities of about 400 bytes, with 5 properties each, using an inequality filter.
But still, one second is definitively too long. Hope the Java gurus can help you! On Dec 3, 5:03 am, Eric Rannaud <[email protected]> wrote: > Crossposting to App Engine Java group: the original thread is > athttp://groups.google.com/group/google-appengine/browse_thread/thread/... > > In a few words: I have a problem with reasonable queries taking a very > long time (several seconds). These queries return 128 entities, from a > total of 500,000 entities of that type in the datastore. Each entity > is about 400 bytes. > > > > On Tue, Dec 1, 2009 at 6:49 PM, Stephen <[email protected]> wrote: > > On Dec 1, 9:12 pm, Eric Rannaud <[email protected]> wrote: > >> On Tue, Dec 1, 2009 at 11:02 AM, Stephen <[email protected]> wrote: > >>> On Dec 1, 9:55 am, Eric Rannaud <[email protected]> wrote: > >>>> SELECT * FROM MessageS where id >= 0 && id < 128 order by id > > >>>> Calendar c = Calendar.getInstance(); > >>>> long t0 = c.getTimeInMillis(); > >>>> qmsgr = (List<MessageS>) qmsg.execute(lo, hi); > >>>> System.err.println("getCMIdRange:qmsg: " + (c.getTimeInMillis() - > >>>> t0)); > > >>> Are you fetching all 128 entities in one batch? If you don't, the > >>> result is fetched in batches of 20, incurring extra disk reads and rpc > >>> overhead. > > >>> Not sure how you do that with the Java API, but with python you pass > >>> '128' to the .fetch() method of a query object. > > >> As far as I can tell, there is no such equivalent in the Java API. The > > > Something like this..? > > > DatastoreService datastore = > > DatastoreServiceFactory.getDatastoreService(); > > > Query query = new Query("MessageS"); > > query.addFilter("id", Query.FilterOperator.GREATER_THAN_OR_EQUAL, 0); > > > List<Entity> messages = datastore.prepare(query) > > .asList(FetchOptions.Builder.withLimit(128)); > > > You might also have to tweak chunkSize and/or prefetchSize, or ask on > > the Java list. > > I did some tests with the code you proposed. The performance remains > essentially the same as with the JDO API, i.e. between 1 and 4 second > per "execute"/"prepare" statement (2.5s on average). > > DatastoreService datastore = > DatastoreServiceFactory.getDatastoreService(); > Query query = new Query("MessageS"); > query.addFilter("id", Query.FilterOperator.GREATER_THAN_OR_EQUAL, lo); > query.addFilter("id", Query.FilterOperator.LESS_THAN, hi); > > long t0 = Calendar.getInstance().getTimeInMillis(); > > List<Entity> r = datastore.prepare(query) > .asList(FetchOptions.Builder > .withLimit(128) > .prefetchSize(128) > .chunkSize(128)); > > System.err.println("LOW:getCMIdRange:qmsg: " > + (Calendar.getInstance().getTimeInMillis() - t0) > + " " + r.size()); > > Thanks. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
