Crossposting to App Engine Java group: the original thread is at http://groups.google.com/group/google-appengine/browse_thread/thread/22018ef2e132ac13/54485b787d5a80b5
In a few words: I have a problem with reasonable queries taking a very long time (several seconds). These queries return 128 entities, from a total of 500,000 entities of that type in the datastore. Each entity is about 400 bytes. On Tue, Dec 1, 2009 at 6:49 PM, Stephen <[email protected]> wrote: > On Dec 1, 9:12 pm, Eric Rannaud <[email protected]> wrote: >> On Tue, Dec 1, 2009 at 11:02 AM, Stephen <[email protected]> wrote: >>> On Dec 1, 9:55 am, Eric Rannaud <[email protected]> wrote: >>>> SELECT * FROM MessageS where id >= 0 && id < 128 order by id >>>> >>>> Calendar c = Calendar.getInstance(); >>>> long t0 = c.getTimeInMillis(); >>>> qmsgr = (List<MessageS>) qmsg.execute(lo, hi); >>>> System.err.println("getCMIdRange:qmsg: " + (c.getTimeInMillis() - t0)); >> >>> Are you fetching all 128 entities in one batch? If you don't, the >>> result is fetched in batches of 20, incurring extra disk reads and rpc >>> overhead. >> >>> Not sure how you do that with the Java API, but with python you pass >>> '128' to the .fetch() method of a query object. >> >> As far as I can tell, there is no such equivalent in the Java API. The > > Something like this..? > > DatastoreService datastore = > DatastoreServiceFactory.getDatastoreService(); > > Query query = new Query("MessageS"); > query.addFilter("id", Query.FilterOperator.GREATER_THAN_OR_EQUAL, 0); > > List<Entity> messages = datastore.prepare(query) > .asList(FetchOptions.Builder.withLimit(128)); > > You might also have to tweak chunkSize and/or prefetchSize, or ask on > the Java list. I did some tests with the code you proposed. The performance remains essentially the same as with the JDO API, i.e. between 1 and 4 second per "execute"/"prepare" statement (2.5s on average). DatastoreService datastore = DatastoreServiceFactory.getDatastoreService(); Query query = new Query("MessageS"); query.addFilter("id", Query.FilterOperator.GREATER_THAN_OR_EQUAL, lo); query.addFilter("id", Query.FilterOperator.LESS_THAN, hi); long t0 = Calendar.getInstance().getTimeInMillis(); List<Entity> r = datastore.prepare(query) .asList(FetchOptions.Builder .withLimit(128) .prefetchSize(128) .chunkSize(128)); System.err.println("LOW:getCMIdRange:qmsg: " + (Calendar.getInstance().getTimeInMillis() - t0) + " " + r.size()); Thanks. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
