Datastore query times are not guaranteed to be consistent, as Datastore is a shared, distributed service. Since you're performing 4 equality filters this can add to the variation.
The question of how to optimize for speed is a bit broad and depends on your data model. With that being said I can offer a few general suggestions: 1) You can speed up queries, trading extra storage and write costs, by defining manual indexes on properties which are frequently queried. See Datastore Indexes <https://cloud.google.com/appengine/docs/standard/java/datastore/indexes> for more details on this as well as the related document Index Selection and Advanced Search <https://cloud.google.com/appengine/articles/indexselection>. . 2) Related to the above, if you only need to fetch a subset of properties instead of the whole entity you can define indexes on those properties and use Projection Queries <https://cloud.google.com/appengine/docs/standard/java/datastore/projectionqueries>, which improves speed by fetching from the index instead of the entity. 3) Use Objectify <https://cloud.google.com/appengine/docs/standard/java/gettingstarted/using-datastore-objectify>. Objectify will automatically cache entities in Memcache <https://cloud.google.com/appengine/docs/standard/java/memcache/>, and will perform a keys-only query + cached get-by-key when possible (also called a "hybrid" query <https://github.com/objectify/objectify/wiki/Caching#hybrid-queries>). Initial queries will be slower with this method but subsequent queries will be significantly faster as the entities will be pulled from Memcache. 4) Try to reduce the actual number of queries to the Datastore if possible. There are a pair of answers on Stack Overflow asked by the same user which mention two strategies for this. One is denormalization <http://stackoverflow.com/questions/10384434/how-to-optimize-one-to-many-queries-in-the-datastore>, which refers to intentionally duplicating properties across different entities (trading more storage for speed). Another is to prefetch and store in memory <http://stackoverflow.com/questions/10368312/how-to-reduce-number-of-requests-to-the-datastore> the results of a query on one kind and use it as a lookup for another (which offloads some of the filtering from the Datastore to your application, which can be faster in certain situations). 5) If possible, alter your data model to reduce the number of filters required for a query. For example, you may not need to have one entity with several foreign key relationships and instead be able split it into different entities / queries. On Sunday, March 5, 2017 at 11:12:53 AM UTC-5, Arjunkumar Udainath wrote: > > My replies take a long time to get posted here since I'm new. > > My application is written in Java and instance class of my app is F4. > There are a total of 922,758 entities of that Kind in the Datastore. > > Here's the function: > > public static Map <String , Object> getEntitiesUsingQueryCursor( String kind > , int limit , int chunkSize , String currentCursor, String account, String > user, Boolean status, String dept ) throws Exception > { > > String nextCursor = null; > > Entity entity = null; > > List <Entity> listOfEntity = new ArrayList <Entity>(); > > Map <String , Object> result = new HashMap <String , Object>(); > > > DatastoreService datastore = > DatastoreServiceFactory.getDatastoreService(); > com.google.appengine.api.datastore.Query q = new > com.google.appengine.api.datastore.Query( kind ); > > List <Filter> listOfFilter = new ArrayList <Filter>(); > Filter filter1 = new FilterPredicate( "accountID" , FilterOperator.EQUAL , > account); > Filter filter2 = new FilterPredicate( "assigneeID" , FilterOperator.EQUAL , > user); > Filter filter3 = new FilterPredicate( "departmentID" , FilterOperator.EQUAL , > dept); > Filter filter4 = new FilterPredicate( "open" , FilterOperator.EQUAL , > status); //Boolean > listOfFilter.add( filter1 ); > listOfFilter.add( filter2 ); > listOfFilter.add( filter3 ); > listOfFilter.add( filter4 ); > Filter filterParams1 = filterParams = CompositeFilterOperator.and( > listOfFilter ); > q.setFilter( filter ); > > PreparedQuery pq = datastore.prepare( q ); > FetchOptions fetchOptions = > FetchOptions.Builder.withLimit(limit).prefetchSize( chunkSize ).chunkSize( > chunkSize );// limit & chunksize - 500 > > if ( !StringUtil.isBlank( currentCursor ) ) > fetchOptions.startCursor( Cursor.fromWebSafeString( > currentCursor ) ); > > QueryResultIterable <Entity> results = pq.asQueryResultIterable( > fetchOptions ); > QueryResultIterator <Entity> iterator = results.iterator(); > > while ( iterator.hasNext() ) > { > entity = iterator.next(); > listOfEntity.add( entity ); > } > > if(listOfEntity.size() == limit) > nextCursor = iterator.getCursor().toWebSafeString(); > > result.put( "cursor" , nextCursor ); > result.put( "entity" , listOfEntity ); > > return result; > } > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/3d8ab756-7d85-4a96-8358-980539bfb5ea%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
