The first query, is a key-only merge-join query. Then I fetch the parents of the results (this is the technique suggested by Brett Slatkin at Building Scalable, Complex Apps on App Engine) When I get those parents, I get a key field from them and fetch it from the datastore. (I am making some changes to avoid this step)
The last two queries have nothing to do with the above, so I can easily fetch them asynchronically. One of those is a count() of the first query. @Max: I really tried to optimize using multiple async gets, by the batch is way much faster. On Fri, Dec 17, 2010 at 9:30 PM, Max Ross (Google) < [email protected] <maxr%[email protected]>> wrote: > Interesting results! I've run similar tests with writes (put and delete) > and my results indicate that in most situations, multiple async writes will > perform better than one batch sync call. I can't think of any reason this > wouldn't be the case for reads as well. Are your entities all in different > entity groups? > > Thanks, > Max > > > On Fri, Dec 17, 2010 at 4:11 PM, Ikai Lan (Google) < > [email protected] <ikai.l%[email protected]>> wrote: > >> Very cool. Thanks for posting! >> >> What kind of processing are you doing after you retrieve the entities? I'm >> surprised it made a difference. >> >> >> -- >> Ikai Lan >> Developer Programs Engineer, Google App Engine >> Blogger: http://googleappengine.blogspot.com >> Reddit: http://www.reddit.com/r/appengine >> Twitter: http://twitter.com/app_engine >> >> >> >> On Fri, Dec 17, 2010 at 2:48 PM, Gal Dolber <[email protected]> wrote: >> >>> Even better :) >>> >>> [image: Screen shot 2010-12-17 at 7.48.44 PM.png] >>> >>> >>> On Fri, Dec 17, 2010 at 7:31 PM, Gal Dolber <[email protected]>wrote: >>> >>>> Here are my results if anyone interested. >>>> >>>> The idea was splitting a batch into multiple async gets so I could start >>>> processing each results while the others were being fetched. >>>> With the batch get I need to wait until I get all the results to do the >>>> next operations. >>>> >>>> At the end (on my situation) batching won. >>>> * >>>> * >>>> *Multiple async gets* >>>> [image: MultipleAsyncs.png] >>>> >>>> *Batching* >>>> >>>> [image: Batch.png] >>>> >>>> >>>> On Thu, Dec 16, 2010 at 4:48 PM, Gal Dolber <[email protected]>wrote: >>>> >>>>> Yes, I am talking about datastore api. >>>>> >>>>> I made some isolated tests and the batch always wins (no by so much). >>>>> >>>>> But in my app, when I break the batch into multiple async gets, I can >>>>> start processing each async get result (and doing another api calls) >>>>> before... and it seems to be perform better. >>>>> >>>>> I'll keep doing tests and post if find anything interesting. >>>>> Thanks! >>>>> >>>>> >>>>> On Thu, Dec 16, 2010 at 2:42 PM, Ikai Lan (Google) < >>>>> [email protected] <ikai.l%[email protected]>> wrote: >>>>> >>>>>> You're talking about the datastore API? Underneath the hood, >>>>>> synchronous gets just make asynchronous calls and block on Future.get() >>>>>> anyway. Making a single synchronous batch get would translate to a single >>>>>> asynchronous RPC with multiple keys vs. multiple asynchronous get RPCs. I >>>>>> suspect the single batch call will perform better from a CPU cost/quota >>>>>> perspective, but I'd defer to AppStats benchmarks as the authoritative >>>>>> answer. >>>>>> >>>>>> -- >>>>>> Ikai Lan >>>>>> Developer Programs Engineer, Google App Engine >>>>>> Blogger: http://googleappengine.blogspot.com >>>>>> Reddit: http://www.reddit.com/r/appengine >>>>>> Twitter: http://twitter.com/app_engine >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Dec 15, 2010 at 3:38 AM, Gal Dolber <[email protected]>wrote: >>>>>> >>>>>>> Should I expect the similar performance? or the batch will always be >>>>>>> better? >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Google App Engine for Java" group. >>>>>>> To post to this group, send email to >>>>>>> [email protected]. >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]<google-appengine-java%[email protected]> >>>>>>> . >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/google-appengine-java?hl=en. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Google App Engine for Java" group. >>>>>> To post to this group, send email to >>>>>> [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]<google-appengine-java%[email protected]> >>>>>> . >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/google-appengine-java?hl=en. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Guit: Elegant, beautiful, modular and *production ready* gwt >>>>> applications. >>>>> >>>>> http://code.google.com/p/guit/ >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Guit: Elegant, beautiful, modular and *production ready* gwt >>>> applications. >>>> >>>> http://code.google.com/p/guit/ >>>> >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Guit: Elegant, beautiful, modular and *production ready* gwt >>> applications. >>> >>> http://code.google.com/p/guit/ >>> >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine for Java" group. >>> To post to this group, send email to >>> [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<google-appengine-java%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine-java?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine for Java" group. >> To post to this group, send email to >> [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<google-appengine-java%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine-java?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine-java%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > -- Guit: Elegant, beautiful, modular and *production ready* gwt applications. http://code.google.com/p/guit/ -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
