great, thanks for the insight max. i have a client that will batch together multiple requests into one RPC call to my app on GAE. each of these individual requests may have one or more datastore accesses. this may include some prefetch requests.
so i want to build a mechanism that will interleave these requests taking advantage of the AsyncDatastoreService for minimum request latency. i've gone through my server-side stack and made it asynchronous by wrapping RPC returns in Future objects. then i've created a FutureChain object that takes one or more Future objects as input, and will return one Future object. i then have some code that will poll the ultimate Future objects until all of them have finished. it ends up being a simple multi-threaded emulation where each individual request in a batch gets a thread and each thread gives up control when it makes an Async request. now for the PreparedQuery, because my app knows how many items i want, i should be able to wrap it in a special Future object that will try to pull in that many items when it is polled...but the problem is, i don't know when the batch has come back, so every time i call next(), i risk blocking on I/O when i could be initiating another I/O asynchronously or processing the results of an async I/O. so until there is explicit knowledge of when the I/O for a batch has finished, i may be able to get away with reducing the poll-rate of queries.... i suppose i could just query for the keys, then i could use an explicit Async method to fetch the entities themselves. if i query for keys, will they be split up in batches? any way to know how many keys will be in one batch? On Nov 29, 11:08 am, "Max Ross (Google)" <maxr+appeng...@google.com> wrote: > Hi Luke, > > First the awesome news: > As of 1.4.0, many queries are implicitly asynchronous. When you call > PreparedQuery.asIterable() or PreparedQuery.asIterator(), we initiate the > query in the background and then immediately return. This lets you do work > while the first batch of results is being fetched. And, when the first > batch has been consumed we immediately request the next batch. If you're > performing a significant amount of work with each Entity as you iterate you > will probably see a latency win as a result of this. > > Now the less awesome news: > We didn't get around to making the List returned by PreparedQuery.asList() > work this same magic, but you can expect this in a future release. > > Some deeper thoughts: > The underlying RPCs between your app and the datastore fetch results in > batches. We fetch an initial batch of results, and once that batch has been > consumed we fetch the next batch. But, there's nothing in the API that maps > to these batches - it's either a List containing the entire result set or an > Iterable/Iterator that returns Entities one at a time. An API that provides > async access to the individual results returned by an Iterable/Iterator > (Iterator<Future<Entity>>) doesn't really make sense since you don't know > which call to hasNext() is going to require a new batch to be fetched, and > without that knowledge, the knowledge of what is going to trigger something > "expensive", you can't really make appropriate use of an asynchronous API. > > Going forward, we're definitely interested in exposing these batches > directly, and an explicitly async API for these batches makes a lot of sense > since fetching these batches would map directly to something "expensive" on > the server side. > > Hope this helps, > Max > > On Fri, Nov 26, 2010 at 4:41 PM, Luke <lvale...@gmail.com> wrote: > > i was taking a look at the 1.4.0 javadoc for AsyncDatastoreService. i > > see the get, put and delete operations return a Future, but the > > prepare methods return a naked PreparedQuery object, and it doesn't > > look like PreparedQuery has any async get methods. > > > does the AsyncDatastoreService not support asynchronous queries, or is > > there something i'm missing? > > > glad to see at lets the get and put methods are async, hoping to get > > async queries too (as well as async interfaces to more services). > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
