Currently hasNext() will block on the first batch of results. There is
currently no way to do what you are asking in Java.
On Tue, Dec 7, 2010 at 2:40 AM, Maxim Veksler <ma...@vekslers.org> wrote:
> Hi Max,
>
> Could you please provide input on the following design:
>
> We design for DataSource redundancy. For this we've built a flow which
> obtains data from 2 sources: First from GAE DataSource using PreparedQuery
> and if that fails (0.5sec timeout) from our backup service feed which we
> access using URLFetch (auto scaled EC2 nodes).
>
> We strive for maximum performance, So we plan to issue async call to each
> of the data sources and later when the data becomes required poll the 2
> sources, using the first one available.
>
> As for the URLFetch service, we will use this patten:
> Future<HTTPResponse> f = fetchService.fetchAsync(request);
> ...
> if(f.isDone()) { ... }
>
> But for PreparedQuery.asIterator(), I'm not sure how this is implement on
> your side. I would love an answer confirming that this code will not block
> on hasNext() (While datasource is asynchronously getting results)?
>
> Iterator<Entity> iterator =
> getAsyncDatastoreService().prepare(findGeoIP).asIterator();
> ....
> if(iterator.hasNext()) { ... }
>
>
>
> Thanks for the insights.
>
> Maxim.
>
> On Mon, Nov 29, 2010 at 9:08 PM, Max Ross (Google) <
> maxr+appeng...@google.com <maxr%2bappeng...@google.com>> wrote:
>
>> Hi Luke,
>>
>> First the awesome news:
>> As of 1.4.0, many queries are implicitly asynchronous. When you call
>> PreparedQuery.asIterable() or PreparedQuery.asIterator(), we initiate the
>> query in the background and then immediately return. This lets you do work
>> while the first batch of results is being fetched. And, when the first
>> batch has been consumed we immediately request the next batch. If you're
>> performing a significant amount of work with each Entity as you iterate you
>> will probably see a latency win as a result of this.
>>
>> Now the less awesome news:
>> We didn't get around to making the List returned by PreparedQuery.asList()
>> work this same magic, but you can expect this in a future release.
>>
>> Some deeper thoughts:
>> The underlying RPCs between your app and the datastore fetch results in
>> batches. We fetch an initial batch of results, and once that batch has been
>> consumed we fetch the next batch. But, there's nothing in the API that maps
>> to these batches - it's either a List containing the entire result set or an
>> Iterable/Iterator that returns Entities one at a time. An API that provides
>> async access to the individual results returned by an Iterable/Iterator
>> (Iterator<Future<Entity>>) doesn't really make sense since you don't know
>> which call to hasNext() is going to require a new batch to be fetched, and
>> without that knowledge, the knowledge of what is going to trigger something
>> "expensive", you can't really make appropriate use of an asynchronous API.
>>
>> Going forward, we're definitely interested in exposing these batches
>> directly, and an explicitly async API for these batches makes a lot of sense
>> since fetching these batches would map directly to something "expensive" on
>> the server side.
>>
>> Hope this helps,
>> Max
>>
>> On Fri, Nov 26, 2010 at 4:41 PM, Luke <lvale...@gmail.com> wrote:
>>
>>> i was taking a look at the 1.4.0 javadoc for AsyncDatastoreService. i
>>> see the get, put and delete operations return a Future, but the
>>> prepare methods return a naked PreparedQuery object, and it doesn't
>>> look like PreparedQuery has any async get methods.
>>>
>>> does the AsyncDatastoreService not support asynchronous queries, or is
>>> there something i'm missing?
>>>
>>> glad to see at lets the get and put methods are async, hoping to get
>>> async queries too (as well as async interfaces to more services).
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Google App Engine for Java" group.
>>> To post to this group, send email to
>>> google-appengine-j...@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com>
>>> .
>>> For more options, visit this group at
>>> http://groups.google.com/group/google-appengine-java?hl=en.
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine for Java" group.
>> To post to this group, send email to
>> google-appengine-j...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com>
>> .
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine-java?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
