I have performed some experiments. I generated test data on which an
IN query was performed. The test data consists of the following:
o an optional parent for all further test entities (i.e. the tests
were performed with and without common parent)
o a set of <n> referred entities, n taking the values 30, 50, 100, 500
o a set of <n> referrer entities, each having a field that contains
the key of one of the referred entities
For each experiment, there were two (no parent) resp. three (with
parent) queries, as follows (I hope the code can still be read after
reformatting in the post):
public static final String parentEntityKind = "TestInQueryParent";
public static final String referredEntityKind =
"TestInQueryReferred";
public static final String referrerEntityKind =
"TestInQueryReferrer";
public static final String referrerField = "ref";
private void queryData(StringBuilder out) {
DatastoreServiceConfig config =
DatastoreServiceConfig.Builder.withDefaults();
DatastoreService datastoreService =
DatastoreServiceFactory.getDatastoreService(config);
if(withParent) {
Query parentQuery = new Query(parentEntityKind);
Entity parent =
datastoreService.prepare(parentQuery).asSingleEntity();
out.append("fetched parent");
}
Query referredQuery = withParent ? new Query(referredEntityKind,
parent.getKey()) : new Query(referredEntityKind);
referredQuery.setKeysOnly();
long startTime = new Date().getTime();
Iterable<Entity> referreds =
datastoreService.prepare(referredQuery).asIterable();
ArrayList<Entity> referredList = new ArrayList<Entity>();
for(Entity referred : referreds) {
referredList.add(referred);
}
long endTime = new Date().getTime();
out.append("<br>fetched ");
out.append(referredList.size());
out.append(" referred entities in ");
out.append(Log.timeDiff(startTime, endTime));
List<Key> keyList = new ArrayList<Key>();
for (Entity entity : referredList) {
keyList.add(entity.getKey());
}
Query inQuery = new Query(referrerEntityKind);
inQuery.addFilter(referrerField, Query.FilterOperator.IN,
keyList);
startTime = new Date().getTime();
List<Entity> result =
datastoreService.prepare(inQuery).asList(FetchOptions.Builder.withDefaults());
endTime = new Date().getTime();
out.append("<br>fetched ");
out.append(result.size());
out.append(" referrer entities in ");
out.append(Log.timeDiff(startTime, endTime));
}
Log.timeDiff(startTime, endTime) computes the difference between start
time and end time and converts it to a String.
The results can be found here:
<https://spreadsheets.google.com/ccc?
key=0AvB6ADVFJKfSdHRWYjhNUWVwLWJDX2lkWEdSVnpNREE&hl=en>
I have enabled appstats, but it is not feasible to add a screen shot
of the experiment with 500 entity pairs here. However, the picture is
very similar to the Python one (except for the lack of a limit of 30,
of course). It looks again as if the IN query is split into 500
individual queries that are executed sequentially, each of which takes
about 2-6ms elapse time, resulting in an RPC total of 2683ms (11305ms
api) and a Grand Total of 3192ms (11305ms cpu+api), which means that
it's not only possible to execute such queries, but moreover they
execute in a reasonnable amount of time. Of course, in this particular
case it would be easier to query the entities directly using a
get(List<Key>) call, but if you have multiple referrers to query (like
an 1 to n relationship), you can't avoid this kind of query.
Cheers, Remigius.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
