I also wanted to add this comment which is probably not helpful, but speaks more to the root of the problem. :) And that is this, "you can't get there from here". The fundamental problem does not appear capable of being reduced beyond looking at a bunch of Journeys. I'm going to summarize a couple of things here, but in the end it just comes down to look at a bunch of data ... unless you change something else.
A quick summary of some points: 1. You can de-normalize Car fields to Journey as it relates to this search. So the problem does reduce, slightly, to looking for Journeys. The "two" lookup aspect of it should not be much of a problem. I know this was mentioned above. 2. You can try to squash aspects of your search into the Key as Stephen suggested. This is a good idea but it does not change the nature of the problem - looking at a whole lot of entities, if only the keys of those entities. These other possibilities are excluded from being viable because of the problem description. 3. Make the processing more parallel by turning it into a counting problem and using a map reduce approach or similar. This is the same as changing the requirements from "query" to "report" in practical terms. There is also the sticky issue of having a reduce implementation on app engine right now. 4. Changing the search criteria to be more restricted in some way that introduces a further relationship between Journeys. This additional information could probably be used to make the search perform better. In fact, one would engineer it in such a way, there would probably be no need to make a change in the way this search worked unless it resulted in this benefit. 5. There is also the "break the fourth wall" kind of solution. Take this part of the problem off of app engine. You would pay a lot more for this. Both in terms of complexity and cost (and maybe some other things too). It may not be realistic, but how realistic is the alternative? I hope you will report back at some stage and tell us. (this might allow for a truly parallel query for example ... come on Datastore Plus!) What I am getting at is that the system probably needs a higher level change to achieve the desired goal given the constraints that exist at this time. That is the "you can't get there from here". I think this has been virtually proven at this point though I would be happy to be proved wrong. It comes to trade-offs like everything. You can often trade space for speed, etc. etc. But it is often a trade. So you may need to look at trades between the existing use cases / user expectation and speed. Thanks Ed for sharing with the group. This has been a good discussion and interesting to think about. -- Jay -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
