I am going to post two replies as I have a small idea that might help a little bit, and then a further post on the whole problem in general.
First, Ed, what you have said is consistent and the problem of identifying the "most recent Journey within a constrained time window" does appear to imply looking at all Journeys. So the question comes up, is there a way to exclude some of the Journeys? Is there perhaps some relationship between Journeys that has not been mentioned yet that could be used to make the data set smaller? The only thing I came up with is the relationship to the search fidelity/granularity. As stated above, there is no differentiation _within_ one minute of time, and further, the boundary of that minute is well defined. So, for any GIVEN minute, the system can at least determine, prior to search, that some Journeys are not the latest in that minute. Therefore, every time a new Journey starts, the system can (asynchronously of course) get all other Journeys that started in that same minute, and mark them as possibly_latest = false. By default, all Journeys are possibly_latest = true. And your search now includes this extra term, to narrow the results. [and one would use the obvious optimization of storing the minute on the Journey as an integer to make this search faster] And of course, the system should also have a cron job that runs at some appropriate frequency to mark all Journeys that are more than 2 months old as possibly_latest = false. This is given the other constraint that the oldest searched Journeys start should be no more than 2 months ago. This is no magic solution. But depending on the data, this could significantly reduce the number of Journeys that are searched (through index). -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
