Hello Richard, I referred you to the best practices because depending on your use case any one of them might be the key to your performance issues. In Cloud Datastore there are only 2 factors that determine the speed of your query.
The first of those is indexing, keeping in mind that you cannot do a query without an index existing for that query. By default every single property of your entities has two indexes created for it, one for ascending and one for descending. Any query using only one of these fields will be fast and no further optimization is required. If your query combines 2 fields or more with equality filters, as is the case in your example, those single property indexes will be queried separately and then merged, which is not a fast operation. That is why you need to make sure that you not only have indexes for both of those properties, but you also need to have a composite index explicitly defined with active and model in that order for your MyModel kind. So something that could explain the difference in speed is if you do not have an explicit composite index for the slow query, but do have one for the fast query. The second factor that determines the speed of your query is the total size of the dataset returned by your query. This total size is determined both by the number of entities returned and the size of those entities. Given this it is normal that a query for a result set twice as large would take twice as long. The other thing to consider is that you should only be querying for fields that you are going to be using. For instance, as explained in the best practices, if the only fields you care about are active and online then you could use a projection query <https://cloud.google.com/datastore/docs/concepts/queries#projection_queries> to retrieve only those fields. I hope this helps answer your questions. On Friday, September 1, 2017 at 7:03:43 AM UTC-4, Richard Cheesmar wrote: > > Interestingly, I have another query which is identical except for one more > entity comparison > > result, next_cursor, more = MyModel.query(MyModel.active == True, > MyModel.offline == False, > MyModel.location.alpha2 == alpha2).fetch_page(pagination, > start_cursor=cursor) > > This query is infinitely faster. > > Are Boolean model properties indexed by default? > > > On Thursday, August 31, 2017 at 11:31:20 AM UTC+3, Richard Cheesmar wrote: >> >> The following query seems to take an way too long >> >> >> t0 = time.time() >> >> # Fetch the entities and return result with cursor >> result, next_cursor, more = MyModel.query(MyModel.active == True, >> MyModel.offline == False).fetch_page(pagination, start_cursor=cursor) >> >> logging.info(time.time() - t0) >> >> >> >> On the local development machine it takes nearly 2 seconds and there are >> less than 150 entities in the model >> >> When live with around 800 entities it takes nearly 2 seconds to retrieve >> the first 250 entities and then around 4 seconds to retrieve the >> remainder. There are two separate requests to retrieve each set on the >> live machine. >> >> I've set the time on the query only, therefore no other processing is >> taken into account in these timings. >> >> >> Enter code here... >> >> >> >> >> >> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/1521b68d-0731-4069-9518-fe0ca276453c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
