I tried a new trick last night.
The assumption is that since the query goes against the "eta" column, and
since entity properties have built-in indexes, then the query should be
really fast. But it's timing-out, and when it works it takes 20+ seconds, so
there maybe something wrong with the index. Here is the query again for
reference.
SELECT * FROM KnownFeed where eta < datetime(2011, 5, 25, 10, 45, 3)
What if I add another index? The built-in one is obviously not doing the
job. But I can't add a second index to the same property, so I added a
composite index as follows:
- kind: KnownFeed
properties:
- name: eta
- name: polling_period
This index goes against the "eta" column and another irrelevant column
"polling_period". And I updated my query as follows:
query = KnownFeed.all().filter('eta <',
end_time).order('eta').order('polling_period')
I'm ordering by "polling_period" to force GAE to use the new index (I
don't really care about polling_period order).
That worked for the last 8 hours. During which I didn't have timeout, and
I thought I'm in the clear. But just now the same kind of timeouts started
to happen. The query starts to time-out, and needs to be retried many times
before it works. I'm running out of tricks to try :(
Waleed
On Wed, May 25, 2011 at 12:09 PM, Waleed Abdulla <[email protected]> wrote:
> Stephen,
> I don't see how your suggestion would help! Can you please elaborate on
> how it's related? Also, I'm not deleting any entities. I'm just updating
> them. And when the query is timing out, it does so even when there is no
> load on the app.
>
> Regards,
> Waleed
>
>
>
> On Wed, May 25, 2011 at 5:41 AM, Stephen <[email protected]> wrote:
>
>> On Wed, May 25, 2011 at 11:53 AM, Waleed Abdulla <[email protected]>
>> wrote:
>> >
>> > In the last 12 hours the issue has been worse. Now I can't even run the
>> > query from the Datastore viewer. This is the query:
>> >
>> > SELECT * FROM KnownFeed where eta < datetime(2011, 5, 25, 10, 45, 3)
>> >
>> > This times out now and breaks the datastore viewer.
>>
>>
>> Instead of frequently deleting these entities, try adding a property
>> 'deleted' and set it to true. Modify the query to check this property.
>> Run a garbage collection task once/month to delete old entities.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to [email protected].
>> To unsubscribe from this group, send email to
>> [email protected].
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
