I am sharing my experience of deleting all the data of one of my apps.
We are talking about 500MB spread over about a few thousands records
and about 15 Models. The summary is that it worked but caused many
errors and depleted my CPU time, read on for the details...
The approach was pretty simple. Iterate though all of a the 15 kinds
and create one task for each kind. Then each of these tasks, queries
the datastore for 10 entities of this kind, db.delete() them and then
queues one more similar task. This continues until the datastore
reports no more records of this kind. I use the default queue. I would
guess that 60-70% records are in two Models so I did notice as I
expected that pretty quickly only two tasks reamained in the queue.
One more thing, all the data have the same ancestor.
At the end of the process the good news is tha the algorithm worked,
no data in the app and the Dashboard reports:
96% of my CPU time depleted
10% of my datastore time depleted
748 tasks queued
other indicators healthy...
Now I do see many many logs with the following two traces which I
think is what caused a lot of tasks to fail and be requeued and
eventually depleted almost all my CPU time. So what could I be doing
wrong? What could I do to avoid these errors?
--------------------------------------------------------------------------------------------------------------------
File "/base/data/home/apps/neatschool-test/
0-7-22-81dafb9.334504032648079489/swplatform/controllers/admin.py",
line 89, in post
db.delete(res)
File "/base/python_lib/versions/1/google/appengine/ext/db/
__init__.py", line 1127, in delete
datastore.Delete(keys)
File "/base/python_lib/versions/1/google/appengine/api/
datastore.py", line 269, in Delete
raise _ToDatastoreError(err)
File "/base/python_lib/versions/1/google/appengine/api/
datastore.py", line 2055, in _ToDatastoreError
raise errors[err.application_error](err.error_detail)
TransactionFailedError: too much contention on these datastore
entities. please try again.
------------------------------------------------------------------------------------------------------------------------
File "/base/data/home/apps/neatschool-test/
0-7-22-81dafb9.334504032648079489/swplatform/controllers/admin.py",
line 86, in post
results = qq.fetch(10)
File "/base/data/home/apps/neatschool-test/
0-7-22-81dafb9.334504032648079489/swplatform/util/db/query.py", line
118, in fetch
ents = self.query.fetch(limit, offset)
File "/base/python_lib/versions/1/google/appengine/ext/db/
__init__.py", line 1426, in fetch
raw = self._get_query().Get(limit, offset)
File "/base/python_lib/versions/1/google/appengine/api/
datastore.py", line 959, in Get
return self._Run(limit, offset)._Get(limit)
File "/base/python_lib/versions/1/google/appengine/api/
datastore.py", line 903, in _Run
_ToDatastoreError(err)
File "/base/python_lib/versions/1/google/appengine/api/
datastore.py", line 2055, in _ToDatastoreError
raise errors[err.application_error](err.error_detail)
Timeout
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
