If you delete many entities it's faster to use a cursor. An entity is
marked as deleted and will physically be removed in the datastore at a
later time. It takes time to skip the marked entities when executing a
query.
The loop can be modified as such, no need to use count() as it uses
CPU and time also, if the result set is empty there are no entities.
try:
cursor = None
while True:
q = Documents.all(keys_only=True)
if cursor:
q.with_cursor(cursor)
result = q.fetch(200)
if not result:
break
cursor = q.cursor()
db.delete(result)
except Exception, e:
self.response.out.write(repr(e)+'\n')
pass
Op 16 september 2011 18:24 heeft sofia <[email protected]> het
volgende geschreven:
> I have the same problem.. Datastore quota at 100% and unable to delete an
> entity either through admin or using map/reduce. What I did is set up a
> script to delete x records at a time. I've managed to decrease data by 45%
> in 2 days but I then hit cpu quota so I'm guessing it's gonna take a few
> more days until I'm able to delete all the data. What i did was set up this
> script, bulkdelete.py:
> #!/usr/bin/env python
> # -*- coding: UTF-8 -*-
> #
> #
> import time
> from google.appengine.ext import webapp
> from google.appengine.ext.webapp.util import run_wsgi_app
> from google.appengine.ext import db
> from lib.model import Documents
> class BulkDelete(webapp.RequestHandler):
> def get(self):
> self.response.headers['Content-Type'] = 'text/plain'
> mod = self.request.get('m')
> if not mod:
> exit
> try:
> while True:
> q = db.GqlQuery("SELECT __key__ FROM Documents ORDER BY date
> ASC")
> assert q.count()
> db.delete(q.fetch(200))
> time.sleep(0.5)
> except Exception, e:
> self.response.out.write(repr(e)+'\n')
> pass
> # init
> application = webapp.WSGIApplication([('/bulkdelete',
> BulkDelete)],debug=True)
> def main():
> run_wsgi_app(application)
> if __name__ == '__main__':
> main()
> and then call it each 5 min through cron.yaml.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
