What I used to do was to use the mapper framework found in GAE articles.
After the task queue was released I found it a lot easier to code and write
tests for a simple handler that updates one at a time. Basically the handler
has a get that is accessible only from admin and get enqueues a task for the
first item in the kind you want to upgrade. The post does the upgrade of the
passed key and finds the next in line to upgrade and queues a task for that.
When there is no entities left to process I send an email to myself that
this is done.
It looks like this:
class Upgrade(webapp.RequestHandler):
def get(self):
self.response.out.write('Started rewriting stuff')
next = MyModel.all(keys_only=True).get()
upgrader_queue.add(
taskqueue.Task(url='/admin/upgrade_mymodel/', params={'next':
str(next)}))
def post(self):
key = db.Key(self.request.get('next'))
next = MyModel.all(keys_only=True).filter('__key__ >', key).get()
def txn():
entity = db.get(key)
if entity:
// do stuff
if next:
upgrader_queue.add(
taskqueue.Task(url='/admin/upgrade_mymodel/', params={'next':
next}))
else:
// mail myself that upgrade is done!!!
db.run_in_transaction(txn)
This has a race condition: if an entity is added during the time you perform
the upgrade you will not process it. This is ok if all "new" entities do not
need to be upgraded which is often the case.
On Mon, Jul 27, 2009 at 3:56 PM, Alexander Tereshkin
<[email protected]>wrote:
>
> Hello All,
>
> Could you please share your wisdom on the following issue: I have
> added a new property to a model and would like to filter the data on
> this property. However, I already have quite a bit of data stored and
> all the entities that were created before the last upgrade don't have
> the new property and hence won't be returned by the queries once the
> filter is added.
> How are you dealing with such situations? Is there some sort of best
> practice here or maybe I just missed something in the docs? Since
> there are no batch updates, it looks like the only option is to cycle
> through all the entities and update them one by one. But then again
> there's too much data to do it in a single request, so I should
> process only this much entities at a time and enqueue the rest of
> processing.
> It just sounds like too much code for something this simple. I'm
> considering writing a generic routine in python to handle this type of
> upgrades, but wanted to check first if I'm missing anything or if such
> routine already exists.
>
> Thanks,
> Alex
>
> >
>
--
Alkis
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
