Eli, I can see a benefit to this approach but isn't there a chance some data would be missed? If customers are using the app while you're testing the new version, wouldn't you need to re-import the data from version 1 and convert it again while the app is down?
On Sep 23, 11:27 am, Eli Jones <[email protected]> wrote: > Yes, you'll need to duplicate your data. > > This is the cleanest way to do this.. in my mind. You would have > two separate versions of your app.. and you could easily switch back to your > old version of you realized some horrible mistake was made after switching > to the new version. > > The main work you would need to do for the changeover from Old Version to > New Version would be: > > 1. Write code that converts old datastore data to the New Version namespace > data. > 2. Test New Version datastore with converted data on New App version. > 3. Schedule a downtime late at night where the app is brought offline (just > upload a dummy app.yaml for the Old Version that points /.* to some message > page.. or you could be official and have a "Maintenance Period" version of > your app that you make live.. so that you make no changes whatsoever to the > "Old Version".), and the datastore conversion code is run after you are > comfortable that no users are using the site and are all viewing the > "maintenance period" messages. > 4. Once complete, make New Version app the live version. > > To me, it seems like a massive headache to try to convert the data in > place.. in the same datastore.. you have to have new model names or at least > some new property for "version" on your models.. and then your key_names > have to be different.. etc.. If you screw something up, that's a potential > headache for your live app.. with the New Version namespace.. you can > freely move ahead with experimenting with your coding.. and even make plenty > of changes to your models (if you think they are beneficial) without > thinking "I don't know.. it's going to be a pain in the ass to figure out > how to keep track of the old version and new version entities and now I have > to add a new model or property definition to my schema code.." > > With a namespace, you just have to add the correct mapping, conversion code > to your datastore converter code, and make schema changes to your New > Version app code.. and just leave the Old Version datastore and code alone. > > You can also get fancy with your converter.. (depending on how the > conversion actually works) and you can have your converter code get a cursor > with keys_only for each model, serialize that cursor and pass it off to a > task that then iterates through the cursor and breaks up the keys into > batches to be converted and fires off tasks to do the batches in parallel. > And, to make the process simpler, you could just used the deferred api.. > (so you don't have to deal with setting up task handler urls or anything > like that). > > However big your datastore is.. and however much money it might cost to have > your data duplicated (it costs $0.01 a day for 2 GB of data above the free > limit).. I think it would be worth it.. it would be valuable to learn to do > it this way, since a large scale site would (or should) do something more > like this. > > > > On Wed, Sep 22, 2010 at 9:39 PM, Kyle Baley <[email protected]> wrote: > > That's an interesting idea. But if you do that, wouldn't you need to > > first copy all the data from the v1 namespace to the v2 namespace? > > > On Sep 22, 7:07 pm, Eli Jones <[email protected]> wrote: > > > It might be useful for you to use a namespace for the new version of the > > > datastore. > > > > Thus, you could have the "new version" of the app deployed as a non-live > > > version of the app.. and code that "new version" to use the "new version > > > datastore" namespace. > > > > Then, when you are ready.. just change the live version of your app to > > the > > > "new version". > > > > Here's a link: > > > >http://code.google.com/appengine/docs/python/multitenancy/multitenanc... > > > > <http://code.google.com/appengine/docs/python/multitenancy/multitenanc.. > > .>So.. > > > you could just think of your version 1 datastore as "that old customer > > who > > > we're going to dump just as soon as our new version 2 datastore customer > > is > > > ready"... or something like that. > > > > It's better (I think) than adding a "version" property to all of your > > models > > > or trying to maintain model consistency between app versions. > > > > On Mon, Sep 20, 2010 at 2:29 PM, Kyle Baley <[email protected]> wrote: > > > > We've just released the first version of our application and are now > > > > looking at a problem we've been avoiding until now. Namely, what is > > > > the best way to upgrade the application to a new version that requires > > > > changes to the datastore. We're looking at two options: > > > > > 1) Big Bang Upgrade > > > > We take the application down and run an upgrade process to update all > > > > entities from version 1 to version 2. > > > > > Pros: Easy to maintain; intuitive > > > > Cons: App has to be taken down for a period of time, which will > > > > increase as time passes and more data is added to the datastore > > > > (potentially hitting the limit for long-running processes eventually) > > > > Question: What's a good way to take the app offline? > > > > > 2) Version Entities Individually > > > > Each entity has a version number and we have a series of commands, > > > > each one responsible for upgrading an entity from one version to the > > > > next. As we request entities, we check to see if it's the latest > > > > version. If not, we run each necessary upgrade command in sequence > > > > until it is the latest version. > > > > > Pros: No need to take the app offline; provides flexibility on whether > > > > to upgrade everything at once or piecemeal > > > > Cons: Not as intuitive; entities with different versions in the > > > > datastore (if that matters) > > > > > What do other people do to upgrade their datastore for a live > > > > application? > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to [email protected] > > . > > > > To unsubscribe from this group, send email to > > > > [email protected]<google-appengine%2Bunsubscrib > > > > [email protected]><google-appengine%2Bunsubscrib > > [email protected]> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%2Bunsubscrib > > [email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
