That's all well and good, but since the datastore docs encouraged using keys in URLs, several of my apps have taken that advice and externalized the keys into other application's databases. I'd guess a lot of my keys are indexed in google's search engine! So the problem is that now I'll either send other apps keys that no longer match the ones I used to generate for a given entity; or I'll get keys in from those apps that aren't for the current application at all.
That means that migrating to HR gives me this extra headache of translating all the keys going in and out the door to deal with this history. Our hope would be that you could do some magic down in the db API to let us continue to use the old App ID, even after migration. Let me give you a concrete example: The third hit when I google "barre zoning board of appeals joshua smith" is http://www.mytowngovernment.org/download?document=ag50b3duZ292ZXJubWVudHIVCxINRG9jdW1lbnRNb2RlbBjrvBgM That's a key. So if I port this site to HR, I'm going to either get an error when someone hits this link, or I have to put sanitization on EVERY HANDLER that takes a key to make sure the key is referencing my app's new, ugly -hr name. A good start would be if you could just update .get() to support doing the mapping from old->new automatically. For example, it'd be cool if there was a way to say: db.always_substitute_the_right_app_id_into_keys_that_I_pass_you_please(), or db.when_you_see_a_key_that_says_appX_make_it_appY(). Of course, it'd be much cooler if I could just keep using the same app ID when I switch to HR. -Joshua On Jul 8, 2011, at 3:17 PM, Ikai Lan (Google) wrote: > The tool *should* preserve keys that are stored as straight keys. That is - > Keys stored in list, String or reference properties. The tool should read > these, decode them, change the app ID (so you don't get the exception about > reading data from a different app ID), and resave these to the new datastore. > > What the tool does NOT do is smart decoding of keys. Some people do things > like sha1 the keys; this functionality will 100% break since the keys will be > different. Remember that keys encode the application ID in them. If the > application ID changes, the hashes will change. > > The tools will not allow migration within the same app ID per se. You will > need to create a new app ID. The tools will allow you to map an alias from > the old app ID to the new one, so for any routing intents and purposes, the > old app ID will still point to the new old. Internally, however, the new app > ID will be stored. This matters if you depend on any functionality that reads > the current app ID from system environment variables in production. All new > HR apps, for instance, prefix a s~ before the app ID. > > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog: http://googleappengine.blogspot.com > Twitter: http://twitter.com/app_engine > Reddit: http://www.reddit.com/r/appengine > > > > On Thu, Jul 7, 2011 at 11:55 PM, Tom Phillips <[email protected]> wrote: > Greg, can you reveal in advance whether the new tools will allow for > migration within the same app id and/or preserve encoded entity keys? > > /Tom > > On Jul 6, 8:01 pm, "Gregory D'alesandre" <[email protected]> wrote: > > We are working on better tools for migrating to HRD (and they are currently > > being tested), I'll post once we have them widely available. > > > > Greg > > > > On Wed, Jul 6, 2011 at 5:52 PM, Waleed Abdulla <[email protected]> wrote: > > > Please star this issue if you agree that Google should make the migration > > > process easier rather than putting the burdon on the developers. After > > > all, > > > most developers signed up to GAE to avoid having to deal with issues like > > > that. > > > > >http://code.google.com/p/googleappengine/issues/detail?id=5250 > > > > > On Wed, Jul 6, 2011 at 8:39 AM, Robert Kluin > > > <[email protected]>wrote: > > > > >> Keys do contain the appid. > > > > >> One solution would be to adjust your code to catch the exception that > > >> gets thrown when "accessing the old apps data", the recreate the key > > >> for the new appid. > > > > >> Robert > > > > >> On Wednesday, July 6, 2011, Joshua Smith <[email protected]> > > >> wrote: > > >> > Another thing that occurs to me is that anyplace I've used a key > > >> external to the application would have to be dealt with. For example, > > >> one > > >> of our apps has an RSS feed that our managers use to keep track of > > >> whether > > >> customers have uploaded new information. This looks like: > > > > >> >http://myapp.com/rss?cust=DAFHP983RPFDSALFHDSKLFJHLSDKAFH > > > > >> > (where the gobledegook is the datastore key) > > > > >> > My managers would need to update all their bookmarks, since I believe > > >> that key has the name of the app pickled in it someplace, right? > > > > >> > Is there any way to avoid having the keys change when I switch to HR? > > > > >> > If not, is there a way that I could decorate database queries so that > > >> they quietly turn old keys into new ones? > > > > >> > -Joshua > > > > >> > On Jul 5, 2011, at 3:35 PM, Joshua Smith wrote: > > > > >> >> I have several apps and I've got this pesky to-do list item for all of > > >> them, to switch them over to the HR datastore. > > > > >> >> In preparation, I've read blog entries by people who have done this, > > >> and I'm still on the fence about whether it is worth the trouble. > > > > >> >> I get very few datastore timeouts in my apps, and periodic maintenance > > >> has never really been an issue. > > > > >> >> I've read about gotchas with eventual consistency, and it seems > > >> unlikely that I could convince myself that I'm safe from this for any > > >> reasonably complex app. (I'm not concerned that I cannot reprogram the > > >> apps > > >> to fix the problem, but rather I'm concerned that I could not ensure > > >> there > > >> weren't going to be problems. If it ain't broke...) > > > > >> >> I'm not clear on how I would go about getting google to grandfather my > > >> old 2000 emails quotas if I create a new -hr version of my app. > > > > >> >> I've also read that migrating uses TONS of CPU, and it is therefore > > >> going to cost me money to do. > > > > >> >> Really, the only reason TO do the switch is that I've read repeated > > >> admonitions from google that I *should* be using HR. > > > > >> >> What's the consensus here? Is it worth the time, risk, and cost? > > > > >> >> -Joshua > > > > >> >> -- > > >> >> You received this message because you are subscribed to the Google > > >> Groups "Google App Engine" group. > > >> >> To post to this group, send email to [email protected] > > >> . > > >> >> To unsubscribe from this group, send email to > > >> [email protected]. > > >> >> For more options, visit this group at > > >>http://groups.google.com/group/google-appengine?hl=en. > > > > >> > -- > > >> > You received this message because you are subscribed to the Google > > >> Groups "Google App Engine" group. > > >> > To post to this group, send email to [email protected]. > > >> > To unsubscribe from this group, send email to > > >> [email protected]. > > >> > For more options, visit this group at > > >>http://groups.google.com/group/google-appengine?hl=en. > > > > >> -- > > >> ------ > > >> Robert Kluin > > >> Ezox Systems, LLC > > > > >> -- > > >> You received this message because you are subscribed to the Google Groups > > >> "Google App Engine" group. > > >> To post to this group, send email to [email protected]. > > >> To unsubscribe from this group, send email to > > >> [email protected]. > > >> For more options, visit this group at > > >>http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Google App Engine" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > > [email protected]. > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
