We will be facing the same problem as we leave app engine. I believe I'm going to attempt to use the bulk downloader to get the data to a local computer. I am slightly worried about the cost of transferring 11TB. I believe that the process make take a couple of days, so I plan on moving inbound records to the new database before I start the d/l, that way I can have reduced functionality over the data transfer period but no data will be lost.
-Daniel On Sep 1, 8:22 am, Millisecond <[email protected]> wrote: > Won't rehash the pricing discussions, but because of the pricing > changes and the way it's been handled we're 90% sure we're going to > move to AWS. > > Anyways... now I'm tasked with figuring out how to get our almost TB > of data over there either into SimpleDB or into an RDS instance, not > sure yet. > > Have other people done the move, how did it go? Did you pull from > AWS / push from GAE? Take an intermediate backup and then load from > some other mechanism (thinking S3) into SDB / RDS? > > Our app is adding over 1 GB / day with 20MM reads and 20MM writes, and > we'd rather not take it offline for too long... Thinking of some > crazy scheme based on descending keys to move data over from key A > "downwards", shut off app, move over everything from key A "upwards" > as we're mostly only writing new data. Maybe special-case a few > object classes and use timestamps to detect deltas. Realize it's > pretty app-specific, but wondering if other people have tackled that > same problem and what their experiences were. > > Thanks for any and all help, > -Casey -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
