App Rocket looks awesome. Assuming I can start it up in a non-default Python instance (I'm normally Java) the same way I've done for some of the Google-supplied python tools that'll really help.
Thanks! -Casey On Sep 1, 7:23 pm, Waleed Abdulla <[email protected]> wrote: > Another tool I used in the past to move my data into GAE (it works both > ways) is App Rocket. It uses timestamps to do real-time replication between > GAE and mysql. You might need to tweak it a little bit to fit your need > exactly, but it gets you most of the way there. > > https://github.com/k7d/approcket/wiki/ > > > > > > > > On Thu, Sep 1, 2011 at 10:15 AM, Daniel <[email protected]> wrote: > > We will be facing the same problem as we leave app engine. I believe > > I'm going to attempt to use the bulk downloader to get the data to a > > local computer. I am slightly worried about the cost of transferring > > 11TB. I believe that the process make take a couple of days, so I plan > > on moving inbound records to the new database before I start the d/l, > > that way I can have reduced functionality over the data transfer > > period but no data will be lost. > > > -Daniel > > > On Sep 1, 8:22 am, Millisecond <[email protected]> wrote: > > > Won't rehash the pricing discussions, but because of the pricing > > > changes and the way it's been handled we're 90% sure we're going to > > > move to AWS. > > > > Anyways... now I'm tasked with figuring out how to get our almost TB > > > of data over there either into SimpleDB or into an RDS instance, not > > > sure yet. > > > > Have other people done the move, how did it go? Did you pull from > > > AWS / push from GAE? Take an intermediate backup and then load from > > > some other mechanism (thinking S3) into SDB / RDS? > > > > Our app is adding over 1 GB / day with 20MM reads and 20MM writes, and > > > we'd rather not take it offline for too long... Thinking of some > > > crazy scheme based on descending keys to move data over from key A > > > "downwards", shut off app, move over everything from key A "upwards" > > > as we're mostly only writing new data. Maybe special-case a few > > > object classes and use timestamps to detect deltas. Realize it's > > > pretty app-specific, but wondering if other people have tackled that > > > same problem and what their experiences were. > > > > Thanks for any and all help, > > > -Casey > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
