I have about 2GB of data that I was copying between applications while porting from a master/slave datastore to a high replication one and the process has completely stopped in the middle.
It's been about 12 hours so far and about $80 worth of CPU, and my mappers are stuck in a running state. There are five entity mappers that aren't doing anything that say they are still running. For the first 2 hours of the copy it looked like there was consistent progress and the number of running jobs kept going down and the number of successful shards kept going up, which was of course good. But now there are 5 mapper jobs left that haven't changed progress in 10 hours. The major ones still show about 3 shards left to complete and one shows 8 shards left to complete (out of 32 shards). So, do I need to start over with my copy? I'm ok with that except that I really don't want to spend another $80 just because it broke the first time. And does anyone know why the mapreduce jobs would just go idle? I'm afraid that if I start it over then the same thing will happen and I'll be out another $80 without any real progress. (updated: I just retried a few of the entities again and each of the 5 types of entities gets stuck at the same point, One entity I tried always gets stuck at 3 shards left out of 32 and another one instantly gets stuck on 1 shard out of 1 shard. I thought maybe the updates had gone through even though they said they didn't, but when I look at the datastore admin it shows a lower count for the number of entities and when I go to the app it throws an error saying a reference property failed to resolve, when the original app doesn't have that error with the same request) A side question: It looks like blobstore values are not copied over and I don't see a way to do that. Is there any way currently to do that? We have about 9GB of blobstore values that we need copied over and a lot of them are over 1MB in size so we can't use the urlfetch trick to do it. Please help as soon as you can because our database is stuck in read only mode until we can figure this out. We would rather not pay another $80 or more later by going back to the the master/slave database. Thanks. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
