I've was able to restore most of my model objects from a local dev
instance to production using bulkloader.py --dump and --restore, but
my last Model object is timing out:
bulkloader.py --restore --kind=RecommendationIndex --url=http://
<appid>.appspot.com/remote_api --filename=RecommendationIndex.bulk --
debug <app-dir>
[INFO ] Logging to bulkloader-log-20100106.140340
[INFO ] Throttling transfers:
[INFO ] Bandwidth: 250000 bytes/second
[INFO ] HTTP connections: 8/second
[INFO ] Entities inserted/fetched/modified: 20/second
[INFO ] Opening database: bulkloader-progress-20100106.140340.sql3
[DEBUG ] [Thread-1] RestoreThread: started
[DEBUG ] [Thread-1] RestoreThread: exiting
[DEBUG ] [Thread-2] RestoreThread: started
[DEBUG ] [Thread-3] WorkerThread: started
[DEBUG ] [Thread-5] WorkerThread: started
[DEBUG ] [Thread-4] WorkerThread: started
[DEBUG ] [Thread-6] WorkerThread: started
[DEBUG ] [Thread-7] WorkerThread: started
[DEBUG ] [Thread-8] WorkerThread: started
[DEBUG ] [Thread-9] WorkerThread: started
[DEBUG ] [Thread-10] WorkerThread: started
[DEBUG ] [Thread-11] WorkerThread: started
[DEBUG ] Configuring remote_api. url_path = /remote_api, servername
= <>.appspot.com
[DEBUG ] [Thread-12] WorkerThread: started
[DEBUG ] Bulkloader using app_id: <>
[INFO ] Connecting to <>.appspot.com/remote_api
Traceback (most recent call last):
File "/usr/local/bin/bulkloader.py", line 67, in <module>
run_file(__file__, globals())
File "/usr/local/bin/bulkloader.py", line 63, in run_file
execfile(script_path, globals_)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 3960, in <module>
sys.exit(main(sys.argv))
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 3956, in main
return _PerformBulkload(arg_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 3814, in _PerformBulkload
return_code = app.Run()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 3125, in Run
self.RunPostAuthentication()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 3230, in
RunPostAuthentication
self.request_manager.IncrementId(list(ancestor_path), kind,
high_id)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 1197, in IncrementId
start, end = datastore.AllocateIds(model_key, 1)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/datastore.py", line 1377, in AllocateIds
raise _ToDatastoreError(err)
google.appengine.api.datastore_errors.Timeout
The data set isn't particularly large (the dump is about 8 megs), and
it has the same number of entries and another model object (~5000)
that restored fine. It's an 'index' object as recommended in the
Google I/O talk, so has some rather large StringListProperty values,
but never more than 500 short strings in a particular object, and of
course the exception occurs before it's actually transferring any
data.
If anyone has any ideas on what may be causing this, I'd love to hear
them. It's always the same exception (datastore.AllocateIds), I guess
I could sprinkle in some logging statements around there in
bulkloader.py, but figured I'd ask here first. I tried --dry_run and
got the standard datastore_v3 error, so it's definitely datastore
related.
Thanks,
Ryan
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
