I had been using python 2.6. I just upgraded to 2.7.1 and I get the same problem. I just downgraded to 2.5 per your suggestion, same problem. I've tried limiting threads, increasing batch size, nothing seems to work. The other weird issue is that I can only run this command once and it has to be right after I have deleted the local datastore files and restarted the app. Otherwise, if I try to run the command again I get authentication failed errors. Here's a more verbose log from my latest attempt.
Can anyone get an 'appcfg.py upload_data' command to work with localhost on a Mac? erik:temp erik.kallevig$ appcfg.py upload_data --application=appname --kind=Page --url=http://localhost:8888/remote_api --filename=./dumpfile --noisy --no_cookies --batch_size=10000 --num_threads=1 Uploading data records. [INFO ] Logging to bulkloader-log-20110420.105013 [INFO ] Throttling transfers: [INFO ] Bandwidth: 250000 bytes/second [INFO ] HTTP connections: 8/second [INFO ] Entities inserted/fetched/modified: 20/second [INFO ] Batch Size: 10000 [INFO ] Opening database: bulkloader-progress-20110420.105013.sql3 [DEBUG ] [Thread-1] RestoreThread: started [DEBUG ] [Thread-1] RestoreThread: exiting [DEBUG ] [Thread-2] RestoreThread: started [DEBUG ] [Thread-3] WorkerThread: started [DEBUG ] Configuring remote_api. url_path = /remote_api, servername = localhost:8888 Please enter login credentials for localhost Email: admin Password for admin: [DEBUG ] Bulkloader using app_id: appname [INFO ] Connecting to localhost:8888/remote_api [DEBUG ] [Thread-4] ProgressTrackerThread: started [DEBUG ] [Thread-5] DataSourceThread: started [INFO ] Starting import; maximum 10000 entities per post [DEBUG ] [Thread-2] RestoreThread: exiting [DEBUG ] [Thread-5] DataSourceThread: exiting [DEBUG ] [Thread-3] Got work item [1-1050] [DEBUG ] Waiting for worker threads to finish... [ERROR ] [Thread-3] WorkerThread: Traceback (most recent call last): File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/adaptive_thread_pool.py", line 176, in WorkOnItems status, instruction = item.PerformWork(self.__thread_pool) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 763, in PerformWork transfer_time = self._TransferItem(thread_pool) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 934, in _TransferItem self.request_manager.PostEntities(self.content) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1393, in PostEntities datastore.Put(entities) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 455, in Put return _GetConnection().async_put(config, entities, extra_hook).get_result() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_rpc.py", line 629, in get_result self.check_success() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_rpc.py", line 599, in check_success rpc.check_success() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 558, in check_success self.__rpc.CheckSuccess() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_rpc.py", line 156, in _WaitImpl self.request, self.response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 248, in MakeSyncCall handler(request, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 386, in _Dynamic_Put 'datastore_v3', 'Put', put_request, put_response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 177, in MakeSyncCall self._MakeRealSyncCall(service, call, request, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 189, in _MakeRealSyncCall encoded_response = self._server.Send(self._path, encoded_request) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appengine_rpc.py", line 358, in Send f = self.opener.open(req) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib2.py", line 380, in open response = meth(req, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/throttle.py", line 526, in http_response self.AddResponse(BANDWIDTH_DOWN, res) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/throttle.py", line 462, in AddResponse content = res.read() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/socket.py", line 290, in read data = self._sock.recv(recv_size) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/httplib.py", line 529, in read s = self.fp.read(amt) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/socket.py", line 308, in read data = self._sock.recv(recv_size) error: (54, 'Connection reset by peer') [DEBUG ] [Thread-3] WorkerThread: exiting [INFO ] An error occurred. Shutting down... [DEBUG ] Waiting for Anonymous_WorkerThread to exit [ERROR ] Error in Thread-3: (54, 'Connection reset by peer') [DEBUG ] Traceback (most recent call last): File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/adaptive_thread_pool.py", line 176, in WorkOnItems status, instruction = item.PerformWork(self.__thread_pool) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 763, in PerformWork transfer_time = self._TransferItem(thread_pool) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 934, in _TransferItem self.request_manager.PostEntities(self.content) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1393, in PostEntities datastore.Put(entities) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 455, in Put return _GetConnection().async_put(config, entities, extra_hook).get_result() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_rpc.py", line 629, in get_result self.check_success() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_rpc.py", line 599, in check_success rpc.check_success() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 558, in check_success self.__rpc.CheckSuccess() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_rpc.py", line 156, in _WaitImpl self.request, self.response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 248, in MakeSyncCall handler(request, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 386, in _Dynamic_Put 'datastore_v3', 'Put', put_request, put_response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 177, in MakeSyncCall self._MakeRealSyncCall(service, call, request, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 189, in _MakeRealSyncCall encoded_response = self._server.Send(self._path, encoded_request) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appengine_rpc.py", line 358, in Send f = self.opener.open(req) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib2.py", line 380, in open response = meth(req, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/throttle.py", line 526, in http_response self.AddResponse(BANDWIDTH_DOWN, res) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/throttle.py", line 462, in AddResponse content = res.read() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/socket.py", line 290, in read data = self._sock.recv(recv_size) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/httplib.py", line 529, in read s = self.fp.read(amt) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/socket.py", line 308, in read data = self._sock.recv(recv_size) error: (54, 'Connection reset by peer') [DEBUG ] Waiting for progress_thread to terminate... [DEBUG ] [Thread-4] ProgressTrackerThread: exiting [DEBUG ] ... done. [INFO ] 1050 entites total, 0 previously transferred [INFO ] 0 entities (1824614 bytes) transferred in 4.7 seconds [INFO ] Some entities not successfully transferred -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
