Hello,
I am using s3ql 2.15 with the swiftks backend with OVH public cloud
storage ( https://www.ovh.com/us/cloud/storage/object-storage.xml ) and
get a ton of log messages of the kind:
|...
2015-11-17 15:58:48.109 3828:Thread-15 s3ql.backends.common.wrapped: Had to
retry 840 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.147 3828:Thread-17 s3ql.backends.common.wrapped: Had to
retry 841 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.197 3828:Thread-18 s3ql.backends.common.wrapped: Had to
retry 842 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.243 3828:Thread-14 s3ql.backends.common.wrapped: Had to
retry 843 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.293 3828:Thread-19 s3ql.backends.common.wrapped: Had to
retry 844 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.385 3828:Thread-18 s3ql.backends.common.wrapped: Had to
retry 845 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.439 3828:Thread-16 s3ql.backends.common.wrapped: Had to
retry 846 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.513 3828:Thread-17 s3ql.backends.common.wrapped: Had to
retry 847 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.523 3828:Thread-11 s3ql.backends.common.wrapped: Had to
retry 848 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.541 3828:Thread-12 s3ql.backends.common.wrapped: Had to
retry 849 times over the last 60 seconds, server or network problem?
2015-11-17 15:58:48.739 3828:Thread-13 s3ql.backends.common.wrapped: Had to
retry 838 times over the last 60 seconds, server or network problem?
...
|
To investigate what the problem might be, I started |mount.s3ql| with
the option |--debug-modules=s3ql.backend|. This gets me logs like this:
|...
2015-11-17 16:48:35.042 28527:Thread-4 s3ql.backends.swift.open_write: started
with s3ql_data_881
2015-11-17 16:48:35.279 28527:Thread-3 s3ql.backends.common.get_ssl_context:
Reading default CA certificates.
2015-11-17 16:48:35.280 28527:Thread-3 s3ql.backends.swift._do_request: started
with 'GET', '/', None, {'limit': 1}, None, None
2015-11-17 16:48:35.280 28527:Thread-3 s3ql.backends.swift._do_request: no
active connection, calling _get_conn()
2015-11-17 16:48:35.281 28527:Thread-3 s3ql.backends.swiftks._get_conn: started
2015-11-17 16:48:35.464 28527:Thread-3 s3ql.backends.swift._do_request_inner:
started with GET /v1/AUTH_prefix/bucket-name/?limit=1
2015-11-17 16:48:35.609 28527:Thread-3 s3ql.backends.swift.open_write: started
with s3ql_data_882
2015-11-17 16:48:37.029 28527:Thread-4 s3ql.backends.s3c.close: started with
s3ql_data_881
2015-11-17 16:48:37.030 28527:Thread-4 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_881', None, None,
CaseInsensitiveDict({'X-Object-meta-001': "'signature':
b'XyhMUIquBROvVR0cTwplHWrQEYsAFSNfdGbaBgu5Cnw=',", 'X-Object-meta-format':
'raw2', 'X-Object-meta-003': "'nonce': b'mZasTnMzcWxfZGF0YV84ODE=',",
'X-Object-meta-002': "'format_version': 2,", 'Content-Type':
'application/octet-stream', 'X-Object-meta-004': "'compression': 'ZLIB',",
'X-Object-meta-006': "'encryption': 'AES_v2',", 'X-Object-meta-md5':
'4s4WbAqYPNX+Qvj3CQBlSg==', 'X-Object-meta-000': "'object_id':
's3ql_data_881',", 'X-Object-meta-005': "'data': b'cVX2AQ==',"}), <_io.FileIO
name=14 mode='rb+'>
2015-11-17 16:48:37.031 28527:Thread-4 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_881
2015-11-17 16:48:37.032 28527:Thread-4 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:37.036 28527:Thread-4 s3ql.backends.common.wrapped: Average
retry rate: 0.02 Hz
2015-11-17 16:48:37.036 28527:Thread-4 s3ql.backends.common.wrapped:
Encountered ConnectionClosed (connection closed unexpectedly), retrying
ObjectW.close (attempt 1)...
2015-11-17 16:48:37.057 28527:Thread-4 s3ql.backends.s3c.close: started with
s3ql_data_881
2015-11-17 16:48:37.057 28527:Thread-4 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_881', None, None,
CaseInsensitiveDict({'X-Object-meta-format': 'raw2', 'X-Object-meta-002':
"'format_version': 2,", 'Expect': '100-continue', 'X-Object-meta-004':
"'compression': 'ZLIB',", 'X-Object-meta-006': "'encryption': 'AES_v2',",
'Host': 'storage.gra1.cloud.ovh.net', 'X-Object-meta-000': "'object_id':
's3ql_data_881',", 'X-Object-meta-005': "'data': b'cVX2AQ==',",
'X-Object-meta-001': "'signature':
b'XyhMUIquBROvVR0cTwplHWrQEYsAFSNfdGbaBgu5Cnw=',", 'X-Object-meta-003':
"'nonce': b'mZasTnMzcWxfZGF0YV84ODE=',", 'Accept-Encoding': 'identity',
'Content-Length': '10443847', 'X-Auth-Token':
'dad22037e66b4809ab4267f60279b061', 'Content-Type': 'application/octet-stream',
'Connection': 'keep-alive', 'X-Object-meta-md5': '4s4WbAqYPNX+Qvj3CQBlSg=='}),
<_io.FileIO name=14 mode='rb+'>
2015-11-17 16:48:37.058 28527:Thread-4 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_881
2015-11-17 16:48:37.089 28527:Thread-4 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:37.121 28527:Thread-4 s3ql.backends.swift._do_request_inner:
writing body data
2015-11-17 16:48:37.612 28527:Thread-3 s3ql.backends.s3c.close: started with
s3ql_data_882
2015-11-17 16:48:37.612 28527:Thread-3 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_882', None, None,
CaseInsensitiveDict({'X-Object-meta-001': "'signature':
b'1H51uVLjykF4Tr5pO26drLxPWANBc7++atVnNb5yNQc=',", 'X-Object-meta-format':
'raw2', 'X-Object-meta-003': "'nonce': b'mZasTnMzcWxfZGF0YV84ODI=',",
'X-Object-meta-002': "'format_version': 2,", 'Content-Type':
'application/octet-stream', 'X-Object-meta-004': "'compression': 'ZLIB',",
'X-Object-meta-006': "'encryption': 'AES_v2',", 'X-Object-meta-md5':
'lrUrSe4D1qZceBsZGKglEQ==', 'X-Object-meta-000': "'object_id':
's3ql_data_882',", 'X-Object-meta-005': "'data': b'aDwO/A==',"}), <_io.FileIO
name=16 mode='rb+'>
2015-11-17 16:48:37.613 28527:Thread-3 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_882
2015-11-17 16:48:37.614 28527:Thread-3 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:37.614 28527:Thread-3 s3ql.backends.common.wrapped: Average
retry rate: 0.03 Hz
2015-11-17 16:48:37.615 28527:Thread-3 s3ql.backends.common.wrapped:
Encountered ConnectionClosed (connection closed unexpectedly), retrying
ObjectW.close (attempt 1)...
2015-11-17 16:48:37.635 28527:Thread-3 s3ql.backends.s3c.close: started with
s3ql_data_882
2015-11-17 16:48:37.636 28527:Thread-3 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_882', None, None,
CaseInsensitiveDict({'X-Object-meta-format': 'raw2', 'X-Object-meta-002':
"'format_version': 2,", 'Expect': '100-continue', 'X-Object-meta-004':
"'compression': 'ZLIB',", 'X-Object-meta-006': "'encryption': 'AES_v2',",
'Host': 'storage.gra1.cloud.ovh.net', 'X-Object-meta-000': "'object_id':
's3ql_data_882',", 'X-Object-meta-005': "'data': b'aDwO/A==',",
'X-Object-meta-001': "'signature':
b'1H51uVLjykF4Tr5pO26drLxPWANBc7++atVnNb5yNQc=',", 'X-Object-meta-003':
"'nonce': b'mZasTnMzcWxfZGF0YV84ODI=',", 'Accept-Encoding': 'identity',
'Content-Length': '10092481', 'X-Auth-Token':
'363937bc5676476aa4842c44ad6e9800', 'Content-Type': 'application/octet-stream',
'Connection': 'keep-alive', 'X-Object-meta-md5': 'lrUrSe4D1qZceBsZGKglEQ=='}),
<_io.FileIO name=16 mode='rb+'>
2015-11-17 16:48:37.636 28527:Thread-3 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_882
2015-11-17 16:48:37.663 28527:Thread-3 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:37.728 28527:Thread-3 s3ql.backends.swift._do_request_inner:
writing body data
2015-11-17 16:48:38.348 28527:Thread-4 s3ql.backends.swift.open_write: started
with s3ql_data_883
2015-11-17 16:48:38.760 28527:Thread-3 s3ql.backends.swift.open_write: started
with s3ql_data_884
2015-11-17 16:48:40.180 28527:Thread-4 s3ql.backends.s3c.close: started with
s3ql_data_883
2015-11-17 16:48:40.181 28527:Thread-4 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_883', None, None,
CaseInsensitiveDict({'X-Object-meta-001': "'signature':
b'QgPiMLuUHSq97OWRCEkcfvBW1TNH2Z9ZEeKmyEI1eT4=',", 'X-Object-meta-format':
'raw2', 'X-Object-meta-003': "'nonce': b'mZasTnMzcWxfZGF0YV84ODM=',",
'X-Object-meta-002': "'format_version': 2,", 'Content-Type':
'application/octet-stream', 'X-Object-meta-004': "'compression': 'ZLIB',",
'X-Object-meta-006': "'encryption': 'AES_v2',", 'X-Object-meta-md5':
'mXTd0E/QCELkF63Q8LHEAA==', 'X-Object-meta-000': "'object_id':
's3ql_data_883',", 'X-Object-meta-005': "'data': b'rCZTAg==',"}), <_io.FileIO
name=14 mode='rb+'>
2015-11-17 16:48:40.181 28527:Thread-4 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_883
2015-11-17 16:48:40.182 28527:Thread-4 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:40.183 28527:Thread-4 s3ql.backends.common.wrapped: Average
retry rate: 0.05 Hz
2015-11-17 16:48:40.183 28527:Thread-4 s3ql.backends.common.wrapped:
Encountered ConnectionClosed (connection closed unexpectedly), retrying
ObjectW.close (attempt 1)...
2015-11-17 16:48:40.203 28527:Thread-4 s3ql.backends.s3c.close: started with
s3ql_data_883
2015-11-17 16:48:40.204 28527:Thread-4 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_883', None, None,
CaseInsensitiveDict({'X-Object-meta-format': 'raw2', 'X-Object-meta-002':
"'format_version': 2,", 'Expect': '100-continue', 'X-Object-meta-004':
"'compression': 'ZLIB',", 'X-Object-meta-006': "'encryption': 'AES_v2',",
'Host': 'storage.gra1.cloud.ovh.net', 'X-Object-meta-000': "'object_id':
's3ql_data_883',", 'X-Object-meta-005': "'data': b'rCZTAg==',",
'X-Object-meta-001': "'signature':
b'QgPiMLuUHSq97OWRCEkcfvBW1TNH2Z9ZEeKmyEI1eT4=',", 'X-Object-meta-003':
"'nonce': b'mZasTnMzcWxfZGF0YV84ODM=',", 'Accept-Encoding': 'identity',
'Content-Length': '10443449', 'X-Auth-Token':
'dad22037e66b4809ab4267f60279b061', 'Content-Type': 'application/octet-stream',
'Connection': 'keep-alive', 'X-Object-meta-md5': 'mXTd0E/QCELkF63Q8LHEAA=='}),
<_io.FileIO name=14 mode='rb+'>
2015-11-17 16:48:40.204 28527:Thread-4 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_883
2015-11-17 16:48:40.233 28527:Thread-4 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:40.334 28527:Thread-4 s3ql.backends.swift._do_request_inner:
writing body data
2015-11-17 16:48:40.759 28527:Thread-3 s3ql.backends.s3c.close: started with
s3ql_data_884
2015-11-17 16:48:40.760 28527:Thread-3 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_884', None, None,
CaseInsensitiveDict({'X-Object-meta-001': "'signature':
b'IvikaA8JFSaEZQlTlnSwpZpAu34CapNbcPRCy/VqbZw=',", 'X-Object-meta-format':
'raw2', 'X-Object-meta-003': "'nonce': b'mZasTnMzcWxfZGF0YV84ODQ=',",
'X-Object-meta-002': "'format_version': 2,", 'Content-Type':
'application/octet-stream', 'X-Object-meta-004': "'compression': 'ZLIB',",
'X-Object-meta-006': "'encryption': 'AES_v2',", 'X-Object-meta-md5':
'djAK0m/+Yy8ZWfQ32X+Hxw==', 'X-Object-meta-000': "'object_id':
's3ql_data_884',", 'X-Object-meta-005': "'data': b'1Km3Yg==',"}), <_io.FileIO
name=16 mode='rb+'>
2015-11-17 16:48:40.768 28527:Thread-3 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_884
2015-11-17 16:48:40.769 28527:Thread-3 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:40.770 28527:Thread-3 s3ql.backends.common.wrapped: Average
retry rate: 0.07 Hz
2015-11-17 16:48:40.771 28527:Thread-3 s3ql.backends.common.wrapped:
Encountered ConnectionClosed (connection closed unexpectedly), retrying
ObjectW.close (attempt 1)...
2015-11-17 16:48:40.792 28527:Thread-3 s3ql.backends.s3c.close: started with
s3ql_data_884
2015-11-17 16:48:40.792 28527:Thread-3 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_884', None, None,
CaseInsensitiveDict({'X-Object-meta-format': 'raw2', 'X-Object-meta-002':
"'format_version': 2,", 'Expect': '100-continue', 'X-Object-meta-004':
"'compression': 'ZLIB',", 'X-Object-meta-006': "'encryption': 'AES_v2',",
'Host': 'storage.gra1.cloud.ovh.net', 'X-Object-meta-000': "'object_id':
's3ql_data_884',", 'X-Object-meta-005': "'data': b'1Km3Yg==',",
'X-Object-meta-001': "'signature':
b'IvikaA8JFSaEZQlTlnSwpZpAu34CapNbcPRCy/VqbZw=',", 'X-Object-meta-003':
"'nonce': b'mZasTnMzcWxfZGF0YV84ODQ=',", 'Accept-Encoding': 'identity',
'Content-Length': '10092514', 'X-Auth-Token':
'363937bc5676476aa4842c44ad6e9800', 'Content-Type': 'application/octet-stream',
'Connection': 'keep-alive', 'X-Object-meta-md5': 'djAK0m/+Yy8ZWfQ32X+Hxw=='}),
<_io.FileIO name=16 mode='rb+'>
2015-11-17 16:48:40.793 28527:Thread-3 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_884
2015-11-17 16:48:40.831 28527:Thread-3 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:40.949 28527:Thread-3 s3ql.backends.swift._do_request_inner:
writing body data
2015-11-17 16:48:41.500 28527:Thread-4 s3ql.backends.swift.open_write: started
with s3ql_data_885
2015-11-17 16:48:42.163 28527:Thread-4 s3ql.backends.s3c.close: started with
s3ql_data_885
2015-11-17 16:48:42.163 28527:Thread-4 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_885', None, None,
CaseInsensitiveDict({'X-Object-meta-001': "'signature':
b'ZhQF9yVs9sTJcwJtAudZtaObZPrI7pZ37k37XNGYPKo=',", 'X-Object-meta-format':
'raw2', 'X-Object-meta-003': "'nonce': b'mZasTnMzcWxfZGF0YV84ODU=',",
'X-Object-meta-002': "'format_version': 2,", 'Content-Type':
'application/octet-stream', 'X-Object-meta-004': "'compression': 'ZLIB',",
'X-Object-meta-006': "'encryption': 'AES_v2',", 'X-Object-meta-md5':
'+T1kqsYjro3Gbw6rQDRM3Q==', 'X-Object-meta-000': "'object_id':
's3ql_data_885',", 'X-Object-meta-005': "'data': b'TAD2Aw==',"}), <_io.FileIO
name=14 mode='rb+'>
2015-11-17 16:48:42.164 28527:Thread-4 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_885
2015-11-17 16:48:42.164 28527:Thread-4 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:42.165 28527:Thread-4 s3ql.backends.common.wrapped: Average
retry rate: 0.08 Hz
2015-11-17 16:48:42.165 28527:Thread-4 s3ql.backends.common.wrapped:
Encountered ConnectionClosed (connection closed unexpectedly), retrying
ObjectW.close (attempt 1)...
2015-11-17 16:48:42.186 28527:Thread-4 s3ql.backends.s3c.close: started with
s3ql_data_885
2015-11-17 16:48:42.186 28527:Thread-4 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_885', None, None,
CaseInsensitiveDict({'X-Object-meta-format': 'raw2', 'X-Object-meta-002':
"'format_version': 2,", 'Expect': '100-continue', 'X-Object-meta-004':
"'compression': 'ZLIB',", 'X-Object-meta-006': "'encryption': 'AES_v2',",
'Host': 'storage.gra1.cloud.ovh.net', 'X-Object-meta-000': "'object_id':
's3ql_data_885',", 'X-Object-meta-005': "'data': b'TAD2Aw==',",
'X-Object-meta-001': "'signature':
b'ZhQF9yVs9sTJcwJtAudZtaObZPrI7pZ37k37XNGYPKo=',", 'X-Object-meta-003':
"'nonce': b'mZasTnMzcWxfZGF0YV84ODU=',", 'Accept-Encoding': 'identity',
'Content-Length': '6331301', 'X-Auth-Token':
'dad22037e66b4809ab4267f60279b061', 'Content-Type': 'application/octet-stream',
'Connection': 'keep-alive', 'X-Object-meta-md5': '+T1kqsYjro3Gbw6rQDRM3Q=='}),
<_io.FileIO name=14 mode='rb+'>
2015-11-17 16:48:42.187 28527:Thread-4 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_885
2015-11-17 16:48:42.218 28527:Thread-4 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:42.224 28527:Thread-3 s3ql.backends.swift.open_write: started
with s3ql_data_886
2015-11-17 16:48:42.838 28527:Thread-3 s3ql.backends.s3c.close: started with
s3ql_data_886
2015-11-17 16:48:42.838 28527:Thread-3 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_886', None, None,
CaseInsensitiveDict({'X-Object-meta-001': "'signature':
b'ViNwNnX/e0nbzysh1TVtuS5RfxIPnhdDISPkQO7FsEU=',", 'X-Object-meta-format':
'raw2', 'X-Object-meta-003': "'nonce': b'mZasTnMzcWxfZGF0YV84ODY=',",
'X-Object-meta-002': "'format_version': 2,", 'Content-Type':
'application/octet-stream', 'X-Object-meta-004': "'compression': 'ZLIB',",
'X-Object-meta-006': "'encryption': 'AES_v2',", 'X-Object-meta-md5':
'FDVCNB0xc/4MGJJr2Vif7A==', 'X-Object-meta-000': "'object_id':
's3ql_data_886',", 'X-Object-meta-005': "'data': b'ACsiNw==',"}), <_io.FileIO
name=16 mode='rb+'>
2015-11-17 16:48:42.839 28527:Thread-3 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_886
2015-11-17 16:48:42.839 28527:Thread-3 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:42.840 28527:Thread-3 s3ql.backends.common.wrapped: Average
retry rate: 0.10 Hz
2015-11-17 16:48:42.840 28527:Thread-3 s3ql.backends.common.wrapped:
Encountered ConnectionClosed (connection closed unexpectedly), retrying
ObjectW.close (attempt 1)...
2015-11-17 16:48:42.861 28527:Thread-3 s3ql.backends.s3c.close: started with
s3ql_data_886
2015-11-17 16:48:42.861 28527:Thread-3 s3ql.backends.swift._do_request: started
with 'PUT', '/s3ql_data_886', None, None,
CaseInsensitiveDict({'X-Object-meta-format': 'raw2', 'X-Object-meta-002':
"'format_version': 2,", 'Expect': '100-continue', 'X-Object-meta-004':
"'compression': 'ZLIB',", 'X-Object-meta-006': "'encryption': 'AES_v2',",
'Host': 'storage.gra1.cloud.ovh.net', 'X-Object-meta-000': "'object_id':
's3ql_data_886',", 'X-Object-meta-005': "'data': b'ACsiNw==',",
'X-Object-meta-001': "'signature':
b'ViNwNnX/e0nbzysh1TVtuS5RfxIPnhdDISPkQO7FsEU=',", 'X-Object-meta-003':
"'nonce': b'mZasTnMzcWxfZGF0YV84ODY=',", 'Accept-Encoding': 'identity',
'Content-Length': '6331367', 'X-Auth-Token':
'363937bc5676476aa4842c44ad6e9800', 'Content-Type': 'application/octet-stream',
'Connection': 'keep-alive', 'X-Object-meta-md5': 'FDVCNB0xc/4MGJJr2Vif7A=='}),
<_io.FileIO name=16 mode='rb+'>
2015-11-17 16:48:42.862 28527:Thread-3 s3ql.backends.swift._do_request_inner:
started with PUT /v1/AUTH_prefix/bucket-name/s3ql_data_886
2015-11-17 16:48:42.889 28527:Thread-3 s3ql.backends.swift._do_request_inner:
waiting for 100-continue
2015-11-17 16:48:42.914 28527:Thread-3 s3ql.backends.swift._do_request_inner:
writing body data
2015-11-17 16:48:43.071 28527:Thread-4 s3ql.backends.swift._do_request_inner:
writing body data
...
|
This looks like every |_do_request_inner| call needs to be done twice
because of a |ConnectionClosed| exception.
Since I suspected that the server closes the connection (despite the
client -- the dugong library -- sets the |Connection: keep-alive|
header) I used Cyberduck to connect to the same server. Here are the
HTTP-Headers of one request:
|> GET /v1/AUTH_prefix/bucket-name?format=xml&prefix=&limit=10000&delimiter=%2F
HTTP/1.1
> X-Auth-Token: token
> Host: storage.gra1.cloud.ovh.net
> Connection: Keep-Alive
> User-Agent: Cyberduck/4.7.3.18498 (Mac OS X/10.11.1) (x86_64)
< HTTP/1.1 200 OK
< Content-Length: 1863
< X-Container-Object-Count: 8
< Accept-Ranges: bytes
< X-Storage-Policy: Policy-0
< X-Container-Bytes-Used: 1086
< X-Timestamp: 1447177557.73320
< Content-Type: application/xml; charset=utf-8
< X-Trans-Id: D908325C:C2B8_052711A1:01BB_564B6540_2A19DBC:7EBF
< Date: Tue, 17 Nov 2015 17:34:56 GMT
< Connection: close
|
Cyberduck (like s3ql) sets the |Connection: keep-alive| header, but the
server chooses to not honor that request and sends the |Connection:
close| response header (and subsequently closes the connection, of course).
This is not nice of the server but perfectly valid according to RFC 2616
Section 8.1.2.1 ( http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html )
Am I right in assuming that s3ql can not handle the server choosing to
close the persistent connection and that's what these retry error
messages are?
Thanks,
Daniel
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
