I'm setting up S3QL 3.0 (from Debian's stretch-backports) + google-auth
installed via "pip3 install --user google-auth" to store some backups from
a Google Compute Engine instance on Google Cloud Storage (in a nearline
bucket). Authentication is with a service account, by setting
GOOGLE_APPLICATION_CREDENTIALS to the JSON key file. The filesystem creates
and mounts without any errors.
After a while (processed the better part of 40GB of data), S3QL crashes,
messages below.
I just fired off an fsck, which appears to be continuing the upload
(writing a lot of messages like WARNING: Writing dirty block 4176 of inode
6 to backend).
Crash messages:
Mar 12 23:55:43 taniguchi mount.s3ql[5294]: mount.s3ql[5294:Thread-3]
root.excepthook: Uncaught top-level exception:
Traceback (most recent call
last):
File
"/usr/lib/s3ql/s3ql/mount.py", line 58, in run_with_except_hook
run_old(*args, **kw)
File
"/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args,
**self._kwargs)
File
"/usr/lib/s3ql/s3ql/block_cache.py", line 445, in _upload_loop
self._do_upload(*tmp)
File
"/usr/lib/s3ql/s3ql/block_cache.py", line 472, in _do_upload
% obj_id).get_obj_size()
File
"/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
return method(*a, **kw)
File
"/usr/lib/s3ql/s3ql/backends/common.py", line 279, in perform_write
return fn(fh)
File
"/usr/lib/s3ql/s3ql/backends/comprenc.py", line 389, in __exit__
self.close()
File
"/usr/lib/s3ql/s3ql/backends/comprenc.py", line 383, in close
self.fh.close()
File
"/usr/lib/s3ql/s3ql/backends/gs.py", line 933, in close
self.metadata,
size=self.obj_size)
File
"/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
return method(*a, **kw)
File
"/usr/lib/s3ql/s3ql/backends/gs.py", line 485, in write_fh
raise
_map_request_error(exc, key) or exc
s3ql.backends.gs.RequestError:
<RequestError, code=401, reason='Unauthorized', message='Invalid
Credentials'>
Mar 12 23:58:41 taniguchi mount.s3ql[5294]: mount.s3ql[5294:Thread-4]
root.excepthook: Uncaught top-level exception:
Traceback (most recent call
last):
File
"/usr/lib/s3ql/s3ql/mount.py", line 58, in run_with_except_hook
run_old(*args, **kw)
File
"/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args,
**self._kwargs)
File
"/usr/lib/s3ql/s3ql/block_cache.py", line 445, in _upload_loop
self._do_upload(*tmp)
File
"/usr/lib/s3ql/s3ql/block_cache.py", line 472, in _do_upload
% obj_id).get_obj_size()
File
"/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
return method(*a, **kw)
File
"/usr/lib/s3ql/s3ql/backends/common.py", line 279, in perform_write
return fn(fh)
File
"/usr/lib/s3ql/s3ql/backends/comprenc.py", line 389, in __exit__
self.close()
File
"/usr/lib/s3ql/s3ql/backends/comprenc.py", line 383, in close
self.fh.close()
File
"/usr/lib/s3ql/s3ql/backends/gs.py", line 933, in close
self.metadata,
size=self.obj_size)
File
"/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
return method(*a, **kw)
File
"/usr/lib/s3ql/s3ql/backends/gs.py", line 485, in write_fh
raise
_map_request_error(exc, key) or exc
s3ql.backends.gs.RequestError:
<RequestError, code=401, reason='Unauthorized', message='Invalid
Credentials'>
Mar 12 23:58:45 taniguchi mount.s3ql[5294]: Unable to flush cache, no
upload threads left alive
Mar 12 23:58:45 taniguchi mount.s3ql[5294]: mount.s3ql[5294:MainThread]
s3ql.block_cache.destroy: Unable to flush cache, no upload threads left
alive
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
