Just asking for advice - I'm hitting the maximum size limit on amazon s3,
e.g. here's the final chunk of the output from fsck.s3ql:
Compressing and uploading metadata...
ERROR: Uncaught top-level exception:
Traceback (most recent call last):
File "/usr/bin/fsck.s3ql", line 11, in <module>
load_entry_point('s3ql==3.0', 'console_scripts', 'fsck.s3ql')()
File "/usr/lib/s3ql/s3ql/fsck.py", line 1292, in main
dump_and_upload_metadata(backend, db, param)
File "/usr/lib/s3ql/s3ql/metadata.py", line 313, in
dump_and_upload_metadata
upload_metadata(backend, fh, param)
File "/usr/lib/s3ql/s3ql/metadata.py", line 322, in upload_metadata
metadata=param, is_compressed=True)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 279, in perform_write
return fn(fh)
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 554, in __exit__
self.close()
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 548, in close
self.fh.close()
File "/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 910, in close
headers=self.headers, body=self.fh)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 528, in _do_request
self._parse_error_response(resp)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 562, in
_parse_error_response
raise get_S3Error(tree.findtext('Code'), tree.findtext('Message'),
resp.headers)
s3ql.backends.s3c.S3Error: EntityTooLarge: Your proposed upload exceeds
the maximum allowed size
Here's some info about the s3ql filesystem in case it's of any use:
Directory entries: 463422387
Inodes: 463422389
Data blocks: 8413937
Total data size: 71.6 TB
After de-duplication: 1.37 TB (1.91% of total)
After compression: 737 GiB (1.01% of total, 52.61% of de-duplicated)
Database size: 54.0 GiB (uncompressed)
Cache size: 96.8 MiB, 1005 entries
Cache size (dirty): 0 bytes, 0 entries
Queued object removals: 0
Based on:
https://github.com/s3ql/s3ql/issues/53
this seems to be an issue just with amazon s3 - woudl it be easiest to
just switch to a different provider?
Has anyone had experience using linode's s3 service with s3ql?
Thanks for s3ql!
Joseph
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/s3ql/alpine.DEB.2.21.2106030959090.2653%40ariadne.
