On Sep 07 2016, Roger Gammans <rgamm...@gammascience.co.uk> wrote:
> apsw.CorruptError: CorruptError: database disk image is malformed
That means you'll have to discard the locally cached metadata. The next
fsck.s3ql will recover whatever was most recently stored in S3.
> Traceback (most recent call last):
> File "/usr/lib/s3ql/s3ql/block_cache.py", line 535, in upload
> obj_id = self.db.rowid('INSERT INTO objects (refcount, size) VALUES(1,
> -1)')
> File "/usr/lib/s3ql/s3ql/database.py", line 104, in rowid
> self.conn.cursor().execute(*a, **kw)
> File "src/cursor.c", line 231, in resetcursor
> apsw.FullError: FullError: database or disk is full
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/usr/lib/s3ql/s3ql/mount.py", line 66, in run_with_except_hook
> run_old(*args, **kw)
> File "/usr/lib/s3ql/s3ql/mount.py", line 795, in run
> self.block_cache.upload(el)
> File "/usr/lib/s3ql/s3ql/block_cache.py", line 573, in upload
> self._unlock_obj(obj_id, noerror=True)
> UnboundLocalError: local variable 'obj_id' referenced before
> assignment
This looks like a bug in S3QL, but it only triggers in the code for
handling database problems. I'll take a look.
>
> That would be clear about something running out of diskspace, - I assume
> (because I was aware of a limit on S3 accounts)
> that is a local filesystem. Any idea which one?
The one that holds the --cachedir directory.
> My guess would be the
> metadata cache, but that seem find at the moment.
>
> Can you shed any light on it ; or give be guidance on metadata sizing
> ?
Required metadata grows linearly with stored data. The proportionaly
factor depends on how big the stored files are, and what block size you
chose.
I'd recommend to simply store a representative subset of your data and
look at the size of the resulting metadata. You can then just scale up
linearly to determine what you'll need for the full data.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to s3ql+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
