On Apr 18 2016, Alexandre Gonçalves <[email protected]> wrote:
> Hello,
>
> In my setup, I use metadata-upload-interval=3600. Which means that the
> cached data is uploaded to the server every hour.
>
> If anything goes wrong, the worst scenario is 1hour of written data
> lost.
Yeah, but I don't think you realize how unlikely this actually is.
To understand why intervals smaller than a few hours do not make sense,
consider what happens during a metadata upload:
1. The file system is frozen (all requests will block)
2. The entire SQLite database holding the metadata is dumped into a more
space efficient format.
3. The file system is unfrozen
4. The dumped database is uploaded
5. The previous metadata is backed up, and backups are rotated
In other words, uploading metadata is a very expensive operation that typically
takes several seconds to several minutes.
On the other hand, consider the situation that the periodic upload is intended
to prevent:
1. Your computer crashes, and the SQLite database in your S3QL cache
directory is irreparably corrupted or lost.
This is a very unlikely situation. When mount.s3ql crashes (but your
computer keeps running), the local metadata will not get corrupted. If
your entire computer crashes (e.g. because of a power outage), fsck.s3ql
is almost always still able to recover the local metadata. You have to
be exceedingly unlucky to depend on the periodically uploaded metadata.
In other words, if you set the metadata upload interval to x hours, what
you are saying is that you consider your system so unstable that there
is a good chance that your local hard disk will be irreparably corrupted
sometime in the next x hours. In such a situation, you should thus be
backing up your entire system every x hours (not just the S3QL
metadata). If the metadata upload interval coincides with the interval
at which you're doing full system backups (not necessarily using S3QL),
you are using the right value. If the metadata upload interval is
smaller (or if you're not doing full system backups at all), you are
probably uploading metadata too often.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
