Hi, I'm using with success S3QL across a variety of backends (s3, local NFS, local RAID6, OVH swift).
Some of these are storing fairly large amounts of data, managed with scripts to cull older backups after X days and months (using a variation of the ideas in the S3QL contrib to expire older copies) to stop the database getting too large. I've noticed these gradually get slower over time (like even a "ls" or "cd" takes minutes or even longer), I am assuming the culprit is sqlite fragmentation. umounting the filesystems and doing a fsck.s3ql, which includes a s3qlite vacuum operation substantially speeds things up for a while, though the performance gradually drops (I'm guessing as the fragmentation comes back). I am storing my s3ql cache files on spinning disk. I guess one obvious thing here might be to try SSD to get a speedup? I'm just about to script some timing the operation of the "s3qlstat $FILESYSTEM" command to determine when s3qlite fragmentation is requiring an umount/fsck to maintain performance. Although due to the fragmentation these fsck operations themselves can take a lot of time. Anyway I'm just mentioning this as something I've observed running S3QL for some time which isn't apparent at the start, but can be managed. -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/s3ql/477d26c5-5d47-45d1-80a2-835762c322f8%40googlegroups.com.
