Updates: One backend filesystem (on Google storage) seems to have been
recovered completely by the suggested method. Thank You :)
The other (on Amazon S3), has failed to fsck fully, giving
the following DB integrity errors. I have tried to repair the DB on the
command line as suggested with:
sqlite3 corrupt.db .recover >data.sql
sqlite3 recovered.db <data.sql
This completes without error but doesn't seem to change the
DB so I suspect these are filesystem Metadata errors not SQLite corruption.
Lessons: At this stage , I'm wishing I had turned on versioning on the S3
bucket so I could roll back to a known good backend version.
Or, kept known good backups of the local Metadata DB.
Can anyone, suggest any other way forward? Note there are actuall only a
handfull of errors out of nearl 1TB of data. Many Thanks!
===========
Checking DB integrity...
ERROR: *** in database main ***
Tree 4028 page 3795 cell 372: Rowid 402442 out of order
Tree 7253 page 7253 cell 21: 2nd reference to page 7343
Tree 7253 page 7253 cell 89: 2nd reference to page 7342
Tree 7253 page 7253 cell 88: 2nd reference to page 7341
Tree 7253 page 7253 cell 87: 2nd reference to page 7340
Tree 7253 page 7253 cell 86: 2nd reference to page 7339
Tree 7253 page 7253 cell 85: 2nd reference to page 7338
Tree 7253 page 7253 cell 84: 2nd reference to page 7337
Tree 7253 page 7253 cell 83: 2nd reference to page 7336
Tree 7253 page 7253 cell 82: 2nd reference to page 7335
Tree 7253 page 7253 cell 81: 2nd reference to page 7334
Tree 7253 page 7253 cell 80: 2nd reference to page 7333
Tree 7253 page 7253 cell 79: 2nd reference to page 7332
Tree 7253 page 7253 cell 78: 2nd reference to page 7331
Tree 7253 page 7253 cell 77: 2nd reference to page 7330
Tree 7253 page 7253 cell 76: 2nd reference to page 7329
Tree 7253 page 7253 cell 75: 2nd reference to page 7328
Tree 7253 page 7253 cell 74: 2nd reference to page 7327
Tree 7253 page 7253 cell 73: 2nd reference to page 7326
Tree 7253 page 7253 cell 72: 2nd reference to page 7325
ERROR: Database file (f{cachepath}.db) is corrupted. Restore from a backup
or try
to repair with the SQLite CLI (cf. https://www.sqlite.org/recovery.html),
then re-run fsck.s3ql
On Wednesday, 30 August 2023 at 2:53:40 pm UTC+10 Xomex wrote:
> Through flakey internet and other problems (now solved) I seem to have
> caused data corruption problems of some kind resulting in Checksum errors
> on fsck or mounting the remote filesystem. (see the attached log). I've
> tried with an empty cache, or --force-remote on fsck but nothing changes.
> Is there any way to continue, or even salvage partial data? Any help
> appreciated.
>
>
> ========================
> Starting fsck of gs://xxxxxx
> Scanning metadata objects...
> Downloading metadata...
> Downloaded 2500/2500 metadata blocks (100%)
> Calculating metadata checksum...
> ERROR: Uncaught top-level exception:
> Traceback (most recent call last):
> File "/root/S3QL5.venv//s3ql-5.0.0/bin//fsck.s3ql", line 21, in <module>
> s3ql.fsck.main(sys.argv[1:])
> File "/root/S3QL5.venv/s3ql-5.0.0/src/s3ql/fsck.py", line 1316, in main
> db = download_metadata(backend, cachepath + '.db', param,
> failsafe=param.is_mounted)
> File "/root/S3QL5.venv/s3ql-5.0.0/src/s3ql/database.py", line 506, in
> download_metadata
> raise DatabaseChecksumError(db_file, params.db_md5, digest)
> s3ql.database.DatabaseChecksumError: File
> /Dump/512G/S3QLCaches.new/gs:=2F=2Fxomex-s3ql=2F.db has checksum
> 09b18dfa48dcc015be6ada8311a9dc11c1622b518cbec41b2e891985571611de, expected
> 307d7eca68c20420906de0f9bcb1fa498980505f63e21a32fac6cc02eff7c55a
>
>
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/s3ql/67ab8eb7-10f4-43bb-8866-b03daa673778n%40googlegroups.com.
