After making recover.db, there is the additional step to make it "live":
whatever your current db is called, move that out of the way then put
recover.db into it's place.
Like (in s3ql cache directory)
mv yourcache.db yourcache.db.bak
mv recover.db yourcache.db
Most likely your filesystem will be back up and running then but you have
the db.bak file just in case. I had this happen when I was using a USB HDD
for the fs and cache, and the USB cable got all flakey causing dropouts and
inconvenient times.. As you'd expect from any robust filesystem, it just
resulted in loss of the last few seconds of stuff copied in, the fsck put a
file or two into lost+found directory and away I went.
On Thursday, August 31, 2023 at 9:25:49 PM UTC-5 Xomex wrote:
> Updates: One backend filesystem (on Google storage) seems to have been
> recovered completely by the suggested method. Thank You :)
>
> The other (on Amazon S3), has failed to fsck fully, giving
> the following DB integrity errors. I have tried to repair the DB on the
> command line as suggested with:
> sqlite3 corrupt.db .recover >data.sql
> sqlite3 recovered.db <data.sql
> This completes without error but doesn't seem to change the
> DB so I suspect these are filesystem Metadata errors not SQLite corruption.
>
> Lessons: At this stage , I'm wishing I had turned on versioning on the S3
> bucket so I could roll back to a known good backend version.
> Or, kept known good backups of the local Metadata DB.
>
> Can anyone, suggest any other way forward? Note there are actuall only a
> handfull of errors out of nearl 1TB of data. Many Thanks!
>
>
> ===========
> Checking DB integrity...
> ERROR: *** in database main ***
> Tree 4028 page 3795 cell 372: Rowid 402442 out of order
> Tree 7253 page 7253 cell 21: 2nd reference to page 7343
> Tree 7253 page 7253 cell 89: 2nd reference to page 7342
> Tree 7253 page 7253 cell 88: 2nd reference to page 7341
> Tree 7253 page 7253 cell 87: 2nd reference to page 7340
> Tree 7253 page 7253 cell 86: 2nd reference to page 7339
> Tree 7253 page 7253 cell 85: 2nd reference to page 7338
> Tree 7253 page 7253 cell 84: 2nd reference to page 7337
> Tree 7253 page 7253 cell 83: 2nd reference to page 7336
> Tree 7253 page 7253 cell 82: 2nd reference to page 7335
> Tree 7253 page 7253 cell 81: 2nd reference to page 7334
> Tree 7253 page 7253 cell 80: 2nd reference to page 7333
> Tree 7253 page 7253 cell 79: 2nd reference to page 7332
> Tree 7253 page 7253 cell 78: 2nd reference to page 7331
> Tree 7253 page 7253 cell 77: 2nd reference to page 7330
> Tree 7253 page 7253 cell 76: 2nd reference to page 7329
> Tree 7253 page 7253 cell 75: 2nd reference to page 7328
> Tree 7253 page 7253 cell 74: 2nd reference to page 7327
> Tree 7253 page 7253 cell 73: 2nd reference to page 7326
> Tree 7253 page 7253 cell 72: 2nd reference to page 7325
> ERROR: Database file (f{cachepath}.db) is corrupted. Restore from a backup
> or try
> to repair with the SQLite CLI (cf. https://www.sqlite.org/recovery.html),
> then re-run fsck.s3ql
>
>
> On Wednesday, 30 August 2023 at 2:53:40 pm UTC+10 Xomex wrote:
>
>> Through flakey internet and other problems (now solved) I seem to have
>> caused data corruption problems of some kind resulting in Checksum errors
>> on fsck or mounting the remote filesystem. (see the attached log). I've
>> tried with an empty cache, or --force-remote on fsck but nothing changes.
>> Is there any way to continue, or even salvage partial data? Any help
>> appreciated.
>>
>>
>> ========================
>> Starting fsck of gs://xxxxxx
>> Scanning metadata objects...
>> Downloading metadata...
>> Downloaded 2500/2500 metadata blocks (100%)
>> Calculating metadata checksum...
>> ERROR: Uncaught top-level exception:
>> Traceback (most recent call last):
>> File "/root/S3QL5.venv//s3ql-5.0.0/bin//fsck.s3ql", line 21, in <module>
>> s3ql.fsck.main(sys.argv[1:])
>> File "/root/S3QL5.venv/s3ql-5.0.0/src/s3ql/fsck.py", line 1316, in main
>> db = download_metadata(backend, cachepath + '.db', param,
>> failsafe=param.is_mounted)
>> File "/root/S3QL5.venv/s3ql-5.0.0/src/s3ql/database.py", line 506, in
>> download_metadata
>> raise DatabaseChecksumError(db_file, params.db_md5, digest)
>> s3ql.database.DatabaseChecksumError: File
>> /Dump/512G/S3QLCaches.new/gs:=2F=2Fxomex-s3ql=2F.db has checksum
>> 09b18dfa48dcc015be6ada8311a9dc11c1622b518cbec41b2e891985571611de, expected
>> 307d7eca68c20420906de0f9bcb1fa498980505f63e21a32fac6cc02eff7c55a
>>
>>
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/s3ql/fc09bb6c-000c-4f69-b182-11d6688d89den%40googlegroups.com.
