On Aug 01 2019, Shannon Dealy <[email protected]> wrote:
> Hello Nikolaus,
>
> I am trying to copy a large S3QL file system from the "local" backend to the
> "s3c" backend
> using "clone_fs.py". I made a small test file system to try it out which
> worked fine, but
> with the real file system it crashes immediately
> with the following exception:
>
> s3ql.backends.common.CorruptedObjectError: Invalid object header:
> b'\x80\x02}q\x00(X\x0b\x00'
>
> The file name it was attempting to process was:
>
> s3ql_metadata_bak_5_pre21
>
> I assume from the "_pre21" suffix in the file name and the 2014 date stamp
> that this is an
> old metadata file backup made while upgrading to version 21 of the file
> system. If my
> assumption is correct, then presumably the exception
> was caused by the fact that this file is from an old/incompatible version of
> the file
> system.
That's right.
> Looking at the top level directory for the S3QL file system, there are a
> number of
> other files with not only the "_pre21" suffix, but with names that include
> "_pre2.13_metadata_" as well as some with "#" in the name such as these:
>
> s3ql_pre2.13_metadata
> s3ql_metadata_bak_2#23037-140201956722432
>
> This leads me to the following questions:
>
> 1 - Am I correct in assuming that all of these files can/should be deleted?
> Presumably this would fix my clone_fs problem.
Yes.
> 2 - Should there be (or is there already) some way of listing and/or cleaning
> up cruft files in the S3QL file system metadata directory.
There is no tool, since there should be almost none of these files.
The files with '#' in there should not exist unless mount.s3ql is
hard-crashed (SIGKILL or power cycle). The _pre files were only created
during one upgrade process.
> 3 - Assuming I am correct and all of the above files should be discarded,
> perhaps some documentation should be added to the clone_fs program about
> this issue (not sure if this affects other programs). Ideally it would
> actually recognize that these files can be skipped, but I know that more
> work is the last thing you need.
Yeah, it'd be a good idea for clone_fs to just skip over the files. Pull
requests welcome :-).
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/s3ql/87v9vfq4wr.fsf%40thinkpad.rath.org.
