Hi Nikolaus
Thanks a lot for your advice. I will definitely refrain from messing with
the metadata in the future :)
I renamed s3ql_metadata to something not starting with 's3ql', but running:
> fsck.s3ql s3://mybucket
results in this exception:
Starting fsck of s3://mybucket
Uncaught top-level exception:
Traceback (most recent call last):
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 253, in lookup
resp = self._do_request('HEAD', '/%s%s' % (self.prefix, key))
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 477, in _do_request
self._parse_error_response(resp)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 494, in
_parse_error_response
raise HTTPError(resp.status, resp.reason, resp.headers)
s3ql.backends.s3c.HTTPError: 404 Not Found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/fsck.s3ql", line 9, in <module>
load_entry_point('s3ql==2.12', 'console_scripts', 'fsck.s3ql')()
File "/usr/lib/s3ql/s3ql/fsck.py", line 1178, in main
param = backend.lookup('s3ql_metadata')
File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 70, in lookup
meta_raw = self.backend.lookup(key)
File "/usr/lib/s3ql/s3ql/backends/common.py", line 46, in wrapped
return method(*a, **kw)
File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 257, in lookup
raise NoSuchObject(key)
s3ql.backends.common.NoSuchObject: Backend does not have anything stored
under key 's3ql_metadata'
I tried adding the --batch argument, but that does not make a difference.
Is that something you can help with?
Thanks in advance!
Eric
On Monday, January 5, 2015 8:51:46 PM UTC+1, Nikolaus Rath wrote:
>
> Eric Eijkelenboom <[email protected] <javascript:>> writes:
> > Hi guys
> >
> > Here’s what happened:
> >
> > 1. Due to a server shutdown, the filesystem was not cleanly umount’ed.
> > Hereafter, fsck resulted in ‘fs still mounted elsewhere, aborting’
> > errors (obviously, the backend was not mounted anywhere else).
>
> Did you run with --batch? Normally, fsck will prompt you and you can
> tell it to continue running anyway. No need to mess with the metadata at
> all.
>
> > 2. I tried restoring from backup by renaming
> > s3://mybucket/s3ql_metadata_bak_0 to to s3://mybucket/s3ql_metadata,
> > hereby overwriting s3ql_metadata (yes, big screw-up).
>
> Yep. That breaks things, because it looks as if some malicious other guy
> is trying to get you to use outdated metadata.
>
> > After discovering s3qladm download-metadata, I tried to:
> >
> > 1. s3qladm download-metadata s3://mybucket
> > 2. Choose a backup
> > 3. Move the generated files to ~/.s3ql
> > 4. Run fsck.
> >
> > fsck does it’s thing, but crashes in the end with the same exception
> > as above:
> >
> > ...
> > Backing up old metadata...
> > Uncaught top-level exception:
> > Traceback (most recent call last):
> > File "/usr/bin/fsck.s3ql", line 9, in <module>
> > load_entry_point('s3ql==2.12', 'console_scripts', 'fsck.s3ql')()
> > File "/usr/lib/s3ql/s3ql/fsck.py", line 1291, in main
> > cycle_metadata(backend)
> > File "/usr/lib/s3ql/s3ql/metadata.py", line 127, in cycle_metadata
> > backend.copy("s3ql_metadata", "s3ql_metadata_bak_0")
> > File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 290, in copy
> > self._copy_or_rename(src, dest, rename=False, metadata=metadata)
> > File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 302, in _
> > copy_or_rename
> > (nonce, meta_old) = self._verify_meta(src, meta_raw)
> > File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 139, in _
> > verify_meta
> > % (stored_key, key))
> > s3ql.backends.common.CorruptedObjectError: Object content does not
> > match its key (s3ql_metadata_new vs s3ql_metadata)
> >
> > Questions:
> > 1. What can I do from here?
>
> Delete the "s3ql_metadata" object from the bucket (or rename it to
> something that does not start with s3ql_).
>
> > 2. How to avoid this in the future?
>
> * Never, ever, modify the bucket contents with anything but the S3QL
> tools.
>
> * Don't use s3qladm download-metadata before you've asked on this list,
> fsck.s3ql should be able to handle 99% of all problems.
>
> * Don't shut down the server while the file system is still mounted :-)
>
> Best,
> -Nikolaus
>
> --
> GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
> Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
>
> »Time flies like an arrow, fruit flies like a Banana.«
>
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
