On Friday, August 8, 2014 8:54:02 AM UTC+10, Daniel Farina wrote: > > > Careful with relying too hard on exactly one version. A dropped > response (successful S3 commit, but for any reason WAL-E was not able > to exit with an exit code to tell Postgres of that fact) will trigger > a retry, and then I think that'll make two versions. >
This would mean that WAL-E attempts a PUT of the WAL segment under the same key, correct? If I'm understanding correctly, even if WAL-E retries and issues a second PUT, both versions will be identical objects so far as WAL-E is concerned? i.e. if I were to grab both objects, they'd hash to the same outut Therefore restoring *either* version before X date in a case where a malicious entity or software bug resulted in thousands of PUTs for the same key would work, as they would be the same segment. This would *not *be true if WAL-E's retry modifies the object in some way. Can you confirm? > > 2. Retrieving the version with a date prior to the issue occurring > > (http://docs.aws.amazon.com/AmazonS3/latest/dev/RetMetaOfObjVersion.html) > > > 3. Restoring the selected version - which will have the same key as the > > original. > > > > Because the key and the object has not changed, WAL-E won't know any > > different - since it only ever operates on the "current" version by > virtue > > of not using the versioning API. > > > > This is hopefully something that I won't have to do (ever!) although > I'll > > see if I can give it a test run this weekend by deleting a bunch of > streamed > > logs and then restoring them in-place. > > You could also maybe make WAL-E GET requests version aware if you find > a pattern that works and it's not too tortuous on the current code. It > may be easier than writing a different tool. Tentatively, I am not > opposed to carrying that feature patch if it didn't get too > complicated unless you think there are reasons why I shouldn't. > > You might find the blackbox integration tests (see test_blackbox.py) a > quick way to iterate on it, also. Plus, I would like those tests if I > carry the patch :) > I'd definitely like to contribute something like this, although my Python is pretty rusty (I'm a telecoms engineer by trade). Once I've worked out the kinks of a manual approach (or a CLI utility to automate it, using Go) I'll at least attempt to document/write a spec for it, even if I don't write the implementation. -- You received this message because you are subscribed to the Google Groups "wal-e" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
