On Thu, Aug 7, 2014 at 4:18 PM, Matt Silverlock <[email protected]> wrote: > > > On Friday, August 8, 2014 8:54:02 AM UTC+10, Daniel Farina wrote: >> >> >> Careful with relying too hard on exactly one version. A dropped >> response (successful S3 commit, but for any reason WAL-E was not able >> to exit with an exit code to tell Postgres of that fact) will trigger >> a retry, and then I think that'll make two versions. > > > This would mean that WAL-E attempts a PUT of the WAL segment under the same > key, correct? If I'm understanding correctly, even if WAL-E retries and > issues a second PUT, both versions will be identical objects so far as WAL-E > is concerned? i.e. if I were to grab both objects, they'd hash to the same > outut
Yeah. Same key. They're identical objects, though. I'm not sure if S3 collapses the two versions in such a case. > Therefore restoring either version before X date in a case where a malicious > entity or software bug resulted in thousands of PUTs for the same key would > work, as they would be the same segment. > > This would not be true if WAL-E's retry modifies the object in some way. Can > you confirm? No modification. etag should be identical. >> You could also maybe make WAL-E GET requests version aware if you find >> a pattern that works and it's not too tortuous on the current code. It >> may be easier than writing a different tool. Tentatively, I am not >> opposed to carrying that feature patch if it didn't get too >> complicated unless you think there are reasons why I shouldn't. >> >> You might find the blackbox integration tests (see test_blackbox.py) a >> quick way to iterate on it, also. Plus, I would like those tests if I >> >> carry the patch :) > > I'd definitely like to contribute something like this, although my Python is > pretty rusty (I'm a telecoms engineer by trade). Once I've worked out the > kinks of a manual approach (or a CLI utility to automate it, using Go) I'll > at least attempt to document/write a spec for it, even if I don't write the > implementation. Cool any which way: glad to have some effortful research. Don't worry too much about being rusty at Python. If you write it with non-idiomatic impediments but the design is basically sound I'll fix it. You can give the instructions here a try https://github.com/wal-e/wal-e#development to hopefully get your iteration cycle short. If you want any help making the iteration loop faster let me know. -- You received this message because you are subscribed to the Google Groups "wal-e" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
