Hi, The etags are checksums calculated over the contents of the data. If the etags change, then this means the content of the object has changed. So this should not happen on backup/restore.
Best, -Nikolaus On Wed, 14 Jun 2023, at 07:49, Peter Marshall wrote: > I'm impressed using s3ql (v 3.8.1) - it does (nearly) exactly what I want! > > I've run into two issues recently with s3 metadata - one I've managed to > fix, the other I'm looking for advice. > > When syncing an s3 bucket containing an s3ql file system to a new one, > and then trying to mount the new one, I was getting errors with the > metadata. The reason was that aws sync will lose metadata if it uses > multipart transfers. > > I fixed this using "aws configure set default.s3.multipart_threshold > 20MB" to increase the multipart threshold above the size of the s3ql > data blocks. > > I also used "aws s3 sync --metadata-directive COPY" but I believe that > is the default anyway, so it was probably superfluous. > > Then I tried to use AWS Backup on s3 buckets to do a similar job. This > is where the problem came in - the ETag is not always preserved on > restore, and is not an MD5 when multipart uploads get used. The decision > on whether AWS uses multi-part uploads on the restore seems to be out of > my control. > > I've not yet looked into the internals of s3ql - is it possible to fix > the ETag issue? I know ETags can't be changed, but could s3ql recover > from the ETags changing? > > Thanks, > > Pete. > > -- > You received this message because you are subscribed to the Google > Groups "s3ql" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/s3ql/8cf45e3e-d5c0-584a-467d-ab2cdd18d0bf%40goteck.co.uk. -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/s3ql/0581d81d-e0d6-4725-b814-f972b556be67%40app.fastmail.com.
