Re: [S3tools-general] question regarding --no-check-md5

2014-05-19 Thread WagnerOne
Hi Matt, I posted in another thread regarding what is used for file comparisons during sync if a multipart ETag is encountered and believe you answered it here already. If a multipart ETag is encountered during sync, s3cmd then reverts to date comparison, but not the S3 stored Last-Modified

Re: [S3tools-general] question regarding --no-check-md5

2014-04-23 Thread Matt Domsch
Yes, I believe it is accurate. it's also fair to ask why we don't follow suit and compare local mtime with S3 LastModified, and upload if mtime is newer. it's been that way since Michal first wrote the initial sync code back in September 2007. Doesn't mean it has to stay that way. On Wed, Apr

Re: [S3tools-general] question regarding --no-check-md5

2014-04-14 Thread WagnerOne
Thanks, Matt. I looked around for the algorithm in the src, but couldn't discern how things worked myself. Thank you for the explanation and reasoning behind how it's done. The aws s3 sync algorithm states it operate like so A local file will require uploading if the size of the local file

Re: [S3tools-general] question regarding --no-check-md5

2014-04-14 Thread Matt Domsch
aws-cli (after a quick perusal of their source code) uses the LastModified value (set by S3 to be the time the upload of the object occurred) on objects in S3, which is obtainable from the ListBucket XML, without doing a HEAD call. They then go on to calculate the difference between LastModified