On May 12 2015, Hans Reinders <[email protected]> wrote: > On Tuesday, April 21, 2015 at 2:58:29 AM UTC+2, Nikolaus Rath wrote: >> > is it possible / advisable to increase the max-obj-size to a very >> > high value (say 1GB)? Because it will be only for backing up, the >> > download granularity argument isn't the case. It also seems to me >> > deduplication will not be an argument to use the default >> > max-obj-size of 10MB. >> >> Why not? >> > Output of s3ql after 4TB with the default max-obj-size of 10MB: > > Directory entries: 18952 > Inodes: 18954 > Data blocks: 449412 > Total data size: 4.23 TB > After de-duplication: 4.23 TB (100.00% of total) > After compression: 4.23 TB (100.00% of total, 100.00% of de-duplicated) > Database size: 62.8 MiB (uncompressed) > Cache usage: 1016 MiB (dirty: 0 bytes) > > So deduplication is indeed not happening.
If de-duplication is not happening with 10 MB objects, it's certainly not going to happenwith 1 GB either. Are you sure that you're actually storing identical blocks? Try something simple, like $ cp /bin/bash <s3ql_mountpoint>/file1 $ cp /bin/bash <s3ql_mountpoint>/file2 and see if that de-duplicates. >> > The backend used will be OpenStack Swift (Hubic). >> >> Are you planning to use the experimental patches from >> https://bitbucket.org/nikratio/s3ql/issue/132/, or do you have a method >> to make it work out of the box? > > I have it working out of the box, For the OAuth2 authentication I use a > PHP script HubiC2SwiftGate which I found on the Hubic forums, and has its > home at https://github.com/oderwat/hubic2swiftgate. With this script it is > also possible to use tools such as Cyberduck for creating the > container. Good to know! I've added the info to the bugreport. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
