On Tuesday, May 12, 2015 at 8:34:00 PM UTC+2, Nikolaus Rath wrote: > > On May 12 2015, Hans Reinders <[email protected] <javascript:>> wrote: > > On Tuesday, April 21, 2015 at 2:58:29 AM UTC+2, Nikolaus Rath wrote: > >> > is it possible / advisable to increase the max-obj-size to a very > >> > high value (say 1GB)? Because it will be only for backing up, the > >> > download granularity argument isn't the case. It also seems to me > >> > deduplication will not be an argument to use the default > >> > max-obj-size of 10MB. > >> > >> Why not? > >> > > Output of s3ql after 4TB with the default max-obj-size of 10MB: > > > > Directory entries: 18952 > > Inodes: 18954 > > Data blocks: 449412 > > Total data size: 4.23 TB > > After de-duplication: 4.23 TB (100.00% of total) > > After compression: 4.23 TB (100.00% of total, 100.00% of > de-duplicated) > > Database size: 62.8 MiB (uncompressed) > > Cache usage: 1016 MiB (dirty: 0 bytes) > > > > So deduplication is indeed not happening. > > If de-duplication is not happening with 10 MB objects, it's certainly > not going to happenwith 1 GB either. Are you sure that you're actually > storing identical blocks? > > Try something simple, like > > $ cp /bin/bash <s3ql_mountpoint>/file1 > $ cp /bin/bash <s3ql_mountpoint>/file2 > > and see if that de-duplicates. >
No, what I meant was that I argued to use a bigger max-obj-size because I could not see deduplication happening with storing of movies and because the download granularity argument was not the case. You replied why not. The output of s3qlstat was my response. > > >> > The backend used will be OpenStack Swift (Hubic). > >> > >> Are you planning to use the experimental patches from > >> https://bitbucket.org/nikratio/s3ql/issue/132/, or do you have a > method > >> to make it work out of the box? > > > > I have it working out of the box, For the OAuth2 authentication I use a > > PHP script HubiC2SwiftGate which I found on the Hubic forums, and has > its > > home at https://github.com/oderwat/hubic2swiftgate. With this script it > is > > also possible to use tools such as Cyberduck for creating the > > container. > > Good to know! I've added the info to the bugreport. > > > Best, > -Nikolaus > > -- > GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F > Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F > > »Time flies like an arrow, fruit flies like a Banana.« > -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
