Hi Peter, A: Because it confuses the reader. Q: Why? A: No. Q: Should I write my response above the quoted reply?
..so please use proper quoting as I'm doing below! On Jun 20 2016, Peter Auyeung <[email protected]> wrote: >> > s3qlstat on google storage: >> > Preparing test data... >> > Measuring throughput to cache... >> > Cache throughput with 4 KiB blocks: 12198 KiB/sec >> > Cache throughput with 8 KiB blocks: 24942 KiB/sec >> > Cache throughput with 16 KiB blocks: 49939 KiB/sec >> > Cache throughput with 32 KiB blocks: 85977 KiB/sec >> > Cache throughput with 64 KiB blocks: 132026 KiB/sec >> > Cache throughput with 128 KiB blocks: 187159 KiB/sec >> > Measuring raw backend throughput.. >> > Backend throughput: 41466 KiB/sec >> > Test file size: 0.00 MiB >> >> Maybe try with a bigger test file? >> >> If the data you are storing is distributed over similarly small files, >> then it is possible that the bad performance is due to latency (as >> opposed to bandwidth). For every file, S3QL needs to make at least one >> PUT request. So if you have millions of tiny files, it will spend a lot >> of time just waiting for the server to reply without ever being able to >> fully utilize the bandwidth. > > Yes it is exactly the case..... > > Wonder if there is a better way to tackle this At some indefinite point in the future, S3QL will be able to store multiple files in one storage object (https://bitbucket.org/nikratio/s3ql/issues/5/support-fragments). Until then, the best workaround is to work with bigger files (e.g. tar them first). Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
