Obviously using an old version (Ubuntu default)...but I see an odd/interesting performance feature.
I am trying to restore 50GB from google cloud. Initially it ran quite fast (approx 1GB/hour; not great, but not killing my link). After 12 hours it was down to 1/10th of that. Investigations of GC, my link usage, CPU and IO rates did not really lead me to any obvious conclusion. The download machine was doing 200kb/sec IO, 25% iowait, otherwise idle, internet was about 250kb/sec. It was creating about 5 small files per second. So...I killed the rsync and replaced it with 4 rsyncs doing different sub-directories. The result was 4 times the throughput: internet link is now at 1Mb/sec, the iowait time is 49% and it is creating small 20 files per second. Is there an improvement in later versions that would improve the single-threaded behavior, and/or is there a parameter I can use to get better performance on the single-threaded version (ie. only one rsync)? I have not played with the 'upload-threads' setting since this seems to be a download problem. Any suggestions would be appreciated! -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/s3ql/3244ef34-3562-4182-871e-6590cb072b1bn%40googlegroups.com.
