Hi Nikolaus, When writing less than 100 MB data, it takes only a few seconds and it finished very quick. But when the file size is larger, the CPU usage keeps climbing and stays there until the file write is complete.
I created a 2 minute video to show you what I mean: http://youtu.be/-HMac-Hr3bw Also, this test does not use debug all. The mkfs is using plain option and the mount is using compression none option. The first file copied in the video is 80 MB in size The second file copied in the video is 5 GB in size Is this normal? I am trying to test this using c3.x2large instance today, want to see how it perform transferring many large files using this instance. Please let me know your insights if there is a work around to reduce the cpu load even more? Thank you! On Wed, Jul 16, 2014 at 12:48 PM, Nikolaus Rath <[email protected]> wrote: > Jimmy Tanzil <[email protected]> writes: > > I am having high CPU load issues when writing large files (above 5 GB) > into > > the mounted s3ql filesystem. > > Could you quantify that? How do you measure it, and how does it depend > on file size? Is there really a jump at 5 GB, or is it rather a smooth > increase? > > > > My mount command: mount.s3ql --compress none --debug all --allow-other > > s3://bucket /backup15 > > Using --debug all is certainly a way to eat up a lot of CPU time. But I > assume the problem also happens with debugging disabled? > > > Best, > -Nikolaus > > -- > GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F > Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F > > »Time flies like an arrow, fruit flies like a Banana.« > > -- > You received this message because you are subscribed to a topic in the > Google Groups "s3ql" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/s3ql/Ts0jgzWRQfI/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Kind Regards, Jimmy R. Tanzil -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
