My concern that it might crash the file system during busy writes. I tried it out last night, the server was writing approximately 4,000 files into the file system.
I got *File system appears to have crashed* when checking the server this morning. mount.log is at http://50.112.251.132/mount.log I am thinking of doing my backups into an EBS drive, then rsync it to the s3ql file system afterwards to avoid file system crashing when doing thousands multiple file writes at the same time. These 4,000+ files are being written at the same time as we backup hundreds of servers into the same location using s3ql file system. Do you have any pointers to writing thousands of large files into s3ql simultanesouly over night without crashing? On Wed, Jul 16, 2014 at 4:50 PM, Nikolaus Rath <[email protected]> wrote: > On 07/16/2014 01:27 PM, Jimmy Tanzil wrote: > >> Is what normal? > > > > The CPU being utilized more than 100% during writing larger files. > > > > My concern is not the timing, the timing is excellent. > > > > I am more concerned about the CPU showing above 100% when writing one 5 > > GB file. > > > > Is there any other tricks that can be done to lower that? > > I do not understand your concern. Why would you want to artifically slow > down S3QL by forcing it not to use all the available CPU cycles? > > You can add a lot of sleep(1) statements to the write() operation in > fs.py. That should bring CPU utilization down to < 1 %. > > > Best, > -Nikolaus > > -- > GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F > Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F > > »Time flies like an arrow, fruit flies like a Banana.« > > -- > You received this message because you are subscribed to a topic in the > Google Groups "s3ql" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/s3ql/Ts0jgzWRQfI/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Kind Regards, Jimmy R. Tanzil -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
