I am having high CPU load issues when writing large files (above 5 GB) into the mounted s3ql filesystem.
Our customers backup data can be as large as 100 GB per file. I am on m1.large EC2 running Ubuntu 14.04 LTS instance for testing. I also use the --plain option for mkfs.s3ql to try reduce cpu load, but it's still high. Is this normal? My mount command: mount.s3ql --compress none --debug all --allow-other s3://bucket /backup15 I got: Using 4 upload threads. Autodetected 4052 file descriptors available for cache entries Using cached metadata. Setting cache size to 42525 MB Mounting filesystem... The mount.logs for transferring 2 test files: http://54.185.24.96/mount.log http://54.185.24.96/mount.log.1 http://54.185.24.96/mount.log.2 http://54.185.24.96/mount.log.3 http://54.185.24.96/mount.log.4 http://54.185.24.96/mount.log.5 Can you please suggest me the proper mount or mksfs command to avoid high CPU usage. The CPU was maxed out just to transfer these 2 test files. I already use compress none options and --plain option for mkfs, doesn’t seem to help. I am concern as we are transferring thousands of files on our productions server and many of them are over 5 GB size per file. -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
