I was doing some benchmarks of backup-fetch runs on an hs1.8xlarge EC2 instance and noticed that I could only hit 500Mbps before wal-e became CPU bound (AFAICT due to SSL). I hacked up something quickly to test using multiprocessing (see https://github.com/tail/wal-e/tree/multiprocessing) and was able to peak at 2.25Gbps before hitting another bottleneck. I graphed a comparison of the two here: https://dl.dropboxusercontent.com/u/153/wal-e/multiprocessing_vs_gevent_1.png .
I wanted to bring this up to the mailing list to see if there was any merits in pursuing this further since it would be a large change. I'm planning on testing backup-push and wal-fetch as well. I'm also curious what performance others are seeing. -- You received this message because you are subscribed to the Google Groups "wal-e" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
