On Sun, Jan 12, 2014 at 5:46 PM, Joe Van Dyk <[email protected]> wrote: > On Sunday, January 12, 2014 5:45:49 PM UTC-8, Joe Van Dyk wrote: >> >> On Sunday, January 12, 2014 5:24:41 PM UTC-8, Daniel Farina wrote: >>> >>> >>> On Jan 12, 2014 4:54 PM, "Joe Van Dyk" <[email protected]> wrote: >>> > >>> > When doing a backup-fetch, I get about 6.5-8 megabytes per second >>> > download from S3. I was expecting it to be a bit faster. Measured it >>> > through >>> > iftop and nethogs. >>> > >>> > I'm noticing that wal-e is using close to 100% of cpu, top shows it >>> > hovering at about 97%. >>> >>> Python 2.6 you say? This version cannot block 3DES encipherment because >>> of a missing feature of the ssl module. This fix helped my surprisingly >>> slow cases a lot. >>> >>> To confirm, try running Linux 'perf top' and see what symbols crop up. >>> The 3DES symbols in openssl were perspicuously named, which is how this bug >>> was diagnosed. >> >> >> I upgraded to Python 2.7.6, no changes in speed. I'll check out perf >> tomorrow.
Unfortunately. The 3DES blocking helped my cases a lot. > Should wal-e be using such a high cpu percentage? I haven't tuned WAL-E very much for CPU usage, but 6 megabytes per second is suspicious. Can you get a faster rate if you send /dev/zero to the disk? My recollection is the bottleneck (as seen by 'perf top') is in "hypervisor_page" or something named like that. That may be correlated to something else that can be optimized, yet this hypothesized indirection obscures the issue somewhat. Searching the web didn't nail a common cause of that bottleneck, and I haven't researched beyond that. -- You received this message because you are subscribed to the Google Groups "wal-e" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
