On 08/10/2010 04:13 PM, Hugo Silva wrote:
> Hello,
>
> I'm backing up a server in Germany from a director in The Netherlands.
> Using bacula, I can't seem to get past ~3000KB/s.
>
> Here's an iperf result:
> [ 3] local [fd-addr] port 16625 connected with [dir-addr] port 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0-10.1 sec 110 MBytes 91.2 Mbits/sec
>
>
> Googling doesn't seem to bring much more than setting "Maximum Network
> Buffer Size" on the SD and FD.
>
> I did (to 1048576 - same value I ran the iperf tests with), but there
> were no changes; anyway, even with my system default socket buffer sizes
> (256K) I manage to see ~90mbits/s.
>
>
> This leads me to think that perhaps the FD just doesn't have that much
> data to send continuously to the SD, hence the low throughput. In theory
> it could also be the storage that's too slow, but a crude test seems to
> rule this out:
>
> # dd if=/dev/zero of=/bacula/x bs=32k
> ^C24030+0 records in
> 24029+0 records out
> 787382272 bytes transferred in 7.044653 secs (111770197 bytes/sec)
>
> Neither server seems to be CPU starved, either (I'm transfering
> encrypted backups on a TLS connection using compression):
>
> director:
> CPU: 3.4% user, 0.0% nice, 1.2% system, 0.0% interrupt, 95.3% idle
> Mem: 58M Active, 425M Inact, 405M Wired, 176K Cache, 316M Buf, 2084M Free
> Swap: 2048M Total, 2048M Free
>
> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
> 11 root 171 ki31 0K 64K CPU1 1 336.1H 100.00% {idle: cpu1}
> 11 root 171 ki31 0K 64K CPU2 2 335.9H 100.00% {idle:
> emcpu2}
> 11 root 171 ki31 0K 64K CPU0 0 335.5H 100.00% {idle: cpu0}
> 11 root 171 ki31 0K 64K RUN 3 336.1H 98.49% {idle: cpu3}
> 46442 bacula 50 0 26796K 7268K select 0 0:01 7.08% {bacula-sd}
> 0 root -68 0 0K 656K - 2 0:36 1.17% {em0 taskq}
>
> FD:
> CPU: 10.9% user, 0.0% nice, 0.6% system, 0.3% interrupt, 88.2% idle
> Mem: 1734M Active, 1807M Inact, 2965M Wired, 1168K Cache, 827M Buf,
> 1393M Free
> Swap: 16G Total, 868K Used, 16G Free
>
> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
> 11 root 171 ki31 0K 128K CPU7 7 5102.6 100.00% {idle: cpu7}
> 11 root 171 ki31 0K 128K CPU6 6 5098.5 100.00% {idle: cpu6}
> 11 root 171 ki31 0K 128K CPU3 3 5097.5 100.00% {idle: cpu3}
> 11 root 171 ki31 0K 128K RUN 0 5076.2 95.07% {idle: cpu0}
> 11 root 171 ki31 0K 128K CPU2 2 5076.6 92.48% {idle: cpu2}
> 11 root 171 ki31 0K 128K RUN 5 5100.7 87.99% {idle: cpu5}
> 11 root 171 ki31 0K 128K CPU4 4 5089.2 87.50% {idle: cpu4}
> 11 root 171 ki31 0K 128K CPU1 1 5088.6 85.35% {idle: cpu1}
> 40102 root 108 0 35016K 9584K CPU5 5 0:51 56.59% {bacula-fd}
> 12 root -28 - 0K 352K WAIT 0 40:34 3.56% {swi5: +}
>
>
>
>
> So my question is, how do I go about working around this? Perhaps an
> option to tell the FD to buffer stuff in memory and then send it in
> bursts is available?
>
> Please advise.
>
Just also another point to check with the tls/encryption used
Did you check what are the available entropy are during your backup ...
cat /proc/sys/kernel/random/entropy_avail ( munin or collectd have plugin to
monitor that )
If you run on low entropy all encryption jobs are slowed down or block until
entropy get higher ...
This can be observed on ssl serveur refusing connexion etc ...
There's http://www.issihosts.com/haveged/ daemon which can help you to maintain
a high level of that.
Headless and diskless servers with limited input have relied on entropy
added by interrupts flagged with IRQF_SAMPLE_RANDOM. However, this
feature will be disappearing from the Kernel soon.
One solution is to run a daemon to add entropy from userspace to the
pool. Example daemons can be found here:
* http://www.vanheusden.com/aed/
* http://www.vanheusden.com/ved/
* http://egd.sourceforge.net/
distributions should provide these or similar daemons as options for users who
require additional entropy sources to keep /dev/random from blocking on
read.
The Kernel thread discussing this thread can be found here:
http://lkml.org/lkml/2009/4/6/283
commit 9d9b8fb0e5ebf4b0398e579f6061d4451fea3242
What: IRQF_SAMPLE_RANDOM
Check: IRQF_SAMPLE_RANDOM
When: July 2009
Why: Many of IRQF_SAMPLE_RANDOM users are technically bogus as
entropy
sources in the kernel's current entropy model. To resolve this, every
input point to the kernel's entropy pool needs to better document the
type of entropy source it actually is. This will be replaced with
additional add_*_randomness functions in drivers/char/random.c
Hope this give you also another way to resolve the case.
--
Bruno Friedmann br...@ioda-net.ch
Ioda-Net Sàrl www.ioda-net.ch
openSUSE Member
User www.ioda.net/r/osu
Blog www.ioda.net/r/blog
fsfe fellowship www.fsfe.org
(bruno.friedmann (at) fsfe.org )
tigerfoot on irc
GPG KEY : D5C9B751C4653227
------------------------------------------------------------------------------
This SF.net email is sponsored by
Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
