Bruno Friedmann wrote:
> On 08/10/2010 04:13 PM, Hugo Silva wrote:
>> Hello,
>>
>> I'm backing up a server in Germany from a director in The Netherlands.
>> Using bacula, I can't seem to get past ~3000KB/s.
>>
>> Here's an iperf result:
>> [ 3] local [fd-addr] port 16625 connected with [dir-addr] port 5001
>> [ ID] Interval Transfer Bandwidth
>> [ 3] 0.0-10.1 sec 110 MBytes 91.2 Mbits/sec
>>
>>
>> Googling doesn't seem to bring much more than setting "Maximum Network
>> Buffer Size" on the SD and FD.
>>
>> I did (to 1048576 - same value I ran the iperf tests with), but there
>> were no changes; anyway, even with my system default socket buffer sizes
>> (256K) I manage to see ~90mbits/s.
>>
>>
>> This leads me to think that perhaps the FD just doesn't have that much
>> data to send continuously to the SD, hence the low throughput. In theory
>> it could also be the storage that's too slow, but a crude test seems to
>> rule this out:
>>
>> # dd if=/dev/zero of=/bacula/x bs=32k
>> ^C24030+0 records in
>> 24029+0 records out
>> 787382272 bytes transferred in 7.044653 secs (111770197 bytes/sec)
>>
>> Neither server seems to be CPU starved, either (I'm transfering
>> encrypted backups on a TLS connection using compression):
>>
>> director:
>> CPU: 3.4% user, 0.0% nice, 1.2% system, 0.0% interrupt, 95.3% idle
>> Mem: 58M Active, 425M Inact, 405M Wired, 176K Cache, 316M Buf, 2084M Free
>> Swap: 2048M Total, 2048M Free
>>
>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
>> 11 root 171 ki31 0K 64K CPU1 1 336.1H 100.00% {idle: cpu1}
>> 11 root 171 ki31 0K 64K CPU2 2 335.9H 100.00% {idle:
>> emcpu2}
>> 11 root 171 ki31 0K 64K CPU0 0 335.5H 100.00% {idle: cpu0}
>> 11 root 171 ki31 0K 64K RUN 3 336.1H 98.49% {idle: cpu3}
>> 46442 bacula 50 0 26796K 7268K select 0 0:01 7.08% {bacula-sd}
>> 0 root -68 0 0K 656K - 2 0:36 1.17% {em0 taskq}
>>
>> FD:
>> CPU: 10.9% user, 0.0% nice, 0.6% system, 0.3% interrupt, 88.2% idle
>> Mem: 1734M Active, 1807M Inact, 2965M Wired, 1168K Cache, 827M Buf,
>> 1393M Free
>> Swap: 16G Total, 868K Used, 16G Free
>>
>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
>> 11 root 171 ki31 0K 128K CPU7 7 5102.6 100.00% {idle: cpu7}
>> 11 root 171 ki31 0K 128K CPU6 6 5098.5 100.00% {idle: cpu6}
>> 11 root 171 ki31 0K 128K CPU3 3 5097.5 100.00% {idle: cpu3}
>> 11 root 171 ki31 0K 128K RUN 0 5076.2 95.07% {idle: cpu0}
>> 11 root 171 ki31 0K 128K CPU2 2 5076.6 92.48% {idle: cpu2}
>> 11 root 171 ki31 0K 128K RUN 5 5100.7 87.99% {idle: cpu5}
>> 11 root 171 ki31 0K 128K CPU4 4 5089.2 87.50% {idle: cpu4}
>> 11 root 171 ki31 0K 128K CPU1 1 5088.6 85.35% {idle: cpu1}
>> 40102 root 108 0 35016K 9584K CPU5 5 0:51 56.59% {bacula-fd}
>> 12 root -28 - 0K 352K WAIT 0 40:34 3.56% {swi5: +}
>>
>>
>>
>>
>> So my question is, how do I go about working around this? Perhaps an
>> option to tell the FD to buffer stuff in memory and then send it in
>> bursts is available?
>>
>> Please advise.
>>
> Just also another point to check with the tls/encryption used
>
> Did you check what are the available entropy are during your backup ...
> cat /proc/sys/kernel/random/entropy_avail ( munin or collectd have plugin to
> monitor that )
>
> If you run on low entropy all encryption jobs are slowed down or block until
> entropy get higher ...
> This can be observed on ssl serveur refusing connexion etc ...
>
> There's http://www.issihosts.com/haveged/ daemon which can help you to
> maintain a high level of that.
>
> Headless and diskless servers with limited input have relied on entropy
> added by interrupts flagged with IRQF_SAMPLE_RANDOM. However, thisain
> feature will be disappearing from the Kernel soon.
> One solution is to run a daemon to add entropy from userspace to the
> pool. Example daemons can be found here:
> * http://www.vanheusden.com/aed/
> * http://www.vanheusden.com/ved/
> * http://egd.sourceforge.net/
> distributions should provide these or similar daemons as options for users
> who
> require additional entropy sources to keep /dev/random from blocking on
> read.
> The Kernel thread discussing this thread can be found here:
> http://lkml.org/lkml/2009/4/6/283
> commit 9d9b8fb0e5ebf4b0398e579f6061d4451fea3242
> What: IRQF_SAMPLE_RANDOM
> Check: IRQF_SAMPLE_RANDOM
> When: July 2009
> Why: Many of IRQF_SAMPLE_RANDOM users are technically bogus as
> entropy
> sources in the kernel's current entropy model. To resolve this, every
> input point to the kernel's entropy pool needs to better document the
> type of entropy source it actually is. This will be replaced with
> additional add_*_randomness functions in drivers/char/random.c
>
> Hope this give you also another way to resolve the case.
>
>
>
I'm not running Linux but your point still stands. I only tried
disabling PKI encryption, not TLS. If I follow you correctly, yours is
still a valid point. Tomorrow I will try again without TLS and share the
result. Thanks!
------------------------------------------------------------------------------
This SF.net email is sponsored by
Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
