Hugo Silva wrote:
> Bruno Friedmann wrote:
>> On 08/10/2010 04:13 PM, Hugo Silva wrote:
>>> Hello,
>>>
>>> I'm backing up a server in Germany from a director in The Netherlands.
>>> Using bacula, I can't seem to get past ~3000KB/s.
>>>
>>> Here's an iperf result:
>>> [ 3] local [fd-addr] port 16625 connected with [dir-addr] port 5001
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 0.0-10.1 sec 110 MBytes 91.2 Mbits/sec
>>>
>>>
>>> Googling doesn't seem to bring much more than setting "Maximum Network
>>> Buffer Size" on the SD and FD.
>>>
>>> I did (to 1048576 - same value I ran the iperf tests with), but there
>>> were no changes; anyway, even with my system default socket buffer sizes
>>> (256K) I manage to see ~90mbits/s.
>>>
>>>
>>> This leads me to think that perhaps the FD just doesn't have that much
>>> data to send continuously to the SD, hence the low throughput. In theory
>>> it could also be the storage that's too slow, but a crude test seems to
>>> rule this out:
>>>
>>> # dd if=/dev/zero of=/bacula/x bs=32k
>>> ^C24030+0 records in
>>> 24029+0 records out
>>> 787382272 bytes transferred in 7.044653 secs (111770197 bytes/sec)
>>>
>>> Neither server seems to be CPU starved, either (I'm transfering
>>> encrypted backups on a TLS connection using compression):
>>>
>>> director:
>>> CPU: 3.4% user, 0.0% nice, 1.2% system, 0.0% interrupt, 95.3% idle
>>> Mem: 58M Active, 425M Inact, 405M Wired, 176K Cache, 316M Buf, 2084M Free
>>> Swap: 2048M Total, 2048M Free
>>>
>>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
>>> 11 root 171 ki31 0K 64K CPU1 1 336.1H 100.00% {idle: cpu1}
>>> 11 root 171 ki31 0K 64K CPU2 2 335.9H 100.00% {idle:
>>> emcpu2}
>>> 11 root 171 ki31 0K 64K CPU0 0 335.5H 100.00% {idle: cpu0}
>>> 11 root 171 ki31 0K 64K RUN 3 336.1H 98.49% {idle: cpu3}
>>> 46442 bacula 50 0 26796K 7268K select 0 0:01 7.08% {bacula-sd}
>>> 0 root -68 0 0K 656K - 2 0:36 1.17% {em0 taskq}
>>>
>>> FD:
>>> CPU: 10.9% user, 0.0% nice, 0.6% system, 0.3% interrupt, 88.2% idle
>>> Mem: 1734M Active, 1807M Inact, 2965M Wired, 1168K Cache, 827M Buf,
>>> 1393M Free
>>> Swap: 16G Total, 868K Used, 16G Free
>>>
>>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
>>> 11 root 171 ki31 0K 128K CPU7 7 5102.6 100.00% {idle: cpu7}
>>> 11 root 171 ki31 0K 128K CPU6 6 5098.5 100.00% {idle: cpu6}
>>> 11 root 171 ki31 0K 128K CPU3 3 5097.5 100.00% {idle: cpu3}
>>> 11 root 171 ki31 0K 128K RUN 0 5076.2 95.07% {idle: cpu0}
>>> 11 root 171 ki31 0K 128K CPU2 2 5076.6 92.48% {idle: cpu2}
>>> 11 root 171 ki31 0K 128K RUN 5 5100.7 87.99% {idle: cpu5}
>>> 11 root 171 ki31 0K 128K CPU4 4 5089.2 87.50% {idle: cpu4}
>>> 11 root 171 ki31 0K 128K CPU1 1 5088.6 85.35% {idle: cpu1}
>>> 40102 root 108 0 35016K 9584K CPU5 5 0:51 56.59% {bacula-fd}
>>> 12 root -28 - 0K 352K WAIT 0 40:34 3.56% {swi5: +}
>>>
>>>
>>>
>>>
>>> So my question is, how do I go about working around this? Perhaps an
>>> option to tell the FD to buffer stuff in memory and then send it in
>>> bursts is available?
>>>
>>> Please advise.
>>>
>> Just also another point to check with the tls/encryption used
>>
>> Did you check what are the available entropy are during your backup ...
>> cat /proc/sys/kernel/random/entropy_avail ( munin or collectd have plugin to
>> monitor that )
>>
>> If you run on low entropy all encryption jobs are slowed down or block until
>> entropy get higher ...
>> This can be observed on ssl serveur refusing connexion etc ...
>>
>> There's http://www.issihosts.com/haveged/ daemon which can help you to
>> maintain a high level of that.
>>
>> Headless and diskless servers with limited input have relied on entropy
>> added by interrupts flagged with IRQF_SAMPLE_RANDOM. However, thisain
>> feature will be disappearing from the Kernel soon.
>> One solution is to run a daemon to add entropy from userspace to the
>> pool. Example daemons can be found here:
>> * http://www.vanheusden.com/aed/
>> * http://www.vanheusden.com/ved/
>> * http://egd.sourceforge.net/
>> distributions should provide these or similar daemons as options for users
>> who
>> require additional entropy sources to keep /dev/random from blocking on
>> read.
>> The Kernel thread discussing this thread can be found here:
>> http://lkml.org/lkml/2009/4/6/283
>> commit 9d9b8fb0e5ebf4b0398e579f6061d4451fea3242
>> What: IRQF_SAMPLE_RANDOM
>> Check: IRQF_SAMPLE_RANDOM
>> When: July 2009
>> Why: Many of IRQF_SAMPLE_RANDOM users are technically bogus as
>> entropy
>> sources in the kernel's current entropy model. To resolve this, every
>> input point to the kernel's entropy pool needs to better document the
>> type of entropy source it actually is. This will be replaced with
>> additional add_*_randomness functions in drivers/char/random.c
>>
>> Hope this give you also another way to resolve the case.
>>
>>
>>
>
>
> I'm not running Linux but your point still stands. I only tried
> disabling PKI encryption, not TLS. If I follow you correctly, yours is
> still a valid point. Tomorrow I will try again without TLS and share the
> result. Thanks!
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by
>
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
No TLS, no PKI:
Rate: 5345.0 KB/s
Termination: Backup Canceled
Ok.. is anyone else using bacula on FreeBSD 8.X, backing up over the
internet?
------------------------------------------------------------------------------
This SF.net email is sponsored by
Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
