On 08/11/10 11:34, Hugo Silva wrote:
> Christian Gaul wrote:
>> Am 11.08.2010 16:49, schrieb Hugo Silva:
>>> Thomas Mueller wrote:
>>>
>>>> Am Tue, 10 Aug 2010 15:13:07 +0100 schrieb Hugo Silva:
>>>>
>>>>
>>>>> Hello,
>>>>>
>>>>> I'm backing up a server in Germany from a director in The Netherlands.
>>>>> Using bacula, I can't seem to get past ~3000KB/s.
>>>>>
>>>>> Here's an iperf result:
>>>>> [ 3] local [fd-addr] port 16625 connected with [dir-addr] port 5001 [
>>>>> ID] Interval Transfer Bandwidth [ 3] 0.0-10.1 sec 110
>>>>> MBytes 91.2 Mbits/sec
>>>>>
>>>> you speak of a server in germany and director in netherlands. the sd is
>>>> also on the director machine. fd sends data to sd directly - could also be
>>>> a routing issue.
>>>>
>>>> and: as in many other threads mentioned, backing up a filesystem with
>>>> thounds or millions of files can't be compared to a sequential read with
>>>> dd.
>>>>
>>>> and: did you ran the btape tests on the sd to check the performance?
>>>>
>>>>
>>>> - Thomas
>>>>
>>> Hi,
>>>
>>> Thank you for your input.
>>>
>>> The SD is also in the director machine, indeed. I don't think it's a
>>> routing issue - the iperf test was done between these two machines with
>>> excellent results.
>>>
>>> I'm using disk storage; btape doesn't seem to be of help:
>>> btape: btape.c:302 btape only works with tape storage.
>>>
>>> I am aware that a dd test vs many small files isn't comparable - but at
>>> least it rules out the SD storage. (and see below)
>>>
>>> My interest is in knowing if there are known ways people use to speed up
>>> the backup process when done over the internet. This is my first bacula
>>> configuration backing up FDs in remote countries.
>>>
>>>
>>> Consider the following:
>>> # zfs create storage/test
>>> # zfs set mountpoint=/test storage/test
>>> # zfs set compression=off storage/test
>>>
>>>
>>> # dd if=/dev/urandom of=/test/testfile bs=128k count=4096
>>> 4096+0 records in
>>> 4096+0 records out
>>> 536870912 bytes transferred in 7.020243 secs (76474691 bytes/sec)
>>>
>>>
>>>
>>> Now at the director, I create a FileSet backing up this one file.
>>> To aid bacula even more, I'll first put it in the OS cache:
>>>
>>> # dd if=/test/testfile of=/dev/null bs=128k
>>> 4222+0 records in
>>> 4222+0 records out
>>> 553385984 bytes transferred in 2.910288 secs (190148180 bytes/sec)
>>>
>>> And finally, the backup job, using this FileSet:
>>> FileSet {
>>> Name = "TestFileSet"
>>> Include {
>>> Options {
>>> #Compression=gzip
>>> Signature=SHA1
>>> Onefs=yes
>>> Honor nodump flag=yes
>>> Noatime=yes
>>> }
>>>
>>> File = /test/testfile
>>> }
>>> }
>>>
>>>
>>> Notice the read bytes/sec on the second dd.
>>>
>>> At this point, consider that:
>>>
>>> * An iperf test used the link at ~93%.
>>> * The SD hdd is capable of writing at least 70MB/s.
>>> * The FD hdd (ok, zfs cache) is capable of reading at least 180MB/s.
>>>
>>> It follows, I believe, that this test should show transfer rates close
>>> to 100mbits. This is one big file, and the hdd is perfectly capable of
>>> sustaining 12.5MB/s sequential read (far more, as demonstrated)
>>>
>>> However..
>>>
>>> Traffic Peak Total
>>> em0 in 4.863MB/s 4.863MB/s 16.461GB
>>> out 137.977KB/s 137.977 KB/s 495.591MB
>>>
>>> To the three points made above, consider that:
>>>
>>> * Bacula is using the network link at ~38.4% during this test.
>>>
>>>
>>>
>>> I had to disable the Maximum Network Buffer Size in the mean time,
>>> coincidence or not the director started throwing out "unknown errors"
>>> while connecting to storage, so this test is run with default buffer
>>> sizes (which shouldn't be a problem - I got 91-93% of the max link
>>> speed with iperf using default buffer sizes)
>>>
>>> This test:
>>> * Uses TLS encryption [encrypted comms]
>>> * Uses PKI encryption [encrypted backup data]
>>> * Does not use compression
>>>
>>> I don't think TLS/PKI is the cause - there's plenty of CPU% while it's
>>> running. Could investigate this further.
>>>
>> On how many cores? AFAIK the FD only uses one thread for TLS / PKI /
>> compression.
>> (At least it never goes over 100% CPU for me, even when running
>> concurrent jobs)
>>
>>
>>> Not sure what to try next. Any suggestions?
>>>
>>> Thanks for reading.
>>>
>>> Hugo
>>>
>>>
>>
> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
> 11 root 171 ki31 0K 128K CPU7 7 5127.9 100.00% {idle: cpu7}
> 11 root 171 ki31 0K 128K CPU5 5 5126.0 100.00% {idle: cpu5}
> 11 root 171 ki31 0K 128K CPU3 3 5122.7 100.00% {idle: cpu3}
> 11 root 171 ki31 0K 128K CPU4 4 5114.2 100.00% {idle: cpu4}
> 11 root 171 ki31 0K 128K CPU1 1 5113.7 100.00% {idle: cpu1}
> 11 root 171 ki31 0K 128K RUN 0 5101.2 100.00% {idle: cpu0}
> 11 root 171 ki31 0K 128K CPU2 2 5101.5 98.39% {idle: cpu2}
> 11 root 171 ki31 0K 128K CPU6 6 5123.7 90.58% {idle: cpu6}
> 61122 root 58 0 27848K 5900K select 7 0:05 9.28% {bacula-fd}
>
>
> FreeBSD will move bacula-fd to another CPU now and then, but as you see
> it's using only about 10% CPU during this test. Core #7, where it was at
> the time of this snapshot, was 100% idle (this is actually a top
> discrepancy - the process was on CPU#6 before, and you can see that one
> is 90.58% idle, which sounds about right)
>
Regardless of the usage you should try disabling PKI encryption on the
FD you are backing up from. I've seen it really slow things down on
even very new CPUs.
------------------------------------------------------------------------------
This SF.net email is sponsored by
Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
