On 03/29/10 06:53 PM, Kern Sibbald wrote:
> On Monday 29 March 2010 18:01:56 Henrik Johansen wrote:
>> On 03/29/10 05:08 PM, Kern Sibbald wrote:
>>> On Monday 29 March 2010 16:59:05 Phil Stracchino wrote:
>>>> On 03/29/10 08:22, Henrik Johansen wrote:
>>>>> We'll top 300+ devices per SD once we are finished migrating.
>>>> I hope you'll post a full writeup somewhere. :)
>>> Oh, I hope not. I am believe that anything more than about 10 devices is
>>> the wrong way to configure Bacula, and with 300+ there is very little
>>> chance that it will work.
>> Would you care to explain why this will not work with Bacula ?
> Assuming you are writing to all of them, or a large number at the same time,
> there are a number of possible problems:
I would most likely write to 100 volumes per SD (300 volumes in total on
3 machines) simultaneously.
> - you may severely fragment your harddisk.
Not a problem in our case.
> - you will have 300 open file descriptors, which is too many on most systems
> especially Solaris systems.
Not a problem :
A 32-bit program using standard I/O is limited to 256 file descriptors.
A 64-bit program using standard I/O can use up to 2 billion descriptors.
The limit for 32 bit apps is easily tunable.
> - you will have 300 device descriptors open in Bacula, which could consume a
> substantial amount of memory, overhead, and lock usage.
I'll do some profiling (thank god for dtrace) to see whether or not this
is a problem. Memory is not a problem since the max on the type of
system we are using is 256 GB.
> - depending on what sort of Pools you have defined, you may start running into
> practical limits for Bacula for the number of simultaneous volumes you are
> using -- anything more than about 20 would concern me.
> - pruning and Volume management within Bacula may become more difficult due to
> the large number of Volumes involved.
This indeed needs a bit of research.
> - you may be starting and stopping the SD often to do device management. This
> could disrupt operations, and we are not planning to add code to the SD to
> reload its conf file while running.
I already scriptet my way out of this one ;-)
> The sum of the above *may* not be important. However, any one of them *could*
> be a real "killer". If you are only running a small number of concurrent
> jobs, some of the things I mention above may have a much smaller impact than
> I am imagining.
> Best regards,
Med venlig hilsen / Best Regards
Tlf. 75 53 35 00
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
Bacula-devel mailing list