On 03/29/10 06:53 PM, Kern Sibbald wrote:
> On Monday 29 March 2010 18:01:56 Henrik Johansen wrote:
>> On 03/29/10 05:08 PM, Kern Sibbald wrote:
>>> On Monday 29 March 2010 16:59:05 Phil Stracchino wrote:
>>>> On 03/29/10 08:22, Henrik Johansen wrote:
>>>>> We'll top 300+ devices per SD once we are finished migrating.
>>>> I hope you'll post a full writeup somewhere.  :)
>>> Oh, I hope not.  I am believe that anything more than about 10 devices is
>>> the wrong way to configure Bacula, and with 300+ there is very little
>>> chance that it will work.
>> Would you care to explain why this will not work with Bacula ?
> Assuming you are writing to all of them, or a large number at the same time,
> there are a number of possible problems:

I would most likely write to 100 volumes per SD (300 volumes in total on 
3 machines) simultaneously.

> - you may severely fragment your harddisk.

Not a problem in our case.

> - you will have 300 open file descriptors, which is too many on most systems
> especially Solaris systems.

Not a problem :

A 32-bit program using standard I/O is limited to 256 file descriptors. 
A 64-bit program using standard I/O can use up to 2 billion descriptors.

The limit for 32 bit apps is easily tunable.

> - you will have 300 device descriptors open in Bacula, which could consume a
> substantial amount of memory, overhead, and lock usage.

I'll do some profiling (thank god for dtrace) to see whether or not this 
is a problem. Memory is not a problem since the max on the type of 
system we are using is 256 GB.

> - depending on what sort of Pools you have defined, you may start running into
> practical limits for Bacula for the number of simultaneous volumes you are
> using -- anything more than about 20 would concern me.
> - pruning and Volume management within Bacula may become more difficult due to
> the large number of Volumes involved.

This indeed needs a bit of research.

> - you may be starting and stopping the SD often to do device management. This
> could disrupt operations, and we are not planning to add code to the SD to
> reload its conf file while running.

I already scriptet my way out of this one ;-)

> The sum of the above *may* not be important.  However, any one of them *could*
> be a real "killer".   If you are only running a small number of concurrent
> jobs, some of the things I mention above may have a much smaller impact than
> I am imagining.
> Best regards,
> Kern

Med venlig hilsen / Best Regards

Henrik Johansen
Tlf. 75 53 35 00

ScanNet Group
A/S ScanNet

Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
Bacula-devel mailing list

Reply via email to