On 14-10-30 07:50 AM, Jeff MacDonald wrote:
Tips/Suggestions?
Jeff.
What is the content of your backups? Some things (ie thousands of tiny
files) will cause a lot of seeks on the machine to be backed up. If you
aren't using attribute spooling then each backed up file also causes a
record to be inserted in to the database, which may take time depending
on your DB environment.
The 'suggestions' for tuning will be different if you are backing up a
few dozen 10GB files versus backing up a million 10kb files.
Its mostly a windows os, with all its sundry smaller files and a few
larger database dumps etc.
I guess what I have to accertain is the slow part getting the data
FROM the servers or the slow part putting the data TO the storage.
I’m not sure which value the rate is in the job report, or if rate is
somehow encompassing both.
jeff
The job report rate will be the final average rate of the job, it
doesn't know/specify the difference between the 'input' rate and the
'output' rate.
Yep, you're going to need to do some investigation on the storage side
of the VM machine you are backing up, the director itself, the storage
daemon itself (though I'm guessing it is on the same system as the
director for you) and the final storage.
Also it's not quite clear from your description, is the final storage on
a different NAS all together from your VMs? (hoping so!) What
virtualization platform are you running?
Finally the question about attribute spooling is a big one - if you are
backing up a lot of small files and you do not have attribute spooling
turned on, you will have abysmal performance especially if the director
is running on the same disks that you are backing up.
Database writes are (almost) always synchronous writes, meaning the
system will stop and wait for the storage layer to say "yes the data is
ACTUALLY committed to disk" before proceeding. If you are seeking all
over backing up a bunch of small files, then trying to do a whole ton of
tiny DB writes at the same time to the same spindles your hard drive
heads are going to be flying around like crazy. An array of 7200 RPM
disks in any sort of parity RAID configuration will not be able to
handle more than 50-90 random IOPs (Operations per Second) at best in
real life, with a DB write or a file read counting as an IOP. If you
are backing up lots of small files randomly distributed around the
storage you are quite likely hitting an IOP wall - an IOP to read the
file and an IOP to write the DB record means not more than 25-45 files
per second. 4kb files = 100-180kb/sec and a completely maxed out
storage layer.
Even WITH attribute spooling enabled you are still going to be in a
less-than-ideal position since the spooled attributes still need to be
written to the same spindles with the hardware configuration you've
described.
Bryn
------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
