Re: [Bacula-users] 350TB backup

Robert M. Candey Mon, 19 Oct 2015 15:28:09 -0700

We have 80TB of heliophysics data that we mirror with rsync daily to another storage server for fast switchover. We use Bacula to make quarterly full backups to LTO-5 tapes that we send to another building (and annually to an Iron Mountain facility with 10 year retention), and incremental and differential backups in between to another tape pool. We split the full backups into 5 jobs by parts of the directory hierarchy in order to keep the backups under a week long, with the 5th job being everything not included in the specific directories of the first 4 jobs, and the PostgreSQL catalog. It made a difference to run the backups from a separate server with dedicated spool RAID array, 48GB RAM, and Fibre-Channel to the tape library, with the servers connected through a 10GbE Ethernet switch. We'll soon be getting 80TB more data a year and so are getting a LTO-7 library and putting the mirrored storage in separate buildings (GlusterFS on top of ZFS). I'm also thinking of using SSDs for the spool area; does anyone have recommendations on that?

Robert Candey

-------- Original Message --------
Subject: Re: [Bacula-users] 350TB backup
From: Thing <[email protected]>
To: [email protected] <[email protected]>
Cc: "[email protected]" <[email protected]>
Date: Mon Oct 19 2015 17:49:23 GMT-0400 (EDT)
Multiple NFS file systems on a NAS array.  500TB total, 350TB used.
Research data, much of it rarely accessed, after 1 year things like climate
data up to 30 years old, probably highly compressible.   Suspect multiple
bacula backup instances to distribute the load? Growth about 30tb a year.
On 20 October 2015 at 10:08, Dimitri Maziuk <[email protected]> wrote:
On 10/19/2015 03:53 PM, Thing wrote:
Hi,

Is anyone backing total volumes of this order?  and if so, what sort of
scaling, design, hardware?
I take it, that's the size of your filesystems? Not the estimated size
of the backup set (i.e. all cycles in retention period)?

--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Multiple NFS file systems on a NAS array. 500TB total, 350TB used. Research data, much of it rarely accessed, after 1 year things like climate data up to 30 years old, probably highly compressible. Suspect multiple bacula backup instances to distribute the load? Growth about 30tb a year.

On 20 October 2015 at 10:08, Dimitri Maziuk <[email protected]> wrote:

On 10/19/2015 03:53 PM, Thing wrote:
> Hi,
>
> Is anyone backing total volumes of this order? and if so, what sort of
> scaling, design, hardware?

I take it, that's the size of your filesystems? Not the estimated size
of the backup set (i.e. all cycles in retention period)?

--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu

------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] 350TB backup

Reply via email to