Re: [Bacula-users] Speed writing to tape drive
Martin, I am not using Gzip in Bacula. The drive I have does hardware compression I thought that would be enough for now. I seem to be getting about 1.2T on an 800G LTO4 tape. Here is the iostat output during a full backup. I will try and do the tar test later this week. avg-cpu: %user %nice %system %iowait %steal %idle 0.280.000.080.200.00 99.43 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 0.41 3.9810.42 23966207 62765414 sdb 11.46 1159.01 623.79 6979515937 3756450992 sr1 0.00 0.00 0.00 8 0 sr2 0.00 0.00 0.00 8 0 avg-cpu: %user %nice %system %iowait %steal %idle 5.300.001.096.600.00 87.00 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 2.50 2.4027.97 24280 sdb 222.88 47629.17 298.10 476768 2984 sr1 0.00 0.00 0.00 0 0 sr2 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 6.260.001.446.090.00 86.21 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 1.00 0.8037.60 8376 sdb 261.60 54036.00 132.80 540360 1328 sr1 0.00 0.00 0.00 0 0 sr2 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 4.740.000.836.380.00 88.05 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 1.40 0.0031.20 0312 sdb 181.70 39100.80 133.60 391008 1336 sr1 0.00 0.00 0.00 0 0 sr2 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 6.270.001.365.590.00 86.78 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 0.50 0.0029.60 0296 sdb 240.30 54162.40 135.20 541624 1352 sr1 0.00 0.00 0.00 0 0 sr2 0.00 0.00 0.00 0 0 -Original Message- From: Martin Simmons [mailto:mar...@lispworks.com] Sent: Wednesday, July 08, 2009 9:13 AM To: Bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Speed writing to tape drive On Mon, 6 Jul 2009 10:14:09 -0700, Hayden Katzenellenbogen said: Hello, Thanks to a patch published two weeks ago I am finally making headway into the wonderful world of Bacula. I have a single machine right now with about 1.2T of data I am backing up. When I run the btape fill test I get write speeds of around 70MB/s when I run a full backup from the local machine I get about 20MB/s is there anyway to figure out where the bottle neck is? Are you using software compression (gzip) in Bacula? What do top and iostat 10 show when running a backup? You could try using tar to check the speed of your filesystem, e.g. time tar cf /dev/null /some/very/large/directory __Martin -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Speed writing to tape drive
John, For now the DB is on the same raid partition. I our DB admin is building a new high availability pair that I will move it onto soon. The data that I am accessing is local but like you said spooling should help I have about 1.3T of data. How much should I spool at a time? 50G? Thanks H -Original Message- From: John Drescher [mailto:dresche...@gmail.com] Sent: Wednesday, July 08, 2009 9:28 AM To: Hayden Katzenellenbogen Cc: Bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Speed writing to tape drive On Wed, Jul 8, 2009 at 12:12 PM, Martin Simmonsmar...@lispworks.com wrote: On Mon, 6 Jul 2009 10:14:09 -0700, Hayden Katzenellenbogen said: Hello, Thanks to a patch published two weeks ago I am finally making headway into the wonderful world of Bacula. I have a single machine right now with about 1.2T of data I am backing up. When I run the btape fill test I get write speeds of around 70MB/s when I run a full backup from the local machine I get about 20MB/s is there anyway to figure out where the bottle neck is? Are you using software compression (gzip) in Bacula? What do top and iostat 10 show when running a backup? You could try using tar to check the speed of your filesystem, e.g. time tar cf /dev/null /some/very/large/directory Could also be the database. I mean if the db is on the same raid array as the source disk do not expect 70MB/s backups. Spooling attributes should help some in this case. John -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Speed writing to tape drive
On Mon, Jul 13, 2009 at 12:26 PM, Hayden Katzenellenbogenhay...@nextlevelinternet.com wrote: John, For now the DB is on the same raid partition. I our DB admin is building a new high availability pair that I will move it onto soon. The data that I am accessing is local but like you said spooling should help I have about 1.3T of data. How much should I spool at a time? 50G? I use 5G for multi-terabyte backups but I am also running multiple concurrent jobs from different servers at the same time. John -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Speed writing to tape drive
On Mon, 6 Jul 2009 10:14:09 -0700, Hayden Katzenellenbogen said: Hello, Thanks to a patch published two weeks ago I am finally making headway into the wonderful world of Bacula. I have a single machine right now with about 1.2T of data I am backing up. When I run the btape fill test I get write speeds of around 70MB/s when I run a full backup from the local machine I get about 20MB/s is there anyway to figure out where the bottle neck is? Are you using software compression (gzip) in Bacula? What do top and iostat 10 show when running a backup? You could try using tar to check the speed of your filesystem, e.g. time tar cf /dev/null /some/very/large/directory __Martin -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Speed writing to tape drive
On Wed, Jul 8, 2009 at 12:12 PM, Martin Simmonsmar...@lispworks.com wrote: On Mon, 6 Jul 2009 10:14:09 -0700, Hayden Katzenellenbogen said: Hello, Thanks to a patch published two weeks ago I am finally making headway into the wonderful world of Bacula. I have a single machine right now with about 1.2T of data I am backing up. When I run the btape fill test I get write speeds of around 70MB/s when I run a full backup from the local machine I get about 20MB/s is there anyway to figure out where the bottle neck is? Are you using software compression (gzip) in Bacula? What do top and iostat 10 show when running a backup? You could try using tar to check the speed of your filesystem, e.g. time tar cf /dev/null /some/very/large/directory Could also be the database. I mean if the db is on the same raid array as the source disk do not expect 70MB/s backups. Spooling attributes should help some in this case. John -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Speed writing to tape drive
Hello, Thanks to a patch published two weeks ago I am finally making headway into the wonderful world of Bacula. I have a single machine right now with about 1.2T of data I am backing up. When I run the btape fill test I get write speeds of around 70MB/s when I run a full backup from the local machine I get about 20MB/s is there anyway to figure out where the bottle neck is? The storage in the machine is a 3ware SATA 9690SA card with eight 1TB drives in a raid6 configuration. The tape drive I have is a Dell TL2000 connected via 4gig qlogic fiber channel card. I am running Ubuntu 8.04.2LTS. Thanks -H Hayden Katzenellenbogen hayd...@nextlevelinternet.com NextLevel Internet 858-836-0700 www.nextlevelinternet.com By the way, we are never too busy for referrals! If you know someone who might be interested in our services (Hosted PBX, Voice, Internet, Metro Ethernet, Co-Location) or who is unhappy with their current communications provider, we will take exceptional care of them! -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
