Re: [Bacula-users] Hardware configuration and off-site backup
On 06/05/10 15:03, Phil Stracchino wrote: On 05/06/10 02:57, Vlamsdoem wrote: On 05/05/10 15:12, Phil Stracchino wrote: On 05/05/10 08:38, John Drescher wrote: Sorry my servers are on gigabit links. How do you come to 9MB/s with a 100Mb link, is it not equals to 12,5 MB/s? Overhead. If it's correct on a gigabit link I would have a rate transfer of 90MB/s You will probably get less than that if you do not use jumbo frames. I did an throughput test with iperf between 2 servers on a gigabit link and it results in a 940Mb/s transfer rate . You tell me that transfer rate will be less than 90MB/s, is there so much overhead in the application layer? It really varies. I routinely get 95Mbit real-world throughput across my 100Mbit network; I know other people using different hardware or different configurations who've never seen 90Mbit. Gigabit is the same way. If you're getting 940Mbit throughput you're doing well, but remember that actual application throughput may not hit that. Ok with all this information I think I'll try with the sata hard disks. They seems fast enough to achieve everthing in time. Thanks to everbody who helped me on this thread. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
On 05/05/10 15:12, Phil Stracchino wrote: On 05/05/10 08:38, John Drescher wrote: Sorry my servers are on gigabit links. How do you come to 9MB/s with a 100Mb link, is it not equals to 12,5 MB/s? Overhead. If it's correct on a gigabit link I would have a rate transfer of 90MB/s You will probably get less than that if you do not use jumbo frames. I did an throughput test with iperf between 2 servers on a gigabit link and it results in a 940Mb/s transfer rate . You tell me that transfer rate will be less than 90MB/s, is there so much overhead in the application layer? and the transfer rate of the sata disks are 3Gb/s(375MB/s). No way are your SATA drives that fast. More likely 1/4 to 1/3 of that speed unless all servers are using high end SSDs. That. Always remember that the data transfer rates specified on disk interfaces are the maximum burst transfer rate FROM a full disk cache or TO an empty one. The actual sustained rates at which the physical mechanism can read or write data to and from the platters are FAR lower. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
On 05/06/10 02:57, Vlamsdoem wrote: On 05/05/10 15:12, Phil Stracchino wrote: On 05/05/10 08:38, John Drescher wrote: Sorry my servers are on gigabit links. How do you come to 9MB/s with a 100Mb link, is it not equals to 12,5 MB/s? Overhead. If it's correct on a gigabit link I would have a rate transfer of 90MB/s You will probably get less than that if you do not use jumbo frames. I did an throughput test with iperf between 2 servers on a gigabit link and it results in a 940Mb/s transfer rate . You tell me that transfer rate will be less than 90MB/s, is there so much overhead in the application layer? It really varies. I routinely get 95Mbit real-world throughput across my 100Mbit network; I know other people using different hardware or different configurations who've never seen 90Mbit. Gigabit is the same way. If you're getting 940Mbit throughput you're doing well, but remember that actual application throughput may not hit that. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
On 04/05/10 13:51, Thomas Mueller wrote: Am Tue, 04 May 2010 08:34:15 +0200 schrieb Vlamsdoem: Hello, I have few questions about my new backup configuration. Before asking my questions I will give you some useful information about what I need to do. I need a backup ± 10 servers with one full backup every week of 3,5 TB and an incremental backup of 100GB 5 times/week. I have 48 hours to do the full backup and about 8 hours to do the incremental backup. Every servers are on a 100Mb ethernet link. Considering all this parameters I need some advice about the choice of my hard disks. I'm planing to buy a StorageWorks array from Hp (DAS) but I don't know what's the best choice for the hard disks. I'm pretty sure that a fast hard disk is better for performance but it's not for money saving :), I'm wondering if big sata disks of 2TB/7.2k rpm are fast enough to achieve my backups in time or is it better to buy faster/sas disks to be sure it's done in time. 100Mb == 100mbit? 100Mbit will result in about 9MB/s . To transfer 3,5TB over ethernet you would need about 108hours. you need _at least_ 1Gbit ethernet on the backup-server. if there are millions of small files involved on the backedup servers, there will be much less throuhput. also do not place the DB data not on the same spindles as the backup volumes. IMHO the backup disks will not be the bottleneck if you go with SATA 7.2k drives. but check the specs - i'm sure HP is providing performance data somewhere on the homepage (or ask your dealer) - Thomas -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users Sorry my servers are on gigabit links. How do you come to 9MB/s with a 100Mb link, is it not equals to 12,5 MB/s? If it's correct on a gigabit link I would have a rate transfer of 90MB/s and the transfer rate of the sata disks are 3Gb/s(375MB/s). In fact the sata hard disks are not be the bottleneck, that's great news :). I'll save my DB data on an other disk, thanks for the advice. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
Sorry my servers are on gigabit links. How do you come to 9MB/s with a 100Mb link, is it not equals to 12,5 MB/s? Overhead. If it's correct on a gigabit link I would have a rate transfer of 90MB/s You will probably get less than that if you do not use jumbo frames. and the transfer rate of the sata disks are 3Gb/s(375MB/s). No way are your SATA drives that fast. More likely 1/4 to 1/3 of that speed unless all servers are using high end SSDs. In fact the sata hard disks are not be the bottleneck, that's great news :). I'll save my DB data on an other disk, thanks for the advice. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- John M. Drescher -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
and the transfer rate of the sata disks are 3Gb/s(375MB/s). No way are your SATA drives that fast. More likely 1/4 to 1/3 of that speed unless all servers are using high end SSDs. I forgot to mention that this ~100MB/s is only for large sequential reads. When reads get small or random the transfer rate can get less than 10 MB/s per drive. John -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
On 05/05/10 08:38, John Drescher wrote: Sorry my servers are on gigabit links. How do you come to 9MB/s with a 100Mb link, is it not equals to 12,5 MB/s? Overhead. If it's correct on a gigabit link I would have a rate transfer of 90MB/s You will probably get less than that if you do not use jumbo frames. and the transfer rate of the sata disks are 3Gb/s(375MB/s). No way are your SATA drives that fast. More likely 1/4 to 1/3 of that speed unless all servers are using high end SSDs. That. Always remember that the data transfer rates specified on disk interfaces are the maximum burst transfer rate FROM a full disk cache or TO an empty one. The actual sustained rates at which the physical mechanism can read or write data to and from the platters are FAR lower. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
If it's correct on a gigabit link I would have a rate transfer of 90MB/s ... and the transfer rate of the sata disks are 3Gb/s(375MB/s). ... No way are your SATA drives that fast. More likely 1/4 to 1/3 of that If the link is 90 MB/s and drives deliver 1/4 of 375 MB/s(94 MB/s) - that's a throughput of about 90 MB/s. That's a whopping speed of 324 GB/hr. You must be going through a fiber switch to a striped array of sorts. Mehma -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
No way are your SATA drives that fast. More likely 1/4 to 1/3 of that If the link is 90 MB/s and drives deliver 1/4 of 375 MB/s(94 MB/s) - that's a throughput of about 90 MB/s. That's a whopping speed of 324 GB/hr. You must be going through a fiber switch to a striped array of sorts. Agreed. That was assuming the unrealistic scenario where all reads (no writes) were large sequential reads on the outer tracks of the hard drive. John -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
Am Tue, 04 May 2010 08:34:15 +0200 schrieb Vlamsdoem: Hello, I have few questions about my new backup configuration. Before asking my questions I will give you some useful information about what I need to do. I need a backup ± 10 servers with one full backup every week of 3,5 TB and an incremental backup of 100GB 5 times/week. I have 48 hours to do the full backup and about 8 hours to do the incremental backup. Every servers are on a 100Mb ethernet link. Considering all this parameters I need some advice about the choice of my hard disks. I'm planing to buy a StorageWorks array from Hp (DAS) but I don't know what's the best choice for the hard disks. I'm pretty sure that a fast hard disk is better for performance but it's not for money saving :), I'm wondering if big sata disks of 2TB/7.2k rpm are fast enough to achieve my backups in time or is it better to buy faster/sas disks to be sure it's done in time. 100Mb == 100mbit? 100Mbit will result in about 9MB/s . To transfer 3,5TB over ethernet you would need about 108hours. you need _at least_ 1Gbit ethernet on the backup-server. if there are millions of small files involved on the backedup servers, there will be much less throuhput. also do not place the DB data not on the same spindles as the backup volumes. IMHO the backup disks will not be the bottleneck if you go with SATA 7.2k drives. but check the specs - i'm sure HP is providing performance data somewhere on the homepage (or ask your dealer) - Thomas -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Hardware configuration and off-site backup
On Tue, 4 May 2010, Thomas Mueller wrote: IMHO the backup disks will not be the bottleneck if you go with SATA 7.2k drives. but check the specs - i'm sure HP is providing performance data somewhere on the homepage (or ask your dealer) IMHO it's unsafe to back up to a single disk (of any type). Using a RAID set gives much greater resilience for the failures that inevitably happen as time goes by. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
