Re: preferred hardware platform
On 9/9/06, Martin Schrvder [EMAIL PROTECTED] wrote: Not if you use rsnapshot. Memory use in rsync scales linearly with the number of files being synced. Hence, any backup app that's calls rsync on large trees will hit this limitation and die a horrible death. I hit this limit on a 300GB fs with ~160k files and 512MB RAM. My data grew, and my backups simply stopped working! Scary!! rdiff-backup however, does not suffer from this problem. And it's a bit more space efficient than rsnapshot as well. Give it a shot and I doubt you'll be disappointed. - Raja
Re: preferred hardware platform
On Sun, Sep 10, 2006 at 11:39:57AM +0530, Raja Subramanian wrote:
rdiff-backup however, does not suffer from this problem.
And it's a bit more space efficient than rsnapshot as
well. Give it a shot and I doubt you'll be disappointed.
I've had an experimental port of rdiff-backup 1.0.4 and librsync
(upon which it depends) available[0] for a month or two. I haven't
looked at it much since the ports tree locked, but it's working well
on several systems running -current.
Feel free to give it a try; I'd be glad to get feedback on it, but
it won't be a priority until after the thaw.
[0]http://www.lfod.us/openbsd/ports.html#rdiff-backup
http://www.lfod.us/openbsd/ports.html#librsync
--
o--{ Will Maier }--o
| web:...http://www.lfod.us/ | [EMAIL PROTECTED] |
*--[ BSD Unix: Live Free or Die ]--*
Re: preferred hardware platform
On 9/10/06, Will Maier [EMAIL PROTECTED] wrote:
On Sun, Sep 10, 2006 at 11:39:57AM +0530, Raja Subramanian wrote:
rdiff-backup however, does not suffer from this problem.
And it's a bit more space efficient than rsnapshot as
well. Give it a shot and I doubt you'll be disappointed.
I've had an experimental port of rdiff-backup 1.0.4 and librsync
(upon which it depends) available[0] for a month or two. I haven't
looked at it much since the ports tree locked, but it's working well
on several systems running -current.
I also have as a local port, because I needed a quick and dirty backup
solution. However, before I trust my system backup to this, I would
have to go through the librsync code and clean it up a lot. It's
scary.
Feel free to give it a try; I'd be glad to get feedback on it, but
it won't be a priority until after the thaw.
[0]http://www.lfod.us/openbsd/ports.html#rdiff-backup
http://www.lfod.us/openbsd/ports.html#librsync
--
o--{ Will Maier }--o
| web:...http://www.lfod.us/ | [EMAIL PROTECTED] |
*--[ BSD Unix: Live Free or Die ]--*
--
What is your function in life? - Killer
Re: preferred hardware platform
Hi Joachim, On 09/09/2006, at 10:02 AM, Joachim Schipper wrote: And seriously, how does one manage to fill a TB of data? video, lossless-compressed music, backups from a bunch of machines, none of our business really (-: I'll grant you the latter, but still... well, let's just say that that 40 GB tapedrive I mentioned is plenty for backing up everything I admin. Just imaging my girlfriends Thinkpad comes out to greater than 40GB (compressed image size). Then there are all my machines and files. I like to have both recent user file backups and also relatively recent (monthly) whole disk image backups to quickly recover from a failed disk. Re-installing Windows anything or even Mac OSX to a much lesser extent, is a pain in the bum. Shane J Pearson shanejp netspace net au
Re: preferred hardware platform
On Sat, 9 Sep 2006 15:58:54 +1000, Shane J Pearson [EMAIL PROTECTED] said: Hi Joachim, On 09/09/2006, at 10:02 AM, Joachim Schipper wrote: And seriously, how does one manage to fill a TB of data? video, lossless-compressed music, backups from a bunch of machines, none of our business really (-: I'll grant you the latter, but still... well, let's just say that that 40 GB tapedrive I mentioned is plenty for backing up everything I admin. Ugh, Joachim is just being silly. :-) There are a myriad of ways of filling up a TB of disk space. Data Mining, anyone?? (tho I don't think too many people are doing that using OBSD) BTW, a program should never, ever have to be larger than 640K. OK, move along. Ignore my late night Saturday drunken postings...
Re: preferred hardware platform
2006/9/9, Benjamin Collins [EMAIL PROTECTED]: mix). In a development environment in which one might have multiple working copies of a large repository (such as OpenBSD's src), all those backups add up, and fast. Not if you use rsnapshot. Best Martin
Re: preferred hardware platform
On Sat, Sep 09, 2006 at 05:33:56PM +0200, Martin Schr?der wrote: 2006/9/9, Benjamin Collins [EMAIL PROTECTED]: mix). In a development environment in which one might have multiple working copies of a large repository (such as OpenBSD's src), all those backups add up, and fast. Not if you use rsnapshot. rsnapshots looks very neat, yes. I do tape backups, myself, so I'll stay with AMANDA. On a side note, aside from it's braindead handling of ports (why one TCP connection doesn't suffice is beyond me), AMANDA is a very good piece of software. I wouldn't recommend it for disk-based backups, though - like many alternatives, it's strongly biased towards tapes. rsnapshot, on the other hand, understands at least a little of the difference between disk and tape (i.e. tape is cheap, disk is random-access). Joachim
Re: preferred hardware platform
On Fri, Sep 08, 2006 at 04:00:20PM +0200, Francois Slabbert wrote: Hi misc, I'm looking to build soho development and storage server, what would be the most stable current hardware configuration. I was thinking of along the lines of: * Intel 945G motherboard * Celeron CPU * 512MB of RAM * LSI Megaraid SATA-6 * 1+ TB of disks Data integrity and stability are my primary concerns. rant *WHAT* are you trying to use that for? Try mine: refurbished Dell Optiplex GX1, 400 MHz Pentium II, 128 MB memory, and two matching pairs of harddisks (6.1 and 4 GB) with a combination of RAIDframe, altroot, and regular backups guaranteeing data consistency. Runs mail, DNS, web, and a couple of other services, and has space to spare - and if it ever needs more, I'll hook up the ~120 GB of external SCSI drives lying around. The one `expensive' piece of hardware in there is the backup solution, a second-hand HP DAT 40GB tape drive and the second-hand Adaptec AH-2940UW driving it. I have a P3, 512 MB RAM, perform much the same functions for an office of six people, and about twenty mail accounts. There, load is about .2; memory usage varies from 1/4 to 1/2. This box was given to us for free; we added two brand-name IDE harddisks, at $100 total. My main workstation, which does all the compilation, has some more power; but my laptop, at which I'm currently sitting, is very similar to the above, but with only one hard disk. Granted, the above is for one person; but OpenBSD will perform admirably on refurb hardware, or pretty much whatever you want to run it on. There are only two things you really should spend some money on: good hard drives (though my four PoSes haven't failed me yet), and a good backup solution. And seriously, how does one manage to fill a TB of data? /rant Otherwise, while others might be more qualified to comment, this doesn't look too bad. I might go with more memory, though - it's cheap, and remarkably useful. Anything that expects to serve a TB of disk should have plenty of memory. Also, consider good network cards. I believe the sk(4) comes highly recommended; gigabit cards are usually preferably, due to better buffering and so on. Don't forget the piece about backup solutions in the rant; having easily accessible *and* off-site tapes is at least as useful as that whole RAID thingy, which can be done in software anyway. Joachim
Re: preferred hardware platform
I was thinking of along the lines of: * Intel 945G motherboard * LSI Megaraid SATA-6 Try and get a BBU for that MegaRAID card... MegaRAID SATA 150-6 will be happier on a PCI-X motherboard (this is _not_ the same thing as PCI-Express). (MegaRAID SATA 300-8x requires PCI-X). I've just noticed there is a new 4-lane PCI-Express SATA card listed on LSI's website: http://www.lsilogic.com/products/megaraid_sata/megaraid_sata_300_8elp.html I don't know if Marco has seen one yet (or has a suitable motherboard to plug one into even). It may work easily (pci id change), or there may be more complicated things to do. * 512MB of RAM ...and be sure to split the disk into smaller partitions unless you want to buy more RAM, fsck will not be amusing otherwise. And seriously, how does one manage to fill a TB of data? video, lossless-compressed music, backups from a bunch of machines, none of our business really (-:
Re: preferred hardware platform
Joachim Schipper wrote: Try mine: refurbished Dell Optiplex GX1, 400 MHz Pentium II, 128 MB memory, and two matching pairs of harddisks (6.1 and 4 GB) with a combination of RAIDframe, altroot, and regular backups guaranteeing data consistency. Runs mail, DNS, web, and a couple of other services, and has space to spare - and if it ever needs more, I'll hook up the ~120 GB of external SCSI drives lying around. Another option if you want cheap reliability is old server hardware. For example, there's a few Proliant 1850s (2x550MHz) on eBay for $70 with RAID and disks and 392mb RAM. Of course, by the time you ship it, you could have purchased three GX1s, and still sleep in the same room (fan noise).
Re: preferred hardware platform
On Fri, Sep 08, 2006 at 08:26:24PM +0100, Stuart Henderson wrote: I was thinking of along the lines of: * Intel 945G motherboard * LSI Megaraid SATA-6 * 512MB of RAM * 1+ TB of disk ...and be sure to split the disk into smaller partitions unless you want to buy more RAM, fsck will not be amusing otherwise. And seriously, how does one manage to fill a TB of data? video, lossless-compressed music, backups from a bunch of machines, none of our business really (-: I'll grant you the latter, but still... well, let's just say that that 40 GB tapedrive I mentioned is plenty for backing up everything I admin. And anyway, in the context of someone asking for advice, I'll gladly shower him with more of same than (s)he ever wanted. ;-) Joachim
Re: preferred hardware platform
And seriously, how does one manage to fill a TB of data? /rant DVB.
Re: preferred hardware platform
On 2006/09/08 20:26, I wrote: I've just noticed there is a new 4-lane PCI-Express SATA card listed on LSI's website: http://www.lsilogic.com/products/megaraid_sata/megaraid_sata_300_8elp.html ...though the 8308ELP which is definitely listed in mfi(4) and supports both SAS and SATA-II drives appears to be available a touch cheaper here (.uk) - who'dathoughtit... The board photos+specs look rather similar. I wonder if the distinction is in marketing to cater for people who, like me until just now, forgot that the SAS cards support SATA too...(I did wonder why there was a PCIE SCSI card and didn't used to be a PCIE SATA since it seemed such an obvious coupling but the penny only just dropped...)
Re: preferred hardware platform
On Fri, Sep 08, 2006 at 08:49:06PM +0200, Joachim Schipper wrote: And seriously, how does one manage to fill a TB of data? Quite easily, if you do daily, weekly, and monthly backups. My group at work doesn't do daily, but we do something like MWF, weekly, monthly, with tapes done weekly and kept up to 6 months, and we fill the better part of a 2TB raid (albeit we have about 25+ people in the mix). In a development environment in which one might have multiple working copies of a large repository (such as OpenBSD's src), all those backups add up, and fast. -- Benjamin Collins [demime 1.01d removed an attachment of type application/pgp-signature]
