[CentOS] Looking for experiences with filesystem choices....
Lots of opinions out there and I have read and read, so I have a home server that I have finally setup with a hot swap hdd cage and what I am planning on doing is copying my data drive every other day and rotating them offsite, haven't figured out how often though. So I did my first test last night and the backup drive was formatted with ext3 but looking to try to speed things up and was wondering what would be my best choice. Most of my data is on VM's and the hdd files on some of them are quite large, I have used JFS and reiser in the past and was leaning on going with JFS but am tempted to look at XFS. So what I was wondering are what are folks experiences (instead of opinions) with different filesystems and while I want speed it needs to be reliable since it will be my back up datarunning centos 5.4 x64 Thanks in advance... ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Looking for experiences with filesystem choices....
On 03/21/2010 04:01 PM, Tom Bishop wrote: Lots of opinions out there and I have read and read, so I have a home server that I have finally setup with a hot swap hdd cage and what I am planning on doing is copying my data drive every other day and rotating them offsite, haven't figured out how often though. So I did my first test last night and the backup drive was formatted with ext3 but looking to try to speed things up and was wondering what would be my best choice. Most of my data is on VM's and the hdd files on some of them are quite large, I have used JFS and reiser in the past and was leaning on going with JFS but am tempted to look at XFS. So what I was wondering are what are folks experiences (instead of opinions) with different filesystems and while I want speed it needs to be reliable since it will be my back up datarunning centos 5.4 x64 Thanks in advance... Hi, in December last year there was a nice thread about choosing the 'right' FS for certain circumstances, which included JFS, XFS, ext3/4 etc. http://lists.centos.org/pipermail/centos/2009-December/086842.html HTH, Timo ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Looking for experiences with filesystem choices....
Thanks Timo I'll go read that... On 3/21/10, Timo Schoeler timo.schoe...@riscworks.net wrote: On 03/21/2010 04:01 PM, Tom Bishop wrote: Lots of opinions out there and I have read and read, so I have a home server that I have finally setup with a hot swap hdd cage and what I am planning on doing is copying my data drive every other day and rotating them offsite, haven't figured out how often though. So I did my first test last night and the backup drive was formatted with ext3 but looking to try to speed things up and was wondering what would be my best choice. Most of my data is on VM's and the hdd files on some of them are quite large, I have used JFS and reiser in the past and was leaning on going with JFS but am tempted to look at XFS. So what I was wondering are what are folks experiences (instead of opinions) with different filesystems and while I want speed it needs to be reliable since it will be my back up datarunning centos 5.4 x64 Thanks in advance... Hi, in December last year there was a nice thread about choosing the 'right' FS for certain circumstances, which included JFS, XFS, ext3/4 etc. http://lists.centos.org/pipermail/centos/2009-December/086842.html HTH, Timo ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Looking for experiences with filesystem choices....
Tom Bishop wrote: Lots of opinions out there and I have read and read, so I have a home server that I have finally setup with a hot swap hdd cage and what I am planning on doing is copying my data drive every other day and rotating them offsite, haven't figured out how often though. So I did my first test last night and the backup drive was formatted with ext3 but looking to try to speed things up and was wondering what would be my best choice. Most of my data is on VM's and the hdd files on some of them are quite large, I have used JFS and reiser in the past and was leaning on going with JFS but am tempted to look at XFS. So what I was wondering are what are folks experiences (instead of opinions) with different filesystems and while I want speed it needs to be reliable since it will be my back up datarunning centos 5.4 x64 Thanks in advance... Not the question you asked, but I'm guessing the choice of backup method for copying the data will have far more effect than the choice of filesystem. How are you backing up the data? Presumably something like rsync will speed up matters considerably over a straight copy once the first pass is done. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Looking for experiences with filesystem choices....
Ned Slider wrote: Tom Bishop wrote: Lots of opinions out there and I have read and read, so I have a home server that I have finally setup with a hot swap hdd cage and what I am planning on doing is copying my data drive every other day and rotating them offsite, haven't figured out how often though. So I did my first test last night and the backup drive was formatted with ext3 but looking to try to speed things up and was wondering what would be my best choice. Most of my data is on VM's and the hdd files on some of them are quite large, I have used JFS and reiser in the past and was leaning on going with JFS but am tempted to look at XFS. So what I was wondering are what are folks experiences (instead of opinions) with different filesystems and while I want speed it needs to be reliable since it will be my back up datarunning centos 5.4 x64 Thanks in advance... Not the question you asked, but I'm guessing the choice of backup method for copying the data will have far more effect than the choice of filesystem. How are you backing up the data? Presumably something like rsync will speed up matters considerably over a straight copy once the first pass is done. Yes, I'd go for 'well-tested' and 'reliable' over speed on a backup drive. Some filesystems are faster at creating/deleting large numbers of files but if you use rysnc you'll only track the changes after the first run. Note that rsync keeps the whole directory tree in memory during the copy (and probably 2 copies for same-machine runs) so if you have a large number of files it will help to have plenty of RAM. This is supposed to be improved in the 3.x versions of rsync (available from rpmforge). -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Looking for experiences with filesystem choices....
On Sun, 2010-03-21 at 14:35 -0500, Les Mikesell wrote: Ned Slider wrote: Tom Bishop wrote: Lots of opinions out there and I have read and read, so I have a home server that I have finally setup with a hot swap hdd cage and what I am planning on doing is copying my data drive every other day and rotating them offsite, haven't figured out how often though. So I did my first test last night and the backup drive was formatted with ext3 but looking to try to speed things up and was wondering what would be my best choice. Yea a lot of opinions and ideas out. Stop keeping your user generated data in VMs and instead use NFS or CIFS for network storage. Or all together run the VMs from the network. Then replicate your data. Rsynce can be cpu bound at times. BUT: Since you have one server get a good raid 1 sata card and use it like arecca. You don't have to invest a fortune in it. Finally rsynce is going to bite you in the a$$ copying live data. It may work perfect for the first few times but in the end it will get you. You have to be willing to sacrifice data speed for reliability. Data speed is crap when it come to reliable data. Ask yourself if you want your precious data from 5 years ago or that 1Gig line speed??? John ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Looking for experiences with filesystem choices....
JohnS wrote: BUT: Since you have one server get a good raid 1 sata card and use it like arecca. You don't have to invest a fortune in it. Finally rsynce is going to bite you in the a$$ copying live data. It may work perfect for the first few times but in the end it will get you. You have to be willing to sacrifice data speed for reliability. Data speed is crap when it come to reliable data. Ask yourself if you want your precious data from 5 years ago or that 1Gig line speed??? Rsync is actually about as good as it gets at copying live data. It can't get a good copy of a file that changes while the copy is made, but neither can anything else and it's speed makes the odds better. One thing you can do with rsync is make a 2nd run with the -v option. This will only copy (and list) files that have changed since the 1st pass so you can see if anything important is open and active. If the 2nd run does not list any files - or if they are just things like growing logfiles, you should have a clean copy. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Looking for experiences with filesystem choices....
So let me add some more detail, I have a hot swap disk cage that currently has my backup data on it almost all of them are in VM's. That disk is my primary data disk, all I need/want to do is to copy that data periodically to another disk that I then can rotate offsite, so I will have 2 more disks and will swap one in and out then during the night either copy or rsync the data to the offsite disk, rinse, wash and repeatI have plenty of cpu, 16cores and lots of memory 32Gigso this disk will not be my primary backup disk but just my offsite disaster recovery in case the other one ever bites the dust... On Sun, Mar 21, 2010 at 3:18 PM, JohnS jse...@gmail.com wrote: On Sun, 2010-03-21 at 14:35 -0500, Les Mikesell wrote: Ned Slider wrote: Tom Bishop wrote: Lots of opinions out there and I have read and read, so I have a home server that I have finally setup with a hot swap hdd cage and what I am planning on doing is copying my data drive every other day and rotating them offsite, haven't figured out how often though. So I did my first test last night and the backup drive was formatted with ext3 but looking to try to speed things up and was wondering what would be my best choice. Yea a lot of opinions and ideas out. Stop keeping your user generated data in VMs and instead use NFS or CIFS for network storage. Or all together run the VMs from the network. Then replicate your data. Rsynce can be cpu bound at times. BUT: Since you have one server get a good raid 1 sata card and use it like arecca. You don't have to invest a fortune in it. Finally rsynce is going to bite you in the a$$ copying live data. It may work perfect for the first few times but in the end it will get you. You have to be willing to sacrifice data speed for reliability. Data speed is crap when it come to reliable data. Ask yourself if you want your precious data from 5 years ago or that 1Gig line speed??? John ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Looking for experiences with filesystem choices....
On Sun, 2010-03-21 at 16:20 -0500, Tom Bishop wrote: So let me add some more detail, I have a hot swap disk cage that currently has my backup data on it almost all of them are in VM's. That disk is my primary data disk, all I need/want to do is to copy that data periodically to another disk that I then can rotate offsite, so I will have 2 more disks and will swap one in and out then during the night either copy or rsync the data to the offsite disk, rinse, wash and repeatI have plenty of cpu, 16cores and lots of memory 32Gigso this disk will not be my primary backup disk but just my offsite disaster recovery in case the other one ever bites the dust... Ok then you skimped on the 16 Cores. Copying that data is fine and dandy with me. But don't count all your eggs until there hatched. It's just you have to understand something here don't rely on copying any data from one single disk here now. At least have for the primary data to = 2 disks mirrored atleast. It's ludicrous to have a 16 core machine with one data disk. That's is just completly insane in the IT evironment I work in. It's unheard of. Check this out: Lets Think: That one single data disk dies or gets corrupted and you have your data Stored Offsite like you plan on doing? You dead in the water already. How long will it take to grab the Offsite data. Really think about it now. I am not being a but about it I'm just telling you what happens in real life and has happened. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
