Re: [SO]HO Software RAID5 server: which implementation should I choice?
Miroslav Lachman a écrit : Arnaud Houdelette wrote: [...] Geom_raid5 is (unfortunatly ?) not part of Freebsd base. You'll have to download and install the module and utility binaries and follow the (simple) instructions from this website : http://home.tiscali.de/cmdr_faako/graid5-howto.html In the meantime, somebody convinced me to give zfs a try, I backed up my data, converted the raid array to raidz pool and I must say I'm not disappointed. + Read performance (~160 Mo/s) + Instant snapshots + zfs filesystems goodness + better support from the community - Stability issues : zfs and kernel need to be tuned - Drive crash scenario may be a bit more complex Do you have any stability issues after tuning? What settings you are using? I am testing ZFS for a short time with these values: vm.kmem_size=1024M vm.kmem_size_max=1024M vfs.zfs.prefetch_disable=1 kern.maxvnodes=40 vfs.zfs.zil_disable=1 (on Sun Fire X2100 with 4GB of RAM and FreeBSD 7.0-RELEASE amd64) It seems to be stable. Miroslav Lachman Hi Box is an AMD64 3200+ with 512MB of RAM. The only tuning I did to get rid of panics : vm.kmem_size=512M vm.kmem_size_max=512M I did not disable prefetch nor zil. But It's only a home NAS : load on the filesystem is reduced : samba, and rarely more than 2 client PC. Arnaud Houdelette ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [SO]HO Software RAID5 server: which implementation should I choice?
Miroslav Lachman wrote: Do you have any stability issues after tuning? What settings you are using? I am testing ZFS for a short time with these values: vm.kmem_size=1024M vm.kmem_size_max=1024M vfs.zfs.prefetch_disable=1 kern.maxvnodes=40 vfs.zfs.zil_disable=1 (on Sun Fire X2100 with 4GB of RAM and FreeBSD 7.0-RELEASE amd64) It seems to be stable. Miroslav Lachman _ I'm also running ZFS and wanted to share my experiences. It doesn't cope well with low-memory environments, but I've successfully run with 2GB ram and 3TB disk with no problems on both i386 and amd64. amd64 needs a little bit of tuning - increasing kmem and whatnot (well documented, not very difficult/stressful) i386 needs a bit more tuning and a kernel recompile (increase KVA_PAGES) but once you get it working it runs fine. I've heard dire warnings that disabling the zil is a terribly bad idea if you're running anything that tries to ensure data file consistency (like a database and nfs or something) To sum up - tune it and it will work wonders for you. Don't try to run it with minimal RAM though - I've had good luck with 2GB+ (and half or more of that allocated to kernel memory) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [SO]HO Software RAID5 server: which implementation should I choice?
On Mon, Apr 21, 2008 at 01:04:20PM -0700, Steven Schlansker wrote: I'm also running ZFS and wanted to share my experiences. It doesn't cope well with low-memory environments, but I've successfully run with 2GB ram and 3TB disk with no problems on both i386 and amd64. amd64 needs a little bit of tuning - increasing kmem and whatnot (well documented, not very difficult/stressful) Since pjd's nokva commit, I have had zero panics on amd64, with no tuning whatsoever. http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/vm/vm_kern.c.diff?r1=1.130;r2=1.131 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [SO]HO Software RAID5 server: which implementation should I choice?
Nenhum_de_Nos wrote: On Wed, Apr 9, 2008 at 9:47 AM, Arnaud Houdelette [EMAIL PROTECTED] wrote: Lev Serebryakov a écrit : Hello, freebsd-stable. Does somebody use some software RAID5 on FreeBSD in real production system? I want to build storage server for my home: RAW photos, multi-layer PhotoShop files and FLAC-encoded music consume a lot of space, and they should be availible both from desktop notebook. Also, all photo-content is unique, so I need some insuranse from single HDD crash. I understand, that I will not safe from fire, PSU failure and thing slike this. I selected hardware platform: Intel Q35-based MoBo with 6xSATA-II ports (all of them is chipset-based, so no SiliconImage/JMicron/Whatever crappy controllers), some low-end Core2Duo, 2Gb of memory. Storage will be 5x500Gb WD HDDs for RAID + one small HDD for boot, system, swap, etc. I want to have 2Tb (ok, not real Tb, I know) of protected storage. I want to have maximum speed via 1Gb network, because graphic files are big and should open fast. Not as fast as local ones, I understand that, but speeds about 12-15Mb/s is not enough for sure :) Only problem I see: which software RAID5 solution should I prefer? FreeBSD-based, of course! I see these variants: (1) FreeBSD 6(7?) + graid3. Slow, one disk for checksums is bottleneck, as far as I understand. (2) FreeBSD 6(7?) + gvinum/radi5. Is it stable enough?! Is it complete? when I try it about 6 months ago in VMWare installation with 5 virtual disks, I got panics and strange behaviour after crashing one of virtual disks. (3) FreeBSD 6(7?) + graid5. Again, is it stable enough? There are THREE versions of it. Which one should I prefer? There was long thread about it some times ago without any clear conclusion. Does something changed? (4) FreeBSD 7 + ZFS zraid. And again: stability. Too many messages about locks, crashes, etc. Code is experemental. Is it only for 32 bit systems? (5) Do I miss something? (6) Solaris + ZFS? I don't want it, I know a little about Solaris administaration, and I already have FreeBSD servers and routers. I know, that 3ware or Areca controllers are very good. I know, that gmirror is very stable. But these variants are too expensive for home server :( Does somebody use some software RAID5 on FreeBSD in real production? Any advices? Hi ! I personally use the 3 option for my personal Home File Server. I got approximatly the same usage for the file server (mostly video, music, photo). I built my own about 12 month ago. I reviewed the about the same variants as the one you propose : (1) Discarded for performance issues. Raid3 is slow. Really. (2) raid5/vinum is also slow. And as I understood at that time, recovery from lost hard drive wasn't easy enough for the freebsd niubee I was then. (4) ZFS wasn't there yet. But I did test it on a test VMWare, and wasn't convinced (mostly stability and memory issues). So I use geom_raid5. I sticked to the main distributions, which seemed more stable at the moment. The kernel module is fairly simple to build/install. Performance is (very) good for a software raid. I successfully switched the raid array from an i386 6.2-RELEASE to an an amd64 7.0-RELEASE (with motherboard and CPU change) without any assle. For the moment, I use one big UFS+SU (and snapshots) on the whole array. I successfuly tried unpplugging then replugging on of the drives, suddent power loss, using the array with a missing disk (degraded mode). All did work fine. (still, I use an UPS on the file server). The sole issue I had is with ataidle. I had to patch ata-disc.c to increase the IO timeout. Without, the raid5 module detected temporary disk loss and constantly launched rebuilds of the array. With 7.0, I wondered if I should use gjournal, but I'm not sure if it's really the way to go on a file system dedicated to store many big files. So I stick to soft updates. Current configuration is : / on a 2GB usb key /tmp on memory ports and source trees (and some portsnap stuff) on a small disk 4x250 GB sata for the raid5 array. AMD A64 3200+ and 512 GB DDRII Realtek Gigabit nics. Copy from raid5 to /dev/null gives about 100MB/s Copy from /dev/random to raid5 about 40MB/s I use samba shares. I get about 40MB/s in both ways from another computer on the network (enabling jumbo-frames gives a big boost). Hope my own story can help you in any way. Regards, Arnaud Houdelette I know its been quite some time from the mail, but if you could say where to find this module. eikipedia says its on freebsd 7 but there is not this module for me (/boot/kernel/ there is no raid5 file). everytime I search the internet I find old stuff about it. If you could point me the site/article/anything :) thanks, matheus Geom_raid5 is (unfortunatly ?) not part of Freebsd base. You'll have to download and install the module and utility binaries and
Re: [SO]HO Software RAID5 server: which implementation should I choice?
Arnaud Houdelette wrote: [...] Geom_raid5 is (unfortunatly ?) not part of Freebsd base. You'll have to download and install the module and utility binaries and follow the (simple) instructions from this website : http://home.tiscali.de/cmdr_faako/graid5-howto.html In the meantime, somebody convinced me to give zfs a try, I backed up my data, converted the raid array to raidz pool and I must say I'm not disappointed. + Read performance (~160 Mo/s) + Instant snapshots + zfs filesystems goodness + better support from the community - Stability issues : zfs and kernel need to be tuned - Drive crash scenario may be a bit more complex Do you have any stability issues after tuning? What settings you are using? I am testing ZFS for a short time with these values: vm.kmem_size=1024M vm.kmem_size_max=1024M vfs.zfs.prefetch_disable=1 kern.maxvnodes=40 vfs.zfs.zil_disable=1 (on Sun Fire X2100 with 4GB of RAM and FreeBSD 7.0-RELEASE amd64) It seems to be stable. Miroslav Lachman ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [SO]HO Software RAID5 server: which implementation should I choice?
On Wed, Apr 9, 2008 at 9:47 AM, Arnaud Houdelette [EMAIL PROTECTED] wrote: Lev Serebryakov a écrit : Hello, freebsd-stable. Does somebody use some software RAID5 on FreeBSD in real production system? I want to build storage server for my home: RAW photos, multi-layer PhotoShop files and FLAC-encoded music consume a lot of space, and they should be availible both from desktop notebook. Also, all photo-content is unique, so I need some insuranse from single HDD crash. I understand, that I will not safe from fire, PSU failure and thing slike this. I selected hardware platform: Intel Q35-based MoBo with 6xSATA-II ports (all of them is chipset-based, so no SiliconImage/JMicron/Whatever crappy controllers), some low-end Core2Duo, 2Gb of memory. Storage will be 5x500Gb WD HDDs for RAID + one small HDD for boot, system, swap, etc. I want to have 2Tb (ok, not real Tb, I know) of protected storage. I want to have maximum speed via 1Gb network, because graphic files are big and should open fast. Not as fast as local ones, I understand that, but speeds about 12-15Mb/s is not enough for sure :) Only problem I see: which software RAID5 solution should I prefer? FreeBSD-based, of course! I see these variants: (1) FreeBSD 6(7?) + graid3. Slow, one disk for checksums is bottleneck, as far as I understand. (2) FreeBSD 6(7?) + gvinum/radi5. Is it stable enough?! Is it complete? when I try it about 6 months ago in VMWare installation with 5 virtual disks, I got panics and strange behaviour after crashing one of virtual disks. (3) FreeBSD 6(7?) + graid5. Again, is it stable enough? There are THREE versions of it. Which one should I prefer? There was long thread about it some times ago without any clear conclusion. Does something changed? (4) FreeBSD 7 + ZFS zraid. And again: stability. Too many messages about locks, crashes, etc. Code is experemental. Is it only for 32 bit systems? (5) Do I miss something? (6) Solaris + ZFS? I don't want it, I know a little about Solaris administaration, and I already have FreeBSD servers and routers. I know, that 3ware or Areca controllers are very good. I know, that gmirror is very stable. But these variants are too expensive for home server :( Does somebody use some software RAID5 on FreeBSD in real production? Any advices? Hi ! I personally use the 3 option for my personal Home File Server. I got approximatly the same usage for the file server (mostly video, music, photo). I built my own about 12 month ago. I reviewed the about the same variants as the one you propose : (1) Discarded for performance issues. Raid3 is slow. Really. (2) raid5/vinum is also slow. And as I understood at that time, recovery from lost hard drive wasn't easy enough for the freebsd niubee I was then. (4) ZFS wasn't there yet. But I did test it on a test VMWare, and wasn't convinced (mostly stability and memory issues). So I use geom_raid5. I sticked to the main distributions, which seemed more stable at the moment. The kernel module is fairly simple to build/install. Performance is (very) good for a software raid. I successfully switched the raid array from an i386 6.2-RELEASE to an an amd64 7.0-RELEASE (with motherboard and CPU change) without any assle. For the moment, I use one big UFS+SU (and snapshots) on the whole array. I successfuly tried unpplugging then replugging on of the drives, suddent power loss, using the array with a missing disk (degraded mode). All did work fine. (still, I use an UPS on the file server). The sole issue I had is with ataidle. I had to patch ata-disc.c to increase the IO timeout. Without, the raid5 module detected temporary disk loss and constantly launched rebuilds of the array. With 7.0, I wondered if I should use gjournal, but I'm not sure if it's really the way to go on a file system dedicated to store many big files. So I stick to soft updates. Current configuration is : / on a 2GB usb key /tmp on memory ports and source trees (and some portsnap stuff) on a small disk 4x250 GB sata for the raid5 array. AMD A64 3200+ and 512 GB DDRII Realtek Gigabit nics. Copy from raid5 to /dev/null gives about 100MB/s Copy from /dev/random to raid5 about 40MB/s I use samba shares. I get about 40MB/s in both ways from another computer on the network (enabling jumbo-frames gives a big boost). Hope my own story can help you in any way. Regards, Arnaud Houdelette I know its been quite some time from the mail, but if you could say where to find this module. eikipedia says its on freebsd 7 but there is not this module for me (/boot/kernel/ there is no raid5 file). everytime I search the internet I find old stuff about it. If you could point me the site/article/anything :) thanks, matheus -- We will call you cygnus, The God of balance
Re: [SO]HO Software RAID5 server: which implementation should I choice?
Nenhum_de_Nos wrote: On Wed, Apr 9, 2008 at 9:47 AM, Arnaud Houdelette [EMAIL PROTECTED] wrote: Lev Serebryakov a écrit : Hello, freebsd-stable. Does somebody use some software RAID5 on FreeBSD in real production system? I want to build storage server for my home: RAW photos, multi-layer PhotoShop files and FLAC-encoded music consume a lot of space, and they should be availible both from desktop notebook. Also, all photo-content is unique, so I need some insuranse from single HDD crash. I understand, that I will not safe from fire, PSU failure and thing slike this. I selected hardware platform: Intel Q35-based MoBo with 6xSATA-II ports (all of them is chipset-based, so no SiliconImage/JMicron/Whatever crappy controllers), some low-end Core2Duo, 2Gb of memory. Storage will be 5x500Gb WD HDDs for RAID + one small HDD for boot, system, swap, etc. I want to have 2Tb (ok, not real Tb, I know) of protected storage. I want to have maximum speed via 1Gb network, because graphic files are big and should open fast. Not as fast as local ones, I understand that, but speeds about 12-15Mb/s is not enough for sure :) Only problem I see: which software RAID5 solution should I prefer? FreeBSD-based, of course! I see these variants: (1) FreeBSD 6(7?) + graid3. Slow, one disk for checksums is bottleneck, as far as I understand. (2) FreeBSD 6(7?) + gvinum/radi5. Is it stable enough?! Is it complete? when I try it about 6 months ago in VMWare installation with 5 virtual disks, I got panics and strange behaviour after crashing one of virtual disks. (3) FreeBSD 6(7?) + graid5. Again, is it stable enough? There are THREE versions of it. Which one should I prefer? There was long thread about it some times ago without any clear conclusion. Does something changed? (4) FreeBSD 7 + ZFS zraid. And again: stability. Too many messages about locks, crashes, etc. Code is experemental. Is it only for 32 bit systems? (5) Do I miss something? (6) Solaris + ZFS? I don't want it, I know a little about Solaris administaration, and I already have FreeBSD servers and routers. I know, that 3ware or Areca controllers are very good. I know, that gmirror is very stable. But these variants are too expensive for home server :( Does somebody use some software RAID5 on FreeBSD in real production? Any advices? Hi ! I personally use the 3 option for my personal Home File Server. I got approximatly the same usage for the file server (mostly video, music, photo). I built my own about 12 month ago. I reviewed the about the same variants as the one you propose : (1) Discarded for performance issues. Raid3 is slow. Really. (2) raid5/vinum is also slow. And as I understood at that time, recovery from lost hard drive wasn't easy enough for the freebsd niubee I was then. (4) ZFS wasn't there yet. But I did test it on a test VMWare, and wasn't convinced (mostly stability and memory issues). So I use geom_raid5. I sticked to the main distributions, which seemed more stable at the moment. The kernel module is fairly simple to build/install. Performance is (very) good for a software raid. I successfully switched the raid array from an i386 6.2-RELEASE to an an amd64 7.0-RELEASE (with motherboard and CPU change) without any assle. For the moment, I use one big UFS+SU (and snapshots) on the whole array. I successfuly tried unpplugging then replugging on of the drives, suddent power loss, using the array with a missing disk (degraded mode). All did work fine. (still, I use an UPS on the file server). The sole issue I had is with ataidle. I had to patch ata-disc.c to increase the IO timeout. Without, the raid5 module detected temporary disk loss and constantly launched rebuilds of the array. With 7.0, I wondered if I should use gjournal, but I'm not sure if it's really the way to go on a file system dedicated to store many big files. So I stick to soft updates. Current configuration is : / on a 2GB usb key /tmp on memory ports and source trees (and some portsnap stuff) on a small disk 4x250 GB sata for the raid5 array. AMD A64 3200+ and 512 GB DDRII Realtek Gigabit nics. Copy from raid5 to /dev/null gives about 100MB/s Copy from /dev/random to raid5 about 40MB/s I use samba shares. I get about 40MB/s in both ways from another computer on the network (enabling jumbo-frames gives a big boost). Hope my own story can help you in any way. Regards, Arnaud Houdelette I know its been quite some time from the mail, but if you could say where to find this module. eikipedia says its on freebsd 7 but there is not this module for me (/boot/kernel/ there is no raid5 file). everytime I search the internet I find old stuff about it. If you could point me the site/article/anything :) thanks, matheus Do you know about freenas at http://www.freenas.org? ___
Re: [SO]HO Software RAID5 server: which implementation should I choice?
Lev Serebryakov wrote: (4) FreeBSD 7 + ZFS zraid. And again: stability. Too many messages about locks, crashes, etc. Code is experemental. Is it only for 32 bit systems? Recently I installed a new server in my local network. [media butcher]# uname -rsm FreeBSD 8.0-CURRENT amd64 [media butcher]# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT video 3,16T 2,01T 1,14T63% ONLINE - [media butcher]# zpool status pool: video state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM video ONLINE 0 0 0 raidz1 ONLINE 0 0 0 label/DISK_01 ONLINE 0 0 0 label/DISK_02 ONLINE 0 0 0 label/DISK_03 ONLINE 0 0 0 label/DISK_04 ONLINE 0 0 0 label/DISK_05 ONLINE 0 0 0 label/DISK_06 ONLINE 0 0 0 label/DISK_07 ONLINE 0 0 0 errors: No known data errors [media butcher]# sysctl hw.physmem hw.physmem: 2136186880 [media butcher]# cat /boot/loader.conf zfs_load=YES vm.kmem_size=1342177280 # 1280 MB vm.kmem_size_max=1342177280 # 1280 MB So, server works very nice on moderate load with vsftpd (with sendfile enabled) - 40..70 users online. -- WBR, Andrey V. Elsukov ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [SO]HO Software RAID5 server: which implementation should I choice?
On Wed, Apr 9, 2008 at 2:45 AM, Lev Serebryakov [EMAIL PROTECTED] wrote: Hello, freebsd-stable. Does somebody use some software RAID5 on FreeBSD in real production system? I want to build storage server for my home: RAW photos, multi-layer PhotoShop files and FLAC-encoded music consume a lot of space, and they should be availible both from desktop notebook. I've used almost all combinations at one time or another. Right now my 32 bit system with disks that are not equally sized is running gmirror/gstripe (raid 10) and my larger 64 bit system has a 6x 750G RAIDZ1 array. We had a discussion of this at our local geek meet last night. The ZFS problems seem to be mostly with complex operations (ie: databases). I personally havn't had a problem. ZFS is also copy-on-write --- so snapshots _should_ preserve original data. My current take on home file servers is that zfs works well for those uses. That said, my postgresql store is on ufs. My current consulting work uses ZFS, but I tend to install opensolaris on the fileserver as the ZFS code is fresher there. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
[SO]HO Software RAID5 server: which implementation should I choice?
Hello, freebsd-stable. Does somebody use some software RAID5 on FreeBSD in real production system? I want to build storage server for my home: RAW photos, multi-layer PhotoShop files and FLAC-encoded music consume a lot of space, and they should be availible both from desktop notebook. Also, all photo-content is unique, so I need some insuranse from single HDD crash. I understand, that I will not safe from fire, PSU failure and thing slike this. I selected hardware platform: Intel Q35-based MoBo with 6xSATA-II ports (all of them is chipset-based, so no SiliconImage/JMicron/Whatever crappy controllers), some low-end Core2Duo, 2Gb of memory. Storage will be 5x500Gb WD HDDs for RAID + one small HDD for boot, system, swap, etc. I want to have 2Tb (ok, not real Tb, I know) of protected storage. I want to have maximum speed via 1Gb network, because graphic files are big and should open fast. Not as fast as local ones, I understand that, but speeds about 12-15Mb/s is not enough for sure :) Only problem I see: which software RAID5 solution should I prefer? FreeBSD-based, of course! I see these variants: (1) FreeBSD 6(7?) + graid3. Slow, one disk for checksums is bottleneck, as far as I understand. (2) FreeBSD 6(7?) + gvinum/radi5. Is it stable enough?! Is it complete? when I try it about 6 months ago in VMWare installation with 5 virtual disks, I got panics and strange behaviour after crashing one of virtual disks. (3) FreeBSD 6(7?) + graid5. Again, is it stable enough? There are THREE versions of it. Which one should I prefer? There was long thread about it some times ago without any clear conclusion. Does something changed? (4) FreeBSD 7 + ZFS zraid. And again: stability. Too many messages about locks, crashes, etc. Code is experemental. Is it only for 32 bit systems? (5) Do I miss something? (6) Solaris + ZFS? I don't want it, I know a little about Solaris administaration, and I already have FreeBSD servers and routers. I know, that 3ware or Areca controllers are very good. I know, that gmirror is very stable. But these variants are too expensive for home server :( Does somebody use some software RAID5 on FreeBSD in real production? Any advices? -- // Black Lion AKA Lev Serebryakov [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [SO]HO Software RAID5 server: which implementation should I choice?
Lev Serebryakov a écrit : Hello, freebsd-stable. Does somebody use some software RAID5 on FreeBSD in real production system? I want to build storage server for my home: RAW photos, multi-layer PhotoShop files and FLAC-encoded music consume a lot of space, and they should be availible both from desktop notebook. Also, all photo-content is unique, so I need some insuranse from single HDD crash. I understand, that I will not safe from fire, PSU failure and thing slike this. I selected hardware platform: Intel Q35-based MoBo with 6xSATA-II ports (all of them is chipset-based, so no SiliconImage/JMicron/Whatever crappy controllers), some low-end Core2Duo, 2Gb of memory. Storage will be 5x500Gb WD HDDs for RAID + one small HDD for boot, system, swap, etc. I want to have 2Tb (ok, not real Tb, I know) of protected storage. I want to have maximum speed via 1Gb network, because graphic files are big and should open fast. Not as fast as local ones, I understand that, but speeds about 12-15Mb/s is not enough for sure :) Only problem I see: which software RAID5 solution should I prefer? FreeBSD-based, of course! I see these variants: (1) FreeBSD 6(7?) + graid3. Slow, one disk for checksums is bottleneck, as far as I understand. (2) FreeBSD 6(7?) + gvinum/radi5. Is it stable enough?! Is it complete? when I try it about 6 months ago in VMWare installation with 5 virtual disks, I got panics and strange behaviour after crashing one of virtual disks. (3) FreeBSD 6(7?) + graid5. Again, is it stable enough? There are THREE versions of it. Which one should I prefer? There was long thread about it some times ago without any clear conclusion. Does something changed? (4) FreeBSD 7 + ZFS zraid. And again: stability. Too many messages about locks, crashes, etc. Code is experemental. Is it only for 32 bit systems? (5) Do I miss something? (6) Solaris + ZFS? I don't want it, I know a little about Solaris administaration, and I already have FreeBSD servers and routers. I know, that 3ware or Areca controllers are very good. I know, that gmirror is very stable. But these variants are too expensive for home server :( Does somebody use some software RAID5 on FreeBSD in real production? Any advices? Hi ! I personally use the 3 option for my personal Home File Server. I got approximatly the same usage for the file server (mostly video, music, photo). I built my own about 12 month ago. I reviewed the about the same variants as the one you propose : (1) Discarded for performance issues. Raid3 is slow. Really. (2) raid5/vinum is also slow. And as I understood at that time, recovery from lost hard drive wasn't easy enough for the freebsd niubee I was then. (4) ZFS wasn't there yet. But I did test it on a test VMWare, and wasn't convinced (mostly stability and memory issues). So I use geom_raid5. I sticked to the main distributions, which seemed more stable at the moment. The kernel module is fairly simple to build/install. Performance is (very) good for a software raid. I successfully switched the raid array from an i386 6.2-RELEASE to an an amd64 7.0-RELEASE (with motherboard and CPU change) without any assle. For the moment, I use one big UFS+SU (and snapshots) on the whole array. I successfuly tried unpplugging then replugging on of the drives, suddent power loss, using the array with a missing disk (degraded mode). All did work fine. (still, I use an UPS on the file server). The sole issue I had is with ataidle. I had to patch ata-disc.c to increase the IO timeout. Without, the raid5 module detected temporary disk loss and constantly launched rebuilds of the array. With 7.0, I wondered if I should use gjournal, but I'm not sure if it's really the way to go on a file system dedicated to store many big files. So I stick to soft updates. Current configuration is : / on a 2GB usb key /tmp on memory ports and source trees (and some portsnap stuff) on a small disk 4x250 GB sata for the raid5 array. AMD A64 3200+ and 512 GB DDRII Realtek Gigabit nics. Copy from raid5 to /dev/null gives about 100MB/s Copy from /dev/random to raid5 about 40MB/s I use samba shares. I get about 40MB/s in both ways from another computer on the network (enabling jumbo-frames gives a big boost). Hope my own story can help you in any way. Regards, Arnaud Houdelette ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [SO]HO Software RAID5 server: which implementation should I choice?
Arnaud Houdelette wrote: Lev Serebryakov a écrit : Hello, freebsd-stable. Does somebody use some software RAID5 on FreeBSD in real production system? I want to build storage server for my home: RAW photos, multi-layer PhotoShop files and FLAC-encoded music consume a lot of space, and they should be availible both from desktop notebook. Also, all photo-content is unique, so I need some insuranse from single HDD crash. I understand, that I will not safe from fire, PSU failure and thing slike this. I selected hardware platform: Intel Q35-based MoBo with 6xSATA-II ports (all of them is chipset-based, so no SiliconImage/JMicron/Whatever crappy controllers), some low-end Core2Duo, 2Gb of memory. Storage will be 5x500Gb WD HDDs for RAID + one small HDD for boot, system, swap, etc. I want to have 2Tb (ok, not real Tb, I know) of protected storage. I want to have maximum speed via 1Gb network, because graphic files are big and should open fast. Not as fast as local ones, I understand that, but speeds about 12-15Mb/s is not enough for sure :) Only problem I see: which software RAID5 solution should I prefer? FreeBSD-based, of course! I see these variants: (1) FreeBSD 6(7?) + graid3. Slow, one disk for checksums is bottleneck, as far as I understand. (2) FreeBSD 6(7?) + gvinum/radi5. Is it stable enough?! Is it complete? when I try it about 6 months ago in VMWare installation with 5 virtual disks, I got panics and strange behaviour after crashing one of virtual disks. (3) FreeBSD 6(7?) + graid5. Again, is it stable enough? There are THREE versions of it. Which one should I prefer? There was long thread about it some times ago without any clear conclusion. Does something changed? (4) FreeBSD 7 + ZFS zraid. And again: stability. Too many messages about locks, crashes, etc. Code is experemental. Is it only for 32 bit systems? (5) Do I miss something? (6) Solaris + ZFS? I don't want it, I know a little about Solaris administaration, and I already have FreeBSD servers and routers. I know, that 3ware or Areca controllers are very good. I know, that gmirror is very stable. But these variants are too expensive for home server :( Does somebody use some software RAID5 on FreeBSD in real production? Any advices? I've been using FreeNAS (http://www.freenas.org) for personal use as well as a couple of places at work. It lives on a USB key and uses the disks for storage. It uses Samba, AFP, NFS, and iSCSI. I haven't really loaded it down, but I've been seeing close to 100Mbit (on gig-e) (this is NFS shares for VMWare ESX) -- Louis Kowolowski[EMAIL PROTECTED] Cryptomonkeys: http://www.cryptomonkeys.com/~louisk Everyone is a genius. It's just that some people are too stupid to realize it. signature.asc Description: OpenPGP digital signature
Re: [SO]HO Software RAID5 server: which implementation should I choice?
Lev Serebryakov wrote: ... I've been using FreeNAS (http://www.freenas.org) for personal use as well as a couple of places at work. Do you use it in RAID5 configuration (which is geom_raid5 - based)? I'm using it in both gmirror and graid5 configurations. -- Louis Kowolowski[EMAIL PROTECTED] Cryptomonkeys: http://www.cryptomonkeys.com/~louisk Everyone is a genius. It's just that some people are too stupid to realize it. signature.asc Description: OpenPGP digital signature
Re[2]: [SO]HO Software RAID5 server: which implementation should I choice?
Hello, Louis. You wrote 9 ?? 2008 ?., 21:26:09: I've been using FreeNAS (http://www.freenas.org) for personal use as well as a couple of places at work. Do you use it in RAID5 configuration (which is geom_raid5 - based)? -- // Black Lion AKA Lev Serebryakov [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
