Re: Dell Perc 6 disk geometry problem with RAID5 (both 6.3 final and 7.0 RC1)
On Sunday 20 January 2008 01:53:30 pm David Wood wrote: Hi there, In message [EMAIL PROTECTED], Erik Trulsson [EMAIL PROTECTED] writes On Sun, Jan 20, 2008 at 04:48:56PM +, David Wood wrote: In message [EMAIL PROTECTED], Aldas Nabazas [EMAIL PROTECTED] writes We bought a new Dell PowerEdge 2950III with Perc 6/i and have the disk geometry problem using 6.3 final or 7.0 RC1. Seems that we are not alone at least one guy has similar problem reported earlier: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-01/msg0 0506.html I do not know if the mfi(4) driver has any problems with large disks, but it is however well known that fdisk(8) and bsdlabel(8) (the tools normally used to partition disks) have problems with volumes larger than 2TB. If you want to use volumes larger than 2TB then gpt(8) is the recommended way to partition the disks. It is however doubtful if the BIOS in your system will allow you to boot from a gpt(8) parttioned volume which is best solved with having a separate - smaller - boot volume where the OS itself is installed. Erik's reminder is timely for those with 2TB volumes. You must use gpt and not fdisk on any disk, be it a single drive (in the future, at least!) or a virtual disk on a RAID setup that are bigger than 2TB. It may be wise to use gpt on any virtual disk that you might grow to 2TB or larger in the future, so long as you're not needing to boot from that virtual disk. fdisk will not work properly with 2TB and larger volumes - the MBR / partition table setup can't cope with these large volumes. You can't boot from a GPT volume - that's a limitation of the BIOS architecture. There is some thought about using EFI on x64 hardware in the future (EFI is used on ia64 hardware; GPT is part of EFI), but we're not there yet. This isn't just about adding GPT support to the BIOS - the whole BIOS setup is wedded to MBR. If you need to boot from a 2TB array, create two virtual disks - one smaller than 2TB to boot from (and use MBR on that), then the residue can be GPT. Actually, using gptboot in HEAD you can now boot from GPT on large volumes (I've booted from a 2 TB volume on a PERC5 using mfi(4) with it). I will see about getting that merged back to 6.x and 7.x in CVS. We use it for large volumes on 6.x and all volumes on 7.x at work. When I said there was nothing relevant waiting for MFC, I was aware of the change that Tom mentioned, but that seems to be about Dell PERC 6 being identified as such rather than a MegaRAID. It doesn't seem that it will change the behaviour of the driver in any way. In our testing at work the 2950 rev 3's worked fine with mfi(4). -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dell Perc 6 disk geometry problem with RAID5 (both 6.3 final and 7.0 RC1)
On Jan 23, 2008 2:19 PM, John Baldwin [EMAIL PROTECTED] wrote: On Sunday 20 January 2008 01:53:30 pm David Wood wrote: Hi there, In message [EMAIL PROTECTED], Erik Trulsson [EMAIL PROTECTED] writes On Sun, Jan 20, 2008 at 04:48:56PM +, David Wood wrote: In message [EMAIL PROTECTED] , Aldas Nabazas [EMAIL PROTECTED] writes We bought a new Dell PowerEdge 2950III with Perc 6/i and have the disk geometry problem using 6.3 final or 7.0 RC1. Seems that we are not alone at least one guy has similar problem reported earlier: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-01/msg0 0506.html I do not know if the mfi(4) driver has any problems with large disks, but it is however well known that fdisk(8) and bsdlabel(8) (the tools normally used to partition disks) have problems with volumes larger than 2TB. If you want to use volumes larger than 2TB then gpt(8) is the recommended way to partition the disks. It is however doubtful if the BIOS in your system will allow you to boot from a gpt(8) parttioned volume which is best solved with having a separate - smaller - boot volume where the OS itself is installed. Erik's reminder is timely for those with 2TB volumes. You must use gpt and not fdisk on any disk, be it a single drive (in the future, at least!) or a virtual disk on a RAID setup that are bigger than 2TB. It may be wise to use gpt on any virtual disk that you might grow to 2TB or larger in the future, so long as you're not needing to boot from that virtual disk. fdisk will not work properly with 2TB and larger volumes - the MBR / partition table setup can't cope with these large volumes. You can't boot from a GPT volume - that's a limitation of the BIOS architecture. There is some thought about using EFI on x64 hardware in the future (EFI is used on ia64 hardware; GPT is part of EFI), but we're not there yet. This isn't just about adding GPT support to the BIOS - the whole BIOS setup is wedded to MBR. If you need to boot from a 2TB array, create two virtual disks - one smaller than 2TB to boot from (and use MBR on that), then the residue can be GPT. Actually, using gptboot in HEAD you can now boot from GPT on large volumes (I've booted from a 2 TB volume on a PERC5 using mfi(4) with it). I will see about getting that merged back to 6.x and 7.x in CVS. We use it for large volumes on 6.x and all volumes on 7.x at work. When I said there was nothing relevant waiting for MFC, I was aware of the change that Tom mentioned, but that seems to be about Dell PERC 6 being identified as such rather than a MegaRAID. It doesn't seem that it will change the behaviour of the driver in any way. In our testing at work the 2950 rev 3's worked fine with mfi(4). -- John Baldwin Hi John, Nice to hear that. Regards, Aldas ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Dell Perc 6 disk geometry problem with RAID5 (both 6.3 final and 7.0 RC1)
Hi, We bought a new Dell PowerEdge 2950III with Perc 6/i and have the disk geometry problem using 6.3 final or 7.0 RC1. Seems that we are not alone at least one guy has similar problem reported earlier: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-01/msg00506.html I was reading the mailing list and found that some of the people are happily using this hardware with the latest FreeBSD: http://lists.freebsd.org/pipermail/freebsd-stable/2008-January/039675.html I just wonder what the status of mfi driver? Maybe it's not fully tested or there will be some important fixes before 7.0 final? We are going to try different RAID combinations but it definitely not working using 6x146GB as RAID5. Maybe someone could share the RAID combinations they successfully are running on? Thanks in advance. Kind Regards, Aldas ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dell Perc 6 disk geometry problem with RAID5 (both 6.3 final and 7.0 RC1)
[ambrisko@ and scottl@ added to CCs] Hi there, In message [EMAIL PROTECTED], Aldas Nabazas [EMAIL PROTECTED] writes We bought a new Dell PowerEdge 2950III with Perc 6/i and have the disk geometry problem using 6.3 final or 7.0 RC1. Seems that we are not alone at least one guy has similar problem reported earlier: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-01/msg00506.html I was reading the mailing list and found that some of the people are happily using this hardware with the latest FreeBSD: http://lists.freebsd.org/pipermail/freebsd-stable/2008-January/039675.html I just wonder what the status of mfi driver? Maybe it's not fully tested or there will be some important fixes before 7.0 final? We are going to try different RAID combinations but it definitely not working using 6x146GB as RAID5. This is extremely disappointing to read, as I was relatively close to buying a Poweredge 2950 III with PERC 6/i. However, it's no good to me if mfi(4) has issues with large virtual disks; the tentative disk configuration is 2x146GB as RAID 1 and 4x750GB (or 1TB) as RAID 5. Looking at CVSweb: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/mfi/ the updates for the 1078 chip which powers the PERC 6 series were contributed by LSI, so you would have hoped things were right. There is a disclaimer on the code, but you would also hope that someone in the know is testing it, especially as my impression has always been that the FreeBSD community is favourably disposed towards LSI storage controllers and that LSI and their vendors try to help the FreeBSD developers. Maybe someone could share the RAID combinations they successfully are running on? I haven't been keeping a very close eye on the problem as I don't currently have any hardware - but is the issue simply one of virtual disk size - there's a cut-off size after which things don't work properly? You could try pulling disks from your server (or removing them from the virtual disk) one by one until things start to work. To save a lot of pain you could create a virtual disk containing one disk as RAID 0 (a single disk) and install the OS on it, leaving the other five disks to play with. I do hope that someone is in a position to investigate this quickly - even if it's too late to get the fix in 7.0-RELEASE now. There's nothing that I can see as relevant that's waiting for MFC, anyway. Of course, it's worth checking whether you've got the latest firmware on the PERC 6/i - the latest version from Dell appears to be 6.0.1-0080. If this doesn't get fixed soon, I'm either going to have to go to another hardware vendor that uses a different RAID controller (HP is a possibility - though we're an all Dell shop) or - sniff - leave FreeBSD in favour of a Linux distribution. I realise that FreeBSD is a volunteer project, and that users can't have any specific expectations - this is not a threat, as I like FreeBSD and want to remain in the community, but I would need a new server to work properly! Looking at CVSweb, it seems that ambrisko@ and scottl@ are the two people most closely associated with the mfi code - I've added them into the CCs. My apologies if that was unwelcome. Best wishes, David -- David Wood [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dell Perc 6 disk geometry problem with RAID5 (both 6.3 final and 7.0 RC1)
On Sun, Jan 20, 2008 at 04:48:56PM +, David Wood wrote: [ambrisko@ and scottl@ added to CCs] Hi there, In message [EMAIL PROTECTED], Aldas Nabazas [EMAIL PROTECTED] writes We bought a new Dell PowerEdge 2950III with Perc 6/i and have the disk geometry problem using 6.3 final or 7.0 RC1. Seems that we are not alone at least one guy has similar problem reported earlier: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-01/msg00506.html I was reading the mailing list and found that some of the people are happily using this hardware with the latest FreeBSD: http://lists.freebsd.org/pipermail/freebsd-stable/2008-January/039675.html I just wonder what the status of mfi driver? Maybe it's not fully tested or there will be some important fixes before 7.0 final? We are going to try different RAID combinations but it definitely not working using 6x146GB as RAID5. I do not know if the mfi(4) driver has any problems with large disks, but it is however well known that fdisk(8) and bsdlabel(8) (the tools normally used to partition disks) have problems with volumes larger than 2TB. If you want to use volumes larger than 2TB then gpt(8) is the recommended way to partition the disks. It is however doubtful if the BIOS in your system will allow you to boot from a gpt(8) parttioned volume which is best solved with having a separate - smaller - boot volume where the OS itself is installed. This is extremely disappointing to read, as I was relatively close to buying a Poweredge 2950 III with PERC 6/i. However, it's no good to me if mfi(4) has issues with large virtual disks; the tentative disk configuration is 2x146GB as RAID 1 and 4x750GB (or 1TB) as RAID 5. Looking at CVSweb: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/mfi/ the updates for the 1078 chip which powers the PERC 6 series were contributed by LSI, so you would have hoped things were right. There is a disclaimer on the code, but you would also hope that someone in the know is testing it, especially as my impression has always been that the FreeBSD community is favourably disposed towards LSI storage controllers and that LSI and their vendors try to help the FreeBSD developers. Maybe someone could share the RAID combinations they successfully are running on? I haven't been keeping a very close eye on the problem as I don't currently have any hardware - but is the issue simply one of virtual disk size - there's a cut-off size after which things don't work properly? You could try pulling disks from your server (or removing them from the virtual disk) one by one until things start to work. To save a lot of pain you could create a virtual disk containing one disk as RAID 0 (a single disk) and install the OS on it, leaving the other five disks to play with. I do hope that someone is in a position to investigate this quickly - even if it's too late to get the fix in 7.0-RELEASE now. There's nothing that I can see as relevant that's waiting for MFC, anyway. Of course, it's worth checking whether you've got the latest firmware on the PERC 6/i - the latest version from Dell appears to be 6.0.1-0080. If this doesn't get fixed soon, I'm either going to have to go to another hardware vendor that uses a different RAID controller (HP is a possibility - though we're an all Dell shop) or - sniff - leave FreeBSD in favour of a Linux distribution. I realise that FreeBSD is a volunteer project, and that users can't have any specific expectations - this is not a threat, as I like FreeBSD and want to remain in the community, but I would need a new server to work properly! Looking at CVSweb, it seems that ambrisko@ and scottl@ are the two people most closely associated with the mfi code - I've added them into the CCs. My apologies if that was unwelcome. -- Insert your favourite quote here. Erik Trulsson [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dell Perc 6 disk geometry problem with RAID5 (both 6.3 final and 7.0 RC1)
Erik Trulsson wrote: On Sun, Jan 20, 2008 at 04:48:56PM +, David Wood wrote: [ambrisko@ and scottl@ added to CCs] Hi there, In message [EMAIL PROTECTED], Aldas Nabazas [EMAIL PROTECTED] writes We bought a new Dell PowerEdge 2950III with Perc 6/i and have the disk geometry problem using 6.3 final or 7.0 RC1. Seems that we are not alone at least one guy has similar problem reported earlier: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-01/msg00506.html This post is related to using BSDLabel and FDISK MBR partition tables with a disk over 2Tb. This is a known limitation of this type of partition table where the max size is 2Tb. If you wish to make a larger partition you must use GPT or the raw device with no partition table (Not recommended). I was reading the mailing list and found that some of the people are happily using this hardware with the latest FreeBSD: http://lists.freebsd.org/pipermail/freebsd-stable/2008-January/039675.html I just wonder what the status of mfi driver? Maybe it's not fully tested or there will be some important fixes before 7.0 final? We are going to try different RAID combinations but it definitely not working using 6x146GB as RAID5. I do not know if the mfi(4) driver has any problems with large disks, but it is however well known that fdisk(8) and bsdlabel(8) (the tools normally used to partition disks) have problems with volumes larger than 2TB. If you want to use volumes larger than 2TB then gpt(8) is the recommended way to partition the disks. It is however doubtful if the BIOS in your system will allow you to boot from a gpt(8) parttioned volume which is best solved with having a separate - smaller - boot volume where the OS itself is installed. I have 5 PERC6/i based systems awaiting deployment atm, might be able to do some very limited testing on one of them if time permits. However I have some very large arrays running on PERC5/e's (6TB Raid 50 - 5 disk spans - 15 * 500Gb disks spread over 2 md1000 shelves) (the systems run RELENG_6_2) and have not seem any issues with them. They are a single gpt partition with UFS2 on them. As far as I have seen in the last ~8 months there have been no issues with the mfi driver and large arrays. However I cannot say at this stage if the same can be said for the PERC6/i. This is extremely disappointing to read, as I was relatively close to buying a Poweredge 2950 III with PERC 6/i. However, it's no good to me if mfi(4) has issues with large virtual disks; the tentative disk configuration is 2x146GB as RAID 1 and 4x750GB (or 1TB) as RAID 5. Looking at CVSweb: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/mfi/ There is one related PCI ID's change to do with DELL sub vendor id's I think that is only on HEAD. Any chance of getting this MFC'd? Tom the updates for the 1078 chip which powers the PERC 6 series were contributed by LSI, so you would have hoped things were right. There is a disclaimer on the code, but you would also hope that someone in the know is testing it, especially as my impression has always been that the FreeBSD community is favourably disposed towards LSI storage controllers and that LSI and their vendors try to help the FreeBSD developers. Maybe someone could share the RAID combinations they successfully are running on? I haven't been keeping a very close eye on the problem as I don't currently have any hardware - but is the issue simply one of virtual disk size - there's a cut-off size after which things don't work properly? You could try pulling disks from your server (or removing them from the virtual disk) one by one until things start to work. To save a lot of pain you could create a virtual disk containing one disk as RAID 0 (a single disk) and install the OS on it, leaving the other five disks to play with. I do hope that someone is in a position to investigate this quickly - even if it's too late to get the fix in 7.0-RELEASE now. There's nothing that I can see as relevant that's waiting for MFC, anyway. Of course, it's worth checking whether you've got the latest firmware on the PERC 6/i - the latest version from Dell appears to be 6.0.1-0080. If this doesn't get fixed soon, I'm either going to have to go to another hardware vendor that uses a different RAID controller (HP is a possibility - though we're an all Dell shop) or - sniff - leave FreeBSD in favour of a Linux distribution. I realise that FreeBSD is a volunteer project, and that users can't have any specific expectations - this is not a threat, as I like FreeBSD and want to remain in the community, but I would need a new server to work properly! Looking at CVSweb, it seems that ambrisko@ and scottl@ are the two people most closely associated with the mfi code - I've added them into the CCs. My apologies if that was unwelcome. ___ freebsd-stable@freebsd.org mailing list
Re: Dell Perc 6 disk geometry problem with RAID5 (both 6.3 final and 7.0 RC1)
Hi there, In message [EMAIL PROTECTED], Erik Trulsson [EMAIL PROTECTED] writes On Sun, Jan 20, 2008 at 04:48:56PM +, David Wood wrote: In message [EMAIL PROTECTED], Aldas Nabazas [EMAIL PROTECTED] writes We bought a new Dell PowerEdge 2950III with Perc 6/i and have the disk geometry problem using 6.3 final or 7.0 RC1. Seems that we are not alone at least one guy has similar problem reported earlier: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-01/msg0 0506.html I do not know if the mfi(4) driver has any problems with large disks, but it is however well known that fdisk(8) and bsdlabel(8) (the tools normally used to partition disks) have problems with volumes larger than 2TB. If you want to use volumes larger than 2TB then gpt(8) is the recommended way to partition the disks. It is however doubtful if the BIOS in your system will allow you to boot from a gpt(8) parttioned volume which is best solved with having a separate - smaller - boot volume where the OS itself is installed. Erik's reminder is timely for those with 2TB volumes. You must use gpt and not fdisk on any disk, be it a single drive (in the future, at least!) or a virtual disk on a RAID setup that are bigger than 2TB. It may be wise to use gpt on any virtual disk that you might grow to 2TB or larger in the future, so long as you're not needing to boot from that virtual disk. fdisk will not work properly with 2TB and larger volumes - the MBR / partition table setup can't cope with these large volumes. You can't boot from a GPT volume - that's a limitation of the BIOS architecture. There is some thought about using EFI on x64 hardware in the future (EFI is used on ia64 hardware; GPT is part of EFI), but we're not there yet. This isn't just about adding GPT support to the BIOS - the whole BIOS setup is wedded to MBR. If you need to boot from a 2TB array, create two virtual disks - one smaller than 2TB to boot from (and use MBR on that), then the residue can be GPT. However, Erik's 6*146 is only 846GB, which should be fine as an MBR / partition table volume (and the use of RAID 5 means you only have 5*146 usable in any case). The referenced problem on freebsd-questions sounds like a failure to use GPT, but Erik's problem sounds different. When I said there was nothing relevant waiting for MFC, I was aware of the change that Tom mentioned, but that seems to be about Dell PERC 6 being identified as such rather than a MegaRAID. It doesn't seem that it will change the behaviour of the driver in any way. There's another change in -CURRENT to do with the log buffer; again, that doesn't seem to be relevant here. Nevertheless, if one or both of these changes are felt stable enough to be MFCed to RELENG_7 and ideally RELENG_7_0, that would be good. I do have my doubts as to whether agreement will be got to merge them to RELENG_7_0 at this stage, however. Erik - can you post dmesg output, details on how you've arranged the physical disks into virtual disks, exactly how you get an anomalous result, and why you think what you're seeing is wrong. I went to look at HP, and I wasn't particularly happy with their offering. The Proliant DL380 G5 isn't a bad machine - but it uses 2.5 inch disks, and I prefer the 6 x 3.5 inch option available with the Dell Poweredge 2950. With the current state of development of disks and my requirement for a high capacity array, a 3.5 inch based machine is more suited to my needs. I also prefer the way that you order a Dell. I do hope we can reassure ourselves over the PERC 6/i. The PERC 5 series seems to have an excellent reputation under FreeBSD, and hopefully the newer PERC 6 series will soon get the same excellent reputation. Best wishes, David -- David Wood [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dell Perc 6 disk geometry problem with RAID5 (both 6.3 final and 7.0 RC1)
David Wood wrote: [ambrisko@ and scottl@ added to CCs] Hi there, What really needs to happen as a result of this thread is the following, and I would appreciate volunteers taking up these tasks in some form: 1) FAQ entries written/updated/clarified/advertised as to how FreeBSD, disklabels, FDISK tables, GPT tables, and the PC BIOS work with regard to disk sizes. Having some press articles and blogs published on the subject in conjunction would be extra good. 2) sysinstall needs to be updated to understand the 2TB problem and provide unambiguous information to the user about it. 3) sysinstall needs to be further updated to give the user options for doing GPT partitioning with MBR compatibility for booting. #1 and #2 are vital to get done. #3 is a very strong nice to have. Please, anyone, feel free to work on these tasks with or without coordination with myself or others. They need to be done. Thanks, Scott ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
