Re: LSI 9240-4i 4K alignment
On 8 Aug, George Kontostanos wrote: The problem: When trying to create a RaidZ pool using gpart and perform a 4K alignment using gnop, we get the follwoing error immediately after exporting the pool and destroying the .nop devices: id: 8043746387654554958 state: FAULTED status: One or more devices contains corrupted data. action: The pool cannot be imported due to damaged devices or data. The pool may be active on another system, but can be imported using the '-f' flag. see: http://illumos.org/msg/ZFS-8000-5E config: Pool FAULTED corrupted data raidz1-0ONLINE 13283347160590042564 UNAVAIL corrupted data 16981727992215676534 UNAVAIL corrupted data 6607570030658834339 UNAVAIL corrupted data 3435463242860701988 UNAVAIL corrupted data When we use glabel for the same purpose with the combination of gnop, the pool imports fine. Might kern/170945 have something to do with this? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On 19 Aug, Josh Paetzel wrote: On 08/19/2012 14:04, Steven Hartland wrote: HBA's are the way to go if your using ZFS to manage the disks, you only need RAID if your using a FS which doesn't manage the disk side well such as UFS. Its often quite common for RAID controllers to actually be slower vs RAID controllers as the RAID stack can get in the way. Any idea of what kind of performance penalty I might see by using the RAID firmware in JBOD mode vs flashing the IT firmware? Just to clear up, The 9240 is a sas2008 based card with the megaraid software on top of it. In it's default config from LSI the FreeBSD mfi will recognize it in later versions of FreeBSD (The upcoming 9.1 for sure) Older versions of mfi will not recognize it. The card can be flashed with IT firmware and then becomes a 9211 HBA, but it's a bit more expensive than a 9211 is so that doesn't make sense to do in many cases. The price difference was pretty minor when I looked. Confusingly enough, the 9211 HBA also has some RAID capabilities. For me, the biggest advantage of the 9211 would be that it would have allowed me to use shorter cables. On the dmesg posted the firmware on the card is phase 11. This *must* be in lockstep with the driver version or the card may not play nicely. FreeBSD 8.3 and 9.0 have v13 of the driver, the upcoming 9.1 will have v14. Note that v14 fixes a *ton* of stability bugs, including issues where bad drives would hang the controller or prevent systems from booting. Where do those version numbers come from? The mfi driver in 9.0-RELEASE claims to be version 3.00 and the the driver in 9.1 claims to be version 4.23. This is what shows up in dmesg on my machine: mfi0: Drake Skinny port 0xce00-0xceff mem 0xfcefc000-0xfcef,0xfce8-0xf ceb irq 18 at device 0.0 on pci1 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: 333 (398082533s/0x0020/info) - Shutdown command received from host mfi0: 334 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 /1000/9240/1000) mfi0: 335 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 mfi0: 336 (boot + 5s/0x0020/info) - Board Revision 04A mfi0: 337 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 /1000/9240/1000) mfi0: 338 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 mfi0: 339 (boot + 5s/0x0020/info) - Board Revision 04A mfi0: 340 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 /1000/9240/1000) mfi0: 341 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 mfi0: 342 (boot + 5s/0x0020/info) - Board Revision 04A mfi0: 343 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 /1000/9240/1000) mfi0: 344 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 mfi0: 345 (boot + 5s/0x0020/info) - Board Revision 04A mfi0: 346 (398759025s/0x0020/info) - Time established as 08/20/12 6:23:45; (25 seconds since power on) mfi0: 347 (398759051s/0x0020/info) - Time established as 08/20/12 6:24:11; (51 seconds since power on) mfi0: 348 (398759078s/0x0020/WARN) - Patrol Read can't be started, as PDs are ei ther not ONLINE, or are in a VD with an active process, or are in an excluded VD % mfiutil show firmware mfi0 Firmware Package Version: 20.5.1-0003 mfi0 Firmware Images: Name VersionDate Time Status BIOS 4.14.00 active PCLI 03.02-001:#%8 Feb 09 2010 13:09:06 active BCON 4.0-22-e_10-RelMar 11 2010 12:38:08 active NVDT 3.04.03-0002 Apr 05 2010 18:50:27 active APP 2.70.04-0862 May 05 2010 18:12:07 active BTBL 2.01.00.00-0019May 14 2009 15:52:08 active The only firmware file on LSI's web site for the 9240-8i is version 20.10.1-107, which appears to be newer than what is on the card if the 20.5.1-0003 is the version number that I should be looking at. Is the BIOS Version 4.14 the v14 version that you mention above? If the FreeBSD mfi driver expects a certain firmware version, shouldn't it complain if it doesn't find it? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On 08/20/2012 00:19, Don Lewis wrote: On 19 Aug, Josh Paetzel wrote: On 08/19/2012 14:04, Steven Hartland wrote: HBA's are the way to go if your using ZFS to manage the disks, you only need RAID if your using a FS which doesn't manage the disk side well such as UFS. Its often quite common for RAID controllers to actually be slower vs RAID controllers as the RAID stack can get in the way. Any idea of what kind of performance penalty I might see by using the RAID firmware in JBOD mode vs flashing the IT firmware? I don't have any current numbers, on ZFS v14 14 RAID controllers were actually a bit faster, but that's all changed dramatically. On our high end stuff we can get HBAs to go over 25% faster than high end RAID controllers, like the 9260/9280, but we don't test with RAID controllers anymore at all, so I don't have up to the minute info. ZFS does block checksums, and so do LSI mfi cards, even when in JBOD mode, you also can't bypass the cache on the card without a huge performance hit, so you end up with 256MB or whatever in between your disks and the OS. In addition because the 9240 is based on the 2008 which lacks hardware assist for RAID5/6 those two modes are done in software, so you take another hit there. Advantages: ZFS doesn't work with hot spares as of this moment on FreeBSD, but LSI controllers do, so if your strategy involves hot spares the RAID card is the better choice. LSI controllers can be set to auto-replace, ZFS can't. Enclosure management works better on RAID controllers than through FreeBSD in many cases. Just to clear up, The 9240 is a sas2008 based card with the megaraid software on top of it. In it's default config from LSI the FreeBSD mfi will recognize it in later versions of FreeBSD (The upcoming 9.1 for sure) Older versions of mfi will not recognize it. The card can be flashed with IT firmware and then becomes a 9211 HBA, but it's a bit more expensive than a 9211 is so that doesn't make sense to do in many cases. The price difference was pretty minor when I looked. Confusingly enough, the 9211 HBA also has some RAID capabilities. For me, the biggest advantage of the 9211 would be that it would have allowed me to use shorter cables. On the dmesg posted the firmware on the card is phase 11. This *must* be in lockstep with the driver version or the card may not play nicely. FreeBSD 8.3 and 9.0 have v13 of the driver, the upcoming 9.1 will have v14. Note that v14 fixes a *ton* of stability bugs, including issues where bad drives would hang the controller or prevent systems from booting. Where do those version numbers come from? The mfi driver in 9.0-RELEASE claims to be version 3.00 and the the driver in 9.1 claims to be version 4.23. I was talking about mps, not mfi. The dmesg I was responding to showed an mps. This is what shows up in dmesg on my machine: mfi0: Drake Skinny port 0xce00-0xceff mem 0xfcefc000-0xfcef,0xfce8-0xf ceb irq 18 at device 0.0 on pci1 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: 333 (398082533s/0x0020/info) - Shutdown command received from host mfi0: 334 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 /1000/9240/1000) mfi0: 335 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 mfi0: 336 (boot + 5s/0x0020/info) - Board Revision 04A mfi0: 337 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 /1000/9240/1000) mfi0: 338 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 mfi0: 339 (boot + 5s/0x0020/info) - Board Revision 04A mfi0: 340 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 /1000/9240/1000) mfi0: 341 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 mfi0: 342 (boot + 5s/0x0020/info) - Board Revision 04A mfi0: 343 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 /1000/9240/1000) mfi0: 344 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 mfi0: 345 (boot + 5s/0x0020/info) - Board Revision 04A mfi0: 346 (398759025s/0x0020/info) - Time established as 08/20/12 6:23:45; (25 seconds since power on) mfi0: 347 (398759051s/0x0020/info) - Time established as 08/20/12 6:24:11; (51 seconds since power on) mfi0: 348 (398759078s/0x0020/WARN) - Patrol Read can't be started, as PDs are ei ther not ONLINE, or are in a VD with an active process, or are in an excluded VD % mfiutil show firmware mfi0 Firmware Package Version: 20.5.1-0003 mfi0 Firmware Images: Name VersionDate Time Status BIOS 4.14.00 active PCLI 03.02-001:#%8 Feb 09 2010 13:09:06 active BCON 4.0-22-e_10-RelMar 11 2010 12:38:08 active NVDT 3.04.03-0002 Apr 05 2010 18:50:27 active APP 2.70.04-0862 May 05 2010 18:12:07 active BTBL 2.01.00.00-0019May 14 2009 15:52:08 active The only firmware file on LSI's web site for the 9240-8i is version
Re: LSI 9240-4i 4K alignment
On Sun, Aug 19, 2012 at 7:34 PM, Josh Paetzel j...@tcbug.org wrote: On the dmesg posted the firmware on the card is phase 11. This *must* be in lockstep with the driver version or the card may not play nicely. FreeBSD 8.3 and 9.0 have v13 of the driver, the upcoming 9.1 will have v14. Note that v14 fixes a *ton* of stability bugs, including issues where bad drives would hang the controller or prevent systems from booting. Oof, good to know. I happen to have a server with v9 9211 firmware and v13 mps drivers that's having stability problems under load; it will be interesting to see if updating the firmware solves the problem. Is there someplace I should have been checking to know that this was a requirement? Thanks, Andy ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On 16 Aug, Steven Hartland wrote: - Original Message - From: George Kontostanos gkontos.m...@gmail.com You are right, the chip specs say: LSISAS2108 RAID-on-Chip The drives are identified as mfisyspd0, mfisyspd1, etc. The following might be interesting to you:- http://forums.servethehome.com/showthread.php?599-LSI-RAID-Controller-and-HBA-Complete-Listing-Plus-OEM-Models Which states:- LSI MegaRAID SAS 9240-4i 1x4 port internal SAS vertical, no cache, no BBU, RAID 0, 1, 10 and 5, can be crossflashed to LSI9211 IT/IR This is insteresting as this is the card we're using but in the 8 port version under mps :) I wish I would have known this earlier. I just put together a ZFS server using LSI MegaRAID SAS 9240-8i cards. The cabling probably would have been cleaner with the 9211-8i, but I went with the 9240 because the vendor that I purchased the cards from listed that 9240 as being PCI-Express 2.0, but didn't say that about the 9211. I also got the impression that the 9240 recognized JBOD drives with the off-the-shelf firmware, whereas the 9211 did not. Even LSI's own site is a bit confusing. They list the 9211 in the HBA section, but its specs don't mention JBOD, whereas the 9240 is listed in the RAID section and its specs do list JBOD. If the only physical difference between the cards is the connector position, it seems odd that they don't offer products with all the combinations of firmware and connector position. I haven't configured the ZFS pool yet, but I didn't have any trouble installing FreeBSD 9.1-BETA on the GPT partitioned boot drive, which shows up as an mfi device. I'm planning on getting the ZFS pool up and running in the next few days. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On 8 Aug, George Kontostanos wrote: Hi all, We have a server with a LSI 9240-4i controller configured in JBOD with 4 SATA disks. Running FreeBSD 9.1-Beta1: Relevant dmesg: FreeBSD 9.1-BETA1 #0: Thu Jul 12 09:38:51 UTC 2012 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz (3200.09-MHz K8-class CPU) Origin = GenuineIntel Id = 0x206a7 Family = 6 Model = 2a Stepping = 7 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x1fbae3ffSSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX AMD Features=0x28100800SYSCALL,NX,RDTSCP,LM AMD Features2=0x1LAHF TSC: P-state invariant, performance statistics real memory = 17179869184 (16384 MB) avail memory = 16471670784 (15708 MB) ... mfi0: Drake Skinny port 0xe000-0xe0ff mem 0xf7a6-0xf7a63fff,0xf7a0-0xf7a3 irq 16 at device 0.0 on pci1 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 ... mfi0: 321 (397672301s/0x0020/info) - Shutdown command received from host mfi0: 322 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073/1000/9241/1000) mfi0: 323 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 mfi0: 324 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073/1000/9241/1000) mfi0: 325 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 mfi0: 326 (boot + 5s/0x0020/info) - Package version 20.10.1-0107 mfi0: 327 (boot + 5s/0x0020/info) - Board Revision 03A mfi0: 328 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) ... mfisyspd0 on mfi0 mfisyspd0: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd0: SYSPD volume attached mfisyspd1 on mfi0 mfisyspd1: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd1: SYSPD volume attached mfisyspd2 on mfi0 mfisyspd2: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd2: SYSPD volume attached mfisyspd3 on mfi0 mfisyspd3: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd3: SYSPD volume attached ... mfi0: 329 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) Info: enclPd=, scsiType=0, portMap=00, sasAddr=44332211, mfi0: 330 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) mfi0: 331 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) Info: enclPd=, scsiType=0, portMap=02, sasAddr=443322110200, mfi0: 332 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) mfi0: 333 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) Info: enclPd=, scsiType=0, portMap=03, sasAddr=443322110100, mfi0: 334 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) mfi0: 335 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) Info: enclPd=, scsiType=0, portMap=01, sasAddr=443322110300, mfi0: 336 (397672376s/0x0020/info) - Time established as 08/07/12 16:32:56; (28 seconds since power on) The problem: When trying to create a RaidZ pool using gpart and perform a 4K alignment using gnop, we get the follwoing error immediately after exporting the pool and destroying the .nop devices: id: 8043746387654554958 state: FAULTED status: One or more devices contains corrupted data. action: The pool cannot be imported due to damaged devices or data. The pool may be active on another system, but can be imported using the '-f' flag. see: http://illumos.org/msg/ZFS-8000-5E config: Pool FAULTED corrupted data raidz1-0ONLINE 13283347160590042564 UNAVAIL corrupted data 16981727992215676534 UNAVAIL corrupted data 6607570030658834339 UNAVAIL corrupted data 3435463242860701988 UNAVAIL corrupted data I'm planning on doing something similar, but I'm curious about how gnop and GPT labels interact. I want to partition the drives for my pool slightly on the small side so that I'm less likely to run into problems if I have to replace a drive in the future. If I used gpart to create and label a GPT partition on the drive, the partition will show up as /dev/gpt/label. The gnop man page says that running gnop on dev creates /dev/dev.nop. What happens if you gnop /dev/gpt/label? Is this what you are doing? When we use glabel for the same purpose with the combination of gnop, the pool imports fine. Any suggestions? It should be sufficient to only gnop one of the devices. You should be able to create the pool with only one gnop device to get the ashift value that you desire, export the pool, destroy the .nop device, and import the pool. If it things the device is corrupted (which seems like a bug of some sort), then ZFS should be able to resilver it. ___ freebsd-stable@freebsd.org mailing list
Re: LSI 9240-4i 4K alignment
On Sun, Aug 19, 2012 at 8:58 PM, Don Lewis truck...@freebsd.org wrote: On 8 Aug, George Kontostanos wrote: Hi all, We have a server with a LSI 9240-4i controller configured in JBOD with 4 SATA disks. Running FreeBSD 9.1-Beta1: Relevant dmesg: FreeBSD 9.1-BETA1 #0: Thu Jul 12 09:38:51 UTC 2012 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz (3200.09-MHz K8-class CPU) Origin = GenuineIntel Id = 0x206a7 Family = 6 Model = 2a Stepping = 7 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x1fbae3ffSSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX AMD Features=0x28100800SYSCALL,NX,RDTSCP,LM AMD Features2=0x1LAHF TSC: P-state invariant, performance statistics real memory = 17179869184 (16384 MB) avail memory = 16471670784 (15708 MB) ... mfi0: Drake Skinny port 0xe000-0xe0ff mem 0xf7a6-0xf7a63fff,0xf7a0-0xf7a3 irq 16 at device 0.0 on pci1 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 ... mfi0: 321 (397672301s/0x0020/info) - Shutdown command received from host mfi0: 322 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073/1000/9241/1000) mfi0: 323 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 mfi0: 324 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073/1000/9241/1000) mfi0: 325 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 mfi0: 326 (boot + 5s/0x0020/info) - Package version 20.10.1-0107 mfi0: 327 (boot + 5s/0x0020/info) - Board Revision 03A mfi0: 328 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) ... mfisyspd0 on mfi0 mfisyspd0: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd0: SYSPD volume attached mfisyspd1 on mfi0 mfisyspd1: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd1: SYSPD volume attached mfisyspd2 on mfi0 mfisyspd2: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd2: SYSPD volume attached mfisyspd3 on mfi0 mfisyspd3: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd3: SYSPD volume attached ... mfi0: 329 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) Info: enclPd=, scsiType=0, portMap=00, sasAddr=44332211, mfi0: 330 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) mfi0: 331 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) Info: enclPd=, scsiType=0, portMap=02, sasAddr=443322110200, mfi0: 332 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) mfi0: 333 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) Info: enclPd=, scsiType=0, portMap=03, sasAddr=443322110100, mfi0: 334 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) mfi0: 335 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) Info: enclPd=, scsiType=0, portMap=01, sasAddr=443322110300, mfi0: 336 (397672376s/0x0020/info) - Time established as 08/07/12 16:32:56; (28 seconds since power on) The problem: When trying to create a RaidZ pool using gpart and perform a 4K alignment using gnop, we get the follwoing error immediately after exporting the pool and destroying the .nop devices: id: 8043746387654554958 state: FAULTED status: One or more devices contains corrupted data. action: The pool cannot be imported due to damaged devices or data. The pool may be active on another system, but can be imported using the '-f' flag. see: http://illumos.org/msg/ZFS-8000-5E config: Pool FAULTED corrupted data raidz1-0ONLINE 13283347160590042564 UNAVAIL corrupted data 16981727992215676534 UNAVAIL corrupted data 6607570030658834339 UNAVAIL corrupted data 3435463242860701988 UNAVAIL corrupted data I'm planning on doing something similar, but I'm curious about how gnop and GPT labels interact. I want to partition the drives for my pool slightly on the small side so that I'm less likely to run into problems if I have to replace a drive in the future. If I used gpart to create and label a GPT partition on the drive, the partition will show up as /dev/gpt/label. The gnop man page says that running gnop on dev creates /dev/dev.nop. What happens if you gnop /dev/gpt/label? Is this what you are doing? When we use glabel for the same purpose with the combination of gnop, the pool imports fine. Any suggestions? It should be sufficient to only gnop one of the devices. You should be able to create the pool with only one gnop device to get the ashift value that you desire, export the pool, destroy the .nop device, and import the pool. If it things the device is corrupted (which seems like a bug of some sort), then ZFS should be able to resilver it. After a few
Re: LSI 9240-4i 4K alignment
- Original Message - From: Don Lewis truck...@freebsd.org To: kill...@multiplay.co.uk Cc: gkontos.m...@gmail.com; freebsd-stable@FreeBSD.org; freebsd-hardw...@freebsd.org Sent: Sunday, August 19, 2012 6:37 PM Subject: Re: LSI 9240-4i 4K alignment On 16 Aug, Steven Hartland wrote: - Original Message - From: George Kontostanos gkontos.m...@gmail.com You are right, the chip specs say: LSISAS2108 RAID-on-Chip The drives are identified as mfisyspd0, mfisyspd1, etc. The following might be interesting to you:- http://forums.servethehome.com/showthread.php?599-LSI-RAID-Controller-and-HBA-Complete-Listing-Plus-OEM-Models Which states:- LSI MegaRAID SAS 9240-4i 1x4 port internal SAS vertical, no cache, no BBU, RAID 0, 1, 10 and 5, can be crossflashed to LSI9211 IT/IR This is insteresting as this is the card we're using but in the 8 port version under mps :) I wish I would have known this earlier. I just put together a ZFS server using LSI MegaRAID SAS 9240-8i cards. The cabling probably would have been cleaner with the 9211-8i, but I went with the 9240 because the vendor that I purchased the cards from listed that 9240 as being PCI-Express 2.0, but didn't say that about the 9211. I also got the impression that the 9240 recognized JBOD drives with the off-the-shelf firmware, whereas the 9211 did not. Even LSI's own site is a bit confusing. They list the 9211 in the HBA section, but its specs don't mention JBOD, whereas the 9240 is listed in the RAID section and its specs do list JBOD. If the only physical difference between the cards is the connector position, it seems odd that they don't offer products with all the combinations of firmware and connector position. I haven't configured the ZFS pool yet, but I didn't have any trouble installing FreeBSD 9.1-BETA on the GPT partitioned boot drive, which shows up as an mfi device. I'm planning on getting the ZFS pool up and running in the next few days. HBA's are the way to go if your using ZFS to manage the disks, you only need RAID if your using a FS which doesn't manage the disk side well such as UFS. Its often quite common for RAID controllers to actually be slower vs RAID controllers as the RAID stack can get in the way. JBOD is generally what HBA's do by default which may be the reason why LSI's site doesn't mention it. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On 08/19/2012 14:04, Steven Hartland wrote: - Original Message - From: Don Lewis truck...@freebsd.org To: kill...@multiplay.co.uk Cc: gkontos.m...@gmail.com; freebsd-stable@FreeBSD.org; freebsd-hardw...@freebsd.org Sent: Sunday, August 19, 2012 6:37 PM Subject: Re: LSI 9240-4i 4K alignment On 16 Aug, Steven Hartland wrote: - Original Message - From: George Kontostanos gkontos.m...@gmail.com You are right, the chip specs say: LSISAS2108 RAID-on-Chip The drives are identified as mfisyspd0, mfisyspd1, etc. The following might be interesting to you:- http://forums.servethehome.com/showthread.php?599-LSI-RAID-Controller-and-HBA-Complete-Listing-Plus-OEM-Models Which states:- LSI MegaRAID SAS 9240-4i 1x4 port internal SAS vertical, no cache, no BBU, RAID 0, 1, 10 and 5, can be crossflashed to LSI9211 IT/IR This is insteresting as this is the card we're using but in the 8 port version under mps :) I wish I would have known this earlier. I just put together a ZFS server using LSI MegaRAID SAS 9240-8i cards. The cabling probably would have been cleaner with the 9211-8i, but I went with the 9240 because the vendor that I purchased the cards from listed that 9240 as being PCI-Express 2.0, but didn't say that about the 9211. I also got the impression that the 9240 recognized JBOD drives with the off-the-shelf firmware, whereas the 9211 did not. Even LSI's own site is a bit confusing. They list the 9211 in the HBA section, but its specs don't mention JBOD, whereas the 9240 is listed in the RAID section and its specs do list JBOD. If the only physical difference between the cards is the connector position, it seems odd that they don't offer products with all the combinations of firmware and connector position. I haven't configured the ZFS pool yet, but I didn't have any trouble installing FreeBSD 9.1-BETA on the GPT partitioned boot drive, which shows up as an mfi device. I'm planning on getting the ZFS pool up and running in the next few days. HBA's are the way to go if your using ZFS to manage the disks, you only need RAID if your using a FS which doesn't manage the disk side well such as UFS. Its often quite common for RAID controllers to actually be slower vs RAID controllers as the RAID stack can get in the way. JBOD is generally what HBA's do by default which may be the reason why LSI's site doesn't mention it. Regards Steve Just to clear up, The 9240 is a sas2008 based card with the megaraid software on top of it. In it's default config from LSI the FreeBSD mfi will recognize it in later versions of FreeBSD (The upcoming 9.1 for sure) Older versions of mfi will not recognize it. The card can be flashed with IT firmware and then becomes a 9211 HBA, but it's a bit more expensive than a 9211 is so that doesn't make sense to do in many cases. On the dmesg posted the firmware on the card is phase 11. This *must* be in lockstep with the driver version or the card may not play nicely. FreeBSD 8.3 and 9.0 have v13 of the driver, the upcoming 9.1 will have v14. Note that v14 fixes a *ton* of stability bugs, including issues where bad drives would hang the controller or prevent systems from booting. Thanks, Josh Paetzel ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
After contacting the manufacturer we got this response: -- My apologies for the wrong information provided in my previous email. I was under the impression that this OS is still supported but after checking with our developer, FreeBSD is currently not supported with the LSI Megaraid Cards due to some issue with the driver we've provided. It will be supported in our upcoming releases which may come by the end of this year. Please check back on our website during that time frame for the FreeBSD driver. Once again please accept my apologies for the inconvenience. - -- George Kontostanos Aicom telecoms ltd http://www.aisecure.net ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
- Original Message - From: George Kontostanos gkontos.m...@gmail.com To: FreeBSD Stable freebsd-stable@freebsd.org; freebsd-hardw...@freebsd.org Sent: Thursday, August 16, 2012 10:38 AM Subject: Re: LSI 9240-4i 4K alignment After contacting the manufacturer we got this response: -- My apologies for the wrong information provided in my previous email. I was under the impression that this OS is still supported but after checking with our developer, FreeBSD is currently not supported with the LSI Megaraid Cards due to some issue with the driver we've provided. It will be supported in our upcoming releases which may come by the end of this year. Please check back on our website during that time frame for the FreeBSD driver. Once again please accept my apologies for the inconvenience. - I would ask them if they have a fix, as if they do there's nothing stopping you building a custom kernel with the update. Its quite common for big companies to not officially support until a release but that doesn't mean they don't have a fix if you know what I mean :) In addition to this we're running LSI 2008 based controllers here on 8.2-RELEASE + manually merged in mps driver with ZFS just fine. They aren't running 4k aligned but they are working well. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On Thu, Aug 16, 2012 at 2:11 PM, Steven Hartland kill...@multiplay.co.uk wrote: - Original Message - From: George Kontostanos gkontos.m...@gmail.com To: FreeBSD Stable freebsd-stable@freebsd.org; freebsd-hardw...@freebsd.org Sent: Thursday, August 16, 2012 10:38 AM Subject: Re: LSI 9240-4i 4K alignment After contacting the manufacturer we got this response: -- My apologies for the wrong information provided in my previous email. I was under the impression that this OS is still supported but after checking with our developer, FreeBSD is currently not supported with the LSI Megaraid Cards due to some issue with the driver we've provided. It will be supported in our upcoming releases which may come by the end of this year. Please check back on our website during that time frame for the FreeBSD driver. Once again please accept my apologies for the inconvenience. - I would ask them if they have a fix, as if they do there's nothing stopping you building a custom kernel with the update. You are right but it seems more cost effective to get a new controller that has support under the mps driver. Its quite common for big companies to not officially support until a release but that doesn't mean they don't have a fix if you know what I mean :) I know. The problem with that controller is that for some reason it doesn't like gpart. I have gnoped the drives directly and it seems to be working so far. In addition to this we're running LSI 2008 based controllers here on 8.2-RELEASE + manually merged in mps driver with ZFS just fine. They aren't running 4k aligned but they are working well. Right, they fall under the mps driver like the following: LSI Logic SAS2004 (4 Port SAS) LSI Logic SAS2008 (8 Port SAS) LSI Logic SAS2108 (8 Port SAS) LSI Logic SAS2116 (16 Port SAS) LSI Logic SAS2208 (8 Port SAS) I have worked in the past with some of those and never had a problem. Regards Steve Cheers -- George Kontostanos Aicom telecoms ltd http://www.aisecure.net ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
- Original Message - From: George Kontostanos gkontos.m...@gmail.com I know. The problem with that controller is that for some reason it doesn't like gpart. I have gnoped the drives directly and it seems to be working so far. It still really smells like something higher up the layers than the controller tbh. In addition to this we're running LSI 2008 based controllers here on 8.2-RELEASE + manually merged in mps driver with ZFS just fine. They aren't running 4k aligned but they are working well. Right, they fall under the mps driver like the following: LSI Logic SAS2004 (4 Port SAS) LSI Logic SAS2008 (8 Port SAS) LSI Logic SAS2108 (8 Port SAS) LSI Logic SAS2116 (16 Port SAS) LSI Logic SAS2208 (8 Port SAS) Confused as your 9240-4i is a SAS2008 based card so you shouldnt be seeing any difference as it should be supported just fine under mps. The cards we have here are 9211-8i if that helps. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On Thu, Aug 16, 2012 at 7:26 PM, Steven Hartland kill...@multiplay.co.uk wrote: - Original Message - From: George Kontostanos gkontos.m...@gmail.com I know. The problem with that controller is that for some reason it doesn't like gpart. I have gnoped the drives directly and it seems to be working so far. It still really smells like something higher up the layers than the controller tbh. We have tried many combinations with different drives. Any other suggestions? In addition to this we're running LSI 2008 based controllers here on 8.2-RELEASE + manually merged in mps driver with ZFS just fine. They aren't running 4k aligned but they are working well. Right, they fall under the mps driver like the following: LSI Logic SAS2004 (4 Port SAS) LSI Logic SAS2008 (8 Port SAS) LSI Logic SAS2108 (8 Port SAS) LSI Logic SAS2116 (16 Port SAS) LSI Logic SAS2208 (8 Port SAS) Confused as your 9240-4i is a SAS2008 based card so you shouldnt be seeing any difference as it should be supported just fine under mps. The cards we have here are 9211-8i if that helps. You are right, the chip specs say: LSISAS2108 RAID-on-Chip The drives are identified as mfisyspd0, mfisyspd1, etc. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. -- George Kontostanos Aicom telecoms ltd http://www.aisecure.net ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
- Original Message - From: George Kontostanos gkontos.m...@gmail.com It still really smells like something higher up the layers than the controller tbh. We have tried many combinations with different drives. Any other suggestions? See below Confused as your 9240-4i is a SAS2008 based card so you shouldnt be seeing any difference as it should be supported just fine under mps. The cards we have here are 9211-8i if that helps. You are right, the chip specs say: LSISAS2108 RAID-on-Chip Hmm LSI's website says its a LSISAS2008 not a LSISAS2108 http://www.lsi.com/channel/products/storagecomponents/Pages/MegaRAIDSAS9240-4i.aspx The drives are identified as mfisyspd0, mfisyspd1, etc. Are you actually using HW RAID volumns then instead of direct disks? If so I would advice configuring the disks as passthrough. What does pciconf -lv say? On boot I see the following here to 2008 chip controller:- == /var/run/dmesg.boot == mps0: LSI SAS2008 port 0xe000-0xe0ff mem 0xfaf3c000-0xfaf3,0xfaf4-0xfaf7 irq 26 at device 0.0 on pci7 mps0: Firmware: 11.00.00.00 mps0: IOCCapabilities: 185cScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR mps0: [ITHREAD] == pciconf -lv == mps0@pci0:7:0:0: class=0x010700 card=0x30201000 chip=0x00721000 rev=0x03 hdr=0x00 vendor = 'LSI Logic (Was: Symbios Logic, NCR)' class = mass storage subclass = SAS I suspect reading around that the MegaRAID cards are have SAS chips on board but are superseed by the RAID part hence mfi vs mps I wonder if you could flash with the IT firmware instead of IR to make it show as mps instead of mfi? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
- Original Message - From: George Kontostanos gkontos.m...@gmail.com You are right, the chip specs say: LSISAS2108 RAID-on-Chip The drives are identified as mfisyspd0, mfisyspd1, etc. The following might be interesting to you:- http://forums.servethehome.com/showthread.php?599-LSI-RAID-Controller-and-HBA-Complete-Listing-Plus-OEM-Models Which states:- LSI MegaRAID SAS 9240-4i 1x4 port internal SAS vertical, no cache, no BBU, RAID 0, 1, 10 and 5, can be crossflashed to LSI9211 IT/IR This is insteresting as this is the card we're using but in the 8 port version under mps :) So based on that a simple firmware flash may be all you need. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On Thu, Aug 16, 2012 at 9:51 PM, Steven Hartland kill...@multiplay.co.uk wrote: - Original Message - From: George Kontostanos gkontos.m...@gmail.com You are right, the chip specs say: LSISAS2108 RAID-on-Chip The drives are identified as mfisyspd0, mfisyspd1, etc. The following might be interesting to you:- http://forums.servethehome.com/showthread.php?599-LSI-RAID-Controller-and-HBA-Complete-Listing-Plus-OEM-Models Which states:- LSI MegaRAID SAS 9240-4i 1x4 port internal SAS vertical, no cache, no BBU, RAID 0, 1, 10 and 5, can be crossflashed to LSI9211 IT/IR This is insteresting as this is the card we're using but in the 8 port version under mps :) So based on that a simple firmware flash may be all you need. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. mfi0@pci0:1:0:0:class=0x010400 card=0x92411000 chip=0x00731000 rev=0x03 hdr=0x00 vendor = 'LSI Logic / Symbios Logic' device = 'MegaRAID SAS 9240' class = mass storage subclass = RAID The controller is configured as jbod. I will follow your advice regarding the firmware. Thanks for your help! -- George Kontostanos Aicom telecoms ltd http://www.aisecure.net ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
Hi, Works fine for me with 9.0-p3 with the same controller, slight firmware differences: mfi0: 2709 (boot + 4s/0x0020/info) - Firmware version 2.120.244-1482 mfi0: 2710 (boot + 5s/0x0020/info) - Package version 20.10.1-0077 mfi0: 2711 (boot + 5s/0x0020/info) - Board Revision 04A I will be trying out 9.1-BETA on a new batch of machines next week ... Regards, Jan. On 08/08/2012, at 7:25 AM, George Kontostanos gkontos.m...@gmail.com wrote: Hi all, We have a server with a LSI 9240-4i controller configured in JBOD with 4 SATA disks. Running FreeBSD 9.1-Beta1: Relevant dmesg: FreeBSD 9.1-BETA1 #0: Thu Jul 12 09:38:51 UTC 2012 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz (3200.09-MHz K8-class CPU) Origin = GenuineIntel Id = 0x206a7 Family = 6 Model = 2a Stepping = 7 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x1fbae3ffSSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX AMD Features=0x28100800SYSCALL,NX,RDTSCP,LM AMD Features2=0x1LAHF TSC: P-state invariant, performance statistics real memory = 17179869184 (16384 MB) avail memory = 16471670784 (15708 MB) ... mfi0: Drake Skinny port 0xe000-0xe0ff mem 0xf7a6-0xf7a63fff,0xf7a0-0xf7a3 irq 16 at device 0.0 on pci1 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 ... mfi0: 321 (397672301s/0x0020/info) - Shutdown command received from host mfi0: 322 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073/1000/9241/1000) mfi0: 323 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 mfi0: 324 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073/1000/9241/1000) mfi0: 325 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 mfi0: 326 (boot + 5s/0x0020/info) - Package version 20.10.1-0107 mfi0: 327 (boot + 5s/0x0020/info) - Board Revision 03A mfi0: 328 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) ... mfisyspd0 on mfi0 mfisyspd0: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd0: SYSPD volume attached mfisyspd1 on mfi0 mfisyspd1: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd1: SYSPD volume attached mfisyspd2 on mfi0 mfisyspd2: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd2: SYSPD volume attached mfisyspd3 on mfi0 mfisyspd3: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd3: SYSPD volume attached ... mfi0: 329 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) Info: enclPd=, scsiType=0, portMap=00, sasAddr=44332211, mfi0: 330 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) mfi0: 331 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) Info: enclPd=, scsiType=0, portMap=02, sasAddr=443322110200, mfi0: 332 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) mfi0: 333 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) Info: enclPd=, scsiType=0, portMap=03, sasAddr=443322110100, mfi0: 334 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) mfi0: 335 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) Info: enclPd=, scsiType=0, portMap=01, sasAddr=443322110300, mfi0: 336 (397672376s/0x0020/info) - Time established as 08/07/12 16:32:56; (28 seconds since power on) The problem: When trying to create a RaidZ pool using gpart and perform a 4K alignment using gnop, we get the follwoing error immediately after exporting the pool and destroying the .nop devices: id: 8043746387654554958 state: FAULTED status: One or more devices contains corrupted data. action: The pool cannot be imported due to damaged devices or data. The pool may be active on another system, but can be imported using the '-f' flag. see: http://illumos.org/msg/ZFS-8000-5E config: Pool FAULTED corrupted data raidz1-0ONLINE 13283347160590042564 UNAVAIL corrupted data 16981727992215676534 UNAVAIL corrupted data 6607570030658834339 UNAVAIL corrupted data 3435463242860701988 UNAVAIL corrupted data When we use glabel for the same purpose with the combination of gnop, the pool imports fine. Any suggestions? -- George Kontostanos Aicom telecoms ltd http://www.aisecure.net ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
On Wed, Aug 8, 2012 at 10:05 AM, Jan Mikkelsen j...@transactionware.com wrote: Hi, Works fine for me with 9.0-p3 with the same controller, slight firmware differences: mfi0: 2709 (boot + 4s/0x0020/info) - Firmware version 2.120.244-1482 mfi0: 2710 (boot + 5s/0x0020/info) - Package version 20.10.1-0077 mfi0: 2711 (boot + 5s/0x0020/info) - Board Revision 04A I will be trying out 9.1-BETA on a new batch of machines next week ... Regards, Jan. Hi can you clarify if this was done using gpart and gnop? Thanks ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: LSI 9240-4i 4K alignment
Hi, On 09/08/2012, at 4:18 AM, George Kontostanos gkontos.m...@gmail.com wrote: On Wed, Aug 8, 2012 at 10:05 AM, Jan Mikkelsen j...@transactionware.com wrote: Hi, Works fine for me with 9.0-p3 with the same controller, slight firmware differences: ... Hi can you clarify if this was done using gpart and gnop? gnop on the raw disks. Before doing the gnop, there is a gpart destroy -F in the script, and dd'ing a bunch of zeros. Regards, Jan. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
LSI 9240-4i 4K alignment
Hi all, We have a server with a LSI 9240-4i controller configured in JBOD with 4 SATA disks. Running FreeBSD 9.1-Beta1: Relevant dmesg: FreeBSD 9.1-BETA1 #0: Thu Jul 12 09:38:51 UTC 2012 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz (3200.09-MHz K8-class CPU) Origin = GenuineIntel Id = 0x206a7 Family = 6 Model = 2a Stepping = 7 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x1fbae3ffSSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX AMD Features=0x28100800SYSCALL,NX,RDTSCP,LM AMD Features2=0x1LAHF TSC: P-state invariant, performance statistics real memory = 17179869184 (16384 MB) avail memory = 16471670784 (15708 MB) ... mfi0: Drake Skinny port 0xe000-0xe0ff mem 0xf7a6-0xf7a63fff,0xf7a0-0xf7a3 irq 16 at device 0.0 on pci1 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 ... mfi0: 321 (397672301s/0x0020/info) - Shutdown command received from host mfi0: 322 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073/1000/9241/1000) mfi0: 323 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 mfi0: 324 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073/1000/9241/1000) mfi0: 325 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 mfi0: 326 (boot + 5s/0x0020/info) - Package version 20.10.1-0107 mfi0: 327 (boot + 5s/0x0020/info) - Board Revision 03A mfi0: 328 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) ... mfisyspd0 on mfi0 mfisyspd0: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd0: SYSPD volume attached mfisyspd1 on mfi0 mfisyspd1: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd1: SYSPD volume attached mfisyspd2 on mfi0 mfisyspd2: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd2: SYSPD volume attached mfisyspd3 on mfi0 mfisyspd3: 1907729MB (3907029168 sectors) SYSPD volume mfisyspd3: SYSPD volume attached ... mfi0: 329 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) Info: enclPd=, scsiType=0, portMap=00, sasAddr=44332211, mfi0: 330 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) mfi0: 331 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) Info: enclPd=, scsiType=0, portMap=02, sasAddr=443322110200, mfi0: 332 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) mfi0: 333 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) Info: enclPd=, scsiType=0, portMap=03, sasAddr=443322110100, mfi0: 334 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) mfi0: 335 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) Info: enclPd=, scsiType=0, portMap=01, sasAddr=443322110300, mfi0: 336 (397672376s/0x0020/info) - Time established as 08/07/12 16:32:56; (28 seconds since power on) The problem: When trying to create a RaidZ pool using gpart and perform a 4K alignment using gnop, we get the follwoing error immediately after exporting the pool and destroying the .nop devices: id: 8043746387654554958 state: FAULTED status: One or more devices contains corrupted data. action: The pool cannot be imported due to damaged devices or data. The pool may be active on another system, but can be imported using the '-f' flag. see: http://illumos.org/msg/ZFS-8000-5E config: Pool FAULTED corrupted data raidz1-0ONLINE 13283347160590042564 UNAVAIL corrupted data 16981727992215676534 UNAVAIL corrupted data 6607570030658834339 UNAVAIL corrupted data 3435463242860701988 UNAVAIL corrupted data When we use glabel for the same purpose with the combination of gnop, the pool imports fine. Any suggestions? -- George Kontostanos Aicom telecoms ltd http://www.aisecure.net ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
