Hi all,
I'm new to the list and OI, but I've been running a Solaris box with some
version of ZFS on it for a couple years. I recently acquired a Supermicro
6047R-E1R36N. It has an X9DRi-LN4F+ motherboard, dual E5-2620s and 48GB of RAM.
I soon learned that I needed a different HBA (it comes with an LSI 2108-based
card that doesn't have any JBOD passthrough mode), so I replaced that with an
LSI SAS9211-8i, flashed with the latest "IT" firmware.
Being new to SAS expanders (which this server has two of), I quickly learned
that some older 2TB drives I had on my older Solaris box simply will not work
reliably on the expander. (They would very quickly lock up the entire file
system once the load got heavy.) However, I've had much better luck (so far)
with some newer 3TB SATA drives (Seagate ST3000DM001). I'm still not convinced
that these are going to be reliable in the long run, having now read about a
few SATA-on-SAS expander horror stories (and my own experience with the 2TB
drives), but this isn't a production system (yet) so I'm willing to see how it
goes.
But the one problem I can't seem to figure out are the Samsung 830 SSD drives I
have for ZIL and ARC. I have 2 mirrored 128GB 830s as the boot pool, but
they're on the built-in SATA ports, not the expander ports. Then I have four
other 128GB 830s which I installed into 3.5" hot-swap adapters and these are
the ones causing problems. The HBA sees the drives just fine—they're listed
during POST and I can access them via the card's built-in BIOS. But once OI
(151a5) is up and running, it only sees ONE of the 830s. And here's where it
gets weird: if I remove all but one of the SSDs from the expander ports, it
will see whichever one SSD is connected—it does not matter which port the
drives are on. Also, regardless of the port, the attachment path always seems
to be the same in the OS. It's like the OS thinks all four drives are the exact
same physical drive.
$ cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c6 scsi-sas connected configured unknown
c6::es/ses0 ESI connected configured unknown
c6::smp/expd0 smp connected configured unknown
c6::w5002538043584d30,0 disk-path connected configured unknown
<-- this is the SSD
c6::w5000c500513a0efe,0 disk-path connected configured unknown
c6::w5000c5005168b2f0,0 disk-path connected configured unknown
c6::w5000c5005169a612,0 disk-path connected configured unknown
c6::w5000c5005169cc6a,0 disk-path connected configured unknown
c6::w5000c500515199bd,0 disk-path connected configured unknown
c6::w5000c500517042d9,0 disk-path connected configured unknown
c8 scsi-sas connected configured unknown
c8::es/ses1 ESI connected configured unknown
c8::smp/expd1 smp connected configured unknown
c9 scsi-sas connected unconfigured unknown
sata0/0::dsk/c4t0d0 disk connected configured ok
sata0/1::dsk/c4t1d0 disk connected configured ok
sata0/2 sata-port empty unconfigured ok
sata0/3 sata-port empty unconfigured ok
sata0/4 sata-port empty unconfigured ok
sata0/5 sata-port empty unconfigured ok
$ format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c4t0d0 <ATA-SAMSUNGSSD830-3B1Q cyl 19932 alt 2 hd 224 sec 56>
/pci@0,0/pci15d9,626@1f,2/disk@0,0
1. c4t1d0 <ATA-SAMSUNGSSD830-3B1Q cyl 19932 alt 2 hd 224 sec 56>
/pci@0,0/pci15d9,626@1f,2/disk@1,0
2. c7t5000C500513A0EFEd0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
/scsi_vhci/disk@g5000c500513a0efe
3. c7t5000C5005168B2F0d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
/scsi_vhci/disk@g5000c5005168b2f0
4. c7t5000C5005169A612d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
/scsi_vhci/disk@g5000c5005169a612
5. c7t5000C5005169CC6Ad0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
/scsi_vhci/disk@g5000c5005169cc6a
6. c7t5000C500515199BDd0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
/scsi_vhci/disk@g5000c500515199bd
7. c7t5000C500517042D9d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
/scsi_vhci/disk@g5000c500517042d9
8. c7t5002538043584D30d0 <ATA-SAMSUNG SSD 830-3B1Q-119.24GB>
/scsi_vhci/disk@g5002538043584d30
This last one is the SSD. If I remove all the SSDs from the expander ports,
disk #8 does not show up (nor does it show in cfgadm). If I put one SSD into
any expander port, disk 8 is back. If I add additional SSDs, I still only have
disk 8. If I remove the first SSD (while others are still attached), disk 8
still appears. In all cases, I can run a read test on the disk (format ->
analyze -> test) without any errors, so I think it's actually accessing one of
the physical drives.
How do I get OI to see the other 3 SSDs?
Any pointers are greatly appreciated. Thanks!
-Scott
_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
