Re: [zfs-discuss] Format returning bogus controller info
On 02/28/11 22:39, Dave Pooser wrote: On 2/28/11 4:23 PM, Garrett D'Amore garr...@nexenta.com wrote: Drives are ordered in the order they are *enumerated* when they *first* show up in the system. *Ever*. Is the same true of controllers? That is, will c12 remain c12 or /pci@0,0/pci8086,340c@5 remain /pci@0,0/pci8086,340c@5 even if other controllers are active? Well, it will so long as the contents of /etc/path_to_inst, /dev/cfg, and /dev/[r]dsk remain untouched. If one is feeling intrepid (or annoyed that the SATA boot drives you want to be c0 are showing up as c4 because the USB controllers are getting enumerated first), one can modify this. For SAS controllers, selectively nuking the contents of /dev/dsk/cX, renaming /dev/cfg/cX to /dev/cfg/cY, updating the archive, and rebooting -r works rather well. For ICH10s, I've discovered that if one writes a for-loop that renames /dev/[r]dsk/c1t0d0s0 - c0t0d0s0 while otherwise cleansing out unwanted entries, the devfsadm reconfig process honors the naming implied by existing symlinks and re-numbers the USB controllers accordingly. Do *not* try this at ... well, actually, try it at home long before you try this on your production box, as this is also a very good way to make your machine completely unbootable - and I wouldn't mention that you're doing this to anyone at S.uh.Oracle... (Maybe there's an easier way. I don't know it, though.) -bacon who is incredibly bugged by the idea of the boot drive being anything BUT c0t0d0s0 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
Date: Mon, 28 Feb 2011 22:02:37 +0100 (CET) From: Roy Sigurd Karlsbakk r...@karlsbakk.net I cannot but agree. On Linux and Windoze (haven't tested FreeBSD), drives connected to an LSI9211 show up in the correct order, but not on OI/osol/S11ex (IIRC), and fmtopo doesn't always show a mapping between device name and slot, since that relies on the SES hardware being properly supported. The answer I've got for this issue is, it's not an issue, since it's that way by design etc. This doesn't make sense when Linux/Windows show the drives in the correct order. IMHO this looks more like a design flaw in the driver code lsiutil can help. The question is whether the physical labels or silkscreen match the slot as reported by lsiutil. Last I checked, it didn't help much. IMHO we need a driver that can display the drives in the order they're plugged in. Like Windoze. Like Linux. Like FreeBSD. I really don't understand what should be so hard to do it like the others. As one said I don't have their sources, both Linux and FreeBSD are OSS software, so the source should be ready quite easuly. OK, so I'm a little behind on this, but... Yes, it is nice to know controller order - well, maybe. That assume the LSI hasn't decided to remap targets because you've moved drives around - the 1068/78 supported target persistence, but the LSI2008 doesn't. Nothing's guaranteed. The SES driver is, I presume, a fairly generic driver for knowing about enclosures. sd(7) on the other hand doesn't really know crap, other than this is the DDI path to what I want to talk to from which it can infer a target, but it doesn't know that that target means anything, and in my humble experience (having a whole pile of SMC hardware on LSI controllers), it really has NFC - and it doesn't need to, since it's just the bit controlling the drive - what does it care what slot it's in? You can use lsiutil advanced commands to query the expander chip and get the topology, but that's a PITA. The best way to deal with it IMHO - assuming you're serious about the need and willing to pay a small fee - is to go to www.santools.com and buy a license for smartmon-ux (no relation to smartmontools). One command, and it will generate a complete topology of your system, mapping drive serial number to WWN to DDI path to enclosure to slot in one nice big file. (It can also do lots of other nifty things but that's not the point. However, being able to map /dev/dsk/cXtYdZsA - encBslotC and toggle LEDs is rather important when you've got a few hundred drives strung to a couple boxes. And at least for SMC 847s, the slots do match the silkscreen (or the schema in the manual). -bacon (it's amazing how many drives you can string off a box, with a little effort...) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
Last I checked, it didn't help much. IMHO we need a driver that can display the drives in the order they're plugged in. Like Windoze. Like Linux. Like FreeBSD. I really don't understand what should be so hard to do it like the others. As one said I don't have their sources, both Linux and FreeBSD are OSS software, so the source should be ready quite easuly. The difference is in the license. While I could look at what a BSD-licensed driver does, I do not want to look at a GPL-licensed driver and create problems for myself or my employer. It's no problem reusing methodics from a GPL driver in CDDL, but the code must be rewritten. Also, copying trivial parts won't break the license, but then, I guess the Solaris driver API is quite different from the one on Linux. Also, perhaps the FreeBSD driver works better at this? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On Thu, 2011-03-03 at 21:02 +0100, Roy Sigurd Karlsbakk wrote: Last I checked, it didn't help much. IMHO we need a driver that can display the drives in the order they're plugged in. Like Windoze. Like Linux. Like FreeBSD. I really don't understand what should be so hard to do it like the others. As one said I don't have their sources, both Linux and FreeBSD are OSS software, so the source should be ready quite easuly. The difference is in the license. While I could look at what a BSD-licensed driver does, I do not want to look at a GPL-licensed driver and create problems for myself or my employer. It's no problem reusing methodics from a GPL driver in CDDL, but the code must be rewritten. Also, copying trivial parts won't break the license, but then, I guess the Solaris driver API is quite different from the one on Linux. Also, perhaps the FreeBSD driver works better at this? What you're not realizing is that FreeBSD and Linux were designed with a different set of assumptions. The basic assumption there is that you won't disturb the probe order, either by adding hardware, or by having hardware that simply is non-deterministic in probing. Solaris doesn't do that. It enumerates the hardware the first time it sees it, and then remembers the physical path to that disk as an alias, usually both in /etc/path_to_instance and in the symbolic links that populate /dev/. The failure here is in your assumption that the Linux and *BSD behavior is correct. It is correct in a small system that doesn't have to concern itself with fallout from changes in the probe order due to e.g. dynamic reconfiguration, but in enterprise systems where devices and buses can be hot swapped, it fails miserably. The Solaris behavior is simply better, once you learn to accept it. - Garrett ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
(Dave P...I sent this yesterday, but it bounced on your email address) A small comment from me would be to create some test pools and replace devices in the pools to see if device names remain the same or change during these operations. If the device names change and the pools are unhappy, retest similar operations while the pools' are exported. I've seen enough controller/device numbering wreak havoc on pool availability that I'm automatically paranoid when I see the controller numbering that you started with. Thanks, Cindy On 02/28/11 22:39, Dave Pooser wrote: On 2/28/11 4:23 PM, Garrett D'Amore garr...@nexenta.com wrote: Drives are ordered in the order they are *enumerated* when they *first* show up in the system. *Ever*. Is the same true of controllers? That is, will c12 remain c12 or /pci@0,0/pci8086,340c@5 remain /pci@0,0/pci8086,340c@5 even if other controllers are active? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
Surprised that one of the most approachable outputs for any customer to use which would enable simple identification/resolution of many of these discussions didn't come up, namely: cfgadm -al for a reasonable physical mapping in which SAS/SATA drives are relatively easy to map out by ID and of course by association with their distinct physical controller paths. And if you want to relate this view to the underlying physical path to the device, then cfgadm -alv will clarify the physical mapping. Both commands can take controller::target arguments if you want to restrict the voluminous output somewhat. BTW, you also should be looking to mpathadm show LU device to successfully decode the virtual device entries. Craig On 1 Mar 2011, at 16:10, Cindy Swearingen wrote: (Dave P...I sent this yesterday, but it bounced on your email address) A small comment from me would be to create some test pools and replace devices in the pools to see if device names remain the same or change during these operations. If the device names change and the pools are unhappy, retest similar operations while the pools' are exported. I've seen enough controller/device numbering wreak havoc on pool availability that I'm automatically paranoid when I see the controller numbering that you started with. Thanks, Cindy On 02/28/11 22:39, Dave Pooser wrote: On 2/28/11 4:23 PM, Garrett D'Amore garr...@nexenta.com wrote: Drives are ordered in the order they are *enumerated* when they *first* show up in the system. *Ever*. Is the same true of controllers? That is, will c12 remain c12 or /pci@0,0/pci8086,340c@5 remain /pci@0,0/pci8086,340c@5 even if other controllers are active? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Craig Morgan Cinnabar Solutions Ltd t: +44 (0)791 338 3190 f: +44 (0)870 705 1726 e: cr...@cinnabar-solutions.com w: www.cinnabar-solutions.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 2/27/11 11:13 PM, James C. McPherson j...@opensolaris.org wrote: /pci@0,0/pci8086,340c@5/pci1000,3020@0 and /pci@0,0/pci8086,340e@7/pci1000,3020@0 which are in different slots on your motherboard and connected to different PCI Express Root Ports - which should help with transfer rates amongst other things. Have a look at /usr/share/hwdata/pci.ids for 340[0-9a-f] after the line which starts with 8086. That's the information I needed; I now have the drives allocated across multiple controllers for the fault-tolerance I was looking for. Thanks for all your help-- not only can I fully, unequivocally retract my failed bit crack, but I just ordered two more of these cards for my next project! :^) -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com http://www.alfordmedia.com/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
I cannot but agree. On Linux and Windoze (haven't tested FreeBSD), drives connected to an LSI9211 show up in the correct order, but not on OI/osol/S11ex (IIRC), and fmtopo doesn't always show a mapping between device name and slot, since that relies on the SES hardware being properly supported. The answer I've got for this issue is, it's not an issue, since it's that way by design etc. This doesn't make sense when Linux/Windows show the drives in the correct order. IMHO this looks more like a design flaw in the driver code lsiutil can help. The question is whether the physical labels or silkscreen match the slot as reported by lsiutil. Last I checked, it didn't help much. IMHO we need a driver that can display the drives in the order they're plugged in. Like Windoze. Like Linux. Like FreeBSD. I really don't understand what should be so hard to do it like the others. As one said I don't have their sources, both Linux and FreeBSD are OSS software, so the source should be ready quite easuly. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 1/03/11 03:00 AM, Dave Pooser wrote: On 2/27/11 11:13 PM, James C. McPhersonj...@opensolaris.org wrote: /pci@0,0/pci8086,340c@5/pci1000,3020@0 and /pci@0,0/pci8086,340e@7/pci1000,3020@0 which are in different slots on your motherboard and connected to different PCI Express Root Ports - which should help with transfer rates amongst other things. Have a look at /usr/share/hwdata/pci.ids for 340[0-9a-f] after the line which starts with 8086. That's the information I needed; I now have the drives allocated across multiple controllers for the fault-tolerance I was looking for. Thanks for all your help-- not only can I fully, unequivocally retract my failed bit crack, but I just ordered two more of these cards for my next project! :^) I'm glad I could help. cheers, James -- Oracle http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 1/03/11 07:02 AM, Roy Sigurd Karlsbakk wrote: ... Last I checked, it didn't help much. IMHO we need a driver that can display the drives in the order they're plugged in. Like Windoze. Like Linux. Like FreeBSD. I really don't understand what should be so hard to do it like the others. As one said I don't have their sources, both Linux and FreeBSD are OSS software, so the source should be ready quite easuly. The difference is in the license. While I could look at what a BSD-licensed driver does, I do not want to look at a GPL-licensed driver and create problems for myself or my employer. James C. McPherson -- Oracle http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 2/28/11 4:23 PM, Garrett D'Amore garr...@nexenta.com wrote: Drives are ordered in the order they are *enumerated* when they *first* show up in the system. *Ever*. Is the same true of controllers? That is, will c12 remain c12 or /pci@0,0/pci8086,340c@5 remain /pci@0,0/pci8086,340c@5 even if other controllers are active? -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 27/02/11 05:24 PM, Dave Pooser wrote: On 2/26/11 7:43 PM, Bill Sommerfeldsommerf...@hamachi.org wrote: On your system, c12 is the mpxio virtual controller; any disk which is potentially multipath-able (and that includes the SAS drives) will appear as a child of the virtual controller (rather than appear as the child of two or more different physical controllers). Hmm... That makes sense, except that my drives are all SATA because I'm cheap^H^H^H fiscally conservative. :^) They're attached to a SAS hba, which is doing translations for them using SATL - SAS to ATA Translation Layer. 'stmsboot -L' displayed no mappings, this is because mpt_sas(7d) controllers - which you have - are using MPxIO by default. Running stmsboot -L will only show mappings if you've enabled or disabled MPxIO but I went ahead and tried stmsboot -d to disable multipathing; ... and now you have disabled MPxIO, stmsboot -L should show mappings. after reboot instead of seeing nine disks on a single controller I now see ten different controllers (in a machine that has four PCI controllers and one motherboard controller): This is a side effect of how your expanders are configured to operate on your motherboard. locadmin@bigdawg2:~# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c10t2d0DEFAULT cyl 9965 alt 2 hd 224 sec 56 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@4/disk@p2,0 1. c13t5000CCA222DF92A0d0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@10/disk@w5000cca222df92a0,0 2. c14t5000CCA222DF8FBEd0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@1/disk@w5000cca222df8fbe,0 3. c15t5000CCA222E006B6d0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@8/disk@w5000cca222e006b6,0 4. c16t5000CCA222DDD7BAd0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340c@5/pci1000,3020@0/iport@2/disk@w5000cca222ddd7ba,0 5. c17t5000CCA222DF3CECd0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@20/disk@w5000cca222df3cec,0 6. c18t5000CCA222DEAFE6d0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@2/disk@w5000cca222deafe6,0 7. c19t5000CCA222E0A3DEd0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@4/disk@w5000cca222e0a3de,0 8. c20t5000CCA222E046B7d0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@8/disk@w5000cca222e046b7,0 9. c21t5000CCA222E0533Fd0DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@20/disk@w5000cca222e0533f,0 So now I'm more baffled than I started. Any other suggestions will be gratefully accepted... If you're lucky, your expanders and the enclosure that they're configured into will show up with one or more SES targets. If that's the case, you might be able to see bay numbers with the fmtopo command - when you run it as root: # /usr/lib/fm/fmd/fmtopo -V If this doesn't work for you, then you'll have to resort to the tried and tested use of dd to /dev/null for each disk, and see which lights blink. James C. McPherson -- Oracle http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 2/27/11 5:15 AM, James C. McPherson j...@opensolaris.org wrote: On 27/02/11 05:24 PM, Dave Pooser wrote: On 2/26/11 7:43 PM, Bill Sommerfeldsommerf...@hamachi.org wrote: On your system, c12 is the mpxio virtual controller; any disk which is potentially multipath-able (and that includes the SAS drives) will appear as a child of the virtual controller (rather than appear as the child of two or more different physical controllers). Hmm... That makes sense, except that my drives are all SATA because I'm cheap^H^H^H fiscally conservative. :^) They're attached to a SAS hba, which is doing translations for them using SATL - SAS to ATA Translation Layer. Yeah, but they're still not multipathable, are they? 'stmsboot -L' displayed no mappings, this is because mpt_sas(7d) controllers - which you have - are using MPxIO by default. Running stmsboot -L will only show mappings if you've enabled or disabled MPxIO but I went ahead and tried stmsboot -d to disable multipathing; ... and now you have disabled MPxIO, stmsboot -L should show mappings. Nope: locadmin@bigdawg2:~# stmsboot -L stmsboot: MPXIO disabled after reboot instead of seeing nine disks on a single controller I now see ten different controllers (in a machine that has four PCI controllers and one motherboard controller): This is a side effect of how your expanders are configured to operate on your motherboard. But there shouldn't be any expanders in the system-- the front backplane has six SFF-8087 ports to control 24 drives, and the rear backplane has three more SFF-8087 ports to control 12 more drives. Each of those ports is connected directly to an SFF-8087 port on an LSI 9211-8i controller, except that the ninth port is connected to the integrated LSI 2008 controller on the motherboard. If you're lucky, your expanders and the enclosure that they're configured into will show up with one or more SES targets. If that's the case, you might be able to see bay numbers with the fmtopo command - when you run it as root: # /usr/lib/fm/fmd/fmtopo -V If this doesn't work for you, then you'll have to resort to the tried and tested use of dd to /dev/null for each disk, and see which lights blink. I can live with that-- but I really want to know what (real, not virtual) controllers disks are connected to; I want to build 3 8-disk RAIDz2 vdevs now (with room for a fourth for expansion later) and I really want to make sure each of those vdevs has fewer than three disks per controller so a single controller failure can degrade my vdevs but not kill them. Probably my next step is going to be to take a look with Nexenta Core or FreeBSD (or maybe SolEx11 for a temporary eval) and see if either of those gives me a saner view, but other suggestions would be appreciated. -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com http://www.alfordmedia.com/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
I can live with that-- but I really want to know what (real, not virtual) controllers disks are connected to; I want to build 3 8-disk RAIDz2 vdevs now (with room for a fourth for expansion later) and I really want to make sure each of those vdevs has fewer than three disks per controller so a single controller failure can degrade my vdevs but not kill them. I cannot but agree. On Linux and Windoze (haven't tested FreeBSD), drives connected to an LSI9211 show up in the correct order, but not on OI/osol/S11ex (IIRC), and fmtopo doesn't always show a mapping between device name and slot, since that relies on the SES hardware being properly supported. The answer I've got for this issue is, it's not an issue, since it's that way by design etc. This doesn't make sense when Linux/Windows show the drives in the correct order. IMHO this looks more like a design flaw in the driver code Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 2/27/11 11:18 AM, Roy Sigurd Karlsbakk r...@karlsbakk.net wrote: I cannot but agree. On Linux and Windoze (haven't tested FreeBSD), drives connected to an LSI9211 show up in the correct order, but not on OI/osol/S11ex (IIRC), and fmtopo doesn't always show a mapping between device name and slot, since that relies on the SES hardware being properly supported. The answer I've got for this issue is, it's not an issue, since it's that way by design etc. This doesn't make sense when Linux/Windows show the drives in the correct order. IMHO this looks more like a design flaw in the driver code Especially since the SAS3081 cards work as expected. I guess I'll start looking for some more of the 3Gb SAS controllers and chalk the 9211s up as a failed bit. -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com http://www.alfordmedia.com/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 28/02/11 07:30 AM, Dave Pooser wrote: On 2/27/11 11:18 AM, Roy Sigurd Karlsbakkr...@karlsbakk.net wrote: I cannot but agree. On Linux and Windoze (haven't tested FreeBSD), drives connected to an LSI9211 show up in the correct order, but not on OI/osol/S11ex (IIRC), and fmtopo doesn't always show a mapping between device name and slot, since that relies on the SES hardware being properly supported. The answer I've got for this issue is, it's not an issue, since it's that way by design etc. This doesn't make sense when Linux/Windows show the drives in the correct order. IMHO this looks more like a design flaw in the driver code Especially since the SAS3081 cards work as expected. I guess I'll start looking for some more of the 3Gb SAS controllers and chalk the 9211s up as a failed bit. LSISAS3081 cards use a different driver, mpt(7d), which still also supports parallel SCSI and has different firmware to what you have in your 9211 card. The 9211 card is also 2nd generation SAS, not 1st generation like the 3081. Personally, having worked on the mpt_sas(7d) project, I'm disappointed that you believe the card and its driver are a failed bit. James C. McPherson -- Oracle http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 28/02/11 03:18 AM, Roy Sigurd Karlsbakk wrote: I can live with that-- but I really want to know what (real, not virtual) controllers disks are connected to; I want to build 3 8-disk RAIDz2 vdevs now (with room for a fourth for expansion later) and I really want to make sure each of those vdevs has fewer than three disks per controller so a single controller failure can degrade my vdevs but not kill them. I cannot but agree. On Linux and Windoze (haven't tested FreeBSD), drives connected to an LSI9211 show up in the correct order, but not on OI/osol/S11ex (IIRC), and fmtopo doesn't always show a mapping between device name and slot, since that relies on the SES hardware being properly supported. The answer I've got for this issue is, it's not an issue, since it's that way by design etc. This doesn't make sense when Linux/Windows show the drives in the correct order. IMHO this looks more like a design flaw in the driver code In Solaris-land we have different design requirements for drivers when compared with Linux and Windows. As to what Linux and Windows do inside their drivers, I cannot say - I don't have their source. It is possible that since Linux and Windows are fairly closely tied to the PC architecture, that perhaps they do some bios calls to try to figure out correct order mappings. James C. McPherson -- Oracle http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 28/02/11 02:08 AM, Dave Pooser wrote: On 2/27/11 5:15 AM, James C. McPhersonj...@opensolaris.org wrote: On 27/02/11 05:24 PM, Dave Pooser wrote: On 2/26/11 7:43 PM, Bill Sommerfeldsommerf...@hamachi.org wrote: On your system, c12 is the mpxio virtual controller; any disk which is potentially multipath-able (and that includes the SAS drives) will appear as a child of the virtual controller (rather than appear as the child of two or more different physical controllers). Hmm... That makes sense, except that my drives are all SATA because I'm cheap^H^H^H fiscally conservative. :^) They're attached to a SAS hba, which is doing translations for them using SATL - SAS to ATA Translation Layer. Yeah, but they're still not multipathable, are they? Not _physically_ multipathable, but the driver and the chip combine their efforts to make this opaque to you. That way you, the user, only have to worry about one path. 'stmsboot -L' displayed no mappings, this is because mpt_sas(7d) controllers - which you have - are using MPxIO by default. Running stmsboot -L will only show mappings if you've enabled or disabled MPxIO but I went ahead and tried stmsboot -d to disable multipathing; ... and now you have disabled MPxIO, stmsboot -L should show mappings. Nope: locadmin@bigdawg2:~# stmsboot -L stmsboot: MPXIO disabled Sorry, forgot one step (it's been a while since I actually disabled MPxIO): you have to re-enable MPxIO :-) after reboot instead of seeing nine disks on a single controller I now see ten different controllers (in a machine that has four PCI controllers and one motherboard controller): This is a side effect of how your expanders are configured to operate on your motherboard. But there shouldn't be any expanders in the system-- the front backplane has six SFF-8087 ports to control 24 drives, and the rear backplane has three more SFF-8087 ports to control 12 more drives. Each of those ports is connected directly to an SFF-8087 port on an LSI 9211-8i controller, except that the ninth port is connected to the integrated LSI 2008 controller on the motherboard. I misread your initial email, sorry. So your disks are connected to separate PHYs on the HBA, by virtue of their cabling. You can see this for yourself by looking at the iport@xx element in the physical paths: 1. c13t5000CCA222DF92A0d0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@10/disk@w5000cca222df92a0,0 2. c14t5000CCA222DF8FBEd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@1/disk@w5000cca222df8fbe,0 The xx part is a bitmask, starting from 0, which gives you an indication of which PHY the device is attached to. Your disk #1 above is connected to iport@10, which is PHY #4 when you have x1 ports: PHY iport@ 0 1 1 2 2 4 3 8 4 10 5 20 6 40 7 80 otoh, if you have x4 wide ports: PHY iport@0 0 f (these are really 0f, but we drop the leading 0) 1 f 2 f 3 f 4 f0 5 f0 6 f0 7 f0 If you're lucky, your expanders and the enclosure that they're configured into will show up with one or more SES targets. If that's the case, you might be able to see bay numbers with the fmtopo command - when you run it as root: # /usr/lib/fm/fmd/fmtopo -V If this doesn't work for you, then you'll have to resort to the tried and tested use of dd to /dev/null for each disk, and see which lights blink. I can live with that-- but I really want to know what (real, not virtual) controllers disks are connected to; I want to build 3 8-disk RAIDz2 vdevs now (with room for a fourth for expansion later) and I really want to make sure each of those vdevs has fewer than three disks per controller so a single controller failure can degrade my vdevs but not kill them. With the information above about the PHY/iport relationship, I hope you can now see better what your physical layout is. Also, please remember that using MPxIO means you have a single virtual controller, and the driver stack handles the translation to physical for you so you don't have to worry about that aspect. Of course, if you want to worry about it, feel free. Probably my next step is going to be to take a look with Nexenta Core or FreeBSD (or maybe SolEx11 for a temporary eval) and see if either of those gives me a saner view, but other suggestions would be appreciated. Not sure where FreeBSD is up to, nor Nexenta Core. If Nexenta Core includes changes from snv_118 onwards, then you'll have pretty much the same mpt_sas(7d) driver that S11x has. James C. McPherson -- Oracle http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 2/27/11 4:07 PM, James C. McPherson j...@opensolaris.org wrote: I misread your initial email, sorry. No worries-- I probably could have written it more clearly. So your disks are connected to separate PHYs on the HBA, by virtue of their cabling. You can see this for yourself by looking at the iport@xx element in the physical paths: 1. c13t5000CCA222DF92A0d0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@10/disk@w5000cca222df92a0,0 2. c14t5000CCA222DF8FBEd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@1/disk@w5000cca222df8fbe,0 The xx part is a bitmask, starting from 0, which gives you an indication of which PHY the device is attached to. Your disk #1 above is connected to iport@10, which is PHY #4 when you have x1 ports: PHY iport@ 01 12 24 38 410 520 640 780 OK, bear with me for a moment because I'm feeling extra dense this evening. The PHY tells me which port on the HBA I'm connected to. What tells me which HBA? That's the information I care most about, and if that information is contained up there I'll do a happy dance and head on in to the office to start building zpools. With the information above about the PHY/iport relationship, I hope you can now see better what your physical layout is. Also, please remember that using MPxIO means you have a single virtual controller, and the driver stack handles the translation to physical for you so you don't have to worry about that aspect. Of course, if you want to worry about it, feel free. Well, I want to make sure that a single controller failure can't cause any of my RAIDz2 vdevs to fault. I know I can do that manually by building the vdevs in such a way that no more than two drives are on a single controller. If the virtual controller is smart enough to do that automagically-- when I'm using SATA disks and a backplane that doesn't support multipathing-- then I have no complaints and I owe you a beer or three the next time you're in the Dallas area. But that seems unlikely to me, and so I think I have to worry about it. I'd love to be wrong, though! Personally, having worked on the mpt_sas(7d) project, I'm disappointed that you believe the card and its driver are a failed bit. I'd like to revise and extend my remarks and replace that with a suboptimal choice for this project. In fact, if I can't make this work my backup plan is to take some of my storage towers that have only one HBA, put the 9211s in them and grab the LSISAS3081 cards out of those towers for this beast. So those cards will still get productive use -- not a failed bit, at worst just not serving the purpose I had in mind. -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 28/02/11 12:46 PM, Dave Pooser wrote: On 2/27/11 4:07 PM, James C. McPhersonj...@opensolaris.org wrote: ... PHY iport@ 01 12 24 38 410 520 640 780 OK, bear with me for a moment because I'm feeling extra dense this evening. The PHY tells me which port on the HBA I'm connected to. What tells me which HBA? The devinfo path that's reported in the non-MPxIO output from format /dev/null (see below) That's the information I care most about, and if that information is contained up there I'll do a happy dance and head on in to the office to start building zpools. I've arranged these by devinfo path: 1st controller c10t2d0/pci@0,0/pci8086,340a@3/pci1000,72@0/iport@4/disk@p2,0 c15t5000CCA222E006B6d0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@8/disk@w5000cca222e006b6,0 c13t5000CCA222DF92A0d0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@10/disk@w5000cca222df92a0,0 c12t5000CCA222E0533Fd0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@20/disk@w5000cca222e0533f,0 The most likely reason why you're seeing a c10t2d0 is because the disk is failing to respond in the required fashion for a particular SCSI INQUIRY command when the disk is attached to the system. 2nd controller c16t5000CCA222DDD7BAd0 /pci@0,0/pci8086,340c@5/pci1000,3020@0/iport@2/disk@w5000cca222ddd7ba,0 3rd controller c14t5000CCA222DF8FBEd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@1/disk@w5000cca222df8fbe,0 c18t5000CCA222DEAFE6d0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@2/disk@w5000cca222deafe6,0 c19t5000CCA222E0A3DEd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@4/disk@w5000cca222e0a3de,0 c20t5000CCA222E046B7d0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@8/disk@w5000cca222e046b7,0 c17t5000CCA222DF3CECd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@20/disk@w5000cca222df3cec,0 With the information above about the PHY/iport relationship, I hope you can now see better what your physical layout is. Also, please remember that using MPxIO means you have a single virtual controller, and the driver stack handles the translation to physical for you so you don't have to worry about that aspect. Of course, if you want to worry about it, feel free. Well, I want to make sure that a single controller failure can't cause any of my RAIDz2 vdevs to fault. I know I can do that manually by building the vdevs in such a way that no more than two drives are on a single controller. If the virtual controller is smart enough to do that automagically-- when I'm using SATA disks and a backplane that doesn't support multipathing-- then I have no complaints and I owe you a beer or three the next time you're in the Dallas area. But that seems unlikely to me, and so I think I have to worry about it. I'd love to be wrong, though! No, the controller doesn't do that for you. That's one bit of AI that we haven't quite got to just yet. Unless you wanted to spend $LOTS on a full frame HDS beast ;-) Personally, having worked on the mpt_sas(7d) project, I'm disappointed that you believe the card and its driver are a failed bit. I'd like to revise and extend my remarks and replace that with a suboptimal choice for this project. Not knowing your other requirements for the project, I'll settle for this version :) [snip] hth, James -- Oracle http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 2/27/11 10:06 PM, James C. McPherson j...@opensolaris.org wrote: I've arranged these by devinfo path: 1st controller c10t2d0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@4/disk@p2,0 c15t5000CCA222E006B6d0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@8/disk@w5000cca222e006b6,0 c13t5000CCA222DF92A0d0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@10/disk@w5000cca222df92a0,0 c12t5000CCA222E0533Fd0 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@20/disk@w5000cca222e0533f,0 The most likely reason why you're seeing a c10t2d0 is because the disk is failing to respond in the required fashion for a particular SCSI INQUIRY command when the disk is attached to the system. That's an inexpensive SSD used as the boot disk, so it's different enough from the other devices I can't say I'm stunned that it behaves differently. 2nd controller c16t5000CCA222DDD7BAd0 /pci@0,0/pci8086,340c@5/pci1000,3020@0/iport@2/disk@w5000cca222ddd7ba,0 3rd controller c14t5000CCA222DF8FBEd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@1/disk@w5000cca222df8fbe,0 c18t5000CCA222DEAFE6d0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@2/disk@w5000cca222deafe6,0 c19t5000CCA222E0A3DEd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@4/disk@w5000cca222e0a3de,0 c20t5000CCA222E046B7d0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@8/disk@w5000cca222e046b7,0 c17t5000CCA222DF3CECd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@20/disk@w5000cca222df3cec,0 So I mentioned I'm dense tonight, right? Is the key there where it says 340x@y, so each controller will have a different letter associated with it and a different number after the @? (That is, presumably in this system there's a 340b@4 and a 340d@6 if I add more drives and try 'format' again?) I'd like to revise and extend my remarks and replace that with a suboptimal choice for this project. Not knowing your other requirements for the project, I'll settle for this version :) Actually at this point I think I have to re-revise it to just fine for this project had I brains enough to comprehend the output of 'format'. :^) -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 28/02/11 02:51 PM, Dave Pooser wrote: On 2/27/11 10:06 PM, James C. McPhersonj...@opensolaris.org wrote: ... 2nd controller c16t5000CCA222DDD7BAd0 /pci@0,0/pci8086,340c@5/pci1000,3020@0/iport@2/disk@w5000cca222ddd7ba,0 3rd controller c14t5000CCA222DF8FBEd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@1/disk@w5000cca222df8fbe,0 c18t5000CCA222DEAFE6d0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@2/disk@w5000cca222deafe6,0 c19t5000CCA222E0A3DEd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@4/disk@w5000cca222e0a3de,0 c20t5000CCA222E046B7d0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@8/disk@w5000cca222e046b7,0 c17t5000CCA222DF3CECd0 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@20/disk@w5000cca222df3cec,0 So I mentioned I'm dense tonight, right? Is the key there where it says 340x@y, so each controller will have a different letter associated with it and a different number after the @? (That is, presumably in this system there's a 340b@4 and a 340d@6 if I add more drives and try 'format' again?) So you've got two controllers /pci@0,0/pci8086,340c@5/pci1000,3020@0 and /pci@0,0/pci8086,340e@7/pci1000,3020@0 which are in different slots on your motherboard and connected to different PCI Express Root Ports - which should help with transfer rates amongst other things. Have a look at /usr/share/hwdata/pci.ids for 340[0-9a-f] after the line which starts with 8086. The addresses are hexadecimal, and based on the probe order for each slot, as well as on the PCI Express device id. Figuring out what the unit address (after the @) is for this part of the device path is deep magic that's contained within the nexus driver for the bus type; in this case, pcie. Your 2nd controller is attached to 340c 5520/X58 I/O Hub PCI Express Root Port 5 while the 3rd is attached to 340e 5520/5500/X58 I/O Hub PCI Express Root Port 7 You mention 340b and 340d - these are 340b 5520/X58 I/O Hub PCI Express Root Port 4 and 340d 5520/X58 I/O Hub PCI Express Root Port 6 Now, just to make things more complex, you can have different root nodes as well as pci-to-pci-X and pci-to-pci-E bridges, and depending on what your BIOS thinks is going on, you'll see different nexus(es) in use. If you want to dig into this more, I suggest that we take the discussion offline and that you send me a copy of the output from prtconf -v and fmtopo -dV run as root. cheers, James -- Oracle http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On Feb 27, 2011, at 9:18 AM, Roy Sigurd Karlsbakk wrote: I can live with that-- but I really want to know what (real, not virtual) controllers disks are connected to; I want to build 3 8-disk RAIDz2 vdevs now (with room for a fourth for expansion later) and I really want to make sure each of those vdevs has fewer than three disks per controller so a single controller failure can degrade my vdevs but not kill them. I cannot but agree. On Linux and Windoze (haven't tested FreeBSD), drives connected to an LSI9211 show up in the correct order, but not on OI/osol/S11ex (IIRC), and fmtopo doesn't always show a mapping between device name and slot, since that relies on the SES hardware being properly supported. The answer I've got for this issue is, it's not an issue, since it's that way by design etc. This doesn't make sense when Linux/Windows show the drives in the correct order. IMHO this looks more like a design flaw in the driver code lsiutil can help. The question is whether the physical labels or silkscreen match the slot as reported by lsiutil. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Format returning bogus controller info
The hardware: SuperMicro 847A chassis (3 drive bays in 4U) -- A means there are 9 SFF-8087 ports on the backplanes, each controlling 4 drives; no expanders here. SuperMicro X8DTH-6F motherboard with integrated LSI 2008 SAS chipset, flashed to IT firmware, connected to one backplane port. Four LSI 9211-8i SAS controllers, flashed to IT firmware, each connected to two backplane ports The OS: OpenSolaris b134, installed off a USB stick created using the instructions at http://blogs.sun.com/clayb/entry/creating_opensolaris_usb_sticks_is The problem: While trying to add drives one at a time so I can identify them for later use, I noticed two interesting things: the controller information is unlike any I've seen before, and out of nine disks added after the boot drive all nine are attached to c12 -- and no single controller has more than eight ports. The output of format: locadmin@bigdawg2:~# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c10t2d0 DEFAULT cyl 9965 alt 2 hd 224 sec 56 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@4/disk@p2,0 1. c12t5000CCA222DDD7BAd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222ddd7ba 2. c12t5000CCA222DEAFE6d0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222deafe6 3. c12t5000CCA222DF3CECd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222df3cec 4. c12t5000CCA222DF8FBEd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222df8fbe 5. c12t5000CCA222DF92A0d0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222df92a0 6. c12t5000CCA222E0A3DEd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222e0a3de 7. c12t5000CCA222E006B6d0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222e006b6 8. c12t5000CCA222E046B7d0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222e046b7 9. c12t5000CCA222E0533Fd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /scsi_vhci/disk@g5000cca222e0533f Specify disk (enter its number): ^C Any suggestions? -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 02/26/11 17:21, Dave Pooser wrote: While trying to add drives one at a time so I can identify them for later use, I noticed two interesting things: the controller information is unlike any I've seen before, and out of nine disks added after the boot drive all nine are attached to c12 -- and no single controller has more than eight ports. on your system, c12 is the mpxio virtual controller; any disk which is potentially multipath-able (and that includes the SAS drives) will appear as a child of the virtual controller (rather than appear as the child of two or more different physical controllers). see stmsboot(1m) for information on how to turn that off if you don't need multipathing and don't like the longer device names. - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Format returning bogus controller info
On 2/26/11 7:43 PM, Bill Sommerfeld sommerf...@hamachi.org wrote: On your system, c12 is the mpxio virtual controller; any disk which is potentially multipath-able (and that includes the SAS drives) will appear as a child of the virtual controller (rather than appear as the child of two or more different physical controllers). Hmm... That makes sense, except that my drives are all SATA because I'm cheap^H^H^H fiscally conservative. :^) 'stmsboot -L' displayed no mappings, but I went ahead and tried stmsboot -d to disable multipathing; after reboot instead of seeing nine disks on a single controller I now see ten different controllers (in a machine that has four PCI controllers and one motherboard controller): locadmin@bigdawg2:~# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c10t2d0 DEFAULT cyl 9965 alt 2 hd 224 sec 56 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@4/disk@p2,0 1. c13t5000CCA222DF92A0d0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@10/disk@w5000cca222df92a0,0 2. c14t5000CCA222DF8FBEd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@1/disk@w5000cca222df8fbe,0 3. c15t5000CCA222E006B6d0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@8/disk@w5000cca222e006b6,0 4. c16t5000CCA222DDD7BAd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340c@5/pci1000,3020@0/iport@2/disk@w5000cca222ddd7ba,0 5. c17t5000CCA222DF3CECd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@20/disk@w5000cca222df3cec,0 6. c18t5000CCA222DEAFE6d0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@2/disk@w5000cca222deafe6,0 7. c19t5000CCA222E0A3DEd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@4/disk@w5000cca222e0a3de,0 8. c20t5000CCA222E046B7d0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340e@7/pci1000,3020@0/iport@8/disk@w5000cca222e046b7,0 9. c21t5000CCA222E0533Fd0 DEFAULT cyl 60798 alt 2 hd 255 sec 252 /pci@0,0/pci8086,340a@3/pci1000,72@0/iport@20/disk@w5000cca222e0533f,0 So now I'm more baffled than I started. Any other suggestions will be gratefully accepted... -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
