On Wed, Jul 8, 2009 at 3:20 PM, Chris Worley<worl...@gmail.com> wrote:
> On Wed, Jul 8, 2009 at 2:57 PM, Erwin Tsaur<erwin.ts...@sun.com> wrote:
>> Chris Worley wrote:
>>>
>>> On Wed, Jul 8, 2009 at 2:42 PM, Erwin Tsaur<erwin.ts...@sun.com> wrote:
>>>
>>>>
>>>> Chris Worley wrote:
>>>>
>>>>>
>>>>> On Wed, Jul 8, 2009 at 1:10 PM, Erwin Tsaur<erwin.ts...@sun.com> wrote:
>>>>>
>>>>>
>>>>>>
>>>>>> Chris Worley wrote:
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> (Sorry for the misleading "Subject" in the initial post.  would like
>>>>>>> to know a more appropriate place to post, since fm is just the
>>>>>>> messenger here.)
>>>>>>>
>>>>>>> More to add: fmadm faulty may be saying something about a bad PCIe
>>>>>>> slot or device (is there an "lspci" in OpenSolaris?):
>>>>>>>
>>>>>>> # fmadm faulty
>>>>>>> --------------- ------------------------------------  --------------
>>>>>>> ---------
>>>>>>> TIME            EVENT-ID                              MSG-ID
>>>>>>> SEVERITY
>>>>>>> --------------- ------------------------------------  --------------
>>>>>>> ---------
>>>>>>> Jul 07 07:55:42 016cf20c-d572-42c1-f217-9eb8d439b73c  PCIEX-8000-KP
>>>>>>>  Major
>>>>>>>
>>>>>>> Fault class : fault.io.pciex.device-interr-corr max 29%
>>>>>>>            fault.io.pciex.bus-linkerr-corr max 14%
>>>>>>> Affects     : dev:////p...@0,0/pci8086,3...@1/pci15d9,1...@0
>>>>>>>            dev:////p...@0,0/pci8086,3...@1/pci15d9,1...@0,1
>>>>>>>            dev:////p...@0,0/pci8086,3...@1
>>>>>>>                faulted but still in service
>>>>>>> FRU         : "MB"
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> (hc://:product-id=X8DTH-i-6-iF-6F:chassis-id=1234567890:server-id=opensolaris/motherboard=0)
>>>>>>>                faulty
>>>>>>>
>>>>>>> Description : Too many recovered bus errors have been detected, which
>>>>>>> indicates
>>>>>>>            a problem with the specified bus or with the specified
>>>>>>>            transmitting device. This may degrade into an unrecoverable
>>>>>>>            fault.
>>>>>>>            Refer to http://sun.com/msg/PCIEX-8000-KP for more
>>>>>>> information.
>>>>>>>
>>>>>>> Response    : One or more device instances may be disabled
>>>>>>>
>>>>>>> Impact      : Loss of services provided by the device instances
>>>>>>> associated
>>>>>>> with
>>>>>>>            this fault
>>>>>>>
>>>>>>> Action      : If a plug-in card is involved check for badly-seated
>>>>>>> cards
>>>>>>> or
>>>>>>>            bent pins. Otherwise schedule a repair procedure to replace
>>>>>>> the
>>>>>>>            affected device.  Use fmadm faulty to identify the device
>>>>>>> or
>>>>>>>            contact Sun for support.
>>>>>>>
>>>>>>> How bad is this error?  I need to put some adapters in, but it sounds
>>>>>>> like the OS doesn't handle the NHM's IOH (or is it really detaining a
>>>>>>> HW issue?).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> OS does handle these issues and unfortunately it is a HW issue.  This
>>>>>> is
>>>>>> likely to eventually cause your system to panic or fill up your hard
>>>>>> drive.
>>>>>>  Assuming you are seeing a lot of btlp and rto errors..  If anything
>>>>>> these
>>>>>> errors are performance killer.  Not only is the RTO/BTLP error telling
>>>>>> you
>>>>>> that many packets require retransmit, the OS also has to constantly go
>>>>>> out
>>>>>> and scan and clean up the fabric.
>>>>>>
>>>>>>
>>>>>
>>>>> This system is triple boot: RHEL5.3, W2008S, and OpenSolaris.
>>>>>
>>>>> The errors in OpenSolaris occur if no cards are installed in the bus.
>>>>>
>>>>> The other OSes don't report any errors w/ or w/o cards in the bus.
>>>>>
>>>>>
>>>>
>>>> This doesn't happen when there are no cards installed, since the error is
>>>> literally complaining about a packets received between 2 devices.  Are
>>>> you
>>>> sure it's you are correctly identifying the right slot?
>>>>
>>>> I believe only OpenSolaris even detects these errors, which is why the
>>>> other
>>>> OSes don't report any errors.  It doesn't mean that errors aren't
>>>> occurring
>>>> though.
>>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>> It would also be nice to throttle the errlog so it doesn't fill the
>>>>>>> disk an hour after boot.  Is this possible?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> no throttling possible, but you could turn it off, though highly not
>>>>>> recommended, it's better to fix the issue.  It really could just be a
>>>>>> badly
>>>>>> seated card.
>>>>>>
>>>>>>
>>>>>
>>>>> How do I disable the errors?
>>>>>
>>>>>
>>>>
>>>> We need to figure out exactly what your error is first, please provide
>>>> the
>>>> "fmdump -eV" log.  If it is huge, just tail the last 500-1000 lines
>>>> should
>>>> be enough.
>>>>
>>>
>>> Would that produce the same as the incantation shown earlier?:
>>>
>>
>> I think all the CE's would produce the same message.  I also need to know
>> the exact device.
>
> Last 1000 lines (of ~20 million) attached.
>
> There are some boards in the bus at this time, but the same error
> occurs w/o them, and their drivers are not yet loaded.  Everything
> else is built-in to the motherboard.

I rebooted w/o any adapters in any slots; the last 1000 lines are attached.

Thanks

Chris
>
> Thanks,
>
> Chris
>
>>>><snip>
>
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x18
        pci_command = 0x47
        pcie_status = 0x11
        pcie_command = 0x2037
        pcie_dev_cap = 0x10008cc2
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62011
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x40
        pcie_ce_mask = 0x0
        remainder = 0x2
        severity = 0x3
        __ttl = 0x1
        __tod = 0x4a550326 0x6030919

Jul 08 2009 14:35:50.100849579 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x85ab13d7e802801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        __ttl = 0x1
        __tod = 0x4a550326 0x602d7ab

Jul 08 2009 14:35:50.100870080 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x85ab1429b102801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1/pci15d9,1...@0,1
        (end detector)

        bdf = 0x101
        device_id = 0x10c9
        vendor_id = 0x8086
        rev_id = 0x1
        dev_type = 0x0
        pcie_off = 0xa0
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x18
        pci_command = 0x47
        pcie_status = 0x11
        pcie_command = 0x2037
        pcie_dev_cap = 0x10008cc2
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62011
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x40
        pcie_ce_mask = 0x0
        remainder = 0x1
        severity = 0x3
        __ttl = 0x1
        __tod = 0x4a550326 0x60327c0

Jul 08 2009 14:35:50.100862233 ereport.io.pciex.dl.btlp
nvlist version: 0
        ena = 0x85ab140aa502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1/pci15d9,1...@0
        (end detector)

        class = ereport.io.pciex.dl.btlp
        dev-status = 0x11
        ce-status = 0x40
        __ttl = 0x1
        __tod = 0x4a550326 0x6030919

Jul 08 2009 14:35:50.100870080 ereport.io.pciex.dl.btlp
nvlist version: 0
        ena = 0x85ab1429b102801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1/pci15d9,1...@0,1
        (end detector)

        class = ereport.io.pciex.dl.btlp
        dev-status = 0x11
        ce-status = 0x40
        __ttl = 0x1
        __tod = 0x4a550326 0x60327c0

Jul 08 2009 14:35:50.623112120 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x85ca34f2f702801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550326 0x2523efb8

Jul 08 2009 14:35:50.623112120 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x85ca34f2f702801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550326 0x2523efb8

Jul 08 2009 14:35:50.623112120 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x85ca34f2f702801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550326 0x2523efb8

Jul 08 2009 14:35:50.687085278 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x85ce0518a502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550326 0x28f416de

Jul 08 2009 14:35:50.687085278 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x85ce0518a502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550326 0x28f416de

Jul 08 2009 14:35:50.687085278 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x85ce0518a502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550326 0x28f416de

Jul 08 2009 14:35:50.710271342 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x85cf66e4d502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550326 0x2a55e16e

Jul 08 2009 14:35:50.713024685 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x85cf90e90e02801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550326 0x2a7fe4ad

Jul 08 2009 14:35:50.713064080 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x85cf9182a902801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550326 0x2a807e90

Jul 08 2009 14:35:50.710271342 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x85cf66e4d502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550326 0x2a55e16e

Jul 08 2009 14:35:50.710271342 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x85cf66e4d502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550326 0x2a55e16e

Jul 08 2009 14:35:50.713024685 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x85cf90e90e02801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550326 0x2a7fe4ad

Jul 08 2009 14:35:50.713024685 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x85cf90e90e02801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550326 0x2a7fe4ad

Jul 08 2009 14:35:50.713064080 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x85cf9182a902801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550326 0x2a807e90

Jul 08 2009 14:35:50.713064080 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x85cf9182a902801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550326 0x2a807e90

Jul 08 2009 14:35:50.894772210 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x85da66277102801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550326 0x355523f2

Jul 08 2009 14:35:50.894772210 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x85da66277102801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550326 0x355523f2

Jul 08 2009 14:35:50.894772210 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x85da66277102801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550326 0x355523f2

Jul 08 2009 14:35:52.015800752 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x861d37af7302801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550328 0xf119b0

Jul 08 2009 14:35:52.015800752 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x861d37af7302801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550328 0xf119b0

Jul 08 2009 14:35:52.015800752 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x861d37af7302801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550328 0xf119b0

Jul 08 2009 14:35:52.310175553 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x862ec37ce602801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550328 0x127ce741

Jul 08 2009 14:35:52.310175553 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x862ec37ce602801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550328 0x127ce741

Jul 08 2009 14:35:52.310175553 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x862ec37ce602801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550328 0x127ce741

Jul 08 2009 14:35:52.324994023 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x862fa5985902801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550328 0x135f03e7

Jul 08 2009 14:35:52.324994023 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x862fa5985902801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550328 0x135f03e7

Jul 08 2009 14:35:52.324994023 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x862fa5985902801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550328 0x135f03e7

Jul 08 2009 14:35:52.531102535 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x863bee910502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550328 0x1fa7fb47

Jul 08 2009 14:35:52.531102535 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x863bee910502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550328 0x1fa7fb47

Jul 08 2009 14:35:52.531102535 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x863bee910502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550328 0x1fa7fb47

Jul 08 2009 14:35:52.570487166 ereport.io.pci.fabric
nvlist version: 0
        class = ereport.io.pci.fabric
        ena = 0x863e47866502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        bdf = 0x8
        device_id = 0x3408
        vendor_id = 0x8086
        rev_id = 0x13
        dev_type = 0x40
        pcie_off = 0x90
        pcix_off = 0x0
        aer_off = 0x100
        ecc_ver = 0x0
        pci_status = 0x10
        pci_command = 0x47
        pci_bdg_sec_status = 0x0
        pci_bdg_ctrl = 0x3
        pcie_status = 0x1
        pcie_command = 0x27
        pcie_dev_cap = 0x8021
        pcie_adv_ctl = 0x0
        pcie_ue_status = 0x0
        pcie_ue_mask = 0x100000
        pcie_ue_sev = 0x62030
        pcie_ue_hdr0 = 0x0
        pcie_ue_hdr1 = 0x0
        pcie_ue_hdr2 = 0x0
        pcie_ue_hdr3 = 0x0
        pcie_ce_status = 0x1000
        pcie_ce_mask = 0x0
        pcie_rp_status = 0x0
        pcie_rp_control = 0x0
        pcie_adv_rp_status = 0x1
        pcie_adv_rp_command = 0x7
        pcie_adv_rp_ce_src_id = 0x8
        pcie_adv_rp_ue_src_id = 0x0
        remainder = 0x1
        severity = 0x1
        __ttl = 0x1
        __tod = 0x4a550328 0x2200f17e

Jul 08 2009 14:35:52.570487166 ereport.io.pciex.dl.rto
nvlist version: 0
        ena = 0x863e47866502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.dl.rto
        dev-status = 0x1
        ce-status = 0x1000
        __ttl = 0x1
        __tod = 0x4a550328 0x2200f17e

Jul 08 2009 14:35:52.570487166 ereport.io.pciex.rc.ce-msg
nvlist version: 0
        ena = 0x863e47866502801
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /p...@0,0/pci8086,3...@1
        (end detector)

        class = ereport.io.pciex.rc.ce-msg
        rc-status = 0x1
        source-id = 0x8
        source-valid = 1
        __ttl = 0x1
        __tod = 0x4a550328 0x2200f17e

_______________________________________________
fm-discuss mailing list
fm-discuss@opensolaris.org

Reply via email to