RE: [openib-general] could not add HCA InfiniHost0

QiWang, Chen Sun, 18 Sep 2005 08:48:28 -0700

lspci -vv : ( 02:00.0 works fine)
02:00.0 PCI bridge: Mellanox Technologies MT23108 PCI Bridge (rev a1) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B-
        Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, Cache Line Size 08
        Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
        Memory behind bridge: e0000000-efffffff
        Secondary status: 66Mhz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
        Capabilities: [70] PCI-X bridge device.
                Secondary Status: 64bit+, 133MHz+, SCD-, USC-, SCO-, SRD- Freq=3
                Status: Bus=2 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, SCO-, SRD-
                : Upstream: Capacity=512, Commitment Limit=512
                : Downstream: Capacity=128, Commitment Limit=128

03:00.0 InfiniBand: Mellanox Technologies MT23108 InfiniHost (rev a1)
        Subsystem: Mellanox Technologies MT23108 InfiniHost
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
        Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, Cache Line Size 08
        Interrupt: pin A routed to IRQ 193
        Region 0: Memory at e8800000 (64-bit, non-prefetchable) [size=1M]
        Region 2: Memory at e8000000 (64-bit, prefetchable) [size=8M]
        Region 4: Memory at e0000000 (64-bit, prefetchable) [size=128M]
        Capabilities: [40] MSI-X: Enable- Mask- TabSize=32
                Vector table: BAR=0 offset=00082000
                PBA: BAR=0 offset=00082200
        Capabilities: [50] Vital Product Data
        Capabilities: [60] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable-
                Address: 0000000000000000 Data: 0000
        Capabilities: [70] PCI-X non-bridge device.
                Command: DPERE- ERO- RBC=3 OST=1
                Status: Bus=3 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, DC=simple, DMMRBC=3, DMOST=1, DMCRS=0, RSCEM-

on failed nodes: lspci -vv

02:01.0 PCI bridge: Mellanox Technologies MT23108 PCI Bridge (rev a1) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B-
        Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, Cache Line Size 08
        Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
        Memory behind bridge: e0000000-efffffff
        Secondary status: 66Mhz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
        Capabilities: [70] PCI-X bridge device.
                Secondary Status: 64bit+, 133MHz+, SCD-, USC-, SCO-, SRD- Freq=3
                Status: Bus=2 Dev=1 Func=0 64bit+ 133MHz+ SCD- USC-, SCO-, SRD-
                : Upstream: Capacity=512, Commitment Limit=512
                : Downstream: Capacity=128, Commitment Limit=128

03:00.0 InfiniBand: Mellanox Technologies MT23108 InfiniHost (rev a1)
        Subsystem: Mellanox Technologies MT23108 InfiniHost
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
        Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, Cache Line Size 08
        Interrupt: pin A routed to IRQ 201
        Region 0: Memory at e8800000 (64-bit, non-prefetchable) [size=1M]
        Region 2: Memory at e8000000 (64-bit, prefetchable) [size=8M]
        Region 4: Memory at e0000000 (64-bit, prefetchable) [size=128M]
        Capabilities: [40] MSI-X: Enable- Mask- TabSize=32
                Vector table: BAR=0 offset=00082000
                PBA: BAR=0 offset=00082200
        Capabilities: [50] Vital Product Data
        Capabilities: [60] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable-
                Address: 0000000000000000 Data: 0000
        Capabilities: [70] PCI-X non-bridge device.
                Command: DPERE- ERO- RBC=3 OST=1
                Status: Bus=3 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, DC=simple, DMMRBC=3, DMOST=1, DMCRS=0, RSCEM-

On Sun, 2005-09-18 at 11:22 +0300, Guy Shevet wrote:

Hi ,

            Can you please send “lspci –vv “ for all Mellanox devices on the system ( the virtual bridge and the simple device ) .

--- Guy

From: QiWang, Chen [mailto:[EMAIL PROTECTED]
Sent: Sunday, September 18, 2005 9:56 AM
To: Grant Grundler
Cc: [email protected]
Subject: Re: [openib-general] could not add HCA InfiniHost0

Hi, Grant
    I can not change the slot, I only have one slot (BladeServer),

--------------------------------------------------------------------
lspci -vvs 02:00.0 :

02:00.0 PCI bridge: Mellanox Technologies MT23108 PCI Bridge (rev a1)
(prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR+ FastB2B-
        Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, Cache Line Size 08
        Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
        Memory behind bridge: e0000000-efffffff
        Secondary status: 66Mhz+ FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
        Capabilities: [70] PCI-X bridge device.
                Secondary Status: 64bit+, 133MHz+, SCD-, USC-, SCO-,
SRD- Freq=3
                Status: Bus=2 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-,
SCO-, SRD-
                : Upstream: Capacity=512, Commitment Limit=512
                : Downstream: Capacity=128, Commitment Limit=128

---------------------------------------------------------------------------------------------------

lspci -vvs 02:01.0

02:01.0 PCI bridge: Mellanox Technologies MT23108 PCI Bridge (rev a1)
(prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR+ FastB2B-
        Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, Cache Line Size 08
        Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
        Memory behind bridge: e0000000-efffffff
        Secondary status: 66Mhz+ FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
        Capabilities: [70] PCI-X bridge device.
                Secondary Status: 64bit+, 133MHz+, SCD-, USC-, SCO-,
SRD- Freq=3
                Status: Bus=2 Dev=1 Func=0 64bit+ 133MHz+ SCD- USC-,
SCO-, SRD-
                : Upstream: Capacity=512, Commitment Limit=512
                : Downstream: Capacity=128, Commitment Limit=128

-----------------------------------------------------------------------------------------------------

Thx

On Thu, 2005-09-15 at 08:50 -0700, Grant Grundler wrote:
 
On Thu, Sep 15, 2005 at 06:11:26PM +0800, QiWang, Chen wrote:
> there are some diff:
> 02:00.0 --> work
> 02:01.0 --> failed
> 
> and first time I install the ib-verbs on node1, It also failed, because 
> lspci= 02:01.0, an I don not know how i change 02:01.0 to 02:00.0, and 
> it works fine for me.
 
You can only change it by removing the Mellanox card and re-installing
in the other slot.
 
Can you post "lspci -vvs 02:01.0" output from the machine that failed?
Can you post "lspci -vvs 02:00.0" output from the machine that worked?
 
grant
 
-- 
QiWang, Chen <[EMAIL PROTECTED]>
Clustars Supercomputing Technology corp.
http://www.Clustars.CN
TEL:+86-0816-2546345-815
FAX:+86-0816-2546370
Mobile:+86-13096497499

-- 
QiWang, Chen <[EMAIL PROTECTED]>
Clustars Supercomputing Technology corp.
http://www.Clustars.CN
TEL:+86-0816-2546345-815
FAX:+86-0816-2546370
Mobile:+86-13096497499

_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general


To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general

RE: [openib-general] could not add HCA InfiniHost0

Reply via email to