Hi,
You were right, ibv_devinfo -v doesn't return something if both card are connected. I didn't checked ibv_* tools, I supposed once IP stack and ibstat OK, the rest should work. I'm stupid 😊 Anyway, once I disconnect one card, ibv_devinfo show me input but with both cards, I don't have any input except "device not found". And what is weird here, it's that it work only when one card are connected, no matter the card (both are similar: model, firmware, revision, company)... Really strange, I will dig more about the issue. Stupid and bad workaround: connected a dual port Infiniband. But production system doesn't wait.. Thank for your help, Frank ________________________________ From: Aaron Knister <[email protected]> Sent: Saturday, June 10, 2017 2:05 PM To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] Infiniband: device mlx4_0 not found Out of curiosity could you send us the output of "ibv_devinfo -v"? -Aaron Sent from my iPhone On Jun 10, 2017, at 06:55, Frank Tower <[email protected]<mailto:[email protected]>> wrote: Hi everybody, I don't get why one of our compute node cannot start GPFS over IB. I have the following error: [I] VERBS RDMA starting with verbsRdmaCm=no verbsRdmaSend=no verbsRdmaUseMultiCqThreads=yes verbsRdmaUseCompVectors=yes [I] VERBS RDMA library libibverbs.so (version >= 1.1) loaded and initialized. [I] VERBS RDMA verbsRdmasPerNode reduced from 1000 to 514 to match (nsdMaxWorkerThreads 512 + (nspdThreadsPerQueue 2 * nspdQueues 1)). [I] VERBS RDMA parse verbsPorts mlx4_0/1 [W] VERBS RDMA parse error verbsPort mlx4_0/1 ignored due to device mlx4_0 not found [I] VERBS RDMA library libibverbs.so unloaded. [E] VERBS RDMA failed to start, no valid verbsPorts defined. I'm using Centos 7.3, Kernel 3.10.0-514.21.1.el7.x86_64. I have 2 infinibands card, both have an IP and working well. [root@rdx110 ~]# ibstat -l mlx4_0 mlx4_1 [root@rdx110 ~]# I tried configuration with both card, and no one work with GPFS. I also tried with mlx4_0/1, but same problem. Someone already have the issue ? Kind Regards, Frank _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<http://spectrumscale.org> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
