Hi, seems that with a change made in upstream to the rdma package (no more udev rules) the infiniband devices get created by the kernel with the wrong permissions.
[root@brux ~]# rpm -qa | grep rdma librdmacm-utils-1.0.15-2.el6.x86_64 librdmacm-devel-1.0.15-2.el6.x86_64 librdmacm-1.0.15-2.el6.x86_64 rdma-3.3-3.el6.noarch [root@brux ~]# cat /etc/redhat-release Scientific Linux release 6.3 (Carbon) [root@brux ~]# rpm -ql rdma /etc/rc.d/init.d/nfs-rdma /etc/rc.d/init.d/rdma /etc/rdma /etc/rdma/fixup-mtrr.awk /etc/rdma/rdma.conf /etc/sysconfig/network-scripts/ifdown-ib /etc/sysconfig/network-scripts/ifup-ib [root@brux ~]# ll /dev/infiniband/ total 0 crw-rw---- 1 root root 231, 64 Aug 13 15:27 issm0 crw-rw---- 1 root root 231, 65 Aug 13 15:27 issm1 crw-rw---- 1 root root 10, 57 Aug 13 15:27 rdma_cm crw-rw---- 1 root root 231, 224 Aug 13 15:27 ucm0 crw-rw---- 1 root root 231, 0 Aug 13 15:27 umad0 crw-rw---- 1 root root 231, 1 Aug 13 15:27 umad1 crw-rw---- 1 root root 231, 192 Aug 13 15:27 uverbs0 This prevents non-root users from using the infiniband. Installing the mellanox provided OFED on top of SL 6.3 gives the devices with the correct permissions: [root@brux ~]# ll /dev/infiniband/ total 0 crw-rw---- 1 root root 231, 64 Aug 16 10:41 issm0 crw-rw---- 1 root root 231, 65 Aug 16 10:41 issm1 crw-rw-rw- 1 root root 10, 56 Aug 16 10:41 rdma_cm crw-rw-rw- 1 root root 231, 0 Aug 16 10:41 umad0 crw-rw-rw- 1 root root 231, 1 Aug 16 10:41 umad1 crw-rw-rw- 1 root root 231, 192 Aug 16 10:41 uverbs0 See the discussion at: https://www.centos.org/modules/newbb/viewtopic.php?topic_id=38586 Can someone with access to TUV bug archive check if this has been fixed already? Thanks, Francesco.
