Re: Unrecognized Inifiniband HCA

2018-05-11 Thread Grzegorz Junka


On 10/05/2018 15:35, Justin Clift wrote:

On 2018-05-10 10:21, Grzegorz Junka wrote:

On 08/05/2018 23:41, Justin Clift wrote:


That's probably a ConnectX (series 1) Mellanox card.  Those can 
operate in

either Infiniband mode, or Ethernet mode.

Which mode are you wanting it to run in? :)

As a thought, the FreeBSD wiki page has a bit of info:

  https://wiki.freebsd.org/InfiniBand

For that card to be recognised at all, it'll need the mlx4 driver(s) 
to load.


I don't remember the exact one off hand (it's been a while), but 
some searching

online for mlx4 and FreeBSD should turn up the right bits.


Many thanks Justin. This is the first time I am hearing about an
Infiniband card operating in Ethernet mode. These cards come with two
CX4/SFF 8470 ports. It's not possible to connect standard Ethernet
cables that I know of (not even SFP modules). Do you mean that they
can operate in Ethernet mode over the CX4 cable?


Yep. :)

Back in the day when these cards were current tech, CX4 was an acceptable
connector for 10GbE.  The Infiniband switches from that era (that I had
access to) were mostly Infiniband only though.

But 10GbE CX4 switches did exist, and can still be found reasonably 
cheaply

on Ebay.  eg:

  * HP ProCurve 6 port CX4 10GBe switch
    https://www.ebay.com/itm/152232294328

  * HP ProCurve 48 port 1GbE switch, with 2x 10GbE CX4 ports on the back
    https://www.ebay.com/itm/281899832599

One of the good things about those HP ProCurve switches... being 
enterprise

gear they just keep working.  For Years.

From memory, HP still releases firmware security updates for them (for
free) to this day. Unlike (say) Cisco. ;)

Note - As a data point, FreeNAS (based on FreeBSD) includes the 
10/40GbE driver

for these cards by default.  With FreeNAS they work as 10GbE "out of the
box". :D

Hopefully that helps. :)


Thanks Justin. That's an amazing piece of tech history :)

GrzegorzJ

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Unrecognized Inifiniband HCA

2018-05-10 Thread Justin Clift

On 2018-05-10 10:21, Grzegorz Junka wrote:

On 08/05/2018 23:41, Justin Clift wrote:


That's probably a ConnectX (series 1) Mellanox card.  Those can 
operate in

either Infiniband mode, or Ethernet mode.

Which mode are you wanting it to run in? :)

As a thought, the FreeBSD wiki page has a bit of info:

  https://wiki.freebsd.org/InfiniBand

For that card to be recognised at all, it'll need the mlx4 driver(s) 
to load.


I don't remember the exact one off hand (it's been a while), but some 
searching

online for mlx4 and FreeBSD should turn up the right bits.


Many thanks Justin. This is the first time I am hearing about an
Infiniband card operating in Ethernet mode. These cards come with two
CX4/SFF 8470 ports. It's not possible to connect standard Ethernet
cables that I know of (not even SFP modules). Do you mean that they
can operate in Ethernet mode over the CX4 cable?


Yep. :)

Back in the day when these cards were current tech, CX4 was an 
acceptable

connector for 10GbE.  The Infiniband switches from that era (that I had
access to) were mostly Infiniband only though.

But 10GbE CX4 switches did exist, and can still be found reasonably 
cheaply

on Ebay.  eg:

  * HP ProCurve 6 port CX4 10GBe switch
https://www.ebay.com/itm/152232294328

  * HP ProCurve 48 port 1GbE switch, with 2x 10GbE CX4 ports on the back
https://www.ebay.com/itm/281899832599

One of the good things about those HP ProCurve switches... being 
enterprise

gear they just keep working.  For Years.

From memory, HP still releases firmware security updates for them (for
free) to this day. Unlike (say) Cisco. ;)

Note - As a data point, FreeNAS (based on FreeBSD) includes the 10/40GbE 
driver

for these cards by default.  With FreeNAS they work as 10GbE "out of the
box". :D

Hopefully that helps. :)

+ Justin
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Unrecognized Inifiniband HCA

2018-05-10 Thread Gary Palmer
On Thu, May 10, 2018 at 09:21:10AM +, Grzegorz Junka wrote:
> 
> On 08/05/2018 23:41, Justin Clift wrote:
> > On 2018-05-08 21:59, Grzegorz Junka wrote:
> >> Hi All,
> >>
> >> pciconf -lv
> >>
> >> gives me
> >>
> >> none1@pci0:3:0:0: class=0x0c0600 card=0x000315b3 
> >> chip=0x634015b3
> >> rev=0xa0 hdr=0x00
> >> ?? vendor = 'Mellanox Technologies'
> >> ?? device = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'
> >> ?? class?? = serial bus
> >>
> >> Does it mean that my card is unrecognized? It supposed to be 10GB x 2
> >> Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware.
> >> Currently the card doesn't show when doing ifconfig. What should I do
> >> to have the proper device name instead of none1 or for the card to
> >> appear in ifconfig?
> >>
> >> # kldstat
> >> Id Refs Address?? Size Name
> >> ??1 28 0x8020 1f67a88?? kernel
> >> ??2?? 1 0x82169000 316708 zfs.ko
> >> ??3?? 2 0x8248 cb78 opensolaris.ko
> >> ??4?? 1 0x8248d000 42c28?? mps.ko
> >> ??5?? 1 0x82621000 bdb0 if_lagg.ko
> >> ??6?? 1 0x8262d000 3650 ums.ko
> >> ??7?? 1 0x82631000 6679 nullfs.ko
> >> ??8?? 1 0x82638000 bdfe unionfs.ko
> >> ??9?? 2 0x82644000 2094f?? mlx5.ko
> >> 10?? 2 0x82665000 103e1?? linuxkpi.ko
> >> 11?? 1 0x82676000 15965?? mlx5en.ko
> >
> > That's probably a ConnectX (series 1) Mellanox card.?? Those can 
> > operate in
> > either Infiniband mode, or Ethernet mode.
> >
> > Which mode are you wanting it to run in? :)
> >
> > As a thought, the FreeBSD wiki page has a bit of info:
> >
> > ?? https://wiki.freebsd.org/InfiniBand
> >
> > For that card to be recognised at all, it'll need the mlx4 driver(s) 
> > to load.
> >
> > I don't remember the exact one off hand (it's been a while), but some 
> > searching
> > online for mlx4 and FreeBSD should turn up the right bits.
> 
> Many thanks Justin. This is the first time I am hearing about an 
> Infiniband card operating in Ethernet mode. These cards come with two 
> CX4/SFF 8470 ports. It's not possible to connect standard Ethernet 
> cables that I know of (not even SFP modules). Do you mean that they can 
> operate in Ethernet mode over the CX4 cable?

You can get CX4 to SFP+ cables, so I would assume so

Regards

Gary

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Missing mlx4, was Re: Unrecognized Inifiniband HCA

2018-05-10 Thread Gary Palmer
On Thu, May 10, 2018 at 08:36:44AM +, Grzegorz Junka wrote:
> 
> On 09/05/2018 00:39, Rodney W. Grimes wrote:
> >> On Tue, May 08, 2018 at 08:59:20PM +, Grzegorz Junka wrote:
> >>> Hi All,
> >>>
> >>> pciconf -lv
> >>>
> >>> gives me
> >>>
> >>> none1@pci0:3:0:0: class=0x0c0600 card=0x000315b3 
> >>> chip=0x634015b3
> >>> rev=0xa0 hdr=0x00
> >>>   ?? vendor = 'Mellanox Technologies'
> >>>   ?? device = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'
> >>>   ?? class?? = serial bus
> >>>
> >>> Does it mean that my card is unrecognized? It supposed to be 10GB x 2
> >>> Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently
> >>> the card doesn't show when doing ifconfig. What should I do to have the
> >>> proper device name instead of none1 or for the card to appear in ifconfig?
> >>>
> >>> # kldstat
> >>> Id Refs Address?? Size Name
> >>>   ??1 28 0x8020 1f67a88?? kernel
> >>>   ??2?? 1 0x82169000 316708 zfs.ko
> >>>   ??3?? 2 0x8248 cb78 opensolaris.ko
> >>>   ??4?? 1 0x8248d000 42c28?? mps.ko
> >>>   ??5?? 1 0x82621000 bdb0 if_lagg.ko
> >>>   ??6?? 1 0x8262d000 3650 ums.ko
> >>>   ??7?? 1 0x82631000 6679 nullfs.ko
> >>>   ??8?? 1 0x82638000 bdfe unionfs.ko
> >>>   ??9?? 2 0x82644000 2094f?? mlx5.ko
> >>> 10?? 2 0x82665000 103e1?? linuxkpi.ko
> >>> 11?? 1 0x82676000 15965?? mlx5en.ko
> >> mlx5en is for ConnectX-4.  I think that's an older card.  Try mlx4en,
> >> which supports ConnectX-2 and ConnectX-3.
> >  From a quick grep this infact should be supported by the mlx4/mlx5en
> > drivers:
> > net/mlx4/main.c:/* MT25408 "Hermon" SDR */
> > net/mlx4/main.c:/* MT25408 "Hermon" DDR */
> > net/mlx4/main.c:/* MT25408 "Hermon" QDR */
> > net/mlx4/main.c:/* MT25408 "Hermon" DDR PCIe gen2 */
> > net/mlx4/main.c:/* MT25408 "Hermon" QDR PCIe gen2 */
> > net/mlx4/main.c:/* MT25408 "Hermon" EN 10GigE */
> > net/mlx4/main.c:/* MT25408 "Hermon" EN 10GigE PCIe gen2 */
> >
> 
> Thank you Gary and Rod for your quick reply. I don't seem to have mlx4 
> in my /boot/kernel. I unloaded mlx5 and mlx5en and tried to load mlx 
> instead, but strangely, it says mlx is already loaded. How?
> 
> # kldload mlx
> kldload: can't load mlx: module already loaded or in kernel
> 
> # kldstat
> Id Refs Address?? Size Name
>  ??1 22 0x8020 1f67a88?? kernel
>  ??2?? 1 0x82169000 316708 zfs.ko
>  ??3?? 2 0x8248 cb78 opensolaris.ko
>  ??4?? 1 0x8248d000 42c28?? mps.ko
>  ??5?? 1 0x82621000 bdb0 if_lagg.ko
>  ??6?? 1 0x8262d000 3650 ums.ko
>  ??7?? 1 0x82631000 6679 nullfs.ko
>  ??8?? 1 0x82638000 bdfe unionfs.ko
> 
> # ls -l /boot/kernel/mlx
> mlx.ko*?? mlx5.ko* mlx5en.ko*
> 
> Anyways, mlx seems to be rather unrelated to this problem: 
> https://www.freebsd.org/cgi/man.cgi?query=mlx
> 
> Looks like mlx4 is available in the kernel: 
> https://github.com/freebsd/freebsd/tree/master/sys/modules/mlx4
> 
> However, it seems that it may not compile: 
> https://forums.freebsd.org/threads/mellanox-mt26448.64350/
> 
> Does it mean, that mlx4 is no longer compiled in the default kernel and 
> I would need to compile the kernel manually? Can I just compile the 
> mlx4/en/ib kernel module without having to compile the whole kernel?

You either need to put "options OFED" into your kernel and recompile
to get the mlx4 module, or go to /sys/modules/mlx4 and run
"make all install"

(more details on the wiki that someone else posted earlier)

Regards,

Gary
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Unrecognized Inifiniband HCA

2018-05-10 Thread Grzegorz Junka


On 08/05/2018 23:41, Justin Clift wrote:

On 2018-05-08 21:59, Grzegorz Junka wrote:

Hi All,

pciconf -lv

gives me

none1@pci0:3:0:0:   class=0x0c0600 card=0x000315b3 chip=0x634015b3
rev=0xa0 hdr=0x00
    vendor = 'Mellanox Technologies'
    device = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'
    class  = serial bus

Does it mean that my card is unrecognized? It supposed to be 10GB x 2
Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware.
Currently the card doesn't show when doing ifconfig. What should I do
to have the proper device name instead of none1 or for the card to
appear in ifconfig?

# kldstat
Id Refs Address    Size Name
 1   28 0x8020 1f67a88  kernel
 2    1 0x82169000 316708   zfs.ko
 3    2 0x8248 cb78 opensolaris.ko
 4    1 0x8248d000 42c28    mps.ko
 5    1 0x82621000 bdb0 if_lagg.ko
 6    1 0x8262d000 3650 ums.ko
 7    1 0x82631000 6679 nullfs.ko
 8    1 0x82638000 bdfe unionfs.ko
 9    2 0x82644000 2094f    mlx5.ko
10    2 0x82665000 103e1    linuxkpi.ko
11    1 0x82676000 15965    mlx5en.ko


That's probably a ConnectX (series 1) Mellanox card.  Those can 
operate in

either Infiniband mode, or Ethernet mode.

Which mode are you wanting it to run in? :)

As a thought, the FreeBSD wiki page has a bit of info:

  https://wiki.freebsd.org/InfiniBand

For that card to be recognised at all, it'll need the mlx4 driver(s) 
to load.


I don't remember the exact one off hand (it's been a while), but some 
searching

online for mlx4 and FreeBSD should turn up the right bits.


Many thanks Justin. This is the first time I am hearing about an 
Infiniband card operating in Ethernet mode. These cards come with two 
CX4/SFF 8470 ports. It's not possible to connect standard Ethernet 
cables that I know of (not even SFP modules). Do you mean that they can 
operate in Ethernet mode over the CX4 cable?

GrzegorzJ
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Missing mlx4, was Re: Unrecognized Inifiniband HCA

2018-05-10 Thread Grzegorz Junka


On 10/05/2018 09:06, Ben RUBSON wrote:

On 10 May 2018 10:36, Grzegorz Junka wrote:


Does it mean, that mlx4 is no longer compiled in the default kernel


It has never been, but I think modules have been added recently, I can 
remember HPS did it :)


Right, I meant that they are not being build as modules along with the 
default kernel, unlike mx5/en currently for example (which are available 
with the default FreeBSD installation) :)




Can I just compile the mlx4/en/ib kernel module without having to 
compile the whole kernel?


Yes, here are at least the instructions for FreeBSD 11.0 which could 
help you :

cd /usr/src/sys/modules/mlx4
make
make install
make clean
cd /usr/src/sys/modules/mlxen
make
make install
make clean



That looks easy, will give it a try. Thanks!
GrzegorzJ

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Missing mlx4, was Re: Unrecognized Inifiniband HCA

2018-05-10 Thread Ben RUBSON

On 10 May 2018 10:36, Grzegorz Junka wrote:


Does it mean, that mlx4 is no longer compiled in the default kernel


It has never been, but I think modules have been added recently, I can  
remember HPS did it :)


Can I just compile the mlx4/en/ib kernel module without having to compile  
the whole kernel?


Yes, here are at least the instructions for FreeBSD 11.0 which could help  
you :

cd /usr/src/sys/modules/mlx4
make
make install
make clean
cd /usr/src/sys/modules/mlxen
make
make install
make clean

Ben
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Missing mlx4, was Re: Unrecognized Inifiniband HCA

2018-05-10 Thread Grzegorz Junka


On 09/05/2018 00:39, Rodney W. Grimes wrote:

On Tue, May 08, 2018 at 08:59:20PM +, Grzegorz Junka wrote:

Hi All,

pciconf -lv

gives me

none1@pci0:3:0:0: class=0x0c0600 card=0x000315b3 chip=0x634015b3
rev=0xa0 hdr=0x00
  ?? vendor = 'Mellanox Technologies'
  ?? device = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'
  ?? class?? = serial bus

Does it mean that my card is unrecognized? It supposed to be 10GB x 2
Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently
the card doesn't show when doing ifconfig. What should I do to have the
proper device name instead of none1 or for the card to appear in ifconfig?

# kldstat
Id Refs Address?? Size Name
  ??1 28 0x8020 1f67a88?? kernel
  ??2?? 1 0x82169000 316708 zfs.ko
  ??3?? 2 0x8248 cb78 opensolaris.ko
  ??4?? 1 0x8248d000 42c28?? mps.ko
  ??5?? 1 0x82621000 bdb0 if_lagg.ko
  ??6?? 1 0x8262d000 3650 ums.ko
  ??7?? 1 0x82631000 6679 nullfs.ko
  ??8?? 1 0x82638000 bdfe unionfs.ko
  ??9?? 2 0x82644000 2094f?? mlx5.ko
10?? 2 0x82665000 103e1?? linuxkpi.ko
11?? 1 0x82676000 15965?? mlx5en.ko

mlx5en is for ConnectX-4.  I think that's an older card.  Try mlx4en,
which supports ConnectX-2 and ConnectX-3.

 From a quick grep this infact should be supported by the mlx4/mlx5en
drivers:
net/mlx4/main.c:/* MT25408 "Hermon" SDR */
net/mlx4/main.c:/* MT25408 "Hermon" DDR */
net/mlx4/main.c:/* MT25408 "Hermon" QDR */
net/mlx4/main.c:/* MT25408 "Hermon" DDR PCIe gen2 */
net/mlx4/main.c:/* MT25408 "Hermon" QDR PCIe gen2 */
net/mlx4/main.c:/* MT25408 "Hermon" EN 10GigE */
net/mlx4/main.c:/* MT25408 "Hermon" EN 10GigE PCIe gen2 */



Thank you Gary and Rod for your quick reply. I don't seem to have mlx4 
in my /boot/kernel. I unloaded mlx5 and mlx5en and tried to load mlx 
instead, but strangely, it says mlx is already loaded. How?


# kldload mlx
kldload: can't load mlx: module already loaded or in kernel

# kldstat
Id Refs Address    Size Name
 1   22 0x8020 1f67a88  kernel
 2    1 0x82169000 316708   zfs.ko
 3    2 0x8248 cb78 opensolaris.ko
 4    1 0x8248d000 42c28    mps.ko
 5    1 0x82621000 bdb0 if_lagg.ko
 6    1 0x8262d000 3650 ums.ko
 7    1 0x82631000 6679 nullfs.ko
 8    1 0x82638000 bdfe unionfs.ko

# ls -l /boot/kernel/mlx
mlx.ko*    mlx5.ko*   mlx5en.ko*

Anyways, mlx seems to be rather unrelated to this problem: 
https://www.freebsd.org/cgi/man.cgi?query=mlx


Looks like mlx4 is available in the kernel: 
https://github.com/freebsd/freebsd/tree/master/sys/modules/mlx4


However, it seems that it may not compile: 
https://forums.freebsd.org/threads/mellanox-mt26448.64350/


Does it mean, that mlx4 is no longer compiled in the default kernel and 
I would need to compile the kernel manually? Can I just compile the 
mlx4/en/ib kernel module without having to compile the whole kernel?


Thanks
GrzegorzJ

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Unrecognized Inifiniband HCA

2018-05-08 Thread Rodney W. Grimes
> On Tue, May 08, 2018 at 08:59:20PM +, Grzegorz Junka wrote:
> > Hi All,
> > 
> > pciconf -lv
> > 
> > gives me
> > 
> > none1@pci0:3:0:0: class=0x0c0600 card=0x000315b3 
> > chip=0x634015b3 
> > rev=0xa0 hdr=0x00
> >  ?? vendor = 'Mellanox Technologies'
> >  ?? device = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'
> >  ?? class?? = serial bus
> > 
> > Does it mean that my card is unrecognized? It supposed to be 10GB x 2 
> > Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently 
> > the card doesn't show when doing ifconfig. What should I do to have the 
> > proper device name instead of none1 or for the card to appear in ifconfig?
> > 
> > # kldstat
> > Id Refs Address?? Size Name
> >  ??1 28 0x8020 1f67a88?? kernel
> >  ??2?? 1 0x82169000 316708 zfs.ko
> >  ??3?? 2 0x8248 cb78 opensolaris.ko
> >  ??4?? 1 0x8248d000 42c28?? mps.ko
> >  ??5?? 1 0x82621000 bdb0 if_lagg.ko
> >  ??6?? 1 0x8262d000 3650 ums.ko
> >  ??7?? 1 0x82631000 6679 nullfs.ko
> >  ??8?? 1 0x82638000 bdfe unionfs.ko
> >  ??9?? 2 0x82644000 2094f?? mlx5.ko
> > 10?? 2 0x82665000 103e1?? linuxkpi.ko
> > 11?? 1 0x82676000 15965?? mlx5en.ko
> 
> mlx5en is for ConnectX-4.  I think that's an older card.  Try mlx4en,
> which supports ConnectX-2 and ConnectX-3.

>From a quick grep this infact should be supported by the mlx4/mlx5en
drivers:
net/mlx4/main.c:/* MT25408 "Hermon" SDR */
net/mlx4/main.c:/* MT25408 "Hermon" DDR */
net/mlx4/main.c:/* MT25408 "Hermon" QDR */
net/mlx4/main.c:/* MT25408 "Hermon" DDR PCIe gen2 */
net/mlx4/main.c:/* MT25408 "Hermon" QDR PCIe gen2 */
net/mlx4/main.c:/* MT25408 "Hermon" EN 10GigE */
net/mlx4/main.c:/* MT25408 "Hermon" EN 10GigE PCIe gen2 */

> 
> Regards,
> 
> Gary

-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Unrecognized Inifiniband HCA

2018-05-08 Thread Justin Clift

On 2018-05-08 21:59, Grzegorz Junka wrote:

Hi All,

pciconf -lv

gives me

none1@pci0:3:0:0:   class=0x0c0600 card=0x000315b3 chip=0x634015b3
rev=0xa0 hdr=0x00
    vendor = 'Mellanox Technologies'
    device = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'
    class  = serial bus

Does it mean that my card is unrecognized? It supposed to be 10GB x 2
Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware.
Currently the card doesn't show when doing ifconfig. What should I do
to have the proper device name instead of none1 or for the card to
appear in ifconfig?

# kldstat
Id Refs Address    Size Name
 1   28 0x8020 1f67a88  kernel
 2    1 0x82169000 316708   zfs.ko
 3    2 0x8248 cb78 opensolaris.ko
 4    1 0x8248d000 42c28    mps.ko
 5    1 0x82621000 bdb0 if_lagg.ko
 6    1 0x8262d000 3650 ums.ko
 7    1 0x82631000 6679 nullfs.ko
 8    1 0x82638000 bdfe unionfs.ko
 9    2 0x82644000 2094f    mlx5.ko
10    2 0x82665000 103e1    linuxkpi.ko
11    1 0x82676000 15965    mlx5en.ko


That's probably a ConnectX (series 1) Mellanox card.  Those can operate 
in

either Infiniband mode, or Ethernet mode.

Which mode are you wanting it to run in? :)

As a thought, the FreeBSD wiki page has a bit of info:

  https://wiki.freebsd.org/InfiniBand

For that card to be recognised at all, it'll need the mlx4 driver(s) to 
load.


I don't remember the exact one off hand (it's been a while), but some 
searching

online for mlx4 and FreeBSD should turn up the right bits.

Hope that helps. :)

Regards and best wishes,

Justin Clift

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Re: Unrecognized Inifiniband HCA

2018-05-08 Thread Gary Palmer
On Tue, May 08, 2018 at 08:59:20PM +, Grzegorz Junka wrote:
> Hi All,
> 
> pciconf -lv
> 
> gives me
> 
> none1@pci0:3:0:0: class=0x0c0600 card=0x000315b3 chip=0x634015b3 
> rev=0xa0 hdr=0x00
>  ?? vendor = 'Mellanox Technologies'
>  ?? device = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'
>  ?? class?? = serial bus
> 
> Does it mean that my card is unrecognized? It supposed to be 10GB x 2 
> Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently 
> the card doesn't show when doing ifconfig. What should I do to have the 
> proper device name instead of none1 or for the card to appear in ifconfig?
> 
> # kldstat
> Id Refs Address?? Size Name
>  ??1 28 0x8020 1f67a88?? kernel
>  ??2?? 1 0x82169000 316708 zfs.ko
>  ??3?? 2 0x8248 cb78 opensolaris.ko
>  ??4?? 1 0x8248d000 42c28?? mps.ko
>  ??5?? 1 0x82621000 bdb0 if_lagg.ko
>  ??6?? 1 0x8262d000 3650 ums.ko
>  ??7?? 1 0x82631000 6679 nullfs.ko
>  ??8?? 1 0x82638000 bdfe unionfs.ko
>  ??9?? 2 0x82644000 2094f?? mlx5.ko
> 10?? 2 0x82665000 103e1?? linuxkpi.ko
> 11?? 1 0x82676000 15965?? mlx5en.ko

mlx5en is for ConnectX-4.  I think that's an older card.  Try mlx4en,
which supports ConnectX-2 and ConnectX-3.

Regards,

Gary

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


Unrecognized Inifiniband HCA

2018-05-08 Thread Grzegorz Junka

Hi All,

pciconf -lv

gives me

none1@pci0:3:0:0:   class=0x0c0600 card=0x000315b3 chip=0x634015b3 
rev=0xa0 hdr=0x00

    vendor = 'Mellanox Technologies'
    device = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'
    class  = serial bus

Does it mean that my card is unrecognized? It supposed to be 10GB x 2 
Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently 
the card doesn't show when doing ifconfig. What should I do to have the 
proper device name instead of none1 or for the card to appear in ifconfig?


# kldstat
Id Refs Address    Size Name
 1   28 0x8020 1f67a88  kernel
 2    1 0x82169000 316708   zfs.ko
 3    2 0x8248 cb78 opensolaris.ko
 4    1 0x8248d000 42c28    mps.ko
 5    1 0x82621000 bdb0 if_lagg.ko
 6    1 0x8262d000 3650 ums.ko
 7    1 0x82631000 6679 nullfs.ko
 8    1 0x82638000 bdfe unionfs.ko
 9    2 0x82644000 2094f    mlx5.ko
10    2 0x82665000 103e1    linuxkpi.ko
11    1 0x82676000 15965    mlx5en.ko

Thanks

GrzegorzJ

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"