How can I remove one interface from lagg, without destroying all lagg?
Hi. I have lagg interface created on my server: [root@h-qa-094 ~]$ ifconfig lagg0 lagg0: flags=8802BROADCAST,SIMPLEX,MULTICAST metric 0 mtu 1500 options=401bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO ether 00:02:c9:19:82:80 nd6 options=21PERFORMNUD,AUTO_LINKLOCAL media: Ethernet autoselect status: active laggproto failover lagghash l2,l3,l4 laggport: igb1 flags=0 laggport: mlxen1 flags=0 laggport: mlxen0 flags=5MASTER,ACTIVE Now, I want to removr igb1 interface from that lag. How can I do it? Regards, Alex Liptsin Software Quality Assurance Engineer | Mellanox Technologies Ltd. Office: +972 (74) 7236141 Mobile: +972(54) 7833986 Fax: +972(74) 7236161 Email: al...@mellanox.commailto:al...@mellanox.com Mellanox, Tel-Hai Industrial Park. Building 7, M.P. Upper Galilee 12100 Israel ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
FreeBSD:: How to set VLAN priority?
Hello. I work with FreeBSD 9.1 RELEASE. I had configured VLANs on my server, but I can't find a way to configure VLAN priority. How can I do it? Thanks. Regards, Alex Liptsin Software Quality Assurance Engineer | Mellanox Technologies Ltd. Office: +972 (74) 7236141 Mobile: +972(54) 7833986 Fax: +972(74) 7236161 Email: al...@mellanox.commailto:al...@mellanox.com Mellanox, Tel-Hai Industrial Park. Building 7, M.P. Upper Galilee 12100 Israel ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Failed to allocate receive buffer problem
Hi. I have a problem that when running a ping (or any other traffic) over IPoIB port, Traffic fails after some time. At destination server DMESG I see that errors: Jun 11 14:42:11 h-qa-033 kernel: ib1: failed to allocate receive buffer 253 Jun 11 14:42:12 h-qa-033 kernel: ib1: failed to allocate receive buffer 254 Jun 11 14:42:13 h-qa-033 kernel: ib1: failed to allocate receive buffer 255 Jun 11 14:42:14 h-qa-033 kernel: ib1: failed to allocate receive buffer 0 Jun 11 14:42:15 h-qa-033 kernel: ib1: failed to allocate receive buffer 1 Jun 11 14:42:16 h-qa-033 kernel: ib1: failed to allocate receive buffer 2 Jun 11 14:42:17 h-qa-033 kernel: ib1: failed to allocate receive buffer 3 Jun 11 14:42:18 h-qa-033 kernel: ib1: failed to allocate receive buffer 4 Jun 11 14:42:19 h-qa-033 kernel: ib1: failed to allocate receive buffer 5 Jun 11 14:42:20 h-qa-033 kernel: ib1: failed to allocate receive buffer 6 Jun 11 14:42:21 h-qa-033 kernel: ib1: failed to allocate receive buffer 7 I work with FreeBSD 9.1. Is it a bug or some configuration issues? Thanks. Regards, Alex Liptsin Software Quality Assurance Engineer | Mellanox Technologies Ltd. Office: +972 (74) 7236141 Mobile: +972(54) 7833986 Fax: +972(74) 7236161 Email: al...@mellanox.commailto:al...@mellanox.com Mellanox, Tel-Hai Industrial Park. Building 7, M.P. Upper Galilee 12100 Israel ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: How to switch Datgram/Connected mtu modes?
Hi. Yes. There is no such entry. The only way I found is to compile inside the kernel options IPOIB_CM . Can I do it manually without compiling the kernel each time I want to switch between the modes? Maybe add it somehow to sysctl or loader.conf? Regards, Alex Liptsin Software Quality Assurance Engineer | Mellanox Technologies Ltd. Office: +972 (74) 7236141 Mobile: +972(54) 7833986 Fax: +972(74) 7236161 Email: al...@mellanox.com Mellanox, Tel-Hai Industrial Park. Building 7, M.P. Upper Galilee 12100 Israel -Original Message- From: John Baldwin [mailto:j...@freebsd.org] Sent: Wednesday, May 29, 2013 9:17 PM To: freebsd-...@freebsd.org Cc: Alex Liptsin; freebsd-questions@freebsd.org Subject: Re: How to switch Datgram/Connected mtu modes? On Sunday, May 26, 2013 7:43:29 am Alex Liptsin wrote: Hello. I work with FreeBSD 9.1 and Mellanox devices. How can I configure MTU in connected mode on FreeBSD 9.1? In Linux to enable connected mode for interface ib0, I enter: echo connected /sys/class/net/ib0/mode Switching between CM and UD mode can be done in run time: echo datagram /sys/class/net/ib0/mode sets the mode of ib0 to UD echo connected /sys/class/net/ib0/mode sets the mode ib0 to CM There is no such directories at FreeBSD. Wat shall I do? Have you tried looking for dev.ib.0 sysctls? It looks like the OFED bits in FreeBSD map Linux sysfs entries to sysctl nodes, but I don't have a box with IB handy to see what it looks like at runtime. -- John Baldwin ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Mellanox NIC names changed, each kldunload/kldload mlx4ib module
Hi. I work with FreeBSD9.1 and Mellanox devices. Every time I unload / load mlx4ib module, NIC names of mellanox devices (ibX) are renamed. Can I prevent it? [root@h-qa-032 mlx4]# ifconfig ib8: flags=8002BROADCAST,MULTICAST metric 0 mtu 65520 options=80018VLAN_MTU,VLAN_HWTAGGING,LINKSTATE lladdr 80.28.0.48.fe.80.0.0.0.0.0.0.0.2.c9.3.0.2e.48.31 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL ib9: flags=8002BROADCAST,MULTICAST metric 0 mtu 65520 options=80018VLAN_MTU,VLAN_HWTAGGING,LINKSTATE lladdr 80.28.0.49.fe.80.0.0.0.0.0.0.0.2.c9.3.0.2e.48.32 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL [root@h-qa-032 mlx4]# kldunload mlx4ib [root@h-qa-032 mlx4]# kldload -v mlx4ib Loaded mlx4ib, id=9 [root@h-qa-032 mlx4]# ifconfig ib10: flags=8002BROADCAST,MULTICAST metric 0 mtu 65520 options=80018VLAN_MTU,VLAN_HWTAGGING,LINKSTATE lladdr 80.30.0.48.fe.80.0.0.0.0.0.0.0.2.c9.3.0.2e.48.31 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL ib11: flags=8002BROADCAST,MULTICAST metric 0 mtu 65520 options=80018VLAN_MTU,VLAN_HWTAGGING,LINKSTATE lladdr 80.30.0.49.fe.80.0.0.0.0.0.0.0.2.c9.3.0.2e.48.32 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL Regards, Alex Liptsin Software Quality Assurance Engineer | Mellanox Technologies Ltd. Office: +972 (74) 7236141 Mobile: +972(54) 7833986 Fax: +972(74) 7236161 Email: al...@mellanox.commailto:al...@mellanox.com Mellanox, Tel-Hai Industrial Park. Building 7, M.P. Upper Galilee 12100 Israel ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: How to compile ipoib module manually?
Thanks a lot. Alex L. Sent from my iPhone On 5 ביונ 2013, at 18:13, John Baldwin j...@freebsd.orgmailto:j...@freebsd.org wrote: On Tuesday, June 04, 2013 5:18:46 am Alex Liptsin wrote: I commented on that lines, because I want to compile and load that modules manually. I had succeed to compile and load mlx4, mlx4ib and mlxen from /sys/modules: [root@h-qa-033 mlxen]# kldstat Id Refs AddressSize Name 1 14 0x8020 13acbd8 kernel 21 0x81612000 21e5 if_mos.ko 33 0x81615000 124ebmlx4.ko 41 0x81628000 e225 mlx4ib.ko 51 0x81637000 ec60 mlxen.ko The problem is that IPOIB module is missing in /sys/modules. 1. Where can I find it? 2. How can I compile ipoib support? You will have to create one. You should be able to use the existing module Makefiles as a guide. -- John Baldwin ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
How to compile ipoib module manually?
run # Ralink Technology RT2700/RT2800/RT3000 NICs. device uath # Atheros AR5523 wireless NICs device upgt # Conexant/Intersil PrismGT wireless NICs. device ural # Ralink Technology RT2500USB wireless NICs device urtw # Realtek RTL8187B/L wireless NICs device zyd # ZyDAS zd1211/zd1211b wireless NICs # FireWire support device firewire # FireWire bus code # sbp(4) works for some systems but causes boot failure on others #devicesbp # SCSI over FireWire (Requires scbus and da) device fwe # Ethernet over FireWire (non-standard!) device fwip # IP over FireWire (RFC 2734,3146) device dcons # Dumb console driver device dcons_crom# Configuration ROM for dcons # Sound support device sound # Generic sound driver (required) device snd_cmi # CMedia CMI8338/CMI8738 device snd_csa # Crystal Semiconductor CS461x/428x device snd_emu10kx # Creative SoundBlaster Live! and Audigy device snd_es137x # Ensoniq AudioPCI ES137x device snd_hda # Intel High Definition Audio device snd_ich# Intel, NVidia and other ICH AC'97 Audio device snd_uaudio # USB Audio device snd_via8233# VIA VT8233x Audio # Mellanox and IB support options OFED # Infiniband protocol options OFED_DEBUG_INIT options SDP # Sockets Direct Protocol for infiniband options SDP_DEBUG # options IPOIB_DEBUG # options IPOIB_CM # Use connect mode ipoib device mthca # Infinihost cards # device mlx4ib # ConnectX Infiniband support #device mlxen # ConnectX Ethernet support # device ipoib # IP over IB devices I commented on that lines, because I want to compile and load that modules manually. I had succeed to compile and load mlx4, mlx4ib and mlxen from /sys/modules: [root@h-qa-033 mlxen]# kldstat Id Refs AddressSize Name 1 14 0x8020 13acbd8 kernel 21 0x81612000 21e5 if_mos.ko 33 0x81615000 124ebmlx4.ko 41 0x81628000 e225 mlx4ib.ko 51 0x81637000 ec60 mlxen.ko The problem is that IPOIB module is missing in /sys/modules. 1. Where can I find it? 2. How can I compile ipoib support? Regards, Alex Liptsin Software Quality Assurance Engineer | Mellanox Technologies Ltd. Office: +972 (74) 7236141 Mobile: +972(54) 7833986 Fax: +972(74) 7236161 Email: al...@mellanox.commailto:al...@mellanox.com Mellanox, Tel-Hai Industrial Park. Building 7, M.P. Upper Galilee 12100 Israel ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
How can I unload/load modules that complied inside the kernel?
Hello. I am using FreeBSD9.1 [root@h-qa-033 ~]# uname -a FreeBSD h-qa-033 9.1-RELEASE FreeBSD 9.1-RELEASE #0: Tue May 28 11:26:45 IDT 2013 root@h-qa-033:/usr/obj/lab/odeds/freebsd/9.1.0/sys/MYKERNEL amd64 OFED and IB support are compiled in kernel. 1. How can I unload/load modules that complied inside the kernel? [root@h-qa-033 ~]# kldstat -v | grep mlx4 -B 5 Id Refs AddressSize Name 1 10 0x8020 13dcbf8 kernel (/boot/kernel/kernel) Contains modules: Id Name 420 mlxen 418 mlx4ib 419 mlx4 I want to unload/load mlx4ib. 2. Is there any way to take it out of kernel and load manually? Like if_lagg for example: [root@h-qa-033 ~]# kldstat Id Refs AddressSize Name 1 10 0x8020 13dcbf8 kernel 31 0x81812000 2197 if_mos.ko 41 0x81815000 690a if_lagg.ko Thanks a lot. Alex. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Create bond on Infiniband ports
Hi. I use FreeBSD 9.1 with OFED compiled on it. There is a Mellanox adapter: [root@qa-h-vrt-030-006 ~]# pciconf -lv |grep mlx4 -A 3 mlx4_core0@pci0:0:5:0: class=0x028000 card=0x005015b3 chip=0x100315b3 rev=0x00 hdr=0x00 vendor = 'Mellanox Technologies' device = 'MT27500 Family [ConnectX-3]' class = network I want to create Bond on the two ports (ib0 and ib1) of this device: [root@qa-h-vrt-030-006 ~]# ifconfig em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=209bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC ether 00:50:56:23:1e:06 inet6 fe80::250:56ff:fe23:1e06%em0 prefixlen 64 scopeid 0x2 inet 10.195.30.6 netmask 0x broadcast 10.195.255.255 nd6 options=23PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL media: Ethernet autoselect (1000baseT full-duplex) status: active lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384 options=63RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff00 nd6 options=23PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL ib0: flags=8043UP,BROADCAST,RUNNING,MULTICAST metric 0 mtu 65520 options=80018VLAN_MTU,VLAN_HWTAGGING,LINKSTATE lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.51 inet 11.195.30.1 netmask 0x broadcast 11.195.255.255 inet6 fe80::250:56ff:fe23:1e06%ib0 prefixlen 64 scopeid 0x4 nd6 options=23PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL ib1: flags=8043UP,BROADCAST,RUNNING,MULTICAST metric 0 mtu 65520 options=80018VLAN_MTU,VLAN_HWTAGGING,LINKSTATE lladdr 80.0.0.49.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.52 inet 12.195.30.1 netmask 0x broadcast 12.195.255.255 inet6 fe80::250:56ff:fe23:1e06%ib1 prefixlen 64 scopeid 0x5 nd6 options=23PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL 1. Is there bond in FreeBSD or its Lagg? 2. I tried to create this way, but failed: [root@qa-h-vrt-030-006 ~]# ifconfig lagg0 create root@qa-h-vrt-031-005 conf]# ifconfig lagg0 laggproto failover laggport ib0 laggport ib1 ifconfig: SIOCSLAGGPORT: Protocol not supported Any ideas? Is it supported on Infiniband ports? Regards, Alex Liptsin ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
How to switch Datgram/Connected mtu modes?
Hello. I work with FreeBSD 9.1 and Mellanox devices. How can I configure MTU in connected mode on FreeBSD 9.1? In Linux to enable connected mode for interface ib0, I enter: echo connected /sys/class/net/ib0/mode Switching between CM and UD mode can be done in run time: echo datagram /sys/class/net/ib0/mode sets the mode of ib0 to UD echo connected /sys/class/net/ib0/mode sets the mode ib0 to CM There is no such directories at FreeBSD. Wat shall I do? Datagram vs Connected modes The IPoIB driver supports two modes of operation: datagram and connected. The mode is set and read through an interface's /sys/class/net/intf name/mode file. In datagram mode, the IB UD (Unreliable Datagram) transport is used and so the interface MTU has is equal to the IB L2 MTU minus the IPoIB encapsulation header (4 bytes). For example, in a typical IB fabric with a 2K MTU, the IPoIB MTU will be 2048 - 4 = 2044 bytes. In connected mode, the IB RC (Reliable Connected) transport is used. Connected mode takes advantage of the connected nature of the IB transport and allows an MTU up to the maximal IP packet size of 64K, which reduces the number of IP packets needed for handling large UDP datagrams, TCP segments, etc and increases the performance for large messages. In connected mode, the interface's UD QP is still used for multicast and communication with peers that don't support connected mode. In this case, RX emulation of ICMP PMTU packets is used to cause the networking stack to use the smaller UD MTU for these neighbours. Thanks a lot Regards, Alex Liptsin Office: +972 (74) 7236141 Mobile: +972(54) 7833986 Fax: +972(74) 7236161 Email: al...@mellanox.commailto:al...@mellanox.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
