Re: missing hw.em.msix support in FreeBSD 14.0-CURRENT?

[Cheng Cui]

Found the RSS support, only if the hardware is ">= e1000_82571" from the
if_em.c.

if (hw->mac.type >= em_mac_min)// #define em_mac_min e1000_82571
> ...
> scctx->isc_txrx = _txrx;
> ...
>
>
Best Regards,
Cheng Cui


On Sun, May 7, 2023 at 7:28 PM Cheng Cui  wrote:

> Hi Yuri,
>
> Thanks for the update. I am using an Emulab testbed node under the same
> type of hardware. Below are the new outputs.
> It is supposed to have multiple tx/rx queues under msix. Why are there
> only rxq0/txq0? Is it because the CPU has only one core? How can I check if
> RSS is supported?
>
> root@s1:~ # sysctl -d dev.em.0.iflib.disable_msix
> dev.em.0.iflib.disable_msix: disable MSI-X (default 0)
> root@s1:~ # sysctl dev.em.0.iflib.disable_msix
> dev.em.0.iflib.disable_msix: 0
> root@s1:~ # sysctl dev.em.0.iflib
> dev.em.0.iflib.rxq0.rxq_fl0.buf_size: 2048
> dev.em.0.iflib.rxq0.rxq_fl0.credits: 1023
> dev.em.0.iflib.rxq0.rxq_fl0.cidx: 959
> dev.em.0.iflib.rxq0.rxq_fl0.pidx: 958
> dev.em.0.iflib.rxq0.cpu: 1
> dev.em.0.iflib.txq0.r_abdications: 0
> dev.em.0.iflib.txq0.r_restarts: 0
> dev.em.0.iflib.txq0.r_stalls: 0
> dev.em.0.iflib.txq0.r_starts: 110443
> dev.em.0.iflib.txq0.r_drops: 0
> dev.em.0.iflib.txq0.r_enqueues: 110444
> dev.em.0.iflib.txq0.ring_state: pidx_head: 1903 pidx_tail: 1903 cidx: 1903
> state: IDLE
> dev.em.0.iflib.txq0.txq_cleaned: 115208
> dev.em.0.iflib.txq0.txq_processed: 115251
> dev.em.0.iflib.txq0.txq_in_use: 42
> dev.em.0.iflib.txq0.txq_cidx_processed: 565
> dev.em.0.iflib.txq0.txq_cidx: 525
> dev.em.0.iflib.txq0.txq_pidx: 568
> dev.em.0.iflib.txq0.no_tx_dma_setup: 0
> dev.em.0.iflib.txq0.txd_encap_efbig: 0
> dev.em.0.iflib.txq0.tx_map_failed: 0
> dev.em.0.iflib.txq0.no_desc_avail: 0
> dev.em.0.iflib.txq0.mbuf_defrag_failed: 0
> dev.em.0.iflib.txq0.m_pullups: 0
> dev.em.0.iflib.txq0.mbuf_defrag: 0
> dev.em.0.iflib.txq0.cpu: 0
> dev.em.0.iflib.override_nrxds: 0
> dev.em.0.iflib.override_ntxds: 0
> dev.em.0.iflib.use_logical_cores: 0
> dev.em.0.iflib.separate_txrx: 0
> dev.em.0.iflib.core_offset: 0
> dev.em.0.iflib.tx_abdicate: 0
> dev.em.0.iflib.rx_budget: 0
> dev.em.0.iflib.disable_msix: 0
> dev.em.0.iflib.override_qs_enable: 0
> dev.em.0.iflib.override_nrxqs: 0
> dev.em.0.iflib.override_ntxqs: 0
> dev.em.0.iflib.driver_version: 7.7.8-fbsd
> root@s1:~ #
>
>
> Best Regards,
> Cheng Cui
>
>
> On Sun, May 7, 2023 at 4:18 PM Yuri  wrote:
>
>> Cheng Cui wrote:
>> > Hello,
>> >
>> > I am using this em(4) driver for some tests. According to the man page
>> > below, there should be a "hw.em.msix" sysctl to enable/disable MSI-X
>> > style interrupts. But why is it missing in my system below?
>> > https://man.freebsd.org/cgi/man.cgi?em(4)
>> > 
>> >
>> > root@s1:~ # dmesg | egrep "8254"
>> > Timecounter "i8254" frequency 1193182 Hz quality 0
>> > Event timer "i8254" frequency 1193182 Hz quality 100
>> > em0:  port 0xdcc0-0xdcff mem
>> > 0xdfae-0xdfaf irq 64 at device 7.0 on pci6
>> > em1:  port 0xccc0-0xccff mem
>> > 0xdf8e-0xdf8f irq 65 at device 8.0 on pci7
>> > em2:  port 0xbcc0-0xbcff
>> > mem 0xdf5e-0xdf5f irq 106 at device 4.0 on pci9
>> > em3:  port 0xbc80-0xbcbf
>> > mem 0xdf5c-0xdf5d irq 107 at device 4.1 on pci9
>> > em4:  port 0xacc0-0xacff
>> > mem 0xdf3e-0xdf3f irq 101 at device 3.0 on pci10
>> > em5:  port 0xac80-0xacbf
>> > mem 0xdf3c-0xdf3d irq 102 at device 3.1 on pci10
>> >
>> > root@s1:~ # ifconfig em5
>> > em5: flags=8843 metric 0 mtu
>> 1500
>> >
>> options=481009b
>> > ether 00:04:23:b7:1e:1d
>> > inet 10.1.1.2 netmask 0xff00 broadcast 10.1.1.255
>> > media: Ethernet 1000baseT 
>> > status: active
>> > nd6 options=29
>> >
>> > root@s1:~ # sysctl hw.em
>> > hw.em.max_interrupt_rate: 8000
>> > hw.em.eee_setting: 1
>> > hw.em.rx_process_limit: 100
>> > hw.em.sbp: 0
>> > hw.em.smart_pwr_down: 0
>> > hw.em.rx_abs_int_delay: 66
>> > hw.em.tx_abs_int_delay: 66
>> > hw.em.rx_int_delay: 0
>> > hw.em.tx_int_delay: 66
>> > hw.em.disable_crc_stripping: 0
>> > root@s1:~ #
>>
>> As if_em was converted to iflib, I guess the related sysctl would now be
>> dev.em.0.iflib.disable_msix (I could be wrong, don't have h/w to check).
>>
>

[Bug 221445] The absence of the accept_rtadv option causes an error "ping6: sendmsg: No buffer space available"

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221445

dereck...@gmail.com changed:

   What|Removed |Added

 CC||dereck...@gmail.com

--- Comment #18 from dereck...@gmail.com ---
(In reply to Sam Frenick from comment #17)

I confirm https://reviews.freebsd.org/D23695 solves the issue.

At OVH, two VMs on VMWare ESXi configured with those 2 routes:
  -  -iface 
  - default 

VM under FreeBSD 13.1: doesn't work, ping6: sendmsg: No buffer space available
VM under FreeBSD 13.2: works

After upgrade from 13.1 to 13.2, the ping6: sendmsg: No buffer space available
message disappears (and outgoing connectivity works).

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: missing hw.em.msix support in FreeBSD 14.0-CURRENT?

[Cheng Cui]

Hi Yuri,

Thanks for the update. I am using an Emulab testbed node under the same
type of hardware. Below are the new outputs.
It is supposed to have multiple tx/rx queues under msix. Why are there only
rxq0/txq0? Is it because the CPU has only one core? How can I check if RSS
is supported?

root@s1:~ # sysctl -d dev.em.0.iflib.disable_msix
dev.em.0.iflib.disable_msix: disable MSI-X (default 0)
root@s1:~ # sysctl dev.em.0.iflib.disable_msix
dev.em.0.iflib.disable_msix: 0
root@s1:~ # sysctl dev.em.0.iflib
dev.em.0.iflib.rxq0.rxq_fl0.buf_size: 2048
dev.em.0.iflib.rxq0.rxq_fl0.credits: 1023
dev.em.0.iflib.rxq0.rxq_fl0.cidx: 959
dev.em.0.iflib.rxq0.rxq_fl0.pidx: 958
dev.em.0.iflib.rxq0.cpu: 1
dev.em.0.iflib.txq0.r_abdications: 0
dev.em.0.iflib.txq0.r_restarts: 0
dev.em.0.iflib.txq0.r_stalls: 0
dev.em.0.iflib.txq0.r_starts: 110443
dev.em.0.iflib.txq0.r_drops: 0
dev.em.0.iflib.txq0.r_enqueues: 110444
dev.em.0.iflib.txq0.ring_state: pidx_head: 1903 pidx_tail: 1903 cidx: 1903
state: IDLE
dev.em.0.iflib.txq0.txq_cleaned: 115208
dev.em.0.iflib.txq0.txq_processed: 115251
dev.em.0.iflib.txq0.txq_in_use: 42
dev.em.0.iflib.txq0.txq_cidx_processed: 565
dev.em.0.iflib.txq0.txq_cidx: 525
dev.em.0.iflib.txq0.txq_pidx: 568
dev.em.0.iflib.txq0.no_tx_dma_setup: 0
dev.em.0.iflib.txq0.txd_encap_efbig: 0
dev.em.0.iflib.txq0.tx_map_failed: 0
dev.em.0.iflib.txq0.no_desc_avail: 0
dev.em.0.iflib.txq0.mbuf_defrag_failed: 0
dev.em.0.iflib.txq0.m_pullups: 0
dev.em.0.iflib.txq0.mbuf_defrag: 0
dev.em.0.iflib.txq0.cpu: 0
dev.em.0.iflib.override_nrxds: 0
dev.em.0.iflib.override_ntxds: 0
dev.em.0.iflib.use_logical_cores: 0
dev.em.0.iflib.separate_txrx: 0
dev.em.0.iflib.core_offset: 0
dev.em.0.iflib.tx_abdicate: 0
dev.em.0.iflib.rx_budget: 0
dev.em.0.iflib.disable_msix: 0
dev.em.0.iflib.override_qs_enable: 0
dev.em.0.iflib.override_nrxqs: 0
dev.em.0.iflib.override_ntxqs: 0
dev.em.0.iflib.driver_version: 7.7.8-fbsd
root@s1:~ #


Best Regards,
Cheng Cui


On Sun, May 7, 2023 at 4:18 PM Yuri  wrote:

> Cheng Cui wrote:
> > Hello,
> >
> > I am using this em(4) driver for some tests. According to the man page
> > below, there should be a "hw.em.msix" sysctl to enable/disable MSI-X
> > style interrupts. But why is it missing in my system below?
> > https://man.freebsd.org/cgi/man.cgi?em(4)
> > 
> >
> > root@s1:~ # dmesg | egrep "8254"
> > Timecounter "i8254" frequency 1193182 Hz quality 0
> > Event timer "i8254" frequency 1193182 Hz quality 100
> > em0:  port 0xdcc0-0xdcff mem
> > 0xdfae-0xdfaf irq 64 at device 7.0 on pci6
> > em1:  port 0xccc0-0xccff mem
> > 0xdf8e-0xdf8f irq 65 at device 8.0 on pci7
> > em2:  port 0xbcc0-0xbcff
> > mem 0xdf5e-0xdf5f irq 106 at device 4.0 on pci9
> > em3:  port 0xbc80-0xbcbf
> > mem 0xdf5c-0xdf5d irq 107 at device 4.1 on pci9
> > em4:  port 0xacc0-0xacff
> > mem 0xdf3e-0xdf3f irq 101 at device 3.0 on pci10
> > em5:  port 0xac80-0xacbf
> > mem 0xdf3c-0xdf3d irq 102 at device 3.1 on pci10
> >
> > root@s1:~ # ifconfig em5
> > em5: flags=8843 metric 0 mtu 1500
> >
> options=481009b
> > ether 00:04:23:b7:1e:1d
> > inet 10.1.1.2 netmask 0xff00 broadcast 10.1.1.255
> > media: Ethernet 1000baseT 
> > status: active
> > nd6 options=29
> >
> > root@s1:~ # sysctl hw.em
> > hw.em.max_interrupt_rate: 8000
> > hw.em.eee_setting: 1
> > hw.em.rx_process_limit: 100
> > hw.em.sbp: 0
> > hw.em.smart_pwr_down: 0
> > hw.em.rx_abs_int_delay: 66
> > hw.em.tx_abs_int_delay: 66
> > hw.em.rx_int_delay: 0
> > hw.em.tx_int_delay: 66
> > hw.em.disable_crc_stripping: 0
> > root@s1:~ #
>
> As if_em was converted to iflib, I guess the related sysctl would now be
> dev.em.0.iflib.disable_msix (I could be wrong, don't have h/w to check).
>

Problem reports for n...@freebsd.org that need special attention

[bugzilla-noreply]

To view an individual PR, use:
  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id).

The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status  |Bug Id | Description
+---+---
In Progress |221146 | [ixgbe] Problem with second laggport  
New |204438 | setsockopt() handling of kern.ipc.maxsockbuf limi 
New |213410 | [carp] service netif restart causes hang only whe 
Open|  7556 | ppp: sl_compress_init() will fail if called anyth 
Open|193452 | Dell PowerEdge 210 II -- Kernel panic bce (broadc 
Open|202510 | [CARP] advertisements sourced from CARP IP cause  
Open|207261 | netmap: Doesn't do TX sync with kqueue
Open|225438 | panic in6_unlink_ifa() due to race
Open|227720 | Kernel panic in ppp server
Open|236888 | ppp daemon: Allow MTU to be overridden for PPPoE  
Open|237072 | netgraph(4): performance issue [on HardenedBSD]?  
Open|237973 | pf: implement egress keyword to simplify rules ac 
Open|238324 | Add XG-C100C/AQtion AQC107 10GbE NIC driver   
Open|238707 | Lock order reversal: rtentry vs "nd6 list"
Open|240944 | em(4): Crash with Intel 82571EB NIC with AMD Pile 
Open|241106 | tun/ppp: panic: vm_fault: fault on nofault entry  
Open|243463 | ix0: Watchdog timeout 
In Progress |118111 | rc: network.subr Add MAC address based interface  
Open|270912 | dns/unbound: issues with ASLR 

19 problems total for which you should take action.

Re: missing hw.em.msix support in FreeBSD 14.0-CURRENT?

[Yuri]

Cheng Cui wrote:
> Hello,
> 
> I am using this em(4) driver for some tests. According to the man page
> below, there should be a "hw.em.msix" sysctl to enable/disable MSI-X
> style interrupts. But why is it missing in my system below?
> https://man.freebsd.org/cgi/man.cgi?em(4)
> 
> 
> root@s1:~ # dmesg | egrep "8254"
> Timecounter "i8254" frequency 1193182 Hz quality 0
> Event timer "i8254" frequency 1193182 Hz quality 100
> em0:  port 0xdcc0-0xdcff mem
> 0xdfae-0xdfaf irq 64 at device 7.0 on pci6
> em1:  port 0xccc0-0xccff mem
> 0xdf8e-0xdf8f irq 65 at device 8.0 on pci7
> em2:  port 0xbcc0-0xbcff
> mem 0xdf5e-0xdf5f irq 106 at device 4.0 on pci9
> em3:  port 0xbc80-0xbcbf
> mem 0xdf5c-0xdf5d irq 107 at device 4.1 on pci9
> em4:  port 0xacc0-0xacff
> mem 0xdf3e-0xdf3f irq 101 at device 3.0 on pci10
> em5:  port 0xac80-0xacbf
> mem 0xdf3c-0xdf3d irq 102 at device 3.1 on pci10
> 
> root@s1:~ # ifconfig em5
> em5: flags=8843 metric 0 mtu 1500
> options=481009b
> ether 00:04:23:b7:1e:1d
> inet 10.1.1.2 netmask 0xff00 broadcast 10.1.1.255
> media: Ethernet 1000baseT 
> status: active
> nd6 options=29
> 
> root@s1:~ # sysctl hw.em
> hw.em.max_interrupt_rate: 8000
> hw.em.eee_setting: 1
> hw.em.rx_process_limit: 100
> hw.em.sbp: 0
> hw.em.smart_pwr_down: 0
> hw.em.rx_abs_int_delay: 66
> hw.em.tx_abs_int_delay: 66
> hw.em.rx_int_delay: 0
> hw.em.tx_int_delay: 66
> hw.em.disable_crc_stripping: 0
> root@s1:~ #

As if_em was converted to iflib, I guess the related sysctl would now be
dev.em.0.iflib.disable_msix (I could be wrong, don't have h/w to check).

[Bug 254596] if_bridge wants LRO turned off, if_vlan insists it remain on

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254596

Kyle Evans  changed:

   What|Removed |Added

 CC||kev...@freebsd.org,
   ||phi...@freebsd.org

--- Comment #2 from Kyle Evans  ---
Maybe CC trouble as well, I think someone mentioned that he's been doing some
bridge work lately.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.

missing hw.em.msix support in FreeBSD 14.0-CURRENT?

[Cheng Cui]

Hello,

I am using this em(4) driver for some tests. According to the man page
below, there should be a "hw.em.msix" sysctl to enable/disable MSI-X style
interrupts. But why is it missing in my system below?
https://man.freebsd.org/cgi/man.cgi?em(4)

root@s1:~ # dmesg | egrep "8254"
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
em0:  port 0xdcc0-0xdcff mem
0xdfae-0xdfaf irq 64 at device 7.0 on pci6
em1:  port 0xccc0-0xccff mem
0xdf8e-0xdf8f irq 65 at device 8.0 on pci7
em2:  port 0xbcc0-0xbcff mem
0xdf5e-0xdf5f irq 106 at device 4.0 on pci9
em3:  port 0xbc80-0xbcbf mem
0xdf5c-0xdf5d irq 107 at device 4.1 on pci9
em4:  port 0xacc0-0xacff mem
0xdf3e-0xdf3f irq 101 at device 3.0 on pci10
em5:  port 0xac80-0xacbf mem
0xdf3c-0xdf3d irq 102 at device 3.1 on pci10

root@s1:~ # ifconfig em5
em5: flags=8843 metric 0 mtu 1500
options=481009b
ether 00:04:23:b7:1e:1d
inet 10.1.1.2 netmask 0xff00 broadcast 10.1.1.255
media: Ethernet 1000baseT 
status: active
nd6 options=29

root@s1:~ # sysctl hw.em
hw.em.max_interrupt_rate: 8000
hw.em.eee_setting: 1
hw.em.rx_process_limit: 100
hw.em.sbp: 0
hw.em.smart_pwr_down: 0
hw.em.rx_abs_int_delay: 66
hw.em.tx_abs_int_delay: 66
hw.em.rx_int_delay: 0
hw.em.tx_int_delay: 66
hw.em.disable_crc_stripping: 0
root@s1:~ #

Best Regards,
Cheng Cui