The branch main has been updated by jhibbits: URL: https://cgit.FreeBSD.org/src/commit/?id=1d445ae68446771c78161a3eca35acdb04ef78ab
commit 1d445ae68446771c78161a3eca35acdb04ef78ab Author: Justin Hibbits <[email protected]> AuthorDate: 2026-08-10 18:24:08 +0000 Commit: Justin Hibbits <[email protected]> CommitDate: 2026-08-22 16:03:31 +0000 dpaa/eth: fast-path single-page TX frame build The TX SG-build loop in dpaa_eth_if_start_locked() walked page boundaries with PAGE_MASK arithmetic even for buffers that lived entirely within one page -- the common case, since MCLBYTES is smaller than PAGE_SIZE. Add a fast path that emits a single SGT entry for wholly-in-one-page segments and skips the inner while entirely. Fix the following bugs while we're here: 1. "if (m->m_len == 0) continue;" in the outer loop never advanced m -- any zero-length mbuf hung the TX path in an infinite loop. Fix this by switching to a for loop, with the advancement in the post-clause. 2. In the inner (page-splitting) loop, the cap "if (m->m_len < ssize) ssize = m->m_len;" compared against the mbuf's original length, not the remaining bytes. A single mbuf whose data started mid-page and ran into a second page would produce a second SGT entry with ssize > rem, over-reading past the buffer end into whatever followed in kernel memory. Fixed by tracking a local rem and capping ssize against it. 3. If the whole mbuf chain consisted of zero-length segments, the final-flag store "fi_sgt[i - 1].final = 1" wrote to index -1. Reject empty frames up front now instead. As part of this, rename the inner counter from dsize to rem for clarity instead of playing double-duty in both inner and outer loops. --- sys/dev/dpaa/dpaa_eth.c | 57 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/sys/dev/dpaa/dpaa_eth.c b/sys/dev/dpaa/dpaa_eth.c index 2fb902c4d300..e2826eb70b14 100644 --- a/sys/dev/dpaa/dpaa_eth.c +++ b/sys/dev/dpaa/dpaa_eth.c @@ -670,7 +670,7 @@ dpaa_eth_tx_add_csum(struct dpaa_eth_frame_info *fi) void dpaa_eth_if_start_locked(struct dpaa_eth_softc *sc) { - vm_size_t dsize, psize, ssize; + vm_size_t psize, ssize; struct dpaa_eth_frame_info *fi; unsigned int i; struct mbuf *m0, *m; @@ -710,44 +710,71 @@ dpaa_eth_if_start_locked(struct dpaa_eth_softc *sc) } i = 0; - m = m0; psize = 0; - dsize = 0; fi->fi_mbuf = m0; - while (m && i < DPAA_NUM_OF_SG_TABLE_ENTRY) { + for (m = m0; m != NULL && i < DPAA_NUM_OF_SG_TABLE_ENTRY; + m = m->m_next) { + vm_size_t rem; + if (m->m_len == 0) continue; - dsize = m->m_len; vaddr = (vm_offset_t)m->m_data; - while (dsize > 0 && i < DPAA_NUM_OF_SG_TABLE_ENTRY) { + rem = m->m_len; + + /* + * Fast path: the whole segment lives inside one + * page. Covers every default-cluster mbuf + * (MCLBYTES < PAGE_SIZE) and skips the split loop + * in the common case. + */ + if ((vaddr & PAGE_MASK) + rem <= PAGE_SIZE) { + fi->fi_sgt[i].addr = dpaa_eth_va_to_phys(vaddr); + fi->fi_sgt[i].length = rem; + fi->fi_sgt[i].extension = 0; + fi->fi_sgt[i].final = 0; + fi->fi_sgt[i].bpid = 0; + fi->fi_sgt[i].offset = 0; + psize += rem; + i++; + continue; + } + + /* + * Slow path: mbuf crosses at least one page + * boundary (jumbo cluster, or an unusually-offset + * external buffer). Emit one SGT entry per + * contiguous physical span. + */ + while (rem > 0 && i < DPAA_NUM_OF_SG_TABLE_ENTRY) { ssize = PAGE_SIZE - (vaddr & PAGE_MASK); - if (m->m_len < ssize) - ssize = m->m_len; + if (rem < ssize) + ssize = rem; fi->fi_sgt[i].addr = dpaa_eth_va_to_phys(vaddr); fi->fi_sgt[i].length = ssize; - fi->fi_sgt[i].extension = 0; fi->fi_sgt[i].final = 0; fi->fi_sgt[i].bpid = 0; fi->fi_sgt[i].offset = 0; - dsize -= ssize; + rem -= ssize; vaddr += ssize; psize += ssize; i++; } - if (dsize > 0) + if (rem > 0) /* SGT full mid-mbuf */ break; - - m = m->m_next; } - /* Check if SG table was constructed properly */ - if (m != NULL || dsize != 0) { + /* + * Reject the frame if we didn't consume the whole chain + * (SGT full mid-frame) or if the chain produced no SGT + * entries at all (all-zero-length mbufs). + */ + if (m != NULL || i == 0) { dpaa_eth_fi_free(sc, fi); m_freem(m0); continue;
