Fishwaldo opened a new pull request, #19735:
URL: https://github.com/apache/nuttx/pull/19735
## Summary
* The arm64 machine directory optimizes thirteen string and memory
functions. `libs/libc/machine/risc-v` optimized three, and one of those
was leaving most of its speed on the table. This closes the gap.
* Measured on rv64 silicon at 1.4 GHz (ESWIN EIC7700X), generic C library
against this series, MB/s:
| function | generic | optimized | speedup |
|----------------------|--------:|----------:|--------:|
| `memcmp` aligned | 30 | 343 | 11.4x |
| `memmove` backward | 438 | 3464 | 7.9x |
| `strncmp` aligned | 32 | 245 | 7.7x |
| `memcpy` aligned | 406 | 3020 | 7.4x |
| `memcpy` mismatched | 408 | 2584 | 6.3x |
| `strlen` | 645 | 3274 | 5.1x |
| `memchr` | 641 | 2497 | 3.9x |
| `strchr` | 644 | 2281 | 3.5x |
| `strcpy` | 601 | 1975 | 3.3x |
| `strrchr` | 402 | 700 | 1.7x |
Each number comes from a harness that verifies its own buffers and
reports its repetition counts; the raw output is in the Testing section.
* The two existing assembly routines learn what they most lacked. `memcpy`
becomes register-width aware, eight bytes a step on RV64 where it always
moved four, and gains a shifting path for the case it used to give up on:
when source and destination disagree about where a register boundary
falls, it now reads the two aligned words straddling each output word and
shifts them together, so no load and no store is ever misaligned and only
the head and tail go byte by byte. `strcmp` stops treating every
unaligned pointer as hopeless: two pointers the same distance past a
boundary are walked up to it bytewise and compared a register at a time
from there, which is the common case for strings carved out of larger
buffers. Only pointers that disagree about the boundary keep the byte
loop, because no single aligned load serves both.
* Ten functions the directory did not cover at all are added as portable
word-at-a-time C, sharing one small header of the old tricks: `memmove`,
`memcmp`, `memchr`, `strchr`, `strchrnul`, `strcpy`, `strlen`, `strncmp`,
`strnlen` and `strrchr`. C rather than assembly because the compiler
renders these scan loops well on both register widths from one source,
and the result is within measurement of what hand assembly reaches for
them. The umbrella option now selects all thirteen.
* The second commit adds `strlcpy` and unrolls the two compare loops.
* **RV32 and RV64 are both supported from one source, but only RV64 has
measured numbers.** RV32 correctness is verified under QEMU; its timings
are not, because QEMU's TCG does not model a real pipeline and any MB/s
figure from it would be meaningless. RV32 should gain too, and for
different reasons per function: the ten new functions go from
byte-at-a-time to word-at-a-time scanning, which is the bulk of the win
above and is register-width independent; `strcmp` gains the same-offset
path; `memcpy` gains only the shifting path, since its aligned loop
already moved four bytes a step on RV32. Anyone with rv32 hardware is
welcome to post numbers.
* **Provenance:** the ten new C files are original, written against the
classic word-at-a-time zero-byte test (`(x - ONES) & ~x & HIGHS`), a
public technique documented in *Hacker's Delight* and used by most C
libraries. newlib's RISC-V machine directory was examined during this
work to see what it covers and under what licence; its files are SiFive
BSD-2-Clause and no code was taken from them. `arch_strcmp.S` was already
in-tree under its original SiFive BSD-2-Clause header, which this series
modifies and preserves.
* **Testing dependency:** the correctness evidence below comes from the
test suite in apache/nuttx-apps#3706, written for this work. That PR is
arch-neutral and stands alone, but this one is best reviewed alongside
it.
* No related issue filed.
## Impact
* Is new feature added? Is existing feature changed? **BOTH.** Ten new
optimized functions, two existing ones reworked, one (`strlcpy`) added by
the second commit.
* Impact on user? **YES, positive**, and opt-in as before. The functions
appear only when `CONFIG_LIBC_ARCH_*` selects them; the umbrella option
now covers all thirteen, so a configuration using the umbrella gets the
new ones automatically.
* Impact on build? **NO** beyond the new objects.
* Impact on hardware? **YES, RISC-V only.** Both RV32 and RV64 build from
one source. RV64 is measured on silicon; RV32 is correctness-verified
under QEMU only, with no performance claim made for it.
* Impact on documentation? **NO.**
* Impact on security? **NO**, though `strlcpy` is the safer-by-design
interface and this makes it fast enough to have no excuse not to use.
* Impact on compatibility? **NO.** Same semantics, verified against the
generic implementations.
* Build-mode dependence? **NO.**
* Anything else? Honest caveat on the unrolling in the second commit: it
helps less than it should. 343 to 420 MB/s for `memcmp`, 245 to 272 for
`strncmp`, while the emitted loop is eight loads, four XORs, three ORs
and a branch per thirty two bytes, and `strlen`'s near identical
single-stream loop runs at 3.3 GB/s. Something about two-stream reads on
this core deserves a profile of its own. The loops are left unrolled
because they are no worse anywhere and the shape is right once that is
understood.
## Testing
I confirm that changes are verified on local setup and works as intended:
* Build Host: macOS 26.5.1, arm64 (Apple Silicon), xPack riscv-none-elf-gcc
15.2.0
* Targets: **RISC-V rv64 on real hardware** (ESWIN EIC7700X EVB, 1.4 GHz,
downstream board port not yet upstream) and **RISC-V rv32 under QEMU**
* Test suite: apache/nuttx-apps#3706
Correctness on rv64 hardware with every function overridden, and the raw
speed report behind the Summary table:
```
########## ROUND 2 (final harness): OPTIMIZED rv64 ##########
== correctness ==
memcpy correctness: ok (0 bad)
memmove correctness: ok (0 bad)
mv-overlap correctness: ok (0 bad)
memset correctness: ok (0 bad)
memcmp correctness: ok (0 bad)
str-scan correctness: ok (0 bad)
strcmp-fam correctness: ok (0 bad)
[compare buffers equal]
memcmp 64K aligned 342.6 MB/s (1376 reps, 0.251s)
memcmp 64K s+1/d+1 337.3 MB/s (1360 reps, 0.252s)
strncmp 4K aligned 244.8 MB/s (15664 reps, 0.250s)
strncmp 4K both+2 241.0 MB/s (15424 reps, 0.250s)
strlen 4K aligned 3274.3 MB/s (209552 reps, 0.250s)
strlen 4K +3 3240.5 MB/s (207392 reps, 0.250s)
memchr 4K aligned 2496.8 MB/s (159792 reps, 0.250s)
strchr 4K absent 2281.0 MB/s (145984 reps, 0.250s)
strchr 4K absent +5 2285.0 MB/s (146240 reps, 0.250s)
strrchr 4K 700.3 MB/s (44816 reps, 0.250s)
memcmp 64K s+1/d+2 41.7 MB/s (176 reps, 0.264s)
memcpy 64K aligned 3020.0 MB/s (12080 reps, 0.250s)
memcpy 64K src+1 2584.0 MB/s (10336 reps, 0.250s)
memmove-bk 64K 3464.0 MB/s (13856 reps, 0.250s)
strcpy 4K aligned 1975.3 MB/s (126416 reps, 0.250s)
strcpy 4K both+1 1822.5 MB/s (116640 reps, 0.250s)
== fails: 0 ==
```
The generic baseline on the same hardware, which is the left column of the
Summary table:
```
########## ROUND 1 final: BASELINE (EVB rv64 generic) ##########
== correctness ==
memcpy correctness: ok (0 bad)
memmove correctness: ok (0 bad)
mv-overlap correctness: ok (0 bad)
memset correctness: ok (0 bad)
memcmp correctness: ok (0 bad)
str-scan correctness: ok (0 bad)
strcmp-fam correctness: ok (0 bad)
== fails: 0 ==
```
The same suite on rv32 under QEMU. Correctness only: QEMU's TCG does not
model a real pipeline, so no timing from it is quoted anywhere in this PR.
```
rv32 built
== string correctness ==
memcpy correctness: ok (0 bad)
memmove correctness: ok (0 bad)
mv-overlap correctness: ok (0 bad)
memset correctness: ok (0 bad)
memcmp correctness: ok (0 bad)
str-scan correctness: ok (0 bad)
strcmp-fam correctness: ok (0 bad)
strlcpy correctness: ok (0 bad)
== string fails: 0 ==
```
Both architectures also pass with the optimizations disabled, against the
generic C library, which is the control.
What the suite covers, since a "0 bad" line is only as good as what
produced it: every source and destination alignment zero through seven,
twenty one sizes from zero up, overlap in both directions for `memmove`,
terminator placement and cap interaction for the n-bounded functions, and
guard bytes around every destination. `strlcpy` additionally: every cap
from zero to past the end, the return always the source length, the result
terminated whenever the cap is nonzero, at most cap-1 bytes copied, and a
cap of zero writing nothing.
The census behind the choice of functions: in this port's kernel image
`strlen` has seventy five call sites, `memcmp` fifty nine and `strcpy`
thirty seven, the three hottest functions the directory did not optimize.
## PR verification Self-Check
* [x] This PR introduces only one functional change.
* [x] I have updated all required description fields above.
* [x] My PR adheres to Contributing Guidelines and Documentation.
* [ ] My PR is still work in progress (not ready for review).
* [x] My PR is ready for review and can be safely merged into a codebase.
---
*Claude (claude-opus-5) assisted with authoring these routines, their code
comments and this PR description. The commits carry `Assisted-by:` tags
per
[CONTRIBUTING.md](https://github.com/apache/nuttx/blob/master/CONTRIBUTING.md)
ยง1.5.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
