https://github.com/python/cpython/commit/60eca2df18fcd47d71ab136e460efabbc2f4a044
commit: 60eca2df18fcd47d71ab136e460efabbc2f4a044
branch: main
author: Cody Maloney <[email protected]>
committer: cmaloney <[email protected]>
date: 2026-11-14T16:31:55-08:00
summary:

gh-161145: Test more bytes simultaneously in find_first_nonascii (#161147)

Long ASCII runs now OR eight aligned words and test the high bits once,
one branch per 64 bytes (32 on 32-bit) instead of one per word.  When
a block contains a non-ASCII byte, the existing one-word loop finds it.
It is plain C: GCC and Clang compile it to SSE2/NEON loads, other
compilers to ORed word loads with the same single branch.

Multiple compiler and architecture codegen sample: 
https://godbolt.org/z/8bM83vx3a

pyperf, PGO+LTO, 2 interleaved rounds:

| Benchmark | main | this |
|---|---:|---:|
| (b'a' * 4096).decode() | 165 ns | 108 ns: 1.53x faster |
| (b'a' * 200).decode() | 28.9 ns | 26.5 ns: 1.09x faster |
| b'a' * 200 + 'é' + ..., .decode() | 60.9 ns | 56.0 ns: 1.09x faster |
| b'a' * 136 + 'é' + ..., .decode() | 53.8 ns | 51.3 ns: 1.05x faster |
| 8-100 byte decodes, 'é' within 100 | | within 1.2 ns |

```python
import pyperf

runner = pyperf.Runner()
tail = 'é'.encode() + b'a' * 50
for n in (8, 100, 200, 4096):
    runner.timeit(f"(b'a' * {n}).decode()", "s.decode()",
                  globals={"s": b'a' * n})
for n in (16, 100, 136, 200):
    runner.timeit(f"(b'a' * {n} + 'é'.encode() + b'a' * 50).decode()",
                  "s.decode()", globals={"s": b'a' * n + tail})
```

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>

files:
A 
Misc/NEWS.d/next/Core_and_Builtins/2026-11-01-16-02-42.gh-issue-161145.5vLdrm.rst
M Objects/unicodeobject.c

diff --git 
a/Misc/NEWS.d/next/Core_and_Builtins/2026-11-01-16-02-42.gh-issue-161145.5vLdrm.rst
 
b/Misc/NEWS.d/next/Core_and_Builtins/2026-11-01-16-02-42.gh-issue-161145.5vLdrm.rst
new file mode 100644
index 00000000000000..5d809624e25f42
--- /dev/null
+++ 
b/Misc/NEWS.d/next/Core_and_Builtins/2026-11-01-16-02-42.gh-issue-161145.5vLdrm.rst
@@ -0,0 +1,2 @@
+Speed up UTF-8 decoding of long ASCII runs, such as in :meth:`bytes.decode`,
+by up to 1.5x.
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
index da1ba44dba8129..0fe765c36848f2 100644
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -5063,6 +5063,16 @@ find_first_nonascii(const unsigned char *start, const 
unsigned char *end)
         }
 #endif
 
+        // Long ASCII runs: test eight words per branch.
+        for (; end - p >= 8 * SIZEOF_SIZE_T; p += 8 * SIZEOF_SIZE_T) {
+            const size_t *w = (const size_t *)p;
+            if ((w[0] | w[1] | w[2] | w[3] |
+                 w[4] | w[5] | w[6] | w[7]) & ASCII_CHAR_MASK)
+            {
+                break;
+            }
+        }
+
         const unsigned char *e = end - SIZEOF_SIZE_T;
         while (p <= e) {
             size_t u = (*(const size_t *)p) & ASCII_CHAR_MASK;

_______________________________________________
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3//lists/python-checkins.python.org
Member address: [email protected]

Reply via email to