Branch: refs/heads/yves/fix_gh19953_add_EXACT_INV
Home: https://github.com/Perl/perl5
Commit: 260703f734aa43e0a498a9ca86e9c12f263b75bd
https://github.com/Perl/perl5/commit/260703f734aa43e0a498a9ca86e9c12f263b75bd
Author: Yves Orton <[email protected]>
Date: 2026-10-15 (Thu, 15 Oct 2026)
Changed paths:
M embed.fnc
M embed.h
M ext/re/t/regop.pl
M ext/re/t/regop.t
M pod/perldelta.pod
M pod/perlreguts.pod
M proto.h
M regcomp.c
M regcomp.h
M regcomp.sym
M regcomp_study.c
M regcomp_trie.c
M regexec.c
M regnodes.h
M t/re/anyof.t
M t/re/re_tests
Log Message:
-----------
regcomp: let non-folding literals join folded nodes and tries
Add EXACT_NF and EXACT_NF_REQ8 to identify text which participates in no
Unicode case fold (NF means "Non Folding"). These nodes execute like EXACT
and EXACT_REQ8, but the optimizer can safely join their text to a folded
exact type, for instance EXACT_FU safely, just as it can join their text
to an old-style EXACT node.
We have long split non-folding text out of folded literals so the
fixed-substring optimizer can use it. However, that split has also prevented
otherwise compatible alternatives from forming a complete trie. It also
interacts with an optimization we do that turns a folded single character into
a character class, which can prevent more powerful TRIE style optimizations
from happening. For example, /foo|bar|a_word|_qux/i should compile into one
TRIE-EXACTFU containing all four words.
Part of the problem is that we were splitting the nodes out as EXACT,
which is actually subtly not ideal. The problem is that EXACT nodes
created normally may (and very often do) contain content that would fold
if it were merged into an EXACTFU for instance. EXACT_NF however are
EXACTish nodes (from the point of view of fixed string extraction) but which
contain content that behaves the same folded or not, and which can be safely
merged together with (more or less) any other EXACT type. By splitting
EXACT into "exact stuff you can NOT merge with any folding types" and
"exact stuff you CAN merge with folding types", we can undo the split
later on, merging the EXACT_NF nodes together with whatever they are next to.
Thus during study we leave non-folding pieces separate until the substring and
length logic has seen them. Then we join each piece to its compatible
predecessor, carring a merge-left flag forward after a successful join so that
the join cascades, meaning that
EXACTFU "foo"
EXACT_NF "_"
EXACTFU "bar"
EXACT_NF "_"
EXACTFU "baz"
becomes one folded EXACTFU containing "foo_bar_baz".
Only globally non-foldable characters get the new types, eg "." or such
characters.
In particular, a ligature whose fold is excluded under /aa stays EXACT_REQ8.
Marking
it non-folding would let a neighboring /u literal absorb it and change its
matching rules.
Includes regression tests for the reported trie, cascading joins before and
after a
nested alternation, single-letter continuations, sharp-S class alternatives,
scoped /aa folding, and mixed narrow and wide non-folding characters.
Fixes #19953.
To unsubscribe from these emails, change your notification settings at
https://github.com/Perl/perl5/settings/notifications