Branch: refs/heads/yves/fix_gh19953_add_EXACT_INV
Home: https://github.com/Perl/perl5
Commit: f58f1cba06e4952a8f852731bf295b6aae3a4984
https://github.com/Perl/perl5/commit/f58f1cba06e4952a8f852731bf295b6aae3a4984
Author: Yves Orton <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M embed.fnc
M embed.h
M ext/re/t/regop.pl
M ext/re/t/regop.t
M pod/perldelta.pod
M pod/perlreguts.pod
M proto.h
M regcomp.c
M regcomp.h
M regcomp.sym
M regcomp_study.c
M regcomp_trie.c
M regexec.c
M regnodes.h
M t/re/anyof.t
M t/re/re_tests
Log Message:
-----------
regcomp: let invariant literals join folded nodes and tries
Add EXACT_INV and EXACT_INV_REQ8 to identify text which participates in no
Unicode case fold. These nodes execute like EXACT and EXACT_REQ8, but the
optimizer can safely join their text to a folded literal.
We have long split non-folding text out of folded literals so the
fixed-substring optimizer can use it. However, that split has also prevented
otherwise compatible alternatives from forming a complete trie. It also
interacts with an optimization we do that turns a folded single character into
a character class, which can prevent more powerful TRIE style optimizations
from happening. For example, /foo|bar|a_word|_qux/i should compile into one
TRIE-EXACTFU containing all four words.
Part of the problem is that we were splitting the nodes out as EXACT,
which is actually subtly not ideal. The problem is that EXACT nodes
created normally may (and very often do) contain content that would fold
if it were merged into an EXACTFU for instance. EXACT_INV however are
EXACTish nodes (from the point of view of fixed string extraction) but which
contain content that behaves the same folded or not, and which can be safely
merged together with (more or less) any other EXACT type. By splitting
EXACT into "exact stuff you can NOT merge with any folding types" and
"exact stuff you CAN merge with folding types", we can undo the split
later on, merging the EXACT_INV nodes together with whatever they are next to.
Thus during study we leave invariant pieces separate until the substring and
length logic has seen them. Then we join each piece to its compatible
predecessor, carring a merge-left flag forward after a successful join so that
the join cascades, meaning that
EXACTFU "foo"
EXACT_INV "_"
EXACTFU "bar"
EXACT_INV "_"
EXACTFU "baz"
becomes one folded EXACTFU containing "foo_bar_baz".
Only globally invariant characters get the new types. In particular, a
ligature whose fold to ASCII is excluded by /aa stays EXACT_REQ8. Marking it
invariant would let a neighboring /u literal absorb it and change its
matching rules.
Includes regression tests for the reported trie, cascading joins before and
after a
nested alternation, single-letter continuations, sharp-S class alternatives,
scoped /aa folding, and mixed narrow and wide invariants.
Fixes #19953.
To unsubscribe from these emails, change your notification settings at
https://github.com/Perl/perl5/settings/notifications