Branch: refs/heads/davem/rx_codeblocks2
Home: https://github.com/Perl/perl5
Commit: b74d82c334268af2b0eebaf2c73435f73e766a11
https://github.com/Perl/perl5/commit/b74d82c334268af2b0eebaf2c73435f73e766a11
Author: David Mitchell <da...@iabyn.com>
Date: 2024-05-08 (Wed, 08 May 2024)
Changed paths:
M regcomp.c
M regexp.h
M t/re/pat.t
Log Message:
-----------
pat codeblocks: add size to struct reg_code_blocks
Split the 'count' field of reg_code_blocks structures into separate
'count' and 'size' fields to make the code less fragile; and as an
intended side-effect, fix GH #16627.
Background:
When a pattern includes embedded perl code, such as /(?{ CODE })/, then
at compile-time the op trees associated with each of those code blocks
are stored within the compiled regex, in a reg_code_blocks structure.
This structure contains some basic info, plus a pointer to an array of
reg_code_block structures, each of which contains a pointer to the
optree for that code block, plus string offsets to where the (?{..}) or
similar expression starts and ends within the pattern string.
For a runtime pattern, perl tries to reuse any original compiled code
blocks rather than recompiling them, to maintain correct closure
behaviour.
So for example, in the following:
my $x = 1;
{ my $x = 2; $r = qr/(??{$x})/ }
my $y = 3;
my $s = '(??{$y})';
my $pat = qr/A (??{$x}) B $r C $s/x;
at perl compile time, the two '$x' code blocks are compiled, and their
optrees stored.
At runtime, when the $pat pattern is compiled, the third code block,
'$y', is compiled, and the two earlier optrees are retrieved. A new
three-element 'struct reg_code_blocks' array is malloc()ed, and the
pointers to the two old, and one new, optrees are stored in it.
Overall, $pat has the same effect as qr/A1B2C3/.
The assembly of this reg_code_blocks array is mostly performed by
S_concat_pat() and S_compile_runtime_code(). It is done incrementally,
since the total number of code blocks isn't known in advance.
Prior to this commit, the array was often realloced() and grown one
element at at a time, as each new run-time code block was discovered,
with a corresponding pRExC_state->code_blocks->count++.
This count field served twin purposes: it indicated both how many code
blocks had been found and stored so far, and the malloc()ed size of the
array. But some parts of the regex compiler allocate more than one slot
at a time, and so the two meanings of the 'count' field temporarily
diverge. This became noticeable when S_concat_pat() recursed to
interpolate the contents of an array, such as qr/$a$b@c/, where
interpolating $a, $b was done iteratively at the top level, then it
recursed to process each element of @c. S_concat_pat() had a local var,
'int n', which counted how many code blocks had been found so far, and
this value sometimes represented the difference between the two meanings
of the 'count' field.
However when it recursed, n started from zero again and things got out
of whack, which led to GH #16627. The bug in that ticket can be reduced
to:
my @x = ( qr/(?{A})/ );
qr/(?{B})@x/;
Here the B code block is stored in pRExC_state->code_blocks->cb[0],
but then S_concat_pat recurses, n is reset to 0, and the A code block is
also stored into slot 0. Then things would start to crash.
The quick and dirty fix would be to share n between recursive calls to
S_concat_pat(), by passing a pointer to it. Instead, this commit takes
the approach of adding a 'size' field to pRExC_state->code_blocks,
so that ->count now only indicates the current number of code blocks
stored (replacing the local var n) while ->size indicates the current
number of slots malloc()ed.
This makes the code more conventional and simpler to understand, and
allows the realloc() to pre-allocate rather than incrementing the array
size by 1 each time. By removing the fragile double meaning of the
'count' field, it should make any future bugs easier to diagnose, at the
cost of this initial commit being more complex.
Commit: 10656067f5a4a481ce355980ecde7425b4fe201f
https://github.com/Perl/perl5/commit/10656067f5a4a481ce355980ecde7425b4fe201f
Author: David Mitchell <da...@iabyn.com>
Date: 2024-05-08 (Wed, 08 May 2024)
Changed paths:
M regcomp.c
M t/op/svleak.t
Log Message:
-----------
avoid leak in regex with overload and code block
When concatenating the components of a run-time pattern, if
a component has concat overloading, then that method is used, and
any previously accumulated code blocks - i.e. (?{...}) and similar -
are discarded. However, the ref counts of of any regex objects
pointed to which contained those code block(s) weren't having
their reference count decremented, and so leaked.
Spotted by code inspection while working on the previous commit.
Compare: https://github.com/Perl/perl5/compare/b74d82c33426%5E...10656067f5a4
To unsubscribe from these emails, change your notification settings at
https://github.com/Perl/perl5/settings/notifications
