Bug#816280: Binary incompatibility between debugperl and perl

[Dominic Hargreaves]

On Sun, Mar 13, 2016 at 01:24:09PM +0200, Niko Tyni wrote:
> On Mon, Feb 29, 2016 at 12:17:39PM +, Dominic Hargreaves wrote:
> > Control: forwarded -1 https://rt.perl.org/Public/Bug/Display.html?id=127212
> > Control: tags -1 + fixed-upstream confirmed
> > 
> > On Mon, Feb 29, 2016 at 12:53:33PM +0100, Nick Wellnhofer wrote:
> > > Package: perl-debug
> > > Version: 5.20.2-2
> > > 
> > > The layout of interpreter variables is different in the debug and normal
> > > version of the perl binary. This means that XS extensions might read from 
> > > or
> > > write to the wrong area of the interpreter variable struct, causing 
> > > crashes
> > > and other strange behavior.
> > > 
> > > Here's an example, originally reported by me at
> > > https://rt.cpan.org/Public/Bug/Display.html?id=111211
> > > 
> > > $ PERL_DESTRUCT_LEVEL=2 debugperl -MList::Util=shuffle -e shuffle
> > > Segmentation fault
> 
> > This was discovered as part of the investigation into 
> >  (which is
> > not quite the same bug) and was fixed upstream. This fix should be in
> > 5.24 which should be in stretch. However, the fix by its nature breaks
> > binary compatibility, so it will unfortunately not be possible to apply
> > it to a stable release.
> 
> The workaround in sid/stretch disabling PERL_TRACK_MEMPOOL for the debug
> build (debian/debugperl-compat-fix.diff) should be applicable to jessie
> too. We should probably push it into a stable update.

Right, I see that the effect is expected to be different on jessie
and applying that patch does fix the problem.

Unfortunately we're too late for the next stable update, but I'll get
it queued up for the one after that.

Dominic.

Bug#816280: Binary incompatibility between debugperl and perl

[Niko Tyni]

On Mon, Feb 29, 2016 at 12:17:39PM +, Dominic Hargreaves wrote:
> Control: forwarded -1 https://rt.perl.org/Public/Bug/Display.html?id=127212
> Control: tags -1 + fixed-upstream confirmed
> 
> On Mon, Feb 29, 2016 at 12:53:33PM +0100, Nick Wellnhofer wrote:
> > Package: perl-debug
> > Version: 5.20.2-2
> > 
> > The layout of interpreter variables is different in the debug and normal
> > version of the perl binary. This means that XS extensions might read from or
> > write to the wrong area of the interpreter variable struct, causing crashes
> > and other strange behavior.
> > 
> > Here's an example, originally reported by me at
> > https://rt.cpan.org/Public/Bug/Display.html?id=111211
> > 
> > $ PERL_DESTRUCT_LEVEL=2 debugperl -MList::Util=shuffle -e shuffle
> > Segmentation fault

> This was discovered as part of the investigation into 
>  (which is
> not quite the same bug) and was fixed upstream. This fix should be in
> 5.24 which should be in stretch. However, the fix by its nature breaks
> binary compatibility, so it will unfortunately not be possible to apply
> it to a stable release.

The workaround in sid/stretch disabling PERL_TRACK_MEMPOOL for the debug
build (debian/debugperl-compat-fix.diff) should be applicable to jessie
too. We should probably push it into a stable update.
-- 
Niko Tyni   nt...@debian.org

Bug#816280: Binary incompatibility between debugperl and perl

[Nick Wellnhofer]

On 29/02/2016 16:52, Dominic Hargreaves wrote:

Just to be clear, do you mean you don't think that your issue
will be fixed by these patches in blead?

http://perl5.git.perl.org/perl.git/commit/5ca5a62834a7fccb204c7fd4b35a11873d1bdd3b
http://perl5.git.perl.org/perl.git/commit/0419d9784dfda8adf6980bd60e782b144126dd72

(from https://rt.perl.org/Ticket/Display.html?id=127212)


This fix should work unless there are other interpreter variables that are 
only enabled with -DDEBUGGING. But I think that my proposal to move debugging 
variables to the end of the struct is better. At least `random_state` which 
can be accessed directly from XS code should be moved forward.


Nick

Bug#816280: Binary incompatibility between debugperl and perl

[Dominic Hargreaves]

On Mon, Feb 29, 2016 at 01:59:39PM +0100, Nick Wellnhofer wrote:
> On 29/02/2016 13:17, Dominic Hargreaves wrote:
> >This was discovered as part of the investigation into
> > (which is
> >not quite the same bug) and was fixed upstream. This fix should be in
> >5.24 which should be in stretch. However, the fix by its nature breaks
> >binary compatibility, so it will unfortunately not be possible to apply
> >it to a stable release.
> 
> Yes, my issue is a bit different and it doesn't seem to be fixed. I'll bring
> it up upstream.

Just to be clear, do you mean you don't think that your issue
will be fixed by these patches in blead?

http://perl5.git.perl.org/perl.git/commit/5ca5a62834a7fccb204c7fd4b35a11873d1bdd3b
http://perl5.git.perl.org/perl.git/commit/0419d9784dfda8adf6980bd60e782b144126dd72

(from https://rt.perl.org/Ticket/Display.html?id=127212)

Dominic.

Bug#816280: Binary incompatibility between debugperl and perl

[Nick Wellnhofer]

On 29/02/2016 13:17, Dominic Hargreaves wrote:

This was discovered as part of the investigation into
 (which is
not quite the same bug) and was fixed upstream. This fix should be in
5.24 which should be in stretch. However, the fix by its nature breaks
binary compatibility, so it will unfortunately not be possible to apply
it to a stable release.


Yes, my issue is a bit different and it doesn't seem to be fixed. I'll bring 
it up upstream.


Nick

Bug#816280: Binary incompatibility between debugperl and perl

[Dominic Hargreaves]

Control: forwarded -1 https://rt.perl.org/Public/Bug/Display.html?id=127212
Control: tags -1 + fixed-upstream confirmed

On Mon, Feb 29, 2016 at 12:53:33PM +0100, Nick Wellnhofer wrote:
> Package: perl-debug
> Version: 5.20.2-2
> 
> The layout of interpreter variables is different in the debug and normal
> version of the perl binary. This means that XS extensions might read from or
> write to the wrong area of the interpreter variable struct, causing crashes
> and other strange behavior.
> 
> Here's an example, originally reported by me at
> https://rt.cpan.org/Public/Bug/Display.html?id=111211
> 
> $ PERL_DESTRUCT_LEVEL=2 debugperl -MList::Util=shuffle -e shuffle
> Segmentation fault
> 
> This segfaults because `shuffle` calls `seedDrand01` which writes to the
> `random_state` interpreter variable. If you have a look at `intrpvar.h` in
> the Perl source, you'll see that `random_state` is at the very end with some
> variables before that are only enabled in debug builds. This causes
> `random_state` to be at a different offset in the debug build. The XS module
> `List::Util` uses the non-debug offset, so any writes to `random_state`
> cause memory corruption.
> 
> I'd propose that all interpreter variables that are only used in the debug
> build should be moved to very end in `intrpvar.h`.

This was discovered as part of the investigation into 
 (which is
not quite the same bug) and was fixed upstream. This fix should be in
5.24 which should be in stretch. However, the fix by its nature breaks
binary compatibility, so it will unfortunately not be possible to apply
it to a stable release.

Cheers,
Dominic.

Bug#816280: Binary incompatibility between debugperl and perl

[Nick Wellnhofer]

Package: perl-debug
Version: 5.20.2-2

The layout of interpreter variables is different in the debug and normal 
version of the perl binary. This means that XS extensions might read from or 
write to the wrong area of the interpreter variable struct, causing crashes 
and other strange behavior.


Here's an example, originally reported by me at 
https://rt.cpan.org/Public/Bug/Display.html?id=111211


$ PERL_DESTRUCT_LEVEL=2 debugperl -MList::Util=shuffle -e shuffle
Segmentation fault

This segfaults because `shuffle` calls `seedDrand01` which writes to the 
`random_state` interpreter variable. If you have a look at `intrpvar.h` in the 
Perl source, you'll see that `random_state` is at the very end with some 
variables before that are only enabled in debug builds. This causes 
`random_state` to be at a different offset in the debug build. The XS module 
`List::Util` uses the non-debug offset, so any writes to `random_state` cause 
memory corruption.


I'd propose that all interpreter variables that are only used in the debug 
build should be moved to very end in `intrpvar.h`.