Thu Jan 21 15:49:36 2016: Request 80322 was acted upon.
Transaction: Correspondence added by BULKDD
Queue: Win32-API
Subject: 'make test' failure - 03_Jim_Shaw.t - invalid unpack type
Broken in: 0.72, 0.73
Severity: Important
Owner: Nobody
Requestors: jdhed...@cpan.org
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=80322 >
After having no clue how to fix this, and having some doubts if this ticket is
real, I reproduced it with Win 7+VC 2013. Perhaps something in EnumChildWindows
changed between Win XP and Win7 that stopped "resetting" the FPU, or the SSE
heavy instead of x87, machine code of VC 2013 wasn't clearing the FPU state the
way the older non-SSE code wouldve, or EnumChildWindows in WinXP used ebp as
base pointer and in Win7 when MS compiled user32.dll, ebp was reused as GPR
with only esp addressing.
Anyways, what was happening was each fld op in the shellcode callback was
pushing one var onto FPU stack. Eventually the FPU stack which has 8 elements
was filled. x87 Stack Fault (bit 6) exception bit came on at that point. After
that, no further FPU math was possible. NAN was the result of every op. That
resulted in this PP division op failing, after 8 calls of the W::A::C callback
https://metacpan.org/source/BULKDD/Win32-API-0.82/Callback.pm#L176 this line in
PP was returning NAN even though its "16/4" which cant possibly be NAN. The NAN
in a SVNV after going through SvUV became 0, so the callback unwound the C
stack by 0 bytes instead of 8 as stdcall requires (EnumChildWindows wants a
stdcall func, not cdecl). Eventually EnumChildWindows poped nonvol register esi
from the wrong location on C stack a couple hundred bytes away from the correct
location of the saved esi register, since every call into the callback screwed
up esp by 8 bytes. The corrupted esi register after E
numChildWindows returned control to W::A causing a crash in Win32::API's
Call_asm.
