Hi guys, a quick search on the archives did not reveal anything - if this is already a known issue, please don't kick me ;)
We encountered an issue using TCG, where self-modifying code is overwriting the executing instruction, causing different behavior in TCG and a real CPU. Clearly, when code is overwriting itself, the QEMU logic will detect and invalidate the TB - and one would assume this is the correct behavior. However, when the current instruction has a REP-prefix, this should cause the ORIGINAL instruction to execute, and NOT generate an exception. Since TCG uses a loop, however, a repeated invocation of the REP'ed instruction will cause an exception, TCG retranslates the TB, and QEMU will execute the new code. For example: seg000:0039083D loc_39083D:......... seg000:0039083D E8 00 00 00 00 call $+5 seg000:00390842 59 pop ecx seg000:00390843 8D BB 00 10 40 00 lea edi, (loc_390000 - 0FFF8F000h)[ebx] seg000:00390849 29 F9 sub ecx, edi seg000:0039084B 83 C1 10 add ecx, 10h seg000:0039084E 30 C0 xor al, al seg000:00390850 FC cld ; here ECX = 0x852 and EDI = 0x390000 ; so at the step 0x851 the instruction rewrites itself ; but EIP stays the same = 0x00390851 ; and instruction becomes invalid ; seg000:00390851 00 AA.. ; when QEMU tries to regenerate TB ; the generated instruction throws an exception seg000:00390851 F3 AA rep stosb seg000:00390853 56 push esi seg000:00390854 C3 retn On a real CPU, the instruction becomes invalid when overwritten, but the CPU continues execution without raising an exception (because it's already been loaded into the execution registers). Getting around this is not trivial, so if it's a new issue, does anyone have an idea how to approach it most meaningfully? I was thinking about detecting this very specific case and marking the TB for re-translation only after the TB is exited, but this is not very clean and also somewhat prone to errors. Thanks! -Clemens -- Clemens Kolbitsch Security Researcher kolbit...@lastline.com Mobile +1 (206) 356-7745 Land +1 (805) 456-7076 Lastline, Inc. 6950 Hollister Avenue, Suite 101 Goleta, CA 93117 www.lastline.com