Re: JIT On Win32: A possible future issue
Jonathan Worthington [EMAIL PROTECTED] wrote: Hi, I was looking over the WinXP Service Pack 2 Changes to functionality document, which you can get here:- http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechn ol/winxppro/maintain/winxpsp2.asp (download available in Word format) JIT was my first thought on reading this, and indeed it gets a specific mention in the document. Turns out you need to use a bit of the Windows API (VirtualAlloc) and pass it flags to say execution of data in that page of memory is OK. I'm not sure how the memory used for JIT'ing is allocated right now - I think there are wrapper functions we call to do it - in which case they can do stuff on a platform specific basis. JIT code is run in malloc'ed memory (s. jit.c:1026) as well as JITted NCI stubs. We already have some system specific stuff for JIT (some architectures need a page flush before execution). So best would be to use a set of spcialized function, e.g.: - mem_alloc_executable - mem_realloc_executable - mem_flush_executable - mem_free_executable Anyways, figured it may be worth a mention now, so there isn't an out-of-the-blue JIT doesn't work on XP SP2 bug report somewhere down the line. Yep, thanks - always good to know, where problems might exist. Jonathan leo
Re: JIT On Win32: A possible future issue
Leopold Toetsch wrote: JIT code is run in malloc'ed memory (s. jit.c:1026) as well as JITted NCI stubs. We already have some system specific stuff for JIT (some architectures need a page flush before execution). So best would be to use a set of spcialized function, e.g.: - mem_alloc_executable - mem_realloc_executable - mem_flush_executable - mem_free_executable Note that JIT already fails under Fedora Core 1 because of 'Exec-Shield' , which is the Linux equivalent of the functionality described by Jonathan. Regards Peter Gibbs EmKel Systems
Re: JIT On Win32: A possible future issue
Peter Gibbs [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: JIT code is run in malloc'ed memory (s. jit.c:1026) as well as JITted NCI stubs. We already have some system specific stuff for JIT (some architectures need a page flush before execution). So best would be to use a set of spcialized function, e.g.: - mem_alloc_executable - mem_realloc_executable - mem_flush_executable - mem_free_executable Note that JIT already fails under Fedora Core 1 because of 'Exec-Shield' , which is the Linux equivalent of the functionality described by Jonathan. Ah, that's the reason for your bug report WRT JIT/NCI. The question is, how can we detect the presence of the exec-shield patch. Your `uname -a` doesn't indicate it. So we need: 1) a config test/option/whatever (e.g. mallocing some mem, fill in a ret instruction and call that. 2) Some means to allocate executable memory. Could you please have a look at fedora (kernel) docs? Regards Peter Gibbs EmKel Systems leo
Re: JIT On Win32: A possible future issue
Ah, that's the reason for your bug report WRT JIT/NCI. The question is, how can we detect the presence of the exec-shield patch. Your `uname -a` doesn't indicate it. What for? We just always do allocating memory from a JIT dedicated heap with execute flas set on it, no matter the presence of the exec-shield patch, or OS is windows XP with service pack 2 installed, or the processor supports NX. 1) a config test/option/whatever (e.g. mallocing some mem, fill in a ret instruction and call that. Calling it and BOOM! STATUS_ACCESS_VIOLATION with the following process termination. 2) Some means to allocate executable memory. Plus, methods to create and free the JIT heap. 0x4C56
Re: JIT On Win32: A possible future issue
Leopold Toetsch wrote:
So we need:
1) a config test/option/whatever (e.g. mallocing some mem, fill in a
ret instruction and call that.
2) Some means to allocate executable memory.
Could you please have a look at fedora (kernel) docs?
There are two ways to flag memory as executable:
1) A flag to mmap
2) Calling mprotect after the memory has been allocated
The second method is POSIX-compliant only when used with mmap'd memory.
It does however work with malloc'd memory under Linux, with the caveat
that if the memory is released and reallocated (within the same
process), the executable flag remains set.
As a quick test, I added the following to src/jit.c (based on 'man
mprotect'):
jit_info-native_ptr = jit_info-arena.start =
mem_sys_allocate_zeroed((size_t)jit_info-arena.size);
{
#include limits.h/* for PAGESIZE */
#ifndef PAGESIZE
#define PAGESIZE 4096
#endif
char *p = (char *)((int) jit_info-native_ptr ~(PAGESIZE-1));
size_t len = ((int) jit_info-native_ptr + jit_info-arena.size -
(int)p);
mprotect(p, len, PROT_READ|PROT_WRITE|PROT_EXEC);
}
With this addition, parrot -j runs without segfaulting, so the above
could be used as a basis for a mem_alloc_executable function. Note that
the pointer passed to mprotect must start on a PAGESIZE boundary, so we
get some area of memory marked executable when it need not be.
I don't think we need to check whether exec-shield is active, only that
there is an mprotect function with the appropriate attribute constants.
Regards
Peter Gibbs
EmKel Systems
Re: JIT On Win32: A possible future issue
Vladimir Lipsky [EMAIL PROTECTED] wrote: Ah, that's the reason for your bug report WRT JIT/NCI. The question is, how can we detect the presence of the exec-shield patch. Your `uname -a` doesn't indicate it. What for? We just always do allocating memory from a JIT dedicated heap with execute flas set on it, no matter the presence of the exec-shield patch, I doubt, that my kernel has an appropriate allocation function (besides mmap() with PROT_EXEC - which isn't resizable AFAIK). ... or OS is windows XP with service pack 2 installed, or the processor supports NX. ... or that an older Win32 has the necessary system call to get executable memory. 1) a config test/option/whatever (e.g. mallocing some mem, fill in a ret instruction and call that. Calling it and BOOM! STATUS_ACCESS_VIOLATION with the following process termination. A *config* test, yes may crash, why not. That's giving a result. 2) Some means to allocate executable memory. Plus, methods to create and free the JIT heap. I did propose a set of these already. 0x4C56 leo
Re: JIT On Win32: A possible future issue
Peter Gibbs [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: So we need: 1) a config test/option/whatever (e.g. mallocing some mem, fill in a ret instruction and call that. 2) Some means to allocate executable memory. Could you please have a look at fedora (kernel) docs? There are two ways to flag memory as executable: 1) A flag to mmap 2) Calling mprotect after the memory has been allocated Good. I don't think we need to check whether exec-shield is active, only that there is an mprotect function with the appropriate attribute constants. Considering the alignment effects[1] and the usage of JITted code for NCI too (where much smaller then page-size chunks are used) I'd prefer to do a config check and 1) allocate aligned memory then and 2) use mprotect only, when necessary. Actuall there is already a test (config/auto/jit/test_c.in) which executes code in data memory. So we don't need much more then adapt this test a bit and add the necessary allocation functions. Regards Peter Gibbs EmKel Systems [1] I don't like to poke holes into this security system, by having unaligned chunks of executable memory around. Thanks for your investigations, leo
Re: JIT On Win32: A possible future issue
From: Leopold Toetsch [EMAIL PROTECTED] Vladimir Lipsky [EMAIL PROTECTED] wrote: Ah, that's the reason for your bug report WRT JIT/NCI. The question is, how can we detect the presence of the exec-shield patch. Your `uname -a` doesn't indicate it. What for? We just always do allocating memory from a JIT dedicated heap with execute flas set on it, no matter the presence of the exec-shield patch, I doubt, that my kernel has an appropriate allocation function (besides So do I. Well, there is, of course, HeapCreate on windows but I'm afraid it marks the memory being reserved as READ WRITE and there is no mean to pass the function any flags. mmap() with PROT_EXEC - which isn't resizable AFAIK). Besides VirtualAlloc with PAGE_EXECUTE. As to resizing, yeah it's bad; VirtualAlloc doesn't do resing either. We will have to rewrite all the things that malloc() does with relative to resizing. Bad. Very bad. ... or OS is windows XP with service pack 2 installed, or the processor supports NX. .. or that an older Win32 has the necessary system call to get executable memory. Old windows indeferently executes data from pages not marked as EXECUTE. So there won't be any problems, at least until Microsoft decides to patch them. leo 0x4C56
Re: JIT for Win32
On Fri, 10 May 2002, Aldo Calpini wrote: hello there (and especially the JIT team), is anybody working on JIT-enabling the Win32 platform? Yes, the new JIT will work (actually, it works) on Windows, probably you didn't read the TODO, says '(currently on hold, waiting for JIT v2)', so, as soon as the the code is in you will be able to use the JIT on windows and help us enhancing it :) Regards, Daniel Grunblatt.
Re: JIT for Win32
Daniel Grunblatt wrote: is anybody working on JIT-enabling the Win32 platform? Yes, the new JIT will work (actually, it works) on Windows, probably you didn't read the TODO, says '(currently on hold, waiting for JIT v2)', so, as soon as the the code is in you will be able to use the JIT on windows and help us enhancing it :) oh well... stupid me, I worked on the parrot-0.0.5 distro, just forgot about peeking at the CVS repository :-) it was really nice to work on it anyway. I have learned alot of things, and would like to suggest an 'enhancement' to Configure.pl. if the compiler is 'cl' (that is, Visual C++), and you invoke Configure.pl with --debugging, it should add '-DDEBUG -D_DEBUG -FAcs -Fa -FR' to CFLAGS in the Makefile. I've found that with these switches debugging in Visual C++ is far more pleasant. also, to inspect core_ops.c in the debugger (instead of the not-so-helpful core.ops), I had to comment all the #line directives in core_ops.c. what is your opinion about these changes? if you think they're worthy, I can submit a patch. cheers, Aldo __END__ $_=q,just perl,,s, , another ,,s,$, hacker,,print;
Re: JIT for Win32
On Fri, 10 May 2002, Aldo Calpini wrote: it was really nice to work on it anyway. I have learned alot of things, and would like to suggest an 'enhancement' to Configure.pl. if the compiler is 'cl' (that is, Visual C++), and you invoke Configure.pl with --debugging, it should add '-DDEBUG -D_DEBUG -FAcs -Fa -FR' to CFLAGS in the Makefile. I've found that with these switches debugging in Visual C++ is far more pleasant. also, to inspect core_ops.c in the debugger (instead of the not-so-helpful core.ops), I had to comment all the #line directives in core_ops.c. I hate that too. what is your opinion about these changes? if you think they're worthy, I can submit a patch. My , not so important, opinion is that a patch would be great. Regards, Daniel Grunblatt.
Re: JIT for Win32
At 12:29 PM -0300 5/10/02, Daniel Grunblatt wrote: On Fri, 10 May 2002, Aldo Calpini wrote: also, to inspect core_ops.c in the debugger (instead of the not-so-helpful core.ops), I had to comment all the #line directives in core_ops.c. I hate that too. If you want to throw in a patch to turn of #line directives if an environment variable, say PARROT_NO_LINE, is true, that'd be cool. On second thought, don't bother--I'll go do it now. :) what is your opinion about these changes? if you think they're worthy, I can submit a patch. My , not so important, opinion is that a patch would be great. Heh. You underestimate your opinion. Still, I think a patch would be great as well, so Aldo, if you would, fire away! -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
