[PATCH]interpreter_array_mutex destroying
At the time the main interpreter(one which has no parent) exits, there
could remain some other interpreters being run by detached threads as
in t/pmc/threads_4.pasm. To execute bytecode successfully an
interpreter needs that things like PIO, class vtables, event loop,
class_count_mutex, interpreter_array_mutex, and maybe others are kept
untouched untill the very last interpreter is done. Now
Parrot_really_destroy when is called by the main interpreter shuts
down PIO, kills the event loop, and destroys interpreter_array_mutex
no matter how many interpreters are still active.
The patch provided moves destroying of interpreter_array_mutex into
the sort of commented out with PARROT_DESTROY_FLAG for the main
interpreter part of Parrot_really_destroy and lets a detached thread
to feel free to lock interpreter_array_mutex. With that fix applied,
t/pmc/threads_4.pasm should be ok. The better patch would require
knowledge of a) how many interpreters being alive and b) who heck is
the fery first interpreter, and would do checks like
if ( the_very_last_interpreter )
{
PIO_internal_shutdown( the_very_first_interpreter );
rather than
if ( !interpreter-parent_interpreter )
{
PIO_internal_shutdown( interpreter );
a) interpreter_array traps how many threads are running, and that's
not the same as how many interpreters are running.
b) though the very first interpretere could be derived from following
the parent_interpreter pointer like
interp-parent_interpreter-parent_interpreter, the global variable
would do the job better and woudn't require serializing access to it.
--- parrot/src/events.c Wed Jun 1 13:35:20 2005
+++ parrot-devel/src/events.c Wed Jun 1 13:32:53 2005
@@ -237,10 +237,6 @@
if (event_queue)
PANIC(event queue already exists - missing parent_interp?);
/*
- * we need a global mutex to protect the interpreter array
- */
-MUTEX_INIT(interpreter_array_mutex);
-/*
* create event queue
*/
event_queue = queue_init(TASK_PRIO);
--- parrot/src/inter_create.c Wed Jun 1 13:35:21 2005
+++ parrot-devel/src/inter_create.c Wed Jun 1 13:34:03 2005
@@ -114,6 +114,10 @@
else {
SET_NULL(interpreter-parent_interpreter);
SET_NULL(interpreter-lo_var_ptr);
+/*
+ * we need a global mutex to protect the interpreter array
+ */
+MUTEX_INIT(interpreter_array_mutex);
MUTEX_INIT(class_count_mutex);
}
interpreter-resume_flag = RESUME_INITIAL;
@@ -408,6 +412,7 @@
mem_sys_free(interpreter-evc_func_table);
/* strings, chartype, encodings */
if (!interpreter-parent_interpreter) {
+MUTEX_DESTROY(interpreter_array_mutex);
MUTEX_DESTROY(class_count_mutex);
string_deinit(interpreter);
/*
--- parrot/src/thread.c Wed Jun 1 13:35:21 2005
+++ parrot-devel/src/thread.c Wed Jun 1 13:34:36 2005
@@ -432,7 +432,6 @@
}
}
UNLOCK(interpreter_array_mutex);
-MUTEX_DESTROY(interpreter_array_mutex);
return;
}
[PATCH]Parrot_dod_sweep fix
This patch prevents from an attempt to add a NULL PMC_EXT structure to the pmc ext pool while in dod sweep run. dod.c.patch Description: Binary data
[PATCH]PackFile_destroy fix
Nested packfile segments of type directory are freed twice by default_destroy(). The first time it happens in directory_destroy(), and the second time in PackFile_Segment_destroy(). This behavior is, of course, incorrect, the patch fixes it. packfile.c.patch Description: Binary data
[PATCH]get_mmd_dispatch_type fix
The patch fixes the bug which appeared in revision 7950 mmd.c.patch Description: Binary data
[PATCH]class_count_mutex
class_count_mutex is used without having been initialized. The fixes that. inter_create.c.patch Description: Binary data
Re: [perl #35855] Problem with Perl 5.6.1
I notice that building with Perl 5.6.1 (on Win32 with Perl 5.6.1 ActiveState-build 635 and MinGW) causes problem. $ parrot Assertion failed: (int)io-image-bufused = 0, file src/pmc_freeze.c, line 478 abnormal program termination This assertion occurs in parrot_get_config_string(). The data parrot_config[] (file src/parrot_config.c) is not valid. This data is the frozen version of runtime/parrot/include/config.fpmc runtime/parrot/include/config.fpmc is a transformation of config_lib.pasm And I don't found a real explanation ... On windows, run-time libraries supplied with compilers distinguish between files open in the text and binary mode and the perl 5.6.* default open mode for plain files is text (this is not true as of perl 5.8.*, correct me if I am wrong). So after the following lines are executed $_ = F; my @c = split ''; @c ends up containing only 208 chars, though config.pmc is of 11588 chars. I can't talk about implementation of fread in all the windows compillers, but MS VC's one does as follows: first it slurps the whole file in the buffer, then it checks if is running in the text mode, and if so, moves the pointer to the begin of the buffer and starts scanning it for CR-LF and CTRL-Z. If fread encounters CR-LF, it maps CR-LF to LF, if encounters CTRL-Z it stops the scanning and assigns the position, where CTRL-Z is found at, to the number of bytes read. In this particular case, fread gets CTRL-Z at 209 position, so the script parrot_config_c.pl produces incomplete frozen pmc representation which is of 208 chars only, and that's why assertion fails when thaw_pmc tries to thaw it. The patch applied fixes the problem at least on my PC which runs perl 5.6.2 parrot_config_c.pl.patch Description: Binary data
Re: [perl #35305] [PATCH] skip threads 'detatch' test on win32
jerry gay [EMAIL PROTECTED] wrote: much better! one failing test now... D:\usr\local\parrot-HEAD\trunkperl t/harness t/pmc/threads.t t/pmc/threadsok 3/11# Failed test (t/pmc/threads.t at line 163) # got: 'start 1 # in thread # done # Can't spawn .\parrot.exe D:\usr\local\parrot-HEAD\trunk\t\pmc\threads_4.pasm: Bad file descriptor at lib/Parrot/Test.pm line 231. # ' # expected: 'start 1 t/pmc/threadsNOK 4# in thread # done # ' # '.\parrot.exe D:\usr\local\parrot-HEAD\trunk\t\pmc\threads_4.pasm' failed with exit code 255 Parrot_really_destroy needs to be fixed
Re: [perl #35305] [PATCH] skip threads 'detatch' test on win32
As stated already, this (and possibly other thread) test(s) can't succeed as long as Win32 has no event loop that passes the terminate event on to the running interpreter. 1) Why the heck --- parrot/config/gen/platform/win32/threads.h Mon May 2 14:40:59 2005 +++ parrot-devel/config/gen/platform/win32/threads.h Mon May 2 14:42:58 2005 @@ -0,0 +1,3 @@ + +#include parrot/thr_windows.h + isn't in the repository? 2) To test both cases(MS compiler and not), I played with the macro #ifdef _MCS_VER in thr_windows.h and forgot 1 at the and of it. The patch applied removes it. Though it couldn't affect the test results as long as thr_windows.h wasn't included at all. mcs_ver.patch Description: Binary data
Re: [Patch] Win32 thread primitives
threads_4 is testing killing threads. This is achieved by scheduling a
terminate event to the running interpreter. This can only succeed, if the
event system is running too.
see src/events.c/Parrot_new_terinate_event()
Though thr_windows.h doesn't contain error checking for now, it luckily
fails when the killed thread returns from runops and tries to LOCK the
interpreter_array_mutex. But before that the thread of the main interp had
time to call Parrot_really_destroy, loop interpreter_array, and free
interpreter_array_mutex. Here I see two solutions:
A) do not free the array and mutex asscociated with it at all; those are
global, malloced once, have no destructor, would it be a memory leak?
B) the very last thread do the work, it iterates thru array and if all tids
are null(except itself), free array and mutex.
On the another hand I didnt see that interpreters task queues and mutexes of
the shared pmc's(sub and Interp, probably) were freed up anywhere. It's that
normal?
And on the third hand, if I understood the code correctly ...
src/threads.c: 40: thread_func()
src/threads.c: 53: interpreter-thread_data-state |= THREAD_STATE_FINISHED;
src/threads.c: 312: pt_thread_join ()
src/threads.c: 321: if (interpreter-thread_data-state ==
THREAD_STATE_JOINABLE ||
src/threads.c: 322: interpreter-thread_data-state ==
THREAD_STATE_FINISHED) {
src/threads.c: 453: detach()
src/threads.c: 462: if (interpreter-thread_data-state ==
THREAD_STATE_JOINABLE ||
src/threads.c: 463: interpreter-thread_data-state ==
THREAD_STATE_FINISHED) {
lines 322, 463 never hold true, because of line 53. So pt_thread_join,
detach are never able do their work on threads that have runops-ed.
Argh, yes. Can you just #undef the CONST after including the windows.h?
Ok, done it.
win32_threads.patch
Description: Binary data
Re: [Patch] Win32 thread primitives
And on the third hand, if I understood the code correctly ...
src/threads.c: 40: thread_func()
src/threads.c: 53: interpreter-thread_data-state |=
THREAD_STATE_FINISHED;
src/threads.c: 312: pt_thread_join ()
src/threads.c: 321: if (interpreter-thread_data-state ==
THREAD_STATE_JOINABLE ||
src/threads.c: 322: interpreter-thread_data-state ==
THREAD_STATE_FINISHED) {
src/threads.c: 453: detach()
src/threads.c: 462: if (interpreter-thread_data-state ==
THREAD_STATE_JOINABLE ||
src/threads.c: 463: interpreter-thread_data-state ==
THREAD_STATE_FINISHED) {
lines 322, 463 never hold true, because of line 53. So pt_thread_join,
detach are never able do their work on threads that have runops-ed.
I don't think so: in line 47 the thread enters the runloop, state is
JOINABLE (except when created detached). Another thread, which has the TID
of the thread can now join or detach it. Only when the thread leaves its
runloop, it's state is set to FINISHED. So at e.g. line 322
It's state is set to PREVIOUS_STATE+FINISHED
So it's never equal to just FINISHED
src/threads.c: 53: interpreter-thread_data-state |=
THREAD_STATE_FINISHED;
src/threads.c: 322: interpreter-thread_data-state ==
THREAD_STATE_FINISHED) {
Typo or what?
[Patch] Win32 thread primitives
This patch defines Win32 thread primitives. Actually it consists of the following files: threads.h.diff thr_windows.h.diff threads.t.diff timer.t.diff The patch had been applied, I mananged to pass all the tests from t/pmc/thread.reast and t/pmc/timer.t but thread_4.pasm(don't know yet why it fails) The other files are needed the patch to compile and work properly: events.c.diff - contains fix for correct deteremination of the tv_nsec value. Now it is always equal to zero. imcc.l.diff, imcc.y.diff et al - contain a dirty hack which prefixes the CONST lexeme with underscore, otherwise CONST conflicts with MACRO from windows header files. Probably the solution wwould be to prefix all the lexemes with IMCC_ or whatever. Prefixing should be done for thread primitives either. events.c.diff Description: Binary data imcc.l.diff Description: Binary data imcc.y.diff Description: Binary data imclexer.c.diff Description: Binary data imcparser.c.diff Description: Binary data imcparser.h.diff Description: Binary data main.c.diff Description: Binary data thr_windows.h.diff Description: Binary data threads.h.diff Description: Binary data threads.t.diff Description: Binary data timer.t.diff Description: Binary data
Re: Prothon 0.1.2 is getting close to Alpha [Prothon]
3) Is there someone on the develpment team who could hold my hand in the beginning to get me going with Parrot in Visual Studio? After a brief There isn't a certain someone. Just put your question on the list. Surely it won't remain unaswered for long, at least as regards the configuringmaking sort of questions period I'm sure I could not only stand on my own but I could become a main support point for Parrot on win32/VS. Mmmm ... stuff like threads, porting signals, async IO is still waiting for a man who isn't that coy and ready to chop off his nice-sized piece of flame 0x4C56
Re: [PATCH] IO fixes for Win32
From: Goplat [EMAIL PROTECTED] flags_to_win32 sets fdwShareMode to FILE_SHARE_DELETE, which is not supported in win98 and will cause the CreateFile to fail, so the ParrotIO is NULL, and A fix for that should be windows version specific and needs support of the config subsystem. The logic is as follows: In the config subsystem ... 1) run a test which defines the os version 2) set the compiler option up if version is windows 95 add -D_WIN32_WINDOWS=0x0400 to cflags else if version is windows 98 then add -D_WIN32_WINDOWS=0x0410 to cflags else if version is windows Me then add -D_WIN32_WINDOWS=0x0500 to cflags else if version is winnt 4.0 then add -D_WIN32_WINNT=0x0400 to cflags else if version is windows 2000 then add -D_WIN32_WINNT=0x0500 to cflags else if version is windows XP then add -D_WIN32_WINNT=0x0501 to cflags else if version is windows server 2003 then add -D_WIN32_WINNT=0x0502 to cflags end In io.h ... #ifdef _WIN32 # if defined(_WIN32_WINDOWS) (_WIN32_WINDOWS = 0x0500) #define PIO_WIN32_SHARE_MODE (FILE_SHARE_READ |\ # FILE_SHARE_WRITE) # else #define PIO_WIN32_SHARE_MODE (FILE_SHARE_READ |\ # FILE_SHARE_WRITE |\ # FILE_SHARE_DELETE) #endif 0x4C56
Re: [PATCH] IO fixes for Win32
From: Vladimir Lipsky [EMAIL PROTECTED] A fix for that should be windows version specific and needs support of the config subsystem. And 0f course in io_win32.c *fdwShareMode = PIO_WIN32_SHARE_MODE; 0x4C56 0x4C56
Re: [PATCH] IO fixes for Win32
From: Goplat [EMAIL PROTECTED] --- Vladimir Lipsky [EMAIL PROTECTED] wrote: From: Goplat [EMAIL PROTECTED] flags_to_win32 sets fdwShareMode to FILE_SHARE_DELETE, which is not supported in win98 A fix for that should be windows version specific and needs support of the config subsystem. If you did that, a version compiled on NT wouldn't work on 9x. I'd say most 9x users don't compile perl themself, they just download a binary from ActiveState (who use NT)... Then there should be different binaries for different versions If you really need FILE_SHARE_DELETE that badly, the check would have to be Yeah and async I/O, and other stuff that aren't avaible in perl5 and windows 95/98/Me/CE. If we didn't yet distinguish windows versions, it's the time to do so. done at runtime. I don't think it's all that necessary though... in fact perl 5 only uses FILE_SHARE_READ, and that only if the file's not opened for write/append. Does perl5 use FILE_SHARE_READ when + and + modes are specified? In any case what you said sounds like a bug to me, since this policy restricts other threads(even threads of the same process) to access the file in the indentical mode.
Re: [PATCH] IO fixes for Win32
From: Goplat [EMAIL PROTECTED] If you really need FILE_SHARE_DELETE that badly, the check would have to be done at runtime. I don't think it's all that necessary though... in fact perl 5 only uses FILE_SHARE_READ, and that only if the file's not opened for write/append. What's more, a value of the parameter CdwDesiredAccess doesn't depend on a value the parameter CdwShareMode and shouldn't do. 0x4C56
Re: cvs commit: parrot/classes perlscalar.pmc
From: Leopold Toetsch [EMAIL PROTECTED] Seems that we got a problem on alphas then. I can see several solutions to accomodate such CPUs: From my point of view, these solutions have the following merits and demerits: - use only PMCs that don't cross cache lines +) No need for memory barriers to preserve the order of read and write operations +) Lock-free read access, and thus, no CPU perfomance degradation -) This would bloat the PMC size - insert rmb_if_needed() for all string vtables that don't lock +) No constraint on PMC's alignment -) In the first place, this variant is a big headache -- it's problematic to say on which architectures such code protection has to be aplied. Even compilers, x86-, and AMD-based hardware do harmless reodering to speed the things up. In addition, you never know which memory ordering model future architectures are going to choose. For example, on Itanium-based hardware, it appears that read and write ordering is not preserved when seen from the perspectives of different processors. It's not only Alpha is that weird on this point. -) It might turn out that not all OSes provide rmb()-like primitives. For instance, KeMemoryBarrierXxx, the windows API functions, are defined in the DDK (Driver Development Kit) only for Windows Server 2003. So they aren't portbale across all Windows and plus defined in the DDK that stands for It's not your business, software develover. What Microsoft advises software developers is sort of When the order is important, use standard operating system locking mechanisms whenever possible. The standard mechanisms have implied memory barriers -) Barriers incur pipeline flushes, the overhead of which increases with pipeline length; ditto to the following variant - lock all string vtable access +) No need for memory barriers, since operating system's locking mechanisms all have implied those -) Acquiring a lock is a damn waste of time, the acquiring CPU is forced to stall until the lock is released and reaches its cache. - move string pointer into PMC_EXT +-) Being a nice idea on the whole, it's a lop-sided variant concerning the other data primitives that a PMC can hold -) It will break the current implementation of perstring.pmc, sarray.pmc, scalar.pmc, and probably other code and pmc classes -) Besides that I might expect that there are more disvantages Summing up the above said, I would vote for use only PMCs that don't cross cache lines leo 0x4C56
Re: cvs commit: parrot/classes perlscalar.pmc
From: Leopold Toetsch [EMAIL PROTECTED] That isn't necessary. One rmb() after resetting Cstr_val ought to be enough. It ensures that in all cases the other CPU reads str_val as NULL. If the vtable is still pointing to the PerlInt vtable, its a user Having an only rmb() on the writing CPU hardly assures that. http://lse.sourceforge.net/locking/wmbdd.html http://www.ussg.iu.edu/hypermail/linux/kernel/0007.3/0481.html http://www.ussg.iu.edu/hypermail/linux/kernel/0110.1/0409.html Yep. But don't care: i386 doesn't need these memory barriers. i386 is not the sole CPU architecture which runs windows. 0x4C56
Re: cvs commit: parrot/classes perlscalar.pmc
Gordon Henriksen [EMAIL PROTECTED] wrote: On Friday, January 30, 2004, at 11:52 , Leopold Toetsch wrote: + /* + * if we morph to a string, first clear str_val + * so that after changing the vtable a parallel + * reader doesn't get a gargabe pointer + */ This is fine on a uniprocessor, but it's not enough on a multiprocessor. There, you need a sync instruction (on a PPC; equivalent elsewhere). It might be not on enough on architectures that don't manage cache lines validity. PPC's Csync seems to be used for managing the coherency of storage shared by the processor and a system component(e.g. I/O device) which doesn't participate in the storage coherency protocol. So, once an architecture defines cache lines in the protocol, the plan would work. Linux rmb() is likely to be a nop on such architecures. 0x4C56
Re: cvs commit: parrot/classes perlscalar.pmc
From: Leopold Toetsch [EMAIL PROTECTED] Gordon Henriksen [EMAIL PROTECTED] wrote: On Friday, January 30, 2004, at 11:52 , Leopold Toetsch wrote: + /* + * if we morph to a string, first clear str_val + * so that after changing the vtable a parallel + * reader doesn't get a gargabe pointer + */ This is fine on a uniprocessor, but it's not enough on a multiprocessor. There, you need a sync instruction (on a PPC; equivalent elsewhere). Yep, that's right. As our PMC size isn't a power of 2, there is a small chance that Cvtable and Cstr_val are in different cache lines and Even if the PMC size were a power of two, it woudn't necessitate Cvtable and Cstr_val being in the same cache line. only the change to the Cvtable is seen by another CPU. For ultimate correctness[1], we would need a Crmb() [2] instruction here. And the wmb() primitive there to ensure that the invalidate corresponding to reseting Cstr_val will appear on the bus ahead of the morphing vtable's invalidate. So the reading CPU can process these invalidates with rbm() in right order. Writing CPU | Reading CPU -|-- lock() | reset Cstr_val | wbm()| morph() | unlock() | | rbm() | get Cstr_val But I don't see a big problem here, we could just configure and insert a Csmp_rmb() from F$linux_kernel_src/include/asm-$arch/system.h. You're lucky, linux folks. leo 0x4C56
Re: Docs and releases
Well, there is always up-to-date documentation, your debugger output ... 0x4C56 Who says that the copy-paste antipattern is bad?
Re: More Windows dev questions: Core dumps
From: Dan Sugalski [EMAIL PROTECTED] On a Unix system, a core dump is a file with a raw (mostly) copy of a process' current memory image that's written whenever a process does something profoundly illegal, like accessing an inaccessible section of memory with no trap handler that allows recovery or something of the sort. They're handy, because it means you can go and debug a process after the fact, and don't have to be there on the spot to figure out what's going on. Really handy for things that die overnight, or when you're not around, and you don't want the defunct process hanging around until you can attach to it with a debugger. Does Windows do this? (I know other OSes, like VMS, do *not* do it) No, Windows does not use core files. If you run your code outside of a debugger and a problem occurs you will be given the option of either debugging the code or killing the executing process. If so, how do I enable it? If not, I presume there's some reasonably simple way to attach a debugger to a process that's died. (I hope) Yeah. That's it. A debugger. Do you have one? If no, try to pull WinDbg about. http://www.microsoft.com/whdc/ddk/debugging/default.mspx?gssnb=1 It's free for downloading. You will find some HOW TO there either. No, I'm not getting turned into a windows programmer, but until I get It's a pity that you ain't =) Dan 0x4C56
Re: More Windows dev questions: Core dumps
0x4C56 - Original Message - From: Dan Sugalski [EMAIL PROTECTED] To: Vladimir Lipsky [EMAIL PROTECTED] Cc: perl6-internals [EMAIL PROTECTED] Sent: Wednesday, December 31, 2003 10:49 PM Subject: Re: More Windows dev questions: Core dumps At 11:51 PM +0300 12/31/03, Vladimir Lipsky wrote: From: Dan Sugalski [EMAIL PROTECTED] On a Unix system, a core dump is a file with a raw (mostly) copy of a process' current memory image that's written whenever a process does something profoundly illegal, like accessing an inaccessible section of memory with no trap handler that allows recovery or something of the sort. They're handy, because it means you can go and debug a process after the fact, and don't have to be there on the spot to figure out what's going on. Really handy for things that die overnight, or when you're not around, and you don't want the defunct process hanging around until you can attach to it with a debugger. Does Windows do this? (I know other OSes, like VMS, do *not* do it) No, Windows does not use core files. If you run your code outside of a debugger and a problem occurs you will be given the option of either debugging the code or killing the executing process. Bummer. Ah, well, one of the few Unix development things that I really like. Well, debug prompts not that bad. Still better than nothing. If so, how do I enable it? If not, I presume there's some reasonably simple way to attach a debugger to a process that's died. (I hope) Yeah. That's it. A debugger. Do you have one? If no, try to pull WinDbg about. http://www.microsoft.com/whdc/ddk/debugging/default.mspx?gssnb=1 It's free for downloading. You will find some HOW TO there either. I've probably got that installed already with the VS/.NET kit, but if Oh! Then you might have got it. I read somewhere that one can do a crash dump, an windows equivalent of unix core dumpes, with windbg. So if you wish, you could look for how to create that thing in docs. Dan 0x4C56
Re: Need some deep non-POSIX thread info (Attention you Windows folks!)
From: Dan Sugalski [EMAIL PROTECTED] So, could someone with some windows experience go digging and find out how one would: 1) Find the address of the base of a thread's stack 3) Find out what a thread's current stack pointer is I would do 1), 3) this way ... thdl = _beginthreadex(NULL, 0, thread_function, arg, CREATE_SUSPENDED, thraddr); #if defined(_X86_) ctx.ContextFlags = CONTEXT_FULL; #elif defined(_ALPHA_) ctx.ContextFlags = CONTEXT_INTEGER; #else /* _MIPS_ and etc */ #endif GetThreadContext(thdl, ctx); #if defined(_X86_) current_stack_ptr = ctx.Esp; #elif defined(_ALPHA_) current_stack_ptr = ctx.IntSp; #else /* _MIPS_ and etc */ #endif VirtualQuery((LPCVOID)current_stack_ptr, mbi, sizeof(mbi)); stack_allocbase_addr = mbi.AllocationBase; 2) Find out how big a thread's stack is By default, OS reserves 1 MB of memory for a thread's stack. One can specify a different size with a linker option or a STACKSIZE statement in the .DEF file. So it's, sort of, up to us how big a thread stack is. No problem here. Dan 0x4C56
Re: Need some deep non-POSIX thread info (Attention you Windows folks!)
And a note for 3): It's importatnt to create a thread with CREATE_SUSPENDED, and at thread runtime we have to suspend thread while checking out its registers so that to get the true values. SuspendThread(thdl); GetThreadContext(thdl, ctx); ... ResumeThread(thdl); 0x4C56
Re: [perl #24769] [PATCH] mem_sys_allocate_executable - initial draft
From: Jonathan Worthington [EMAIL PROTECTED]
Here is my attempt at a patch for executable memory allocation, which
makes
+void *
+mem_alloc_executable(size_t size)
+{
+ void *ptr = VirtualAlloc(NULL, size, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
^^^
MEM_RESERVE | MEM_COMMIT
or more preferrable MEM_RESERVE | MEM_TOP_DOWN | MEM_COMMIT
+ if (memInfo.RegionSize = size)
+ {
+ CopyMemory(newBlock, memblock, memInfo.RegionSize);
+ }
+ else
+ {
+ CopyMemory(newBlock, memblock, size);
+ }
memInfo.RegionSize is always either greater or equal size. So just
CopyMemory(newBlock, memblock, size);
would be enough
+void
+mem_free_executable(void *addr)
+{
+ /* We need to decommit, then release the pages. */
+ VirtualFree(addr, 1, MEM_DECOMMIT); No need
+ VirtualFree(addr, 0, MEM_RELEASE);
+}
There is no need to decommit pages if next you release the region.
0x4C56
Re: JIT On Win32: A possible future issue
Ah, that's the reason for your bug report WRT JIT/NCI. The question is, how can we detect the presence of the exec-shield patch. Your `uname -a` doesn't indicate it. What for? We just always do allocating memory from a JIT dedicated heap with execute flas set on it, no matter the presence of the exec-shield patch, or OS is windows XP with service pack 2 installed, or the processor supports NX. 1) a config test/option/whatever (e.g. mallocing some mem, fill in a ret instruction and call that. Calling it and BOOM! STATUS_ACCESS_VIOLATION with the following process termination. 2) Some means to allocate executable memory. Plus, methods to create and free the JIT heap. 0x4C56
Re: JIT On Win32: A possible future issue
From: Leopold Toetsch [EMAIL PROTECTED] Vladimir Lipsky [EMAIL PROTECTED] wrote: Ah, that's the reason for your bug report WRT JIT/NCI. The question is, how can we detect the presence of the exec-shield patch. Your `uname -a` doesn't indicate it. What for? We just always do allocating memory from a JIT dedicated heap with execute flas set on it, no matter the presence of the exec-shield patch, I doubt, that my kernel has an appropriate allocation function (besides So do I. Well, there is, of course, HeapCreate on windows but I'm afraid it marks the memory being reserved as READ WRITE and there is no mean to pass the function any flags. mmap() with PROT_EXEC - which isn't resizable AFAIK). Besides VirtualAlloc with PAGE_EXECUTE. As to resizing, yeah it's bad; VirtualAlloc doesn't do resing either. We will have to rewrite all the things that malloc() does with relative to resizing. Bad. Very bad. ... or OS is windows XP with service pack 2 installed, or the processor supports NX. .. or that an older Win32 has the necessary system call to get executable memory. Old windows indeferently executes data from pages not marked as EXECUTE. So there won't be any problems, at least until Microsoft decides to patch them. leo 0x4C56
[PATCH] Embedding interface change
Beware :) This patch changes Parrot_loadbc to Parrot_set_packfile. 0x4C56 diff Description: Binary data
Re: pthreads-win32
From: Dan Sugalski [EMAIL PROTECTED] At 2:01 PM +0100 12/14/03, Leopold Toetsch wrote: Can we use this lib: http://sources.redhat.com/pthreads-win32/ citedistributed under the GNU Lesser General Public License (LGPL)/cite I think so, yes. I'd rather use windows native threading if we can If by native threading you mean bare Win32 API functions, I'd rather use the multithread version of the Microsoft C Run Time library. First of all, 'cause we have already been linking against it, and the second, it provides a proper initialization for threads, and the third, it's been supporting by Microsoft, not by a single man. Well, I surmise that the pthreads lib either uses native threading, the main difference beetween CRT and pthreads being that the later just is a new wheel conforming the POSIX standard Hope the pthreads library won't turn out a way to go for Parrot. 0x4C56
Re: Incorrect scoping of constants in IMCC
From: Melvin Smith [EMAIL PROTECTED] At 04:20 PM 12/9/2003 -0500, Dan Sugalski wrote: which is .included) and visible through the rest of the compilation unit. Parser will not allow .const outside of a compilation unit and make it global to the compilation. Hmm... What do you mean by a compilation unit? a file? Will it be global to all compilation units or rather to the one where the .const was defined? 0x4C56 Happy .~. /V\ // \\ /( )\ ^`~'^ 2004 keeps comin'
Re: Segfault on Win32, the
From: Leopold Toetsch [EMAIL PROTECTED] Did you recheck that after my fix to vtable freeing? My patch covered encoding and chartype too. No, I didn't. Did you fix the Parrot_loadbc function name? If didn't, I'll have a go for Parrot_set_PackFile() leo 0x4C56 Happy .~. /V\ // \\ /( )\ ^`~'^ 2004 keeps comin'
Re: Segfault on Win32, the
Some time ago I mentioned I'd been gettin' one more segfault while the Parrot debug mode was on. In fact, it proved to be an effect of the same cause -- we have global resources sharable among interpreters, that we try to free more than once. This time it's encoding_array. If you're going to disable the encoding_array freeing for now, just make sure you disable the chartype_array one either. 0x4C56
Just a thought
Have a look at the following code fragment Parrot_PackFile pf; char *bc = file.pbc; pf = Parrot_readbc(interpreter, bc); Parrot_loadbc(interpreter, pf); Did you catch the difference between the 2nd actual parameter and the function name? Maybe it's worth renaming? E.g. Parrot_loadpf() 0x4C56
Re: Segfault on Win32, The
From: Dan Sugalski [EMAIL PROTECTED] Sent: Tuesday, December 02, 2003 5:27 PM Well, bah. I'll disable the table freeing for now and that should take care of the bug at the moment, though it certainly won't cure the underlying memory leak. Arguably, and this is getting into the realm of embedding, since this is something that's set up by the global initialization, it should stay in existence until the embedder calls the global destruction code What about a counter of the vtable users(interpreters)? , since it's possible (though I'm not sure we want to support it) that after destroying all the extant interpreters that the embedding program will want to create another one. As it has already been told, it's no problem -- you just call make_interpreter() and all the initialization is done for your new interpreter all over again including creating a vtable 0x4C56
Re: Segfault on Win32, The
From: Leopold Toetsch [EMAIL PROTECTED] Sent: Wednesday, December 03, 2003 12:05 PM The bug is already fixed. But we have to separate interpreter globals and real globals finally. Some time ago, I suggested to change the init functions slightly to take a parent interpreter argument: Do you think you could clarify in what way it relates to separating different sorts of globals. Thanks in advance. leo 0x4C56
Re: Segfault on Win32, The
From: Leopold Toetsch [EMAIL PROTECTED] Sent: Thursday, November 27, 2003 12:04 PM PMC (in gdb speak): b dod.c:525 r t/pmc/pmc_61.pasm p *(PMC *) b p *((PMC *) b)-vtable c ... Umm .. the problem is that this pmc doesn't have the vtable; (*((PMC *)b)).vtable points to a block of memory which belonged to the child interpreter's pool and has already been freed, I deduce from examining the adresses. So it ends up with Access Violation in ntdll!RtlFreeHeap when VTABLE_destroy is called. Now I'm just trying to find out what type of a pmc it is, and track down when the pmc is created. WRT type, all what I know is ((*((Buffer *)b)).obj).flags = 72353280(PObj_is_PMC_FLAG|PObj_is_PMC_EXT_FLAG| PObj_custom_mark_FLAG|PObj_active_destroy_FLAG| b_PObj_is_special_PMC_FLAG) Do you know what type could have these flags set up? I'm no square but it could be the managedstruct type. And one more thing to say, I get one more segfault there (dod.c, line 525) while testing with interpreter.flags | PARROT_DEBUG_FLAG [1] valgrind has no hint about a problem with this test. what does valgrind exactly do? leo 0x4C56 [1] Parrot_set_flag(interpreter, PARROT_DEBUG_FLAG);
Re: clean checkout on Win2K
t\pmc\pmc...NOK 62# Failed test (t\pmc\pmc.t at line 1497) # got: 'All names and ids ok. # Can't spawn .\parrot.exe --gc-debug -b t\pmc\pmc_62.pasm: Bad file descripto r at lib/Parrot/Test.pm line 62. I've no clue, what's going on here. Does the test-file exist after make test? Can you run above command manually? It's just a common response of the Test::Harness thing to the segfault. We simply need to fix test #61 0x4C56
Re: Segfault on Win32, The
From: Leopold Toetsch [EMAIL PROTECTED] Sent: Monday, December 01, 2003 5:47 PM Which child interpreter? Parrot_destroy_vtable() is called after free_unused_pobjects(). Okay. I'll try to reword the problem all over again. Well you know test 61 in t/pmc/pmc.t causes a segfault. To get things more clear, I'll just simplify the code of test 61 and only extract the problematic place. new P0, .ParrotInterpreter end At the current time being, the above code ends up with the segfault on Win32. When I refered to a child interpreter, I meant the interpreter being created by the new opcode. Now I'll outline you some apprx. addresses I have: For the parent(0x002f3c88) interpreter: (*(*(*interpreter).arena_base).pmc_pool).start_arena_memory 0x002fb7f0 (*(*(*interpreter).arena_base).pmc_pool).end_arena_memory 0x002fe7f0 (*(*(*interpreter).arena_base).pmc_ext_pool).start_arena_memory 0x00531f80 (*(*(*interpreter).arena_base).pmc_ext_pool).end_arena_memory 0x00535f80 For the child(0x00544a90) interpreter: (*(*(*new_interp).arena_base).pmc_pool).start_arena_memory 0x00565c90 (*(*(*new_interp).arena_base).pmc_pool).end_arena_memory 0x00568c90 (*(*(*new_interp).arena_base).pmc_ext_pool).start_arena_memory 0x00568ce0 (*(*(*new_interp).arena_base).pmc_ext_pool).end_arena_memory 0x0056cce0 Before the segfault happens: Parrot_base_vtables[0x28] 0x0053cbb0 (*(Parrot_base_vtables[0x28])).base_type 0x0028 (SArray) (*(Parrot_base_vtables[0x28])).destroy 0x0049a2b0 Parrot_SArray_destroy(Parrot_Interp *, PMC *) When segfault happens: (PMC *)b 0x002fdc80 (*((PMC *)b)).vtable 0x0053cbb0 (*((PMC *)b)).pmc_ext 0x00535ab0 Parrot_base_vtables[0x28] 0x0053cbb0 (*(Parrot_base_vtables[0x28])).base_type CXX0030: Error: expression cannot be evaluated (*(Parrot_base_vtables[0x28])).destroy CXX0030: Error: expression cannot be evaluated So a big question raises up: Do interpreter share the same copy of Parrot_base_vtables? Though I know the answer -- they do. Now if you consider the following LIFO queue of parrot_on_exit registered functions Parrot_really_destroy(int 0x, void * 0x00544a90) line 953 print_profile(int 0x, void * 0x002f3c88) line 375 print_debug(int 0x, void * 0x002f3c88) line 433 Parrot_really_destroy(int 0x, void * 0x002f3c88) line 953 the bug will get as clear as ... I'm out of english words! I'm done with it for now and I'm not the dan to say which way is the parrot way to fix this bug 0x4C56
Re: Object stuff
From: Dan Sugalski [EMAIL PROTECTED] Sent: Monday, December 01, 2003 9:53 PM *) I've made the Parrot_base_vtable array movable again, as it needs to be resized. This is a temporary hack, since there are horrible threading issues here. (Not to mention the fact that this table is global and really ought not be, or we need a per-interpreter-family version that's actually used) Is actually used? D'oh. I didn't notice that today. 0x4C56
Re: Segfault warning boxes on Win32
At Wed, 26 Nov 2003 05:20:50 +0300, Vladimir Lipsky wrote: If it blocks the automated test builds, then just turning the box off is okay. It must be admitted that, the truth is that I don't know anybody who does send those reports More important -- is the Segfault reported in any other manner. If not, thats bad. It's not, I'm afraid 0x4C56
Segfault on Win32, The
Well, currently, test 61 in t/pmc/pmc.t causes an abrupt stop of
the test session with the segfault box popping up. I haven't yet
taken a time to figure out what actually is the cause of trouble,
but I've already detected the precise place where that all happens.
It happens here
void
free_unused_pobjects(struct Parrot_Interp *interpreter,
struct Small_Object_Pool *pool)
{
struct Small_Object_Arena *cur_arena;
...
/* if object is a PMC and needs destroying */
if (PObj_is_PMC_TEST(b)) {
/* then destroy it here
*/
if (PObj_active_destroy_TEST(b))
VTABLE_destroy(interpreter, (PMC *)b);
^
while testing the ParrotInterpreter PMC when DOD calls a destroy method,
the buffer being passed contains an invalid reference and refering to it
produces an access violation.
Below is the call stack
free_unused_pobjects(Parrot_Interp * 0x002f39f8, Small_Object_Pool *
0x002f4340) line 525 + 14 bytes
Parrot_really_destroy(int 0, void * 0x002f39f8) line 964 + 22 bytes
Parrot_exit(int 0) line 48 + 16 bytes
main(int 1, char * * 0x002f2bb4) line 555 + 7 bytes
PARROT! mainCRTStartup + 227 bytes
KERNEL32! 77e9ca90()
It would be a time-saver for me to fix the bug if you lay out
some thoughts on this. So feel free to push the reply button.
0x4C56
[PATCH] pio_win32_open fix
Hey unix folks, does test 10 in t/src/io.t succed on your OS? 0x4c56 io_win32.diff Description: Binary data
Re: PIO_eof
Currently it isn't prohibitied. That is the only way to detect an error
from PIO_open* at this time.
io = PIO_open(interpreter, NULL, file[i], flags[j]);
if ( (PIO_eof(interpreter, io) ? 0:1) != expected[i][j] )
{
0x4c56
Re: Segfault warning boxes on Win32
If it blocks the automated test builds, then just turning the box off is okay. It must be admitted that, the truth is that I don't know anybody who does send those reports 0x4C56 - Original Message - From: Dan Sugalski [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, November 25, 2003 7:58 PM Subject: Segfault warning boxes on Win32 Folks, I just committed a patch to disable the Your program has crashed, want to send a report to Microsoft box that pops up on Win32 when Parrot segfaults -- it blocks the automated test builds which as you might expect happens with some regularity. :) However I'm not sure if this ought be the default for parrot. (Though definitely an option) Not being a Windows Guy, I'll defer the question to folks that do windows programming -- what should the default be (on or off) and why? Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
PIO_eof
Hi everyone! In t/src/io.c, specifically test 9 and 10, I wonder if the PIO_eof check up works anywhere; because I didn't manage to find any place where the PIO_F_EOF flag is set up when the PIO_*_open function fails neither in io_stdio.c, io_unix.c nor io_win32.c Is the io == NULL testing prohibited in Parrot embedding system? 0x4c56
Win32 building
D:\build\parrotnmake ... d:\build\parrot\src\encoding.c(39) : warning C4090: 'function' : different 'const' qualifiers d:\build\parrot\src\encoding.c(39) : warning C4022: 'mem_sys_free' : pointer mismatch for actual parameter 1 ... d:\build\parrot\src\chartype.c(231) : warning C4090: 'function' : different 'const' qualifiers d:\build\parrot\src\chartype.c(231) : warning C4028: formal parameter 3 different from declaration d:\build\parrot\src\chartype.c(296) : warning C4028: formal parameter 3 different from declaration d:\build\parrot\src\chartype.c(314) : warning C4028: formal parameter 3 different from declaration ... d:\build\parrot\src\hash.c(443) : warning C4098: 'new_hash' : 'void' function returning a value d:\build\parrot\src\hash.c(455) : warning C4098: 'new_cstring_hash' : 'void' function returning a value ... d:\build\parrot\io\io_stdio.c(250) : warning C4761: integral size mismatch in argument; conversion supplied ... d:\build\parrot\imcc\imc.c(59) : warning C4700: local variable 'ins_next' used without having been initialized d:\build\parrot\imcc\imc.c(60) : warning C4700: local variable 'ins' used without having been initialized ... 0x4c56
