Re: Docs and releases
* Harry ([EMAIL PROTECTED]) [040118 05:06]: --- Paul Cochrane [EMAIL PROTECTED] wrote: Yeah, I don't think you're going crazy. The funny thing is that about a three weeks ago my cvs checkout worked fine. I did a cvs update, and then Configure.pl didn't work[1], so I re-checkedout the whole source tree again, and basically that was when the problems started. So, my guess is that something snuck in only recently. I'm using Gentoo linux, and I'm not going to install Yellowdog or whatever just to sort this problem out. (sorry). Chromatic has currently got the exact same problem as you now and Leo is trying to help get it sorted. Email the list if you are still getting the problem with version numbers of gcc etc. I know I'm a bit late replying, but yeah, parrot hangs on the tests on my machine too, although, where it once hung on the arithmetic tests, it now hangs on the t/src/extend test. System info: Gentoo Linux PPC uname -a: Linux snip 2.4.19-r6 #7 Tue Apr 22 16:54:53 EST 2003 ppc 740/750 GNU/Linux gcc -v: gcc version 2.95.3 20010315 (release) perl -v: This is perl, v5.8.0 built for powerpc-linux I hope that this information is of use. If anyone wants any more info, or more detailed stuff, then just let me know. Later Paul
Re: Docs and releases
* Andrew Dougherty ([EMAIL PROTECTED]) [040120 02:19]: Wordsize errors are one common type of error that show up on PPC (and SPARC) more readily than on x86, due to byte-order issues. When reporting problems, it's often a good idea to include the ./myconfig file in the parrot build directory -- it includes information about the variable types and sizes used in building parrot. These sizes may depend on your gcc version, how it was configured, and how it was called. Ok. I should have put this into my reply to Harry Jackson, but here 'tis anyway. cat myconfig: Summary of my parrot 0.0.13 configuration: configdate='Tue Jan 20 17:51:28 2004' Platform: osname=linux, archname=powerpc-linux jitcapable=1, jitarchname=ppc-linux, jitosname=LINUX, jitcpuarch=ppc execcapable=1 perl=/usr/bin/perl Compiler: cc='gcc', ccflags='-DPERL5 -DDEBUGGING -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', Linker and Libraries: ld='gcc', ldflags=' -L/usr/local/lib', cc_ldflags='', libs='-lpthread -lnsl -ldl -lm -lcrypt -lutil' Dynamic Linking: so='.so', ld_shared='-shared -L/usr/local/lib', ld_shared_flags='' Types: iv=long, intvalsize=4, intsize=4, opcode_t=long, opcode_t_size=4, ptrsize=4, ptr_alignment=4 byteorder=4321, nv=double, numvalsize=8, doublesize=8 Later Paul
Re: Start of thread proposal
Dan Sugalski [EMAIL PROTECTED] wrote: =head2 Guarantees Maybe this isn't strictly a threading thing, but are we going to make any guarantees about event orderings? For example, will we guarantee that a sequence of events send from one thread to another will always be received in the order they are sent? (Obviously no guarantees about interleaving of events from other threads). This is usually only important in distributed environments where multiple paths exists between a pair of endpoints, but it would be nice for user-code to be able to rely on it. Dave.
Re: Start of thread proposal
=item MUTEX This is a low level, under the hood, not exposed to users, thing that can be locked. They're non-recursive, non-read/write, exclusive things. When a thread gets a mutex, any other attempt to get that mutex will block until the owning thread releases the mutex. The platform-native lock construct will be used for this. Will this be macroised to hide the platform native implementation from the main body of the code? The sleep until something pings me construct. Useful for queue construction. Not conceptually associated with a MUTEX. (POSIX threads require this, so we're going to hide it there behind macros and/or functions) Could you elaborate on the nature of what would constitute a ping? =item POSIX threads The threading scheme must be sufficient to support a POSIX share-everything style of threading, such as is used in perl 5's pthread model, as well as the thread models for Ruby and Python. Thankyou:) =item Process-type' threads The scheme must also support the perl 5 iThreads threading model. In this model no data is shared implicitly, and all sharing must be done on purpose and explicitly. It very much resembles the Unix fork-process-with-shared-memory-segment model, not a surprise as it was originally developed with Pseudo forks? An interesting extract from the Windows Services for Unix docs. The Interix subsystem shows improvements in all aspects of performanceranging from 30 percent improvements in fork and exec performance... It's sales pitch, but I'm trying to verify it (and build parrot using it). =item System memory allocation routines are threadsafe We are assuming that the memory allocation system of the base OS is threadsafe. While some of the C runtime libraries are notoriously thread dangerous, memory allocation code generally is threadsafe, and we'll assume that on all platforms. (Though we will, in general, manage our own memory) Is there any likelyhood that memory allocation will be hidden behind macros at two levels: - ALLOCPOOL() for allocating large chunks of memory (ppols) that are later sub-allocated (and managed) by: - SUBALLOC() for sub allocating within a pool ? =item Each interpreter will have a unique id This ID will be independent of the process or OS thread, and will be constant across the lifetime of the interpreter. Interpreter IDs Imay be reused as interpreters are destroyed and recreated, and as such are only guaranteed valid while an interpreter is in use. The provision of method(s) to obtain the native TIDs/HANDLES would make life for those writing implementation specific extensions easier. [SNIP] - Too much to understand before commenting... --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk Nigel
Re: PDB [Was: Re: Calling conventions, IMC]
Will Coleda [EMAIL PROTECTED] wrote: bash-2.05a$ make exit ../../../parrot ../tcl.pbc exit.tcl branch_cs: illegal resume offsetmake: *** [exit] Error 1 This is very likely an undefined label. imcc tries to find that at runtime (it could have come out from eval) and fails. So, I'm trying to track down where this badness is happening: bash-2.05a$ ../../pdb tcl.pbc Parrot Debugger 0.0.2 The pdb isn't up yet, to solve such problems. Try to trace it: $ parrot -t tcl.pbc exit.tcl leo
Re: Memory corruption
Steve Fink [EMAIL PROTECTED] wrote: Here's another obnoxious test case. [ ... ] So, in case anyone is curious (hi leo!), attached is a 56KB (9KB gzipped) imc file. It crashes on a memcpy inside compact_pool Some remarks what I could find out: - the damaged Buffer is in a pool of object_size 60 - Array/IntList (i.e. List) buffer_likes have that size - valgrind reports a lot of unitialised values during DOD [1] - some of these are in mark_context - the context contains all kinds of aggregates - current COW copying of stacks is AFAIK borken - both copies of one COWed stack share the same Buffer header - I don't think the the GC/compact_pool in resources.c does correctly deal with such Buffers. COWed strings have distinct (String)Buffer headers. It could be that e.g. lex Pads (which contain Arrays) aren't marked properly (valgrind has indications for that): ==16728== Conditional jump or move depends on uninitialised value(s) ==16728==at 0x80C4BE0: pobject_lives (src/dod.c:91) ==16728==by 0x81C1B72: mark_stack (src/method_util.c:162) ==16728==by 0x81BD2B8: mark_context (src/sub.c:59) And finally I have to admit, that the list code is too complex. I don't think, that the error is directly caused by the list code (DOD/GC is turned off during each allocation). But it seems to be overkill for the average usage (mainly small Arrays). It can deal with huge sparse arrays, tries to be space efficient, with small arrays and is pretty fast. But all current array usage (except a few tests) just put a few items into arrays and do not much mess around. leo [1] some of these are during stack trace, where valgrind doesn't see what Parrot is doing - we can ignore these
Re: Start of thread proposal
Dan Sugalski [EMAIL PROTECTED] wrote: =item SHARED THREAD A thread that's part of a group of threads sharing a common interpreter environment. I presume that this group of threads is one thread pool or interpreter pool. Could you expand the definition to cover pool. =item Each interpreter show the same process id The term process id is really misleading. Again I presume, that one pool is ment here: All the interpreters within a process will share a process ID. I'd vote for keeping a process ID what it is and use pool id or such here. leo
Re: Start of thread proposal
On Monday, January 19, 2004, at 07:58 , [EMAIL PROTECTED] wrote: Is there any likelyhood that memory allocation will be hidden behind macros at two levels: - ALLOCPOOL() for allocating large chunks of memory (ppols) that are later sub-allocated (and managed) by: - SUBALLOC() for sub allocating within a pool Are you wanting something akin to Apache 2 pools, which are hierarchical and designed to reduce path length when freeing blocks of objects? For instance, all objects and sub-pools allocated during an HTTP request cycle can be deallocated just by free()'ing the top-level request pool. I don't think parrot could make use of that model, since it can't very well guarantee that the user cannot retain references past the lifetime of the pool. Apache trusts modules not to make such errors; parrot can't trust the byte-code it's executing any further than it can throw it. A generational collector is a more likely means by which parrot might reduce memory-related overhead. Gordon Henriksen [EMAIL PROTECTED]
Re: Start of thread proposal
At 1:46 PM +0100 1/20/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: =item SHARED THREAD A thread that's part of a group of threads sharing a common interpreter environment. I presume that this group of threads is one thread pool or interpreter pool. Could you expand the definition to cover pool. Ah, damn, I needed to fix that up before sending it out. Should've been SHARED INTERPRETER. A shared interpreter is one that's in an interpreter pool. An interpreter pool is a set of interpreters that share common resources. They're essentially threads in the OS sense, and they have shared access to pretty much everything. The memory pools, arenas, and global namespace are shared--pretty much everything except what's in the interpreter structure itself. =item Each interpreter show the same process id The term process id is really misleading. Nope. It's the process ID. Nothing misleading there, unless you've done threading work under linux, since for a while it gave each thread a separate PID. Again I presume, that one pool is ment here: All the interpreters within a process will share a process ID. Nope, again, that's really process. (Again, because of Linux' past bad behavior reporting prorcess IDs) -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
This week's summary
The Perl 6 Summary for the week ending 20040118 I hope you'll forgive the lack of banter before we start in on perl6-internals. Threads. Again. Still more proposals about threads this week. Jeff Clites offered some notes based on the Java Virtual Machine's threading model. Surprisingly, this was the week's only threads thread. Next week: Dan starts to outline the design that's going to be implemented. http://tinyurl.com/2fk4f Questions about abstract PMCs Stéphane Payrard had some questions about abstract PMCs and whether they were needed in core_pmcs.h and pmctypes.pasm to make PMC type checking work in IMCC. Leo Tötsch answered questions, but didn't think they were actually needed in those files. Discussion ensued. http://tinyurl.com/3bqxg Docs and releases Tim Bunce wondered whether a date had been set for the next release. He also pointedly wondered if the docs were up to date with best practises and whether having them up to date would be a goal for the next release. Dan answered: No, no and yes, bordering on a requirement and not just a goal. Discussion ensued again. For some reason this thread flushed out a few 'shy lurkers' so let's extend a big hello to Paul Cochrane, Herbert Snorrason, Matt Diephouse, Robin Redeker, Richard Holden and Mark Solinski. http://tinyurl.com/yucb2 Making continuations work properly The work of getting continuations to close over the various bits and pieces the should close over continues; it seems there's rather more to doing the Right Thing than meets the eye. http://tinyurl.com/2bvpj new_noinit and init ops Luke Palmer was trying to implement a loop using a continuation. He wanted to be able to defer initialization of a continuation so he implemented two stage instantiation initialization; wrapped it up in two new ops, new_noinit and init, and posted the resulting patch to the list. Michal Wallace thought it best to just use a lexical. Then he perpetrated a spectacularly extended metaphor about Parrot entitled *Mr Parrot's Neighborhood*, which probably works best if you don't automatically correct the spelling of neighbourhood. http://tinyurl.com/24myr http://tinyurl.com/3xrus -- Mr Parrot's neighborhood Namespace stuff Jeff Clites revisited a thread from a while back about namespaces. His discussion centred on whether the namespace part of a name should be, logically a string Global::Namespace::Hierarchy or list of strings [Global, Namespace, Hierarchy]. He argued that it made sense to just use a simple thing and asked what we actually gained from having a hierarchy. Dan wants a hierarchy because it makes cross language sharing of namespaces easier. Larry wants a hierarchy 'cos it makes all sorts of things easier. Tim Bunce offered another proposal which received qualified approval from both Dan Leo. http://tinyurl.com/2j8gb http://tinyurl.com/3fecc -- Tim's proposal Parrot string docs Robert Eaglestone had a bunch of questions about Parrot's String documentation. Answers were forthcoming. http://tinyurl.com/ysz54 IMCC v1 feature freeze Melvin Smith announced a feature freeze for IMCC version 1 and called for bug reports for it. He plans to get imcc1 working as correctly as possible and frozen within a couple of weeks before starting the really major work (and deprecation of features) on IMCC 2. There was a certain amount of wrangling about CVS issues, but it was generally thought to be a good idea. http://tinyurl.com/2ooy5 Managed and unmanaged structs Dan had some thoughts about accessing and generally monkeying around with C structs and added a couple of related tasks to the todo list. Leo pointed out that quite a bit of it was done, and pointed out where further work was needed. http://tinyurl.com/2a4jl Loading bytecode at runtime Dan did some more design work on how runtime loading of bytecode should be handled. http://tinyurl.com/22rst The todo list Dan was reminded that we have a full, working, RT installation so he's started creating tickets for each todo. This should make for better tracking and ownership of tasks. Hurrah. He asked for a volunteer or two to manage the todo queue. Dave Pippenger and Stephane Peiry stepped up to the plate with heartening alacrity. Go guys. http://tinyurl.com/2wxou http://bugs6.perl.org/ Numeric formatting More design from Dan. This time he was thinking about numeric formatting. His initial plan was to lift the formatting rules from SQL, but I'm not sure if that plan survived contact with Michael Scott who pointed out that ICU (the Unicode library that's already included in the Parrot tree) has its own number formatting API. After some discussion in which Dan pointed out
Re: Start of thread proposal
Dan Sugalski [EMAIL PROTECTED] wrote: At 1:46 PM +0100 1/20/04, Leopold Toetsch wrote: The term process id is really misleading. Nope. It's the process ID. Nothing misleading there, unless you've done threading work under linux, since for a while it gave each thread a separate PID. $ ps | grep [p]arrot 17472 pts/000:00:00 parrot 17473 pts/000:00:00 parrot 17474 pts/000:00:00 parrot So the unless applies ;) This is a single parrot interpreter, with main, thread-manager, and event-handler thread. 3 PIDs. leo
Re: Start of thread proposal
At 16:24 +0100 1/20/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: At 1:46 PM +0100 1/20/04, Leopold Toetsch wrote: The term process id is really misleading. Nope. It's the process ID. Nothing misleading there, unless you've done threading work under linux, since for a while it gave each thread a separate PID. $ ps | grep [p]arrot 17472 pts/000:00:00 parrot 17473 pts/000:00:00 parrot 17474 pts/000:00:00 parrot So the unless applies ;) This is a single parrot interpreter, with main, thread-manager, and event-handler thread. 3 PIDs. This _all_ depends on which version of Linux you're using. Older versions don''t do it that way, and newer versions don't do it either (the specific versions escape me at the moment, but I know RH9 does _not_ have pids for threads). I know, because my Thread::Signal module depends on threads having pids. But fewer and fewer Linux systems support it (and Linux was the only one who worked this way to begin with). ;-( Liz
Re: Start of thread proposal
At 4:24 PM +0100 1/20/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: At 1:46 PM +0100 1/20/04, Leopold Toetsch wrote: The term process id is really misleading. Nope. It's the process ID. Nothing misleading there, unless you've done threading work under linux, since for a while it gave each thread a separate PID. $ ps | grep [p]arrot 17472 pts/000:00:00 parrot 17473 pts/000:00:00 parrot 17474 pts/000:00:00 parrot So the unless applies ;) This is a single parrot interpreter, with main, thread-manager, and event-handler thread. 3 PIDs. Right. Linux is, as I said, arguably broken here, but I don't want to argue that point. That's why I specified that the PID is the process id of whatever instantiated parrot -- in this case, no matter which thread you're in, when you ask for the pid from parrot you should get 17472. (In this case, at least) Parrot needs to grab the pid at global initialization time and store it away for later pid queries. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Start of thread proposal
Dan Sugalski [EMAIL PROTECTED] wrote: =item Interpreter pools will share allocation pools All the interpreters in an interpreter pool will share header and memory allocation pools. What about temporary PMCs (or strings)? Evaluating a non-trivial expression can have lot of these. Each new temp would need a lock on the memory sub-system. Combined with the cost of: =item All shared PMCs must have a threadsafe vtable The first thing that any vtable function of a shared PMC must do is to aquire the mutex of the PMCs in its parameter list i.e. typically 3 mutexes, it could be that the vtable of a shared PMC should just grab one interpreter lock (of the owner of that PMC) and that not all is shared (especially not the temps). leo
Re: Start of thread proposal
At 5:16 PM +0100 1/20/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: =item Interpreter pools will share allocation pools All the interpreters in an interpreter pool will share header and memory allocation pools. What about temporary PMCs (or strings)? Those won't get marked as shared unless we're unconditionally marking things as shared. (Though we may just give 'em a mutex anyway) Evaluating a non-trivial expression can have lot of these. Each new temp would need a lock on the memory sub-system. Only on allocation. We could have a per-thread freelist if we wanted. Wouldn't be unreasonable. Combined with the cost of: =item All shared PMCs must have a threadsafe vtable The first thing that any vtable function of a shared PMC must do is to aquire the mutex of the PMCs in its parameter list i.e. typically 3 mutexes, it could be that the vtable of a shared PMC should just grab one interpreter lock (of the owner of that PMC) and that not all is shared (especially not the temps). Yeah, but since PMCs aren't owned by any one interpreter in the pool but are rather pool-wide you run into either the global-lock problem (which kills performance on SMP systems) or interpreters potentially deadlocking on unrelated data access. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Shared vs non shared PMC timings
To estimate the costs of shared PMCs I have run this program[1], once with .Ref and once with .SharedRef. There are 2 major issues: 1) PMC initialization (new_p_ic_p): The shared PMC needs additionally the allocation of the synchronize structure and the MUTEX_INIT. 2) PMC access (set_p_i): locking/unlocking the mutex Here are snippets from the profile: with SharedRef CODE OP FULL NAMECALLS TOTAL TIME AVG T. ms - --- -- -- 753 new_p_ic_p 100.157785 0.0016 905 set_p_i100.049269 0.0005 with Ref CODE OP FULL NAMECALLS TOTAL TIME AVG T. ms - --- -- -- 753 new_p_ic_p 100.051330 0.0005 905 set_p_i100.011356 0.0001 (Overall timings aren't really comparable, the SharedRef also does a LOCK for mark, which slows that down as well) Linux 2.2.16, Athlon 800, unoptimized Parrot build. leo [1] set I0, 10 set I1, 0 lp: new P0, .PerlInt new P1, .Ref, P0 # or .SharedRef set P1, I1 inc I1 lt I1, I0, lp end
RE: Shared vs non shared PMC timings
Leopold Toetsch wrote: (Overall timings aren't really comparable, the SharedRef also does a LOCK for mark, which slows that down as well) ?? Why'd you do that? Competetive threads MUST be suspended (most likely with their cooperation, not with an OS suspend call) during the mark phase. -- Gordon Henriksen IT Manager ICLUBcentral Inc. [EMAIL PROTECTED]
Re: Shared vs non shared PMC timings
At 5:48 PM +0100 1/20/04, Leopold Toetsch wrote: To estimate the costs of shared PMCs I have run this program[1], once with .Ref and once with .SharedRef. There are 2 major issues: 1) PMC initialization (new_p_ic_p): The shared PMC needs additionally the allocation of the synchronize structure and the MUTEX_INIT. This is a one-time cost. If a PMC has one, it should stick around after the PMC is destroyed and put on the free list. 2) PMC access (set_p_i): locking/unlocking the mutex with SharedRef 905 set_p_i100.049269 0.0005 with Ref 905 set_p_i100.011356 0.0001 Yeah, that's about right. There is, unfortunately, little to be done about it. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Start of thread proposal
Dan Sugalski [EMAIL PROTECTED] wrote: At 5:16 PM +0100 1/20/04, Leopold Toetsch wrote: What about temporary PMCs (or strings)? Those won't get marked as shared unless we're unconditionally marking things as shared. (Though we may just give 'em a mutex anyway) This needs either one check per PMC, if its shared or not, or additional costs for locking temps. Both is rather expensive, compared to the raw working functionality of a vtable. Evaluating a non-trivial expression can have lot of these. Each new temp would need a lock on the memory sub-system. Only on allocation. Yes: Each *new* temp .. ... We could have a per-thread freelist if we wanted. Wouldn't be unreasonable. That already smells like separate memory sub-systems. The freelist has to be filled first. During DOD runs, it has to be refilled. To achieve some locality of the temp PMCs, you can't just give these to arbitrary intpreters. Separate free lists seem also to imply separate DOD runs to cleanup the temps. i.e. typically 3 mutexes, it could be that the vtable of a shared PMC should just grab one interpreter lock (of the owner of that PMC) and that not all is shared (especially not the temps). Yeah, but since PMCs aren't owned by any one interpreter in the pool the owner is for me the interpreter, that did allocate the arena. ... but are rather pool-wide you run into either the global-lock problem (which kills performance on SMP systems) or interpreters potentially deadlocking on unrelated data access. Could be. But the performance is the overall throughput. When a lot of fine grained locks (plus memory subsystem locks for temps) take much longer then one global lock, then that scheme can nethertheless be slower on SMP machines. It would scale better though for more CPUs. Deadlocks shouldn't be a problem, if exactly one vtable (like in SharedRef) just grabs one and only one lock. leo
Re: Shared vs non shared PMC timings
Dan Sugalski [EMAIL PROTECTED] wrote: At 5:48 PM +0100 1/20/04, Leopold Toetsch wrote: 1) PMC initialization (new_p_ic_p): The shared PMC needs additionally the allocation of the synchronize structure and the MUTEX_INIT. This is a one-time cost. If a PMC has one, it should stick around after the PMC is destroyed and put on the free list. Ah yep. Good idea. leo
Re: Shared vs non shared PMC timings
Gordon Henriksen [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: (Overall timings aren't really comparable, the SharedRef also does a LOCK for mark, which slows that down as well) ?? Why'd you do that? I didn't do it :) Pmc2c.pm is too dumb, it just puts a LOCK/UNLOCK pair around each vtable mthod, and no one is telling it that there are exceptions ;) leo
Re: Shared vs non shared PMC timings
At 6:49 PM +0100 1/20/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: At 5:48 PM +0100 1/20/04, Leopold Toetsch wrote: 1) PMC initialization (new_p_ic_p): The shared PMC needs additionally the allocation of the synchronize structure and the MUTEX_INIT. This is a one-time cost. If a PMC has one, it should stick around after the PMC is destroyed and put on the free list. Ah yep. Good idea. I'll admit, I'm tempted to make it part of the arena initialization cost -- when a new PMC arena is allocated because we're out of headers we just unconditionally give 'em all sync structs if we're running threaded. The only worry there is resource exhaustion. Rumor has it that some systems have a limited number of mutexes, but I've never actually seen one of them. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: [RESEND] Q: Array vs SArray
At 2:17 PM +0100 1/11/04, Leopold Toetsch wrote: One further note, while at Array/PerlArray: the class dependency is suboptimal. PerlArray isa Array isa list. The underlying list is auto-extending and does no bounds checking. Array does bounds-checking. PerlArray doesn't bounds check. So for better performace and simpler code, the dependency of PerlArray and Array should be swapped. Okay, at this point we've a pile of different array classes with fuzzy requirements and specified behaviours which sort of inherit from each other except when they don't. Before we go any further we need to figure out what we want. Then we need to see what we have. Then it's time to kill off the unneeded stuff and fix the rest. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: The todo list
In the mean time, if anyone else has todo list items, send them (*ONE* per e-mail!) to bugs-parrot at bugs6.perl.org to get 'em in the queue and we'll start sorting them out from there. If we're lucky and have sufficient web luck we might even get 'em into a web-accessible TODO list (so make sure the subjects are descriptive too!) I need to repair the web-accessible TODO list. Someone nag me periodically. -R
[DOCS] Updated documentation in Perl scripts
I've committed updates to the documentation in the Perl scripts in build_tools, classes and tools/dev. http://homepage.mac.com/michael_scott/Parrot/docs/html All scripts now run with -w (turned up a harmless bug in Parrot::Vtables, which I fixed). CVS $Id and copyright notices were also added where necessary. Mike
Re: [DOCS] CVS $Id
Mike~ I just recently added a file src/generic_register.c. I think that I simply copied the CVS $Id from the file register.c which I used as a template. Matt Michael Scott wrote: I've committed updates to the documentation in the Perl scripts in build_tools, classes and tools/dev. http://homepage.mac.com/michael_scott/Parrot/docs/html All scripts now run with -w (turned up a harmless bug in Parrot::Vtables, which I fixed). CVS $Id and copyright notices were also added where necessary. Mike
open issue review (easy stuff)
Here are 177 currently outstanding parrot issues in the RT system. You can see more detail on them by going to: http://rt.perl.org/ clicking on guest access, and then typing the number into the upper right hand corner box. what we're mostly looking for are issues that can already be marked as closed. Once that's done, we can then take another pass and look at the outstanding issues. If you respond to this message, with the ticket id and brief reason why it can be closed, i.e. patch applied, patch rejected, bad idea.. I'll do the labor of updating the ticketing system. -R 726: -fno-strict-aliasing (fwd) 760: Parrot_warn doesn't work with a NULL interpreter 801: [PATCH] PerlArray in scalar context 15267: example/pxs is broken 15308: Dans Feedback Integrated into Documentation 15357: Read write ops in core.ops are broken 15748: [SPAM=07.30] [PATCH] added function in pmc.c 15877: [SPAM=07.30] [PATCH] genclass.pl 15988: Make.pl might load the wrong Make.pm 16020: Re: Array vs. PerlArray 16087: [PATCH] Scratchpad pmc 16114: [PATCH] faster assembler 16237: [PATCH] register window flush for sparc 16258: [PATCH] The Great Renaming 16300: [BUG] hash clone hangs 16414: Interpreter PMC 16622: RE: [PATCH PDD07] Document struct naming conventions 16755: imcc requires Parrot_dlopen but HAS_DLOPEN is never defined 16763: parrot hash test fail on sparc Linux 16764: [PATCH] library building win32 16772: [PATCH] Add format for UINTVAL 16789: 'make test' for parrot fails 16797: make subs closures 16842: Solaris8/gcc warnings/errors 16935: [PATCH] more regex stack manipulation 17072: warnings while compiling parrot 17158: [PACTH] reduce size of core_ops_prederef.* by a fair amount 17159: imcc / Mac OS X problem 17244: [PACTH] hash ops at runtime for op_code() lookup 17405: [PATCH] correct make pdb on Win32 17490: Magic is useless unless verifiable. 17491: One-queens problem 17492: Leaning tower of Hanoi 17562: Segfault in stack handling code running ELIZA 17646: [PATCH] Add stone-age exception handling 17702: [PATCH] resources.c - growing allocations 17739: [PATCH] Tests for assign ops 17817: [PATCH] Parrot_sprintf-related stuff, part 2 17876: [PTACH] Parrot_snprintf writes 1 char too much 17931: [PATCH] DOD/GC related 17974: make clean removes lib/Parrot/PMC.pm 18003: Make test failures under Win32 MSVC++ 18044: Assembler doesn't return a useful exit status 18064: test 75 of t/pmc/pmc.t fails 18097: [PATCH] allow NULL interpreter in sprintf like functions 18142: [PACTH] Parrot_destroy 18170: [PATCH] very complete lexical scope implementation 18189: Test failures with 'long long' on i386/linux 18319: [PATCH] Re: How to portably link on Win32 (all flavors), OS/2 and VMS? 18320: PerlArray, GC or string bug 18637: PDD06: Incomplete sentence at end of section 18762: nci.t hates me 18782: [PTACH] long double support i386/linux 18832: [PATCH] nci test lib 18892: Perl_Parse from JNI Native code crashing... 18897: Error running Configure.pl 19090: [PATCH] make parrot_v[sfn]*printf behave itself 19143: Another GC bug? 19163: configure probe for va_list* 19183: languages/perl6/t/compiler.t -- multiple ways to spell Inf 19184: languages/perl6/t/rx/call test error 19192: JIT fails 3 tests on AMD K5 19328: bug/feature (?) in perlarray of perlarrays 19331: [PATCH]befunge 0.1.0 released 19356: [PATCH] creating string_destroy() 19467: [PATCH] win32.h - MinGW #pragma warnings 19516: imcc whitespace sensitive parsing error 19599: [pasm bug] PerlHash: cannot delete a key 19670: [PASM] bug in i/o (read on stdin) 19671: [PATCH] befunge debugger supports the delete instruction 19728: [PATCH] SPARC jit fix for restart 19834: [PATCH] sub, add, mul, div with combinations of INT, NUM, PMC 19871: Fwd: Introduction and cygwin results 19872: Fwd: Re: Introduction and cygwin results 19873: Fwd: perl6 testing on mingw32 19874: Patch for pdump 20298: Operators neg and abs in core.ops 20358: [BUG] disassemble still 20666: Assemble.pl Reports Incorrect Line Numbers after Removal of Macros 20707: Major packfile(?) breakage 21577: [PATCH] sun4 vtable jit support 21600: [PATCH] Enable buffer io in PIO 21665: [BUG] Incompatible return type in io/io_unix.c 21668: APL doesn't use sigils 21679: [PATCH] Implement use for P6C. 21729: IMCC doesn't handle scientific notation constants 21988: More Small Memory Leak Fixes 22119: [PATCH]Parrot PPC JIT: More ops and bugfixes. 22135: NCI and Jit push arguments in wrong order 22173: test t/native_pbc/number.t fails on powerpc linux 22174: [PATCH] snprintf link errors on Win32 22175: [PATCH] Static libparrot should built by target 'all' 22183: imcc doesn't support \e 22214: set Nx, Sx with non-numeric Sx results in number 22218: Segfault in Parrot_really_destroy 22281: fix rebuild_other so that it does not make chunks bigger than MAX_ITEMS 22316: [PATCH] Truncated entry in glossary.pod 22319: [PATCH] extra-large is too big for varargs 22321: [PATCH] debug tests when debugging 22324: [PATCH] fixed test and
Re: open issue review (easy stuff)
Robert~ Thanks for taking this on. Matt 22558: Another Hack at constant propogation Aplied or Fixed and then applied (I forget which) 24847: [patch] simplify register stacks (remove code duplication) Reject (resubmitted late as a different patch which was applied)
Re: open issue review (easy stuff)
Thanks for taking this on. No problemo. 22558: Another Hack at constant propogation Aplied or Fixed and then applied (I forget which) 24847: [patch] simplify register stacks (remove code duplication) Reject (resubmitted late as a different patch which was applied) Noted. (FYI, for the future, I may not ack all the changes, although it depends how many there are.) -R
Re: open issue review (easy stuff)
Here are a few that can be closed; I'm sure this is far from a complete list. Simon 15357: Read write ops in core.ops are broken Close - the broken ops no longer exist. 16114: [PATCH] faster assembler Close - we don't use assemble.pl any more. 17974: make clean removes lib/Parrot/PMC.pm Close - the bug was fixed. 18637: PDD06: Incomplete sentence at end of section Close - fixed. 18892: Perl_Parse from JNI Native code crashing... Close (or redirect) - Perl 5.6.1 bug 19143: Another GC bug? Close - fixed. 21668: APL doesn't use sigils Close - not a bug 22316: [PATCH] Truncated entry in glossary.pod Close - fixed. 22856: IMCC fails to build Close - fixed. 22867: Popbottom ops Close - not a bug, just unclear documentation, which I fixed. 23767: parser_util.c doesn't compile Close - fixed. 24056: [PATCH] disassemble crashes Close - fixed. 24063: [PATCH] Add C test info to tests.pod Close - patch was applied. 24267: Inconsistent behaviour of logical or op Close - fixed.
Re: Start of thread proposal
On Tuesday, January 20, 2004, at 07:56 , [EMAIL PROTECTED]
wrote:
I believe that parrot already has the concept of memory pools in it's
memory management. The idea is that by allocating similarly sized
objects from separate (large) allocations, you can reduce the
fragmentation of chunks and reduce the incidences where the memory need
to be GC'd and compacted.
Don't presume that. General purpose memory allocators tend to handle
this scenario... generically. Use of custom allocators is usually a
premature optimization; the best generic memory allocators are very,
very good.
But the ultimate effect of the overuse of custom allocators or pools is
to move memory fragmentation to a higher level, preventing the generic
allocator from addressing it even when the memory allocation patterns
would otherwise allow it.
If the allocation of pools, and the allocation of bit-of-a-pool, are
macroised, it makes it possible for OS's where there are multiple APIs
for memory allocation to bypass the CRT memory allocation routines and
use which ever native APis are best suited for the type of allocation.
Would much rather see them abstracted, as they are now, beneath an API.
This reduces the number of ways in which compiled parrot extensions can
be incompatible with the parrot core.
Personally, I would like to see memory allocation for each class type
be managed by the class constructor itself. This would theoretically
allow each class that has a fixed instance size to manage it's own pool
on OS's where that makes sense. The class would allocate a pool for
itself when loaded and then allocate instances from that pool on new()
and deallocate upon DESTROY. If it's memory pool was exhausted when new
was called, it would invoke the GC on *it's pool only*.
A PMC class is free to do precisely that. Only the PMC header cannot be
thus managed, and that's already pooled.
This separation would mean that each run of the GC would have a much
smaller pool of memory to compact and garbage collect when it was
invoked.
This is false. The mark phase will still need to run over the entire
process, else it cannot detect all references into the pool. (Unless you
can provide a type-safety guarantee that your type can only be
referenced by other instances of itself. In which case, your type is
necessarily garbage and the best optimization strategy would be
dead-code elimination. :)
I predict the best overall throughput would be with the sweep or copy
phase run immediately after the mark phase, across the entire process:
It would be wasteful to do all that marking and yet leave known garbage
uncollected.
It would also be less likely to be called, as each allocation from a
pool of fixed sized sub allocations will only ever need to call the GC
when it's pool is entirely exhausted.
Statistically, multiple pools will individually exhaust themselves
*MORE* frequently than a global pool. The ideal case for collection
frequency is that there is only one pool, or that all pools rise to
capacity concurrently. In these ideal cases, the segregated pools will
exhaust themselves precisely *AS* frequently as a global pool. In any
case, there is no possibility for a decrease in collection events
through the use of pooled memory. As per above, collection events will
also not become less expensive. Thus, expect garbage collection to have
a negative net impact on performance if pooled memory is used.
But that is a radical departure :), so if would just like to see
separate calls for pool allocation/reallocation and element
allocation/reallocation, rather than having calls to malloc() scattered
through out the codebase.
Uh. See memory.c and smallobject.c...
Digression:
Most copying collectors avoid free space fragmentation by design.
(That's one of their benefits.) An allocator as might be employed by a
standard compacting, generational system (Java, MS CLR), might resemble
the following:
#define SYS_ALIGN sizeof(double) /* for instance */
struct generation {
volatile void *next;
void *top;
void *base;
}
struct generation *generation_init(size_t capacity) {
struct generation *g = (struct generation *) malloc(sizeof(struct
generation));
g-next = g-base = malloc(capacity);
g-top = g-base + capacity;
}
void* generation_alloc(struct generation* p, size_t len) {
len = (len + SYS_ALIGN - 1) ~(SYS_ALIGN - 1); /* round up */
do {
void *ptr = ATOMIC_ADD(p-next, len);
if (ptr = p-top) {
return ptr - len; /* break the loop */
}
gc();
} while (true); /* try again */
}
+ No free space fragmentation at all.
+ Super-duper fast.
+ Threadsafe.
+ Lock-free unless ATOMIC_ADD cannot be implemented without a mutex.
- There is copying overhead when the generation is exhausted.
- One could say that the generation is fragmented by garbage.
+ It is no more fragmented by garbage than a GC system which uses a
freelist allocator.
Gordon Henriksen
Re: Start of thread proposal
20/01/2004 13:29:35, Gordon Henriksen [EMAIL PROTECTED] wrote: On Monday, January 19, 2004, at 07:58 , [EMAIL PROTECTED] wrote: Is there any likelyhood that memory allocation will be hidden behind macros at two levels: - ALLOCPOOL() for allocating large chunks of memory (ppols) that are later sub-allocated (and managed) by: - SUBALLOC() for sub allocating within a pool Are you wanting something akin to Apache 2 pools, which are hierarchical and designed to reduce path length when freeing blocks of objects? For instance, all objects and sub-pools allocated during an HTTP request cycle can be deallocated just by free()'ing the top-level request pool. Nothing to do with Apache memory pools. I believe that parrot already has the concept of memory pools in it's memory management. The idea is that by allocating similarly sized objects from separate (large) allocations, you can reduce the fragmentation of chunks and reduce the incidences where the memory need to be GC'd and compacted. Allocating an 8 byte chunk from a common memory pool is quite likely to nip a little off from a previously freed large chunk. When it comes time reallocate another chunk the same size as that large, freed chunk, although there is enough room in the over all freespace chain to accommodate it, the largest available chunk is now 8 bytes or so too small for the requirement. That induces either a compaction cycle or the need to extend the memory pool by the size of the large request. Allocating all small requests from the same pool, and large from another pool means that you are less likely to fragment the memory and more likely to be able to re-use an existing slot in the free-space chain for any given request. If the allocation of pools, and the allocation of bit-of-a-pool, are macroised, it makes it possible for OS's where there are multiple APIs for memory allocation to bypass the CRT memory allocation routines and use which ever native APis are best suited for the type of allocation. Personally, I would like to see memory allocation for each class type be managed by the class constructor itself. This would theoretically allow each class that has a fixed instance size to manage it's own pool on OS's where that makes sense. The class would allocate a pool for itself when loaded and then allocate instances from that pool on new() and deallocate upon DESTROY. If it's memory pool was exhausted when new was called, it would invoke the GC on *it's pool only*. This separation would mean that each run of the GC would have a much smaller pool of memory to compact and garbage collect when it was invoked. It would also be less likely to be called, as each allocation from a pool of fixed sized sub allocations will only ever need to call the GC when it's pool is entirely exhausted. But that is a radical departure :), so if would just like to see separate calls for pool allocation/reallocation and element allocation/reallocation, rather than having calls to malloc() scattered through out the codebase. I don't think parrot could make use of that model, since it can't very well guarantee that the user cannot retain references past the lifetime of the pool. Apache trusts modules not to make such errors; parrot can't trust the byte-code it's executing any further than it can throw it. A generational collector is a more likely means by which parrot might reduce memory-related overhead. Gordon Henriksen [EMAIL PROTECTED] Nigel/
Churchill's Parrot Still Swearing
i think this is on topic. :) uri F*** THE NAZIS, SAYS CHURCHILL'S PARROT Jan 19 2004 EXCLUSIVE By Bill Borrows SHE WAS at Winston Churchill's side during Britain's darkest hour. And now Charlie the parrot is 104 years old...and still cursing the Nazis. [Picture] Her favourite sayings were F*** Hitler and F*** the Nazis. And even today, 39 years after the great man's death, she can still be coaxed into repeating them with that unmistakable Churchillian inflection. Many an admiral or peer of the realm was shocked by the tirade from the bird's cage during crisis meetings with the PM. But it always brought a smile to the war leader's face. Churchill bought Charlie - giving him a boy's name despite the fact she was female - in 1937. She took pride of place in a bizarre menagerie of pets including lambs, pigs, cattle, swans and, at one point, a leopard. He immediately began to teach her to swear - particularly in company - and she is keeping up the tradition today. The blue and gold macaw is believed to be Britain's oldest bird. The title was previously thought to belong to 80-year-old Cokky the cockatoo. But it can be proved Charlie is at least 104 and was born in the 19th century. Peter Oram bought her for his pet shop after Churchill died in 1965. But he was forced to move her into his home after she kept swearing at children. For the last 12 years, she has lived at Mr Oram's garden centre in Reigate, Surrey. Centre worker Sylvia Martin said: If truth be told, Charlie is looking a little scruffy but she is very popular with the public. We are all very attached to her. James Humes, an expert on the late PM, said: Churchill may no longer be with us but that spirit and those words of defiance and resolve continue. Charlie's story is in this month's Jack Magazine, on sale Thursday. -- Uri Guttman -- [EMAIL PROTECTED] http://www.stemsystems.com --Perl Consulting, Stem Development, Systems Architecture, Design and Coding- Search or Offer Perl Jobs http://jobs.perl.org
