Re: Start of thread proposal
[EMAIL PROTECTED] [EMAIL PROTECTED] wrote: I believe that parrot already has the concept of memory pools in it's memory management. The idea is that by allocating similarly sized objects The problem is not in the fixed sized header pools, its with the *memory* pool used e.g for string memory. During GC we are walking the *header* pools, and if we find its buffer memory being used, we move that buffer memory to a new store, thereby compacting it. The old memory store(s) are freed then. So string memory can move around beyond your code. Nigel/ leo
Re: [RESEND] Q: Array vs SArray
Dan Sugalski [EMAIL PROTECTED] wrote: Okay, at this point we've a pile of different array classes Before we go any further we need to figure out what we want. 1) Unify setting/getting element count - the elements() vtable is unused (not accessible by opcode) - we use get_integer() - push for Array is unusabale currently - reserve array store size before filling can have performance benefits by avoiding reallocation So e.g. elements I0, P0 # @a.elements # scalar @a elements P0, I0 # @a.elements = I0 # $#a = I0-1 set I0, P0 # @a.internal_capacity set P0, I0 # @a.internal_capacity = I0 2) Array vs PerlArray - PerlArray does autoexpand and is derived from Array. - Array throws exceptions if you access elements beyond capacity - the underlying List does not do bounds checking So the class inheritance is wrong. 3) SArray - Should have been a simple typed array (At these times PerlHash store was typed too). Its obsolete now. 4) list.c - is too complicated for the current usage patterns A currently better implementation could be e.g. - one buffer array - a start index to accomodate shift/unshift - resize by 2 n until a certain limit, then by that limit - after pop/shift shrink buffer by limit if limit+threshold is free leo
[DOCS] C code documentation
PDD 7 Conventions and Guidelines for Parrot Source Code has a section on Code Comments that has been followed for C code. I'm about to change this. The existing documentation headers will be replaced with pod headers contained within C multi-line comment delimiters. I'm going to stick to exactly the same style that I used in the Perl scripts. Functions will be proceeded with similarly formatted descriptions. No information will be lost. Perhaps the most controversial feature of all this is that I'm using rows of 80 '#'s as visual delimiters to distinguish documentation sections from code. This may seem like overkill to some. I'm basing it on what looks right to me in BBEdit and Xcode. If it turns out that it doesn't work for everyone, I'll change it. If anyone feels strongly about this then speak. Mike
Q: Sub vs Closure lexical pads
While trying to generate a small example that shows the memory corruption problem reported by Steve, I came along these issues: a) [1] is .Sub, [2] is turned off The subroutine prints main's $m - very likely wrong. Q: Should the Sub get a NULL scratch pad, or a new empty scratch pad stack? b) [1] is .Closure, [2] is turned off The closure prints main - probably ok c) [1] is .Closure, [2] is newpad 0 Lexical '$m' not found d) [1] is .Closure, [2] is newpad 1 prints main Q: What is correct? .sub _main new_pad 0 new $P0, .PerlString set $P0, main\n store_lex -1, $m, $P0 .sym pmc foo newsub foo, .Sub, _foo # [1] .pcc_begin prototyped .pcc_call foo .pcc_end pop_pad end .end .sub _foo prototyped # new_pad 1# [2] find_lex $P0, $m print $P0 # pop_pad .pcc_begin_return .pcc_end_return .end leo
Re: Q: Sub vs Closure lexical pads
Leopold Toetsch writes:
While trying to generate a small example that shows the memory
corruption problem reported by Steve, I came along these issues:
a) [1] is .Sub, [2] is turned off
The subroutine prints main's $m - very likely wrong.
Well, Subs don't do anything with pads, so I'd say this is correct.
Q: Should the Sub get a NULL scratch pad, or a new empty scratch pad stack?
Or just keep the existing one like current subs do. A top-level sub
should be a closure under an empty pad stack.
The following correspond to Perl code (approximately):
b) [1] is .Closure, [2] is turned off
The closure prints main - probably ok
eval 'my $m = main\n; ' . 'print $m';
c) [1] is .Closure, [2] is newpad 0
Lexical '$m' not found
sub foo { print $m }
{ my $m = main\n; foo() }
d) [1] is .Closure, [2] is newpad 1
prints main
my $m = main\n; sub { print $m }-();
Q: What is correct?
It looks to me as though they all are.
Luke
.sub _main
new_pad 0
new $P0, .PerlString
set $P0, main\n
store_lex -1, $m, $P0
.sym pmc foo
newsub foo, .Sub, _foo # [1]
.pcc_begin prototyped
.pcc_call foo
.pcc_end
pop_pad
end
.end
.sub _foo prototyped
# new_pad 1# [2]
find_lex $P0, $m
print $P0
# pop_pad
.pcc_begin_return
.pcc_end_return
.end
leo
Re: Start of thread proposal
21/01/2004 02:12:09, Gordon Henriksen [EMAIL PROTECTED] wrote: This is false. The mark phase will still need to run over the entire process, else it cannot detect all references into the pool. If by reference, you mean address, then that is true. If when a reference is taken, the address of the referent is stored in arbitrary other data structures, then all memory be scanned by the GC, However, if references were not addresses, but an index into a table of addresses, then only the table need be scanned. When a reference, or the thing holding it, is deleted, then the indexed slot in the address table is blanked and subsequent GC passes looking for references to a referent will no longer find the reference in the table. With the addition of a reference count to the table, all references to a single entity can share the same index, but now I'm groping back in the direction of reference counting, which is not flavour of the month:) Nigel.
Re: [DOCS] C code documentation
My vote goes for the simplest that will still parse; /* =head1 foo */ After all, arent't we all using editors that can highlight the scructure of our code to our satisfaction ? Surely even VIM et al can stick in dividers or something to make them stand out if the coder desires? I've already got __ | |__(_)__ _ | '_ \ / _` | |_.__/_\__, | fonts set for my POD headings, |___/ which is a lot nicer to work with (IMHoHO). ASCII art delimiters in code are SO 20th century ;-) ___ __ _ / \ | | ( ( ( 2ยข ) | | ___ \__/ |_| On Wed, 21 Jan 2004 22:42, Michael Scott wrote; PDD 7 Conventions and Guidelines for Parrot Source Code has a section on Code Comments that has been followed for C code. I'm about to change this. The existing documentation headers will be replaced with pod headers contained within C multi-line comment delimiters. I'm going to stick to exactly the same style that I used in the Perl scripts. Functions will be proceeded with similarly formatted descriptions. No information will be lost. Perhaps the most controversial feature of all this is that I'm using rows of 80 '#'s as visual delimiters to distinguish documentation sections from code. This may seem like overkill to some. I'm basing it on what looks right to me in BBEdit and Xcode. If it turns out that it doesn't work for everyone, I'll change it. If anyone feels strongly about this then speak. Mike -- Sam Vilain, [EMAIL PROTECTED] Real software engineers don't debug programs, they verify correctness. This process doesn't necessarily involve executing anything on a computer, except perhaps a Correctness Verification Aid package.
RE: [perl #25129] IO Buffer Test
This is a test for the buffer bug that was discussed on p6i a few days ago. It does not remove the test file it makes because I do not know a good way to do this. You may use teardown() defined in t/src/io.t The attached patch contains stripped down versions of your program that I used to simulate the buffer IO bugs (perl #24941). Test cases for: a) PIO_flush on buffer full b) turn off buffering Leo, even with old versions of the repository, I couldn't simulate the 'filling the disk on flush' bug. My box is i386-linux with gcc. /Stefan Thanks, Arvindh biotests.patch Description: biotests.patch This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorised review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. Visit us at http://www.cognizant.com
Re: open issue review (easy stuff)
Robert Spier wrote: 24941: [PATCH] RE: More Buffer IO Bugs (was: Strangeness when printing to file) patch applied. Arvindh __ Do you Yahoo!? Yahoo! Hotjobs: Enter the Signing Bonus Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus
Re: Memory corruption
Leopold Toetsch [EMAIL PROTECTED] wrote: Steve Fink [EMAIL PROTECTED] wrote: It crashes on a memcpy inside compact_pool Some remarks what I could find out: - current COW copying of stacks is AFAIK borken - both copies of one COWed stack share the same Buffer header I have now more signs for that: - First: I have a smaller program, that shows a simimlar corruption bug (a Random PMC turns suddenly into a RetContinuation PMC - while its called Random it shouldn't expose randomness on itself :) - Turning GC off stops that bug - as well as in your program - Using the LEA/system allocator (Configure.pl --gc=libc) makes the bug vanish the GC/compact_pool in resources.c does correctly deal with such Buffers. COWed strings have distinct (String)Buffer headers. I'm quite sure now, that the bug is caused by the new COWed register chunks, which share *one* Buffer header. leo
how to subclass dynamic PMCs?
Hi all,
I'm hoping this is just a simple linker option, but
I've been reading ld manuals for the past few
hours and I just don't get it. :)
I'm trying to make a dynamically loaded PMC that
subclasses another dynamically loaded PMC.
I made two files in parrot/dynclasses/ :
// file 1: pisequence.pmc
#include parrot/parrot.h
#define enum_class_PiSequence -1
pmclass PiSequence need_ext dynpmc {
INTVAL get_integer () {
return 0;
}
}
// file 2: piobject.pmc
#include parrot/parrot.h
#define enum_class_PiString -1
pmclass PiString extends PiSequence need_ext dynpmc {
}
I added these to the Makefile, ran make -C dynclasses
and now, I have two *.so files:
% find runtime -name pi*.so
runtime/parrot/dynext/pisequence.so
runtime/parrot/dynext/pistring.so
Then I try to run this:
.sub __main__
loadlib P0, pisequence
find_type I0, PiSequence
print I0
print \n
loadlib P1, pistring
find_type I1, PiString
print I1
print \n
end
.end
Output is:
Couldn't load 'runtime/parrot/dynext/pistring': \
runtime/parrot/dynext/pistring: cannot open shared object file: \
No such file or directory
51
Couldn't load 'runtime/parrot/dynext/pistring': \
runtime/parrot/dynext/pistring: cannot open shared object file: \
No such file or directory
0
If I copy the get_integer function to PiString
instead of trying to inhert, everything works:
51
52
It seems the problem is that pistring.so can't
load because it can't find a definition for
Parrot_PiSequence_get_integer... Even though
it's defined in the other *.so file that's
already loaded.
I don't think this is a bug... I think I just
don't know what I'm doing. Can someone help? :)
Sincerely,
Michal J Wallace
Sabren Enterprises, Inc.
-
contact: [EMAIL PROTECTED]
hosting: http://www.cornerhost.com/
my site: http://www.withoutane.com/
--
Re: [RESEND] Q: Array vs SArray
At 9:38 AM +0100 1/21/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: Okay, at this point we've a pile of different array classes Before we go any further we need to figure out what we want. 1) Unify setting/getting element count - the elements() vtable is unused (not accessible by opcode) - we use get_integer() - push for Array is unusabale currently - reserve array store size before filling can have performance benefits by avoiding reallocation Okay. So, let's do the following: *) Expose elements as an op. Two, actually, as we should have a get and set version. *) Expand elements to get_elements and set_elements 2) Array vs PerlArray 3) SArray If SArray is obsolete, let's kill it. I'd like to ultimately have a fixed-size PMC array, a variable size PMC array, fixed-size typed arrays, and variable-sized typed arrays. For the moment the fixed and variable sized PMC arrays are fine. Folks seem to want a mixed-type array as well, and while we'll pay performance for that I don't see any reason to not do it, though we can live with a fixed-size version for now. So, lets do the classes as: *) Array - fixed-size, mixed-type array *) vPArray - variable-sized PMC array *) PArray - Fixed-size PMC array *) vSArray - variable-sized string array *) SArray - fixed-size string array And so on, for N and I arrays. I'm not particularly attached to the names. (Actually I think they suck, but they are reasonably descriptive) Personally I'd leave Array as it is, since it does one of the things that we need it to do. PerlArray can be a full subclass of vPArray, since right now they're the same thing. That may change at some point later on. 4) list.c - is too complicated for the current usage patterns A currently better implementation could be e.g. - one buffer array - a start index to accomodate shift/unshift - resize by 2 n until a certain limit, then by that limit - after pop/shift shrink buffer by limit if limit+threshold is free If you want to hack in on this, go ahead, though it looks like something that we'll ultimately need--its the guts of a sparse mixed-type array, which is useful and we may well need at some point. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Start of thread proposal
At 10:25 AM +0100 1/21/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: One more question: =head2 Guarantees ... doesn't have anything about user data integrity. So when 2 threads access a PerlNum, they could get a mixture of the typically 2 involved words. Potentially, yeah, though it's really unlikely. But: =item All shared PMCs must have a threadsafe vtable The first thing that any vtable function of a shared PMC must do is to aquire the mutex of the PMCs in its parameter list ... seems to indicate that even whole ops like add P,P,P are atomic. Yep. They have to be, because they need to guarantee the integrity of the pmc structures and the data hanging off them (which includes buffer and string stuff) And how does user level locking play together with that? I've not decided -- That was something I thought we might hash out as we abused the first half of the design doc. Personally I'm all for the user and low-level lock being the same thing, but that doesn't have to be the case. There are advantages and disadvantages to either way of doing things. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: [RESEND] Q: Array vs SArray
All~ So, lets do the classes as: *) Array - fixed-size, mixed-type array *) vPArray - variable-sized PMC array *) PArray - Fixed-size PMC array *) vSArray - variable-sized string array *) SArray - fixed-size string array I suggest using Array to mean fixed size and Vector to mean variable size. Thus yielding *) Array - fixed-size, mixed-type array *) Vector - variable-size, mixed-type array *) PArray - Fixed-size PMC array *) PVector - variable-sized PMC array *) SArray - fixed-size string array *) SVector - variable-sized string array etc. One could also use List for variable sized, but I have C++ based biases for vector... Matt
Re: [RESEND] Q: Array vs SArray
Dan Sugalski writes: At 9:38 AM +0100 1/21/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: Okay, at this point we've a pile of different array classes Before we go any further we need to figure out what we want. 1) Unify setting/getting element count - the elements() vtable is unused (not accessible by opcode) - we use get_integer() - push for Array is unusabale currently - reserve array store size before filling can have performance benefits by avoiding reallocation Okay. So, let's do the following: *) Expose elements as an op. Two, actually, as we should have a get and set version. I'd like it to be possible to return infinity here. If that means setting a PMC or returning a PMC, I think that's fine. As long as there are also Ireg variants. Luke
Re: [RESEND] Q: Array vs SArray
*) Array - fixed-size, mixed-type array *) vPArray - variable-sized PMC array *) PArray - Fixed-size PMC array *) vSArray - variable-sized string array *) SArray - fixed-size string array And so on, for N and I arrays. I'm not particularly attached to the names. (Actually I think they suck, but they are reasonably descriptive) i think they are not that bad... calling some vectors would mean having to remember which of the two synonyms is used for what - this can be hard. the 'modifiers' are on the front so its easier to parse them (as the regexp apocalypse explains) if i follow this correctly, variable-size means they autoeXpand, so maybe an x instead of v might also be understandable... (we would then have xp arrays ;) LF
Re: Start of thread proposal
Dan Sugalski [EMAIL PROTECTED] wrote: [ No Guarantees WRT data access } ... seems to indicate that even whole ops like add P,P,P are atomic. Yep. They have to be, because they need to guarantee the integrity of the pmc structures and the data hanging off them (which includes buffer and string stuff) But isn't that a contradiction? Or better: When even an opcode like above is atomic, that an access to a shared PerlNum should be guaranteed being atomic too. leo
Re: [RESEND] Q: Array vs SArray
Dan Sugalski [EMAIL PROTECTED] wrote: 4) list.c If you want to hack in on this, go ahead, though it looks like something that we'll ultimately need--its the guts of a sparse mixed-type array, which is useful and we may well need at some point. Actually that was the reason, I implemented it like that. But I'd like to have it a bit simplified, that is: - for small element counts use that simplified scheme - else use the current chunk based one with constant chunk size This could simplify the code by a fare amount. leo
Re: [RESEND] Q: Array vs SArray
Dan Sugalski [EMAIL PROTECTED] wrote: So, lets do the classes as: *) Array - fixed-size, mixed-type array *) vPArray - variable-sized PMC array *) PArray - Fixed-size PMC array *) vSArray - variable-sized string array *) SArray - fixed-size string array Actually I forgot one: We already have IntList, which is INTVAL only and doesn't bounds checking - vIArray in that naming convention. BTW SArray nicely meats current main usage as @argv :) And so on, for N and I arrays. I'm not particularly attached to the names. Nor am I. But the scheme sounds reasonable. leo
Re: Start of thread proposal
On Wed, Jan 21, 2004 at 01:14:46PM -0500, Dan Sugalski wrote: ... seems to indicate that even whole ops like add P,P,P are atomic. Yep. They have to be, because they need to guarantee the integrity of the pmc structures and the data hanging off them (which includes buffer and string stuff) Personally, I think it would be better to use corruption-resistant buffer and string structures, and avoid locking during basic data access. While there are substantial differences in VM design--PMCs are much more complicated than any JVM data type--the JVM does provide a good example that this can be done, and done efficiently. Failing this, it would be worth investigating what the real-world performance difference is between acquiring multiple locks per VM operation (current Parrot proposal) vs. having a single lock controlling all data access (Python) or jettisoning OS threads entirely in favor of VM-level threading (Ruby). This forfeits the ability to take advantage of multiple CPUs--but Leopold's initial timing tests of shared PMCs were showing a potential 3-5x slowdown from excessive locking. I've seen software before that was redesigned to take advantage of multiple CPUs--and then required no less than four CPUs to match the performance of the older, single-CPU version. The problem was largely attributed to excessive locking of mostly-uncontested data structures. - Damien
