quivalent to a
minus option and the absolute length. The following patch has been
applied in revision 29735.
Appropriate tests to be added later.
Regards
Peter Gibbs
Index: src/spf_render.c
===
--- src/spf_render.c(revision 297
- Original Message -
From: "Geoffrey Broadwell" <[EMAIL PROTECTED]>
To: "Peter Gibbs" <[EMAIL PROTECTED]>
Cc: "Christoph Otto" <[EMAIL PROTECTED]>;
Sent: Friday, July 25, 2008 6:09 PM
Subject: Re: [perl #57260] [BUG] Segfaults in spr
e a simple array, but there are
other tests defined in sprintf_tests with more complicated expressions
inside square brackets, so it was more work than I had time for.
We need either a full parser for the perl-style tests, or a simpler test
harness that doesn't try to do the perl-speci
- Original Message -
From: "Will Coleda" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Friday, July 25, 2008 10:31 PM
Subject: Re: [perl #57260] [BUG] Segfaults in sprintf opcode
On Fri, Jul 25, 2008 at 4:23 PM, Peter Gibbs via RT
<[EMAIL PROTECTED]
these implementions to our
version of sprintf.
Regards
Peter Gibbs
ly works in a specific time zone. I have
committed (rev 29938) an alteration to this patch, which calculates the
date/time corresponding to -1.
This should now be self-adjusting for various time zones.
Regards
Peter Gibbs
David
In your last change (splitting buffer allocation from string) I assume you
also intended to shorten the initial allocation.
Peter Gibbs
EmKel Systems
Index: string.c
===
RCS file: /home/perlcvs/parrot/string.c,v
retrieving
free()
calls should go in free_string() rather than one in string_destroy() but I
don't know exactly what you and/or Dan intended here.
Peter Gibbs
EmKel Systems
Index: resources.c
===
RCS file: /home/perlcvs/parrot/res
to COW strings - would the buffer-related parameters (eg
bufstart, buflen, bufused) then not be linked to the buffer rather than the
string? ie should we have two structs (STRING & BUFFER) rather than one
combined one, with STRING pointing to BUFFER rather than directly to memory?
Peter Gibbs
I suspect that the length of the output buffer for string_transcode should
be based on the output encoding, not on the input encoding.
Peter Gibbs
EmKel Systems
Index: string.c
===
RCS file: /home/perlcvs/parrot/string.c,v
Another correction to string_transcode; this function now seems to work okay
(tested using a dummy 'encode' op added to my local copy of core.ops)
Peter Gibbs
EmKel Systems
Index: string.c
===
RCS file: /home/perl
udes buffer re-use (copy-on-write) for
substrings. On the other hand, passing strings to native C library routines
requires the termination. One compromise would be for 'unique' strings to be
terminated, and do 'copy-on-read' if a non-unique string (i.e. one borrowing
a buffer)
Jeff
If I understand the STRING struct correctly, buflen is the physical
allocation of the buffer, and bufused is the number of bytes actually used
by the string; therefore the latter would be correct for key_hash?
Peter Gibbs
EmKel Systems
Index: key.c
ion will be happening
on fixed boundaries, it may make sense to round the buffer length up
to match.
--
Peter Gibbs
EmKel Systems
Index: include/parrot/string.h
===
RCS file: /home/perlcvs/parrot/include/parrot/string.h,v
retrieving
t" to life.pasm in the
middle of the stats output lines, and on my linux x86 system this gives a
segfault, which seems to confirm that something has gone awry somewhere.
--
Peter Gibbs
EmKel Systems
Dan
I assume you meant pmc_pool not buffer_header_pool ??
--
Peter Gibbs
EmKel Systems
Index: resources.c
===
RCS file: /home/perlcvs/parrot/resources.c,v
retrieving revision 1.30
diff -c -r1.30 resources.c
*** resources.c 18 Mar
low fixes it by clearing the buffer pointer on allocation of a new
header; it could alternatively be done when the header is added to the free
list.
--
Peter Gibbs
EmKel Systems
Index: resources.c
===
RCS file: /home/perlcvs/parrot/r
nce both these problems involve design considerations, I am leaving it to
Dan to fix them.
Quick hacks to bypass the problems show that the pushp/popp test then works
fine.
--
Peter Gibbs
EmKel Systems
Clinton
The following patch seems to fix both these problems. It makes some slight
changes to the logic, so should be considered a temporary fix until Dan has
time to take a look at the code.
--
Peter Gibbs
EmKel Systems
Index: resources.c
again. Since the free pools are not
compressed by go_collect, perhaps they should be allocated independently and
not copied around all the time??
After fixing the above, the test program still abends, this time with
"subend somehow is less than substart" - I have not yet followed
reallocate.
A patch to create such a function and use it within add_pmc_to_free and
add_header_to_free follows; since all uses of mem_realloc are potentially at
risk, it might be a good idea to change any such places also.
--
Peter Gibbs
EmKel Systems
Index: incl
bc3, as the reference stays intact following
the re-assignment to $foo. Assuming that this will stay the same for perl 6,
we must either keep the original PMC address, or find and correct all
references to it.
--
Peter Gibbs
EmKel Systems
n the same place
as only creating a new string if the final parameter is NULL; the
implementations again differ.
I agree that the whole issue of garbage collection relating to newborns and
temporaries is problematic. One option might be to always flag newly-created
objects in some way, then have a single function call to release them; at
least this means just one function call at the end of each procedure. This
still leaves a problem with nested procedures.
--
Peter Gibbs
EmKel Systems
as discussed above, so "once a cow, always a cow", which will
have some negative impact.
--
Peter Gibbs
EmKel Systems
f useless blocks, or should we just let them be created and
discarded?
--
Peter Gibbs
EmKel Systems
epeat; this could be modified to do a simple
string_copy if the repeat count was one, and to act in-place if possible
Since then, string_replace has been added; for now, I will just change this
to do a straight un-cow (i.e. make a copy of the buffer if it is marked as
COW)
> Mike Lambert
>
--
Peter Gibbs
EmKel Systems
>> On Tuesday 02 April 2002 04:36, Peter Gibbs wrote:
>"Bryan C. Warnock" <[EMAIL PROTECTED]> wrote:
> buflen should be 16, not 0. bufused should be 0.
Currently, string_make sets the value of buflen. It does not know about
Parrot_allocate's rounding rule,
on; since this is only for a mode string, the
chances are very small, but not zero. Why are we passing a C-style string
around inside parrot anyway?
--
Peter Gibbs
EmKel Systems
uch), which would then
be called by both Parrot_allocate and Parrot_reallocate. The
alignment/padding rules would then be implemented in _allocate and
_reallocate, but nowhere else.
Of course, putting the 'correct' buflen into the string header messes up my
evil use of the last byte for
, after the DOD run, there is still less than N headers
available, allocate more even though we can satisfy the immediate
requirement. This would improve performance by reducing the number of DOD
runs, but at the cost of additional memory - a classic tradeoff!
--
Peter Gibbs
EmKel Systems
2412 collection runs were made
Copying a total of 0 bytes
There are 81 active Buffer structs
There are 256 total Buffer structs
--
Peter Gibbs
EmKel Systems
The memory_collected GC statistic does not get updated at present. Patch
below fixes.
Note that a 5000-generation run of life.pasm allocates 32K, and copies
almost 58MB.
--
Peter Gibbs
EmKel Systems
Index: resources.c
===
RCS file
ction runs were made
Copying a total of 57801936 bytes
Since the three pools occupy 1024 bytes each, 33576960 bytes of the above
copying is due to the pools.
--
Peter Gibbs
EmKel Systems
- Original Message -
From: "Michel J Lambert" <[EMAIL PROTECTED]>
To: "Brya
The string_grow function (currently used only by string_replace) does not
allocate a new buffer if there are no bytes to be copied from old buffer to
new buffer. Patch below fixes this.
--
Peter Gibbs
EmKel Systems
Index: string.c
al of 57801936 bytes
so a 97% decrease in the number of collection runs only turns into an 11%
improvement in total performance.
[166MHz Pentium; linux 2.2.18]
--
Peter Gibbs
EmKel Systems
kes 91 to 92 seconds (without JIT)
So, thus far, we aren't doing too badly. I am sure the perl version can be
optimised, but as it stands I think it is a reasonable equivalent of what
the pasm version does.
--
Peter Gibbs
EmKel Systems
#!/usr/bin/perl
#
# life.perl
#
# Play conway's (no
simplest way for now is probably just to add return statements at the
beginning of Parrot_do_dod_run and/or Parrot_go_collect in resources.c
--
Peter Gibbs
EmKel Systems
tring,
we know the actual length, instead of using the maximum possible.
I haven't yet looked to see why basic needs to transcode string anyway.
--
Peter Gibbs
EmKel Systems
Index: string.c
===
RCS file: /home/perlcvs/parrot/
one, but it does mean that the string
transcoding routines are getting called when nobody expects it. There may
well be other places in string.c where this causes problems; I will look at
this shortly.
--
Peter Gibbs
EmKel Systems
Note that string_grow still has the problem with not bothering to allocate a
new buffer if copysize is zero, e.g. if we are expanding a previously empty
buffer.
I have submitted a patch for this previously, but since string_grow has now
changed, herewith a resynced patch.
--
Peter Gibbs
EmKel
#x27; behaviour of Parrot_allocate, and makes the
additional allocated space available for subsequent use for string
expansion.
--
Peter Gibbs
EmKel Systems
Index: string.c
===
RCS file: /home/perlcvs/parrot/string.c,v
retrievin
the real allocated size (which means the size
adjustment logic in go_collect can be removed); then the COW string handling
routines would have to ensure that buflen-1 is always used as the available
buffer size, which is not a major problem.
--
Peter Gibbs
EmKel Systems
referenced it
is stored in the (old copy of the) buffer, and the in-buffer flag is set; if
the same buffer is encountered later, the second header is simply adjusted
to point to the new location.
--
Peter Gibbs
EmKel Systems
/* Run through all the string header pools and copy */
for (cur_string_
do we start/stop trusting ourselves? string_replace calls string_grow
which calls Parrot_reallocate which calls mem_allocate. If we say that
string_replace (or other string_* functions) cannot be trusted to call
string_grow correctly, surely we must then modify mem_allocate to handle a
negative request length.
--
Peter Gibbs
EmKel Systems
st time it is called for any given string,
even though the physical allocation would have been sufficient.
--
Peter Gibbs
EmKel Systems
at this patch gives compiler warnings in string.c because of the
'const' attribute on the parameters, and therefore should not be applied in
its current form; I'm sure somebody can figure out how best to resolve the
warnings.
--
Peter G
at
present. These functions have also re-introduced the string growth problem
(see http://www.mail-archive.com/perl6-internals@perl.org/msg09322.html);
the patch and sample program below can be used to illustrate this.
--
Peter Gibbs
EmKel Systems
Index: core.ops
access to the innards
of a STRING struct can be restricted to string.c? This would mean the logic
to access a COWed substring would be confined to one place. After the
introduction of string_to_cstring, there are very few places that access
s->bufstart directly anyway.
--
Peter Gibbs
EmKel Systems
have become the centre of the allocation
process. If anybody has any problems with this basic concept, please let me
know asap.
The next stage will be the creation of a header arena and memory pool for
constant strings.
Files changed: resources.h, resources.c, memory.c
--
Peter Gibbs
EmKel System
ther person to be keeping a close eye on what I'm doing, to
try to avoid too many major disasters. Hopefully, once the current exercise
is finished, we will have a pretty tight and stable memory management
system, which shouldn't need much hand-holding.
--
Peter Gibbs
EmKel Systems
ry allocation of the multiple pools, rather than any inherent
improvements.
--
Peter Gibbs
EmKel Systems
phase2.patch
Description: Binary data
that is pure
duplication; this can be moved to a separate common function.
--
Peter Gibbs
EmKel Systems
phase2.patch
Description: Binary data
buffer += pool->unit_size;
> +cur_buffer += (Buffer *)((char *)cur_buffer + pool->unit_size);
That should be
> +cur_buffer = (Buffer *)((char *)cur_buffer + pool->unit_size);
--
Peter Gibbs
EmKel Systems
Index: resources.c
omewhat improved for string-intensive programs (eg
life). Because of the split pools, non-string buffers do not suffer any
additional overhead, so performance of non-string-intensive programs should
be unaffected.
Feedback welcome.
--
Peter Gibbs
EmKel Systems
phase3.patch
Description: Binary data
at means putting the original buffer length in the
tail, and using
true_bufstart = (char *)s->bufstart - (tail->buflen - s->buflen)
in compact_string_pool?
--
Peter Gibbs
EmKel Systems
a COWed string will still believe that
the buffer is shared until a GC collection run occurs, and therefore could
result in buffers being copied unnecessarily. Your system eliminates this
problem; however, I believe that Dan may be averse to using a linked list -
we'll see.
--
Peter Gibbs
gram sometime:
substr S0, "constant", 0, 8, "variable"
print "This is a "
print "constant"
print " constant.\n"
end
This can be disallowed at assembly time (see patch below); but, as a general
rule, where should we be preserving our cons
ortality problem; a previous patch
fixed one cause of it (which is why the comment is there), but another
problems remains - see
http://www.mail-archive.com/perl6-internals@perl.org/msg09311.html for a
post on this subject, which never elicited any response.
--
Peter Gibbs
EmKel Systems
could be generally useful anyway.
[Steve - it seems to me that the 'normal' buffer pool should just be
replaced by the size 0 pool in your new system? I would think twice about
incorporating strings, as that might complicate COW, if it ever happens.]
--
Peter Gibbs
EmKel Systems
combined.patch
Description: Binary data
Chris Ball wrote:
> t/op/stacks.ok 28/29#Failed test (t/op/stacks.t at line 592)
> # got: ''
> # expected: '43210-1
This test has a missing 'end' in the code and segfaults. Patch below.
--
Peter Gibbs
E
# New Ticket Created by "Peter Gibbs"
# Please include the string: [netlabs #583]
# in the subject line of all future correspondence about this issue.
# http://bugs6.perl.org/rt2/Ticket/Display.html?id=583 >
Chris Ball wrote:
> t/op/stacks.ok 28/29#Failed test (
not. If we were to eliminate the separation of
pools based on type, perhaps we need to consider some other means of having
multiple pools, purely for performance reasons. This could, for example, be
based on the buffer header size, so each resource pool would have its own
memory pool.
--
Peter Gibbs
EmKel Systems
handling these situations soon. If anybody is
interested, I will resync my previous 'neonate' patch - it needs a bit of
work to fit with the latest changes to resources.c
--
Peter Gibbs
EmKel Systems
ocked DOD runs
are effectively cancelled, whereas they should just be postponed.
--
Peter Gibbs
EmKel Systems
# New Ticket Created by "Peter Gibbs"
# Please include the string: [netlabs #601]
# in the subject line of all future correspondence about this issue.
# http://bugs6.perl.org/rt2/Ticket/Display.html?id=601 >
Attached is a simplified version of a previous patch to allow buf
# New Ticket Created by "Peter Gibbs"
# Please include the string: [netlabs #602]
# in the subject line of all future correspondence about this issue.
# http://bugs6.perl.org/rt2/Ticket/Display.html?id=602 >
Attached patch to core.ops implements neonate protection for the '
ns, and
re-implement the original concept of telling a PMC to move its data to a new
location. That would require scanning PMCs during memory pool compaction -
at which point we are pretty close to unification of buffers and PMCs
anyway??
Comments?
--
Peter Gibbs
EmKel Systems
# New Ticket Created by "Peter Gibbs"
# Please include the string: [netlabs #607]
# in the subject line of all future correspondence about this issue.
# http://bugs6.perl.org/rt2/Ticket/Display.html?id=607 >
Attached is another patch to implement copy-on-write strings.
Summa
h bit holds the real string length. That way you
start where you need to, but you can still find the COW marker off
the end of the original buffer.
[End quote]
--
Peter Gibbs
EmKel Systems
resource
types - what do you think?
--
Peter Gibbs
EmKel Systems
mstances, which is accepted as an empty string
by some, but not all, string handling code. In this case, attempting to
store the string into an array is invoking string_copy, which assumes the
input to be valid. IIRC Dan stated some time ago that checks for null were
not to be included, but I don't know what the current status is.
--
Peter Gibbs
EmKel Systems
string_pool() to increase the pool size by a factor of
> >(0.5 - pct_freed_last_time) if it reclaimed less than 50% of memory
> >reduced this to 21 collections, much fewer of which reclaimed abysmally
> >small amounts of memory.
And Dan Sugalski replied:
> Interesting. Could
while blocked could set a flag, and the unblocking would
trigger the blocked collection - this should reduce the impact of blocking.
--
Peter Gibbs
EmKel Systems
For now, just delete config_h.in from the STICKY_FILES line in
config/gen/makefiles/root.in and re-run Configure.pl
I'm sure Brent will sort it out properly later.
--
Peter Gibbs
EmKel Systems
integer, the
flag stays set; this could also cause unpredictable behaviour.
For now, just assume that any attempt to turn one type of PMC into another
is likely to have undesirable consequences, and you should be safe.
--
Peter Gibbs
EmKel Systems
# New Ticket Created by "Peter Gibbs"
# Please include the string: [netlabs #619]
# in the subject line of all future correspondence about this issue.
# http://bugs6.perl.org/rt2/Ticket/Display.html?id=619 >
The attached patch improves performance for programs that allocate
te any ideas you have (or, better still, just write the
thing - I really don't mind!)
--
Peter Gibbs
EmKel Systems
I managed to send the wrong version of the patch on the previous post!
Herewith the correct (I hope) one.
Apologies to all.
--
Peter Gibbs
EmKel Systems
reclaim.patch
Description: Binary data
and 4
Starting quicksort with 3 and 3
Starting quicksort with 5 and 4
Starting quicksort with 6 and 5
dd
aa
bb
ee
cc
Am I missing something somewhere?
--
Peter Gibbs
EmKel Systems
esolved, albeit not optimally, in the current version.
--
Peter Gibbs
EmKel Systems
Index: include/parrot/interpreter.h
===
RCS file: /home/perlcvs/parrot/include/parrot/interpreter.h,v
retrieving revision 1.45
diff -u -r1.45 interpreter.h
-
# New Ticket Created by "Peter Gibbs"
# Please include the string: [netlabs #628]
# in the subject line of all future correspondence about this issue.
# http://bugs6.perl.org/rt2/Ticket/Display.html?id=628 >
Following patch adds dependencies entry for hash.c to Makefile.
Stop
# New Ticket Created by "Peter Gibbs"
# Please include the string: [netlabs #629]
# in the subject line of all future correspondence about this issue.
# http://bugs6.perl.org/rt2/Ticket/Display.html?id=629 >
Attached patch does some fairly radical things to the memory manag
# New Ticket Created by "Peter Gibbs"
# Please include the string: [netlabs #691]
# in the subject line of all future correspondence about this issue.
# http://bugs6.perl.org/rt2/Ticket/Display.html?id=691 >
Index: RESPON
aster machine
and see what happens.
--
Peter Gibbs
EmKel Systems
doesn't want to use them due to the problems that occur when we
> start using exceptions (and longjmp, etc).
If performance has to halve in order to implement such features, I hope
somebody plans to write Parrot::Lite!
--
Peter Gibbs
EmKel Systems
> Finallythe unification of buffers and PMCs means that buffers can now
> point to things of their own accord, without requiring that they be
> surrounded by an accompanying PMC type.
How about the other way round? If the one-size-fits-all PMCs were to be
replaced by custom structures, then everything could be a PMC, and
buffer headers as a separate resource could just disappear!
--
Peter Gibbs
EmKel Systems
ed block, first allocate 16K, discard the amount before the alignment
point, and dish out the rest as 15 (or 16 if you're really lucky) 1K aligned
pages. I seriously considered this when I changed my buffer memory to be
paged instead of a single allocation per memory pool; but I haven't act
and perlstring that I also left out originally.
--
Peter Gibbs
EmKel Systems
ontext and act
accordingly. If context was an enumeration, a simple switch statement
would suffice. In this case, it might be interesting to benchmark the
consequences of replacing some of the other get_X vtable functions.
e.g. get_integer(pmc) -> get_value(pmc, CONTEXT_INTEGER)
Comments, any
Peter Gibbs wrote:
> vtable method get_scalar(pmc)
> vtable method get_value(pmc, context)
Having broken the rule of posting before drinking coffee in the morning,
this is obviously nonsense - neither of these will work, because we don't
know the return type of these fu
t) {
case CONTEXT_SCALAR: return BASETYPE_INTVAL;
}
}
void set_pmc(PMC* value)
{
switch (value->get_preferred_type(INTERP, value, CONTEXT_SCALAR)) {
case BASETYPE_INTVAL:
set_integer(INTERP, SELF, value);
break;
}
}
--
Peter Gibbs
EmKel Systems
C in any way. The future 'assign Px, Py' will call a vtable function
(set_pmc); however, in the quoted example, the pure register level
behaviour is all that is intended.
--
Peter Gibbs
EmKel Systems
PerlInt - in future, the perl compiler
will have to create a PMC with the correct type initially, and this
morphing will be removed for typed variables.
--
Peter Gibbs
EmKel Systems
to somewhere close to
that again.
--
Peter Gibbs
EmKel Systems
to the Buffer header structure
arose in a thread started by Dan himself, with contributions from Mike
Lambert and myself:
http:[EMAIL PROTECTED]/msg11424.html
The specific message from Dan prohibiting it was:
http:[EMAIL PROTECTED]/msg11570.html
--
Peter Gibbs
EmKel Systems
both fields to determine if transcoding is required. I do not
believe that the two fields are orthogonal, and therefore the
number of charsets would be less than #chartypes * #encodings.
b) some alterations to the single vtable thus created, in
particular the addition of a find_substring method.
--
sed on point 2. I will be sending
my patch to Mike so he can see if there is anything he can use.
--
Peter Gibbs
EmKel Systems
to do a diff) is an experimental version of string_repeat that
tries to reduce the number of calls to memcpy. I can't find my notes
at the moment as to what benefit it gave - perhaps you might like
to try it sometime. (Note that the string_make call will need to be
changed to the split chartype/encodin
an improvement on Hanoi of about 15% using your
COW patch, and your COW is better on both tests than mine.
--
Peter Gibbs
EmKel Systems
of the
loop. (patch attached, but not fully tested)
On my machine, that changes 5000 lives from 168 seconds to 133
(tested for one run each only)
I'm sure there is more that can be done, but that may help for now.
--
Peter Gibbs
EmKel Systems
dod.patch
Description: Binary data
1 - 100 of 159 matches
Mail list logo
