Proposal: Test::Refcount
I've been hunting through my code, eventually coming to a bug where I had
a closure that referred to itself, as well as an object of mine. This
prevented my object's DESTROY handler from working, causing bugs.
I've been playing with Test::Memory::Cycle, but unfortunately that won't
solve this particular problem.
my $hash = {};
$hash-{object} = $some_object;
$hash-{cycle} = $hash;
No amount of cycle checking in $some_object is going to find this cycle.
If the cycled object is one like this; containing the only reference to
itself, then almost by definition there's nothing that could be walked
for memory cycles.
This is where I consider a refcount-assertion module instead; something I
could do this with:
use Test::Refcount;
my $object = Some::Class-new( ... );
has_onlyref( $object, '$object has only 1 reference' );
# Synonym for
has_refcount( $object, 1, '$object has only 1 reference' );
There's a very simple implementation for this I can think of; using:
use B qw( svref_2object );
sub refcount {
my $sv = svref_2object( $_[0] );
return $sv-REFCNT - 1; # Because @_ refers to it too
}
Does this sound good?
--
Paul LeoNerd Evans
[EMAIL PROTECTED]
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
signature.asc
Description: PGP signature
Re: Proposal: Test::Refcount
On 14 Jul 2008, at 18:12, Paul LeoNerd Evans wrote: No amount of cycle checking in $some_object is going to find this cycle. If the cycled object is one like this; containing the only reference to itself, then almost by definition there's nothing that could be walked for memory cycles. I've used Devel::Leak::Object[1] successfully in that kind of situation. It counts the number of blesses and destroys. It works well for the 'object makes a closure that references itself' scenario. [1] http://search.cpan.org/dist/Devel-Leak-Object/ -- Andy Armstrong, Hexten
Re: Port to PyConstructs
On Sat, Jul 12, 2008 at 1:50 AM, Shlomi Fish [EMAIL PROTECTED] wrote: http://construct.wikispaces.com/ In short, it's a library that parse binary structs. Cool! http://dev.perl.org/perl6/rfc/122.html
Re: Proposal: Test::Refcount
On Jul 14, 2008, at 12:12 PM, Paul LeoNerd Evans wrote: Does this sound good? Yes, and if it sounds like it would actually supercede Test::Memory::Cycle, I would be glad to abandon T:M:C in favor of yours. xoxo, Andy -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Proposal: Test::Refcount
use B qw( svref_2object );
sub refcount {
my $sv = svref_2object( $_[0] );
return $sv-REFCNT - 1; # Because @_ refers to it too
}
This sounds like a very good idea to me, I was looking for a module like
this in my last job. I would suggest a two modules though say
Devel::Refcount which exports the refcount function and the Test::Refcount
for writing tests.
Ivan Wills
--
email/jabber: [EMAIL PROTECTED]
/
/ _ _
/ \ / | | | |
/ \/ \_| | |
Re: Proposal: Test::Refcount
On Tue, 15 Jul 2008 05:01:14 +1000
Ivan Wills [EMAIL PROTECTED] wrote:
I would suggest a two modules though say Devel::Refcount
Is this Devel::Refcount really necessary, given how easy it is to build
using B::SV-REFCNT ?
Plus, the more levels of code, the more difficult it is to ensure the
count is correct. Consider
sub is_1ref
{
my ( $object, $name ) = @_;
my $count = refcount($object);
...
}
vs
sub is_1ref
{
my ( undef, $name ) = @_;
my $count = refcount($_[0]);
...
}
The $object in the first code creates a second reference, so you have to
subtract 1, whereas the @_ array seems special and doesn't have that
side-effect.
--
Paul LeoNerd Evans
[EMAIL PROTECTED]
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
signature.asc
Description: PGP signature
Re: Proposal: Test::Refcount
On Mon, 14 Jul 2008 13:46:42 -0500 Andy Lester [EMAIL PROTECTED] wrote: Yes, and if it sounds like it would actually supercede Test::Memory::Cycle, I would be glad to abandon T:M:C in favor of yours. OK; see what you think to the attached initial attempt. Not CPANned it yet; will wait and see some comments on it first. -- Paul LeoNerd Evans [EMAIL PROTECTED] ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ Test-Refcount-0.01.tar.gz Description: GNU Zip compressed data signature.asc Description: PGP signature
Re: Proposal: Test::Refcount
On Mon, Jul 14, 2008 at 08:32:52PM +0100, Paul LeoNerd Evans wrote:
sub is_1ref
{
my ( undef, $name ) = @_;
my $count = refcount($_[0]);
...
}
The $object in the first code creates a second reference, so you have to
subtract 1, whereas the @_ array seems special and doesn't have that
side-effect.
Until you take a reference to it, or various other things (I cant' remember the
canonical list, but push is in it, whereas shift doesn't seem to be (at least
a simple shift)):
$ perl -MDevel::Peek -e 'sub f {Dump $_[0]; warn [EMAIL PROTECTED]; Dump $_[0];
} f(\$v)'
SV = RV(0x817e18) at 0x800168
REFCNT = 1
FLAGS = (ROK)
RV = 0x800f18
SV = NULL(0x0) at 0x800f18
REFCNT = 2
FLAGS = ()
ARRAY(0x800f00) at -e line 1.
SV = RV(0x817e18) at 0x800168
REFCNT = 2
FLAGS = (ROK)
RV = 0x800f18
SV = NULL(0x0) at 0x800f18
REFCNT = 2
FLAGS = ()
It would be dangerous to rely on this reference counting behaviour remaining
the same.
Nicholas Clark
Re: Proposal: Test::Refcount
On Tue, Jul 15, 2008 at 05:01:14AM +1000, Ivan Wills wrote: This sounds like a very good idea to me, I was looking for a module like this in my last job. I would suggest a two modules though say Devel::Refcount which exports the refcount function and the Test::Refcount for writing tests. Can't you use Devel::Peek to get the refcount? -- David Cantrell | http://www.cantrell.org.uk/david Blessed are the pessimists, for they test their backups
Re: Proposal: Test::Refcount
On Mon, 14 Jul 2008 20:59:34 +0100 David Cantrell [EMAIL PROTECTED] wrote: Can't you use Devel::Peek to get the refcount? I use B: use B qw( svref_2object ); my $SV = svref_2object($ref); my $refcount = $SV-REFCNT; -- Paul LeoNerd Evans [EMAIL PROTECTED] ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ signature.asc Description: PGP signature
Re: Proposal: Test::Refcount
On Mon, 14 Jul 2008 20:59:20 +0100 Nicholas Clark [EMAIL PROTECTED] wrote: It would be dangerous to rely on this reference counting behaviour remaining the same. Ah. Then perhaps it may be safer to use the normal my ( $object, $count, $name ) = @_; list assign idiom, then use -REFCNT - 1 -- Paul LeoNerd Evans [EMAIL PROTECTED] ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ signature.asc Description: PGP signature
Re: Proposal: Test::Refcount
I would suggest a two modules though say Devel::Refcount Is this Devel::Refcount really necessary, given how easy it is to build using B::SV-REFCNT ? Plus, the more levels of code, the more difficult it is to ensure the count is correct. I see your point although I would contend that a Devel::Refcount module would be useful even if Test::Refcount didn't use it as knowing about B::SV-REFCNT is not easy. The B:: namespace not a place I would think that may perl programers would go and try searching on CPAN for reference counting and you will have a hard time finding out about B::svref_2object call let alone the REFCNT parameter. This may be a problem with CPAN and general lower level that reference counting exists in. If Paul LeoNerd Evens doesn't want to create the module Devel::Refcount along with Test::Refcount I would be happy to do it my self. Ivan -- email/jabber: [EMAIL PROTECTED] / / _ _ / \ / | | | | / \/ \_| | |
Re: Proposal: Test::Refcount
On Tue, 15 Jul 2008 06:11:25 +1000
Ivan Wills [EMAIL PROTECTED] wrote:
If Paul LeoNerd Evens doesn't want to create the module Devel::Refcount
along with Test::Refcount I would be happy to do it my self.
Well, it's not that I don't want to as such.. I'm just not sure it's
really justified.
It's kindof a personal thing, but I have a disliking for the mass
accumulation of really small one-liner modules around the place. Would
anyone consider
package String::CaseInsensitive;
sub casecmp { lc $_[0] cmp lc $_[1] }
1;
as a module, for example?
When the module starts to get this small, I begin to wonder if it's more
of a documentation issue. Perhaps somewhere in the documentation about
objects and reference counts:
The reference count of an object can be obtained by
use B qw( svref_2object );
my $count = svref_2object($ref)-REFCNT;
--
Paul LeoNerd Evans
[EMAIL PROTECTED]
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
signature.asc
Description: PGP signature
Re: Proposal: Test::Refcount
OK; see what you think to the attached initial attempt. I just tried it and the tests fail on anonymous references in t/01count.t and t/02one.t, I'm using perl 5.8.8 on linux if that is of any help. Ivan Wills -- email/jabber: [EMAIL PROTECTED] / / _ _ / \ / | | | | / \/ \_| | |
Re: Proposal: Test::Refcount
On Tue, 15 Jul 2008 06:33:26 +1000 Ivan Wills [EMAIL PROTECTED] wrote: I just tried it and the tests fail on anonymous references in t/01count.t and t/02one.t, I'm using perl 5.8.8 on linux if that is of any help. Ahh.. 5.10 here. OK; I've changed it to use a normal lexical $object, then subtract 1 from the refcount, rather than rely on @_ not incrementing the count. Perhaps that'll be more portable. Try again attached... -- Paul LeoNerd Evans [EMAIL PROTECTED] ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ Test-Refcount-0.01.tar.gz Description: GNU Zip compressed data signature.asc Description: PGP signature
Re: Proposal: Test::Refcount
On Mon, 14 Jul 2008 21:42:23 +0100 Paul LeoNerd Evans [EMAIL PROTECTED] wrote: OK; I've changed it to use a normal lexical $object, then subtract 1 from the refcount, rather than rely on @_ not incrementing the count. Perhaps that'll be more portable. Actually it occurs to me this may not help, because the reference will still be in @_ as well. If it still fails, perhaps try my $object = shift; my $count = shift; my $name = shift; that way it'll remove the ref. from @_ as it goes. -- Paul LeoNerd Evans [EMAIL PROTECTED] ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ signature.asc Description: PGP signature
Re: Proposal: Test::Refcount
* On Mon, Jul 14 2008, Paul LeoNerd Evans wrote:
On Tue, 15 Jul 2008 06:11:25 +1000
Ivan Wills [EMAIL PROTECTED] wrote:
If Paul LeoNerd Evens doesn't want to create the module Devel::Refcount
along with Test::Refcount I would be happy to do it my self.
Well, it's not that I don't want to as such.. I'm just not sure it's
really justified.
It's kindof a personal thing, but I have a disliking for the mass
accumulation of really small one-liner modules around the place.
I like these. It's as important to reuse simple things as it is to
reuse complicated things.
Would anyone consider
package String::CaseInsensitive;
sub casecmp { lc $_[0] cmp lc $_[1] }
1;
as a module, for example?
No, but that's because lc is the hard part, not the call to cmp. If
Perl didn't have lc built in, you'd bet I'd want the lc function
from a module, rather than the simple (but wrong) tr/[A-Z]/[a-z]/.
When the module starts to get this small, I begin to wonder if it's more
of a documentation issue. Perhaps somewhere in the documentation about
objects and reference counts:
The reference count of an object can be obtained by
use B qw( svref_2object );
my $count = svref_2object($ref)-REFCNT;
Considering B is core, there is probably no need for a module to do
this. But, does this always return correct results? Does the
svref_2object call affect the REFCNT? I am too lazy to check, but these
are nice questions for the documentation to answer. If svref_2object
affects its argument, then it's probably best to write a module that
compensates for this behavior.
Regards,
Jonathan Rockway
--
print just = another = perl = hacker = if $,=$
Re: Proposal: Test::Refcount
On Mon, 14 Jul 2008 18:28:57 -0500
Jonathan Rockway [EMAIL PROTECTED] wrote:
Considering B is core, there is probably no need for a module to do
this. But, does this always return correct results? Does the
svref_2object call affect the REFCNT? I am too lazy to check, but these
are nice questions for the documentation to answer. If svref_2object
affects its argument, then it's probably best to write a module that
compensates for this behavior.
Actually, now I look at it there seems to be more complicated things
going on.
Originally I tried testing it with just the HASH or ARRAY based object
types people tend to use. I thought I'd try out all the reference types,
and here's the behaviour I get:
A new SCALAR ref has REFCNT 2
A new ARRAY ref has REFCNT 1
A new HASH ref has REFCNT 1
A new CODE ref has REFCNT 2
A new GLOB ref has REFCNT 3
A new Regex ref has REFCNT 1
[see attached program]
It seems interesting that a new SCALAR or CODE ref has 2 references
already, and a GLOB has 3. I guess the CODE one -might- be explained by
its PAD or something, and maybe the GLOB one has more references in the
Perl symbol table, but these are pure guesses. I also cannot explain the
SCALAR one.
This behaviour seems to complicate the idea of simply asserting
REFCNT == 1.
--
Paul LeoNerd Evans
[EMAIL PROTECTED]
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
#!/usr/bin/perl -w
use strict;
use B qw( svref_2object );
my %refs = (
SCALAR = do { \my $var },
ARRAY = [],
HASH = +{},
CODE = sub {},
GLOB = \*SomeNewName,
Regex = qr/foo/,
);
foreach my $type (qw( SCALAR ARRAY HASH CODE GLOB Regex )) {
my $REFCNT = svref_2object($refs{$type})-REFCNT;
printf A new %6s ref has REFCNT %d\n, $type, $REFCNT;
}
signature.asc
Description: PGP signature
Re: Proposal: Test::Refcount
On Tue, 15 Jul 2008 00:44:12 +0100
Paul LeoNerd Evans [EMAIL PROTECTED] wrote:
SCALAR = do { \my $var },
It seems I can make the SCALAR ref have refcount 1 by changing this to
SCALAR = do { my $var; \$var },
Various initialisations {e.g my $var = 1} also keep it happy.
Just the CODE and GLOB to go then...
--
Paul LeoNerd Evans
[EMAIL PROTECTED]
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
signature.asc
Description: PGP signature
Re: Proposal: Test::Refcount
* On Mon, Jul 14 2008, Paul LeoNerd Evans wrote:
On Tue, 15 Jul 2008 00:44:12 +0100
Paul LeoNerd Evans [EMAIL PROTECTED] wrote:
SCALAR = do { \my $var },
It seems I can make the SCALAR ref have refcount 1 by changing this to
SCALAR = do { my $var; \$var },
Various initialisations {e.g my $var = 1} also keep it happy.
Just the CODE and GLOB to go then...
Hmm, it might be a good idea to ask about this on p5p. It would be good
to get this right Once And For All and make it really easy to use.
Memory cycles are the most common cause of memory leaks in Perl, and
they're really easy to fix once you know about them. So a module that
makes them easy to find would be very good for everyone :)
Thanks for your work so far ;)
Regards,
Jonathan Rockway
--
print just = another = perl = hacker = if $,=$
Begging for money - too tacky?
A friend recently reminded me of the RRDB author's vast list of donations he's received for his work, and I was thinking howzabout me? I was wondering if anyone had ever put anything in their module POD asking for donations, and if it worked. I wrote something up I was thinking of adding to mine, and I was wondering if people thought this was too tacky.. Here's what I wrote ... =head1 DONATIONS If you'd like to thank me for the work I've done on this module, please consider making a donation to me via PayPal (send in thanks. I spend a lot of free time creating free software, and would appreciate any support you'd care to offer. Please note that I am Bnot suggesting that you Bmust do this in order for to continue working on this module. I will continue to do so, inasmuch as I have in the past, for as long as it interests me. Similarly, a donation made in this way will probably not make me work on this module much more, unless I get so many donations that I can consider working on free software full time, which seems unlikely at best. /*== VegGuide.Org Your guide to all that's veg ==*/
