Re: [ Memory ] Re: Thought
Michael G Schwern <[EMAIL PROTECTED]> writes: > On Thu, Oct 03, 2002 at 10:43:43AM +0100, Nicholas Clark wrote: > > The reason I'm saying it might not be much use to people unfamiliar with > > the internals of unix is precisely because it does return a precisely defined > > but not directly useful value - the highest address on the heap. > > If you read it, call perl code that creates a known perl structure, and read > > it again, doesn't go up directly and exactly related to the amount of memory > > that structure needed. Depending on how much other memory there is free, it > > may not go up at all, or it may go up more if the allocation system grabs a > > big block of free space. > > Ok, now I'm really confused. > > - sbrk() returns, effectively, the size of the heap. > - the heap may not go up when you create a perl structure That's true. There may be formerly-used and now free memory in the "heap" which can be reused. > > but MemUsed() is shown to be used like this: > > use Devel::Internals; > > my @array = (1..1); > MemUsed "creation of array"; > my @dup = @array; > MemUsed "duplication of array"; > > and MemUsed, in scalar context, returns the sbrk value, the size of the > heap. So I presume in the above code it simply returns the difference > between the sbrk value between the two calls. But Nick just said this isn't > actually the memory used, it's just the different in heap size. But the > function is called MemUsed! Well, it won't return the amount of memory for newly-created perl structures, but it should reflect the amount of memory the perl process takes from the operating system's virtual memory system. Hence the name MemUsed seems OK to me. For other systems which do not use sbrk(), MemUsed should be implemented differently. Regards, Slaven -- Slaven Rezic - [EMAIL PROTECTED] tknotes - A knotes clone, written in Perl/Tk. http://ptktools.sourceforge.net/#tknotes
Re: [ Memory ] Re: Thought
[EMAIL PROTECTED] writes:
> En op 4 oktober 2002 sprak Michael G Schwern:
> > The problem is when you put those two next to each other, one
> > promising a friendly interface, one a bare-metal interface,
> > it confuses the intent of the module. Is it for Joe End User
> > or is it for Joe Core Hacker?
>
> A module to report memory usage of any process (not just yourself),
> is do-able: on some Unices you might trample thru /proc file system;
> on Windows, the Performance Registry, and so on. But has it been done?
> I searched CPAN and could not find such a module.
Not a module, but a function which should work on FreeBSD and Linux:
=item currmem([$pid])
=for category System
Return ($mem, $realmem) of the current process or process $pid, if $pid
is given.
=cut
sub currmem {
my $pid = shift || $$;
if (open(MAP, "/proc/$pid/map")) { # FreeBSD
my $mem = 0;
my $realmem = 0;
while() {
my(@l) = split /\s+/;
my $delta = (hex($l[1])-hex($l[0]));
$mem += $delta;
if ($l[11] ne 'vnode') {
$realmem += $delta;
}
}
close MAP;
($mem, $realmem);
} elsif (open(MAP, "/proc/$pid/maps")) { # Linux
my $mem = 0;
my $realmem = 0;
while() {
my(@l) = split /\s+/;
my($start,$end) = split /-/, $l[0];
my $delta = (hex($end)-hex($start));
$mem += $delta;
if (!defined $l[5] || $l[5] eq '') {
$realmem += $delta;
}
}
close MAP;
($mem, $realmem);
} else {
undef;
}
}
__END__
> And, if you were to
> write such a module, what should you call it?
>
> I would find such a module generally useful in that it gives a sysadmin
> view of a Perl process (is it a memory pig compared to a Java process,
> say). Admittedly, such a module may not be very useful in figuring out
> how much memory a perl data structure is using, but I don't see how
> sbrk is any better in that regard. For a more fine-grained view, you
> need hooks into Perl internals (such as the Perl malloc).
>
Regards,
Slaven
--
Slaven Rezic - [EMAIL PROTECTED]
BBBike - route planner for cyclists in Berlin
WWW version: http://www.bbbike.de
Perl/Tk version: http://bbbike.sourceforge.net
Re: [ Memory ] Re: Thought
On Thu, Oct 03, 2002 at 02:35:14PM -0400, Benjamin Goldberg wrote: > If "what it does" is merely return the value of the C sbrk() function, > then IMHO, sbrk() is a perfectly good name. > > However, as to other possible names -- how about ProcessSize() ? I'm > not sure if this is really a valid description of what sbrk() returns, > though. I'm not certain it's as accurate as we might like to think. I am under the impression (quite possibly wrong, as I don't have the source code handy) that 1: Doug Lea's malloc will do anonymous mmap() to satisfy really big requests 2: glib malloc is a variant derived from Doug Lea's code Hence is it plausible that systems exist where the malloc that perl calls may normally get memory from the system with sbrk(), but occasionally use somewhere other than the system heap. So perl code might cause more memory to be used, but if we report back sbrk() as process size we're actually claiming a bit more for our "fact" than we can actually be sure of. If we call it sbrk() we can divorce ourselves from actually claiming that it is the one true source of memory :-) Not that this is helpful - I guess we want a direct sbrk() interface if sbrk() is on the system, and a process size interface that makes the best estimate it can. But I'm not writing this thing, and so far I guess I've not wanted it *that* much. [mm. maybe I have. number 3 on my hitlist of "what slows perl" is RAM. So seeing if a data structure change reduces RAM is interesting to me, the encode compiler, and everyone building perl from source] Nicholas Clark -- Even better than the real thing:http://nms-cgi.sourceforge.net/
