Re: Failing to understand getrusage()
On Sat, Mar 11, 2006 at 01:49:50AM +0300, Yar Tikhiy wrote:
On Tue, Mar 07, 2006 at 06:12:59PM +0200, Kostik Belousov wrote:
It may be desirable to add ru_maxrss sampling at the calcru time too.
Something like this:
Index: sys/kern/kern_resource.c
===
RCS file: /usr/local/arch/ncvs/src/sys/kern/kern_resource.c,v
retrieving revision 1.156
diff -u -r1.156 kern_resource.c
--- sys/kern/kern_resource.c22 Feb 2006 16:58:48 - 1.156
+++ sys/kern/kern_resource.c7 Mar 2006 16:10:27 -
@@ -853,9 +853,16 @@
struct rusage *rup;
{
struct proc *p;
+ struct vmspace *vm;
+ long rss;
p = td-td_proc;
PROC_LOCK(p);
+ vm = p-p_vmspace;
+ rss = pgtok(vmspace_resident_count(vm));
+ if (rup-ru_maxrss rss)
+ rup-ru_maxrss = rss;
+
switch (who) {
case RUSAGE_SELF:
Please excuse me for a dumb question, but what makes ru_maxrss so
different from other ru_ fields that it deserves special handling
in kern_getrusage()? Perhaps the all-or-nothing approach will be
better for the sake of consistency...
Current resource usage accounting is inaccurate (i.e. done at sampling
points) only for several fields, ru_maxrss being one of them. E.g.,
ru_nsignals is precise.
Sure, the best would be implementing approach like solaris microaccounting
(AFAIR, solaris could measure used parts of the time-slice where
the thread runs on CPU, and do this measure on demand, not stressing
the system when the exact numbers are not needed).
My small fix just add little more sence to result of maxrss calculation,
making it to never return meaningless values like 0. Better, pmaps
shall be modified to correctly set maxrss (this seems to be not hard,
could someone look at the patch if I implement it ?).
pgp4ig0oR3kx0.pgp
Description: PGP signature
Re: Failing to understand getrusage()
On Tue, Mar 07, 2006 at 06:12:59PM +0200, Kostik Belousov wrote:
On Tue, Mar 07, 2006 at 06:06:31PM +0300, Yar Tikhiy wrote:
On Tue, Mar 07, 2006 at 12:11:56PM +0200, Kostik Belousov wrote:
On Tue, Mar 07, 2006 at 02:15:56AM +0300, Yar Tikhiy wrote:
Personally, I'd like to say a me too. /me too fails to see why
in a quiet, idle system ru_maxrss is very unpredictable over numerous
runs of the test program, both before and after the malloc+memset.
Filling the memory with a non-zero value doesn't matter. Is it the
Heizenberg daemon at work? :-)
I think that this is a statclock in work :). Just add some busy loops
before each calls to getrusage like
for (x = 0; x 0x100; x++)
getpid();
and you would get statisically stable results:
deviant% ./1mb
before: 424, after: 1548
deviant% ./1mb
before: 424, after: 1548
See,
% sysctl kern.clockrate
kern.clockrate: { hz = 1000, tick = 1000, profhz = 666, stathz = 133 }
133 Hz is very slow on 3GHz machine, and curproc-p_stats-p_ru is
updated on statclock tick, see sys/kern/kern_clock.c.
This sounds very clear and reasonable. I shouldn't have forgotten
about the driving role of statclock in collecting all rusage stats,
including those related to memory consumption.
It may be desirable to add ru_maxrss sampling at the calcru time too.
Something like this:
Index: sys/kern/kern_resource.c
===
RCS file: /usr/local/arch/ncvs/src/sys/kern/kern_resource.c,v
retrieving revision 1.156
diff -u -r1.156 kern_resource.c
--- sys/kern/kern_resource.c 22 Feb 2006 16:58:48 - 1.156
+++ sys/kern/kern_resource.c 7 Mar 2006 16:10:27 -
@@ -853,9 +853,16 @@
struct rusage *rup;
{
struct proc *p;
+ struct vmspace *vm;
+ long rss;
p = td-td_proc;
PROC_LOCK(p);
+ vm = p-p_vmspace;
+ rss = pgtok(vmspace_resident_count(vm));
+ if (rup-ru_maxrss rss)
+ rup-ru_maxrss = rss;
+
switch (who) {
case RUSAGE_SELF:
Please excuse me for a dumb question, but what makes ru_maxrss so
different from other ru_ fields that it deserves special handling
in kern_getrusage()? Perhaps the all-or-nothing approach will be
better for the sake of consistency...
--
Yar
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Failing to understand getrusage()
On Tue, Mar 07, 2006 at 02:15:56AM +0300, Yar Tikhiy wrote:
On Thu, Mar 02, 2006 at 11:50:29PM +, Nick Barnes wrote:
At 2006-03-02 22:24:17+, Nik Clayton writes:
I'm failing to understand how getrusage() works, which is a bit
perplexing,
because it doesn't seem like it would be terribly complicated.
ru_maxrss is the maximum resident set size, not the heap size.
malloc(big) doesn't grow the resident set. Touching the memory you
have allocated will grow the resident set. Try this:
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
p = malloc(SIZE);
assert(p)
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
for (i=0; iSIZE; ++i) {
p[i] = 0;
}
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
Well, there was a call to memset() in the original Nik's program
while your code just does the same by itself.
Personally, I'd like to say a me too. /me too fails to see why
in a quiet, idle system ru_maxrss is very unpredictable over numerous
runs of the test program, both before and after the malloc+memset.
Filling the memory with a non-zero value doesn't matter. Is it the
Heizenberg daemon at work? :-)
I think that this is a statclock in work :). Just add some busy loops
before each calls to getrusage like
for (x = 0; x 0x100; x++)
getpid();
and you would get statisically stable results:
deviant% ./1mb
before: 424, after: 1548
deviant% ./1mb
before: 424, after: 1548
See,
% sysctl kern.clockrate
kern.clockrate: { hz = 1000, tick = 1000, profhz = 666, stathz = 133 }
133 Hz is very slow on 3GHz machine, and curproc-p_stats-p_ru is
updated on statclock tick, see sys/kern/kern_clock.c.
pgpKxJflUdGZG.pgp
Description: PGP signature
Re: Failing to understand getrusage()
On Tue, Mar 07, 2006 at 12:11:56PM +0200, Kostik Belousov wrote:
On Tue, Mar 07, 2006 at 02:15:56AM +0300, Yar Tikhiy wrote:
Personally, I'd like to say a me too. /me too fails to see why
in a quiet, idle system ru_maxrss is very unpredictable over numerous
runs of the test program, both before and after the malloc+memset.
Filling the memory with a non-zero value doesn't matter. Is it the
Heizenberg daemon at work? :-)
I think that this is a statclock in work :). Just add some busy loops
before each calls to getrusage like
for (x = 0; x 0x100; x++)
getpid();
and you would get statisically stable results:
deviant% ./1mb
before: 424, after: 1548
deviant% ./1mb
before: 424, after: 1548
See,
% sysctl kern.clockrate
kern.clockrate: { hz = 1000, tick = 1000, profhz = 666, stathz = 133 }
133 Hz is very slow on 3GHz machine, and curproc-p_stats-p_ru is
updated on statclock tick, see sys/kern/kern_clock.c.
This sounds very clear and reasonable. I shouldn't have forgotten
about the driving role of statclock in collecting all rusage stats,
including those related to memory consumption.
Thank you for resolving our doubts! :-)
--
Yar
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Failing to understand getrusage()
On Tue, Mar 07, 2006 at 06:06:31PM +0300, Yar Tikhiy wrote:
On Tue, Mar 07, 2006 at 12:11:56PM +0200, Kostik Belousov wrote:
On Tue, Mar 07, 2006 at 02:15:56AM +0300, Yar Tikhiy wrote:
Personally, I'd like to say a me too. /me too fails to see why
in a quiet, idle system ru_maxrss is very unpredictable over numerous
runs of the test program, both before and after the malloc+memset.
Filling the memory with a non-zero value doesn't matter. Is it the
Heizenberg daemon at work? :-)
I think that this is a statclock in work :). Just add some busy loops
before each calls to getrusage like
for (x = 0; x 0x100; x++)
getpid();
and you would get statisically stable results:
deviant% ./1mb
before: 424, after: 1548
deviant% ./1mb
before: 424, after: 1548
See,
% sysctl kern.clockrate
kern.clockrate: { hz = 1000, tick = 1000, profhz = 666, stathz = 133 }
133 Hz is very slow on 3GHz machine, and curproc-p_stats-p_ru is
updated on statclock tick, see sys/kern/kern_clock.c.
This sounds very clear and reasonable. I shouldn't have forgotten
about the driving role of statclock in collecting all rusage stats,
including those related to memory consumption.
It may be desirable to add ru_maxrss sampling at the calcru time too.
Something like this:
Index: sys/kern/kern_resource.c
===
RCS file: /usr/local/arch/ncvs/src/sys/kern/kern_resource.c,v
retrieving revision 1.156
diff -u -r1.156 kern_resource.c
--- sys/kern/kern_resource.c22 Feb 2006 16:58:48 - 1.156
+++ sys/kern/kern_resource.c7 Mar 2006 16:10:27 -
@@ -853,9 +853,16 @@
struct rusage *rup;
{
struct proc *p;
+ struct vmspace *vm;
+ long rss;
p = td-td_proc;
PROC_LOCK(p);
+ vm = p-p_vmspace;
+ rss = pgtok(vmspace_resident_count(vm));
+ if (rup-ru_maxrss rss)
+ rup-ru_maxrss = rss;
+
switch (who) {
case RUSAGE_SELF:
pgpMImMfLdJf7.pgp
Description: PGP signature
Re: Failing to understand getrusage()
On Thu, Mar 02, 2006 at 11:50:29PM +, Nick Barnes wrote:
At 2006-03-02 22:24:17+, Nik Clayton writes:
I'm failing to understand how getrusage() works, which is a bit perplexing,
because it doesn't seem like it would be terribly complicated.
ru_maxrss is the maximum resident set size, not the heap size.
malloc(big) doesn't grow the resident set. Touching the memory you
have allocated will grow the resident set. Try this:
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
p = malloc(SIZE);
assert(p)
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
for (i=0; iSIZE; ++i) {
p[i] = 0;
}
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
Well, there was a call to memset() in the original Nik's program
while your code just does the same by itself.
Personally, I'd like to say a me too. /me too fails to see why
in a quiet, idle system ru_maxrss is very unpredictable over numerous
runs of the test program, both before and after the malloc+memset.
Filling the memory with a non-zero value doesn't matter. Is it the
Heizenberg daemon at work? :-)
--
Yar
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
Failing to understand getrusage()
I'm failing to understand how getrusage() works, which is a bit perplexing,
because it doesn't seem like it would be terribly complicated.
I've attached the code. My aim is to verify that I can use getrusage() to
do (admittedly crude) instrumentation of which functions in my program are
allocating lots of memory[1]. So I figure I can call getrusage() at various
points and look at the ru_maxrss member.
I'm confused because the results I'm getting don't make a great deal of
sense. The attached program, when run 10 times one after the other,
produces the following output;
before: 0, after: 1300
before: 0, after: 0
before: 0, after: 0
before: 188, after: 188
before: 0, after: 0
before: 452, after: 452
before: 0, after: 0
before: 0, after: 1316
before: 0, after: 0
before: 0, after: 0
'before' is ru_maxrss before 'malloc(1024 * 1024)', 'after' is ru_maxrss
afterwards. Different runs produce slightly different numbers -- the point
is that they vary widely, and in some cases are '0' for both cases.
Those results don't look sane to me. From doing some googling it looks like
there may be a delay between the process's resident set size increasing and
the stats that getrusage() uses being updated.
If that's the case, can I (portably?) wait for the getrusage() stats to
synchronise before I read them? If it's not the case, what am I doing
wrong? Is there another (portable) mechanism I use to find out how big my
process is?
N
[1] I'm actually instrumenting Perl applications, using Perl's built-in
debugger, instrumenting at the Perl level, so solutions of the form Use
this profiling application aren't going to be applicable. I'm using
BSD::Resource, which wraps getrusage(), and odd results from that led me to
this C test.
/* Call getrusage() before and after allocating 1MB of RAM, and see what
the maxrss figure is */
#include sys/types.h
#include sys/time.h
#include sys/resource.h
#include stdio.h
int
main(void)
{
struct rusage ru;
char *ptr;
intrc;
rc = getrusage(RUSAGE_SELF, ru);
if(rc == -1) {
perror(getrusage(): );
return 1;
}
printf(before: %ld, , ru.ru_maxrss);
ptr = malloc(1024 * 1024);
if(ptr == NULL) {
perror(malloc(): );
return 1;
}
memset(ptr, 0, 1024 * 1024); /* Zero-fill the memory */
rc = getrusage(RUSAGE_SELF, ru);
if(rc == -1) {
perror(getrusage(): );
return 1;
}
printf(after: %ld\n, ru.ru_maxrss);
free(ptr);
return 0;
}
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Failing to understand getrusage()
At 2006-03-02 22:24:17+, Nik Clayton writes:
I'm failing to understand how getrusage() works, which is a bit perplexing,
because it doesn't seem like it would be terribly complicated.
ru_maxrss is the maximum resident set size, not the heap size.
malloc(big) doesn't grow the resident set. Touching the memory you
have allocated will grow the resident set. Try this:
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
p = malloc(SIZE);
assert(p)
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
for (i=0; iSIZE; ++i) {
p[i] = 0;
}
getrusage(RUSAGE_SELF, ru);
printf(%lu\n, ru.ru_maxrss);
Nick B
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
