Hi. Looks like you moved clock doc as well.
01.07.2010 21:44 пользователь <[email protected]> написал: Revision: 1320 Author: [email protected] Date: Thu Jul 1 10:43:41 2010 Log: Start distilling various bits of info on memory profiling into a handy reference for the future. http://code.google.com/p/perl-devel-nytprof/source/detail?r=1320 Added: /trunk/MemoryProfiling.pod Modified: /trunk/HACKING ======================================= --- /dev/null +++ /trunk/MemoryProfiling.pod Thu Jul 1 10:43:41 2010 @@ -0,0 +1,74 @@ +=head1 Ideas and Plans for Memory Profiling with NYTProf + +XXX I<This mostly a dumping ground for thoughts, and links at the moment>. +Somewhere to gather the info so it'll be easier to find when someone +wants to work on it. + +Generalize the concepts of clocks. Have a structure defining a 'clock' with +pointers to functions to get the time, subtract times to get ticks, return +the resolution etc. Give them names and attributes (cpu, realtime etc). +User could then pick a clock by name. By default we'd pick the best available +realtime clock (or best available cputime clock if usecputime=1 option set). + +Conjectural terminology: + + "clock" means some measuring mechanism like get_clock(), times(), getrusage(), + that may yield multiple pieces of information with a single call. + + "measure" is one specific item generated by a clock. + + Clock "time"=times(), measures: "time.user", "time.user+sys" etc + Clock "clock"=clock_gettime(), measures: "clock.realtime", "clock.monotonic" etc + Clock "rusage"=getrusage(), measures: "rusage.majflt", "rusage.nvcsw" etc + Clock "memory" measures: "memory.bytes", "mem.allocs" etc + Clock "arena" measures: "arena.svs", "arena.bytes" etc + +Clocks need to be abstracted. Probably via a structure with function pointers +plus ways to copy clock output (struct) and get the difference between two +clock outputs. + +Use the differencing logic where we currently handle times in the statement and +subroutine profilers. + +=head1 Email threads + +XXX find the various threads + +XXX plus Nicholas's experimental patch + +=head1 Possibly Relevant Perl Modules + +Per-process memory information: + + http://search.cpan.org/perldoc?Devel::Mallinfo + +Arena, stash, and pad based memory reporters: + + http://search.cpan.org/perldoc?Devel::Gladiator + http://search.cpan.org/dist/Internals-DumpArenas/ + http://search.cpan.org/dist/Internals-GraphArenas/ + http://search.cpan.org/perldoc?Devel::DumpSizes + http://search.cpan.org/perldoc?Devel::Arena + +Per-object memory size reporters: + + http://search.cpan.org/perldoc?Devel::Size + http://search.cpan.org/perldoc?Devel::Size::Report + +Others: + + http://search.cpan.org/dist/Devel-Memalyzer/ + http://search.cpan.org/perldoc?Devel::Memalyzer::Plugin::ProcSmaps + +=head1 Other Items of Interest + +"Memory Efficient Perl" slides by jjore + + http://docs.google.com/present/view?id=dg7kgpct_24cjs3c9fv + http://diotalevi.isa-geek.net/~josh/090402/frontend.png + +memory visualization tools + + http://netjam.org/spoon/viz/ + +=cut ======================================= --- /trunk/HACKING Mon Jun 7 08:37:11 2010 +++ /trunk/HACKING Thu Jul 1 10:43:41 2010 @@ -133,27 +133,14 @@ *** For core only +See MemoryProfiling.pod file + Store raw NYTPROF option string in the data file. Include parsed version in report index page. Add actual size and mtime of fid to data file. (Already in data file as zero, just needs the stat() call.) Don't alter errno. -Generalize the concepts of clocks. Have a structure defining a 'clock' with -pointers to functions to get the time, subtract times to get ticks, return -the resolution etc. Give them names and attributes (cpu, realtime etc). -User could then pick a clock by name. By default we'd pick the best available -realtime clock (or best available cputime clock if usecputime=1 option set). - -[Conjectural terminology: "clock" means some measuring mechanism, like -get_clock(), times(), getrusage(), that may yield multiple pieces of -information, and "measure" is one specific item. -Clock "time"=times(), measures: "time.user", "time.user+sys" etc -Clock "clock"=clock_gettime(), measures: "clock.realtime", "clock.monotonic" etc -Clock "rusage"=getrusage(), measures: "rusage.majflt", "rusage.nvcsw" etc -Clock "memory" measures: "memory.bytes", "mem.allocs" etc -] - Add help option which would print a summary of the options and exit. Could also print list of available clocks for the clock=N option (using a set of #ifdef's) @@ -178,14 +165,6 @@ the regex. If it does then save the current $DB::profile value and set a new one. When the sub exits restore the previous $DB::profile value. -Could optionally track resource usage per sub. Data sources could be perl sv -arenas (clone visit() function from sv.c) to measure number of SVs & total SV -memory, plus getrusage()). Abstract those into a structure with functions to -subtract the difference. Then use the same logic to get inclusive and exclusive -values as we use for inclusive and exclusive subroutine times. -Also possibly track the memory allocated to lexical pad SVs -(for given sub at given depth). - Work around OP_UNSTACK bug ( http://rt.perl.org/rt3/Ticket/Display.html?id=60954) while ( foo() ) { # all calls to foo should be from here ... @@ -346,9 +325,6 @@ 'self-contained' and can be archived and thrown around as a tarball/zip and still used for further analysis. -The pseudo-sub "main::BEGIN" doesn't appear to be 'called' in NYTProf data. -Perhaps it should. - To stress-test NYTProf using perl's own test suite, set env vars: NYTPROF='file=/tmp/nytprof.out:addpid=1:nameanonsubs=0:nameevals=0' PERL5OPT='-d:NYTProf' @@ -393,8 +369,6 @@ Fix inconsistency in results of href_for_*() subs -Make generation of call graph .dot files optional and/or optimize to make faster - String evals could/should be tied into the subroutine profiler. That would give inclusive timings which we don't have at the moment. The evals appear in the html report as if they're calls but the timings are @@ -414,19 +388,4 @@ xs calling optimization) it'll be the most recent sub entry that was recorded. Generate extra report pages for xsubs in packages that don't have source code. - -USE_SITECUSTOMIZE causes two "Unable to determine line number" warnings at start of -every program: - (-e:0) nextstate - Unable to determine line number in -e - (-e:0) gvsv(main::!) - (-e:0) nextstate - Unable to determine line number in -e - (-e:0) const(PV("/usr/local/perl512-dev/lib/site_perl/5.12.0/ sitecustomize.pl"\0)) -because it unshifts some code onto PL_preambleav (like -M/-V etc): -"BEGIN { do {local $!; -f '%s/sitecustomize.pl'} && do '%s/sitecustomize.pl' }" -which toke.c then concats into PL_linestr in the 'PL_preambled' block -(where PL_perldb, PL_minus_E, PL_minus_n, PL_minus_p are handled). -Would be nice to find a way to silence the warnings in this case. -Even better if the profiler could know it was profiling preamble code. -I can't see an obvious way but I'm fairly clueless re toke.c +They're currently all dumped into the 'main' file. -- You've received this message because you are subscribed to the Devel::NYTProf Development User group. Group hosted at: http://groups.google.com/group/develnytprof-dev Project hosted at: http://perl-devel-nytprof.googlecode.com CPAN distribution: http://search.cpan.org/dist/Devel-NYTProf To post, email: [email protected] To unsubscribe, email: [email protected] -- You've received this message because you are subscribed to the Devel::NYTProf Development User group. Group hosted at: http://groups.google.com/group/develnytprof-dev Project hosted at: http://perl-devel-nytprof.googlecode.com CPAN distribution: http://search.cpan.org/dist/Devel-NYTProf To post, email: [email protected] To unsubscribe, email: [email protected]
