On Fri, Jul 02, 2010 at 12:35:55AM +0400, Ruslan Zakirov wrote: > Hi. > > Looks like you moved clock doc as well.
Yes, because a key part of the work on memory profiling is generalizing what gets measured, which means generalizing "clocks". A better name would be good. Perhaps "probe". I'll update the file. Thanks. Tim. > 01.07.2010 21:44 пользователь <[1][email protected]> > написал: > > Revision: 1320 > Author: [2][email protected] > Date: Thu Jul  1 10:43:41 2010 > Log: Start distilling various bits of info on memory profiling into a > handy reference for the future. > > [3]http://code.google.com/p/perl-devel-nytprof/source/detail?r=1320 > > Added: >  /trunk/MemoryProfiling.pod > Modified: >  /trunk/HACKING > > ======================================= > --- /dev/null > +++ /trunk/MemoryProfiling.pod  Thu Jul  1 10:43:41 2010 > @@ -0,0 +1,74 @@ > +=head1 Ideas and Plans for Memory Profiling with NYTProf > + > +XXX I<This mostly a dumping ground for thoughts, and links at the > moment>. > +Somewhere to gather the info so it'll be easier to find when someone > +wants to work on it. > + > +Generalize the concepts of clocks. Have a structure defining a 'clock' > with > +pointers to functions to get the time, subtract times to get ticks, > return > +the resolution etc. Give them names and attributes (cpu, realtime etc). > +User could then pick a clock by name. By default we'd pick the best > available > +realtime clock (or best available cputime clock if usecputime=1 option > set). > + > +Conjectural terminology: > + > +   "clock" means some measuring mechanism like get_clock(), times(), > getrusage(), > +   that may yield multiple pieces of information with a single call. > + > +   "measure" is one specific item generated by a clock. > + > +   Clock "time"=times(), measures: "time.user", "time.user+sys" etc > +   Clock "clock"=clock_gettime(), measures: "clock.realtime", > "clock.monotonic" etc > +   Clock "rusage"=getrusage(), measures: "rusage.majflt", > "rusage.nvcsw" etc > +   Clock "memory" measures: "memory.bytes", "mem.allocs" etc > +   Clock "arena" measures: "arena.svs", "arena.bytes" etc > + > +Clocks need to be abstracted. Probably via a structure with function > pointers > +plus ways to copy clock output (struct) and get the difference between > two > +clock outputs. > + > +Use the differencing logic where we currently handle times in the > statement and > +subroutine profilers. > + > +=head1 Email threads > + > +XXX find the various threads > + > +XXX plus Nicholas's experimental patch > + > +=head1 Possibly Relevant Perl Modules > + > +Per-process memory information: > + > +   [4]http://search.cpan.org/perldoc?Devel::Mallinfo > + > +Arena, stash, and pad based memory reporters: > + > +   [5]http://search.cpan.org/perldoc?Devel::Gladiator > +   [6]http://search.cpan.org/dist/Internals-DumpArenas/ > +   [7]http://search.cpan.org/dist/Internals-GraphArenas/ > +   [8]http://search.cpan.org/perldoc?Devel::DumpSizes > +   [9]http://search.cpan.org/perldoc?Devel::Arena > + > +Per-object memory size reporters: > + > +   [10]http://search.cpan.org/perldoc?Devel::Size > +   [11]http://search.cpan.org/perldoc?Devel::Size::Report > + > +Others: > + > +   [12]http://search.cpan.org/dist/Devel-Memalyzer/ > +   > [13]http://search.cpan.org/perldoc?Devel::Memalyzer::Plugin::ProcSmaps > + > +=head1 Other Items of Interest > + > +"Memory Efficient Perl" slides by jjore > + > +   [14]http://docs.google.com/present/view?id=dg7kgpct_24cjs3c9fv > +   [15]http://diotalevi.isa-geek.net/~josh/090402/frontend.png > + > +memory visualization tools > + > +   [16]http://netjam.org/spoon/viz/ > + > +=cut > ======================================= > --- /trunk/HACKING    Mon Jun  7 08:37:11 2010 > +++ /trunk/HACKING    Thu Jul  1 10:43:41 2010 > @@ -133,27 +133,14 @@ > >  *** For core only > > +See MemoryProfiling.pod file > + >  Store raw NYTPROF option string in the data file. >  Include parsed version in report index page. > >  Add actual size and mtime of fid to data file. (Already in data file > as zero, >  just needs the stat() call.) Don't alter errno. > > -Generalize the concepts of clocks. Have a structure defining a 'clock' > with > -pointers to functions to get the time, subtract times to get ticks, > return > -the resolution etc. Give them names and attributes (cpu, realtime etc). > -User could then pick a clock by name. By default we'd pick the best > available > -realtime clock (or best available cputime clock if usecputime=1 option > set). > - > -[Conjectural terminology: "clock" means some measuring mechanism, like > -get_clock(), times(), getrusage(), that may yield multiple pieces of > -information, and "measure" is one specific item. > -Clock "time"=times(), measures: "time.user", "time.user+sys" etc > -Clock "clock"=clock_gettime(), measures: "clock.realtime", > "clock.monotonic" etc > -Clock "rusage"=getrusage(), measures: "rusage.majflt", "rusage.nvcsw" > etc > -Clock "memory" measures: "memory.bytes", "mem.allocs" etc > -] > - >  Add help option which would print a summary of the options and exit. >  Could also print list of available clocks for the clock=N option >  (using a set of #ifdef's) > @@ -178,14 +165,6 @@ >  the regex. If it does then save the current $DB::profile value and set > a new one. >  When the sub exits restore the previous $DB::profile value. > > -Could optionally track resource usage per sub. Data sources could be > perl sv > -arenas (clone visit() function from sv.c) to measure number of SVs & > total SV > -memory, plus getrusage()). Abstract those into a structure with > functions to > -subtract the difference. Then use the same logic to get inclusive and > exclusive > -values as we use for inclusive and exclusive subroutine times. > -Also possibly track the memory allocated to lexical pad SVs > -(for given sub at given depth). > - >  Work around OP_UNSTACK bug > ([17]http://rt.perl.org/rt3/Ticket/Display.html?id=60954) >  while ( foo() ) {  # all calls to foo should be from here >    ... > @@ -346,9 +325,6 @@ >  'self-contained' and can be archived and thrown around as a > tarball/zip and >  still used for further analysis. > > -The pseudo-sub "main::BEGIN" doesn't appear to be 'called' in NYTProf > data. > -Perhaps it should. > - >  To stress-test NYTProf using perl's own test suite, set env vars: >  NYTPROF='file=/tmp/nytprof.out:addpid=1:nameanonsubs=0:nameevals=0' >  PERL5OPT='-d:NYTProf' > @@ -393,8 +369,6 @@ > >  Fix inconsistency in results of href_for_*() subs > > -Make generation of call graph .dot files optional and/or optimize to > make faster > - >  String evals could/should be tied into the subroutine profiler. >  That would give inclusive timings which we don't have at the moment. >  The evals appear in the html report as if they're calls but the > timings are > @@ -414,19 +388,4 @@ >  xs calling optimization) it'll be the most recent sub entry that was > recorded. > >  Generate extra report pages for xsubs in packages that don't have > source code. > - > -USE_SITECUSTOMIZE causes two "Unable to determine line number" warnings > at start of > -every program: > -  (-e:0)    nextstate > -  Unable to determine line number in -e > -  (-e:0)    gvsv(main::!) > -  (-e:0)    nextstate > -  Unable to determine line number in -e > -  (-e:0)    > const(PV("/usr/local/perl512-dev/lib/site_perl/5.12.0/[18]sitecustomize.pl"\0)) > -because it unshifts some code onto PL_preambleav (like -M/-V etc): > -"BEGIN { do {local $!; -f '%s/[19]sitecustomize.pl'} && do > '%s/[20]sitecustomize.pl' }" > -which toke.c then concats into PL_linestr in the 'PL_preambled' block > -(where PL_perldb, PL_minus_E, PL_minus_n, PL_minus_p are handled). > -Would be nice to find a way to silence the warnings in this case. > -Even better if the profiler could know it was profiling preamble code. > -I can't see an obvious way but I'm fairly clueless re toke.c > +They're currently all dumped into the 'main' file. > > -- > You've received this message because you are subscribed to > the Devel::NYTProf Development User group. > > Group hosted at:  [21]http://groups.google.com/group/develnytprof-dev > Project hosted at:  [22]http://perl-devel-nytprof.googlecode.com > CPAN distribution:  [23]http://search.cpan.org/dist/Devel-NYTProf > > To post, email:  [24][email protected] > To unsubscribe, email:  > [25][email protected] > > -- > You've received this message because you are subscribed to > the Devel::NYTProf Development User group. > > Group hosted at: [26]http://groups.google.com/group/develnytprof-dev > Project hosted at: [27]http://perl-devel-nytprof.googlecode.com > CPAN distribution: [28]http://search.cpan.org/dist/Devel-NYTProf > > To post, email: [email protected] > To unsubscribe, email: [email protected] > > References > > Visible links > 1. mailto:[email protected] > 2. mailto:[email protected] > 3. http://code.google.com/p/perl-devel-nytprof/source/detail?r=1320 > 4. http://search.cpan.org/perldoc?Devel::Mallinfo > 5. http://search.cpan.org/perldoc?Devel::Gladiator > 6. http://search.cpan.org/dist/Internals-DumpArenas/ > 7. http://search.cpan.org/dist/Internals-GraphArenas/ > 8. http://search.cpan.org/perldoc?Devel::DumpSizes > 9. http://search.cpan.org/perldoc?Devel::Arena > 10. http://search.cpan.org/perldoc?Devel::Size > 11. http://search.cpan.org/perldoc?Devel::Size::Report > 12. http://search.cpan.org/dist/Devel-Memalyzer/ > 13. http://search.cpan.org/perldoc?Devel::Memalyzer::Plugin::ProcSmaps > 14. http://docs.google.com/present/view?id=dg7kgpct_24cjs3c9fv > 15. http://diotalevi.isa-geek.net/~josh/090402/frontend.png > 16. http://netjam.org/spoon/viz/ > 17. http://rt.perl.org/rt3/Ticket/Display.html?id=60954 > 18. http://sitecustomize.pl/ > 19. http://sitecustomize.pl/ > 20. http://sitecustomize.pl/ > 21. http://groups.google.com/group/develnytprof-dev > 22. http://perl-devel-nytprof.googlecode.com/ > 23. http://search.cpan.org/dist/Devel-NYTProf > 24. mailto:[email protected] > 25. mailto:[email protected] > 26. http://groups.google.com/group/develnytprof-dev > 27. http://perl-devel-nytprof.googlecode.com/ > 28. http://search.cpan.org/dist/Devel-NYTProf -- You've received this message because you are subscribed to the Devel::NYTProf Development User group. Group hosted at: http://groups.google.com/group/develnytprof-dev Project hosted at: http://perl-devel-nytprof.googlecode.com CPAN distribution: http://search.cpan.org/dist/Devel-NYTProf To post, email: [email protected] To unsubscribe, email: [email protected]
