(this mail has been sent 10 hours ago, seems it was too large. I send it again without attached docs)
Le 30/03/2011 16:47, Ian Walls a écrit : > Fellow Developers, Hello everybody, > Last night, I stayed up late running circ/circulation.pl > <http://circulation.pl> through NYTProf, to get an idea where we may > be able to optimize circulation for speed. After much frustration > (darn session IDs...), I was able to get a report. The results > were... not exactly what I expected. For those who have used or developed on Koha 2.x, you probably have noticed that Koha 2.x was much more slower than Koha 3.x So, yesterday and tonight i've taken my laptop and made a lot of testings. First of all, i'll thank forever ... The New York Times ... Why will you tell me ? Because of http://search.cpan.org/~adamk/Aspect-Library-NYTProf-1.00/lib/Aspect/Library/NYTProf.pm This tool is really amazing. You run perl -d:NYTProf mainpage.pl ; nytprofhtml --open And you get a highly detailled HTML report with the time spent in each line/sub/module to run mainpage.pl. Here is the result : http://depot.biblibre.com/ppoulain/nytprof.ini/ (the .ini is to keep track of the perf before anything optimized) Note : tests done on my laptop, that is not a server. 1st test with master: Profile of mainpage.pl for 1.73s (of 2.18s), executing 155108 statements and 41041 subroutine calls in 249 source files and 64 string evals. I have investigated to find the longest timings & if we can do something. (note All the following improvement have been done incrementally) First of all, C4::Context and C4::Koha are loaded on everypage, so it's worth investigating == C4::Context == * spent 185ms just reading the config file. Ouch ! that's almost 10% of the time !!! (http://depot.biblibre.com/ppoulain/nytprof.ini/C4-Context-pm-13-line.html#230) I tried to hardcode the hashref instead of XMLin-ing the config file, the results are astonishing : after = Profile of mainpage.pl for 1.59s (of 1.93s), executing 117399 statements and 26939 subroutine calls in 241 source files and 63 string evals. => PROPOSAL = add a YAML version of the XML config file that we could use in // => PROPOSAL 2 = memcache the config file (hint : the config file in koha 2 was not xml !) * C4::Context, sub db_scheme2dbi, the i flag in /mysql/i cost 84ms !!! with /i, the duration is 84.4ms, without, it's 10µs !!! after = Profile of mainpage.pl for 1.55s (of 1.91s), executing 117398 statements and 26938 subroutine calls in 241 source files and 63 string evals. => PROPOSAL = just 'return "mysql"' at the beginning of this sub, as everything else is not working anyway ! == C4::Languages == * C4::Languages::getAllLanguages 43ms => heavily under optimized mysql query = returns all languages from language_subtag_registry and language_descriptions even if only a few localisations are installed. * C4::Languages::_build_languages_arrayref 15ms => same as for getAllLanguages I know there is a patch (from hdl) pending to improve this sub, i haven't tested it yet (http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6000) (hint : this sub appeared in 3.0, was not in 2.2 !) == Other packages == * C4::Koha.pm 15ms for use Business::ISBN, that is used only in 1 sub used only in a few pages (and none if no cover images are enabled unless i'm wrong) * C4::Letters use MIME::Lite spend 26ms, that are useless 90% of the time (why is C4::Letters loaded ! answer = because C4::Reserves is loaded by Items.pm that is loaded by Biblio.pm that is loaded by AuthoritiesMarc.pm that is loaded (uselessly) by mainpage.pl * CGI/Session/Driver/mysql.pm needs 79ms i've tested by switching to driver::File::Serializer (dropped to 21ms). this seems to work, but probably requires more testing than what I did * Date::Manip needs requires 46ms and is useless most of the time. after = Profile of mainpage.pl for 1.31s (of 1.60s), executing 103643 statements and 25380 subroutine calls in 215 source files and 55 string evals. => PROPOSAL = WHERE it's interesting we could replace the use XXXX by a require XXXX just before the lines that need the package. The require is loaded at run tim, so, when unneeded, the package won't be loaded == mainpage.pl == Mainpage.pl load some authorities, we do nothing with it. Commenting the use C4::Authorities and all related code results in: after = Profile of mainpage.pl for 1.27s (of 1.57s), executing 103000 statements and 25194 subroutine calls in 212 source files and 55 string evals. == Packages nested == * I also noticed that Auth.pm loads Members.pm (to retrieve patron informations and display them on the top right). Members.pm loads Reserves that loads Biblio.pm, Circulation.pm and Items.pm All those packages are the biggest to load. I've written a script, that display, for each file in C4 which package is loaded and which sub in the package loaded are used. It shows, for example, that there is a "use Koha.pm;" that is useless in AuthoritiesMarc.pm It also shows which sub are used, and how many times. For example, Circulation.pm calls "use C4::Biblio" and : In Circulation.pm the Biblio.pm, sub GetBiblioItemData is used 1 times In Circulation.pm the Biblio.pm, sub GetBiblioFromItemNumber is used 4 times That will be handy to find what can be cleaned & removed. I've attached the script and the result. If you want to run it yourself, just export PERL5LIB, KOHA_CONF, put the script in $KOHA directory, and run it ! == Other == * packages that uses the biggest amount of CPU are not C4 packages : * utf8_heavy-pl is the most cpu consuming thing : 132ms, I don't know if we can do something * Template::Parser needs 123ms, not sure we can do something * all XML stuff is highly consuming = loading MARC::File::XML requires 47ms in Biblio.pm, XML::Sax::Base require 40ms to be loaded => definetly, the best way to reduce CPU time is by pre-loading things, we already knew that == Next step == * i'll file a bug for the various improvements that can be made easily * i'll continue investigate & report my findings I'm aware it's a 1st step, and the long-term road is to reach data&code persistency. But cleaning what we already have may ease the next big step. And any speed improvement is a good thing, isn't it ! HTH -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
check_subs.pl
Description: application/perl-module
_______________________________________________ Koha-devel mailing list [email protected] http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
