Hi all,
1) global.inc seems to be a bit to heavy for use in ajax code etc. We
could try to split the global.inc in for example a common.inc and a
real global.inc. The common.inc could then be responsible for the
oblivious code from the global.inc and could require the global.inc.
Through this way the real pages on the system would use common.inc and
have all the code, and the ajax and other requests are using
global.inc which would only define the basic paths, autoloaders,
install check etc. This would avoid having the online, visit events to
be triggered on every ajax call. It's enough to trigger them for each
request.
Excellent idea, chop the ******* into smaller pieces ;-)
2) The css files are loaded dynamically each time. Though they are
never changed, except when the registrations change. They are also
built on php files which reads the active registrations and then
render the css files with the import statement. It could be a solution
to statically generate these import files when the registrations are
changing (which is not very often) so that the css file can be sent
statically. Through this way we could already save 3 calls to the
global.inc and the registrations table.
If and when you're able to take all variables into account that render
these dynamic css files, I see no problems with this. Do keep in mind
that the one for content objects also includes a "trick" to get around
the (ludicrous) CSS file limit in IE.
3) There are a lot of javascript / ajax calls that are called already
when the page is rendered the first time. For example for paths,
translations, themes etc. We think this should be optimized by caching
the utilities.js file and remove some oblivious calls to some
settings. These settings (like web path, theme etc) could be given to
the javascript file by printing them as html in the header. We could
also optimize this by avoiding that the system would do ajax calls
when a page just finished rendering. In our opinion it should be best
that ajax calls should only be called when a user interaction has
happened.
I would strongly argue against adding them in the HTML, but we could
simply add caching to utilities.js for these values or whatever else we
don't want to constantly repeat. (there must be a jquery plugin for
that, no?)
... and I'll just continue here with Laurent's remarks:
- If not already done apache can be configured to emit a expiracy header
in the far future for static files - i.e. css and js. If you hit the
refresh in the browser - F5 - it will refresh cached files by emiting a
"if modified since" header. In this case apache will return either a
unmodified html header or the modififed file. So it is always possible
to refresh those files in dev.
Probably something which could be a "suggestion" in an installation /
optimization manual since it's Apache inherent, but ...
- The php files that dynamically emit css and js should emit an html
expiracy header in the - very - far future in production and in not so
far future in dev. There is already a parameter for that. Plus they
should check the "if modified since" header and should return either 304
or file's content accordingly
... this would obviously be something we can do because of the dynamic
nature of the thing.
- js files are not minified. That is remove "unnecessary" chars like
spaces, comments, etc for production. Several facilities exist for that.
There you have it ... for production. Could we possibly fix this by
using the production / dev setting AND having both minified versions and
regular (dev-friendly) versions of the javascript? It's a bit more in
terms of files and disk space, but it does avoid having to manually do
the switch for releases or packaging. It could be done at runtime and
cached in some way (apache headers, on the server hdd, whatever...) but
IMO that would create unnecessary overhead. We do need minified versions
for production though.
Adding a note to self: go trough the list of included javascript files
and filter out some which are more then likely not used anymore.
- Have one entry point for js as well as css. That would avoid having
too many html headers in the page.
*votes in favour*
- I believe it should be possible to generate one big file for css, and
possibly for js. That may be done dynamicaly if we emit the expiracy
headers and handle the if modified since request.
I can see that working for CSS, not so sure about JS either. Whether
it's done dynamically or actually compiled to a file when registrations
change ... either one works for me if it triggers the right response in
terms of caching. The advantage of a compiled file on the server would
be not having to dynamically render it for every single user / session.
You just compile it when it's vanished for some reason or whenever the
administrator adds / updates / removes a package or when he explicitly
asks for it or ... At any rate it's a pretty good idea.
- Cache admin content. That would require to install an object cache -
i.e. memcache, etc - We could cache there frequent queries such as
platform settings instead of retrieving from the db. For that to work we
would need to invalid the cache when modifying platform parameters. A
possible approach would be create an Cache class to abstract possible
cache implementations. Add cache pools - i.e. admin, weblcms, user,
whatever. Another approach could be to cache the user object in session
with frequently accessed parameters.
In general I agree with this, but as Sven stated in his reply to your
reply on his post ;-) we should be carefull about adding external
dependencies. If and when we implement something like this and it's
nothing more then the flick of a switch to make it work, then I'm all
for it. A lot of things still need caching in one way or another right
now, most of them pretty simple.
- implement a message queue and process events asynchronously.
Small line, big implications ;-)
Isn't there something about this on the support website already as well?
At any rate (and I think you already posted something like that as well)
the info messages should be less annoying. Already have some nice ideas
for that, will put them on the support website as soon as possible.
That's it for me ... thank you both for the interesting feedback. Keep
up the good work!
Best regards,
Hans
On 9/03/2011 16:30, Sven Vanpoucke wrote:
Hello all
We at hogent are currently working at improving the performance of the
chamilo platform. We have found a very good software package which
makes it possible to profile our database queries which is called
Jetprofiler. We stumbled upon some issues where a lot of queries were
used to render one page (for example the browser of the repository
application). A lot of these issues where due to some basic structures
like registrations (which were contacted for each active application
separately), menu items (same as with registrations) and some other
unnecessary calls to the database. Adding a bit of caching here and
there (in tables where there are no more then 150 records) we were
able to limit the amount of queries drastically (for example in the
repository we went from 150+ queries for 10 content objects to only 37
queries).
Although optimizing these things improved the performance already by
30% (measured by stress tests with JMeter) we noticed in jetprofiler
that the top used queries where still the administration / settings
tables. After a deep analysis of the problem we noticed that the
queries as such were good queries, but they were listed as top queries
because they where called every often. More then once per request...
So we started digging into the code and soon noticed that global.inc
is called for a lot of side scripts that are launched either through
css files or javascript ajax. For example in the repository there are
3 calls to css files which are dynamically built (and even work
recursively) and there are 4 calls to an ajax file => utilities.php.
This means that global.inc.php is loaded at least 8 times per request,
thus retrieving all the settings, all the registrations and even add
some tracking that is used on each page. This means that there is a
lot of overhead created for one request. We think it should be
possible to minimize this overhead, especially since it's something
very generic that will happen on almost every page throughout the
system. We propose the following steps in order to optimize these
problems.
1) global.inc seems to be a bit to heavy for use in ajax code etc. We
could try to split the global.inc in for example a common.inc and a
real global.inc. The common.inc could then be responsible for the
oblivious code from the global.inc and could require the global.inc.
Through this way the real pages on the system would use common.inc and
have all the code, and the ajax and other requests are using
global.inc which would only define the basic paths, autoloaders,
install check etc. This would avoid having the online, visit events to
be triggered on every ajax call. It's enough to trigger them for each
request.
2) The css files are loaded dynamically each time. Though they are
never changed, except when the registrations change. They are also
built on php files which reads the active registrations and then
render the css files with the import statement. It could be a solution
to statically generate these import files when the registrations are
changing (which is not very often) so that the css file can be sent
statically. Through this way we could already save 3 calls to the
global.inc and the registrations table.
3) There are a lot of javascript / ajax calls that are called already
when the page is rendered the first time. For example for paths,
translations, themes etc. We think this should be optimized by caching
the utilities.js file and remove some oblivious calls to some
settings. These settings (like web path, theme etc) could be given to
the javascript file by printing them as html in the header. We could
also optimize this by avoiding that the system would do ajax calls
when a page just finished rendering. In our opinion it should be best
that ajax calls should only be called when a user interaction has
happened.
As we already have a first release we think that this is a really
important topic. We would like to fix these issues but are welcoming
all remarks / feedback.
Best regards
_______________________________________________
Dev mailing list
[email protected]
http://lists.chamilo.org/listinfo/dev
