Cheers, I would like to add a few comments on top of what Sven already said.
First tracking some simple ajax calls with a profiler - i.e. user leave - we have about 90% of the work done by global and 10% of actual work. Digging into "global" a lot of the work is taken by admin calls - i.e. platform settings. Which obviously do not change a lot. The pages that generates the css and js files do not answer the "if modified since" html header and do not generate a cache header in the far future. They do but in the very near future. There are a lot of small css and js files generating a lot of chatty calls. Even with the cache turned on for css and js the browser seems to suffer. Plus we have several page headers for js that takes uncesseray space. Js files are not minimized. Possible solutions: - If not already done apache can be configured to emit a expiracy header in the far future for static files - i.e. css and js. If you hit the refresh in the browser - F5 - it will refresh cached files by emiting a "if modified since" header. In this case apache will return either a unmodified html header or the modififed file. So it is always possible to refresh those files in dev. - The php files that dynamically emit css and js should emit an html expiracy header in the - very - far future in production and in not so far future in dev. There is already a parameter for that. Plus they should check the "if modified since" header and should return either 304 or file's content accordingly - js files are not minified. That is remove "unnecessary" chars like spaces, comments, etc for production. Several facilities exist for that. - Have one entry point for js as well as css. That would avoid having too many html headers in the page. - I believe it should be possible to generate one big file for css, and possibly for js. That may be done dynamicaly if we emit the expiracy headers and handle the if modified since request. - Cache admin content. That would require to install an object cache - i.e. memcache, etc - We could cache there frequent queries such as platform settings instead of retrieving from the db. For that to work we would need to invalid the cache when modifying platform parameters. A possible approach would be create an Cache class to abstract possible cache implementations. Add cache pools - i.e. admin, weblcms, user, whatever. Another approach could be to cache the user object in session with frequently accessed parameters. - implement lazy loading in Global to avoid unnecessary calls. That is retrieve data from cache/db on first access instead of retrieving them upfront. - implement a message queue and process events asynchronously. - I agree with Sven's proposition we certainly need a lighgter "global" include for light calls such as ajax. I am only wondering if lazy loading would be enough or if we actually need to split the file in two. Talking about Message Queue I would like to take the opportunity to introduce Daniel to the community. Daniel is a student here at the University of Geneva and will work on how we can implement a Message Queue for events. I already sent him the remarks I got but feel free to add more requirements. Le 09.03.2011 16:30, Sven Vanpoucke a écrit : > Hello all > > We at hogent are currently working at improving the performance of the > chamilo platform. We have found a very good software package which > makes it possible to profile our database queries which is called > Jetprofiler. We stumbled upon some issues where a lot of queries were > used to render one page (for example the browser of the repository > application). A lot of these issues where due to some basic structures > like registrations (which were contacted for each active application > separately), menu items (same as with registrations) and some other > unnecessary calls to the database. Adding a bit of caching here and > there (in tables where there are no more then 150 records) we were > able to limit the amount of queries drastically (for example in the > repository we went from 150+ queries for 10 content objects to only 37 > queries). > > Although optimizing these things improved the performance already by > 30% (measured by stress tests with JMeter) we noticed in jetprofiler > that the top used queries where still the administration / settings > tables. After a deep analysis of the problem we noticed that the > queries as such were good queries, but they were listed as top queries > because they where called every often. More then once per request... > > So we started digging into the code and soon noticed that global.inc > is called for a lot of side scripts that are launched either through > css files or javascript ajax. For example in the repository there are > 3 calls to css files which are dynamically built (and even work > recursively) and there are 4 calls to an ajax file => utilities.php. > This means that global.inc.php is loaded at least 8 times per request, > thus retrieving all the settings, all the registrations and even add > some tracking that is used on each page. This means that there is a > lot of overhead created for one request. We think it should be > possible to minimize this overhead, especially since it's something > very generic that will happen on almost every page throughout the > system. We propose the following steps in order to optimize these > problems. > > 1) global.inc seems to be a bit to heavy for use in ajax code etc. We > could try to split the global.inc in for example a common.inc and a > real global.inc. The common.inc could then be responsible for the > oblivious code from the global.inc and could require the global.inc. > Through this way the real pages on the system would use common.inc and > have all the code, and the ajax and other requests are using > global.inc which would only define the basic paths, autoloaders, > install check etc. This would avoid having the online, visit events to > be triggered on every ajax call. It's enough to trigger them for each > request. > > 2) The css files are loaded dynamically each time. Though they are > never changed, except when the registrations change. They are also > built on php files which reads the active registrations and then > render the css files with the import statement. It could be a solution > to statically generate these import files when the registrations are > changing (which is not very often) so that the css file can be sent > statically. Through this way we could already save 3 calls to the > global.inc and the registrations table. > > 3) There are a lot of javascript / ajax calls that are called already > when the page is rendered the first time. For example for paths, > translations, themes etc. We think this should be optimized by caching > the utilities.js file and remove some oblivious calls to some > settings. These settings (like web path, theme etc) could be given to > the javascript file by printing them as html in the header. We could > also optimize this by avoiding that the system would do ajax calls > when a page just finished rendering. In our opinion it should be best > that ajax calls should only be called when a user interaction has > happened. > > As we already have a first release we think that this is a really > important topic. We would like to fix these issues but are welcoming > all remarks / feedback. > > Best regards > -- _____________________________________ Meilleures salutations. Laurent Opprecht
<<attachment: laurent_opprecht.vcf>>
_______________________________________________ Dev mailing list Dev@lists.chamilo.org http://lists.chamilo.org/listinfo/dev
