Jonathan, I don't know if this is the proper place to post this, unless you are looking to have it added to core. I have a feeling core contributors are going to chime in and say this more than likely belongs the in django-users mailing list. When you post it there though, I for one think this sounds like a great little project and will definitely e interested in taking a look at this, if your company is willing to open-source it.
On Fri, Oct 15, 2010 at 10:35 AM, Jonathan S <[email protected]>wrote: > Hi all, > > At my current job, I spent a week of programming on a preprocessor for > Django templates. > In short, what it does is compiling the templates in a more compact > version, dropping all useless information and preprocessing > templatetags where possible. The Apache benchmark tools showed a page > load improvement up to twice as fast in Django 1.2, and 4 times as > fast in the older Django version and HTML pages are now only half as > big in filesize now. > > It is completely transparent, there are no changes needed to the > templates, and should only be installed as a template loader (wrapping > around the original loaders). > > What is does, in the Django language is: > * preprocess {% trans %} and {% blocktrans %} when they don't take > variables parameters. (the template loader will have a 'cache' for > each language) > * preprocess inheritance. Combining parent and child templates by > replacing the blocks, and filling in {{block.super}}. {% block %} tags > can be removed because all inheritance has been determined. (Note, > this doesn't work with {% extends variable_name %}, but I think it's a > bad idea to have variable inheritance. (but can be disabled if you > would need it.) > * preprocess {% include "path" %}, if path is not a variable. > * grouping {% load statements %} > * resolve all {% url %} when they don't take variable parameters. > * preprocess {% now "Y" %} (we will restart our server at least once, > after a year transition. :p ) > * We have a few tags, like {% google_analytics %}, which constantly > output the same code, this is implemented as a custom preprocessor > extension for ourself. > > Now, if the template is HTML, we do a little more optimizations: > * Remove all whitespace between block-level HTML tags > * Removing empty class attributes, like class="", (which may appear as > a result of class="{% block name %}{% endblock %}", where the child > template din't implement the block.) > * Merge all <style type="text/css"> into the first <style/> node. > * replace all multiple whitespaces, by one space, but not in a <pre> > or <textarea> node. > * Remove HTML comments, but not conditional comments > * We have to option to merge <script/> nodes as well, but it seems > that it's not desirable in all situations. > > Further, we minify embedded CSS and javascript (renaming variable > names to be as short as possible, and removing really all whitespace > where allowed.) Javascript and CSS comments are removed too. > > ==Technically==. We wrote a hybrid parse tree, having Django, HTML, > CSS and Javascript nodes in the same parse tree. (first parsing Django > templatetags, then with another lexer, parsing the HTML in the django > tree, etc...) This made it pretty easy to do all the necessary > operations. (Actually, it's based on a lexer that I wrote several > years ago in a dead hobby project, but improved for this: > http://code.google.com/p/python-pages/ ) > > This 'compilation' is not really fast (can be improved), but that's > not really important because it only has do be done once for every > template. So, for every following request, the performance is still > much better than what we noticed when using a run-time compressor like > {% compress %} before. > > For parsing javascript or HTML, it's important to note that there are > a few limitations. It's still a tree structure, and like <b><i>....</ > b></i> is a very bad nesting habbit in HTML, we have the same > limitation. For instance: > > Don't do: > {% if test %} <a ... {% else %} <a ... {% endif %} .... >link</a> > but do: > <a {% if test %}... {% else %}...{% endif %}>link</a> > It's not good to open a tag twice, while closing it only once, so go > for the second. > Also don't open a javascript ' { ' in another 'scope' than were you > close the ' } '. > > This are bad programming habbits anyway. Our codebase of templates is > already pretty large, and almost everything worked out of the box > after applying the preprocessor, even the Dango Debug toolbar and > Django Admin. This means that chances are small you hit these > limitations, and if you do, the preprocessor will tell you. > > Isn't it cool to have a parser, which when he encounters a javascript > syntax error, is being able to tell you at which line in the django > template source file it's caused. :) > > The preprocessor has not yet been released as open-source. But I'm > willing to ask my company to do, if there is a need for this in the > Django community. Personally, I'm really satisfied with the results. > > What do you guys think? > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<django-developers%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- Brendan Smith, IT Coordinator National Priorities Project http://www.nationalpriorities.org http://www.costofwar.com http://www.facebook.com/nationalpriorities 413 584 9556 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
