On Fri, Apr 8, 2011 at 9:15 AM, Andrey Zubko <[email protected]> wrote:
> Hello,
> I'm Andrey Zubko, student faculty of Computer Science in Zaporozhye
> State Engineering Academy. I'm senior chief developer of a local
> Internet Service Provider "Telza" which provides Internet and
> telephony services. My responsibilities includes enhancing, improving
> existent Billing system that is written by myself.
>
> In GSOC program I want to improve template subsystem by integrating
> template compilation into python bytecode.
>
> Goals and challenges
> Integrating of template compilation has 2 goals. The first goal and
> the chief goal is to provide backward-compatibility to templates,
> custom tags, custom filters that are already written. The second
> goal is to minimize modifications of Django sources to provide more
> stability and faster integrating in trunk.
> Implementation
> Support of template compilation can be achieved by adding method
> 'compile' to every Node-derived class, and some other classes,
> functions, such as django.template.Variable,
> django.template.FilterExpression, django.template.NodeList, and others
> described further.
>
> 1. Node and Node-derived classes modifications
> Class Node and derived from it classes should have optional method –
> compile, it returns list of python bytecode commands for write in
> compiled template. It receives parameter parent_name that will
> contains name of uppermost tag/block. Parent_name is a name of
> uppermost tag/block that identifies entry in hash blocks_output that
> is used for caching all outputs. Only one case can provide parent_name
> to be overridden – meet of the '{% block %}' tag in a parent tag, this
> tag should override parent_name to its unique name. Approach for
> caching all output is used for providing template inheritance ability.
> Use of template Inheritance provides following cases :
> child template has one sequence of blocks, but parent template has
> another sequence of blocks
> child template has blocks that haven't specified in parent's template
>
> For solving this cases I provided list blocks_sequence_output that
> keeps sequence of blocks outputting. Implementation of 'compile'
> method in Node class :
> def compile(self,parent_name):
> self.generate_name(parent_name)
> compiled_code = ["t = Template('%s')" % self.template,
> "blocks_output['%s'] = blocks_output['%s'] +
> t._render(context)" % (self.parent_name,self.parent_name)]
> return compiled_code
>
> As you saw, in purposes of providing backward-compatibility all Node-
> derived classes, that haven't implemented 'compile' method, will use
> current rendering approach.
> Node-derived classes with implemented 'compile' method will be looked
> like :
> class TextNode(Node):
> def compile(self,parent_name):
> return ["blocks_output['%s'] = blocks_output['%s'] + \"\"\"%s
> \"\"\"" % (self.parent_name,
>
> self.parent_name,
>
> self.s)]
>
> As shown before 'compile' method is not so difficult to implement.
> To recursively compilation of Node-derived classes is used method
> 'compile' in the class NodeList :
> class NodeList(list):
> def generate_name(self,node):
> return id(node)
>
> def compile(self, parent_name=None):
> bits = []
> for node in self:
> if isinstance(node, Node):
> if parent_name is None or isinstance(node,
> BlockNode) :
> if isinstance(node, BlockNode) :
> parent_name_new = node.name
> else :
> parent_name_new = self.generate_name(node)
>
> bits.append("blocks_output['%s']=''" %
> parent_name_new)
> else :
> parent_name_new = parent_name
>
> bits.append(self.compile_node(node, parent_name_new))
> else:
> bits.append(node)
> return mark_safe(''.join([force_unicode(b) for b in bits]))
>
> Also this function is used for initialising blocks_output hashes for
> uppermost nodes.
>
> 2. Blocks and dynamical inheritance
> I have described before Node-derived classes improvements, but
> django.template.loader_tags.BlockNode, django.template.loader_tags,
> django.template.loader_tages.ExtendsNode and others have big influence
> on compiled template architecture design, that's why I have allocated
> whole section for them. Internal variables blocks_output,
> blocks_output_sequence are used for providing support of dynamic
> inheritance. They specify the way how template should be rendered, in
> what sequence, what blocks should be outputted to user.
> 'blocks_output' hash collects blocks or tags outputs, and
> blocks_output_sequence saves sequence of block outputting. Because
> parent template can contain its own sequence of blocks that is saved
> in blocks_output_sequence, we must update the child
> blocks_output_sequence without right of overriding it. Because parent
> template can contain own blocks that is not present in child template,
> we must update child blocks_output hash with not existing items in it.
> By solving this requirements class BlockNode will be looked like :
>
> class BlockNode(Node):
> def super_compile(self):
> compiled_code = ["blocks_output['%s'] = blocks_output['%s'] +
> blocks_output_parent['%s']" % (self.name,self.name,self.name)]
> return compiled_code
>
> def compile(self,parent_name):
> compiled_childs = self.nodelist.compile()
> return compiled_childs
>
> blocks_output_parent – is blocks_output hash received from parent
> template that contains cached parent output.
> For providing support of static and dynamic inheritance class
> ExtendsNode should have 'compile' method that will be looked like :
>
> class ExtendsNode(Node):
> def get_parent_compiled(self):
> if self.parent_name_expr: # dynamical inheritance
> parent = self.parent_name_expr.compile()
> if not parent :
> if self.parent_name_expr:
> error_msg = " Got this from the '%s' variable." %
> self.parent_name_expr.token
> raise TemplateSyntaxError(error_msg)
> else : # static inheritance
> parent = self.parent_name
> if not parent:
> error_msg = "Invalid template name in 'extends' tag:
> %r." % parent
> raise TemplateSyntaxError(error_msg)
> if hasattr(parent, 'compile'):
> return parent.compile() # parent is a Template object
>
> code = ["blocks_output, blocks_sequence_output_parent =
> get_compiled_template(%s,no_output=True)" % self.parent_name,
> "blocks_output_parent = blocks_output"]
>
> return code
>
> def compile(self,parent_name):
> compiled_parent = self.get_compiled_parent()
> return compiled_parent
>
> 3. Variables and filters modifications
> Support of variables and filters are implemented by
> django.template.FilterExpression class. It is wrapped by VariableNode
> class – Node-derived class that implements simple call of
> FilterExpression instance. For supporting compilation into python
> bytecode VariableNode class should contain 'compile' function that
> calls 'compile' method of FilterExpression instance :
>
> class VariableNode(Node):
> def __compile__(self,parent_name):
> return self.filter_expression.compile()
>
> And FilterExpression should have implemented 'compile' method in which
> it calls filters compiled functions and variable resolving :
>
> class FilterExpression(object):
> def compile(self, ignore_failures=False):
> if isinstance(self.var, Variable):
> obj =
> self.var.compile(ignore_failures,settings.COMPILED_TEMPLATE_STRING_IF_INVALID)
> else:
> obj = self.var
>
> for func, args in self.filters:
> arg_vals = ""
> i = 0
> for lookup, arg in args:
> if i > 0 :
> arg_vals = arg_vals + ","
> if not lookup:
> arg_vals = arg_vals + "\"%s\"" % mark_safe(arg)
> else:
> arg_vals = arg_vals + "%s" % arg.compile(context)
>
> if getattr(func, 'needs_autoescape', False):
> new_obj = "%s(%s,autoescape=context.autoescape,%s)" %
> (func.__name__,obj,arg_vals)
> else:
> new_obj = "%s(%s,%s)" % (func.__name__,obj,arg_vals)
> if getattr(func, 'is_safe', False) and isinstance(obj,
> SafeData):
> obj = "mark_safe(%s)" % new_obj
> elif isinstance(obj, EscapeData):
> obj = "mark_for_escaping(%s)" % new_obj
> else:
> obj = new_obj
> return [obj]
>
> By the way, 'compile' method FilterExpression has no parent_name param
> – it is omitted because FilterExpression can't produce a block – it
> can be called only from a block – VariableNode.
> And django.template.Variable receives parameters ignore_failures, and
> default value for variable :
>
> class Variable(object):
> def compile(self,ignore_failures,default_value):
> if self.lookups is not None:
> value = "resolve(context,'%s',%s,'%s')" %
> (self.lookups,ignore_failures,default_value)
> else:
> # We're dealing with a literal, so it's already been
> "resolved"
> value = "'%s'" % self.literal
>
> if self.translate:
> value = "_(%s)" % value
>
> Using the Variable class introduced us to inner compiled template
> function named 'resolve'. 'resolve' function implements finding entry
> in context with looking name, value of entry can be simple value,
> class, function. 'resolve' receives params :
> 1.context – context variable that is passed to compiled template
> 'render' method
> 2.key – name of looking entry
> 3.ignore_failures – if ignore_failures is true and finding entry is
> failed, then it will return default_value or if it is omitted – None,
> otherwise it will raise VariableDoesNotExist exception
> 4.default_value – default value
>
> Compiled Template Architecture
> Compiled template contains two sections :
> 1.Dependencies section
> 2.Compiled code section
> Dependencies section is section that has default imports, and external
> imports of user-written custom tags, filters. External imports are
> determined by calling {% load %} tag.
> Compiled code section is a section that contains template's python
> byte code.
> Structure of the compiled code :
>
> # -- DEPENDENCIES SECTION
> # ----- Default imports
> from django.template import Template, get_compiled_template
> from django.template.defaultfilters import *
> # ----- User custom imports
> # from ... import ...
> # -- END OF DEPENDENCIES SECTION
>
> def resolve(context,ignore_failures,default_value):
> """ should be looked like method _resolve_lookup in
> django.template.Variable """
>
> blocks_output = {}
> blocks_output_sequence = []
> blocks_sequence_output_parent = None
>
> def output():
> for block_name in blocks_output_sequence :
> print blocks_output.get(block_name,'')
>
> def render(context,no_output=False):
> # START OF COMPILED TEMPLATE PYTHON BYTECODE
>
> # ...
>
> # END OF COMPILED TEMPLATE PYTHON BYTECODE
> if blocks_sequence_output_parent is not None :
> blocks_sequence_output = blocks_sequence_output_parent
> if no_output :
> return blocks_output, blocks_output_sequence
> else :
> output()
>
> Every compiled template has 3 functions :
> resolve – discussed earlier
> output – task of this function is to output blocks output cache in the
> determined sequence
> render – entrypoint of compiled template, that contains template's
> compiled python bytecode. It supports one optional parameter –
> 'no_output' – it is used only when called parent template – not to
> output, to return cached data.
>
> Examples
> Examples based on already written code :
> compiling variables and filters :
>
> from django.template import Parser, FilterExpression
>
> token = 'variable|default:"Default value"|date:"Y-m-d"'
> p = Parser('')
> fe = FilterExpression(token, p)
> fe.compile()
> =>
> 'date(default(resolve(context,variable,None,''),"Default value"),"Y-m-
> d")'
> template compilation sample :
> child.html :
> {% extends parent %}
> {% block first_block %}
> {% if True %}
> some_info
> {% endif %}
> {% endblock %}
> {% block second_block %}
> {% endblock %}
>
> parent.html :
> TextNode
> {% block first_block %}
> {% endblock %}
>
> =>
> Compiled child template :
> # -- DEPENDENCIES SECTION
> # ----- Default imports
> from django.template import Template, get_compiled_template
> from django.template.defaultfilters import *
> # ----- User custom imports
> # from ... import ...
> # -- END OF DEPENDENCIES SECTION
>
> def resolve(context,key,ignore_failures,default_value):
> """ should be looked like method _resolve_lookup in
> django.template.Variable """
>
>
> blocks_output = {}
> blocks_output_sequence = []
> blocks_sequence_output_parent = None
>
> def output():
> for block_name in blocks_output_sequence :
> print blocks_output.get(block_name,'')
>
> def render(context,no_output=False):
> blocks_output, blocks_sequence_output_parent =
> get_compiled_template(resolve(context,'parent',None,''),no_output=True)
> blocks_output_parent = blocks_output
>
> blocks_output['first_block'] = '' # entering block tag in NodeList
> compile
> blocks_sequence_output.append('first_block')
> if True :
> blocks_output['first_block'] = blocks_output['first_block'] +
> "some_info"
>
> blocks_output['second_block'] = '' # entering block tag in
> NodeList compile
>
> if blocks_sequence_output_parent is not None :
> blocks_sequence_output = blocks_sequence_output_parent
>
> if no_output :
> return blocks_output, blocks_output_sequence
> else :
> output()
>
> Compiled parent template :
> # -- DEPENDENCIES SECTION
> # ----- Default imports
> from django.template import Template, get_compiled_template
> from django.template.defaultfilters import *
> # ----- User custom imports
> # from ... import ...
> # -- END OF DEPENDENCIES SECTION
>
> def resolve(context,key,ignore_failures,default_value):
> """ should be looked like method _resolve_lookup in
> django.template.Variable """
>
>
> blocks_output = {}
> blocks_output_sequence = []
> blocks_sequence_output_parent = None
>
> def output():
> for block_name in blocks_output_sequence :
> print blocks_output.get(block_name,'')
>
> def render(context,no_output=False):
> blocks_output['first_block'] = '' # entering block tag in NodeList
> compile
> blocks_sequence_output.append('first_block')
>
> if blocks_sequence_output_parent is not None :
> blocks_sequence_output = blocks_sequence_output_parent
>
> if no_output :
> return blocks_output, blocks_output_sequence
> else :
> output()
>
> Milestones
> 1. Designing concept - current step
> 2. Coding "template compilation" subsystem
> 3. Writing unit-tests
> 4. Optimization of written code
A quick word of advice -- this timeline is probably enough to ensure
that you *won't* get selected for the Django GSoC. It doesn't provide
any evidence that you've done serious thought about how long the work
involved will take. A serious proposal needs to have a work breakdown
with 1-week granularity at the *most*. It should also give a clear
mechanism for evaluating progress by indicating the deliverables at
various stages of the project (e.g., after week 3, it will be possible
to write a compiled template containing only {% if %} and {% block %}
clauses).
As an additional complication for your project -- you should take a
look at your competition. You're proposing to work on a project that
has also been proposed by Armin Ronacher. Armin has a long and
demonstrated history of expertise when it comes to compiled template
engines. If you're proposing a simliar project, then you would be well
advised to clearly explain why:
1) your proposal is better than his, or
2) your proposal is a good compliment to his.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
