Hello, Russell Keith-Magee
Thanks for your advices.
*1. Detailed Timeline :*
- Designing concept : this step requires fully design writing for
achieving BACKWARD compatibility for all tags, and filters. Providing
backward-compatibility is a simple task. I've described it earlier by using
FilterExpression class.
- Coding "template compilation" subsystem :
- 4 days term - django.template.loader, django.template.loaders.* -
adding support for importing already compiled templates.
- 1 week term - implementing 'compile' method to
django.template.Template class, in this method will be processed
calling of
child nodelist compiling and writing to compiled template file.
- 2 weeks term - implementing 'compile' method to django.templates
- 1 weeks term - implementing 'compile' method to
django.template.loader_tags Node-derived classes.
- 3 days term - implementing support in
django.template.FilterExpression and django.template.Variable.In 5
weeks term will be produced such possibilities :
1. compiling, writing, importing compiled tempaltes files
2. support for compilation of default filters from
django.template.defaultfilters, and default tags from
django.template.defaulttags : , Node, Node-derived classes : TextNode,
VariableNode, BlockNode, ExtendsNode, ConstantIncludeNode,
IncludeNode,
AutoEscapeNode, CommentNode, CsrfTokenNode, CycleNode, FilterNode,
FirstOfNode, ForNode, IfChangedNode, IfEqualNode,
RegroupNode, SsiNode,
LoadNode, NowNode, SpacelessNode, URLNode, WidthRatioNode, WithNode.
- 5 days term - implementing 'compile' method for
django.template.defaulttags.IfNode and implementing compilation
support for
django.template.smartif.
- Writing unit-tests :
- 1 week term - writing unit-tests for compilation process for
django.template.defaultfilters.
- 2 week term - writing unit-tests for django.template.defaulttags.
- 4 days term - writing unit-tests for django.template.loader_tags.
- 1 week term - writing unit-tests for compilation process for
django.template.Template, django.template.loader,
django.template.loaders.*.
Testing compiled tempates writing,import'ing and raw template compiling.
- 1-2 weeks term - resolving all errors founded by using unit-tests.
By this step I will produce fully tested template compilation code.
- 3-5 weeks term - optimization of written code
*2. Armin Ronacher has good template engine that supports template
compilation, and other features. I can not argue against his template engine
solution. But, society should be pointed on :*
- The default syntax of Jinja2 matches Django syntax in many ways.
However this similarity doesn’t mean that you can use a Django template
unmodified in Jinja2. For example filter arguments use a function call
syntax rather than a colon to separate filter name and arguments.
Additionally the extension interface in Jinja is fundamentally different
from the Django one which means that your custom tags won’t work any
longer.
- Armin Ronacher wrote about backward-compatibilty: > Are you planning on
keeping the API for registering template tags > compatible?
(django.template.Library.register.tag) It will be supported because
otherwise upgrading won't be possible, but how well this works in detail
would have to be investigated and what a cleaner implementation looks like.
Jinja2 has an expression parser and encourages people to create a AST that
does what they want in extensions. However in Jinja2 I never encouraged
people to write custom tags so the API is very limited there. ---- As for
me, he haven't made decision how it would be looked like in Django. I have
already proposed solution of template subsystem's improvement.
- My solution will come with django 'out of the box' against Armin
Ronacher solution.
On Fri, Apr 8, 2011 at 4:23 AM, Russell Keith-Magee <[email protected]
> wrote:
> On Fri, Apr 8, 2011 at 9:15 AM, Andrey Zubko <[email protected]> wrote:
> > Hello,
> > I'm Andrey Zubko, student faculty of Computer Science in Zaporozhye
> > State Engineering Academy. I'm senior chief developer of a local
> > Internet Service Provider "Telza" which provides Internet and
> > telephony services. My responsibilities includes enhancing, improving
> > existent Billing system that is written by myself.
> >
> > In GSOC program I want to improve template subsystem by integrating
> > template compilation into python bytecode.
> >
> > Goals and challenges
> > Integrating of template compilation has 2 goals. The first goal and
> > the chief goal is to provide backward-compatibility to templates,
> > custom tags, custom filters that are already written. The second
> > goal is to minimize modifications of Django sources to provide more
> > stability and faster integrating in trunk.
> > Implementation
> > Support of template compilation can be achieved by adding method
> > 'compile' to every Node-derived class, and some other classes,
> > functions, such as django.template.Variable,
> > django.template.FilterExpression, django.template.NodeList, and others
> > described further.
> >
> > 1. Node and Node-derived classes modifications
> > Class Node and derived from it classes should have optional method –
> > compile, it returns list of python bytecode commands for write in
> > compiled template. It receives parameter parent_name that will
> > contains name of uppermost tag/block. Parent_name is a name of
> > uppermost tag/block that identifies entry in hash blocks_output that
> > is used for caching all outputs. Only one case can provide parent_name
> > to be overridden – meet of the '{% block %}' tag in a parent tag, this
> > tag should override parent_name to its unique name. Approach for
> > caching all output is used for providing template inheritance ability.
> > Use of template Inheritance provides following cases :
> > child template has one sequence of blocks, but parent template has
> > another sequence of blocks
> > child template has blocks that haven't specified in parent's template
> >
> > For solving this cases I provided list blocks_sequence_output that
> > keeps sequence of blocks outputting. Implementation of 'compile'
> > method in Node class :
> > def compile(self,parent_name):
> > self.generate_name(parent_name)
> > compiled_code = ["t = Template('%s')" % self.template,
> > "blocks_output['%s'] = blocks_output['%s'] +
> > t._render(context)" % (self.parent_name,self.parent_name)]
> > return compiled_code
> >
> > As you saw, in purposes of providing backward-compatibility all Node-
> > derived classes, that haven't implemented 'compile' method, will use
> > current rendering approach.
> > Node-derived classes with implemented 'compile' method will be looked
> > like :
> > class TextNode(Node):
> > def compile(self,parent_name):
> > return ["blocks_output['%s'] = blocks_output['%s'] + \"\"\"%s
> > \"\"\"" % (self.parent_name,
> >
> > self.parent_name,
> >
> > self.s)]
> >
> > As shown before 'compile' method is not so difficult to implement.
> > To recursively compilation of Node-derived classes is used method
> > 'compile' in the class NodeList :
> > class NodeList(list):
> > def generate_name(self,node):
> > return id(node)
> >
> > def compile(self, parent_name=None):
> > bits = []
> > for node in self:
> > if isinstance(node, Node):
> > if parent_name is None or isinstance(node,
> > BlockNode) :
> > if isinstance(node, BlockNode) :
> > parent_name_new = node.name
> > else :
> > parent_name_new = self.generate_name(node)
> >
> > bits.append("blocks_output['%s']=''" %
> > parent_name_new)
> > else :
> > parent_name_new = parent_name
> >
> > bits.append(self.compile_node(node, parent_name_new))
> > else:
> > bits.append(node)
> > return mark_safe(''.join([force_unicode(b) for b in bits]))
> >
> > Also this function is used for initialising blocks_output hashes for
> > uppermost nodes.
> >
> > 2. Blocks and dynamical inheritance
> > I have described before Node-derived classes improvements, but
> > django.template.loader_tags.BlockNode, django.template.loader_tags,
> > django.template.loader_tages.ExtendsNode and others have big influence
> > on compiled template architecture design, that's why I have allocated
> > whole section for them. Internal variables blocks_output,
> > blocks_output_sequence are used for providing support of dynamic
> > inheritance. They specify the way how template should be rendered, in
> > what sequence, what blocks should be outputted to user.
> > 'blocks_output' hash collects blocks or tags outputs, and
> > blocks_output_sequence saves sequence of block outputting. Because
> > parent template can contain its own sequence of blocks that is saved
> > in blocks_output_sequence, we must update the child
> > blocks_output_sequence without right of overriding it. Because parent
> > template can contain own blocks that is not present in child template,
> > we must update child blocks_output hash with not existing items in it.
> > By solving this requirements class BlockNode will be looked like :
> >
> > class BlockNode(Node):
> > def super_compile(self):
> > compiled_code = ["blocks_output['%s'] = blocks_output['%s'] +
> > blocks_output_parent['%s']" % (self.name,self.name,self.name)]
> > return compiled_code
> >
> > def compile(self,parent_name):
> > compiled_childs = self.nodelist.compile()
> > return compiled_childs
> >
> > blocks_output_parent – is blocks_output hash received from parent
> > template that contains cached parent output.
> > For providing support of static and dynamic inheritance class
> > ExtendsNode should have 'compile' method that will be looked like :
> >
> > class ExtendsNode(Node):
> > def get_parent_compiled(self):
> > if self.parent_name_expr: # dynamical inheritance
> > parent = self.parent_name_expr.compile()
> > if not parent :
> > if self.parent_name_expr:
> > error_msg = " Got this from the '%s' variable." %
> > self.parent_name_expr.token
> > raise TemplateSyntaxError(error_msg)
> > else : # static inheritance
> > parent = self.parent_name
> > if not parent:
> > error_msg = "Invalid template name in 'extends' tag:
> > %r." % parent
> > raise TemplateSyntaxError(error_msg)
> > if hasattr(parent, 'compile'):
> > return parent.compile() # parent is a Template object
> >
> > code = ["blocks_output, blocks_sequence_output_parent =
> > get_compiled_template(%s,no_output=True)" % self.parent_name,
> > "blocks_output_parent = blocks_output"]
> >
> > return code
> >
> > def compile(self,parent_name):
> > compiled_parent = self.get_compiled_parent()
> > return compiled_parent
> >
> > 3. Variables and filters modifications
> > Support of variables and filters are implemented by
> > django.template.FilterExpression class. It is wrapped by VariableNode
> > class – Node-derived class that implements simple call of
> > FilterExpression instance. For supporting compilation into python
> > bytecode VariableNode class should contain 'compile' function that
> > calls 'compile' method of FilterExpression instance :
> >
> > class VariableNode(Node):
> > def __compile__(self,parent_name):
> > return self.filter_expression.compile()
> >
> > And FilterExpression should have implemented 'compile' method in which
> > it calls filters compiled functions and variable resolving :
> >
> > class FilterExpression(object):
> > def compile(self, ignore_failures=False):
> > if isinstance(self.var, Variable):
> > obj =
> >
> self.var.compile(ignore_failures,settings.COMPILED_TEMPLATE_STRING_IF_INVALID)
> > else:
> > obj = self.var
> >
> > for func, args in self.filters:
> > arg_vals = ""
> > i = 0
> > for lookup, arg in args:
> > if i > 0 :
> > arg_vals = arg_vals + ","
> > if not lookup:
> > arg_vals = arg_vals + "\"%s\"" % mark_safe(arg)
> > else:
> > arg_vals = arg_vals + "%s" % arg.compile(context)
> >
> > if getattr(func, 'needs_autoescape', False):
> > new_obj = "%s(%s,autoescape=context.autoescape,%s)" %
> > (func.__name__,obj,arg_vals)
> > else:
> > new_obj = "%s(%s,%s)" % (func.__name__,obj,arg_vals)
> > if getattr(func, 'is_safe', False) and isinstance(obj,
> > SafeData):
> > obj = "mark_safe(%s)" % new_obj
> > elif isinstance(obj, EscapeData):
> > obj = "mark_for_escaping(%s)" % new_obj
> > else:
> > obj = new_obj
> > return [obj]
> >
> > By the way, 'compile' method FilterExpression has no parent_name param
> > – it is omitted because FilterExpression can't produce a block – it
> > can be called only from a block – VariableNode.
> > And django.template.Variable receives parameters ignore_failures, and
> > default value for variable :
> >
> > class Variable(object):
> > def compile(self,ignore_failures,default_value):
> > if self.lookups is not None:
> > value = "resolve(context,'%s',%s,'%s')" %
> > (self.lookups,ignore_failures,default_value)
> > else:
> > # We're dealing with a literal, so it's already been
> > "resolved"
> > value = "'%s'" % self.literal
> >
> > if self.translate:
> > value = "_(%s)" % value
> >
> > Using the Variable class introduced us to inner compiled template
> > function named 'resolve'. 'resolve' function implements finding entry
> > in context with looking name, value of entry can be simple value,
> > class, function. 'resolve' receives params :
> > 1.context – context variable that is passed to compiled template
> > 'render' method
> > 2.key – name of looking entry
> > 3.ignore_failures – if ignore_failures is true and finding entry is
> > failed, then it will return default_value or if it is omitted – None,
> > otherwise it will raise VariableDoesNotExist exception
> > 4.default_value – default value
> >
> > Compiled Template Architecture
> > Compiled template contains two sections :
> > 1.Dependencies section
> > 2.Compiled code section
> > Dependencies section is section that has default imports, and external
> > imports of user-written custom tags, filters. External imports are
> > determined by calling {% load %} tag.
> > Compiled code section is a section that contains template's python
> > byte code.
> > Structure of the compiled code :
> >
> > # -- DEPENDENCIES SECTION
> > # ----- Default imports
> > from django.template import Template, get_compiled_template
> > from django.template.defaultfilters import *
> > # ----- User custom imports
> > # from ... import ...
> > # -- END OF DEPENDENCIES SECTION
> >
> > def resolve(context,ignore_failures,default_value):
> > """ should be looked like method _resolve_lookup in
> > django.template.Variable """
> >
> > blocks_output = {}
> > blocks_output_sequence = []
> > blocks_sequence_output_parent = None
> >
> > def output():
> > for block_name in blocks_output_sequence :
> > print blocks_output.get(block_name,'')
> >
> > def render(context,no_output=False):
> > # START OF COMPILED TEMPLATE PYTHON BYTECODE
> >
> > # ...
> >
> > # END OF COMPILED TEMPLATE PYTHON BYTECODE
> > if blocks_sequence_output_parent is not None :
> > blocks_sequence_output = blocks_sequence_output_parent
> > if no_output :
> > return blocks_output, blocks_output_sequence
> > else :
> > output()
> >
> > Every compiled template has 3 functions :
> > resolve – discussed earlier
> > output – task of this function is to output blocks output cache in the
> > determined sequence
> > render – entrypoint of compiled template, that contains template's
> > compiled python bytecode. It supports one optional parameter –
> > 'no_output' – it is used only when called parent template – not to
> > output, to return cached data.
> >
> > Examples
> > Examples based on already written code :
> > compiling variables and filters :
> >
> > from django.template import Parser, FilterExpression
> >
> > token = 'variable|default:"Default value"|date:"Y-m-d"'
> > p = Parser('')
> > fe = FilterExpression(token, p)
> > fe.compile()
> > =>
> > 'date(default(resolve(context,variable,None,''),"Default value"),"Y-m-
> > d")'
> > template compilation sample :
> > child.html :
> > {% extends parent %}
> > {% block first_block %}
> > {% if True %}
> > some_info
> > {% endif %}
> > {% endblock %}
> > {% block second_block %}
> > {% endblock %}
> >
> > parent.html :
> > TextNode
> > {% block first_block %}
> > {% endblock %}
> >
> > =>
> > Compiled child template :
> > # -- DEPENDENCIES SECTION
> > # ----- Default imports
> > from django.template import Template, get_compiled_template
> > from django.template.defaultfilters import *
> > # ----- User custom imports
> > # from ... import ...
> > # -- END OF DEPENDENCIES SECTION
> >
> > def resolve(context,key,ignore_failures,default_value):
> > """ should be looked like method _resolve_lookup in
> > django.template.Variable """
> >
> >
> > blocks_output = {}
> > blocks_output_sequence = []
> > blocks_sequence_output_parent = None
> >
> > def output():
> > for block_name in blocks_output_sequence :
> > print blocks_output.get(block_name,'')
> >
> > def render(context,no_output=False):
> > blocks_output, blocks_sequence_output_parent =
> > get_compiled_template(resolve(context,'parent',None,''),no_output=True)
> > blocks_output_parent = blocks_output
> >
> > blocks_output['first_block'] = '' # entering block tag in NodeList
> > compile
> > blocks_sequence_output.append('first_block')
> > if True :
> > blocks_output['first_block'] = blocks_output['first_block'] +
> > "some_info"
> >
> > blocks_output['second_block'] = '' # entering block tag in
> > NodeList compile
> >
> > if blocks_sequence_output_parent is not None :
> > blocks_sequence_output = blocks_sequence_output_parent
> >
> > if no_output :
> > return blocks_output, blocks_output_sequence
> > else :
> > output()
> >
> > Compiled parent template :
> > # -- DEPENDENCIES SECTION
> > # ----- Default imports
> > from django.template import Template, get_compiled_template
> > from django.template.defaultfilters import *
> > # ----- User custom imports
> > # from ... import ...
> > # -- END OF DEPENDENCIES SECTION
> >
> > def resolve(context,key,ignore_failures,default_value):
> > """ should be looked like method _resolve_lookup in
> > django.template.Variable """
> >
> >
> > blocks_output = {}
> > blocks_output_sequence = []
> > blocks_sequence_output_parent = None
> >
> > def output():
> > for block_name in blocks_output_sequence :
> > print blocks_output.get(block_name,'')
> >
> > def render(context,no_output=False):
> > blocks_output['first_block'] = '' # entering block tag in NodeList
> > compile
> > blocks_sequence_output.append('first_block')
> >
> > if blocks_sequence_output_parent is not None :
> > blocks_sequence_output = blocks_sequence_output_parent
> >
> > if no_output :
> > return blocks_output, blocks_output_sequence
> > else :
> > output()
> >
> > Milestones
> > 1. Designing concept - current step
> > 2. Coding "template compilation" subsystem
> > 3. Writing unit-tests
> > 4. Optimization of written code
>
> A quick word of advice -- this timeline is probably enough to ensure
> that you *won't* get selected for the Django GSoC. It doesn't provide
> any evidence that you've done serious thought about how long the work
> involved will take. A serious proposal needs to have a work breakdown
> with 1-week granularity at the *most*. It should also give a clear
> mechanism for evaluating progress by indicating the deliverables at
> various stages of the project (e.g., after week 3, it will be possible
> to write a compiled template containing only {% if %} and {% block %}
> clauses).
>
> As an additional complication for your project -- you should take a
> look at your competition. You're proposing to work on a project that
> has also been proposed by Armin Ronacher. Armin has a long and
> demonstrated history of expertise when it comes to compiled template
> engines. If you're proposing a simliar project, then you would be well
> advised to clearly explain why:
> 1) your proposal is better than his, or
> 2) your proposal is a good compliment to his.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
