Date: 2004-12-05T04:15:57 Editor: DanielFagerstrom <[EMAIL PROTECTED]> Wiki: Cocoon Wiki Page: TemplateEngineDesign URL: http://wiki.apache.org/cocoon/TemplateEngineDesign
no comment New Page: = Template Engine Design = For the original post, see [http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=110132582421156&w=2]. The idea is to keep this design document updated to reflect the implementation, this far it is just a slightly editet version of the post. Here are some ideas about how to implement JXTG 2.0. I'll focus on describing how pre-compilation can be implemented and how to connect to Java written executable tags. After having evaluated Jelly and taglib, I don't think it is worthwhile to trying to implement JXTG2 on top of them. It is certainly a good idea to reuse ideas and code from JXTG. One could possibly start from JXTG and go to JXTG2 through a sequence of refactoring steps, but the monsterous size of the implementation of JXTG makes that less atractive, at least for me. I would probably start from Jonas taglib as it contains some things that we are going to need any way and is a good starting point. Anyway it is IMHO a good idea to take a look at it, and see that a taglib implementation can be small and lean and does not have to be +150kB source code. ---o0o--- I'll continue to describe a possible design of a template generator (TG) that uses pre-compilation and has customer tags. This will be done in three steps: We start with a trivial pre-compiled template language (TL) ''without'' expressions. In the next step we add an expression language to it and we finish by adding executable tags. == A Trivial Pre-Compiled "Template Generator" == This step is not that useful in it self, the result will just be an unnesecarilly complicated implementation of the file generator. This is for introducing some ideas. The pre-compiling TG works in two steps: First it compiles the content of the input source to a script, and store the script in its cache together with an approriate cache key. Then the cached script will be used as long as the cache key is valid. It is important that the script is thread safe otherwise it will be unsafe to reuse. As store, Cocoon's transient store could be used. In the second step the script is executed and SAX output is generated. === The Trivial Script === The task of the trivial "template generator" is to output its input template document as is :) For this task there allready is a feature complete script implementation in Cocoon; {{{o.a.c.xml.SAXBuffer}}}. The {{{SAXBuffer}}} contains a list of {{{SAXBits}}} where each {{{SAXBit}}} is an (immutable) representation of a SAX event. The {{{SAXBit}}} interface contains the method {{{send(ContentHandler)}}}. And the {{{SAXBuffer}}} contains an inner class implementing {{{SAXBit}}} for each SAX event type. The {{{SAXBuffer}}} implements {{{XMLConsumer}}} which records the input events as list of {{{SAXBits}}}. It also implements {{{XMLizable}}} with the method {{{toSAX(ContentHandler)}}} that iterates through the recorded {{{SAXBits}}} and execute {{{send(ContentHandler)}}} on them. This far compiling a script means sending the input source to a {{{SAXBuffer}}} and executing a script means calling the {{{toSAX}}} method on the SAX buffer. JXTG contains code that is similar to the {{{SAXBuffer}}}. == A Real Template Generator == The defining property of an XML TL is that you can embed expressions in the XML attributes and text. These expression are replaced with the result of evaluating the expressions in some context. To add this to our TG we need to define the context, get a expression evaluator and handle the expression embeding in the script. === Expression Language === It would be best if we could decide to use JXPath everywhere in Cocoon ;) But as that not is going to happen we need to support several expression languages (EL). This is not just a need in JXTG but also in CForms, input modules and many other places. So we have a need for plugable EL in Cocoon. Dmitri Plotnikov, the main author of JXPath have written a common API for EL [http://www.plotnix.com/jex/index.html], and commited the code to Jakarta Commons Sandbox [http://cvs.apache.org/viewcvs.cgi/jakarta-commons-sandbox/jex/]. AFAICS the project never took off so we shouldn't use it. But we could probably steal some good ideas from it, although I think that we should keep it much simpler. ELs often has the possibility to create a compiled expression object from the expression string. When that is possible, (and given that the compiled expression is thread safe), we can get better performance by storing compiled expressions in the script. Besides plugable EL we also want to be able to plug in a locale sensitive and configurable convertor component for handling the results from the EL. Now, we don't have to implement everything at once. We can start with suporting one EL and refactor to using the plugable EL, when/if someone implements it. === Expression Context === The expresion must be evaluated in some context. JXTG have a lot of code that package the different objects in the Cocoon object model so that it looks the same way as in flowscripts, but from JXPath or JEXL. Carsten have factored out the code from JXTG to {{{o.a.c.environment.TemplateObjectModelHelper}}} in scratchpad. It is marked as "work in progress" and I don't know the current state of it. But IMHO we should base JXTG2 on that (and possibly improve it) rather than having still another implementation of the script object model. === Embeding Expressions in the Template Language === In the TL we are using expressions like this: {{{ <tag attr1="text ${expr1} text ${expr2}"> Text ${expr3} more text ${expr4}, even more text. </tag> }}} We can see that it is the SAX startElement and the characters events that are affected. During compilation of the TL into a script, we need to parse the attribute content and characters content and create a list of text elements and compiled expression elements. During execution of the script the expression context must be available when executing the EL parts of the script. In JXTG this is implemented in the {{{TextEvent}}} and the {{{StartElement}}} classes, where both contains list with interspersed expression and text objects. If we continue to base our script implementation on the {{{SAXBuffer}}}, we need to extend it with new handling of startEvent and characters like in JXTG. We need new {{{StartElement}}} and {{{Characters}}} classes that can handle expressions like I described above. During script compilation time they need a EL factory. During script execution time they need a expression context, so their interface must be extended with something like {{{send(ContentHandler, ExpressionContext)}}} and simillary the script (extended {{{SAXBuffer}}}) need a method {{{toSAX(ContentHandler, ExpressionContext)}}}. == Adding Executable Tags == There are numerous ways to add executable tags (ET) to a TL. Of those I have seen I prefer Jelly's way, and therefore I'll describe how to implement something similar. Take a look on the implementation of some Jelly tags [http://cvs.apache.org/viewcvs.cgi/jakarta-commons/jelly/src/java/org/apache/commons/jelly/tags/core/] and compare them with the corresponding tags in e.g. taglib and decide for yourself what you prefer. For concretenes lets say that we have a for-tag in our TL (example from Jonas): {{{ <core:for var="index" begin="0" end="8" step="2"> <p>${index}</p> </core:for> }}} We have a {{{TagRepository}}} where URI and tag name is mapped to a tag object. Tags that are registred in the {{{TagRepository}}} (executable tags) gets special treatment. All others are handled as described in the previous sections. When a ET is executed it will have access to: * the execution context (including the current expression context) * the content of its attributes * its body in form of a script * the current XMLConsumer for output * its parent ET (this is less important, but is used for implementing e.g. choose/when, take a look at the Jelly tags) === Execution Context === The execution context contains the expression context as before but now we want the expression context to be implemented as a stack of (variable name, value) bindings, searched from top and downwards for a variable binding, (see Jonas code e.g. for details). This is needed for creating local variables in template "macros" and for handling context info in recursive templates. Recursive templates are needed as soon as you have recurively defined data structures like the class stuff in CForms. The execution context gives access to the tag repository so that it is possible to add tags and use tags in user defined tags. It might also contain access to Cocoon components, and the source resover. === The Executable Tag === The most complicated question in deciding how the ETs should work is if we should require them to be thread safe or not. If they are thread safe they can be made part of the script and be completely prepared at compile time. But OTH they will be harder to write as one cannot use member variables for state info. For the moment I think not requireing thread safety would be enough and that we can see more efficient treatment of thread safe tags as a future optimization. In Jelly, tags can implement {{{CompilableTag}}}, which give them the possiblity to do some work at compile time, I've no idea about how it work though. Anyway the tag will be created at execute time and need access to the different data specified above. Jelly uses the following inteface for tags: {{{ public interface Tag { public Tag getParent(); public void setParent(Tag parent); public Script getBody(); public void setBody(Script body); public JellyContext getContext(); public void setContext(JellyContext context) throws JellyTagException; public void doTag(XMLOutput output) throws MissingAttributeException, JellyTagException; public void invokeBody(XMLOutput output) throws JellyTagException; } }}} and setter injection for the attributes. Or this is rather the default way, Jelly also use various reflection based tricks and tries to handle almost any bean as a tag. The "for" tag above would be implemented like this: {{{ public class For extends TagSupport { String var; int begin, end, step; public void setBegin(int begin) { this.begin = begin } // other setters and getters public void doTag(XMLOutput output) throws JellyTagException { getContext().variables().push(); for (int i = begin; i <= end; i+= step) { if (var != null) getContext().variables().put(var, new Integer(i)); invokeBody(output); } variables.pop(); } } }}} (I'm using Jelly interfaces, so the example isn't completely consistent with other discussion about intefaces in the mail). I have no strong opinions about if we should user setter injection, getAttribute methods in doTag, etc. === Compiling the Script === Now we continue to extend our script implementation so that it can contain an {{{ExecutableTag}}} element as well as the previous ones. An {{{ExecutableTag}}} object is set up with attributes, the script for its body, its parent and a reference to the factory for creating its tag bean during compile time. During execution we need a {{{send(ContentHandler, Context)}}} method that creates and instantiates the tag object (as described above) and then call its {{{doTag}}} method. The script also need a {{{toSAX(ContentHandler, Context)}}} for executing all of the script. Now we need a template parser its not enough to feed the input source into the {{{SAXBuffer}}} anymore as we are going to create a new script for every body of an executable tag. The parser need stacks for keeping track of the current tag and the current script, take a look at e.g. Jonas {{{TemplateTransformer}}} to get an idea about how such parsing can be implemented. === Implementing the Tags === The idea is that all tags in JXTG are implemented as executable tags in the above described framework. Also tags for defining new tags e.g. macros. Besides implementing the JXTG tags, we would need to implement the ESQL tags (or a replacement of them), and porting (if that is needed) the CForms tags. After that we would have a replacement for XSP, AFAIU :)
