Hi Paul,
This is a shorter reply to a much longer version that I started writing over
the weekend. It quickly got long and rambling with many diversions so I
decided it was better to write it up as a series of design documents. So at
some point in the not-too-distant future I'll be addressing your points in
more detail, but in the mean time, here are the salient points.
Starting at the end:
I know this email probably sounds really critical.
Not at all. Well, only in the constructively critical sense which I'm always
happy to hear. I'm glad there is someone taking an acute interest in what's
going on and asking questions that I need to make sure I've got good answers for.
Just as long as you appreciate (which I'm sure you do), that I might not have
answers for you right here and now. If it's a question of me spending time
writing code for you to look at/hack on or writing detailed explanations of
what I'm going to do, then I'm sure you'll agree the former is preferable.
Especially when it's generally easier and less ambiguous to write code than
trying to write about it.
Anyway, nothing is set in stone in the architecture. Well, not much. But
everything is certainly up for discussion, so keep it coming.
Template::Alloy internally uses _filename to store the "uri" of the file.
Some have expressed concern that they don't want the full pathname of their
file available to template users.
Yep, that's a good point. You'll be able to configure the uri that the
template provider tags onto the local (to the INCLUDE_PATH) template name.
This relates to the other point you made:
> I'm imagining you'll be wanting to have [% (file:my/file.tt).template %].
There won't be a file: or template: available from the template unless you
explicitly map a template provider/path to a namespace.
More on that to follow in a design document about providers, paths,
filesystems, namespaces and how it all fits together. But in summary, yep,
that'll be catered for by a translation/mapping layer between template and
filesystem names.
I'm imagining part of the magic is a shared cache so that you
don't have to load the filter list twice, or scan the plugin dirs twice, or
load a template twice.
Yep. There will be a central hub, effectively an uber-context, rather similar
to TT2's context. Most of the major TT components (template providers,
filters, plugins, etc) will hang off here.
print $t->process( thing => 'Badger' );
That is somewhat confusing. Template::process returns true if successful but
Template::Template::process returns the processed data. You may want to
change method names to keep things more clear.
Yes, it's confusing. And there's also the TT2 context process method which
returns the output.
$context->process('hello', name => 'Badger');
This is something that I'll address in detail in a design doc, but in summary,
my plan is for all the process() methods to return the output by default.
print $tt->process('hello', name =>'Badger');
print $tt->template('hello')->process(name => 'Badger')
Errors will be throw as exceptions by default so there will be no need to
explicitly check the return value (I've long since learnt the hard way why
returning undef to indicate error is now considered bad practice).
Like TT2, you'll be able to specify the output:
$tt->process('hello', { name => 'Badger' }, \*STDOUT);
Or use the print() method which will do what the TT2 process() method
currently does.
$tt->print('hello', name => 'Badger');
For backwards compatibility, there will be a host of ways to say that you want
the default output to be STDOUT just like TT2. More on that in the
"Configuration" design doc.
[% "Hello [% thing %]".template %] I can make this work - quite easily, but I
think it is a security hole to expose the template objects to the language by
default.
Yes, .template and .eval won't be enabled by default. There's a big issue
relating to the homogenisation of virtual methods and filters. Some virtual
methods and filters need access to the context, while other don't. Then we
also have the matter of static vs dynamic filters. But I'll save that for
yet another design doc.
This is nice and I'm sure it is infinitely flexible, but the common operation
is that you want your nested templates to share the same configuration as
your top level template/context.
Yes, that's right. Most, if not all of that config will be stored in the
Template::Templates provider (hanging off the hub).
Part of the
question is will Template users want and/or need to have the different
get/setters for different templates. Again it is a neat feature, but I think
that it will rarely be used.
It's more a question of allowing different template language authors the
flexibility to implement their own strategies for getting/setting variables,
templates, and other resources. The same goes for different implementations
of a language. You might want to use a fast but somewhat feature-stripped
version of TT3 for some of your templates, in order to eek out the best
performance, for example, but use the regular version for the others. Or you
might want to use some of your old TT2 templates alongside some shiny new TT3
templates, or some HTML::Template templates, or whatever.
In TT2 templates are limited to calling $context->process(), $stash->get(),
etc. So they're effectively stuck with whatever implementations the
T::Context and T::Stash objects provide them with.
In TT3, the template will be calling $self->process() and $self->get(), or
$self->another_method_altogether(). Subclasses of T::Template will be free to
re-implement/augment those methods however they see fit, or inherit the
defaults from the base class.
However, there is the important issue of templates playing nicely with each
other. So if they want to access each others variables, or pass variables
between them, then they will have to follow certain rules and stick to the
Playing Nicely Together API.
I think it would be interesting to do a straw poll to find how may people
routinely use INCLUDE vs PROCESS. I don't ever use INCLUDE.
I do, often because I have no choice.
Part of the problem that requires me to turn to this particular solution is
that there's no way for a template to declare local variables. If you use a
temporary variable (even indirectly, like in a FOREACH loop) in a template
that you PROCESS then it'll trample your current variable namespace.
So TT3 will have something like a MY directive to declare variables as
properly local. I've got a prototype implementation which uses regular Perl
'my' variable in the generated code. It seems to work well and has the
benefit of being more efficient (effectively allowing you to bypass the stash
in simple cases).
So I think that will eliminate most of the problems that cause me to turn to
INCLUDE. Can't speak for anyone else, though.
I think there's also a need for a version of PROCESS that only passes the
arguments to the template, and doesn't provide access to any of the other
variables currently defined. For those times when you want a template
processed in "stand-alone" mode. This often goes hand-on-hand with the
DEFAULT directive:
header:
[% DEFAULT title='Hello World' %]
...blah blah [% title %]...
I can't think of a good name for it, so JFDI will have to do for now:
[% JFDI header %] # vars => { }
[% JFDI header title="My Example" %] # vars => { title => ... }
In that first call, you really don't want the DEFAULT finding any other
'title' variable you happen to have lying around. So 'header' is processed
with no variables defined whatsoever. In the second example, we explicitly
pass a title, but that's all it gets.
some people will have to
use INCLUDE because they'll be using templates they don't trust (sometimes I
wonder about these people - who will admint to using templates they don't
trust ?).
It's not so much that I'll use templates I don't trust. But rather that I'll
use templates that may (or may not) use some variables that happen to clash.
This can happen, for example, when two different designers/developers wrote
the templates and happened to use the same 'x' variable. Or when I'm re-using
some of my own templates from a previous project. Or, in the pathological
case, when I'm trying to claw my way through a thousand templates written by 9
different people over 6 years of rather haphazard evolution of a web site/app.
Even adopting a naming convention doesn't really solve the problem. You just
end up with variable clash over my_x instead. Re-engineering the whole site
to organise variables properly is probably the more "correct" solution, but
using INCLUDE is often the more appropriate one when a quick fix is required.
Alas :-(
People in the know (like yourself) already know better than to use
INCLUDE, and take steps to avoid it except where strictly necessary. It's
those people who don't know any better that we also have to account for. :-)
So I think some kind of local variable mechanism is a must and a way of
processing a template in an isolated environment (i.e. detached from the
current context) would be nice to have.
The part I find interesting about the TT3 get/set fallback proposal is that it
takes complexity from one point (the localization of the variable stash) and
it inserts it at another point -- the lookup of a variable.
Partly, yes. One of the benefits (as I perceive them) are that you can reuse
the same (or similar) variable localisation mechanism for templates and other
resources if necessary. There's no need for the current ugly BLOCKS hack in
TT2 if localisation is implicit. Having said that, variables will always be a
bit special because they have to be fast.
Either way, there is similar overhead to the TT2 way of
cloning the stash - but now it applies to PROCESS directives as well as
INCLUDE directives.
No, definitely not. TT3 PROCESS will be as fast as TT2 PROCESS. It's an
important feature of the underlying architecture that we can process a
template in the current context (PROCESS), in which case all things are shared
and there's no fallback cost, or in a child context (INCLUDE), in which case
you get the safely blanket but take a performance hit.
The overhead hasn't been removed, it has only been
relocated. And the overhead grows for variable access each additional nested
scope.
First, let me reiterate that this only applies to INCLUDE.
The performance depends on a number of things:
1) The implementation (I've got about 6 or 7 different variants which
I'm using to explore the problem and benchmark different scenarios)
2) How deep you nest.
3) How many variables you have defined.
4) What kind of variables you have defined. Large chunks of text take
significantly longer to copy than refs, for example.
5) How many different variables you use in a template, what type
they are and how many levels up they're defined.
6) How many times you reference the same variable in a template.
7) How many (and what kind) of INCLUDE parameters you use.
There's probably some other factors, too, but they seem to be the main ones.
Different strategies win in different situations, so it'll be a question of
picking the one that keeps most people happy most of the time. Which means
we'll undoubtedly make some people unhappy.
But I'm not *too* worried about INCLUDE. I *am* worried about it, of course,
but not half as much as I am about PROCESS because that's the one that claims
to be fast. But given that I'm really not worried about PROCESS being fast
(because it'll be doing no more than TT2 process or even less), then it
follows that I'm not worried about INCLUDE at all because less than half of
nothing is nothing.
So the undeniable logic is that there's nothing to worry about. Nevertheless,
I have worried it enough about it to reach to the conclusion that allowing
templates to define custom get/set/etc methods will be a Good Thing. I agree
that most people won't use it most of the time (or not consciously at
least), but I see it as an important "do your own thing" hook for people who
want it.
The new TT3 way clones the entire context.
Nah. Just bits of it, if and only if you want them cloned, like when you use
INCLUDE instead of PROCESS.
Do you think I'm made of memory? :-)
Cheers
A
_______________________________________________
templates mailing list
[email protected]
http://lists.template-toolkit.org/mailman/listinfo/templates
