Sorry for responding twice to the same email. I have almost written this
email half a dozen times since Andy's response. I wanted to put it out now
to discuss some of the ramifications of the the proposed TT3 model.
> [aside]
> The uri() method is worth mentioning because it ties in with another
> proposal you made regarding CACHE_STR_REF. If a template is loaded from a
> file then uri() will return the filename, or possible a more canonical URI
> like "file:/blah/blah" or even "file://localhost/blah/blah".
Template::Alloy internally uses _filename to store the "uri" of the file.
Some have expressed concern that they don't want the full pathname of their
file available to template users. Alloy avoids this by using the "PRIVATE"
key name. Having a uri method will become an issue with the "everything is a
template object" concept - I'll discuss it more below.
> In TT3, you'll be able to use templates as stand-alone objects. For
> example:
>
> my $f = Template->template( file => 'example.tt3' );
> my $t = Template->template( text => 'Hello [% thing %]' );
>
> print $t->process( thing => 'World' );
> print $t->process( thing => 'Badger' );
>
> This is one important difference between TT2 and TT3. TT2 templates
> require a separate context object in order to do anything useful. TT3
> templates will effectively *be* their own context, by virtue of the fact
> that
> Template::Template will be a subclass of Template::Context (or something
> like it - I'm still hacking on that bit ATM).
This is how Template::Alloy works - except the context and self are the same
thing.
> That means you can do things like this, in HTML-Template style:
>
> $t->set( thing => 'Badger' );
> $t->get('thing');
Pretty close. HTML::Template uses the combined "param" method
$t->param(thing => 'Badger');
$t->param(\%THINGS);
my $val = $t->get('thing');
Template::Alloy supports this via the HTE role. However Alloy only uses
parameters set this way when using the HTE output method (all the same as the
HTML::Template interface).
> Each template will also have it's own methods for fetching and using things
> other than variables:
>
> $t->template('header');
> $t->plugin('table');
> $t->filter('html');
>
> And so on. (there's magic behind that which I'll skip over for now)
Thats good. I'm imagining part of the magic is a shared cache so that you
don't have to load the filter list twice, or scan the plugin dirs twice, or
load a template twice.
> What happens in the background when you call process(), is that the
> template takes responsibility for parsing and compiling itself into a
> runtime form, and then runs itself. How it does all that it up to the
> template (and friends, like compilers, caches, stores, etc).
>
> At the low-level end of things, this gives you a lot more flexibility. It
> means you can bypass the whole Template front-end mechanism (e.g. the
> service layer) if you want, and just get a template object that spits some
> text back out when you feed it variables.
>
> print $t->process( thing => 'Badger' );
That is somewhat confusing. Template::process returns true if successful but
Template::Template::process returns the processed data. You may want to
change method names to keep things more clear.
> It makes things like this much easier to implement:
>
> [% "Hello [% thing %]".eval %]
> [% "Hello [% thing %]".template %]
> [% "Hello [% thing %]".process(thing = 'Badger') %]
[% "Hello [% thing %]".eval %] Currently works in Template::Alloy.
[% 'Hello $thing'.eval(interpolate => 1) %] Currently works in Template::Alloy
and I'll submit an eval filter patch that will let it work in
Template::Parser::CET.
[% "Hello [% thing %]".template %] I can make this work - quite easily, but I
think it is a security hole to expose the template objects to the language by
default. At the least there is the .uri method which will let you see full
template filenames of values. I'm imagining you'll be wanting to have [%
(file:my/file.tt).template %]. That would be good - but then you can
call .uri and see where on the filesystem the file was found. You would have
to go to pretty good lengths to make .uri callable if you are in perl and
not-callable if you are in a template. The other issue, is that TT's method
setting is very powerful and I can manipulate a perl based object - so for
example I could do [% t = (file:my/file.tt).template; t.CONFIG.EVAL_PERL =
1 %]. You will either have to make your structure be tied to keep this from
happening, or you will have to use "_config" which then re-introduces the
hole if they enable using private variables. While it is sort of neat, I
don't think it is a good idea to expose the template objects. VMethods and
filters can already do all of this in Alloy and with a little work in
Template::Parser::CET on TT2.
[% "Hello [% thing %]".process(thing = 'Badger') %] This would actually be a
very simple filter to implement. It is very close to the current eval
filter. And again Alloy and Template::Parser::CET do already autobox strings
so it is possible to try "'[% 1 + 3 %]'.eval" now (I think Parser::CET may
require you to write '[% 1 + 3 %]'|eval to make sure you access the filter
properly in TT2).
> The other key benefit is that a template object can implement whatever kind
> of parsing *and* runtime strategy that it likes.
This is nice and I'm sure it is infinitely flexible, but the common operation
is that you want your nested templates to share the same configuration as
your top level template/context. Any time you nest and you are using a
separate template object or context you have to pass along your current
configuration - or you have to use the same lookup fallback method that you
proposed below - but that imposes other issues.
> It's almost exactly the same as in TT2, except we're using $self rather
> than $context, and there's a built-in method for get() rather than going
> through the stash.
>
> The difference is that the template ($self) can implement it's own get()
> method, or do something else entirely to handle variables. It's no longer
> limited by what the $context provides. Nor is it even required to compile
> itself to Perl code. It can implement itself using an optree like TT1 or
> CET.
This is somewhat already available in TT now with object methods. Part of the
question is will Template users want and/or need to have the different
get/setters for different templates. Again it is a neat feature, but I think
that it will rarely be used.
> Having said that, a key part of TT is getting a bunch of templates to play
> nicely with each, rather than skulking around by themselves. When we
> INCLUDE a template, for example, we want it to get all the variables that
> we've currently got without us having to explicitly pass them over.
>
> TT2 operates on the idea of a shared context. All variables are available
> everywhere unless you take measures to ensure that they localised. For
> example, by using INCLUDE rather than PROCESS. Internally this involves
> cloning the stash (which eats memory and time) and messing with the BLOCKS
> definition list (which is just plain ugly). And all it really achieves is
> to protect you from a SET in the called template or a BLOCK definition
> which would otherwise screw up your own variable/templates.
>
> The outer template passes a reference to itself as the 'context' for the
> inner template. The inner get() method can then call the $context->get()
> method to fetch any variables from the outer template and cache them
> locally. Although that first get() on a variable takes a shade longer, the
> local cache means you only take the hit once. And you win by only doing
> this for the variables you actually use, whereas the current stash cloning
> code copies all your variables, every time, regardless.
>
> When you set() a variable, you set it in the local variable cache and don't
> affect the calling context. So you effectively get the protection that
> INCLUDE gives you for free. It's more like Perl in that sense: don't
> export anything by default! However, to implement the way PROCESS
> currently works, there will be some mechanism (got a couple of candidates
> but no firm code yet) to allow the template to set stuff in the context.
>
> The other nice thing is that the same strategy works for all the resources:
> variables, templates, plugins, filters, and so on. There's no need for any
> more messing about with BLOCK definition lists and other special cases like
> that.
I think it would be interesting to do a straw poll to find how may people
routinely use INCLUDE vs PROCESS. I don't ever use INCLUDE. Where I work if
we have variables that are set inside processed templates we use namespace
prefixes to guard them. Not only does this fix the same issue and prevents
us from needing to clone the stash - it allows for us to dump the stash to
show what has occurred. But this is a side issue - some people will have to
use INCLUDE because they'll be using templates they don't trust (sometimes I
wonder about these people - who will admint to using templates they don't
trust ?).
The part I find interesting about the TT3 get/set fallback proposal is that it
takes complexity from one point (the localization of the variable stash) and
it inserts it at another point -- the lookup of a variable. Well, it
actually is worse than variable lookups -- it also applies to the lookup of
every configuration item as well. As I've been mentioning through this
response, every sub template will want to use the base configuration of the
top level template. This means that every sub template will need a copy of
the top level configuration. You either have to clone the Template object
itself when you process sub templates, or you have to do the lookup fallback
on configuration variables also. (Note: Thinking about it you might also
possibly pass along the top level object which keeps this configuration -
that would work also - but then you've given up having separate configuration
for sub templates). Either way, there is similar overhead to the TT2 way of
cloning the stash - but now it applies to PROCESS directives as well as
INCLUDE directives. The overhead hasn't been removed, it has only been
relocated. And the overhead grows for variable access each additional nested
scope.
It was mentioned that the new TT3 get/set fallback with sub contexts is closer
to how perl does it. I thought so also at my first glance. But when you go
to a subscope in Perl, it maintains the same context, it just allows you to
temporize a variable in the current lexical scope if a "my" declaration is
used. Otherwise each sublevel scope is the same as the preceding one. The
new TT3 way clones the entire context. If I had to suggest a way to do it,
I'd suggest the way Alloy is doing it, and that is in the situations such as
INCLUDE or WHILE or one type of FOREACH simply do a local $self->{'stash'} =
{ %{ $self->{'stash'} } }. If you call the parse_tree method (to start
parsing new text) it localizes all configuration items that can change using
the CONFIG directive using the same "local" method (remember the proposed
CONFIG directive? - it is actually very useful). The nice thing about using
this method is that it places the entire copying of the stash into the
lowlevel perl builtin function area. Doing anything iteratively in Perl will
have a hard time coming close to working as efficiently as using builtins to
do the work for you. And you don't ever have "unlocalise" because perl will
do it for you (granted the proposed TT3 sub context way wouldn't have to
either).
For another staw poll, I'd be interested how often people use variables vs
process other templates. People obviously do both, but variable lookup
probably happens far more often than the including of other files. Again,
variable lookup needs to be kept as fast as possible.
So - to sum up:
I'd be careful introducing full blown template objects into the language by
default.
1). They expose system level details
2). They expose security issues.
3). They are neat, but the extra features will probably be seldom used.
I'd be careful about doing the get/set fallback.
1). It doesn't actually reduce overhead, it only relocates it.
2). If you have a global cache for templates (which you'll need) you'll
have to localize its stash or re-initialize it every time you use it.
3). You can't view all available variables in the current scope (via [%
DUMP %]).
4). It accomplishes the same thing that other algorithms can probably do
faster.
5). It is neat, but the extra features will probably be seldom used.
I know this email probably sounds really critical. I didn't intend it that
way. I am excited to see so many neat things planned for TT3. I just wanted
to warn of potential pitfalls that I see with the currently selected approach
(there are dozens of approaches that can be used). You have mentioned before
that you want to make sure that TT3 is done "the right way (tm)." All of
your proposals probably fall into this realm. I just want to bring to the
table that I think there are several "right ways."
One thing that I would really like to see would be an email thread asking what
features people on the list really want. In the end Andy Wardley gets to
decide. But I really like how Larry Wall let everybody put in their requests
for Perl 6. He ultimately decided what was in or out. There are so many
people on the list TT list that use TT everyday that would be able to say
what they actually need vs what things are neat. Maybe I'll start another
thread.
Paul
_______________________________________________
templates mailing list
[email protected]
http://lists.template-toolkit.org/mailman/listinfo/templates
