On Sep 17, 11:52 pm, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Wed, 2008-09-17 at 14:42 +0100, Ben Ford wrote:
> > I take it that most are aware of:
>
> >http://lucumr.pocoo.org/cogitations/2008/09/16/why-jinja-is-not-djang...
>
> > It seems like a very well thought out and thorough write up.
>
> Um .. welll. :-(
>
> Parts of it are very well thought out and if it had been a post on "how
> Jinja works" it would have been excellent. Other parts are completely
> unconstrained by facts or acknowledgement that Django's standard
> templates and Jinja's ones have different goals (which is important,
> since this isn't an apples to apples comparison at all).
There are philosophical differences between Django and Jinja
templating about how much power there should be in the templating
engine. Django's philsophy is to keep the power to the minimum
required - "templates are not programs" - and I think we all
understand this, but apart from that point, can you [or one of the
other core devs] be more specific about what you think these different
goals are?
>
> The good news is that, as far as I can see, all the actual bugs he notes
> are already in tickets. I went through and checked quickly yesterday and
> found all but one, which I'm pretty sure is in there, so I'll look again
> today.
>
> Whilst reading that, keep in mind that Armin apparently misunderstands
> (or possibly just omits to clearly state) what a template object
> instance is in Django. It's a state machine for rendering templates and
> includes the current state as well as the transitions. This isn't an
> invalid or particularly novel approach to executing a domain specific
> language. In fact, it makes a lot of sense if you want to write
> self-contained node-based state instances. Jinja and Django templates
> have different compilation strategies and return different types of
> objects and the difference is a vital distinction in any discussion of
> appropriate usage.
I don't think that template users care about the internals as much as
they care about correct behaviour, including in a threaded
environment.
> The blog post appears to imply that it would be
> natural for any random data structure to be used as a global variable
> (by extrapolation, since he doesn't establish that a Template instance
> -- not a class, the instance of a compiled template -- is special in any
> sense, so it must be somehow "normal") and that those structures will
> work beautifully when multiple threads cause its internal state to be
> modified.
>
> This has two problems: (1) it's pretty much false by default (Python !=
> Haskell, Erlang, etc and mutable objects are very normal in Python), and
> (2) it would imply that using lots of globals to store stuff in
> multi-threaded programs is a good idea that people should be doing more
> often.
>
> There's a name for that sort of programming style and it's not
> "Excellent, Maintainable Code". :-)
>
> Using the same logic, Python is not thread-safe at all and shouldn't be
> used, since lists, dictionaries, sets, and pretty much any random
> instance of a class object contain state in the instance that will
> change upon use. Tuples and strings are specially noted to be immutable
> and so are usable in those situations without extra locking, but they're
> special-cases, not the norm, in a pass-by-reference language. Using
> module-level globals sparingly is something we certainly encourage.
I'm not sure how relevant these references to Haskell, Erlang etc.
are. Python, like most other languages, does not guarantee intrinsic
thread safety of its data structures, nor do we expect this. Armin's
point relates to *shared* data across threads, viz. the templates
themselves. When concurrent requests are handled by different threads,
they create separate instances of instrinsically unsafe data
structures, e.g. context and response objects, but as these are not
shared among threads, we don't care that they're not thread-safe
implementations. However, the templates *are* shared, and so they have
a responsibility not to store state in themselves, especially as the
context is there for precisely that purpose. Otherwise, when two
concurrent requests invoke the same template containing a cycle tag,
the cycle tag will not perform properly because the different threads
trample over each other's state. This is, it seems to me, quite
different to needing to set the cycle tag to a known state at the
start of template execution.
> Fortunately, this isn't a real issue, since there's nowhere that we
> suggest the assumption of immutability would be valid (and, really, a
> decent programmer is going to know that any arbitrary data structures
> have the same behaviour unless it's noted as immutable; minimal
> experience says that making assumptions contrary to that always leads to
> trouble). You create the template object as part of a view and render it
> and things are (mostly -- see next para) fine. There's no real need to
> put template instances into globals.
>
> The "mostly" bit is because we do have some reset-ability issues with
> things like the cycle tag -- so rendering it multiple times in the same
> thread of execution, such as creating mass mail, will lead to differing
> results.
>
> Also, so that it's on the record: Armin denigrating Eric Holscher for
> ticket #7501 was a low-blow for multiple reasons. One being that Armin
> either misunderstood or ignored the type of object that a Template
> instance is, as noted above. Another being that nominally avoiding
> mentioning specifics whilst giving enough information that somebody
> could easily find it out anyway is simply sleazy. It devalues Eric's
> work for that part of the audience who don't know the history and makes
> the author of the piece just look small. To fill in the missing pieces,
> I explicitly asked Eric to change the title at the Lawrence Sprint, as I
> was busy doing something else at the time and had been meaning to change
> it for a while and Eric was going through doing the scut work of making
> the changes through the Web browser interface whilst Jacob and I fed him
> decisions on various points. I made this change in full awareness of the
> difference between multi-threaded and multi-run use-cases and knowing
> that Template instances are just like lists, dictionaries,
> sets, ...(insert 150 other data structures here).
See my comment above. I don't believe it is relevant to invoke the
lack of intrinsic thread safety in lists, dictionaries etc. as the
myriad instances of these objects are not shared between threads
during concurrent processing of requests. The templates, however, are.
Therefore, they should be thread-safe, in the sense of allowing the
same instance to be used by multiple concurrent threads, without any
surprises. This can be easily achieved by storing any needed state in
the context [or other private object created per template execution].
If it is not, I'm not sure that Django templates are thread-safe, and
Armin would appear to have a valid point.
Armin could certainly have chosen better words when commenting on the
change to the #7501 ticket summary, but there is a real point here -
there are actually (it seems) two bugs, which have been conflated by
the change in title. The bug Malcolm is talking about is not setting
the cycle tag to a known state at the start of template execution.
This is not the same as the bug Armin reported - that of the cycle
state being trampled on by different threads during their concurrent
execution, due to the cycle tag's state being stored in the template
itself rather than in a per-template-execution object such as the
context.
Looking at the code of the cycle tag,
class CycleNode(Node):
def __init__(self, cyclevars, variable_name=None):
self.cycle_iter = itertools_cycle([Variable(v) for v in
cyclevars]) # Malcolm's bug (possibly)
self.variable_name = variable_name
def render(self, context):
value = self.cycle_iter.next().resolve(context) # Armin's bug
if self.variable_name:
context[self.variable_name] = value
return value
The first commented line may be Malcolm's bug - the cycle_iter
instance should be initialised to a known state, which should happen
whenever a template is executed, which may or may not be when the
CycleNode instance is created (which, I would guess, is at template
load/parse time).
The second commented line is probably Armin's bug. The cycle_iter
instance should not be held in the CycleNode instance, because that,
as part of the template, is shared between threads. Instead, the
cycle_iter should be held in the context or another separate object
which is specifically for use in one template execution only.
In general, couldn't one assert that any statement in the render
method which mutates self [ above, this is done by
self.cycle_iter.next() ] could lead to incorrect behaviour? It would
be useful to know what other examples Armin has found. For example, I
see in IfChangedNode.render the assignments "self._last_seen =
compare_to" and "self._last_seen = None" which could potentially cause
the same kind of problem, if the same node instance is accessed by
multiple threads.
I'm not looking to step on anyone's toes here - just trying to clarify
my understanding of the thread safety (or otherwise) of the current
template engine implementation.
Regards,
Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
