Hi Armin,
>> First, different kinds of newlines ('\n', '\r' and '\r\n' in Python)
>> are replaced by '\n' by
>> source = '\n'.join(unicode(source).splitlines())
>
> That's done because the lexer is using lookbehinds and negative lookbehinds
> in regular expressions which in Python are fixed width. As such the
> newlines have to be normalized to a fixed length form (and for simplicities
> sake I chose to normalize to a unix newline). That said, the function also
> currently drops the trailing newline and the information if such a newline
> was there or not.
Aha, that is the reason behind that! Now I understand that the change
would not be just as trivial as I thought ... Thanks for the
explanation!
> I will update the documentation for sure and consider adding a flag that
> controls the trailing one.
Great. I would not think the option for just that to be necessary, but
it might be useful for someone.
> Jinja2 currently only normalizes newline whitespace. The rest is kept
> unchanged. On top of that it supports a wide range of unicode supported
> whitespace characters as token separator inside the Jinja2 blocks. Someone
> was joking last time that this makes it impossible to use Jinja2 to generate
> "whitespace" (the programming language) sourcecode, but so far that was the
> only use case where the normalization of newlines caused problems. If you
> have some more use cases I will consider changing the lexer to operate on
> arbitrary newlines instead of normalizing them upfront.
You are of course right -- the newline replacement hardly ever matters.
My two reasons for digging into this were the idea of generating
binary-like data (for which Jinja is not exactly suited anyway) and
that I like my favourite tools to work in a well-defined and
straightforward way ;-)
Thanks for being patient with me.
Best regards and good luck,
Tomas
--
You received this message because you are subscribed to the Google Groups
"pocoo-libs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/pocoo-libs?hl=en.
