Re: Template-based widget rendering

[Preston Timmons]

Carl,

The default renderer is backwards-compatible with no settings changes, 
> and does not require Jinja2, but it still allows overrides/additions of 
> widget templates. It's also standalone-compatible, in that it is 
> self-contained and has no dependency on TEMPLATES config. 


I like this idea. It's explicit and predictable. A lot simpler than our 
current approach.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/952f690d-bfc9-4aba-ba46-0501fd7f27bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Problem with caching template loader

[Preston Timmons]

Yep, that looks wrong. Looks like it was added in this commit:

https://github.com/django/django/commit/f33db5a09acfc3df3085235a5712c46094eb9a0d

The test case could be improved also by checking the appropriate key is set.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/de745f58-3ad6-4688-881f-1379a0e9a4ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Template-based widget rendering

[Preston Timmons]

Hey Curtis,

I think you're asking how this patch will help with form and field layouts?
If so, not that much. It only addresses moving the widget HTML that
currently is hardcoded in Python into templates.

For example, compare:

https://github.com/django/django/blob/master/django/forms/widgets.py#L272

to the template version:

https://github.com/django/django/blob/15667/django/forms/templates/django/forms/widgets/input.html

It also enables easier custom widgets, like the admin clearable file input:

https://github.com/django/django/blob/15667/django/contrib/admin/templates/admin/widgets/clearable_file_input.html

There's nothing in this patch that would hinder further development to
convert the form rendering methods, like `Form.as_p()` to be template
based also, or providing better rendering methods altogether.

With that said, yes the renderer class is able to be set per form class
and as an argument to `Form.__init__()`.

Preston



On Tuesday, May 10, 2016 at 10:32:30 PM UTC-5, Curtis Maloney wrote:
>
> Sorry for the late entry to the discussion, but I was looking over the 
> code and wondered about something. 
>
> In projects where I've used my django-sniplates for form rendering, it's 
> been helpful that I can have several different form widget sets within 
> the one project -- for instance, for side-by-side labels, or top-labels, 
> etc. 
>
>  From what I can see so far of the code/docs, there's no way to override 
> which set of form widgets get used on a per-form basis... let alone 
> per-field. 
>
> Is this correct? 
>
> The only possible avenue I see is a custom renderer class that somehow 
> mangles the widget template paths... 
>
> -- 
> Curtis 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/856af7b0-9d1e-485b-ac57-e37f4d4cee04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Template-based widget rendering

[Preston Timmons]

+1. I like the simpler fallback solution Carl has suggested also.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a5f5ad63-71f0-4ba5-bd21-028d79b0a3b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Make template caching a feature of the Django template engine

[Preston Timmons]

When I created my initial branch for this, it was easy enough to auto 
reload a file once it changed, but it was difficult to deal with template 
directory precedence. For instance, if `admin/base.html` was cached, but a 
custom `admin/base.html` was added in a loader or directory with higher 
precedence, the cached version was still used. My concern is that it's not 
clear to the user they need to restart the process to see the new template. 
A similar situation happens when templates are deleted.

I can see why some people would still want the ability to turn this feature 
on anyway, but without a solution for the above cases it seems better to me 
as an opt-in rather than a default.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/65d011fc-469c-4244-925d-ce8b2cc5a80d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Allow custom "BoundFields" on forms and make BoundField public API

[Preston Timmons]

>
> Yes I know that you can access the Field instance but it wouldn't be 
> possible to do {{ field.field.city }} since a Field is not supposed to 
> have any data stored with it. 
>

This makes sense to me. Feel free to make a ticket and attach your PR once 
it's ready for review.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3394b9e8-a8b4-468b-82eb-6979091e1de8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Allow custom "BoundFields" on forms and make BoundField public API

[Preston Timmons]

Hi Moritz,

Is the purpose of this patch to have a more convenient form of this:

{% for choice in form.foo.field.choices %}
  {{ choice.0 }} {{ choice.1 }}
{% endfor %}

I'm not necessarily against the idea, but your patch seems to encourage 
looping over the field choices rather than the widget choices. These two 
values don't always match. A looping API should also account for the case 
of nested choices, as well.

Tim, #15667 makes it easier to loop through choices in the widget template, 
but doesn't provide anything beyond the currently documented API when 
accessing widget choices from the form instance:

https://docs.djangoproject.com/en/1.8/ref/forms/widgets/#radioselect

I'm happy to see improvement here if the current API is a pain point. I'm 
not sure yet this is a better solution than the current API for accessing 
widget choices.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9541ed1f-5070-4753-a046-9cf745cb79f2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: deprecate and remove egg template loader

[Preston Timmons]

+1. Eggs have reached the point of obsolescence.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/421c5490-e440-43b4-b1a8-d69638923f07%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Template-based widget rendering

[Preston Timmons]

Hi Carl,

Thanks for the feedback. I agree with your suggestions. I didn't think about
running a check for a combination of INSTALLED_APPS and a loader with
APP_DIRS. That would be sufficient for customizing loading.

I'll update and convert this to a PR.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7e2e0274-aea5-42c0-92ba-f4b54346dd6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Template-based widget rendering

[Preston Timmons]

Hi all,

I've been working through solutions for #15667 -- template based widget
rendering. This is a problem that was close to a solution at one time, but
stalled out due to performance concerns and difficulties with defining a
workable API to create configurable template loaders.

Now that Jinja2 is supported, performance isn't as much a concern. So,
I'd like to tackle the API portion of this.

This problem has two sides:

1) Converting individual widgets to be rendered with templates.
2) Deciding how to instantiate a user-customizable template engine.

The first is easy to accomplish, but the second isn't. I've written a
proposal for the second problem available here:

https://gist.github.com/prestontimmons/24a2a835bea590afb70b

In addition, a WIP branch is available here:

https://github.com/prestontimmons/django/tree/ticket-15667

Hopefully, we can finally get this in. :)

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c0ea2593-05c2-4307-86ea-1b34c05d0f7a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two phase cleaning of models/forms

[Preston Timmons]

Yep, I think you're right now. Given existing solutions there's not warrant
enough to add another built-in validation hook.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1dd15f29-271a-46ca-b041-e77f48c0a567%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two phase cleaning of models/forms

[Preston Timmons]

Since I think this can be a useful feature, I'll explain why.

One use case is for validating address forms. We deal with a lot of
them with varying levels of validation based on country, state, zip
code, etc. Sometimes, multiple sets of address fields appear on the
same form. We can't simply reuse the fields without worrying about
disparate validation routines specified on the form in addition to the
fields. This leads to a meticulous set of mixins. It gets the job done,
but I don't think that's great api.

If fields had a second clean method that was called with the form
data, we could do something simpler like:

state1 = StateField(country_field="country1")
state2 = StateField(country_field="country2")

The field would then look like:

StateField():

def __init__(self, country_field=None):
self.country_field = country_field

def clean_full(self, cleaned_data):
if self.country_field:
do_something(cleaned_data[country_field])

Yes, clean_FOO() can be made to work. No, it doesn't make reusing
fields with complex validation in multiple forms and contexts easy,
especially if the field names may need to differ in certain forms.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/473fc160-dd08-4899-90c2-36589cafb264%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two phase cleaning of models/forms

[Preston Timmons]

There are times when I've definitely wanted this feature. Particularly, 
when multiple fields on a form have this type of constraint. Putting all 
the logic in the clean method gets convoluted.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2c61b6fb-801b-4c1a-9fd7-9f42dec3d1c7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: GSoC 2015: Template engine optimisation

[Preston Timmons]

>
> I've done a couple of days of investigation into template performance 
> recently trying to speed up our site and my main takeaway was that there 
> was no silver bullet - no particular node taking up all of the time. I was 
> mostly trying to optimise a particularly complicated template we render a 
> lot in a loop so I was modifying the template, Django code and making 
> custom tags to see what would make a difference rather than trying to 
> decipher profiles. We use the cached template loader so compile time wasn't 
> really considered.
>

Thanks, Sam. That's helpful information.

I'd be interested to know the template you used, or at least one 
representative of the template you used. Templates that highlight 
real-world pain points would help to benchmark the things that matter.

Did you apply all the internal Django changes you mentioned cumulatively? 
If so, what was your overall rendering time improvement? How did the 
speedups from changing nodes compare to the speedups from modifying the 
engine? It sounds like the engine changes were a factor because of multiple 
calls to the include tag.

While the changes you mentioned don't sound like ones that could go into 
Django, they do shed light on what can potentially be done or not for 
optimization.

Preston


 

> We tried the following things and none of them made more than a couple of 
> percent of difference each:
>
>  - we made a cut down {% url %} tag that just does what we need - the 
> built in tag can handle a lot more at a performance cost
>  - I grouped {% with %} statements together - ideally grouping into {% 
> include ... a=b c=d %} - to avoid extra layers of context
>  - I ditched TextNodes that just contained whitespace (both by removing 
> the whitespace and by automatically removing the Nodes on compile) - it's 
> easy for these to build up when you have code like the following and every 
> extra node slows things down a bit (whitespace is sometimes meaningful so 
> we would have only implemented this for particular bits of code)
>
> {% if whatever %}
>{{ my_var }}
> {% endif %}
>
>  - I also tried commenting out (or replacing with "return '' ") chunks of 
> the template engine code and our template and it just seemed that the more 
> I commented out, the faster it ran - no particular jumps in speed, just  a 
> gradual change as more was removed. Escaping was one of the first things I 
> commented out and it made a surprisingly small amount of difference.
>
> I'd be very happy to be proven wrong but thought it was worth sharing my 
> findings since - particularly in the context of Preston's suggestion that 
> we might find a bottleneck - I don't think there is one particular 
> bottleneck.
>
> Our "solution" for now has been to speed up the processors in our servers 
> and investigate switching to pypy - we'll probably be looking at Jinja2 
> once we upgrade to 1.8 as well.
>
> Sam
>
> On Thu, 12 Mar 2015 at 15:57 Preston Timmons <preston...@gmail.com 
> > wrote:
>
>> After a while I believe layers and layers of caution have accrued, and 
>>> nobody is sure any more where these have overlapped excessively.
>>>
>>
>> Do you have examples of which layers these are?
>>
>> Escaping seems to happen in Variable, VariableNode, FilterExpression, and 
>> render_value_in_context. I don't see a lot of work being done twice there.
>>
>> If you think the escape implementation is slow, though, it wouldn't be 
>> hard to simply remove that and benchmark with no escape code running. That 
>> would at least reveal the theoretical limit to which the escape code could 
>> be improved.
>>
>> On another note, in my benchmark I see the difference to render a 
>> somewhat complex template between Django and Jinja2 at 8-9 times.
>>
>> From one run, just grabbing the minimum time to render a template:
>>
>> Django: 9.08e-05
>> Jinja2: 1.38e-05
>>
>> The big difference here is because Django uses a recursive node-based 
>> renderer, whereas Jinja2 just translates the template into Python. That 
>> means a lot less overhead when rendering happens.
>>
>> Optimizing Django rendering means either:
>>
>> 1) Identifying the nodes which are a bottleneck and reducing the work 
>> they do
>> 2) Replacing node rendering with something that's faster.
>>
>> Option 2 probably has the biggest opportunity for gain, but it would 
>> require some real creativity to maintain backwards-compatibility with 
>> existing tags.
>>
>> Preston
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers 

Re: GSoC 2015: Template engine optimisation

[Preston Timmons]

>
> After a while I believe layers and layers of caution have accrued, and 
> nobody is sure any more where these have overlapped excessively.
>

Do you have examples of which layers these are?

Escaping seems to happen in Variable, VariableNode, FilterExpression, and 
render_value_in_context. I don't see a lot of work being done twice there.

If you think the escape implementation is slow, though, it wouldn't be hard 
to simply remove that and benchmark with no escape code running. That would 
at least reveal the theoretical limit to which the escape code could be 
improved.

On another note, in my benchmark I see the difference to render a somewhat 
complex template between Django and Jinja2 at 8-9 times.

>From one run, just grabbing the minimum time to render a template:

Django: 9.08e-05
Jinja2: 1.38e-05

The big difference here is because Django uses a recursive node-based 
renderer, whereas Jinja2 just translates the template into Python. That 
means a lot less overhead when rendering happens.

Optimizing Django rendering means either:

1) Identifying the nodes which are a bottleneck and reducing the work they 
do
2) Replacing node rendering with something that's faster.

Option 2 probably has the biggest opportunity for gain, but it would 
require some real creativity to maintain backwards-compatibility with 
existing tags.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/957b9840-b537-4861-9390-e7b6f44dd72c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: GSoC 2015: Template engine optimisation

[Preston Timmons]

Hi Oleksii,

I found that cProfile isn't that helpful when rendering templates. There 
are a lot
of function calls and the output is too verbose to really reveal where 
Django
spends it's time.

Also, keep in mind that rendering is only one step of the template cycle, 
and
usually only a small part of it. There are these steps to consider:

* Template loading
* Lexing, parsing and compiling
* Rendering

Here are some recent benchmarks I've done on the template engine:

https://groups.google.com/d/msg/django-developers/VFBLAoPSplI/pPzOYm3PUVQJ

>From what I can tell, if we compare Django templates to Jinja2, which are
considered quite fast, the biggest visible difference doesn't come because
Jinja2 has a faster parser or renderer. It's because it maintains an 
internal
cache. Jinja2 only recompiles templates when it has to.

Depending how things go with ticket #15053, internal caching might become
part of Django, though. If that's so, your proposal will need to hone in on 
identifying other specific areas you think performance can be improved.

The Django parser and lexer are parts that could be completely rewritten,
for example, while easily maintaining backward compatibility. Changing the
rendering layer is much more difficult because multiple 3rd-party libraries
depend on the Node class.

If you're serious about working on this, I suggest digging into the 
benchmarks,
identifying an area that can be improved, and providing a proposal for how
you think it can be made faster.

Good luck.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c16befb5-c12d-43cc-84a2-6931c245b8a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

My branch is updated. This now combines the origin classes into one. It also
includes a simpler cache algorithm than before. This same algorithm can be
used if we implement an internal cache to the engine, which is now a small
addition to this patch.

I'd like to see origin.reload go away but it's left intact in this branch. 
My other
PR to rework template debug information for multiple engines removes this.

A related goal for both of these branches is to simplify the debug view to 
only
deal with the presentation of the debug information rather than creating or
replaying information. It looks like we can simplify it a lot.

Preston


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6115dbc2-e8d3-43f7-a819-1b77fb52326b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Drop the TEMPLATE_DEBUG setting

[Preston Timmons]

Hey Curtis,

I was working to remove origin.reload and ended up with a fork that 
combines the debug implementation:

https://github.com/django/django/pull/4254

On a complex template with debug off I'm measuring about a 5% increase in 
template compile time compared to master. Turning debug on adds another 1% 
on top of that. That's a lot better than the current debug implementation.

80-90% of the additional time seems to be in the lexer, not the parser. 
This is due to the finditer loop being used to annotate the start and end 
position of each token in the template source.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8c7e0662-71c3-4fde-b4e7-8f334bd38112%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

A small update:

I have a local branch working again, but nothing ready to review.

I have decided the origin.status and origin.tried apis I mentioned above 
aren't
going to work. For templates to be safely cacheable in a multi-threaded
environment, they basically need to be treated as immutable. Anything that
manipulates a template or it's subobjects is problematic.

I think I can work with this restraint, though.

The next things I'll probably submit is to remove the origin.reload method. 
As
mentioned above, we should prefer template loading to go through the loaders
instead. I've made progress here but have went down the rabbit hole of 
cleaning
up the debug implementation.

It looks promising that we can combine the debug parsing, as brought up in 
another
thread, with minimal overhead. Most of the extra time is due to extra 
function calls,
not tokenizing. These calls aren't necessary if there's a single parser and 
lexer
class. I think we can avoid copying debug info to every token and node as 
well.
This can also address #24119.

Unfortunately, this part of the code isn't tested well. When I have 
additional tests
written I'll submit a PR.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c6a1e8fe-2438-4b88-bd23-38a2e50c6577%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

> In that case, would it be possible to do the caching improvements before 
> the main
> refactoring? I'm trying to keep patches at a reviewable size :-)
>
> -- 
> Aymeric.
>

Maybe. The existing loaders don't provide a nice way to add 
Origin.uptodate. It would
require an informal hack that only supported Django builtin loaders 
initially.

I'll start by separating out some of the cleanup/groundwork changes I did 
into separate
PRs. If those can go in first, then there'll be a lot less noise in the 
patch.

At the moment, I think it will be easiest to land the cache improvements 
after the
loaders are updated.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/db492cc9-082c-4d76-9a8f-3aff14b5d018%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

>
> Yes, I do. 
>
> This is an independent follow-up to the big refactoring we’re talking 
> about, isn’t it?


Cool.

It depends. The cached loader is what I'm least happy with in my 
refactoring. This 
internal cache idea I think is simpler, more performant, and easier to 
understand.
Adding it makes the cached loader changes unnecessary.

Because the changes to support it are minimal, I'll at least make a 
side-branch that
replaces the cached loader changes with an internal cache. If it pans out 
and doesn't
balloon into further unrelated changes, it might be easier to include it in 
this refactoring.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3f0f9cca-ef69-4528-acef-2c4067035606%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

>
> Thanks for your continued work on this and sorry for my slow answers. 
>

No worries. I've been taking the time to get a better grasp of Django 
templates as a whole.
 

> > Does that makes sense, or should I look for a way to incorporate that 
> > into LoaderOrigin? 
>
> Looking at the implementation, it feels weird to add attributes to the 
> origin 
> that are only used by the loader… If origins were dumb bags of data this 
> would be all right but they have a reload() method that calls the loader. 
>
> Such mutual dependencies never end well :-/ That’s a broader design 
> problem not directly related to your refactoring, but while you’re 
> breaking 
> all the things, perhaps we could fix it too.
>

Yes. I'll look at this. The reload method isn't documented and I think it 
can
go away if the debug view is updated.
 

> By the way, I would like to check if we can deprecate the dirs argument of 
> LoaderOrigin. It may only be used by deprecated code as of Django 1.8. 
>

Deprecate or remove? :)

This argument is one I don't think is documented anywhere. 
 

> In fact context.template.engine would be much better than context.engine. 
> I’ll try 
> to fix that in 1.8 before it’s too late. 


Sure. I forgot to mention it but my latest patch actually solves this by 
passing origin
in context.render_context. This makes sense because render_context is 
available
only in the current render scope, rather than modifying the global context. 
I'm just
as happy with accessing context.template, though.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a9a02345-feb9-4d5e-b14c-09e61ae386e4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Feature proposal: Conditional block tags

[Preston Timmons]

After looking at this, there's nothing special about blocks compared to any 
other node. They could just as well be evaluated at render time.

Here's a branch that implements this change:

https://github.com/prestontimmons/django/compare/conditional-blocks?expand=1

Before returning blocks into block_context to be used as overrides, it 
loops through any if nodes and checks the conditions.

The one case that's kinda funny is if you do something like:

{% if var %}
{% block content %}...{% endblock %}
{% else %}
{% block content %}...{% endblock %}
{% endif %}

Currently, the second block node will raise a TemplateSyntaxError because 
it's defined twice. Does that matter?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/71aa2d20-d162-4620-b4c6-8cb915a1da5d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Drop the TEMPLATE_DEBUG setting

[Preston Timmons]

Here are some benchmarks I added here:

https://github.com/prestontimmons/templatebench

This is the cumulative time to do 1000 iterations:

Basic.do_init: 0.1423451900
BasicDebug.do_init: 0.1941769123
35% increase in parsing time

Basic.do_parse_complex: 1.2230978012
BasicDebug.do_parse_complex: 1.4190740585
15.5% increase in parsing time

FileSystem.do_get_template_complex: 1.4923889637
FileSystemDebug.do_get_template_complex: 1.7524909973
17.4% increase in parsing time

FileSystem.do_get_template_index: 0.5193221569
FileSystemDebug.do_get_template_index: 0.5603711605
9.8% increase

If my benchmarks are right, the increase is measurable, although probably 
not enough to notice in most usage of Django templates. Increasing parsing 
time isn't ideal, though, since that's where Django templates seems to 
spend most of their cpu time.

>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9cc2bb2d-0335-4d49-8529-9c3dbdd07467%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

Hi Aymeric,

I'm thinking of proposing an alternative to the cached loader. This new
approach makes Django faster in general.

To start, I put together some benchmarks here:

https://github.com/prestontimmons/templatebench

The goal was to identify where Django spends it's time. Is it the loaders 
that
are slow? The parsing? The rendering? Something else?

Here are some basic timings from my Macbook air. This is the cumulative time
to run 1000 iterations:

Instantiating a basic template, i.e. Template("hello"):
0.0344369411

Parsing a complex template with extends and includes:
0.3044617176

Unsurprising so far, but the time for parsing measurably grows as the 
template
has more to parse.

Running get_template with a simple template, like "hello":
0.1308078766

Running get_template on a complex template:
0.4068300724

With a simple template, more time is spent finding the template than parsing
it. As template contents grow, though, the parsing time far outgrows the
template loading time.

Running get_template on a template with 200 includes:
12.2357971668

Here's a classic case where Django bombs. The parsing time really adds up.

Time to render a basic template:
0.0240666866

Time to render a complex template:
0.1018106937

In this case, the rendering of a complex template takes four times more than
a simple template. This is compared to a 10 times increase in parsing time
from the previous benchmark. A chunk of this time is also parsing, though, 
due
to extends and include nodes. All in all, the parsing time grows much 
quicker
than the render time does.

Based on these benchmarks, I've come to believe most of the time in Django
templates is spent on parsing, not on loading templates or rendering. The
cached loader is effective because it removes the need to reparse templates
more than once.

Interesting enough, Jinja2 has different results:

Running get_template with a simple template, like "hello":
0.0112802982

Running get_template with a complex template:
0.0122888088

Even complex templates make little difference in parsing time for Jinja2.

Running get_template on a template with 200 includes:
0.0110247135

Many includes don't make a difference.

Time to render a basic template:
0.0134618282

Time to render a complex template:
0.0217206478

For a complex template, Jinja2 rendering is about 50% faster than Django.
Even so, the overall time difference is small since rendering is quick
anyway.

After digging into Jinja2, I think this is because the Jinja2 environment
keeps an internal cache of templates. If a template is in the cache, it
calls the template "uptodate" method. If "uptodate" is true, the cached
template is used. For filesystem loaders, this incurs a filesystem hit each
time, but that's fine. File system calls aren't the bottleneck. Parsing is.

With that, I wondered if we couldn't do something similar in Django. I made 
an
experimental commit here, based on my branch:

https://github.com/prestontimmons/django/commit/4683300c60033ba0db472c81244f01ff932c6fb3

This adds internal caching to django.template.engine.Engine and to the 
extends
node. It also adds an "uptodate" method to the template origin so templates 
are
reparsed when modified. This is different than the cached loader, which 
never
checks if templates are changed. That means it's also viable in development.

Running get_template with a simple template, like "hello":
Before: 0.1308078766
After:  0.0192785263
Jinja2: 0.0112802982

Running get_template on a complex template:
Before: 0.4068300724
After:  0.0204186440
Jinja2: 0.0122888088

By parsing only when necessary these benchmarks see a 10-20x speed up.

Running get_template on a template with 200 includes:
Before: 12.2357971668
After:  0.0179648399

Using include many times is now an option.

So far, all the tests pass, and I've been testing with other templates. The
implementation seems almost too easy for the increase in speed.

Granted there's not a dealbreaker I haven't noticed yet, I'd like to propose
that we follow Jinja2's example by adding internal caching in place of the
cache loader. It has a nice speed increase and simplifies things for my
recursive loader branch as well.

There is one risk I can think of. External template tags can store state
on the Node instance rather than context.render_context. This is warned
against in the docs and is not thread-safe. In practice though, if the 
cached
loader isn't used, a developer could be unaware that they have a problem
at all. Switching to an internal cache would cause those to be revealed.

Even so, I think that can be handled with documentation.

Do you think it's worth making an attempt to formalize this?

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email 

Re: Formalizing template loader and debug api's

[Preston Timmons]

My pull request is updated with a simplified cache loader and docs. I also 
found
some nicer solutions to some of the hackier things, like making origin 
available
to the ExtendsNode.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fa62be63-544c-4191-94a0-adc58315bd58%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: discontinue shipping tests with contrib apps?

[Preston Timmons]

The trickiest tests to move will be the gis tests.

When gis is enabled, django/contrib/gis/tests is added to the discovery 
path. This doesn't affect which tests are discovered per se--all gis tests 
are discovered anyway by discovery on the parent app 
django.contrib.gis--but it causes the later side-effect of 
django.contrib.gis.tests.geo3d, django.contrib.gis.tests.geoapp, etc. being 
added to INSTALLED_APPS during the test run. The purpose of this logic is 
to avoid running migrations when gis isn't enabled.

If the test apps stay as subdirectories in the gis test folder, runtests.py 
will need to be updated accordingly. If the namespace is made flat instead, 
we'd need to update runtests.py to do something like filter out gis_* apps 
from INSTALLED_APPS when not available.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5482299a-b900-49f0-ba4a-cf26dd1165fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: discontinue shipping tests with contrib apps?

[Preston Timmons]

I think the "need" is mainly conceptual--whether tests are more 
appropriately grouped with their app or with the other tests. With the 
discover runner it's uncommon that contrib tests are included in any local 
test runs.

I do prefer moving all tests into the tests directory. The logic to get 
test_modules in runtests.py would be simplified quite a bit from it.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ba79f1b9-678e-4fdf-8f7a-26319e5ac4d3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

Hi Aymeric,

It took me awhile, but I haz an update:

https://github.com/django/django/pull/3475

Loader cleanup

This addresses the code duplication between get_internal and
load_template_sources. I also made the app directories loader inherit
from the filesystem loader, since the only real difference between the two
are the directories they look in.

Cached loader

The cached loader didn't clean up so nicely. I thought I could remove
the custom load_template implementation, but a problem arises if the cached
loader is configured with both recursive and non-recursive loaders. 
Therefore,
I kept the old implementation in place for now.

Updating the cached loader was a bit tricky. The old loader caches templates
and TemplateDoesNotExist exceptions with a key defined by template_name and
template_dirs. This isn't as straightforward when the skip argument is added
to the equation.

The options were then to create a custom cache key per skip argument, or 
defer
caching to the lower-level methods. During extending, skip always includes 
the
originating template. That would lead to many more cache misses, so I chose 
to
do the latter instead. When skip isn't provided, I use the simple algorithm.
There may be a simpler approach, here.

I've done some performance testing without noticing much difference, but 
this
is something I'll test some more. I also experimented with using lru_cache
instead of local dictionaries. This simplifies the implementation but is
measurably slower.

Origin hash

The cache algorithm I added uses the origin object as a cache key. The 
"name"
attribute is used as part of the key value. Therefore, the underlying loader
must be sure to set name to a unique value per template.

You mention above that "name" must be optional for loaders that don't load
templates from the filesystem, but I'm not sure this is true. Either name,
or another attribute on origin, needs to uniquely identify the template 
within
a loader. Otherwise, there's not a way to implement __eq__ and __hash__. The
locmem loader is a non-filesystem loader that is able to do this. I also
implemented a sample db loader that does as well:

https://github.com/prestontimmons/dbloader/blob/master/dbloader/loader.py

Can you think of a loader where we wouldn't want to require a unique name?

Egg and db loader origin

Some loaders use different identifiers for templates. The filesystem loaders
are just file paths and the locmem loader is just a string key, but the egg
loader is a tuple, (app_config.name, pkg_name) and the sample db loader is
the model instance.

The egg and db loaders don't have an obvious attribute on origin to save the
extra information to. You'll see in those cases that I subclassed origin as
EggOrigin and DbOrigin.

Does that makes sense, or should I look for a way to incorporate that
into LoaderOrigin?

LoaderOrigin vs StringOrigin

I looked at combining these, but didn't yet since it doesn't impact the 
feature
this patch is implementing. If they're combined to a DTLOrigin, should that
live in django.template.base? It can't live in django.template.loader since
that module isn't safe for django.template.base to import.

context.origin

In order for the origin to be available to the ExtendsNode, I also used the
context.origin = self.origin hack in the Context.render method. I saw the 
logic
you used to only set engine on toplevel_render. When I moved this assignment
into the if statement or not, it didn't seem to make a difference.

Is that a hack we can live with for now?

Debug view

This branch includes an update to the debug views. The best way to view the
output is to run the debug page in a browser. I added a sample project here
with various scenarios:

https://github.com/prestontimmons/project-15053

There's one case where the debug postmortem isn't optimal. If multiple 
engines
are configured and a template is found by the second engine that fails
extending, the postmortem won't include the templates that were tried by the
initial get_template call to the first engine. I'm not seeing an obvious way
to persist that information at the moment.

Things to do

The next things I plan to do is measure the cached loader performance more
closely and add the docs.

-- 

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5aad6c45-70eb-45cb-92d9-4e975e3dc50a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

>
> Perhaps we’ll bikeshed names at some point but let’s leave the “naming
> things” problem aside for now.
> “name” must be optional for loaders that don’t load templates from 
> files.
> “loadname” must be optional too for templates instantiated from 
> strings.
> Here optional means that things don’t break if the value is None.


If we combine LoaderOrigin and StringOrigin, then yes, it makes sense for
these to be optional. The origin will then gain an additional "source"
attribute from StringOrigin.

I don’t like the code duplication between get_internal and
> load_template_source. I assume your design ended up that way because
> you needed get_internal and load_template_sources will be deprecated.
> Is this correct?


Correct.

To minimize duplication I can modify load_template_sources to use 
get_internal.
The test suite won't hit this code path anymore, though. That means I should
probably add additional tests to cover the deprecated methods.

That leaves the cached loader. If we modify Loader.load_template to
call Loader.get_template when get_internal is available, then I think the
cached.Loader.load_template and cached.loader.find_template methods can be
safely removed.

I'll go ahead and make these updates in the branch.

TemplateDoesNotExist is used both internally by the DTL and in generic
> APIs such as django.template.loader.get_template. If it’s designed to
> accept a list of tried origins, then we need:
> - a generic Origin without engine, loader, status and tried (at least)
> - a DTL Origin as described above.
> Does this make sense?


I think this is fine.  "tried" and "loader" are internal concepts to Django.

If an engine is able to support the post-mortem an origin needs only to be 
able
to return a string representation of the load method and a status. These are
optional if the engine can't provide this information.

Yes. Maintaining compatibility with LoaderOrigin as documented in 1.7
> appears to be a lost cause anyway.


Okay.

I think the appropriate way for origins to return template code would 
> be to
> call back into the template loaders. Reading template contents in the 
> job
> of the loader. It should be possible to write a single origin class 
> that works
> for most loaders.
> The sole purpose of the reload() method is to display tracebacks in 
> templates.
> This logic should be moved out of the debug view and into the template
> backends. See #24119. As a consequence, we should leave it outside of 
> this
> refactoring and simply preserve it for DTL origins until that ticket 
> gets fixed.


Sounds good.

If you’re going to change both its arguments and its return type, you 
> can change
> the method's name :-) The second solution looks better to me. 
> Hopefully we can
> write a “just do this” guide to help people maintaining custom 
> template loaders
> upgrading.


Okay. I'll post back when I've implemented things further.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1bbd1aed-5732-470c-bc52-00a78052b10e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

Hi Aymeric,

I got a chance to update my patch with the use of origins. The good news is
that it's simpler than the old implementation. I have a few api questions
below. The updated branch is here for now:

https://github.com/prestontimmons/django/compare/ticket-15053-origin?expand=1

As a quick recap:

First, this branch uses origins everywhere: for extends history, for
TemplateDoesNotExist errors, etc. There's no more 4-tuple of statuses.

Second, I use these attributes on the origin:

engine
loader
loadname -> template name
name -> full template path
status -> Skipped, Found, Source does not exist
tried -> a list of origins tried up to this one
reload -> gets template contents
__eq__
__hash__ -> added so the cache loader can use origin as a cache key

Third, loaders implement these methods:

get_template_sources -> yields origins
get_internal -> takes an origin and returns the template contents

Fourth, TemplateDoesNotExist takes an optional tried argument which is also
a list of origins. This will be used in the debug views. I didn't change the
debug views in this branch yet, though.

*Questions*

*Origin attributes*

I'd like to do some renaming:

origin.loadname -> origin.template_name

I think it's hard to remember the difference between origin.name and
origin.loadname. I think "template_name" is more conventional.

origin.reload -> origin.read

If the method to read a template contents lives on the origin, I chose to 
prefer using the origin method rather than the loader method directly. 
There's
not much consequence here, but it means it's no longer a special method just
for the debug view.

*tried and status*

The nice thing about putting the tried and status on origin are that it 
works
well with a recursive api. It even means I could get rid of the extra 
context
variable used to store extends history. The sucky things are that tried and
status are set and manipulated outside the origin constructor. The cached
loader also has to be sure to reset the origin before returning the cached
template. That might be warranted, but part of me wonders if it's lousy api.

*Deprecation path*

If we want to maintain the load_template api, I can change it to accept the
skip argument. Within load_template I can add a hasattr check for
"get_internal". If it exists, use the new code. Otherwise, use the old
"load_template_sources" method.

Since adding a kwarg to load_template could potentially break 3rd-party
loaders that override it, I could add an _accepts_skip_kwarg property to
base.Loader like you've done with _accepts_engine_in_init = True. This
would enable Engine.find_template to exclude that if necessary.

If we do this, I'd like to deprecate the (template, template_origin)
return value in favor of just template. Returning a tuple is redundant.
Ditto for Engine.find_template.

If we use get_template instead, we don't need to deprecate the return value 
and
we don't need _accepts_skip_kwargs. Instead we can just recommend use of the
new method until load_template is removed.

Either way, load_template_sources is obsoleted.

Which deprecation path do you think is preferable?

*Reading template contents*

If we have an origin.read() function, an option to consider is updating each
loader to return a custom origin object rather than adding "get_internal"
or "load_template_source" to the loader. These could be something like
FileOrigin, EggOrigin, CacheOrigin, DbOrigin, etc.

I can't think of any deeper advantages this holds, though. So I leave it at
that.

*Adding skip to Engine.get_template/Engine.find_template*

I wondered whether skip should also be added to Engine.get_template. Doing 
so
eliminates some looping logic from Engine.find_template that is 
reimplemented
in the extends node. After implementing it though, I've decided against it:

1) In general, users don't work with origins directly. Therefore I don't 
think
it has much applicability outside the internal loaders.

2) It takes quite a bit of fiddling around to maintain a linear list of
templates that were found or skipped. Even though the new implementation 
removed some duplication, it also wasn't any simpler than the old.

*Conclusion*

If the api so far makes sense, I'll update the debug view post-mortem
messages and add in the docs and deprecation warnings.

Thanks,

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/70225dbc-0316-4c5f-9e13-7297df860d78%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Formalizing template loader and debug api's

[Preston Timmons]

>
> The debugging information is two dimensional data: for each level of
> template extending, for each template loader, you have one entry —
> entries are 4-tuples in your current proposal.


Yes, this is correct. Although there may not be an entry for each loader on
each level of extending. The looping stops if a matching template is found.

At this point, I think we should talk about template origins. What
> is their purpose? What is the complete abstract Origin API? The
> current API only includes a `name` attribute and an unspecified
> `reload()` method; I don't think that's sufficient.


Origin currently has these attributes:

name

For filesystem and app loaders this is the full path to the template.
For the egg loader this is "egg::".
For locmem this is just the template name.

loader

This is the load_template_source method of the loader that found the 
template.

loadname

This is the original template_name used for load_template_source.

dirs

This is a deprecated value set when dirs is passed to get_template.

reload

This is used by the debug views to get the source content of the template. 
It
calls self.loader--which is really load_template_source--with loadname and
dirs.

My branch makes some changes to this class. (Note: this will change a bit if
get_template_sources returns an origin instead)

1. name is the value as returned by get_template_sources. For filesystem and
   app loaders this is still the full path. For the egg and cached loaders 
it's
   a tuple (app_config.name, template_name) or (loader, source).

2. loader is changed to reference the loader instance rather than the
   load_template_source method.

3. A tried attribute is added. This is the same format as passed to
   TemplateDoesNotExist.

4. reload is updated to call loader.get_internal when available, otherwise
   it falls back to loader.load_template_source.

This is what I use origin for:

1. I add origin.name to the loader extends history. This enables the current
template to be skipped rather than re-extended.

2. If an extended template is found, the new origin.name is added to the
extends history. Also, the original origin.tried and new origin.tried are
combined to give a complete list of templates found before and after 
extending.

3. If an extended template isn't found, the TemplateDoesNotExist tried value
is combined with the original origin.tried to give a complete list of what
templates were tried before failing.

4. Loader.get_template sets tried on origin with the values returned by
get_template_sources.

## Potential changes

If get_template_sources is changed to return an origin, it seems to make 
sense
that "tried" would simply be a list of origins instead of a 4-tuple. There
would need to be a status attribute added. This would default to None since 
we
don't know the status until get_internal is called.

"tried" currently lives on origin. This seems weird to me since origin.tried
would contain a reference to itself. It would probably be better to move 
that
to the template object. The extends node would then access the list of 
origins
up to that point, rather than just the current origin.

The extends history would be simplified a bit. Currently, it's a dict that
tracks history for each loader. This is done to eliminate any possible 
clashes
if two template loaders return the same source value. This could be updated 
to
just be a list of origins.

It's sufficient for my patch if origin has these values:

origin.name -> the source path - may be a string, tuple, model instance, 
etc.
origin.loader -> the template loader
origin.loadname -> the template_name value
origin.status -> the status of the template. This is set after get_internal 
is
called.
origin.loader_name -> returns a string representation of the loader class. 
This
is used in the debug view postmortem.

## Other template engines

If "tried" is a list of origins, the other template engines would need to 
use
the origin class if they display a postmortem.

In order to get a correct string representation of the underlying loader in 
the
postmortem, engines could subclass Origin and implement a custom
loader_name method. Using Origin then should be similar to using the tuple.
I don't think it will cause any tie-in to Django-specific loaders.

Would origin.reload be used for other engines? If so, it could also be 
added in
the subclass. I'll have to take some time to understand how the traceback 
works
before I could comment much on this, though.

Thanks.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 

Re: Formalizing template loader and debug api's

[Preston Timmons]

Hi Aymeric,

Thanks for the feedback! 

>
Even if it isn’t consistently implemented, the loader API is documented:
> 
> https://docs.djangoproject.com/en/1.7/ref/templates/api/#using-an-alternative-template-language
> To sum up:
> load_template_source(template_name) —> (template_string, 
> template_origin)
> load_template(template_name) —> (template, template_origin)
> The behavior is loosely defined; anything that matches the signature 
> is fine.
> template_origin is especially ill-defined.
> Are you proposing to change the API outright? Or are you planning to
> implement a deprecation path for existing third-party template loaders?


The deprecation is gradual. Old loaders continue to work but can't take 
advantage
of recursive extending until they implement get_template_sources and
get_internal.

How would a loader that loads templates from a database implement this
> method?
> It seems to me that the proper way to define such a method is “yields 
> some
> opaque objects that implement the following API: …” I suspect that API
> would be quite similar to the Origin API — maybe an idealized version 
> if
> the current version doesn’t suffice.


My original thought was it would yield a model instance which is then 
passed to
get_internal. Your right, though. It'd be better to return an origin or
other opaque object. This is a nice change. I'll update the loaders so they
have a consistent return value.

Attaching this method to the objects yielded by get_template_sources 
> would
> provide better encapsulation than attaching it to the Loader itself.


Agreed.

At first sight adding a “skip” optional argument to Engine.get_template,
> Engine.find_template, BaseLoader.__call__ and BaseLoader.load_template
> may work. Could you explain where the difficulty lies?


There are a few reasons I chose to go with a new api:

First, the advantage of "get_template" over "load_template" is that it calls
"get_template_sources" directly. This means the skip and debug information 
is
readily available in a single place.

If "skip" is added to load_template instead, it also needs to be added to
"load_template_source". That means the skip and debug logic needs to be
duplicated in each of the loaders, including third-party ones. This is 
because
"load_template_source" is where the looping through template sources 
currently
exists.

Second, the cached loader reimplements most of load_template with additional
logic. If "get_template_sources" can be counted on, this cleans it up quite 
a
bit.

Third, returning (template, origin) is redundant if origin is a property of
template. If we change load_template, that ends up sticking around, unless
we later introduce another deprecation for it.

With that said, it's possible to refactor "load_template" to stop using
"load_template_source" and instead call "get_template_sources". If it does
this I can address 1 and 2. I'm not sure if we can do 3. We'd still need to
do the following:

* Add a deprecation period because get_template_sources is required to 
support
  template recursion.
* If get_template_sources is used, there still needs to be a get_internal
  method of some sort. I could try to reuse load_template_source, but it's
  a pretty big change in behavior of what that function currently does. I'm
  not sure this lends itself to a nice upgrade path.

Therefore, I see the deprecation period for 3rd-party loaders being one of:

1. Add get_template_sources and get_internal
2. Add get_template_sources and modify load_template_source

Do you still prefer refactoring load_template instead? If so, should I add
get_internal or try to modify load_template_source?

Your design makes sense. My main suggestion is to fit it in the current 
> APIs.
> I would try to reuse load_template instead of introducing get_template 
> and to
> hook to a revamped Origin API. The Origin API was documented in Django
> 1.7; changing it it less disruptive than changing the Loader API. Did 
> you try
> this, and if you did, why didn’t it work?


Okay. I think this is how the api would look then:

get_template_sources(template_name) -> yields origin(s)
get_internal(origin) -> template_string
get_template(template_name) -> template
--or--
load_template(template_name) -> (template, template_origin)

This API is still tied to the Django Template Language. I would prefer 
> a generic
> list of (“template identifier”, “loading mechanism”, “reason for 
> failure”) that every
> backend could implement.


Yes, this is an easy change. I'll add it into my branch.

Furthermore, I’m not sure what this list represents. Is it a “template 
> call stack”
> i.e. the chain of templates included up to the point where loading a 
> template
> failed? Or is it a list of ways the template backend tried to load the 
> failing
> template before giving up?


Sorry. I didn't explain that example clearly, and 

Re: Formalizing template loader and debug api's

[Preston Timmons]

Hi Unai. Yes, your previous work was helpful for me in developing this 
solution. Some of Aymeric's recent refactorings have simplified things 
since you last worked on it as well.

Hi Riccardo. Thanks for the suggestion. I don't mind changing the name of 
get_internal if others agree. That's an easy fix.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0c4a6459-3312-4a35-bfa8-dad24c4ebf8f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Migrations in Django 1.7 make unit testing models harder

[Preston Timmons]

At work we use a --no-migrate flag on our test runner. It's helpful when 1) 
we're developing a new app but the models aren't nailed down yet, and 2) it 
speeds up the test suite time significantly.

The --keepdb option is sufficient for scenario 2, but doesn't do much for 
scenario 1. Admittedly though, scenario 1 isn't as common an occurrence.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/63e2abd2-23a2-447c-a0e7-0efde0a689ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Formalizing template loader and debug api's

[Preston Timmons]

Hi all,

I've been working on #15053, which enables Django template loaders to extend
templates of the same name recursively. The most common use-case for this is
extending the admin templates without needing to copy the application 
templates
into a project.

To illustrate the new algorithm, here are some examples:

(-> indicates the next file that is extended)
(fs indicates filesystem template folders)

1. Extend admin template from filesystem directory

fs/admin/base.html # extends admin/base.html
-> django/admin/base.html

2. Extend in multiple filesystem directories

fs/base.html # extends base.html
-> fs2/base.html # extends base.html
-> fs3/base.html

3. Extend using both a filesystem and app loader with multiple directories

fs/base.html # extends base.html
-> fs2/base.html # extends base.html
-> app1/base.html # extends base.html
-> app2/base.html # extends base2.html
-> fs/base2.html # extends base2.html
-> fs2/base2.html

I found it difficult to support this algorithm with the existing template
loaders, and as you can see from example 3, the extends algorithm isn't
necessarily linear with respect to template loaders anymore.

Therefore, the patch I submitted changes three main areas. The first two are
mostly separate from Aymeric's recent changes, but the third is not.

1. Added a new loader api and made it consistent across all loaders.
2. Modified the extends node to track extended templates and passes
them as a skip argument to the loaders.
3. Updated the debug view template postmortem api to work with the new
algorithm, as well as added support for the egg and cached loader.

Below is an explanation of what I changed in 1 and 2 and a proposal for
3 that hopefully works with multiple template engines.

## 1. Add new template loader apis

I tried to solve this patch without changing the template loader apis, but I
eventually decided this was inevitable for two reasons:

1. The loaders don't implement a consistent api. For instance, the
filesystem and app loaders define a get_template_source method that is
used elsewhere for debug information, whereas the egg and cached loaders do
not. The cached loader implements load_template but not 
load_template_source.

2. The loaders need to be able to skip templates that have already been
extended, but adding a new argument to load_template is difficult to do in a
backwards-compatible way.

This led me to the following loader api:

### Loader API

Loader.get_template_sources

A method that yields all paths the loader will look at for a given template
name. For example, if the filesystem loader receives "index.html" as an
argument, this yields the full path of "index.html" for each directory on 
the
search path. The cached loader yields tuples of (loader_class, 
template_paths)
for each configured loader.
This method already exists for the filesystem and app loaders. This patch
implements it for all loaders.

Loader.get_internal

Returns the contents for a template given a ``source`` as returned by
``get_template_sources``.

This is where a filesystem loader would read contents from the filesystem,
or a database loader would read from the database. If a matching template
doesn't exist this should raise a ``TemplateDoesNotExist`` error.

BaseLoader.get_template

Returns a ``Template`` object for a given ``template_name`` by looping
through results from ``get_template_sources`` and calling ``get_internal``.
This returns the first matching template. If no template is found,
``TemplateSyntaxError`` is raised.

This method takes an optional ``skip`` argument. ``skip`` is a set 
containing template sources. This is used when extending templates to
allow enable extending other templates of the same name. It's also used
to avoid recursion errors.

In general, it will be enough to define ``get_template_sources`` and
``get_internal`` for custom template loaders. ``get_template`` will
usually not need to be overridden.

This loader api enables the following changes:

## 2. Update the extends tag

The new extends tag keeps track of which sources were tried in the local
context. These sources are passed to the loader ``get_template`` method as
the ``skip`` argument. In doing so, the extends tag never extends the same 
source file twice. This enables recursive extending, and also avoids 
filesystem recursion errors when extending is truly circular.

The main caveat here is that I changed Template.origin to always be 
available
on the template--not just when TEMPLATE_DEBUG is true. Without this, the
extends tag has no way to know the source path of the current template.

I think this change is okay, but I don't know the original reasons for
excluding this in the first place. It could be simply because there was no
use-case presented outside of debug.

## 3. Debug api

Django displays a template postmortem message when a matching template isn't
found. This is a linear list grouped by template loader. This api is 
currently
supported only by the filesystem and 

Re: Multiple template engines for Django - week 12

[Preston Timmons]

I agree about the select_template() change as well.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/286ede07-d0e7-447f-9da9-4b3e9e49089e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Multiple template engines for Django - week 4 - DEP ready for review!

[Preston Timmons]

Aymeric,

Great work on this. I have a few more questions:

First, if multiple engines are configured, how do you see the error being 
displayed if a template isn't found in any of them? Currently, this 
originates from the template loaders. For instance, the filesystem loader 
raises a TemplateDoesNotExist with a list of templates it tried.

Second, if I want to get a template from within a script, do I have to 
manually loop through the engines? Or is there a higher-level get_template 
that encapsulates this?

Third, if I understand right, extending templates would only happen within 
the template engine that found the template. That means if multiple Django 
template engines were specified, the extends tag would not extend templates 
from other Django engines, even when specified. Is this correct?

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f14a854b-3f3a-41e8-9ccd-810378682cc0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Multiple template engines for Django - week 1

[Preston Timmons]

Hi Aymeric,

With the new template update, do you foresee 
django.utils.setup_test_template_loader and 
django.utils.restore_template_loaders still working?

Those aren't officially public api's, but they're useful nonetheless. 
Perhaps the way to do it in the future would be with override_settings and 
a dict loader?

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/837097fd-7d90-486d-a85f-7161232766fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: unittest2 discovery proposal

[Preston Timmons]

Hi Shai,

The discover runner does discovery based on pattern.

So, if your tests are named, test_*.py, they would by be discovered by
default. Test discovery is recursive, under the root, so it doesn't
matter if you have tests in a tests directory.

The __init__.py imports would be redundant, and actually ignored by
the discover process.

If your test files don't match the default pattern of "test*.py", then
you have three options:

1) Rename the test files
2) Invoke the test command with a custom pattern
3) Specify a custom load_tests in the __init__.py file that tells
discovery what to do.

Preston


On Apr 1, 2:28 pm, Shai Berger  wrote:
> Hi,
>
> +1 in general. One concern, and one idea:
>
> > -- Backwards-incompatible changes --
>
> > * Some valid test structures in Django don't work with unittest2. For
> >   instance, tests in `tests/__init__.py` don't match a patter than
> >   unittest would recognize if running discovery on a module.
>
> I have a complex app with many tests; so many, that we felt the need to break
> the tests module into modules in a package. So we have
>         tests/test_frobnication.py
>         tests/test_grobulation.py
> etc., and
>         tests/__init__.py
> which imports them all.
>
> It isn't quite clear to me how this applies to such a structure; it seems like
> I would need a special pattern, "tests.test_*.py" or some such. This, in turn,
> raises an idea: an app should be able to specify the discovery parameters for
> its own tests in the source, not just on the command line.
>
> Just to be clear: In view of backwards-compatibility concerns, I don't think
> the load_tests protocol supported by unittest2 is enough for this. It should
> be much simpler, IMO; something like (pseudo code):
>
>         try:
>                 from app import tests
>                 if hasattr(tests, 'root'):
>                         set_discovery_root(tests.root)
>                 if hasattr(tests, 'pattern'):
>                         set_discovery_pattern(tests.pattern)
>         except ImportError:
>                 pass
>
> This way, my case is handled simply by adding these lines into
> tests/__init__.py:
>
> root=app.tests
>
> Thanks,
>         Shai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

unittest2 discovery proposal

[Preston Timmons]

Hi all,

There are some important design decisions to be made if we include
unittest2 discovery into Django.

This message documents a proposed API, the pros and cons, and
decisions that need to be made.

https://code.djangoproject.com/ticket/17365


-- Limitation of current test setup: --

* Tests can only be in tests.py.
* Tests must be included inside the app.
* Full dotted paths to test cases don't work.


-- How unittest2 discovery works: --

Unittest discovery has two advantages:

1) Better discovery for the existing case of testing an app

Besides the old model and doc tests, current test discovery is limited
to
tests in tests.py.

Unittest2 discovery allows files to be matched by pattern. For
instance,
these are all valid test modules:

tests.py
test_views.py
test_models.py

Further, a custom pattern can be set if the test files don't match the
naming convention "test*.py".


2) Custom root folders can be specified for test discovery

If the tests aren't in the app directory--a common case for reusable
apps--
a custom root folder can be specified for discovery.

./manage.py test --root=../tests


3) Can use Python dotted paths to modules.

These would all work:

./manage.py test myapp
./manage.py test myapp.tests
./manage.py test myapp.test_views
./manage.py test myapp.tests.MyTest
./manage.py test myapp.tests.MyTest.test_method

** The current "myapp." notation is not part of unittest2,
and
would go away.


-- The Proposed API --

The unittest2 discovery is roughly-compatible with the existing custom
Django discovery. Here's what the API could look like:

# Discover tests under current directory
./manage.py test

# Test with discovery under different root
./manage.py test root="../tests

# Test with a custom top level directory
./manage.py test --root="../tests --top_level=../../top

# Test with different pattern matching
./manage.py test --pattern="tests_*"

# Test single installed app with discovery
./manage.py test contact

# Test multiple apps with discovery
./manage.py test myapp1 myapp2 myapp3

# Test specific test module
./manage.py test myapp1.tests
./manage.py test myapp1.tests_views
./manage.py test myapp1.test_models

# Test with mixed arguments
./manage.py test myapp.test_views myapp.test_models

# Test single testcase
./manage.py test contact.tests.ContactTest

# Test single method
./manage.py test contact.tests.ContactTest.test_contact

# Test all installed apps with discovery
./manage.py test --installed


-- Notes on above API: --

* Running `./manage.py test myapp` does discovery within the app and
runs the tests. That means test in existing `tests.py` will be found,
as well as in any file name "test*.py"

* Root, pattern, and top-level options are passed from the `test`
command
to the unittest discovery.

* Running a single test case uses a full dotted path, rather than the
short
path that Django currently uses (myapp.tests.MyTest vs myapp.MyTest)

* The old behavior of running all installed app tests can be triggered
by
using the `--installed` flag.


-- Opting into the new discovery --

To allow a deprecation cycle, the new discovery is an opt-in.

To use the discover runner, a user must update their settings.py:

TEST_RUNNER = 'django.test.runner.DiscoverRunner'

Until Django 1.8, the 'django.test.simple.DjangoTestSuiteRunner' is
still
available, including it's ability to run doctest, model tests, etc.


-- Backwards-incompatible changes --

* The new runner doesn't run doctests or tests in models.py

* Some valid test structures in Django don't work with unittest2. For
  instance, tests in `tests/__init__.py` don't match a patter than
  unittest would recognize if running discovery on a module.


Design Decisions


1) Any changes to the API?

I think the above API covers all the common use-cases.

Is anything missing?


2) Should this be opt-in or not?

The current patches are set up as opt-in. Until Django 1.8, a user has
to manually update their test runner setting to use the
DiscoverRunner. This lets current setups continue working, doctests
and model tests included.

Is there a reason not to follow the normal deprecation cycle?


3) What do to about doctests?

There is discussion to roll doctest removal into this patch.

https://code.djangoproject.com/ticket/18727

On one hand that's okay, but removing doctests has no bearing on using
unitest2 discovery. It's simply a backwards-compatibly break to the
old `django.test.simple.DjangoTestSuiteRunner`.

Do we really want to do a hard cut-off for that? Should it really be
part of the new discovery branch?

If we remove doctests, why not remove support for discovery in
models.py at the same time?


4) Option names for the `test` command.

What should the discovery option names be?

Above, I proposed this:

./manage.py test --root=../tests --top-level=../../top --

Reason behind hiding the template origin

[Preston Timmons]

Hi,

I have a question for somebody more experienced with the Django
template system internals.

The template loader load_template method is documented as such:

"The load_template() method of the Loader class retrieves the template
string by calling load_template_source(), instantiates a Template from
the template source, and returns a tuple: (template,
template_origin)."

The BaseTemplateLoader differs from this behavior. It looks like it
was purposefully written to hide the template origin. Instead it
returns None, unless TEMPLATE_DEBUG is true, then it return the name
of the template.

The CachedTemplateLoader differs slightly from this, returning the
origin object, rather than the template name, but again only when
TEMPLATE_DEBUG is True.

Is there a reason the template origin isn't always returned?

Knowing the template origin is nice in certain cases, and removing the
special logic would make these two tickets easy to implement:

https://code.djangoproject.com/ticket/16096
https://code.djangoproject.com/ticket/17199

Thanks.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Form Rendering API Proposal

[Preston Timmons]

This looks excellent so far.

Do {% formfield %} and {% formrow %} accept context like {% form %}
does?

Is there a way with {% formfield %} or {% formrow %} to set custom
attributes like placeholder, autocorrect, etc.? I find this common
requirement when optimizing forms for mobile devices.

Thanks,

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Templatetag API for form rendering

[Preston Timmons]

This looks interesting so far.

How does setting the form layout affect rendering individual fields in
a form?

Is the output of {% renderform my_form "first_name" %} equivalent to
{{ form.first_name }}, or is it more like the output of "as_p" or
"as_table"? Which templates are involved in a form layout?

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Direction on #10899

[Preston Timmons]

Hi,
Could one of the core devs take a look over #10899 and give some
guidance on what the best step to take next is? Thanks.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: New context template tag

[Preston Timmons]

I'm a fan of this proposal. I find that most of the tags I build
return a variable to the context rather than just rendering a string.
And while a few of the tags I have built require more complexity than
the decorator approach could handle, the majority of them would be
trivial to make if this were in place.

The proposal because it covers a common use case and it fits in with
the pattern that Django already employs for tags rather than trying to
introduce something new. It's also not trying to over-generalize the
problem but make the simple things simple. Creating a tag that takes
one or two arguments, processes them, and returns a value to the
context feels like it should be simple but right now it's not.

For illustration, I'll word the proposal in terms of how it would read
in the Django documentation:

SHORTCUT FOR SIMPLE TAGS
Many template tags take a number of arguments -- strings or a template
variables -- and return a string after doing some processing based
solely on the input argument and some external information.

CONTEXT TAGS
Many template tags take a number of arguments -- strings or a template
variables -- and return a variable to the context after doing some
processing based solely on the input argument and some external
information.

This feels to me like a natural enhancement to the Django template
library.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Revised form rendering

[Preston Timmons]

Hey Russ,

I think this is a great proposal so far!

Is there a way with the proposed solution for the template designer to
add custom attributes to a form field? If so, do you envision that
happening in the chrome layer?

Here's a use case:

The designer wants to render an email input field. They also want to
set the ``autocapitalize`` attribute to "off" so Mobile Safari doesn't
capitalize the first letter of the email while it's being typed. In
addition, they want to set the placeholder attribute.


Internally, we solve this problem through a custom template tag doing
something like this:

Form Template::

{% load form_tags %}

{% with "forms/text_field.html" as text_field_template %}
  {% text_field form.email placeholder="Email"
autocapitalize="off" autocorrect="off" %}
{% endwith %}

Field Template::


  {{ field }}



We use the template tag because calling
``form_field.field.widget.attrs.update(attrs)`` is the only way I know
to get the {{ field }} object to render custom attributes on the input
field from the template. Other use cases involve overriding the label
defined in the form definition and adding custom classes to the input
field.

Thanks,

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.2 release candidate available

[Preston Timmons]

Congratulations! Thanks for all your hard work on this. There's not
another framework I'd rather use.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: admin javacripts

[Preston Timmons]

Hi Rob,

On Feb 19, 11:58 am, Rob Hudson  wrote:
> On Fri, Feb 19, 2010 at 8:58 AM, Luke Plant  wrote:
> 1) Do we put jQuery in base.html and have Django's widgets and plugins
> assume it will be there?
>
> * Benefits: Javascript admin customizations are simpler and can use
> the jQuery already on the page.
> * Trade off: If someone wants to override base.html and put in
> mootools, e.g., Django's widgets break.

You can do this without breaking other javascript libraries or jQuery
versions by scoping Django's version of jQuery immediately after it is
included in base.html. An an example to illustrate:

If this is in the head:



  var $jQD = jQuery.noConflict();


And somebody else includes this:



Then Django's version of jQuery would be available to all widgets as
$jQD and the other jQuery version would still be available as $ or
jQuery. This would work whether the other jQuery include or any other
javascript library was included before or after the Django include.
The only possible problem is if someone's custom javascript happens to
use $jQD as a global variable. Using a unique enough variable name
would mitigate potential conflicts.


> 2) Or do we put jQuery in each widget's media setting and encapsulate
> both jQuery and the plugin code in a closure?  Which seems like where
> Django has been heading by default and the one I would advocate.
>
> * Benefit: Users can override base.html and add their own JS library
> of choice, even if it's jQuery for easy development or things outside
> of widgets.
> * Trade off: A bit trickier to develop admin widgets.  But perhaps
> this could be worked on -- something like jQuery UI's widget factory
> methods?

I agree that this makes a nice sandbox for the widgets. A second
advantage is that if a new widget was created it could use a new
version of jQuery, or even a different javascript library, without
interfering with anything else that exists. Whereas, if a global
Django admin jQuery is ever updated then the person who does it has to
test that all of the widgets depending upon it are still working.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Dallas 1.1 sprint - dates?

[Preston Timmons]

Yes, either weekend works.

On Apr 3, 2:54 pm, Nizam Sayeed <ni...@nomadjourney.com> wrote:
> Yeah. We're open for either weekend.
>
> On Apr 3, 1:22 pm, Brett Hoerner <bretthoer...@gmail.com> wrote:
>
>
>
> > On Fri, Apr 3, 2009 at 12:38 PM, Nizam Sayeed <ni...@nomadjourney.com> 
> > wrote:
> > > Count me in and a friend of mine as well.
>
> > > On Apr 3, 12:33 pm, Preston Timmons <prestontimm...@gmail.com> wrote:
> > >> I would be interested in coming as well.
>
> > I assume you guys are OK with either weekend, then?
>
> > Brett
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Dallas 1.1 sprint - dates?

[Preston Timmons]

I would be interested in coming as well.



On Apr 3, 9:50 am, Jeremy Dunck  wrote:
> Gary, Justin?
>
> On Fri, Apr 3, 2009 at 9:47 AM, Alex Robbins
>
>
>
>  wrote:
>
> > I live in the Dallas area and would be interested in coming, whenever
> > it happens.
>
> > On Apr 2, 12:45 pm, Jeremy Dunck  wrote:
> >> Hey all, I was considering putting on a Dallas sprint for 1.1.  I'm
> >> not sure exactly when 1.1 will ship, but soon-ish, so I was thinking
> >> about trying to make the sprint the weekend of 4/10 (Easter weeked) or
> >> 4/17.
>
> >> Any preference?  Who can make it or will consider making it?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Patch status for ticket #9122

[Preston Timmons]

Thanks, Brian. I appreciate you putting time into that.

The existing documentation explains the generic inline classes as
behaving the same as the normal inlines.
http://docs.djangoproject.com/en/dev/ref/contrib/admin/#using-generic-relations-as-an-inline

Right above that is documented the normal inline options.
http://docs.djangoproject.com/en/dev/ref/contrib/admin/#inlinemodeladmin-options

I thought I would mention that because it seems like a feature is
already documented as working.

Preston




On Mar 19, 8:54 pm, Brian Rosner <bros...@gmail.com> wrote:
> On Mar 19, 2009, at 5:47 PM, Preston Timmons wrote:
>
> > Might somebody be able to review the patch and tests for this ticket
> > to see if they are acceptable? I am hoping it can get in as a bug fix
> > for 1.1. If something is lacking here I would like to try to fix it.
>
> The patch looks generally acceptable. I'd like to see some  
> documentation on it. I will definitely review this in time for 1.1.  
> Thanks for the heads up.
>
> Brian Rosnerhttp://oebfare.com
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Patch status for ticket #9122

[Preston Timmons]

Ticket #9122 Inline admin on generic relations ignores exclude and
max_num
http://code.djangoproject.com/ticket/9122

Might somebody be able to review the patch and tests for this ticket
to see if they are acceptable? I am hoping it can get in as a bug fix
for 1.1. If something is lacking here I would like to try to fix it.
Thanks.

Preston

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---