Re: [GSoC] Revised form rendering

[Gregor Müllegger]

I finally submitted my proposal to the google melange homepage. The core of
the proposal is very much unchanged to the first draft, however I've included
passages that indicate that the syntax might change in further discussions
about this topic and that the concept of a chrome will be propably dropped in
favour of more flexible template based widgets -- I didn't wanted to
include a new detailed view on this topic without knowing how the final
implementation for template based widgets will look like.


For all that are interested, here are some links with more information about
my submission:

1. The official submission on the google-melange.com homepage:

http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/gregmuellegger/1

2. The latest revision (and its change history) of the proposal on github
(maybe also nicer formated):

https://gist.github.com/898375


I will get back to the list kicking of some more discussions about the syntax
next week and doing some syntax "testing" like we previously discussed in this
thread.

Gregor

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC] Revised form rendering

[Gregor Müllegger]

Sorry for the quite late response, everything is getting pretty close to the
deadline. I'm not happy with it but I need to tackle some personal issues at
once.

2011/4/5 Russell Keith-Magee :
>
> ...
>
> In particular, I have two objections:
>  * it feels like it's trying to cram too much into a single template tag.
>  * it requires a lot of duplication of "form myform configure" or
> "formblock myform using"
>
> Consider a slight variation that uses more tags:
>
> {% form myform configure %}
>    {% layout "uni_form" %}
>    {% widget "calendar" for DateField %}
>    {% autocomplete "lastname" from "/users/lastnames" %}
> {% endform %}

Yes that looks clean and easy to follow. I just don't like the lone keyword
'configure' (or 'using') at the end of the tag. But I guess we need it to
determine if we search for a closing endform tag or if it's a standalone one
(or we use the 'trick' you described for the media handling).

> Then, when it comes time to render:
>
> {% form myform using %}
>    {% field "firstname" %}
>    And some HTML decoration
>    {% field "lastname" %}
> {% endform %}

Looks good. A problem would be with using multiple forms:

{% form myform using %}
{% form another_form using %}
{% field "name" %}
{% endform %}
{% endform %}

Is "name" a field of "myform" or "another_form"? But thats easy to work around
by using {% field myform.name %} for example.

> or, using an iterative approach:
>
> {% form myform using %}
>    {% for field in myform %}
>        {% field field %}
>    {% endfor %}
> {% endform %}

I wonder how it would look like to change the layout and outputting the whole
form like {{ form.as_ul }}. Your example is for more than basic scenarios good
but very verbose for just changing the layout. Maybe using the form tag would
inherit the configuration from the wrapping form tag::

{% form myform using %}
{% layout "ul" %}
{% form myform %}
{% endform %}

Skipping the wrapping {% form myform using %} is also possible, layout would
configure the whole template by setting a context variable. However this might
have sideeffects on included templates etc. Explicit is usually better than
implicit. But we will find a way to deal with it.

> or, using default layout:
>
> {% form myform %}
>
> This approach:
>  * cuts down on the duplication, either in the form of indented form
> blocks, or in duplicated 'tag opening' content
>  * avoids the need for a DSL embedded in the tag definition
>  * avoids potential grammar ambiguities -- the less grammar there is
> in a tag, the less likely there will be edge cases in the parsing.
>
> ...
>
>>> Rendering modifiers
>>> 
>>>
>> But like I already wrote above: I see your concerns with adding yet another
>> registry that might not be used very often. I would agree on dropping that
>> feature if you think its better to skip this until users demand it 
>> afterwards.
>
> I think this might be the better approach. Your proposal is already
> fairly ambitious, especially if it's going to include a reworking of
> admin's rendering. And, to that end, reworking admin's rendering
> should prove a useful testbed for whether it is needed at all.
>
> Also -- some food for thought; the reason that a registry is needed
> here is because you're defining an extension api for a monolithic {%
> form %} tag. If the rendering strategy used multiple tags, modifiers
> could also be defined as template tags:
>
> {% form myform %}
>    {% mymodifier foo %}
> {% endform %}
>
> Then you don't need a separate registry; a "modifier" just becomes a
> template tag operating on the API exposed by the form rendering API
> (presumably a set of known objects in the local context of the form
> tag). If an end user wants a rendering modifier, they just define
> their own template tag and use it in a {% form configure %} block.

Ha! That makes me happy to have a possibility for custom behaviour in the form
rendering and you happy because we don't need another abstraction. I like this
approach and will go for it.

>>> Form rows
>>> ~
>>>
>>> If you look at the existing as_* form layout tools, there is a
>>> distinction between gross form layout, and the layout of an individual
>>> form row.
>>>
>>> {{ form_header }}
>>> {% for field in form %}
>>>    {{ pre_field }}}
>>>    {{ field }}
>>>    {{ post_field }}
>>> {% endfor %}
>>> {{ form_footer }}
>>>
>>> Conceptually, the rendering for pre/post field could vary depending on
>>> the field. How is this handled in your proposal? Is it part of the
>>> form renderer? Or part of the widget rendering? This overlaps with
>>> your discussion about error templates -- since errors (and their
>>> rendering) is one of the major pre/post rendering activities for both
>>> forms and fields.
>>
>> I'm not sure if I get your example right. Is it representing the output of a
>> "as_*" method? I looked into the django/forms/forms.py code again and I'm
>> 

Re: [GSoC] Revised form rendering

[Russell Keith-Magee]

On Sun, Apr 3, 2011 at 11:46 PM, Gregor Müllegger  wrote:
> Hi,
>
> 2011/4/2 Russell Keith-Magee :
>> On Fri, Apr 1, 2011 at 11:57 PM, Gregor Müllegger  
>> wrote:
>> Firstly, while it looks fine for a small example, I can see how it
>> would rapidly deteriorate if you have lots of fields, or lots of
>> custom field requirements for each field. Django's template language
>> doesn't allow you to split tags over multiple lines, so what happens
>> when my tag runs over 80 characters (this simple example is already a
>> 69 characters)?
>
> (Having the possibility of multiline tags would be nice in many other cases as
> well ... but that's another topic :-)

I'm not convinced it would be nice. Template tags are intended to be
short and pithy, not space in which you can define your own
domain-specific language. My feeling is that if a template tag
proposal needs to invoke the "if only tags could span multiple lines"
clause, the tag is already doing too much in it's definition.

The one exception I will grant to this is comments -- requiring {%
comment %} {% endcomment %}, or {# #} on every line is just daft,
IMHO. However, it's a daftness that hasn't progressed into enough of
an itch to drive me to scratch it myself. It's also unrelated to your
proposal :-)

> I don't see the use of more than two or three modifiers in a template tag in a
> day to day use.

Well.. I gave you an example -- a form with 10 fields, and I want to
render 8 of them.

> However it's of course possible to produce complex statements
> that would need to span multiple lines. And it's already possible with the
> syntax described in the proposal.
>
> You can use a {% formblock %} to extract the modifiers out of the form tag:
>
>    {% formblock using layout "uni_form" %}
>    {% formblock using widget template "calendar" for myform.birthday %}
>        {% form myform using fields "firstname" "lastname" "birthday" %}
>    {% endformblock %}
>    {% endformblock %}
>
> Some might not like the syntax since you would encourage another level of
> indention and another two lines for the closing tags. But you can use the
> "configure" statement as described in the media handling section of my
> proposal:
>
>    {% form myform configure layout "uni_form" %}
>    {% form myform configure widget template "calendar" for myform.birthday %}
>    {% form myform using fields "firstname" "lastname" "birthday" %}

I hadn't quite grasped how you were using formblock in your previous
example. I now see what you're trying to do, and it makes a lot more
sense. However, I'm not completely sold on certain aspects of this
syntax.

In particular, I have two objections:
 * it feels like it's trying to cram too much into a single template tag.
 * it requires a lot of duplication of "form myform configure" or
"formblock myform using"

Consider a slight variation that uses more tags:

{% form myform configure %}
{% layout "uni_form" %}
{% widget "calendar" for DateField %}
{% autocomplete "lastname" from "/users/lastnames" %}
{% endform %}

Then, when it comes time to render:

{% form myform using %}
{% field "firstname" %}
And some HTML decoration
{% field "lastname" %}
{% endform %}

or, using an iterative approach:

{% form myform using %}
{% for field in myform %}
{% field field %}
{% endfor %}
{% endform %}

or, using default layout:

{% form myform %}

This approach:
 * cuts down on the duplication, either in the form of indented form
blocks, or in duplicated 'tag opening' content
 * avoids the need for a DSL embedded in the tag definition
 * avoids potential grammar ambiguities -- the less grammar there is
in a tag, the less likely there will be edge cases in the parsing.

>> Secondly, you appear to be using a single template tag for rendering
>> both forms *and* fields. Is there a motivation behind this?
>
> Yes there is :-) The motiviation is, that {% form myform.birthday %} is just a
> shorthand for {% form myform using fields "birthday" %}. You can also use it
> to iterate over a form (like in your example below) and outputting it field by
> field:
>
>    {% for field in myform %}
>        {% form field %}
>    {% endfor %}
>
> I also don't see a need for an extra {% formfield %} tag. I don't see any
> cases in which a {% formfield %} would be shorter, more flexible, easier to
> use. It might be more readable, but I think it would end up as a stripped down
> copy/alias of the form tag.

See my previous comments about cramming everything into a single tag.
Also, there's an implementation issue here; if the first argument to
form can be a field or a form, then your implementation will need to
have logic to differentiate the two -- either isinstance checks, or
duck-type checks. Either way, it's a problem that is avoided by being
explicit: {% form %} operates on forms, {% field %} operates on
fields.

>> Rendering modifiers
>> 
>>
> But 

Re: [GSoC] Revised form rendering

[Gregor Müllegger]

2011/4/4 Daniel Greenfeld :
> Anyway, as the current lead on Django Uni-Form I think its great that Gregor
> is picking up the torch for refactoring form rendering in Django 1.40. He's
> obviously done his homework and put a lot of thought into this critical part
> of Django.
> I'm not a core developer my vote doesn't count, but I'm giving it anyway. +1
> from me.
> Daniel Greenfeld

Hi Daniel, I really appreciate your support. It shows that I might be on the
right path for a successfull refactoring. Thanks.

@all:

I've updated my proposal. Mainly the media tag part and the timeline estimates
(nothing really exciting for the moment). A revision of the chrome section
will follow tomorrow. Have a look at the proposal online [1] or look below,
I've included the changed bits:



GSoC 2011 Proposal - Revised form rendering


...

Media aka. JS/CSS
-

One of the other mainpoints in the discussions I reviewed for this proposal was
the use of JS and CSS files that must be maintained somehow to display them
how we already do through the media framework (e.g. ``{{ form.media }}``).

The problem with this is that with the new template tag we can change some
of the widgets in the template and introducing new dependencies. Thats why I
would like to have an alternative for the ``using`` argument in the
``{% form %}`` tag.

If ``using`` is replaced with ``configure``, the ``{% form %}`` tag will _not_
output the HTML in the current place. However it will record and remember the
usage of widgets and fields to determine which media files are required. An
example template would look like::

{% block extrahead %}
{% form myform configure widget "CalendarInput" for myform.birthday %}
   ^--- The new widget for the field birthday will be
recorded, but the form will not be rendered.
{{ myform.media.css }}
   ^--- Outputting all necessary css files.
{% endblock %}

{% block content %}
{% form myform %}
   ^--- The form will be rendered as usual but with the
"CalendarInput" widget that was specified in the other tag.
{% endblock %}

{% block footer %}
{{ myform.media.js }}
   ^--- Outputting all necessary js files at the end of the document.
{% endblock %}

I will also check the possibility and difficulty of a new ``{% formmedia %}``
tag that hooks into the template parsing system, reading until the end of the
template and analyzing the use of the ``{% form %}`` tag. This way it could
determine all changes that will be applied to the form before it gets
rendered, including all the necessary CSS dependencies that needs to be
imported in the header of the page.

It is not clarified yet, if the ``{% formmedia %}`` is possible at all with
the current template parsing implementation. There might be some risks that
need to be sorted out before starting with the implementation:

* By parsing from the ``{% formmedia %}`` tag until the end of the template
  might result in that all content after this tag is represented as a child
  node of it. What side effects are implied? Does it produce backwards
  incompatibilities with thirdparty template tags?

* What happens if the ``{% form %}`` tag is changing the widget of the form
  based on a context variable?

Estimates
-

...

1st week: Starting to layout the documentation. The form tag syntax based on
discussions from the mailing list should already be finalized.

2nd week: Examing what unittests are available for the current form rendering
and making sure they are stable for testing backwards compatibility during the
project.

3rd week: I will attend DjangoCon EU, hopefully giving a talk about the
revised form rendering and collecting more feedback in an open space.

4th week: Converting the current layouts into template based renderers, ensuring
backwards compatibility.

Goal: no HTML should be left now in the python source.

5th week: Mainly working on documentation and integrating the very last changes
based on discussions from DjangoCon EU.

Goal: All public APIs should be stable.

6th week: Starting to write tests and implementing the ``{% form %}`` tag to
be able to emulate all the rendering that is currently possible.

7th week: Implementing the necessary rendering modifiers like "fields",
"layout" etc. and the API for chrome.

8th week: Implementing the ``{% formmedia %}`` tag.

Goal: Project should be feature complete.

9th - 11th week:

* Validating backwards compatibility for the ``{% formmedia %}`` parsing
  implementation with thirdparty modules (see Media section).
* Converting the admin to use the new form rendering.
* Integrating lessons learned from the admin.
* Bugfixes and regression tests for problems that showed up in the work with
  the admin.

Goal: Code should be ready to be used in sample projects.

12th week: Finalizing, bugfixes and 

Re: [GSoC] Revised form rendering

[Daniel Greenfeld]

First off, for some arcane reason Google is forcing formatting on me. 
Hopefully that doesn't make this email that ugly.

Anyway, as the current lead on Django Uni-Form I think its great that Gregor 
is picking up the torch for refactoring form rendering in Django 1.40. He's 
obviously done his homework and put a lot of thought into this critical part 
of Django. 

I'm not a core developer my vote doesn't count, but I'm giving it anyway. +1 
from me.

Daniel Greenfeld
pyda...@gmail.com


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC] Revised form rendering

[Chris Beaven]

Haven't got time for a full review of this proposal right now, but I've done 
a lot of thinking about this area already and am more than happy for anyone 
to steal my ideas and compare thoughts.

Shoot me an email off-list or catch me on #django (as SmileyChris) if you 
want to discuss more personally.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC] Revised form rendering

[Gregor Müllegger]

Hi,

2011/4/2 Russell Keith-Magee :
> On Fri, Apr 1, 2011 at 11:57 PM, Gregor Müllegger  
> wrote:
>> I suggest reading this proposal online: https://gist.github.com/898375
>> It's exactly the same as below but formated nicely.
>>
>>
>> GSoC 2011 Proposal - Revised form rendering
>> 
>>
>> Hi my name is Gregor Müllegger. I'm a Computer Science student in Germany at
>> the University of Augsburg currently in the fourth year of my studies. I 
>> first
>> came to django shortly before 0.96 was released and a lots of awesomeness was
>> introduced with the magic removal branch.
>
> Hi Gregor,
>
> Firstly, I'd like to echo Carl's sentiments -- this is a strong
> proposal that shows you've researched the history of Django
> discussions on this topic, and given it some thought of your own.
>
> Carl has already raised several of the points that I noticed on my
> read through. However, I do have some of my own queries.
>
> Syntax
> ~~
>
> You've proposed a syntax that goes something like:
>
> {% form myform using layout "p" and fields "firstname" "lastname" %}
> {% form fancyform.favourite_color using layout "p" %}
>
> I have two questions about this syntax.
>
> Firstly, while it looks fine for a small example, I can see how it
> would rapidly deteriorate if you have lots of fields, or lots of
> custom field requirements for each field. Django's template language
> doesn't allow you to split tags over multiple lines, so what happens
> when my tag runs over 80 characters (this simple example is already a
> 69 characters)?

(Having the possibility of multiline tags would be nice in many other cases as
well ... but that's another topic :-)

I don't see the use of more than two or three modifiers in a template tag in a
day to day use. However it's ofcourse possible to produce complex statements
that would need to span multiple lines. And it's already possible with the
syntax described in the proposal.

You can use a {% formblock %} to extract the modifiers out of the form tag:

{% formblock using layout "uni_form" %}
{% formblock using widget template "calendar" for myform.birthday %}
{% form myform using fields "firstname" "lastname" "birthday" %}
{% endformblock %}
{% endformblock %}

Some might not like the syntax since you would encourage another level of
indention and another two lines for the closing tags. But you can use the
"configure" statement as described in the media handling section of my
proposal:

{% form myform configure layout "uni_form" %}
{% form myform configure widget template "calendar" for myform.birthday %}
{% form myform using fields "firstname" "lastname" "birthday" %}

> Secondly, you appear to be using a single template tag for rendering
> both forms *and* fields. Is there a motivation behind this?

Yes there is :-) The motiviation is, that {% form myform.birthday %} is just a
shorthand for {% form myform using fields "birthday" %}. You can also use it
to iterate over a form (like in your example below) and outputting it field by
field:

{% for field in myform %}
{% form field %}
{% endfor %}

I also don't see a need for an extra {% formfield %} tag. I don't see any
cases in which a {% formfield %} would be shorter, more flexible, easier to
use. It might be more readable, but I think it would end up as a stripped down
copy/alias of the form tag.

> Rendering modifiers
> 
>
> I share Carl's concern about exactly how and why these are necessary;
> even after your most recent post, I'm still not completely clear on
> what they mean in practice.
>
> On the one hand, I can see that there may be a use case for passing
> arguments into the template rendering process. However, you seem to be
> proposing a mechanism by which developers could define Python code
> that alter widget rendering by performing such tasks as:
>
>  * selecting a different rendering directory
>  * changing the template that is used for a widget
>  * changing the layout scheme
>  * prepending/appending content onto a widget
>
> I think I understand your motivation here, but my inclination is that
> this will end up being a very complex system that won't end up being
> useful in practice.

The rendering modifiers came up in the proposal because I thought about the
possible implementations of the {% form ... using ... and ... %} syntax. I had
the idea that these modifiers after "using" could be seperated from the tag
itself. I had the analogy to queryset methods in mind. E.g. the filter() method
changes the internal state of the queryset. The same would be true for
rendering modifiers, analogy: the "fields" modifiers limit the rendered
fields. These modifiers could be implemented easily as methods on the template
tag.

But with querysets you can extend the QuerySet base class and add other
convenience methods. I wanted the same for the form rendering. Why shouldn't

Re: [GSoC] Revised form rendering

[Gregor Müllegger]

Hi Yuri,
thanks for your toughts.

2011/4/2 burc...@gmail.com :
> Gregor,
>
> Regarding proposal itself,
>
> the idea to make django form rendering based on templates is good, but
> I think it should be simple, modular and not much verbose.
> I'd like more to see more modular, but easier syntax:
> 1) If you have to render form in your custom way, you have to write
> all form fields (you'll tweak their order and widgets anyway later, so
> it's a plus).
> 2) If you have to render field in your custom way, you have to write
> your own widget or widget template.
> 3) Any advanced form customization can be made using templates.
>
> Something like this:
> {% renderform form1 using layout "uni_form" %} {# errors output could
> be changed with layout templates only #}
>  {# will use template layouts/app1/uni_form/charfield/x.html if it
> was charfield. #}
>  {% render field1 using widget template "x" set class="field1" %}
>  {% render field3 %}
>  {% render field2 %}
> {% endrenderform %}

If I get your ideas right, then my proposed {% form %} and {% formblock %}
tag already supports what you suggested. I try to rewrite your example with
my proposed syntax:

{% formblock form1 using layout "uni_form" %}
{% form form1.field1 using widget template "x" and addclass "field1" %}
{% form form1 using fields "field3" "field2" %}
{% endformblock %}

The things that are different to your example are just some minor syntax
things. Additionally, the "rendering modifiers" (in this case "widget" and
"addclass") are not specified yet and I don't settled on a fixed set yet. I
would like to defer this until a later point because thats something that need
a broader discussion and IMO shouldn't take place in this thread, discussing
the more basic concerns regarding my proposal.

Or did I misunderstand what you tried to express? If thats the case, sorry for
that and please clarify that for me. I would appreciate your feedback.

(btw: a difference to your example is that I use the form tag to render forms
and single fields. I will include why I choose to do this in my response later
to Russell's post -- sorry I need to get offline but will respond again in the
next few hours)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC] Revised form rendering

[burc...@gmail.com]

On Sat, Apr 2, 2011 at 10:45 PM, Russell Keith-Magee
 wrote:
> Secondly, it may be possible to play a trick on the template compiler.
> Jonas Obrist suggested this trick to me at Djangocon (US) last year;
> however, I haven't had a chance to dig into it any more.
>
> The trick goes something like this:
>
> When you parse a template tag like an {% if %}, the template parser
> scans forward until it reaches an {% endif %}. All the node content
> that is discovered is then provided to the if node as the subtemplate
> for the if.
>
> However, there's no specific requirement that you need to parse until
> you reach a specific tag -- you could just scan to the end of the
> document, and treat the rest of the document as if it were the nested
> content of an if that runs to the end of the page.
>
> So -- you should be able to write a "CSS" template tag that reads to
> the end of the template, and in the process, discovers all the {% form
> %} nodes on the template. From that, you can determine all the CSS
> requirements that your HTML document has.
>
> It requires a bit of a twist in the document tree -- the entire
> document effectively becomes a child of the CSS node in the template
> -- but it should allow you to get access to form rendering
> requirements before the form is actually rendered on the page.

Hi Russell,

Maybe a bit different thing, but if we're talking about forms media
customization,

I'd love the following solution implemented for css and js:
Tag {% media for form1 %} or {% css for form1 %} which would remember
all previous css/js references, displaying only ones not rendered yet.
For example, lots of my widgets are referring jquery js and css, and
there's no need to load jquery 10 times.
And if I loaded jquery before forms, I would like to be able to say
{% js "{{STATIC_URL}}/media/jquery-1.4.2.min.js" %} or {% css
"/media/css/jquery-tweaks.css" %}

Gregor,

Regarding proposal itself,

the idea to make django form rendering based on templates is good, but
I think it should be simple, modular and not much verbose.
I'd like more to see more modular, but easier syntax:
1) If you have to render form in your custom way, you have to write
all form fields (you'll tweak their order and widgets anyway later, so
it's a plus).
2) If you have to render field in your custom way, you have to write
your own widget or widget template.
3) Any advanced form customization can be made using templates.

Something like this:
{% renderform form1 using layout "uni_form" %} {# errors output could
be changed with layout templates only #}
  {# will use template layouts/app1/uni_form/charfield/x.html if it
was charfield. #}
  {% render field1 using widget template "x" set class="field1" %}
  {% render field3 %}
  {% render field2 %}
{% endrenderform %}

-- 
Best regards, Yuri V. Baburov, Skype: yuri.baburov, MSN: bu...@live.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC] Revised form rendering

[Russell Keith-Magee]

On Fri, Apr 1, 2011 at 11:57 PM, Gregor Müllegger  wrote:
> I suggest reading this proposal online: https://gist.github.com/898375
> It's exactly the same as below but formated nicely.
>
>
> GSoC 2011 Proposal - Revised form rendering
> 
>
> Hi my name is Gregor Müllegger. I'm a Computer Science student in Germany at
> the University of Augsburg currently in the fourth year of my studies. I first
> came to django shortly before 0.96 was released and a lots of awesomeness was
> introduced with the magic removal branch.

Hi Gregor,

Firstly, I'd like to echo Carl's sentiments -- this is a strong
proposal that shows you've researched the history of Django
discussions on this topic, and given it some thought of your own.

Carl has already raised several of the points that I noticed on my
read through. However, I do have some of my own queries.

Syntax
~~

You've proposed a syntax that goes something like:

{% form myform using layout "p" and fields "firstname" "lastname" %}
{% form fancyform.favourite_color using layout "p" %}

I have two questions about this syntax.

Firstly, while it looks fine for a small example, I can see how it
would rapidly deteriorate if you have lots of fields, or lots of
custom field requirements for each field. Django's template language
doesn't allow you to split tags over multiple lines, so what happens
when my tag runs over 80 characters (this simple example is already a
69 characters)?

Secondly, you appear to be using a single template tag for rendering
both forms *and* fields. Is there a motivation behind this?

Rendering modifiers


I share Carl's concern about exactly how and why these are necessary;
even after your most recent post, I'm still not completely clear on
what they mean in practice.

On the one hand, I can see that there may be a use case for passing
arguments into the template rendering process. However, you seem to be
proposing a mechanism by which developers could define Python code
that alter widget rendering by performing such tasks as:

 * selecting a different rendering directory
 * changing the template that is used for a widget
 * changing the layout scheme
 * prepending/appending content onto a widget

I think I understand your motivation here, but my inclination is that
this will end up being a very complex system that won't end up being
useful in practice.

For example -- consider the case of validation. In an ideal world, it
would be nice to be able to just throw a switch and say "turn on
client side validation for this field", and then have that validation
work regardless of the rendering of the field itself. However, in
practice, it seems to me that the implementation of validation will be
tightly bound to the rendering of the field itself -- i.e, your widget
template would really need to be a "HTML4 text input with validation",
not a "text widget" with the HTML4 option and the validation option
enabled.

I can certainly see value in allowing template designers to pass in
keyword arguments when rendering fields on a template, allowing chrome
developers to define templates that implement various features (e.g.,
define a TextInput template that optionally has validation). However,
making this a complex registry and code solution seems like overkill
to me.

Form rows
~

If you look at the existing as_* form layout tools, there is a
distinction between gross form layout, and the layout of an individual
form row.

{{ form_header }}
{% for field in form %}
{{ pre_field }}}
{{ field }}
{{ post_field }}
{% endfor %}
{{ form_footer }}

Conceptually, the rendering for pre/post field could vary depending on
the field. How is this handled in your proposal? Is it part of the
form renderer? Or part of the widget rendering? This overlaps with
your discussion about error templates -- since errors (and their
rendering) is one of the major pre/post rendering activities for both
forms and fields.

CSS/Media
~

I'm not wild about the idea of having to include the same form content
twice in order to get CSS and JS included as where it is required.

I can see two options here.

Firstly, follow the lead of the {% cycle %} tag -- use the {% form %}
tag to define what the form will look like, but not actually render
where it is defined; assign the rendered properties to a context
variable, then reference that context variable when you want to
actually render something related to the form.

{% form myform  as myform_rendered %}

{{ myform_rendered.media.css }}

{{ myform_rendered.media.js }}

{{ myform_rendered.html }}

This would be fairly easy to implement, but I'm not sure it would be
especially flexible.

Secondly, it may be possible to play a trick on the template compiler.
Jonas Obrist suggested this trick to me at Djangocon (US) last year;
however, I haven't had a chance to dig into it any more.

The trick goes something like this:

When you parse a 

Re: [GSoC] Revised form rendering

[Gregor Müllegger]

Hi, thanks for your fast feedback. I was really looking forward to your input,
I haven't reached you in IRC before.

2011/4/1 Carl Meyer :
> Hi Gregor,
>
> As you've probably seen in past threads, this is an area where I'm quite
> motivated to see some improvement. I think you've got quite a strong
> proposal here in general, and you've clearly done your homework, so my
> comments below dive directly into the details:

(I've shorten my proposal a bit in the quotes to make the whole email more
readable)

> On 04/01/2011 11:57 AM, Gregor Müllegger wrote:
>> I suggest reading this proposal online: https://gist.github.com/898375
>> It's exactly the same as below but formated nicely.
>>
>>
>> GSoC 2011 Proposal - Revised form rendering
>> 
>>
>> ...
>>
>> Motiviation
>> ---
>>
>> ...
>>
>>    I want to tell the designer how he can do this without my help in the
>>    template.
>
> I agree with your Motivations section - in particular this final line,
> which sums up the core motivation as I see it.
>
>> Goals I want to accomplish
>> --
>>
>> ...
>>
>> **1. No HTML in python source**
>>
>> ...
>>
>
> This all looks quite good to me.
>
>> **2. Reordering/skipping fields**
>>
>> ...
>>
>> The goal will be to make these modifiers defineable. Which means that you
>> can create your own modifiers to support some of your extravagant form
>> rendering needs. To support this we will need to have a rendering modifier
>> *registry* or something similiar, where you can ideally load new ones with 
>> the
>> existing ``{% load %}`` tag. The same will apply for custom *chrome*,
>> described below.
>
> This last bit worries me; it has the potential to add unneeded
> complexity. Do you have any actual use cases for the "custom rendering
> modifier" that can't be handled adequately by your proposal otherwise?
> The concept of what a "rendering modifier" actually _is_ and what it can
> do both seem poorly defined here. I'd rather see this level of
> flexibility left out (and potentially added later) than done without a
> clear driving concept and motivation.
>

I understand your worries and I clearly don't want to invent a YAGNI case. But
I think it fits nicely into the proposal (of it's current state). The reasons
are chrome. Chrome are just a specialized versions of "rendering modifiers"
(btw: I don't like the name but it's the best I came up with yet). And if you
can make it possible to register your own chrome, then it might be only a very
small overhead (if any) to do the same for rendering modifiers.


I want to make clear what a rendering modifier would be on the python level. I
thought about a FormRenderer class that handles the rendering of the form
based on the some predefined rules like what default layout is used. How to
render widgets and so on.

Rendering modifiers are then decorated around the FormRenderer. An example:
In the template::

{% form myform using layout "p" and widget "CalendarInput" for
myform.birthday %}

The FormRenderer iterates over the fields in myform and wants to render them
with layout "default" and with the widgets defined in the form itself. It
asks then the form modifiers "layout" and "widget" that were used in the form
tag how they want to modify the rendering of a specific field. The "layout"
modifier would always return "use layout p", the "widget" modifier always
returns "use the default widget" except when FormRenderer reaches the birthday
field, then "widget" would return "use CalendarInput widget".

(Sorry for the very easy language, maybe my english is to limited to express
it in a not so verbose way.)


I haven't thought in detail about additional usecases for the rendering
modifiers that might life in the app ecosystem. But while writing this
response, there came some of the possibilities into my mind. Maybe there is
the potential for a doctype modifier::

   {% form myform using doctype "html4" %}

This will add a new template directory to the template path while rendering
the form. So that it renders HTML4 compatible widgets if available in the new
template directory. And if not it falls back to the default templates used by
the new template based widgets.

Something more exotic (and complex to implement but still possible) is a
"validation" rendering modifier::

   {% form myform using validation %}

This will walk through the form and outputs at the end some javascript that
validates the form on the client side based on the used form fields.
This will need a port of python validators into javascript, but that isn't
that complex (I already did a proof of concept for that kind of behaviour in a
template tag).
And thats something that wouldn't get into django core but could life nicely
in **django-clientsidevalidation** :-)


This is why I think that the registry for rendering modifiers is useful. We
would need a registry (i.e. a dict that holds the "name to modifier" mapping)

Re: [GSoC] Revised form rendering

[Carl Meyer]

Hi Gregor,

As you've probably seen in past threads, this is an area where I'm quite
motivated to see some improvement. I think you've got quite a strong
proposal here in general, and you've clearly done your homework, so my
comments below dive directly into the details:

On 04/01/2011 11:57 AM, Gregor Müllegger wrote:
> I suggest reading this proposal online: https://gist.github.com/898375
> It's exactly the same as below but formated nicely.
> 
> 
> GSoC 2011 Proposal - Revised form rendering
> 
> 
> Hi my name is Gregor Müllegger. I'm a Computer Science student in Germany at
> the University of Augsburg currently in the fourth year of my studies. I first
> came to django shortly before 0.96 was released and a lots of awesomeness was
> introduced with the magic removal branch.
> 
> I'm also doing some django freelancing work since 2008 to finance my studies
> and attended DjangoCon EU in 2011. This year I would like to apply to
> Google Summer of Code helping to improve Django with something that bugs me
> since quite a while: It's builtin ability to render a form straight away into
> HTML.
> 
> Motiviation
> ---
> 
> Why would I like to change the current behaviour? There are some reasons for
> this:
> 
> 1. It's hard to change the default rendering.
> 
>It is very easy to use e.g. django's builtin rendering of a ``ul`` based
>output like ``{{ myform.as_ul }}``. But what if you want to use your own
>*layout* like ``{{ myform.as_dl }}``? You can create a new method on your
>existing forms, but that involves writing python code (out of the designers
>hand) and might not be possible for thirdparty forms.
> 
> 2. Maybe the ``as_ul`` rendering is fine for me. But maybe I want to skip that
>unnecessary field that the thirdparty form is providing. The solution would
>be to write down every single field in your template, reimplementing the
>``ul`` layout::
> 
> {{ form.field.label_tag }}: {{ form.field }}
> {{ form.field.help_text }} {{ form.field.errors }}
> {# skipping field2 here #}
> {{ form.field3.label_tag }}: {{ form.field3 }}
> {{ form.field3.help_text }} {{ form.field3.errors }}
> ...
> 
>We all love DRY, so this is not acceptable.
> 
> 3. The designers I worked with are often interested on adding custom css class
>or an attribute to a form field. Most of the time this is really a pain to
>do if you don't have control over the python form code. Imagine a reusable
>app that ships with urls, views, forms. To add a single class you would
>need to: (a) overwrite the predefined url because you want (b) to specify
>an additional parameter for the view which is (c) your custom subclass of
>the thirdparty form::
> 
>class ChangedAttributeForm(ThirdPartyForm):
>field = forms.CharField(max_length=50,
>widget=TextInput(attrs={'class': 'fancy-text-input'}))
> 
>btw: This also violates the DRY principle since you have to redefine the
>used field type, it's attributes (like ``max_length=50``) and the widget.
> 
>I want to tell the designer how he can do this without my help in the
>template.

I agree with your Motivations section - in particular this final line,
which sums up the core motivation as I see it.

> Goals I want to accomplish
> --
> 
> After showing some of the problems that I see, are here the higher goals I
> want to achieve during the summer:
> 
> 1. All of the rendering formats should be extracted from django's python 
> source
>into a well defined structure of customizable templates.
> 2. Make it possible to reorder the use of form fields in the template without
>needing to write down the complete form.
> 3. Support for *chrome* that can be added dynamically in the template to a set
>of fields, changing some aspects of the form output.
> 4. Ensuring that the DRY principle is applyable to your form templates (in
>case of reordering/skipping some fields, like shown above)
> 5. Ensuring total backwards compatibility for ``{{ form.as_p }}`` etc. but
>deprecating it.
> 6. Converting the admin to use all of the new goodness. First, to make django
>eating its own dogfood. Second, to prove that the stuff I have developed is
>really a step forward and to show up big problems that occur in reallife
>scenarios, not taken into account in the blueprint.
> 7. Documenting all the newly introduced templatetags, the builtin chromes,
>how to write custom chrome, how to create your own form layout etc...
> 
> Let's get a bit more detailed. How do I want to implement these goals?
> 
> **1. No HTML in python source**
> 
> I will push the formating of ``as_table``, ``as_ul`` and ``as_p`` into a
> specified template structure. A formating (e.g. ``ul``) will be called a
> layout and will live in the template directory ``forms/layouts//...``.
>