Re: Controlling form/widgets output

[Mike Amy]

The FormRenderer idea is a good one.

But how about generalised Renderer objects that take or inherit a
template list, build a context and then can be passed around to fill
in the details. Update-able at construction, processing or rendering,
and able to be passed straight into another context for rendering?

Like:

from django.template import loader, Context

def update_context(context, *dicts, **kwargs):
   for dict in dicts:
   context.update(dict)
   if kwargs:
   context.update(kwargs)
   return context

class Renderer(object):
   "builds and renders a context"
   def __init__(self, templates = None, *dicts, **kwargs):
   "takes a list of templates, and any extra context"
   self.context = dict()
   self.update(*dicts, **kwargs).template = loader.select_template
(templates or self.templates)

   def update_context(self, *dicts, **kwargs):
   "update the context that will be rendered"
   update(self.context, *dicts, **kwargs)
   return self

   def render(self, *dicts, **kwargs):
   "renders with any extra context, without updating, enabling
flyweight, e.g. reuse by a widget in a form or a search result list."
   return self.template.render(update_context(Context
(self.context), *dicts, **kwargs))

   def __unicode__(self):
   "calls render so we can pass the renderer directly into another
context, as if a string"
   return self.render()


Didn't test it. ;)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Controlling form/widgets output

[Jacob Kaplan-Moss]

On Thu, Jan 22, 2009 at 4:57 PM, catsclaw  wrote:
>   Well, it seems to me that makes for an *extremely* tight coupling
> between the model and the view.

I'm sorry to be so blunt, but your perception is misguided. Forms have
no dependancy upon models, nor do models on forms, nor must views use
forms, models, or anything, really.

*Model*Forms do of course have dependancies on both, but you can use a
form without using models at all. I have Django apps in the wild that
have no database connection, and I use forms just fine.

I really think you need to do more research here. Maybe if you share
exactly what you're trying to do we can help figure out why it seems
so hard.

> And it's a little difficult for me to
> understand what value there is in such a tight coupling--if I've got a
> DateField, why can't I have it represented in an HTML page by a
> javascript calendar pop-up, or a text box, or three select boxes
> (month, day, and year).

It's difficult to understand because you've assumed it's impossible;
it's not. A quick google search for "django datefield select" turns up
http://www.djangosnippets.org/snippets/399/ as the first result;
there's a few other recipes showing similar widgets.

Using custom widgets is in fact incredibly easy. Start by reading
http://docs.djangoproject.com/en/dev/ref/forms/widgets/#specifying-widgets,
and follow on from there.

>   Plus, any time you collect a password you need to display it in a
> form using a password input, not the stock text input.

Again, this is in fact not only possible, but easy::

class MyForm(forms.Form):
password = forms.CharField(widget=forms.PasswordInput)

> And I might
> very well have two separate form fields which should be displayed as a
> single unit.

This one's our fault: ``forms.ComboForm`` is what you want here, but
it's not yet documented. Sorry!

> There's a need to have the display decoupled from the
> form logic.  Django just doesn't provide much help in doing that.

It's pretty decoupled. If you want a custom widget, write a custom ``Widget``.

> In other words, if I create a
> widget which renders a CharField and I'd like to validate the maximum
> length, I either *must* inherit from TextInput or PasswordInput, edit
> the CharField class, or write a different CharField class that's aware
> of my new widget.

No, you don't::

class MyForm(forms.Form):
password = forms.CharField(widget=forms.PasswordInput, max_length=30)

> If I were writing it, I'd make the Form class responsible for
> defining the order of the fields, the labels, the validation logic and
> errors.  The Fields would be part of that.

You just described exactly how Django's form package works. Seriously.
The ``Form`` defines the order of fields (they're sorted in definition
order), the validation logic (in the form of ``clean_*`` methods), and
errors (as ``Form.errors``).

-

Why don't we start over here: what is the problem? What did you try do
do? What did you expect to happen? What actually happened?

Jacob

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Controlling form/widgets output

[catsclaw]

On Jan 21, 5:37 pm, Russell Keith-Magee 
wrote:
> >   Here's just one example: when fields are written out to HTML, they
> > get asked "what attributes does this kind of widget need?".
>
> Unless I'm misunderstanding your intent with this statement - No, they
> aren't. A form contains fields, which describe the type of data that
> is being collected and the validation procedures for said data. Each
> field provides a widget, which describes a HTML element that allows
> that data to be collected. When the form is rendered, it binds some
> data to a field instance in a BoundField, and renders the BoundField
> using the widget for the field.

   Well, it seems to me that makes for an *extremely* tight coupling
between the model and the view.  And it's a little difficult for me to
understand what value there is in such a tight coupling--if I've got a
DateField, why can't I have it represented in an HTML page by a
javascript calendar pop-up, or a text box, or three select boxes
(month, day, and year).
   Plus, any time you collect a password you need to display it in a
form using a password input, not the stock text input.  And I might
very well have two separate form fields which should be displayed as a
single unit.  There's a need to have the display decoupled from the
form logic.  Django just doesn't provide much help in doing that.

> The field isn't asked anything
> during the rendering process beyond "what is your widget?".

   Not true.  There's a "widget_attrs" method on the base Field class
which passes a widget and expects to be passed back a dictionary of
attributes to be added to the widget.  In other words, if I create a
widget which renders a CharField and I'd like to validate the maximum
length, I either *must* inherit from TextInput or PasswordInput, edit
the CharField class, or write a different CharField class that's aware
of my new widget.

> >   Is there any work being done to clean up the way Django creates
> > forms?  I saw there was something being done with DOCTYPE
> > declarations, but it didn't look like that was making very extensive
> > changes.
>
> In short, no. We completed a major refactor of the forms framework
> just before v1.0 landed, and we're not really looking to make another
> major change.
>
> However - our commitment is to backwards compatibility, not to no
> change at all. If you have a specific problem that can't be solved,
> feel free to make a specific suggestion on how to improve things.

   If I were writing it, I'd make the Form class responsible for
defining the order of the fields, the labels, the validation logic and
errors.  The Fields would be part of that.  I'd make a FormRenderer
class which was responsible for drawing a form in different formats
(as a list, as a table, etc ...) and Widgets would act as a bridge
between a Field (or group of fields) and the FormRenderer.

-- Chris
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Controlling form/widgets output

[Malcolm Tredinnick]

On Thu, 2009-01-22 at 08:37 +0900, Russell Keith-Magee wrote:
> On Thu, Jan 22, 2009 at 7:39 AM, catsclaw  wrote:
[...]

> > Adding a
> > widget therefore requires rewriting the field classes you want the
> > widget to be used with, rather than letting the widget interrogate the
> > field classes.  That's not great design, and it means any changes to
> > form rendering becomes a major pain.
> 
> Again - examples abound in the Django source tree that demonstrate
> that this simply isn't true.

In the larger scheme, I think this is one of those areas where people's
initial intuition differs. Django's form classes generally push
downwards, with the greater intelligence lying at the higher levels
(forms know the most, then form field classes have more localised
knowledge and the widgets have the least knowledge and are simply given
the information that they should present).

There's a little leakage, in the sense that field classes can pass down
attributes, but its' not required. Also, the attributes that are passed
down are a pretty common set (and, by and large, arbitrary, as they're
just a dictionary, so you can pass in whatever you like via the form or
fields). If there's a concrete example of what you can't get, that would
be good to outline.

I suspect that that is why some people might prefer the opposite
direction for the dependencies, with widgets able to query fields.
However that would lead to a different and, likely, greater set of
problems with, for example, the over-arching form subclasses not able to
get a hold of the bigger picture of what they're managing. To solve that
and allow the reverse-dependencies, you end up having mutual
dependencies, which bring their own set of problems to the table, in the
design sense.

Again, specific issues would be easier to address here, rather than
general complaints. We have a lot of accumulated experience to suggest
the current design isn't a total disaster and I'm yet to find anything
that isn't possible in a fairly neat fashion with the current dependency
direction. So what sort of things are you trying to do where you can't
think of a way to implement them at the moment?

Regards,
Malcolm


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Controlling form/widgets output

[Russell Keith-Magee]

On Thu, Jan 22, 2009 at 7:39 AM, catsclaw  wrote:
>
> Hi all --
>
>   I've started to poke around in customizing the generated form code
> and providing my own widget instances, but the current Django form
> code is written in such a way that it makes things nearly impossible.

First off - as a general guide, hyperbolae will get you nowhere. I can
point to several places in the Django codebase where custom widgets
are used to override the default widget, so "nearly impossible" isn't
a fair comment.

>   Here's just one example: when fields are written out to HTML, they
> get asked "what attributes does this kind of widget need?".

Unless I'm misunderstanding your intent with this statement - No, they
aren't. A form contains fields, which describe the type of data that
is being collected and the validation procedures for said data. Each
field provides a widget, which describes a HTML element that allows
that data to be collected. When the form is rendered, it binds some
data to a field instance in a BoundField, and renders the BoundField
using the widget for the field.

Yes - there are entry points where HTML attributes can be injected
into this rendering process, and there are some entry points where the
field can push attributes into the widget, but otherwise it is the
widget that is rendered, not the field. The field isn't asked anything
during the rendering process beyond "what is your widget?".

> Adding a
> widget therefore requires rewriting the field classes you want the
> widget to be used with, rather than letting the widget interrogate the
> field classes.  That's not great design, and it means any changes to
> form rendering becomes a major pain.

Again - examples abound in the Django source tree that demonstrate
that this simply isn't true.

>   Is there any work being done to clean up the way Django creates
> forms?  I saw there was something being done with DOCTYPE
> declarations, but it didn't look like that was making very extensive
> changes.

In short, no. We completed a major refactor of the forms framework
just before v1.0 landed, and we're not really looking to make another
major change.

However - our commitment is to backwards compatibility, not to no
change at all. If you have a specific problem that can't be solved,
feel free to make a specific suggestion on how to improve things.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Controlling form/widgets output

[catsclaw]

Hi all --

   I've started to poke around in customizing the generated form code
and providing my own widget instances, but the current Django form
code is written in such a way that it makes things nearly impossible.
   Here's just one example: when fields are written out to HTML, they
get asked "what attributes does this kind of widget need?".  Adding a
widget therefore requires rewriting the field classes you want the
widget to be used with, rather than letting the widget interrogate the
field classes.  That's not great design, and it means any changes to
form rendering becomes a major pain.
   Is there any work being done to clean up the way Django creates
forms?  I saw there was something being done with DOCTYPE
declarations, but it didn't look like that was making very extensive
changes.

-- Chris

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

ManyToManyField in both models/forms

[Evgeniy Ivanov (powerfox)]

Hi list,
I had to implement M2M editing in both admin forms (something like
editing userlist for group and grouplist for user). I wasn't able to
find any fine solution, but specifying M2M in both models and creating
tables manually (since syncdb tries to create two tables).
I herd in the IRC, that such thing (editing in both forms) is
abnormal, but I disagree.
So I tried to use intermidiary model like described in the docs, but
it is really another thing (not "pure" m2m). Another solution is
hardcording the form, but it will require extra save code, I don't
like it too.

I think that if you add special arg (create_table) to ManyToManyField
it can be easily implemented using models/fields/related.py:751.

If you think it can go, I will create a ticket.

Also there is (a kind of offtop) a very cute thing:
http://www.djangosnippets.org/snippets/962/
Why isn't it in the native ManyToManyField?


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Token parsing

[Jacob Kaplan-Moss]

On Thu, Jan 22, 2009 at 3:58 AM, Peter2108  wrote:
> Not quite sure if this is the place to post this. Anyway, in the
> module template.loader.tags the do_extends function parses a tokens
> content like this:bits = token.contents.split()  which does not
> work correctly if the
> extends path has a folder name with a space in it (as one of mine
> did). I read in the docs on adding tags (http://docs.djangoproject.com/
> en/dev/howto/custom-template-tags/) that token.split_contents() should
> be used so this looks like a bug.

Yup, it is -- please file a bug in the ticket tracker. Thanks!

Jacob

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Token parsing

[Peter2108]

Not quite sure if this is the place to post this. Anyway, in the
module template.loader.tags the do_extends function parses a tokens
content like this:bits = token.contents.split()  which does not
work correctly if the
extends path has a folder name with a space in it (as one of mine
did). I read in the docs on adding tags (http://docs.djangoproject.com/
en/dev/howto/custom-template-tags/) that token.split_contents() should
be used so this looks like a bug.

Thanks, Peter

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Joins and aggregates

[Alex Gaynor]

On Wed, Jan 21, 2009 at 9:01 AM, Ludvig Ericson wrote:

>
> On Jan 21, 2009, at 14:10, Russell Keith-Magee wrote:
> > Any opinions? Any other options that I may have missed?
>
> Sorry if I'm stating the obvious, but would it be
> impossible to error out if two aggregates cause this
> unexpected behavior?
>
> I mean then I'd say raising an exception for such queries
> would be acceptable. Something like "unsatisfiable query".
>
> - Ludvig
>
> >
> Is there a way to programatically determine if an aggregate is causing such
a problem(without generating false positive) and therefore we could only do
the subqueries where necessary.  I don't think it's fair to the users of
properly working aggregates, of which there are an awful lot, to impose
either unneeded restrictions or overhead.

Alex


-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." --Voltaire
"The people's good is the highest law."--Cicero

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Joins and aggregates

[Tim Chase]

>> 1) Prevent joins at the query level. Keep track of the joins
>> that are being used for aggregates, and only allow one join.
>> This would mean killing a lot of queries that work correctly
>> right now, but would prevent the class of invalid queries.
>> 
>> 2) Push aggregates into subqueries. Rather than try to 
>> accomodate aggregates using GROUP BY in the main query, 
>> generate a subquery for each aggregate and join the main 
>> query table onto the aggregate subquery table to provide the
>> annotation.  Obviously, this is going to get very expensive
>> at the query level.
>> 
>> Any opinions? Any other options that I may have missed?

Is there a mid-way point?  It looks like aggregates work
with _one_ join, but not _more_ than one join.  So possibly
use aggregates as they currently stand for the _first_ one,
and then use #2 above (aggregates in subqueries) for
subsequently joined data.  It allows you to maintain the
efficient grouping queries for the common (single-table)
joins, and only invokes the expense of #2 if there's a 2nd
table for aggregation.

I haven't waded into the QSRF+Aggregate code lately, so I
don't know how easy it would be to add in "this is how many
tables we've joined so far" tracking.

-tim






--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Joins and aggregates

[Ludvig Ericson]

On Jan 21, 2009, at 14:10, Russell Keith-Magee wrote:
> Any opinions? Any other options that I may have missed?

Sorry if I'm stating the obvious, but would it be
impossible to error out if two aggregates cause this
unexpected behavior?

I mean then I'd say raising an exception for such queries
would be acceptable. Something like "unsatisfiable query".

- Ludvig

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Joins and aggregates

[Russell Keith-Magee]

Hi all,

I've been looking at ticket #10060, The problem is caused by join
behavior in aggregates, but I'm uncertain about the best way to
address the problem.

Using the example from the ticket, the problem goes like this: You can
use double underscore notation for form joins in aggregates:

>>> Branch.objects.annotate(total=Sum('center__client__loan__amount'))

This will join tables as expected, and return objects annotated with a
correct total.

However, if a second deep-joined aggregate is added to the same query:

>>> Branch.objects.annotate(
total=Sum('center__client__loan__amount'),
repaid=Sum('center__client__loan__payment_schedule__payments__principal'))

the annotated `total` is much larger than it should be, by a factor
determined by the cardinality of payment_schedules in each loan.

Obviously, this is less than ideal. Unfortunately, the solution isn't
trivial. The incorrect value for total is caused by the fact that
total and repaid share the same chain of joins. However, if you split
it out so that repaid uses a unique join chain, you still get an
incorrect result - with a much larger multiplication factor
(determined by the cardinality of center, rather than
payment_schedules).

Fundamentally, the issue here is abstraction leakage - SQL aggregates
are more about the grouping procss than they are about the aggregate
itself; the aggregate()/annotate() puts the emphasis on the
aggregates, and implies the grouping.

Now, the existing syntax works fine, as long as you are aware of the
potential limitations. Unfortunately requires a little knowledge of
the underlying SQL. This has always been true of Django's query
language to an extent - my question to the group is how much do we
want to lean on this limitation?

If we decide to accept the limitations, obviously there is a big block
of documentation required.

I can see two alternatives to documentation:

 1) Prevent joins at the query level. Keep track of the joins that are
being used for aggregates, and only allow one join. This would mean
killing a lot of queries that work correctly right now, but would
prevent the class of invalid queries.

 2) Push aggregates into subqueries. Rather than try to accomodate
aggregates using GROUP BY in the main query, generate a subquery for
each aggregate and join the main query table onto the aggregate
subquery table to provide the annotation. Obviously, this is going to
get very expensive at the query level.

Any opinions? Any other options that I may have missed?

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---