removing "milestone" field in Trac

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi all,

The milestone field came up again in IRC discussion tonight between
Julien and Paul and I -- namely, that it doesn't seem all that useful
and causes noise in Trac.

It might have made sense back when we had more feature-driven releases
and a feature voting process, but as we're (trying to) move towards
date-based releases and we've moved away from feature voting, I don't
see what purpose it serves. Tickets marked "release blocker" must get
fixed before release, any other accepted ticket is always a candidate to
get fixed if someone works on it in time. Milestone doesn't add anything
but noise.


Julien already raised this topic a while back [1] and Luke more or less
agreed. Can we just go ahead and remove the field, or does somebody want
to present an impassioned defense of its usefulness (in which case you
should explain what it means, which tickets should get a milestone set,
who should set it, and under what conditions it should be removed or
bumped)?

Thanks,

Carl

[1]
http://groups.google.com/group/django-developers/browse_thread/thread/bf7da09f1f5881dc/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5393IACgkQ8W4rlRKtE2eVTgCgrAS6xav1CDdbzwIdBF1MHy3M
G4AAnRDATInx+eFLqWlj6czhFUeT+49x
=a3GB
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django test-runner annoyances

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 09/13/2011 08:46 AM, mvr wrote:
> Why doesn't the django test management command / test builder allow
> fully-qualified package names instead of just app-relative ones?
> 
> At work we've been using the method below to monkey-patch the test
> builder, so that
> 
>  $ django-admin.py test my_module.my_app.tests.some_test_file
> 
> always works as expected.  We'd like to get rid of this monkey-patch,
> and since this functionality can be added in such a way that it's
> completely backwards compatible, where is the harm?  I'm also willing
> to submit a diff that modifies django in-place, but the monkey patch
> below should be easy to read and first I wanted to hear if anyone has
> any thoughts on why the existing behaviour really is exactly what it
> should be.

I'm generally in favor of updating Django's test runner to be more
consistent with what the rest of the Python testing world does. Being
able to reference test files, suites, and tests by
fully-qualified-dotted-path rather than magical-applabel-path would be a
good start, IMO.

Another piece would be adding support for unittest2 test discovery, to
lift the requirement that all tests have to live directly in tests.py or
models.py. It's not that I think putting tests inside a tests package is
a bad convention, but if you have a lot of tests it's unfortunate that
you currently have to manually import all the suites into tests/__init__.py.

But I'm getting off-track -- these should be separate tickets anyway. If
you'd like to file the first one and upload a backwards-compatible
patch, I'm +1 on the concept.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5xC54ACgkQ8W4rlRKtE2eNFgCg7gVEkO6Y+tmXcsWlidmh67ge
SQwAn0PqFg74dy1yLsSPDYab1Jj+jNZ+
=bAbE
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #7198 - Better error message when app is missing models.py

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Gary,

On 09/12/2011 12:04 AM, Gary Wilson Jr. wrote:
> I'm a fan of not requiring a models.py, as IMHO it shouldn't be any
> different than other common files found in an app e.g. urls.py,
> templatetags dir, etc.  If I don't need any models for my app, then
> why must I still have a models.py?  That said, it also seems there
> could be some backwards incompatibilities if code or external tools
> rely on a valid app including a models.py file.

Actually, I think there's generally consensus that requiring models.py
is not ideal. There's already an existing GSoC branch (app-loading),
which already fixes the models.py issue (AFAIK) but is somewhat
languishing for lack of attention. So I think the best path towards
getting that fixed is to check out that branch and help get it in
merge-ready shape. Jannis was the mentor for that GSoC, he or Arthur
Koziel (the student) can probably comment most knowledgeably about what
needs to be done.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5ub0QACgkQ8W4rlRKtE2cISACgh/lYqhb4OR4aqllMPR4xyG4P
c8MAnRyh+tdcXCpxTq6Z8g5L+MNNC7ZX
=pVuz
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF token not validated?

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Jens,

On 09/12/2011 10:20 AM, Jens Diemer wrote:
> 
> I wonder that the CSRF token send from the client didn't be validated.

Well, it is sanitized to only alphanumeric characters, but you're right
that the length is never checked.

> Don't know if a DOS attack is possible by sending many request with very
> long CSRF tokens?
> 
> IMHO it's a good idea to check the length before do anything with it.

Sanity-checking the length sounds reasonable to me - do you mind opening
a ticket for this and attaching your patch?

Thanks,

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5ubHQACgkQ8W4rlRKtE2frrQCgr8HhCPKaPGKyTocUGnmiU9Ku
ekYAoNgZqJ/n4SJnd1tD2Zkpeb/+du47
=ZWv6
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: https://www.djangoproject.com/download/ is down!

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 09/11/2011 12:46 PM, Wim Feijen wrote:
> The download page for Django is down.
> 
> I also filed a ticket, so this is a duplicate message.

Thanks for the alert, but it seems to be up for me, and
"downforeveryoneorjustme.com" confirms that its up. Either its a
localized issue, or if it was down, it isn't any longer. I've closed the
ticket "worksforme".

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5tBX0ACgkQ8W4rlRKtE2fY8ACgl0nnp5ymGV0QXrI6n0XWc3AI
3/wAnjuSafU/ymyh2qlmoOIZuWuiZV1I
=GKeE
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Improved password hashing for 1.4

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



On 09/09/2011 09:54 PM, Paul McMillan wrote:
> In conjunction with Justine Tunney, Isaac Kelly and Russell KM, I'd
> like to introduce our plan of attack for including significantly
> better password hashing in Django 1.4. One of the key goals with this
> push is to include just enough functionality that we can improve this
> particular aspect of Django. 
[snip]
> I'm really excited that we finally have the momentum to bring this
> important change to Django!

Me too. The plan looks great to me.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5q+vgACgkQ8W4rlRKtE2dGKwCfXzm56E+whpY6Ns58l6HLrXlD
5CAAn3+UWmP+vtTyPIUFNARWiGcvR7hz
=Pwrt
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: class based views: object instead of dictionary as context?

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 09/02/2011 01:01 AM, Greg Donald wrote:
> Planning for designers to work in code is an extreme edge case.  In 15
> years of web development I've never once worked with a single designer
> who was interested in touching any code.  As far as my own web
> projects go, the designer is done the moment they give me a layered
> .psd.

Please be careful with the unwarranted generalizations based on your
personal experience. I'm sure there are designers who never go beyond
the PSD, but my personal experience happens to be the opposite - I've
never worked with a web designer who didn't handle all of the HTML, CSS,
and templates themselves; almost all of the designers I know are
perfectly competent in front-end coding (and some in Python, too). It
seems a bit rude to dismiss them all as an "extreme edge case." If you'd
like to broaden your horizons, I'd be happy to recommend some names ;-)

> How is something like {{ view.myvar }} any harder than {{ myvar }} for
> those two designers in the whole world who actually want to touch the
> code?

I agree that {{ view.myvar }} is just fine.

But please lay off the overblown rhetoric. I can name fifteen designers
who code off the top of my head (more than I could name who don't).

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5hBdwACgkQ8W4rlRKtE2dmjACfT9EURJhvwTLW/S21ak7TzXvI
U84AoMzbnDACXDf0oTVjtovcen+ybv+6
=Sz6+
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Form Rendering API Proposal

[Carl Meyer]

Hi Chris,

On 07/09/2011 02:50 AM, Chris Beaven wrote:
> If we're going to keep things simple, why are we introducing the idea of
> inline "using" templates?

That's a good question. I wouldn't be gutted at all if we dropped
inline-using from the initial scope, too, because I really think
separate template files is not a bad thing. But I'm also not as
concerned about having it in because in my mind it really doesn't have
the complexity downsides of inline-extends; its just a pretty normal
scoped tag, much like "with" or other existing tags.

> You go a long way of convincing me that the confusion introduced by
> 'extends' isn't worth it.
> You're right, my example suited a separate template file just fine. But
> do I really need to create a new template file for all specific cases?
> It seems like the following templates would be much better to just be
> kept inline with the main template the form is being used in.
> 
> "forms/p-custom-help-text-for-displayname.html"
> {% extends "forms/p.html" %}
> {% block config %}
> {% formconfig field using
> "checkbox-custom-help-text-for-displayname.html" for form.display_name %}
> {% endblock %}
> 
> "forms/fields/checkbox-custom-help-text-for-displayname.html"
> {% extends "forms/fields/checkbox.html" %}
> {% block label %}Show posts by me as from {{ user.username }}
> rather than {{ user.get_full_name }}{% endblock %}
> 
> Perhaps I'm not quite getting how we'd do this kind of thing with the
> current proposal.

The only issue I see here is naming :-) When a template gets that
specific in purpose, it doesn't make sense to pretend its reusable and
try to name it in a "generic" way. In general I don't see any problem
with making specific included form templates for specific needs. I'd
probably just call these "profile_form.html" and "display_name.html", if
those are names that make sense in the context of the project. I don't
know about your projects, but mine already include plenty of template
partials with very project-specific, non-generic purposes, so I don't
see this as that different.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Thoughts on solution to forward references in MySQL (#3615)

[Carl Meyer]

Hi Russell and Jim,

On 07/06/2011 05:34 PM, Russell Keith-Magee wrote:
> On Thu, Jul 7, 2011 at 5:05 AM, Jim D.  wrote:
>> * There's a DB feature can_defer_constraint_checks .  I couldn't find much
>> by way of documentation or or usage of this feature. But I was trying to
>> figure out if the work we are doing here is what this feature refers to, and
>> if so, if we should be marking this as True for MySQL and SQLite with this
>> implementation. I'm not sure there is other behavior that is required or
>> expected in that attribute. Anyhow, that might also be a path forward for
>> the issue I raise above (ie. we could skip the test if
>> can_defer_constraint_checks is not True).
> 
> This feature flag exists essentially to verify whether MySQL InnoDB is
> currently in use. It isn't used during normal runtime; it's purely a
> test skipping flag. It's used to identify tests that need to be
> skipped because the test data requires a circular reference which
> (historically) hasn't been possible under InnoDB because constraints
> aren't checked.
> 
> Essentially, this feature flag shouldn't need to exist at all; it only
> exists so that InnoDB passes the test suite without errors. If we are
> able to resolve the issue that allows MySQL to use forward references
> in data, then this feature flag shouldn't be required at all.

This isn't true since the introduction of the new deletion code last
fall. The feature flag is used in db/models/deletion.py to detect
whether it is necessary to null out nullable FKs prior to deletion. On
backends which can defer constraints, this nulling-out would add a
needless extra query. On backends which can't defer, nulling-out means
the nullable FK relationship doesn't dictate a deletion-ordering
constraint, which lets us handle circular FK references as long as at
least one of them is nullable (otherwise we'd have to just throw up our
hands and hope for the best, likely resulting in an integrity error).

Since this use of the flag is unrelated to fixture loading, the
fixture-loading fix should not change the value of the flag for any backend.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Form Rendering API Proposal

[Carl Meyer]

On 06/23/2011 04:43 PM, Jacob Kaplan-Moss wrote:
> 1. Performance: it looks, to me, like rending a basic form is going to
> cause dozens of template includes and dozens of sub-renders (the form
> loads a form template which loads row templates which load widget
> templates). That's dozens of disk hits, and a lot of overhead for form
> rendering. I worry about this overhead a lot. Django's performance has
> slipped lately, and I'm really afraid this'll make things a lot worse.
> 
> So I'm going to need to see some benchmarks -- particularly in how a
> simple {% form myform %} compares to {{ form.as_* }}.
> 
> The wrong performance benchmarks could result in a veto from me; this
> is important.

We've had a fair bit of discussion on this, and some benchmarks, around
Bruno's work porting django-floppyforms (templated widgets) to a core
patch (https://code.djangoproject.com/ticket/15667). The disk hits
aren't a big concern to me - if you are at a point where you have to
care about template speed, you already need to be using the cached
template loader regardless. But just plain rendering speed is a problem
- even with just widgets in templates, in some pathological cases (e.g.
a ChoiceField with tons of choices) it can be quite significantly
slower. Enough so that I put #15667 in the icebox for now, pending
seeing the impact of Armin's GSoC.

I really think templated form-rendering is a massive improvement in
Django for front-end devs, so I'm very hopeful that Armin's work can
make rendering speed a non-issue. No pressure, Armin ;-)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: WTForm should be inbuilt to Django, and make admin & others use it.

[Carl Meyer]

On Jun 20, 12:13 pm, Andre Terra  wrote:
> Have we moved forward with this issue at all? What is its current status?

I also maintain an external package that implements fieldsets (django-
form-utils [1]). Both django-form-utils and WTForm take the approach
of adding a Python data structure to Form.Meta that represents the
fieldsets. At one point I had thought I would push a patch to get a
fieldsets API like this into django.forms, but I'm no longer convinced
that's a good plan.

For the admin, representing fieldsets in a Python data structure in
the ModelAdmin is necessary and appropriate, since the admin is auto-
generating form rendering for a wide variety of Python models, and a
Python programmer should be able to generate these fieldsets by
touching only Python code. That's a special case. In general,
rendering of forms with fieldsets is a purely presentational issue
(the fieldsets don't impact the data submitted by the form), so I
think having fieldsets represented in Python code is the wrong
approach (and will inevitably lack the flexibility the designer needs
in their markup, just like the current hardcoded form field rendering
does). A template author and designer should be able to make changes
to the markup presentation of a form, including fieldsets, without
needing to touch Python code at all.

Gregor Müllegger is currently working on a Google Summer of Code
project to provide a templates-based flexible form rendering system. I
think this system will be flexible enough to allow template authors to
use fieldsets with forms easily and non-repetitively.

Carl

[1] http://pypi.python.org/pypi/django-form-utils

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Logging configuration and handle_uncaught_exception

[Carl Meyer]

In case anyone's interested in this but isn't following the ticket,
I've attached an updated patch there using a variant of Vinay's
suggestion, and with some shim code to address backwards-compatibility
concerns. Reviews welcome.

Ticket is at https://code.djangoproject.com/ticket/16288

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Logging configuration and handle_uncaught_exception

[Carl Meyer]

Hi Julien,

On 06/17/2011 12:08 PM, Julien Phalip wrote:
> Just noting that another "filtering" functionality has recently been
> added to trunk [1]. It is a different kind of filtering than what's
> being discussed here -- it is to filter out sensitive information from
> error reports when they're being produced. Maybe the naming of one of
> those functionalities might need to be reconsidered in order to avoid
> confusion in the docs and APIs. I'm not sure. Just thought I'd point
> it out ;)

Since "filters" are already a built-in feature of the logging package in
Python's standard library, and we're just making use of that existing
feature, we don't have the option of changing naming here: i.e., the
keys in LOGGING must be named "filters" or they just won't work with
logging.dictConfig. (Even if we could do it, it'd be a mistake to try to
impose alternate Django terminology on a feature of a Python standard
library component).

In any case, we already have plenty of uses of "filter" in Django:
there's the ORM .filter() method, and ModelAdmin.list_filter, just off
the top of my head. I don't think there's a problem here: "filter" is a
generic term, and if there's any potential for confusion it must be
qualified. "Error filtering" vs "logging filters" vs "admin changelist
filters", etc.

> (By the way, "production" is a common generic term used for when DEBUG
> is False)

True that it's used for that, but it doesn't exclusively mean that;
therefore I think DebugFalseFilter is a clearer name than
ProductionFilter. I would have to look at docs or implementation to be
confident I knew what the latter did; not so for the former.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Logging configuration and handle_uncaught_exception

[Carl Meyer]

Hi Matt,

On 06/17/2011 07:48 AM, Matt Bennett wrote:
> Since logging Filters are not specific to an individual logger or
> handler, I've just called it DebugFalseFilter. It's not a pretty name,
> but I couldn't come up with anything better - I decided it should
> convey what the filter allows through, but AFAIK there's no name for
> "not debug mode".

The name is fine - thanks for the patch! I realized there's still a bit
of a back-compat issue, I've commented on the ticket with a possible
solution. I'd like to get Russ' take on it, if he's ok with the plan I
can take it from here.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Logging configuration and handle_uncaught_exception

[Carl Meyer]

On 06/16/2011 08:47 AM, Russell Keith-Magee wrote:
> The behavior that is implemented in the 500 handler is a bit
> convoluted, but it was designed to be a drop-in replacement for the
> behavior that existed in 1.2 -- i.e., server error emails were not
> sent in DEBUG mode. I remember looking at this exact problem; as I
> recall, I was faced with the option of filtering in the handler (which
> would mean providing a email handler that wouldn't send emails under
> certain circumstances) or filtering at the logging call level. In the
> interest of making the admin email handler general purpose, I opted
> for the second option.
> 
> Now, I'm not absolutely bound to this specific implementation, only to
> the preservation of existing behavior without hobbling the email
> handler as a general purpose tool.

This is a case for a custom Filter object [1]. The filter object
implementation would only be a few lines, to reject logging when DEBUG
is True, and can be attached to the admin email handler in the default
logging configuration. [2] This way the logging call can occur in all
code paths, and the admin email handler itself can remain
general-purpose, but the backwards-compatible behavior can be maintained.

Matt, if you'd be willing to open a ticket for this, that'd be helpful.
If you feel like putting together a patch using the Filter approach,
that'd be even better ;-)

Carl

[1] http://docs.python.org/library/logging.html#filter-objects
[2]
http://docs.python.org/library/logging.config.html#dictionary-schema-details

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Undocumented feature for INSTALLED_APPS settings

[Carl Meyer]

On 06/12/2011 03:35 PM, Aymeric Augustin wrote:
> I checked the SVN history. This "feature" was never documented, even
> before the reorganization at r8506. It appears in
> django/conf/__init__.py when magic-removal is merged (r2809).
> 
> We just discussed it on IRC, and the consensus is that it dates back
> to before Django was open-sourced. It was probably only used at World
> Online.
> 
> In my opinion, it's an anti-feature: 1 - It's un-pythonic: in
> essence, it's equivalent to an filesystem-based implementation of
> "from  import *", which was not rejected in Python for a
> good reason [1] 2 - like "from  import *", it's not
> explicit, 3 - you don't add apps to your settings file every day, so
> there's little to gain.
> 
> I think it should be deprecated; since it was never documented, we
> could even remove it outright.

I agree with all of this. I would be in favor of simply removing the
wildcard feature with a note in the release notes, unless someone pops
up to argue that it's more widely-used than we think and it should be
deprecated instead, in which case I think a normal deprecation path is a
fine alternative.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Deprecation policy for IE6

[Carl Meyer]

On 06/09/2011 05:32 AM, Idan Gazit wrote:
> I'm looking at admin tickets, and I realize that some defined policy
> for when we can safely start to break IE6 would be very helpful.
> 
> I'd like to simply declare that going forward, the admin need not
> work perfectly in IE6. That leaves our support footprint for the
> Admin at "modern browsers" + IE>7.
> 
> * contrib.admin is contrib, and thus not covered by Django's
> deprecation policy
> 
> * This isn't a change which affects any other frontend product built
> with Django. The only audience this affects is users of the admin. I
> think it's reasonable to require administrative users to have IE7 if
> all they have is IE.
> 
> The admin is already using the HTML5 doctype (see
> https://groups.google.com/d/topic/django-developers/wJ9dnUDHUVI/ for
> background), but not any of the new HTML5 elements.
> 
> This change would mainly open up the ability to use PNGs and remove
> hacks and workarounds from admin CSS/HTML
> 
> Any objections?

Hearty +1 from me, for purely pragmatic reasons. In 2011, IE6 support is
simply an unreasonable burden to place on volunteer front-end
development work, IMHO. It's hard enough getting front-end work done
without tripling (quadrupling? more?) the pain factor like that. In my
mind, asking front-end developers to support IE6 is roughly similar to
asking Python devs to support Python 1.5, perhaps not in terms of usage,
but in terms of the additional development pain.

I think it needs to be stated clearly that the effective choice is
between maintaining IE6 support and making major improvements to the
admin. If someone wants to argue that admin IE6 support should be
maintained for another release, they should acknowledge that the
implication is that there probably won't be significant upgrades to the
admin UI for at least that long.

If there are Django deployments whose administrators really can't use
any browser other than IE6, Django 1.3 will be around as long as they
need it. It's not a reasonable tradeoff for that (frankly somewhat
ridiculous - IE6 is how many years old now?) edge case to continue to
hold the rest of the community hostage.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: 5-for-1?

[Carl Meyer]

Hi Stephen,

On 06/07/2011 02:37 AM, Stephen Burrows wrote:
> Hi - is the 5-for-1 deal still active on ticket reviews? [1]
> If so, I've reviewed the following tickets:
> 
> 3624
> 16152
> 16157
> 16158
> 16166
> 
> And would love it if someone could have a look at ticket 14082 [2].
> 
> If not, ah well. :-)

Thanks for the ticket reviews, and your work on the patch. Committed in
r16334 [1]

Carl

  [1] https://code.djangoproject.com/changeset/16334

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: support for custom django-admin commands written as packages

[Carl Meyer]

Hi Marwan,

On 06/05/2011 10:15 AM, Marwan Al-Sabbagh wrote:
> Hello,
>   Currently one can create a custom command by implementing a Command
> object in a python module such as
> "polls/management/commands/closepoll.py". It would be great if Django
> also supported these commands being implemented in a python package so
> in this case Command would be found in
> "polls/management/commands/closepoll/__init__.py". This would be
> useful when a command contains a lot of logic or code and the
> developer would like the ability to split up the logic into multiple
> files to make the code base more manageable. I faced this issue
> recently when I implemented a command that would be run as a cron job
> to periodically import data into Django from a legacy system that had
> some pretty hairy logic to deal with.
>I've already made the changes to support the feature, it only
> required a change to the function
> django.core.management.find_commands() to look for packages as well as
> modules in each commands directory. I thought I'd run it by
> django-developers before creating the ticket on trac with the patch.
> What do you guys think?

I'm not a big fan of the way Django finds management commands. Searching
directly on the filesystem makes Django apps with management commands
incompatible with legitimate builtin Python features such as import
hooks or zipfile imports. IMHO, regardless of what you think of those
Python features, this makes Django somewhat less than "just Python."

That said, I don't have a great alternative in mind at the moment, but I
don't like the idea of taking the current bad system and extending it
with even more complexity.

Generally my approach to the problem you describe is to make the
management command itself a minimal wrapper around the core code,
implemented as functions and classes within appropriately-named
submodules of the app itself, not within the
myapp.management.commands.mycommand namespace. I almost always at some
point want to reuse that code as library code, so I actually find it
preferable if the bulk of the code lives outside the management.commands
namespace, and is just imported and used by the management command. This
can also encourage a separation of responsibilities that makes testing
easier.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django and the new EU anti-cookie law

[Carl Meyer]

On 05/27/2011 08:18 AM, Hanne Moa wrote:
> "From 26th May 2011 websites in the UK need to ask for permission
> before they can set cookies not required for ‘essential’ means"
> 
> http://blog.silktide.com/2011/05/cookie-law-makes-most-uk-websites-illegal-what-you-need-to-know/
> 
> What cookies in Django are "essential"? When not logged in I see that
> the csrf-token cookie is set, and when logged in there's the
> session-cookie. Should one strive only to use csrf-protection (and
> thus the cookie) only when it is needed? Are there other cookies set
> by contrib apps? Should there be something about this in the docs?
> " cookies is essential, you need need to get consent for these
> other ones: "

Django itself sets only these cookies: CSRF, language, contrib.sessions,
and contrib.messages (if you're using one of the cookie-using backends).
I am not a lawyer, and I haven't read the new EU guidelines in depth.
Based on the description in the blog post you link, ISTM that CSRF and
language cookies are likely defensible as "essential" to the
user-requested service. I think the status of the session cookie (and
probably the messages ones as well) are very much dependent on how they
are used by the specific site in question.

I'd be fine with a documentation patch listing more clearly what parts
of Django may set cookies, to make it easier for Django users to comply
with this ruling.

I'd be opposed to any specific mention of this ruling in the patch, and
certainly opposed to any attempt in the documentation to define which
Django-set cookies are "essential" and which require user consent -
legal advice related to particular legal jurisdictions is WAY out of
scope for Django's documentation.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Template.origin

[Carl Meyer]

On 05/24/2011 11:53 AM, Jeremy Dunck wrote:
> I was wondering how people would feel about template origin being an
> attribute of the Template instance.
> 
> The point to me would be to make it easier to see which location a
> template is coming from; I think the compile_string func could do it--
> after constructing the template, the loader would annotate with
> origin.
> 
> It seems obviously useful, so I think there must be a good reason it
> wasn't already done.  :)

Looking at the code only briefly, it seems pretty reasonable to me.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Templatetag API for form rendering

[Carl Meyer]

On 05/24/2011 09:35 AM, Gregor Müllegger wrote:
>> Which templates are involved in a form layout?
> 
> This is not yet defined in detail. I would suggest a schema like:
> 
> forms/layouts//.html
> 
> Resulting in templates for the layout "table" like:
> 
> forms/layouts/table/row.html
> forms/layouts/table/label.html
> forms/layouts/table/field.html
> forms/layouts/table/errors.html
> forms/layouts/table/help_text.html

This looks like a pretty good starting point to me.

Since we're building on top of Bruno's templated widgets, I think it
would also be quite useful if you could override the default widget
templates in your layout, just by placing the properly named file at the
right location. So if a widget uses the template
forms/widgets/textarea.html, and you're using "mycustom" layout, it
would first look for a template at
"forms/layouts/mycustom/widgets/textarea.html".

> A note on something different: We haven't specified yet how it will look like
> to render just parts of a single field like errors. If we keep the
> {{ field.errors }} syntax (which is unlikely since the template variable has
> no access to the layout set by {% formlayout %}) or if we use an extra
> templatetag for that. A proposal for these details will follow after we have
> agreed on the higherlevel things like described in the RFC.

Yeah. I guess it would be possible to squeeze this into the {% form %}
(nee {% renderform %}) tag with some syntactical modifiers, e.g.:

{% form "myfield:errors" %}

Or even with a filter?

{% form "myfield"|fielderrors %}

But it may be cleaner just to do it as a separate tag.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Templatetag API for form rendering

[Carl Meyer]

Hi Gregor,

On 05/24/2011 09:25 AM, Gregor Müllegger wrote:
>
> Yes, defining a global default is really useful, we shouldn't skip that.

Yep, I'm definitely converted to that position ;-)

>> For the case-by-case override, though, I'd still much rather write
>>
>> {% form "table" %}
>>  {% renderform myform %}
>> {% endform %}
>>
>> than
>>
>> {% form %}
>>  {% formlayout "table" %}
>>  {% renderform myform %}
>> {% endform %}
>>
>>
>> What if instead of allowing form modifier tags to appear unenclosed, and
>> making them then implicitly global, we had a {% formdefaults %} tag that
>> paralleled the {% form %} tag, except it defined your defaults for form
>> rendering:
>>
>> {% formdefaults "table" %}
>>  {% widget ... %}
>> {% endformdefaults %}
>>
>> This is much more explicit, which means that a) a random new designer
>> reading your templates is more likely to notice that global defaults are
>> being defined, and b) you're less likely to accidentally define global
>> defaults because you omitted an enclosing block tag.
> 
> A question for my own understanding: The difference between your
> {% formdefaults %} variant and my "modify a global scope" is just a
> syntactical difference i.e. the following examples are equal?

That's right.

> (using your syntax)
> {% formdefaults %}
> {% widget ... %}
> {% endformdefaults %}
> 
> {% form %}
> {% renderform my_form %}
> {% endform %}
> 
> (using the RFC syntax)
> {% widget ... %}
> 
> {% form %}
> {% renderform my_form %}
> {% endform %}
> 
> And you propably want to raise a TemplateSyntaxError if the {% widget ... %}
> tag is used outside a {% formdefaults %} or {% form %} enclosed block?

Also correct.

The other difference is that if we have {% formdefaults %} we can
specify the layout as an argument to {% formdefaults %} as well as an
argument to {% formconfig %} or whatever we name it (presuming we listen
to Russ on that naming, and I think he's right), and that means we
really wouldn't need {% formlayout %} as a separate tag anymore. Which I
think is a plus.

> I'm not sure yet, if it's worth the extra {% formdefaults %} tag. Ofcourse it
> prevents template authors from excidentally overriding defaults, but I think
> who will use these utilities must know about these implications anyway. It's
> somehow just reflecting the behaviour of a python module scope.
> But maybe I'm a bit too much programmer here and reflecting too much
> of my habits
> onto the template authors mind.

There are some key differences between this and Python module scope,
besides the usual "template authors aren't necessarily Python
programmers." The main one is that Python module scope is contained
within a single file; here we're talking about a global scope that
potentially extends across many template files, what with inheritance
and includes, and even into Python code files too. I think because of
this action-at-a-distance factor, there's a much greater need to make
the modification of global scope explicit, clear, and hard to do by
accident or overlook.

It also makes it easy to "grep" for places where the global defaults are
being set. Otherwise, there'd be no easy way to find them and separate
them out from scoped uses of the modifier tags.

Given that this is something you'd likely only do once in your project,
not over and over again, I think the downside of typing the extra tag is
minimal.

> After all I have no objections against the {% formdefaults %} proposal and
> would be happy implementing it if thats your prefered way.

I do think it's clearly better, though if you disagree of course you
should argue, not just do it my way ;-)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: TemplateResponse and decorator_from_middleware

[Carl Meyer]

Hi Luke,

On 05/23/2011 05:14 PM, Luke Plant wrote:
> On 23/05/11 18:33, Carl Meyer wrote:
>>> It is not nearly as invasive as the other changes you suggested.
>>> However, it does fix the failing tests in my project (i.e. fixes
>>> csrf_protect), and leaves TemplateResponse objects unrendered as long as
>>> possible, without having to change one line of my own code/tests. And
>>> the implementation doesn't make me puke either.
> 
> Just re-read this, and realised it might come over as suggesting that
> your idea *did* make me puke, which isn't what I was saying at all - I
> was saying that this (limited) implementation of what was essentially
> your idea turns out quite nicely.

I didn't read it that way, but thanks for the clarification ;-)

>> Unfortunately, this doesn't do anything for the problem with cache_page
>> and Vary headers, which isn't related to TemplateResponse. I'm not sure
>> we can wait for Django 2000 to find a fix for that.
> 
> OK, but I think it is less pressing. #16004 is significant regression
> introduced in 1.3, which is now causing more bugs in trunk due to actual
> use of TemplateResponse. #15855 has existed since the dawn of time (0.91
> at least, as far as I can tell from eye-balling the code).
> 
> I'm happy to wait around for a fuller solution to be thrashed out, but I
> worried that the pursuit of an ideal might make it actually more fragile
> than it is. I think long term we should move towards a single processing
> phase (e.g. decorators always just mark for action later). But to
> attempt that in a backwards compatible way requires making functions
> that are eager now lazy (in effect). That addition of laziness is very
> likely to add more bugs - as the addition of TemplateResponse highlighted.
> 
> decorator_from_middleware is broken, but it has been broken for a long
> time, and its very likely that a lot is depending on its eager behaviour.

Yes, fair enough. I think your current patch is the best plan we have
for now, and it wouldn't make sense to hold it; I'll keep noodling on
the longer-term options.

>>> There was also the general
>>> assumption of just one view being involved, one response object. We
>>> don't want to break code that is slightly more inventive, like this:
>>>
>>> http://docs.djangoproject.com/en/dev/ref/contrib/csrf/#view-needs-protection-for-one-path
>>
>> I don't like seeing that recommended in the documentation. For one
>> thing, it's highly dependent on detailed knowledge of the implementation
>> of csrf_protect and csrf_exempt. The response is still going to pass
>> through both of those decorators; a naive reader would be as likely to
>> assume "last-one-wins" as anything else. And I find it quite hard to
>> read, compared to just having a function available that takes the
>> request as argument and performs the CSRF check directly and immediately.
> 
> It doesn't depend on anything in the implementation, as far as I can
> see. csrf_protect protects a view from CSRF attacks. csrf_exempt exempts
> a view from the checks the CSRF middleware performs. That is how they
> are documented, and that's all you need to know. That section of the
> documentation is pointing out a solution that you could deduce simply
> from the documentation of the parts.
> 
> I guess whether it is obvious depends on what your model is of how these
> work. I'd argue that by far the most obvious model is that the
> csrf_protect does what it says eagerly i.e. immediately, especially
> given that is supposed to work when the middleware is absent.

Yes, you're right here as well. I still think that pattern is ugly and
hard to read, but given the documented behavior of those decorators, its
function is clear.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: TemplateResponse and decorator_from_middleware

[Carl Meyer]

On 05/23/2011 11:31 AM, Luke Plant wrote:
> I think your core idea is quite interesting. I haven't had time to think
> through all the implications.
> 
> It does make the result of what is returned from view functions rather
> obscure, and this could affect things like testing.

I don't think it's any more obscure - an HttpResponse is still an
HttpResponse - just different. It changes the semantics of
decorator_from_middleware so you can't expect the middlewares to be
applied immediately without further response handling (more like actual
middleware).

> For example, the new tests I wrote for decorator_from_middleware, which
> pass with my first patch on #16004, will not pass with this
> implementation. This means the semantics are quite difference. It also
> means that tests using RequestFactory could be broken, because they
> won't pass through the machinery of request handling.

Yes, the semantics would be different, and this would break some current
tests that assume the current semantic of immediate application as part
of the view function call.

The entire issue, though, is that the current semantic of
decorator_from_middleware is broken, because middlewares can't reliably
be converted into something that runs immediately rather than in the
middleware phase, and still work properly.

One other option, I suppose, would be to introduce this as a new
function, delayed_decorator_from_middleware, and convert cache_page and
the csrf stuff to use it, to un-break them, without changing the current
behavior of decorator_from_middleware.

> However, with one modification, we can get something quite a bit better.
> Basically, if it is a TemplateResponse, we defer the process_response
> handling as you say. Otherwise, we do it the way we currently do it.
>
> In fact, we *almost* have machinery in place to do this -
> 'TemplateResponse.add_post_render_callback' - and I've got a patch based
> on that which works. The only problem is that currently
> 'process_response' is allowed to return a completely new response
> object, whereas that is not supported with add_post_render_callback.
> 
> That would be easy to change, however - or we could add a similar hook
> which does support it. Either way, it would require changing code like this:
> 
>  response.render()
> 
> to
> 
>  response = response.render()
> 
> This way, the only change people have to cope with in tests is adding
> 'response = response.render()' to things that have 'render', rather than
> having to invoke some other machinery to get the expected result -
> TemplateResponse has all the machinery itself. Since they already have
> to invoke 'render' for anything that has changed to TemplateResponse,
> this change adds no extra burden.
> 
> (BTW, we do need to document the change introduced by [16087] in the 1.4
> release notes as a breaking change - any tests against customized admin
> views that use RequestFactory will currently break if they don't add
> '.render()' calls.)
> 
> Putting all these things together, I've uploaded a new patch to #16004:
> 
> http://code.djangoproject.com/attachment/ticket/16004/16004.fix.alternative.diff
> 
> It is not nearly as invasive as the other changes you suggested.
> However, it does fix the failing tests in my project (i.e. fixes
> csrf_protect), and leaves TemplateResponse objects unrendered as long as
> possible, without having to change one line of my own code/tests. And
> the implementation doesn't make me puke either.
>
> I agree that we ought to be doing something about middleware ordering
> and things like that, but perhaps that is something for Django 2000 -
> when we can sit down and work out how everything should fit together.

This does seem like a relatively nice solution for TemplateResponse and
decorator_from_middleware. Isn't the patch missing a modification to
BaseHandler to do "response = response.render()" rather than just
"response.render()"?

Unfortunately, this doesn't do anything for the problem with cache_page
and Vary headers, which isn't related to TemplateResponse. I'm not sure
we can wait for Django 2000 to find a fix for that.


> One little note while I remember - I'm really not happy about decorator
> functions looking at settings to see how they should work - this is
> fixing fragility by adding more fragility. We want to be getting rid of
> this kind of thing, not adding more of it. 

I agree, of course. I only proposed that as a temporary deprecation-path
check, not a permanent behavior.


There was also the general
> assumption of just one view being involved, one response object. We
> don't want to break code that is slightly more inventive, like this:
> 
> http://docs.djangoproject.com/en/dev/ref/contrib/csrf/#view-needs-protection-for-one-path

I don't like seeing that recommended in the documentation. For one
thing, it's highly dependent on detailed knowledge of the implementation
of csrf_protect and csrf_exempt. The response is still going to pass
through both of those 

Re: TemplateResponse and decorator_from_middleware

[Carl Meyer]

Hi Luke,

On May 11, 7:00 pm, Luke Plant  wrote:
> Yeah, I guess I was seeing TemplateResponse as the culprit, on the basis
> of Last In First Out, but maybe it's decorators we need to be worrying
> about.
>
> I've actually documented a specific instance of #15855 in the CRSF docs

A possible fix for decorator_from_middleware just occurred to me this
morning. What if, instead of running process_response immediately, it
annotated the response with metadata saying "please run this
process_response middleware on me!" And then the running of that
middleware actually occurred later.

There'd be a couple possible variants of "later": right before your
actual middleware runs, right after, or somewhere in the middle (you
pick when). This latter option would have to be controlled by
something like a "ViewDecoratorMiddleware" which you put into your
MIDDLEWARE_CLASSES. I think this control might be necessary to get the
ordering right in some cases; there are other middlewares that should
always be first/last, so I don't think either "before all" or "after
all" would suffice. This ViewDecoratorMiddleware would also give us a
backwards-compatible way to introduce the change: if you've got this
ViewDecoratorMiddleware in your MIDDLEWARE_CLASSES,
decorator_from_middleware annotates. If you don't, it sticks with
current behavior (and raises a PendingDeprecationWarning).

I think we could even do something similar for process_view, if the
decorator annotates the actual view function with which additional
process_view middlewares should be run on it. Process_request can't be
helped; since request middleware can change the urlconf, thus
resolving to a different view, there's no way we can run request
middleware when it ought to have been run, if its applied as a
decorator. In any case, though, I think the most serious problems here
have to do with when process_response runs; process_request and
decorator_from_middleware doesn't seem as problematic.

It seems to me that this approach could potentially fix the cache_page/
Vary headers problem and the CSRF issues, make both of them work
nicely with TemplateResponse, and generally make
decorator_from_middleware less problematic.

Let me get a bit crazy now: If we want to fix the problem even more
fully, get rid of ViewDecoratorMiddleware, and allow full ordering
control of when the various middlewares-as-decorators are applied,
another option would be a way to include a specific middleware in your
MIDDLEWARE_CLASSES, but annotated as "decorator-only". So you'd go
ahead and include e.g. CsrfViewMiddleware in MIDDLEWARE_CLASSES but
marked decorator-only, and then it would only apply to requests/
responses where the resolved view has been annotated with the
csrf_protect decorator. Again, for backwards-compatibility,
decorator_from_middleware would check to see if you have the given
middleware in your MIDDLEWARE_CLASSES, marked decorator-only: if you
do, it would annotate, if you don't, it would run eagerly as it does
now.

Thoughts? Does something along these lines seem worth pursing as a
fix? Or am I off my rocker?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Templatetag API for form rendering

[Carl Meyer]

Hi Jonathan,

On 05/23/2011 04:30 AM, Jonathan Slenders wrote:
> 1. Like Carl said, I always prefer template tags which alter the
> context to create a scope. (I hate {% url ... as varname %})
> 
> {% form "table" %}
> {% renderform my_form %}
> {% endform %}

Well, in any case, not all context-altering tags will create their own
scope; even if we do ditch {% formlayout %}, {% widget %} still alters
context without creating a new scope. However, if my {% formdefaults %}
proposal is used, we'd at least maintain that the context-altering tags
can only be used within a scoped tag, either {% formconfig %} or {%
formdefaults %} (though in the latter case, although there's a scope
within which the modifications must be made, they have global impact).

> 2. Also totally agreed with Russell that we need consistency about
> when template tag parameters need to be quoted.
> That is, quoted in case of constants, and unquoted when they need to
> be resolved.

Yes. I'm still not entirely understanding which part of the RFC made it
appear that it didn't follow this rule. It already does.

> And even if this seems to cover all the use cases, ensure
> extensibility.

Definitely.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Templatetag API for form rendering

[Carl Meyer]

On 05/22/2011 08:54 PM, Russell Keith-Magee wrote:
> My argument: I'm trying to think of another example in Django's
> template language where the template tag is an "action" in this way.
> To my reading, outside of the tags used for logic (for, if, etc), and
> tags that define a contextual block (autoescape, comment, etc),
> template tags have the flavor of {% this block will be replaced with X
> %}, not {% do X here %}. For example, it's {% csrf_token %}, not {%
> render_csrf_token %}; {% cycle %}, not {% render_cycle_value %}; and
> so on.

Point taken. Yeah, renderform -> form does seem like the right thing. {%
formconfig %} doesn't quite sit right with me yet, but we can ponder
that name.

 Another tag to modify the rendering is the widget tag::

{% widget [] [using ] for
  [with = ...] %}

 In this syntax description  means that you can specify one of
 three things as argument:

 1. A bound field, means that the widget will be rendered only for this 
 field.
 2. A field class/type that will match for all fields that are using this
   formfield class. Example::

{% widget widgets.PasswordInput for formfields.CharField %}

   will render a  for all charfields.

 3. A field name (string). This will apply to all fields that have that
   particular field name. It's only useful if the option should apply to 
 more
   than one form or as convenience for short reference of a field.
>>>
>>> Broadly, I like what you've described here for the widget field. My
>>> only concern is a subtle one, regarding the way that quotes are
>>> interpreted.
>>>
>>> Over time, Django's template language has been slowly moving to a
>>> position where any user-specified argument can be interpreted as a
>>> variable or as a constant, with quotation used to differentiate
>>> between the two. For example, the {% url %} tag was modified in 1.3
>>> (using the future import syntax) so that the argument is interpreted
>>> literally if it is quoted, an as a variable if it isn't. This opens up
>>> all sorts of options for dynamic template programming, making the
>>> decision of where a link will land based on view logic, rather than
>>> template logic. Looking at your examples:
>>>
{% widget widgets.Textarea for my_form.comment %} (1. case)
{% widget widgets.DatePicker for formfields.DateField %} (2. case)
{% widget widgets.PasswordInput for "password" %} (3. case)
>>>
>>> In these examples, for X is being interpreted as a context variable in
>>> case 1, an interpreted class name in case 2, and a string that will
>>> match against a name in case 3.
>>>
>>>  * Case 1, but determining in the view which field will be a text area.
>>>  * Case 2, but determining the class that will be matched in the view.
>>>  * Case 3, but matching against a dynamic string (i.e., determine in
>>> the view the string that will be used for a match).
>>>
>>> Now, this isn't a case where I have an obvious use case I can point at
>>> -- these examples feel contrived, even to me. The third case is the
>>> only one that seems likely in practice. What I'm really arguing for
>>> here is consistency with the broad direction of the template language
>>> as a whole -- if you're specifying a constant, it should be quoted.
>>
>> Oh, I'm all about maintaining our newfound consistency in template tag
>> argument quoting. Which is why that was part of my feedback to the
>> original proposals, and Gregor and I spent quite a while working out the
>> solution which you apparently read a bit too quickly to notice ;-)
>>
>> If you read again, you'll note that "widgets" and "formfields" are not
>> some kind of magical syntactical marker, they are just template
>> variables (dictionaries of widgets and formfields, respectively) that
>> would be provided by a new context processor. So in every case above,
>> the argument is either a quoted string or a normal template variable;
>> there are no syntactic special cases.
> 
> I saw that explanation; I just wasn't sure how it would play out in
> practice. In particular, I wasn't sure how:
> 
>   {% widget widgets.PasswordInput for foo %}
> 
> would be interpreted, since foo could be a string, a field, or a
> widget class; 

It could be a string, a BoundField instance, or a Field subclass.

> or how
> 
>   {% widget widgets.PasswordInput for "foo" %}
> 
> would be interpreted as anything other than a string (i.e., is there a
> "constant" interpretation for the first two use cases).

No, if its a quoted string, then it's always a fieldname. That's why we
provide the "formfields" dictionary in the template context (via context
processor), so you have an easy way to pass in actual Field subclasses
and don't need any special "constant form" for that. (Same for the
"widgets" dictionary and the first argument; for that argument using a
quoted string would always be wrong).

> 
> Is the intention to run make the interpretation of 

Re: RFC: Templatetag API for form rendering

[Carl Meyer]

On 05/22/2011 07:22 PM, Russell Keith-Magee wrote:
> On Mon, May 23, 2011 at 6:21 AM, Carl Meyer <c...@oddbird.net> wrote:
>> Just had a quick conversation with Gregor and Chris Beaven on IRC;
>> based on a comment of Chris', we discussed the possibility of ditching
>> the {% formlayout %} tag in favor of specifying the layout as an
>> argument to the {% form %} block tag. E.g. instead of
>>
>>> {% form %}
>>> {% formlayout "table" %}
>>> {% renderform my_form %} {# will use table layout #}
>>> {% endform %}
>>
>> You'd say:
>>
>> {% form "table" %}
>>{% renderform my_form %}
>> {% endform %}
>>
>>> The description of the form tag implies that the modifier tags are able to
>>> set/modify the global state in the current template. That is something that 
>>> is
>>> explicitly wanted. This way you can set a "formlayout" in the head of your
>>> base template and any other extending template will have this default for 
>>> the
>>> form rendering.
>>
>> And perhaps if the layout can be specified that way, with minimal
>> boilerplate, we don't need the global-context-modifying version of
>> formlayout either.
>>
>> Not sure if Gregor was entirely convinced, but I think I'd probably
>> lean towards doing it this way.
> 
> I'm not sure I am convinced. It seems to me that there would be three
> common use cases for form rendering:
> 
>  1) Render this form using defaults
>  2) Render most forms using the defaults, but render for XXX using layout YYY
>  3) Render all forms on this form using YYY
> 
> It seems weird to me that in order to hit use case 3, you need to wrap
> your entire template in a {% form %} block. Having a global form
> layout context so that {% formlayout %} will work without an
> encompassing {% form %} block makes more sense to me.

I guess I was comparing

{% form %}
  {% renderform myform %}
{% endform %}

to

{% form "table %}
  {% renderform myform %}
{% endform %}

and thinking the latter didn't seem too comparatively onerous, even if
you were doing it for every form render. But I'd forgotten that for
simple cases you could otherwise just do {% renderform myform %} with no
block tag; it is unfortunate to require the block tag every time in case 3.

For the case-by-case override, though, I'd still much rather write

{% form "table" %}
  {% renderform myform %}
{% endform %}

than

{% form %}
  {% formlayout "table" %}
  {% renderform myform %}
{% endform %}


What if instead of allowing form modifier tags to appear unenclosed, and
making them then implicitly global, we had a {% formdefaults %} tag that
paralleled the {% form %} tag, except it defined your defaults for form
rendering:

{% formdefaults "table" %}
  {% widget ... %}
{% endformdefaults %}

This is much more explicit, which means that a) a random new designer
reading your templates is more likely to notice that global defaults are
being defined, and b) you're less likely to accidentally define global
defaults because you omitted an enclosing block tag.

Global state is serious business - it means action-at-a-distance. It
should be obvious when it's happening, and hard to do accidentally.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Templatetag API for form rendering

[Carl Meyer]

On 05/22/2011 07:18 PM, Russell Keith-Magee wrote:
> I like this. Simple, covers all the common use cases that I can see.
> My only feedback here would be mostly bikeshedding -- the fact that {%
> form %} is a configuration action, and {% renderform %} is the
> rendering action. It feels to me like {% form %} should be the
> rendering action, and {% formconfig %} (or something similar) should
> be the configuration action.

Hmm. In the current proposal, {% form %} isn't really an action at all,
it's just a scope. Which is why a generic name seemed sensible for it.
Whereas {% renderform %} is an action - it renders the form - and it
seemed good for its name to be explicit about that action.

I don't care too much about that naming, though. If the "common case" is
just {% renderform my_form %} with no enclosing configuration scope at
all, it would be nice to keep that common case as short as possible.

>> Another tag to modify the rendering is the widget tag::
>>
>>{% widget [] [using ] for
>>  [with = ...] %}
>>
>> In this syntax description  means that you can specify one of
>> three things as argument:
>>
>> 1. A bound field, means that the widget will be rendered only for this field.
>> 2. A field class/type that will match for all fields that are using this
>>   formfield class. Example::
>>
>>{% widget widgets.PasswordInput for formfields.CharField %}
>>
>>   will render a  for all charfields.
>>
>> 3. A field name (string). This will apply to all fields that have that
>>   particular field name. It's only useful if the option should apply to more
>>   than one form or as convenience for short reference of a field.
> 
> Broadly, I like what you've described here for the widget field. My
> only concern is a subtle one, regarding the way that quotes are
> interpreted.
> 
> Over time, Django's template language has been slowly moving to a
> position where any user-specified argument can be interpreted as a
> variable or as a constant, with quotation used to differentiate
> between the two. For example, the {% url %} tag was modified in 1.3
> (using the future import syntax) so that the argument is interpreted
> literally if it is quoted, an as a variable if it isn't. This opens up
> all sorts of options for dynamic template programming, making the
> decision of where a link will land based on view logic, rather than
> template logic. Looking at your examples:
> 
>>{% widget widgets.Textarea for my_form.comment %} (1. case)
>>{% widget widgets.DatePicker for formfields.DateField %} (2. case)
>>{% widget widgets.PasswordInput for "password" %} (3. case)
> 
> In these examples, for X is being interpreted as a context variable in
> case 1, an interpreted class name in case 2, and a string that will
> match against a name in case 3.
> 
>  * Case 1, but determining in the view which field will be a text area.
>  * Case 2, but determining the class that will be matched in the view.
>  * Case 3, but matching against a dynamic string (i.e., determine in
> the view the string that will be used for a match).
> 
> Now, this isn't a case where I have an obvious use case I can point at
> -- these examples feel contrived, even to me. The third case is the
> only one that seems likely in practice. What I'm really arguing for
> here is consistency with the broad direction of the template language
> as a whole -- if you're specifying a constant, it should be quoted.

Oh, I'm all about maintaining our newfound consistency in template tag
argument quoting. Which is why that was part of my feedback to the
original proposals, and Gregor and I spent quite a while working out the
solution which you apparently read a bit too quickly to notice ;-)

If you read again, you'll note that "widgets" and "formfields" are not
some kind of magical syntactical marker, they are just template
variables (dictionaries of widgets and formfields, respectively) that
would be provided by a new context processor. So in every case above,
the argument is either a quoted string or a normal template variable;
there are no syntactic special cases.

Our idea was that there would be public API for adding custom widgets
and formfields into these dictionaries, so that a third-party app can
make its own custom widgets and fields available for use in your
templates without requiring you to add yet another context processor.
I'm realizing now that we need to think carefully about the implications
of introducing a global flat namespace for widgets and formfields like
this; it could cause some nasty bugs if apps are stomping on each
others' names, or even built-in field/widget names. Maybe requiring an
additional context processor for any third-party app that wants to
provide custom fields and widgets into the template context isn't such a
bad idea after all.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to 

Re: RFC: Templatetag API for form rendering

[Carl Meyer]

Just had a quick conversation with Gregor and Chris Beaven on IRC;
based on a comment of Chris', we discussed the possibility of ditching
the {% formlayout %} tag in favor of specifying the layout as an
argument to the {% form %} block tag. E.g. instead of

>     {% form %}
>         {% formlayout "table" %}
>         {% renderform my_form %} {# will use table layout #}
>     {% endform %}

You'd say:

{% form "table" %}
{% renderform my_form %}
{% endform %}

> The description of the form tag implies that the modifier tags are able to
> set/modify the global state in the current template. That is something that is
> explicitly wanted. This way you can set a "formlayout" in the head of your
> base template and any other extending template will have this default for the
> form rendering.

And perhaps if the layout can be specified that way, with minimal
boilerplate, we don't need the global-context-modifying version of
formlayout either.

Not sure if Gregor was entirely convinced, but I think I'd probably
lean towards doing it this way.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Composite fields API

[Carl Meyer]

Hi Michal,

I'm looking forward to seeing this project take shape! Comments below:

On 05/12/2011 06:41 AM, Michal Petrucha wrote:
[..]
> The constructor of a CompositeField will require at least two
> positional parameters, each positional parameter will be a single
> atomic field. The order of this parameters will be important as
> explained below. The parameters will have to be field instances, lazy
> loading won't be necessary (the recommended place of composite field
> definitions will be after atomic fields).

This sounds fine.

> CompositeField will accept these three field options:
> - db_index (creates a multi-column index across the underlying fields)
> - primary_key (creates a composite primary key in the model)
> - unique (creates a unique constraint for the set of fields)
> 
> Other field options either wouldn't make sense or would be too
> difficult to implement.
> 
> There is a clash with the current API here, in the ``unique`` option.
> This would supersede the current ``unique_together`` Meta option. I
> see three options possible:
> 
> 1) Leave out the ``unique`` option and live with ``unique_together``.
>This would pribably imply also leaving out ``db_index``, otherwise
>the API would be a complete mess.
> 
> 2) Allow ``CompositeField.unique`` but also keep ``unique_together``.
>The problem I see with this approach is that there would be two
>quite different ways to achieve the same effect.
> 
> 3) Make ``CompositeField.unique`` the way to go and deprecate
>``unique_together``.
>This way, specifying a unique constraint on a tuple of fields would
>work the same way it works on single fields which is IMO a
>significant benefit. There's, however, the issue of breaking
>backwards compatibility. Furthermore, one would have to add a new
>field, albeit virtual, just to create a simple constraint, which
>may seem weird to some.

I agree with Javier - I favor option 2. In my mind, although the final
result at the database level may be the same (a unique index across
multiple database columns), in conceptual terms at the ORM level it is
really two different things. There are many cases where I want to
specify that two fields should be unique together, but they really are
two separate fields; I'm never going to want to access it as a single
field or composite value. In this case, specifying a CompositeField
would confuse the intent and be more verbose than unique_together. I
think the conceptual distinction is clear, and it will actually be less
confusing to users to have both options available than to have
CompositeField become the only way to specify an index on multiple columns.

> One minor detail, should the field silently ignore invalid options or
> should it issue warnings?

Explicit is better than implicit, and errors should never pass silently
unless explicitly silenced. If the option is invalid, it should not just
be a warning, it should be an outright failure (though if the check is
expensive, it could possibly happen in model-validation rather than
always at runtime).

> The value of a CompositeField will be represented by an instance of a
> CompositeValue class. This will be a descendant of tuple and will
> resemble namedtuple present in Python >= 2.5. It will support
> iteration, numbered indexing and access to individual field values
> using attributes corresponding to underlying field names. The order of
> values will be the same as the order of fields specified in the model
> definition.

Yes, Tom is right of course - now that Python 2.5 is minimal version, we
can just use namedtuple.

> Assigning a value to a CompositeField will be possible using any
> iterable as long as its length equals the number of atomic fields (and
> the values can be assigned to the corresponding fields, obviously).

I mentioned this in an earlier thread, but I'd really like to see the
API allow me to specify my own class as the value class, as long as it
satisfies some basic API requirements. In my mind the long-term goal
here is that GFKs should be reasonably implementable as a CompositeField
or a CompositeField subclass without exploiting undocumented internal APIs.

If there are implementation complexities that push this feature out of
scope for GSoC, that's fine - but I want to make sure we don't make that
future expansion difficult by design choices we make now.

[...]
> I'm also thinking about implementing an abstract class, VirtualField.
> This could be useful mainly as a base class for fields with no direct
> database column. That means, it would mainly handle things like
> add_to_class (adding itself to the list of virtual fields instead of
> local ones), specifying arbitrary lookup filters when asked for one
> etc. CompositeField could then be a descendant of this class.
> 
> However, I can't currently imagine any other use-case for this
> abstract class than CompositeField. The question is, then, is there
> any interest in having an abstract 

Re: TemplateResponse and decorator_from_middleware

[Carl Meyer]

On 05/11/2011 05:37 PM, Luke Plant wrote:
> The only solution I can see that isn't awful is providing two views - a
> default and documented one that has all the right decorators applied,
> and one that is obviously 'use at own risk', called something like
> 'login_unsafe', for which we document the extra things you need.
> 
> I can see even less room for process_template_response - it just isn't
> useful to add a middleware for the sole purpose of messing with the body
> of any response, only to find that in many cases you can't mess with
> them after all, since the developer had to do something like add caching
> to a particular view etc.
> 
> So where does this leave us? I don't know. I did have some bad feelings
> about TemplateResponse and the idea of lazy responses, but didn't voice
> them at the time because I didn't have anything but a vague feeling of
> unease (which is not something you can either defend or argue against),
> and I wasn't involved in reviewing that feature much.

Yeah, this is bad. I don't have a lot to add at the moment, except to
also point out #15855 - cache_page as a decorator breaks proper setting
of Vary headers.

The reason I bring it up here is just to point out another case,
unrelated to TemplateResponse, where we promote the idea that you can
use a decorator as a view-specific equivalent to a middleware, but doing
so breaks the semantics of the original middleware in subtle but
significant ways (because it changes the order things happen in).

Not sure what that implies, except that maybe decorator_from_middleware
and the pattern it encourages is ultimately at least as problematic as
TemplateResponse...

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #1342 Allow customization of MAX_SHOW_ALL_ALLOWED. Reopen or new ticket?

[Carl Meyer]

Hi Jim,

On 05/10/2011 02:09 PM, Jim D. wrote:
> I'm looking at a five-year-old ticket here (http://
> code.djangoproject.com/ticket/1342) that suggests MAX_SHOW_ALL_ALLOWED
> in the admin be configurable. As of now it is hard coded at 200 in
> contrib/admin/views/main.py .
> 
> The ticket is, in my opinion, somewhat erroneously marked as "fixed",
> as someone pointed out 14 months ago. The patch that supposedly fixed
> it is the one that added the list_per_page attribute; however, that
> patch only addressed half of the issue (ability to configure results
> per page) and not the other half related to the show all.
> 
> Clearly this is not an issue of overwhelming importance or I'd imagine
> it would have already come up in the last five years. Be that as it
> may, I have an important use case where I'd like to preserve
> pagination but allow for a show_all link to appear, which can only be
> accomplished gracefully by changing the MAX_SHOW_ALL_ALLOWED. It also
> strikes me as bad form to hard code an arbitrary value like that
> without providing any recourse to amend it.

Agreed.

> First question is administrative -- should I reopen this old ticket or
> start a new ticket?

I think I'd open a new one, in this case, with reference to the existing
one.

> Second question is how to approach the patch. Is it acceptable to move
> this to global_settings under an "Admin" section at the bottom and
> leave it undocumented? Given that it's already undocumented and there
> are other settings in global_settings that are also undocumented, this
> would seem to be the quickest approach that would hopefully ruffle the
> fewest feathers. If that's not kosher, I'd be happy to contribute a
> patch with full documentation if need be, or to take a different
> approach, e.g. make this configurable at the site or model admin
> level.

I think having it configurable at (ideally) both the admin-site and
model-admin level (and having that documented) would be much better than
a new setting.

> Third question, is there some reason this issue has not been fixed
> before or should not be fixed? Mostly I just want to make sure this
> issue has not been discussed before and some conclusion reached that
> I'm not aware of (searched, but couldn't find anything).

You've searched the mailing list and the ticket tracker - that's due
diligence. I haven't been around as long as some, but I don't know of
any discussions about this. I think it's just been low priority and
overlooked.

> If anyone has any thoughts regarding any of these questions I'd
> appreciate hearing them. My main objective is to find the most direct
> path to something that you guys will find acceptable. Thanks.

Thanks for contributing!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: model fields options

[Carl Meyer]

Hi Eduardo,

On 05/06/2011 12:22 PM, legutierr wrote:
> You're probably right about this, but (while we are on the subject)
> aren't there some things that shouldn't be part of the model field
> options that currently are?  Why is help_text part of the field
> definition?  This is a ui-specific thing--what does it have to do with
> the database?  All abstractions are leaky, sure, but this seems
> inappropriate.  The same thing goes for editable, error_messages:
> these options are not part of the ORM, they are parts of the forms
> subsystem that have somehow ended up in the ORM.  Why should the ORM
> know anything about forms, or any other part of Django for that
> matter?

I think error_messages may actually belong in the ORM now, since
model-validation. Totally agreed that editable and help_text don't
belong there - I think the fact that they're still there is just a
matter of historical inertia, and that nobody's been sufficiently
motivated to deal with the backwards-compatibility difficulties to
change it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: deprecation of AUTH_PROFILE_MODULE

[Carl Meyer]

On 05/02/2011 11:38 AM, Jeremy Dunck wrote:
> Given a blank slate, what would Auth look like these days?  And can we
> work towards that?

Now, why you gotta get all constructive and forward-thinking and stuff? ;-)

Here's my list of core ideal-contrib.auth desiderata that I keep seeing
crop up in related threads:

1. A specification of the minimal useful interface for a User (perhaps
in the form of an abstract base model?)

2. A way to provide your own User class that extends the minimal spec,
in a single table (i.e. MTI need not apply).

3. A way for reusable apps to access and point foreign keys at the User
model (solutions that involve magically transporting models into other
modules need not apply - Django's been there, not going back).

4. A migration path that meets our backwards-compatibility standards.

Abstract base models already can handle 1 & 2, but if we take that
approach we still lack solid proposals for 3 & 4. And of course there
are many questions left untouched here (groups? permissions?), but these
are the core elements that I don't think I've heard much, if any,
disagreement on. I'd be interested to know if anyone sees a problem with
any of the above, or what people would add as core requirements.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: deprecation of AUTH_PROFILE_MODULE

[Carl Meyer]

On 05/02/2011 07:15 AM, Russell Keith-Magee wrote:
>>> what do you think about deprecating AUTH_PROFILE_MODULE and .get_profile()
>>> or removing the suggestion to use it from the docs in 1.4 release?
>>> There are broader issues with extending User model but I think this one
>>> can be handled separately.
>>
>> -1 on deprecating it before having something better to replace it.
> 
> Completely agreed. get_profile may not be ideal, but it's better than
> providing nothing, and cleaning up get_profile will be a logical
> extension of any good auth.User refactor.

Deprecating get_profile() is pretty low on my priority list, but I'm a
bit confused by these two comments. What exactly does get_profile()
offer that needs to be "replaced" or is actually "better than providing
nothing"? Or, more specifically, better than recommending a
OneToOneField and the usual ORM accessor (even leaving aside the
multiple-profiles case)? In every way that I can see, it is worse than
that: it's a special case, requires additional cognitive overhead to
learn, more characters to type, more runtime processing, more code in
Django to maintain, and adds zero value.

The only possible argument for it I can see is that it provides a
standardized way for reusable apps to get at a project's user profile
model. But given that no assumptions can be made about what is on the
profile model, the range of useful uses for that is pretty much limited
to one case: providing some pre-built URLs etc for creation and editing
of profiles; i.e. django-profiles. And even that case gains nothing from
AUTH_PROFILE_MODULE being a built-in feature of Django as opposed to an
external setting documented by django-profiles itself.

I don't mind leaving get_profile() around, I guess, but IMO the best
argument that can be made for that is "it's useless but mostly harmless,
and it's not worth making people change their code to remove it." I'll
continue to recommend in IRC and elsewhere that people avoid using it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ModelForm validation - a better way?

[Carl Meyer]

On 04/29/2011 10:02 AM, Yishai Beeri wrote:
> Of course, cleanup need not be simplistic. In fact, I think the common
> coder would never expect a CM to actually save an object on __exit__ -
> and will be surprised by the proposed behavior.

Could be - the name "finish()" was intended to give the impression that
the purpose of the CM was to wrap up everything the modelform needs to
do. That currently includes saving the object.

I'm open to the idea that we change the name of the CM to
"form.validate()" and it never saves the object; you have to call
obj.save() yourself. In a way, it feels like making users of the API do
extra work for no good reason, and opening the door to mis-use of the
API (since all changes to the object should be completed within the body
of the context manager), but perhaps this is worth it to avoid
unexpected behavior.

For reference, here's where we'd be in that case (I still prefer the
context manager over the idea of two separate calls to something named
"validate"):

def my_view(request):
form = MyModelForm(request.POST or None)
try:
with form.validate(tweak=True) as obj:
obj.user = request.user
except ValidationError:
pass
else:
obj.save()
return redirect(obj)
return render(request, "foo.html", {"form": form})


Or in the simple case, where no modifications to the object are needed:


def my_view(request):
form = MyModelForm(request.POST or None)
try:
obj = form.validate()
except ValidationError:
pass
else:
obj.save()
return redirect(obj)
return render(request, "foo.html", {"form": form})


Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ModelForm validation - a better way?

[Carl Meyer]

Hi Johannes,

On 04/29/2011 09:02 AM, Johannes Dollinger wrote:
> Here's my take on the API:
> 
> def my_view(request):
> form = MyModelForm(request.POST or None)
> try:
> with form.finish() as obj:
> obj.user = request.user
> return redirect(obj)
> except ValidationError:
> return render(request, "foo.html", {"form": form})
> 
> The context manager would validate the form on __enter__  (only
> included fields) and raise ValidationError if it does not validate.
> On __exit__, it would raise ValidationError if model validation
> fails. This let's you defer expensive computations/queries until
> after a sanity check. 

I like this. It's a little weird that we re-use ValidationError for
something rather different than its usual use (normally it represents a
specific validation error and carries data about that error - in this
case we're effectively re-raising a sort of meta-ValidationError to
represent that the entire form has failed validation. Actually I think
that's probably fine, it just took me a moment to think through).

I think it would also be more correct to use try-except-else, and put
the success-case handling in the else clause.

It looks like this view is supposed to handle both GET and POST, so I
guess your assumption is that the context manager would also raise
ValidationError in case of an unbound form? That feels odd, but it
certainly simplifies the view code. The more-explicit alternative would
be something like this:

def my_view(request):
form = MyModelForm(request.POST or None)
if request.method == "POST":
try:
with form.finish() as obj:
obj.user = request.user
except ValidationError:
pass
else:
return redirect(obj)
return render(request, "foo.html", {"form": form})

That's not too bad either.

Optionally, finish() might take a flag to defer
> all validation until __exit__, which may be required if you want to
> display all validation erros at once. 

Yes, I think that would be useful.

As an added benefit, this makes
> dealing with multiple forms and formset in a single view straight
> forward, as you can simply add more `with` blocks inside the try
> block, and validation across forms can simple raise a ValidationError
> on its own.

Yes - in the cross-form-validation case, the ValidationError might want
to carry a message for the template, but I suppose the developer can
handle that themselves.

I like this direction. I guess the main objection here might be that it
requires try-except handling for every form-handling view. I don't
necessarily consider that a problem: there's something that feels right
about exception-handling, rather than if-else clauses, as the idiom for
managing form validation errors. (Clearly it has something to recommend
it, since that's already the way we handle the details of validation
internally).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ModelForm validation - a better way?

[Carl Meyer]

Hi Yishai,

On 04/29/2011 08:53 AM, Yishai Beeri wrote:
> First, the logic tied into the context manager does not match the
> idiomatic use of context managers in Python. One expects a context
> manager to "clean up" on exit, rather than *save* a new object. 

I'd argue it's not totally off base. When the action you're performing
in the context manager is "tweaking this model instance before its
saved", I think it's reasonable to consider "save it if it validates,
otherwise populate form._errors" to be appropriate and expected
"cleaning up" from that action.

> For
> instance, what if I throw an exception inside the with block? The idiom
> tells us the object will be saved - but in this suggested approach I
> probably want the opposite. 

The fact that context managers imply cleanup doesn't mean that cleanup
has to be defined simplistically ("save the object no matter what"). The
context manager performs appropriate cleanup. That might be saving, or
populating form._errors, or (if you raise an exception inside the body)
probably neither.

Also unclear is what happens if the form
> fails validation (inside the __enter__ on form.finish); an exception?

In the original proposal there would be no form validation on __enter__,
only full validation on __exit__. The proposal could be modified to do
both - that gets into the territory of your and Johannes' alternative
proposals, which are interesting.

> Second, and this is a general issue underlying partial validation -
> probably part of what makes this issue so hairy - the full-model
> validation, and the resulting error messages, run the risk of being
> pretty remote from what the user actually did. It feels to me that form
> validation needs to be a step that focuses on values the user entered in
> the form, and that full-model validation should come as a second step,
> possibly adding more messages and tying them to particular form
> elements.

It would have to be up to the developer tweaking the model instance to
ensure that they don't do so in a way that results in validation errors
that are confusing to the user or that the user can't fix. This is
really no different from the current situation, where if you tweak the
model before saving you're responsible to avoid IntegrityError.

That said, I do see reasons why it would be nice to have the partial
sanity check of the current style of form validation before doing the
extra work that might be involved in tweaking the model for form
validation. Both your and Johannes' proposals do that.

 I think in many cases the two types of validation deserve
> separation in code; model validation might need to be more expensive
> (e.g., hit the DB), 

Already, modelform validation can itself just as easily hit the DB, if
you have unique constraints involving fields included in the form.

and obviously model validation cannot rely on
> per-field idioms like form validation does.

I'm not sure what you mean by this last bit - model validation and
modelform validation are actually pretty close to the same thing in the
current code (modelform validation calls model validation and catches
any ValidationErrors), with the exception of possibly excluding some
fields (and doing additional form-specific validation).

> As a very rough sketch, perhaps something along the lines of:
> 
> try:
>form.validate()
>obj = form.save(commit=False)
>obj.user = request.user
>obj.validate(form=form)
>obj.save()
>redirect(obj)
> except FormValidationError, ModelValidationError:
>redisplay the form
> 
> of course, the above can also be written with a couple of if/else
> clauses, but perhaps routing validation errors as exceptions may feel
> more natural in Python.
> 
> I don't really like the obj.validate(form=form) line; perhaps it needs
> to be form.validate_object(obj), or even a context manager such as:
> 
> with form.validation():
>form.validate()
>...
>obj.validate() # having the form.validation CM allows the object
> validation to somehow tie the errors to the form fields, if so desired
>...

This ends up being pretty similar to Johannes' idea - I'll reply to his,
feel free to note any differences you think are important.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

Hi Lior,

(moved from another thread)
On Apr 29, 12:16 am, Lior Sion  wrote:
> I looked at the sample you wrote on the other thread (unique together
> on username and date, and having a null username with a given date) of
> when the old behavior is the right one and it didn't quite convince me
> - I do believe that the right implementation would fail a case of
> NULLed username and repeating dates, when a unique together exists.

That would clearly violate the currently-established consistent
semantic for modelform validation, which is that the modelform only
validates the fields that are actually included in the form. Any
fields that are not included in the form, it must be assumed that they
might change before saving, and their current value is unreliable (it
might just be the default value for a new object, or something else
entirely, it's not necessarily a value that's actually intended to be
validated or saved). In essence, what you are proposing is validation
of "possibly-junk" data.

Because of this, any fix has to be absolutely 100% sure that it will
never generate a false positive on the unique_together check because
of the non-included field's value, or else we're violating the
documented expectation that non-included fields' values are not part
of validation. The exceptions for default and blank in the current
patch was my best attempt to achieve that, but I don't even think
that's sufficient - really, modelform validation can't assume anything
about the data on self.instance for fields that aren't in the form,
regardless of whether the field is blank or has a default.

This is why I'm saying that the current expected semantics, that a
modelform can provide meaningful validation while ignoring some fields
of the model, is inherently broken. No matter what you do with
constraints that include some included and some excluded fields
(whether they are unique_together or custom clean methods, as in
Mikhail's example), you're doing the wrong thing - you can't validate
the constraint because you can't use some of the data that's needed to
validate it, but if you drop the constraint entirely (as we do now)
you require the developer to re-validate everything themselves later
(and probably stuff those later model-validation errors back into the
form, too, and then break out of the is_valid clause because its no
longer valid), which is really ugly and painful in custom code and
currently not done at all in the admin.

And that's why I think the only long-term solution, that isn't just
going to cause more problems, is to begin moving towards a new
expectation for validating modelforms, where its expected that full
model validation is run, and any tweaks to the model must be made
before validation rather than after.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

Hi Mikhail,

On Apr 24, 7:46 am, Mikhail Korobov  wrote:
> The issue is not only with unique_together indeed. Please correct me if I'm
> wrong, but it seems there is no way currently to use model validation with
> fields dependent on each other when one of these fields is not POSTed
> directly by user. Example:
>
> # models.py
> class Ticket(models.Model):
>     created_by = models.ForeignKey(User, related_name='created_tickets')
>     responsible = models.ForeignKey(User, related_name='todo')
>
>     def clean():
>         # we don't want to allow tickets where created_by == responsible for
> some reason
>         from django.core.exceptions import ValidationError
>         if self.created_by == self.responsible:  
>             raise ValidationError('Responsible is incorrect.')
>
> # views.py
> class TicketForm(forms.ModelForm):
>     class Meta:
>         model = Ticket
>         exclude = ['created_by']
>
> @login_required
> def add_ticket(request):
>     form = TicketForm(request.POST or None)
>     if form.is_valid():
>         ticket = form.save(commit=False)
>         ticket.created_by = request.user
>
>         # todo: handle ticket.full_clean()
>
>         ticket.save()
>         return redirect('ticket_list')
>     return TemplateResponse(request, 'tickets/add.html', {'form': form})
>
> Model.clean method is always called on form validation (unlike field
> validators that are excluded if field is excluded from form). And we can't
> write validator for 'responsible' field or for 'created_by' field (so that
> it will be excluded from validation on form.is_valid) because this validator
> won't have an access to other model fields. So the only option here seems to
> move validation from model level to form level.
>
> Please consider this use case while rethinking model validation. The
> proposed context manager seems to be a very good idea and I hope it will fix
> this issue as well.

Sorry, I missed this somehow earlier. I agree with you that this is
another example of why attempting to do partial model validation is
simply broken. I think the context manager proposal (which I've posted
in a new thread) does fix this case as well, and I'd be happy for your
comments on it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ModelForm validation - a better way?

[Carl Meyer]

Hi Lior, thanks for commenting.

On 04/29/2011 12:16 AM, Lior Sion wrote:
> I think that forcing programmers to remember quite a long process of
> for validation each time is a wrong approach, especially if only done
> to support backward code that behaves in "unnatural" way.

I'm not sure why you think it's "quite long," other than that one of the
examples above included a full view and the other one didn't. For
reference, here are equivalent side-by-side examples for the case where
you don't want to modify the object before saving:

current:

if form.is_valid():
obj = form.save()
redirect(obj)

proposed:

obj = form.finish()
if form.is_valid():
redirect(obj)

And for the case where you do want to modify the object before saving:

current:

if form.is_valid():
obj = form.save(commit=False)
obj.user = request.user
obj.save()
redirect(obj)

proposed:

with form.finish(tweak=True) as obj:
obj.user = request.user
if form.is_valid():
redirect(obj)

(This is assuming we use a flag argument to determine whether you want
the context manager. I'm not a huge fan of this, but I like it better
than requiring the context manager and using "pass").

The proposal here is the same length as current code in the first case,
and one line shorter in the second case. So there may be valid
objections to the proposal, but I won't accept length as one of them ;-)

> I looked at the sample you wrote on the other thread

I'm happy to continue conversation on that, but I'll do it in the other
thread for clarity.

Thanks,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

ModelForm validation - a better way?

[Carl Meyer]

Hi all,

We have a number of tickets open (at least #12028, #13249, #13091,
#15326, and #15860 -- #13091 is the active one) reporting problems with
unique_together constraints in our attempts to validate arbitrary
partial models, when validating a ModelForm with some fields excluded.

Eduardo Gutierrez and I have spent some time looking at this problem
recently, and my main feeling is that validation of arbitrarily partial
models in the face of unique_together constraints has no reliably right
answer, and we'd do better to move away from it altogether.

Fortunately, I think we can do better. The reason we have to validate
partial models is because of this idiom:

if form.is_valid():
obj = form.save(commit=False)
obj.user = request.user # for instance
obj.save()

But there is no reason those tweaks to the model have to happen after
form validation. If we encouraged an idiom where the tweaks happen
before form validation, we could just run full model validation and
avoid all the error-prone complexity of validating partial models.

Alex and I talked over some possibilities for a context manager
available from a new method on ModelForms, that you'd use like this
(idea originally from Łukasz Rekucki [1], somewhat modified):

def my_view(request):
if request.method == "POST":
form = MyModelForm(request.POST)
with form.finish() as obj:
obj.user = request.user
if form.is_valid():
return redirect(obj)
else:
form = MyForm()
return render(request, "foo.html", {"form": form})

form.finish() returns a context manager which returns form.instance from
its __enter__ method, as "obj" here, allowing the user to do any
tweaking they like within the body of the context manager. On exit, the
context manager performs _full_ model validation. Any validation errors
are saved to the form, as usual. If validation succeeds, the model
instance is saved.

The following check to form.is_valid(), then, is just for purposes of
managing view logic (redirect, or redisplay form?). Actual validation
has already happened, so it would just be checking self._errors (this
isn't a change, that's already how .is_valid() works).

For backwards compatibility, there would be no change to the existing
behavior if you don't use the new .finish() context manager -
form.is_valid() would trigger possibly-partial validation just as it
does now, and do the best it can. But the admin could be converted to
use the new idiom (with a new ModelAdmin method that would be called
from within the context manager to allow model-tweaking before
validation), which would solve the admin-related bugs. And the
documentation could recommend the new idiom over the old, particularly
for incomplete modelforms with unique_together constraints.

Open questions:

1. What if you don't need to tweak the instance, but you do want to
trigger the new full validation behavior (i.e. your form excludes some
fields, but you initially passed an instance to the form that you're
confident has the excluded fields already set correctly - this would be
the case for e.g. the parent FK in admin inlines)? Perhaps form.finish()
could take an argument that determines whether it returns the context
manager or just returns form.instance directly?

2. Is it going to be too weird for people to adjust to the idea that
they get their model instance out of the form before they (explicitly)
call form.is_valid()?

Other issues we've overlooked, or ways this could be improved? Use cases
this doesn't handle?

Thanks,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

On 04/28/2011 07:12 PM, legutierr wrote:
> I'm up for working on the new idiom now.  I've put this much time into
> it, I don't want to waste the momentum.  What's the approach you are
> thinking of, and how can I get started in the implementation?

Much appreciated. I have a half-written post about exactly that to begin
a new thread - should be up shortly!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

Hi Eduardo,

On 04/28/2011 06:36 PM, legutierr wrote:
> This is extraordinarily discouraging.  

I can understand why.

I've also spent a number of hours thinking about this, reviewing the
patch, considering alternatives, coming up with cases that might break,
etc. I'd like to set aside those sunk costs (which I don't think were
wasted in either case) and keep the focus on the best way to solve the
issue in Django moving forward - that's what I owe to the rest of the
core development team and to the community.

> This is the second time that I
> have devoted tremendous energy to a patch, trying to coordinate with
> core developers, not doing any work until I get the green light from
> core developers regarding an implementation plan (trying to avoid this
> very same eventuality), only to be told, after working code + tests +
> docs have been attached to the ticket, after several iterations of
> feedback: nope, this is not the way that we want to do this policy-
> wise, there's this other approach we want to take, so never mind.

I'm not certain what the other situation is that you're referring to, so
I can't speak to that. My observation has been that this isn't the
common experience (unfortunately, getting no attention to a bug/patch in
the first place has at times been a more common one, though that too is
getting better -- unreviewed bug count is currently zero!), but I'm
sorry you've experienced it, and I regret having contributed to it in
this case. I will certainly be more careful in the future about
expressing optimism that an approach might be workable, especially if
(as in this case) I have reservations about it from the start.

> I can understand going through the bureaucratic rigmarole that comes
> with contributing to Django--in fact, I support it--but to go through
> all of the discussion, justification, and *time* required to get a
> simple bug fix checked in (no, this really *is* a bug--look, there are
> five other tickets filed. sure, let's analyze the problem from every
> angle. sure, I'll rewrite it so it matches exactly your
> specifications.), only to be told that someone who wasn't even
> involved in this ticket and discussion *at all* until now thinks it
> isn't worth it, makes a guy like me want to tear his hair out.  You
> say that this is "in the best interests of Django", but you must know
> that Django will suffer if people like me stop wanting to contribute
> because of things like this.

Indeed, and I hope that you don't lose interest in contributing.

I don't think that the time spent discussing and analyzing this, even
writing and reviewing a patch, is wasted. From my perspective, it has
clearly revealed that the current approach of trying to do
partial-model-validation is broken in concept and not reliably fixable.
That's useful information, and moves us (has already moved us) towards a
better solution.

I can't agree that this is a simple bug fix. The current behavior is
wrong in some cases. The behavior after this "fix"  would still be wrong
in some cases, although fewer. A simple bug fix is one where the fix is
clear, obviously correct, and definitively solves the reported problem.
I don't think that describes this case. Model and form validation is a
complex area, and it's easy for seemingly small changes to have
unintended effects that cause more maintenance burdens down the road.

> How often has it been the case that some really cool new feature gets
> delayed because the core developer who was working on it was unable to
> dedicate the time they thought they would be able to?  Can I help move
> it along, can you work with me to write it?  Why can't we check this
> one in, close two tickets (as well as satisfying three or four
> duplicates) and then move on to the more definitive fix?

I'm committed to having these tickets closed one way or another before
Django 1.4 is released (and neither fix here would be a candidate for
backporting to a 1.3.X release anyway), so let's focus on making the
best fix we can. If the ideas we have in mind for that turn out to be
unworkable for some reason, I still think that the current patch would
be preferable to no change.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

Hi Eduardo,

On 04/28/2011 12:35 PM, legutierr wrote:
> I just added a new patch in response to Carl's comments on the ticket.
> 
> http://code.djangoproject.com/ticket/13091

So, in the process of reviewing and tweaking this patch for commit, I
checked in the #django-dev IRC channel for any other core dev opinionsm
since I didn't feel 100% confident that we were doing the right thing
here. I talked through the issue with Alex Gaynor, and he successfully
convinced me that we aren't. Specifically:

1. We have an idea in mind, as I mentioned above, for a new
modelform-validation idiom that would solve this problem fully, by
requiring tweaks to the model to happen pre-validation and then
validating the full model.

2. If we implement the new idiom, and convert the admin to use it, then
anyone who runs into the problems with the current partial-validation
scheme in their own code can simply switch to the new recommended idiom.
Nobody will be stuck.

3. The current proposed patch on #13091 only improves the current
situation very marginally; there are a lot of cases it still wouldn't
catch (anytime a field involved in a unique-together is modified
post-validation and pre-save, and the odd exclusions for default/blank
fields). It's very much an incomplete fix, and yet it introduces new
complexity both to the code and the documentation, when we already have
a better alternative fix in mind.

So I apologize for leading you to spend time on that patch and then
switching gears. In terms of what's best for Django, I think Alex is
right on this one, so I plan to just work on the better fix rather than
committing the current patch on #13091.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Permission Duplication

[Carl Meyer]

Hi Valentin,

On 04/27/2011 10:33 PM, Valentin Golev wrote:
> 1. I've run into something that seems like a bug. If it really is a bug,
> I'll file a ticket, if it's not, please clarify the behaviour, and, in
> this case, I think a better error message will be awesome.
> 
> Basically, if there are two permissions for a model with same codenames
> and different descriptions, Django tries to add both in the database,
> but there is a unique index on codename so it all crashes horribly.

This sounds like a bug to me - Django should detect the duplicate
codename and give a reasonable error before trying to save it to the
database. Please do file a bug (if there isn't one already).

> 2. Another idea being in my head lately is that since there is a better
> logging support in django nowadays, maybe it's time to try adding a
> dozen of warnings and suggestions to humble Django users. I'd like to
> try to improve overall error-reporting and in Django. Several ideas from
> the top of my head:
> 
> * Handling apps with no "models.py". It seems like Django silences this
> error and removes the app from the app list. It doesn't play nice with
> several things, like testing. I think there should be a warning in
> runserver's console.

I agree that we could do something better here. The first thing I'd want
to check is how this is handled in the app-loading branch that Jannis
Leidel and Arthur Koziel are currently working on, from Arthur's GSoC
work last summer. That will hopefully be merged soon, and will change a
fair bit of behavior around model-loading and INSTALLED_APPS.

> * I think there should be ready to use runserver-console handler for
> logging (seems like just a normal StreamHandler), and it could be
> enabled by default in settings.py

This sounds to me worth filing a ticket for, if there isn't one already.
I'm not sure about enabling it by default, but perhaps with a flag to
runserver.

> * I also think there should be some interesting logging namespaces, like
> django.db.query for logging all queries.

My understanding is that there's a general intention to do more useful
logging now that the logging framework is in place, so I think specific
tickets and patches in this area would be welcome.

> * An awesome feature will be helping with solving O(n) queries errors
> (and other database mistakes) - gentle warnings in console when two
> almost same queries are running during the same request. It doesn't seem
> like a simple task to do, but definitely like an interesting one.

My immediate reaction is that we don't need to get this tricky; just
making it easier to log queries and see that log in console will greatly
improve immediate awareness of O(n) query situations.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Use cases for a OneToMany field

[Carl Meyer]

Hi Charlie,

On 04/27/2011 02:35 PM, Charlie DeTar wrote:
[snip]
> Is this something that others see value in?  Cause I sure think it
> would be awesome.

I actually think that sounds quite useful; I'd never thought of that
alternative to reverse FKs when you don't want to modify the model that
would normally have the FK on it.

It looks to me like something that ought to be implementable outside of
Django, as a subclass of ManyToManyField. I think that's probably the
best path forward, filing bugs if there are things in ManyToManyField
that make it unreasonably difficult to do this as a subclass.

I'm not sure about having this in core - the thought of trying to
document it and explain clearly why it exists and when you might want to
use it instead of a ForeignKey makes me cringe a bit ("There should be
one-- and preferably only one --obvious way to do it."). But if you want
it in core, a battle-tested external implementation is a pretty good
place to start the conversation.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

On 04/27/2011 02:02 PM, legutierr wrote:
> Ok, I'll create a patch soon (with tests + documentation) that
> hopefully works for you.  I don't think it will be very complicated
> implementation-wise, just a few additional lines, I think.  With
> regards to the documentation, I'll add a note here:
> 
> http://docs.djangoproject.com/en/1.3/ref/models/options/#unique-together
> 
> and here:
> 
> http://docs.djangoproject.com/en/1.3/ref/models/instances/#django.db.models.Model.validate_unique
> 
> Including a note saying that the behavior has changed

Great, thanks. I think this behavior change only needs to be described
in one place (the validate_unique docs), but the text at the former link
is actually inaccurate ever since model validation - it should be
updated to mention that unique_together is also checked by model
validation, with a link to the validate_unique docs.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

Hi Eduardo,

On 04/26/2011 02:47 PM, legutierr wrote:
>> With your proposed change, if I happen to have a FavoriteBirthdayPresent
>> in the database for a given year with an empty "username" field, it
>> would incorrectly prevent any user from creating their own
>> FavoriteBirthdayPresent for that year.
>>
>> I'll readily admit that's a corner case that requires several
>> perhaps-unusual conditions:
>> - the excluded field that's part of a unique_together has a default
>> value which is also a valid value in the database, and is not None/NULL
>> (because NULL != NULL in SQL), and
>> - there actually might be a record in the database with that default value.
>>
>> And I think there are probably many more cases where your proposed
>> behavior would be correct. I'd just be happier marking #13091 Accepted
>> if I could see a solution that seemed more clearly correct in all cases.
> 
> Regarding this, I have two somewhat contradictory responses:
> 
> 1) It would be feasible to treat the case where a default value is
> defined on a field (or where the field is allowed to be null) as being
> distinct from the case where the default value is not defined and the
> field is not permitted to be null.  In other words, in the case that
> you cite the current behavior could be maintained (unique_together
> tuples containing any field with a default value would be ignored in
> model validation), while my proposed behavior would only be
> implemented when model validation is certain not to create the
> circumstances you describe.  I would be happy to write a patch for
> this + tests if you are OK with the approach.

Hmm, that's interesting. I'm not super-enthused about the complexity
there (Zen of Python: "if the implementation is hard to explain, it's a
bad idea"), but I think you're right that it's feasible. Note that
nullable fields would be ok to go ahead with (because NULL is not equal
to NULL in SQL, it won't cause false positives on the uniqueness check);
it's just fields with non-null defaults that could cause the false
positive if they are excluded from the form but included in a
unique-together check.

If the implementation (and documentation) for that patch doesn't look
too terrible, I'd consider it - I do think it gets the behavior closer
to right than what we do now, and I'm not sure it's really possible to
get it fully right in all cases as long as we're trying to do validation
on a partial model. I'd be interested in others' thoughts, of course.

> 2) I am not sure, however, that the case you site is really a
> problem.  So what if the user is told that the "year" data they have
> supplied in such a case is not "sufficiently unique"?  It would be
> true (would it not?) that the default "username" would already have
> their favorite birthday present assigned for that year (even if the
> default is null), and it seems to me that such a fact is intelligible
> to the user (The error message could read: "The data you supplied for
> field 'year' is not sufficiently unique for username 'default'," or
> perhaps simply "The 'year' you specified is not sufficiently
> unique.").

That error message is not intelligible to the user, because they aren't
trying to save data for the user "default" at all, they are trying to
save data for their own user (and they can't change the user, because
its excluded from the form and assigned in the view code after
validation). The error is wrong because its checking uniqueness for user
"default" when it ought to be checking for user "janedoe", and it
prevents Jane from saving data for herself for any year that "default"
has data for.

  It would also be fully within the power of the user to
> modify the form in order to get it to pass model validation and be
> saved. 

Only if they give up on saving any data for themselves for that year.

>> This is really giving me the itch to build a new context-manager-based
>> idiom for ModelForm validation in 1.4 that would allow modification of
>> the to-be-saved object within the context manager and always perform
>> full validation of the model on the way out. This idea was raised
>> briefly in discussions right around the time model-validation landed,
>> but was tabled due to the need (at that point) to support Python 2.4.
>> Seems like that could neatly resolve all these knotted issues around
>> validation of partial ModelForms.
> 
> I am sure that whatever idiom you create will be an improvement over
> the current approach, but unfortunately, I think that what you are
> describing is a little over my head.  Regardless, I hope that the
> prospect of introducing this new idiom will not prevent #12082 and
> #13091 from being resolved without the acceptance of a new approach.

I don't think it needs to.

> While we are on the subject of new idioms, I am curious as to what
> might be wrong with this slight amendment to the current idiom:
> 
> form = ModelForm(data)
> form.instance.excluded_field = programatic_data
> if form.is_valid():
>   

Re: #14891, a.k.a. "related managers don't work how we claim they do"

[Carl Meyer]

On 04/19/2011 01:47 AM, Ivan Sagalaev wrote:
> OK, may be not reverse OneToOne… What I mean is that although it seems
> natural to treat all relations equally they are actually used for
> different use cases where different defaults make sense. For example:
> 
> - Reverse many-to-one (topic.article_set) access is used to access a
> (limited) list of children which is expected to behave as any other list
> of such objects and hence should use the default manager.
> 
> - Direct one-to-one or many-to-one (article.topic or profile.user)
> access is used to access a parent object and in most real cases it
> doesn't make sense if it's absent for example. Usually you're just
> dealing with a "deleted" child accessing its "deleted" parent which is
> OK. In this case it makes sense to use a pure manager to build the
> relation.
> 
> As for reverse one-to-one I'm really not sure because I can't recall any
> real example to lay upon. Speaking about documentation simplicity (which
> directly influences sanity of its readers) it can be made as simple as
> that:
> 
> - pure manager is used whenever there is a clear child-parent realation
> (direct OneToOne and ForeignKey access)
> - all other relations use default managers
> - explicit use_for_related_fields overrides default behavior
> 
> What do you think?

I do find this pretty convincing. I don't see a good reason why a
default manager should not be used by default for many-related access.
In other words, I think the current behavior is probably better default
behavior more of the time than the documented behavior would be.

So I guess I've changed my position. I'm leaning towards fixing the docs
to match what we currently do - and I'm also feeling more and more like
the "use_for_related_fields" Manager class attribute is an ugly hack,
and open to looking at patches to replace it with per-relation override
of the default manager-selection.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #14891, a.k.a. "related managers don't work how we claim they do"

[Carl Meyer]

On 04/19/2011 04:58 AM, Johannes Dollinger wrote:
>> Do you have real-world use-cases in mind that require per-relation 
>> manager configuration? I can't think of any uses I've run across
>> that weren't workable with a single default manager used by all
>> relations.
> 
> The only time I used a custom manager for a relationship was for a
> TagField – making .add() accept bare strings. It's been more useful
> for me to use a custom descriptor, e.g. this TagField's descriptor
> delegates __set__ to an .assign() method on the manager. Another
> use-case for custom descriptors was an experimental ListField, where
> the related manager behaves like a list of related objects (e.g.
> supports slicing, iteration, assignment).
> 
> I don't think these examples necessarily need a public API, although
> it'd be nice to reduce the the dependencies on internal API in my
> code.

Yeah... it'd be interesting to see a full proposal for a public API for
use-cases like this, but my sense is that some advanced uses are always
going to depend on implementation details.

> Let me turn the question around: do you have real-world use-case that
> benefits from use_for_related_fields? Especially one that cannot be
> solved more cleanly with custom queryset methods? 

Depends whether you mean use_for_related_fields=True or
use_for_related_fields=False. For the former, sure - I have utility
managers that add extra chainable methods both to the manager and its
returned queryset, and I'd like to have the added method(s) available
everywhere, including on related-object managers.

I haven't yet had a case where I wanted a manager to be the default
manager for normal operation, but wanted to specify
use_for_related_fields=False.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

Hi Florian,

On 04/23/2011 08:18 AM, Florian Apolloner wrote:
> Nice, something like that would be great. One of my bigger problems
> regarding modelforms lately was that I wanted something like: "If you
> don't fill out the user it's set to the current user", in the admin,
> with as little modifications as possible. So save_model sounded like
> the right spot but modelvalidation will make the form invalid (I want
> the user field on the form…). Still no idea on how to solve that
> properly and nicely; if the context managers can take care of that too
> (though I doubt it since I can't change the is_valid call in the
> admin) I am all in for it. Btw any threads around that subject I could
> read?

http://groups.google.com/group/django-developers/msg/3014f29c5125653a is
where it was briefly mentioned by Lukasz, I haven't seen any discussion
since.

As Joseph says later in that thread, two design constraints of model
validation were to avoid double-validation and avoid storing metadata on
the model about which validations have already been run. I think a
context manager might be able to avoid that problem by storing data
about which validations have been run temporarily on the context manager
itself, rather than on the model. This might allow form-validation on
the way in, full model validation on the way out. If there are problems
with that, it might work to just do no validation on the way in and full
model validation on the way out, though it'd be nicer if the code inside
the context manager could rely on at least validation of the fields
included in the form.

In any case, if we have this, I could see switching the admin to use it,
and perhaps adding an overridable method that's called from within the
context manager, to allow you to complete/tweak the model instance
before full validation is run. Seems like this would solve your problem?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)

[Carl Meyer]

On 04/22/2011 02:42 AM, legutierr wrote:
> However, in the case of a tuple of fields that are "unique together",
> the proper behavior should be that if *any* of those fields are
> editable in the form, the constraint should be validated by
> is_valid().  In the current implementation, *all* of the unique-
> together fields have to be editable for the constraint to be
> validated; THAT is the real bug here.  It is always fully within the
> power of a user to resolve a "unique together" constraint violation
> simply by modifying any one of the fields subject to that constraint;
> the only thing required is that the user be told which editable
> field(s) are causing the constraint violation.  Furthermore, such
> violations are not uncommon, and should not require the intervention
> of the developer.
> 
> Note that this is not just a problem for me, but is also a problem at
> the root of several tickets that have already been accepted, several
> over a year old.  These two open tickets pertain to admin and content
> types, and have the same root cause as I am describing above:
> 
> http://code.djangoproject.com/ticket/12028
> http://code.djangoproject.com/ticket/13091
> 
> also:
> 
> http://code.djangoproject.com/ticket/13249
> http://code.djangoproject.com/ticket/15326

Having reviewed all these related tickets, I do think that in _many_
cases it would be useful and expected to have ModelForm validate a
unique_together constraint if any of the unique_together fields are
included in the ModelForm, as you're proposing. I don't think it's
cut-and-dried, though -- I can imagine cases where the new behavior
would be wrong. For instance, this model:

class FavoriteBirthdayPresent(models.Model):
username = models.CharField(max_length=30)
year = models.IntegerField()
description = models.CharField(max_length=200)

class Meta:
unique_together = [["username", "year"]]

and this ModelForm:

class FavoriteBirthdayPresentForm(forms.ModelForm):
class Meta:
model = FavoriteBirthdayPresent
exclude = ["username"]

and this view code:

if form.is_valid():
fave = form.save(commit=False)
fave.username = request.user.username
fave.save()

With your proposed change, if I happen to have a FavoriteBirthdayPresent
in the database for a given year with an empty "username" field, it
would incorrectly prevent any user from creating their own
FavoriteBirthdayPresent for that year.

I'll readily admit that's a corner case that requires several
perhaps-unusual conditions:
- the excluded field that's part of a unique_together has a default
value which is also a valid value in the database, and is not None/NULL
(because NULL != NULL in SQL), and
- there actually might be a record in the database with that default value.

And I think there are probably many more cases where your proposed
behavior would be correct. I'd just be happier marking #13091 Accepted
if I could see a solution that seemed more clearly correct in all cases.

This is really giving me the itch to build a new context-manager-based
idiom for ModelForm validation in 1.4 that would allow modification of
the to-be-saved object within the context manager and always perform
full validation of the model on the way out. This idea was raised
briefly in discussions right around the time model-validation landed,
but was tabled due to the need (at that point) to support Python 2.4.
Seems like that could neatly resolve all these knotted issues around
validation of partial ModelForms.

> Something to take note of regarding #13091, which seems to be the
> canonical ticket: the current patch attached to this ticket would
> eliminate *all* exclusions from the call to the validate_unique()
> model validations.  This I now disagree with (as I am sure you do,
> too), although I originally proposed doing just that.  I also think
> that in the case of a "unique together" tuple where *none* of the
> fields are editable, model validation should be skipped inside
> is_valid().  However, I do think that a modification is warranted to
> resolve the underlying error of the above-listed tickets, as well as
> my own.

Yes, I agree that the current patch on #13091 is too aggressive.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Help review tickets, get a prize!

[Carl Meyer]

On 04/20/2011 05:38 PM, Jacob Kaplan-Moss wrote:
> * It clearly *is* a bug, and there seems to be a fair bit of
> information (steps to reproduce, the traceback, etc.); these I quickly
> mark "accepted", fix the metadata, and move on.

So you don't necessarily reproduce it yourself before marking Accepted?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Help review tickets, get a prize!

[Carl Meyer]

Hi Tino,

On 04/20/2011 05:14 PM, TiNo wrote:
> It takes me, being a newbie at reviewing tickets, quite some more time.
> Would you (or any other core dev / speed reviewer) mind sharing your
> workflow? Any scripts to create environments at certain revisions or
> something alike? Or to quickly run the relevant tests?

It takes me longer than 5 mins per ticket, too. I tend to dive in pretty
deep to make sure I'm understanding the full context for a ticket before
accepting it. That said, here are some things I do for convenience; some
of them may be obvious:

- I have virtualenvs for all supported Python versions (2.4-2.7; trunk
has dropped 2.4 support, but I still may need it for backports), and
each one has my development copy of Django already installed in it (with
"pip install -e ." so it's actually using the working tree code and will
see my changes). Combined with "git checkout" this covers "creating
environments at certain revisions".

- I have stock settings modules for running the tests with each database
I have available (MySQL, SQLite3, Postgres), and I actually have these
packaged up with a setup.py and also installed in each Django-dev
virtualenv (with "pip install -e") so they're always on sys.path and
accessible as "testconf.sqlite" etc.

- I have a bare-bones "test project" with a settings module that has the
admin in INSTALLED_APPS and set up in urls.py, and an installed
"testapp" that already has __init__.py, models.py, and admin.py. So for
those cases where I need to try out something manually (often in the
admin), it's just a matter of copy-pasting some things into the
models.py and admin.py, syncing the db, and firing up runserver. And I
actually have this version-controlled so I can commit the "manual test"
setup for a given bug (in a branch named with the ticket number) in case
I need it later for working on the bug.

HTH,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Help review tickets, get a prize!

[Carl Meyer]

On 04/20/2011 04:27 PM, Alex Gaynor wrote:
> Consider me in on the 5-1 offer.

Same here.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: new backports policy

[Carl Meyer]

On 04/20/2011 04:37 PM, Tobias McNulty wrote:
> I'd be more comfortable if the policy stated that any new bugs
> introduced by the last release would be backported to that release.
>  It's possible that "major functionality bugs in newly-introduced
> features" will equate to virtually the same thing, but I'm not clear
> what would constitute a major functionality bug (it sounds big, and like
> it might be a difficult criterion to meet).

I like this. In general we treat regressions as release blockers anyway,
so this fits with the "backport if it would have blocked the release"
spirit of the proposal.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Admin search behavior changed from 1.2.5 to 1.3

[Carl Meyer]

Hi Ryan,

On 04/20/2011 10:42 AM, Ryan K wrote:
> Today I discovered behavior similar to that originally reported in
> #15819 (http://code.djangoproject.com/ticket/15819). I've updated it
> with a simple way to reproduce the issue. Could anyone confirm this
> behavior? It's nothing major but it does seem that the admin search
> behavior changed from 1.2.5 to 1.3.

I can reproduce the issue (and confirm that it is a regression in 1.3),
and I've accepted the ticket.

> Would it be possible to add a boolean, say 'search_distinct', to
> ModelAdmin so distinct() is called after a search here:
> http://code.djangoproject.com/browser/django/trunk/django/contrib/admin/views/main.py#L271
> ?

Is there any reason why someone would _want_ duplicate search results? I
don't think so, so there is no reason for a ModelAdmin option here - it
should just be fixed.

> I haven't yet contributed to the project but would like to if this is
> indeed something that should be fixed.

That'd be great! Patch should be against SVN trunk, and include a test
that fails before the fix and passes afterwards, in
tests/regressiontests/admin_views/tests.py. Feel free to hit me up on
IRC (carljm) or here if you have questions working on the patch. I made
some additional comments on the ticket. Thanks!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: new backports policy

[Carl Meyer]

On 04/19/2011 05:24 PM, Luke Plant wrote:
> Another issue with regards to backporting bug fixes that Jacob didn't
> mention is the fact that bug fixes often introduce subtle regressions
> and backwards incompatibilities.
> 
> Personally, I find that backporting a fix very often takes only a
> minute, by appropriate use of DVCS features (e.g. hg transplant) and
> some scripting. (This doesn't include the time for running tests again,
> and doesn't include cases where the merge fails or the patch has to be
> effectively re-written). However, the problems caused by introducing
> these regressions onto supposedly stable branches can be much more time
> consuming and tricky.
> 
> By keeping backporting of bugs down to a minimum, we avoid these
> problems, and help keep our stable branch genuinely stable. Not
> introducing regressions is much more important than fixing existing
> bugs, since most people will have worked around existing bugs where
> necessary for their needs anyway.

+1 to all of this. IMO the bigger issue is the regressions that
inevitably get introduced when we backport all bugfixes, not the extra
committer time to do backports (which I also agree I haven't found
problematic).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Right way to do formfield-less fields?

[Carl Meyer]

On 04/19/2011 09:44 AM, Jonas H. wrote:
> On 04/10/2011 03:14 PM, Jonas H. wrote:
>> What's the preferred way to do a custom model field that has no
>> corresponding formfield? Intuitively, I'd say raise a
>> NotImplementedError, but that would make the whole model unusable in the
>> admin even if that field is excluded from being displayed.

I haven't tried this, but based on the implementation of
`fields_for_model` it looks to me like returning None might work (it
would cause that field to not be included in the SortedDict returned by
fields_for_model, so you wouldn't even need to list that field in
`exclude` - in fact, you couldn't or it would fail validation).

>> So, are fields without a formfield possible already? If yes, how to do
>> it? If not, would you accept a patch that makes the `ModelAdmin.exclude`
>> attribute validation not call into `Field.formfield`?

In theory, yes, although a brief look at the relevant code doesn't
suggest any obvious clean fixes to my mind.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #14891, a.k.a. "related managers don't work how we claim they do"

[Carl Meyer]

Hi Johannes,

On 04/18/2011 01:45 PM, Johannes Dollinger wrote:
> Deprecate `use_for_related_fields` and always use the default manager
> for related managers. Then add the possibility to specify custom
> mangers for individual relations:
> 
> ForeignKey(Foo, related_manager=RSpecialManager) ManyToManyField(Foo,
> manager=SpecialManger, related_manager= RSpecialManager)
> 
> More fine grained control would especially be useful for subclasses
> of ForeignKey and ManyToManyField fields. It comes at the expense of
> verbosity, but it appears to be a rarely used feature (given that the
> bug was discovered only now). And thus, explicitness might actually
> be a good idea.

Do you have real-world use-cases in mind that require per-relation
manager configuration? I can't think of any uses I've run across that
weren't workable with a single default manager used by all relations.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #14891, a.k.a. "related managers don't work how we claim they do"

[Carl Meyer]

On 04/18/2011 04:35 PM, Ivan Sagalaev wrote:
> Not exactly… I mean that when use_for_related_fields is set explicitly
> Django should respect it. Right now, as I understand from your first
> mail, it doesn't work as False when set to False. So I agree that this
> should definitely be fixed.
> 
> What I was saying is that when this attribute is not set current
> behavior does make sense:
> 
> - use default manager as a base for *_set managers
> - use pure manager as a base for OneToOne and parent lookups
> 
> However it can't be described with neither "False by default" nor "True
> by default". I think it's fine and we could just thoroughly document
> this behavior.

Hmm. Why does it make sense for OneToOneField lookups to behave
differently from *_set managers? If I've got a default manager that
hides "deleted" objects, for instance: why should deleted objects by
default "not exist" when I traverse a reverse FK, but "exist" when I
traverse a OneToOneField?

Simply from a complexity-of-documentation standpoint I don't like the
idea that the effective "default" for use_for_related_fields would be
neither True nor False, so to counterbalance that I'd want a pretty
strong case for why it's the best option.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #14891, a.k.a. "related managers don't work how we claim they do"

[Carl Meyer]

On 04/18/2011 12:47 PM, Luke Plant wrote:
>> So - do we (A) fix the behavior to match our documented semantics, note
>> it in the release notes, and hope for the best? Or (B) bow to
>> backwards-compatibility and change the documentation to match the actual
>> behavior? Or (C) find some middle ground (a deprecation path for the
>> current behavior)?
> 
> I vote for A - fix the bug.

That's my leaning, too.

The biggest pain this would cause would be for people using third-party
custom managers that don't set use_for_related_fields, and relying on it
being used for related fields anyway. Since use_for_related_fields
currently must be set as a class attribute, not an instance attribute,
they would either need to subclass the third-party manager just in order
to add use_for_related_fields=True, or they would need to monkeypatch it on.

In my mind, this reveals a different problem: the author of a custom
Manager subclass is not necessarily the best person to decide whether
that manager should be used for related fields in a particular use of
it. Making Manager.__init__ accept a use_for_related_fields argument is
problematic for backwards-compat with existing subclasses, but we could
respect it if set directly as an instance attribute:

   objects = MyCustomManager()
   objects.use_for_related_fields = True

I may open that as a separate ticket.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #14891, a.k.a. "related managers don't work how we claim they do"

[Carl Meyer]

Hi Ivan,

On 04/18/2011 01:07 PM, Ivan Sagalaev wrote:
>> even if "use_for_related_fields" is absent or explicitly set to
>> False on your Manager subclass.
> 
> … the default manager should not be used as a base class. Fixing just
> this would be the best option because it retains current behavior by
> default while allowing other uses if needed.

By "just this" I presume you actually mean just the second half of what
you quoted ("explicitly set to False")? In other words, you're proposing
to make use_for_related_fields work as advertised, but make it default
to True instead of False?

If I were designing this from scratch, I think I would also prefer
use_for_related_fields to default to True. But that would be a
non-bugfix change that would impact backwards-compatibility for things
that currently do properly respect use_for_related_fields. For example,
people might suddenly start getting ObjectNotFound from a OneToOneField
descriptor where previously they got the "hidden-by-default-manager"
object back. So I think we could only do that with some kind of
deprecation path for the current default of False.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Filter via related_name in inherited model not working .. Bug?

[Carl Meyer]

Hi Alex,

On 04/18/2011 02:44 AM, robim42 wrote:
> thank you for your answer. I also thought that owner_partner should
> work ... but when you try you get this:

Sorry, not owner__partner - since partner is only a field on Buyer, not
on Base, you'd need owner__buyer__partner.

In any case, I don't see any evidence of a bug in Django here (we've got
a pretty good test suite for filter traversal and inheritance), so this
thread should move to django-users.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Accidental logging disabling

[Carl Meyer]

Hi Ivan,

On 04/17/2011 10:38 PM, Ivan Sagalaev wrote:
> On 04/15/2011 02:20 AM, akaariai wrote:
>> I have been using setup_environ in my projects, and the lazy
>> initialization in can cause some weird problems, for example if you do
>> manual timing using:
>> start = datetime.now()
>> access settings
>> print 'Used %s' % (datetime.now() - start)
>>
>> You might get weird results as accessing settings can change your
>> timezone. Would it be wise that setup_environ() would access the
>> settings so that they are no more lazy? Or does this cause other
>> problems?
> 
> I didn't have in mind changing the settings object to be non-lazy. It
> looks like a rather intrusive change. And I can't really predict
> consequences. 

FWIW, I don't think changing settings in general to be non-lazy is an
option - it will suddenly make a bunch of parts of Django dependent on
DJANGO_SETTINGS_MODULE at import time, where currently they only require
it at runtime (and even then perhaps only when certain functionality is
used).

I do, however, think that it would be fine to have the setup_environ
function explicitly trigger settings-loading. This should never be a
problem, because setup_environ already sets DJANGO_SETTINGS_MODULE. It's
also not an invasive change at all - it should be a one-liner, I think.

My gut feeling tells me that changing things from being
> immediate to be lazy is the source of many hidden problems. But it
> should be safe vice versa because if the environment already works with
> a lazy object it couldn't count on the order of execution. So immediate
> instantiation shouldn't break the logic. What it could do though is
> affect performance…

I don't think there would be any performance issues. Any code that's
calling setup_environ is quite likely accessing settings not long
thereafter, so it would shift a rather small bit of work that would
happen anyway to happen sooner rather than later.

> Anyway I still think strongly suggesting not disabling existing loggers
> would suffice.

That might be reasonable as well - I'm interested to hear Russell speak
to that, as he did the logging work. I still think making setup_environ
non-lazy is a reasonable change that will address a broader range of
problems.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: The model option's verbose_name_raw is not really that 'raw'

[Carl Meyer]

Hi Viktor,

On 04/18/2011 09:14 AM, Viktor Kojouharov wrote:
> I think I might have stumbled upon a small bug. According to its docs,
> the 'verbose_name_raw' property is supposed to return an untranslated
> version of the 'verbose_name' property. However, if LANGUAGE_CODE is
> set, then 'verbose_name' would already be translated, provided that the
> user wrote the meta option as "verbose_name = _('Some name')"

That depends on what "_" is in the relevant models.py. If it is one of
the lazy variants (i.e. "django.utils.translation.ungettext_lazy") [1],
which it should be, then the verbose_name is not translated until it is
coerced to unicode, and the definition of the verbose_name_raw property
is correct.

If you use a non-lazy translation function for your verbose_name, it
almost certainly won't work as expected at all, because it will be
translated once when your models.py is imported and not translated to
the appropriate locale for each user.

Carl


  [1] see
http://docs.djangoproject.com/en/1.3/topics/i18n/internationalization/#lazy-translation

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

#14891, a.k.a. "related managers don't work how we claim they do"

[Carl Meyer]

Hi all,

Our documentation on "automatic managers" [1] and related managers [2]
is quite clear that automatically-created managers for related-objects
access will be plain-vanilla Manager instances, unless you set
"use_for_related_fields = True" on the custom Manager subclass that you
use as a default manager.

Unfortunately, this documentation is not true; near as I can tell, it
never has been since it was introduced.

Vanilla managers are used as documented in a few places: OneToOneField
related-object access, and traversing FKs and M2Ms for internal
cascade-deletion purposes. But in general, related-object access for
reverse-FKs and M2Ms, contrary to the documentation, will _always_ use
your default manager (actually, a dynamic subclass of it) and will
respect any restrictions in the get_query_set method of your default
manager, even if "use_for_related_fields" is absent or explicitly set to
False on your Manager subclass.

I have more details, and the fix (if we want it) on ticket #14891 [3]

The dilemma here is that fixing this bug is quite likely to break
significant amounts of code that is relying on it. As evidence of that,
when fixing it I had to also fix two places in Django's own test suite
where we assumed that a custom default manager would be used for
related-object access, without setting use_for_related_fields=True on
the manager.

So - do we (A) fix the behavior to match our documented semantics, note
it in the release notes, and hope for the best? Or (B) bow to
backwards-compatibility and change the documentation to match the actual
behavior? Or (C) find some middle ground (a deprecation path for the
current behavior)?

Personally, I don't find option (B) very appealing. The current behavior
isn't crippling (you can generally work around it if you need to by just
not making your custom manager the default one), but it is inconsistent,
hard to explain, and means "use_for_related_fields" is quite
misleadingly named.

Input welcome,

Carl

  [1]
http://docs.djangoproject.com/en/dev/topics/db/managers/#set-use-for-related-fields-when-you-define-the-class
  [2]
http://docs.djangoproject.com/en/dev/topics/db/managers/#managers-for-related-objects
  [3] http://code.djangoproject.com/ticket/14891

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Accidental logging disabling

[Carl Meyer]

On 04/15/2011 04:20 AM, akaariai wrote:
> I have been using setup_environ in my projects, and the lazy
> initialization in can cause some weird problems, for example if you do
> manual timing using:
> start = datetime.now()
> access settings
> print 'Used %s' % (datetime.now() - start)
> 
> You might get weird results as accessing settings can change your
> timezone. Would it be wise that setup_environ() would access the
> settings so that they are no more lazy? Or does this cause other
> problems?

I think this would be worth experimenting with. AFAIK the only real
reason for settings to be lazy is to avoid the dreaded "Settings cannot
be imported, because environment variable DJANGO_SETTINGS_MODULE is
undefined." when importing modules that import settings (instead you
only get it when settings are actually accessed).

But that's not an issue for setup_environ, because it takes the module
as parameter and actually sets DJANGO_SETTINGS_MODULE. So I don't think
there would be any harm (and there would be significant predictability
benefit) in having setup_environ trigger the actual loading of settings
immediately.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Filter via related_name in inherited model not working .. Bug?

[Carl Meyer]

Hi Alex,

On 04/15/2011 07:51 AM, robim42 wrote:
> --- models.py code --- snip ---
> from django.db import models
> 
> class Base(models.Model):
> name = models.CharField(max_length=20)
> 
> def __unicode__(self):
> return self.name
> 
> class Seller(Base):
> childa = models.CharField(max_length=20)
> 
> def __unicode__(self):
> return u'%s: %s' %(self.name, self.childa)
> 
> class Buyer(Base):
> childb = models.CharField(max_length=20)
> partner = models.ForeignKey(Base, related_name = 'buyer_partner',
> blank = True, null = True)
> 
> def __unicode__(self):
> return u'%s: %s, %s' %(self.name, self.childb, self.partner)
> 
> class Item(models.Model):
> name = models.CharField(max_length=20)
> owner = models.ForeignKey(Base, related_name='item_owner',
> blank=True, null=True)
> 
> def __unicode__(self):
> return u'%s: %s' %(self.name, self.owner)
> --- snap ---
> 
> I'm using postgresql in the real application and the sqlite3 backend
> on this example here, both behave the same.
> To add some data I do the following on the python shell (python manage
> shell):
> --- snip ---
 from multi.models import Seller, Buyer, Item, Base
 seller = Seller(name='Seller1', childa='ChildA')
 seller.save()
 buyer = Buyer(name='Buyer1', childb='ChildB', partner=seller)
 buyer.save()
 item = Item(name='Nice Thing', owner = buyer)
 item.save()
> 
> --- snap ---
> 
> After this I do some query to verify that everything is in the
> database (also on the shell):
> --- snip ---
 from multi.models import Seller, Buyer, Item, Base
 Buyer.objects.all()
> 
> []
 Seller.objects.all()
> 
> []
 Item.objects.all()
> 
> []
> --- snap ---
> 
> So all necessary data is in the database ... now a few queries to get
> data back out and assigned to some vars ... and finally to filter
> queries ... the first one is working, the second one seems to bee
> 'broken' (gives no result = []) and I don't know why ...
> --- snip ---
 from multi.models import Seller, Buyer, Item, Base
 s = Seller.objects.get(name='Seller1')
 b = Buyer.objects.get(name='Buyer1')
 s
> 
> 
 b
> 
> 
 Item.objects.filter(owner=b)
> 
> []
 Item.objects.filter(owner__buyer_partner = s)
> 
> []
> --- snap ---
> 
> Could anyone point me to the right direction or give me an example how
> it is supposed to work?

If I'm reading this right, I think you're misunderstanding which side of
the relationship related_name applies to. The way you have your models
set up, "buyer_partner" is the name you would use to traverse from the
partner targeted by the "partner" FK back to the Buyer containing that
FK. To traverse from the Buyer to its partner, you would just use
"partner" (the name of the FK) not "buyer_partner" (the related_name).

The owner of your Item is your Buyer, so trying to filter by
owner__buyer_partner is using the wrong name; I would expect
owner__partner=s to work.

If the Item was owned by the Seller in this example, then I would expect
a filter on owner__buyer_partner=b to return the item.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC] Composite fields

[Carl Meyer]

Hi Michal,

On 04/01/2011 09:22 PM, Michal Petrucha wrote:
> I propose to implement a CompositeField with the following properties:
> 
> - will be represented as a namedtuple containing the values of the
>   actual fields it encapsulates
> 
> - will support retrieval and assignment (assignment sets the values of
>   encapsulated fields)
> 
> - will be possible to use in __exact QuerySet lookups
> 
> - support for Field.db_index and Field.unique creating the appropriate
>   indexes and constraints in the database
> 
> - support for Field.primary_key

I haven't had time yet to sit down and look at your implementation
questions for a CompositeField (how it works with lookups and Qs, etc),
but I do think that one design goal for a CompositeField implementation
is that we should be able to reimplement the Generic Foreign Key as a
CompositeField. I think your proposal could get there fairly easily --
all that's needed is for the mapping to and from the Python
representation to be customizable by a CompositeField subclass. So a
namedtuple representation is a sane default, but a GFK implementation
could override that to map to and from actual model objects instead.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revised form rendering

[Carl Meyer]

Hi Mikhail,

On 04/01/2011 05:09 PM, Mikhail Korobov wrote:
> Hi Carl and Gregor,
> 
> On 2 апр, 01:17, Carl Meyer <c...@oddbird.net> wrote:
>>
>>> 3. The designers I worked with are often interested on adding custom css 
>>> class
>>>or an attribute to a form field. Most of the time this is really a pain 
>>> to
>>>do if you don't have control over the python form code. Imagine a 
>>> reusable
>>>app that ships with urls, views, forms. To add a single class you would
>>>need to: (a) overwrite the predefined url because you want (b) to specify
>>>an additional parameter for the view which is (c) your custom subclass of
>>>the thirdparty form::
>>
>>>class ChangedAttributeForm(ThirdPartyForm):
>>>field = forms.CharField(max_length=50,
>>>widget=TextInput(attrs={'class': 'fancy-text-input'}))
>>
>>>btw: This also violates the DRY principle since you have to redefine the
>>>used field type, it's attributes (like ``max_length=50``) and the widget.
>>
>>>I want to tell the designer how he can do this without my help in the
>>>template.
>>
>> I agree with your Motivations section - in particular this final line,
>> which sums up the core motivation as I see it.
>>
> 
> This goal can be achieved without changing django, see
> https://bitbucket.org/kmike/django-widget-tweaks
> 
> Each widget is instantiated by field in 'as_widget' method now, and
> the field passes 'attrs' argument to widget with extra attributes.
> 
> The idea behind django-widget-tweaks is simple: it provides template
> filters that take field instance and wrap its 'as_widget' bound method
> so it receives extra attributes (and pass these extra attributes to
> widget). This way widget attributes (including input types and css
> classes) can be altered in templates without touching python, e.g.:
> 
> {{ myform.field|add_class:"fancy-text-input" }}
> {{ search_form.query|attr:"type:search" }}
> 
> Implementation: 
> https://bitbucket.org/kmike/django-widget-tweaks/src/0e9bac3c71bd/widget_tweaks/templatetags/widget_tweaks.py

That's quite a neat app - I have some similar template filters
implemented in some of my own projects. It doesn't cover the full range
of Gregor's proposal -- but I actually think most of Gregor's proposal
would be possible to implement outside Django. Perhaps we should
consider whether it would be preferable to start it off that way, or
focus the proposal more on where Django itself really needs to change.

Personally, I think HTML generation in Python code is a bug, and I'd
like to see it eradicated from Django, so I'm pretty open to seeing this
stuff make it to trunk.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC] Revised form rendering

[Carl Meyer]

Hi Gregor,

As you've probably seen in past threads, this is an area where I'm quite
motivated to see some improvement. I think you've got quite a strong
proposal here in general, and you've clearly done your homework, so my
comments below dive directly into the details:

On 04/01/2011 11:57 AM, Gregor Müllegger wrote:
> I suggest reading this proposal online: https://gist.github.com/898375
> It's exactly the same as below but formated nicely.
> 
> 
> GSoC 2011 Proposal - Revised form rendering
> 
> 
> Hi my name is Gregor Müllegger. I'm a Computer Science student in Germany at
> the University of Augsburg currently in the fourth year of my studies. I first
> came to django shortly before 0.96 was released and a lots of awesomeness was
> introduced with the magic removal branch.
> 
> I'm also doing some django freelancing work since 2008 to finance my studies
> and attended DjangoCon EU in 2011. This year I would like to apply to
> Google Summer of Code helping to improve Django with something that bugs me
> since quite a while: It's builtin ability to render a form straight away into
> HTML.
> 
> Motiviation
> ---
> 
> Why would I like to change the current behaviour? There are some reasons for
> this:
> 
> 1. It's hard to change the default rendering.
> 
>It is very easy to use e.g. django's builtin rendering of a ``ul`` based
>output like ``{{ myform.as_ul }}``. But what if you want to use your own
>*layout* like ``{{ myform.as_dl }}``? You can create a new method on your
>existing forms, but that involves writing python code (out of the designers
>hand) and might not be possible for thirdparty forms.
> 
> 2. Maybe the ``as_ul`` rendering is fine for me. But maybe I want to skip that
>unnecessary field that the thirdparty form is providing. The solution would
>be to write down every single field in your template, reimplementing the
>``ul`` layout::
> 
> {{ form.field.label_tag }}: {{ form.field }}
> {{ form.field.help_text }} {{ form.field.errors }}
> {# skipping field2 here #}
> {{ form.field3.label_tag }}: {{ form.field3 }}
> {{ form.field3.help_text }} {{ form.field3.errors }}
> ...
> 
>We all love DRY, so this is not acceptable.
> 
> 3. The designers I worked with are often interested on adding custom css class
>or an attribute to a form field. Most of the time this is really a pain to
>do if you don't have control over the python form code. Imagine a reusable
>app that ships with urls, views, forms. To add a single class you would
>need to: (a) overwrite the predefined url because you want (b) to specify
>an additional parameter for the view which is (c) your custom subclass of
>the thirdparty form::
> 
>class ChangedAttributeForm(ThirdPartyForm):
>field = forms.CharField(max_length=50,
>widget=TextInput(attrs={'class': 'fancy-text-input'}))
> 
>btw: This also violates the DRY principle since you have to redefine the
>used field type, it's attributes (like ``max_length=50``) and the widget.
> 
>I want to tell the designer how he can do this without my help in the
>template.

I agree with your Motivations section - in particular this final line,
which sums up the core motivation as I see it.

> Goals I want to accomplish
> --
> 
> After showing some of the problems that I see, are here the higher goals I
> want to achieve during the summer:
> 
> 1. All of the rendering formats should be extracted from django's python 
> source
>into a well defined structure of customizable templates.
> 2. Make it possible to reorder the use of form fields in the template without
>needing to write down the complete form.
> 3. Support for *chrome* that can be added dynamically in the template to a set
>of fields, changing some aspects of the form output.
> 4. Ensuring that the DRY principle is applyable to your form templates (in
>case of reordering/skipping some fields, like shown above)
> 5. Ensuring total backwards compatibility for ``{{ form.as_p }}`` etc. but
>deprecating it.
> 6. Converting the admin to use all of the new goodness. First, to make django
>eating its own dogfood. Second, to prove that the stuff I have developed is
>really a step forward and to show up big problems that occur in reallife
>scenarios, not taken into account in the blueprint.
> 7. Documenting all the newly introduced templatetags, the builtin chromes,
>how to write custom chrome, how to create your own form layout etc...
> 
> Let's get a bit more detailed. How do I want to implement these goals?
> 
> **1. No HTML in python source**
> 
> I will push the formating of ``as_table``, ``as_ul`` and ``as_p`` into a
> specified template structure. A formating (e.g. ``ul``) will be called a
> layout and will live in the template directory ``forms/layouts//...``.
> 

Re: GSOC : Django auth

[Carl Meyer]

On 03/30/2011 07:43 PM, Russell Keith-Magee wrote:
> ... and this is the exact model that has been proposed, and rejected
> in #3011 (albeit with a more complex dance around default values). It

I just re-read #3011, and it is far from clear from reading through the
ticket that that approach was ever rejected as such, even less why.
There is mention on the ticket about it being dropped from the 1.1
feature discussions, but even there there is more confusion than clarify
on the ticket.

Anyway, the rejection and reasons may be clear from a thorough search of
django-developers archives, which I haven't done, and is a reasonable
thing to expect a GSoC applicant to do - but it seems to me that if any
core devs have a clear idea that the approach in the #3011 tickets is
not workable, it would probably be worthwhile Trac gardening to say so
(and briefly why) on the ticket itself.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Single Table Inheritance

[Carl Meyer]

On 03/29/2011 02:46 PM, Shawn Milochik wrote:
> I'm not proposing a change to Django itself or suggesting that this
> should be a standard practice. I do think that this is a fairly clean
> solution for an individual to use to solve this problem if they have
> it.
> 
> They can create a custom manager on the abstract class that would
> return an iterable, perhaps using itertools.chain() of the querysets.

Ah, I didn't realize that's the direction you were headed. Yeah, you can
do this, and I've done it; it starts to hurt as soon as you want, say,
sorting + pagination without pulling all of both tables into memory.

> It depends on what they expect to do with the output of this custom
> manager, and they'd obviously lose the ability to treat this output as
> a queryset by using additional sorts & filters and such. But if the
> goal is to be able to get instances from all subclasses at once then
> this is a viable solution, FWIW.

Yup.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Single Table Inheritance

[Carl Meyer]

Hi Shawn,

On 03/29/2011 01:28 PM, Shawn Milochik wrote:
> Hopefully someone on the core dev team can let me know if this is
> possible in Django. If so, it will solve this problem.
> I am not familiar with custom metaclass stuff done within models.Model.
> 
> 1. Create a custom metaclass as described in "Pro Python," page 124.
> 
> 2. Add this metaclass to the abstract model.
> 
> This would allow the base-class to be aware of its subclasses and have
> a class method that returns a queryset of them.
> 
> Further detail:
> This simply amounts to creating a list object in the abstract base
> class. Each time a subclass is instantiated, that class is added to
> the parent class's list. That's all. That list could be used in a
> custom manager to get the desired queryset.
> 
> What I don't know is how nicely this will play with the existing
> metaclass work in Django. It seems that a metaclass can be easily made
> by subclassing the one used for models.Model instead of the default
> 'type,' but this is beyond my experience level. Is this a reasonable
> approach?

What you've outlined here is certainly possible (and yes, you'd need to
subclass the ModelBase metaclass). I haven't looked at the abstract
inheritance stuff recently, but I think there would be some alternative
ways for the abstract base to know about its children that wouldn't
require the metaclass assignment. However: getting the list of
subclasses is (less than) half the battle; the trickier part is giving
the ORM the capability to do UNION queries on similar tables, so you can
get results from multiple tables in a single QuerySet.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Single Table Inheritance

[Carl Meyer]

On 03/29/2011 12:40 PM, Jeremy Dunck wrote:
> What about keeping abstract inheritance in this case, but allowing
> Document.objects.* to work by returning instances of the subclasses.
> Filtering, etc. would only work based on the Document base class.
> 
> It would mean doing some unions, but would still fit the use case
> pretty well, I think.

I'd certainly be intrigued to look at a patch that implemented that.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: switch to HTML5 for bundled templates

[Carl Meyer]

I think it would be helpful here to clearly distinguish three distinct
varieties of "using HTML5," two of which are clear wins and one which I
don't see any reason to do:

1. Switching to the HTML5 doctype in those few places where Django
actually renders a full page with doctype (the admin, databrowse, CSRF
failure page). This is fully backwards and forwards compatible, and
there is no reason (AFAICS) why we should not simply do it. It gives
people writing custom admin widgets more flexibility, without losing
anything. IMO "configurability" here is overkill; the admin (and
databrowse) are standalone apps: they just need to be internally
consistent and work well in all browsers.

2. Providing HTML5 form widget types (email, date, etc) in django.forms.
These are very useful in newer browsers, and fall back gracefully to
input type="text" in older browsers, so again I don't see a downside.
Unlike HTML5-only elements (see #3), these input types don't cause a
problem with CSS in IE.

3. Introducing HTML5-only elements (nav, section, footer, etc) anywhere
in Django-provided templates. This requires Javascript help to work with
most IE versions. Given that the current markup in Django works fine and
is already valid HTML5, I don't see why we would want to do this, short
of possibly in a full admin redesign later on.

My understanding is that Luke is proposing 1 & 2, but not 3 - and I
agree with that. Like others, I'm interested in hearing from Idan on this.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Complains about FileField not deleting files in 1.3.

[Carl Meyer]

Hi Alex,

On 03/29/2011 01:36 AM, Alex Kamedov wrote:
> I think, cron jobs is an overhead in many simple cases where old
> behaviour was useful and more simpler.
> Why you don't want include DeletingFileField[1] in django?
> 
> [1] https://gist.github.com/889692

Because, as mentioned above, it is known to cause data loss in certain
situations (rolled-back transactions, overlapping upload-to
directories), and we are not very fond of including things in Django
that cause some Django users to lose their data. If you understand those
risks and want to use DeletingFileField in your projects, it's not hard
to do so.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: switch to HTML5 for bundled templates

[Carl Meyer]

Hi Luke,

On 03/28/2011 12:38 PM, Luke Plant wrote:
> Overall, I think the advantages outweigh the disadvantages, that we have
> to make the move sometime, and now is about the right time, or perhaps
> slightly late.

100% agreed, for all the reasons you outlined. We've been using the
HTML5 doctype exclusively for a year and a half already, with no issues,
and quite a lot of benefits (i.e. Firefox 4 and latest Chrome will now
do client-side validation of things like input type="email").

It's quite clear by now that HTML5 is the future, we may as well get on
board sooner rather than later.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Trac components cleanup

[Carl Meyer]

On 03/28/2011 02:33 AM, Russell Keith-Magee wrote:
> As with the other thread on Trac changes -- I agree this is worth
> doing, but would like to hear some other core dev voices before making
> any changes.

These changes look to me like a gain in sanity on every front. +1

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Complains about FileField not deleting files in 1.3.

[Carl Meyer]

On 03/27/2011 06:42 AM, -RAX- wrote:
> I am referring to this: 
> http://docs.djangoproject.com/en/dev/releases/1.3/#filefield-no-longer-deletes-files
> Instead of preventing the data loss from happening a very usefull
> feature has been removed.

Well, it does also prevent the data loss from happening ;) This data
loss is not a hypothetical problem; we had bug reports from users
affected by it.

> Why not simply letting the developer decide when to enable or disable
> it with a constructor boolean parameter?
> 
> My company sells multimedia web applications normally handling over
> 1 files over various models.
> I am sorry to say that but to me the idea of running a cron job to
> remove orphaned files does not seam to be practical. Shall I make a
> query for each file?

I don't see why that would be necessary. One query for each model
containing one or more FileFields is enough to build a list of the files
that ought to exist, and any file not in that list can presumably be
removed.

> The roll back data loss problem could have been solved by copying the
> file into a temporary file and by restoring it if necessary.

Emulating the transactional behavior of a relational database is not
that trivial. We considered this approach carefully and decided that if
we tried to go down that road, we'd be continually finding and fixing
edge-case bugs in it, and any bug in it would be likely to be a
data-loss bug. Deleting files when we can't be sure it's the right thing
to do is a very dangerous business to be in.

> Am I the only one who would like to see the previous behaviour
> restored? Can we at least re-enable this feature from the file-field
> constructor?

If you want the previous behavior, it's not at all difficult to restore
it with a post-save signal handler. You can make your own trivial
subclass of FileField that attaches this post-save handler in the
contribute_to_class method: that's precisely what FileField used to do.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Wrong error message when user having is_staff=False tries to login to admin

[Carl Meyer]

On Mar 16, 12:11 am, Tai Lee  wrote:
> Assuming that any authenticated user might be an attacker who has
> brute forced a password and presenting obscure error messages to
> authenticated users is not helping anybody.

I agree with this, and with the many people in this thread who have
come to the conclusion that there is no significant security benefit
to the obfuscated error message here. So I would be +1 to change it.

However, if there are core devs who still feel that there's some
security issue here (Russell?), Paul presented early on what I think
is a pretty good compromise: keep the error message the same in all
cases, but append "or you don't have permission to log in here." so it
actually covers the possible cases and isn't quite so misleading. I
think that change should be uncontroversial.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: template-based widget rendering

[Carl Meyer]

Hi Bruno,

On 03/23/2011 08:56 AM, Bruno Renié wrote:
> I'm not sure, however, how django's backwards-compatibility applies in
> this case. The widgets API has never been publicly documented but the
> official documentation encourages people to look at the code and
> create their own widgets based on it. Is it fine to touch some
> internal methods like build_attrs()? There will be some minor
> differences in the rendered code, with things like the order of
> attributes (name='foo' id='bar' versus id='bar' name='foo') and
> linebreaks (I have an EOL at the end of every template so each widget
> gets a "\n" at its end).

Aesthetic differences in the rendered HTML (like attribute order or
linebreaks) is not a backwards-compatibility issue, IMO - though the
default templates should still be functionally identical to the current
output.

I think widgets should maintain the same external API (i.e. the way they
are used by Django's forms code should not change), or else follow the
normal deprecation path for any well-justified changes there. As far as
I can tell from a quick scan, the external API consists only of the
signatures of the __init__() and render() methods, and the .attrs
attribute (which is tweaked directly by BoundField). Any methods or
attributes that are a) not documented, and b) not used by Django's form
code outside of widgets.py, are IMO fair game for change (though
obviously still only with good reason).

The area where this is most likely to break user code is if people have
subclassed some of the more complex widgets and are depending on
internal implementation details in their subclass. My feeling here is
that this is just like any other use of an internal API; all bets are
off. If it turns out that this breaks a lot of code, the other option
would be to introduce the new widgets in a new namespace, make them the
default ones, and leave the old ones alone. I'd really like to not do
this unless we absolutely have to.

> Currently the base `Widget` class has a render() method that raises
> NotImplemented. I think the rendering logic should happen here, the
> base Widget class should accept a template_name attribute and a
> render() method that renders the template. This way there is (almost)
> no need to touch any widget's render() method since template_name and
> get_context_data provide the needed extension points.

This sounds reasonable to me.

> I'll post some updates here and on the ticket, in the meantime I'm
> open to comments and suggestions.

Great! Thanks for working on this.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: secret key from file...

[Carl Meyer]

On 03/22/2011 07:05 PM, Ian Kelly wrote:
> On Tue, Mar 22, 2011 at 4:49 PM, Matt Robenolt
>  wrote:
>> Why not just do an import for your custom settings?
>>
>> try:
>>from site_settings import *
>> except ImportError:
>>pass
> 
> No particularly compelling reason that I know of, the import machinery
> is just unnecessary in this case.  The site_settings.py is viewed as
> an extension of the settings.py, so it doesn't need to be loaded as a
> module in its own right.  And for the same reason we know exactly
> where we expect the file to be, so there's no need to consult
> sys.path.
> 
> I suppose it just comes down to a matter of taste.

Interesting. I would have assumed that the reason is so that code in
site_settings.py has access to the previously defined values in the main
settings.py, and can actually modify them (i.e. append to
MIDDLEWARE_CLASSES or whatnot). With an import this is not possible.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Improving Class based views names

[Carl Meyer]

On 03/21/2011 09:14 AM, daonb wrote:
> On Mar 20, 4:49 am, Carl Meyer <c...@oddbird.net> wrote:
>> Those last five characters in "get_context_data" actually serve quite a
>> useful purpose, IMO. They clarify that the return value is just the data
>> that will go into building a context (a dictionary), as opposed to being
>> the Context or RequestContext object itself, which is what I'd expect
>> from a method named "get_context".
> 
> Good point. I might be splitting hairs, but _data isn't clear enough -
> both a dict and a Context objects satisfy `data`. Looking at
> RequestContext code, I found __init__ gets a `dict` parameter, so how
> about making it get_context_dict?

Sure - except that these decisions were made several months ago, and
these kinds of considerations needed to have been raised then. We've had
a beta and an RC release with these names, and will have 1.3 any time
now; there's simply zero chance that we'll break things for everyone
who's already started working with CBVs in order to make tiny
improvements to method names.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Improving Class based views names

[Carl Meyer]

Hi Benny,

On 03/19/2011 05:41 PM, daonb wrote:
> Migration to the beta was quite smooth except for two names that broke
> my code: `pk` & `get_context_data`. The first comes from `models` and
> is now used instead of `object_id` in urls and views. It also broke
> compatibility with django-backlinks and I was forced to support both
> `pk` and `object_id` in the view wrapper, ugly. The second name can be
> rinsed and lose it's last 5 chars.

Those last five characters in "get_context_data" actually serve quite a
useful purpose, IMO. They clarify that the return value is just the data
that will go into building a context (a dictionary), as opposed to being
the Context or RequestContext object itself, which is what I'd expect
from a method named "get_context".

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Homogenization of User and UserProfile

[Carl Meyer]

On 03/18/2011 08:09 AM, Alexander Schepanovski wrote:
> I think using subclass of User model for your fields has the same
> flexibility as separate profile model (since you can have only one).

You can have multiple subclasses of the User model (not that I recommend
subclassing).

> contrib.auth can be fairly simply adjusted to use custom User model
> from settings instead of always using its own User.

Yes, it could, but the entire ecosystem of reusable apps with FKs
pointing to contrib.auth.models.User can't.

> In that sense I am totally for homogenization because it will make
> code cleaner and eliminate unnecessary sql request not because of
> serialization.

Subclassing doesn't solve the extra SQL request unless the base User
model is turned into an abstract model, which would completely break
third-party FKs.

"Custom User model" is definitely a problem I'd like to see solved, but
to do it in a way that's backwards-compatible and allows reusable code
to point FKs at User is a difficult problem that will require adding
significant new indirection capabilities to the ORM, and no-one has yet
proposed a full solution AFAIK.

> Such code is really an eyesore:
> first_name = user.first_name
> middle_name = user.get_profile().middle_name
> last_name = user.last_name

I agree it's ugly, but if it bothers you it's fairly easy to wrap it up
into properties on your profile that proxy to the User model, and then
just always use your profile (you can even add a middleware that creates
a lazy-profile attribute directly on the request, if you want).

> What do you do in such situation? Create a profile class containing
> all required fields? And then if you update some app you need to
> update your composite UserProfile model?

Don't use AUTH_PROFILE_MODULE or .get_profile(). As far as I'm concerned
they bring almost nothing to the table except for the "there can be only
one" restriction, and I'd be just as happy to see them deprecated at
some point. The only justification I've heard for them is that they
allow reusable code to access the user profile, but I'm not sure what
reusable code is going to usefully do with a custom profile model when
it has no idea what properties it has.

Just use OneToOneField and the regular ORM access descriptors, and you
can have as many "user profiles" as you need.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Default project layout / directory structure

[Carl Meyer]

On 03/17/2011 05:36 AM, Tom Evans wrote:
> I strongly disagree here. Django shouldn't be doing magic with my
> PYTHONPATH, if I want stuff in my PYTHONPATH, I'll darn well put it
> there.

Hear, hear! I'm only a weak +0 on the entire idea of documenting a
"standard" project layout, as I think it's highly idiosyncratic and
dependent on the nature of the project. But I am strongly -1 on any
proposal to introduce any more sys.path-mangling anywhere in Django. I'm
eager to remove the bit that is already there.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django urls in JavaScript

[Carl Meyer]

Hi Marco,

On 03/17/2011 10:44 AM, Marco Louro wrote:
> I don't really know how to approach this issue the best way, so I'm
> just going to be direct, I'd love to see Django provide support for
> django.core.urlresolvers.reverse (at least) on the client (in
> JavaScript).

I think this is really interesting, and I'd like to play with your
approach. It does seem to me like something that, while quite useful for
some projects, not all projects will need, and that it ought to stand on
its own as a reusable app for quite a while to prove its usefulness and
shake out any issues (and probably just stay that way).

Just to give you a point of comparison, database migrations are a
feature that is much more core to Django than this, and even those were
left out of core for years to allow several different approaches to sort
themselves out. Even now that South is the clear winner in that space,
only a portion of South is currently being considered for core inclusion.

I don't immediately see anything that would prevent implementing your
proposal as a reusable app outside core. If you do run into problems
where a small change in core could make your life a lot easier in
implementing this as a reusable app, please do bring it up here or file
a bug.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

[Carl Meyer]

On Mar 16, 11:43 am, Christophe Pettus  wrote:
> On Mar 16, 2011, at 2:24 AM, Johannes Dollinger wrote:
>
> > I would be nice if support for composite primary keys would be implemented 
> > as a special case of general composite fields.
>
> It's appealing, but the reality is that no existing back-end actually has 
> such an animal as a composite field.  In all of these cases, what we're 
> really creating is a composite index on a set of standard fields.  
> Introducing a more powerful index-creation syntax into Django isn't a bad 
> idea, but we shouldn't call it a "field" if it is not.

I'm not expressing an opinion one way or another on composite primary
key syntax, but I don't agree here that a Django model "field" must
map one-to-one to a database column. It already does not, in the case
of ManyToManyField, and at some point I would like to introduce
(irrespective of composite primary keys) a more general ORM
abstraction for composite fields (i.e. model Fields that map to more
than one database column) as a path to cleaning up the implementation
of GenericForeignKey.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: QuerySet subclass based on namedtuple()

[Carl Meyer]

Hi Alexander,

On 03/15/2011 03:30 AM, Alexander Schepanovski wrote:
> It would be nice to have a QuerySet subclass based on namedtuple().
> namedtuples takes less memory than dicts (from ValuesQuerySet), much
> more convenient  and induce more readable code than tuples
> (ValuesListQuerySet).
> 
> Namedtuples use same dot notation as model instances. So the same
> simple enough code can operate on both. You can easily upgrade/
> downgrade your queryset when needed.

This is an interesting idea. I think it falls into the category of
things that can be implemented outside core as a reusable library to
prove its usefulness before being considered as a core change.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: template-based widget rendering

[Carl Meyer]

Hi Bruno,

On 03/14/2011 06:33 PM, Bruno Renié wrote:
> Although Django 1.3 is not released yet I'd like to take advantage of
> the pycon sprints to discuss a proposal for 1.4: render form widgets
> using Django templates instead of python code.
> 
> This approach is implemented in django-floppyforms [0] (I'm the
> author): each widget gets a template_name argument, and the render
> method of each widget renders this template. Floppyforms also provides
> hooks to inject data in the template context, making it quite easy to
> implement custom widgets: add some context data, define a custom
> template and you're done.
> 
> This approach has a couple of advantages and drawbacks.
> 
> Advantages:
> 
>  - Templates give you escaping for free, giving us better protection against 
> XSS
>  - Template authors can alter widget rendering more easily
>  - This would also fix the "how do I specify my own doctype?" issue:
> Django can provide XHTML-style  elements and people can switch
> to html4-style s by providing their own set of templates.
> 
> Drawbacks:
> 
>  - Template rendering is slower than string interpolation. There is a
> benchmark in the floppyforms source tree, for a simple form the
> rendering takes 9 milliseconds instead of 1.5 milliseconds. With
> template caching enabled, it drops to 3 milliseconds. It still takes
> no time but I can see it being an issue.
>  - We need to add a template loader to provide the default form
> templates without asking the users to add 'django.forms' to their
> INSTALLED_APPS.
> 
> Backwards-compatibility is not going to be a big issue: the default
> templates can implement django's current behaviour.
> 
> We've been discussing it with Jannis and Carl and I'm trying to start
> a broader discussion. A side-effect of such a change would be to make
> the widgets API public and provide hooks for customization. In
> floppyforms I've been trying to follow the same conventions as the
> class-based views, using template_name and get_context_data as
> extension points.
> 
> I'm happy to work on the patch and convert the admin widgets but I'd
> like to hear people's voices about:
> 
> 1) If such a change is desired, and how we can make it faster,
> 2) how the API should differ from floppyforms,
> 3) make sure no significant use case is forgotten.

As we've already discussed here at PyCon, I'm +1 on this change. It
makes forms far more flexible and usable by template authors, and I
think that will benefit almost all Django users. It's more consistent
with Django's philosophy that presentation issues should be accessible
to template designers who are not necessarily Python programmers.

There is the speed tradeoff, and I sure hate to do anything that we know
is slower. But I feel like this is a case of "do it right, then make it
fast" - and I just think this is clearly the right way to do it.
Probably the right way to make it fast is to make the template engine
fast (Alex Gaynor had some interesting proposals for that for last
year's GSoC, though it got dropped in favor of the no-SQL stuff). In
absolute terms, I think the speed difference is already small enough
that only a small fraction of users will be impacted by it, and they
would always have the option to replace the default widgets in their
forms with their own faster versions.

The auto-escaping is another significant advantage - it's quite easy to
forget to escape something that should be escaped in a custom widget (we
had such a bug in Django itself until the latest security release), and
I think it can only be good for security to discourage people ever
building HTML strings in Python code.

I already use floppyforms on all projects, and don't have any issues
with the widget API it provides.

> This could be part of a broader change related to form rendering (e.g.
> fieldsets, looping over fields, etc) but I haven't thought about this
> enough to have a concrete proposal. For more information, see that
> discussion that happened at Djangocon:
> 
> https://github.com/SmileyChris/django-forms/wiki/Django-form-API-ideas

Although these proposals are related, and will involve some similar
changes (i.e. the need to be able to load default form templates), I
think the broader questions about form-rendering tags still need more
work, and converting the built-in widgets to use templates doesn't need
to be delayed while we hammer all of that out.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Expensive queryset cloning

[Carl Meyer]

Hi Alex,

On 03/14/2011 08:49 PM, Alexander Schepanovski wrote:
> Personally, I would like all querysets mutate not clone by default.
> And when one need a clone just make it explicitly.

This is not an option. It will break quite a lot of existing code, and
often in highly confusing ways. You'll need to find other ways to
optimize. I'd be surprised if the cloning of querysets is actually a
significant bottleneck relative to the database query itself, unless
you're doing loops with hundreds or thousands of clones, in which case
it can almost certainly be optimized via Q objects.

If you really think it will make a significant difference for your app,
you can probably achieve this for yourself by using your own subclass of
QuerySet and overriding the _clone method.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: type-field model inheritance

[Carl Meyer]

Hi Simon,

On Mar 4, 3:27 am, Simon Meers  wrote:
> +1 for better polymorphic support in Django core; it is a very common
> problem which could do with an efficient and elegant solution.
> Regarding efficiency, if you can keep track of your subclasses
> effectively (potentially using a registry if not introspection), you
> can use select_related to retrieve heterogeneous multi-table
> inheritance models in a single query (see example in [1]). I haven't
> looked deeply enough into django_polymorphic to see if it includes
> such optimisations. I'm sure we could further tweak the internals to
> make things more efficient and developer-friendly in this regard.

FWIW, django-model-utils [1] includes an InheritanceManager that
implements polymorphic queries in a single query via select_related.

  [1] https://github.com/carljm/django-model-utils

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: type-field model inheritance

[Carl Meyer]

Hi Craig,

On Mar 4, 1:03 am, Craig de Stigter  wrote:
> It looks like django_polymorphic does what I want. I'm not yet sure why it
> says it takes one query per type of model in a queryset. Unless it is
> talking about multi-table inheritance, in which each type would require a
> different join. But I'll look in more detail in the next few days and no
> doubt it will become clear.

Since multi-table-inheritance is the only kind of inheritance (apart
from abstract/proxy) supported by Django's ORM, I don't know what
other type of inheritance django_polymorphic would be referring to...

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Problem due to contenttype cache

[Carl Meyer]

On Mar 4, 5:55 am, Rahul  wrote:
> When i ran test cases of multiple_databases model of regressiontests
> then "test_generic_key_deletion" test case gave Error at the point
> where it was trying to create Review object using "other" db,
> but when i ran "test_generic_key_deletion" test case only then it
> passed perfectly.
>
> When i narrowed the problem and run this test case with
> "test_generic_key_cross_database_protection" then even it gives Error
> at same line.
>
> In my investigation i found that in
> "test_generic_key_cross_database_protection" test case, one Reiview
> object is getting created and because there exist FK relationship with
> ContentType, content type record for this is getting populated in DB
> and content type lookup are cached. At the end of this test case,
> content type record gets cleared by rollback, but content type lookup
> still remain in cache, so that when inside "test_generic_key_deletion"
> it tries to create Review object it gets content type from the cache
> and does not have  to go to  DB. Next when it tries to populate review
> table, Integrity error is raised as it has not made entry to content
> type table.
>
> I also tested it with InnoDB and getting same error here as well.
>
> Could you please let me know if you wants to let it gives error for
> this test case for InnoDB or I am running this wrong way.

Any failed test on a supported backend (which includes InnoDB) is a
bug. Please file a ticket, including this information and the full
settings with which you're running the tests. Thanks!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Delete cascade up for Inherited models

[Carl Meyer]

Hi Rahul,

On Feb 21, 12:56 am, Rahul  wrote:
> When i ran the test cases ( "modeltests/delete/
> test_inheritance_cascade_up" ) which were doing cascade up for
> inherited models, it failed for DB2 cause of the fact that DB2 doesn't
> support initial deferred constraint checks like behavior, hence it is
> must to execute delete statement from child to parent, however when I
> traced the sql execution for "modeltests/delete/
> test_inheritance_cascade_up" test case, what I find is Django tried to
> execute delete sql from parent to child. I already had the flag
> "can_defer_constraint_checks" set to false inside DatabaseFeatures.

What version of Django are you developing against? This sounds very
much like #15118 [1], which is present in the 1.3 beta but was fixed
in trunk over a month ago by r15246 [2].

Carl

  [1] http://code.djangoproject.com/ticket/15118
  [2] http://code.djangoproject.com/changeset/15246

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.