Re: Don't assume that missing fields from POST data are equal to an empty string value.

[Tai Lee]

>From my perspective, this issue is about silent data loss, which is about
one of the worst bugs you can have and one that Django typically tries very
hard to avoid or fix, even if it breaks compatibility.

It is extremely easy to cause silent data loss with the behaviour being
discussed. For whatever reason, if a field is not provided in the request
data, then Django is assuming it is an empty string and overwriting
existing data.

Sure, if you're in complete control over your backend and frontend and you
explicitly tell your model forms which fields to *include* (not exclude),
you can protect yourself against *one* scenario where additional fields are
added to a model but the developer forgets to update the corresponding form
class.

But that's not the only scenario. If you *exclude* fields, then new fields
added to the model will still be included automatically, and if the form
data doesn't include them, there could be silent data loss.

And the client submitting data could be completely outside the control of
the backend developer. Either it's managed by another developer or another
team, or it's entirely 3rd party code that the backend developer doesn't
even know exists. Explicitly defining which fields to include or exclude
won't help prevent silent data loss here.

Cheers.
Tai.


On Tue, Jan 23, 2018 at 10:13 AM, Curtis Maloney 
wrote:

> On 01/22/2018 06:03 PM, Anupam Jain wrote:
>
>> Wow - I just realised that we have been losing data for sometime on our
>> web platform since there were some fields in the ModelForm that were hidden
>> and not being sent. They were all being overwritten as blank values. Thank
>> God, we use django-reversion to track changes. Will take us sometime to
>> recover the data though.
>>
>
> Just something vaguely related that this post prompted in me ...
>
> My general guide is... if you're using hidden fields, then you're probably
> doing it wrong.
>
> There are very few valid cases for passing data in hidden fields. In most
> cases you really want to exclude the fields from the model. This has the
> added benefit of protecting from a malicious user who edits the values of
> the hidden fields.
>
> A ModelForm will _only_ update fields on the model that it has fields for.
> No matter what the user - or your code [such as clean functions] - manages
> to get into the cleaned_data dict.
>
> --
> Curtis
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/to
> pic/django-developers/w8UKCLjOMpg/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/django-developers/f06e134e-f596-3938-0bdf-daea0a56d505%40tinbrain.net.
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAEJB-EOZX8BuTAXKX63zUmP3XwtaPuwSgajXF7gO%2BVBd%2BHJ2hQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: "Project" vs "App"

[Tai Lee]

I'm not sure about saying that a project is a "container for apps", and 
that a project contains "only" settings and "no database models".

At a basic level, a "project" can be just a settings module, but it may 
include other things.

Apps that are sub-packages within a project are generally not pluggable or 
re-usable in other projects. Pluggable apps that are developed along side a 
project can and should ideally be in their own repo, but definitely should 
have their own top level namespace.

A project should only "contain" apps that are tightly coupled to the 
project and/or each other. In this case the use of apps is more to to make 
navigating a large code base easier, than to facilitate re-use and the 
creation of pluggable apps.

A "project" can also be an "app" and define database models. You just add 
your "project" to `INSTALLED_APPS`.

I often find it convenient for the "project" to also be the main "app". It 
seems redundant to have two packages, one for the "project" and one for the 
"app" when they would both logically derive their name from the site.

To avoid a top level namespace conflict, I have often seen generic top 
level package name like `project` or `djangosite`, which make it difficult 
to then install one site's code as a dependency for another site. Or we end 
up needlessly repeating the name and having the app as a sub-package (e.g. 
`foo.foo`).

The best options if they are to remain separate seems to be to have a 
generic sub-package for the project (e.g. `foo.project`) or app (e.g. 
`foo.base` or `foo.core` or `foo.main`), which allows `foo` to be more 
easily used as a dependency in another project.

The former seems a little counter intuitive, as one might assume that a 
project is a container for apps (though it is not always and doesn't have 
to be).

The latter is problematic because "base", "core" and "main" are not very 
good app names. In that case, you'd likely want to set the app label for 
your models and the verbose name for your app config to "foo" anyway, so 
you might as well just make `foo` your app as well as being your project.

The things most likely to be confusing when treating a project as an app 
are:

1) The project's root URLconf and the app's URLconf.

An app's URLconf doesn't normally include URLconfs from other apps, but a 
project's root URLconf does. This confusion could largely be eliminated by 
changing the name of the default root URLconf in the startproject template 
and in the docs to `root_urls.py`.

This makes it clear what's what, and would allow the app URLs to be used 
more easily in other projects.

2) Project `TEMPLATE_DIRS` vs the `templates` directory in an app.

Although to be fair, I've seen many developers struggle just as much (if 
not more) with where to save and where to find templates when the project 
and main app are completely separate. It can take a while for newcomers to 
understand the difference, and then they often remain unsure of where they 
should save a new template and where they should look to find a template 
they need to edit.

I think this problem would be reduced by having the project also be the 
main app, and for its `templates` directory to be `TEMPLATE_DIRS[0]`. 
Ancillary apps would still have their own `templates` directories, but for 
the most part people would only need to look in one place to edit a 
template for the project, and they would be less likely to save a new 
template in the wrong place.

When used as a dependency in another project, all the templates would be 
available (not only app templates), but they would no longer be in 
`TEMPLATES[0]` so the other project could easily override anything. With 
separate projects and apps, the other project might not have access to all 
the templates, unless they explicitly add the first project to 
`TEMPLATE_DIRS`.

Cheers.
Tai.


On Thursday, November 19, 2015 at 8:54:40 PM UTC+11, guettli wrote:
>
> I created a ticket to find a better definition of "Project" vs "App"
>
> https://code.djangoproject.com/ticket/25748
>
> I am happy since Tim Graham accepted it.
>
> Here are the current docs: 
> https://docs.djangoproject.com/en/1.8/ref/applications/#projects-and-applications
>
> Here is my view of Project" vs "App". It would be nice to find a consensus 
> and update the docs.
>
> Project
> ==
> A project is a container for apps.
> It contains only settings, no database models.
> Since it contains no database models it does not contain database schema 
> migrations.
> It can contain migrations which fill a database with project specific data.
> It is common that there is only one production installation of one project.
> It is common to have several stages (dev, test, prod) for one project.
> A project might contain a sitecustomize.py
>
> App
> ===
> An app can have models, views and code.
> It should be re-usable.
> An app can depend on other apps.
> It must not depend on a project.
> An app can contain a settings.py for testing, but it contain

Re: future of QuerySet.extra()?

[Tai Lee]

I think that while `extra()` gets a bad rap, it is extremely useful when 
creating generic pluggable apps, and I for one would miss it if it were to 
be removed.

In a comment on #7231 a few years ago I wrote:

I can't reply to the discussion linked on Google Groups anymore, but wanted 
> to add my comment that I have found many uses for .extra() and also hit the 
> limitation of being unable to add left outer joins to a query generated by 
> the ORM. I would not like to see it disappear in favour of raw SQL.
>

> My current use case is for joining reverse generic relations on queries 
> against a model that does not have a GenericRelation field back to the 
> generic model (because I don't know which models it will be used with -- I 
> can't add models to 3rd party code).
>

> To require the whole query in raw SQL in order to join generic relations 
> would mitigate the pluggability of the generic model in question.


While `raw()` may be allow you to get model instances from any custom SQL, 
it will often be prohibitively complicated or impossible to provide generic 
queryset methods that extend 3rd party apps. The RawSQL expression example 
given would appear to execute a subquery for each row of the result set. It 
wouldn't seem to help with the example above, unless I'm missing something?

Cheers.
Tai.


On Monday, August 3, 2015 at 5:27:14 PM UTC+10, Anssi Kääriäinen wrote:
>
> You can annotate raw SQL with expressions. I think the only case that 
> can't be done with expressions is addition of extra tables to the query. I 
> am certain we will get a solution to this too in future releases.
>
>  - Anssi
>
> On Friday, July 31, 2015 at 9:01:45 PM UTC+3, Andres Osinski wrote:
>>
>> Would the expressions API be able to define fields that are aggregations 
>> of complex foreign relationships? I keep using it when I need fields that 
>> are the result of weird aggregations over multiple JOINs with some unusual 
>> stuff like CASE fields and subqueries.
>>
>> On Fri, Jul 31, 2015 at 2:58 PM, Tim Graham  wrote:
>>
>>> In light of the new expressions API, the idea of deprecating 
>>> QuerySet.extra() has been informally discussed in IRC and elsewhere. I 
>>> wonder if there is consensus to mark extra() as "unmaintained" and to 
>>> suggest filing feature requests for functionality that can be performed 
>>> through extra() but not through other existing QuerySet methods? There are 
>>> at least several tickets (examples below) of edge cases that don't work 
>>> with extra(). It seems like a waste of time to leave these tickets as 
>>> accepted and to triage new issues with extra() if they won't be fixed.
>>>
>>> https://code.djangoproject.com/ticket/24142
>>> https://code.djangoproject.com/ticket/19434
>>> https://code.djangoproject.com/ticket/12890
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/6e1be326-3b17-49ca-accf-03eec5ad41ef%40googlegroups.com
>>>  
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> -- 
>> Andrés Osinski
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0613b9fc-358d-4aec-99e3-cec2d4fe9894%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: URL namespaces

[Tai Lee]

I still think that even though `get_absolute_url` might be disliked these 
days for various reasons, it is very common in the wild and is a perfect 
example of a legitimate need to be able to reverse URLs in code that 
executes in response to a request, but that doesn't have the request passed 
to it as an argument.

So I'd still like to consider the option of adding a default `current_app` 
hint to thread local storage (which has a life cycle of one request), and 
for `reverse()` and therefore also `{% url %}` to use that hint as a 
default. It could still be overridden by explicitly passing `current_app` 
to `reverse()` (and also `{% url %}`).

This could be done with a middleware class, and in fact a quick search 
reveals there are many django snippets and blog posts out where users have 
re-implemented a `ThreadLocalRequestMiddleware` or similar. But I would 
prefer to see it done before middleware is applied to make it non-optional, 
since the value will be read in `reverse()` which can and is often called 
completely outside the request/response cycle.

For any such code that executes outside the request/response cycle, I think 
we could provide a context manager that sets the `current_app` default hint 
on `__enter__` and restores it on `__exit__`.

Django appears to use thread locals already to set the current active 
language. It's been said that is more of a wart than a design pattern to 
follow, but clearly it can be used safely in Django, and just like the 
current active language, having apps that are able to resolve their own 
URLs properly and consistently is an important feature.

Here's a relevant conversation from a couple years 
back: https://groups.google.com/forum/#!topic/django-developers/VVAZMWi76nA

Cheers.
Tai.


On Monday, June 8, 2015 at 5:29:56 AM UTC+10, Marten Kenbeek wrote:
>
> So 2) has been fixed (#24904 and #24906), and I got a PR for 1) (
> https://github.com/django/django/pull/4724). 
>
> About #24127: I see a few options here:
>
>
>1. A context processor or similar that sets `current_app` if it isn't 
>set. By default `current_app` doesn't exist, so there's a good distinction 
>between not set and set to `None`. This would be 100% backwards 
> compatible, 
>but IMO it's not the best solution in the long term. 
>2. Provide a new flag to {% url %} to reverse with current_app set to 
>request.resolver_match.namespace. This feels slightly less awkward than 
> the 
>current method recommended by the docs.
>3. Deprecating a non-existing current_app to force an explicit choice 
>between the old and new behaviour, then add the new behaviour after the 
>deprecation period. This would be coupled with option 1 for ease-of-use. 
>4. A setting? A setting! Yay, another setting...
>5. Document the slight backwards-incompatible change? And provide an 
>easy way to retain the old behaviour, of course. 1 through 4 are all far 
>from ideal, and I don't think there's an easy, backwards-compatible fix 
>here. 
>
> Thoughts?
>
> Op dinsdag 2 juni 2015 14:32:42 UTC+2 schreef Marten Kenbeek:
>>
>> The admin proves problematic. Looking at the uses of `AdminSite.name`, 
>> it's not easily replaced. There are quite a few places that use the name to 
>> reverse urls, but don't have access to the request and the current 
>> namespace. I think the best solution for now is to document that you should 
>> pass `admin.site.urls` directly to `url()`. `include()` does a bunch of 
>> stuff, but none of it is needed in the case of the admin urls. This allows 
>> the new `include()` API to be as intended, but it doesn't put an 
>> unnecessary burden of providing the "correct" namespace for an admin 
>> instance on the end-developer. 
>>
>> In time I'd like to see a method on the AdminSite that returns an actual 
>> resolver object, using the new API. The default URLconf template would 
>> simply look something like this:
>>
>> urlpatterns = [
>> admin.site.get_resolver('^admin/'),  # or 
>> get_resolver(RegexPattern('^admin/')) to be more explicit
>> ]
>>
>> This would solve the namespace issue for the admin, but it would still 
>> allow for only obvious way to set the instance namespace, and keep the 
>> `include()` API clean.
>>
>> Op zaterdag 30 mei 2015 14:28:22 UTC+2 schreef Tai Lee:
>>>
>>> Right. So if I understand correctly, you can avoid problems even when 
>>> installing two different apps into the same app namespace as long as you 
>>> *always* supply `current_app` when reversing URLs? In which case, I don't 
>>> think we need to worry about

Re: URL namespaces

[Tai Lee]

>
> An app doesn't necessarily have to use its own patterns. With that said, 
> it's up to the user to make sure it works if they are to mess with the 
> app_name. Two conflicting apps can be installed with different namespaces, 
> but there will always be a default (the last one, if there is no namespace 
> with the same name). With a consistent use of `current_app` (even if it's 
> not the *current* application used in the request), you can avoid problems.
>

Right. So if I understand correctly, you can avoid problems even when 
installing two different apps into the same app namespace as long as you 
*always* supply `current_app` when reversing URLs? In which case, I don't 
think we need to worry about being able to change the app namespace to 
avoid conflicts (maybe we should even disallow it), and instead just make 
it easier to *always* supply `current_app`, which brings me to my next 
point.

If possible, I'd like to add `current_app` to the request by default, but 
> that has been attempted before, and backwards compatibility is a problem. 
> That would allow an app to more reliably reverse its own urls, provided 
> that their code and templates are called from within their own application.
>

I'd take this a step further. In this comment -- 
https://code.djangoproject.com/ticket/21927#comment:9 -- I made a 
suggestion that Django set a `current_app` hint in thread local storage 
with a middleware class or even before middleware runs, and then have 
`reverse()` and `{% url %}` fallback to that default hint ONLY if no 
current app is explicitly provided via the current mechanisms, so it should 
be a relatively safe change.

This should make it much more convenient and possible to reverse URLs 
correctly regardless of conflicting app namespaces, even in code that 
doesn't have access to the request object. For example, model methods that 
are executed in templates and do not have access to a `request` or 
`RequestContext` object. As far as I know, 1 thread = 1 request, so using 
thread local storage should be safe to provide such a hint for any code 
that executes in response to a request. For management commands, it would 
still need to be supplied explicitly. What do you think?

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ba7b83e0-8961-45f9-89f0-e45ce25224fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: URL namespaces

[Tai Lee]

Thanks for taking the time to write up this proposal. I agree on pretty 
much all counts, but I do have a few comments:

1)

If an app has templates and view code that are reversing 
:, then changing the app name (e.g. by an undocumented 
feature, passing a (patterns, app_name) tuple), then all those URLs will 
break. Unless I'm missing something, an app *must always* be installed with 
the app name that is hard coded in its templates and view code, so there is 
no workaround for app namespace conflicts?

Does or would it work to allow two apps with the same app namespace, as 
long as they both are installed with customised instance names, so that the 
"default app" namespace lookup doesn't return anything, and Django must 
fallback to the `current_app` to determine which app is being reversed?

2)

I'd rather see the "app name, period" (as opposed to a default app name) 
defined as a variable in an app's URLconf as per #11642 or as an attribute 
of the `urlpatterns` object, that Django looks at when a URLconf is 
included with a dotted path string, instead of requiring users to import 
and then include a `(patterns, app_name)` tuple. Basically, the app name 
defined by the app should be used for both of:

include(r^foo/', 'foo.urls')

and

from foo.urls import some_tuple
include(r'^foo/', some_tuple),

3)

Not exactly related to the issue at hand, but if we are reworking URL 
namespaces it might be worth some consideration. One reason why I always 
avoid using URL namespaces these days is that it becomes impossible to 
override an app URL at the project level. Project authors are stuck with 
the exact URLs defined by the app author, all located below the same root 
path where the URLs are included and with the same view behaviour.

Without namespaced URLs, the project author can include their own version 
of a URL at any location (as long as it comes before the app's included 
URLs) and change the view behaviour. For namespaced URLs, I'd like to be 
able to do something like:

urlpatterns = patterns('',
url(r'^my_foo_login/$', 'myapp.views.my_foo_login', name='login', 
app='foo', namespace='foo'),
url(r'^foo/', include('foo.urls', app='foo', namespace='foo')),
)

Then when templates or view code in the `foo` app does reverse('foo:login') 
they will get `/my_foo_login/` instead of `/foo/login/` and I can change 
the behaviour of the view.

Cheers.
Tai.


On Friday, May 29, 2015 at 1:21:58 AM UTC+10, Marten Kenbeek wrote:
>
> Sure, I'll go through the list of url tickets on Trac. 
>
> The key difference with #11642 is "default". What I propose is that 
> urls.py specifies the app_name, period. The only reason to change app_name 
> is conflicting app names, and there are easy workarounds that I feel 
> shouldn't be part of the API itself. The namespace would be specified in 
> include(), and default to the app_name. 
>
> Op donderdag 28 mei 2015 15:37:20 UTC+2 schreef Tim Graham:
>>
>> Point 1 sounds like https://code.djangoproject.com/ticket/11642 -- and 
>> that ticket says it may be superseded by 
>> https://code.djangoproject.com/ticket/21927. Could you review those 
>> tickets as well as the others in the "Core (URLs)" component of Trac? It 
>> would be good if you could assign yourself any tickets that your project 
>> will address.
>>
>> On Wednesday, May 27, 2015 at 5:58:00 PM UTC-4, Marten Kenbeek wrote:
>>>
>>> Hi all,
>>>
>>> URL namespaces have a few quirks that make them a bit difficult to use 
>>> and understand. I hope to create a patch that clears up these issues. 
>>>
>>> First up: the distinction between an application namespace and an 
>>> instance namespace. The docs say on the application namespace:
>>>
>>> This describes the name of the application that is being deployed. Every 
 instance of a single application will have the same application namespace.
>>>
>>>
>>> And on the instance namespace: 
>>>
>>> This identifies a specific instance of an application. Instance 
 namespaces should be unique across your entire project. However, an 
 instance namespace can be the same as the application namespace. This is 
 used to specify a default instance of an application. 
>>>
>>>
>>> The current implementation requires that both are specified in the same 
>>> place: either in the included url configuration by returning a 3-tuple, or 
>>> in the including url configuration through the arguments to include(). The 
>>> first case generally does not allow multiple deployments of the same app, 
>>> unless the included url configuration contains specific logic to return 
>>> different instance namespaces. The second case does not in any way enforce 
>>> that multiple deployments in fact have the same application namespace. 
>>>
>>> I'd like to get the semantics and the implementation in line, and 
>>> provide one obvious way to specify namespaces. Including a set of url 
>>> patterns under a namespace involves two parts: the including urlconf that 
>>> calls include(), and th

Re: Proposal: Manually login without authenticate

[Tai Lee]

I agree, why not always pass the single auth backend into the login 
function?

If the backend is inferred by a single value in the settings and not stored 
alongside the user ID, what would happen to existing users who are already 
logged in when a second backend is added to the settings and deployed? 
Django would no longer know which backend to use to fetch the authenticated 
user?

Cheers.
Tai.


On Saturday, May 23, 2015 at 6:58:35 AM UTC+10, Unai Zalakain wrote:
>
> Hi Paulo! 
>
> >If the application has only one backend we always infer it in the login 
> >function. If it isn't, the client needs to provide one explicitly. 
>
> Why not pass the single auth backend into the login function? It makes 
> the design and the documentation much simpler. 
>
> -- 
> unai 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0319a35d-b870-4edd-8805-a2707a8b0f58%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: An easier path to upgrade from one LTS release to another

[Tai Lee]

This sounds good. But will it significantly slow down the rollout of new 
features into Django that require deprecation? Also, could features that 
are deprecated in an LTS be dropped in the next non-LTS release? e.g. if 
1.8 still had a feature that was deprecated in 1.5, could it finally be 
removed in 1.9?


On Friday, May 8, 2015 at 1:53:59 AM UTC+10, Carl Meyer wrote:
>
> On 05/07/2015 08:53 AM, Tim Graham wrote: 
> > I think there is some merit to reconsidering the deprecation schedule as 
> > Anssi suggests. What I have seen is that most third-party apps didn't 
> > consider dropping support for the previous LTS (1.4) until the next LTS 
> > (1.8) was released. This meant that all these projects had to implement 
> > their own compatibility shims (instead of using Django's) once they 
> > wanted to add support for Django 1.6+ because the compatibility shims 
> > for deprecated features in 1.4 were dropped in Django. This resulted in 
> > ugly code with lots of conditional version branches, etc. I'll keep 
> > thinking about this as we decide the release schedule going forward. 
>
> I agree with this. I think by far the biggest thing we could do to make 
> it easier to go from LTS to LTS is to make it easier for third-party 
> apps to support two LTS releases at once. Guaranteeing that no API which 
> is un-deprecated in one LTS will be removed until after the following 
> LTS would make that massively easier. 
>
> Carl 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ebcfc899-fe81-481d-9c00-9e66b5967c56%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Explicit relative imports

[Tai Lee]

I also don't see much benefit in using relative imports. Code is read many 
more times than it is written, and I find having the full path makes it 
much easier to sort and visually scan imports, and to easily see at a 
glance exactly where to find what you need.

If they are to be used, I would recommend against any relative imports from 
parent packages. It should be limited to siblings only.

Cheers.
Tai.


On Friday, 14 November 2014 00:57:15 UTC+11, Tom Christie wrote:
>
> Contrary voice here, but I don't dig explicit relative imports, it's just 
> adds an extra decision point for no real benefit.
>
> Personally I always recommend full absolute imports, ordered strictly 
> alphabetically - there's then never any room for confusion or decision 
> making around how the imports should be written, and it's always tidy and 
> consistent.
>
> Not looking to necessarily change the Django project's stance on that, but 
> that's the style I use throughout my projects and it's trivial to stick to. 
>
> Cheers,
>
>   Tom
>
> On Wednesday, 12 November 2014 21:59:42 UTC, Jannis Leidel wrote:
>>
>>
>> > On 11 Nov 2014, at 22:51, Aymeric Augustin <
>> aymeric@polytechnique.org> wrote: 
>> > 
>> > Hello, 
>> > 
>> > We’ve started using explicit relative imports in newer parts of the 
>> Django source tree. They’re short and readable. That’s good. 
>> > 
>> > I would like to add guidelines about imports in the coding style guide 
>> in order to improve consistency. 
>> > 
>> > My inclination would be to recommend relative imports within 
>> “components” but avoid them between “components”, where a component is: 
>> > 
>> > - a well-defined sub-framework (django.core.cache, django.db, 
>> django.forms, django.template, etc.) 
>> > - a contrib app 
>> > - an app in the tests/ directory 
>> > - etc. 
>> > 
>> > I would discourage going back into parent modules with relative imports 
>> because statements such as `from ...spam import eggs` are hard to parse. 
>> > 
>> > You can see an example of this style in django.apps which has only 
>> three files. 
>> > 
>> > What do you think? 
>>
>> Yup, the way to go. 
>>
>> Jannis
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/36767be0-78fd-40f0-afdc-c3d0c7c121e9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Automatically get default value for "current_app", for easier namespace support.

[Tai Lee]

Right now it seems that for a generic app to support the possibility of 
being installed in a URLconf with a namespace, the app author will need to 
take care to explicitly define a `current_app` for every call to 
`reverse()`, `TemplateResponse`, `RequestContext`, `Context` and `{% url 
%}`.

Django already adds a `ResolverMatch` object to every request. Shouldn't 
Django just use this to get a default value for "current_app", whenever 
users don't explicitly define one?

This should make it almost a non-issue to define the current app for every 
case except explicit calls to `reverse()`, `Context` and templates that are 
rendered without a `RequestContext` object (as none of these have access to 
the request).

We could even get a sensible default in those cases as well by storing the 
current app in thread local storage, and using that as a default in 
`reverse()`.

I've made a ticket for this, but it was closed as wontfix because thread 
local storage is global state and Django is at war with global state.

https://code.djangoproject.com/ticket/22203

As suggested, I'm bringing this to django-developers to ask for any 
alternative suggestions that don't involve global state, and also to try 
and make my case for the ticket as originally described.

Django already uses `threading.local()` in a number of places such as 
`urlresolvers._prefixes`, `urlresolvers._urlconfs`, `CacheHandler._caches`, 
`ConnectionHandler._connections`, `trans_real._active`, `timezone._active`.

The most notably similar use case is probably for timezone support, which 
allows users to call `activate()` to tell Django what timezone they are in, 
and then other parts of the code call `get_current_timezone()` to get the 
value stored in thread local storage.

I think it would be along the same lines to have the ability to set a 
current app and have other parts of the code get the current app, without 
having to pass an object around as an argument every step of the way, which 
is practically impossible.

For example, models with a `get_absolute_url` method (or perhaps multiple 
`get_foo_url` methods) that are rendered in templates. These functions 
can't take arguments when rendered as context variables in a template, and 
have no way to obtain the current namespace from the `request` object.

This would make it super easy for users (via middleware) or Django itself 
to inspect the `request.resolver_match` object and set the current app 
early in the request/response cycle, and then `reverse()` and `{% url %}` 
would just work without generic app authors having to explicitly build in 
support for multiple instances of their app being deployed within a single 
project.

Does anyone else think this would be a good idea, or does anyone have an 
alternative suggestion?

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a7a8b2fb-91db-4b0a-ac1b-d76c5df38aa7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Update on localflavor move

[Tai Lee]

Hi Aymeric & Adrian,

I didn't see any further discussion or consensus on the issue of the 
generic localflavor. The 1.5 docs and (accelerated) deprecation of 
localflavor are a little hazy regarding the generic localflavor.

The 1.5 docs say that it hasn't been removed (yet), but doesn't say it will 
remain where it is, or move into core, or simply disappear with no 
equivalent external package.

The docs also said that (all of) `django.contrib.localflavor` would be 
removed, and any import below `django.contrib.localflavor` currently raises 
a warning with a message to use `django-localflavor-*` packages instead. 
But for generic, there is none.

So as it stands, anyone who uses `django.contrib.localflavor.generic` (all 
my projects do) will find that it has been removed in 1.6 with no 
equivalent in core and no equivalent external package.

I would prefer to see these reinstated in django core for 1.6, and the 
1.5.x docs updated with an explanation on where they can be imported from 
when upgrading to 1.6.

It's probably a bit much to ask for the non-biased international format 
fields to become the defaults, but I would definitely like to see these 
move into core before `django.contrib.localflavor` goes away.

It would seem a bit silly to me to create an external 
`django-localflavor-generic` package with 3 international (not locale 
specific) format fields and no other localisation. Django has always had 
great international support in core, and I doubt these three fields are 
much of a burden to maintain.

Also, as a separate issue I noticed that only a handful of the 
`django-localflavor-*` packages are on PyPI. This makes the method 
inconsistent and potentially more difficult (depending on your locale) to 
add them as requirements to a project.

Cheers.
Tai.


On Tuesday, 16 October 2012 07:17:43 UTC+11, Aymeric Augustin wrote:
>
>
> 1) Can we move the fields defined in 
> django.contrib.localflavor.generic.forms to django.forms? 
>
> Currently the US-biased fields are in django.forms and the non-biased ones 
> in django.contrib.localflavor.generic.forms — an interesting perspective :) 
>
> The alternative is to deprecate them entirely. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Streaming HttpResponse revisted. Any core devs, please take a look if you can :)

[Tai Lee]

Docs have been added (patch should be complete now) and pull request has 
been opened, at Anssi's request.

https://groups.google.com/d/topic/django-developers/RrNPfuJxnlM/discussion

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/iOkzMIonkdsJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Streaming HttpResponse revisted. Any core devs, please take a look if you can :)

[Tai Lee]

Hi Aymeric & Anssi,

Thanks for your feedback. I have updated my branch with an implementation 
of a new `HttpStreamingResponse` class plus tests and updated the ticket. 
Please take a look and let me know if this is moving in the right 
direction, and if there is any other feedback. If the approach is OK, I 
will add docs as well.

https://code.djangoproject.com/ticket/7581#comment:32
https://github.com/thirstydigital/django/tree/tickets/7581-streaming-response-middleware

Thanks.
Tai.


On Sunday, 26 August 2012 06:22:02 UTC+10, Aymeric Augustin wrote:
>
> Hi Tai, 
>
> Thanks for your work and sorry for the late answer. I just had the time to 
> review the discussion. 
>
>
> In my opinion, option 2 is reasonable. I'm -0 on a stream_content keyword 
> argument for HttpResponse. I +1 on adding a separate HttpStreamingResponse 
> class, because it better reflects the difference in behavior, and because 
> it makes it possible to define a different API from the regular 
> HttpResponse (for instance, we can chose not to define response.content). 
>  Then we can use a deprecation path to remove support for streaming 
> responses in the regular HttpResponse class. 
>
> Once the deprecation path completes, I expect that: 
> - HttpResponse will be simpler and more predictable (as it'll no longer 
> contain code to support streaming) 
> - both HttpResponse and HttpStreamingResponse will have intuitive and 
> well-behaved APIs (which isn't true right now) 
> - the implementations will be easier to understand and to maintain — 
> HttpResponse was one of the hardest classes to port to Python 3, so much 
> that I created a ticket saying it needs a refactoring. 
>
> At a high level, I think this is a good compromise. It is fully backwards 
> compatible for people that don't use streaming — the vast majority of 
> Django users. It provides a robust foundation for the minority who does. It 
> takes a bit more work than your current patch :) 
>
>
> In addition, I'm -1 on encoding include or exclude filters in the 
> MIDDLEWARE_CLASSES setting. Chains of includes and excludes are never a 
> good idea — anyone who's struggled with a complex Apache configuration 
> knows what I'm talking about! 
>
>
> Best regards, 
>
> -- 
> Aymeric. 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/PzjhRg-tlXMJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Streaming HttpResponse revisted. Any core devs, please take a look if you can :)

[Tai Lee]

Hi Ryan,

I don't think what you are suggesting would be necessary. Stream-capable 
middleware could avoid having to handle both cases by simply wrapping 
response.content with a new iterator in all cases, if they want 
to. Non-streaming responses can still be iterated. They just have fewer 
iterations (as few as one).

The only reason for a stream-capable middleware to access response.content 
explicitly in this case is for improved efficiency when the user doesn't 
care about streaming the response. I think GZipping the whole content is 
probably more efficient than GZipping chunks of it.

Cheers.
Tai.


On Friday, August 24, 2012 2:48:53 AM UTC+10, Ryan Hiebert wrote:
>
> Rather than requiring stream-capable middleware to check if it is stream 
> content, and also require them to handle non-stream content separately, can 
> we have .stream_content return an auto created generator with some sane 
> automatic divisions (at newlines or every x bytes/characters). This would 
> allow stream-aware middleware to act as if everything they do is streams, 
> even if in reality it is not. 
>
> If we don't do this, it seems likely enough that middlewares that are 
> stream-aware might end up doing this conversion anyway. If that's the case, 
> moving that work into .stream_content can save them some common boilerplate 
> code. 
>
> One possible problem with this approach would be that if there are many 
> middlewares, alternating back and forth between stream aware and stream 
> unaware, that the overhead might be significant. If it turned out that was 
> the case, then proper ordering of middlewares would be part of optimizing a 
> project, if it isn't already. 
>
> Ryan 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/0JVwyu9PAUQJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Streaming HttpResponse revisted. Any core devs, please take a look if you can :)

[Tai Lee]

I have updated the patch on my GitHub repository after some discussion on 
IRC with Anssi. The patch applies on master again now, it consumes 
generator content on access instead of on assignment, and raises a 
PendingDeprecationWarning if you create an HttpResponse with 
stream_content=True and then you try to access response.content. This 
should alert people to the fact that they have explicitly asked for a 
streaming response, but some piece of code somewhere has broken the ability 
to stream the response by accessing it's content directly.

So the end result should be that we fix the buggy response.content != 
response.content behaviour, we preserve the behaviour of existing responses 
and middleware when a generator is passed as content (stream if nothing 
accesses content directly, otherwise don't stream but still execute 
middleware), and either stream or don't stream but issue a warning when 
streaming is explicitly requested.

In 1.7 we could raise a loud exception when stream_content=True and 
response.content is accessed directly. Middleware can do nothing if they 
don't care about or need to worry about streaming responses. If they are 
capable of functioning with a streaming response, they can check 
response.stream_content and wrap the content generator with 
`response.content = wrap(response)`.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/PFXTllthoTkJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Streaming HttpResponse revisted. Any core devs, please take a look if you can :)

[Tai Lee]

After discussion with akaarai and mYk on IRC, I have two updated proposals.


OPTION 6:

1. Consume `HttpResponse.content` on first access, so that it can be read 
multiple times and eliminate the current buggy behaviour (e.g. 
`response.content != response.content`) when a generator is passed in as 
content.

2. Create a new `HttpStreamingResponse` class, which has no `.content` 
attribute but instead has `.stream` attribute, and can be iterated only 
once. If it is iterated a second time, an exception will be raised to 
trigger a loud failure.

3. Middleware that ships with Django that can work with streaming 
responses, is updated. E.g. GZipMiddleware can do `if isinstance(response, 
HttpStreamingResponse): response.stream = 
compress_sequence(response.stream)`.

4. The `MIDDLEWARE_CLASSES` takes a new optional syntax that allows project 
authors to nominate certain middleware classes for conditional execution. 
This means we don't need to add Yet Another Setting, but the project author 
-- who is the only person in a position to know what combination of 3rd 
party middleware is incompatible with responses returned by other 3rd party 
views -- is able to conditionally execute middleware classes by listing 
compatible (or incompatible) view functions and response classes.

MIDDLEWARE_CLASSES = (
'some.middleware',
'another.middleware',
('some.conditional.middleware', {
'exclude': ['some.incompatible.view.function', 
'some.incompatible.response.class'],
'include': ['some.compatible.view.function', 
'some.compatible.response.class'],
}),
)

This proposal should be backwards compatible, fix the current buggy 
behaviour for people who pass a generator as content to `HttpResponse` but 
don't explicitly want a streaming response, and allow people who do 
explicitly want a streaming responses to have them.

There may be other use cases for conditionally executing middleware, where 
the decision lies with the project author instead of the middleware author 
(who won't know in advance which apps their middleware will have to work 
with). I haven't tried to think of any.

If people return a `HttpStreamingResponse` from their view, and they have 
enabled some old or 3rd party middleware that doesn't know anything about 
this, and which expects to access `response.content`, the project author 
will *have* to nominate that combination of view/response/middleware as 
incompatible in their `MIDDLEWARE_CLASSES` setting to avoid a loud failure.

Which leads me to my next and favourite option...


OPTION 7:

1-2. As for option 6.

3. Add support for a new middleware method specifically for streaming 
responses. E.g. have middleware authors create a 
`.get_streaming_response()` method if their middleware is able to support 
streaming responses. Only `HttpResponse` objects will be passed to 
`.get_response()` middleware methods, and only `HttpStreamingResponse` 
objects will be passed to `.get_streaming_response()` middleware methods.

4. Add `.get_streaming_response()` methods to middleware that ship with 
Django that are able to support streaming responses, e.g. GZipMiddleware.

This should be backwards compatible. Existing code will behave exactly as 
it does now with the exception that `response.content == response.content` 
will be True, whether a generator was passed in as content or not.

No existing middleware will need to be updated unless they explicitly want 
to add support for the new `HttpStreamingResponse` objects.

It will be much clearer in the documentation what middleware authors need 
to do to support streaming responses (implement a new method, vs add a 
conditional branch of execution in their existing method).

It will be much clearer for project authors to see if a middleware class 
does support streaming responses (check for a `.get_streaming_response()` 
method vs hunting in the docs or inside the `.get_response()` method for a 
conditional branch of execution).

It will be clearer for for developers who want to explicitly stream a 
response (use `HttpStreamingResponse` instead of `HttpResponse`).

In the case of existing response middleware that doesn't care about 
accessing the content of a response, they will continue to work as they do 
now for `HttpResponse` objects, but they will need to be updated if they 
also want to work with `HttpStreamingResponse` objects. In that case, if 
they don't access the response content, `.get_streaming_response()` could 
just be an alias for `.get_response()`.


Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/wNAoFyXRuGQJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Streaming HttpResponse revisted. Any core devs, please take a look if you can :)

[Tai Lee]

It sounds like the preferred solution is still "don't do that", which I 
think corresponds most closely with option 1. Simply consuming generator 
content in `HttpResponse.__init__()` in all cases would prevent some of the 
surprising behaviour we are seeing, but it would completely break any 
support for streaming responses.

I thought that Jacob's "It's clear to me that we have to do *something* 
about this" comment when he marked #7581 as "accepted" would take "don't do 
that" off the table.

>
It might not count as a "backwards incompatible" change to revoke all 
support for streaming responses, as I couldn't find explicit support for 
them mentioned in the docs, but streaming responses have been around for a 
long time and people are relying on on this behaviour, as evidenced by 
several tickets and reported use cases (not limited to "large" responses, 
but also including "long running" responses).

Those people just have to avoid some core middleware that currently break 
streaming responses by prematurely consuming content, and Django doesn't 
provide a generic or documented way for people to hook into this part of 
the framework on a per-response basis. They just have to choose "all or 
nothing" when it comes to those middleware, unless they also have control 
over the middleware (which they don't for core, contrib and third party 
middleware).

I don't see a huge difference between wrapping or replacing the content of 
response, compared to wrapping or replacing the entire response. The key 
issue (for me) still remains, being that middleware would still have no way 
to know when it is appropriate for them to wrap or replace the entire 
response. It may also cause new complications with headers or properties of 
the wrapped response not being preserved.

Removing `HttpResponse.content` entirely is probably a non-starter, as it 
would be a backwards incompatible change. It has been part of the public 
API since at least 1.0.

Could somebody please explain their primary objections to option 2? This 
has a working up-to-date implementation already (just missing tests and 
docs, which I would be happy to contribute), is backwards compatible, fixes 
buggy behaviour in the general case where iterators are passed to 
`HttpResponse` as content but the developer does not explicitly want a 
streaming response, and it provides a hook for developers to explicitly 
tell middleware that the response should be streamed with 
`HttpResponse(stream_content=True)`.

I've seen it asked, but not answered yet (unless I missed it), if there is 
really a need for more fine grained control than "is it streaming, or not?" 
(e.g. a capabilities based API on responses). If not, I think this is 
currently the most likely candidate for inclusion.

Thanks.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/F5cUHy1oOnsJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Streaming HttpResponse revisted. Any core devs, please take a look if you can :)

[Tai Lee]

I'd like to re-visit the discussion surrounding #7581 [1], a ticket about 
streaming responses that is getting quite long in the tooth now, which 
Jacob finally "accepted" 11 months ago (after a long time as DDN) and said 
that it is clear we have to do *something*, but *what* remains to be seen.

I'd like to try provide a little refresher and summarise the options that 
have been suggested, and ask any core devs to please weigh in with their 
preference so that I can work up a new patch that will be more likely to 
gain acceptance.


THE PROBLEM:

1. There are bugs and surprising behaviour that arise when creating an 
HttpResponse with a generator as its content, as a result of "quantum 
state" of `HttpResponse.content` (measuring it changes it).

>>> from django.http import HttpResponse
>>> r = HttpResponse(iter(['a', 'b'])) 
>>> r.content 
'ab'
>>> r.content 
''
>>> r2 = HttpResponse(iter(['a', 'b'])) 
>>> print r2.content
ab
>>> print r2.content

>>> r3 = HttpResponse(iter(['a', 'b']))
>>> r3.content == r3.content
False

2. Some middleware prematurely consume generator content by accessing 
`HttpResponse.content`, which can use a lot of memory and cause browser 
timeouts when attempting to stream large amounts of data or 
slow-to-generate data.

There have been several tickets [2] [3] [4] and django-developers 
discussions [5] [6] [7] [8] about these issues.


SOME USE CASES FOR STREAMING RESPONSES:

A. Generating and exporting CSV data directly from the database.

B. Restricting file access to authenticated users for files that may be 
hosted on external servers.

C. Drip-feeding chunks of content to prevent timeout when requesting a page 
that takes a long time to generate.


OPTION 1:

Remove support for "streaming" responses. If an iterator is passed in as 
content to `HttpResponse`, consume it in `HttpResponse.__init__()` to 
eliminate buggy behaviour. Middleware won't have to worry about what type 
of content a response has.

Now that Jacob has accepted #7581 and said that it is clear we need to do 
*something*, I hope we can rule out this option.


OPTION 2:

Make `HttpResponse.__init__()` consume any iterator content, and add an 
`HttpResponseStreaming` class or an `HttpResponse(streaming=False)` 
argument. Allow middleware to check via `hasattr()` or `isinstance()` 
whether or not the response has generator content, and conditionally skip 
code that is incompatible with streaming responses.

Some middleware will have to be updated for compatibility with streaming 
responses, and any 3rd party middleware that prematurely consumes generator 
content will continue to work, only without the bugs (and potentially with 
increased memory usage and browser timeouts).


OPTION 3:

Build a capabilities API for `HttpResponse` objects, and have middleware 
inspect responses to determine "can I read content?", "can I replace 
content?", "can I change etag?", etc. This will likely become a bigger and 
more complicated design decision as we work out what capabilities we want 
to support. Some have argued that it should be sufficient to know if we 
have generator content or not, for all the cases that people have reported 
so far.


OPTION 4:

Provide a way for developers to specify on an `HttpResponse` object or 
subclass that specific middleware should be skipped for that response. This 
would be problematic because 3rd party views won't know what other 
middleware is installed in a project in order to name them for exclusion.


OPTION 5:

Add Yet Another Setting that would allow developers to define 
`CONDITIONAL_MIDDLEWARE_CLASSES` at a project level. At the project level, 
developers would know which middleware classes they are using, and when 
they should be executed or skipped. This would give very fine grained 
control at a project level to match middleware conditionally with 
`HttpResponse` subclasses, without requiring any changes to existing or 3rd 
party middleware. This could look something like this:

MIDDLEWARE_CLASSES = (
'django.middleware.common',
)

CONDITIONAL_MIDDLEWARE_CLASSES = {
'exclude': {
'django.http.HttpResponseStreaming': ['django.middleware.common', 
'otherapp.othermiddleware', ...],
},
'include': {
'myapp.MyHttpResponse': ['myapp.mymiddleware', ...],
},
}


MY TAKE:

I think that option 1 and option 4 are non-starters.

I think option 3 is perhaps a little overkill and will be more difficult to 
get committed once we start thinking about what capabilities we want to 
support.

I think option 2 is probably going to be the easiest solution. It's 
practically implemented and up-to-date already (missing docs and tests).

Although it does involve Yet Another Setting, I think option 5 provides the 
most flexibility, where it is most needed. It gives developers working at 
the project level a way to override and conditionally skip or execute 3rd 
party middleware without having to make any changes to 3rd party middleware.

I would be happy with eith

Re: ModelForm.Meta.overrides

[Tai Lee]

Overall, I like this patch. I think it's a little unbalanced that we 
provide `Meta.widgets` but no clearly documented way to override other 
properties on fields without replacing entire fields.

I think the doc changes are pretty helpful to users who are looking for a 
way to do this, and it is a good thing to treat all field attributes 
equally instead of special casing widgets.

I also think a class with overrides defined this way will be easier to read 
visually and easier to detect and manipulate programmatically than hacking 
`self.fields` inside `__init__()`, which people do now (myself included).

My only concern is the suggestion to use a callback as the key. I can see 
that this provides more flexibility for users who want to apply a single 
set of overrides to multiple fields, but I can see it being misused and 
abused. If people really need to do that, I think they can just construct 
their overrides dict programmatically and assign it to `Meta.overrides`.

Cheers.
Tai.


On Friday, 3 August 2012 17:04:56 UTC+10, DrMeers wrote:
>
> A couple of months ago Jannis closed #/17924 [1] as wontfix, stating "I'm 
> violently -1 on the whole topic "meta programming form fields after they've 
> been instantiated", it's a mess. Yes it would be more DRY, but also much 
> harder to know how the hell the form fields are composed as. Just override 
> the form field, it's not a huge deal code wise and much easier to grok." This 
> has since been contested [2] (though for invalid reasons in my opinion).
>
> I'm not sure that Jannis and I were talking about the same thing; the 
> solution I had in mind did not involve changing form-fields after their 
> instantiation, but rather providing a more elegant and flexible mechanism 
> that works in a similar way to ModelForm.Meta.widgets and 
> formfield_callback.
>
> I've put a draft patch together and attached it to [1] which should give a 
> better indication of the intended approach and its flexibility and 
> benefits. There are still a few details to iron out in evolving the ideal 
> implementation, but this at least demonstrates the gist of it. Does anyone 
> else think this is worth exploring?
>
> Simon
>
> [1] https://code.djangoproject.com/ticket/17924
> [2] *
> https://groups.google.com/d/msg/django-developers/x_nJ5epfG18/ZSKcPW0_DvAJ
> *
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/V-9BdhvaeKwJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GitHub migration planning

[Tai Lee]

This seems odd to me. Django is generally a very open and community 
oriented project, which strives to consult with the community and achieve a 
consensus, resorting to a BDFL decision when necessary, after all sides 
have put their case.

Maybe I wasn't following closely enough (apologies if that is the case), 
but my impression was that this decision was reached behind closed doors 
without a great deal of focussed community discussion or attempt to reach 
consensus. I just remember Adrian basically saying (and I'm paraphrasing 
here): "I've been away too long, but I'm back now and I've decided that 
we're moving to GitHub!"

Yes, there have been threads about moving to this or that environment, but 
I don't remember a thread started by Django core letting the community know 
that a move was seriously on the cards, and giving the community a chance 
to have some formal input before a decision was made.

Even if Django core feels that a BDFL decision will be required, I still 
think it is appropriate to raise such a change with the community and ask 
for some structured proposals, and then if necessary make a BDFL decision 
that shows which elements from which proposals have been blessed.

There's always a possibility that the community has something to say that 
could sway the decision, even if Adrian or Jacob are 99% decided. And even 
if it doesn't change the outcome, at least there is a clear public record 
of the decision making process.

I don't necessarily disagree with the decision that was made, but I would 
have liked to see more openness in the discussion and decision making 
process.

Cheers.
Tai.


On Saturday, 21 April 2012 22:52:01 UTC+10, Luke Plant wrote:
>
> On 20/04/12 19:58, Daniel Sokolowski wrote:
>
> > Was BitBucket (mercurial system which is python based) not considered?
> > And could someone point me to a url where I can read the discussion on
> > this migration; I am rather curious why it’s happening – the current
> > system works so I see no reason to fix it.
>
> Some of the discussion happened on django-core.
>
> One of the reasons for this was that it affects core developers most of
> all, so Adrian wanted their opinions first.
>
> Another reason was that this kind of change is almost certainly going to
> require a BDFL decision, because we will never come to consensus on Git
> vs Mercurial etc. - even within the core developers they are strong
> preferences in both directions, and even strong preferences to stick
> with Subversion. And I guess that's the reason that we didn't have
> further discussion on django-devs - since it already needed a BDFL
> decision, there was no point making the pretence of discussion in a
> wider forum. (Adrian/Jacob feel may correct me if I'm guessing wrongly).
>
> Luke
>
> -- 
> "My capacity for happiness you could fit into a matchbox without
> taking out the matches first." (Marvin the paranoid android)
>
> Luke Plant || http://lukeplant.me.uk/
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/zZIneXzTTvcJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Admin site: Appropriateness of HTTP 500 when using non-allowed query strings

[Tai Lee]

I agree with this. HTTP 500 error should not occur due to users attempting 
to subvert the system somehow. HTTP 500 errors should only be returned when 
an unhandled exception occurs (which shouldn't happen).

Cheers.
Tai.


On Tuesday, 10 April 2012 21:34:07 UTC+10, 3point2 wrote:
>
> The admin site allows the use of certain query strings to filter 
> change list pages. The syntax follows queryset field lookups, for 
> example http://mysite.com/admin/myapp/mymodel/?field__exact=test. 
> Lookups that are not specified on the ModelAdmin's list_filter option 
> raise a SuspiciousOperation exception. This is done to prevent a 
> normal user from obtaining sensitive information (e.g. password 
> hashes). 
>
> In production use, I'm not sure that returning an HTTP code of 500 
> (internal server error) and emailing the server admins is an 
> appropriate response to a user manipulating the query string. 
>
> I think that 403 (forbidden) would be more accurate. In my mind, 500 
> suggests that something went wrong on the server, for example an 
> unexpected condition or exception in the application code. In this 
> situation, this is not the case. Django is deliberately forbidding a 
> user from accessing information for which they have not been 
> authorized. 
>
> Any thoughts?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/Rs6SW74LU2QJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.user refactor: the profile aproach

[Tai Lee]

Sure. But will how will users be guided to making that distinction when 
developing their pluggable apps? I haven't seen anything that would prevent 
developers from doing this, and if the admin itself does it, isn't that a green 
light for everyone to do it?

Another problem with pluggable apps (like the admin) adding fields to the 
`User` model is that they make an assumption that *every* user is a user of 
that particular pluggable app.

I use the admin, but not *every* user has access to the admin. Those users 
shouldn't need to have values (even default values, e.g. is_staff=False) for 
admin-specific fields, and admin-specific fields shouldn't be selected every 
time any user is retrieved from the database.

Cheers.
Tai.


On 11/04/2012, at 8:05 AM, Donald Stufft wrote:

> I think swappable user models should be used as a replacement for 
> get_profile() not per app profiles.
> 
> It should be used for generic-esque data about a User. e.g. Email, phone 
> number, name, etc.
> 
> It should not be used for app specific data about a user, e.g. Default 
> Gallery, Notification Settings, etc.
> On Tuesday, April 10, 2012 at 6:01 PM, Tai Lee wrote:
> 
>> Alex,
>> 
>> I think the problem with this aspect of your proposal is that it signals a 
>> green light for other pluggable apps to follow Django's lead and provide 
>> mixing which must be added to their `User` model in order to use the 
>> pluggable app, instead of creating a profile model for their pluggable app.
>> 
>> Django's admin is a pluggable app, and it should follow the best practices 
>> that we recommend for authors of other pluggable apps.
>> 
>> It has been suggested that "nothing is stopping pluggable app authors from 
>> continuing to use profiles", but on the flip side, nothing is going to stop 
>> them from using mixins and requiring a schema migration to install their 
>> pluggable app.
>> 
>> If we allow swappable models, we should strongly recommend (in 
>> documentation, and by example with the admin) that pluggable app authors 
>> continue to use profiles, and that only project authors use the ability to 
>> swap in a new `User` model for their own purposes, and not to support a 
>> pluggable app.
>> 
>> Cheers.
>> Tai.
>> 
>> 
>> On 11/04/2012, at 3:25 AM, Alex Ogier wrote:
>> 
>>> Tom,
>>> 
>>> I proposed mixins to solve the specific problem: there is an app that needs 
>>> a specific contract from a model it wants to authenticate or otherwise 
>>> interact with, how can we make it easy for developers to implement that 
>>> contract?
>>> 
>>> Most apps don't actually need that much though. There are a bunch of 
>>> standard ways to relate to a model that don't invasively change it. They 
>>> are all still available, and in fact preferred because no matter how easy 
>>> it is to use a mixin, doing nothing is even easier.
>>> 
>>> Best, 
>>> Alex Ogier
>>> 
>>> On Apr 10, 2012 10:58 AM, "Tom Evans"  wrote:
>>>> On Tue, Apr 10, 2012 at 3:13 PM, Ian Lewis  wrote:
>>>> > Hi,
>>>> >
>>>> > I'm not getting why you *have* to add fields to the User model to store 
>>>> > data
>>>> > pertaining to the user. There is nothing in the proposal for pluggable 
>>>> > user
>>>> > models that says you can never have a seperate model with a foreign key 
>>>> > to
>>>> > the user model. It just means that you can define your user model the way
>>>> > you want it to be.
>>>> 
>>>> That is perfectly fine. The problem comes when there is a simple
>>>> system to add fields to the user model, people will use it to add
>>>> fields to the user model in their pluggable apps, for 'simplicity' and
>>>> 'ease of use'.
>>>> 
>>>> > Why can't third party apps have a model with a foreign key to the user 
>>>> > table
>>>> > with the pluggable models approach? I imagine you are right that every 
>>>> > app
>>>> > and it's brother adding fields to the user model is not realistic but I
>>>> > don't think that anyone has proposed that. Certainly not me.
>>>> 
>>>> The proposed solution as decided by BDFL diktat is 2a from [1]. I quote:
>>>> 
>>>> Split off as much as possible of auth.User into orthogonal mixins that
>>>> can be reused.
>>>> Mo

Re: auth.user refactor: the profile aproach

[Tai Lee]

Alex,

I think the problem with this aspect of your proposal is that it signals a 
green light for other pluggable apps to follow Django's lead and provide mixing 
which must be added to their `User` model in order to use the pluggable app, 
instead of creating a profile model for their pluggable app.

Django's admin is a pluggable app, and it should follow the best practices that 
we recommend for authors of other pluggable apps.

It has been suggested that "nothing is stopping pluggable app authors from 
continuing to use profiles", but on the flip side, nothing is going to stop 
them from using mixins and requiring a schema migration to install their 
pluggable app.

If we allow swappable models, we should strongly recommend (in documentation, 
and by example with the admin) that pluggable app authors continue to use 
profiles, and that only project authors use the ability to swap in a new `User` 
model for their own purposes, and not to support a pluggable app.

Cheers.
Tai.


On 11/04/2012, at 3:25 AM, Alex Ogier wrote:

> Tom,
> 
> I proposed mixins to solve the specific problem: there is an app that needs a 
> specific contract from a model it wants to authenticate or otherwise interact 
> with, how can we make it easy for developers to implement that contract?
> 
> Most apps don't actually need that much though. There are a bunch of standard 
> ways to relate to a model that don't invasively change it. They are all still 
> available, and in fact preferred because no matter how easy it is to use a 
> mixin, doing nothing is even easier.
> 
> Best, 
> Alex Ogier
> 
> On Apr 10, 2012 10:58 AM, "Tom Evans"  wrote:
> On Tue, Apr 10, 2012 at 3:13 PM, Ian Lewis  wrote:
> > Hi,
> >
> > I'm not getting why you *have* to add fields to the User model to store data
> > pertaining to the user. There is nothing in the proposal for pluggable user
> > models that says you can never have a seperate model with a foreign key to
> > the user model. It just means that you can define your user model the way
> > you want it to be.
> 
> That is perfectly fine. The problem comes when there is a simple
> system to add fields to the user model, people will use it to add
> fields to the user model in their pluggable apps, for 'simplicity' and
> 'ease of use'.
> 
> > Why can't third party apps have a model with a foreign key to the user table
> > with the pluggable models approach? I imagine you are right that every app
> > and it's brother adding fields to the user model is not realistic but I
> > don't think that anyone has proposed that. Certainly not me.
> 
> The proposed solution as decided by BDFL diktat is 2a from [1]. I quote:
> 
> Split off as much as possible of auth.User into orthogonal mixins that
> can be reused.
> Modify auth.User to inherit these mixins. Care must be taken to ensure
> that the database expression of the new User model is identical to the
> old User model, to ensure backwards compatibility.
> Unrelated and third-party apps can indicate that they depend on
> various orthogonal mixins. For example, contrib.admin can specify that
> it works with auth.User out of the box, and with any model
> implementing PermissionsMixin if you supply your own login forms.
> 
> At the moment, you cannot change the user model, so we do not have
> issues relating to third party apps changing the user model. With the
> proposed solution, you would be able to change the user model, so we
> may have issues.
> 
> It's also enlightening to read the code from Alex's Django branch,
> which is an initial implementation of option 2a.
> 
> > The thing I
> > want to be able to is define user models suitable for my project. Third
> > party apps adding their own fields wasn't proposed by anyone AFAIK, nor was
> > specifically requiring that you add them yourself. Some might require that
> > your user has something like an 'email' field because that would be a common
> > field across apps but app specific data can easily go on a seperate model
> > included with the app that simply has a FK to user. You can then only fetch
> > that data on requests that need it.
> >
> > I'm sorry but doing a JOIN every request is a BAD idea. You will run into
> > problems there quickly and have no way out of it besides ditching auth
> > completely (and thus all the thirdparty apps you use that depend on it).
> 
> I completely disagree, but I'm not here to try and convince people how
> to design their databases. A JOIN every request will not end the
> world. Besides, it is far more likely to be a separate query than a
> JOIN, and would only happen on views that required that data.
> 
> More to the point, what basis are you making this claim on? People
> love to pipe up "JOINs are slow and evil", but have you actually
> analysed the cost compared to monolithic tables?
> 
> > Assuming the user table and profile tables are small is awfully short
> > sighted.
> 
> To be fair, I was slightly ambiguous with my use of the word 'small'. I said:
> 
> >> Quer

Re: auth.user refactor: the profile aproach

[Tai Lee]

Tom,

Thanks for raising those issues. I would just like to add that I hope to see 
fields currently in `User` that are required by the admin (is_staff, etc.), 
moved to an admin profile model, and not simply made available through mixins 
that are still required by every `User` model that gets swapped in.

Even if we technically allow the `User` model to be swapped in, and therefore 
allow new fields to be added to it, I think that this would set a dangerous 
example and precedent for authors of other pluggable apps.

The admin is arguably *the* premiere example of a "pluggable" app, held up as a 
reference on how it should be done. I think it's important that we clearly 
define when it is appropriate for a pluggable app to require schema changes to 
a project's `User` model, and that should only be in rare and specific 
circumstances.

Generally, fields should only be added to the `User` model by a project author 
(not a pluggable app author), because pluggable app authors won't know the 
details of the environment (other pluggable apps and project) they will be 
installed into.

I also hope to see a more fleshed out proposal from Adrian, before an actual 
implementation is delivered, hopefully containing answers to some of the 
questions raised here.

Cheers.
Tai.


On 10/04/2012, at 7:47 PM, Tom Evans wrote:

> On Fri, Apr 6, 2012 at 7:31 PM, Alex Ogier  wrote:
>> Tai, read https://gist.github.com/2289395 for a summary of many reasons why
>> I think profiles are a bad idea, and unifying multiple profiles is an even
>> worse idea.
>> 
>> Best,
>> Alex Ogier
> 
> Hi Alex
> 
> Is https://gist.github.com/2289395 the complete proposal for what is
> to be implemented? It seems more of a point by point rebuttal of
> another proposal.
> 
> Is the approved solution to have mixins which contribute to a user
> class? Are pluggable apps expected to provide mixins that contribute
> to the user model?
> 
> Does this proposal fix the current issues with the user model in new projects?
> 
> My biggest concerns with this approach:
> 
> 1) Monolithic user tables.
> 
> Adding apps that want their own user storage with this system requires
> new columns in your user model. This is not the best approach. Foreign
> keys to separate user data tables per app is much more appropriate.
> 
> Monolithic tables kill performance. As an example, a C++ ORM I have
> used had this approach, and it was common to come across a user table
> with 100+ columns, including some massive fields (unbounded text
> fields, blobs). On most page requests, only a few columns were looked
> at, but every page paid the cost of retrieving massive user objects.
> 
> The most common complaint against profiles is that they are 'slow' as
> you have to join to many tables, or issue multiple queries. Queries
> against monolithic tables are much slower than a few queries on much
> smaller tables.
> 
> 2) Constant flux on my user model as a site develops
> 
> Adding new apps which require user storage would require schema
> change. If you have lots of users, this is a real pain. When adding an
> app profile, all that is required is a new table to be created, which
> would not lock a critical table like 'user' for a long period.
> 
> 
> I've no objection to allowing project managers more control over the
> user model, but I don't think we should encourage the majority of
> pluggable apps to pollute the user model - in fact, probably only apps
> dealing with AAA.
> 
> Eg, a photo gallery app may want to store the preferred thumbnail size
> and whether to open images in a new window. These do not belong on the
> user model, but on a photo gallery app profile. Most users may not
> have preferences, and for those that do, it is only relevant to access
> that info on a photo gallery page.
> 
> A twitter auth app may want to store an oauth token. This should
> belong on the user model, as it is used for AAA, and may be accessed
> on any request.
> 
> Cheers
> 
> Tom
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.user refactor: the profile aproach

[Tai Lee]

Alex Ogier,

Is it really better to require users to create their own User model that 
behaves like an admin user, instead of just shipping with a self contained 
admin user (as a profile model) without the auth component?

If the auth app was purely a stub to connect different profiles and 
authentication systems from different apps (or the project), but doesn't 
actually define any identity or authentication or profile models itself (not 
counting base abstract classes), isn't that effectively achieving the 
separation that you want? Being able to use admin without the current cruft in 
auth or with any completely different authentication credentials, and similarly 
using any other pluggable app without any cruft needed by the admin.

In any case, I don't think it will actually be possible to use the admin or any 
other pluggable app that relies on the concept of a central user who might 
access to multiple apps (instead of every app having its own users and auth) 
without *an* auth app and a central User model.

Like Adrian, I don't actually use the User model for auth or identity (name, 
email, etc.) anymore. But unless you have authored *all* the apps in your 
project and they know how to talk to *your* User model, you still need a User 
from Django, because that is what all 3rd party pluggable apps will need.

If I want to use any 3rd party apps that use the central user, I will still 
need to create a Django User and fake or sync the that are only there for the 
admin, even if I don't use the admin. If you still require swapped in User 
models to assume a minimal interface and fields, people will still have this 
problem.

This is why I think the central User should not contain any auth or identity 
data, so there is no cruft required only for apps you may not even be using, 
and so you are not tied to any particular auth system.

Cheers.
Tai.


On 06/04/2012, at 3:44 PM, Alex Ogier  wrote:

> I think this proposal will make more sense if people stop thinking "If 
> someone wants to use contrib.auth, then why do we need another crufty 
> interface to swap out auth.User?" Instead think of someone who wants to use 
> contrib.admin but not be stuck with contrib.auth.
> 
> The point of this proposal isn't to make contrib.auth larger and better, it's 
> to make it unnecessary. A lot of people find contrib.auth's models 
> unsatisfactory. Adrian Holovaty stated that he hasn't used auth.User for 
> several years. When you have a complex site with multiple authentication 
> methods and requirements that don't fit django's idea of a user, it stops 
> being worth it to bend yourself to django's will. The problem is that 
> contrib.auth and contrib.admin are currently intimately linked. This 
> proposal's purpose is to give an official way to break these two apart.
> 
> I don't know of a single framework out there besides Django that ships with a 
> fixed model for its users. The reason is that authentication and identity are 
> radically different for every site so it's tremendously important to support 
> whatever people decide to do, and not force decisions on them.
> 
> So, stop thinking just in terms of contrib.auth.models.User. If you're 
> already using that with a profile and it's all working fine, then this change 
> isn't for you. This is for everyone who just wishes auth.User would go away 
> without totally borking admin.
> 
> Respectfully,
> Alex Ogier
> 
> On Apr 6, 2012 1:21 AM, "Donald Stufft"  wrote:
> Nothing about this proposal prevents this.
> 
> And in that case, no those 2 apps would not be able to be used together. But 
> this is hardly the first
> time that 2 apps cannot be used together. because of choices made like that 
> on the app owner.
> On Friday, April 6, 2012 at 1:18 AM, Harris Lapiroff wrote:
> 
>> I very much share Tai's concerns about the swappable user model introducing 
>> incompatibilities. Imagine two apps, each of which requires an "age" 
>> attribute on the user model. But suppose one of those apps expects age to be 
>> the number of years since that user's birth and one of those apps expects 
>> the age to be the number of years since the user registered for the website. 
>> The user model must provide the same attribute to both apps, but it is 
>> supposed to have a different value for each app. A developer will be unable 
>> to use these two apps together without patching one of them.
>> 
>> A bit of a contrived example, maybe, but I can imagine this 
>> same-name-different-purpose issue coming up over and over again, making 
>> otherwise pluggable apps incompatible with each other.
>> 
>> I think we should go with a pared down user model and allow each app to 
>> manage whatever data it needs on each user through profiles and signals. 
>> Developers will end up with some data duplication, but I think that is 
>> preferable to confusion about the source and purpose of data. Profiles are 
>> essentially a way for each app to namespace its own data and I think that's

Re: auth.user refactor: the profile aproach

[Tai Lee]

Thanks Anssi, but I think the interface contract for a pluggable app *should* 
be on the pluggable app's profile, instead of relying on an assumption that 
developers will have created a user model that is duck typed to suit every 
pluggable app installed in the project (which might not be practical or even 
possible). 

For the sake of avoiding duplication, which could be managed with signals and 
the save() method of a project level profile, you potentially overload fields 
that are not analogous across pluggable apps by sharing a single namespace. 
E.g. is_active for the admin may not mean is_active for app2 or app3.

Jacob, by having to define a minimum set of fields that users must implement in 
their swappable User model to support contrib apps like the admin, don't we end 
up exactly where we are now? Isn't this exactly what we have already?

The only difference I see then is that the interface would be defined in docs 
instead of a model and re-implemented by developers for every project, and 
pluggable apps will often require a schema migration on the swapped-in project 
level User model when adding a pluggable app to an established project.

In my ideal scenario, User would be a glue model with just a PK, and maybe some 
convenience methods. AdminProfile would not have username or password fields, 
but would have is_active, is_superuser, and maybe optionally a name, etc.

Developers would need to create at least one model in their project, which 
would be a subclass of BaseAuthModel in most cases, which stores authentication 
credentials.

They would also need to create an auth backend, which would be a subclass of 
BaseAuthBackend in most cases.

Pluggable apps could also define auth models and backends if they require 
specific auth to function (e.g. an app that adds twitter integration.)

Then people could login to the admin and any other pluggable apps using 
whatever credentials and authentication system they like (username/email and 
password, twitter, Facebook, openid, etc.) Developers wouldn't need to 
implement a minimal set of fields or do a database schema migration to support 
or install any pluggable apps.

If app1 or app2 require access to an email address (for example), their 
profiles should have a required email field. It should be up to the project 
developer to make sure that an email address is set when creating an app1 or 
app2 profile for a User, and synchronizing them between app1 and app2 (if 
appropriate).

I think the data duplication issue is much easier for project developers to 
manage explicitly without resorting to magic and assumptions than sharing or 
combining namespaces for profile data, doing database scheme migrations, and 
duck typing a single model for use with multiple pluggable apps.

Cheers.
Tai.


On 06/04/2012, at 7:42 AM, Anssi Kääriäinen  wrote:

> On Apr 5, 11:29 pm, Tai Lee  wrote:
>> But I still don't see how a swapped in `User` model which *has* to behave in 
>> a specified way so that it can work with the Django admin and any other 
>> pluggable apps that might have special requirements, is any better than 
>> simply allowing the admin and other pluggable apps to have their profile and 
>> authentication needs self-contained?
>> 
>> If Django's `User` model was just a stub (without even username and password 
>> fields), and Django shipped with an abstract `BaseAuth` model with a 
>> `username` field that was email compliant and a `password` field, and 
>> corresponding `BaseAuthForm` and `BaseAuthBackend`, then user's can still 
>> create their own `User` model with literally *whatever* fields they want in 
>> it, they can use the standard auth fields, form and backend provided by 
>> Django, or roll their own.
>> 
>> Instead of creating a custom `User` model that quacks like an admin duck, 
>> and quacks like every pluggable app that is installed as well, all they need 
>> to do is create/update an an `AdminUser` whenever their custom `User` is 
>> saved.
>> 
>> This is explicit, the admin and other pluggable apps know where to access 
>> information that they need (from their own models), and the developer has 
>> control over how the data is kept in sync across the pluggable apps used in 
>> the project, at the project level.
> 
> If every application provides a profile which matches it needs, you
> will get a serious case of data-duplication. Every application which
> needs email-address should then create a profile containing the email.
> Another way to solve this would be to check the existing profiles for
> the email field, and now you have introduced some magic, and in
> addition you have mostly just moved the interface contract from the
> User model to its profile. So, you have the same problem again. This
> is the failing of the profil

Re: auth.user refactor: the profile aproach

[Tai Lee]

On 06/04/2012, at 3:09 AM, Ian Lewis wrote:

> The good part about swappable user models is that you don't need to 
> necessarily fix the model's DB fields in advance. Your identifier and 
> password's length and other properties can be user defined and can be 
> reflected in the DB.
> 
> Django can also leave migration of user models an data up to the developer so 
> they an use south or whatever to manage the user model since they "own" it. 
> Django Devs wouldn't necessarily need to support a DB table that can 
> essentially never change as is the case currently.
> 
> Fixing that stuff ahead of time and just making a FK means the user model 
> itself is fixed. I don't want to be able to just add fields. I want to be 
> able to change the primary key to be a uuid, or a email, or use the default 
> auto incrementing integer. I want to be able to use an existing model or DB 
> table and, with some work, plug it into Django auth.
> 
> Now if I'm customizing the user model itself anyway, why not just tack on 
> whatever other fields I want? I don't need a FK since those fields are on the 
> user model and I can create however many one to one or one to many 
> relationships for "profiles" or user data as makes sense for my project. That 
> may be a lot or it may be zero. 
> 
> (now whether that is realistic given the needs of the admin is a different 
> story)

But I still don't see how a swapped in `User` model which *has* to behave in a 
specified way so that it can work with the Django admin and any other pluggable 
apps that might have special requirements, is any better than simply allowing 
the admin and other pluggable apps to have their profile and authentication 
needs self-contained?

If Django's `User` model was just a stub (without even username and password 
fields), and Django shipped with an abstract `BaseAuth` model with a `username` 
field that was email compliant and a `password` field, and corresponding 
`BaseAuthForm` and `BaseAuthBackend`, then user's can still create their own 
`User` model with literally *whatever* fields they want in it, they can use the 
standard auth fields, form and backend provided by Django, or roll their own.

Instead of creating a custom `User` model that quacks like an admin duck, and 
quacks like every pluggable app that is installed as well, all they need to do 
is create/update an an `AdminUser` whenever their custom `User` is saved.

This is explicit, the admin and other pluggable apps know where to access 
information that they need (from their own models), and the developer has 
control over how the data is kept in sync across the pluggable apps used in the 
project, at the project level.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.user refactor: the profile aproach

[Tai Lee]

Are we really sure that we can or should stomach a swappable User model as 
a special case?

It's highly likely that I am missing something (or a few things), but 
aren't the main benefits of swapping the User model just to avoid making a 
backwards incompatible change, and to avoid making `select_related()` 
queries?

I'm not suggesting that backwards incompatibility is not important, but I'm 
not convinced by the argument about inefficient `select_related()` queries, 
and whatever we do will involve manual schema migration for many users.

Do we really need to allow users to swap in their own model with a 
combination of additional project level fields as well as additional fields 
required by various pluggable apps as the primary `User` model, instead of 
simply stripping out authentication and identifying fields from the primary 
`User` model and allowing users to implement whatever they need as app or 
project level profile models?

The more I think about it, the more I think this will just lead to more 
fragmentation and incompatibilities with pluggable apps, when the user 
"interface" is abused or causes conflicts.

What's going to happen when this is rolled out and people start developing 
an eco-system of pluggable apps without a concrete `User` model in Django? 
All they can rely on is that there will be some form of `User` model and 
that it *should* provide at least a minimal set of fields and behaviours as 
specified by some guidelines in Django docs?

Won't we just be back at square 1 in deciding what to include in the 
minimal fields and guidelines for the this "contract"? Isn't the only 
sensible minimal set of fields, no fields at all?

Pluggable apps will have to either dictate to developers in the 
installation instructions that their particular `User` model must have 
certain fields or behaviours that are required by the pluggable app, which 
may even conflict with other pluggable apps. Or they will still have to 
fallback to using an app-profile model, which brings us back to using 
`select_related()` anyway. I don't like the idea that a pluggable app might 
require users to change their existing models like this.

If pluggable apps will still need to use app-profile models anyway, if we 
can get past the backwards incompatibility issue, what is so bad about 
simply having a `User` model which has no auth or identity data in it, but 
is just there as glue for for apps to share data for a single "user"?

Bundled apps like the admin would define `AdminProfile` and any pluggable 
apps that require or support the admin could access `user.admin_profile`. 
Pluggable apps that don't use the admin and/or have their own auth or 
identifying requirements can just ship with their own profile and/or auth 
models and optional auth backend.

The only real issue I have with Django users/auth as it is right now is 
that there are redundant (for me) or non-compliant (with RFC) fields in 
`auth.User` that are required and that I have to fake in order to hook into 
the admin, groups and permissions or re-specify in my own profile for 
RFC-compliance.

The other problem I have seen mentioned with the profile approach is 
managing common fields (e.g. two pluggable apps that have a "name" field). 
I'm quite happy for this to be managed by the developer at a project level, 
either using signals or forms or a `save()` method on their profile model 
or whatever else they like to keep that data in sync, if it needs to be 
kept in sync.

I don't think pluggable apps requiring that developers have a "name" (for 
example) field on their primary `User` model is really a good solution to 
this problem, because the pluggable app doesn't know what other purpose 
that field is used for, and doesn't know if it is changed in app1 if it 
will have any consequences for app2.

If developers really want a single project level `User` model, they can 
still create that (with an FK back to Django's `User` model), and simply 
update it's `save()` method to sync any common fields on all the pluggable 
app's profile models (which may even have different field names). E.g. 
`project.User` could have `first_name` and `last_name`, but app1 has only 
`name`, and app2 has `given_name` and `family_name`. In 
`project.User.save()`, the developer of a project can determine how to sync 
this data.

Cheers.
Tai.


On Thursday, 5 April 2012 00:57:57 UTC+10, Jacob Kaplan-Moss wrote:
>
> On Wednesday, April 4, 2012 at 9:44 AM, Russell Keith-Magee wrote:
>
> My point is that there is nothing about this problem that is unique to 
> User. Django's own codebase contains another example of exactly the same 
> pattern -- Comments. 
>
> As the original author and designer of that pattern, I should probably 
> point out that I now think it's a mistake. Have you actually tried using 
> it? It doesn't really work very well. Every time I've introduced any sort 
> of "swappable model" mechanism I've come to regret it.
>
> I'm -1 on the idea of generalized

Re: auth.user refactor: the profile aproach

[Tai Lee]

I'm not so sure that it's necessary or even desirable to solve the 
"general" problem of swappable models. If anyone can swap any model by 
changing a setting, that sounds like a recipe for confusion to me.

Seems to me that the main benefit of the swappable models approach is just 
to avoid backwards incompatibility issues by making the change opt-in, but 
I think it's also opening Pandora's box. I think that leaving it up to 
developers to ensure that their swapped-in models simply behave enough like 
the original is going to cause subtle and difficult to trace bugs.

App developers won't be able to have confidence that models they have 
defined will actually be used as defined.

This is why I would prefer to simply have an empty `User` model that is 
basically just a PK, which is used only as the glue that connects all the 
related bits and pieces (auth models, profile models, group and permissions 
models, etc.)

The rest of the existing `User` fields can be implemented in an `AdminUser` 
model, and allow multiple authentication backends, which defaults to just 
an `AdminUserBackend` authentication backend.

Apps and projects can just use this if they like, or they can define 
whatever models they like to link to the user (if they want to hook into 
other data that is linked to the user), along with their own auth backend. 
Then users can login with either admin credentials, or your project/apps 
credentials. A few abstract user base classes or auth backend classes could 
help make this part easier.

I'm not convinced that there should even be this concept (like the "data 
bag") that an app can rely on "some app" ("any app") defining a value for a 
user related field (e.g. name, email, twitter name, etc.), without knowing 
or caring which app has defined it. If you don't know which app defines the 
data, how can you know that it is the correct data for your app and your 
purpose? Likewise if you change a value, how do you know it won't 
inadvertently impact another app that relies on it? You won't know what 
other apps will be installed along side your own, what fields they will 
define, or when they will want to overwrite those fields.

There is such a thing as too generic. People can and do already define 
their own user/profile models and authentication backends. They can and do 
create their own models to store arbitrary data like name, twitter name, 
longer email address. The only problem is that there are then required and 
redundant or insufficient vestigial fields left over in `auth.User`. I have 
my own user model with username field, but I have to fake an 
`auth.User.username` to connect my user to groups and permissions. I can't 
use signals to sync my username/email address to `auth.User.email` for the 
Django admin because it's only 75 characters long and not RFC compliant.

If `auth.User` was just a glue model, with just a PK, and the other fields 
were moved to `AdminUser` (and `AdminUser.email increased in length), this 
would be a non-issue because I would not need to even have an `AdminUser` 
at all to create users for my app/project. I would only need to create one 
for users of the admin app, and in that case I should adhere to the 
AdminUser model's field requirements.

Cheers.
Tai.


On Wednesday, 4 April 2012 15:25:40 UTC+10, Eric Florenzano wrote:
>
> I completely agree with Adrian's proposal, with these amendments suggested 
> by Russell in order to build something slightly more generic, like 
> LazyForeignKey, instead of UserField.  In fact, a UserField coud end up 
> being provided as a subclass of LazyForeignKey with a default name provided 
> (e.g. 'user') and any other user-domain-specific customizations needed.
>
> Thanks,
> Eric Florenzano
>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/gMdsJCSMmKIJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.user refactor: the profile aproach

[Tai Lee]

I like this proposal because I am a big fan of a stripped down `User` model 
which is basically just an ID, whic provides a common hook into 
groups/permissions and other django and 3rd party profiles.

What I don't like is the magical `data` bag (accessing `User.data` and 
filter lookups), the new `AUTH_PROFILES` setting, and the idea of Django 
automagically or lazily creating profiles for me. I would rather see Django 
rely solely on its `OneToOneField` field to access user profile data.

Any pluggable app (app1) will know what fields it's own profile model has 
and how to access them, relative to the central `User` object.

If app1 relies on fields defined by another app (app2), that other app 
should be a requirement of app1 and app1 would know how to access app2's 
fields.

I am happy to use project-level signals for everything else (syncing common 
profile data from app1 and app2, or auto-creating user profiles), because 
project-level signals give me explicit control over what is going to happen 
and when. I don't need any more magic here.

Cheers.
Tai.


On Tuesday, 3 April 2012 10:35:41 UTC+10, Jacob Kaplan-Moss wrote:
>
>  Hi folks -- 
>
> I've written up a proposal for how *I* would like to address refactoring 
> auth.user: https://gist.github.com/2245327.
>
> In essence, this does two things:
>
> * Vastly "prunes" the required fields on auth.user. The only things left 
> are an "identifier" (which could be username, email, url, uuid, whatever), 
> and a password.
> * Introduces a new "profile" system that provides a way to contribute 
> extra related fields. Multiple profiles are supported, along with some 
> syntactic sugar for dealing with multiple profiles in a reasonably reusable 
> way.
>
> And that's about it. I'm deliberately trying to find a middle ground 
> between "do the minimum to allow people to move on" and "throw out and 
> rewrite django.contrib.auth entirely". I'm not expecting everyone to be 
> thrilled by this idea, but I'm hoping that this is "Good Enough" for almost 
> everyone.
>
> For more please see the document. Please do try to read the whole thing: 
> I've had a few rounds of feedback incorporated already, and there's even an 
> FAQ at the end.
>
> I'm not using BDFL fiat here, at least not yet. This is a proposal, and I 
> very much want to hear feedback, objections, and alternatives. I'm 
> particularly interested in hearing from people who've got complicated auth 
> needs and think this absolutely won't work for them. 
>
> I *have* reviewed all the other proposals and I'm between -0 and -1 on all 
> of them. This means that if you don't like my proposal, you'll probably 
> have to come up with a complete *new* idea to have any chance of getting my 
> vote.
>
> Thanks!
>
> Jacob
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/PMpcPCKgTuoJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django should not use `force_unicode(..., errors='replace')` when parsing POST data.

[Tai Lee]

I agree that it is not an ideal user experience to raise an exception while 
decoding POST data. However, I think the alternatives are arguably just as bad, 
or even worse.

I consider silently replacing characters in user data to avoid an exception 
while decoding to be a silent data loss / corruption issue.

Depending on the actual data being submitted and the web site in question, this 
may be a non-issue, it may be inconvenient but acceptable, it may be critical, 
and it may cause errors down the line (like the `DatabaseError` exception).

However, neither users or developers currently have any choice in the matter, 
and most won't even know it is happening.

If simply switching to `errors='strict'` is not acceptable, what do people 
think about adding a `dirty` or `raw` property to `request.FILES` and 
`request.POST` when a decoding error occurs, and storing the original byte 
strings in there while keeping the forcibly decoded strings in `request.FILES` 
and `request.POST` as they are now?

This would give developers a chance to deal with this in a way that suits them. 
They could use middleware to re-raise the silenced exception, or they could try 
to decode with common alternative encodings before falling back to forcibly 
decoding (or raising an exception), or they could return a form validation 
error in their view, or they could still use the forcibly decoded data but warn 
users that their data was altered.

If we were to go down this route, I would still prefer to see Django ship with 
such a middleware that re-raises the `UnicodeDecodeError` exception and have it 
enabled by default, simply because this issue involves silent changes to user 
supplied data. If it causes an error for anyone, and it shouldn't normally as 
this appears to be an edge case, the issue will be easily diagnosed and the 
developer can then choose if they want to silently replace data, or attempt 
alternative decoding, or display an error to users.

The Django docs for file uploads say:

> The content-type header uploaded with the file (e.g. text/plain or 
> application/pdf). Like any data supplied by the user, you shouldn't trust 
> that the uploaded file is actually this type. You'll still need to validate 
> that the file contains the content that the content-type header claims -- 
> "trust but verify."


I think Django should follow the "trust but verify" principle when decoding all 
POST data. It's true that we can't accurately detect what character encoding is 
being used for supplied data, and this is precisely why we shouldn't forcibly 
decode POST data. We can at least inform developers (if not users) when the 
specified character encoding is proven to be wrong, and allow them to choose 
how to handle it.

Cheers.
Tai.


On 30/03/2012, at 5:01 AM, Waylan Limberg wrote:

> I'm not sure which approach is the way to go here. However, forcing
> users to deal with encodings is generally a bad idea. Besides, you
> never can trust a browser to give you what it says it is giving you.
> In other words, the user may not be able to get the browser to send
> the correct encoding anyway. For those reasons I'm leaning toward #1.
> Of course, that begs the question: should Django be doing a better job
> escaping the data used to build the SQL statement? I guess we won't
> know unless we get the bad SQL statement. Which takes us back to #1.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Django should not use `force_unicode(..., errors='replace')` when parsing POST data.

[Tai Lee]

I've just created an essay *cough* I mean ticket about the way Django 
silently mangles POST data that is incorrectly encoded, and how this can 
send someone (like me) down the rabbit hole trying to debug it, thanks to 
"current transaction aborted" database errors.

This shouldn't normally happen, but it seems that I am unlucky and someone 
somewhere is able to get their browser to submit POST data that is encoded 
incorrectly.

I don't want to repeat myself too much and end up writing another essay 
here, so I'll just link to the ticket. Just wanted to post this here as 
well because jezdez said it might be better suited to discussion on 
django-developers.

https://code.djangoproject.com/ticket/18004

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/ByLiu7RzHtIJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Tagging 1.4 django release in Subversion

[Tai Lee]

How come? The release that can be downloaded from the site already must 
correspond to an SVN revision number, right? Why not tag it as such so that 
people can easily get the same code from SVN as from the release tarball?

Cheers.
Tai.


On Sunday, 25 March 2012 22:02:30 UTC+11, Florian Apolloner wrote:
>
> Hi,
>
> it's not tagged yet on purpose.
>
> Cheers,
> Florian
>
> On Sunday, March 25, 2012 8:26:17 AM UTC+2, jdetaeye wrote:
>>
>>
>> Can a developer please tag the 1.4 release in the SVN repository please? 
>> Ie create 
>> http://code.​​djangoproject.com/svn/django/​​tags/releases/1.4
>> Looks like it was forgotten...
>>
>> Johan
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/TWBkN0anP1oJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Pretty Django model instaces updating

[Tai Lee]

This is easy to achieve in your own code. Create your own BaseModel
class that all your models subclass. Instead of using
`.filter().update()`, do this:

def update(self, **kwargs):
"""
Updates the fields specified in `kwargs` and then call `save()`.
"""
if kwargs:
for k, v in kwargs.iteritems():
setattr(self, k, v)
self.save(force_update=True)

This will still fire the `save` signals, and your model instance will
have the new values as well as having them saved to the database.

I think it is also easier than assigning values to each field and then
calling `.save(only_fields[field1, field2])`, which I don't think is
any better than filter().update(). If `save(only_fields=...)` were to
only write specified fields to the database, this might cause
unexpected side effects with `save` signals?

If you don't like the name `update`, or it clashes with something else
in your model namespaces, just use something else.

I don't think this warrants inclusion in Django core.

Cheers.
Tai.


On Mar 2, 8:48 am, Kee  wrote:
> Fully agree with Anssi Kääriäinen,
>
> the best approach is ``only_fields`` for save()``
>
> On Mar 1, 11:45 pm, Anssi Kääriäinen  wrote:
>
>
>
>
>
>
>
> > On Thursday, March 1, 2012 9:29:34 PM UTC+2, Carl Meyer wrote:
>
> > > 
>
> > > Thanks for the suggestion. I agree that that's a useful pattern and one
> > > I frequently use myself. The problem is that the model instance
> > > namespace is a precious resource, and using up any more of it than we
> > > already do can easily lead to backwards-compatibility problems. In this
> > > case, I don't think turning a one-liner into a shorter one-liner
> > > justifies using up more of that namespace.
>
> > > Carl
>
> > I suggest implementing .save(only_fields=list_of_fields).
>
> > That should not be hard to implement and will not consume the namespace of
> > models. Signals are fired properly. In addition, update() would be a bit
> > strange instance method: it updates the specified fields by the kwargs (the
> > instance's field values do not matter). In addition, after the update the
> > instance's fields are not updated to reflect what was just saved.
>
> >  - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Newline stripping in templates: the dnl way

[Tai Lee]

On Feb 29, 8:15 am, Leon Matthews  wrote:
> Would it be feasible to add some logic, something along the lines of:
>
> "Template lines containing just tags with no literal content do not
> produce a line in the output  (unless of course the tag itself
> produces one)"

I believe that is what has been suggested, and tickets produced by
SmileyChris and worked on by others to that end, at
https://code.djangoproject.com/ticket/2594

The ticket is still accepted (and is now 6 years old), and the
consensus from this thread now appears to be that the "dnl" and multi-
line comment solutions put forward here are not preferred.

I think the next step is for any interested parties to look at the
latest patches on that ticket, update them for trunk and make sure
they still work, have the required tests and/or docs, and try to get
it committed.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Newline stripping in templates: the dnl way

[Tai Lee]

I think this discussion is focusing on template tags, not template 
variables. Maybe even a subset of template tags (e.g. block level tagsif, 
for, block, filter, etc). Template variables and inline tags (e.g. now) 
shouldn't have white space stripped.

In 100% of cases I can think of I either wouldn't care (in HTML templates) 
or I would want lines that only contain block level tags and no actual 
content removed from the rendered output of the template.

The only time I would explicitly NOT want this to happen is when I have an 
existing template where white space matters and it has been carefully 
crafted to work around Django's white space issues. I think SmileyChris' 
solution from the ticket is the way to go, but in a backwards compatible 
way. Make it opt-in with a new template tag, similar to the way auto 
escaping is handled, and optionally a deprecation path that will make this 
the default eventually.

I don't think anyone has suggested stripping ALL white space around ALL 
template tags, or even ANY template variables.

Cheers.
Tai.


On Saturday, 25 February 2012 21:04:26 UTC+11, Florian Apolloner wrote:
>
> Hi,
>
> On Saturday, February 25, 2012 10:04:21 AM UTC+1, Anssi Kääriäinen wrote:
>>
>> In most situations white space matters: 
>> {{ user.lastname }} {{ user.firstname }} 
>>
>
> Right, but
> """
> {{ user.lastname }} 
> {{ user.firstname }}
> """
> would have produced exactly the same output in HTML, hence my statement 
> that you usually don't have to care about it that much. Eg, you need to 
> make sure that there is whitespace but it doesn't matter how much since 
> it's collapsed anyways (and outside of tags it doesn't matter at all…)
>
> Cheers,
> Florian
>

On Saturday, 25 February 2012 21:04:26 UTC+11, Florian Apolloner wrote:
>
> Hi,
>
> On Saturday, February 25, 2012 10:04:21 AM UTC+1, Anssi Kääriäinen wrote:
>>
>> In most situations white space matters: 
>> {{ user.lastname }} {{ user.firstname }} 
>>
>
> Right, but
> """
> {{ user.lastname }} 
> {{ user.firstname }}
> """
> would have produced exactly the same output in HTML, hence my statement 
> that you usually don't have to care about it that much. Eg, you need to 
> make sure that there is whitespace but it doesn't matter how much since 
> it's collapsed anyways (and outside of tags it doesn't matter at all…)
>
> Cheers,
> Florian
>

On Saturday, 25 February 2012 21:04:26 UTC+11, Florian Apolloner wrote:
>
> Hi,
>
> On Saturday, February 25, 2012 10:04:21 AM UTC+1, Anssi Kääriäinen wrote:
>>
>> In most situations white space matters: 
>> {{ user.lastname }} {{ user.firstname }} 
>>
>
> Right, but
> """
> {{ user.lastname }} 
> {{ user.firstname }}
> """
> would have produced exactly the same output in HTML, hence my statement 
> that you usually don't have to care about it that much. Eg, you need to 
> make sure that there is whitespace but it doesn't matter how much since 
> it's collapsed anyways (and outside of tags it doesn't matter at all…)
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/aKbJjYcK4QIJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Newline stripping in templates: the dnl way

[Tai Lee]

Adding more symbols to existing tags (e.g. {^ for x in y ^} or {% for
x in y -%}), multi-line comment tags that don't actually include a
comment, and half baked comment tags (where the closing tag is
assumed) are all going to make templates uglier, and harder to read.

The comment tag based solutions don't even solve the problem
completely, because leading white space is still there, and users are
still required to carefully position comment tags all around block
tags to remove whitespace.

What's better about {% for x in y -%} or {^ for x in y ^} over {%
stripwhitespace on %} at the top of your template, followed by {% for
x in y %}? I think the latter is far more readable.

My ideal solution is not to add new ways to mark each individual
template tag that should have surrounding white space stripped, but to
simply enable the removal of lines that have only block tags and no
actual content. 99% of the time this is the right thing to do, and we
just need a deprecation path and new template tag so that template
authors can opt-in to the new behaviour now.

Cheers.
Tai.


On Feb 25, 4:37 am, Tobia  wrote:
> Tai Lee wrote:
> > I don't think adding {# to the end of lines is easy to understand at a
> > glance, it doesn't even convey a hint of meaning like "dnl" does
>
> I beg to differ.  {# is already recognized by template authors as
> meaning "start of comment", and they know (or should know) that it
> cannot extend through more than one line.  Therefore I'd think it
> intuitive that it will "eat" till the end of the line and not beyond.
>
> Look:
>
> Here are your subscriptions:
> {% for thing in things %}{#
>  - {{ thing.name }}
>    You added it on {{ thing.date }}
> {% endfor %}{#
> you can manage your subscriptions...
>
> Tom Evans wrote:
> > I'd be strongly -1 on anything that makes template language look more
> > like m4!
>
> I'll tell you, m4 can be quite addictive once you grasp the basics! :)
>
> > This could be addressed by having a different open/close tag for tags
> > which chomp the preceeding/next character if it is a newline. Eg:
> > {^ for item in folder ^}
>
> I don't think adding new reserved characters would make the language
> simpler for template authors, nor for the the template parser, nor for
> the sake of backwards compatibility.  {# is already part of it.
>
> But I can see the need to chomp whitespace before a template tag, as
> well as the newline after it.
>
> Martin J. Laubach wrote:
> > For this (avoiding newlines) the currently discussed multiline tags
> > would work pretty well too without adding more cruft to the template
> >language:
>
> > foo bar {#
> > #}baz
>
> If this can be accomplished without massive performance penalties, I
> agree with you.
>
> Maybe {# #} could be made multiline at first, with special code, while
> waiting for a proper implementation of generic multiline tags.  This
> would certainly be more forward-compatible than my proposal above
> and it would solve more whitespace problems, if not all.
>
> Tobia

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Newline stripping in templates: the dnl way

[Tai Lee]

I definitely want to see a resolution to this issue. Django templates
when white space matters (e.g. plain text emails) are painful. But, I
don't think adding {# to the end of lines is easy to understand at a
glance, it doesn't even convey a hint of meaning like "dnl" does, but
either option is pretty ugly to my eyes.

I'd personally like to see a new template tag that will switch on the
new behaviour, and a deprecation path for this to become the default.

Cheers.
Tai.


On Feb 25, 1:10 am, Tobia  wrote:
> Hi all,
> regarding issue #2594 "Template system should handle whitespace
> better" explaining the need for some kind of (optional) whitespace
> stripping from the output of templates, I've been looking at the
> proposed solutions and compared them to other template and macro
> engines.
>
> In particular, a historical but still widely used and general-purpose
> macro engine is m4. Its own feature for controlled newline stripping
> is the "dnl" reserved word, "Discard to Next Line." It works by
> "chomping" all input until and including the next newline.
>
> For example, to call a macro foo() without copying over the newline
> that appears after the macro call in the template, a m4 programmer
> would write:
>
> foo('bar`, `whatever')dnl
>
> An equivalent feature in Django templates would enable template
> developers to strip newlines from specific lines, while keeping
> backwards compatibility with existing templates.
>
> So if the general idea is well-accepted, I propose the "{#" token. The
> example from the issue would become:
>
> 
> {% for item in items %}{#
>     {{ item }}
> {% endfor %}{#
> 
>
> It is already a reserved token in the template system, so it's not
> going to break anything. The existing comment syntax {# ... #} is
> already restricted to single-line comments, so there are no multi-line
> comments that would break. Plus, I'd wager it could be implemented
> quite efficiently.
>
> What do you think?
>
> Tobia

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User usernames

[Tai Lee]

It's not that hard to just set up a OneToOneField back to User, and
use signals to automatically create a User when you create your own
User/Profile. Then you can still make use of 3rd party apps that rely
on contrib.auth or contrib.sessions, and also make use of groups from
contrib.auth, etc.

Cheers.
Tai.


On Feb 17, 9:13 pm, Jonathan Slenders 
wrote:
> On 16 fév, 13:05, Tom Evans  wrote:
>
> > 75 isn't large enough these days for either email or username. We run
> > a patched version of django for some time that has changed both these
> > fields to 255 characters in order to accommodate the needs of our
> > users. See RFC 3696.
>
> This and other issues made us moving away from contrib.auth and
> contrib.sessions. It's not too hard to write your own custom
> authentication and session middleware, and you can migrate whereever
> you want to. The main problem is if you depend on other libraries with
> rely on the existance of auth.models.User, like contrib.admin.
>
> Personally, I think a lot of the apps in django.contrib have a lack of
> flexibility. Maybe it's good to leave these apps as they are, but
> start something like contrib_v2, as the improved version.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User usernames

[Tai Lee]

I created a generic `accounts` app which has (among other things) it's
own `Profile` model with a username field and a OneToOneField pointing
at `User`. I added an authentication backend to my settings that
checks usernames from my model. Of course there are other supporting
components, forms, signals, etc., but this has removed any Django
username limitations for me pretty easily.

This might be a little inconvenient, but it works well here and I
still have a `User` for every `Profile` so I can hook into Django
sessions and groups. If you are having trouble with `User.username`
right now or anticipate issues in the near future, I suggest you
implement something similar yourself, or find an open source generic
app that does it for you, as I think it will likely be a very long
time until this is resolved in Django.

Cheers.
Tai.


On Feb 16, 9:57 am, Donald Stufft  wrote:
> On Wednesday, February 15, 2012 at 5:49 PM, James Bennett wrote:
> > On Wed, Feb 15, 2012 at 4:37 PM, Donald Stufft  > (mailto:donald.stu...@gmail.com)> wrote:
> > > I know this has been discussed before, but I wanted to bring it up again 
> > > in
> > > light of the oncoming Djnago 1.4 beta.
>
> > So, here's the thing: you're asking for a fairly significant,
> > massively backwards-incompatible change which requires every Django
> > install on the planet to do a schema migration... about four hours
> > before we feature-freeze 1.4. There is simply no way this is going to
> > get in on that time scale; 1.5, maybe, if the inherent problems can
> > get ironed out, and if you're strongly interested in making this
> > happen I'd invite you to help out with that.
>
> > But 1.4 beta -- and thus feature freeze -- happens tonight, and it's
> > flat impossible to land something of this magnitude before that
> > happens.
>
> 1.5 would work as well ;) Sorry I sometimes speak before I think things 
> through,
> thoroughly. django.auth in general is something that i'm interested in and I 
> want
> to try and improve to be more flexible, I just hadn't though of a general 
> solution yet
> and a "easy" (code wise) fix appealed in a shorter term "provide some 
> improvement"
> sort of way.
>
>
>
>
>
>
>
> > --
> > "Bureaucrat Conrad, you are technically correct -- the best kind of 
> > correct."
>
> > --
> > You received this message because you are subscribed to the Google Groups 
> > "Django developers" group.
> > To post to this group, send email to django-developers@googlegroups.com 
> > (mailto:django-developers@googlegroups.com).
> > To unsubscribe from this group, send email to 
> > django-developers+unsubscr...@googlegroups.com 
> > (mailto:django-developers+unsubscr...@googlegroups.com).
> > For more options, visit this group 
> > athttp://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature Request: Client side validation classes for forms

[Tai Lee]

I found that Alex's `django-ajax-validation` works pretty well for
this and I think it works the way you described. Perhaps it could be
updated and included into Django core, if there is good support for
it.

https://github.com/alex/django-ajax-validation/

Cheers.
Tai.


On Feb 4, 8:03 am, Adrian Holovaty  wrote:
> On Fri, Feb 3, 2012 at 11:46 AM, Karthik Abinav
>
>  wrote:
> >   I was thinking about a feature that could be implemented. For common
> > fields like username having only alphanumeric , or phone numbers having only
> > numbers, a client side validation need not be written every time.Instead one
> > could directly write something like,
>
> > forms.CharField(validator = "usernamevalidation")
>
> > in the forms definition and the client side validation for that field would
> > automatically be taken care of by the validator class. This will save a lot
> > of time while making large websites with lot of registration forms and in
> > general be helpful to people who dont really know javascript and yet want
> > some amount of frontend validation in place.
>
> I like the idea of having a JavaScript version of form validation.
> Basically we could make a view class that takes a Form object in
> __init__() and returns JSON of the errors in a consistent way -- this
> would be very easy to do. Then we could provide some standard
> JavaScript to parse that JSON and add the error messages to the
> appropriate fields in the form in a consistent way.
>
> Good idea! It's a bit too late now to add it to Django 1.4, but I'd
> like to implement this for the next version.
>
> Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Custom managers in reverse relations

[Tai Lee]

Instead of trying to dynamically change the manager used for reverse
relations or any manager/queryset (either by getting and then throwing
away the default manager, or some other method), or applying manual
filters to achieve the same result, there is another alternative.

Instead of:

{{{
reporter.article_set.manager('published_articles')
}}}

Do:

{{{
Article.published_articles.filter(reporter=reporter)
}}}

This is probably not ideal, but is still easy to read and should be an
easy work-around in most cases.

Regarding option 1-4 above, only option 2 appeals to me. I prefer my
example above to option 1, and option 3-4 are a bit confusing. But
even for option 2, I think it will fall down when you have multiple
foreign keys to the same model. This is why we have `related_name` on
`ForeignKey` instead of `Manager`, and why it should not be moved to
`related_name` on `Manager`.

Cheers.
Tai.


On Jan 17, 9:12 am, Sebastian Goll  wrote:
> Hi all,
>
> I don't know if using the queryset's hypothetical use_manager() method
> would be the right approach to the reverse manager problem. In fact,
> I'm not entirely happy with the current manager() proposal either: it
> feels very much like an after-the-fact change to the default manager.
> That's how it is implemented; first, you retrieve the default manager,
> then you decide that's not what you wanted after all.
>
> I don't think that is how selecting the reverse manager should feel.
> Also, as long as we don't unify managers and querysets entirely, I
> think that whatever returns to us the reverse manager should, in fact,
> return a manager, i.e. we should be able to call all manager methods
> defined on that manager, not just the @queryset ones (given that those
> would be made available in the querysets derived from that manager).
>
> Unfortunately, I'm not sure what would be a better way, or syntax, for
> setting the reverse manager. In #3871, russellm suggested the manager()
> method approach to select which manager is used. To me that feels very
> much like an after-the-fact approach. I also allows you to chain
> manager() calls which doesn't seem very useful at all:
>
> {{{
>   reporter.article_set.manager('published_articles').manager('articles')
>
> }}}
>
> I think we need something different. Just brainstorming, these are some
> thoughts how choosing reverse managers _could_ be implemented; this is
> not yet related to any actual implementation, only some ideas:
>
>   1. use a dict-like approach on the current `related_name` reverse
>     manager, e.g.
>
> {{{
>   reporter.article_set['published_articles']
>
> }}}
>
>     This is not verbose, and probably not intuitive at all; it uses
>     array indexing for something that's not at all an array. However,
>     `article_set` could be thought of as a dict mapping manager names
>     to managers, so maybe it makes at least some sense after all.
>
>   2. add `related_name` to models.Manager as well, e.g.
>
> {{{
> class Article(models.Model):
>     reporter = models.ForeignKey(Reporter, related_name='article_set')
>     ...
>     articles = models.Manager()
>     published_articles = PublishedManager(related_name='published_articles')
>
> }}}
>
>     `reporter.article_set` returns the default manager as before, but
>     now `reporter.published_articles` returns related objects via
>     PublishedManager.
>
>     It would be an error to specify both `related_name` on the
>     ForeignKey and on the default manager. In fact, `related_name` on
>     the ForeignKey could be translated into the `related_name`
>     attribute on the default manager at model import time, keeping
>     `ForeignKey(related_name=(…))` for backwards compatibility as well
>     as for simple models which do not need a manager other than the
>     default models.Manager().
>
>     In fact, the more intuitive and unified way to write the above
>     example (with two managers) would be to move the `related_name`
>     from ForeignKey to the default manager:
>
> {{{
> class Article(models.Model):
>     reporter = models.ForeignKey(Reporter)
>     ...
>     articles = models.Manager(related_name='article_set')
>     published_articles = PublishedManager(related_name='published_articles')
>
> }}}
>
>     Non-default managers without the `related_name` attribute would not
>     be accessible in reverse relations.
>
>   3. make `related_name` on ForeignKey a dict mapping attributes to
>     manager names, e.g.
>
> {{{
> class Article(models.Model):
>     reporter = models.ForeignKey(Reporter, related_name={'article_set': 
> '_default_manager', 'published_articles': 'published_articles'})
>     ...
>     articles = models.Manager()
>     published_articles = PublishedManager()
>
> }}}
>
>     This doesn't seem sensible at all. Lots of boilerplate and overly
>     verbose.
>
>   4. add `reverse_managers` to ForeignKey, specifying which managers
>     (by name) should be made available on the reverse relation, e.g.
>
> {{{
> class Article(mod

Re: Don't assume that missing fields from POST data are equal to an empty string value.

[Tai Lee]

Ian,

I agree that there are a lot of different ways that form data can be
submitted to Django, including near endless combinations of multiple
Django forms, multiple HTML forms, AJAX, etc.

If we reduce the scope of our discussion and consider only a Django
form (forgetting HTML forms and AJAX) and two possible scenarios:

1. Partial form data is bound to a Django form, and it is not expected
to result in a call to the form's `save()` method and a change to the
database. It is only intended to examine the form's validation state
relating to the partial data that is bound to the form. In this case,
the proposed change should have no impact.

2. Full form data is bound to a form, and it IS expected to validate
and result in a call to the form's `save()` method and a change to the
database. In this case, why would we ever want to assume that a
character field which has no bound data, is actually bound to an empty
string?

This is a dangerous assumption, I think. If we intend to obtain
complete data from the user, bind it to a form and save it to the
database, we should be sure (as much as we can be) that the data is
actually complete.

Take the following pure Django example:

{{{
class Profile(models.Model):
first_name = models.CharField(blank=True, max_length=50)
last_name = models.CharField(blank=True, max_length=50)
address = models.CharField(blank=True, max_length=50)

class ProfileForm(ModelForm):
class Meta:
model = Profile

profile = Profile.objects.create(first_name='Tai', last_name='Lee',
address='Sydney')
form = ProfileForm({}, instance=profile)
if form.is_valid():
form.save()
}}}

The profile will have no first name, last name or address. The form
will produce no validation errors and will save the updated model to
the database.

This is very surprising and counter-intuitive to me.

I think the only time we could safely make the assumption that fields
with empty string values will be omitted by the UA is when we are
confident that the source of the form data also makes a similar
assumption that the back-end will re-normalise those missing fields
back to an empty string.

I'd be surprised if any UAs actually do make that assumption, because
the way Django treats missing character fields does not appear to be
based on any spec, and Django is not the only form processing back-end
(or even the most popular one).

I'm not sure I follow your simplest case example of one HTML form in a
browser window and one Django Form in a view. If optional fields are
left off the HTML form deliberately, without change the Form class or
the view code, this is exactly when data loss will currently occur. I
think you are confusing optional as in "may not be specified by the
user" with optional as in "may not be processed by the form"?

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Don't assume that missing fields from POST data are equal to an empty string value.

[Tai Lee]

Donald,

Thanks for sharing your feedback with everyone.

I do believe that there should be some kind of validation error or at
least a loud warning to the console or logs when a form is bound to
incomplete data. The only time when this should occur is for
"unsuccessful controls" such as unchecked radio buttons and
checkboxes. These shouldn't be a problem for Django, which already
normalises no value for a checkbox to False and any value for it to
True.

I'm not sure that there is a widespread practice of submitting partial
forms with JS and still expecting the entire form to validate and save
is widespread, or even valid according to the RFC and HTML4 specs
which expect every successful control to be included in the form data.

Sure, I can see that forms could be fully or even partially submitted
via JS to perform AJAX validation in real time, but I don't see how
the form as a whole being invalid and validation errors on the missing
fields would impact that.

If we can't find a clear definition or distinction between null and an
empty string in the RFC or HTML4 specs, and we go to our browser (you
mentioned Chrome), I think you will find that they consider text input
fields with an empty string as the value are "successful controls", as
these are included in the form data.

Before knocking back this proposal on the grounds of ambiguous specs
(multiple interpretations), I would like to know if there are actually
any UAs that behave as you fear they might. If there are, then this
change would definitely be a problem for those UAs.

However, even in the event that this is deemed a backwards
incompatible change and the potential silent data loss issue is not
serious enough to override that, then a loud warning could still be
added without changing the current behaviour.

Cheers.
Tai.


On Jan 12, 3:29 am, Donald Stufft  wrote:
> I'm very much -1 on this change.
>
> To "fix" this change would require throwing an error anytime an incomplete 
> dictionary was passed as the data arg to a form. This would break any 
> existing code that relies on this (in particular it's common practice to 
> accept a subset of the fields via json). So this would be a backwards 
> incompatible change.
>
> Further more I disagree with the interpretation of the RFC as provided. The 
> RFC states that any UA may choose to not send along a form field if it 
> contains a null value. So the question then becomes what is a null value in 
> regards to the RFC? As I cannot find any mention of what constitutes a null 
> value in the RFC I went to my browser. Using javascript I executed 
> ``document.getElementById('textfield').value = null`` in the js console. The 
> result was that the value was set to "". So in my browser (Chrome on OS X) it 
> is treating null and "" with equivalence.
>
> Going by my personal interpretation of the RFC, and Chrome's behavior in my 
> javascript test I can only conclude that the proposed change would cause 
> Django forms to violate the RFC spec and while Violating the RFC spec in and 
> of itself isn't always the wrong thing to do I do believe that it should only 
> be done when RFC and implementations are at odds in a way that are 
> incompatible with each other. In this case they are not, and the RFC is more 
> permissive and should be followed as Django does not have a list of supported 
> browsers so we must strive to follow the RFC's where we can (and where they 
> are actually defined) and deviate only when the alternative is being broken 
> in major browsers.
>
> Additionally I believe in this case there are two major error conditions.
>
> A) The proposed change is made, a visitor is using a UA that I believe 
> follows the RFC and any Django forms with optional, and unfilled in values 
> stop working for this visitor.
> B) The proposed change is not made, and when an optional form field is left 
> off of a form (or json, or any partially incomplete dictionary of values) the 
> form assumes the default initial value of "".
>
> Neither error condition is optimal, however A has the additional downside 
> that this error is completely outside of the control of the developer whereas 
> B is the result of developer error and is under his control.
>
>
>
>
>
>
>
> On Tuesday, January 10, 2012 at 8:38 PM, Tai Lee wrote:
> > There is a potential for data loss with optional form fields that are
> > (for whatever reason) omitted from an HTML template.
>
> > For example, if we take an existing model form and template that
> > works, add an optional character field to the model but fail to add a
> > corresponding field to the HTML template (e.g. human error, forgot
> > about a template, didn't tell the templat

Re: Don't assume that missing fields from POST data are equal to an empty string value.

[Tai Lee]

Tom, the problem is not caused by using `exclude`, it is caused by not
using `fields`. If you use `exclude` or simply don't specify `exclude`
or `fields` then all fields will be included. Thanks for your other
example and description of the problem, I just wanted to clarify that
it is not only a problem when using `exclude`.

Cheers.
Tai.


On Jan 12, 2:40 am, Tom Evans  wrote:
> Are there any other situations where this can happen? The problem is
> in fact caused by using 'exclude' to choose the fields presented in a
> model form, using 'fields' and explicitly listing the fields seems
> much safer.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Don't assume that missing fields from POST data are equal to an empty string value.

[Tai Lee]

Optional or required is not the same thing as important. It is
entirely valid to have important but optional fields on a model.

For example, a field that contains a string reference to a python
callback function that modifies the behaviour of some model methods
when set.

We use this on a Broadcast model in a mailing list application. The
callback is used to generate additional context for the email template
for personalisation when sending out email newsletters.

This is important data that should not be lost as it will change the
behaviour of model methods (in this case, potentially sending the same
email newsletter without personalisation to many recipients), but it
may not be required for every instance of the model.

Cheers.
Tai.


On Jan 12, 1:11 am, Babatunde Akinyanmi  wrote:
> -1
> I think a programmer should not specify a field that would contain
> important data as optional in the first place. If the data loss from
> not including it in the form is going to cause problems then it should
> not be optional.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Don't assume that missing fields from POST data are equal to an empty string value.

[Tai Lee]

There is a potential for data loss with optional form fields that are
(for whatever reason) omitted from an HTML template.

For example, if we take an existing model form and template that
works, add an optional character field to the model but fail to add a
corresponding field to the HTML template (e.g. human error, forgot
about a template, didn't tell the template author to make a change,
didn't realise a change needed to be made to a template), when that
form is submitted Django will assume that the user has provided an
empty string value for the missing field and save that to the model,
erasing any existing value. This is not a bug, but it is relatively
easy to trigger silent and unexpected data loss.

I have briefly discussed this with PaulM and dstufft on django-dev,
but could did not reach any consensus.

RFC1866 [1] says:

> The fields are listed in the order they appear in the
> document with the name separated from the value by `=' and
> the pairs separated from each other by `&'. Fields with null
> values may be omitted. In particular, unselected radio
> buttons and checkboxes should not appear in the encoded
> data, but hidden fields with VALUE attributes present
> should.

The HTML4 spec at W3C.org [2] says:

> Users interact with forms through named controls.
>
> A control's "control name" is given by its name attribute. The scope of the
> name attribute for a control within a FORM element is the FORM element.
>
> Each control has both an initial value and a current value, both of which are
> character strings. Please consult the definition of each control for
> information about initial values and possible constraints on values imposed by
> the control. In general, a control's "initial value" may be specified with the
> control element's value attribute. However, the initial value of a TEXTAREA
> element is given by its contents, and the initial value of an OBJECT element
> in a form is determined by the object implementation (i.e., it lies outside
> the scope of this specification).
>
> The control's "current value" is first set to the initial value. Thereafter,
> the control's current value may be modified through user interaction and
> scripts.
>
> A control's initial value does not change. Thus, when a form is reset, each
> control's current value is reset to its initial value. If a control does not
> have an initial value, the effect of a form reset on that control is
> undefined.
>
> When a form is submitted for processing, some controls have their name paired
> with their current value and these pairs are submitted with the form. Those
> controls for which name/value pairs are submitted are called successful
> controls.

as well as [3]:

> A successful control is "valid" for submission. Every successful control has
> its control name paired with its current value as part of the submitted form
> data set. A successful control must be defined within a FORM element and must
> have a control name.
>
> However:
>
> * Controls that are disabled cannot be successful.
> * If a form contains more than one submit button, only the activated submit
>   button is successful.
> * All "on" checkboxes may be successful.
> * For radio buttons that share the same value of the name attribute, only the
>   "on" radio button may be successful.
> * For menus, the control name is provided by a SELECT element and values are
>   provided by OPTION elements. Only selected options may be successful. When
>   no options are selected, the control is not successful and neither the name
>   nor any values are submitted to the server when the form is submitted.
> * The current value of a file select is a list of one or more file names. Upon
>   submission of the form, the contents of each file are submitted with the
>   rest of the form data. The file contents are packaged according to the
>   form's content type.
> * The current value of an object control is determined by the object's
>   implementation.
>
> If a control doesn't have a current value when the form is submitted, user
> agents are not required to treat it as a successful control.
>
> Furthermore, user agents should not consider the following controls
> successful:
>
> * Reset buttons.
> * OBJECT elements whose declare attribute has been set.
>
> Hidden controls and controls that are not rendered because of style sheet
> settings may still be successful.

I interpret the above to mean that any text input with a value
attribute (even `value=""`) is successful control, and should be
included in the encoded form data. This is what current versions of
Chrome and Firefox do, at least. I have not found any examples of
browsers which are known not to do this.

What I would like to change in Django is for it to stop assuming that
missing POST data for a character field is actually an empty string,
and instead raise a form validation error that would prevent the model
instance from being saved to the database (potentially causing data
loss for that field).

I d

Re: RFC: Query Methods

[Tai Lee]

I have had difficulty in finding an easy way to add common methods to
querysets and managers for multiple models in Django.

I solved the problem in my projects by defining a custom `Manager`
class which defines `use_for_related_fields = True` and also overrides
`__getattr__()` to look for queryset and manager methods on an inner
`QuerySet` class on the model class, similar to the inner `Admin`
class.

{{{
from django.db import models
from django.db.models import query
from turbia import models as turbia_models

class Client(models.Model):
is_deleted = models.BooleanField(default=False)
objects = turbia_models.Manager()

class QuerySet:
def not_deleted(self):
return self.filter(is_deleted=False)

class Brand(models.Model):
client = models.ForeignKey(Client)
is_deleted = models.BooleanField(default=False)
objects = turbia_models.Manager()

class QuerySet:
def not_deleted(self):
return self.filter(client__is_deleted=False, 
is_deleted=False)
}}}

I use this custom manager on an abstract base model class that all my
models are derived from, so it becomes trivial to add manager and
queryset methods when defining a new model. I can also define an inner
`QuerySet` class that is a subclass of another models inner `QuerySet`
class, when there are common queryset methods.

I'm sure there are problems with my implementation and it may not be
flexible or general enough to be relevant here (especially regarding
multiple manager support), but I would love to see improved support
built into the core of Django that would allow methods to be added to
both the manager and queryset for a model, and shared between
different (but similar) models much more easily.

I agree with Russell and others that these methods should not be
defined directly on the model (with a decorator).

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Request for eyes familiar with ORM internals and defer/only (attn: Malcolm)

[Tai Lee]

I've run into a bug that is exposed when using defer() or only() with
select_related(). A couple others have come across it, and there is an
existing ticket.

Exceptions appear to be silenced somewhere under normal circumstances
when evaluating `queryset` objects, which made it difficult to track
down, initially. I'm still not sure where the exceptions are silenced,
or if that should be changed. The exceptions appear to be raised when
running the test suite or when coercing a `query` object to `str`.

The symptom is an empty queryset being returned, when there are
actually results.

I've written a patch with tests that I think fixes the problem. I just
need somebody familiar with the ORM and defer/only to give feedback or
bump to RFC.

Malcolm, if you have a moment for a quick review that would be great
since you implemented the defer/only functionality and no doubt have
much greater. If you Otherwise, I'd really appreciate anyone familiar
with the ORM and defer/only taking a quick look :)

https://code.djangoproject.com/ticket/14694

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DecimalField model validation

[Tai Lee]

Why is ROUND_HALF_EVEN superior? Perhaps for statistics, where
rounding all halves up would skew results, but I guess not for most
other cases.

If the default rounding behaviour produces different results than a
regular calculator, common spreadsheet and accounting software or even
human (mentally adding numbers) would, then I think the default
behaviour is surprising to many users. I was certainly surprised when
I first encountered it, not being familiar with Python's default of
ROUND_HALF_EVEN.

I don't see an easy way for users to change the default rounding
behaviour for `DecimalField`. Would it be considered backwards
incompatible to change the default to ROUND_HALF_UP?

If so, would it be easy enough to make it possible for users to easily
change the default behaviour for a single field or even project-wide?

I hate to suggest adding another setting, but might it be necessary
for some users to change the rounding behaviour of DecimalField that
is defined in a 3rd party app?

On a related note, the `floatformat` template tag explicitly uses
ROUND_HALF_UP. I think we should at least be consistent. And if
there's cause to allow users to change the behaviour for DecimalField,
they should probably be able to change `floatformat` as well.

I agree that a no_rounding argument would also be good, in addition to
being able to specify the rounding method to use.

Cheers.
Tai.


On Oct 6, 11:45 am, Paul McMillan  wrote:
> > .. (A) silent rounding issue:
> >    when a decimal value is saved in db
> >    its decimal digits exceeding decimal_places are rounded off using
> >    .quantize(). Rounding defaults to ROUND_HALF_EVEN (python default).
> >    There is no mention of this behavior in docs.
>
> Docs patches welcomed. This rounding behavior is generally superior to
> ROUND_UP (what most of us are taught in gradeschool) for a number of
> reasons that are inappropriate to get into here. If we're going to
> round, that's the behavior we should use as default.
>
> > .. (B) no model validation for decimal digits:
> >    .full_clean() does not raise any exception:
> >       - if the value has too many decimal digits
> >       - or if value has too many digits
> >    other fields, like CharField, do not validate values that exceed fields
> > constraints
>
> There's no clearly defined way to deal with overlong CharFields.
> Rounding overlong floating point numbers is common and not unexpected.
> Decimals work similarly to floats, and so it's not unreasonable to
> have similar behavior.
>
> Reading the docs on Python's decimal module, it makes sense to think
> of the decimal_places as an analog to python's decimal notion of
> precision.http://docs.python.org/library/decimal.html#module-decimal
>
> > .. (C) errors on save:
> >   decimal values exceeding field's digits boundaries (decimal or total) make
> > .save() raise decimal.InvalidOperation exceptions
>
> This does sound like an error in validation. If it's possible to pass
> a value through form validation to the point where it has errors on
> save, that is a bug.
>
> > In my opinion they should be fixed in a backwards-incompatible way!:
>
> Thank you for your opinion, we don't do backwards-incompatible fixes.
> There's usually a compatible way to fix behavior.
>
> >  (A) django shuld not round decimal values silently before saving, if the
> > value has too many decimal digits,
> >      raise an exception (just like for values that have too many digits)
>
> In the general use case, the existing behavior is more user friendly.
> Many people would run out and re-add the rounding behavior if we
> changed it. Users will enter over-long strings into decimal fields.
> It's really unfriendly to paste an 32 place decimal number in and be
> told it can only be 17 decimal digits long (and then have to go count
> out till you get the correct number). Since there's not a good way to
> make this the default value and retain backwards compatibility, this
> is unlikely to change. This behavior should be documented.
>
> That said, a no_rounding kwarg sounds like a perfectly reasonable feature.
>
> >  (B) decimalfield shoud not validate values that exceed its max_digits and
> > decimal_places constraints
>
> I agree that decimalfield should not validate values that exceed
> max_digits. If it does, that is a bug we should fix. We need to be
> careful not to create situations where very large numbers validate but
> very small but precise decimal numbers get rounded unexpectedly.
> Unfortunately, these two parameters overlap in difficult ways. I
> disagree about decimal_places, since the expected behavior is rounding
> in most other real-world circumstances.
>
> Does the combination of documenting the existing rounding behavior,
> fixing the error on save() with max_digits, and adding the no_rounding
> feature address your concerns?
>
> -Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to

Re: deprecation vs removal

[Tai Lee]

On Oct 4, 11:17 am, Russell Keith-Magee 
wrote:
> I'm completely agreed that the 'soft' deprecation is useful. I'm just
> complaining about the ambiguity in the language: "We're deprecating
> this feature by marking it PendingDeprecation...".

What about just changing "PendingDeprecation..." to
"SoftDeprecation..." or "QuietDeprecation..."? It's not really pending
deprecation. It is deprecated already, but we just don't loudly
complain (yet) if people haven't updated their code immediately.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Patch for using custom managers in a reverse relation. (#3871)

[Tai Lee]

Is this really much better than using your own custom manager as the
default automatic manager (used for reverse relations) with
`use_for_related_fields = True`?

Your custom manager could do nothing to filter results by default and
so behave the same as the default automatic manager, but provide
additional methods that filter the results as you need.

Cheers.
Tai.


On Oct 3, 3:25 pm, Vivek Narayanan  wrote:
> Hi,
>
> I've added a patch that provides functionality for selecting a custom
> manager in a reverse relation , rather than the default manager. For
> example:
>
>          author = Author.objects.get(id=1)
>          # Selects the manager 'foobar' of Post.
>          mgr = author.post_set.managers("foobar")
>
> https://code.djangoproject.com/ticket/3871
>
> Would be great if someone could review it.
>
> Vivek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Consistent exception handling in templates.

[Tai Lee]

On Jul 9, 12:24 pm, Karen Tracey  wrote:
> I'm strongly against that idea. Swallowing errors makes it incredibly
> difficult to debug errors. I would prefer for the `TEMPLATE_DEBUG` setting to
> turn off much of the error-silencing currently done in template processing.
> Yes, that means different behavior during development compared to
> production. But running with DEBUG (template or otherwise) on IS different
> from production, that's a given. And it this case it would allow for more
> effective development since errors wouldn't be hidden away in nearly
> impossible-to-find places. In my mind that's a tradeoff well worth making.
>
> See ticket #13167 for one example of where "template errors" have not been
> silenced since before 1.0. If that case is to be be "fixed" now, I would
> very much want that silencing to ONLY happen when TEMPLATE_DEBUG is False.
> In my opinion silencing these during development would be a huge step
> backwards.

I think we are in agreement here. Swallowing errors makes it
incredible difficult to debug errors. I would like to see the
circumstances when exceptions are swallowed be more clearly defined,
and ensure that only exceptions that are expected to be swallowed, are
swallowed. If we can do that, then there should be no need for the
behaviour to change based on the `TEMPLATE_DEBUG` setting.

The problem with the inconsistent behaviour associated with this
setting is exceptions that are expected to be swallowed, that we want
to be swallowed, are not being swallowed in some cases when
`TEMPLATE_DEBUG = True`. This can make it impossible to access some
URLs on a website where such a condition is present, even though there
is no actual error and the exception is expected to be silenced. It
also results in some exceptions being transformed into
`TemplateSyntaxError` sometimes, and the true exception raised other
times.

See ticket #16147 for an example where `TemplateDoesNotExist` is
raised when the template DOES exist, because a template tag inside the
template being loaded raises `TemplateDoesNotExist` at compile time,
only when `TEMPLATE_DEBUG = True`.

This can make it impractical to develop locally with `TEMPLATE_DEBUG =
True`, because it can prevent access to working pages, which means you
miss out on all the enhanced debugging that `TEMPLATE_DEBUG = True`
provides when you encounter an actual error that needs developer
attention.

With the intent being that template authors should not be able to
trigger HTTP 500 errors on a production site, couldn't we achieve that
by silencing exceptions raised when an invalid argument was passed to
a template tag (e.g. `ValueError` or `TypeError` in the template tag
code), but allowing exceptions raised in the code for the object that
was passed in to the template tag as an argument?

For example, if an object that calculates and returns a number when
evaluated is passed to `{% widthratio %}`, but there is a bug in the
code of that object and it raises some kind of exception when
evaluated, that should not be silenced. It should bubble up so that
the developer is alerted to a problem in the code for that object (not
an error that the template author has made by passing an invalid
argument). However, if there was no exception raised in evaluating the
object, but it simply returned something other than a number, then
that should behave the same as if an empty string or missing template
variable was passed in and no exception should be raised.

I think that a `TemplateSyntaxError` should be raised at compile time
if an invalid template tag is used (unknown, too few or too many
arguments). The `render()` function for a template tag should silence
exceptions raised when arguments with the incorrect type or value are
provided, but should not silence exceptions raised when evaluating the
arguments themselves (or properties on the arguments).

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Consistent exception handling in templates.

[Tai Lee]

On Jul 8, 11:31 pm, Jacob Kaplan-Moss  wrote:
> > [...] but some of Django's own template tags (e.g. `widthratio`) don't 
> > adhere to
> > this policy.
>
> These are bugs, and should be fixed.

Thanks for this clarification. I will try to review the built-in tags
and submit some patches to fix these bugs. For added clarity, should a
template tag behave the same when a string or variable is passed as
it's argument?

E.g. if the first argument to a template tag is expected to be a
number, we can validate that and raise a `TemplateSyntaxError` at
compile time if it is passed in as a literal, but we can only validate
it and fail silently at render time if it is passed in as a template
variable. I believe that this behaviour is inconsistent and confusing,
and can lead to unexpected and difficult to trace problems.

To avoid this, should the validation of arguments to template tags
that can be passed in as either literals or template variables be
limited to checking that an argument was passed, and delay type or
value checking until render time, which will always fail silently?


> I'm against this, for a whole host of reasons — the crux of which
> comes down to the philosophy that templates are *not* a programming
> language, and introducing a try/except concept into them is about
> seven steps too far.
>
> The point of the template language is to give front end designers and
> developers the space to explore without the persnicketiness a
> so-called "real" programming language introduces. What you're asking
> for here is to fundamentally change the philosophy of the template
> language. If you disagree that fundamentally with the ideas behind the
> template language I really think you'll be better off finding another
> one than trying to swim upstream.

I agree that what I suggested would be a fundamental change, and not
likely to be adopted in full. However, I would still like to hear
other ideas from the community about smaller steps that we could take
that would make exception handling more consistent and selective,
without changing the fundamental philosophy of the template language.

Firstly, Jonathan Slenders makes the point that perhaps not ALL
exceptions should be silenced.

Secondly, there is the inconsistent behaviour between production and
development environments related to the `TEMPLATE_DEBUG` setting. I
think that if an exception is intentionally silenced in production, it
should also be intentionally silenced with `TEMPLATE_DEBUG = True`, to
allow for consistent behaviour during development.

If an exception IS worth raising when `TEMPLATE_DEBUG = True`, it
should also raise in production, and these types of exceptions should
be caused by an underlying problem in the template tag or the
evaluation of an object passed to the template tag, rather than
something that a template author is likely to trigger because they are
not programmers.

Does anybody else have ideas or suggestions about these two issues, or
other small improvements we could make to exception handling in
templates without fundamentally changing the philosophy of the
template language?

Cheers.
Tai

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Consistent exception handling in templates.

[Tai Lee]

I'd like to raise the topic of exception handling in templates, to see
what people think of the current state and discuss possible
improvements.

I personally find it confusing to use and create template tags at
times, due to exception handling. Sometimes exceptions are supposed to
be silenced, and sometimes not.

The docs say that when writing custom template tags, the `render()`
function should never raise `TemplateSyntaxError`, but some of
Django's own template tags (e.g. `widthratio`) don't adhere to this
policy.

I have also had difficulty with inconsistent behaviour between
development and production environments, due to the `TEMPLATE_DEBUG`
setting. For example, if `TEMPLATE_DEBUG` is `False`, the `include`
template tag will fail silently when passed a string or context
variable. If `TEMPLATE_DEBUG` is `True` (often the case on local
development servers) it will raise an exception at compile time only
when passed a literal string argument, otherwise it will raise an
exception at render time (again, against the advice in the docs) when
passed a context variable. In order to mimic production behaviour in a
development environment, you must set `TEMPLATE_DEBUG` to `False`, and
then you miss out on useful debug information.

When it comes time to test, and you have a template tag that does
happen to raise an unhandled exception (e.g. `ValueError`), it will be
transformed into `TemplateSyntaxError` if `TEMPLATE_DEBUG` is `True`.
So you need to test for either exception in your tests, or override
the `TEMPLATE_DEBUG` setting so that you can expect one exception or
the other, or add a try/except block to the `render()` function of
your template tag and either raise `TemplateSyntaxError` all the time
(against the advice of the docs) or silence all exceptions, with
either option possibly hiding problems that exist elsewhere in the
code (in a function called by the template tag).

I'd like to see a move towards consistent exception handling in
templates for template tags and filters. I'd like to see no exceptions
silenced by default, but provide a way for template authors to
conditionally silence exceptions in fragments of their templates.
Perhaps with a block tag.

This should allow for consistent behaviour, make it easier to spot
errors during development, and give developers the option of silencing
specific exceptions when they need to do something conditionally, e.g.
include a template that may or may not exist.

I imagine that this would be a huge backwards incompatible change. Are
there any less severe alternative steps we could take to improve the
situation generally?

Would it be possible to make such a change by following a deprecation
path?

Should built-in template tags follow the advice given in the docs, and
never raise `TemplateSyntaxError` in their `render()` function?

Cheers.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: required=True for BooleanFields

[Tai Lee]

On Jun 17, 3:41 pm, David Cramer  wrote:
> I'm not suggesting changing the behavior (again due to the
> compatibility concerns), but I completely agree with the original
> poster(s).
>
> Also, in my experience it's a much less common case that you're
> wanting an "I agree" checkbox in your form, versus a "Boolean" field
> which can be positive or negative.

You can do that easily right now without changing Django by adding
`required=False` the BooleanField on your form. This is less
repetitive than writing the same validation method over and over
again.

def validate_somebooleanfield(self):
value = self.cleaned_data.get('somebooleanfield')
if not value:
raise forms.ValidationError('This field is required.')
return value

If this behaviour were to change, `required` would become a completely
meaningless argument to BooleanField because no value in the GET or
POST data is normalised to `False` because of the way checkbox values
are submitted in HTML forms.

The nature of boolean fields means that it's impossible to NOT provide
a value. NullBooleanField is used in those cases.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: required=True for BooleanFields

[Tai Lee]

This has been discussed many times in the past. For better or worse,
we're stuck with the current behaviour for backwards compatibility.

I personally think it's better this way. Without this behaviour, it
would be a PITA to implement forms that have a "required" boolean
field (must be ticked) without repeating yourself constantly by
writing custom validation for those fields. Most forms I work with
have a "Yes, I have agreed to the terms and conditions" or "Yes, I
want to receive emails from XYZ" type fields.

If your boolean fields are not "required" (as per the definition in
Django forms, "must be ticked"), why can't you just put
`required=False` in your form?

I'd try to avoid patching your local Django with a change like this
unless absolutely necessary so that you can cleanly upgrade and don't
end up writing code that does the opposite of what everyone else
expects.

Cheers.
Tai.


On Jun 17, 5:14 am, Michael Blume  wrote:
> In Django BooleanFields, the required flag is used to mean that the field
> must be checked for the form to validate. Required is True by default for
> all Fields, so this is the default behavior.
>
> I strongly suspect that this violates principle of least surprise for most
> people including Boolean Fields in forms. It did for me.
>
> I've patched it in my local Django checkout. I realize this is a
> backwards-incompatible change, so it might not be eligible for trunk any
> time soon, but FWIW, here's the patch:
>
> --- i/django/forms/fields.py
> +++ w/django/forms/fields.py
> @@ -606,6 +606,11 @@ class URLField(CharField):
>  class BooleanField(Field):
>      widget = CheckboxInput
>
> +    def __init__(self, *args, **kwargs):
> +        if not args and 'required' not in kwargs:
> +            kwargs['required'] = False
> +        return super(BooleanField, self).__init__(*args, **kwargs)
> +
>      def to_python(self, value):
>          """Returns a Python boolean object."""
>          # Explicitly check for the string 'False', which is what a hidden
> field

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Vote on {% include %} behaviour.

[Tai Lee]

On Jun 3, 9:32 pm, Jonathan Slenders 
wrote:
> I really never want to have the {% block %} names of B/C in previous
> example to be available for overriding in templates which inherit from
> A. This would even cause unexpected collisions between block names.
> The author of the include B, is not supposed to know where his
> template will be included, and we can't expect him to choose block
> names which don't collide with those of A.

I agree, and my preferred fix (to remove `ConstantIncludeNode`) would
make it clear that the actual include is to occur at render time, and
any blocks contained therein are completely separate to blocks in the
parent template. It should not be possible to replace a block in an
included template by defining the same block in the including (parent)
template.

> Further, I'm not aware of any inconsistencies in include behaviour
> when using variable template names. Maybe anyone else?

One inconsistency is that when a literal quoted string is used as the
path to the template to be included, `ConstantIncludeNode` is used
instead of `IncludeNode` as a special case. This tries to include the
specified template when the parent template is compiled instead of
when it is rendered. This is supposed to be functionally the same as
using a variable path argument (which is executed when the template is
rendered), but slightly optimised. Unfortunately, it's not
functionally the same.

One example of this difference causing a problem is that when
`TEMPLATE_DEBUG` is `True`, a `TemplateDoesNotExist` exception is
raised when compiling a template that does exist, if it contains `{%
include "template_that_doesnt_exist.html" %}`. This means that any
code that checks for the existence of a template (with
`select_template()`) could break and behave differently in production
and development environments based on the content of the template.

If the template tries to include the same non-existant template, but
passed to `{% include %}` as a variable, no exception is raised.

Another example cited in the tickets is that it is impossible to
conditionally include a template that uses a literal string as the
path argument. E.g. `{% if some_false_var %}{% include "..." %}{%
endif %}` will raise `TemplateDoesNotExist` because the include is
executed when the parent is compiled, even though it is never rendered
because the context at render time causes the condition to fail.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Vote on {% include %} behaviour.

[Tai Lee]

G'day,

There are several open tickets (some getting quite old now) that all
stem from the inconsistent behaviour of the {% include %} tag. When a
quoted string is used for the path, it is treated as a special case
and the include is executed at compile time. Otherwise, it is executed
as the template is rendered.

The docs don't seem to mention this special case, and a few examples
of buggy behaviour and confusion stemming from this include:

 - Included templates with a missing path sometimes fail silently, and
sometimes loudly.

 - Template loaders raise TemplateDoesNotExist when the specified
template *does* exist, if it happens to use an include with a quoted
path that references another template that does not exist.

 - Included templates do not render in the full context of their
parent template as implied by the docs. This means you can't use `{%
block %}` tags inside included templates and override them in
templates that extend the parent template.

My feeling is that including templates should be consistent. If we
support variables as the path argument to the include tag, then ALL
uses of the include tag should be processed when the templates are
rendered.

This means that it will not be possible to use block tags inside
includes. This is not possible now, but there is an open ticket that
would like to make it possible (only for the special case quoted
string path includes).

I would also like to see includes either always fail loudly, or always
fail silently. I think that the most common and expected behaviour
currently is for includes to fail silently, but I don't really care
either way.

The best solution that I can see is to remove `ConstantIncludeNode`
entirely (as has been suggested in at least one of the tickets).
Includes would always be processed at render time, it would not be
possible to extend blocks defined inside an included template, and
including templates that does not exist would no longer raise a
TemplateDoesNotExist exception when getting the parent template (which
does exist).

Could we get a quick vote from any core committers and other
interested parties, so that we can move forward to fixing and closing
these tickets?

https://code.djangoproject.com/ticket/16147
https://code.djangoproject.com/ticket/12064
https://code.djangoproject.com/ticket/12008
https://code.djangoproject.com/ticket/598

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Field lookup types and backwards incompatibility

[Tai Lee]

I thought Django already did option 1. If it doesn't, why not? What
are the possible edge cases? Using a date field named "date" as a
foreign key to another model that also contains a "date" field?


On May 10, 1:47 am, Ulrich Petri  wrote:
> Currently there are at least 4 open tickets [1-4] that propose the addition
> of new lookup types. Two of them [1,2] are in DDN because of backward
> compatibility concerns.
> Following that (IMHO valid) logic makes it impossible to _ever_ add new
> lookup types to django without breaking backwards compatibility since field
> lookup types and field names share the same namespace.
>
> Interestingly there has been one instance [5] where a lookup type was added
> after the 1.0 release (as far as I could gather from SVN). Why no backward
> comp. concerns were voiced in that case is not clear from the ticket.
>
> Also there is currently no validation preventing field names from clashing
> with the already existing lookup types (date, year, month, day, etc. are
> IMHO "easy targets"). Whether that would be a desirable feature to have is
> also debatable since it would "artificially" limit a developers naming
> choices.
> That using conflicting field names usually causes no problems but blows up
> with a rather confusing error message (e.g. "TypeError: Related Field has
> invalid lookup: [...]") when querying for that field through a relation
> without an explicit lookup isn't helpful either.
>
> To solve this situation I think there are two possibilities:
>
>    1. Make the current implementation "smarter": Don't automatically assume
>    that the last filter part is a lookup type but instead check first if there
>    is a field of that name.
>    2. Separate the field name and lookup namespaces:
>    Either change the separator for lookup types (e.g. to 3 underscores) or
>    introduce a completely different way of specifying lookups (e.g.
>    Thing.objects.filter(some_field__other_field=lookups.Contains("blah")))
>
> Option one clearly is far less invasive (and possible to implement short
> term) but has the potential of user / developer confusion and maybe some
> hard-to-get-right edge cases.
> Option two is a big hammer and possibly (probably?) not something the core
> devs are willing to change even in the long term?
>
> So what do people think?
>
> Bye Ulrich
>
> [1]http://code.djangoproject.com/ticket/8424
> [2]http://code.djangoproject.com/ticket/9596
> [3]http://code.djangoproject.com/ticket/10911
> [4]http://code.djangoproject.com/ticket/12489
> [5]http://code.djangoproject.com/changeset/9818

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: math tag

[Tai Lee]

On May 3, 9:43 am, Russell Keith-Magee 
wrote:
> This stems back to the design motivation of Django's template language
> -- you shouldn't be doing math in the template. Instead, you should be
> doing your math in the view, and providing the template with a
> pre-calculated result.
>
> So, my inclination would be to say no, this doesn't have a place in
> the default list of tags.

If it a rule that people should never do math in the template, Django
wouldn't ship with add and divisibleby filters, possibly also the
widthratio template tag.

While the use case posted elsewhere in this thread is practically
unreadable, I think there are some legitimate use cases where it is
preferable to perform some simple maths in the template (presentation
logic).

The times when I've wished for more than slightly more maths than an
"add" filter have been when iterating through data structures in the
template and displaying as tabular data.

It's not very nice to first iterate through these structures in the
view (which may otherwise be lazy until accessed in the template) and
add extra attributes that relate to presentation to the object before
it gets passed to the template.

Don't get me wrong, I don't think we need advanced maths capability.
But a little more than what we have now would be useful to some
template authors.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Add current_app to request and pass it to template loaders.

[Tai Lee]

On Mar 28, 4:08 pm, Justin Holmes  wrote:
> By "current app," do you mean the app which contains the view to which
> the current URL is mapped?

I mean the "namespace" (instance name) for the requested URL or the
"current_app" attribute of a context object which is supposed to be
used as a hint for reversing named URLs. I think it should also be
passed to template loaders as a hint, so that templates can be
overridden for a specific app instance.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Add current_app to request and pass it to template loaders.

[Tai Lee]

Now that 1.3 is out, does any core dev have an opinion, feedback or
suggestions on this?

I've solved my immediate need with two template loaders (subclasses of
the app_directories loader) that use thread locals. One prefixes the
requested template name with the app name and the other prefixes it
with the namespace.

But I'd still like to see Django improved so that users can create
template loaders which have access to the current app. It would also
be nice for process_view middleware to have access to the other
attributes of the ResolverMatch object (app name, namespace, etc) to
avoid having to resolve URLs twice.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Default project layout / directory structure

[Tai Lee]

If we're talking about the lowest common denominator and keeping
things simple, this is what I think we'd have:

myapp1/
myapp2/
myproject/
-django.wsgi.sample
-manage.py
-management/
--commands/
---myproject_mycommand.py.sample
-media/
--myproject/
---readme.txt
-models.py
-settings.py
-static/
--myproject/
-templates/
--myproject/
-templatetags/
--myproject_tags.py.sample
-urls.py
-vendor/
--readme.txt
-views.py
otherapp1/
otherapp2/

Basically it's what we have now plus three things, samples of common
optional files (which could include a link to the relevant docs page
so people can get started right away), suggestions for naming
conventions (use the app name as a prefix to avoid clashes with other
apps) and a vendor folder for 3rd party reusable apps:

 - a sample wsgi wrapper
 - a sample management command showing that management commands should
be prefixed with the app name to avoid clashes with other apps
 - a readme.txt reminding people that the media folder is for uploaded
content (not static content) and suggesting that they use an
`upload_to` value of `app_name/model_name/field_name` on their model
file and image fields as a starting point
 - a static folder showing that static content should be placed in a
folder named after the app to avoid clashes with other apps
 - a templates folder showing that templates should be placed in a
folder named after the app to avoid clashes with other apps
 - a sample template tag library showing that template tag libraries
should be prefixed with the app name to prevent clashes with other
apps and suffixed with "_tags" to prevent quirky import issues (e.g.
myproject_tags.py, myproject_form_tags.py)
 - a vendor folder that is automatically added to the python path (at
the beginning, or after the project folder) and a readme.txt
explaining that 3rd party reusable apps or specific versions of apps
that exist elsewhere on the system should be placed here and imported
with `from otherapp... import...`

Another thing to point out is that a "project" is just an "app" with a
settings.py file, a media folder, and perhaps a vendor folder.

My big thing is that we should drive home the importance of decoupling
or loosely coupling your apps. Unless you are 100% certain that you
will ALWAYS want to bundle an app with your project AND and that you
will NEVER want to use an app without your project, your other apps
should exist as separate modules that can be placed anywhere on your
python path.

If Django were to automatically add the vendor folder to the python
path as the second folder (after the folder containing your
settings.py file), it would provide a standard way for users to
include specific versions of 3rd party apps or their own reusable apps
that may or may not be installed elsewhere on the hosting environment.
For example, bundling a specific version of Django to be used even if
another version of Django is already installed.

I've recently decoupled a bunch of apps that were initially too
tightly coupled to a common library that I use in most projects. As
the apps grew and as we started working on more projects, it became
troublesome to have to bundle this increasingly large library of apps
when only a handful would be used by a project, and there were also
times when I'd have liked to make use of just one in a project that a
3rd party had source code access to without giving them access to
everything.

I wish that I'd learned sooner to always try and build smaller,
simpler apps with more limited and specific scope that were decoupled
or more loosely coupled to other apps so that they could be more
easily reused. Anything that Django can do to make this point more
obvious would be a good thing.

Cheers.
Tai.


On Mar 17, 2:21 pm, Simon Litchfield  wrote:
> On Mar 15, 9:46 am, Russell Keith-Magee 
> wrote:
>
> > On Fri, Mar 11, 2011 at 1:14 PM, Simon Litchfield  wrote:
> > > Who votes we should come up with a django-blessed 'official' default 
> > > project layout / directory structure?
>
> > Sure -- no disagreement that it would be good to have some common
> > ground with regards to project layout. All we need now is to agree
> > what that blessed project layout should be. :-)
>
> Lowest common denominator is what we're after here I think. Eg --
>
> manage.py
> --conf
> settings.py
> settings_dev.py
> --lib
> other_app1
> other_app2
> --my_app1
> --my_app2
> --media
> --templates
> --templatetags (project tags, non-app specific, if you have any)
> --views.py (project views, non-app specific, if you have any)
> --urls.py (project urls, non-app specific)
>
> Or maybe this --
>
> manage.py
> --conf
> settings.py
> settings_dev.py
> (wsgi files)
> --lib
> other_app1
> other_app2
> --project
> my_app1
> my_app2
> templatetags (project tags, non-app specific, if you have any)
> views.py (project views, non-app specific, if you have any)
> urls.py (project urls, non-app specific)
> --media
> css
> img
> 

Re: Wrong error message when user having is_staff=False tries to login to admin

[Tai Lee]

I also don't think it should be considered a security vulnerability to
reveal that an authenticated user does not have permission to access
the admin (or any other) app.

If the credentials are valid and they authenticate against the defined
authentication backends, then we should assume that we are talking to
a trusted authenticated user and we should present error messages that
are at the very least not misleading.

Assuming that any authenticated user might be an attacker who has
brute forced a password and presenting obscure error messages to
authenticated users is not helping anybody.

Cheers.
Tai.


On Mar 16, 3:41 am, "Brian O'Connor"  wrote:
> 2011/3/15 Juan Pablo Martínez 
>
> > The admin is not "one more app" is (if I may) the app with more weight
> > on most sites. Someone who has access to the admin has access to most
> > or all information. There is no "one more app. "
>
> This has nothing to do with the argument here.  The account in question, as
> already stated many times, has no access to the admin site.  That's the
> whole point of this discussion.
>
> Carelessness or neglect of a click in the admin should't call into> question 
> the admin with the justification "that does not happen
> > again."
>
> This has to do with deliberately misleading users.  I've been stuck by this
> at least once in my django career, and artemy has too.  People make
> mistakes, it happens.
>
> > I think if everyone is going to fix "contrib" to your needs the
> > contrib lost all independence.
>
> I especially don't understand this statement.  The whole point of
> django-developers is to discuss development of django, and by extension
> (because there are no other lists, as far as I'm aware) the contrib
> modules.  Everyone comes here to help make the project better, to help fit
> their needs.  That's the whole point, as far as I'm concerned.
>
> A reasonable suggestion was made, in which a few people came back and said
> that by doing this improvement, it would open a security issue.  Myself, and
> others have stated that in fact, this would not be a security issue, and
> have provided examples.
>
> At this point, I'll absolutely never forget to check the is_staff flag
> purely because I've been following this discussion.  What I don't understand
> is why there is such a huge opposition to the change.
>
> --
> Brian O'Connor

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Proposal: Add current_app to request and pass it to template loaders.

[Tai Lee]

I have a generic app which includes base templates (for HTML
documents, emails, etc.) and also template tags that render a template
(for form fields, pagination forms/links, filter forms/links, etc.)

Sometimes I have a project that has several other apps installed which
both use the base templates and template tags from the generic app.

Sometimes I need to be able to override the base templates or the
templates rendered by template tags, but only for a single app,
without affecting the other apps.

Currently I believe the only way to override these templates for a
single app only is to pass a list of template names instead of a
single template name to a loader in the views for that app, with the
first template name having a prefix of the current app. But this only
works in my views and my template tags (not with built in template
tags like `extends` and `include`).

I could also duplicate these templates and put them in one of my
TEMPLATE_DIRS, but this would affect all apps within my project.

I would like to propose a few things to make this easier (or possible
at all).

Currently, the `process_view()` method of request middleware is passed
the request, the view function, args and kwargs, which are obtained by
the `resolve()` method, which also returns `url_name`, `app_name` and
`namespace`.

If `namespace` was also passed to `process_view()`, it would be
trivial for users to create a middleware that adds `namespace` to
request objects (or Django could ship with one), which could then
easily be passed through to ``Context`, `RequestContext` and
`TemplateResponse` as `current_app` in the user's views without having
to look it up again (which Django already did with `resolve()`) or
hard code it.

`RequestContext` and `TemplateResponse` could even use
`request.namespace` (if set) as a default for `current_app` (if not
explicitly passed as an argument).

Currently the docs recommend that users pass `current_app` to
`Context`, `RequestContext` and `TemplateResponse`. Why not just do it
for them by default? I can't think of a use case where users would
want to pass a `current_app` that does not match the `namespace` of
the requested URL, but even if they did that would also still be
possible.

If template loaders also accepted a `current_app` argument, which
Django could pass automatically (if set on the context) when using
built in template tags (e.g. `extends`, `include`, etc.), then users
could write their own template loaders that looked for templates in a
folder prefixed with `current_app` first. For example, if
`current_app` on my context is "myapp1" and I do `{% extends "otherapp/
base.html" %}`, it could try to load `['myapp1/otherapp/base.html',
'otherapp/base.html']`.

To summarise:

Ideally I would like Django to automatically add `namespace` (or
`current_app`) to request objects via middleware (and give
`process_view()` access to other attributes returned by `resolve()`),
I would like `RequestContext` and `TemplateResponse` to automatically
use this (if set on `request`, and if `current_app` is not explicitly
passed in as an argument), and I would like built in template tags
like `extends` and `include` to automatically pass `current_app` from
the context (if set) to the template loader(s).

Does anyone have any reason why such a thing would be a bad idea and
likely to be rejected, before I start working on a patch?

If we don't want this to be the default behaviour, the very least I
think we would need to make this feature possible is to allow
`current_app` to be passed as an argument to template loaders and for
built in template tags like `extends` and `include` to pass the
`current_app` (if set on the context).

I can already create a middleware to add `namespace` to `request` (by
resolving `request.path` a second time, not ideal but possible). I can
already add `current_app` to all my `RequestContext` and
`TemplateResponse` objects (but it'd be nicer if Django did it for
me). I can create my own helper function that will return a template
name for the current app, if it exists in the current app's template
folder, given a template name for another app. But I can't create a
template loader that built in template tags like `extends` and
`include` would use to look for templates from the current app's
templates folder. At least, not without resorting to storing the
request in thread locals or something.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Intermittent IOError exception raised and emailed to admins during file upload.

[Tai Lee]

As a datapoint, I've had 9 of these errors from 5 different people (2
people retried 3 times each) and 30 successful uploads from 14
different people in the past 7 days on a low traffic site (300
visits).

That's a pretty low success rate. Seven of the errors had a user agent
indicating MSIE 8, one was MSIE 7, and one was Chrome on OS X. If
almost 25% of upload attempts are failing, and 33% of users are
experiencing difficulty, that seems to indicate that web browsers
(especially MSIE) are extremely unreliable at performing uploads.

I'd love to see some data on the reliability of uploads on other
higher traffic sites across different browsers and using different
server side technologies, to see if this really is typical.

Cheers.
Tai.


On Feb 5, 2:23 am, Jacob Kaplan-Moss  wrote:
> On Fri, Feb 4, 2011 at 2:45 AM, Tai Lee  wrote:
> > It seems to happen with very small file uploads as well. I've seen it 
> > reported with 30KB uploads, while at the same time 60MB uploads work. If I 
> > can't find a problem with the server causing disconnects, and it is 
> > actually on the client side, I'll just have to continue to ignore the 
> > exception reports Django emails as a result :)
>
> Based on my experience with this error (which, sadly, is extensive)
> it's almost certainly client-side. When there are server-side
> disconnects it's not intermittent: every server-side disconnect issue
> I've seen has been reproducible and thus fixable.
>
> But there's a whole bunch of client-side issues that could be causing
> this, and you probably can't do anything about it. Some examples of
> where the problem can be:
>
> * Browser: you can get this when a user just clicks the "stop" button
> or navigates away. Some browsers close the connection cleanly; others
> don't.
> * Browser: some versions of IE, especially on Vista, randomly
> terminate uploads. Most of the time it's seemed to be related to the
> windows firewall thing.
> * Browser: various development versions of Firefox, Chrome, and Safari
> have all had upload/disconnect issues. To my knowledge none of these
> issues have made it into shipping versions, but many users of these
> browsers use the cutting edge.
> * Dodgy 'net connections, especially public ones. For example, I can
> pretty consistently trigger an IOError by uploading a file from Kansas
> City International Airport.
> * Badly-configured intermediary caches, like those run by ISPs. AOL
> dialup users (yes, there are still some of these!) seem to throw lots
> of IOErrors. (I traced this town by noting that certain /24 and /32 IP
> spaces seemed more prevalent in the error tracebacks and tracked down
> those block owners.)
>
> So yeah, all of those are out of your control, which is why I (and
> many others) usually just say to ignore the error.
>
> However, I think it's a bad idea to automatically suppress these
> errors (which you could do with a piece of middleware or a custom log
> handler in 1.3). That's because this error could *also* be caused by
> something misconfigured on your end (a messed up HTTP load balancer,
> for example). So if you just suppress the error you could end up
> missing something that *is* fixable.
>
> So we're back to "read and then ignore," sadly. I would *love* a
> better way, but I can't think of one.
>
> Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Intermittent IOError exception raised and emailed to admins during file upload.

[Tai Lee]

It seems to happen with very small file uploads as well. I've seen it reported 
with 30KB uploads, while at the same time 60MB uploads work. If I can't find a 
problem with the server causing disconnects, and it is actually on the client 
side, I'll just have to continue to ignore the exception reports Django emails 
as a result :)

Cheers.
Tai.


On 04/02/2011, at 7:41 PM, -RAX- wrote:

> Have you considered the possibility that it would be a client/browser
> side error?
> 
> Both windows7 and chrome have a known open bug with large file
> uploads. Many of my customers have reported this bug, all of them
> using windows7 and chrome.
> 
> Has anyone seen this happening from a Mac or from Linux?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Intermittent IOError exception raised and emailed to admins during file upload.

[Tai Lee]

Thanks for your detailed explanation, Graham. I'll try to approach
this from another angle and see if I can determine what is actually
causing the connections to be dropped, as I'm seeing this error a few
times a day on a low traffic site using Apache at the front for static
media, with proxy pass to Django under Apache/mod_wsgi.

Cheers.
Tai.


On Feb 4, 6:17 pm, Graham Dumpleton 
wrote:
> The problem is you can't just catch IOError, because the presence of WSGI
> middleware may mean that IOError could be generated for other reasons when
> reading wsgi.input. To base detection on the description in the IOError,
> ie., 'request data read error', would mean having to know what that
> description could be on all WSGI hosting mechanisms, and then just hope that
> there aren't other reasons why it could arise from WSGI middleware.
>
> So, no simple solution. This unfortunately is one of the short comings of
> the WSGI specification. That is, there is no standardised exception type to
> represent actual real problems with reading original wsgi.input. Part of the
> reason is that there is no standard module associated with the WSGI
> specification to hold the exception type, so nothing to compare against. The
> only solution would be for WSGI specification to be enhanced to require a
> unique exception type to be used for wsgi.input exceptions such as aborted
> connection, with the type of that exception being passed through in the WSGI
> environ dictionary so it could be compare against. For example something
> like:
>
>   try:
>     data = environ['wsgi.input'].read()
>   except wsgi['wsgi.input_exception], e:
>     # ignore it
>
> In summary, no real practical solution at this time for this.
>
> Graham
>
>
>
>
>
>
>
> On Friday, February 4, 2011 1:55:25 PM UTC+11, Tai Lee wrote:
>
> > There are many questions about this on django-users. The usual answer
> > is to ignore the errors, which are probably generated as a result of
> > the client prematurely terminating a connection, usually during an
> > upload.
>
> > Nobody seems to be able to reliably reproduce the error, and it seems
> > to happen with large and small uploads alike, but always
> > intermittently.
>
> > It has been suggested that a client dropping the connection should be
> > within what one can expect from a networked client, and thus Django
> > should not email the exception and traceback to the admins in this
> > case. From my preliminary investigation with this issue, I tend to
> > agree.
>
> > Is it feasible to handle this exception in Django, or are there other
> > issues that would prevent us from isolating this case from other
> > legitimate exceptions that may be raised at this point?
>
> > Here's a recent traceback:
>
> > Traceback (most recent call last):
>
> >  File "/home/tailee/django/core/handlers/base.py", line 105, in
> > get_response
> >    response = middleware_method(request, callback, callback_args,
> > callback_kwargs)
>
> >  File "/home/tailee/django/middleware/csrf.py", line 224, in
> > process_view
> >    request_csrf_token = request.POST.get('csrfmiddlewaretoken', '')
>
> >  File "/home/tailee/django/core/handlers/wsgi.py", line 209, in
> > _get_post
> >    self._load_post_and_files()
>
> >  File "/home/tailee/django/http/__init__.py", line 207, in
> > _load_post_and_files
> >    self._post, self._files = self.parse_file_upload(self.META, self)
>
> >  File "/home/tailee/django/http/__init__.py", line 169, in
> > parse_file_upload
> >    return parser.parse()
>
> >  File "/home/tailee/django/http/multipartparser.py", line 192, in
> > parse
> >    for chunk in field_stream:
>
> >  File "/home/tailee/django/http/multipartparser.py", line 314, in next
> >    output = self._producer.next()
>
> >  File "/home/tailee/django/http/multipartparser.py", line 468, in next
> >    for bytes in stream:
>
> >  File "/home/tailee/django/http/multipartparser.py", line 314, in next
> >    output = self._producer.next()
>
> >  File "/home/tailee/django/http/multipartparser.py", line 375, in next
> >    data = self.flo.read(self.chunk_size)
>
> >  File "/home/tailee/django/http/multipartparser.py", line 405, in read
> >    return self._file.read(num_bytes)
>
> >  File "/home/tailee/django/http/__init__.py", line 231, in read
> >    return self._stream.read(*args, **kwargs)
>
> > IOError: request data read error

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Intermittent IOError exception raised and emailed to admins during file upload.

[Tai Lee]

There are many questions about this on django-users. The usual answer
is to ignore the errors, which are probably generated as a result of
the client prematurely terminating a connection, usually during an
upload.

Nobody seems to be able to reliably reproduce the error, and it seems
to happen with large and small uploads alike, but always
intermittently.

It has been suggested that a client dropping the connection should be
within what one can expect from a networked client, and thus Django
should not email the exception and traceback to the admins in this
case. From my preliminary investigation with this issue, I tend to
agree.

Is it feasible to handle this exception in Django, or are there other
issues that would prevent us from isolating this case from other
legitimate exceptions that may be raised at this point?

Here's a recent traceback:

Traceback (most recent call last):

 File "/home/tailee/django/core/handlers/base.py", line 105, in
get_response
   response = middleware_method(request, callback, callback_args,
callback_kwargs)

 File "/home/tailee/django/middleware/csrf.py", line 224, in
process_view
   request_csrf_token = request.POST.get('csrfmiddlewaretoken', '')

 File "/home/tailee/django/core/handlers/wsgi.py", line 209, in
_get_post
   self._load_post_and_files()

 File "/home/tailee/django/http/__init__.py", line 207, in
_load_post_and_files
   self._post, self._files = self.parse_file_upload(self.META, self)

 File "/home/tailee/django/http/__init__.py", line 169, in
parse_file_upload
   return parser.parse()

 File "/home/tailee/django/http/multipartparser.py", line 192, in
parse
   for chunk in field_stream:

 File "/home/tailee/django/http/multipartparser.py", line 314, in next
   output = self._producer.next()

 File "/home/tailee/django/http/multipartparser.py", line 468, in next
   for bytes in stream:

 File "/home/tailee/django/http/multipartparser.py", line 314, in next
   output = self._producer.next()

 File "/home/tailee/django/http/multipartparser.py", line 375, in next
   data = self.flo.read(self.chunk_size)

 File "/home/tailee/django/http/multipartparser.py", line 405, in read
   return self._file.read(num_bytes)

 File "/home/tailee/django/http/__init__.py", line 231, in read
   return self._stream.read(*args, **kwargs)

IOError: request data read error

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small decision needed on #15025

[Tai Lee]

On Jan 13, 5:06 am, Don Spaulding  wrote:
>
> I can't speak to the number of templates actually in the wild that
> rely on the old behavior.  I will say that the debug template has been
> "relying on the bug" since 2005.  By my estimation, that makes nearly
> every template ever created a candidate for relying on the old
> behavior.  And for a python dev to treat callables as first-class
> objects is hardly unusual.
>
> ;-)

Not quite, because as detailed elsewhere the debug page is not a
normal template, it's a special case because we need to inhibit the
normal exception handling behaviour of the template system for all
exceptions. Anyone who has a similar special case view/template in
their own app should also be escaping potentially dangerous variables
before they reach the template.


> What do you think about throwing in a nod towards backwards
> compatibility by special-casing attribute access on callables?
> Basically, keep the functionality of [14992] that says "Callables are
> always called" and just add " unless the next lookup is for an
> attribute that exists on the callable".  There's a potential for name
> clashes if you were expecting the template lookup to happen on the
> return value of the callable instead of the callable itself, but I
> would think that's a much smaller incompatibility than the one created
> by [14992].

This feels too magical to me, and it still leaves the possibility for
ambiguous behaviour as you note. Without knowing all of the attributes
on your variables, you won't know if a template variable will resolve
to an attribute of the callable or it's return value. And when a clash
does happen, there's no easy way a template author to debug or resolve
it. I also don't think the degree of compatibility is part of the
equation. If the old behaviour is a bug (inconsistent treatment of
dict, object and list variables), we don't need to retain
compatibility. If it's not a bug, we do need to retain full backwards
compatibility.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small decision needed on #15025

[Tai Lee]

If we can, I would be in favour of treating the old behaviour as a bug
and not having to support it while it follows a deprecation path.

However, either way, if the new behaviour stays (and I definitely
think it should) I think we should update the docs to clarify that
there was a change in behaviour in 1.3, that the new behaviour is to
try a dictionary lookup, an attribute lookup, a list index lookup, and
then to try calling the resulting object, and that the reason for the
change was for consistency (where previously Django assumed that only
object attributes could be callable).

I've created a new ticket and uploaded a patch to do just that.

http://code.djangoproject.com/ticket/15057


On Jan 12, 1:31 pm, Luke Plant  wrote:
>
> Thanks for your explanations and research on this. I think it is a case
> where the docs don't specify the process exactly enough, so it is
> somewhat borderline.
>
> The docs don't actually make sense, as they talk about "method call" as
> a "lookup", when technically there is no such thing in Python - just
> attribute lookup, followed by calling a callable - and at the level of
> detail we are arguing that distinction matters.
>
> So we then have to think about what is sensible behaviour, and I think
> the new behaviour is much simpler and saner, and much more consistent
> with itself. Also, given the nature of the Django's template system
> (i.e. usable for designers), it is difficult to come up with a use case
> for the old behaviour. The use case linked by Tai is very unconvincing -
> to me, using 'mycallable.func_name' is a gross thing to do in a template
> - a fairly blatant case of logic that belongs in the view function.
>
> If we decide that the old behaviour was a bug, it doesn't (technically)
> matter that the fix breaks some people's templates, because our policy
> is to fix bugs and not to support bugs. The only exception would be when
> a large fraction of people are relying on the bug, and I don't think
> that is likely to be the case here.
>
> So, yes, you were relying on a bug IMO - Django was treating objects
> retrieved by dictionary and list index lookup inconsistently compared to
> objects retrieved by attribute lookup, which is not a sensible thing to
> do in Python.
>
> We may want to put a note in the backwards incompatible changes section,
> as a courtesy to those whose templates may stop working. But I don't
> think the deprecation path proposed by Tai would be useful - it would
> just confuse things more, and it isn't appropriate if we treat the
> original behaviour as a bug. I also think that mrmachine's analysis and
> patch on #15025 is correct.
>
> Thanks,
>
> Luke
>
> --
> "Whom have I in heaven but You?
> And there is none upon earth that I desire besides You." Psalm 73:25
>
> Luke Plant ||http://lukeplant.me.uk/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small decision needed on #15025

[Tai Lee]

I just discussed this with Don on IRC. An example of the use case
described is at:

http://djangosnippets.org/snippets/1016/

The fix for this use case is easy. Pass a list of `(button.func_name,
button.short_description)` tuples as `buttons` in the context instead
of passing a list of callables. But the problem is that a previously
working template stopped working, with no obvious reason why.

The docs say that template variables are resolved by trying a (in this
order, with shortcut logic) a dictionary lookup, an attribute lookup,
a method call and a list index lookup. Technically the docs weren't
even correct prior to [14992] because there was no shortcut between an
attribute lookup and a method call, but that's a very minor pedantic
issue. Now, it tries a dictionary lookup, an attribute lookup and a
list index lookup, and then tries to call the resulting value.

To avoid rolling back [14992] as a backwards incompatible change, I
propose a deprecation path.

Callables obtained from a dictionary or list index lookup can continue
to return the raw callable (along with a deprecation warning) unless
`getattr(callable, 'template_callable', False)` is `True` so that
people can opt-in to the new behaviour.

In 1.5 that changes to `getattr(callable, 'template_callable', True)`
so that the new behaviour becomes default, while still allowing people
to retain the old behaviour by setting a `template_callable` attribute
to `False` on their callable.

This should allow us to move to the new (more consistent) behaviour
and avoid the problem of a silent change in template rendering that
leaves developers with no indication as to why their previously
working templates are suddenly broken.

Another problem (described at #15025 and fixed by the patch there) is
with the debug page. It is now resolving callable stack trace frame
local variables instead of just describing them. If such a variable
raises an exception, a raw traceback is displayed in the browser and
the original traceback is lost. If it doesn't raise an exception, it's
still going to display incorrect information to the developer, e.g. a
form instance instead of a form class.

Cheers.
Tai.


On Jan 12, 3:30 am, Don Spaulding  wrote:
> Hi django-devs,
>
> Prior to r14992 (which is the fix for #7153), the way a callable
> context variable was resolved depended on where it was in the lookup
> chain.  Sometimes it was invoked, other times it was treated as a
> first-class function.  r14992 changed that behavior so that the
> callable is always invoked before resolving the next bit of the
> variable.  I'm starting to see the other side of the coin, that it is
> more consistent for the template language to behave in this manner.
>
> The thing I can't decide is how much of a corner case my template
> usage is.  I certainly rely on both styles of callable resolution in
> my templates.  Basically, they both Just Worked for me.  I never gave
> "sometimes I get a callable, sometimes I get the result of calling a
> callable" a second thought.  I'm looking for some feedback on whether
> I've just been relying on a bug in the template engine and need to fix
> my templates, or whether r14992 is pulling the rug out from under my
> (working, IMO) templates.
>
> #15025:http://code.djangoproject.com/ticket/15025
> r14992:http://code.djangoproject.com/changeset/14992
> #7153:http://code.djangoproject.com/ticket/7153

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Some tickets that need love before 1.3 feature freeze.

[Tai Lee]

There are a few tickets that I believe are or should be RFC that I
would like to see committed before the 1.3 feature freeze kicks in,
which I just heard was very soon now. If anyone has time, could they
please review the following and bump to RFC or provide additional
feedback if they're not yet ready?

http://code.djangoproject.com/ticket/11206

This one just prevents the `floatformat` filter from returning some
numbers in scientific notation.

http://code.djangoproject.com/ticket/14820

This one is a documentation change that more clearly explains the
compatibility issues between `object_id` and primary key fields in
generic relations.

http://code.djangoproject.com/ticket/9029

And this one fixes an ugly inconsistency in `get_or_create`.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Enabling context access in simple_tag

[Tai Lee]

I've been using the patch from #1105 in my local branch of Django for
a long time now. To be honest, I've hit the need to have access to the
existing context in a simple_tag far more frequently than the need to
return a single value as a named variable in the context, and I'm
definitely +1 on adding a `takes_context` argument to `simple_tag` (or
something else that achieves the same thing).

However, regarding the `register.assignment_tag` proposal, we still
have the same problem -- some people will want access to the context
in there, too.

One suggestion from #1105 was to split out this functionality into
individual decorators, @takes_context, @takes_block. I'm not sure how
easy this would be technically to implement, but I think it would
solve the `takes_context_plus` sink problem Malcolm describes as we
potentially add more special case tag types (simple, inclusion,
assignment, etc.)

However, I guess this would be more involved and may involve some
backwards incompatibility and a deprecation path. I don't think any
full blown ideal solution should prevent us form committing just the
`takes_context` argument to `simple_tag` immediately (bringing it
inline with `inclusion_tag`, which has had this ability forever), a
request that has been around for a long time that would solve both use
cases mentioned here.

Even though it doesn't have the "as X" syntactic sugar of an
`assignment_tag`, it is more versatile -- you can update more than
just one variable, and if you need to name the variable it will go
into you could always just use the first argument to your simple tag
to define it e.g. `{% getfunction_as "varname" %}`

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Add a "needinfo" state to triaging

[Tai Lee]

I believe that only the reporter, owner, and CCs are notified when an
update is made, not anyone who added a comment. Unless a reviewer adds
themselves to the CCs when providing feedback or assigns the ticket to
themselves (which would be unlikely, when leaving feedback that the
reporter needs to take further action), the reviewer won't be
notified. But other people who stumble across the ticket might think
that since feedback was been provided, and possibly even actioned
already by the reporter, that the ticket will be monitored by the
reviewer and therefore leave the ticket alone themselves.

The problem is, as you've mentioned, that is is super easy for busy
people to skim through a large volume of trac email notifications when
the majority are just people adding a comment or adding themselves to
the CCs. There's nothing that makes it stand out to indicate who is
responsible for the next step, whether the next step is pushing it
back to the reporter, closing it, finding someone else to review it,
marking it RFC or actually committing it.

I think there needs to be more ownership of tickets and a more direct
way to make it clear to reporter, reviewer, and any other interested
parties, in whose court the ball lies for the next steps to be taken.
Similar to the way major features need to be championed by at least
one core committer, I'd like to see more of a mentorship type scenario
between reporter and reviewer, sort of like remote pair programming,
to avoid stagnation.

Explicitly, this would involve a commitment from the reporter to
action feedback given to them and a commitment from the reviewer to re-
visit the ticket once that feedback has been actioned and update its
status accordingly. At the moment all these tickets just sit in a pool
waiting for the lucky day when some random person will stumble across
it and hope that they don't just assume someone else is taking care of
it and leave it as is.

This doesn't have to be done with new flags on trac, but maybe a
change to the contributing guide that tells people that when they
review a ticket and provide feedback on it, they can expect the
reporter to assign the ticket back to them when the feedback has been
actioned making it clear that they should revisit it? I'm not sure if
this would trigger an email notification to the new "owner", if they
didn't take ownership of the ticket themselves?

Cheers.
Tai.


On Nov 18, 12:17 am, Gabriel Hurley  wrote:
> > Maybe trac can be improved in this respect by notifying
> > reviewers when tickets that they have closed, or accepted, or provided
> > feedback on, are updated.
>
> In my experience Trac already does that... when a ticket is changed
> (by anyone), the reporter, the owner, anyone on the cc list, and
> anyone who has changed the ticket (by adding a comment, triaging,
> etc.) gets an email. I get quite a few notifications every day from
> Django's Trac install. I can only imagine what it's like for the most
> active devs.
>
> The issue which has already been alluded to is that these are
> unfiltered streams of information. Getting 30 emails that say someone
> added themselves as a CC on this ticket or that ticket makes it less
> likely that you'll see the one email where someone reviewed a patch
> and marked it as RFC. This is where good Trac queries come in.
>
> As always, the rest comes down to the real hours we're all willing and
> able to put in :-)
>
> All the best,
>
>     - Gabriel

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Add a "needinfo" state to triaging

[Tai Lee]

This has been my experience, as well. I know this is an open source
project and that we're all volunteers, but I have found it it
extremely disheartening and a discouragement to further contribution
when I have invested the time to do the right thing and submit a
detailed report, with patch and docs and tests, and the tickets remain
untouched or merely accepted without feedback (or without further
feedback once initial feedback is addressed) for a prolonged period of
time.

Yes, it's possible to find tickets with many combinations of flags
that need certain types of attention, but the big problem is that
these tickets aren't being found. All the processes in the world won't
help if at the end of the day we just say that sorry, nobody was
interested in reviewing that particular combination, or a reviewer
wasn't notified or failed to follow up on a ticket where they provided
feedback that has been addressed and other reviewers simply left the
ticket alone because they assumed that it would be reviewed again by
the person who initially provided feedback.

I think that while core committers are only volunteers as well, they
should lead the community by example and follow up tickets that they
have accepted, or provided feedback on, or simply accepted without
feedback. Maybe trac can be improved in this respect by notifying
reviewers when tickets that they have closed, or accepted, or provided
feedback on, are updated.

Cheers.
Tai.


On Nov 17, 6:17 am, George Sakkis  wrote:

> I admit I am guilty of breaking the (unknown to me) rule/etiquette of
> marking my own tickets RFC as a last resort to move them forward.
> Unlike your example workflow, my experience is often like this:
>
> * I create a ticket and submit a patch plus passing tests (no need for
> docs if it's a bug or a promise to add them once it is reviewed and
> accepted, i.e. it passes the DDN stage).
> * The ticket stays unreviewed for days, weeks or months or it is
> marked as accepted at best, without actual feedback how to proceed,
> one way or another.
> * At some point I mark it RFC since as far as I am concerned it *is*
> ready for checkin.
> * More time passes and still nobody bothers.
>
> If I'm doing it wrong, please educate me.
>
> George

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Add a "needinfo" state to triaging

[Tai Lee]

I'm in favour of closing tickets with "need more info, re-open when
you have it or if you disagree" because it clearly puts the onus back
on the reporter to re-open the ticket after addressing the feedback in
order to see any progress.

Leaving tickets "open" with potentially yet another status or flag
indicating that it needs some type of action will just contribute to
the problem we already have with many tickets stagnating in some
combination of "open" for literally years without any action.

If a ticket is marked as "needs more info" while remaining open, it
may be that only the original reporter (or somebody who is seriously
motivated to dig) will be able to provide that more info, and if it's
off their radar for whatever reason the ticket will likely remain in
that state forevermore.

Cheers.
Tai.


On Nov 17, 1:01 am, Andrew Godwin  wrote:

> It was ticket state, not closing state (sorry about the reply lag!). I
> guess it's more that I don't like closing bugs purely because they've
> not got enough information - it gives the wrong impression in my mind (I
> only close once I have positive affirmation that it is invalid,
> worksforme, etc.)
>
> This is, I suspect, more a relic of the way I use Trac. I'm fine with a
> closing state as well, I just feel like closing tickets before we even
> know what they are sends the wrong impression.
>
> On the other hand, we want to keep the trac as cruft-free as possible,
> so I suppose closing tickets is fine in that regard. The ticket state
> proposal has merit in that it can be crafted so that someone replying to
> a ticket in needsinfo state moves it back into a "normal" state, thus
> needing the minimum level of Trac competency from the reporter, and not
> presenting a rather imposing "ticket closed!!!" interface - getting to a
> project's ticket and finding it closed deters me from replying a lot
> more than finding it in a "needsinfo" state.
>
> Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Doc. patch

[Tai Lee]

The following two tickets are fairly trivial, with docs and tests,
both accepted by a core dev 7 plus months ago.

http://code.djangoproject.com/ticket/12398
http://code.djangoproject.com/ticket/13291

This one is 2 years old and has no docs (but I'm not sure if they are
required). It was marked as DDN by Jacob who thought there was no bug,
but subsequent comments seem to indicate that there is. It also has
several dupes.

http://code.djangoproject.com/ticket/8898

This one is a year old and needs a decision, but is fairly trivial so
should not require too much debate to reach a consensus.

http://code.djangoproject.com/ticket/11206

This one is also a year old, was accepted by a core committer, and I
think I've addressed the feedback received to date.

I'd appreciate it if anybody could review these tickets and provide
further feedback on what action I need to take to get things moving
again, make a decision or call for a vote on the DDN tickets, or to
bump them up to RFC where appropriate.

Cheers,
Tai.


On Nov 17, 8:36 am, SmileyChris  wrote:
> On Nov 17, 10:10 am, Luke Plant  wrote:
>
> > Well, if your ticket hasn't received the attention you think it
> > deserves, then it is, but you waited less than 10 minutes from filing
> > the ticket!
>
> Agreed, perhaps a bit hasty :)
> But in any case, I just committed it.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Add a "needinfo" state to triaging

[Tai Lee]

I like the idea of needmoreinfo as a resolution, which makes it clear
to the reporter that they need to take the next step to re-open the
ticket with more info. I don't think that closed with "invalid" and a
comment makes this as clear.

However, I think there's another problem area where we need to be able
to make it clear that someone needs to take the next step, and that is
when a ticket has been "accepted", feedback has been given to the
reporter, and the reporter has actioned that feedback (e.g. by
uploading a patch with tests and docs as per the feedback). In this
case the ticket will often languish in "accepted" for months (or
years). Since it is frowned upon for a reporter to mark their own
ticket as RFC, there's no way for the reporter to flag the ticket as
feedback actioned and put it back in the court of the original triager
or core developer who accepted it and gave their feedback, who can
then either push it up to RFC or commit it themselves.

Cheers.
Tai.


On Nov 15, 12:35 pm, Russell Keith-Magee 
wrote:
> On Sun, Nov 14, 2010 at 5:22 PM, Andrew Godwin  wrote:
> > On 13/11/10 16:52, Daniel Moisset wrote:
>
> >> Hi,
> >>    while working on the sprint today doing triaging we noticed that a
> >> lot of tickets were in the "Unreviewed" state because actually there's
> >> not enough information to move it to any other state (they can not be
> >> neither accepted/DDNd nor closed). In most cases we sent a reply back
> >> to the submitter asking for more details about their problem, but the
> >> ticket remains in the "Unreviewed" state, still taking the time of
> >> other triagers looking for tickets to review.
>
> >> Many projects have a "need information" state to report this
> >> situation. I think adding this to the ticket state diagram [1] would
> >> be helpful for triagers, and to developers reviewing the ticket list
> >> and wanting to know how many tickets actually need to be looked at
> >> before a release.
>
> >> What are the thoughts of the core team on this?
>
> > We have this on the South trac, and it helps me greatly in filtering views
> > to just tickets that need my attention/responses instead of just getting
> > confronted with "all open tickets". You can also set it up so the default
> > action once you're in infoneeded is to move out of it, so it shouldn't even
> > cause more work for responders as they have to change the state back - it
> > just does.
>
> > I'm +1 on adding this, but that's definitely biased by the fact that it
> > makes the two Tracs I use on a regular basis more similar.
>
> To be clear, Andrew -- are you +1 to a "Needs Info" closing condition,
> or a "Needs info" ticket state?
>
> For me, it all hinges on whether a you consider that a reported issue
> should be open or closed by default. I tend towards "closed" by
> default, simply because our Trac instance is already a graveyard of
> old tickets. The history of Django's Trac instance doesn't suggest
> that an incomplete bug report will be updated after initial reporting;
> unless the reporter is a known member of the community, if a ticket
> isn't complete at time of reporting, it isn't going to get any better.
>
> To that end, I'm not sure what a Needs Info ticket state gains us. If
> someone opens a ticket that doesn't contain enough useful debugging
> information, we should be closing the ticket, not keeping it open for
> eternity in the hope that they'll come back and provide the detail
> they should have provided in the first place.
>
> If the reporter *is* inclined to provide more information, a "needs
> info" close state gives them an indication that the ball is in their
> court, differentiating between closed "Invalid" (i.e., You're wrong)
> and closed "WorksForMe" (i.e., can't reproduce), while keeping the
> ticket out of our working list until such time as they provide the
> missing details.
>
> So; I'm +0 to a close state, -0 to a ticket state. I'm +0 because this
> is already possible right now -- close "invalid" with a comment that
> indicates the ticket should be reopened if the reporter can provide
> more details.  However, I can see that there might be a slight
> advantage to communicating that with something other than a comment.
>
> Yours,
> Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model proxy instance does not equal the respective model instance

[Tai Lee]

I think that since Proxy models only affect the methods, properties
and way the data is used (not the actual underlying data itself), it
is appropriate to compare User and UserProxy as equal. To not treat
them as equal, you will need to be sure that you never pass a
UserProxy object to a 3rd party function that will attempt to compare
it to a User object. If you need to in your own code, you can always
use isinstance in combination with equality to determine how equal the
two objects are.

Cheers.
Tai.


On Oct 21, 7:21 am, Gabriel Hurley  wrote:
> Speaking on a semantic level, a "proxy" is a stand-in acting on behalf
> of (or in place of) another entity. There is an implied near-
> equivalence, but inherent in the idea of a proxy is that it is *not*
> the same as the original. As in the case of assigning a proxy to vote
> for you in corporate elections: the proxy you delegate your right to
> vote to is the functional equivalent of you, but they are recognized
> as not *actually* being you.
>
> In my mind the same reasoning applies to Proxy Models. While they are
> a stand-in for accessing the same underlying data, they may have very
> different properties (ordering, methods, etc.). So while they may pass
> a rough equivalence test, they are in specific ways dissimilar. The
> underlying table/data stored by the model is only one piece of what
> makes up the sum total of a model.
>
> That said, I can certainly see the use case for comparing proxy models
> as equal to the original model...
>
> All the best,
>
>     - Gabriel
>
> On Oct 20, 6:24 am, Byron  wrote:
>
>
>
>
>
>
>
> > Continuing from...http://code.djangoproject.com/ticket/14492
>
> > > the real issue here is "What is a Proxy", and "what is equality"
>
> > I agree, that is the real issue. I based most of my reasoning off of
> > what the docs already state in that a proxy model can be written to
> > extend the behavior of the model being proxied, i.e. the data model
> > cannot be changed. That being said, I feel the default behavior when
> > dealing with equality, other data related things... should inherit
> > from there parent. If there is a need to override this behavior, I am
> > sure other hooks can be implemented to customize proxy models
> > differently then there parent. With regards to my primary issue of
> > equality, in this context two objects are being compared to determine
> > whether they represent the same row of data. It seems less likely to
> > compare two model objects to only be interested in whether they are
> > equal data structures.
>
> > Regarding permissions, the arguments presented 
> > inhttp://code.djangoproject.com/ticket/11154
> > all can be accomplished with writing a custom manager for the proxy
> > model or writing a custom ModelAdmin class for the admin. I agree with
> > Malcolm that a proxy model's permission should shadow its parent.
>
> > On Oct 19, 9:08 am, Byron  wrote:
>
> > > Added ticket and patch:http://code.djangoproject.com/ticket/14492
>
> > > On Oct 17, 12:00 pm, Byron  wrote:
>
> > > > I came across an issue when comparing a model proxy instance and a
> > > > model instance, that is:
>
> > > >     obj1 = Entry.objects.get(id=1)
> > > >     obj2 = EntryProxy.objects.get(id=1)
>
> > > >     obj1 == obj2 # False
>
> > > > This I feel is a bug and the culprit is in ``Model.__eq__`` where it
> > > > tests to see if the object being compared against is an instance of
> > > > the same class of the Model itself.. which will always be false for
> > > > proxy models being compared to there non-proxied instance
> > > > counterparts. It should test if it is a proxy and whether the object
> > > > is an instance of the parent class as well.
>
> > > > Was this a design decision or overlooked?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ANN: Improving our decision-making and committer process

[Tai Lee]

Hi Russ,

On Oct 5, 11:48 am, Russell Keith-Magee 
wrote:

> There have been a couple of suggestions recently that the contributing
> guide should be distilled into a specific HOWTO for new users. I
> suppose the idea here would be for the contribution guide to be the
> letter of the law, and the HOWTO to be the spirit of the law -- a
> lightweight guide pointing people at some specific activities they
> could engage in as a new contributor.

I see. Sounds reasonable. I can see that it might be difficult to
extract the relevant information from the full contributing guide for
newbies, as well as for long time users wanting a quick reference.

> > Perhaps we need another triage stage for these tickets, "Needs final
> > review" or something?
>
> That's essentially what RFC is. This is an area where our documented
> procedures have lagged behind in-practice procedures, but we've
> essentially accepted that once a patch has been written by one person,
> and reviewed by another and found to be ok, then it's RFC, which means
> a core committer needs to take a look.

I'm happy for RFC to indicate that a ticket is ready for final review
by a core committer, after being reviewed by at least someone other
than the patch author. But, I think there's still a problem of
accepted tickets that have a patch with docs and tests and no feedback
indicating that it needs improvement, being lost in the shuffle, as
well as how and why we ever end up with tickets in this state and how
can the patch author get someone to review their work.

It could happen that a ticket is accepted on merit, then it gets a
patch with docs and tests, and then sits there getting no attention
because nobody knows it needs a review. The patch author could just
move it to RFC themselves as it had previously been accepted and he or
she subsequently added the required docs and tests, in order to get
somebody to look at it. But I think this is generally frowned upon and
might take up too much core committer attention before the ticket is
really ready.

It could also happen that a ticket is unreviewed and already has a
patch with docs and tests, if it is reviewed by a core committer (or
not) and "accepted" without feedback about any necessary improvements,
shouldn't it have simply gone straight to RFC instead? What should the
original patch author do when his or her patch with docs and tests is
reviewed by someone else and moved to "accepted" like this? There's no
feedback that they need to incorporate into the patch, and they can't
move their own work to RFC, even though someone else has reviewed and
"accepted" the patch?

I'd like to see the ticket system work smoothly in and of itself, and
for tickets to be reviewed without the patch authors having to ask
around on IRC or django-developers for someone to take a look. I don't
want to make the ticket system too complicated, but I think it is
already a bit muddled by determining whether or not a ticket needs
review by checking 5 or 6 different properties, instead of a single
clear indicator set by the patch author that he or she would like
somebody (not necessarily a core committer) to review the work. This
could also help in eliminating another problem of not knowing if a
ticket is still being worked on or if the patch author considers it
finished.

> As a general principle, DDN isn't usually a situation where more
> information is needed; it's usually just time that is lacking.
>
> Sprints definitely help push this sort of thing along. However, that's
> more because we have two core developers in a room at the same time,
> not because of the presence of the rest of the community (although the
> community got lots of other great work done). Malcolm and I spend a
> day at the recent sprints just doing DDN tickets, and we managed to
> clear out quite a few DDNs. We made vague plans to hook up on
> IRC/Skype as a semi-regular activity to try and clear out the backlog,
> but we haven't made that happen yet.

Are at least two core committers required for all DDN tickets? Can a a
single core committer make decisions on minor or trivial tickets, and
the reporter be given the option to request a vote or further
consideration if they feel the decision is wrong (possibly based on
lack of information)? Perhaps some of the people who are well known
and long standing in the community (but who don't have commit access)
could make some of these decisions?

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ANN: Improving our decision-making and committer process

[Tai Lee]

Hi Jacob,

Thanks for your feedback.

> For (1) check out http://code.djangoproject.com/wiki/Reports(it's
> linked in the nav). If there's anything missing there, please feel
> free to add it -- it's a wiki page. Let me know if you need help
> figuring out the linked query syntax.

I wasn't able to edit this page. Other wiki pages have an "Edit this
page" and "Attach file" buttons at the bottom, but not the Reports
page.

I was just going to add a link to tickets that are Accepted with a
patch that doesn't need docs, tests or improvement as tickets that
just need a fresh pair of eyes to review and either bump up to Ready
for checkin or back down to Patch needs improvement (with feedback).

I also noticed that 3 of the 4 existing links under "Tickets needing
some work" are incorrect. They indicate that they are accepted
tickets, but only one of these links is filtering by triage stage.

> Work on (2) began at some point over 
> athttp://code.djangoproject.com/wiki/TicketChangeHelp. It's pretty rough
> still, but once it gets better I'd love to link sections from the
> ticket page and/or move it into the docs directly.

This is good, but it looks like mostly copy and paste from the
"Contributing to Django" docs (duplication of data, also the Wiki is
not authoritative, especially when not authored by a core committer)
for the descriptions of each triage stage and patch flag, with the
addition of next steps. I'm also not sure if this Wiki page is linked
to from anywhere (e.g. actual ticket pages)?

Perhaps we'd be better off simply adding the next steps information to
the "Contributing to Django" docs directly, and to also update the
diagram there to include the various patch flags (has patch, needs
docs, needs tests, patch needs improvement) between the Accepted and
Ready for checkin triage stages?

At the moment all the docs / diagram say is that tickets which have a
patch with docs and tests are moved to ready for checkin, but a quick
search on trac revealed 358 tickets with this combination of triage
stage and patch flags. These tickets should all be either "patch needs
improvement" with feedback, or ready for checkin, right?

Perhaps we need another triage stage for these tickets, "Needs final
review" or something?

This is where I personally feel one of the big problems is with
perceived responsiveness of the core committers. These tickets are
sitting there as Accepted, and don't appear to require any further
action from the community or patch author (don't need a patch, docs,
tests or improved patch), and they are ignored by people wanting to
contribute and core committers alike.

> Ideally, I'd like each ticket page to have a "what's next?" box on it
> indicating where the ticket is and how to move it along. Technically
> this isn't hard -- we've got plenty of folks in our community who
> could write a Trac plugin -- but getting the language right is
> important. If we get that worked out to some extent I'll figure out
> what the next steps are technically.

Perhaps another boolean flag that simply says if the next step lies
with the patch author / community, or with the core committers?
Similar to regular support tickets which might say "waiting on
customer" or "waiting on support staff". This could help avoid
situations where the patch author thinks they've done everything they
need to do, satisfied all the requirements (docs, tests, code style),
and yet their ticket is never pushed up to Ready for checkin, which
only makes them less likely to contribute again in future and more
likely to simply fix or work around their problem locally instead and
save writing docs and tests for Django that go unused.

> For (3), well, feel free to bring things up here or on IRC, I s'pose.
> I'd really appreciate it if we didn't get 'em all brought up at once,
> of course. But yes, if there are tickets marked DDN that you feel are
> critical, please feel free to ask for a yay/nay.

Are only critical tickets eligible for a vote? I think there are a lot
of non-critical (even trivial) tickets that are stuck in DDN hell :)
Maybe we could organise regular DDN and Ready for checkin triage
sprints?

Also as DDN tickets must be decided by core committers, the community
and patch authors can't really do anything with these tickets and they
are stick in limbo. Perhaps there needs to be another flag to indicate
whether the core committers need more information or a proposal for
why a ticket should be included before they can make their decision,
or if it has been added to the queue of tickets to be decided on
(maybe with a position in the queue), and make sure that the queue is
worked through in order, and either sent back to the patch author for
more information or put to a vote if a decision can't be made?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email 

Re: ANN: Improving our decision-making and committer process

[Tai Lee]

On Sep 30, 7:22 pm, Russell Keith-Magee 
wrote:

> What is also needed is a whole lot more people volunteering. Any
> suggestions on how to get more people doing the entirely unglamorous,
> but completely necessary work will be gratefully accepted.

I'd like to suggest (1) easy to find and use pre-defined searches to
find tickets at each stage of triage, (2) a clearer indication of the
next steps and the person responsible for it whenever a ticket is
reviewed, and (3) tickets that have been sitting on DDN for a while be
put to a vote.

The "stage" is not just unreviewed, DDN, accepted, RFC, someday/maybe,
and fixed on a branch. The "accepted" stage itself is broken down into
has patch, needs docs, needs tests, patch needs improvement.

As an example, http://code.djangoproject.com/ticket/13291 and
http://code.djangoproject.com/ticket/12398 both have a patch with docs
and tests, and were marked as "accepted" by a core committer (not that
it matters who reviewed them), without providing any actual feedback
and without stating that they need a better patch. These tickets are
fairly trivial and have been in limbo for 6-8 months.

As the original reporter and developer, I assumed that since these
tickets were reviewed by a core committer and "accepted" without any
feedback about needing a better patch, that they would be committed
shortly, or at least be moved to RFC shortly. These tickets are ideal
candidates for anyone doing triage (not only core committers) to
review and move to RFC, but they've been forgotten about by the
reviewer, assumed no further action was required by the developer, and
ignored in the sea of "accepted" tickets by everyone else.

So for (1), I'd like to see the how to contribute documentation asking
for people to triage specific sets of tickets with specific goals. For
example, to review tickets that are almost RFC and just waiting on
review (with a link to tickets that are accepted, with a patch that
has docs and tests and doesn't need improvement), or to review tickets
that just need docs or test (with links to appropriate searches), etc.

And for (2), I'd like to see tickets remember who reviewed them (who
accepted it, or who decided it needed a better patch, etc.) and for
the system to ping both the original reporter (or recently active
developer) and the reviewer after a period of inactivity. This should
hopefully catch tickets which are initially "accepted" and then
forgotten about.

Lastly I also have several tickets that have been DDN for 1-2 years,
so for (3), the reporter and recently active developers could be
notified that the DDN ticket will be put to a vote in say 1 or 2 weeks
time and allow them time to prepare any arguments or improved patch
that may sway the decision, or try to reach a consensus on django-
developers prior to the vote.

I've been working with Django for several years now and I have tried
to contribute useful tickets with patches including docs and tests
when possible, and it has at times been extremely discouraging to see
this effort go unnoticed for literally years without review (beyond a
passing glance to "accept" a ticket, in an attempt to reduce the
"unreviewed" backlog).

I've also felt that at busy times, some tickets that had merit but
didn't have an immediately apparent and elegant solution were rejected
simply save time leading up to a release.

I was pleased to see the recent announcement from Jacob about the
changes to the decision making process, but I think we also need to
improve the triage process so that tickets requiring action (including
decisions) can be directed to or found by the people responsible for
making those next steps happen.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: HttpResponse: freeze API, read_once

[Tai Lee]

After re-reading the rest of those previous discussions, I see at
least one thing wrong with the "disabled_middleware" attribute
approach. 3rd party apps won't know which middleware is installed that
might break their responses, and end-users won't have a mechanism to
disable middleware for responses in 3rd party apps (just their own
apps).

The last post (by me) on the previous discussion thread that was
linked has a suggestion to work around this by adding a
CONDITIONAL_MIDDLEWARE_CLASSES setting, where end-users can map
response middleware that should or shouldn't execute for a particular
HttpResponse class.

This should be completely backwards compatible, will not require
duplicate, repetitive or error prone code in each middleware function
to interrogate the capabilities (or frozen headers, etc) for each
response.

Perhaps this could even also map response middleware functions to view
functions in addition to HttpResponse classes?

Regarding the freeze API discussed above, how would one make use of
this in a view that generates and returns a response with
direct_to_template (or similar)? Would users simply be required to
construct and alter their own HttpResponse manually and return that,
rather than using a helper function like direct_to_template?

Cheers.
Tai.


On Sep 15, 2:25 pm, Tai Lee  wrote:
> I'm going to try and preempt a possible objection that has been raised
> in previous discussions on this subject.
>
> Won't this change require a lot of repetitive logic to be shifted into
> middleware functions? All middleware authors will now need to inspect
> the response and find out if they can make the changes they want to
> make.
>
> If an old middleware hasn't been updated and a developer tries to
> freeze a header that the old middleware needs to alter, an exception
> will be raised. The only option for the developer in this case would
> be to alter the old 3rd party middleware themselves?
>
> As an alternative (that has been raised in one of the previous
> discussions), what's wrong with the "disabled_middleware" (or whatever
> name is deemed appropriate) attribute on HttpResponse? This would make
> it easy for developers to disable or skip certain response middleware
> on a per-response basis, or create their own HttpResponse subclasses
> that skip certain incompatible middleware, and would not require any
> changes to existing middleware functions nor require middleware
> authors to check in their functions if it is allowed to make any
> changes. The code that determines whether or not changes (middleware)
> are applied to responses would be in one place, the Django code that
> applies middleware, rather than the 3rd party middleware code itself.
>
> Cheers.
> Tai.
>
> On Sep 15, 1:33 pm, Forest Bond  wrote:
>
>
>
> > Hi,
>
> > On Tue, Sep 14, 2010 at 01:22:37AM +0100, Luke Plant wrote:
> > > It would be extremely useful to me if you could go through the use cases
> > > linked on the previous thread and see if your solution addressed them
> > > all.
>
> > Sorry, I'll have to get this out tomorrow.  I ended up being a little short 
> > on
> > time today.
>
> > For what it's worth, looking at the use cases again has been instructive 
> > and I
> > think the proposed API can be simplified some and still be able to address 
> > each
> > of them.
>
> > Thanks,
> > Forest
> > --
> > Forest Bondhttp://www.alittletooquiet.nethttp://www.pytagsfs.org
>
> >  signature.asc
> > < 1KViewDownload

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: HttpResponse: freeze API, read_once

[Tai Lee]

I'm going to try and preempt a possible objection that has been raised
in previous discussions on this subject.

Won't this change require a lot of repetitive logic to be shifted into
middleware functions? All middleware authors will now need to inspect
the response and find out if they can make the changes they want to
make.

If an old middleware hasn't been updated and a developer tries to
freeze a header that the old middleware needs to alter, an exception
will be raised. The only option for the developer in this case would
be to alter the old 3rd party middleware themselves?

As an alternative (that has been raised in one of the previous
discussions), what's wrong with the "disabled_middleware" (or whatever
name is deemed appropriate) attribute on HttpResponse? This would make
it easy for developers to disable or skip certain response middleware
on a per-response basis, or create their own HttpResponse subclasses
that skip certain incompatible middleware, and would not require any
changes to existing middleware functions nor require middleware
authors to check in their functions if it is allowed to make any
changes. The code that determines whether or not changes (middleware)
are applied to responses would be in one place, the Django code that
applies middleware, rather than the 3rd party middleware code itself.

Cheers.
Tai.


On Sep 15, 1:33 pm, Forest Bond  wrote:
> Hi,
>
> On Tue, Sep 14, 2010 at 01:22:37AM +0100, Luke Plant wrote:
> > It would be extremely useful to me if you could go through the use cases
> > linked on the previous thread and see if your solution addressed them
> > all.
>
> Sorry, I'll have to get this out tomorrow.  I ended up being a little short on
> time today.
>
> For what it's worth, looking at the use cases again has been instructive and I
> think the proposed API can be simplified some and still be able to address 
> each
> of them.
>
> Thanks,
> Forest
> --
> Forest Bondhttp://www.alittletooquiet.nethttp://www.pytagsfs.org
>
>  signature.asc
> < 1KViewDownload

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Revised form rendering

[Tai Lee]

On Jul 14, 11:15 pm, Russell Keith-Magee 
wrote:
>
> I'm not wildly enthusiastic about this. You've posted this snippet (or
> something like it) a couple of times, and every time my immediate
> reaction has been "You're in a twisty maze of endtemplatedir tags, all
> alike" :-)
>
> I can see what you're trying to do in terms of setting default
> directories; but the syntax you are proposing:
>
>  * makes the representation of setting a default over a wide range quite 
> elegant
>  * makes the representation for overriding a default on a single
> field/form quite verbose.
>
> To me, this seems like priorities the wrong way around. Overriding a
> single default should be the simple case, not the verbose case.
> Consider the template where you want to override the default widget on
> every field on a form - your entire form will potentially be a nest of
> {% templatedir %}{% endtemplatedir %} tags.

Thanks for the feedback, Russ. I'm not set on any particular syntax, I
just wanted to demonstrate the idea of augmenting the path that
template loaders use as a way of customising presentation for a group
of pages, a single page, a group of elements on a page, or a single
element on a page.

I admit that several instances of {% templatedirs "mypath" %}
{{ form.field %}{% endblock %} or nested calls to {% templatedirs %}{%
endtemplatedirs %} in a template could get messy. I'm not sure how
else it could be done though, without tying it to a positional
argument to {% form %} and {% field %} tags, in which case it would
need to be added to every template tag you might want to override
templates for in this way. It might also be more difficult to override
when you have pluggable apps that include templates which already use
the {% form %} and {% field %} templates.

One thing to keep in mind is that you could actually wrap your entire
form (or page, or master template) in a {% templatedirs %} block tag
and then simply add/remove templates from your file system, without
having to make any other changes to existing or pluggable templates in
order to explicitly replace the templates for specific elements.

The {% form %} and {% field %} template tags could even be updated to
look for multiple templates, based on their name as well as their
type. I'm thinking of something like the admin here, where you can
define templates at different locations in order to apply them with
more or less granularity.

For example:

In your projects master template you wrap the whole thing in {%
templatedirs "myforms" %}{% endblock %}. Then, in any of the templates
derived from your master template (but not templates that belong to
pluggable apps that extend their own master template), the template
engine will look there first when loading templates. Effectively
adding a folder to the TEMPLATE_DIRS setting, but only for templates
that extend your master template, without affecting other pluggable
apps that you might be using.

Then, if you tried to render a form that had been assigned a name or
id of "contact", it would try to load:

myforms/django/forms/contact.html
myforms/django/forms/default.html
django/forms/contact.html
django/forms/default.html

And if you tried to render a field on the contact form that has a name
of "email", it would try to load:

myforms/django/formfields/contact/email.html
myforms/django/formfields/contact/default.html
myforms/django/formfields/email.html
myforms/django/formfields/default.html
django/formfields/contact/email.html
django/formfields/contact/default.html
django/formfields/email.html
django/formfields/default.html

Perhaps the block tag could either accept multiple paths to search, to
avoid ugly nesting. Or alternatively it could require two arguments,
the first being the path to search, and the second a filter so that
search paths are not searched unecessarily. E.g. if you knew you only
wanted to load temlates from "myforms" instead of "django/forms", you
could do {% templatedirs "myforms" "django/forms" %}...{%
endtemplatedirs %}? Or, perhaps it doesn't need to be a block level
tag. Perhaps it just just applies to the whole template, like {% load
%}?

Just food for thought.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Revised form rendering

[Tai Lee]

Hi Carl,

On Jul 14, 11:44 am, Carl Meyer  wrote:
>
> I totally agree with this. I think very similar results can be
> achieved in the form-rendering system just with an optional parameter
> to a "form/render_form" tag that specifies a "base path" for finding
> template widgets for this form render, as I mentioned above. I think
> it's better to keep the form-rendering proposal specific to forms, and
> evaluate {% templatedirs %} separately as a feature proposal on its
> own merits.

To me this sounds like basically the same thing, but limited to the {%
form %} and possibly {% field %} tags as a positional argument,
instead of being a generic tag that can be used for multiple forms and
form fields, as well as other (non-form) purposes.

I don't want to hijack this thread and take the focus away from
enhanced form rendering, but if such a feature is going to be tied to
forms specifically I'd like to see that it can be applied to multiple
{% form %} and {% field %} tags, without having to define a new "base
path" for each individual form or field that needs to be customised,
which is what led me to the use of a block tag instead of positional
arguments, and it just so happens that it could be useful outside of
form rendering so I tried to come up with a generic name :)

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Revised form rendering

[Tai Lee]

Hi Russ and Carl,

On Jul 14, 5:55 am, Carl Meyer  wrote:
> Hi Russ,
>
> On Jul 13, 12:11 pm, Russell Keith-Magee 
> wrote:
>
> > My manifestation of this problem is slightly different -- it's the
> > issue of how to ship a widget library that can be used anywhere. I
> > suppose one argument here is that you just need to advertise your
> > chrome with doctype support notes.
>
> > Alternatively, since this is a template language, you could always
> > introduce an {% if %} block and render conditionally on doctype...
>
> Exactly. I can imagine a number of different ways a reusable widget
> library might approach this problem: simply advertise support for a
> certain doctype (don't discount the value of supporting this low-
> barrier-to-entry option, even if it isn't ideally reusable), or use
> template conditionals, or provide separate widget templates for
> separate doctypes... I find it preferable to make the full power of
> the template language available, and then let template authors use the
> tools they know to iterate towards best practices in this area, rather
> than hardcoding into Django particular ideas of which doctypes are
> relevant and what changes have to be made in templates to support
> them. It's not just a matter of closing slashes, also which attributes
> are valid (of which there may be more added later that we don't even
> know about now)... I think hardcoding current assumptions about
> doctypes is just asking for maintenance headaches. I'd much rather
> leave that concern where it belongs, fully in the hands of template
> authors.

I wouldn't like to see widget libraries being packaged up with
templates including conditional templates based on doctype. This seems
messy, not friendly to override in your own templates, and effectively
combining what should be multiple separate templates into one.

Wouldn't a mechanism that allows the path from which templates are
loaded to be changed benefit both widget libarary authors and the
designers who are implementing those widget libraries? A widget
library author would include and advertise templates for each doctype
they support, e.g. "html4/mywidgetlibrary/forms/datetime.html" and
"xhtml1/mywidgetlibrary/forms/datetime.html", then the designer would
use:

{% templatedirs "html4/mywidgetlibrary" %}
{{ myform }}
{% endtemplatedirs %}

or:


{{ form.myregulartextfield %}
...
{% templatedirs "html4/mywidgetlibrary" %}
{{ myform.mydatetimefield }}
{% endtemplatedirs %}


to load the appropriate template from the widget library templates.
This gives a lot of flexibility and allows widget library authors to
support multiple doctypes or flavours (not just specific to doctypes),
and designers to pick and choose (or even override at the project
level) the widget templates according to their doctype or preferred
flavour, without Django needing any specific machinery for doctypes or
designers needing to edit any Python code.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Revised form rendering

[Tai Lee]

I really like the idea of using Django templates instead of generating
markup in Python, which should enable designers (who don't know and
don't want to know Python) to customise markup in almost any way they
like. We just need to add a few hooks that will make it easier to
override specific templates (per form, per page, per app, rather than
per project).

To this end, I'd like to propose a new stackable block tag (let's call
it `templatedirs` and `endtemplatedirs` for now) that designers can
use to signal that any templates loaded within the block tag should be
tried from the specified folder first.

For example, if {{ myform }} renders "forms/form.html" by default, and
{{ myform.mydatetimefield }} renders "forms/datetime.html" by default,
then:

{% templatedirs "html401strict" %}
{{ form }}
{% endtemplatedirs %}

would try to load "html401strict/forms/form.html" before trying "forms/
form.html", and:

{% templatedirs "html401strict" %}
{% templatedirs "fancyforms" %}
{{ myform.mydatetimefield }
{% endtemplatedirs %}
{% endtemplatedirs %}

would try to load "html401strict/fancyforms/forms/datetime.html",
"fancyforms/forms/datetime.html" and "html401strict/forms/
datetime.html" before trying "forms.datetime.html".

This would allow flexible overriding of templates from inside
templates themselves, with whatever granularity is required by the
designer, without the designer needing to learn Python or register
template tags.

Templates for a specific doctype could be applied to an entire site by
adding {% templatedirs "html401strict" %} to the base template, and
adding the appropriate templates to the "html401strict" folder in your
app or project templates folder. Custom templates could be applied to
a specific page or a specific form or a specific form element the same
way.

I'm sure this could be useful outside of the forms framework, too. For
example, {% templatedirs request.user.profile.client.slug %}...{%
endtemplatedirs %} to render customised templates for each client (or
category of user, e.g. staff, admin, client, etc.)

Cheers.
Tai.


On Jul 13, 8:25 am, Idan Gazit  wrote:
> Russ/Carl:
>
> Finally got a chance to catch up on the thread, and found Carl penned
> a (lovely, much more detailed) version of what I had in mind.
>
> In the end, forms is a repository of unusually common fail because
> designers must figure out Python and a lot of how django works in
> order to customize forms. One of django's strongest assets is the
> separation-of-roles philosophy, and this is a common use-case where
> that philosophy is not implemented.
>
> As a general design goal, improving forms should be about moving
> Django towards a state where designers do not need to write python in
> order to customize how forms render.
>
> A big +1 on the template approach.
>
> -I
>
> On Jul 12, 10:24 pm, Carl Meyer  wrote:
>
>
>
> > Hi Russ,
>
> > First of all, thanks very much for this proposal! Form rendering has
> > been a major pain point for us (thus the existence of
> > django-form-utils), and improving it is tops on my 1.3 wishlist. I
> > will also be at DjangoCon and eager to sprint in this area.
>
> > Django has a really good template system. With template inheritance,
> > includes, and overrides, it allows the designer I work with a
> > remarkable amount of flexibility in producing exactly the HTML he
> > wants, without repeating himself. He's not a programmer (and thus
> > doesn't follow django-developers), but he works full-time with the
> > Django template system and loves it. His (frequent) complaints to me
> > about Django form rendering are always the same: "Why is this markup
> > generated deep inside Python code, and why can't I override a template
> > to fix it? I thought we'd moved on from PHP!"
>
> > I think it's possible to solve the problems you're aiming at here
> > without so much new Python machinery, just by delegating more to
> > templates (as Eric and Danny have already mentioned briefly). HTML is
> > the native language of web designers, and I'm firmly convinced that
> > the less we hide their HTML behind Django-specific abstractions, the
> > more usable the system will be.
>
> > I'll first outline in brief what I think a fully template-oriented
> > approach might look like, address some possible concerns with it, and
> > follow that with some specific comments on your proposal.
>
> > A fully template-based approach to form rendering
> > =
>
> > You have a form object in your template. You render it using a simple
> > "render_form" filter:
>
> > {{ form|render_form }}
>
> > "render_form" just renders a default form rendering template (at some
> > sensible template path, the bikeshed could be painted
> > "django/forms/default.html" or some such), with the form object itself
> > passed into the context.
>
> > This template just does the usual iterating over form fields, and the
> > default one provided by Django could mimic form.as_

Re: Proposal: Revised form rendering

[Tai Lee]

Regarding the DOCTYPE problem, I don't think it's appropriate to set
the DOCTYPE as a setting for an entire project. As many have pointed
out, each bundled app could define a different DOCTYPE in their base
templates.

It doesn't make sense to apply a DOCTYPE setting to the context of a
form either, as the DOCTYPE is not set at the form level but the page
level, and all template tags that render HTML (not only forms) should
render markup that is appropriate for the DOCTYPE in use.

What about adding a {% doctype %} template tag for use in base
templates which will not only render the appropriate DOCTYPE, but
store it in the context? Then templates that extend from the base
template will have access to this context variable and know whether or
not to include a trailing forward slash for self closing elements, or
whether or not `email` fields should be rendered as `text` or `email`
type?

Regarding the application of settings and substitution of widgets
inside templates, I'd like to see an easy way for designers to create
bundles of presentational logic (templates, stylesheets, images) which
they can pass in or refer to as arguments to {% form %} or {%
formfield %} template tags. This could take the form of {% form myform
myformpresentationbundle %} or {% formblock %}{% formbundle
myformpresentationbundle %}{% fieldbundle mydatefield mycalendarbundle
%}{{ myform }}{% endformblock %} or {% formfield myform.mydatefield
mycalendarbundle %}.

I'm not sure how these bundles would be defined or made available to
and loaded from the templates. I don't think having to create template
tags and load them (especially if they override built ins) is going to
be user friendly for designers. Perhaps the {% form %} and {%
formfield %} template tags could just be inclusion template tags, but
instead of having the template that they include defined with the
template tag, they load it from the templates folder (or another
folder) according to the arguments?

Cheers.
Tai.


On Jul 12, 12:31 pm, André Eriksson  wrote:
> Good proposal overall. One thought I have in order to try and combat
> the massive parameter list of {% form %} is to optionally add an
> ending tag, as well as sub-tags:
>
> {% form myform %}
>     {% using birthday=calendar %}
>     {% renderer "as_ul" %}
>     {% autocomplete "name_autocomplete" %}
>     {% doctype xhtml1 %}
> {% endform %}
>
> I see this as having several advantages:
>
> 1) It introduces a clearer way of laying out settings.
> 2) It leverages the template engine for specifying defaults in a
> simple but ingenious way:
> {% form myform %}
>     {% include "form_settings.django" %}
> {% endform %}
> -- form_settings.django:
>     {% doctype xhtml1 %}
>     {% replace_widget datetime=calendar %}
>
> And because of the flexibility of the template engine we can easily
> specify either section-, site- or project-specific form settings, with
> the use of block tags and overriding them. This can be accomplished in
> a few ways:
>
> 1) Keeping the current {{ myform }} method for displaying forms
> without settings attached.
> 2) Converting to always using {% form myform %}, where the {% form %}
> tag parses until it either finds a matching {% endform %} tag or a new
> {% form %} tag; a new {% form %} tag means that there are no settings
> attached.
> 3) Adding a parameter to {% form %} for settings: {% form myform
> detailed %} {# settings #} {% endform %}
> 4) Adding two template tags, one for simple form rendering, another
> for detailed form rendering. Naming them is left as an exercise to the
> reader.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing middleware/view decorator handling of content iterators

[Tai Lee]

I didn't think about the author of a `HttpResponse` subclass needing
to know all potential middleware that will not work with that
particular response class. That is definitely a problem. However, I
don't think that exposing a capabilities based API on `HttpResponse`
is the answer.

If we do that, then ALL middleware that already exists or is yet to be
written will need to interrogate every response and conditionally
execute based on the capabilities that the response has (and does not
have), which will be very repetitive and prone to error and still
likely result in 3rd party middleware not working as intended with
some responses.

>From what I understand of previous discussions [1], this is what
Malcolm was strongly opposed to. Middleware should not need to examine
the response to determine if they can or cannot execute. E.g. if it is
possible for a response to have generator content that cannot be
consumed, then ALL middleware must check for this possibility.

Perhaps a better alternative is to provide a way to map middleware to
be skipped (or executed) for a specific `HttpResponse` class. I
hesitate to recommend Yet Another Setting, but this would give
developers control over which middleware execute (or do not execute)
for which responses, even if both the middleware and `HttpResponse`
class are 3rd party code.

MIDDLEWARE_CLASSES = (
'django.middleware.common',
)

CONDITIONAL_MIDDLEWARE_CLASSES = {
'exclude': {
'django.http.HttpResponseStreaming':
['django.middleware.common', 'otherapp.othermiddleware', ...],
},
'include': {
'myapp.MyHttpResponse': ['myapp.mymiddleware', ...],
},
}

Middleware specified in `MIDDLEWARE_CLASSES` would run all the time,
except when the middleware function is linked to the `HttpResponse`
class in `CONDITIONAL_MIDDLEWARE_CLASSES['exclude']`. Middleware that
is NOT specified in `MIDDLEWARE_CLASSES` would run only when the
middleware function is linked to the `HttpResponse` class in
`CONDITIONAL_MIDDLEWARE_CLASSES['include']`.

Cheers.
Tai.

[1] 
http://groups.google.com/group/django-developers/browse_thread/thread/3e0d78b98498c159/f08036fb1e7fe6b9

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing middleware/view decorator handling of content iterators

[Tai Lee]

On Feb 21, 7:46 am, Forest Bond  wrote:
> Okay, I think "disabled_middleware" is a bad name because it doesn't actually
> imply that all middleware should be disabled.  Anyway, it seems like 
> middleware
> could just check if isinstance(response, StreamingHttpResponse).

I'm not sure I follow your reasoning here. If you just don't like the
name, that's a bike shed argument. I'm proposing that
`disabled_middleware` be a list or tuple of middleware that should NOT
be called on the response, because the author of the `HttpResponse`
subclass that defined `disabled_middleware` knows that those
middleware are not needed or will not work as intended.

The point that I gleaned from earlier discussions is that we cannot
fix this problem in middleware. We cannot have middleware check `if
isinstance(response, StreamingHttpResponse)` and conditionally change
its behaviour because we would force ALL Django and 3rd party
middleware in existence to check for the possibility that a response
has generator or string content and change their behaviour
accordingly.

I believe the answer is to define explicit `HttpResponse` subclasses
and provide a mechanism by which each subclass can explicitly disable
certain middleware. That way, all existing Django and 3rd party
middleware do not need to change or know or care about what kind of
content the response has. Then the developer can simply disable any
middleware that is causing a problem with a particular response,
regardless if it is a problem with streaming, or they don't want an
ETag, or they don't want a response compressed, etc.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing middleware/view decorator handling of content iterators

[Tai Lee]

See also #7581, which has a patch that I have been using in my local
Django branch for over a year without issue. My use case is that I
need to generate large amounts of data in CSV format and stream the
response to avoid timeouts. I wouldn't mind `HttpResponse` consuming
`content` immediately if we had a `Http StreamingResponse` or similar
subclass, but the issue I found after discussion with Malcolm and Ivan
Sagalaev and ccahoon who worked on the http-wsgi-improvements branch
relates to middleware needing to be aware of when they should or
should not consume the content in a response, when they should replace
the content or generator in a response, or when they should be skipped
entirely.

Here's my suggestion taken from a recent comment in #7581:

I suggest that we simply add an attribute
`HttpResponse.disabled_middleware` (list or tuple). Any installed
middleware specified there should be bypassed when applying response
middleware. Django could then ship with an `HttpStreamingResponse`
class that disables any included middleware that is known not to work
with generator content, and this could easily be subclassed and
updated by users who need to disable any 3rd party middleware to
create a streaming response.

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Design Czar

[Tai Lee]

It feels like a catch 22 situation. We need someone to champion and
shepherd design changes into trunk, but we can't assign such a role to
somebody who hasn't met the criteria that each core committer must
meet. To quote the Django documentation, any design czar or core
designer should have "a long history of contributions to Django's
codebase, a solid track record of being polite and helpful on the
mailing lists, and a proven desire to dedicate serious time to
Django's development."

At the same time, everybody agrees that design by committee simply
does not work.

But what's stopping people from re-designing the admin outside of
django as a standalone app (e.g. grapelli), at least to begin with,
and if/when they run into issues that can only be solved by improving
or changing Django internals, raise proposals to make those changes?

If/when such an application has gained enough momentum and has
demonstrated that the people involved are committed to it, invite them
to roll those changes into Django trunk, or invite them to suggest
changes that can be made to Django internals that will further aid
design changes, or offer them a design czar position independent of
their standalone app, so that they might encourage others to
contribute design changes to Django?

I agree that the design czar need not necessarily be a rock star
designer who needs to re-design everything himself (or herself). But
he or she must have a firm grasp on interface design and be familiar
with and to the Django community. I've not noticed many people who fit
the bill on the mailing lists, but from the recent discussions it
seems to me that Jeff Croft has the interface design experience, is
active in the design community and counts as his peers several other
great designers, is familiar with and to the Django community, and has
been passionate in exposing these issues and trying to improve Django
and make designers more welcome and able to contribute.

He'd get my vote as a champion of these issues, and I think he'd do a
good job of reviewing design contributions, suggesting changes to
Django internals that will help increase design contributions,
encouraging good interface design rather than bike shed aesthetic
contributions, without needing to do the design work himself (which he
has indicated he might not be inclined to do).

Cheers.
Tai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Tai Lee]

On Jan 9, 12:36 am, Honza Král  wrote:
> On Fri, Jan 8, 2010 at 11:59 AM, koenb  wrote:
>
> > On 8 jan, 10:03, James Bennett  wrote:
>
> >> Suppose I have a ModelForm and call save(commit=False) to get the
> >> instance so I can do some more work on it. I'm basically saying to
> >> Django "I don't think this object is ready to be saved yet, but I need
> >> the object so I can do stuff to it". If Django goes ahead and does
> >> implicit model validation there and raises an exception, it's just
> >> telling me what I already knew: the object's not ready to be saved.
> >> But now I get to do extra work to catch the exception, and that's bad
> >> and wrong.
>
> >> Calling ModelForm.save(commit=False) should simply disable model
> >> validation, period. If I want to validate the instance before saving
> >> it, I'll validate the instance before saving it, but commit=False is
> >> as clear a way as we have of saying "I'm not going to save this yet"
> >> and model validation should respect that.

I agree with this completely. Calling ModelForm.save(commit=False)
should disable model validation, period. We're explicitly telling
Django not to save the model because it's not ready to be saved,
therefore no model validation needs to be performed.

> > The problem is that in the idiom the is_valid() call on the modelform
> > tries to full_validate the instance, it is not in the save
> > (commit=False) call.
>
> Yes

This is what I think should be changed. ModelForm.is_valid() should
only validate the form. Model validation errors should not be (and I
believe are not currently) returned to the user as form errors,
because they're not form errors and the user can't do anything about
them. They're errors for the developer at the point when he or she is
ready to save a model. Model validation should be moved out of
ModelForm.is_valid() and into ModelForm.save(), but only if
`commit=True`. Otherwise, model validation should be performed when
explicitly requested.

Cheers.
Tai.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Tai Lee]

It makes sense to me that the developer should first check that their
form is valid and second check that their model object is valid when
calling ModelForm.save(commit=False). This could be done by having the
developer check the result of model.full_validate() before calling
model.save(), or by having model objects returned by ModelForm.save
(commit=False) automatically trigger model validation when model.save
() is called, even if model.save() does not normally trigger model
validation?

Cheers.
Tai.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Reversing URL for a view with namespaces

[Tai Lee]

Speaking of the ways in which a namespace can be specified or
provided, #11642 might be relevant here.

http://code.djangoproject.com/ticket/11642

It feels a little cumbersome to me to require users to either manually
specify a namespace or import a different object (than just the usual
urls module) which includes an embedded namespace. The attached patch
allows app authors to specify an "app_name" in their urls module which
will be used as the default app_name and namespace if none are
specified in the users urls module. The user can still specify their
own namespace or tuple with embedded namespace.

Cheers.
Tai.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Session/cookie based messages (#4604)

[Tai Lee]

On Oct 14, 10:45 am, Tobias McNulty  wrote:
>
> Okay, let's pick something other than 'messages' and stick to it.  I'm
> fine with "notifications", "notices", and maybe even "notes", but the
> last one is a little ambiguous.  Other ideas?

My personal bike shed is named "alerts", but other than that I have
nothing to add that hasn't already been mentioned. I've been using
django-notify for a while now, and think it's great. Can't wait to see
something like it make it's way into core.

Cheers.
Tai.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: HttpResponseStreaming, ticket #7581, and [soc2009/http-wsgi-improvements] Final(?) status update

[Tai Lee]

On Aug 15, 3:19 am, ccahoon  wrote:
> This is the case, yes, if the HTTP and GZip middleware are not used.
> We had to modify both of those to make sure they didn't consume the
> content generator to find the Content-Length. I imagine, IF we wanted
> to, setting the Content-Length header could be done in the base
> handler. One thing I am not certain about (I am about to head out the
> door) is if it isn't set BECAUSE the middleware can modify it. If so,
> then it would certainly have to be set after the response middleware
> were applied.

We could add a method to HttoResponse.response_fixes that sets the
Content-Length header if not already set, but remove that fix from
HttpResponseStreaming. The response fixes are applied after response
middleware, so this should be a good place to ensure that all
HttpResponse objects have a Content-Length applied while respecting
any existing headers set by middleware and any changes to the response
content made by middleware.

We could also then simplify ConditionalGetMiddleware and
GZipMiddleware, neither of which would need to set the Content-Length
anymore. I think that right now depending on the order of these two
middleware, the Content-Length might be calculated twice. It makes
sense to do this only once after all middleware that might alter the
response have been applied.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---