Re: patch for # 7270, select_related cannot follow reverse one-to-one relationship

[Russell Keith-Magee]

On Fri, Sep 18, 2009 at 2:49 AM, Ben Davis  wrote:
> I noticed this ticket is currently assigned to Malcolm; in the comments he
> says it's on his list of patches to work on.   It looks like the last thing
> he did was set the milestone to 1.2 back in March,  but haven't seen any
> comments on the ticket from him since then.   Is it considered fair game to
> re-assign this (or mark it as unassigned so that someone else can pick it
> up)?  I don't want to step on anyone's feet...

If the ticket had been claimed by a general member of community, the
answer would be "sure, go for it". However, in this case, the question
you need to answer here is a purely practical one - what benefit
exists in assigning the ticket to yourself? A casual look at the
ticket will reveal that you have been doing a lot of work on the
ticket over the last few months, and that is a lot more informative
than the official ticket assignment.

Malcolm is a core developer, and while he may be in a period of low
activity at the moment, the list of tickets he has assigned to himself
is a good list of stuff that has been tacitly approved by someone with
the commit bit.

You are free to keep adding patches without assigning the ticket to
yourself. You can keep discussing the ticket without assigning it to
yourself. Being assigned to Malcolm doesn't stop myself (or any other
core dev) from closing the ticket.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Graham Dumpleton]

On Sep 17, 6:25 pm, Simon Willison  wrote:
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
> log under the mod_python handler and environ['wsgi.errors'] under WSGI
> (meaningmod_wsgiwill write to the Apache error log as well).

It isn't necessarily practical to use environ['wsgi.errors'] as that
exists only for life of that request. Thus, anything done at time of
module imports or in background threads wouldn't have access to it.
You are better of just using sys.stderr.

Graham
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: CSRF proposal and patch ready for review

[Luke Plant]

Hi Simon,

OK, here is my response.  I hope this doesn't turn into a personal my-code-vs-
your-code match (especially as most of "my code" is really other people's code 
and ideas :-) but I want to make sure we do this right, as it potentially has 
big implications, so the following will be "sleeves rolled up" (getting into 
details) and "gloves off" when it comes to criticisms of the code itself :-)

Advantages of SafeForm
--

I can see the value of having contrib apps secured whatever the user has in 
their settings.py

Unfortunately, that is the only advantage of any significance that I can 
see.  And there are *much* easier ways to achieve that goal if that is our 
main priority (see the bottom of my e-mail for details).

There is the 'advantage' that Russell mentioned of not having to touch the 
tutorials.  This is a pretty dubious advantage IMO because it means that the 
tutorials will still lead to insecure code.  Far from 'raising the profile' of 
CSRF as Simon rightly suggested we should do, this is just keeping quiet about 
it.

Disadvantages of SafeForm
-

1) Big changes are required to use it.

The changes to use SafeForm are much more intrusive than using the template 
tag.  It requires quite extensive changes to both templates and views:

 * different API and different flow control for SafeForm compared to Form (it
   may be better, but it's different, which means a big upgrade cost)
 * a decorator for views
 * a decorator for forms
 * quite commonly, significant changes to the template:
   * Often you have to explicitly add the csrf token field.  And you've 
 got to worry about whether you need to or not.
   * If you weren't displaying form.non_field_errors before (which can be
 common if you don't need it), you have to add it.
 * For the multiple forms in a  case, you've got even more changes
   to your view function and your template, and you have to learn about
   CsrfForm on top of everything else.

Using SafeForm is a *lot* more complex than the template tag method.  The 
basic instructions for day-to-day usage of the template tag method (i.e. not 
including change-once settings) go like this:

 If the target of a  is internal:
   * add {% load csrf %} to the template and {% csrf_token %} to the form
   * use RequestContext in the corresponding view.

And that's it.

2) More than one way to do CSRF protection.

Although you provide a low level mechanism for forms, it still means you have 
Two Ways To Do CSRF Protection, and they are quite different (from the users 
point of view).  Once you've added in the multiple forms in a  problem, 
you've now got Three Ways To Do CSRF Protection, all of which will be needed 
in quite common circumstances.

(Yes, the template tag also provides a lower level mechanism (using get_token 
directly), but developers will only ever need to use it if they not using 
templates to generate pages, which is very uncommon).

3) More than one way to do Forms

Switching between Form and SafeForm requires API changes for the form itself 
and significant flow control changes in the view. So we have "Two Ways To 
Do Forms".

Every user now faces a choice - use Form or SafeForm?  Form seems to be 
simpler; the API is possible to play with at the interactive prompt (see 
tests/regressiontests/forms.py, for example), unlike SafeForm which requires 
an HttpRequest object; and SafeForm sounds like something for the paranoid, 
ultra-secure types.  So you can bet that tutorials and code across the 
internet are going to use Form.

Even if people standardise on using SafeForm everywhere, you will have the 
case where they use Django forms to generate a form even when it is targeted 
externally.  Because they have switched to using SafeForm, that form will now 
include the CSRF token, which is leaked to external sites, which is a security 
vulnerability.  Further, it's really not obvious, because the place where the 
use of SafeForm is defined (in their forms.py or views.py somewhere) is a very 
long way from the information that defines whether they need it or not (the 
target attribute of the  in the template).

4) CSRF is baked in to view functions.

Every view function now has CSRF baked in, either 'hand rolled' or via 
SafeForm.  While this is the major advantage of SafeForm, it is also a major 
disadvantage, because:

 * If we need to change anything about the API in the future, we will require
   extensive changes to Django code and everyone else's.

 * Developers can't 'swap out' the implementation of CSRF protection.  With
   the template tag method, this is easy to do.  If someone has custom
   requirements, or because of their situation they can just do
   strict Referer checking or Origin checking or something, they can implement
   their own middleware and template tag, and switch *all* the Django apps
   over to their system simply by changing some settings.

 * Testing view functions becomes a lot harder.  

Re: Proposal for 1.2: built-in logging with django.core.log

[Simon Willison]

On Sep 17, 4:04 pm, Russell Keith-Magee 
wrote: 
> I've seen several attempts to wrap Java loggers in a "nicer"
> interface, and every one of them ended up hobbling some of the power
> features of the logger. There is also the issue of our wrapper playing
> nicely with the loggers already being used in the wild.

I should clarify - by "lightweight wrapper" I basically mean a pre-
configured log setup and a standard place to import the logger from -
and maybe a tiny bit of syntactic sugar if it will make the common
case more palatable. I'm mostly just interested in making the logging
module an encouraged technique within the Django world. It should
definitely play nicely with any already-existant logging code.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[apollo13]

On Sep 17, 10:25 am, Simon Willison  wrote:
> That's 0.0006 of a second overhead for a page logging 100 SQL
> statements. The performance overhead will go up if you attach a
> handler, but that's fine - the whole point of a framework like
> 'logging' is that you can log as much as you like but only act on
> messages above a certain logging level.
I woudln't worry about performance, lately I had a project doing
massive logging, which resulted in a performance loss, but replacing
my logging functions with simple lambda functions doing nothing (eg
replace django.core.log.warn with lambda *args, *kwargs: pass if
loglevel is error) gave me quite a performance boost.

Aide from that: +1 on the proposal.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Change list default sort modification proposal

[Joshua Russo]

I don't have any say in what gets accepted or not but I do believe that your
solution is a decent one. I'm continuing with mine because it addresses a
current need simply and can be (fairly) easily tested. If you can create the
regression tests for yours then I believe you might get more traction.
Then again there may be some hesitation to the level of complexity you are
introducing. There is always a strong inclination in the the project for
simplicity.


On Thu, Sep 17, 2009 at 5:42 PM, Ben Davis  wrote:

> I'm not sure if anyone's seen this,  but I've written a patch for
> multi-sort columns in the changelist ui here:
>
> http://code.djangoproject.com/ticket/11868
>
> Not sure if this is what you guys had in mind, but I'd be interested in
> your feedback.
>
>
> On Thu, Sep 17, 2009 at 12:57 PM, Joshua Russo wrote:
>
>>
>> Sorry for dragging this back out of the depths but I was reviewing my
>> code for regression testing and I noticed I had missed this comment
>> way back when.
>>
>> Where is this named sort option documented?
>>
>> It seems to me that the implementation that I have here would still be
>> useful even if you could associate multiple sort columns with a single
>> displayed column. I think that it's confusing to the developer not to
>> have the list display using the sort order that is defined within the
>> model.
>>
>>
>> On Aug 13, 2:49 pm, Rock  wrote:
>> > My suggestion for handling the UI for multi-column sorts is to allow
>> > the definition
>> > of named sorts in a manner similar to the way that the default sorting
>> > is defined.
>> > With this in place, additional multi column orders can be accomplished
>> > simply
>> > by using this name as a sort selection and applying the corresponding
>> > named sort.
>> >
>> > If this is an acceptable approach, I can flesh out the API for it.
>> >
>> > On Aug 11, 10:10 pm, Malcolm Tredinnick 
>> > wrote:
>> >
>> >
>> >
>> > > For those, like me, wondering what this proposal was about, it's
>> > > concerning changing sorting in the admin interface to initially use
>> the
>> > > full set of fields specified in Meta.ordering on the model.
>> >
>> > > What I can't work out yet, due to difficulty in reviewing the patch,
>> > > mentioned below, is whether it allow sorting by more than one column
>> via
>> > > the column-clicking method. That would almost certainly seem to be a
>> > > request that is going to arrive 10 seconds after a feature like this
>> is
>> > > committed, although the difficulty has always been with how the UI
>> works
>> > > for that situation.
>>
>>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Question on ticket triage process

[Jacob Kaplan-Moss]

On Thu, Sep 17, 2009 at 1:38 PM, Ben Davis  wrote:
> So by documentation,  do you mean updating the official django docs,  or
> better documentation in the code itself?

Yes :)

As a general rule:

* Self-documenting code is fine sans comments, but anything
non-obvious needs explanation in the code.
* All public APIs must be documented in the official docs. No exceptions.

Jacob

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: patch for # 7270, select_related cannot follow reverse one-to-one relationship

[Ben Davis]

I noticed this ticket is currently assigned to Malcolm; in the comments he
says it's on his list of patches to work on.   It looks like the last thing
he did was set the milestone to 1.2 back in March,  but haven't seen any
comments on the ticket from him since then.   Is it considered fair game to
re-assign this (or mark it as unassigned so that someone else can pick it
up)?  I don't want to step on anyone's feet...



On Fri, Sep 4, 2009 at 7:09 AM, Russell Keith-Magee
wrote:

>
> On Fri, Sep 4, 2009 at 6:30 AM, Ben Davis wrote:
> > Hi I've submitted a patch for this bug that I'd like to have considered
> for
> > some trunk action...  I can't take the credit for all the work on this,
> but
> > it seems I'm taking the initiative to get it resolved.
>
> ... which is exactly what is needed. Having patches is great, but what
> we really need is people willing to engage the core and the
> development community over the long run to see fixes like this one
> home. Persistence will be required :-)
>
> > I really only had one question about a particular change that was needed
> in
> > order for the  .values() query to work with reverse one-to-one  (e.g.,
> > User.objects.values('username', 'userprofile__zip') ) .   It looks like
> > mtredinnick wrote the original code for this that was modified.  The
> change
> > was fairly simpe,   In the setup_joins function in
> db/models/sql/query.py:
> >
> > ---
> >
> >  raise FieldError("Cannot resolve keyword %r into
> field.
> > "
> >  "Choices are: %s" % (name, ",
> ".join(names)))
> >
> >
> > -if not allow_many and (m2m or not direct):
> > +if not allow_many and m2m:
> >  for alias in joins:
> >  self.unref_alias(alias)
> >  raise MultiJoin(pos + 1)
> >
> >
> > ---
> >
> > Looking at the original code, I'm not sure why "indirect" fields were not
> > allowed if they weren't many-to-many relationships. This is basically
> what
> > was keeping the reverse one-to-one lookups from working. I'm not 100%
> sure
> > of the consequences of this change, but all model tests seemed to pass
> with
> > this change.
> >
> > thoughts?
>
> My immediate thought is that the effect of the (m2m or not direct)
> logic is to remove the 'many' sides of fields from consideration. The
> comparison you're looking at is operating on data that comes from line
> 1725:
>
> field, model, direct, m2m = opts.get_field_by_name(name)
>
> In this contect, 'field' can be one of 5 things:
>
>  1. Natural fields (charfield)
>  2. ForeignKey fields (the local end)
>  3. OneToOneFields (the local end)
>  4. ManyToManyField (the local end)
>  5. RelatedObject (the remote 'many' end of a FK, or the remote end of an
> m2m)
>  6. RelatedObject (the remote end of a OneToOne Field)
>
> (1) is not of interest here. select_related() currently works with (2)
> and (3). These are the values where direct=True (since they are
> locally defined) but m2m is False. (4) always returns m2m; (5) and (6)
> always return direct=False.
>
> (6) is the case you're trying allow, but you can't do so at the
> expense of excluding (5).
>
> The fact that the tests don't fail when you make this change is
> promising. (I presume your statement about model tests includes the
> regression tests too, not just the modeltest directory) However,
> there's no particular guarantee of branch coverage in Django's test
> suite.
>
> Just be sure, my suggestion here it to try and be pathological. Work
> your way backwards from that if statement, and try to work out what
> set of models will cause that join condition to be invoked.
>
> Essentially, try to reverse engineer the test case that will fail if
> you modify that line. If it turns out to not be possible, then explain
> why; if it is possible, then you need to either find another way to
> manufacture the join you need.
>
> Yours,
> Russ Magee %-)
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Change list default sort modification proposal

[Ben Davis]

I'm not sure if anyone's seen this,  but I've written a patch for multi-sort
columns in the changelist ui here:

http://code.djangoproject.com/ticket/11868

Not sure if this is what you guys had in mind, but I'd be interested in your
feedback.

On Thu, Sep 17, 2009 at 12:57 PM, Joshua Russo wrote:

>
> Sorry for dragging this back out of the depths but I was reviewing my
> code for regression testing and I noticed I had missed this comment
> way back when.
>
> Where is this named sort option documented?
>
> It seems to me that the implementation that I have here would still be
> useful even if you could associate multiple sort columns with a single
> displayed column. I think that it's confusing to the developer not to
> have the list display using the sort order that is defined within the
> model.
>
>
> On Aug 13, 2:49 pm, Rock  wrote:
> > My suggestion for handling the UI for multi-column sorts is to allow
> > the definition
> > of named sorts in a manner similar to the way that the default sorting
> > is defined.
> > With this in place, additional multi column orders can be accomplished
> > simply
> > by using this name as a sort selection and applying the corresponding
> > named sort.
> >
> > If this is an acceptable approach, I can flesh out the API for it.
> >
> > On Aug 11, 10:10 pm, Malcolm Tredinnick 
> > wrote:
> >
> >
> >
> > > For those, like me, wondering what this proposal was about, it's
> > > concerning changing sorting in the admin interface to initially use the
> > > full set of fields specified in Meta.ordering on the model.
> >
> > > What I can't work out yet, due to difficulty in reviewing the patch,
> > > mentioned below, is whether it allow sorting by more than one column
> via
> > > the column-clicking method. That would almost certainly seem to be a
> > > request that is going to arrive 10 seconds after a feature like this is
> > > committed, although the difficulty has always been with how the UI
> works
> > > for that situation.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[SeanOC]

+1 on the logging proposal.  The stock python logging module is
definitely a bit of a finicky and confusing creature, especially for
people coming to Python for the first time with Django.

-0 On the signals based approach.  I would be wary of the potential
performance overhead of replacing logging with signals and/or
implement logging in certain high traffic areas with signals.  It
would definitely be interesting to see some proof of concept
performance tests (i.e. what Simon did with the handlerless logging)
before going too far down that path.

-Sean O'Connor

On Sep 17, 1:40 pm, Eric Florenzano  wrote:
> On Sep 17, 1:25 am, Simon Willison  wrote:
>
> > 1. We'll be able to de-emphasise the current default "e-mail all
> > errors to someone" behaviour, which doesn't scale at all well.
>
> I'm a big fan of this proposal, for exactly this reason.
>
> +1
>
> Thanks,
> Eric Florenzano
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Question on ticket triage process

[Ben Davis]

So by documentation,  do you mean updating the official django docs,  or
better documentation in the code itself?

On Sat, Sep 12, 2009 at 2:15 PM, Eric Holscher wrote:

> At first glance, tests and documentation. Everything needs both of these
> things before they go into trunk. Having a complete patch like that will
> make it a lot easier for someone to see what you're doing, and verify that
> you have fixed it.
>
> Cheers,
> Eric
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Change list default sort modification proposal

[Joshua Russo]

Sorry for dragging this back out of the depths but I was reviewing my
code for regression testing and I noticed I had missed this comment
way back when.

Where is this named sort option documented?

It seems to me that the implementation that I have here would still be
useful even if you could associate multiple sort columns with a single
displayed column. I think that it's confusing to the developer not to
have the list display using the sort order that is defined within the
model.


On Aug 13, 2:49 pm, Rock  wrote:
> My suggestion for handling the UI for multi-column sorts is to allow
> the definition
> of named sorts in a manner similar to the way that the default sorting
> is defined.
> With this in place, additional multi column orders can be accomplished
> simply
> by using this name as a sort selection and applying the corresponding
> named sort.
>
> If this is an acceptable approach, I can flesh out the API for it.
>
> On Aug 11, 10:10 pm, Malcolm Tredinnick 
> wrote:
>
>
>
> > For those, like me, wondering what this proposal was about, it's
> > concerning changing sorting in the admin interface to initially use the
> > full set of fields specified in Meta.ordering on the model.
>
> > What I can't work out yet, due to difficulty in reviewing the patch,
> > mentioned below, is whether it allow sorting by more than one column via
> > the column-clicking method. That would almost certainly seem to be a
> > request that is going to arrive 10 seconds after a feature like this is
> > committed, although the difficulty has always been with how the UI works
> > for that situation.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Eric Florenzano]

On Sep 17, 1:25 am, Simon Willison  wrote:
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.

I'm a big fan of this proposal, for exactly this reason.

+1

Thanks,
Eric Florenzano
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #9886: HttpRequest refactoring

[Ivan Sagalaev]

Jacob Kaplan-Moss wrote:
> On Thu, Sep 17, 2009 at 10:09 AM, Ivan Sagalaev
>  wrote:
>> Ticket [9886] is one of the things that I've been holding for 1.2 time
>> frame to ask for some discussion since Jacob marked it DDN.
> 
> At first glance, you need docs and tests before you get a second glance :)

I believe my follow up about this that I've posted a bit erlier will 
eventually get to Google Groups :-).

I'm going to make them of course. Now I just wanted a general answer 
along the lines "okay this looks sane" or "no we don't need it".

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #9886: HttpRequest refactoring

[Ivan Sagalaev]

Ivan Sagalaev wrote:
> This comment[1] in the ticket is a summary of what had changed. Thanks 
> for looking!

Forgot to add... There's no docs & tests changes in the patch yet. I'll 
add them after the decision if this looks good at all.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #9886: HttpRequest refactoring

[Jacob Kaplan-Moss]

On Thu, Sep 17, 2009 at 10:09 AM, Ivan Sagalaev
 wrote:
> Ticket [9886] is one of the things that I've been holding for 1.2 time
> frame to ask for some discussion since Jacob marked it DDN.

At first glance, you need docs and tests before you get a second glance :)

Jacob

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

#9886: HttpRequest refactoring

[Ivan Sagalaev]

Hello everyone!

Ticket [9886] is one of the things that I've been holding for 1.2 time 
frame to ask for some discussion since Jacob marked it DDN.

This has started as a feature allowing to .read() directly from a 
request object. However this has lead to a rather nice refactoring 
removing a lot of code duplication in wsgi and modpython handlers.

This comment[1] in the ticket is a summary of what had changed. Thanks 
for looking!

[9886]: http://code.djangoproject.com/ticket/9886
[1]: http://code.djangoproject.com/ticket/9886#comment:1

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: proposal: decouple admin index page from INSTALLED_APPS

[patrickk]

just for the records ...

as russell commented on the ticket, this can all be done with
subclassing AdminSite.

so ... here´s how grouping and reordering app with the admin-interface
can be achieved:
http://code.google.com/p/django-grappelli/wiki/customizingindex

and here´s the code (subclassing AdminSite), just in case someone
wants to do something similar without grappelli:
http://code.google.com/p/django-grappelli/source/browse/branches/grappelli_2/sites.py

thanks for all the feedback/help,
patrick


On 17 Sep., 12:55, patrickk  wrote:
> I´ve opened a ticket with my patches (and a 
> screenshot):http://code.djangoproject.com/ticket/11895
>
> I´ve given up on changing the app-index page since this seems to break
> the overall coherence of the admin-interface. therefore, my patch doesn
> ´t change the breadcrumbs. it only gives you the ability to group (and
> reorder) the apps shown on the index-page of every admin-site.
>
> I´ve not included a patch for index.html ... basically, one has to
> loop the group-list first and then the app-list (every app which is
> not part of a group is displayed at the bottom of the index-page). if
> one doesn´t define a group, all apps are listed as is.
>
> the file admin.py show how the apps can be defined for an admin-site.
>
> thanks,
> patrick
>
> On 6 Sep., 11:38, patrickk  wrote:
>
> > my thoughts:
> > 1. defining groups within admin.py of each app makes reordering groups
> > complicated.
> > 2. to assign an app to a group, I have to register that app with a
> > group - if I want (for example) to assign django.contrib.auth to the
> > group "User Management", I have to change that app (which breaks
> > future svn-updates). this is true for almost every 3rd-party app (if I
> > ´m using an svn-checkout).
>
> > instead of using a bottom-up structure (registering apps with groups),
> > why not using a top-down structure (defining groups and assigning
> > apps).
> > e.g., you could have a file admin.py within your project and this file
> > could define the groups:
>
> > class UserManagement(AdminGroup):
> >     label = "User Management"
> >     ...
> >     'apps': ('django.contrib.auth', 'user',)
>
> > regards,
> > patrick
>
> > On 5 Sep., 20:22, Russ Ryba  wrote:
>
> > > I'm coming late to the coversation but wanted to add by thoughts on  
> > > the admin index issue. I'm at a campground on a cell phone so  
> > > bandwidth and typing is limited.
>
> > > I'll try to be specific in my comments.
>
> > > 1. I think that this should be handled by individual app admin.py.  I  
> > > like the idea of just modifying one line in installed_apps to turn an  
> > > application on or off.
>
> > > 2. I think settings.py should only know about applications.  You  
> > > should never specify model sorting or model anything in settings.py.
>
> > > 3. Admin.py per app should specify grouping and sorting per model.  
> > > First by registering a modrladmin group, then later assigning  
> > > modeladmins to that group.
>
> > > I would extend the previous example as follows.
>
> > > class UserManagement(AdminGroup):
> > >     label = "User Management"
> > >     sort_order = 100
> > >     merge_with_existing_group = True
>
> > > class UserAdmin(ModelAdmin):
> > >        Meta:
> > >               sort_order = 50
> > > class FooAdmin(ModelAdmin):
> > >       Meta:
> > >               sort_order = 60
>
> > > admin.site.register(User, UserAdmin, group=UserManagement)
> > > admin.site.register(Foo, FooAdmin, group=UserManagement)
>
> > > Then in the index template app groups would be sorted by sort order  
> > > values, then alphanumeric.
>
> > > - This would keep grouping defined in each app admin class.
> > > - This would let you sort apps/groups by numerical values.
> > > - mergewexisting would let apps join together or not, being false by  
> > > default so you could see when name collisions occur with app or group  
> > > names.  It could also let complimentary apps extend each other by  
> > > adding new models.
>
> > > Further thoughts.
>
> > > - SQL tables are currents based on appname_model, as are the URL  
> > > mappings. Changing the index grouping is essentially the same as  
> > > merging applications.
>
> > > I love the sound of it if we focus on the index.  However I foresee  
> > > the next logical request would be to "fix" the SQL and URL mappings to  
> > > match the index display. That would be practically impossible or very  
> > > ugly.
>
> > > Name collisions suddenly have to be accounted for, SQL migrations to  
> > > rename tables, as well as permission names.
>
> > > 4. Thinking about how to implement the URL and SQL mappings leads me  
> > > to suggest a new file App/meta.py.  This would contain application  
> > > name override, Application sort order and model grouping would be  
> > > defined here.
>
> > > The meta class would know about the applications models, and admin  
> > > interfaces 

Re: Patch Review - #9638

[lakin]

Sweet,  thanks a bunch!

Lakin

On Sep 17, 8:31 am, Jacob Kaplan-Moss  wrote:
> On Thu, Sep 17, 2009 at 9:26 AM, lakin  wrote:
> > Does anyone have the time to review the patch, please?
>
> Looks good; I've marked it ready for commit.
>
> Jacob
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Andrew Gwozdziewycz]

On Thu, Sep 17, 2009 at 10:04 AM, Russell Keith-Magee
 wrote:

> As for likely roadblocks: I've been led to believe that Adrian has
> objections to framework-level logging. I have no idea as to the nature
> of his objection, but ticket #5415 indicates that he is (or has been,
> historically) in favor of adding signals that could be used for
> logging or debugging purposes.

I'm in favor of the signals approach as part of the core framework. In
addition, django.contrib.logging, which could provide a generic
solution, good for the
80 or 90% case would really rock.

-- 
http://www.apgwoz.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Patch Review - #9638

[Jacob Kaplan-Moss]

On Thu, Sep 17, 2009 at 9:26 AM, lakin  wrote:
> Does anyone have the time to review the patch, please?

Looks good; I've marked it ready for commit.

Jacob

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Patch Review - #9638

[lakin]

I'd like to help facilitate the application of the patch on
http://code.djangoproject.com/ticket/9638 to trunk and potentially in
any releases that it would make sense to include it in.  I've been
told that I should politely ask for a patch review on the mailing
list.

Does anyone have the time to review the patch, please?

Thanks in Advance,

Lakin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Russell Keith-Magee]

On Thu, Sep 17, 2009 at 4:25 PM, Simon Willison  wrote:
>
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
> log under the mod_python handler and environ['wsgi.errors'] under WSGI
> (meaning mod_wsgi will write to the Apache error log as well).
>
> Benefits of logging as a core Django service
> 
>
> Adding logging to Django core would provide the following benefits:
>
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.
>
> 2. Having logging in the core framework will mean people start
> actually using it, which will make it easier for people to debug their
> Django apps. Right now adding "print" statements to a Django app is a
> common debugging technique, but it's messy (you have to remember to
> take them out again) and error prone - some production environments
> throw errors if an app attempts to write to stdout. It's also not
> obvious - many developers are surprised when I show them the
> technique.
>
> 3. Logging in Django core rather than a 3rd party app will encourage
> reusable applications to log things in a predictable way, standard
> way.
>
> 4. 3rd party debugging tools such as the debug toolbar will be able to
> hook in to Django's default logging behaviour. This could also lead to
> plenty of additional 3rd party innovation - imagine a tool that looks
> out for logged SQL that took longer than X seconds, or one that groups
> together similar log messages, or streams log messages to IRC...
>
> 5. Built-in support for logging reflects a growing reality of modern
> Web development: more and more sites have interfaces with external web
> service APIs, meaning there are plenty of things that could go wrong
> that are outside the control of the developer. Failing gracefully and
> logging what happened is the best way to deal with 3rd party problems
> - much better than throwing a 500 and leaving no record of what went
> wrong.
>
> 6. Most importantly from my point of view, when a sysadmin asks where
> Django logs errors in production we'll have a good answer for them!
>
> 7. As a general rule, I believe you can never have too much
> information about what's going on with your web application. I've
> never thought to myself "the problem with this bug is I've got too
> much information about it". As for large log files, disk space is
> cheap - and pluggable backends could ensure logs were sensibly
> rotated.

No disagreement here with any of these assertions.

> Places logging would be useful
> ==
>
> - Unhandled exceptions that make it up to the top of the Django stack
> (and would cause a 500 error to be returned in production)
> - The development web server could use logging for showing processed
> requests (where currently these are just printed to stdout).
> - Failed attempts at signing in to the admin could be logged, making
> security audits easier.
> - We could replace (or complement) django.connection.queries with a
> log of executed SQL. This would make the answer to the common question
> "how do I see what SQL is being executed" much more obvious.
> - Stuff that loads things from INSTALLED_APPS could log what is being
> loaded, making it much easier to spot and debug errors caused by code
> being incorrectly loaded.
> - Likewise, the template engine could log which templates are being
> loaded from where, making it easier to debug problems stemming from an
> incorrectly configured TEMPLATE_DIRS setting.
> - We could use logging to address the problems with the template
> engine failing silently - maybe some template errors (the ones more
> likely to be accidental than just people relying on the fail-silent
> behaviour deliberately) should be logged as warnings.
>
> Most of the above would be set to a low log level which by default
> would not be handled, displayed or stored anywhere (logging.info or
> similar). Maybe "./manage.py runserver --loglevel=info" could cause
> such logs to be printed to  the terminal while the development server
> is running.
>
> Problems and challenges
> ===
>
> 1. The Python logging module isn't very nicely designed - its Java
> heritage shines through, and things like logging.basicConfig behave in
> unintuitive ways (if you call basicConfig twice the second call fails
> silently but has no effect). This is why I suggest wrapping it in our
> own higher level interface.

In the absence of specifics, this makes me a little bit nervous. The
Python logging interface may be very Java-heavy and complex, but it is
a thoroughly known quantity, and it houses a lot of features.

I've seen several attempts to wrap Java loggers in a "nicer"
interface, and every one of them ended up hobbling some of the power
features of the 

Re: Proposal for 1.2: built-in logging with django.core.log

[Russell Keith-Magee]

On Thu, Sep 17, 2009 at 6:53 PM, Ivan Sagalaev
 wrote:
>
> Hi Simon,
>
> Simon Willison wrote:
>> 1. We'll be able to de-emphasise the current default "e-mail all
>> errors to someone" behaviour, which doesn't scale at all well.
>
> In a recent thread[1] on a similar topic Russel has also emphasized that
> we should improve documentation about doing logging.

To clarify - I think that documentation is the very least we should
do. As your comments indicate, there are a lot of things you need to
do in order to get logging right, so we should at the very least
provide some documentation on how to do it right.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: CSRF proposal and patch ready for review

[Simon Willison]

All good points - the change in function signature naturally fell out
of the CSRF work (since the form needs access to the request object in
both cases) but you've convinced me that it's a step too far - I'll
see if I can fix that.

Cheers,

Simon

On Sep 17, 2:10 pm, Russell Keith-Magee 
wrote:
> On Thu, Sep 17, 2009 at 3:11 PM, Simon Willison  
> wrote:
>
> > On Sep 15, 2:57 pm, Luke Plant  wrote:
> >> OK, I'll wait and see.
>
> > Here's the code:http://github.com/simonw/django-safeform
>
> >>  * it requires using Django's form system.  I've got plenty of views that
> >> don't (e.g. anything with a dynamic number of controls).  People not using
> >> django.Forms are left out in the cold, or having to learn another way to do
> >> things.
>
> > I've covered this in django-safeform by exposing a low level interface
> > to the CSRF token creation and validation routines (in csrf_utils) -
> > see the readme for documentation on this.
>
> >>  * it's off by default.  Turning it on by default will require making
> >> django.forms.Form subclass SafeForm, which will almost certainly require a
> >> huge and immediate upgrade effort, because SafeForm cannot have the same 
> >> API
> >> as Form.  If it's off by default, I see no point at all in this entire
> >> exercise.  If we don't arrive at a point where the POST form created in the
> >> tutorial is safe from CSRF, I think we've failed.
>
> > My priority for this is a little different: while I would dearly love
> > to see all Django applications secure against CSRF by default, I can't
> > see a way of doing it that doesn't break existing apps. More important
> > for me though is that the Django admin (and other reusable apps, such
> > as Pinax stuff) MUST be secure against CSRF out of the box, no matter
> > what the user puts in their settings.py file. If developers fail to
> > protect themselves (especially when the solution is well documented)
> > then at least it isn't our fault.
>
> > I think this is subtly different from the XSS escaping issue simply
> > because the solution to CSRF is much more intrusive.
>
> >> And it will still require changes to templates, just like the template tag,
> >> and it will also require changes to views, only significantly more complex
> >> than simply using RequestContext.  It's got at least as many and at least 
> >> as
> >> intrusive 'moving parts' AFAICS.
>
> > The SafeForm API actually simplifies views - you go from this:
>
> > def change_password(request):
> >    form = ChangePasswordForm()
> >    if request.method == 'POST':
> >        form = ChangePasswordForm(request.POST)
> >        if form.is_valid():
> >            return HttpResponse('Thank you')
> >    return render_to_response('change_password.html', {
> >        'form': form,
> >    })
>
> > To this:
>
> > @csrf_protect
> > def change_password(request):
> >    form = ChangePasswordForm(request)
> >    if form.is_valid():
> >        return HttpResponse('Thank you')
> >    return render_to_response('change_password.html', {
> >        'form': form,
> >    })
>
> > The SafeForm deals with the request.method == 'POST' bit, meaning one
> > less conditional branch within the view function itself.
>
> Did we discuss this bit at the sprint? I'm completely on board with
> everything else, but this bit makes me nervous:
>
>  * It calls is_valid() on the form during a GET (which isn't currently
> done as part of normal practice).
>
>  * It isn't possible to have any special view handling based on the
> request method.
>
>  * It implies that the method of construction the form will always be
> the same, regardless of whether we are GETting or POSTing. This isn't
> always true - in particular, I'm thinking of the construction of
> FormSets, which can use a queryset to populate the forms on GET, but
> don't require the queryset on POST.
>
> > I'm pretty happy with the SafeForm interface now that I've fleshed it
> > out - it's a lot less clunky than I was originally expecting.
>
> Another advantage that Simon hasn't mentioned - it requires no
> modifications to the tutorial. We can present the basic ideas of form
> handling in tutorial 4 without mentioning CSRF. We then have scope to
> introduce a new tutorial on security issues, that can both educate on
> the nature of XSS and CSRF attacks, plus what you have to do in order
> to protect against them.
>
> Yours,
> Russ Magee %-)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: CSRF proposal and patch ready for review

[Russell Keith-Magee]

On Thu, Sep 17, 2009 at 3:11 PM, Simon Willison  wrote:
>
> On Sep 15, 2:57 pm, Luke Plant  wrote:
>> OK, I'll wait and see.
>
> Here's the code: http://github.com/simonw/django-safeform
>
>>  * it requires using Django's form system.  I've got plenty of views that
>> don't (e.g. anything with a dynamic number of controls).  People not using
>> django.Forms are left out in the cold, or having to learn another way to do
>> things.
>
> I've covered this in django-safeform by exposing a low level interface
> to the CSRF token creation and validation routines (in csrf_utils) -
> see the readme for documentation on this.
>
>>  * it's off by default.  Turning it on by default will require making
>> django.forms.Form subclass SafeForm, which will almost certainly require a
>> huge and immediate upgrade effort, because SafeForm cannot have the same API
>> as Form.  If it's off by default, I see no point at all in this entire
>> exercise.  If we don't arrive at a point where the POST form created in the
>> tutorial is safe from CSRF, I think we've failed.
>
> My priority for this is a little different: while I would dearly love
> to see all Django applications secure against CSRF by default, I can't
> see a way of doing it that doesn't break existing apps. More important
> for me though is that the Django admin (and other reusable apps, such
> as Pinax stuff) MUST be secure against CSRF out of the box, no matter
> what the user puts in their settings.py file. If developers fail to
> protect themselves (especially when the solution is well documented)
> then at least it isn't our fault.
>
> I think this is subtly different from the XSS escaping issue simply
> because the solution to CSRF is much more intrusive.
>
>> And it will still require changes to templates, just like the template tag,
>> and it will also require changes to views, only significantly more complex
>> than simply using RequestContext.  It's got at least as many and at least as
>> intrusive 'moving parts' AFAICS.
>
> The SafeForm API actually simplifies views - you go from this:
>
> def change_password(request):
>    form = ChangePasswordForm()
>    if request.method == 'POST':
>        form = ChangePasswordForm(request.POST)
>        if form.is_valid():
>            return HttpResponse('Thank you')
>    return render_to_response('change_password.html', {
>        'form': form,
>    })
>
> To this:
>
> @csrf_protect
> def change_password(request):
>    form = ChangePasswordForm(request)
>    if form.is_valid():
>        return HttpResponse('Thank you')
>    return render_to_response('change_password.html', {
>        'form': form,
>    })
>
> The SafeForm deals with the request.method == 'POST' bit, meaning one
> less conditional branch within the view function itself.

Did we discuss this bit at the sprint? I'm completely on board with
everything else, but this bit makes me nervous:

 * It calls is_valid() on the form during a GET (which isn't currently
done as part of normal practice).

 * It isn't possible to have any special view handling based on the
request method.

 * It implies that the method of construction the form will always be
the same, regardless of whether we are GETting or POSTing. This isn't
always true - in particular, I'm thinking of the construction of
FormSets, which can use a queryset to populate the forms on GET, but
don't require the queryset on POST.

> I'm pretty happy with the SafeForm interface now that I've fleshed it
> out - it's a lot less clunky than I was originally expecting.

Another advantage that Simon hasn't mentioned - it requires no
modifications to the tutorial. We can present the basic ideas of form
handling in tutorial 4 without mentioning CSRF. We then have scope to
introduce a new tutorial on security issues, that can both educate on
the nature of XSS and CSRF attacks, plus what you have to do in order
to protect against them.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: SoC merge plans?

[Russell Keith-Magee]

On Thu, Sep 17, 2009 at 4:35 PM, Simon Willison  wrote:
>
> On Sep 13, 12:52 pm, Russell Keith-Magee 
> wrote:
>> Although the branch isn't ready for merge yet, the code that is
>> present is all up to spec. I think the sprints have produced workable
>> solutions for the outstanding problems; fixes implementing the
>> solutions we discussed should be forthcoming shortly.
>
> With my cowboy hat on... what are the disadvantages to merging this to
> trunk /now/ and fixing the remaining outstanding issues there? It
> might make it easier to get the other SoC branches compatible with
> this one (which sounds to me like it makes the most far-reaching
> changes) and it would encourage the wider community to help check that
> everything really is backwards compatible.
>
> 

Howdy Pardner... :-)

My hesitation here is that there one of the outstanding changes is a
fairly big refactor of the way the SQL backend handles queries, and I
want to make sure it's right before we inflict it on the world. In
particular, I need to make sure that the needs of Oracle and
contrib.GIS aren't broken by the refactor. I don't want to commit the
patch only to have to roll large parts of it back in the near future.

To keep your inner cowboy happy - I'm not actually anticipating that
much of an overlap problem with the other GSoC branches. Although the
changes are fairly extensive, they are mostly with internals, and not
in areas that the other GSoC branches should be touching, such as
database backends, details of connection handling, use of connections
by management commands.

The m2m refactoring patch (which is a prerequisite for the multi-db
work) could be committed now without any major problems.

Yours,
Russ (The lone cowboy of the West) %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: proposal: decouple admin index page from INSTALLED_APPS

[patrickk]

I´ve opened a ticket with my patches (and a screenshot):
http://code.djangoproject.com/ticket/11895

I´ve given up on changing the app-index page since this seems to break
the overall coherence of the admin-interface. therefore, my patch doesn
´t change the breadcrumbs. it only gives you the ability to group (and
reorder) the apps shown on the index-page of every admin-site.

I´ve not included a patch for index.html ... basically, one has to
loop the group-list first and then the app-list (every app which is
not part of a group is displayed at the bottom of the index-page). if
one doesn´t define a group, all apps are listed as is.

the file admin.py show how the apps can be defined for an admin-site.

thanks,
patrick


On 6 Sep., 11:38, patrickk  wrote:
> my thoughts:
> 1. defining groups within admin.py of each app makes reordering groups
> complicated.
> 2. to assign an app to a group, I have to register that app with a
> group - if I want (for example) to assign django.contrib.auth to the
> group "User Management", I have to change that app (which breaks
> future svn-updates). this is true for almost every 3rd-party app (if I
> ´m using an svn-checkout).
>
> instead of using a bottom-up structure (registering apps with groups),
> why not using a top-down structure (defining groups and assigning
> apps).
> e.g., you could have a file admin.py within your project and this file
> could define the groups:
>
> class UserManagement(AdminGroup):
>     label = "User Management"
>     ...
>     'apps': ('django.contrib.auth', 'user',)
>
> regards,
> patrick
>
> On 5 Sep., 20:22, Russ Ryba  wrote:
>
> > I'm coming late to the coversation but wanted to add by thoughts on  
> > the admin index issue. I'm at a campground on a cell phone so  
> > bandwidth and typing is limited.
>
> > I'll try to be specific in my comments.
>
> > 1. I think that this should be handled by individual app admin.py.  I  
> > like the idea of just modifying one line in installed_apps to turn an  
> > application on or off.
>
> > 2. I think settings.py should only know about applications.  You  
> > should never specify model sorting or model anything in settings.py.
>
> > 3. Admin.py per app should specify grouping and sorting per model.  
> > First by registering a modrladmin group, then later assigning  
> > modeladmins to that group.
>
> > I would extend the previous example as follows.
>
> > class UserManagement(AdminGroup):
> >     label = "User Management"
> >     sort_order = 100
> >     merge_with_existing_group = True
>
> > class UserAdmin(ModelAdmin):
> >        Meta:
> >               sort_order = 50
> > class FooAdmin(ModelAdmin):
> >       Meta:
> >               sort_order = 60
>
> > admin.site.register(User, UserAdmin, group=UserManagement)
> > admin.site.register(Foo, FooAdmin, group=UserManagement)
>
> > Then in the index template app groups would be sorted by sort order  
> > values, then alphanumeric.
>
> > - This would keep grouping defined in each app admin class.
> > - This would let you sort apps/groups by numerical values.
> > - mergewexisting would let apps join together or not, being false by  
> > default so you could see when name collisions occur with app or group  
> > names.  It could also let complimentary apps extend each other by  
> > adding new models.
>
> > Further thoughts.
>
> > - SQL tables are currents based on appname_model, as are the URL  
> > mappings. Changing the index grouping is essentially the same as  
> > merging applications.
>
> > I love the sound of it if we focus on the index.  However I foresee  
> > the next logical request would be to "fix" the SQL and URL mappings to  
> > match the index display. That would be practically impossible or very  
> > ugly.
>
> > Name collisions suddenly have to be accounted for, SQL migrations to  
> > rename tables, as well as permission names.
>
> > 4. Thinking about how to implement the URL and SQL mappings leads me  
> > to suggest a new file App/meta.py.  This would contain application  
> > name override, Application sort order and model grouping would be  
> > defined here.
>
> > The meta class would know about the applications models, and admin  
> > interfaces and define how to group then all together.
>
> > Accessible via something like
>
> >     Appname.meta.property = value
>
> > Since it's define at the app level the SQL and URL processors should  
> > have access to it.
>
> > This could also be a good place to define things I use a lot like  
> > custom permissions specific to the application per app or per app model.
>
> > Maybe I'm getting ahead of myself with URL and SQL stuff?
>
> > Comments?
>
> > Russ Ryba
>
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 

Re: Proposal for 1.2: built-in logging with django.core.log

[Ivan Sagalaev]

Hi Simon,

Simon Willison wrote:
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.

In a recent thread[1] on a similar topic Russel has also emphasized that 
we should improve documentation about doing logging.

> 3. Logging in Django core rather than a 3rd party app will encourage
> reusable applications to log things in a predictable way, standard
> way.

Talking about predictable and standard way I want to be sure that we 
don't break existing logging.

I.e. we at Yandex now have many reusable Django apps that don't setup 
loggers themselves but just log things into named loggers and expect 
them to be setup by a project that uses them.

What I gather from your proposal is that you want the same model ("an 
app logs, a project setups") plus a nice declarative syntax in 
settings.py instead of boring creation of handlers, formatters and 
loggers. Right?

> - We could replace (or complement) django.connection.queries with a
> log of executed SQL. This would make the answer to the common question
> "how do I see what SQL is being executed" much more obvious.

In the thread that I was referring to[1] we kind of agreed on using a 
signal there. Then hooking a logger onto the signal is simple.

> 5. People might leave logging on, then find their server disk has
> filled up with log files and caused their site to crash.

We had this problem with standard logging. Then we switched to a 
RotatingFileHandler which wasn't very good however because its behavior 
is simplistic and is not controllable by admins with an interface that 
they know, namely logrotate. Setting up logrotate also wasn't without 
problems. When it rotates a file it should let an app know about it and 
it uses SIG_HUP for that. However this instantly terminates Django's 
flup-based FastCGI server which we use.

Now we've settled on a WatchedFileHandler ported from Python 2.6 logging 
module. It watches for file descriptor change and doesn't require 
SIG_HUP to pick up a new file. May be we should port it to Django and 
use it as a default handler for logging to file system.

> 6. Logging levels are confusing - what exactly is the difference
> between warn, info, error, debug, critical and fatal? We would need to
> document this and make decisions on which ones get used for what
> within the framework.

May be we can just leave it for users to decide. It depends so much on 
how much an app actually wants from logging.

The only standard thing I can think of is to have DEBUG = True imply 
level = logging.DEBUG (which includes everything more sever). DEBUG = 
False will imply logging.INFO then. What do you think?

> # If you want custom log handlers - not sure how these would interact
> with
> # channels and log levels yet
> LOG_HANDLERS = (
> 'django.core.log.handlers.LogToDatabase',
> 'django.core.log.handlers.LogToEmail',
> )

This is a hard problem really. Most handlers require different set of 
arguments. File names, email credentials, system idents for SysLog etc. 
Also there should be different formatters. For example there's no point 
to waste space in a syslog message on a timestamp since syslog tracks it 
itself...

[1]: 
http://groups.google.com/group/django-developers/browse_frm/thread/9d0992e800cf7d68#

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Horst Gutmann]

Definitely a +1 from me.

-- Horst

On Thu, Sep 17, 2009 at 10:25 AM, Simon Willison
 wrote:
>
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
> log under the mod_python handler and environ['wsgi.errors'] under WSGI
> (meaning mod_wsgi will write to the Apache error log as well).
>
> Benefits of logging as a core Django service
> 
>
> Adding logging to Django core would provide the following benefits:
>
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.
>
> 2. Having logging in the core framework will mean people start
> actually using it, which will make it easier for people to debug their
> Django apps. Right now adding "print" statements to a Django app is a
> common debugging technique, but it's messy (you have to remember to
> take them out again) and error prone - some production environments
> throw errors if an app attempts to write to stdout. It's also not
> obvious - many developers are surprised when I show them the
> technique.
>
> 3. Logging in Django core rather than a 3rd party app will encourage
> reusable applications to log things in a predictable way, standard
> way.
>
> 4. 3rd party debugging tools such as the debug toolbar will be able to
> hook in to Django's default logging behaviour. This could also lead to
> plenty of additional 3rd party innovation - imagine a tool that looks
> out for logged SQL that took longer than X seconds, or one that groups
> together similar log messages, or streams log messages to IRC...
>
> 5. Built-in support for logging reflects a growing reality of modern
> Web development: more and more sites have interfaces with external web
> service APIs, meaning there are plenty of things that could go wrong
> that are outside the control of the developer. Failing gracefully and
> logging what happened is the best way to deal with 3rd party problems
> - much better than throwing a 500 and leaving no record of what went
> wrong.
>
> 6. Most importantly from my point of view, when a sysadmin asks where
> Django logs errors in production we'll have a good answer for them!
>
> 7. As a general rule, I believe you can never have too much
> information about what's going on with your web application. I've
> never thought to myself "the problem with this bug is I've got too
> much information about it". As for large log files, disk space is
> cheap - and pluggable backends could ensure logs were sensibly
> rotated.
>
> Places logging would be useful
> ==
>
> - Unhandled exceptions that make it up to the top of the Django stack
> (and would cause a 500 error to be returned in production)
> - The development web server could use logging for showing processed
> requests (where currently these are just printed to stdout).
> - Failed attempts at signing in to the admin could be logged, making
> security audits easier.
> - We could replace (or complement) django.connection.queries with a
> log of executed SQL. This would make the answer to the common question
> "how do I see what SQL is being executed" much more obvious.
> - Stuff that loads things from INSTALLED_APPS could log what is being
> loaded, making it much easier to spot and debug errors caused by code
> being incorrectly loaded.
> - Likewise, the template engine could log which templates are being
> loaded from where, making it easier to debug problems stemming from an
> incorrectly configured TEMPLATE_DIRS setting.
> - We could use logging to address the problems with the template
> engine failing silently - maybe some template errors (the ones more
> likely to be accidental than just people relying on the fail-silent
> behaviour deliberately) should be logged as warnings.
>
> Most of the above would be set to a low log level which by default
> would not be handled, displayed or stored anywhere (logging.info or
> similar). Maybe "./manage.py runserver --loglevel=info" could cause
> such logs to be printed to  the terminal while the development server
> is running.
>
> Problems and challenges
> ===
>
> 1. The Python logging module isn't very nicely designed - its Java
> heritage shines through, and things like logging.basicConfig behave in
> unintuitive ways (if you call basicConfig twice the second call fails
> silently but has no effect). This is why I suggest wrapping it in our
> own higher level interface.
>
> 2. There may be some performance overhead, especially if we replace
> mechanisms like django.connection.queries with logging. This should be
> negligble: here's a simple benchmark:
>
> # ("hello " * 100) gives a 600 char string, long enough for a SQL
> statement
 import timeit, logging
 t = timeit.Timer('logging.info("hello " * 100)', 'import 

Re: Proposal for 1.2: built-in logging with django.core.log

[Mat Clayton]

+1 for this, another random thought which doesn't do your long post justice.
But what are everyone's thoughts about log aggregation, taking logs from X
app servers and combining them into a single location, something like
Facebook's Scribe. I assume this could be built in as a separate log
handler, but it would be nice for the guys who need this functionality to be
able to achieve it easily, probably not right for core though.

Mat

On Thu, Sep 17, 2009 at 9:25 AM, Simon Willison wrote:

>
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
> log under the mod_python handler and environ['wsgi.errors'] under WSGI
> (meaning mod_wsgi will write to the Apache error log as well).
>
> Benefits of logging as a core Django service
> 
>
> Adding logging to Django core would provide the following benefits:
>
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.
>
> 2. Having logging in the core framework will mean people start
> actually using it, which will make it easier for people to debug their
> Django apps. Right now adding "print" statements to a Django app is a
> common debugging technique, but it's messy (you have to remember to
> take them out again) and error prone - some production environments
> throw errors if an app attempts to write to stdout. It's also not
> obvious - many developers are surprised when I show them the
> technique.
>
> 3. Logging in Django core rather than a 3rd party app will encourage
> reusable applications to log things in a predictable way, standard
> way.
>
> 4. 3rd party debugging tools such as the debug toolbar will be able to
> hook in to Django's default logging behaviour. This could also lead to
> plenty of additional 3rd party innovation - imagine a tool that looks
> out for logged SQL that took longer than X seconds, or one that groups
> together similar log messages, or streams log messages to IRC...
>
> 5. Built-in support for logging reflects a growing reality of modern
> Web development: more and more sites have interfaces with external web
> service APIs, meaning there are plenty of things that could go wrong
> that are outside the control of the developer. Failing gracefully and
> logging what happened is the best way to deal with 3rd party problems
> - much better than throwing a 500 and leaving no record of what went
> wrong.
>
> 6. Most importantly from my point of view, when a sysadmin asks where
> Django logs errors in production we'll have a good answer for them!
>
> 7. As a general rule, I believe you can never have too much
> information about what's going on with your web application. I've
> never thought to myself "the problem with this bug is I've got too
> much information about it". As for large log files, disk space is
> cheap - and pluggable backends could ensure logs were sensibly
> rotated.
>
> Places logging would be useful
> ==
>
> - Unhandled exceptions that make it up to the top of the Django stack
> (and would cause a 500 error to be returned in production)
> - The development web server could use logging for showing processed
> requests (where currently these are just printed to stdout).
> - Failed attempts at signing in to the admin could be logged, making
> security audits easier.
> - We could replace (or complement) django.connection.queries with a
> log of executed SQL. This would make the answer to the common question
> "how do I see what SQL is being executed" much more obvious.
> - Stuff that loads things from INSTALLED_APPS could log what is being
> loaded, making it much easier to spot and debug errors caused by code
> being incorrectly loaded.
> - Likewise, the template engine could log which templates are being
> loaded from where, making it easier to debug problems stemming from an
> incorrectly configured TEMPLATE_DIRS setting.
> - We could use logging to address the problems with the template
> engine failing silently - maybe some template errors (the ones more
> likely to be accidental than just people relying on the fail-silent
> behaviour deliberately) should be logged as warnings.
>
> Most of the above would be set to a low log level which by default
> would not be handled, displayed or stored anywhere (logging.info or
> similar). Maybe "./manage.py runserver --loglevel=info" could cause
> such logs to be printed to  the terminal while the development server
> is running.
>
> Problems and challenges
> ===
>
> 1. The Python logging module isn't very nicely designed - its Java
> heritage shines through, and things like logging.basicConfig behave in
> unintuitive ways (if you call basicConfig twice the second call fails
> silently but has no effect). This is why I suggest wrapping it 

Re: Customizing URL resolver

[twold]

On Sep 16, 4:14 pm, matehat  wrote:
> Oh, and trying to make the url resolving mechanism look in the
> database is really a bad idea because it would trigger a bunch of
> database queries every time a request is processed by your
> application, which would really slow things down.

What do you mean? With get_absolute_url I get to magically resolve the
url without looking into db?
Regex just don't give me enough flexibility -- I need to make url
changes in admin (indirectly through renaming and moving categories)
and later have to look it up in db. No way around this. Or is it?
Well, maybe I can see one: automatically generate urls.py in admin
category post-save. But I don't like this at all.

>
> mathieu
>
> On 16 sep, 10:11, matehat  wrote:
>
> > Hi,
>
> > If the url only depends on the object's id (which I suppose don't
> > change with time), you should not have it put in the database at
> > first, because the object's id alone takes care of its persistence.
> > Then, plugging it into Django's url resolving mechanism becomes really
> > easy with the well-documented "get_absolute_url()" model method and
> > the "permalink" decorator, for instance.
>
> > Also, if you want better advices on how to fit your database model
> > into django's url resolving mechanism, you should post a question
> > (together with more details) onto the django-users group.
>
> > Mathieu
>
> > On 16 sep, 07:07, twold  wrote:
>
> > > Hello,
> > > I am having little difficulties with django url system and I hope
> > > someone can clear it up. Here's what I want to achieve: we're going to
> > > have a tree-based catalog in our app and we'd like to index it like
> > > this: name1/name2/name3/... I've created a database lookup table url <-> 
> > > id with urls being automatically generated at catalog save time. Now
>
> > > I would like to plug this in django url resolving process (first I put
> > > in view to which I passed whole path, but that is simply wrong for a
> > > couple of reasons). Also, I don't want to completely replace
> > > RegexURLResolver, because other parts of our site will be using it. So
> > > what I'd like would probably be some refactoring of the resolvers. For
> > > the time being I subclassed RegexURLPattern and implemented custom
> > > resolve method -- it is hacky, but works straight away when I put my
> > > subclass in url_patterns. The problem is, there seems to be no easy
> > > way, to make reverse work. I was thinking about all of it a lot, also
> > > googling a lot and I found no answers. So I wonder: what's the best
> > > way to make database URL resolving play nicely with the default regex
> > > resolving? Is there any way at all?
>
> > > With regards,
> > > Marek Bernat
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Customizing URL resolver

[twold]

On Sep 16, 4:11 pm, matehat  wrote:
> Hi,
>
> If the url only depends on the object's id (which I suppose don't
> change with time),

The only problem is, it doesn't :-) We'd like to be able to handle
redirects.
E.g. category "wine" is renamed to "wines", but we still want to be
able to redirect bots and whatnot from old url to the new one. 404 is
just bad for SEO.

 you should not have it put in the database at
> first, because the object's id alone takes care of its persistence.
> Then, plugging it into Django's url resolving mechanism becomes really
> easy with the well-documented "get_absolute_url()" model method and
> the "permalink" decorator, for instance.

Thanks, I didn't know about that, will look into it. Can it also
handle the above-mentioned redirects?

>
> Also, if you want better advices on how to fit your database model
> into django's url resolving mechanism, you should post a question
> (together with more details) onto the django-users group.

Sorry about that, I only noticed later that there is a -users group :-
(

>
> Mathieu
>
> On 16 sep, 07:07, twold  wrote:
>
> > Hello,
> > I am having little difficulties with django url system and I hope
> > someone can clear it up. Here's what I want to achieve: we're going to
> > have a tree-based catalog in our app and we'd like to index it like
> > this: name1/name2/name3/... I've created a database lookup table url <-> id 
> > with urls being automatically generated at catalog save time. Now
>
> > I would like to plug this in django url resolving process (first I put
> > in view to which I passed whole path, but that is simply wrong for a
> > couple of reasons). Also, I don't want to completely replace
> > RegexURLResolver, because other parts of our site will be using it. So
> > what I'd like would probably be some refactoring of the resolvers. For
> > the time being I subclassed RegexURLPattern and implemented custom
> > resolve method -- it is hacky, but works straight away when I put my
> > subclass in url_patterns. The problem is, there seems to be no easy
> > way, to make reverse work. I was thinking about all of it a lot, also
> > googling a lot and I found no answers. So I wonder: what's the best
> > way to make database URL resolving play nicely with the default regex
> > resolving? Is there any way at all?
>
> > With regards,
> > Marek Bernat
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Replacing get_absolute_url, I am against it

[Simon Willison]

On Sep 15, 12:04 pm, Ivan Sagalaev  wrote:
> James Bennett wrote:
> > Except I can't help thinking this is an awfully arbitrary distinction
> > to draw. In effect you're saying that nearly every question about an
> > object should be answerable by interrogating it directly, *except* for
> > "what's a URL I can use for you?"
>
> May be I can explain this distinction with an example.
>
> We once had two different web sites about "events". They both had a
> "core" set of models but each one had their own set of views & urls. So
> for a core Event model a question "what's your URL" just didn't make
> sense. It had two different URLs depending on the project it was
> imported in.

For me, this one boils down to pragmatism v.s. purity.

>From a purity point of view, having models that know what their URLs
are when Django's concept of a URL is defined in the urls.py file is
an obvious violation of separation of concerns.

>From a pragmatism point of view, being able to ask an object it's URL
is incredibly convenient - and in 90% of the projects I've worked on
in the past few years each object has had one and only one URL (in
fact REST principles actively encourage this).

I'm OK with .get_url() and .get_url_path() (I still prefer the
semantics and names used in my original proposal to any of the
proposed alternatives) only being useful 90% of the times, with
projects where objects can have more than one URL needing to find some
alternative method.

As for running code, allow me to point to http://code.google.com/p/django-urls/
:)

Cheers,

Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: SoC merge plans?

[Simon Willison]

On Sep 13, 12:52 pm, Russell Keith-Magee 
wrote:
> Although the branch isn't ready for merge yet, the code that is
> present is all up to spec. I think the sprints have produced workable
> solutions for the outstanding problems; fixes implementing the
> solutions we discussed should be forthcoming shortly.

With my cowboy hat on... what are the disadvantages to merging this to
trunk /now/ and fixing the remaining outstanding issues there? It
might make it easier to get the other SoC branches compatible with
this one (which sounds to me like it makes the most far-reaching
changes) and it would encourage the wider community to help check that
everything really is backwards compatible.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Proposal for 1.2: built-in logging with django.core.log

[Simon Willison]

I think we should add logging to Django in version 1.2, implemented as
a light-weight wrapper around the Python logging module
(django.core.log maybe?) plus code to write errors to the Apache error
log under the mod_python handler and environ['wsgi.errors'] under WSGI
(meaning mod_wsgi will write to the Apache error log as well).

Benefits of logging as a core Django service


Adding logging to Django core would provide the following benefits:

1. We'll be able to de-emphasise the current default "e-mail all
errors to someone" behaviour, which doesn't scale at all well.

2. Having logging in the core framework will mean people start
actually using it, which will make it easier for people to debug their
Django apps. Right now adding "print" statements to a Django app is a
common debugging technique, but it's messy (you have to remember to
take them out again) and error prone - some production environments
throw errors if an app attempts to write to stdout. It's also not
obvious - many developers are surprised when I show them the
technique.

3. Logging in Django core rather than a 3rd party app will encourage
reusable applications to log things in a predictable way, standard
way.

4. 3rd party debugging tools such as the debug toolbar will be able to
hook in to Django's default logging behaviour. This could also lead to
plenty of additional 3rd party innovation - imagine a tool that looks
out for logged SQL that took longer than X seconds, or one that groups
together similar log messages, or streams log messages to IRC...

5. Built-in support for logging reflects a growing reality of modern
Web development: more and more sites have interfaces with external web
service APIs, meaning there are plenty of things that could go wrong
that are outside the control of the developer. Failing gracefully and
logging what happened is the best way to deal with 3rd party problems
- much better than throwing a 500 and leaving no record of what went
wrong.

6. Most importantly from my point of view, when a sysadmin asks where
Django logs errors in production we'll have a good answer for them!

7. As a general rule, I believe you can never have too much
information about what's going on with your web application. I've
never thought to myself "the problem with this bug is I've got too
much information about it". As for large log files, disk space is
cheap - and pluggable backends could ensure logs were sensibly
rotated.

Places logging would be useful
==

- Unhandled exceptions that make it up to the top of the Django stack
(and would cause a 500 error to be returned in production)
- The development web server could use logging for showing processed
requests (where currently these are just printed to stdout).
- Failed attempts at signing in to the admin could be logged, making
security audits easier.
- We could replace (or complement) django.connection.queries with a
log of executed SQL. This would make the answer to the common question
"how do I see what SQL is being executed" much more obvious.
- Stuff that loads things from INSTALLED_APPS could log what is being
loaded, making it much easier to spot and debug errors caused by code
being incorrectly loaded.
- Likewise, the template engine could log which templates are being
loaded from where, making it easier to debug problems stemming from an
incorrectly configured TEMPLATE_DIRS setting.
- We could use logging to address the problems with the template
engine failing silently - maybe some template errors (the ones more
likely to be accidental than just people relying on the fail-silent
behaviour deliberately) should be logged as warnings.

Most of the above would be set to a low log level which by default
would not be handled, displayed or stored anywhere (logging.info or
similar). Maybe "./manage.py runserver --loglevel=info" could cause
such logs to be printed to  the terminal while the development server
is running.

Problems and challenges
===

1. The Python logging module isn't very nicely designed - its Java
heritage shines through, and things like logging.basicConfig behave in
unintuitive ways (if you call basicConfig twice the second call fails
silently but has no effect). This is why I suggest wrapping it in our
own higher level interface.

2. There may be some performance overhead, especially if we replace
mechanisms like django.connection.queries with logging. This should be
negligble: here's a simple benchmark:

# ("hello " * 100) gives a 600 char string, long enough for a SQL
statement
>>> import timeit, logging
>>> t = timeit.Timer('logging.info("hello " * 100)', 'import logging')
>>> t.timeit(number=100) # one hundred statements
0.00061702728271484375
>>> t.timeit(number=100) # one million statements
6.458014965057373

That's 0.0006 of a second overhead for a page logging 100 SQL
statements. The performance overhead will go up if you attach a
handler, but that's 

Re: CSRF proposal and patch ready for review

[Simon Willison]

On Sep 15, 2:57 pm, Luke Plant  wrote:
> OK, I'll wait and see.

Here's the code: http://github.com/simonw/django-safeform

>  * it requires using Django's form system.  I've got plenty of views that
> don't (e.g. anything with a dynamic number of controls).  People not using
> django.Forms are left out in the cold, or having to learn another way to do
> things.

I've covered this in django-safeform by exposing a low level interface
to the CSRF token creation and validation routines (in csrf_utils) -
see the readme for documentation on this.

>  * it's off by default.  Turning it on by default will require making
> django.forms.Form subclass SafeForm, which will almost certainly require a
> huge and immediate upgrade effort, because SafeForm cannot have the same API
> as Form.  If it's off by default, I see no point at all in this entire
> exercise.  If we don't arrive at a point where the POST form created in the
> tutorial is safe from CSRF, I think we've failed.

My priority for this is a little different: while I would dearly love
to see all Django applications secure against CSRF by default, I can't
see a way of doing it that doesn't break existing apps. More important
for me though is that the Django admin (and other reusable apps, such
as Pinax stuff) MUST be secure against CSRF out of the box, no matter
what the user puts in their settings.py file. If developers fail to
protect themselves (especially when the solution is well documented)
then at least it isn't our fault.

I think this is subtly different from the XSS escaping issue simply
because the solution to CSRF is much more intrusive.

> And it will still require changes to templates, just like the template tag,
> and it will also require changes to views, only significantly more complex  
> than simply using RequestContext.  It's got at least as many and at least as
> intrusive 'moving parts' AFAICS.

The SafeForm API actually simplifies views - you go from this:

def change_password(request):
form = ChangePasswordForm()
if request.method == 'POST':
form = ChangePasswordForm(request.POST)
if form.is_valid():
return HttpResponse('Thank you')
return render_to_response('change_password.html', {
'form': form,
})

To this:

@csrf_protect
def change_password(request):
form = ChangePasswordForm(request)
if form.is_valid():
return HttpResponse('Thank you')
return render_to_response('change_password.html', {
'form': form,
})

The SafeForm deals with the request.method == 'POST' bit, meaning one
less conditional branch within the view function itself.

I'm pretty happy with the SafeForm interface now that I've fleshed it
out - it's a lot less clunky than I was originally expecting.

Cheers,

Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---