Re: GSoC 2012: Security Enhancements

2012-04-23 Thread Paul McMillan 
On Wed, Apr 18, 2012 at 3:50 PM, Luke Plant  wrote:
> One query: are you sure it is harder to manipulate? In particular, I
> remember from a while back that Flash allowed some headers to be
> manipulated, which caused problems, and they fixed it by blacklisting
> some headers, I think including referer. Did they also fix Origin?

In very old browsers and very old versions of flash*, the origin
header isn't on the blacklist. This is why I propose to only use it as
a negative signal (if present, and does not match, fail) rather than
also as a positive signal. In a year or two (maybe 2014 when IE6 is
finally, truly, laid to rest), we can revisit the idea of using it as
a positive signal.

-Paul

* It has been patched in version 7 and onward. If you are running
unpatched flash, you probably have so many viruses it doesn't
matter... 
http://helpx.adobe.com/flash-player/kb/actionscript-error-send-action-contains.html

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

2012-04-20 Thread Rohan Jain 
On 16:03 +0100 / 18 Apr, Luke Plant wrote:
> On 15/04/12 05:23, Rohan Jain wrote:
> > On 22:50 +0100 / 13 Apr, Luke Plant wrote:
> >> The reason for the strict referer checking under HTTPS is set out here:
> >>
> >> https://code.djangoproject.com/wiki/CsrfProtection
> >>
> >> Particularly, it is to fix the 'CSRF + MITM' attack that is possible
> >> under HTTPS. The key elements are set out in the following scenario
> >> (although it is not the only variation):
> >>
> >>  - a client connects to a site via HTTP: http://example.com/
> >>  - in the response, an active MITM attacker sets a cookie over
> >>HTTP for example.com
> >>  - this cookie will be used by the client when it connects over HTTPS
> >>to the same domain (this is the fundamental problem, but we can't
> >>fix it - it's what browsers do).
> >>  - the MITM also inserts a POST form in the HTTP response.
> >>The form has a CSRF token that matches the cookie that was
> >>set by the attacker.
> >>The forms targets https://example.com/ and is automatically
> >>submitted by javascript.
> >>
> >> Without strict referer checking, the POST request will succeed, even
> >> though it is forged.
> >>
> >> Signing the cookie or token value does no good at all, because the
> >> attacker can retrieve a valid cookie/token pair directly from
> >> https://example.com/.
> > 
> > I am a bit confused about this. How can an attacker extract the token
> > out of the signed cookie without the private key? There is a
> > possibility that the attacker deletes and sets the cookie right out,
> > but since that case won't be correctly signed server can handle it
> > securely.
> 
> They don't need to extract the token, they just need to replay an
> existing good token/cookie pair, which they can get directly from the
> server any time they want.
> 
> (Also, signing does nothing to hide the token. Are you actually talking
> about encryption here? But even if you encrypt it, it doesn't help - the
> only thing that matters is that the CSRF form token 'matches' the CSRF
> cookie, whatever your definition of 'match', and that can be achieved by
> getting the pair from the server.)

Sorry, my bad. I didn't realize swaying towards encryption. Encryption
is something the SoC Ideas page suggests against for CSRF. I am no
crypto expert so shouldn't waste time over this I guess.

--
Rohan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

2012-04-19 Thread Rohan Jain 
I hosted a simple app which responds with the request details for
testing purposes:
https://request-mirror.herokuapp.com/
(source: https://github.com/crodjer/request-mirror)

On 12:05 -0700 / 18 Apr, Paul McMillan wrote:
> There seems to be  some confusion about CORS (a hairy draft spec that
> is not fully implemented in any browser, and not appropriate for
> inclusion in Django at this time) and the "Origin" header (aka Web
> Origin, rfc6454).
> 
> http://tools.ietf.org/html/rfc6454
> https://wiki.mozilla.org/Security/Origin
> http://www.w3.org/TR/access-control/#origin-request-header
> 
> The Origin header defined in rfc6454 is compatible with the CORS
> origin header, but does not require full CORS support from the browser
> or the server to be useful.
> 
> In my tests with Firefox and Chrome, both sent the origin header
> properly when making potentially state-changing cross-origin requests
> (both AJAX and standard form POST requests).
> 

How did you test the form POST requests with firefox?

For me, with a normal POST, Iceweasel (firefox for debian) 10 does not
seem to be sending any origin headers but Chrome (18) is.
MDN says that this should work firefox >= 3.5
(https://developer.mozilla.org/en/http_access_control#The_HTTP_request_headers)
and the Origin header should be present:
https://wiki.mozilla.org/Security/Origin#Origin_header_format

I tried testing these cross domain (between the secure remote and
local, between local aliases) through HTTP Post, but the browser never
complained or did a Preflighted request (HTTP OPTIONS request) or sent
back the origin header.

But it apparently isn't with mine (Hopefully I am doing something
wrong here)

> My suggestion here is to include optional support for the Origin
> header as follows:
> - if present and null, fail the CSRF check
> - if present and not null, use in alongside the Referer header
> - if absent, keep current behavior
> 
> As a general rule, if a browser sends an origin header, that value is
> more reliable (harder for malicious sites to manipulate, less often
> stripped by firewalls, less often disabled by users) than the referer
> header. This addition won't improve CSRF protection for older
> browsers, but it also won't break anything for them. For users with
> newer browsers, it should prevent CSRF even in cases when the CSRF
> token is stolen due to misconfiguration or user error.
> 
> -Paul

I second this. The selective origin checking, though not completely,
will improve CSRF protection for some clients. We can then be sure
about an automatic increase in security with the shift towards such
clients.

--
Rohan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

2012-04-18 Thread Luke Plant 
On 18/04/12 20:05, Paul McMillan wrote:

> My suggestion here is to include optional support for the Origin
> header as follows:
> - if present and null, fail the CSRF check
> - if present and not null, use in alongside the Referer header
> - if absent, keep current behavior
> 
> As a general rule, if a browser sends an origin header, that value is
> more reliable (harder for malicious sites to manipulate, less often
> stripped by firewalls, less often disabled by users) than the referer
> header. This addition won't improve CSRF protection for older
> browsers, but it also won't break anything for them. For users with
> newer browsers, it should prevent CSRF even in cases when the CSRF
> token is stolen due to misconfiguration or user error.

That does sound like a useful additional behaviour.

One query: are you sure it is harder to manipulate? In particular, I
remember from a while back that Flash allowed some headers to be
manipulated, which caused problems, and they fixed it by blacklisting
some headers, I think including referer. Did they also fix Origin?

The vulnerability I'm thinking about is this one I think:

http://lists.webappsec.org/pipermail/websecurity_lists.webappsec.org/2011-February/007533.html

Luke


-- 
"My capacity for happiness you could fit into a matchbox without
taking out the matches first." (Marvin the paranoid android)

Luke Plant || http://lukeplant.me.uk/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

2012-04-18 Thread Paul McMillan 
There seems to be  some confusion about CORS (a hairy draft spec that
is not fully implemented in any browser, and not appropriate for
inclusion in Django at this time) and the "Origin" header (aka Web
Origin, rfc6454).

http://tools.ietf.org/html/rfc6454
https://wiki.mozilla.org/Security/Origin
http://www.w3.org/TR/access-control/#origin-request-header

The Origin header defined in rfc6454 is compatible with the CORS
origin header, but does not require full CORS support from the browser
or the server to be useful.

In my tests with Firefox and Chrome, both sent the origin header
properly when making potentially state-changing cross-origin requests
(both AJAX and standard form POST requests).

My suggestion here is to include optional support for the Origin
header as follows:
- if present and null, fail the CSRF check
- if present and not null, use in alongside the Referer header
- if absent, keep current behavior

As a general rule, if a browser sends an origin header, that value is
more reliable (harder for malicious sites to manipulate, less often
stripped by firewalls, less often disabled by users) than the referer
header. This addition won't improve CSRF protection for older
browsers, but it also won't break anything for them. For users with
newer browsers, it should prevent CSRF even in cases when the CSRF
token is stolen due to misconfiguration or user error.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

2012-04-18 Thread Luke Plant 
Sorry to reply twice, a comment on a different part:

On 15/04/12 05:23, Rohan Jain wrote:
> On 22:50 +0100 / 13 Apr, Luke Plant wrote:
>> .. At the moment, it seems that few browsers send the
>> 'Origin' header for normal HTML requests. (Recent versions of Chrome,
>> Firefox and Opera do not, I don't know about IE).
> 
> Page, http://caniuse.com/cors mentions the browsers and their versions
> which support CORS. A big share of browser does support it and another
> big one (constituting old IE and Opera) does not. We cannot expect
> these browsers to go away anytime soon, so we have to keep maintaining
> a compatibility system.
> 
> Since http referrer is already being used in case of secure requests,
> how about something similar in normal requests? Isn't the argument
> about referrer being absent only in 0.2% case or less valid here? Is
> 0.2% too significant for normal requests?

The page you linked to shows browsers that support CORS, which is quite
different from browsers that send Origin with normal HTTP requests (i.e.
non XMLHttpRequests requests) - in my own tests I can't find any browser
that sends it for normal HTML requests.

This means that it might be useful as an alternative way of addressing
the problem with AJAX, but even there I think it is of dubious value: if
we deploy a method that means that AJAX works automatically (without,
for example, the jQuery fix we include in our docs) on the basis of the
Origin header, everything will work fine in development, where most
people don't test IE regularly, and will fail for Internet Explorer in
production, which is not very nice. It is better to fail early.

Regards,

Luke

-- 
"My capacity for happiness you could fit into a matchbox without
taking out the matches first." (Marvin the paranoid android)

Luke Plant || http://lukeplant.me.uk/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

2012-04-18 Thread Luke Plant 
On 15/04/12 05:23, Rohan Jain wrote:
> On 22:50 +0100 / 13 Apr, Luke Plant wrote:
>> The reason for the strict referer checking under HTTPS is set out here:
>>
>> https://code.djangoproject.com/wiki/CsrfProtection
>>
>> Particularly, it is to fix the 'CSRF + MITM' attack that is possible
>> under HTTPS. The key elements are set out in the following scenario
>> (although it is not the only variation):
>>
>>  - a client connects to a site via HTTP: http://example.com/
>>  - in the response, an active MITM attacker sets a cookie over
>>HTTP for example.com
>>  - this cookie will be used by the client when it connects over HTTPS
>>to the same domain (this is the fundamental problem, but we can't
>>fix it - it's what browsers do).
>>  - the MITM also inserts a POST form in the HTTP response.
>>The form has a CSRF token that matches the cookie that was
>>set by the attacker.
>>The forms targets https://example.com/ and is automatically
>>submitted by javascript.
>>
>> Without strict referer checking, the POST request will succeed, even
>> though it is forged.
>>
>> Signing the cookie or token value does no good at all, because the
>> attacker can retrieve a valid cookie/token pair directly from
>> https://example.com/.
> 
> I am a bit confused about this. How can an attacker extract the token
> out of the signed cookie without the private key? There is a
> possibility that the attacker deletes and sets the cookie right out,
> but since that case won't be correctly signed server can handle it
> securely.

They don't need to extract the token, they just need to replay an
existing good token/cookie pair, which they can get directly from the
server any time they want.

(Also, signing does nothing to hide the token. Are you actually talking
about encryption here? But even if you encrypt it, it doesn't help - the
only thing that matters is that the CSRF form token 'matches' the CSRF
cookie, whatever your definition of 'match', and that can be achieved by
getting the pair from the server.)

Luke

-- 
"My capacity for happiness you could fit into a matchbox without
taking out the matches first." (Marvin the paranoid android)

Luke Plant || http://lukeplant.me.uk/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

2012-04-14 Thread Rohan Jain 
On 22:50 +0100 / 13 Apr, Luke Plant wrote:
> Hi Rohan,
> 
> Sorry for the slow reply on this one, I've had a busy time recently.
> Please see my comments on some parts of this proposal.

No worries about this.
> 
> On 31/03/12 19:10, Rohan Jain wrote:
> > Hi,
> > 
> > I am Rohan Jain, a 4th (final) year B.Tech undergraduate Student from
> > Indian Institute of Technology, Kharagpur. I have been using django
> > since over a year and generally look into the code base to find about
> > various implementations. I have made attempts to make some minor 
> > contributions and if selected this would be my first major one.
> > 
> > More about Me:  IRC, Github: crodjer
> > 
> > I am interested in contributing some security enhancements to django 
> > as my Summer of Code project. Below is the 1st draft of my proposal 
> > regarding this. A pretty version of this is available at: 
> > https://gist.github.com/2203174
> > 
> > 
> > #Abstract
> > 
> > Django is a reasonably secure framework. It provides an API and 
> > development patterns which transparently take care of the common web 
> > security issues. But still there are security features which need 
> > attention. I propose to work on integration of existing work on 
> > centralized token system and improved CSRF checking without any 
> > compromises. If time permits I will also attempt on integration of 
> > django-secure.
> > 
> > #Description ##Centralized tokenization There are multiple places in
> > django which use some or other kinds of tokens:
> > 
> > - contirb.auth (random password, password reset) - formtools -
> > session (backends) - cache - csrf - etags
> > 
> > Token generation is pretty common around the framework.  So, instead 
> > of each application having its own token system, and hence needs to
> > be maintained separately. There should be centralized token system,
> > which provides an abstract API for everyone to loose. In fact, I have
> > seen that some apps use `User.objects.make_random_password` from 
> > contrib.auth, which they can be sure of being maintained in the
> > future for random generation. To me this looks kind of weird. In last
> > djangocon, a lot of work regarding this was done over [Yarko's 
> > Fork][yarko-fork].
> > 
> > I had a discussion with Yarko Tymciurak regarding this. The work is 
> > nearly ready for a merge, only some tasks left. In the initial
> > period my SoC I can work over these to insure that the already done 
> > significant work gets in django and is updated for 1.5.
> > 
> > - Porting more stuff to the new system (README.sec in [yarko's
> > fork][yarko-fork]) - Testing - See if the current coverage of the
> > tests is enough, write them if not. - Compatibility issues - API
> > Documentation
> > 
> > I will study the changes done at djangocon and then attempt the
> > tasks mentioned above.
> > 
> > ##CSRF Improvements
> > 
> > Cross-Origin Resource Sharing (CORS): W3C has a working draft
> > regarding [CORS][w3c-cors-draft], which opens up the possibility for
> > allowing client-side request cross-origin requests. This directly
> > triggers in mind the capability to develop API which can be exposed
> > directly to the web browser. This would let us get rid of proxies and
> > other hacks used to achieve this. Currently all the major browsers
> > support this: Chrome (all versions), Firefox (> 3.0), IE (> 7.0),
> > Safari (> 3.2), Opera (> 12.0). Introduced it here as some further
> > parts of the post refer to this.
> > 
> > ###Origin checking
> > 
> > With CORS around need for using CSRF token can be dropped, at least
> > in some browsers. [Ticket #16859][orig-check-ticket], is an attempt
> > for that. But this was rejected because of neglecting the case for 
> > presence of `CSRF_COOKE_DOMAIN` (Refer to the closing comment on the 
> > ticket for details). So to handle this we need to simulate checking
> > of CSRF cookie domain as web browsers do it. Maybe:
> > 
> > ```python 
> > reqest.META.get('HTTP_ORIGIN').endswith(settings.CSRF_COOKIE_DOMAIN) 
> > ```
> 
> I'm very cautious about making the logic here more complex. It can be
> done, but every additional code path increases the possibility of a
> security hole. ..

Yes, this will touch critical code of the framework and cannot afford
to expose any possible vulnerabilities. That is why I am planning of a
thorough security level testing and of course any patch relating this
(or anything else in this security related proposal) will need careful
reviews before being considered safe.

> .. At the moment, it seems that few browsers send the
> 'Origin' header for normal HTML requests. (Recent versions of Chrome,
> Firefox and Opera do not, I don't know about IE).

Page, http://caniuse.com/cors mentions the browsers and their versions
which support CORS. A big share of browser does support it and another
big one (constituting old IE and Opera) does not. We cannot expect
these browsers to go away anytime soon, so we have t

Re: GSoC 2012: Security Enhancements

2012-04-13 Thread Luke Plant 
Hi Rohan,

Sorry for the slow reply on this one, I've had a busy time recently.
Please see my comments on some parts of this proposal.

On 31/03/12 19:10, Rohan Jain wrote:
> Hi,
> 
> I am Rohan Jain, a 4th (final) year B.Tech undergraduate Student from
> Indian Institute of Technology, Kharagpur. I have been using django
> since over a year and generally look into the code base to find about
> various implementations. I have made attempts to make some minor 
> contributions and if selected this would be my first major one.
> 
> More about Me:  IRC, Github: crodjer
> 
> I am interested in contributing some security enhancements to django 
> as my Summer of Code project. Below is the 1st draft of my proposal 
> regarding this. A pretty version of this is available at: 
> https://gist.github.com/2203174
> 
> 
> #Abstract
> 
> Django is a reasonably secure framework. It provides an API and 
> development patterns which transparently take care of the common web 
> security issues. But still there are security features which need 
> attention. I propose to work on integration of existing work on 
> centralized token system and improved CSRF checking without any 
> compromises. If time permits I will also attempt on integration of 
> django-secure.
> 
> #Description ##Centralized tokenization There are multiple places in
> django which use some or other kinds of tokens:
> 
> - contirb.auth (random password, password reset) - formtools -
> session (backends) - cache - csrf - etags
> 
> Token generation is pretty common around the framework.  So, instead 
> of each application having its own token system, and hence needs to
> be maintained separately. There should be centralized token system,
> which provides an abstract API for everyone to loose. In fact, I have
> seen that some apps use `User.objects.make_random_password` from 
> contrib.auth, which they can be sure of being maintained in the
> future for random generation. To me this looks kind of weird. In last
> djangocon, a lot of work regarding this was done over [Yarko's 
> Fork][yarko-fork].
> 
> I had a discussion with Yarko Tymciurak regarding this. The work is 
> nearly ready for a merge, only some tasks left. In the initial
> period my SoC I can work over these to insure that the already done 
> significant work gets in django and is updated for 1.5.
> 
> - Porting more stuff to the new system (README.sec in [yarko's
> fork][yarko-fork]) - Testing - See if the current coverage of the
> tests is enough, write them if not. - Compatibility issues - API
> Documentation
> 
> I will study the changes done at djangocon and then attempt the
> tasks mentioned above.
> 
> ##CSRF Improvements
> 
> Cross-Origin Resource Sharing (CORS): W3C has a working draft
> regarding [CORS][w3c-cors-draft], which opens up the possibility for
> allowing client-side request cross-origin requests. This directly
> triggers in mind the capability to develop API which can be exposed
> directly to the web browser. This would let us get rid of proxies and
> other hacks used to achieve this. Currently all the major browsers
> support this: Chrome (all versions), Firefox (> 3.0), IE (> 7.0),
> Safari (> 3.2), Opera (> 12.0). Introduced it here as some further
> parts of the post refer to this.
> 
> ###Origin checking
> 
> With CORS around need for using CSRF token can be dropped, at least
> in some browsers. [Ticket #16859][orig-check-ticket], is an attempt
> for that. But this was rejected because of neglecting the case for 
> presence of `CSRF_COOKE_DOMAIN` (Refer to the closing comment on the 
> ticket for details). So to handle this we need to simulate checking
> of CSRF cookie domain as web browsers do it. Maybe:
> 
> ```python 
> reqest.META.get('HTTP_ORIGIN').endswith(settings.CSRF_COOKIE_DOMAIN) 
> ```

I'm very cautious about making the logic here more complex. It can be
done, but every additional code path increases the possibility of a
security hole. At the moment, it seems that few browsers send the
'Origin' header for normal HTML requests. (Recent versions of Chrome,
Firefox and Opera do not, I don't know about IE).

> I would also suggest making CSRF cookie as http only. There doesn't 
> seem a reason currently why the cookies would be needed to be
> accessed in browser.

Currently the CSRF cookie has to be accessible in browser if the AJAX
workaround in our docs is to work at all:

https://docs.djangoproject.com/en/dev/ref/contrib/csrf/#ajax



> ###Less restrictive secure requests
> 
> The current CSRF system is pretty much secure as it is. But CSRF 
> protection poses too much restriction to https. It says no to all
> the request, without honouring any tokens. It kind of has to, thanks
> to the way browsers allow cookie access. A cookie accessible through 
> subdomains mean that any subdomain secure or insecure can set the
> CSRF token, which could be really serious for the site security. To
> get around this, currently one has to comple

Re: GSoC 2012: Security Enhancements

2012-04-06 Thread Rohan Jain 
Hi Russel,

That is a good news for me. I have added a timeline and posted it over
melange.

Public Gist for the same: https://gist.github.com/2203174

-- Rohan


On 16:14 +0800 /  6 Apr, Russell Keith-Magee wrote:
> 
> On 06/04/2012, at 3:54 PM, Rohan Jain wrote:
> 
> > Hi Russel,
> > 
> > Thanks for the reply.
> > 
> > On 14:42 +0800 /  6 Apr, Russell Keith-Magee wrote:
> >> 
> >> Hi Rohan,
> >> 
> >> Apologies for the lack of response. Anyone who has put effort into writing 
> >> up a proposal certainly deserves a response of some kind, so we've dropped 
> >> the ball here.
> >> 
> >> In our defence, here's a couple of the reasons why your proposal probably 
> >> hasn't got a wild response:
> >> 
> >> * You've picked a project on your own, rather than one that is on the list 
> >> of suggested projects. Part of the GSoC process is mentoring, and you need 
> >> to have a mentor that can spare the time, and has the technical skills to 
> >> review your work. Most of the projects on the suggested list already have 
> >> champions inside the community, so if you pick one of them, you're likely 
> >> to get a response. If you pick your own project, you also have to find 
> >> someone to get sufficiently enthused about it. 
> > 
> > I had picked the project from the list of suggested projects
> > (https://code.djangoproject.com/wiki/SummerOfCode2012#SecurityEnhancements).
> > In fact, I have tried to take on each of the issues mentioned there,
> > point by point. I did talk to Paul over the IRC about this.
> 
> Ah. Now I look a little stupid :-)
> 
> That project has evidently been added to the list since I last looked at the 
> list. Apologies for not keeping up to date.
> 
> >> Regarding your project proposal itself: I can't really address the 
> >> technical merits, because I don't have any expertise on CORS, or the 
> >> subtleties of the CSRF changes your proposing. What you've proposed 
> >> certainly sounds interesting on the surface, but I'd really want to see 
> >> someone like Paul McMillan comment on the technical specifics. Ideally, 
> >> Paul would also mentor the project, since he's Django's resident security 
> >> expert, and he'd need to sign off on anything that was bound for trunk.
> > 
> > I had some discussions with Paul over this earlier, but since a week
> > days I haven't been able to find him online.
> 
> It's good that you've discussed this with Paul -- he's certainly the man to 
> speak to.
> 
> >> So - apologies for not responding sooner. Unfortunately, I suspect that 
> >> while your project probably has merit, the community isn't in a position 
> >> to support your ambition at the moment. If Paul wants to swoop in at the 
> >> last minute and prove me wrong, I'd be a very happy man -- I hate seeing 
> >> someone enthusiastic get turned away -- but absent of that, it's only fair 
> >> that we be honest to you about your chances.
> > 
> > No worries. I am at a blame too for my own activity. I myself have
> > learnt a lot of interesting stuff from this. I would anyway be glad to
> > give something back to django.
> 
> The fact that this project *is* in fact on our suggested projects list means 
> I probably have to alter my advice. 
> 
> If you're still interested in doing this project, it's worth submitting it. 
> You've clearly put in a bunch of effort already, and it would be a pity for 
> that to go to waste. As an organization, we have a certain amount of 
> discretion as to the projects we pick, and if someone is clearly working with 
> the community to improve API or sort out fine details, we may be able to 
> overlook small problems with an "official" application. However, the 
> "official" application still needs to be submitted in order for the GSoC 
> process to go ahead. What you've got here looks solid enough to demonstrate 
> you've thought about the problem and have an good idea of how you're going to 
> solve it.
> 
> That said, ultimate selection of the project will probably have more to do 
> with Paul's availability and enthusiasm to mentor than the technical aspects 
> of your proposal.
> 
> All I would suggest is:
> 
>  * Add a detailed work breakdown (granularity ~1 week)
> 
>  * Push the "merging" activities to the status of "stuff I'll do if the 
> schedule allows at the end of the project". That removes the emphasis from 
> the merging activity, making it look less suspect from the GSoC process 
> point-of-view.
> 
>  * Submit the proposal, and hope Paul is watching this thread :-)
> 
> Yours,
> Russ Magee %-)
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
> 

-- 
You received this message because you are subscribed to t

Re: GSoC 2012: Security Enhancements

2012-04-06 Thread Russell Keith-Magee 

On 06/04/2012, at 3:54 PM, Rohan Jain wrote:

> Hi Russel,
> 
> Thanks for the reply.
> 
> On 14:42 +0800 /  6 Apr, Russell Keith-Magee wrote:
>> 
>> Hi Rohan,
>> 
>> Apologies for the lack of response. Anyone who has put effort into writing 
>> up a proposal certainly deserves a response of some kind, so we've dropped 
>> the ball here.
>> 
>> In our defence, here's a couple of the reasons why your proposal probably 
>> hasn't got a wild response:
>> 
>> * You've picked a project on your own, rather than one that is on the list 
>> of suggested projects. Part of the GSoC process is mentoring, and you need 
>> to have a mentor that can spare the time, and has the technical skills to 
>> review your work. Most of the projects on the suggested list already have 
>> champions inside the community, so if you pick one of them, you're likely to 
>> get a response. If you pick your own project, you also have to find someone 
>> to get sufficiently enthused about it. 
> 
> I had picked the project from the list of suggested projects
> (https://code.djangoproject.com/wiki/SummerOfCode2012#SecurityEnhancements).
> In fact, I have tried to take on each of the issues mentioned there,
> point by point. I did talk to Paul over the IRC about this.

Ah. Now I look a little stupid :-)

That project has evidently been added to the list since I last looked at the 
list. Apologies for not keeping up to date.

>> Regarding your project proposal itself: I can't really address the technical 
>> merits, because I don't have any expertise on CORS, or the subtleties of the 
>> CSRF changes your proposing. What you've proposed certainly sounds 
>> interesting on the surface, but I'd really want to see someone like Paul 
>> McMillan comment on the technical specifics. Ideally, Paul would also mentor 
>> the project, since he's Django's resident security expert, and he'd need to 
>> sign off on anything that was bound for trunk.
> 
> I had some discussions with Paul over this earlier, but since a week
> days I haven't been able to find him online.

It's good that you've discussed this with Paul -- he's certainly the man to 
speak to.

>> So - apologies for not responding sooner. Unfortunately, I suspect that 
>> while your project probably has merit, the community isn't in a position to 
>> support your ambition at the moment. If Paul wants to swoop in at the last 
>> minute and prove me wrong, I'd be a very happy man -- I hate seeing someone 
>> enthusiastic get turned away -- but absent of that, it's only fair that we 
>> be honest to you about your chances.
> 
> No worries. I am at a blame too for my own activity. I myself have
> learnt a lot of interesting stuff from this. I would anyway be glad to
> give something back to django.

The fact that this project *is* in fact on our suggested projects list means I 
probably have to alter my advice. 

If you're still interested in doing this project, it's worth submitting it. 
You've clearly put in a bunch of effort already, and it would be a pity for 
that to go to waste. As an organization, we have a certain amount of discretion 
as to the projects we pick, and if someone is clearly working with the 
community to improve API or sort out fine details, we may be able to overlook 
small problems with an "official" application. However, the "official" 
application still needs to be submitted in order for the GSoC process to go 
ahead. What you've got here looks solid enough to demonstrate you've thought 
about the problem and have an good idea of how you're going to solve it.

That said, ultimate selection of the project will probably have more to do with 
Paul's availability and enthusiasm to mentor than the technical aspects of your 
proposal.

All I would suggest is:

 * Add a detailed work breakdown (granularity ~1 week)

 * Push the "merging" activities to the status of "stuff I'll do if the 
schedule allows at the end of the project". That removes the emphasis from the 
merging activity, making it look less suspect from the GSoC process 
point-of-view.

 * Submit the proposal, and hope Paul is watching this thread :-)

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC 2012: Security Enhancements

2012-04-06 Thread Rohan Jain 
Hi Russel,

Thanks for the reply.

On 14:42 +0800 /  6 Apr, Russell Keith-Magee wrote:
> 
> Hi Rohan,
> 
> Apologies for the lack of response. Anyone who has put effort into writing up 
> a proposal certainly deserves a response of some kind, so we've dropped the 
> ball here.
> 
> In our defence, here's a couple of the reasons why your proposal probably 
> hasn't got a wild response:
>  
>  * You've picked a project on your own, rather than one that is on the list 
> of suggested projects. Part of the GSoC process is mentoring, and you need to 
> have a mentor that can spare the time, and has the technical skills to review 
> your work. Most of the projects on the suggested list already have champions 
> inside the community, so if you pick one of them, you're likely to get a 
> response. If you pick your own project, you also have to find someone to get 
> sufficiently enthused about it. 

I had picked the project from the list of suggested projects
(https://code.djangoproject.com/wiki/SummerOfCode2012#SecurityEnhancements).
In fact, I have tried to take on each of the issues mentioned there,
point by point. I did talk to Paul over the IRC about this.

> 
>  * You've picked a very gnarly problem. Security issues are the very model of 
> an 'anti-bikeshed'. If you read the original discussion about Bikesheds [1], 
> it's all about how everyone gives their opinion on "simple" topics, but 
> everyone leaves hard problems alone. In Django's context -- *everyone* has an 
> opinion about contrib.auth.User because it seems like a simple problem. 
> However, security is all about subtle issues and expert knoweledge. 
> Therefore, your pool of experts is much smaller.  
> 
> [1] http://bikeshed.com/

I recently generated some interest in this field, so picked up what I
would like to work on the best. For being in field of web development,
I believe it is necessary to have a good understanding of security.

> 
> Regarding your project proposal itself: I can't really address the technical 
> merits, because I don't have any expertise on CORS, or the subtleties of the 
> CSRF changes your proposing. What you've proposed certainly sounds 
> interesting on the surface, but I'd really want to see someone like Paul 
> McMillan comment on the technical specifics. Ideally, Paul would also mentor 
> the project, since he's Django's resident security expert, and he'd need to 
> sign off on anything that was bound for trunk.

I had some discussions with Paul over this earlier, but since a week
days I haven't been able to find him online.

> 
> What I can do is point at the things that look like problems from a GSoC 
> perspective. In particular, of the three sections to your project plan, two 
> of them (unified tokenization and django-secure) involve merging existing 
> projects into trunk. This is problematic, because one of the conditions of 
> GSoC is that the student writes the bulk of the code. While integrating these 
> two code pools may well be very valuable contributions to Django, they're not 
> good from the perspective of a GSoC project.

Yes, these two will be more calling out to my `git merge` skills. I
was assuming the most of my time would have spanned over CSRF
improvements, while using some functionality from centralized token
generation.

> 
> So - apologies for not responding sooner. Unfortunately, I suspect that while 
> your project probably has merit, the community isn't in a position to support 
> your ambition at the moment. If Paul wants to swoop in at the last minute and 
> prove me wrong, I'd be a very happy man -- I hate seeing someone enthusiastic 
> get turned away -- but absent of that, it's only fair that we be honest to 
> you about your chances.

No worries. I am at a blame too for my own activity. I myself have
learnt a lot of interesting stuff from this. I would anyway be glad to
give something back to django.

-- Rohan

> 
> Yours,
> Russ Magee %-)
> 
> On 06/04/2012, at 2:09 PM, Rohan Jain wrote:
> 
> > Hi again,
> > 
> > I really couldn't understand the response this post has got. It
> > deserved at least a little feedback, positive or negative. I guess I
> > wont be submitting this over melange.
> > 
> > Still, I have put some effort and research in the proposal. So if
> > possible I would like to know if it had anything of value. Maybe some
> > one could work over that, even me if I get the time.
> > 
> > --
> > Rohan
> > 
> > 
> > On 23:40 +0530 / 31 Mar, Rohan Jain wrote:
> >> Hi,
> >> 
> >> I am Rohan Jain, a 4th (final) year B.Tech undergraduate Student
> >> from Indian Institute of Technology, Kharagpur. I have been using
> >> django since over a year and generally look into the code base to find
> >> about various implementations. I have made attempts to make some minor
> >> contributions and if selected this would be my first major one.
> >> 
> >> More about Me: 
> >> IRC, Github: crodjer
> >> 
> >> I am interested in contributing some security

Re: GSoC 2012: Security Enhancements

2012-04-05 Thread Russell Keith-Magee 

Hi Rohan,

Apologies for the lack of response. Anyone who has put effort into writing up a 
proposal certainly deserves a response of some kind, so we've dropped the ball 
here.

In our defence, here's a couple of the reasons why your proposal probably 
hasn't got a wild response:
 
 * You've picked a project on your own, rather than one that is on the list of 
suggested projects. Part of the GSoC process is mentoring, and you need to have 
a mentor that can spare the time, and has the technical skills to review your 
work. Most of the projects on the suggested list already have champions inside 
the community, so if you pick one of them, you're likely to get a response. If 
you pick your own project, you also have to find someone to get sufficiently 
enthused about it. 

 * You've picked a very gnarly problem. Security issues are the very model of 
an 'anti-bikeshed'. If you read the original discussion about Bikesheds [1], 
it's all about how everyone gives their opinion on "simple" topics, but 
everyone leaves hard problems alone. In Django's context -- *everyone* has an 
opinion about contrib.auth.User because it seems like a simple problem. 
However, security is all about subtle issues and expert knoweledge. Therefore, 
your pool of experts is much smaller.  

[1] http://bikeshed.com/

Regarding your project proposal itself: I can't really address the technical 
merits, because I don't have any expertise on CORS, or the subtleties of the 
CSRF changes your proposing. What you've proposed certainly sounds interesting 
on the surface, but I'd really want to see someone like Paul McMillan comment 
on the technical specifics. Ideally, Paul would also mentor the project, since 
he's Django's resident security expert, and he'd need to sign off on anything 
that was bound for trunk.

What I can do is point at the things that look like problems from a GSoC 
perspective. In particular, of the three sections to your project plan, two of 
them (unified tokenization and django-secure) involve merging existing projects 
into trunk. This is problematic, because one of the conditions of GSoC is that 
the student writes the bulk of the code. While integrating these two code pools 
may well be very valuable contributions to Django, they're not good from the 
perspective of a GSoC project.

So - apologies for not responding sooner. Unfortunately, I suspect that while 
your project probably has merit, the community isn't in a position to support 
your ambition at the moment. If Paul wants to swoop in at the last minute and 
prove me wrong, I'd be a very happy man -- I hate seeing someone enthusiastic 
get turned away -- but absent of that, it's only fair that we be honest to you 
about your chances.

Yours,
Russ Magee %-)

On 06/04/2012, at 2:09 PM, Rohan Jain wrote:

> Hi again,
> 
> I really couldn't understand the response this post has got. It
> deserved at least a little feedback, positive or negative. I guess I
> wont be submitting this over melange.
> 
> Still, I have put some effort and research in the proposal. So if
> possible I would like to know if it had anything of value. Maybe some
> one could work over that, even me if I get the time.
> 
> --
> Rohan
> 
> 
> On 23:40 +0530 / 31 Mar, Rohan Jain wrote:
>> Hi,
>> 
>> I am Rohan Jain, a 4th (final) year B.Tech undergraduate Student
>> from Indian Institute of Technology, Kharagpur. I have been using
>> django since over a year and generally look into the code base to find
>> about various implementations. I have made attempts to make some minor
>> contributions and if selected this would be my first major one.
>> 
>> More about Me: 
>> IRC, Github: crodjer
>> 
>> I am interested in contributing some security enhancements to django
>> as my Summer of Code project. Below is the 1st draft of my proposal
>> regarding this. A pretty version of this is available at:
>> https://gist.github.com/2203174
>> 
>> 
>> #Abstract
>> 
>> Django is a reasonably secure framework. It provides an API and
>> development patterns which transparently take care of the common web
>> security issues. But still there are security features which need
>> attention. I propose to work on integration of existing work on
>> centralized token system and improved CSRF checking without any
>> compromises. If time permits I will also attempt on integration of
>> django-secure.
>> 
>> #Description
>> ##Centralized tokenization
>> There are multiple places in django which use some or other kinds of
>> tokens:
>> 
>> - contirb.auth (random password, password reset)
>> - formtools
>> - session (backends)
>> - cache
>> - csrf
>> - etags
>> 
>> Token generation is pretty common around the framework.  So, instead
>> of each application having its own token system, and hence needs to be
>> maintained separately. There should be centralized token system, which
>> provides an abstract API for everyone to loose. In fact, I have seen
>> that some apps use `User.objects.make

Re: GSoC 2012: Security Enhancements

2012-04-05 Thread Rohan Jain 
Hi again,

I really couldn't understand the response this post has got. It
deserved at least a little feedback, positive or negative. I guess I
wont be submitting this over melange.

Still, I have put some effort and research in the proposal. So if
possible I would like to know if it had anything of value. Maybe some
one could work over that, even me if I get the time.

--
Rohan


On 23:40 +0530 / 31 Mar, Rohan Jain wrote:
> Hi,
> 
> I am Rohan Jain, a 4th (final) year B.Tech undergraduate Student
> from Indian Institute of Technology, Kharagpur. I have been using
> django since over a year and generally look into the code base to find
> about various implementations. I have made attempts to make some minor
> contributions and if selected this would be my first major one.
> 
> More about Me: 
> IRC, Github: crodjer
> 
> I am interested in contributing some security enhancements to django
> as my Summer of Code project. Below is the 1st draft of my proposal
> regarding this. A pretty version of this is available at:
> https://gist.github.com/2203174
> 
> 
> #Abstract
> 
> Django is a reasonably secure framework. It provides an API and
> development patterns which transparently take care of the common web
> security issues. But still there are security features which need
> attention. I propose to work on integration of existing work on
> centralized token system and improved CSRF checking without any
> compromises. If time permits I will also attempt on integration of
> django-secure.
> 
> #Description
> ##Centralized tokenization
> There are multiple places in django which use some or other kinds of
> tokens:
> 
>  - contirb.auth (random password, password reset)
>  - formtools
>  - session (backends)
>  - cache
>  - csrf
>  - etags
> 
> Token generation is pretty common around the framework.  So, instead
> of each application having its own token system, and hence needs to be
> maintained separately. There should be centralized token system, which
> provides an abstract API for everyone to loose. In fact, I have seen
> that some apps use `User.objects.make_random_password` from
> contrib.auth, which they can be sure of being maintained in the future
> for random generation. To me this looks kind of weird.
> In last djangocon, a lot of work regarding this was done over [Yarko's
> Fork][yarko-fork].
> 
> I had a discussion with Yarko Tymciurak regarding this. The work is
> nearly ready for a merge, only some tasks left. In the initial period
> my SoC I can work over these to insure that the already done
> significant work gets in django and is updated for 1.5.
> 
>  - Porting more stuff to the new system (README.sec in
>[yarko's fork][yarko-fork])
>  - Testing - See if the current coverage of the tests is enough, write
>them if not.
>  - Compatibility issues
>  - API Documentation
> 
> I will study the changes done at djangocon and then attempt the tasks
> mentioned above.
> 
> ##CSRF Improvements
> 
> Cross-Origin Resource Sharing (CORS):  
> W3C has a working draft regarding [CORS][w3c-cors-draft], which opens
> up the possibility for allowing client-side request cross-origin
> requests. This directly triggers in mind the capability to develop
> API which can be exposed directly to the web browser. This would let
> us get rid of proxies and other hacks used to achieve this.
> Currently all the major browsers support this: Chrome (all versions),
> Firefox (> 3.0), IE (> 7.0), Safari (> 3.2), Opera (> 12.0).
> Introduced it here as some further parts of the post refer to this.
> 
> ###Origin checking
> 
> With CORS around need for using CSRF token can be dropped, at least in
> some browsers. [Ticket #16859][orig-check-ticket], is an attempt for
> that. But this was rejected because of neglecting the case for
> presence of `CSRF_COOKE_DOMAIN` (Refer to the closing comment on the
> ticket for details). So to handle this we need to simulate checking of
> CSRF cookie domain as web browsers do it. Maybe:
> 
> ```python
> reqest.META.get('HTTP_ORIGIN').endswith(settings.CSRF_COOKIE_DOMAIN)
> ```
> 
> As the closing comment points it out, we can't do this with secure
> requests. They need to be essentially checked against the referrer or
> origin, at least for now. We can not be sure that some untrusted or
> insecure subdomain has not already set the cookie or cookie domain.
> d
> To deal with this, we have to consider https separately as it is
> being done now. So it will be something like:
> 
> 
> ```python
> def process_view(self, request, ):
> 
> # Same initial setup
> 
> if request.method not in ('GET', 'HEAD', 'OPTIONS', 'TRACE'):
> 
> host = request.get_host()
> origin = reqest.META.get('HTTP_ORIGIN', "")
> cookie_domain = settings.CSRF_COOKIE_DOMAIN
> 
> if request.is_secure():
> good_referer = 'https://%s/' % host
> referer = origin or request.META.get('HTTP_REFERER')
> # Do the same ori

GSoC 2012: Security Enhancements

2012-03-31 Thread Rohan Jain 
Hi,

I am Rohan Jain, a 4th (final) year B.Tech undergraduate Student
from Indian Institute of Technology, Kharagpur. I have been using
django since over a year and generally look into the code base to find
about various implementations. I have made attempts to make some minor
contributions and if selected this would be my first major one.

More about Me: 
IRC, Github: crodjer

I am interested in contributing some security enhancements to django
as my Summer of Code project. Below is the 1st draft of my proposal
regarding this. A pretty version of this is available at:
https://gist.github.com/2203174


#Abstract

Django is a reasonably secure framework. It provides an API and
development patterns which transparently take care of the common web
security issues. But still there are security features which need
attention. I propose to work on integration of existing work on
centralized token system and improved CSRF checking without any
compromises. If time permits I will also attempt on integration of
django-secure.

#Description
##Centralized tokenization
There are multiple places in django which use some or other kinds of
tokens:

 - contirb.auth (random password, password reset)
 - formtools
 - session (backends)
 - cache
 - csrf
 - etags

Token generation is pretty common around the framework.  So, instead
of each application having its own token system, and hence needs to be
maintained separately. There should be centralized token system, which
provides an abstract API for everyone to loose. In fact, I have seen
that some apps use `User.objects.make_random_password` from
contrib.auth, which they can be sure of being maintained in the future
for random generation. To me this looks kind of weird.
In last djangocon, a lot of work regarding this was done over [Yarko's
Fork][yarko-fork].

I had a discussion with Yarko Tymciurak regarding this. The work is
nearly ready for a merge, only some tasks left. In the initial period
my SoC I can work over these to insure that the already done
significant work gets in django and is updated for 1.5.

 - Porting more stuff to the new system (README.sec in
   [yarko's fork][yarko-fork])
 - Testing - See if the current coverage of the tests is enough, write
   them if not.
 - Compatibility issues
 - API Documentation

I will study the changes done at djangocon and then attempt the tasks
mentioned above.

##CSRF Improvements

Cross-Origin Resource Sharing (CORS):  
W3C has a working draft regarding [CORS][w3c-cors-draft], which opens
up the possibility for allowing client-side request cross-origin
requests. This directly triggers in mind the capability to develop
API which can be exposed directly to the web browser. This would let
us get rid of proxies and other hacks used to achieve this.
Currently all the major browsers support this: Chrome (all versions),
Firefox (> 3.0), IE (> 7.0), Safari (> 3.2), Opera (> 12.0).
Introduced it here as some further parts of the post refer to this.

###Origin checking

With CORS around need for using CSRF token can be dropped, at least in
some browsers. [Ticket #16859][orig-check-ticket], is an attempt for
that. But this was rejected because of neglecting the case for
presence of `CSRF_COOKE_DOMAIN` (Refer to the closing comment on the
ticket for details). So to handle this we need to simulate checking of
CSRF cookie domain as web browsers do it. Maybe:

```python
reqest.META.get('HTTP_ORIGIN').endswith(settings.CSRF_COOKIE_DOMAIN)
```

As the closing comment points it out, we can't do this with secure
requests. They need to be essentially checked against the referrer or
origin, at least for now. We can not be sure that some untrusted or
insecure subdomain has not already set the cookie or cookie domain.
d
To deal with this, we have to consider https separately as it is
being done now. So it will be something like:


```python
def process_view(self, request, ):

# Same initial setup

if request.method not in ('GET', 'HEAD', 'OPTIONS', 'TRACE'):

host = request.get_host()
origin = reqest.META.get('HTTP_ORIGIN', "")
cookie_domain = settings.CSRF_COOKIE_DOMAIN

if request.is_secure():
good_referer = 'https://%s/' % host
referer = origin or request.META.get('HTTP_REFERER')
# Do the same origin checks here

# We are insecure, so care less
# A better way for this check can be used if needed
elif origin.endswith(cookie_domain):
# Safe, accept request

# Some unsupported browser
# Do the conventional checks here
```

If the above were to be implemented, the setting `CSRF_COOKIE_DOMAIN`
should be deprecated for something like `CSRF_ALLOWED_DOMAIN` which
makes more sense.

I would also suggest making CSRF cookie as http only. There doesn't
seem a reason currently why the cookies would be needed to be accessed
in browser.

###Less restrictive secure requests

The current CSRF system is