Re: Enforcing a max size for form field values read into memory (review/determination of next steps needed)

2016-04-20 Thread Cristiano Coelho
Hi,

In particular I'm interested in this new setting: DATA_UPLOAD_MAX_MEMORY_SIZE 
[1]
that only seems to be checked against mutlparts [2] and url encoded[3] 
request bodies.

It could be good that this setting is also checked against other types 
where request.body is read directly, as you can still get the 
content-length from the body right? Please correct me if I'm wrong, but 
when in already django code all body data is always loaded in memory except 
for files and multi-part uploads which are streamed.
So JSON, XML or even plain text post requests could benefit from the 
DATA_UPLOAD_MAX_MEMORY_SIZE setting and it could be very convenient for 
example if an attacker sends a huge json, the python (at least 2.7) 
json.loads call usually crashes with an out of memory error when the string 
is too big while still creating a huge RAM spike.


[1] 
https://github.com/django/django/pull/6447/files#diff-ba8335f5987fcd81d41c28cd1879a9bfR291
[2] 
https://github.com/django/django/pull/6447/files#diff-ba8335f5987fcd81d41c28cd1879a9bfR291
[3] 
https://github.com/django/django/pull/6447/files#diff-0eb6c5000a61126731553169fddb306eR294


El martes, 19 de abril de 2016, 13:06:27 (UTC-3), Tom Christie escribió:
>
> > If you are using django-rest-framework, how would the fields counter 
> work?. It would be a shame if only multi part and urlencoded uploads would 
> have the benefit of these checks, while still allowing json, xml and others 
> still be "exploited".
> Note I didn't really read the code changes completely so I'm talking with 
> almost no knowledge on the proposed change.
>
> They wouldn't be respected by anything other than multi-part or urlencoded 
> requests.
> Tim's correct in noting that accessing `request.body` or `request.stream` 
> won't apply these checks (which is for example, what REST framework does).
>
> Even so I think this is probably a reasonable approach. We could add 
> support for respecting these settings in REST framework too, once they 
> exist.(Although I think we'd have need to have a stricter consideration of 
> backwards compat wrt. folks POSTing large amounts of JSON data)
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c8cb7752-b7b5-483c-a896-614f41cd203b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: typeshed stubs for mypy

2016-04-20 Thread Tim Graham
My understanding is that adding type annotations must wait until we drop 
support for Python 2 which will happen after we cut the stable/1.11.x 
branch in January 2017.

On Wednesday, April 20, 2016 at 7:28:25 PM UTC-4, Pradip Caulagi wrote:
>
> Hello, 
>
> Is there any plans to add type annotations to Django itself?  Or to 
> provide typeshed stubs so that we can run mypy on django projects? 
>
> Thanks, 
> Pradip 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/417351d3-63ae-40ec-b6a0-12378867786b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


typeshed stubs for mypy

2016-04-20 Thread Pradip Caulagi

Hello,

Is there any plans to add type annotations to Django itself?  Or to 
provide typeshed stubs so that we can run mypy on django projects?


Thanks,
Pradip

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/57180059.2080307%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Making Django more PaaS-friendly

2016-04-20 Thread andrew
We deploy django use AWS Elastic Beanstalk and use credstash for secrets:

https://github.com/fugue/credstash

So we can just do PASSWORD = credstash.getSecret('password') which works 
pretty well with the AWS Key Management Service.

What is a pain is that the AWS Health Check does not set the Host header 
and we don't know until runtime what the IP address of the server will be. 
We have to do a call to the local metadata server to find out the IP 
address then add it to ALLOWED_HOSTS which is really quite hideous. There 
is an explanation here:

https://dryan.com/articles/elb-django-allowed-hosts/

Personally I would quite like to see an equivalent of the @csrf_exempt 
decorator perhaps @allowed_hosts_exempt that could be use to mark the 
health check method.

Andrew



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5868cb37-2d59-4792-95b6-146a56309ad8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


SubQuery without using RawSQL

2016-04-20 Thread schinckel
I started some work late last night on attempting to replace some RawSQL() 
calls that do a sub query with a new Expression.

It actually worked!

Even in the cold light of day, it doesn't seem _too_ bad, so after working 
on it a bit today with jarshwah, I decided to stick up a WIP PR.

https://github.com/django/django/pull/6478

I'd love to have some further input.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f11385f8-9147-4f3b-8f5d-5b9000fbecd4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.