Fellow Report - May 7, 2016

[Tim Graham]

Triaged

---

https://code.djangoproject.com/ticket/26555 - makemigrations serializer 
isinstance() check fails for subclasses of Decimal (and DateTime etc) 
(accepted)

https://code.djangoproject.com/ticket/26568 - in_bulk when run on a query 
set should not need a list of IDs (duplicate)

https://code.djangoproject.com/ticket/26566 - Wrong example of 
Cache-Control in the docs (accepted)

https://code.djangoproject.com/ticket/26564 - Migration for renaming M2M 
field intermediate model causes exception (accepted)

https://code.djangoproject.com/ticket/26582 - Better ArrayField/list 
formatting in the admin (accepted)

https://code.djangoproject.com/ticket/26583 - Add an option to silence 
collectstatic clashing files warnings (accepted)

Authored



https://github.com/django/django/pull/6493 - Refs #22936 -- Moved more of 
Field.get_db_prep_lookup() to lookups.

https://github.com/django/django/pull/6501 - Middleware implementation as 
per DEP0005 (polishing, updating tests, documentation)

Reviewed/committed

--

https://github.com/django/django/pull/6477 - Refs #16508 -- Made 
Model.__init__() aware of virtual fields.

https://github.com/django/django/pull/6537 - Fixed #26561 -- Improved 
admin's JavaScript SelectBox performance on large lists.

https://github.com/django/django/pull/6542 - Fixed #26567 -- Updated 
references to obsolete RFC2616.

https://github.com/django/django/pull/6555 - Fixed #26558 -- Removed need 
for request context processor on admin login page.

https://github.com/django/django/pull/5460 - Fixed #25588 -- Added spatial 
lookups to RasterField.

https://github.com/django/django/pull/6377 - Fixed #25986 -- Fixed crash 
sending email with non-ASCII in local part of the address.

https://github.com/django/django/pull/6561 - Fixed #26566 -- Rewrote an 
incorrect Cache-Control example.

https://github.com/django/django/pull/6522 - Fixed #26508 -- Clarified docs 
for various FieldFile methods.

https://github.com/django/django/pull/6558 - Fixed #26582 -- Added prettier 
admin display for list values.

https://github.com/django/django/pull/6330 - Fixed #26402 -- Added relative 
path support in include/extends template tags.

https://github.com/django/django/pull/6553 - Fixed #26449 -- Merged admin's 
FORMFIELD_FOR_DBFIELD_DEFAULTS with formfield_overrides.

Reviews of core dev work



https://github.com/django/django/pull/6499 - Fixed #22936 -- Obsoleted 
Field.get_prep_lookup()/get_db_prep_lookup()

https://github.com/django/django/pull/6552 - Fixed #26577 -- Disabled 
implicit wait of Selenium tests where appropriate.

https://github.com/django/django/pull/6551 - Stopped truncating 
AdminEmailHandler message subjects

https://github.com/django/django/pull/6566 - Fixed #26544 -- Fixed 
translations of password validators help_text in forms.
https://github.com/django/django/pull/4889 - Fixed #25004 -- Added an 
OpenLayers 3 map widget

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c5fcee6f-dceb-45c8-bec1-f308a4e20fe2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Enforcing a max size for form field values read into memory (review/determination of next steps needed)

[Tim Graham]

After discussion on the PR, we concluded that the reasons that the 
request.body check was removed weren't valid, so this check is reinstated. 
I believe I've addressed all Tom Christie's concerns by now. If anyone else 
would like to take a look, now is the time. Thanks!

https://github.com/django/django/pull/6447

On Tuesday, April 26, 2016 at 6:47:09 PM UTC-4, Tim Graham wrote:
>
> It seems there was request.body checking in previous iterations of the PR 
> but it was dropped for reasons that aren't entirely clear to me:
> https://github.com/django/django/pull/3852#discussion_r35350372
>
> On Wednesday, April 20, 2016 at 9:30:48 PM UTC-4, Cristiano Coelho wrote:
>>
>> Hi,
>>
>> In particular I'm interested in this new setting: 
>> DATA_UPLOAD_MAX_MEMORY_SIZE 
>> [1]
>> that only seems to be checked against mutlparts [2] and url encoded[3] 
>> request bodies.
>>
>> It could be good that this setting is also checked against other types 
>> where request.body is read directly, as you can still get the 
>> content-length from the body right? Please correct me if I'm wrong, but 
>> when in already django code all body data is always loaded in memory except 
>> for files and multi-part uploads which are streamed.
>> So JSON, XML or even plain text post requests could benefit from the 
>> DATA_UPLOAD_MAX_MEMORY_SIZE setting and it could be very convenient for 
>> example if an attacker sends a huge json, the python (at least 2.7) 
>> json.loads call usually crashes with an out of memory error when the string 
>> is too big while still creating a huge RAM spike.
>>
>>
>> [1] 
>> https://github.com/django/django/pull/6447/files#diff-ba8335f5987fcd81d41c28cd1879a9bfR291
>> [2] 
>> https://github.com/django/django/pull/6447/files#diff-ba8335f5987fcd81d41c28cd1879a9bfR291
>> [3] 
>> https://github.com/django/django/pull/6447/files#diff-0eb6c5000a61126731553169fddb306eR294
>>
>>
>> El martes, 19 de abril de 2016, 13:06:27 (UTC-3), Tom Christie escribió:
>>>
>>> > If you are using django-rest-framework, how would the fields counter 
>>> work?. It would be a shame if only multi part and urlencoded uploads would 
>>> have the benefit of these checks, while still allowing json, xml and others 
>>> still be "exploited".
>>> Note I didn't really read the code changes completely so I'm talking 
>>> with almost no knowledge on the proposed change.
>>>
>>> They wouldn't be respected by anything other than multi-part or 
>>> urlencoded requests.
>>> Tim's correct in noting that accessing `request.body` or 
>>> `request.stream` won't apply these checks (which is for example, what REST 
>>> framework does).
>>>
>>> Even so I think this is probably a reasonable approach. We could add 
>>> support for respecting these settings in REST framework too, once they 
>>> exist.(Although I think we'd have need to have a stricter consideration of 
>>> backwards compat wrt. folks POSTing large amounts of JSON data)
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c8f7d421-6c33-4803-a9d8-159dd9ce4d92%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: re-thinking middleware

[Tim Graham]

Thanks, I have what I hope is a minimally mergable patch here: 
https://github.com/django/django/pull/6501

As noted on the PR, there a more things that should be before the 1.10 
feature freeze but I'm hoping I can ticket them out and get some help from 
the community after merging this first step so that I can continue spending 
most of my time reviewing patches.

On Friday, May 6, 2016 at 7:59:38 PM UTC-4, Carl Meyer wrote:
>
> I agree with Simon on both counts. We do usually continue to test 
> deprecated code paths until they are removed, but I also think the 
> duplication in cases of tests overriding MIDDLEWARE_CLASSES might not be 
> necessary in _all_ cases; I think some discretion could be used 
> depending on to what extent the middleware is incidental to the tests vs 
> the direct subject of the test. But it might be simpler to just do them 
> all than to make that determination. 
>
> Carl 
>
> On 05/04/2016 08:57 PM, charettes wrote: 
> > Hi Tim, 
> > 
> > I think we should favor displaying a message in accordance with the 
> > setting the user is using as it will make the transition less confusing. 
> > In the case of the documented check message I think using the form 
> > "MIDDLEWARE/MIDDLEWARE_CLASSES" would make it easier to read then 
> > mentioning the two possible variants. We already alter the document 
> > messages anyway to account for their dynamic nature. 
> > 
> > In the case of the tests I believe both code path should continue to be 
> > tested. From the top of my head I can't think of an alternative to 
> > subclasses using @override_settings. I suggest we make the *legacy* 
> > tests class extend the MIDDLEWARE using test class and not the other way 
> > around as it will make the MIDDLEWARE_CLASSES code removal clearer. 
> > 
> > Simon 
> > 
> > Le mercredi 4 mai 2016 19:59:05 UTC-4, Tim Graham a écrit : 
> > 
> > I've been working on this and wanted to raise a couple points for 
> > discussion. 
> > 
> > How should we treat error messages in place like system checks where 
> > we have phrases like "Edit your MIDDLEWARE_CLASSES" ... of course 
> > the check can easily check both MIDDLEWARE and MIDDLEWARE_CLASSES 
> > without much effort but should we make the error message "smart" and 
> > display the version of the setting that the user is using? 
> > Alternatively, we could always reference MIDDLEWARE (the 
> > non-deprecated version) or use some variation like 
> > "MIDDLEWARE(_CLASSES)" or "MIDDLEWARE/MIDDLEWARE_CLASSES" until the 
> > deprecation period ends. 
> > 
> > Another point for discussion is whether we need to duplicate a lot 
> > of tests so we test that the middleware continue to work with both 
> > the old-style MIDDLEWARE_CLASSES and the new style MIDDLEWARE 
> > response handling. I guess a subclass of anything that uses 
> > @override_settings(MIDDLEWARE=...) that uses 
> > @override_settings(MIDDLEWARE_CLASSES=...) might work. Just putting 
> > it out there in case anyone has a better idea. 
> > 
> > On Monday, January 18, 2016 at 9:20:03 PM UTC-5, Carl Meyer wrote: 
> > 
> > I've updated DEP 5 with a new round of clarifications and tweaks 
> > based on the most recent feedback: 
> > https://github.com/django/deps/compare/62b0...master 
> >  
> > 
> > Carl 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> > Groups "Django developers (Contributions to Django itself)" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> > an email to django-develop...@googlegroups.com  
> > . 
> > To post to this group, send email to django-d...@googlegroups.com 
>  
> > . 
> > Visit this group at https://groups.google.com/group/django-developers. 
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/django-developers/eb1cf3f4-c021-40f6-be65-35427b2bf5c5%40googlegroups.com
>  
> > <
> https://groups.google.com/d/msgid/django-developers/eb1cf3f4-c021-40f6-be65-35427b2bf5c5%40googlegroups.com?utm_medium=email_source=footer>.
>  
>
> > For more options, visit https://groups.google.com/d/optout. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/04f2e0f0-aa41-4a24-abea-85d8c292bcc5%40googlegroups.com.
For more options, visit