Re: Drop CSRF middleware from the settings template

[Florian Apolloner]

On Thursday, April 20, 2023 at 1:00:05 PM UTC+2 Jure Erznožnik wrote:

OK, I'll bite:

For the first issue, my problem revolved around this code:
@property def POST(self): # Ensure that request.POST uses our request 
parsing. if not _hasattr(self, '_data'): self._load_data_and_files() if 
is_form_media_type(self.content_type): return self._data 

return QueryDict('', encoding=self._request._encoding) 


That code looks correct. I cannot tell you why self._data would be lost, 
but the empty QueryDict for request.POST makes sense. Please note that my 
comments only apply to the Django codebase itself, I don't really know what 
DRF does aside from that

IIUC, this code tries to match the token received from the headers with one 
that's supposed to be in the form data payload. The code is allowed to fail 
just fine, but in this case it has the side-effect mentioned: the form 
payload will have been parsed and cannot be parsed again - while at the 
same time rejecting the parsed data because it is not form payload type. 


It compares the value from the cookie with either the value from the header 
or the form payload. There is no need to parse it again, 
_load_data_and_files from above does this and sets it: 
https://github.com/encode/django-rest-framework/blob/38a74b42da10576857d6bf8bd82a73b15d12a7ed/rest_framework/request.py#L283
 
If you use application/json you need to access request.data in DRF, 
request.POST will always be empty.
 

What I was trying to say with that paragraph was that I'd like to actually 
figure out a way to START doing token rotation because my observation is 
that it's currently NOT rotating and is therefore a lot less useful as a 
security measure.


Oh then I did misread you. It is not neccessary to rotate the token every 
request, reusing the token that you obtain once is just fine. 

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/263b4b04-f986-42fc-9e1d-1fc7a4e4b264n%40googlegroups.com.

Re: Drop CSRF middleware from the settings template

[Jure Erznožnik]

OK, I'll bite:

For the first issue, my problem revolved around this code:

@property def POST(self): # Ensure that request.POST uses our request 
parsing. if not _hasattr(self, '_data'): self._load_data_and_files() if 
is_form_media_type(self.content_type): return self._data return 
QueryDict('', encoding=self._request._encoding)


The second `if` asks whether the payload is form media type, and, if so, 
returns the parsed content, otherwise it returns a blank QueryDict. In 
my case, the content-type was set to application/json. It parsed 
successfully and self._data was still populated when the code reached 
the second if. However, the second if rejected the content and skipped 
to returning the empty QueryDict, which then resulted in payload 
consequently being served as the empty QueryDict also for DRF. I can't 
really say where the actual request._data got reset because of this, but 
I did (get reset). I'm guessing DRF tries to parse the data from 
scratch, but the stream was consumed already or some such. I didn't 
nearly debug this enough to KNOW what went wrong.


I was unable to find the actual culprit for this behaviour, so I 
ultimately just exempt the view. But the circumstances of the actual 
issue were very unclear: I have a commit where CSRF middleware chose 
this path and a commit where it did not. I was unable to find the 
culprit by being unable to find the condition that made the middleware 
go into the check at all - contrary to the commit before where it did 
not. Needless to say, the commits did not deal with anything I perceived 
as anything near CSRF or AJAX. They were super small too...


This is the actual section of the code that triggers the POST processing 
in the middleware:


if request.method == "POST": try: request_csrf_token = 
request.POST.get('csrfmiddlewaretoken', '') except OSError:


IIUC, this code tries to match the token received from the headers with 
one that's supposed to be in the form data payload. The code is allowed 
to fail just fine, but in this case it has the side-effect mentioned: 
the form payload will have been parsed and cannot be parsed again - 
while at the same time rejecting the parsed data because it is not form 
payload type.


As for the second issue, I must admit I don't understand your reply at 
all. I'm a pretty bad n00b while you're one of the Django đombas (tough 
big guys) and it's entirely possible I am misinterpreting what I'm 
seeing here. I just stated my observation that Django now only serves 
the index.html for me, generating a CSRF token along the way and then I 
use that CSRF token for all subsequent AJAX requests. I don't even 
understand the "visual representation changes" and "looks like it's 
different". So, sorry, I'm afraid I can't be of assistance here.


What I was trying to say with that paragraph was that I'd like to 
actually figure out a way to START doing token rotation because my 
observation is that it's currently NOT rotating and is therefore a lot 
less useful as a security measure.


Now I have bit my chunk off and I'm chewing. Hopefully no hooks in there ;)

LP,
Jure


On 20. 04. 23 11:51, Florian Apolloner wrote:

Hi,

On Tuesday, April 18, 2023 at 10:57:55 PM UTC+2 jure.er...@gmail.com 
wrote:


Well, TBH, I've just completed dealing with CSRF form in my
projects. I ended up exempting the particular view from CSRF
because I didn't know how to get the stuff to work. The problem
was that django parsed the body payload, which was JSON and thus
rejected its contents (because it wasn't form payload type – POST
method). As a result, DRF then had no payload to work with… I
shouldn't go into too much detail as it's irrelevant to the point.


I do not think this is true. Django only parses the POST data if the 
content-type is a form type, so if you are sending JSON properly 
Django will not parse the data and DRF can handle it just fine.


 But, I've been considering I need a modernised CSRF: currently it
works by generating a new token every page served. But we have
switched our front-end to SPA and that doesn't make much sense any
more since CSRF token itself doesn't change at all, since Django
template system only ever serves one page. AFAIK, DRF doesn't
ganerate new tokens in its pipelines.


That is not accurate either. The token does not change for every page 
served, only it's visual representation does. That means that you can 
keep using the same CSRF token even though it looks like it is 
different (note: This assume that you are not triggering a codepath 
that is rotating the token).


Cheers,
Florian
--
You received this message because you are subscribed to the Google 
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: Django's automatic admin interface.

[Dipankar]

Thank you all or your comments .. I am happy to hear about wire or
htmx @Matthew.
I want to do it for myself if I get a little bit clue from you. Thanks a
lot.

On Wed, Apr 19, 2023 at 7:25 PM Matthew Pava  wrote:

> I agree with your sentiment, Tom. I would add that we could get a more
> “SPA-feel” by using HTML over the wire or htmx, which requires minimal
> JavaScript.
>
>
>
> *From:* django-developers@googlegroups.com <
> django-developers@googlegroups.com> *On Behalf Of *Tom Carrick
> *Sent:* Wednesday, April 19, 2023 5:07 AM
> *To:* django-developers@googlegroups.com
> *Subject:* Re: Django's automatic admin interface.
>
>
>
> IMO, if we were going to modernise the admin (which is laudable), it
> wouldn't be by using JS frameworks or Tailwind, but by simplifying things
> further, by removing the last bits of JQuery, simplifying the HTML and
> making it more semantic, and rewriting the CSS to use a grid based layout
> and cut down the amount of code that is needed to achieve the same result.
>
>
>
> I don't have anything against Tailwind or Vue per së, but forcing every
> Django project to have them in the backend seems too opinionated and too
> much of a maintenance burden.
>
>
>
> As you mentioned, there are already opinionated packages out there, I'm
> happy they exist, but in my opinion they belong in external packages, not
> in core.
>
>
>
> Tom
>
>
>
> On Wed, 19 Apr 2023 at 11:45, Dipankar  wrote:
>
> Sorry if my question is wrong.. .. Not exactly technology I wanted to know
> about the frontend framework like tailwindCSS,react or Vue.
>
>
>
> In nutshell I want admin interface with tailwindCSS/React/Vue. any
> suggestion ?
>
>
>
> On Wed, Apr 19, 2023 at 3:01 PM David Sanders <
> shang.xiao.sand...@gmail.com> wrote:
>
> Hi Dipankar,
>
>
>
> Not being rude but serious question: What's the latest front end
> technology? :)
>
>
>
> On Wed, 19 Apr 2023, 7:27 pm Dipankar,  wrote:
>
> Is there any plan to replace Django's automatic admin interface with the
> latest front end technology?
>
> There are several packages available but what if Django itself provides
> the same as core.
>
>
> --
>
> Warm Regards,
> Dipankar B.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFdBwp_N0remvp8zAPFVda6iyFWVWR%3DZh0EtfE9fzYcPQVixkQ%40mail.gmail.com
> 
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CADyZw-61oX4oh3apDatm_MKCdoYCMLxkk8O4krKMZuPZF2LpNg%40mail.gmail.com
> 
> .
>
>
>
> --
>
> Warm Regards,
> Dipankar B.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFdBwp-cO_g_JCTQhMQVEJe%2BN4FjJD5F-D%2B7LKugRK%2B1-Pq3Rg%40mail.gmail.com
> 

Re: Drop CSRF middleware from the settings template

[Florian Apolloner]

Hi,

On Tuesday, April 18, 2023 at 10:57:55 PM UTC+2 jure.er...@gmail.com wrote:

Well, TBH, I've just completed dealing with CSRF form in my projects. I 
ended up exempting the particular view from CSRF because I didn't know how 
to get the stuff to work. The problem was that django parsed the body 
payload, which was JSON and thus rejected its contents (because it wasn't 
form payload type – POST method). As a result, DRF then had no payload to 
work with… I shouldn't go into too much detail as it's irrelevant to the 
point.


I do not think this is true. Django only parses the POST data if the 
content-type is a form type, so if you are sending JSON properly Django 
will not parse the data and DRF can handle it just fine. 
 

 But, I've been considering I need a modernised CSRF: currently it works by 
generating a new token every page served. But we have switched our 
front-end to SPA and that doesn't make much sense any more since CSRF token 
itself doesn't change at all, since Django template system only ever serves 
one page. AFAIK, DRF doesn't ganerate new tokens in its pipelines.


That is not accurate either. The token does not change for every page 
served, only it's visual representation does. That means that you can keep 
using the same CSRF token even though it looks like it is different (note: 
This assume that you are not triggering a codepath that is rotating the 
token).

Cheers,
Florian 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f64582bd-63ea-483a-aebc-2acc5dac6cdbn%40googlegroups.com.

Re: Production Django use and "real ip"

['Adam Johnson' via Django developers (Contributions to Django itself)]

> Are there any deployment scenarios where META.REMOTE_ADDR is ever even
correct?

Yes, when running a WSGI server that faces the internet, with no
intermediate proxies.

On Wed, Apr 19, 2023 at 3:51 PM Arthur Pemberton  wrote:

> At this point, I'm not even suggesting that Django handle this internally.
> I'm suggesting that the behaviour/expectation be documented, at least in
> the deployment guide.
>
> Are there any deployment scenarios where META.REMOTE_ADDR is ever even
> correct?
>
> Arthur Pemberton
>
> On Wed, Apr 19, 2023 at 3:37 AM 'st...@jigsawtech.co.uk' via Django
> developers (Contributions to Django itself) <
> django-developers@googlegroups.com> wrote:
>
>> As someone whose worked on various projects in different languages over
>> the last 15 years that heavily involved deciphering IP sent in headers to
>> try to determine the "real" IP address of a connection, I would urge
>> caution with anything around determining a "real IP". There is no standard
>> in terms of where to look and what to trust. The X-FORWARDED-FOR is not
>> always right, can easily be spoofed, it can include multiple IPs of which
>> the order is not consistent. Sometimes the left most element is the first
>> and "true IP", sometimes it's the right, sometimes its a value in the
>> middle. It all depends what's included, what appended to the request, what
>> the values are and what you want to trust/ignore. I've seen requests that
>> include internal network IPs, then router/gateway IPs, proxy IPs and load
>> balancers all within that head all in different orders.   It's especially
>> messy when dealing with requests on mobile network where the carrier uses
>> proxies, sometimes 3rd parties, and where your website is hosted behind
>> both load balancers and webserver as each may manipulate the header in
>> different ways.
>>
>> One of the best packages within the Django eco-system for trying to
>> identify a users actual external IP that I've come across is
>> django-ipware . It allows you to
>> choose the precedence order that matches your use case, yo have private IP
>> prefixes, to configure how many proxies you wish to ignore etc. They also
>> have a handy notice/disclaimer
>>  on
>> the subject.
>>
>> IMO Django core should leave this 3rd party packages and individual
>> deployments to decide and determine what they deem as being the source of
>> the "real IP" for their individual project.
>>
>>
>>
>> On Friday, 14 April 2023 at 10:13:22 UTC+1 Adam Johnson wrote:
>>
>>> It's surprisingly complex to interpret x-forwarded-for:
>>> https://www.brainonfire.net/blog/2022/03/04/understanding-using-xff/ .
>>> We will never be able to safely add automated handling.
>>>
>>> I *guess* we could add a note to the deployment guide like "check your
>>> HTTP_X_FORWARDED_FOR setting". I'm concerned it would be a step towards
>>> making the guide too long, and filled with irrelevant details. Most sites
>>> don't care about recording the user's IP. On those that do, it should be
>>> easy enough to discover the setting.
>>>
>>> On Sat, Apr 1, 2023 at 4:39 AM Arthur Pemberton 
>>> wrote:
>>>
 I have read previous discussions (most recent I could find was Dec 2013
  [1] ) on the inclusion of `HTTP_X_FORWARDED_FOR` based logic to get the
 "real" IP address of an HttpRequest. From what I can see, currently there
 is currently no automatic handling of `HTTP_X_FORWARDED_FOR` in Django.

 However, I do notice that Django acknowledges `X_FORWARDED_HOST`,
 `X_FORWARDED_PORT` and (indirectly) `X_FORWARDED_PROTO`
 (though SECURE_PROXY_SSL_HEADER).

 If there is still opposition to having some built-in handling for
 `HTTP_X_FORWARDED_FOR`, I think that the deployment guide [1] should at
 least mention the need for the developer to handle this explicitly.

 Regards,
 Arthur P.

 

 [1]
 https://groups.google.com/g/django-developers/c/J5O28jB5D3Q/m/KLLgllFS7v0J
 [2] https://docs.djangoproject.com/en/4.1/howto/deployment/

 --
 You received this message because you are subscribed to the Google
 Groups "Django developers (Contributions to Django itself)" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-develop...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/django-developers/96d735ee-4ac0-4bf4-9850-a49f287e6e2an%40googlegroups.com
 
 .

>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to