Re: Drop CSRF middleware from the settings template

[Florian Apolloner]

On Thursday, April 20, 2023 at 1:00:05 PM UTC+2 Jure Erznožnik wrote:

OK, I'll bite:

For the first issue, my problem revolved around this code:
@property def POST(self): # Ensure that request.POST uses our request 
parsing. if not _hasattr(self, '_data'): self._load_data_and_files() if 
is_form_media_type(self.content_type): return self._data 

return QueryDict('', encoding=self._request._encoding) 


That code looks correct. I cannot tell you why self._data would be lost, 
but the empty QueryDict for request.POST makes sense. Please note that my 
comments only apply to the Django codebase itself, I don't really know what 
DRF does aside from that

IIUC, this code tries to match the token received from the headers with one 
that's supposed to be in the form data payload. The code is allowed to fail 
just fine, but in this case it has the side-effect mentioned: the form 
payload will have been parsed and cannot be parsed again - while at the 
same time rejecting the parsed data because it is not form payload type. 


It compares the value from the cookie with either the value from the header 
or the form payload. There is no need to parse it again, 
_load_data_and_files from above does this and sets it: 
https://github.com/encode/django-rest-framework/blob/38a74b42da10576857d6bf8bd82a73b15d12a7ed/rest_framework/request.py#L283
 
If you use application/json you need to access request.data in DRF, 
request.POST will always be empty.
 

What I was trying to say with that paragraph was that I'd like to actually 
figure out a way to START doing token rotation because my observation is 
that it's currently NOT rotating and is therefore a lot less useful as a 
security measure.


Oh then I did misread you. It is not neccessary to rotate the token every 
request, reusing the token that you obtain once is just fine. 

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/263b4b04-f986-42fc-9e1d-1fc7a4e4b264n%40googlegroups.com.

Re: Drop CSRF middleware from the settings template

[Florian Apolloner]

Hi,

On Tuesday, April 18, 2023 at 10:57:55 PM UTC+2 jure.er...@gmail.com wrote:

Well, TBH, I've just completed dealing with CSRF form in my projects. I 
ended up exempting the particular view from CSRF because I didn't know how 
to get the stuff to work. The problem was that django parsed the body 
payload, which was JSON and thus rejected its contents (because it wasn't 
form payload type – POST method). As a result, DRF then had no payload to 
work with… I shouldn't go into too much detail as it's irrelevant to the 
point.


I do not think this is true. Django only parses the POST data if the 
content-type is a form type, so if you are sending JSON properly Django 
will not parse the data and DRF can handle it just fine. 
 

 But, I've been considering I need a modernised CSRF: currently it works by 
generating a new token every page served. But we have switched our 
front-end to SPA and that doesn't make much sense any more since CSRF token 
itself doesn't change at all, since Django template system only ever serves 
one page. AFAIK, DRF doesn't ganerate new tokens in its pipelines.


That is not accurate either. The token does not change for every page 
served, only it's visual representation does. That means that you can keep 
using the same CSRF token even though it looks like it is different (note: 
This assume that you are not triggering a codepath that is rotating the 
token).

Cheers,
Florian 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f64582bd-63ea-483a-aebc-2acc5dac6cdbn%40googlegroups.com.

Re: Select API choice when starting new project.

[Florian Apolloner]

Hi, this is something which is already supported via the template argument 
to startproject/startapp.

Cheers,
florian

On Sunday, April 16, 2023 at 11:47:44 PM UTC+2 Daniel Azubuine wrote:

> When starting a new Django project, the user can select if he wants to 
> build an API or use the Django template. 
> If he chooses API, then Django-rest framework will be installed, and then 
> serializers.py will be added to the folders.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e6df6bae-b417-4a0c-97bd-c2de927728dcn%40googlegroups.com.

Re: Proposal: Constructing urls outside the request cycle

[Florian Apolloner]

On Sunday, April 16, 2023 at 10:21:20 AM UTC+2 Adam Johnson wrote:

One question though, how will we support projects that are served at 
multiple domains? Would Django only support the "main" site through 
BASE_URL and require you to perform URL construction for other domains as 
required?


As usual it depends (tm) :D As long as `reverse` keeps generating URLs 
without a host then we'd only use BASE_URL as a fallback in certain cases 
where no request is available. In the longrun I'd love to support something 
along the lines of 
https://werkzeug.palletsprojects.com/en/2.2.x/routing/#werkzeug.routing.Subdomain
 and 
then `reverse` would probably be able to generate the proper URLs. But yes, 
there are certainly many things to think about.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/bd70d579-9ad9-4102-bcb2-20bab28f1783n%40googlegroups.com.

Re: Integrate dj-database-url into Django

[Florian Apolloner]

The thread that Carlton refers to is 
https://groups.google.com/g/django-developers/c/q20_Cxske88 -- don't be 
deterred by the title of it, it really talks about handling settings as 
well :)

I kinda agree with James. I'd like to have a proper answer for handling 
settings rather then go setting by setting. The main difficulty I see is to 
decide whether we really go low-level in the sense of providing a simple 
API that can get settings from a variety of sources (the `get("key", 
default)`) or if we go further and provide a newer opinionated high level 
API like pydantic etc… Both approaches have their ups and downs. But before 
repeating everything from the other thread here, I'd like to ask that we 
decide on one spot to continue this discussion and then stick to that one 
(the settings one, as noted by James I am also rather -0 on adding 
DATABASE_URL support now). Titlewise neither of the two threads seem to fit 
very well so we can just as well open a topic on the forum ;)

Cheers,
Florian
On Tuesday, November 29, 2022 at 8:09:17 AM UTC+1 carlton...@gmail.com 
wrote:

> This must be do-able now. 
>
> There was some discussion of something along these lines a couple of years 
> ago, with the idea of doing it for GSoC. 
> There was much back-and-forth but no consensus. 
>
> It has then come up again with Peter's suggestions about making it easier 
> to get to a deployment ready state. 
> The discussion with Florian there touches on this exact point.
>
> It looks like there's two sides to it: 
> * A user-facing API, perhaps like ...get("key" default) 樂
> * A pluggable backend API for that, so folks can use things like Vault and 
> temporary secrets files as well as the environment. 
>
> If I followed correctly, it would be OK to ship an environment backend 
> with Django (perhaps with some simpler ones for development/testing) 
> leaving more advanced options open. 
> I think Florian has ideas here. 
>
>
> On Mon, 28 Nov 2022 at 23:20, James Bennett  wrote:
>
>> It feels like this needs to be a broader conversation about not just
>> changing DATABASES to accept a URL, or integrating a particular
>> database-settings project, but to re-think how Django does
>> settings-from-environment as a whole.
>>
>> When I'm setting up new Django-based projects, django-environ
>> (https://pypi.org/project/django-environ/) is now one of the first
>> packages I reach for, and it has a much more comprehensive set of
>> helpers for configuring Django from environment variables and doing
>> appropriate parsing and type casts.
>>
>> There also is Pydantic's settings model
>> (https://pydantic-docs.helpmanual.io/usage/settings/) which uses type
>> annotations to drive parsing/validation/type-casting and defaults to
>> reading from environment variables.
>>
>> I'd rather see some sort of comprehensive "make settings-from-env
>> easier" in Django than one-off pieces like a database URL.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CAL13Cg-cqJi1JanmhN0S3rf1MYME9qH4XJsCEjLdB5wFXEQeAA%40mail.gmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dd5a98e9-a7f7-4db8-a61d-697210bd6970n%40googlegroups.com.

Re: Technical Board vote on DEP 0012: The Steering Council

[Florian Apolloner]

Hi,

+1 from me, but I'd like to ask the wider community (ie DSF members) 
whether they support this change.

While there has been some opposition on whether a change likes this will 
actually change things, I think that given the overall good reception of 
the proposal it is at least worth to try it.

Cheers,
Florian
On Friday, November 25, 2022 at 10:04:36 AM UTC+1 Mariusz Felisiak wrote:

> Thanks!
>
> Reminder: according to DEP 10 voting will end on *December 2nd, 2022* 
> *AoE* (members of the Technical Board may change their votes at any time 
> prior to closing of the final voting period).
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/77e508b4-8264-4997-b531-5da0416b6171n%40googlegroups.com.

Re: [Technical Board?] Project Ideas, and beginning GSoC 2023.

[Florian Apolloner]

I do have ideas but no idea about how viable they are in a GSoC context. 
Nevertheless I will put write them down here, maybe we can find smaller 
scopes if needed and if not, it still serves as a list of things that I'd 
think to be interesting:

 * Probably my number one since it kinda is a blocker for me: We need a 
connection pool in Django for async to work. That said connection pools are 
hard to get right ( 
https://github.com/brettwooldridge/HikariCP/blob/dev/documents/Welcome-To-The-Jungle.md
 
and https://www.psycopg.org/articles/2021/01/17/pool-design/ ).
 * Production ready webserver 
(https://groups.google.com/g/django-developers/c/q20_Cxske88). Maybe only 
focus on a smaller part first, like reading env variables / config files.
 * Depending on how far we get with `request.data` in the meantime, maybe 
put the second steps (adjustable parsers etc) in as GSoC project?
 * I haven't talked with anyone about this one yet, so it might be 
completely bonkers: I think openid connect is here to stay for a while and 
I'd love to see first class support in core for it. I am looking at this 
from an enterprise perspective though; I do not expect a user to choose 
where to login out of many options but rather provide a replacement for the 
default username/password login in an enterprise environment. Most 
solutions there support openid connect. Please note that I am not 
suggesting to support generic OAuth2/SAML and whatnot -- there are great 
3rd party packages for that already (which also include support for openid 
connect). I'd just love to be able to install arbitrary Django projects and 
point them to the central authentication server.

I hope this provides some ideas to get more ideas rolling :)

Cheers,
Florian 


On Tuesday, November 15, 2022 at 10:11:45 AM UTC+1 carlton...@gmail.com 
wrote:

> Hi all. 
>
> Google Summer of Code (GSoC) for 2023 has just been announced. 
>
> https://opensource.googleblog.com/2022/11/get-ready-for-google-summer-of-code-2023.html
>
> Django has participated many times, and it's been a real boon. Recent 
> successes include: 
>
> * The django-stubs mypy plugin. 
> * The cross-db JSON field. 
> * The Redis cache backend. 
> * Per model-class (field instance) custom lookups
> * Setup of the django-asv benchmarking project. 
> * ... 
>
> Application deadline for Orgs is February 7. I'll begin working on it 
> after the New Year. 
>
> Main bit is an ideas list for projects. The GSoC guide has a Writing a 
> Good Ideas List
> section. 
>
> > Projects should take ~175 hours or ~350 hours for GSoC contributors to 
> complete. 
>
> i.e. "short" or "long" projects. 
> https://google.github.io/gsocguides/mentor/defining-a-project-ideas-list
>
> I'm writing here *to solicit input on project ideas*. 
>
> I put "Technical Board?" in the subject because we're short a list of 
> project
> ideas for the technical direction of Django going forward, and maybe 
> thinking
> in terms of GSoC could be a way of bootstrapping that. The "?" is because 
> that's not 
> meant to exclude other input.
>
> Here's last year's list for reference: 
> https://code.djangoproject.com/wiki/SummerOfCode2022
>
> - Some of those were done: benchmarking (but that could be built on) and 
> per-instance 
>   field lookups.
>
> - Rate-limiting is a no-go I think: we couldn't come to any decent 
> agreement on scope, 
>   so it's better to live as a third-party package. 
>
> - I've tried to include folks from the wider ecosystem in previous years. 
> Two years ago 
>   we had both Wagtail and django-stubs projects. Wagtail then (last year)
>   applied in their own right, to great success. I'd like to offer that help
>   again to e.g. Jazzband or other established projects, assuming 
> maintainers
>   feel they have the capacity to mentor. (It's a minor increment to my 
> effort
>   for a good return I think.)
>
>
> No urgency but, can I ask you to put your grey cells into action? 
>
>
> Thanks. 
>
> Kind Regards,
>
> Carlton
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2b3650c6-d340-4464-96f2-e6722a8e415an%40googlegroups.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

The speckenv example highlights the problems when using `literal_eval`: 
A=[] will parse as list but A=[a] will parse as string again (if I read the 
code correctly). This ambiguity makes it really hard to explain. When using 
`literal_eval` one imo has to be consistent and don't fall back to a str 
and require strings to be written as A='[]' (with quotes). Then again this 
is rather ugly and hard to get right/explain if you try to pass that in via 
a shell ala `A='[]' django-admin runserver` and now A is a list again :/

Cheers,
Florian

On Wednesday, November 2, 2022 at 8:20:00 AM UTC+1 m...@feinheit.ch wrote:

> On Tue, Nov 1, 2022 at 10:34 AM Florian Apolloner  
> wrote:
>
>> Right, that would work. I am wondering though if we want to go all in on 
>> a typed config object like that or in a first step only have a simple API 
>> like `config.get('DEBUG', cast=bool)`.
>>
>>
> I found a neat trick in a 12factor library somewhere (I don't remember 
> where unfortunately) of using ast.literal_eval for config variables:
> https://docs.python.org/3/library/ast.html#ast.literal_eval
>
> https://github.com/matthiask/speckenv/blob/94722bcc95b94a9e6b1a1aada2bd977c43bfb767/speckenv.py#L63
>
> This may be too cute or magical for Django. It certainly works well and 
> avoids having to be overspecific with types. I find it boring to tell the 
> config loader that True is a bool and "hello" is a string. That being said 
> I don't really have a strong preference here apart from having a strong 
> dislike for the "several settings files" pattern. It's much more obvious to 
> me if e.g. `MIDDLEWARE` is configured in a single location and not modified 
> in other files. 
>
> Thanks,
> Matthias
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ad38315d-2a80-49de-9aa3-a93ec4ca567an%40googlegroups.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

Right, that would work. I am wondering though if we want to go all in on a 
typed config object like that or in a first step only have a simple API 
like `config.get('DEBUG', cast=bool)`.

On Tuesday, November 1, 2022 at 3:35:46 AM UTC+1 pe...@lincolnloop.com 
wrote:

> On Mon, Oct 31, 2022 at 11:52 AM Florian Apolloner  
> wrote:
>
>> On Monday, October 31, 2022 at 5:27:02 PM UTC+1 pe...@lincolnloop.com 
>> wrote:
>>
>>> In my ideal scenario, the default is a hard-coded settings file for the 
>>> project (deprecating DJANGO_SETTINGS_MODULE env var) and we have 
>>> CONFIG_LOADERS defined in the settings that could do env, toml, etc. 
>>> Perhaps things like django.setup, 
>>>
>> django.core.wsgi.get_wsgi_application, and 
>>> django.core.management.execute_from_command_line could accept the settings 
>>> module as an argument? django-admin could accept a --settings flag?
>>>
>>
>> I doubt that will ever fly. There is no project so to say in Django. 
>> Currently literally everything depends on knowing a 
>> `DJANGO_SETTINGS_MODULE` from which everything else can follow (even the 
>> wsgi app…). So without a settings file how would you know that hard-coded 
>> path for it? Django has no concept of a project that you could import. You 
>> also cannot easily put `CONFIG_LOADERS` in a settings file like we 
>> currently have because then how are you going to evaluate that while also 
>> executing all the other module level stuff. And django-admin command all 
>> already support a `--settings` flag (as alternative to 
>> DJANGO_SETTINGS_MODULE ;))
>>
>> Either way I doubt we will deprecating DJANGO_SETTINGS_MODULE any time 
>> soon. And I don't see it such of a problem with being there. Projects can 
>> set it (and startproject already does this) to a fixed value in manage.py & 
>> wsgi.py. For a project that installs a custom entrypoint script, you can 
>> also preset it to whatever you like.
>>
>> Please note that we are trying to load stuff from files & env vars in the 
>> first step, not throw out settings.py at the same time…
>>
>
> I figured it wouldn't fly :)
>
> I misunderstood your chicken-egg question above. What I've been doing is 
> defining a "Config" class and instantiating it in the settings file. 
> Instantiation of the class (or calling an explicit method on it) would do 
> the env/file loading and the instance becomes your `env` object. Rather 
> than having a CONFIG_LOADERS setting, you could define them on the Config 
> class or when you instantiate it. This is pretty similar to how goodconf 
> works today. https://github.com/lincolnloop/goodconf#quick-start
>  
>
>>
>> Cheers,
>> Florian
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/b0cc8e42-0781-4a69-b8a1-12cc6929c2f7n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/b0cc8e42-0781-4a69-b8a1-12cc6929c2f7n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/bab31aa7-5351-488b-9b6e-b7264da7c711n%40googlegroups.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

On Monday, October 31, 2022 at 5:27:02 PM UTC+1 pe...@lincolnloop.com wrote:

> In my ideal scenario, the default is a hard-coded settings file for the 
> project (deprecating DJANGO_SETTINGS_MODULE env var) and we have 
> CONFIG_LOADERS defined in the settings that could do env, toml, etc. 
> Perhaps things like django.setup, 
>
django.core.wsgi.get_wsgi_application, and 
> django.core.management.execute_from_command_line could accept the settings 
> module as an argument? django-admin could accept a --settings flag?
>

I doubt that will ever fly. There is no project so to say in Django. 
Currently literally everything depends on knowing a 
`DJANGO_SETTINGS_MODULE` from which everything else can follow (even the 
wsgi app…). So without a settings file how would you know that hard-coded 
path for it? Django has no concept of a project that you could import. You 
also cannot easily put `CONFIG_LOADERS` in a settings file like we 
currently have because then how are you going to evaluate that while also 
executing all the other module level stuff. And django-admin command all 
already support a `--settings` flag (as alternative to 
DJANGO_SETTINGS_MODULE ;))

Either way I doubt we will deprecating DJANGO_SETTINGS_MODULE any time 
soon. And I don't see it such of a problem with being there. Projects can 
set it (and startproject already does this) to a fixed value in manage.py & 
wsgi.py. For a project that installs a custom entrypoint script, you can 
also preset it to whatever you like.

Please note that we are trying to load stuff from files & env vars in the 
first step, not throw out settings.py at the same time…

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b0cc8e42-0781-4a69-b8a1-12cc6929c2f7n%40googlegroups.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

Hi Tobias,

On Sunday, October 30, 2022 at 2:02:36 AM UTC+2 tob...@caktusgroup.com 
wrote:

> Regarding DJANGO_ENV, why not ship the template with two settings files 
> and use the existing DJANGO_SETTINGS_MODULE, instead of adding a new 
> environment variable?
>

That is certainly one way to do it and probably the most controversial 
point. Ie what does DJANGO_ENV=dev|staging|prod offer over 
`DJANGO_SETTINGS_MODULE=your_project.settings.dev|staging|prod`

Maybe that would be controversial too, I'm not sure, but perhaps less so 
> than introducing a new paradigm? In any event, perhaps a pluggable backend 
> would allow everyone to be happy, or very nearly so... 
>

If we can come up with a reasonable API :) I wonder if a setuptools 
entrypoint to automatically enable your backend solely by installing it 
alongside the product would be one way to allow for easy customization… We 
might need two things to get this all started: An documented API contract 
that we can discuss and also a set of usecases that we do want to allow?

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1e0a0540-fcfa-45b0-bedc-f213aab9b267n%40googlegroups.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

Hi Peter,

On Saturday, October 29, 2022 at 6:07:16 PM UTC+2 pe...@lincolnloop.com 
wrote:

> Since you asked for feedback, I'll tell you what I'd do differently (and 
>> this is also a sign that production environment vary widely).
>>
>
> Production environments do vary widely, but I don't think that should stop 
> us from providing defaults that are one-size-fits-most. startproject 
> already does this and users are welcome to adjust it how they see fit.
>

Agreed, I should have written this better. I expect there to be plenty of 
different opinions on this topic and it might be hard to figure out the 
one-size-fits-most without being to limiting.
 

> * As soon as health checks check something aside from simply returning a 
>> 200 status code I'd want to be able to limit access to them
>>
>
> Yep, that seems reasonable. Although depending on the level of failure, 
> you may not be able to log in to see them. Starting with just a view that 
> responds with a 200 is reasonable.
>

Sure and with limiting access I was more thinking about source IPs than 
actual logins (ie open health endpoint to the internal network from which 
the loadbalancer connects)
  

>  * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently 
>> my projects often piggy-back on DEBUG but that can be rather hard from time 
>> to time
>>
>
> I'm open to other suggestions here, but am kind of in the same boat. There 
> may be times when you briefly want DEBUG=True in a non-public deployed 
> environment or DEBUG=False in a development environment. Overloading that 
> makes those scenarios a challenge.
>

Jupp, that is also the reason why I am in a love/hate relationship with it 
:)
 

>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env 
>> variables that leak down the whole process-tree. django-environ does 
>> support this; that said there are other options like dynaconf which also 
>> support hashicorp vault which is a massive plus imo. Just being able to 
>> utilize vault for free from every django application out there would be 
>> stellar.
>>
>
> I'm 100% in agreement here. I wrote goodconf [1] for this before I knew 
> about dynaconf which does a lot of the same things. I would love to see 
> Django adopt a split between "settings" (which I see as things that don't 
> change across environments) and "configuration" (things that you set per 
> environment). I did switch to django-environ's FileAwareEnv [2] which 
> allows loading from files, but not in the yaml./toml sense. I didn't go all 
> in here because I felt like it was a more controversial stance and at the 
> end of the day, you probably still need to support environment variables 
> because many (most?) PaaS providers don't have support for files like that.
>

Yes, being able to override stuff via env variables should probably be 
always allowed. And as long as we do have a "schema" of some kind (be that 
via config.get_bool or typing annotation on a configuration object) it is 
also relatively easy to convert env variables to their correct form (ie 
boolean).

With the goal of getting something merged into Django eventually, I felt 
> like environment variables were the lowest bar. We have precedence for 
> loading from environment variables with DJANGO_SETTINGS_MODULE. Do you 
> think a more full-featured library like dynaconf would make it harder to 
> push some of this upstream? Users that want to use dynaconf or goodconf or 
> whatever can still edit their settings and use those like they do today.
>

Yes and no. A 3rd party dependency will make it certainly harder to get 
this merged upstream. On the other hand I wouldn't want to support env 
variables only in the first iteration if that makes adding support for 
"properly" typed schemes (yaml/toml) harder in the future. From a viability 
point I'd like to see that we have an API that can handle hierarchical 
typed config and env variables. If we get that right the reasons to use 
something else will be rather small I guess. 
 
> Had another thought here... what about providing a pluggable 
configuration backend? Django could ship with some native backends like 
environment variables and toml (when it's available). Third party packages 
could provide Vault, secret files, etc. Same as how storages currently 
works and how you can easily drop in something like django-storages.

Yes, the main question though is how the api contract would look like and 
how we are going to "solve" the chicken egg problem of how those backends 
are activated? We cannot really put `SETTING_LOADERS = 
["django.settings.loaders.env", "django.settings.loader.toml"]` into a 
`settings.py` file and then use `config.get("DATABASE_URL")` just below to 
use the DATABASE_URL. So like `DJANGO_SETTINGS_MODULE` this would be 
something that has to be in an env var before Django starts. But I guess 
that would mostly be okay and projects might be able to pass their own 
loaders to `django.setup()` or configure 

Re: #21978 include production-ready web server

[Florian Apolloner]

Hi Jure,

My point was mostly about hieracrhy, not the exact format. That said on a 
technical level it is true that the same can achieved via python -- after 
all this is what settings.py is currently. But the hard truth is that many 
people prefer a established configuration file format (for some definition 
of established). Having executable files as config files can also be 
dangerous. Assume a software that reads `env.py` from `[CWD, ~/.config/app, 
/etc/app]`. Now if someone manages to convince you to run the software in a 
directory that they preconfigured with an `env.py` you will execute the 
code they want you to execute. This is part of the reason why stuff like 
`direnv` and others explicitly require you to allow a directory before 
setting env variables from `.envrc` for you (the most common offender 
obviously being someone cloning a git repo and in the case of direnv simply 
stepping into the directory executes code).

How many of the standard python tooling ala pip, flake8, black, pytest, 
tox, isort allow you to have a python file as config file? At some point 
not supporting one of the more common config file format (.ini,.yaml,.toml) 
feels like swimming against the current without any real benefit. So yes, a 
python file would suffice but I guess that battle is lost already.

Cheers,
Florian


On Wednesday, October 26, 2022 at 10:26:55 AM UTC+2 jure.er...@gmail.com 
wrote:

> IMHO, adding YAML support would introduce a dependency that's not needed: 
> why not just use Python, instead of ini/yaml/toml?
>
> Projects could have a (.gitignore) env.py that settings.py imports and 
> then sets the appropriate variables from it. You can use straight 
> assignment, e.g. 
>
> LANGUAGE_CODE = env.LANGUAGE_CODE
>
> or a getattr for optional settings:
>
> LANGUAGE_CODE = getattr(env, 'LANGUAGE_CODE', 'en-gb')
>
> No dependencies at all, not even internal python ones.
>
> But maybe I'm just being ignorant: there must be good reasons if core 
> python accepted toml. In that case, I hope to be enlightened (no sarcasm 
> here).
>
> LP,
> Jure
>
> On 26. 10. 22 09:30, Florian Apolloner wrote:
>
> Hi Pete,
>
> this does look interesting. And I agree that this is something that would 
> be nice to have in core in one form or another. That said I think we have 
> to do it iteratively (ie in smaller parts) and evaluate along the way.
>
> Since you asked for feedback, I'll tell you what I'd do differently (and 
> this is also a sign that production environment vary widely).
>
>  * As soon as health checks check something aside from simply returning a 
> 200 status code I'd want to be able to limit access to them
>  * I like my dev env to be as close as possible to production so I usually 
> also use whitenoise in dev. I had to many times where dev worked and 
> whitenoise in prod failed then :D
>  * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently 
> my projects often piggy-back on DEBUG but that can be rather hard from time 
> to time
>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env 
> variables that leak down the whole process-tree. django-environ does 
> support this; that said there are other options like dynaconf which also 
> support hashicorp vault which is a massive plus imo. Just being able to 
> utilize vault for free from every django application out there would be 
> stellar.
>
> Another thing about settings, and this is something I really like in 
> spring-boot is their relaxed binding for configuration variables. You can 
> look at 
> https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0 
> for an example. They describe there how hierarchical variables are handled 
> and translated between yaml and env variables. Of course this binding is 
> not exactly bidirectional always (I think) but it allows for a few nice 
> things. First off you can start with a config file in yaml (one might 
> dislike yaml but I find it easier to read than say a .env file with 100 
> settings):
>
> ```
> db:
>   host: myhost
>   port: 1234
>
> caches:
>   default:
>  type: memory
>   second:
> type: redis
> ```
>
> which would translate to the following env variables:
> ```
> DB_HOST: myhost
> DB_PORT: 1234
> CACHES_DEFAULT_TYPE: memory
> CACHES_SECOND_TYPE: redis
> ``` 
>
> This is imo really a powerful concept and allows for so much nicer and 
> easier definitions.
>
> Cheers,
> Florian
>
> On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com 
> wrote:
>
>> I put together a first stab at this at 
>> https://github.com/lincolnloop/django-production. Feedback is 
>> appreciated!
>>
>> On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com

Re: #21978 include production-ready web server

[Florian Apolloner]

Hi Pete,

this does look interesting. And I agree that this is something that would 
be nice to have in core in one form or another. That said I think we have 
to do it iteratively (ie in smaller parts) and evaluate along the way.

Since you asked for feedback, I'll tell you what I'd do differently (and 
this is also a sign that production environment vary widely).

 * As soon as health checks check something aside from simply returning a 
200 status code I'd want to be able to limit access to them
 * I like my dev env to be as close as possible to production so I usually 
also use whitenoise in dev. I had to many times where dev worked and 
whitenoise in prod failed then :D
 * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently 
my projects often piggy-back on DEBUG but that can be rather hard from time 
to time
 * I prefer settings in settings files (ini/yaml/toml) as opposed to env 
variables that leak down the whole process-tree. django-environ does 
support this; that said there are other options like dynaconf which also 
support hashicorp vault which is a massive plus imo. Just being able to 
utilize vault for free from every django application out there would be 
stellar.

Another thing about settings, and this is something I really like in 
spring-boot is their relaxed binding for configuration variables. You can 
look at 
https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0 for 
an example. They describe there how hierarchical variables are handled and 
translated between yaml and env variables. Of course this binding is not 
exactly bidirectional always (I think) but it allows for a few nice things. 
First off you can start with a config file in yaml (one might dislike yaml 
but I find it easier to read than say a .env file with 100 settings):

```
db:
  host: myhost
  port: 1234

caches:
  default:
 type: memory
  second:
type: redis
```

which would translate to the following env variables:
```
DB_HOST: myhost
DB_PORT: 1234
CACHES_DEFAULT_TYPE: memory
CACHES_SECOND_TYPE: redis
``` 

This is imo really a powerful concept and allows for so much nicer and 
easier definitions.

Cheers,
Florian

On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com wrote:

> I put together a first stab at this at 
> https://github.com/lincolnloop/django-production. Feedback is appreciated!
>
> On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com <
> pe...@lincolnloop.com> wrote:
>
>> I hadn't seen django-simple-deploy. At first glance, it looks 
>> interesting, but more platform-specific than what I was thinking.
>>
>> I'll play around with creating a third-party app that encapsulates some 
>> of these thoughts and see where I end up.
>>
>> Thanks for your time and feedback!
>>
>> On Tuesday, August 23, 2022 at 1:17:12 PM UTC-6 carlton...@gmail.com 
>> wrote:
>>
>>> Hey Pete. 
>>>
>>> Yes, there can be a lot of steps... (and trouble is every one has a 
>>> different opinion once you get to specifics :) 
>>>
>>> django-webserver looks good — in a similar ballpark to what I had in 
>>> mind, yes, but I'd probably not want to bundle the options for each server 
>>> myself, as that seems a lot to maintain over time... (Better to outsource 
>>> to the individual projects no?)  
>>>
>>> Have you seen what Eric Matthes is doing with django-simple-deploy? 
>>>
>>> https://github.com/ehmatthes/django-simple-deploy
>>>
>>> I really like the idea: in an opinionated way, it applies the project 
>>> changes you need to deploy (and it's plugable so folks could provide a 
>>> *template* for their own flavour of deployment if they wanted, which is 
>>> kind of important given the number of options in the landscape, even if you 
>>> only to *ship* a beginner option.) 
>>>
>>> Tim Allen also recently raised improving the Django project template, 
>>> which I think is related here. (That's on my list too: async this QTR, then 
>>> hoping to take on Adam's proposal to modernise the Request object for the 
>>> end of year, then 爛 swinging back here for "OK, what does it look like, 
>>> what can we do?" after that: there's a bunch of related tickets around 
>>> #21978 that it would be very sweet if we could clear up...) 
>>>
>>> I think all these thoughts are really pursuable outside of core in the 
>>> very short run, even if the goal is to merge them — it's much easier to 
>>> experiment that way, and then say, "This worked". 
>>>
>>> Anyhow, other than a "I've been thinking along these lines", and a 
>>> generally positive emote, I don't have too much more concrete at this 
>>> stage. 
>>> 
>>> C.
>>>
>>> On Tue, 23 Aug 2022 at 19:00, Peter Baumgartner  
>>> wrote:
>>>
 Thanks for the additional background Carlton! I like the idea of
 having some sort of protocol that folks can follow to make
 ready-to-use Django server packages (django-webserver [1] is in the
 same vein), but I'd strive for more. pip install + INSTALLED_APPS is
 great, but getting 

Re: Changing the role of the Technical Board

[Florian Apolloner]

Hi James,

On Tuesday, October 25, 2022 at 12:27:03 AM UTC+2 James Bennett wrote:

> On Mon, Oct 24, 2022 at 2:24 PM Andrew Godwin  wrote:
>
>> Proposing features - this is already in DEP 10, so I more just want to 
>> get that aspect of the Board actually going (and, as a side-effect, have 
>> something to aid fundraising). I am talking with the current Board 
>> separately on an internal thread, where the current stance (not everyone 
>> has responded) is that I am personally happy to take on all the work here 
>> for now - but I want to make sure it's not just me in the long run, be that 
>> merely proving that the idea works or attracting board members who want to 
>> specifically mediate such discussions and interaction with the wider 
>> community.
>>
>
> I admit I haven't been following Django development all that closely since 
> I mic-dropped after DEP 10 passed, but this is worrying, because canvassing 
> for feature proposals is not an optional thing -- DEP 10 *requires* the 
> Technical Board to do this at least once per feature release of Django. Has 
> that not been occurring? Because if it hasn't, then we have a major 
> problem, and I don't see how the current proposal would resolve it.
>

To be honest that is not how I understood and read the DEP. In my 
understanding the goal of the DEP was to make contributions easier and put 
the power into the community. The role of the technical board as I 
understood it was to assist or step in when our other decision making 
processes fail. As such I didn't read the DEP 10 as a requirement for the 
technical board to put out calls for proposals and ideas as long as those 
ideas are around anyways. With the powers the technical board has, I was 
always rather hesitant to use them unless it seemed necessary.

> And it's not just the lack of canvassing for features that's worrying; if 
members of the Technical Board didn't feel they were up to the job, they 
should have let someone know that. Getting burned out or overcommitted is a 
thing that happens, and a thing that was anticipated in drafting the 
governance -- DEP 10 has a procedure for it!
> Why did no member of the Technical Board do that?

Because on one hand, even with a process for it out there there might be a 
mental barrier to do so and admit failure to yourself. On the other hand, 
and this is probably my reason, is that we apparently have a different 
understanding of how the technical board is supposed to work. This is 
probably fully my fault, but the impression I got from the previous 
technical boards was that the role was to be rather passive and a 
tie-breaker when all else fails. I also think that I am not the only one 
who thought so.

Obviously the way I see (saw) the role of the technical board is not how 
you and probably others see it. As such I have no problem stepping down 
from the technical board if people feel I over- or understepped.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6eece557-85ae-41ee-b3cf-9c2307082b37n%40googlegroups.com.

Re: Supporting Oracle's renamed cx_Oracle driver in Django

[Florian Apolloner]

Hi Chris,

On Tuesday, June 7, 2022 at 11:30:46 AM UTC+2 christopher@gmail.com 
wrote:

> The question is how to add support cleanly so that both names are 
> supported in 4.1?  Is there a preference?  Particularly how can it be be 
> done to reuse code without (temporary) duplication?
>

Good question, no idea :) I am having the same problem with psycopg3 and 
this is what I did 
https://github.com/django/django/pull/15687/files#diff-01f6880f77beca32ee83e011072ba73dc7eed7f9f3efdebd935af693a4fac7b3
 
-- basically I added a compatibility module which does imports from the 
respective locations, with the idea of being able to simply run sed over 
this in the future. I know this isn't much of an answer and the differences 
between psycopg2 & 3 are rather minimal but maybe a similar approach is 
viable for oracle as well?
 

> Regarding connection options, yes some things are supported with the Easy 
> Connect Plus syntax.  There are some connection properties that aren't 
> supported, e.g  application contexts (this was also true of cx_Oracle). And 
> maybe more in future depending what users ask for. A generic way to set 
> these, e.g with OPTIONS is a good direction.  
>

Yes, I'd use the existing backends as guidelines, common stuff like HOST 
etc fits into the toplevel imo.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8f4fb80c-b0b6-4125-9816-f31651d55c9dn%40googlegroups.com.

Re: Feature Idea: Allow setting session cookie name dynamically

[Florian Apolloner]

Hi Maciej,

You can already customize the cookie name via a setting. What this request 
is asking is customization based on the request object which is not that 
common. Did you check that you configured your applications correctly to 
use different cookie names 
(https://docs.djangoproject.com/en/4.0/ref/settings/#session-cookie-name)?

> I wonder if using a Django project code name as part of session cookie 
for new projects would have a potential to be considered as an accepted 
feature.

has not been considered (as far as I know) and is something I'd be strongly 
against.

Cheers,
Florian

On Monday, June 6, 2022 at 11:51:24 PM UTC+2 macie...@gmail.com wrote:

> Hi Dan and Carlton,
>
> In my current company I am impacted by conflicting session cookie name. We 
> have several internal tools built on Django, available in the internal 
> network under the same top-level domain. The scope of session cookies 
> apparently was set on more than one service to a wildcard. Majority of 
> users use only one of services in day-to-day work, so it's not impacting a 
> big number of people.
>
> I personally would be +1 for exposing a method to easily and without 
> copying much code one was able to change it.
>
> I wonder if using a Django project code name as part of session cookie for 
> new projects would have a potential to be considered as an accepted feature.
>
> Kind regards,
> Maciej
>
> pon., 6 cze 2022 o 22:19 Dan Strokirk  napisał(a):
>
>> Hi Carlton, thanks for the response.
>>
>> An external package might be useful, although the code majority of the 
>> code would be the copied SessionMiddleware code and the tiny changes to 
>> allow a dynamic cookie name, so my thoughts is that this might be "too 
>> small" for a published pypi package?
>>
>> But since I haven't found a similar request earlier on this mailing list 
>> or the issue tracker, it seems that it might be really niche, as you say!
>>
>> Best regards,
>> Dan
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/01a91019-8d54-4322-a295-dbfdc12dfab9n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/08a438ff-9d7a-47e0-abe2-d68a7dd20935n%40googlegroups.com.

Re: Supporting Oracle's renamed cx_Oracle driver in Django

[Florian Apolloner]

Hi Chris,

a thin client seems like great news to me :)

On Friday, May 27, 2022 at 3:05:51 PM UTC+2 christopher@gmail.com wrote:

> 1a. What about completely replacing cx_Oracle with python-oracledb 
> (oracledb)? Python-oracledb is just a major release of cx_Oracle with a new 
> name.  We have run Django tests.  The new driver is 'thin' by default so 
> it's easier for almost everyone to use.
>

Assuming the differences between the two drivers are not to big I'd would 
be great if the existing backend would support both names for now. Then we 
can drop the cx_oracle imports relatively easily after a release or two.
 

> 1b. Alternatively a configuration option needs to allow either cx_Oracle 
> or python oracledb to be used.  Perhaps this could be done by creating 
> separate sub-modules, one for each name space, inside 
> django.db.backends.oracle? If so, then the ‘ENGINE’ values could be ...
>

I am against this, if python-oracledb is the new cx_oracle I guess 
cx_oracle won't really be developed anymore (correct me if I am wrong) and 
as such I do not see much point in supporting both. Oracle is a beast on 
it's own, one oracle driver to handle is more than enough :)

*2. Letting users choose whether to use python-oracledb Thin or Thick modes*
>
> 2a. It could be documented that users need to explicitly call 
> init_oracle_client() in their own code.  This may be easiest.
>

Would be fine for me if the testsuite passes with thin mode.
 

> 2b. Or how about adding a new parameter, say ‘DRIVER_MODE’ or simply 
> ‘MODE’, in settings.dict to allow users to use the ‘Thick’ mode of 
> python-oracledb?  Django would need to make a call to init_oracle_client() 
> internally.  This would need to take an optional parameter to be passed as 
> the lib_dir argument to init_oracle_client()
>

Should be fine I think, lets see what others think about that. But the 
config approach would certainly be the preferred way of doing things over 
the documentation approach (in the longrun at least). Then again I am not 
really sure which features thick mode offers and whether Django developers 
will actually need that often… You probably know that better than I do.

*3. Supporting some new connection parameters and using service names 
> instead of SIDs*
>
> One new feature of the python-oracledb driver are additional connect() 
> keyword arguments like hostname, port, service_name, tcp_connect_timeout.  
> (The makedsn() function is now deprecated in python-oracle because of 
> this change.)  How can new parameters best be passed to python-oracledb?
>

I'd say use get_connection_params & get_new_connection 
https://github.com/django/django/blob/8c0886b068ba4e224dd78104b93c9638b860b398/django/db/backends/oracle/base.py#L245-L259
 
-- or do I miss something here.

It would be nice if apps could do something like:
>
>DATABASES = {
>'default': {
>'ENGINE': 'django.db.backends.oracle',
>'USER': 'scott',
>'PASSWORD': '',
>'OPTIONS': {
>'host': 'example.com',
>'port': 1521,
>'service_name': 'orclpdb',
>'tcp_connect_timeout': 10,
>   },
>}
>}
>

I think obvious keys like host & port should be kept in the toplevel like 
we already do 
https://docs.djangoproject.com/en/4.0/ref/databases/#connecting-to-the-database-1
 
and the NAME could become the service_name (though I do see how this could 
clash with the current usage)? Then again we are not 100% consistent all 
the time here, so whatever feels more natural I guess?

I hope this helps somewhat. Let us know if there is more you'd like to know.

Cheers,
Florian


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9032cdc0-5c59-4df8-a595-820434b33cddn%40googlegroups.com.

Re: Status of 4.1 pre-release.

[Florian Apolloner]

On Thursday, May 12, 2022 at 1:33:38 PM UTC+2 thinkwel...@gmail.com wrote:

> > Next step would be someone to pick the preliminary work up and push it 
> forward.
>
> I guess I thought the "preliminary work" was already done. There's a 
> driver written, and a PR for a django backend.
>

I don't think there is a PR anywhere yet. The preliminary work is indeed 
done (or mostly, I do not know the full state of 
https://github.com/dvarrazzo/django-psycopg3-backend/ ), but as Carlton 
pointed out it now needs someone to make a proper PR out of it. That one 
could be you!

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0e25fd8a-6298-4ff7-bb68-a003f6b28a81n%40googlegroups.com.

Re: general interest in faster bulk_update implementation

[Florian Apolloner]

Hi Jörg,

Regarding your question about database version support: 
https://docs.djangoproject.com/en/4.0/ref/databases/ lists the supported 
versions. Granted not as support grid and you have to scroll to every 
database, but it is usually in the first paragraph.

If f-expressions are currently supported your implementation could fall 
back to a slower implementation if it encounters them?

Cheers,
Florian 

On Sunday, April 24, 2022 at 10:19:16 PM UTC+2 Jörg Breitbart wrote:

> Hi there,
>
> I have recently tried to tackle the known performance issues of 
> bulk_update. For more context plz see the original thread: 
> https://code.djangoproject.com/ticket/31202
>
> In another issue we kinda ended up at the same idea again for a 
> different reason (https://code.djangoproject.com/ticket/33647).
>
> My early attempt for a faster implementation: 
> https://github.com/netzkolchose/django-fast-update
>
> This impl has several drawbacks/restrictions which I'd like to discuss, 
> before wasting more time to get things more aligned with the ORM, esp 
> these aspects:
> - missing f-expression support
> - only support for recent db engines
>
> @f-expressions
> This is not directly possible from a constant table (thats how postgres 
> calls tables created from VALUES(...)). At this point idk if 
> f-expressions is an actively used thing with bulk_update at all, or if 
> support for those would just penalize the update throughput for no good 
> reason. My impl above currently skips such a workaround. Would like to 
> hear some more opinions about that.
>
> @db engine support
> Are there any strict db support promises made by django releases? I see 
> that some django versions have notes about certain db engines and 
> versions, but I cant find a "support grid" or an explicit doc for that. 
> If django does not officially restrict db versions, then the impl above 
> would need some sort of a fallback for unsupported db versions.
>
> Plz also tell me, if this all sounds too wild for getting considered 
> into django (tbh the db version dependencies bug me alot), then it 
> prolly is better kept in a 3rd party package.
>
> Regards,
> jerch
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7f6e3d0b-1bf0-4c41-8877-f9a28913d2ccn%40googlegroups.com.

Re: MFA (2FA)

[Florian Apolloner]

Hi Yonas,

that is an unfair characterization of WebAuthn. WebAuthn supports 
passwordless authentication as strong first factor (albeit often supporting 
a limited number of credentials because it requires storage on the device). 
But Webauthn also (and this is imo more widely used) supports a strong 
unphishable second factor. So no, we are not going to treat it as an 
alternative for the auth system; it is the one 2FA system that we want the 
most.

Cheers,
Florian

On Saturday, April 9, 2022 at 2:31:54 AM UTC+2 Yonas wrote:

> Hi Florian,
>
> WebAuthn promotes password-less authentication, so let’s treat it as an 
> alternative to the Django auth system while implementing 2FA for the 
> password-based Django auth. 
>
> On Friday, April 8, 2022 at 8:56:18 PM UTC+3 f.apo...@gmail.com wrote:
>
>> Hi Yonas,
>>
>> On Friday, April 8, 2022 at 3:18:23 AM UTC+2 Yonas wrote:
>>
>>> There are multiple ways to implement MFA, as you mentioned. But the goal 
>>> here is to provide a simple mechanism. It's "not necessary" to cover every 
>>> use case, and I believe that's where third-party packages come in.
>>>
>>
>> While it is not required to  cover every usecase, WebAuthn would be at 
>> the top of the list. I do not think adding MFA to core without having 
>> support for WebAuthn is going to  get much traction.
>>
>> Cheers,
>> Florian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e6e03feb-3c77-4cfa-b318-2b0354f2faefn%40googlegroups.com.

Re: Ticket #21289 - Login rate limiting

[Florian Apolloner]

Hi Jacob,

I am afraid this does not help much at all. Assuming a malicious client 
wants to attack you, they can still just issue one request to get this 
"other hidden field". Then they wait 5 seconds and are free to send 
thousands of requests with that token (Well till it expires, then they need 
a new one. You can probably also not easily make a single use token because 
that would require state storage on the server -> back to square one). Even 
if we were to bind this token to source IPs etc, all you are doing is 
slowing down the whole attack by five seconds. And this is a static one 
time fee to pay for the client, which compared to the number of tries they 
need (over days or so) is not much.

Or do I miss something important here?

Cheers,
Florian

On Tuesday, April 5, 2022 at 4:04:54 PM UTC+2 jacob...@gmail.com wrote:

> How about this proposal?
>
> Someone opens the login page. In addition to the visible 
> fields username and password and the hidden field csrftoken we add another 
> hidden field. This field contains the earliest (server-)timestamp a user 
> might login, and lies in the near future, for instance now() + 
> timedelta(seconds=5). That value is cryptographically signed 
> 
> .
>
> In addition to this, we disable the submit button and add a small 
> Javascript function which sets an interval corresponding to the mandatory 
> login delay. After that interval expired, the submit button is enabled 
> again.
>
> A malicious client who bypasses the disabled button and attempts to submit 
> the login for, will receive a HTTP response with an error code > 400.
>
> What are the advantages?
>
>- Django doesn't have to store any state of users and/or IP addresses 
>attempting to log in.
>- Django doesn't have to delay itself to throttle requests. This btw. 
>is a DoS attack vector by blocking server threads.
>- We transfer responsibility for delaying login requests to the client 
>– who can't bypass them.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3ca88e7e-70a9-4ba0-8521-cac5f231cb42n%40googlegroups.com.

Re: MFA (2FA)

[Florian Apolloner]

Hi Yonas,

On Friday, April 8, 2022 at 3:18:23 AM UTC+2 Yonas wrote:

> There are multiple ways to implement MFA, as you mentioned. But the goal 
> here is to provide a simple mechanism. It's "not necessary" to cover every 
> use case, and I believe that's where third-party packages come in.
>

While it is not required to  cover every usecase, WebAuthn would be at the 
top of the list. I do not think adding MFA to core without having support 
for WebAuthn is going to  get much traction.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/797b982e-d5d9-4637-8f3e-5a29154a6aa3n%40googlegroups.com.

Re: Revisiting MSSQL and Azure SQL Support in Django

[Florian Apolloner]

Hi Tim,

On Friday, April 1, 2022 at 5:02:00 PM UTC+2 Tim Allen wrote:

> The DB popularity index at db-engines.com has regularly listed the top 
> four as Oracle, MySQL, Microsoft SQL Server, and PostgreSQL, in that order. 
> I notice some comments in this thread about Microsoft being for-profit... 
> well, what about Oracle? I don't see Oracle on the Support Django page 
> either, yet two of their databases have support in core. MSSQL is the only 
> one of the big-four RDBMS's without support in core Django. That seems to 
> be a pretty big hole in Django's offering.
>

I understand that comparing to existing databases seems like to make an 
argument for MSSQL but it is not that easy. First off, Django was released 
way earlier than Oracle acquired MySQL -- I think it is understandable that 
we do not simply drop support for a database just because Oracle buys it. 
Postgres and MySQL (or now MariaDB) are both easily installable via every 
Linux distribution and have been there since the beginning. Support for 
Oracle itself (iirc) was added because the team at that point in time 
thought it would enable Django to get access to areas where it hadn't 
access before. Oracle itself has (imo) been proven to be quite a burden 
over time and there had been discussion about removing it from core more 
than once.

Truth to be told, if the inclusion request for MSSQL gets serious we will 
have to start a discussion about whether or not we simply nuke all database 
backends (aside from sqlite maybe) from core. I'd be slightly in favor of 
simply setting a policy for core that it should only include backends of 
OSS databases. Why? Because it is way easier to install those on various 
systems than their commercial alternatives (even if there are test licenses 
and possibly free containers around there).
 

> They have put a lot of time and effort into this project, and I think 
> they're well on their way to where they need to be for the long-term goal 
> of being in core Django.


I applaud to that, but I still do not understand why a well maintained 
database backend needs to be inside core? And I am not just talking about 
database backends here, we are saying no to pretty much every library 
inclusion.
 

> A lot of the questions being asked of Microsoft in this thread just don't 
> seem fair to me - we're not asking the same of Oracle, Redis Enterprise 
> Software, or any of the other commercial products that Django has built-in 
> support for. Why Microsoft and not the others?
>

We are asking Oracle but we are also not getting far, and simply kicking it 
out is not something we do easily. As for Redis, as far as I am concerned 
(and to the extend we support it) is open source.

Cheers,
Florian

>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/58642523-6754-4869-b85b-f065042a4e6bn%40googlegroups.com.

Re: Revisiting MSSQL and Azure SQL Support in Django

[Florian Apolloner]

Hi Waren,

On Thursday, March 31, 2022 at 6:30:06 PM UTC+2 vwa...@gmail.com wrote:

> We'd love to hear thoughts and feedback around the possibility of moving 
> forward with a DEP enhancement proposal, with a commitment from Microsoft 
> to providing continued dedicated support for the 1st party backend through 
> the Django project itself (rather than the 3rd party repo).
>

As pointed out by Adam I do not think the backend is ready for core. At the 
minimum it would need to support most of what Django offers. Then there is 
also the question on how to ensure continued commitment from Microsoft -- 
what happens when Microsoft (or the current PM/PO or whatever) looses 
interest? The backend would be in Django then and we can't easily kick it 
out.

Given the backend APIs and the recent enhancements by Tim to allow backends 
to define which tests they can't pass etc I do not see any strong technical 
argument for including a new backend in core at all. Aside from a selling 
argument for Microsoft that Django includes the backend in core, what do we 
gain? It is not like one usually uses Django without installing any 3rd 
party dependencies, so installing one more certainly won't hurt.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/826cc91f-1621-4255-8679-ef7e8da1cc09n%40googlegroups.com.

Re: RFC #33561 -- Synchronize user attributes on every authentication with RemoteUserBackend

[Florian Apolloner]

Hi Adrian,

On Saturday, March 5, 2022 at 5:13:14 PM UTC+1 ator...@redhat.com wrote:

> - Existing RemoteUserBackend implementations won't need to change 
> signatures whenever backwards compatibility is removed
> - RemoteUserBackend implementations won't need to do anything in order to 
> support Django versions in which the feature doesn't exist (e.g. 3.9) and 
> versions in which the feature exists and is not backwards-compatible (e.g. 
> 5.1)
>

While this is true, the "migration" path for implementors is simply to 
change the function to something like:

```
def configure_user(self, request, user, created=False):
if not created: return
   … do whatever you already did 
```

and it will stay compatible with current behavior and the new behavior.
 

> - The code footprint within Django, not counting documentation and tests, 
> is like 3 LOC
>

I do understand that, but I don't think it is as simple as that. With the 
separated methods (unless we pass created to synchronize_user as well) 
you'd do synchronization always even when the user was just created even 
though configure_user could take care of that in one go.

- Anyone who wants to extend a RemoteUserBackend implementation can easily 
> and cleanly extend/replace the synchronization and initial setup 
> independently of each other, if everything is done in the same method this 
> can get messy
>

I do not really think this will be the case since those methods are empty 
by default… Those backends are so simple that at some point one can simply 
write their own if subclassing might become to much of a hazzle.
 

> def configure_user(self, request, user, created):
> if created:
> user = self.initial_configuration(request, user)
> user = self.synchronize(request, user)
> return user
>
> Which is the same as having two separate methods for initial configuration 
> and synchronization, but with extra steps.
>

I do not think this will be common though. I rather think that usually one 
would do the same thing on creation and on sync. 
 

> Have a good weekend
>

Thanks, you as well!

Florian 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a5cc89c2-1dbf-4a97-87e6-8744e8e5b64fn%40googlegroups.com.

Re: RFC #33561 -- Synchronize user attributes on every authentication with RemoteUserBackend

[Florian Apolloner]

Hi Adrian,

I agree this would be nice to have.

On Friday, March 4, 2022 at 8:03:09 PM UTC+1 ator...@redhat.com wrote:

> Another idea would be to use configure_user() for both initial 
> configuration and synchronization by passing an extra parameter "created" 
> to it, and calling it just before the authenticate method's return line, 
> but I figured this change would be more disruptive for existing 
> implementations. 
>

I do prefer that approach. It is not more disruptive (or at least only 
marginally) and only means more work (backwards compat warnings) when 
implementing this. We can easily inspect the existing signature and only 
pass the boolean when supported and in the backwards compatibility period 
simply do not support created=False when the user didn't change their 
configure_user signature. I'd hate seeing two methods that basically do the 
same.

Cheers,
Florian 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9db3b605-a8fc-40ab-a318-0ff1e7a76eb3n%40googlegroups.com.

Re: Google Group's django-updates issue

[Florian Apolloner]

Oh. AWS just hates us :) Thanks, we are looking into it.

On Friday, March 4, 2022 at 5:55:00 PM UTC+1 charettes wrote:

> Hi everyone,
>
> I'm not sure if we're already aware of the issue but it seems that Trac 
> hasn't published any changes to the Django Updates Google Group[0] since 
> March 27th while there was clear activity that took place since then[1].
>
> It seems the updates were published over SMTP[2] and I assume something 
> broke around the time the latest changes were pushed to the repository on 
> the 27th[3].
>
> Cheers,
> Simon
>
> [0] https://groups.google.com/g/django-updates
> [1] https://code.djangoproject.com/timeline
> [2] 
> https://github.com/django/code.djangoproject.com/blob/19a7a14206dea8485ef8e2befed2b8e406208b53/trac-env/conf/trac.ini#L111
> [3] 
> https://github.com/django/code.djangoproject.com/commit/19a7a14206dea8485ef8e2befed2b8e406208b53
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/165b49b1-e2cd-4f70-a14f-bfddb11368den%40googlegroups.com.

Re: Proposal on how add configuration options to Email Backends

[Florian Apolloner]

Okay then,

some of the things like sender reputation and different bounce hooks came 
to my mind as well, but it is good to hear confirmation from others. I 
think the next steps would be to create a new ticket to add support for 
*multiple* email backends and then work from that (I would only link the 
old ticket since it's scope was mainly putting the config into a dict as 
opposed to multiple backends). Given that there are plenty of +1 here 
already I think we already have our implementors? :)

Some items that I like to see addressed in a PR:
 * Backwards compat
 * Similarity to Caches & Databases (ie so we don't invent yet another 
syntax)
 * Support for connection aliases (default/…) in send_email % friends 
(basically everything taking a connection now should probably take aliases 
as well)

Cheers,
Florian

On Sunday, January 30, 2022 at 11:07:56 PM UTC+1 Adam Johnson wrote:

> Another situation you might want multiple backends is when switching 
> providers. Rather than big-bang swap *all* email sends to a new provider, 
> you might want to move only low-value emails first, or a percentage of your 
> user base, and iterate.
>
> On Sun, 30 Jan 2022 at 20:59, Steven Mapes  wrote:
>
>> I should add I also have other instances my clients use a custom AWS SES 
>> backend I wrote used for various types of sends (bulk, newsletters) but 
>> generally sends where they need SNS notifications to trigger HTTPS 
>> postbacks for certain results from the send such as bounced mails, spam 
>> reports etc where those the requests go back to the django server to be 
>> handled whereas other mail sends may go via O365 / Google for general, 
>> often internal, notifications and emails send.
>>
>> I have another client who has two mail providers where one is the primary 
>> and if that fails due to the SMTP server being down, the platform othen 
>> fails over to re-send via the backup SMTP server.
>>
>> *Steve Mapes*
>> Steven Mapes T/A Jigsaw Tech
>> E: st...@jigsawtech.co.uk 
>> <https://link.getmailspring.com/link/cf847694-1156-4ba3-bc92-5f42d434b...@getmailspring.com/0?redirect=mailto%3Asteve%40jigsawtech.co.uk=ZGphbmdvLWRldmVsb3BlcnNAZ29vZ2xlZ3JvdXBzLmNvbQ%3D%3D>
>> P: +44(0)7974220046
>> W: https://www.jigsawtech.co.uk/ 
>> <https://link.getmailspring.com/link/cf847694-1156-4ba3-bc92-5f42d434b...@getmailspring.com/1?redirect=https%3A%2F%2Fwww.jigsawtech.co.uk%2F=ZGphbmdvLWRldmVsb3BlcnNAZ29vZ2xlZ3JvdXBzLmNvbQ%3D%3D>
>> [image: LinkedIn] 
>> <https://link.getmailspring.com/link/cf847694-1156-4ba3-bc92-5f42d434b...@getmailspring.com/2?redirect=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fstevemapes%2F=ZGphbmdvLWRldmVsb3BlcnNAZ29vZ2xlZ3JvdXBzLmNvbQ%3D%3D>[image:
>>  
>> Twitter] 
>> <https://link.getmailspring.com/link/cf847694-1156-4ba3-bc92-5f42d434b...@getmailspring.com/3?redirect=https%3A%2F%2Ftwitter.com%2Fstevenamapes=ZGphbmdvLWRldmVsb3BlcnNAZ29vZ2xlZ3JvdXBzLmNvbQ%3D%3D>
>> The entire content of this email message is confidential. This also 
>> applies to any files attached to it. This email is intended for an 
>> individual or entity to whom they are addressed. In case you are not the 
>> addressee of this email, and you have received it in error, contact the 
>> system manager immediately. The information could be very sensitive, and it 
>> is intended for the specific addressee. This email should not be 
>> disseminated, distributed or copied. If you have received this email and it 
>> was not for you, please notify the sender by email immediately and 
>> afterward delete this email from your system. Disclosing, copying, 
>> distributing, or taking any action in reliance on the email content is 
>> strictly prohibited.
>> On Jan 30 2022, at 3:38 pm, Florian Apolloner  wrote:
>>
>> I do not understand why you would need multiple email backends to send 
>> from different addresses. Can you elaborate on that?
>>
>> [image: Sent from Mailspring]
>> On Sunday, January 30, 2022 at 1:18:48 PM UTC+1 st...@jigsawtech.co.uk 
>> wrote:
>>
>> I've a big +1 on changing email config to a dictionary to support 
>> multiple backends as it's very much a common occurrence for both clients of 
>> mine and for my own businesses. Most of the use cases are when they main 
>> site sends emails from no-reply@ such as for password resets but then when 
>> alternative email are required for sales and/or customer service email 
>> address where it's handled via the website. Currently I end up creating a 
>> custom settings.py dictionary to store the settings so I can then refer to 
>> that using the connection for swapping the backend to send from.
>>
>> On Sunday, 30 

Re: Proposal on how add configuration options to Email Backends

[Florian Apolloner]

I do not understand why you would need multiple email backends to send from 
different addresses. Can you elaborate on that?

On Sunday, January 30, 2022 at 1:18:48 PM UTC+1 st...@jigsawtech.co.uk 
wrote:

> I've a big +1 on changing email config to a dictionary to support multiple 
> backends as it's very much a common occurrence for both clients of mine and 
> for my own businesses. Most of the use cases are when they main site sends 
> emails from no-reply@ such as for password resets but then when alternative 
> email are required for sales and/or customer service email address where 
> it's handled via the website. Currently I end up creating a custom 
> settings.py dictionary to store the settings so I can then refer to that 
> using the connection for swapping the backend to send from.
>
> On Sunday, 30 January 2022 at 11:14:54 UTC f.apo...@gmail.com wrote:
>
>> Hi Jacob,
>>
>> I wouldn't be opposed to move email configuration into a dictionary 
>> (somewhere between -0 and +0). Although if we plan to do that we should 
>> rethink all the existing session variables and other as well I guess and 
>> figure out if we should move more settings to dictionaries. 
>>
>> > why shouldn't it makes sense to have different email backends? If you 
>> have a staging system you may want to use you local SMTP-relay, while in 
>> production
>> you may for instance use AWSs SES 
>>  service.
>>
>> This specific example at hand is imo not a good motivator to add support 
>> for multiple backends because the settings would imo be different. Take 
>> databases as an example: You also don't have staging/production in there 
>> but switch the actual values in the default database.
>>
>> > `EMAIL = [...]`
>>
>> I am not sure a list makes sense here and would go for similarity with 
>> CACHES & DATABASES since you'd usually identify the backend via a unique 
>> name or so. Also DATABASES & CACHES have an OPTIONS dict which is the 
>> passed on to the backend, I think we should follow suit here.  
>>
>> > Personally, I would prefer SMTP = {...}
>>
>> I do not think SMTP would be a good fit because many services allow HTTP 
>> submission, so what we are sending is actually an email and smtp is just a 
>> protocol implementation in the backend of AWS SES or so.
>>
>> As for other email backends that do require different options: I do not 
>> see an issue when they simply take `EMAIL_AWS_SES_KEY` and document it as 
>> such; this doesn't require us to add more flexibility to email backends…
>>
>> So I guess it boils down to the following questions:
>>
>>  * Do we want to support multiple (at the same time) email backends, if 
>> yes we would move to a settings dict anyways…
>>  * If the answer to the above is no, what value does putting it into a 
>> single dict give us?
>>
>> In the past I think I have argued for a SECURITY_HEADERS (or similar) 
>> dict because it allows us to check the dictionary keys easily for typos; 
>> emails probably don't suffer from that problem as badly as security related 
>> settings.
>>
>> I hope this can get the discussion going.
>>
>> Cheers,
>> Florian
>> On Sunday, January 30, 2022 at 9:29:27 AM UTC+1 jacob...@gmail.com wrote:
>>
>>> Well, that ticket is 8 years old and in the meantime other email 
>>> backends have emerged, requiring different configuration options.
>>> I made this proposal after attempting to fix a 14 year old open ticket 
>>> #6989 but this was ultimately postponed, see comment by
>>> Carlton Gibson on 
>>> https://github.com/django/django/pull/13728#issuecomment-987762791
>>>
>>> To summarize the discussion from 7 years ago
>>>
>>> Collin Anderson wrote:
>>>
 I don't see any benefit to moving email settings to a dictionary. It is 
 helpful for databases and caches because there can be multiple backends.  
>>>
>>> It makes the popular "from local_settings import *" convention harder to 
 use. What's wrong with 6 individual settings? If the goal is to allow
>>>
>>> multiple email backends, then let's make that the ticket goal.
>>>
>>>  
>>> and Carl Meyer replied:
>>>
 I agree with Collin; unless we are adding new capabilities (i.e. 
 multiple configured email backends, which it seems nobody really wants), 
 it's hard 
>>>
>>> to find any actual benefit from this change to justify the churn (and 
 the additional complexities of working with dictionary settings in 
 partial-override scenarios).
>>>
>>>  
>>> why shouldn't it makes sense to have different email backends? If you 
>>> have a staging system you may want to use you local SMTP-relay, while in 
>>> production
>>> you may for instance use AWSs SES 
>>>  service. That 
>>> service may require additional configuration settings not available in the 
>>> local smtp backend.
>>> I can also imagine that in some situations it may make sense to have two 
>>> email backends 

Re: Proposal on how add configuration options to Email Backends

[Florian Apolloner]

Hi Jacob,

I wouldn't be opposed to move email configuration into a dictionary 
(somewhere between -0 and +0). Although if we plan to do that we should 
rethink all the existing session variables and other as well I guess and 
figure out if we should move more settings to dictionaries. 

> why shouldn't it makes sense to have different email backends? If you 
have a staging system you may want to use you local SMTP-relay, while in 
production
you may for instance use AWSs SES 
 service.

This specific example at hand is imo not a good motivator to add support 
for multiple backends because the settings would imo be different. Take 
databases as an example: You also don't have staging/production in there 
but switch the actual values in the default database.

> `EMAIL = [...]`

I am not sure a list makes sense here and would go for similarity with 
CACHES & DATABASES since you'd usually identify the backend via a unique 
name or so. Also DATABASES & CACHES have an OPTIONS dict which is the 
passed on to the backend, I think we should follow suit here.  

> Personally, I would prefer SMTP = {...}

I do not think SMTP would be a good fit because many services allow HTTP 
submission, so what we are sending is actually an email and smtp is just a 
protocol implementation in the backend of AWS SES or so.

As for other email backends that do require different options: I do not see 
an issue when they simply take `EMAIL_AWS_SES_KEY` and document it as such; 
this doesn't require us to add more flexibility to email backends…

So I guess it boils down to the following questions:

 * Do we want to support multiple (at the same time) email backends, if yes 
we would move to a settings dict anyways…
 * If the answer to the above is no, what value does putting it into a 
single dict give us?

In the past I think I have argued for a SECURITY_HEADERS (or similar) dict 
because it allows us to check the dictionary keys easily for typos; emails 
probably don't suffer from that problem as badly as security related 
settings.

I hope this can get the discussion going.

Cheers,
Florian
On Sunday, January 30, 2022 at 9:29:27 AM UTC+1 jacob...@gmail.com wrote:

> Well, that ticket is 8 years old and in the meantime other email backends 
> have emerged, requiring different configuration options.
> I made this proposal after attempting to fix a 14 year old open ticket 
> #6989 but this was ultimately postponed, see comment by
> Carlton Gibson on 
> https://github.com/django/django/pull/13728#issuecomment-987762791
>
> To summarize the discussion from 7 years ago
>
> Collin Anderson wrote:
>
>> I don't see any benefit to moving email settings to a dictionary. It is 
>> helpful for databases and caches because there can be multiple backends.  
>
> It makes the popular "from local_settings import *" convention harder to 
>> use. What's wrong with 6 individual settings? If the goal is to allow
>
> multiple email backends, then let's make that the ticket goal.
>
>  
> and Carl Meyer replied:
>
>> I agree with Collin; unless we are adding new capabilities (i.e. multiple 
>> configured email backends, which it seems nobody really wants), it's hard 
>
> to find any actual benefit from this change to justify the churn (and the 
>> additional complexities of working with dictionary settings in 
>> partial-override scenarios).
>
>  
> why shouldn't it makes sense to have different email backends? If you have 
> a staging system you may want to use you local SMTP-relay, while in 
> production
> you may for instance use AWSs SES 
>  service. That 
> service may require additional configuration settings not available in the 
> local smtp backend.
> I can also imagine that in some situations it may make sense to have two 
> email backends concurrently. We maybe should rethink about that.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/27c2ed9d-92f4-49e3-b770-a74e37cff368n%40googlegroups.com.

Re: Why are session cookies not considered sensitive?

[Florian Apolloner]

Hi,

I agree that we should treat session cookies as sensitive and hide them 
like we do with passwords. That said, please be aware that all the 
SafeException reporters are best effort and it is generally not possible to 
have a "safe" exception.

In that sense, patches welcome but we are not going to treat this as 
security issue (ie no backporting).

Cheers,
Florian

On Monday, January 17, 2022 at 10:09:07 PM UTC+1 Tobias Bengfort wrote:

> Hi,
>
> AFAIU, SafeExceptionReporterFilter takes care of removing any sensitive 
> data from logs. However, I today realized that this does not cover 
> session cookies.
>
> In a ticket about this issue[1] it was treated not as a security issue 
> but more as a request for customization. That puzzled me a bit. Why are 
> session cookies not treated as sensitive, just like passwords are?
>
> thanks,
> tobias
>
>
> [1]: https://code.djangoproject.com/ticket/29714
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a2b299d1-6700-487d-ae45-37c6612e33ffn%40googlegroups.com.

Re: Django security releases issued: 4.0.1, 3.2.11, and 2.2.26

[Florian Apolloner]

Hi,

On Tuesday, January 4, 2022 at 2:09:18 PM UTC+1 Sjoerd Job Postmus wrote:

> Unfortunately (at least for us), this breaks the case where dictsort was 
> used with a static argument that looked up a callable. A quick code search 
> showed that the pattern dictsort.*get​ (
> https://github.com/search?q=%22dictsort.*get%22=Code) is less used 
> than I would expect it to be used, but used nonetheless. 
>

I expected as much; literally every security fix nowadays will break some 
code. Sorry about that. You can change it to a property or write your own 
filter I guess.

On the other hand, searching publicly viewable code for cases where 
> dictsort gets a dynamic value yields very little results: 
> https://grep.app/search?q=dictsort%3A%5B%5E%22%270-9%5D=true=true,
>  
> only one which I can recognize as a Django template: 
> https://github.com/crodas/Haanga/blob/develop/tests/assert_templates/regroup.tpl#L3
>  .
>

Yes, that said there is no easy way for a template filter to determine 
whether it is a variable or a literal (I'd say it is simply impossible). 
Our first approach would have been to limit only those values provided via 
variables… I do not think that is possible in a nice way though and 
probably to much for a security patch.

Since the previous behaviour of allowing callables was in place already in 
> 2005, (I could find 
> https://github.com/django/django/commit/ed114e15106192b22ebb78ef5bf5bce72b419d13#diff-e05e2e8efbf1fa5eea1ffee16cc8b740cba7b1bff746b2e55cebf968a0983f2cR192,
>  
> where the filter is introduced), I would argue that even though it may not 
> have been explicitly documented that this syntax allows callables, I don't 
> think it's far fetched to consider it to always have supported.
>

I would argue that the Django team always said that the documentation is 
the public API. Everything else works by luck. I even went as far as 
supporting lookups on objects (ie a list of objects as opposed to a list of 
dicts) because I assumed that people would use that often (and the docs 
clearly say this filter is for dicts).

However, before creating a ticket, I was wondering what the sentiment of 
> django-developers is here. My own sentiments are summarized by what's also 
> mentioned in the announcement blogpost.
>

>From a security PoV I think this is not going to fly. The main issue here 
is that dictsort should perform a rather limited subset of normal template 
variable resolution -- if we were going to support callables again we'd 
have to also support `alter_data` etc (which is forbidden in templates) and 
then we are basically back to what the previous code did. We opted for the 
most limited subset possible while allowing a relatively wide range of code 
to keep working, I don't think adding more features to that filter is 
feasible.
 

> >  As a reminder, all untrusted user input should be validated before use.
>
> As an example, even with the change, {% for user in 
> users|dictsort:"password" %} would still be supported, which is still be 
> counted as potential information disclosure.
>

There is a massive difference though. Being able to sort by every index 
inside the password means that you need to control far less password hashes 
than in your example to get a useful result. To be honest I do not think it 
is very realistic to use that attack on the password; but it might be 
usable for shorter api  tokens or so that you can also view in plaintext.

 I hope that helps.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6ee7a4a3-f0b3-451e-a250-71b2a31b4722n%40googlegroups.com.

Re: select_for_update and transactions (bad dev UX in a feature)

[Florian Apolloner]

Hi Aymeric,

On Saturday, November 20, 2021 at 12:39:17 PM UTC+1 Aymeric Augustin wrote:

> I'm not trying to disagree for the sake of disagreement; I'm just trying 
> to bring some contextual awareness and avoid  the "core devs say 
> ATOMIC_REQUESTS is bad" effect. I hope we can agree on this?
>

Absolutely, my comment was mostly there to show that not all core devs 
agree that ATOMIC_REQUESTS is something that should be always enabled. As 
usual it depends on your project, database etc… I should have added more 
context to that, thanks for pointing it out.
   

> Finally — yes, I know that my implementation of this feature is a wormhole 
> across layers. I won't even try to justify it. I wish we had something 
> better but we haven't found that yet.
>

I still think it is good (in the sense that we worked with what we had 
available). Sure the layering isn't exactly nice, but we have worse things 
in Django's codebase :) I even have a possible fix for that, but it 
requires us to rewrite middlewares again :/  

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6d65a33b-32ad-46a0-9b75-9c7f96c1fe03n%40googlegroups.com.

Re: select_for_update and transactions (bad dev UX in a feature)

[Florian Apolloner]

On Wednesday, November 17, 2021 at 11:09:58 PM UTC+1 Adam Johnson wrote:

> FWIW I always recommend enabling ATOMIC_REQUESTS and ensuring transactions 
> are used by default in other code paths such as background tasks.
>

FWIWI I always recommend disabling ATOMIC_REQUESTS and using transactions  
as needed :) Honestly long transactions are imo bad, one should evaluate 
what the view needs and then act accordingly.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ef5689c4-506a-4533-ab31-89ce496e2e43n%40googlegroups.com.

Re: Proposal for a transaction.on_before_commit

[Florian Apolloner]

Because the scoping of the transactions makes it hard I imagine. A view 
could start more than one transaction for instance, and one of those might 
succeed and the other might get rolled back. Thinking more about this, a 
system like this might get hard to implement. Even if thread locals are 
used, one might have to clear the list in cases of rollbacks or at least 
partially clear it for savepoint rollbacks. And all that doesn't even 
consider autocommit yet :) 

On Sunday, October 10, 2021 at 5:39:18 PM UTC+2 Shai Berger wrote:

> Just to clarify the use-case: 
>
> Why is a before-commit signal preferable to a vanilla Python 
> context-manager around the code? Or, if it is in the context of 
> requests, a middleware? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/549f5570-8cf3-4a1a-b381-e72a5d15d66en%40googlegroups.com.

Re: Why can't range function be added in templates?

[Florian Apolloner]

On Friday, October 8, 2021 at 6:14:19 PM UTC+2 dhruv...@gmail.com wrote:

> Ik range function can be added but I also mentioned that there are some 
> cases range function can't be direct generated from the view like this  
> https://github.com/Tanzanite-lpu/tgl-2.0.0/blob/ddce5ef87a4b5248b8cb8cd5fa3df4adb3f00b31/main/templates/groups.html#L36
>

Why was "|range" added there? You are not using {{ j }} there at all but {{ 
forloop.counter }}. There doesn't seem to be any reason to use "|range" 
there. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/987fbb10-1575-4897-b699-1bbdb0cf817an%40googlegroups.com.

Re: [TB] Proposal to add Florian Apolloner to Releasers team.

[Florian Apolloner]

On Wednesday, July 21, 2021 at 11:18:41 AM UTC+2 carlton...@gmail.com wrote:

> As such, I would like to nominate Florian as a new Releaser, and ask the 
> TB to discuss/vote on that in accordance with DEP 10. 
>

Given that I am on the TB I will not participate in the vote -- even though 
I do not expect my vote to change anything in the outcome.

(I didn't discuss this with him but, I hope that's OK with you Florian?) 
>

Given that I'd mostly serve as fallback I guess it would be fine (assuming 
we have a good documented release process :D)

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ffd80d2c-ee04-49fd-a0e0-c1fc6959e4b0n%40googlegroups.com.

Re: CTRL-BREAK still make sense on Windows for `runserver`?

[Florian Apolloner]

As long as runserver (usually) properly shuts down when CTRL-C is hit, then 
it imo makes sense to change the text -- especially if that is the only 
thing accessible on some keyboards.

The usually above is because someone might have a bare except that is hit 
that moment and as such the process might not shut down.

On Tuesday, June 22, 2021 at 7:58:47 AM UTC+2 carlton...@gmail.com wrote:

> I too realise I don't have a(n obvious) break key but have been happily 
> hitting CTRL-C. 
>
> On Monday, 21 June 2021 at 23:53:15 UTC+2 Adam Johnson wrote:
>
>> Would it be bad to have Django respond to both shortcuts?
>>
>
> I think as a matter of fact it does. At least using PowerShell/Win10/Etc — 
> Looking at Matthew's link, we don't do either of the things that would 
> inhibit this. 樂
>
> Looking at it from Will's POV (writing introductory guides) the output 
> text of runserver might be a bit problematic:
>
> "Quit the sever with CTRL-BREAK" — Errrmmm... Where's that then? 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/33caf74f-73d9-4886-bf11-d9727531cdb0n%40googlegroups.com.

Re: Transaction APIs do not consult the DB router to choose DB connection

[Florian Apolloner]

On Tuesday, June 1, 2021 at 2:35:17 PM UTC+2 gojeta...@gmail.com wrote:

> I don't see any reason for why providing a hook seems so difficult. 
>

It is more code to maintain, needs tests etc and increases complexity. Just 
because something is easy on the surface, doesn't mean it will be easy in 
the longrun.

A simple implementation can be: (From message-3 of this conversation):
>
>
> > #settings.py
>
> TRANSACTION_DB_SELECTOR = "path_to_some_callable"
>
> #transaction.py
> ...
> transaction_db_selector = import_string(settings.TRANSACTION_DB_SELECTOR)
> def get_connection():
> if transaction_db_selector:
> using = transaction_db_selector()
> if using is None:
> using = DEFAULT_DB_ALIAS
> return connections[using]
>

I do not think that completely ignoring the `using` that was passed in by 
the user would be a very good idea (for the case when there is also a 
`TRANSACTION_DB_SELECTOR` set).

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/016c2d90-302c-45d5-82de-48de66404146n%40googlegroups.com.

Re: Transaction APIs do not consult the DB router to choose DB connection

[Florian Apolloner]

On Tuesday, June 1, 2021 at 1:45:29 PM UTC+2 Aymeric Augustin wrote:

> I would recommend running without persistent database connections 
> (CONN_MAX_AGE = 0) and switching settings.DATABASE["default"] in a 
> middleware at the beginning of every request. Modifying settings at runtime 
> isn't officially supported, but this looks like the closest option to the 
> intent you expressed.
>

Funny that you should mention this. I thought about this when doing 
something similar for ARA, but settled down on a custom database backend: 
https://github.com/ansible-community/ara/blob/master/ara/server/db/backends/distributed_sqlite/base.py
 
-- it does work surprisingly well and I think it might be more resilient 
than settings shenanigans. Not that I am particularly proud of that backend 
but I thought I'd share it here for the off-chance someone finds your post 
via a search engine and wonders how to implement such a thing.

Cheers,
Florian  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/42ea1452-8699-41f5-b6c8-2b2d319477den%40googlegroups.com.

Re: Transaction APIs do not consult the DB router to choose DB connection

[Florian Apolloner]

On Monday, May 31, 2021 at 12:13:58 PM UTC+2 Adam Johnson wrote:

> I'm also -1 on changing anything in Django right now.
>

-1 as well, I simply see no way how something like:

```
with transaction.atomic():
 Model1.objects.create()
 Model2.objects.create()
```

will allow for any useful behavior via the router. Unless 
transaction.atomic could inspect the contents and figure out the involved 
dependencies, but this is surely way more than what could go into core for 
now.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6292ccbb-bc83-48d0-b08e-aa765a29ce32n%40googlegroups.com.

Re: APPEND_SLASH behavior

[Florian Apolloner]

Hi Chris,

nice hearing from you.

On Friday, May 7, 2021 at 6:20:44 PM UTC+2 chris.j...@gmail.com wrote:

> With the suggested work-around of having a view call other views, would a 
> view be able to continue URL resolution in that case? 
>

Not without many code changes I fear and I am not sure about the gains.
 

> To avoid the problems with the bad interaction with the request machinery, 
> another approach that came to mind would be to allow inserting / including 
> a function at any point in the URLconf. The function would return whether 
> the pattern should be skipped or claimed. That would have the advantage of 
> taking place before any view is started.
>

I'll put it like this: Interesting idea but I am afraid of the outcome :D 
Before I'd support such a change we'd really want to gather usecases first 
and think hard what it could/would break… And now I cannot stop thinking 
about such a function ala "lambda: random.choice([True, False])". Thank you 
for that ;)

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0c72e667-0223-4a28-8e4f-83c5d0f3c758n%40googlegroups.com.

Re: APPEND_SLASH behavior

[Florian Apolloner]

On Thursday, May 6, 2021 at 10:19:42 PM UTC+2 Adam Johnson wrote:

> That said, I think doing this in process_response would be preferable over 
>> doing it in process_request so the extra checks when the URL is valid (the 
>> common case) are reduced. Resolving URLs can take a bit, especially when 
>> the urlconf is long and as such I'd like to get that check out of the "hot" 
>> code path. Only doing the redirect in the case of a failing resolve in the 
>> first place would probably make this more efficient.
>
>
> I agree. I guess it might need a deprecation cycle to move it to 
> process_response though?
>

Maybe, maybe not. It already (partially?) does exist in process_response as 
well. If we can show that this doesn't break existing stuff we can do 
without deprecation.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e0fc2fa7-2b94-41f1-8136-cf8fa2e15bc7n%40googlegroups.com.

Re: APPEND_SLASH behavior

[Florian Apolloner]

Hi,

I took a quick glance (literally just that) at the pull requests. I do like 
the one that offers a way to abort early inside a prefix -- this is a nice 
optimization and as well might open interesting options for specialized 
catch all views. I am not convinced about the backtracking PR, which 
problems does this solve in reality? What does this mean for features like 
atomic requests -- it is still just one request after all… 

Cheers,
Florian

On Wednesday, May 5, 2021 at 7:21:01 PM UTC+2 atd...@gmail.com wrote:

> Ok I see better.
> Talking about efficiency, I take this opportunity to link here the 
> following draft PR I made: Backtracking URL Resolver 
>  and Provide ability to 
> abort URL resolution early , 
> which, if implemented in the right way, may help make URL resolving more 
> efficient and more flexible too.
>
> I understand that you may not have time to review  or maybe this is not 
> something you are planning to put into Django right now, but I would be 
> very happy to have any feedback on it!
>
> Le mercredi 5 mai 2021 à 17:19:43 UTC+2, f.apo...@gmail.com a écrit :
>
>> On Thursday, April 29, 2021 at 4:23:57 PM UTC+2 atd...@gmail.com wrote:
>>
>>> In both cases however, the current check being done in the 
>>> process_response method for the CommonMiddleware(here 
>>> )
>>>  
>>> seems irrelevant to me unless I am missing *something*. 
>>>
>>
>> You are most likely not missing anything and we also noticed that when 
>> fixing the admin enumeration stuff. But we did not get around to fixing it 
>> yet. That said, I think doing this in process_response would be preferable 
>> over doing it in process_request so the extra checks when the URL is valid 
>> (the common case) are reduced. Resolving URLs can take a bit, especially 
>> when the urlconf is long and as such I'd like to get that check out of the 
>> "hot" code path. Only doing the redirect in the case of a failing resolve 
>> in the first place would probably make this more efficient.
>>
>> Cheers,
>> Florian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4550509b-e628-47f9-b635-b4d6b74414e5n%40googlegroups.com.

Re: APPEND_SLASH behavior

[Florian Apolloner]

On Thursday, April 29, 2021 at 4:23:57 PM UTC+2 atd...@gmail.com wrote:

> In both cases however, the current check being done in the 
> process_response method for the CommonMiddleware(here 
> )
>  
> seems irrelevant to me unless I am missing *something*. 
>

You are most likely not missing anything and we also noticed that when 
fixing the admin enumeration stuff. But we did not get around to fixing it 
yet. That said, I think doing this in process_response would be preferable 
over doing it in process_request so the extra checks when the URL is valid 
(the common case) are reduced. Resolving URLs can take a bit, especially 
when the urlconf is long and as such I'd like to get that check out of the 
"hot" code path. Only doing the redirect in the case of a failing resolve 
in the first place would probably make this more efficient.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/baf9fa2e-c339-4fed-8d57-643ed2830385n%40googlegroups.com.

Re: Adding ability to choose AutoField type (signed vs unsigned)

[Florian Apolloner]

On Tuesday, April 6, 2021 at 7:08:06 PM UTC+2 Adam Johnson wrote:

> I also don't think this is necessary any more and can be closed.
>  
>
>> An explicit `id = PositiveBigIntegerField(...)` workaround would be fine 
>> to be honest, for those that need it.
>
>
> I also would like to meet the django app that *does* need it (for 
> non-silly reasons like deciding to start ID's near 2^63).
>

Since you have asked: It is required if you want to store a x509 
certificate serial number as is. We indeed get queries from integrators 
that have previously stored serial numbers in a big integer field. After 
the "recent"  cert library updates which now properly produce 64 bit 
positive integers like the CAB forum requires, they ran into plenty of 
problems with the first certificate serial that was larger than 2^63.

Map that to Django and I think changing to PositiveBigIntegerField is a 
fair workaround. Not sure if it is worth changing the default, but the 
requirements do exist for some people.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2a5a9a02-70d6-4f77-9af0-c9217cb9eb3an%40googlegroups.com.

Re: Update - MSSQL support for Django

[Florian Apolloner]

Maybe wait with featuring till the licensing issues are cleared up. 
https://github.com/microsoft/mssql-django/issues/15 does not leave a good 
impression on a first glance.

On Friday, March 26, 2021 at 1:48:50 PM UTC+1 wi...@wsvincent.com wrote:

> Yes, what Adam said! We'll certainly feature the project next week in 
> Django News and would happily also include a blog post etc on the project :)
>
> On Friday, March 26, 2021 at 7:56:11 AM UTC-4 Adam Johnson wrote:
>
>> Great work Warren and co! I see you already have a number of stars on 
>> GitHub.
>>
>> A promotional blog post could help get the word out as well. If you do 
>> write one, submit your blog's Django-specific feed to "Community blog 
>> posts" on https://www.djangoproject.com/community/ and it will get 
>> picked up by many readers. You can also send it to Jeff Triplett and Will 
>> Vincent who run Django News ( https://django-news.com/ ).
>>
>> On Fri, 19 Mar 2021 at 23:56, Warren Chu  wrote:
>>
>>> Hi All, 
>>>
>>> We've released the preview version 1.0b1 of the Microsoft SQL Server 3rd 
>>> Party Backend for Django. This version supports Django 2.2, 3.0 and 3.1.
>>>
>>> What's coming up next between now and GA release? We are re-adding 
>>> optional regex support, so that the regex DLL file is downloadable from a 
>>> trusted Microsoft downloads page. This file can then be installed in SQL 
>>> Server via our project script. This follows our organizational security 
>>> best practices.
>>>
>>> If you have any questions, feel free to post here in the Django 
>>> Developer forum, or via Github issues at: 
>>> https://github.com/microsoft/mssql-django
>>>
>>> Thanks,
>>> Warren
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/93188f91-c638-4e4d-89ee-e4b63585ab38n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>
>>
>> -- 
>> Adam
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4cec142a-49a2-4046-8e5b-0754d7c382d0n%40googlegroups.com.

Re: request to reopen ticket #24522: --random test option

[Florian Apolloner]

Thank you for diving into how unittest discovery tests!

On Friday, March 12, 2021 at 12:14:05 PM UTC+1 chris.j...@gmail.com wrote:

> But, regardless of the default, to get the full benefit I think you'd want 
> at least one of the test runs in CI to be a random one.
>

This is were I somewhat disagree. We have to run it as often with as many 
different seeds as possible to have an effect. The space of tests is so big 
that I am not sure we'd find something otherwise (though I have to agree 
that is theoretical thinking I did not try in practice).

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4034f3db-e6ee-4d78-b140-553c73c1a170n%40googlegroups.com.

Re: request to reopen ticket #24522: --random test option

[Florian Apolloner]

Btw while I am fine with making it default (I really do not care much 
either way), we certainly need a way to disable it and keep the current 
algorithm (even though it might not be guaranteed stable). When I develop I 
often run one suite and fix the errors as they occur. Having a stable order 
there ensures that the tests I fixed before are still fixed when the next 
test fails (I quite often also run with failfast there). I think having a 
stable test output can be useful for development. Having randomization in 
CI might be interesting, but then we probably need a way to derive the salt 
from the PR number or so, because I think the number of tests inside a PR 
is small enough that you probably do not see isolation failures, but you 
really want to see progress in a PR without random failures due to 
isolation.
On Friday, March 12, 2021 at 10:47:56 AM UTC+1 Florian Apolloner wrote:

> Is the current order defined in any way -- if not one should consider it 
> randoms anyways and as such actually randomizing should not make much of a 
> difference. FWIW I am fine with making it the default and outputting the 
> seed at the end of the run so it can reproduced. I am curious if it finds 
> many remaining issues.
>
> On Thursday, March 11, 2021 at 12:15:44 AM UTC+1 chris.j...@gmail.com 
> wrote:
>
>> For those that would rather it be opt-out, remember that we can always 
>> make it opt-out later, once we have more experience with the feature.
>>
>> Regarding the suggestion to have `--random 0` mean not to use 
>> randomizing, I also think that could be confusing. 0 is also a valid 
>> integer seed, and it seems nice to have the option of passing 0 as a simple 
>> seed if one wants. We could use --no-random for disabling if we ever wanted 
>> to make the feature opt-out in the future.
>>
>> --Chris
>>
>>
>>
>> On Wednesday, March 10, 2021 at 4:42:08 AM UTC-8 Mariusz Felisiak wrote:
>>
>>> I usually agree with new features being opt-in, but perhaps this case is 
>>>> different?
>>>>
>>>> If I had tests that are breaking if executed randomly, I’d want to know 
>>>> about it yesterday. IOW, I’m having difficulty imagining a scenario where 
>>>> the user would be thankful for not activating this feature by default. So 
>>>> personally, I’d like to see an opt-out setting for this in settings.py.
>>>>
>>>> /$0.02 
>>>> Fran
>>>>
>>>
>>> `--reverse`  will catch 95% of test isolation issues for you. It's 
>>> highly more likely that running tests in reverse order will catch isolation 
>>> issue for you than running them in a non-deterministic order. 
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4ec1e24a-917b-4256-9949-6f5deded1a80n%40googlegroups.com.

Re: request to reopen ticket #24522: --random test option

[Florian Apolloner]

Is the current order defined in any way -- if not one should consider it 
randoms anyways and as such actually randomizing should not make much of a 
difference. FWIW I am fine with making it the default and outputting the 
seed at the end of the run so it can reproduced. I am curious if it finds 
many remaining issues.

On Thursday, March 11, 2021 at 12:15:44 AM UTC+1 chris.j...@gmail.com wrote:

> For those that would rather it be opt-out, remember that we can always 
> make it opt-out later, once we have more experience with the feature.
>
> Regarding the suggestion to have `--random 0` mean not to use randomizing, 
> I also think that could be confusing. 0 is also a valid integer seed, and 
> it seems nice to have the option of passing 0 as a simple seed if one 
> wants. We could use --no-random for disabling if we ever wanted to make the 
> feature opt-out in the future.
>
> --Chris
>
>
>
> On Wednesday, March 10, 2021 at 4:42:08 AM UTC-8 Mariusz Felisiak wrote:
>
>> I usually agree with new features being opt-in, but perhaps this case is 
>>> different?
>>>
>>> If I had tests that are breaking if executed randomly, I’d want to know 
>>> about it yesterday. IOW, I’m having difficulty imagining a scenario where 
>>> the user would be thankful for not activating this feature by default. So 
>>> personally, I’d like to see an opt-out setting for this in settings.py.
>>>
>>> /$0.02 
>>> Fran
>>>
>>
>> `--reverse`  will catch 95% of test isolation issues for you. It's highly 
>> more likely that running tests in reverse order will catch isolation issue 
>> for you than running them in a non-deterministic order. 
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/281504ae-18af-4d74-aca4-9d89de9b1eb9n%40googlegroups.com.

Re: 'npm init django' works, maybe let's stop that?

[Florian Apolloner]

Mhm, I expected such an answer when I hit send :) I maybe should have 
worded it better. While you are right, that ultimately this is an issue for 
the DSF board I think that most of those issues can be solved by asking 
authors/etc nicely without having to invoke the thread of trademarks or 
lawyers. When that fails then we can surely put it forward to the DSF.

On Monday, March 8, 2021 at 9:04:49 AM UTC+1 James Bennett wrote:

> It's not really a discussion of whether "we" (this list) want 
> something removed; how to enforce Django's trademark is purely the 
> domain of the DSF. The DSF also has the ability to wave its ownership 
> of the trademark around and hire lawyers in cases where people are 
> reluctant to take something down, which has happened in the past. So 
> again I'd say that if people are putting packages up that seem to 
> conflict with the Django trademark, just notify the DSF board and let 
> them handle it. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dbbb7440-8eea-4d4a-9f3f-85aba577edb1n%40googlegroups.com.

Re: 'npm init django' works, maybe let's stop that?

[Florian Apolloner]

I am in favor of removing such a package. FWIW I have been contacting the 
authors of https://pypi.org/project/Django-2.2.19/ and 
https://pypi.org/project/Django-504/ as well a few weeks ago (their 
packages list before or next to Django when searching for Django on PyPI 
and might create some confusion). Given that I received no answer in weeks 
I'll guess I'll ask the PyPI team to remove them. For me it is not as much 
a trademark issue in this case but mostly a security issue.

Cheers,
Florian

On Monday, March 8, 2021 at 1:56:17 AM UTC+1 Adam Johnson wrote:

> I thought of this, but I also wanted to have the discussion out in the 
> open before the decision that it’s a use of the trademark we want to stop. 
> I guess we don’t have any other place for such a discussion?
>
> On Mon, 8 Mar 2021 at 00:05, James Bennett  wrote:
>
>> Any time this happens, just notify the DSF Board; they're the ones
>> with the legal standing to enforce the trademark.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CAL13Cg_%2B8qTkAFF82_6PrYt3BcTh_j6HtgHRQzKnn-D1XMD%2BMQ%40mail.gmail.com
>> .
>>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f560529b-9853-4b36-ad76-f95af17fe3ccn%40googlegroups.com.

Re: Revisiting Python support for after Django 3.2 LTS

[Florian Apolloner]

> Except for our LTS versions, which would never drop support. 

Mhm, I'd support such an approach only if we have a clear idea for security 
issues. Assume LTS supports Python 3.x to 3.z. Python 3.x goes EOL and has 
a security issue that affects Django LTS -- what do we do? On one hand 
there are distributions that actively support and backport to EOL versions, 
others are rolling release version and always on the latest which we simply 
cannot support in LTS. As much as I hate it, containers are more and more 
becoming the answer (FWIW I think podman is doing a great job in that 
regard with systemd integration etc).

No matter which way we will choose, there will be dragons.

Cheers,
Florian
On Wednesday, January 27, 2021 at 8:50:49 PM UTC+1 
in...@markusholtermann.eu wrote:

> I think I need to go through all proposed options a few more times to 
> understand their differences and implications.
>
> But what about a more pragmatic approach: Django supports the currently 
> supported Python versions at any given time. Except for our LTS versions, 
> which would never drop support. That means, a Django 2.2 might need to get 
> fix in 2.2.x to support e.g. Python 3.11. But if Python support for 3.8 is 
> dropped, why not drop it from Django' non-LTS versions. Where "drop" is 
> meant as a , we won't add fixes to support an old Python version, and we 
> won't add new features from a new Python version that's around. But if a 
> new Python version requires a fix, let's back port it.
>
> Cheers,
>
> Markus
>
> On Wed, Jan 27, 2021, at 4:22 PM, Carlton Gibson wrote:
> > OK... urgh. I don't think there's a perfect answer here. 
> > 
> > As you say Tim, assuming the "support unless EOL before the Django 
> > version's EOL" we end up dropping the Python version before the LTS, 
> > which is going to be just as "premature" as we have now, but for the 
> > x.0. 
> > 
> > If I've not counted wrong (again) the 4.2 LTS, for example, due April 
> > 2023, would drop Python 3.8 and 3.9, which have until Oct 2024 and 2025 
> > to run. 
> > 
> > So we'd be trading extra life here for probably more complaints there. 
> > 
> > I've played around with various variations of the cut-off date without 
> > any real ground being made. 
> > 
> > Thus, we probably need to stick as we are, in the main. 
> > 
> > A last option would be to say that the x.0 will support one extra older 
> > version after the LTS, in cases like this where the otherwise dropped 
> > version is supported for the whole lifetime of the x.0 release. 
> > 
> > That I think would look like this, assuming we backport support for new 
> > versions within the mainstream support period: 
> > 
> > 4.0 : 3.7, 3.8, 3.9, 3.10
> > 4.1 : 3.8, 3.9, 3.10, 3.11
> > 4.2 LTS : 3.8, 3.9, 3.10, 3.11, 3.12
> > 
> > 4.2 is in danger of taking 3.13 as well, but that's released after 4.2 
> > leave mainstream support.
> > It's akin for what we did for Django 2.2 and Python 3.9
> > That's independent of whether we keep support for 3.7 a bit longer. 
> > 
> > Having looked it through now, I think, fully, that's my final thought. 
> > I'd need to think about the __exact wording, but I hope the idea is 
> clear.
> > 
> > I take Claude's point about contributing but, for me, the desire to 
> > support slightly more versions if we can is for beginners, picking up a 
> > couple of year old laptop, with SOME version of Python on it, and being 
> > able to get going, with hopefully the latest version. I appreciate they 
> > can install an older version, but it's hard enough to get started 
> > without hitting subtle version changes. I'd like to support that 
> > use-case as best we can. 
> > 
> > The current policy begins, "Typically...". I'd suggest supporting 
> > Python 3.7 for Django 4.0 is merely leveraging that. 
> > 
> > We should decide. I'd be +1 to maintain the current policy but keep 3.7 
> > support for Django 4.0, dropping it at Django 4.1. 
> > 
> > Thanks all, and especially you Tim for taking the time to explain and 
> > explore the options. 
> > C.
> > 
> > 
> > 
> > On Tuesday, 26 January 2021 at 22:00:31 UTC+1 timog...@gmail.com wrote:
> > > Looking at this again, I'm not sure it would be six versions. 
> Carlton's suggested policy has the effect of dropping a lot of Python 
> versions right before the LTS since it's supported for 3 years. For 
> example, in Django 5.2 LTS, I think it's incorrect that Python 3.10 and 
> 3.11 would be supported since their support expires in Oct 2026 & 7, before 
> Django 3.2's expiration in April 2028)? The current policy means we have 
> Django LTS's supporting some Python's well after they expire. I never liked 
> that aspect of it. Anyway, the decision won't affect my life.
> > > On Tuesday, January 26, 2021 at 5:32:50 AM UTC-5 Mariusz Felisiak 
> wrote:
> > >> Hi y'all,
> > >> 
> > >> I think we should keep the current policy. There is never a good time 
> to drop support for anything and extending support especially for Python 
> 

Re: Update returning

[Florian Apolloner]

Hi Simon,

On Wednesday, January 27, 2021 at 5:54:42 AM UTC+1 charettes wrote:

> I think that's the best option here if we want to elegantly add support 
> for this feature while maintaining backward compability. Something along 
> the lines of ...
>

That is certainly an interesting approach. It kinda breaks the "there 
should be one way of doing things" rule, but…

The usage of `returning` bring another set of questions though. Since 
> UPDATE are not ordered RETURNING data has little value without the primary 
> key associated with the updated rows. Maybe the return value of 
> `returning=[f1, ..., fn]` should be a dict mapping the primary key to list 
> of returning values.
>

I am not sure I like that. For things where you update just one row and 
want to know the new values the primary key doesn't make much sense. 
Granted for multiple rows it would maybe easier to have it automatically 
keyed by the pk, but returning something always (the pk) without having an 
option to disable  it seems kinda wrong to me. Not sure what the best 
option would be.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4eea605a-57d7-4ce3-b233-3eb88c91e110n%40googlegroups.com.

Re: Snowflake db backend

[Florian Apolloner]

Hi Scott,

Thank you for your response, this is very helpful.

On Tuesday, January 26, 2021 at 11:38:18 PM UTC+1 foug...@apps.disney.com 
wrote:

> Snowflake does not support lastrowid.  So, we grab the last ID inserted 
> with a 'SELECT MAX(pk_name) FROM table_name'.  This is obviously prone to 
> failure.  Assigning an ID during the INSERT would provide similar results 
> on all backends.
>

U, the 'SELECT MAX()' is not going to fly well, you are right. 
Assigning an ID during INSERT has it's own problems. In postgresql it would 
be possible because you could just select the next value from the created 
sequence (more or less), but with IDENTITY columns this might get harder. I 
do not think there is a sensible way to do this in MySQL at all. While 
lastrowid support is a nice to have, Django should work (mostly?) without 
it: 
https://github.com/django/django/blob/464a4c0c59277056b5d3c1132ac1b4c6085aee08/django/db/models/sql/compiler.py#L1372-L1387
 
-- the requirement here is that your database is at least able to return 
values after insert. From the looks of it, it does not I think? Or 
differently put: Which ways does snowflake offer to get an ID? Solely by 
providing it directly into insert?

The feature flag `supports_transactions` really means 
> `supports_nested_transactions`.  Snowflake supports a single level of 
> transaction, BEGIN + (ROLLBACK|COMMIT).  Multiple BEGINS contribute to the 
> current (only) transaction.  Since I have to set this flag to False, no 
> transactions are used, even ones that are supported and testing grinds to a 
> crawl with all of the table truncations and repopulation.  Since Django 
> *normally* operates in auto-commit mode, this isn't a huge deal. 
>  Snowflake also doesn't support save points, but the feature flag seems to 
> do what is expected when disabled.
>

Hu, which database support nested BEGIN? As far as I am aware Django does 
never nest BEGINs -- do you have an example for me? I'd very much like to 
fix this. From a quick glance at the code we only start a transaction if we 
are not already in one: 
https://github.com/django/django/blob/master/django/db/transaction.py#L196-L208

Snowflake does not support column references without a FROM or JOIN clause. 
>  I've only encountered this used in the unit tests.
>

Ok, see below.

I've been able to work around most of the function differences by adding 
> as_snowflake methods to the appropriate classes.  There are a few cases 
> like JSON_OBJECT that don't translate well, which work, but not entirely as 
> expected.
>

Perfect, this sounds as if our extension system works as intended in this 
area.

A search for 'connection.vendor == ' in the core code shows one example of 
> where core backends are given privileged access to core Django inner 
> workings that 3rd party backends don't.
>

We have been working to get rid of those: 
https://github.com/django/django/commit/275dd4ebbabbbe758c7219a3d666953d5a7b072f#diff-69f332030b6f25f8f4d95842548d847139ee2cc163aef18f1c8d83b90f3f9e5f
 
-- This is solely in 3.2, but Tim can suggest a workaround for earlier 
versions (he was doing something similar in his cockroachdb tests before 
3.2).

Please take these comments as observations, not complaints.  I understand 
> project evolution and the resistance to change something that doesn't seem 
> broken.  Maybe keep some of these thoughts in mind when a change is made to 
> the core backends or the compiler and consider how other backends will need 
> to implement that new feature.
>

No offense taken at all. If somehow possible I'd like to encourage you to 
work with us on your pain points. I think at least some of those are 
addressed or at least addressable. I am happy to offer code and fixes, but 
I'll need a few more details especially wrt transaction handling.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/de030559-7053-42af-8020-68c4213c0a1an%40googlegroups.com.

Re: Update returning

[Florian Apolloner]

On Tuesday, January 26, 2021 at 5:26:02 PM UTC+1 Adam Johnson wrote:

> Not that I am completely convinced that the following is a good idea; but 
>> what about: 
>
> QuerySet.objects.update(name="Rob").values("id", "name")
>>
>
> That's not possible since update() directly performs the update - it's not 
> lazy in any way. It could be done in the other order like 
> `QuerySet.objects.values("id", "name").update(name="Rob")` but I don't see 
> the necessity to define "returning" fields in a chainable manner.
>

Ha, not sure what I was thinking. The sentence below I noted that update() 
would return something but I didn't think that this would break chaining. 
My bad.

I looked further around and `get_or_create` has the nice workaround of 
being able to use `defaults__exact` if it clashes with the `defaults` 
keyword. Sadly we do not have that option here. Truth to be told I do not 
think that many people have fields called returning

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/37efe4e9-4ca7-4ad7-b370-94dd2d27fc86n%40googlegroups.com.

Re: Update returning

[Florian Apolloner]

Not that I am completely convinced that the following is a good idea; but 
what about:

QuerySet.objects.update(name="Rob").values("id", "name")

On second thought I'd like an .returning() more than values, but I am not 
sure if it makes sense to add a new function just for the sake of a small 
backwards compatibility.

> I'd expect it to return a list of dicts.

Currently .update returns the number of rows affected I think. This 
information should still be there even if returning extra data.
On Tuesday, January 26, 2021 at 3:00:01 PM UTC+1 Adam Johnson wrote:

> I think we could do the most logical:
>
> QuerySet.objects.update(name="Rob", returning=["id', "name"])
>
> There is a precedent for small backwards incompatible changes like this, 
> for example when "named" was added to "values_list()". However maybe 
> backwards compatibility is worth keeping. We can be backwards compatible by 
> special-casing models with a field called 'returning', in which case the 
> field is updated. This blocks the 'returning' functionality but users can 
> always rename a field.
>
> I'd rather not add a new method or otherwise change the signature of 
> update() more radically.
>
> On Tue, 26 Jan 2021 at 13:27, Tom Carrick  wrote:
>
>> Hi,
>>
>> I found myself with a use-case today of wanting to return some data from 
>> an update, as I want to make sure I return exactly what is in the database 
>> without making an extra query.
>>
>> I've found https://code.djangoproject.com/ticket/28682 and agree with 
>> the resolution there.
>>
>> I suppose there is a way to do this in a backwards compatible way, 
>> something like:
>>
>> Foo.objects.update(["id", "name"], name="Rob")
>>
>> But it's very ugly. But how about a new method:
>>
>> Foo.objects.update_returning(["id", "name"], name="Rob")
>>
>> Doesn't seem quite so bad. There's also a possibility of something like:
>>
>> Foo.objects.update_returning(updates={"name": "Rob"}, returning=["id", 
>> "name"])
>>
>> I'd expect it to return a list of dicts. I'm not sure what's best, if 
>> anything. It could be it's a bit too niche, but it is sometimes quite 
>> useful.
>>
>> Tom
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CAHoz%3DMaDz7VLXqRk01D_bUOLN%3D5TBiqVrw-BzyLogeaTMtXM4g%40mail.gmail.com
>>  
>> 
>> .
>>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/abde0155-3eb9-4330-994a-c5f9133b236an%40googlegroups.com.

Re: Snowflake db backend

[Florian Apolloner]

On Saturday, January 23, 2021 at 12:41:35 AM UTC+1 t...@tomforb.es wrote:

> > Honestly, I don't think we'd release this as a third party package.  We 
> have had to code around a lot of core db specific code to get Snow SQL to 
> work
>
> This is good feedback that is valuable for us to hear. Even if you don’t 
> release it as a fully maintained package just understanding the pain points 
> you encountered while developing this is great feedback that can help us 
> improve.
>
> Would you be willing to share specific code snippets here, or just general 
> areas that you found challenging while developing this backend?
>

+1, if it were possible to show the full code (even if you are not going to 
maintain it in the longrun) would be very much helpful to improve.

Like Tim I wrote one or two database backends in the past (informix and the 
likes) and it was not to bad. That said, mine where relatively easy to 
implement due to their closeness to existing database and (somehwat) 
conformance with SQL standards. I'd love to see the code for snowflake and 
maybe we can identify a few areas that would need improvement.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fdd01c9d-041f-4206-b529-701fde694202n%40googlegroups.com.

Re: Technical Board Decision Needed: Admin append_slash behaviour.

[Florian Apolloner]

On Thursday, January 7, 2021 at 2:16:57 PM UTC+1 carlton...@gmail.com wrote:

> 1. Add the catch-all view to admin to stop the unauthenticated probing, as 
> per the Security Teams initial idea, but not the AdminSite.append_slash 
> option.
> 2. Don't even add the catch-all, and close the ticket as wontfix. 
>

I think the catch-all view is certainly a worthwhile addition, it is a low 
hanging fruit that makes fast probing if auth.user is installed impossible.

* It SEEMS to me that the catch-all view does serve it's purpose as as the 
> AdminSite.admin_view decorator redirects all non-staff requests equally to 
> login (whether they exist or not, because the catch-all view exists.) This 
> is prior to any per-view timing variation. (I think )


Technically you could already mount a timing attack because url resolving 
is not constant time, the first matching view wins :þ

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/03910826-32d4-44c9-a3d5-a35f984c05e7n%40googlegroups.com.

Re: Status of 3.2 release blockers.

[Florian Apolloner]

Hi Carlton,

I'd like to put 
https://github.com/django/django/pull/12553
https://github.com/django/django/pull/13800

onto this list as well. They are imo basically finished and I'd love 
reviews from our fellows. Let me know if I can help somewhere else in 
exchange ;)

Cheers,
Florian

On Monday, January 4, 2021 at 10:56:42 AM UTC+1 carlton...@gmail.com wrote:

> Hi all,
>
> Happy New Year! 
>
> Time to begin release process for the next major release, Django 3.2!
>
> Reference: https://code.djangoproject.com/wiki/Version3.2Roadmap
>
> The 3.2 feature freeze is scheduled for January 14. 
> We'll probably release the alpha that day.
>
> We have a few larger patches we want to finish reviewing:
>
> https://github.com/django/django/pull/11929 - Fixed #26167 -- Added 
> support for functional indexes.
> https://github.com/django/django/pull/13618 - Fixed 30360 -- Support 
> rotation of secret keys.
> https://github.com/django/django/pull/13435 - Fixed #32018 -- Extracted 
> admin colors into CSS variables.
> https://github.com/django/django/pull/11026 - Fixed #29010, Fixed #29138 
> -- Added limit_choices_to and to_field support to autocomplete fields.
>
> We'll update this thread for any schedule changes. 
>
> Kind regards, 
> Carlton
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/22232a00-2beb-40ef-97ea-5cd944316b39n%40googlegroups.com.

Re: Changing the discovery method for management commands

[Florian Apolloner]

On Saturday, December 19, 2020 at 3:16:51 PM UTC+1 diptesh@gmail.com 
wrote:

> and having to add a bunch of unnecessary files to the source code


A bunch? You can literally get away with one file:

➜  testing git:(dc_message_signing) ✗ tree testabc 
testabc
└── management
└── commands
└── my_cmd.py

This is a complete (!) django app. While I agree that in theory it would be 
nice to support some other mechanisms, I do not think it is worth the 
effort given how easy it is to add a command (one file & two directories)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6dc1d664-aee3-4bdf-9939-bf52f842c881n%40googlegroups.com.

Re: Changing the discovery method for management commands

[Florian Apolloner]

Nothing is stopping pytest-django to also ship a django app for those 
management commands though

On Saturday, December 19, 2020 at 2:27:57 PM UTC+1 diptesh@gmail.com 
wrote:

> As you said, most cases. The remaining cases have absolutely no way of 
> defining management commands. One example is *pytest-django *which 
> doesn't provide a django app but could benefit with a mangement command 
> *python 
> manage.py setup_pytest*.
>
> On Saturday, December 19, 2020 at 2:11:32 PM UTC+5:30 Adam Johnson wrote:
>
>> Why? I don't see an impetus to avoid creating a Django app. In most use 
>> cases there are related models or other Django bits to go with a management 
>> command.
>>
>> On Sat, 19 Dec 2020 at 01:48, Diptesh Choudhuri  
>> wrote:
>>
>>> As of now, if you need to create a management command, it is necessary 
>>> to create a file *app_name/management/commands/my_command.py, *and then 
>>> add *app_name *to *INSTALLED_APPS *in *settings.py. *This prevents 
>>> non-django packages from defining their own management commands, because it 
>>> explicitly requires them to create a django app which just adds a bunch of 
>>> unnecessary files to their source code.
>>>
>>> I propose we overhaul the existing management command discovery system 
>>> so that it is easier to write management commands. Also I suggest we keep 
>>> the default discoverer in place so as to maintain backwards compatibility.
>>>
>>> All of this will require documentation and I am ready to make a PR for 
>>> that too. Please tell me if the idea is feasible, and I will get to work on 
>>> it ASAP.
>>>
>>> Best
>>> Diptesh Choudhuri
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/a7f9bf60-da49-404b-ac70-192220149059n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>
>>
>> -- 
>> Adam
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0df083cf-140d-4122-be68-42c9b4d56cfbn%40googlegroups.com.

Dogfeeding our signing API

[Florian Apolloner]

Hi there,

Craig Smith is working on making cookie-based messages RFC compliant 
(https://code.djangoproject.com/ticket/32191). Our prefered approach would 
be to encode messages via `signing.dumps` and this is (excuse the wording) 
where the shitshow begins. As it currently stands we cannot use 
`signing.dumps` because it creates it's own `Signer` instance [1] and the 
messages framework uses `get_cookie_signer` [2] to get yet another signer 
instance. This means we would have to replicate that functionality or 
extract it somehow, which leads to duplication and more complexity. Craig 
tried to do the latter in [3] which leads to a kind of encoding/decoding 
mess because (understandably) the serializers used by `dumps/loads` operate 
on bytes, while many other things in Django use strings.

My way out of this mess would be to move `signing.dumps` & `signing.loads` 
onto the `Signer` class itself; this way this functionality would become 
reusable while allowing to switch how signing is performed. My preference 
would be to call those methods `sign_object` & `unsign_object` which would 
make it clear that it will have to run some kind of serializer over it. 
`signing.dumps/loads` could just call those methods then.

Another option would be to extract the serialization & compression part out 
of `signing.dumps/loads` into another set of functions, but that would 
(while giving more flexibility) give even more ways on how to do things. I 
rather have a public & documented way on how to use `Signer` instances to 
sign an object structure (with optional compression) and say "this is the 
way [of doing things in Django]" ;) The "raw" string signing methods 
`sign/unsign` can still be used if people need to do "more".

What do you all think? Does this make any sense?

Thanks,
Florian

1: 
https://github.com/django/django/blob/8f384505eee8ce95667d77cfc2a07d4abe63557c/django/core/signing.py#L124
2: 
https://github.com/django/django/blob/8f384505eee8ce95667d77cfc2a07d4abe63557c/django/core/signing.py#L75
3: https://github.com/django/django/pull/13732

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fd4d4d57-666b-4a0f-bf7d-4bc35df39803n%40googlegroups.com.

Re: Improving MSSQL and Azure SQL support on Django

[Florian Apolloner]

Hi Warren,

> A) django-mssql is viewed as the Phase 1 focus and current preferred 
option with a longer term Phase 2 goal of including MSSQL as a supported 
backend for Django core

Please keep in mind that Phase 2 is something that might never happen. We 
have a tendency to not bloat Django and there is no reason why a database 
backend cannot live outside of core. Tim Graham is currently working on a 
cockroach backend and is running against the builtin Django testsuite (+ 
submitting patches to Django where we need to alter tests due to reliance 
on PKs etc…). This approach works imo very well (I did the same when 
writing a backend for Informix). We are very supportive of 3rd party db 
backends in the sense that we usually quickly address issues in our 
testsuite etc to make testing those external backends against Django 
easier. In that sense there is no strong reason to include the MSSQL 
backend in core. Package management in Python works well enough to allow 
the backend to stay external. 

Personally I think having the django-mssql backend with MS support behind 
it (be that test infra for the existing django-mssql fork etc) is already 
the icing on the cake.

Cheers,
Florian


On Friday, December 4, 2020 at 2:59:43 AM UTC+1 vwa...@gmail.com wrote:

> Thanks for the responses.
>
> @Florian - I've now reached out to the maintainers of ESSolutions as well 
> as the relevant previous Microsoft engagement owners, thanks for the 
> suggestion
>
> @Chris - This is excellent news indeed, we are excited to make progress 
> beginning with some small steps
>
> @r...@whidbey.com - 
> A) django-mssql is viewed as the Phase 1 focus and current preferred 
> option with a longer term Phase 2 goal of including MSSQL as a supported 
> backend for Django core
> B) I've now logged both your pain points as issues in our private repo 
> (which will be made public after internal compliance reviews)
> C) Happy to hear about your scaled production app using Django and MSSQL. 
> Could I ask you to send a "Hello" email to myself at v-wa...@microsoft.com, 
> so that we can follow up with eventual testing?
>
> @dans...@gmail.com - Your security concerns are important and noted, I've 
> flagged this for investigation as a priority
>
> Cheers,
> Warren
> On Friday, 27 November 2020 at 10:57:09 UTC-8 dans...@gmail.com wrote:
>
>> I'd suggest someone talk with professional DBAs for MSSQL.  In my work, 
>> which is Federal government, the DBA told me that disconnecting from PSQL 
>> as "appuser" and attempting to connect to database "postgres" in order to 
>> create the test database violated FISMA.  I had to subclass my own 
>> postgresql backend to create the test user while connected to my actual 
>> database.  I would imagine that the original reasons for doing it this way 
>> harkens back to a time when people used the same database server for 
>> production, staging, qa, and integration, and some of these environments 
>> may not have existed. While in some organizations we have only production, 
>> staging/qa, and development/integration, I would guess there are few cases 
>> where the same database server is used for production and the other 
>> environments.
>>
>> On Fri, Nov 27, 2020 at 1:18 PM r...@whidbey.com  
>> wrote:
>>
>>> Good news.  I've been using Django on MSSQL for about 8 years.  Couple 
>>> of things:
>>> - I've been using pyodbc, not django-mssql.  I note your messages from 
>>> 2015 include it as a library to check for compatibility; what was the 
>>> outcome of that?  Is it proposed that django-mssql become the 
>>> "best-practices" interface for SQL Server?
>>> - Couple of persistent pain points:
>>>   1. Testing.  The Django code that sets up test databases fails with 
>>> MSSQL, while it succeeds with PostGRE, MySQL and SQLite.  The issues seem 
>>> to revolve around setting constraints as the tables are generated, rather 
>>> than holding off and enabling the constraints at the end of the process.
>>>   2. Stored Procedures.  These need to be loaded as an additional step 
>>> in the creation of a database, and don't really have any representation in 
>>> Django per se so migrations, etc don't generally have an idea that they 
>>> exist.
>>>
>>> I'd be happy to test out what you come up with against our system.  It's 
>>> currently serving a custom REST interface with 2-million-plus rows of 
>>> transactions,with clients world-wide, along with a customer-facing web app, 
>>> a staff site and a suite of Tableau reports.
>>>
>>> On Thursday, November 26, 2020 at 5:40:18 PM UTC-8 vwa...@gmail.com 
>>> wrote:
>>>
 Hi All,

 Microsoft has now committed ongoing resources towards improving MSSQL 
 and Azure SQL support for Django. We're currently focused on internal 
 compliance and forking the ESSolutions django-mssql-backend 
 , adding testing 
 pipelines, refactoring the Django DB engine naming convention, and 
 

Re: Improving MSSQL and Azure SQL support on Django

[Florian Apolloner]

Hi Warren,

given that the current django-mssql-backend maintainers might not be 
reading here I'd suggest you open a ticket on the repository to get in 
contact with them. It also might be a good idea to re-evaluate this thread 
(and possibly at Microsoft internally) to see why the last approaches 
failed and what can be done better this time. Personally I'd also suggest 
to work "openly" with the existing fork from ESSolutions and submit PRs 
there for testing pipelines etc (I assume you'd want to run them on azure 
or something and having that integration developed in the open would 
certainly provide good guidance for other projects). I am not sure what 
internal compliance entails, but please keep in mind that this is an open 
source project used already by people and compliance to some internal 
microsoft standards might not be something people care about or need.

I am excited to see your PRs against the repo.

Cheers,
Florian

On Friday, November 27, 2020 at 2:40:18 AM UTC+1 vwa...@gmail.com wrote:

> Hi All,
>
> Microsoft has now committed ongoing resources towards improving MSSQL and 
> Azure SQL support for Django. We're currently focused on internal 
> compliance and forking the ESSolutions django-mssql-backend 
> , adding testing 
> pipelines, refactoring the Django DB engine naming convention, and 
> addressing current test suite errors.
>
> We'd love to hear from current mssql-backend maintainers as well as 
> mssql-backend users about the existing issues and feature requests that we 
> should be prioritizing.
>
> We looking forward to engaging the community and working towards MSSQL as 
> a first-class supported backend for Django.
>
> -Warren
>
> On Wednesday, 4 December 2019 at 07:05:25 UTC-8 Tim Allen wrote:
>
>> Hi Sean, just an update from what I know.
>>
>> We are still waiting for a reply from Microsoft. They're a large company, 
>> so understandably, it takes a little while.
>>
>>
>> For now, if people need to get onto Django 2.2 for long term support 
>> (which will last until April, 2022), you can use this package:
>>
>> https://github.com/ESSolutions/django-mssql-backend I've been running it 
>> in production for months without incident. Of course, YMMV.
>>
>>
>> If Microsoft and/or the DSF end up wanting to bring support under the 
>> Django umbrella, the django-mssql-backend repository is a possible 
>> starting point, IMHO.
>>
>> The django-mssql-backend is currently being developed and support for 
>> Django 3.0 is being worked on: ESSolutions/django-mssql-backend#18 
>> 
>>
>>
>> Regards,
>>
>>
>> Tim
>>
>> On Monday, December 2, 2019 at 11:03:56 AM UTC-5, Sean Martz wrote:
>>>
>>> Hello,
>>>
>>> It seems like this issue has lost momentum. Is this still something 
>>> that's on anyones radar? It looks like django-pyodbc-azure is not actively 
>>> maintained anymore (it looks like Michaya has taken a hiatus from GitHub). 
>>> It also looks like there's a small community potentially popping up that's 
>>> interested in first class MSSQL Server support for Django. (
>>> https://github.com/FlipperPA/django-mssql-backend). Is Microsoft still 
>>> interested in committing resources to this goal? In my situation, it would 
>>> be a lot easier to sell stakeholders and decision makers on Django if it 
>>> had first class support for MSSQL Server.
>>>
>>> For what it's worth, Django-pyodbc-azure is still working well.
>>>
>>> Cheers,
>>> Sean
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0eb67fc9-8edd-4774-9e60-7233a2cabb4dn%40googlegroups.com.

Re: Revisiting Python support for after Django 3.2 LTS

[Florian Apolloner]

I think Redhat realized that they have old versions and this is not going 
to fly well during the lifetime of their system. This is why they 
introduced modules in v8 and strongly pushing podman. That said, it is a 
major hurdle for many companies if upstream drops support for $x (be that 
in python, php or whatever). I think this is also one of the reasons why 
containers are so popular. But do we really want to push users that much 
towards containers? Are there any big Django users our there that provide 
RPM packages and could give us some insights?

Cheers,
Florian
On Friday, November 20, 2020 at 11:52:51 PM UTC+1 Tim Allen wrote:

> I often hear Ubuntu thrown around during these discussions, and it is my 
> distro of choice for personal projects. But like many of us, I work at a 
> RedHat / CentOS shop, and trying to maintain a current Python version is a 
> much more difficult proposition. Unfortunately, IUS Community has stopped 
> providing yum-installable versions of Python in an attempt to get EPEL to 
> be more up to date, but that hasn't happened. To get a version of Python 
> greater than 3.6 on RedHat / CentOS, AFAIK, you currently must build from 
> source with altinstall.
>
> I agree with Andrew's statement that we should consider each version. I 
> can see dropping Python 3.5 support - it would allow us to use a feature 
> like f-strings, which improves readability and speed throughout the 
> codebase, and is ubiquitous in Python. But what does dropping Python 3.6 
> support really achieve? Do we need data classes?
>
> I realize there is a need to move forward, especially for wonderful things 
> like better async support. I just ask that we also consider those of us 
> using Django in corporate or academic settings where the pace of upgrading 
> Python is a bit more glacial.
>
> On Thursday, November 19, 2020 at 11:51:29 AM UTC-5 Andrew Godwin wrote:
>
>> I agree we should not be quite so beholden to our existing Python version 
>> policy - that was mostly to get us out of the early 3.x era. Now things are 
>> more stable, I'd support a policy that is much more like "any stable 
>> version of Python currently out there and supported".
>>
>> Andrew
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1e1085e8-a175-4707-be04-cd3073ce7febn%40googlegroups.com.

Re: DB persistent connection

[Florian Apolloner]

Hi,

I am not sure what you are trying to tell us?! Opening a connection to 
Postgresql is somewhat time-consuming and 10ms in a cloud env does not seem 
to be surprising at all.

Cheers,
Florian

On Monday, October 26, 2020 at 2:28:28 PM UTC+1 pra...@fampay.in wrote:

> https://groups.google.com/g/django-developers/c/rH0QQP7tI6w/m/yBusiFTNBR4J
>
> We are running django with gunicorn + gevent. We have set Max con age to 
> 10 min. We use newrelic for monitoring. 
>
> Even though we have set it to use persistent connection it;s showing in 
> new relic that
> On average it takes 10ms for db connection
> [image: Screenshot 2020-10-26 at 3.54.08 PM.png]
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5130fcd5-5f2e-485e-b784-c8c6c9ef05b6n%40googlegroups.com.

Re: Oddity found in password creation

[Florian Apolloner]

So, I have been digging a little bit more and it seems there was a 
conscious decision to not include an entropy check or character classes: 
https://groups.google.com/g/django-developers/c/9GBhgGXmEKs/m/toKKgGhaqewJ 
-- But I have to admit that this is all I could find and I was rather 
surprised when you brought up the topic. I certainly would also have 
expected stronger validation by default (then again I tend to customize 
those settings very early on usually) -- maybe some kind of entropy check 
(but it's hard to write a good error message on those).

I just double checked and / are indeed in the common 
password list. Funny that they seem to be a) common enough to be included 
there and b) commoner than let's say .

Cheers,
Florian
On Tuesday, October 6, 2020 at 5:59:30 PM UTC+2 hcharpent...@gmail.com 
wrote:

>
> Oh I see...
> Thank you!
> On Tuesday, October 6, 2020 at 4:30:34 PM UTC+2 f.apo...@gmail.com wrote:
>
>> Seem to be expected behaviour (albeit somewhat suboptimal :D). The 
>> default validators are: 
>> https://github.com/django/django/blob/999cddd58d30469f3ee85278985313fdf528323d/django/conf/project_template/project_name/settings.py-tpl#L87-L100
>>  
>> and from the looks of it / is in the list of common 
>> passwords, whereas / is not. 
>>
>> On Tuesday, October 6, 2020 at 3:32:51 PM UTC+2 hcharpent...@gmail.com 
>> wrote:
>>
>>>
>>> Hi,
>>> When you create or modify a user account, the password has to be min. 8 
>>> characters long, not only numbers, not too simple.
>>> But is it normal that you can use 8 times the same character?
>>> E.g.: "", "" are not allowed as a password ; "", 
>>> "" are allowed...
>>> Thank you and have a nice day!
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0f381205-ba70-4b89-9f34-5a2c459ac5f5n%40googlegroups.com.

Re: Oddity found in password creation

[Florian Apolloner]

Seem to be expected behaviour (albeit somewhat suboptimal :D). The default 
validators are: 
https://github.com/django/django/blob/999cddd58d30469f3ee85278985313fdf528323d/django/conf/project_template/project_name/settings.py-tpl#L87-L100
 
and from the looks of it / is in the list of common 
passwords, whereas / is not. 

On Tuesday, October 6, 2020 at 3:32:51 PM UTC+2 hcharpent...@gmail.com 
wrote:

>
> Hi,
> When you create or modify a user account, the password has to be min. 8 
> characters long, not only numbers, not too simple.
> But is it normal that you can use 8 times the same character?
> E.g.: "", "" are not allowed as a password ; "", 
> "" are allowed...
> Thank you and have a nice day!
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4835f8ee-27ed-468a-9404-5904cfc6c533n%40googlegroups.com.

Re: ConditionalGetMiddleware MD5

[Florian Apolloner]

Hi Adam,

On Monday, September 14, 2020 at 2:00:06 PM UTC+2 Adam Johnson wrote:

> one thing to consider might be that md5 is usually disabled for FIPS 
>> enabled system
>>
>
> This is true, but if no one has complained, why "fix" it? As covered above 
> one can always implement custom ETag header generation.
>

Well there are open tickets, so in that sense someone has "complained". I 
also know that pulp tries/tried to work around md5 usage in Django etc…  
But with regard to fips the proper way is probably the new `useforsecurity` 
argument in python3.9. 

The arguments about performance are all good and I think that if we change 
away from md5 it should probably something which is on par performance-wise.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/961696ec-0d0f-4030-acf7-deb80854f87fn%40googlegroups.com.

Re: ConditionalGetMiddleware MD5

[Florian Apolloner]

Hi,

one thing to consider might be that md5 is usually disabled for FIPS 
enabled system (ie https://code.djangoproject.com/ticket/28401 ). So if we 
are changing something here we might also consider this.

Cheers,
Florian

On Friday, September 11, 2020 at 9:41:51 AM UTC+2 gertb...@gmail.com wrote:

> I'm not so sure this is a problem (wrt to using md5 hash of response 
> content for ETags and likely also for cache keys). The probability of a 
> naturally occurring collision with MD5 is 1.47*10-29 [1] so the risk of 
> this scenario occurring by accident is extremely remote.
>
> If we assume that User 2 is a malicious actor and can reliably generate 
> collisions (which is not practically possible within typical cache 
> durations), then User 1 is likely still no worse off. Maybe some security 
> researchers can comment on the threat model in this case.
>
> Lastly, the practical use of content hash based ETags are limited due to 
> the processing overhead required for them (as implemented in 
> ConditionalGetMiddleware). It might be useful for a low traffic/value 
> deployments and then the risks are likely not a problem. Switching to a 
> more cryptographically secure hashing algorithm will increase the overhead 
> and provide little additional benefit to such installations. As mentioned 
> earlier, production deployments will often use Apache or Nginx, and neither 
> generate ETags using file content hashes.
>
> [1] https://www.avira.com/en/blog/md5-the-broken-algorithm
>
> On Fri, 11 Sep 2020 at 08:25, Francisco Couzo  wrote:
>
>> If changing ConditionalGetMiddleware to use SHA-256
>> It also might be a good to change it on FileBasedCache, 
>> _generate_cache_key, and generate_cache_header_key too
>> Also, _generate_cache_key is just blindly concatenating hashes, so 
>> ['foo', 'bar'] is equal to ['fo', 'obar'], I don't know it might be a 
>> problem, but it just doesn't looks right
>>
>>
>> On Thu, Sep 10, 2020 at 10:14 PM Taymon A. Beal  
>> wrote:
>>
>>> That attack doesn't work with the recommended production setup because
>>> Django doesn't serve uploaded files in that setup.
>>>
>>> That being said, some users might be doing that anyway since setting
>>> up production-worthy upload hosting is such a pain, and even if they
>>> don't, they might have other views that somehow allow users to control
>>> the response body. So I think this should be treated as a
>>> not-super-severe-but-still-worth-fixing security issue.
>>>
>>> What backwards-compatibility considerations exist? Do we consider it
>>> normal for upgrading to a different Django version to bust users'
>>> caches? I can't immediately think of any bad consequences of changing
>>> the hash function, apart from that one. Busting users' caches doesn't
>>> sound that terrible, given that, even if the hash function were
>>> changed on every release (which of course it wouldn't be; SHA-2 has
>>> been the most generally-recommended hash function for 15 years and
>>> there are no signs that this will change), it would still only happen
>>> once every eight months, and it's fairly rare for anything to be
>>> cached that long in the first place, I think.
>>>
>>> Taymon
>>>
>>>
>>> On Thu, Sep 10, 2020 at 1:16 PM Francisco Couzo  
>>> wrote:
>>> >
>>> > User 1 uploads a file
>>> > User 2 downloads it, and caches it
>>> > User 1 uploads a new file to the same URL, with the same MD5 hash
>>> > User 2 will keep using the old file indefinitely
>>> >
>>> > Sure, user 1 has to upload two files with the same hash on purpose
>>> >
>>> > On Thu, Sep 10, 2020 at 11:07 AM Adam Johnson  wrote:
>>> >>
>>> >> What would this protect against?
>>> >>
>>> >> On Thu, 10 Sep 2020 at 03:56, Francisco Couzo  
>>> wrote:
>>> >>>
>>> >>> I think it would be a good idea to make ConditionalGetMiddleware use 
>>> a hash function that's not as easy to find a collision as MD5, most 
>>> probably SHA-256 or BLAKE2.
>>> >>> I don't see a problem with just changing it, it will just invalidate 
>>> the old cache.
>>> >>> If there's an agreement on changing the hash function, I can make 
>>> the PR.
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>>
>>> >>>
>>> >>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> >>>
>>> >>>
>>> >>> To unsubscribe from this group and stop receiving emails from it, 
>>> send an email to django-develop...@googlegroups.com.
>>> >>>
>>> >>>
>>> >>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/ff591d46-97fc-43d6-9d1c-d0ba24d7b1a8n%40googlegroups.com
>>> .
>>> >>>
>>> >>>
>>> >> --
>>> >> Adam
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, 
>>> send an email to 

Re: What the purpose of having function that is not working correctly?

[Florian Apolloner]

On Thursday, September 10, 2020 at 11:16:56 AM UTC+2 Alexander Lyabah wrote:

> The problem with the function is that it is actually working, but not 
> always, and because of that, other people are suggesting it on 
> StackOverflow, using it in prod, and may, eventually catch weird 
> exceptions, which leads to a bad experience with Django in general.
>

No, this function is never working in a useful way. It does client side 
interpolation of query params which should be done by the drivers instead, 
even when it works it is potentially dangerous. The actual problem is that 
people on StackOverflow recommend to use it. FWIW Since it is solely a 
debugging aid I'd actually break any usage of it by adding "--" to the 
start of it (or similar)
 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/75c7da85-1e69-40d6-a9e2-d901fb867e41n%40googlegroups.com.

Re: Logging in from one browser logs me out from other browsers (after any change in PBKDF2PasswordHasher.iterations)

[Florian Apolloner]

On Thursday, September 3, 2020 at 11:10:39 AM UTC+2 Adam Johnson wrote:

> You could also move to use the Argon2 hasher, which does not have any 
> chagnes between versions to log out users: 
> https://docs.djangoproject.com/en/3.0/topics/auth/passwords/#using-argon2-with-django
>

Or a custom subclass of a PBKDF2 which sets the wanted iteration count… I 
think the defaults in Django are fine :) 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/525158b1-19fb-4a20-9b2a-67fd7507e809n%40googlegroups.com.

Re: Logging in from one browser logs me out from other browsers (after any change in PBKDF2PasswordHasher.iterations)

[Florian Apolloner]

I do not think there is anything to reopen because it works as designed. 
Password changes cause other browser sessions to be terminated because the 
session auth hash no longer matches.  You can use a custom user model and 
override `get_session_auth_hash` but the defaults won't change, sorry.

On Thursday, September 3, 2020 at 4:56:13 AM UTC+2 Uri wrote:

> Django developers,
>
> I would like to reopen #31970 
> . In short, the problem is - 
> if a user is logged in with more than one browser, and when we upgrade 
> Django to any version which *PBKDF2PasswordHasher.iterations* changes 
> (which is *any* new version), and then the user logs in again - this logs 
> them out from all other browsers. I think this is a bug.
>
> I found out that this can be avoided by changing *def must_update*, for 
> example if you change it to something like:
>
> def must_update(self, encoded):
> # Update the stored password only if the iterations diff is at least 
> 250,000.
> algorithm, iterations, salt, hash = encoded.split('$', 3)
> iterations_diff = abs(self.iterations - int(iterations))
> return ((int(iterations) != self.iterations) and (iterations_diff >= 
> 25))
>
> Or even simply:
>
> def must_update(self, encoded):
> return False
>
>
> אורי
> u...@speedy.net
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/87b16804-3da2-46b7-8ff5-466cd2f38aa2n%40googlegroups.com.

Re: Django default input validation accepts special caracters

[Florian Apolloner]

On Wednesday, August 19, 2020 at 2:01:51 AM UTC+2 cur...@tinbrain.net wrote:

> Seems related to something I learned early in my web career... about not 
> storing values escaped, because you don't know which medium it needs 
> escaping for -- e.g. HTML needs different escaping than URLs.
>

Exactly, this is the way to go. I'd switch any security firm that complains 
about "allowing special characters" in security audits in a heartbeat.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5bfa7411-67e5-414d-bd52-f5162b26db73n%40googlegroups.com.

Re: Status of 3.1 release blockers.

[Florian Apolloner]

On Saturday, August 1, 2020 at 12:53:09 PM UTC+2 Florian Apolloner wrote:

> And also, how to define this setting to its default value in Django 
>> versions <= 3.0.
>>
>
> You don't, this is not supported.
>

Sorry I probably didn't fully understand the question. Either way; this 
setting is not used (and will not be) in Django <= 3.0. You can only set it 
on 3.1 to `sha1` to get the pre 3.1 behavior in 3.1. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/905b4500-39f1-4545-b099-4b3b57a24ee8n%40googlegroups.com.

Re: Status of 3.1 release blockers.

[Florian Apolloner]

On Saturday, August 1, 2020 at 7:16:57 AM UTC+2 Uri wrote:

> How do you propose running several versions of Django 3.0 and 3.1 with 
> this PR and setting? Does it have to be defined, or is it enough using the 
> default setting? 
>

You have to set it to `sha1` and then when all your instances upgraded to 
3.1 you will can remove it at which point all new signatures will use the 
new algorithm.

And when upgrading Django to 3.1, what is the default setting if this 
> setting is not explicitly defined? And how will it affect sessions in case 
> of downgrading back to 3.0? Will it be possible to downgrade also with the 
> default setting?
>

Downgrading will __never__ be supported. This is only for the transitional 
period where you run 3.0 and 3.1 in tandem. Once you switched to 3.1 and 
changed fully to `sha256` there will be no way (never ever!) to downgrade 
that.

And also, how to define this setting to its default value in Django 
> versions <= 3.0.
>

You don't, this is not supported.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7f76827b-a259-437c-8c22-2c4b18e5753cn%40googlegroups.com.

Re: Making startproject's settings more 12-factor-y

[Florian Apolloner]

Comments like these are inappropriate for this mailing list. Stop spamming 
your whatsapp number all over the place.

On Sunday, July 26, 2020 at 6:00:47 PM UTC+2 mithileshr...@gmail.com wrote:

> ok whatsapp +918709440658 <+91%2087094%2040658>
>
> On Fri, 10 Jul 2020, 2:54 pm '1337 Shadow Hacker' via Django developers 
> (Contributions to Django itself),  wrote:
>
>> All right, thank you for your feedback.
>>
>> May I throw in the idea of using DJ_ instead of DJANGO_ as prefix ?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers  (Contributions to Django itself)" group.
>
>
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/PBiCVsKC2UAJFHuD-tEkUh2063D7DgV1DcAxdlcFI2tSqpPxK-8XVWSPbSnyvG5spVf_1QYHZeyNN2dimPNPk7NrS0eh9zq-PhAQxSHAsMk%3D%40protonmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9cc9da3e-daf6-4006-9c90-339f1b52c7a8n%40googlegroups.com.

Re: Should the docs suggest namespace packages?

[Florian Apolloner]

There are also various issues in python itself like 
https://bugs.python.org/issue23882 -- so I strongly recommend against 
namespace packages unless you actually need them.

On Monday, July 20, 2020 at 1:14:41 AM UTC+2 re...@fleschenberg.net wrote:

> Hi,
>
> On 7/19/20 10:25 PM, Tim Graham wrote:
> > Another commenter remarked, "We had a similar issue with test discovery.
> > One of our developers read an article that __init__.py files are not
> > required on Python3 and started removing them. Everything seemingly
> > worked but some tests were not discovered and did not run in CI (which
> > made it difficult to spot). I think Django should not required them if
> > technically possible or at least it should be made clear in the
> > documentation (sorry if I overlooked it)."
> > 
> > Supporting namespace packages without a real use case seems contrary to
> > the consensus in this thread (which I see as not promoting implicit
> > namespace packages). Based on that consensus, my inclination wouldn't be
> > to try to make Django work with as few __init__.py files as possible.
> > What do you think?
>
> I agree. The situation of "One of our developers read an article that
> __init__.py files are not required on Python3" is exactly the one we
> should act against, as far as we can. This is a common misunderstanding
> that we should try to clear up, not promote.
>
> The person you quoted gives an example of why this is not purely
> academical. Maybe it's debatable whether Python should behave
> differently, but that's not something that Django can fix.
>
> I am in favour of failing early and loudly here. IMHO Django should
> ignore migration packages without an __init__.py file, maybe with a
> warning, or maybe even with an error.
>
> -- 
> René Fleschenberg
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c59be0e9-3773-4421-895e-c66d31f220a3n%40googlegroups.com.

Re: Making startproject's settings more 12-factor-y

[Florian Apolloner]

As promised the PR: https://github.com/django/django/pull/13120 I also had 
to touch TokenResetGenerator since it stores the secret in a class level 
variable and that would prevent the server startup either way… 

Let's see what the full test suite says, but I cannot really imagine many 
issues.

On Sunday, June 28, 2020 at 10:27:44 AM UTC+2 Florian Apolloner wrote:

> On Sunday, June 28, 2020 at 1:23:25 AM UTC+2 timog...@gmail.com wrote:
>
>> It seems like it could be insecure to move that to a system check as "For 
>> performance reasons, checks are not run as part of the WSGI stack that is 
>> used in deployment." (Also, it seems impossible to write a system check 
>> that determines whether or not a project will consult SECRET_KEY.)
>>
>
> The check if it is empty can be done on access in LazySettings without any 
> real overhead. We are even calling url validators when access 
> MEDIA_URL/STATIC_URL (
> https://github.com/django/django/blob/62d85a283500e9abb0e1c9ec53c59be468f056a0/django/conf/__init__.py#L152-L158)
>  
> -- so we really don't have to talk about overhead here :D
>
> I will try to remove those properties and move the checks into 
> `__getattr__`, this should result in a (small) one time overhead. PR(s) to 
> follow.
>
> I think a deployment system check for non-empty SECRET_KEY might also make 
> sense.
>
> Cheers,
> Florian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/596f6a5b-7ca0-4dd3-b765-f4d559758920n%40googlegroups.com.

Re: Making startproject's settings more 12-factor-y

[Florian Apolloner]

On Sunday, June 28, 2020 at 1:23:25 AM UTC+2 timog...@gmail.com wrote:

> It seems like it could be insecure to move that to a system check as "For 
> performance reasons, checks are not run as part of the WSGI stack that is 
> used in deployment." (Also, it seems impossible to write a system check 
> that determines whether or not a project will consult SECRET_KEY.)
>

The check if it is empty can be done on access in LazySettings without any 
real overhead. We are even calling url validators when access 
MEDIA_URL/STATIC_URL 
(https://github.com/django/django/blob/62d85a283500e9abb0e1c9ec53c59be468f056a0/django/conf/__init__.py#L152-L158)
 
-- so we really don't have to talk about overhead here :D

I will try to remove those properties and move the checks into 
`__getattr__`, this should result in a (small) one time overhead. PR(s) to 
follow.

I think a deployment system check for non-empty SECRET_KEY might also make 
sense.

Cheers,
Florian 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/688bcec8-ed3f-4c35-bdf2-ad5fe4929ca4n%40googlegroups.com.

Re: Making startproject's settings more 12-factor-y

[Florian Apolloner]

Hi Tom,

On Friday, June 26, 2020 at 12:43:19 PM UTC+2 t...@carrick.eu wrote:

> I do have a use-case where having a default SECRET_KEY makes things much 
> easier - docker.
>
> Normally you can't run management commands in a Dockerfile if there's no 
> secret key (and often other things) set, and usually it's best to run 
> collectstatic as a build step.
>

 I do not see that as an argument for a default SECRET_KEY. Maybe we should 
just defer the check for SECRET_KEY till it is accessed? There is no reason 
Django should complain a missing secret key when it does not need one at 
all.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/83630d2b-f51b-4ec3-b92a-838feb452fa1n%40googlegroups.com.

Re: Making startproject's settings more 12-factor-y

[Florian Apolloner]

Hi there,

On Thursday, June 25, 2020 at 7:52:31 PM UTC+2 kit@gmail.com wrote:

> Personally, I think that *at minimum* providing Django-builtin "get from 
> env"  helpers would be great; beyond that, I'd love to have them be 
> included around `DEBUG` and `SECRET_KEY` with the current values as 
> defaults, so they're optional. Once we see how this gets used, we can see 
> about passing it a file instead of `os.environ`, or borrowing other ideas 
> from any of the various supporting projects that have been suggested.
>

I am all for minimal variants, but I do not think this would could it. your 
"get from env" helpers would basically be os.environ.get("SECRET_KEY", 
"secret_default") which does provide much usefulness on it's own imo. So 
even if we started out with a minimal implementation we'd at least have to 
have a good plan on what to do in the future.

And there are plenty more things to consider; for instance I do not agree 
that it makes sense to have "SECRET_KEY" default to a value when missing in 
the env. It is way to easy to type "SECRT_KEY" and never realize that. So 
if "SECRET_KEY" is taken from the environment it should fail loudly if it 
is not present. "DEBUG" is in a similar category there but could default to 
False to be safe.

I personally rather have no solution in Django itself before forcing a 
half-baked one down everyone. Also please note that the bar to add this to 
Django is very high since it can (at least for things like django-environ) 
easily live outside of Django with no realy downside.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/68e6a5e0-ea75-43e9-b34a-24f18ced73d4n%40googlegroups.com.

Re: Overriding template blocks without copy/pasting entire template files

[Florian Apolloner]

Ah, might be that you got hold up in a moderation queue or similar. Google 
groups is weird sometimes -- sorry!

On Tuesday, June 16, 2020 at 3:20:25 PM UTC+2, Arvind Nedumaran wrote:
>
> Something must be faulty with my email delivery I suppose. I sent that 
> email a good 8 hours ago. :) 
>
> Onward, 
> Arvind 
>
> Sent from Outlook Mobile <https://aka.ms/blhgte>
>
> --
> *From:* django-d...@googlegroups.com  <
> django-d...@googlegroups.com > on behalf of Florian 
> Apolloner >
> *Sent:* Tuesday, June 16, 2020 6:46:45 PM
> *To:* Django developers (Contributions to Django itself) <
> django-d...@googlegroups.com >
> *Subject:* Re: Overriding template blocks without copy/pasting entire 
> template files 
>  
> If you look at the mails beyond the first one, you will see that Josh & 
> Carlton already came to that conlusion ;)
>
> On Tuesday, June 16, 2020 at 2:16:31 PM UTC+2, Arvind Nedumaran wrote: 
>
> Could you elaborate on how this is different from extending a base 
> template and re-defining the necessary blocks? 
>
> #base.html
> {% block title %}{{ section.title }}{% endblock %}
> . 
>
> {% extends "base.html" %}{% block title %} My custom title that isn't 
> section.title {% endblock %}
> 
>
> This much is already possible right? 
>
>
> On Tue, Jun 16, 2020 at 10:34 AM Josh Smeaton  wrote:
>
> Something that has bugged me for awhile is the requirement to copy and 
> paste an entire template when you would just like to override a single 
> block. This arises mostly when overriding admin templates, like 
> `admin/base.html`.
>
> In my ideal world, I'd be able to do something like this:
>
> # myapp/templates/admin/base.html
> {% override "admin/base.html" %}
> {% block footer %}this site is restricted, blah blah legal text blah{% 
> endblock %}
>
>
> And then the template loading system would find the next `admin/base.html` 
> in the chain and use my overrides.
>
> There is prior art too. https://pypi.org/project/django-apptemplates/ 
> allows you to override a template from a specific app using this syntax:
>
> # myapp/templates/admin/base.html
> {% extends "admin:admin/base.html" %}
> {% block footer %}this site is restricted, blah blah legal text blah{%
>  endblock %}
>
>
> I think this kind of functionality should be included within Django 
> itself. If others agree, should there be a new name such as override, or 
> would overloading extends be good enough?
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-d...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/6cf43971-1ea9-4b93-9a35-aaab5908327co%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/django-developers/6cf43971-1ea9-4b93-9a35-aaab5908327co%40googlegroups.com?utm_medium=email_source=footer>
> .
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-d...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/89b8d89a-d7e4-44fa-a5b8-cb8e1c910785o%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/django-developers/89b8d89a-d7e4-44fa-a5b8-cb8e1c910785o%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a357fb57-55b3-4f65-bf59-cf64c3e412bbo%40googlegroups.com.

Re: Overriding template blocks without copy/pasting entire template files

[Florian Apolloner]

If you look at the mails beyond the first one, you will see that Josh & 
Carlton already came to that conlusion ;)

On Tuesday, June 16, 2020 at 2:16:31 PM UTC+2, Arvind Nedumaran wrote:
>
> Could you elaborate on how this is different from extending a base 
> template and re-defining the necessary blocks? 
>
> #base.html
> {% block title %}{{ section.title }}{% endblock %}
> . 
>
> {% extends "base.html" %}{% block title %} My custom title that isn't 
> section.title {% endblock %}
> 
>
> This much is already possible right? 
>
>
> On Tue, Jun 16, 2020 at 10:34 AM Josh Smeaton  > wrote:
>
>> Something that has bugged me for awhile is the requirement to copy and 
>> paste an entire template when you would just like to override a single 
>> block. This arises mostly when overriding admin templates, like 
>> `admin/base.html`.
>>
>> In my ideal world, I'd be able to do something like this:
>>
>> # myapp/templates/admin/base.html
>> {% override "admin/base.html" %}
>> {% block footer %}this site is restricted, blah blah legal text blah
>> {% endblock %}
>>
>>
>> And then the template loading system would find the next 
>> `admin/base.html` in the chain and use my overrides.
>>
>> There is prior art too. https://pypi.org/project/django-apptemplates/ 
>> allows you to override a template from a specific app using this syntax:
>>
>> # myapp/templates/admin/base.html
>> {% extends "admin:admin/base.html" %}
>> {% block footer %}this site is restricted, blah blah legal text blah
>> {% endblock %}
>>
>>
>> I think this kind of functionality should be included within Django 
>> itself. If others agree, should there be a new name such as override, or 
>> would overloading extends be good enough?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-d...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/6cf43971-1ea9-4b93-9a35-aaab5908327co%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/89b8d89a-d7e4-44fa-a5b8-cb8e1c910785o%40googlegroups.com.

Re: What's wrong with the jenkins testrun failing on ASGI (I think?)

[Florian Apolloner]

Ok, so rebasing PRs to current master will fix this (leaving this here as 
note for others who run into this).

On Tuesday, June 16, 2020 at 10:39:13 AM UTC+2, Mariusz Felisiak wrote:
>
> It's an issue with the asgiref==3.2.8, see 
> https://github.com/django/asgiref/issues/170. We temporarily pinned 
> asgiref==3.2.7 [1].
>
> Best,
> Mariusz
>
> [1] 
> https://github.com/django/django/commit/dcb4d79ef719d824431a8b3ff8ada879bbab21cc
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c775e1a0-9485-46da-b4d9-015e8ee019b9o%40googlegroups.com.

What's wrong with the jenkins testrun failing on ASGI (I think?)

[Florian Apolloner]

Hi,

do we have any idea what is happening on 
https://djangoci.com/job/pr-mariadb/database=mysql,label=mariadb,python=python3.8/6459/
 
? It seems as if (mostly) the same set of tests is failing over and over. 
Is this ASGI/concurrency related?

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/69e9c9a2-7945-4a5f-96e2-e1f594f773ebo%40googlegroups.com.

Re: Implement QuerySet.__contains__?

[Florian Apolloner]

On Tuesday, June 2, 2020 at 11:28:34 AM UTC+2, Adam Johnson wrote:
>
> If you already fetched the queryset, `if obj in queryset` will make a new 
>> database query
>
>
> I don't see why we couldn't implement __contains__ to do a walk through 
> _result_cache if it has been fetched?
>

we are doing the same for len() IIRC. I have to say it sometimes makes it a 
little bit hard to debug, but if it is documented as such it should be okay.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fa0eae17-2c25-4ae2-befb-c20ebfb8d1da%40googlegroups.com.

Re: Django Version 3.2 Roadmap

[Florian Apolloner]

On Wednesday, May 13, 2020 at 4:40:55 PM UTC+2, Adam Johnson wrote:
>
> This looks good to me, it fits the preset release cadence. Although the 
> final release will be on "April fools day," it's something we've done 
> several times before.
>

While we might have done it (did we actually do that several times 
before?), I'd like to suggest moving it  to April 10th. (or whatever suits 
us). Every year the first of April seems to be a high tension day, if there 
is no necessity to release on such a day I'd strongly recommend against it. 
It is just easier on the brain if you read a news article and you don't 
have to decide if it is a possible prank simply because it is fools day. I 
realize that you cannot trust news any other day either, by why pile onto 
that if you don't have a reason…

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f9e3c147-06d8-4b3d-a250-cd3a8d00af90%40googlegroups.com.

Re: Rename request.GET and POST, and lowercase COOKIES, FILES, and META

[Florian Apolloner]

On Saturday, May 9, 2020 at 1:41:41 PM UTC+2, Aymeric Augustin wrote:
>
> - Users who don't know that GET queries mustn't have side effects and who 
> don't want to bother will have an incentive to use a POST query to get the 
> data in form_data; this is the safe option with regards to CSRF;
>

This is imo the best argument in this thread and convinces me a little bit 
:D But I still think we should accompany this with documentation that 
explains why GET-powered forms do not show up in form_data. What irks me a 
little bit is that request.GET/POST nicely stood out in code -- ie I could 
find user input handling on a first glance.

As for disruption, can we just alias them for now and drop GET/POST from 
the docs and not immediately start with warnings?

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ce0ea3c0-0a73-401e-a2ca-8427b4551f7d%40googlegroups.com.

Re: Rename request.GET and POST, and lowercase COOKIES, FILES, and META

[Florian Apolloner]

On Wednesday, May 6, 2020 at 5:49:40 PM UTC+2, Adam Johnson wrote:
>
> Would just like to point out - that's not the proposal. The proposal is to 
> rename the existing request.POST to request.form_data, which is based on 
> parsing application/x-www-form-urlencoded data from request.body. Browsers 
> only send such data in the body on POST.
>

I am not convinced. As Carlton point out, forms with method="GET" are 
legitimate and wouldn't end up in `form_data` which is imo quite confusing. 
Do we have any other options? (Hard I know, but…) 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6bcf7fca-470e-453d-b832-4eac53f8dae2%40googlegroups.com.

Re: [Probably BUG] set_password and check_password accept values other than string as parameters

[Florian Apolloner]

oh, I somehow missed that. thanks for the heads-up.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/497793c5-d4f8-4ab7-8e01-cbfd6822ca48%40googlegroups.com.

Re: [Probably BUG] set_password and check_password accept values other than string as parameters

[Florian Apolloner]

On Thursday, March 12, 2020 at 6:16:31 PM UTC+1, Dawid Czeluśniak wrote:
>
> I think that the root question here is: should we allow users to create 
> passwords from anything that is not str?
>

I would reduce it to allow strings & bytes and am willing to review a patch 
and push it through. It is security sensitive enough that I think we can 
ignore the (imo non-existant) backwards compatibility issues.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b667a3cd-6334-4496-a174-5b41351450a3%40googlegroups.com.

Re: Discuss ticket 20264: URLValidator should allow underscores in local hostname

[Florian Apolloner]

Hi Adam,

On Wednesday, March 25, 2020 at 7:27:58 PM UTC+1, Adam Johnson wrote:
>
> I think that would make Florian happy, although it *has* been seven years 
> since his closing comment on the ticket.
>

You should know me better :D No this would not make Florian happy and he is 
still against it. By all means add a lenient=False flag which can be turned 
to True to enable lenient parsing but the defaults should imo stay.

It might be true that for the sole purpose of __displaying__ URLs that an 
underscore will not hurt, but in the greater scheme of things it simply 
does not work:

 * java.net.URI will not parse it: new 
java.net.URI("http://test_host.com;).getHost -> null
 * While you laugh about me mentioning java the more relevant argument is 
that we are going towards a HTTPs world and there you have to play by a 
different set of rules namely CA/Browser Forum Baseline Requirements. These 
requirements require you to follow RFCs (especially RFC 5280) which in turn 
requires subjectAltNames to follow the preferred style of RFC 1034 which 
finally disallows the use of underscores. So for this reason CAs won't 
allow you to issue certs for those hostnames, you can only make those work 
via wildcard certs, which in turn only work for subdomains and not TLDs.

So this limits the usefulness of underscores in URLs to mainly http-only 
sites or sites that went around extra hoops to get it working. In that 
sense I do not see a strong requirement to be lenient in parsing by default.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e6a7c79d-f53f-4893-bf05-06fa5475f915%40googlegroups.com.

Re: Deprecating logout via GET

[Florian Apolloner]

UX wise the intermediary page is annoying. So if we can just provide some 
docs and upgrade notes I'd be fine with just dropping the get part.

On Thursday, March 5, 2020 at 5:40:25 PM UTC+1, René Fleschenberg wrote:

> Hi, 
>
> On 04.03.20 12:13, Sam Willis wrote: 
> > Why not have the logout link take the user to a page asking them to 
> confirm the logout, and have it as a POSTed form button from there? 
> > 
> > That adds a helpful confirmation page, removes the difficulties of 
> styling a button as a link constantly (or changing the header design to a 
> button). 
> > 
> > One downside would be the change in behaviour, people will be used to 
> the logout link immediately logging them out and so may not read the next 
> page... 
>
> Personally I am fine with an intermediate page or without one. 
>
> It would just be nice to reach some decision on this so we can then move 
> forward with https://github.com/django/django/pull/12505, or file 
> another PR that introduces the intermediate page. 
>
> I agree with Aymeric that some accessibility testing on this would be 
> good, however. 
>
> Regards, 
> René 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e402a6b5-88ca-4834-bd2c-e69462b75815%40googlegroups.com.

Re: Deprecating logout via GET

[Florian Apolloner]

I found an example on stackoverflow on how we could do this in the admin 
without JS (with a bit of styling): https://stackoverflow.com/a/33880971 -- 
I personally would prefer it if we would not need javascript for a 
fundamental functionality like this.

On Saturday, February 29, 2020 at 9:26:23 AM UTC+1, Adam Johnson wrote:
>
> Google (=Gmail): GET, but with a security token in the URL
> Facebook: POST
> Instagram: POST
> Twitter: POST
>
> ‪On Sat, 29 Feb 2020 at 08:08, ‫אורי‬‎ > 
> wrote:‬
>
>> I'm interested: Google, Gmail, Facebook, Instagram, Twitter: How do they 
>> use logout? POST or GET?
>> אורי
>> u...@speedy.net 
>>
>>
>> On Thu, Feb 27, 2020 at 7:10 PM René Fleschenberg > > wrote:
>>
>>> Hi everyone,
>>>
>>> there seems to be consensus that logging the client out on GET requests
>>> to the logout view is not great. Clients may try to prefetch links (this
>>> came up on IRC today). Attackers might annoy users by logging them out
>>> with embedded links to the logout URL.
>>>
>>> The reason this was not changed yet is backwards compatibility.
>>>
>>> I'd like to propose deprecating logging out via GET with the usual
>>> deprecation cycle, so that we have the possibility to stop supporting it
>>> in Django 4.0.
>>>
>>> In my opinion, this is a fairly easy change for Django users to make
>>> upon a Django update. Some changes to Django itself will also have to be
>>> made (the admin templates), but I think it should be possible to do
>>> those before the 4.0 release. And even if that doesn't happen, emitting
>>> a DeprecationWarning in 3.1+ wouldn't harm, right?
>>>
>>> What do you think?
>>>
>>> I went ahead and created a PR for this:
>>> https://github.com/django/django/pull/12504
>>>
>>> -- 
>>> René Fleschenberg
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers  (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-d...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/9baf0189-fc7a-80a0-6543-559c2b2b186a%40fleschenberg.net
>>> .
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-d...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CABD5YeHh%2BNvu4vtwaD8pC9526bAVpyA1iwo94c5bVSjRUa5eNg%40mail.gmail.com
>>  
>> 
>> .
>>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2598e4a0-1b0d-4d34-a807-2b243af72053%40googlegroups.com.

Re: Deprecating logout via GET

[Florian Apolloner]

On Friday, February 28, 2020 at 5:08:07 PM UTC+1, Maher, Brian wrote:
>
> Are any current browsers dumb enough to prefetch logout links these days? 
> I would assume that most prefetch algorithms are smart enough to not 
> pre-fetch these. 
>

Not sure what heuristics browsers use.

I have also seen the argument floated around that it’s not “correctly 
> restful” to modify the state (session data) via a GET request. I’d say that 
> sessions themselves are not restful by nature. 
>

It has nothing do to with restful, logout by GET is not CSRF protected…

I just don’t see the benefit, in return for breaking practically every 
> logout button on every installation of Django around.
>

Not necessarily, if you GET the logout page an intermediary page can be 
displayed (though that would need a new template).

 is currently a perfectly nice way to 
perform a logout against your django installation if you are viewing this 
google group in the same browser ;)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a08a0f8b-dd51-48a8-9a62-ef730f90bd35%40googlegroups.com.

Re: GitHub Actions

[Florian Apolloner]

On Wednesday, November 6, 2019 at 4:48:11 PM UTC+1, Tom Forbes wrote:
>
> The usage limits are documented here: 
> https://help.github.com/en/github/automating-your-workflow-with-github-actions/about-github-actions#usage-limits.
>  
> I’m not sure how they apply to organisations (maybe just “free”?), but I 
> think Github would allow us an increased limit.
>

If not we can still work towards using our own workers, we have the 
capacity for it. 
 

> What do we think?
>

Fantastic! 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9e728efd-a8ca-4cdf-98e0-b99dc91c83c9%40googlegroups.com.

Re: GitHub Actions

[Florian Apolloner]

On Wednesday, November 6, 2019 at 8:43:21 AM UTC+1, Shai Berger wrote:
>
> Is there benefit enough in GitHub Actions (over Jenkins) to justify a 
> move from an open-source based solution? 
>

Yes, less server costs (even if sponsored). Less things to maintain for us 
(Jenkins is a beast). Better and more reliable integration with github. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d34795bc-7490-43b0-bdcb-8a3023939c9b%40googlegroups.com.