Re: select_for_update and transactions (bad dev UX in a feature)

[Aymeric Augustin]

Hello,

> On 18 Nov 2021, at 11:11, Florian Apolloner  wrote:
> 
> FWIWI I always recommend disabling ATOMIC_REQUESTS and using transactions  as 
> needed :)

Investing engineers' time into evaluating the exact transactional integrity 
requirements of every view may be appropriate in some contexts. Getting a 
blanket safeguard from ATOMIC_REQUESTS can be a fair tradeoff in other contexts.

Engineer time is expensive. PostgreSQL horsepower can be pretty cheap on low or 
even medium traffic websites, which are a substantial part of Django's target 
audience.

On these websites, it can be entirely reasonable to pay a limited cost for 
ATOMIC_REQUESTS and to get the benefit that every view succeeds completely or 
fails completely.

Also you don't depend on engineers getting the analysis right every time, 
including when they make a change — the situation that triggered this 
discussion in the first place.

> Honestly long transactions are imo bad, one should evaluate what the view 
> needs and then act accordingly.


Since most views on most websites are short, it isn't completely bonkers to use 
a blacklist approach, that is, to review the situation closely on long views 
and apply non_atomic_requests appropriately.

I'm not trying to disagree for the sake of disagreement; I'm just trying to 
bring some contextual awareness and avoid  the "core devs say ATOMIC_REQUESTS 
is bad" effect. I hope we can agree on this?

Finally — yes, I know that my implementation of this feature is a wormhole 
across layers. I won't even try to justify it. I wish we had something better 
but we haven't found that yet.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/16779272-37F9-4BB6-A5F7-4CE5B226E2A9%40polytechnique.org.

Re: Proposed change in ORM model save logic

[Aymeric Augustin]

Hello,

> On 15 Oct 2021, at 08:49, Barry Johnson  wrote:
> 
> Instead of resolving this difference in keys by executing:
> setattr(self, field.attname, obj.pk)
> We believe it should instead:
> setattr(self, field.name, obj)

For anyone reading this, this code is 
https://github.com/django/django/pull/13589/files#diff-1c8b882c73bfda668d5451d4578c97191b0ebc0f088d0c0ba2296ab89b428c44R939
 


I think the proposed change is correct. I reviewed this pull request and 
related ticket, as well as prior art in the area, and I'm not seeing a reason 
for assigning just the PK (= PK set, related cache not set) rather than the 
entire object (= PK and related cache both set).

Disclaimer — while I was involved in this area long ago, I didn't follow what 
happened after commit 5980b05c, where the "prevent data loss with unsaved 
related objects" checks were moved from the moment a related object is assigned 
to the moment the object is saved.

I reviewed Carlton's wontfix, which made sense to me, to understand why we 
didn't reach the same conclusion. Carlton thinks the code has been here for a 
very long time and would be tricky to touch; I believe that we're only touching 
a line introduced in 519016e5, 2.5 years ago, which doesn't feel _that_ old to 
me.

Hope this helps!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3F76E6A2-439B-4510-AF32-F9ED766A34CB%40polytechnique.org.

Re: Proposal for a transaction.on_before_commit

[Aymeric Augustin]

Hello Raphael,

Oh - a use case for django-transaction-signals 
 ;-) I'm bringing up 
this elaborate joke because you're essentially asking for a "pre-commit" signal 
here and the README contains a good list of things that can go wrong with 
transaction signals. (Please ignore how this package demonstrates a way to do 
it as third-party code *cough* *cough* *cough*)

> I figured a perfect way to do this would be using `save()` or
> `post_save` to add the changed model instance to some kind of
> thread-local list, and then using `transaction.on_commit` to "schedule"
> the aggregation and create the log entries when all changes have been
> made. However, this obviously is not a good enough because `on_commit`
> runs *after* the `COMMIT` statement and thus we're not guaranteed that
> all log entries are persisted to the database.


In my opinion "saving the log entries may fail after a successful transaction" 
isn't the main design problem here. The bigger problem is "log may contain 
entries for writes that don't actually happen, because some savepoints were 
rolled back, typically due to atomic blocks exiting with an exception". And 
then you get dragged into the whole complexity that the README of 
django-transaction-signals outlines and that we're trying to avoid in Django.

(If you don't have any savepoint rollbacks, then your code sounds sufficiently 
constrained to implement logging of changes explicitly at the application layer 
rather than at the framework layer.)

If you run with ATOMIC_REQUESTS, I would suggest to replace it by a custom 
middleware that wraps get_response(request) in an atomic block. Then you know 
that this is the outermost traction and you can do whatever needed before 
exiting the atomic block. You also need the same in all management commands, 
which could be a problem if you depend on third-party management commands.

Failing that, in order to log changes with transactional correctness enforced 
by the ACID guarantees of PostgreSQL, I'd recommend doing it at that database 
level with triggers — which always execute in the same transaction. I realize 
it may be difficult to perform the kind of aggregation you have in mind.

As a last resort, I'd try a custom database backend to track accurately 
transactions and savepoints and maintain an up-to-date record of changes that 
will happen when the transaction is committed.

Hope this helps!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/E028A2F2-9E00-4599-B1F0-E1406F5810A6%40polytechnique.org.

Re: Custom Groups model and interoperability

[Aymeric Augustin]

Hello,

The semi-official position is "create a separate model with a FK to the 
auth.Group". I know that's less than helpful. At least you know that there's 
nothing better built-in.

-- 
Aymeric.

PS: `{% block content %}` doesn't provide a meaningful level of 
interoperability. The semi-official position on this is "there is no reasonable 
way to do pluggable templates".

> On 5 Oct 2021, at 19:13, Scot Hacker  wrote:
> 
> 
> No official (or semi-official) position on this? Is "The right way to create 
> custom Groups" simply undefined in Django? 
> 
> Thanks.
> 
> 
> On Sunday, October 3, 2021 at 10:27:36 PM UTC-7 Scot Hacker wrote:
> I asked this question on django-users and got no response. Thought I might 
> have better luck getting an "official" opinion on this here.
> 
> You can call blocks in your templates anything you like, but if you intend to 
> share your software with the world, you'd better use `{% block content %}` if 
> you expect interoperability between your project and 3rd party apps.
> 
> My question is along similar lines, but relates to using custom Groups. There 
> is no `settings.AUTH_GROUPS_MODEL` equivalent to `AUTH_USER_MODEL`. If you 
> make a custom Group model that subclasses Django Group, you can still use 
> `user.groups.all()`. But if a project uses a Group model that that does not 
> subclass `Group`, that doesn't work of course.
> 
> I can't seem to find anywhere in the docs where Django recommends doing it 
> one way or another. I've always thought that a custom group should subclass 
> Group, and that is common, but some projects apparently don't do it that way.
> 
> I am asking because I am the maintainer of a reusable app that requires 
> groups compatibility, and have come across an app that that uses a 
> non-Django-derived groups system. We have come to impasse about the best way 
> to support this (or whether to support it all).
> 
> Are there any best practices on this question in the Django docs or elsewhere?
> 
> Thanks,
> Scot
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/6d593b84-e0d9-4a08-8225-c49bbfa3bfacn%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6482479E-AADA-4851-B031-F6F5CD02E8E6%40polytechnique.org.

Re: Feature request : Load signals automatically

[Aymeric Augustin]

Hello,

I'd like to to chime in quickly as I'm responsible for this boilerplate (see 
this talk , slide 33).

This is really a question of convention vs. configuration. Django tends to 
prefer configuration over convention, because there's value is seeing what 
happens just when you read the code, especially when you read other people's 
code. Other frameworks take a different approach to this, perhaps because they 
care more about writing less code or because they focus more on the "single 
person team" use case.

As Adam points out in his reply on the forum, Django autoloads models, admins, 
and apps. Most Django users and maintainers love models and admins; hopefully 
they admit that apps solve a problem that needed solving. At the same time, 
many of them are also skeptical of signals, having witnessed excessive use of 
signals in projects, usually in the name of decoupling, and sometimes because 
the person writing the code didn't find any other way to break an import loop. 
(It's easy: import modules instead of objects, then reference all objects as 
mymodule.MyObject, and you're good.)

Signals result in inversion of control: you declare a function call in the 
callee rather than in the caller. This tends to make it more difficult to 
figure out what the code is doing. In my opinion, the only valid use case for 
signals is when the code sending the signal is provided by a third-party 
library that you don't control. If you have signals inside your project, I'd 
suggest refactoring them to good old function calls. Then perhaps you have 
fewer signals and don't feel such a need to autoload them. If you still have 
too many, Adam's suggestion works :-)

-- 
Aymeric.



> On 14 Sep 2021, at 17:37, 'Adam Johnson' via Django developers (Contributions 
> to Django itself)  wrote:
> 
> I've replied on the forum.
> 
> On Tue, 14 Sept 2021 at 13:21, Roxane Bellot  > wrote:
> 
> 
> Hi guys !
> 
> First of all, i’m new here, so if this issue has already been discussed or if 
> here is not the place to talk about it (I also opened a forum post 
> ) please 
> don’t hesitate to tell me. Also, please be aware English is not my first 
> language.
> 
> I would want to discuss about signals and how they are loaded in Django.
> 
> Very often, i create my signals.py, write my signal and wonder for a while 
> why it does not work. Of course, the file is not imported and so never even 
> read by python !
> So i create an apps.py, on this template :
> 
> from django.apps import AppConfig
> 
> 
> class CustomAppConfig(AppConfig):
> name = "app.subapp"
> 
> def ready(self):
> import app.subapp.signals
> 
> I would like to sugest importing signals.py automatically at app loading 
> (maybe django.apps.config.AppConfig.ready() ?)
> 
> Pros
> 
> no more wandering for hours what is the problem here
> no more boilerplate apps.py everywhere
> Cons
> 
> file with signals would have to be named signals.py
> maybe some rely on the file being imported when they decided ? (don’t see a 
> use case here, specially when loading signals does not equals activating them)
> Alternatives
> 
> If loading (and so activating) signals is too mush of a change, maybe others 
> solutions will fix my problem. For know, all I can think of :
> 
> Log if a signals.py file is detected but the AppConfig is not overriden (but 
> we would not know if imported elsewhere)
> Log how many signals are detected for the app (but from a dozen apps the log 
> will be too heavy to be read)
> not really an alternative, but the loading could be conditionnal to a 
> project-wide setting
> i would like to know if this would benefits others (I think so) and if it is 
> worthy of filing an Issue (as a developper I may find time to dev on it if 
> needed)
> 
> 
> 
> Looking forward to reading from you,
> 
> Roxane Bellot
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAAvJ%2BFzx%3DFZ01BvBhEtH8SOZQA3imM4q3j1FNA7EF6ec1BKrPA%40mail.gmail.com
>  
> .
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> 

Re: Django models `EmailField` default is empty string - Not `None` (or Null) . Why ?

[Aymeric Augustin]

Hello,

https://docs.djangoproject.com/en/3.2/ref/models/fields/#django.db.models.Field.null
 

 should clarify the situation.

-- 
Aymeric.



> On 13 Jun 2021, at 13:17, Carlos Leite  wrote:
> 
> 
> Yesterday I realized that, Django email field default value is not "Null" 
> actually is an empty string ("")  - Although I'm not code often, I was 
> surprised.
> 
> I wrote some tests for a model where an  `emailfield` was supposed to be 
> `null=False`. 
> 
> But during the tests, it didn't raise an Exception (when persist on DB). Or 
> for a ModelForm `is_valid` in fact was True.
> 
> I did some (not a lot) research, and it seems the default value for an empty 
> value for EmailField is ""
> Isn't it counter productive? (at least for me, it was)
> I believe, the empty value,  is a special case for a Model Field 
> and I couldn't find any  information on docs. (I told I did SOME research) 
> I did check 
> https://docs.djangoproject.com/en/3.2/ref/models/fields/#emailfield 
> 
> 
> Please let me know my expectations make sense, I'll do my best for help, 
> if not, I know its not the place for "help", just tell me I'm wrong. 
> 
> Thanks everyone.
> 
> 
> Cadu Leite
> | Twitter 
> | @cadu_leite 
> 
> http://people.python.org.br/ 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAEM6-Q%2B1VHddFaXWNFgdd3fU%2B1BHOVo0U%3De-SOt2-ptFq24CcQ%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/BDFE41D4-0087-4EA3-970D-3F20D0C30502%40polytechnique.org.

Re: Transaction APIs do not consult the DB router to choose DB connection

[Aymeric Augustin]

> On 3 Jun 2021, at 12:03, N Aditya  wrote:
> 
> 1. If atomic(using=...) is the way to go and the same has been implemented 
> for ORM queries as well, why introduce something like the routers framework 
> in the first place ?

You can meaningfully route individual ORM queries based on the model being 
queried and it's often useful to do so. So Django supports it.

It would be convenient to do the same for transaction blocks, but it isn't 
possible in a generally useful way because you don't know yet which models will 
be queried. So Django doesn't support it.

> 2. Also generally speaking, was the intention of building the routers 
> framework only to allow routing based on models and other hints available as 
> part of the args list

Yes, it was.

> or to expose hooks that can achieve routing of DB queries based on custom 
> logic that could either depend on the args list or not (The use-cases are 
> plenty. It can be from some state being pushed into a KV store based on some 
> logic that gets executed in the view etc., not just thread-local state. Also, 
> considering thread-local state here to be globals seems completely unfair 
> since in actuality, it is tied to the request's scope and not global for all 
> requests).

No, it wasn't.

> Also, if possible, I'd like to hear opinions about the above from the authors 
> of the routers framework as well. 

You could read what Russell wrote at that time:

https://github.com/django/django/commit/1b3dc8ad9a28486542f766ff93318aa6b4f5999b
https://code.djangoproject.com/ticket/12540
https://code.djangoproject.com/ticket/12541


> Also, I believe you quoted this in your previous message:
> > Indeed, you'd need something slightly smarter, but since connections are 
> > thread-local in Django, it should be manageable.
> 
> (...)

Let's say my proposal doesn't work, then. I didn't try it!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/35082C11-6797-43D9-9951-DC4935DEE62C%40polytechnique.org.

Re: Transaction APIs do not consult the DB router to choose DB connection

[Aymeric Augustin]

> On 2 Jun 2021, at 07:49, N Aditya  wrote:
> 
> Below are a few things I'd like to clarify:
> Are you referring to thread-locals as `global state/variables` in your 
> previous message ?
Yes.
> If so, why suggest something which you consider bad practise ?
You rejected the good practice before — atomic(using=...) so I'm looking for 
something that matches your constraint.
> As a consequence, if using thread-locals is considered bad software engg, 
> could you please give me a way by which I could pass on request metadata to 
> the router layer cleanly ?
Pass it explicitly with `atomic(using=...)`. I'm aware that it doesn't work for 
third-party libraries that you don't control. I don't have a perfect solution.
> The final recommendation you gave is something that isn't officially 
> supported and would fail when a threaded server or a fully async request path 
> is used.
> (Even your favourite search engine might not be of much help here)
Indeed, you'd need something slightly smarter, but since connections are 
thread-local in Django, it should be manageable.

I don't think you can meaningfully put "fully async request path" and "database 
transaction managed by the Django ORM" in the same sentence. You'd run the 
transaction in a thread pool, which brings you back to by previous case.

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/37A77A89-74F1-498D-AE41-9772528E4E32%40polytechnique.org.

Re: Transaction APIs do not consult the DB router to choose DB connection

[Aymeric Augustin]

Hello,

> On 1 Jun 2021, at 14:35, N Aditya  wrote:
> 
> All I'm looking for is a hook that the transaction APIs can call before 
> deciding on which database to use. I don't see any reason for why providing a 
> hook seems so difficult. 

Just because something is easy to implement doesn't mean it gets added to 
Django. It also needs to be generally useful and well designed. Your proposal 
doesn't meet the second criterion and I'm skeptical about the first one.

Indeed, the get_connection() API you're proposing doesn't take any argument. As 
a consequence, the only way to make it return a different value at different 
times would be to depend on global variables. This is bad software engineering. 
It doesn't meet Django's standards.

(If you don't know why having functions that return different values based on 
global variables is a bad idea, you can search for "global state" in your 
favorite search engine. I'm pretty sure you can find a good blog post 
explaining why. And yes Django suffers from some legacy in this area. We are 
trying to avoid taking more debt.)

Four committers rejected your proposal and suggested alternatives. We're trying 
to help. Pushing the same idea again won't work any better. Please take into 
account the objections and try to find a design that addresses them, or 
implement one of the alternatives we offered.

Thank you,

-- 
Aymeric.




-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/940A0E63-8879-4F6E-AF27-ED5F8880611D%40polytechnique.org.

Re: Transaction APIs do not consult the DB router to choose DB connection

[Aymeric Augustin]

Hello,

The first item in Django's design philosophies is Loose coupling 
.
 Per this principle, the database layer shouldn't know about the HTTP layer. 
This is a strong reason for keeping the HTTP request object away from the 
database layer (e.g. database routers or the transactions API). This is why "a 
hook which can be invoked with the request object as its param to make a 
decision" is extremely unlikely to be accepted.

If you disagree with the consequences of this philosophy, you can store the 
current HTTP request as a thread-local variable in your application and access 
it from anywhere, for example from a database router.

Since I'm the original author of the transactions API, I've been thinking about 
your suggestion for the last few days. However, I'm not seeing what context 
could meaningfully be passed to `transaction.atomic()` besides the existing 
`using` parameter. Passing the HTTP request object is not an option.

I would recommend running without persistent database connections (CONN_MAX_AGE 
= 0) and switching settings.DATABASE["default"] in a middleware at the 
beginning of every request. Modifying settings at runtime isn't officially 
supported, but this looks like the closest option to the intent you expressed.

Best regards,

-- 
Aymeric.



> On 1 Jun 2021, at 12:09, N Aditya  wrote:
> 
> Hey all, 
> 
> I believe there's a major misunderstanding of what I've been trying to 
> convey. Let me rephrase:
> 
> Problem: Include the request object as a decision param in the routers 
> framework/transaction APIs based on which a DB can be chosen i.e all 
> queries(reads/writes) and transactions are directed to a specific database 
> without having to explicitly mention it. This could be location based, IP 
> based, domain based or based on any other metadata available in the request. 
> 
> Explanation: 
> 1. This use-case is an extension of the examples provided in Django docs for 
> multi-database support wherein based on the model (or) based on the type of 
> operation(read/write) the database is chosen via the routers framework. I 
> need to achieve the same but this time it wouldn't be based on the model or 
> type of operation, rather it would be based on some metadata in the request. 
> 2. The multi-tenancy library which I've built just adds solidity to this 
> use-case. Simply put, it achieves routing based on request headers. 
> 
> The gap: I believe the essence of the routers framework is to provide a hook 
> by which ORM queries can be directed to a DB based on some condition without 
> having to explicitly specify it in every query. A similar hook is missing for 
> all of the transaction APIs. This is the gap I'm trying to bridge. To add 
> further context, in the library that I built, though the routers framework 
> currently doesn't receive the request object, I was able to work around using 
> contextvars but the essence is that the hooks were there. I'm unable to do 
> the same for the transaction APIs without monkey patching since such a hook 
> is unavailable.
> 
> > I don't think this is a strong argument by any mean. To me this is 
> > basically saying "it might be useful in the future so lets add it now" 
> > without providing any details about why it might be useful in the first 
> > place.
> 
> I thought I made it clear but reinstating anyway. I'd like the routers 
> framework to include the request object as part of the arguments list to each 
> of its methods so that decisions made can be based on it as well. If so, 
> there could be a separate method(hook) for transaction APIs as well, like 
> db_for_transaction which can take in the request object to make a decision. 
> It could include something else as well in the future. (Mentioned in 
> message-1 of this thread. Probably I forgot to mention the request object 
> part).
> If the above is difficult to achieve, then we could at the least expose a 
> hook which can be invoked with the request object as its param to make a 
> decision. I believe this should be fairly simple to implement and maintain. 
> (Mentioned in message-3 of this thread).
> 
> If you guys feel that this use-case is still invalid, I'd like to know a 
> logical explanation apart from maintenance overhead issues and waiting for 
> more people to come up with the same problem. 
> 
> 
> 
> Simon, regd the multi-tenancy thing, I've done some extensive research on the 
> libraries available currently to achieve multi-tenancy (including yours) in 
> Django and none of them have adopted the isolated DB approach and declared it 
> production ready. There are multiple libraries using the schema per tenant 
> approach but the downsides are: 
> 
> 1. It is DB implementation specific i.e only DBs' which are using schemas 
> (like PostgreSQL) can take advantage of it. If they're using MongoDB or other 
> data s

Re: Let's simplify coding with html

[Aymeric Augustin]

Hello,

There are several projects providing alternate syntax for generating HTML, 
along the lines of your suggestion, and other projects for integrating these 
with Django. Here's one such project: https://github.com/nyaruka/django-hamlpy 
. I believe we're happy with this 
functionality being provided by third-party projects. As a consequence this 
doesn't seem likely to be accepted for GSoC.

-- 
Aymeric.



> On 21 Mar 2021, at 17:08, manasdeep borole  wrote:
> 
> Hi Everyone,
> I know coding could get tough sometimes(experienced it).
> But thanks to amazing people like you and per built framework like Django.
> At least not everyone has to know everything now.
> The Open Source support is amazing as well.
> Thanks a lot.
> 
> Last time while working with html I was annoyed by the syntax. 
> You have to deal with so many end tags they tend to confuse you.
> eg 
> 
> NOW...
> 
> 
> 
> 
> this is most inner class
> 
> this is quite outer class
> 
> this is most outer class
> 
> 
> With Improvement
> 
> div class='upper':
> div class='middle':
> div class='lower':
> "this is most inner class
>  "this is quite outer class"
> "this most outer class"
> 
> advantages:
> Python like indentation
> simple to read (developer/debugger friendly)
> 50% lowers the chances of mistake while coding (most errors happen because 
> coders forget end tags with html)
> no end tags required
> supports python based logic building like use of for, if
> programers are free to let computer do the repetitive tasks like logic based 
> id for multiple div with similar functionality 
> plans to support direct support for bootstrap( working on it)
> 
> disadvantages:
> needs time and effort to build such a system
> expert support and guidance may be needed
> adoption time and some effort on part of programmers 
> If someone could mentor me I am willing to build on this Idea
> Actually I have built a basic version.
> I understand you would have a lot of requests for GSOC application.
> Can someone guide me if this Idea could be made into a GSOC proposal.
> If you have any suggestion or ideas to make this project better you are most 
> welcome.
> Thank you 
> For your 'time' and 'consideration'
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/024e7422-8242-497c-9a32-f5b67b4fcae3n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/F7ED4035-51FD-4F64-AB3C-791C57C83B8F%40polytechnique.org.

Re: Proposal: Drop dependency on pytz in favor of zoneinfo

[Aymeric Augustin]

Hi Carlton,

> IIUC-at-first-pass: your suggestion differs in jumping straight to the 
> end-point with a fallback for those who need it. (In contrast to getting 
> folks to opt-in early if they want it.)


Indeed. The majority will switch when the default changes; better earlier than 
later.

> what did you think about allowing an early opt-in for Django v3.2?


The more options we create, the more confusion we may generate e.g. when users 
try to understand release notes or to ask for support. This is why I tried to 
keep options to a minimum (1. initial situation, 2. target situation, 3. with 
shim) and to make as few users as possible go through option 3.

Implementing USE_PYTZ_DEPRECATION_SHIM is Django 3.2 makes the situation a bit 
more complex, as the setting will mean "use pds instead of pytz" in 3.2 and 
"use pds instead of zoneinfo" in 4.0. We'd have four options instead of three.

I'm pretty confident that the transition will go smoothly and I don't think the 
flaws in pytz are bad enough to warrant urgent action. As a consequence, 
offering opt-in in Django 3.2 doesn't seem all that valuable to me.

I'm OK with starting the migration in one year in Django 4.0. I'm also OK with 
offering the opt-in in Django 3.2 if others have a different value assessment!

Cheers,

-- 
Aymeric.



> On 3 Jan 2021, at 09:12, Carlton Gibson  wrote:
> 
> Hi Aymeric. 
> 
> Thanks for inputting! 
> 
> I need to read in-depth what you’ve said here, but IIUC-at-first-pass: your 
> suggestion differs in jumping straight to the end-point with a fallback for 
> those who need it. (In contrast to getting folks to opt-in early if they want 
> it.) This sounds better if we can. 
> 
> The Autumn hasn’t allowed time to make progress here but, what did you think 
> about allowing an early opt-in for Django v3.2? (I’m not sure we have 
> capacity to get the in in-time now anyway, so it may be moot.)
> 
> Kind regards, 
> Carlton.
> 
> 
> 
> On Sat, 2 Jan 2021 at 10:29, Aymeric Augustin 
>  <mailto:aymeric.augus...@polytechnique.org>> wrote:
> Hello,
> 
> As the original author of support for timezone aware datetimes in Django, 
> I've been meaning to review this... for six months... Better late than never 
> I guess?
> 
> In this discussion, we're assuming settings.USE_TZ = True.
> 
> 
> The original design 
> <https://groups.google.com/g/django-developers/c/zwQju7hbG78>, which still 
> matches 80% of the current implementation, was very much based on advice 
> found in the documentation of pytz. This has three consequences:
> 
> 1. By default, users manipulate aware datetimes in UTC, which limits the 
> breakage to cases where they explicitly switch to another timezone. This is 
> auditable e.g. "look for timezone.activate or timezone.override in your code".
> 
> 2. Django provides custom wrappers in order to take care of the pitfalls of 
> pytz automatically e.g. timezone.make_aware. Backwards-compatibility can be 
> implemented transparently there.
> 
> 3. A strategy designed for migrating from pytz to zoneinfo should be 
> applicable to Django.
> 
> 
> I'm seeing three areas that need care:
> 
> A. APIs returning a tzinfo object, currently a pytz timezone (other than UTC 
> — we switched from Django's custom definition of the UTC tzinfo to pytz' 
> definition in the past without trouble).
> 
> B. APIs returning aware datetimes, currently in a pytz timezone (other than 
> UTC).
> 
> C. APIs performing datetime conversions in the database, which is typically 
> used for aggregating by day in a given timezone. This depends on the timezone 
> name. I think we're fine on this front since we're keeping the same timezone 
> names.
> 
> So the primary concern is leaking pytz tzinfo objects (either directly, or 
> via aware datetime objects), to user code that requested it explicitly. I may 
> sound like I'm belaboring the point. However, I think we can make a better 
> backwards-compatibility decision with an accurate estimate of the extent of 
> the breakage.
> 
> 
> Django currently references pytz in the following places:
> 
> - timezone.get_default/current_timezone => see case A above.
> - timezone.get_default/current_timezone_name => shouldn't be an issue since 
> time zone names don't change.
> - BaseDatabaseWrapper.timezone => for timezone conversions in Python code via 
> make_aware / make_naive; see case B above.
> - BaseDatabaseWrapper.timezone_name => for timezone conversions in the 
> database via database-specific SQL functions; see case C above.
> 
> With SQLite, the SQL conversion functions are implemented in Python and use 

Re: Proposal: Drop dependency on pytz in favor of zoneinfo

[Aymeric Augustin]

Hello,

As the original author of support for timezone aware datetimes in Django, I've 
been meaning to review this... for six months... Better late than never I guess?

In this discussion, we're assuming settings.USE_TZ = True.


The original design 
, which still 
matches 80% of the current implementation, was very much based on advice found 
in the documentation of pytz. This has three consequences:

1. By default, users manipulate aware datetimes in UTC, which limits the 
breakage to cases where they explicitly switch to another timezone. This is 
auditable e.g. "look for timezone.activate or timezone.override in your code".

2. Django provides custom wrappers in order to take care of the pitfalls of 
pytz automatically e.g. timezone.make_aware. Backwards-compatibility can be 
implemented transparently there.

3. A strategy designed for migrating from pytz to zoneinfo should be applicable 
to Django.


I'm seeing three areas that need care:

A. APIs returning a tzinfo object, currently a pytz timezone (other than UTC — 
we switched from Django's custom definition of the UTC tzinfo to pytz' 
definition in the past without trouble).

B. APIs returning aware datetimes, currently in a pytz timezone (other than 
UTC).

C. APIs performing datetime conversions in the database, which is typically 
used for aggregating by day in a given timezone. This depends on the timezone 
name. I think we're fine on this front since we're keeping the same timezone 
names.

So the primary concern is leaking pytz tzinfo objects (either directly, or via 
aware datetime objects), to user code that requested it explicitly. I may sound 
like I'm belaboring the point. However, I think we can make a better 
backwards-compatibility decision with an accurate estimate of the extent of the 
breakage.


Django currently references pytz in the following places:

- timezone.get_default/current_timezone => see case A above.
- timezone.get_default/current_timezone_name => shouldn't be an issue since 
time zone names don't change.
- BaseDatabaseWrapper.timezone => for timezone conversions in Python code via 
make_aware / make_naive; see case B above.
- BaseDatabaseWrapper.timezone_name => for timezone conversions in the database 
via database-specific SQL functions; see case C above.

With SQLite, the SQL conversion functions are implemented in Python and use 
pytz, but the end result is the same.


My suggestion would be:

- Switch everyone to zoneinfo (or backports.zoneinfo) by default in 4.0
- Provide a temporary, immediately deprecated setting USE_PYTZ_DEPRECATION_SHIM 
in 4.0 and remove it in 5.0

Why?

- Most users don't use timezone.activate, timezone.override, or the 
DATABASES.TIME_ZONE setting to get aware datetimes in a timezone other than UTC 
for the purpose of doing datetime arithmetic in local time. Let's keep the 
upgrade instructions simple for the majority of users! "If you don't do 
anything fancy with timezones, you don't need to worry about this!"

- But some users need a more gradual path, especially those with large 
codebases. Conscious use of pytz_deprecation_shim looks like the best plan for 
them. Since it isn't easy to monkey-patch, let's instruct them to set 
USE_PYTZ_DEPRECATION_SHIM = True so they can upgrade now and fix deprecation 
warning later. This would be Paul's pull request 
, but conditional on the setting.

I considered using "can we import pytz_deprecation_shim?" as a signal instead 
of "is USE_PYTZ_DEPRECATION_SHIM set?" but this wouldn't work well if 
pytz_deprecation_shim becomes an indirect dependency. Then shims could get 
accidentally activated and the user wouldn't have a good solution.


I hope this helps!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1AA84619-74BA-4A7D-A5A8-DC7210885BE9%40polytechnique.org.

Re: Ticket #21289 - Login rate limiting

[Aymeric Augustin]

Hello,

Having some basic throttling built-in would be an improvement for the vast 
majority of websites. Also it would plug one of the big holes in 
django.contrib.auth (another big one being 2FA).

Some DoS concerns were expressed on the pull request. I believe the options are:

1. use a global form validation error — this is my preference;

2. return a HTTP 429 — this requires making the protection very conservative 
(i.e. users should never see it in normal circumstances, like CSRF errors) or 
adding a new error page (which every website then has to style).

Furthermore, I think:

- We should consult with the authors of existing libraries in this space to 
check if the simple mechanism we're considering don't have downsides we missed 
and they know;

- We should focus this on usernames and ignore IP addresses, as most sites are 
behind a reverse proxy of some kind and no one handles X-Forwarded-For headers 
right (even Heroku doesn't care — when I reported they were vulnerable to XFF 
injection, their security team [or, more accurately, their subcontractors] 
didn't understand the report, even after several rounds of explanation and a 
working proof of concept)

- We must make sure this doesn't get in the way of users with more advanced 
needs. Currently the feature can be disabled with DELAY_AFTER_FAILED_LOGIN = 0, 
which seems fine.

Best regards,

-- 
Aymeric.



> On 27 Jul 2020, at 20:21, Claude Paroz  wrote:
> 
> Hi Adam,
> 
> Le lundi 27 juillet 2020 16:55:35 UTC+2, Adam Johnson a écrit :
> Hi Claude,
> 
> A delay of 5 seconds seems quite long. Often I fail to log into a site due to 
> mis-selection of credentials from my password manager, so I can resubmit a 
> login form within 1-2 seconds.
> 
> That 5-secs choice is debatable, we can agree on a shorter delay. We may also 
> wait for the second or third failure before throttling (suggested on the PR).
>  
> A real rate-limiting solution has the advantage of buckets of requests per 
> time period, allowing users a few rapid attempts before being locked.
> 
> Additionally, the default PBKDF2 hasher already enforces a (smaller) 
> arbitrary delay via its algorithm iterations. I can't find a source but I 
> think I remember reading it should be tuned to take about 100ms. This is 
> about 1.5 orders of magnitude less than a 5 second delay, which is perhaps 
> not so significant in terms of password brute-forcing (less difference than 
> one extra password character). Not sure if 100ms is where Django's current 
> default ends up on a modern CPU, but we probably aren't far off given we 
> increase the iterations according to a formula that roughly tracks Moore's 
> law.
> 
> I'd rather see something like django-ratelimit merged to core. It's more 
> general purpose so users can reuse and customize it, and we could potentially 
> use it for other features in Django too.
> 
> Sure, that might be the better long-term solution. However, I'm afraid this 
> will be delayed forever... (note the ticket is already 7 years old). In my 
> opinion, a "battery-included" framework like Django should include at least a 
> basic brute force protection. That's why I'd like to push some minimal 
> mitigation for Django 3.2, then we can always add a more elaborate tooling 
> set later.
> 
> Claude
>  
> On Mon, 27 Jul 2020 at 12:13, Claude Paroz > wrote:
> Hi all,
> 
> I thought a bit about login rate limiting again in recent times.
> https://code.djangoproject.com/ticket/21289 
> 
> 
> We know that there are some packages (django-ratelimit, django-defender,
> etc.) that can do the job, but the main issue here is to provide a
> *default* behavior for any fresh new Django project.
> 
> A must-read on this subject is:
> https://owasp.org/www-community/controls/Blocking_Brute_Force_Attacks 
> 
> 
> I would like to suggest one mitigation measure for default Django, which
> seems to me the least controversial, considering that hard-locking by
> username and/or ip address can open Denial of Service vectors which may
> or may not be acceptable for some sites.
> 
> My suggestion is to add a time delay of 5 seconds in the
> contrib.auth.forms.AuthenticationForm after the first failure on any
> username. This choice of 5 seconds is a compromise between not too much
> annoying users after a failed login attempt, and still set a significant
> throttling limit for some brute force attacks. You can consider that
> after a failed login, a real user will spend at least 2-3 seconds just
> to re-enter a new password and re-submit the form, so the real wait
> penalty should not be more than 2-3 seconds.
> 
> This is of course NOT the panacea against all type of brute force
> attacks, as you can read on the OWASP article above. But it appears to
> me as a reasonable measure that can be widely accepted by most Django
> projects that use the default authentication form.
> 
> 

Re: Should the docs suggest namespace packages?

[Aymeric Augustin]

I was still planning to revert ccc25bf. It's just a matter of finding time.

-- 
Aymeric.


> On 19 Jul 2020, at 22:25, Tim Graham  wrote:
> 
> On the topic of namespace packages, I noticed this line in the 3.1 release 
> notes:  "Migrations are now loaded also from directories without __init__.py 
> files." https://code.djangoproject.com/ticket/30300 
> <https://code.djangoproject.com/ticket/30300>
> 
> The ticket's rationale seems to be "We've just finished migrating our 
> codebase to all-python 3, and nuked all our __init__py files. We are now 
> upgrading to Django 2.1, and noticing the various places where they're still 
> de-facto required by Django. This particular case seemed unnecessary and easy 
> to fix."
> 
> Another commenter remarked, "We had a similar issue with test discovery. One 
> of our developers read an article that __init__.py files are not required on 
> Python3 and started removing them. Everything seemingly worked but some tests 
> were not discovered and did not run in CI (which made it difficult to spot). 
> I think Django should not required them if technically possible or at least 
> it should be made clear in the documentation (sorry if I overlooked it)."
> 
> Supporting namespace packages without a real use case seems contrary to the 
> consensus in this thread (which I see as not promoting implicit namespace 
> packages). Based on that consensus, my inclination wouldn't be to try to make 
> Django work with as few __init__.py files as possible. What do you think?
> 
> 
> On Thursday, May 21, 2020 at 7:52:49 AM UTC-4 Adam Johnson wrote:
> +1 for reverting ccc25bf .
> 
> On Wed, 20 May 2020 at 19:54, Aymeric Augustin  > wrote:
> Hello,
> 
> Commit ccc25bf refers to ticket #23919 in the commit message. In that ticket, 
> I argued that the __init__.py files should be kept: 
> https://code.djangoproject.com/ticket/23919#comment:102 
> <https://code.djangoproject.com/ticket/23919#comment:102>. No one brought a 
> counter argument.
> 
> It's weird that the __init__.py files were removed anyway without further 
> discussion (that I can find with my browser's search on that page, at least).
> 
> It's fairly minor, but I think thatccc25bf should be reverted.
> 
> Best regards,
> 
> -- 
> Aymeric.
> 
> 
> 
>> On 20 May 2020, at 14:58, René Fleschenberg > > wrote:
>> 
>> On Github, Tim pointed out that with
>> https://github.com/django/django/commit/ccc25bfe4f0964a00df3af6f91c2d9e20159a0c2
>>  
>> <https://github.com/django/django/commit/ccc25bfe4f0964a00df3af6f91c2d9e20159a0c2>,
>> various __init__.py files in Django's own code were removed.
>> 
>> I tend to say it would be good to revert that change. Or we could update
>> the docs only, since they serve much more directly as an example for
>> user code.
>> 
>> -- 
>> René Fleschenberg
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/5d689d5c-01ab-d017-249c-a53a6d9b1f87%40fleschenberg.net
>>  
>> <https://groups.google.com/d/msgid/django-developers/5d689d5c-01ab-d017-249c-a53a6d9b1f87%40fleschenberg.net>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/C9E429E5-DC4C-4C47-8126-17B32D31D245%40polytechnique.org
>  
> <https://groups.google.com/d/msgid/django-developers/C9E429E5-DC4C-4C47-8126-17B32D31D245%40polytechnique.org?utm_medium=email&utm_source=footer>.
> 
> 
> -- 
> Adam
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/d725cf0c-b0bd-4fe0-83b0-7315d0c742ben%40googleg

Re: Custom collation support

[Aymeric Augustin]

Hello Tom,

Just wanted to give you some encouragement here, as you've been monologuing for 
two months ;-) The PR looks promising!

Regarding migrations, I'm not seeing how the collation for a column will be set 
or unset in the database if you change it in the code e.g.:

MySQL : ALTER TABLE ... MODIFY ... VARCHAR(...) COLLATE ...
PostgreSQL : ALTER TABLE ... ALTER COLUMN ... TYPE varchar(...) COLLATE ...

I think this needs to be in scope, else it will be hard for maintainers of 
existing projects to take advantage of this new feature.

-- 
Aymeric.



> On 18 Jul 2020, at 13:29, Tom Carrick  wrote:
> 
> I've written a proof of concept: https://github.com/django/django/pull/13207 
> 
> 
> The diff is quite small, though I'm not sure if there's something I'm doing 
> wrong - this is my first foray into schema migration internals so I'm 
> learning as I read the codebase.
> 
> Tom
> 
> On Fri, 17 Jul 2020 at 13:55, Tom Carrick  > wrote:
> I've had a deeper look at this now and think I have an API proposal. First, 
> the state of supported vendors:
> 
> 1. All vendors support adding a collation to text/varchar fields.
> 2. The syntax is more or less the same.
> 3. However, the collation names themselves are different.
> 4. PostgreSQL is the only vendor that allows creating custom collations at 
> runtime.
> 
> So I'm thinking we add a new `collation` parameter to `CharField` and 
> `TextField`, that simply takes a string of the collation ID. I'm not quite 
> sure on the implementation as I don't know the ORM that well, but my naive 
> approach would be to just add a new format string to the `data_types` dict 
> that is calculated during the field __init__(), either an empty string to use 
> the default collation, or e.g. ' collate '. There's may well be 
> a better approach.
> 
> By using this - because the collation names are not the same across vendors - 
> the user is saying "I'm okay with this only working on one database vendor", 
> so there should be a warning in the docs. There is perhaps some scope in the 
> future to make this take a callable that can figure out the collation 
> per-database. This would be useful for getting case-insensitive lookups 
> working across all backends, for example. But I want to keep that out of the 
> scope because it's some extra work and I'm not sure on the implementation.
> 
> Another downside is that people like to use CharField as a base class for 
> other column types that might not support collations, but I think this should 
> be in the user's hands to make sure they aren't doing that.
> 
> We should also add a `CreateCollation` operation for Postgres, similar to the 
> `CreateExtension` operation that currently exists. If the user wants to use a 
> custom collation they must create it first, similar to using extensions 
> currently.
> 
> The advantage to this is that users can use collations without having to make 
> SQL migrations, which I think would be nice. The really nice thing is the 
> ability to have case-insensitive lookups that work across all database 
> vendors, rather than only Postgres as it currently is. And as I mentioned in 
> the previous message, Postgres is discouraging our current method of using 
> the citext extension in favour of this approach.
> 
> Cheers,
> Tom
> 
> On Wed, 27 May 2020 at 14:17, Tom Carrick  > wrote:
> I think it would be useful to be able to create collations and use them with 
> model fields. The motivation here is mostly that citext is somewhat 
> discouraged  in favour of 
> creating a collation. I'm not sure how this would work on other databases,or 
> what the API would look like. I didn't see a ticket for it or another 
> discussion, but maybe there is one.
> 
> Perhaps a Collation class would make sense, and it could be added to a Field 
> with a new parameter. I'm not sure how easy it would be to only create a 
> collation once, so perhaps it would need a CreateCollation migration as well, 
> but I know very little about the internals of migrations.
> 
> Cheers,
> Tom
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAHoz%3DMbyJ1qLNCxmTh6JdsP-e1UoYwdWJm8KL3q8MgRAQaD5sA%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" grou

Re: Django Query optimizing performance (HUGE DATA)

[Aymeric Augustin]

Hello,

I think you've found the wrong mailing list for this post. This mailing list is 
for discussing the development of Django itself, not for support using Django. 
This means the discussions of bugs and features in Django itself, rather than 
in your code using it. People on this list are unlikely to answer your support 
query with their limited time and energy.

For support, please follow the "Getting Help" page: 
https://docs.djangoproject.com/en/3.0/faq/help/ 
 . This will help you find 
people who are willing to support you, and to ask your question in a way that 
makes it easy for them to answer.

Thanks for your understanding!

-- 
Aymeric.



> On 19 Jul 2020, at 05:38, karthik challa  wrote:
> 
> Hi Experts,
> 
> I am trying to execute the below query and the query is taking more than 5 
> minutes.
> 
> Here are the details
> 
> Model.py
> 
> class Url(models.Model):
>  subdomain = models.ForeignKey(Subdomain, null=True, blank=True, 
> related_name='url_subdomain', on_delete=models.SET_NULL,db_index=True)
>  full_url = models.CharField(max_length=1000, unique=True, db_index=True)
>  class Meta:
>  ordering = ['full_url']
> 
>  def __str__(self):
>  return self.full_url
> 
> 1>Django query ORM 
> 
> subdomains = 
> Subdomain.objects.all().annotate(numItems=Count('url_subdomain')).order_by('name')
> 
> 2>SQL Query 
> 
> SELECT "urls_subdomain"."id", "urls_subdomain"."created_at", 
> "urls_subdomain"."name", COUNT("urls_url"."id") AS "numItems" 
>  FROM "urls_subdomain" LEFT OUTER JOIN 
> "urls_url" ON ("urls_subdomain"."id" = "urls_url"."subdomain_id") GROUP BY 
> "urls_subdomain"."id" ORDER BY "urls_subdomain"."name" ASC
> 
> 3>Templates
> 
> {% if subdomains %}
>  
>  Subdomain
>  
>  {% for item in subdomains %}
>  item
>  {% endfor %}
>  
>  
> 
>  {% endif %}
> 
> Please let me know how can i optimize the preformance
> 
> Thanks & Regards,
> Karthik
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/996db826-fe18-4557-977b-58493c1f9481o%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CB478AA7-1F00-4E93-BAD6-423C78E702FA%40polytechnique.org.

Re: The blacklist / master issue

[Aymeric Augustin]

Hello,

In the context of access control, blacklist / whitelist makes sense only if the 
reader has a preconceived assumption that black = bad, illegal, forbidden / 
white = good, legal, authorized. You can probably see where I'm going.

Sure, blacklist / whitelist has nothing to do with race to start with, but I 
find the parallel with Apartheid sufficiently obvious to make it embarrassing, 
certainly because I'm not a native English speaker and I don't have enough 
background on what has racial overtones and what doesn't.

I mean, I had been living in the US for several months whet someone had to tell 
me the difference between "to screw" and "to screw up". (I'm grateful.) Do you 
really expect a guy like me to know that "blackface" has racial overtones but 
"blacklist" doesn't, and thus interpret the words correctly?

Besides, the connection didn't exist in the first place, but when people start 
making it, can we still pretend it doesn't exist? If I wanted to troll a 
linguist, I'd say it's akin to pretending that words people actually use don't 
exist until they're written in a dictionary ;-)

Lastly, another argument for the statu quo is that humans are good at 
interpreting words based on context, so "black" in "blacklist" isn't a problem. 
However, I counter that humans are even better at making connections and 
detecting patterns, even subconsciously and sometimes even when the pattern 
doesn't actually exist. That's quite likely to happen here.

I agree that this isn't as clear cut as master / slave. That must be why it 
took us six years to go from the master / slave discussion to the blacklist / 
whitelist discussion.

No one's gonna get confused on the meaning regardless of whether we make the 
change or not. This is "just" a political marker. It doesn't have one correct 
answer. It has several answers whose correctness vary over time.

I think we'll make the change at some point. Some progressives will hate us for 
taking so much time. Some conservatives will hate us for being snowflakes. 
Since we already started spending time on this discussion, we might just as 
well do the change while we're there, take some flak for a couple days, and 
move on.

Best regards,

-- 
Aymeric.

> On 15 Jun 2020, at 21:56, Daniele Procida  wrote:
> 
> Tom Carrick wrote:
> 
>> I don't think there is an easy answer here, and I open this can of worms
>> somewhat reluctantly. I do think Luke is correct that we should be
>> concerned with our credibility if we wrongly change this, but I'm also
>> worried about our credibility if we don't.
> 
> There are plenty of black-something terms in English that are both negative 
> and have nothing whatsoever to do with race. The black and the dark are those 
> things that are hidden and sinister, as contrasted with those that are in the 
> light and open to scrutiny (black magic, dark arts, black legs, blackguards, 
> blackmail, etc).
> 
> I think it would look pretty silly to confuse meanings that refer to what's 
> shadowy and obscure with things that have racial overtones, and I think we 
> should steer well clear of that. It's not at all like metaphors such as 
> master/slave. 
> 
> If we made such a change and tried to justify it on the grounds of a 
> connection between race and the word "black" in those terms, we'd be rightly 
> laughed at.
> 
> 1000 neckbeards would immediately come out of the woodwork having done some 
> basic web searches going 'neeer neeer neeer, the Django Software Foundation 
> overflowing with snowflakes who think that "blacklist" means [etc etc etc]', 
> and who has the stomach for that? 
> 
> Even choosing to do it on the basis of the potential for offence seems to be 
> a fairly flimsy argument.
> 
> On the other hand, we can do whatever the hell we like. 
> 
> We don't have to justify anything to anyone. If we want to change words in 
> *our* framework, it's absolutely nobody's business but our own.
> 
> If black members of the DSF or the community are disheartened that the word 
> "black" gets to refer to so many negative things and are bothered when they 
> see them in Django, then that alone is sufficient justification. 
> 
> If we want a reason for changing "blacklist" (or whatever), it's that people 
> in our community said they would feel better about it and asked to have it 
> changed. 
> 
> Acknowledging how someone feels about something and acting because you care 
> about their feelings seems to be a respectful thing to do.
> 
> "We did it because we felt like it" is an utterly unanswerable justification.
> 
> The DSF has credibility because the software is first rate, the foundation is 
> well-governed and the community is an international example of decency and 
> kindness. Things like this become credible because the DSF chooses to do them 
> - it's not the other way round.
> 
> Daniele
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" 

Re: ./manage.py settings

[Aymeric Augustin]

Django already attempt to mask security-sensitive settings in the debug page.

It would make sense to do the same by default for the output of diffsetting: 
obfuscate secrets.

A --include-secrets flag (on any other name) would  show the secrets.

-- 
Aymeric.



> On 12 Jun 2020, at 13:59, Jure Erznožnik  wrote:
> 
> There was a GSoC proposal 
> 
>  this year for a SecretsManager that could conceivably make what you suggest 
> possible. Wasn't accepted though.
> 
> Not sure if it's a security issue with beginners that require help setting up 
> settings.py though. Should be easy enough to just paste the file (modified) 
> somewhere.
> 
> LP,
> Jure
> 
> On 12/06/2020 13:51, René Fleschenberg wrote:
>> Hi,
>> 
>> Just a thought that came to my mind: It would be useful to have a
>> command that dumps the run-time settings, but automatically replaces
>> secrets with dummy values.
>> 
>> I think this should not be too hard to do for Django's own secret
>> settings, and maybe it can also be done for some known common third
>> party settings (AWS_SECRET_ACCESS_KEY, for example).
>> 
>> Even better would be to have a standardized way of marking settings as
>> secret, though I am not sure if this is feasible.
>> 
>> Regards,
>> René
>> 
>> On 6/12/20 5:09 AM, '1337 Shadow Hacker' via Django developers
>> (Contributions to Django itself) wrote:
>>> Hi all,
>>> 
>>> So, just on #django IRC channel there was a user trying to help another
>>> one, asking for some settings through ./manage.py shell etc ... A
>>> discussion that went kind of like "Print out your settings" "How would I
>>> print, I tried that, I'm in settings.py" "With ... print()" "but in the
>>> shell, __file__ is not defined" and so on, and 20 minutes later the user
>>> couldn't print his settings and left.
>>> 
>>> TBH I'm pretty fine because the users I support are on projects where I
>>> added djcli to the requirements, so I can always ask the to do the djcli
>>> setting command to print out a setting. It's very useful useful to me, I
>>> think Django should a command to dump runtime settings because that
>>> should make the life easier of Django users trying to support newbies in
>>> their own projects, where they don't have installed a custom command,
>>> it's pretty cheap to make and maintain and should save quite some
>>> frustration from both sides of the story.
>>> 
>>> What do you think ?
>>> 
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-developers+unsubscr...@googlegroups.com 
>>> 
>>>  
>>> .
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/X2HkH0yasxu2XVxBuVkbTfdSHctL7y3pW9seQ8CHFcVOpmsiyN80JlDYH2g5F6IfPvEdN4zyG6vuxj2HEMJaXUSErMUM5IT70iLvkxsrc7U%3D%40protonmail.com
>>>  
>>> 
>>> 
>>>  
>>> .
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/4370f935-9d2d-283d-3156-14c878fc1c93%40gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A8423AE2-2149-4948-88DC-59E503B6F235%40polytechnique.org.

Re: Implement QuerySet.__contains__?

[Aymeric Augustin]

> If people agree QuerySet.contains() should be added, how do we move forward?

Yes, if there's no major new argument in a couple days, we can assume consensus 
on adding .contains().

* Create a ticket in Trac. Point to this discussion and to the previous ticket 
where .contains() was suggested.
* Submit a pull request in GitHub with code, tests, docs. This is worth 
mentioning in the changelog.

There's a lot of details in the docs, in the Contributing Guide, about how to 
do allD this right.

-- 
Aymeric.



> On 5 Jun 2020, at 13:58, Johan Schiff  wrote:
> 
> Shai does make a good point about changing a well documented behaviour. That 
> argument wins me over.
> 
> Adding .contains() and updating the documentation goes a long way to make it 
> easier for developers.
> 
> Best case would be that Dajngo does not make assumptions about what the 
> developer wants, but to implement __len__(), __bool__() and __contains__(), 
> we have to assume one method or the other. I still think the wrong call was 
> made here, but changing it now is too much pain. (Assuming queryset should be 
> evaluated is probably correct in most cases, but sometimes adds a big 
> performance hit when the code goes into production and the dataset grows - 
> code that looks reasonable and works like a charm in dev will cause trouble 
> in production. Assuming the opposite would have added a tiny performance hit 
> in most cases, but avoided the big one.)
> 
> Since I'm new here:
> If people agree QuerySet.contains() should be added, how do we move forward?
> I'd be willing to write some code with tests and add a ticket, if that's 
> helpful.
> 
> ‪Le ven. 5 juin 2020 à 11:42, ‫אורי‬‎  > a écrit :‬
> Hi,
> 
> I'm thinking, maybe instead of:
> 
> if obj in queryset:
> 
> Users should write:
> 
> if obj in list(queryset):
> 
> Which I understand is already working? Why does the first line (without 
> list()) should work anyway?
> 
> And if users want performance, why not write:
> 
> if queryset.filter(pk=obj.pk ).exists():
> אורי
> u...@speedy.net 
> 
> On Tue, Jun 2, 2020 at 11:57 AM Johan Schiff  > wrote:
> Is there a specific reason Djangos QuerySet does not implement __contains__? 
> It doesn't seem very complicated to me, but maybe I'm missing something.
> 
> When checking if an object is in e queryset I always use code lite this:
> if queryset.filter(pk=obj.pk).exists():
> 
> The pythonic way would be:
> if obj in queryset:
> 
> The way I understand it this latter version will fetch all objects from 
> queryset, while the first one is a much leaner db query.
> 
> So, is there a reason QuerySet doesn't have a __contains__ method, or would 
> it be a good addition? My guess is that a lot of people use the "obj in 
> queryset" syntax, believing it does "the right thing".
> 
> //Johan
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/88c83b8d-658c-47cc-9162-fcfebebe9c4a%40googlegroups.com
>  
> .
> 
> -- 
> You received this message because you are subscribed to a topic in the Google 
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/django-developers/NZaMq9BALrs/unsubscribe 
> .
> To unsubscribe from this group and all its topics, send an email to 
> django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABD5YeE7_ORUfJcnt6f4aB4J0j-%3DyDK_BowEt_fefcaFMGdB1g%40mail.gmail.com
>  
> .
> 
> 
> -- 
> Vänligen, Johan Schiff
> Radkompaniet AB
> 072-229 61 19
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAD69rUBKOse%2Be2RedPT%2B-KWkcdC%3DiaG0zyt1Hakefp6gQ7vdLA%40mail

Re: What happens if a Fellow has a holiday?

[Aymeric Augustin]

Hello,

It would be best if fellows took 5 weeks off per year — or whatever is common 
in their country.

Our usual strategy for tackling fragile processes is to see what breaks then 
figure out a solution.

Enjoy your holidays. We'll learn how much we depend on you :-) 

-- 
Aymeric.



> On 4 Jun 2020, at 12:47, Carlton Gibson  wrote:
> 
> Hi all. 
> 
> Short answer is we don't really know, since we never take holidays. 🙂
> 
> I though am having a much needed week off from tomorrow, to be back on the 
> 16th. 
> 
> Mariusz will need assistance as available Approving changes, from Claude (our 
> third merger) and the Technical Board, who can also Approve PRs. 
> Mostly that's not needed for contributor PRs, but Mariusz makes a lot of his 
> own PRs as he's working, and those do need approval.
> Both these points by DEP 10.
> 
> To facilitate this I have added the Django Technical Board GitHub team to the 
> django/django repo, with the non-write "Triage" role. 
> In this context, mostly this allows Mariusz to ask the TB by team handle for 
> a review. 
> (I trust this is OK.) 
> 
> 3.1b1 is due on the 15th. We can delay that but... 
> 
> Please can I ask the 6 of you to be aware that you might be needed. 
> Thanks. 
> 
> 
> Kind Regards,
> 
> Carlton
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/0c3b30db-4431-4fb1-ae5c-da87fe28a17d%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/459789D2-BB6F-4CF6-B63C-6204A922F4FE%40polytechnique.org.

Re: Implement QuerySet.__contains__?

[Aymeric Augustin]

> On 2 Jun 2020, at 19:42, Tim Graham  wrote:
> 
> And here's some past discussion:
> https://code.djangoproject.com/ticket/24141 - contains() method for QuerySets 
> (closed as needsinfo due to no mailing list discussion to find a consensus)
> https://github.com/django/django/pull/3906 - Efficient QuerySet.__contains__ 
> (closed as wontfix due to the behavior change)

Thanks. Your memory is amazing :-)

Since the issue keeps coming back, I think we should do one of the following:

1. make simultaneous and consistent changes to __nonzero__, __len__, and 
__contains__ to eliminate the "fetch the whole table" behavior when calling `if 
qs`,  `len(qs)` and `if obj in qs` — and check if any other special methods 
require similar treatment
2. failing that, add .contains(), as suggested in 
https://code.djangoproject.com/ticket/24141

We're talking about a trade-off between preserving optimisations in existing 
code bases and expertise of advanced users versus doing the right thing by 
default for less experienced users. I think we're increasingly willing to 
consider changes that make Django safer to use, so perhaps what Carl said in 
2015 could be open for discussion. If I forget what I know about querysets for 
a minute, the Principle of Least Astonishment says we should go for option 1.

if qs:  # test in qs isn't empty
if list(qs):  # fetch and cache qs, then test if it isn't empty
len(qs): # count the number of objects in qs
len(list(qs)):  # fetch and cache qs, then count the number of objects
if obj in qs:  # test if qs contains obj
if obj in list(qs):  # fetch and cache qs, then test if qs contains obj

objects = list(qs) is a well known pattern when you need to trigger the 
database query, perhaps because you'll do further manipulations in Python.

In https://code.djangoproject.com/ticket/18702#comment:8 
 Anssi says he was in 
favor of committing the optimized version (my option 1). Just before finalizing 
his patch, he changed his mind because:

- "This approach optimizes use cases that are rare": well, since they keep 
coming up, in fact they aren't so rare
- "Special casing some uses makes the code complex": sure, but the whole point 
of Django is to include _some_ complexity so that users have less to worry 
about.

I'm interested in what others think :-)

-- 
Aymeric.




-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/90C68EF9-7343-44D6-9BB0-33972198685A%40polytechnique.org.

Re: Implement QuerySet.__contains__?

[Aymeric Augustin]

> On 2 Jun 2020, at 13:30, Florian Apolloner  wrote:
> 
> On Tuesday, June 2, 2020 at 11:28:34 AM UTC+2, Adam Johnson wrote:
> If you already fetched the queryset, `if obj in queryset` will make a new 
> database query
> 
> I don't see why we couldn't implement __contains__ to do a walk through 
> _result_cache if it has been fetched?
> 
> we are doing the same for len() IIRC. I have to say it sometimes makes it a 
> little bit hard to debug, but if it is documented as such it should be okay.

These are good points.

As far as I can tell, the current implementation of __len__ is to fetch the 
whole queryset:

def __len__(self):
self._fetch_all()
return len(self._result_cache)

I'm almost sure doing an automatic .count() was discussed in the past. I don't 
remember why we rejected it — perhaps because in most cases, when you need the 
length of a queryset, you will also iterate it?

The suggestion for __contains__ is similar: doing an automatic 
.filter(pk=obj.pk).exists(). We could do it only if the queryset isn't fetched 
yet, but I'm afraid this may be too magical. Is it reasonable to have `total = 
len(queryset); found = obj in queryset` make one SQL query while `found = obj 
in queryset; total = len(queryset)` makes two queries?

Now I'm worried that we may be making inconsistent decisions between __len__ 
and __contains__... Let's make sure we're consistent across dunder methods.

-- 
Aymeric


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9B25C8F3-A282-4C7F-BB4C-252364514A03%40polytechnique.org.

Re: Implement QuerySet.__contains__?

[Aymeric Augustin]

Hello Johan,

You explained the upside. There's one downside to be aware of. If you already 
fetched the queryset, `if obj in queryset` will make a new database query, 
which will be slower than walking through the already fetched data (unless the 
queryset is really large and the database really close in terms of network 
latency). Making this change may throw off some assertNumQueries tests. 
However, people who consider this a regression can do `obj in list(queryset)` 
or perhaps `obj in iter(queryset)` to reuse the cached list.

Judgment call: I think the upside is worth the downside and 
backwards-incompatibility, so I'm in favor of the change.

Best regards,

-- 
Aymeric.



> On 2 Jun 2020, at 10:57, Johan Schiff  wrote:
> 
> Is there a specific reason Djangos QuerySet does not implement __contains__? 
> It doesn't seem very complicated to me, but maybe I'm missing something.
> 
> When checking if an object is in e queryset I always use code lite this:
> if queryset.filter(pk=obj.pk).exists():
> 
> The pythonic way would be:
> if obj in queryset:
> 
> The way I understand it this latter version will fetch all objects from 
> queryset, while the first one is a much leaner db query.
> 
> So, is there a reason QuerySet doesn't have a __contains__ method, or would 
> it be a good addition? My guess is that a lot of people use the "obj in 
> queryset" syntax, believing it does "the right thing".
> 
> //Johan
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/88c83b8d-658c-47cc-9162-fcfebebe9c4a%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/570372E7-2949-426C-BAAE-DBEF1B17FBC0%40polytechnique.org.

Re: Should the docs suggest namespace packages?

[Aymeric Augustin]

Hello,

Commit ccc25bf refers to ticket #23919 in the commit message. In that ticket, I 
argued that the __init__.py files should be kept: 
https://code.djangoproject.com/ticket/23919#comment:102 
. No one brought a 
counter argument.

It's weird that the __init__.py files were removed anyway without further 
discussion (that I can find with my browser's search on that page, at least).

It's fairly minor, but I think that ccc25bf should be reverted.

Best regards,

-- 
Aymeric.



> On 20 May 2020, at 14:58, René Fleschenberg  wrote:
> 
> On Github, Tim pointed out that with
> https://github.com/django/django/commit/ccc25bfe4f0964a00df3af6f91c2d9e20159a0c2,
> various __init__.py files in Django's own code were removed.
> 
> I tend to say it would be good to revert that change. Or we could update
> the docs only, since they serve much more directly as an example for
> user code.
> 
> -- 
> René Fleschenberg
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/5d689d5c-01ab-d017-249c-a53a6d9b1f87%40fleschenberg.net.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/C9E429E5-DC4C-4C47-8126-17B32D31D245%40polytechnique.org.

Re: Clear all filters

[Aymeric Augustin]

On 18 May 2020, at 11:10, Mariusz Felisiak  wrote:

> Link to the right of the search box ("X total") already removes the entire 
> query string, so ...


Ah, this is a bug I'm hitting!

I have a small personal app where I classify data. Some classification is 
automated with regexes. New data needs to be classified manually.

My typical workflow is:

- filter on unclassified data
- find something interesting
- use the search field to find all similar data
- classify it (directly in the list view, thanks to list_editable)
- clear the search => I lose my filter :-(
- filter on unclassified data again
- etc.

Having to set the filter again and again is a minor annoyance so I haven't 
bothered to report a bug. But it's the wrong behavior in this case.

Best regards,

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/D962DF7E-2B2F-4D37-9F8C-609ACE4120E9%40polytechnique.org.

Re: Management of static assets

[Aymeric Augustin]

Thanks David for investigating the topic thoroughly! I wasn't expecting all 
that when I filed a one-line ticket six years ago :-) So, here's a bunch of 
opinions.


Before I start, I'd like to quote the intro to the Media class 
:

> Rendering an attractive and easy-to-use Web form requires more than just HTML 
> - it also requires CSS stylesheets, and if you want to use fancy « Web2.0 » 
> widgets, you may also need to include some JavaScript on each page. The exact 
> combination of CSS and JavaScript that is required for any given page will 
> depend upon the widgets that are in use on that page.


That was the reasonable thing to do before single-page apps, asset pipelines, 
bundlers and code splitting. It's still a fairly reasonable thing to do on a 
website that relies on good old HTML forms but would benefit from "fancy « 
Web2.0 » widgets".


1. Django shouldn't do anything (besides what django.contrib.staticfiles 
already does) for projects that use webpack (or any other bundler)

The correct approach with a bundler is to bundle all JavaScript code and, if 
needed, to optimize with code splitting. Attempting to include only what's 
needed on each page, like Media does, will usually be counterproductive, 
because it will general different bundle for different pages and defeat caching 
by the browser.

Then I know two techniques for integrating the frontend code with Django:

- the single page app 

 (website and API on separate domains) — this is clearly out of scope of this 
discussion as Django only provides an API in this scenario
- what I call the hybrid app 

 (website and API on the same domain) — here django.contrib.staticfiles helps; 
it comes after the JavaScript bundler in the deployment pipeline (and I prefer 
plain django.contrib.staticfiles over django-webpack-loader, but that's another 
story)

(NB: while the blog posts I just linked to focus on create-react-app, the 
concepts apply to any modern JavaScript toolchain.)

Regardless, Django already provides more than we need. For example, both 
webpack and django.contrib.staticfiles add hashes to file names to ensure cache 
invalidation.

So my answer to questions 3, 4 and 5 is "no, except maybe documentation".

Now, let's leave this brave new world and remember the jQuery era.


2. Media makes sense

Although pluggable apps are fantastic, the concept doesn't work well for 
templates, which is where  and 

Re: Technical Board statement on type hints for Django

[Aymeric Augustin]

Sorry for the late answer, I didn't have time to read this mailing list for the 
last month :-( Here's my position on the matter.

Adding type annotations only makes sense if a type checker checks them. 
Otherwise, they're unlikely to be correct and usable, so it would be 
counterproductive to add them. Unfortunately, I don't think we have a good 
enough type checker yet.

I'm having a rough experience maintaining type checking with mypy in websockets 
, a library that is very much smaller 
and simpler than Django. (Some problems are related to async, which isn't a 
concern for Django, but that doesn't explain everything.)

* Besides adding the type annotations, type checking required many other code 
changes. Importing types in modules that need them created circular 
dependencies. Fixing these import loops required moving imports to the bottom 
(ugh), splitting code in small modules (meh), or always importing modules 
rather than objects inside modules (perhaps a good practice, but changing this 
throughout a project creates a lot of noise). Also, import loops tend to 
reappear when implementing new features or refactoring.

* It's an uphill struggle to keep up with subtle changes of behavior in mypy — 
which is still 0.x software. I also encountered regressions that required 
pinning to an old version. Supporting more than one mypy version seems out of 
reach. I'm very pessimistic about supporting more than one type checker.

* I don't remember mypy catching interesting bugs. I'm keeping it only because 
I'm failing to deal with the sunk cost logical fallacy :-( Overall, adding type 
annotation to websockets has decreased my ability and willingness to maintain 
it. I believe it increased the barrier to entry for other contributors as well.

So, I think the ecosystem needs to stabilize a lot before Django can adopt 
static typing, even just as a best practice for new code. Comments like this 
one  make me 
uncomfortable about living that close to the edge.

Also, given that static typing was bolted on Python very late in its history 
and that there's a strong tradition of relying on duck typing, I'm not yet 
convinced static typing will ever be widely accepted as a best practice.

tl;dr I'm recommending to wait and see.

For the record, I'm a huge fan of good type systems. My second programming 
language after Basic was Caml , which has fantastic type 
inference: you never declare types but everything is strongly statically typed 
at compile time. I loved it. In contrast, having to declare types manually to 
get passable type checking isn't good enough for me.

Finally, I have to say that I was impressed by the effort that Maksim (author 
of the DEP), along with others, put into making mypy work with the more dynamic 
parts of Django. For example, typing of QuerySet arguments is impressive: 
https://github.com/django/deps/pull/65/files#diff-afab5a339bc83e08032ffb9446fb5103R185
 
.
 The documentation mentions a list of things that work, which suggests that 
edge cases not mentioned in the list may not work, so it isn't clear to me if 
it's possible to build a 100% correct solution with this approach. Still, it's 
impressive!

Best regards,

-- 
Aymeric.



> On 17 Apr 2020, at 14:19, charettes  wrote:
> 
> Thanks for sharing your answer publicly Adam and Markus. Here are the points 
> I brought up in the discussion
> 
> From my limited experience with type annotation on a large monolithic code 
> base that was ported from Python 2 to 3 even gradual typing has been a lot of 
> trouble for the few benefits it brought us.
> 
> This partially explains why I had reservations about the benefits of the 
> proposal .
> 
> I also think that it's a bit too early to bundle type annotations in the core 
> repository and I'd favour an approach where we make adjustments to the source 
> to add entry points allowing third parties to implement and maintains such 
> type annotations. That's the approach we've taken in the past when a new 
> feature is suggested for inclusion in core; we add missing entry points for 
> the feature to be implemented externally instead. This should allow different 
> tools to compete organically in the ecosystem without having to choose the 
> best candidate right now.
> 
> I'm -0 on allowing stdlib type annotations in the mean time since I feels 
> like it would open a can of worms that we want to keep sealed for now. What 
> if a function accepts/return a non-stdlib type (or an Union of a stdlib and 
> non-stdlib), do we type annotate it? What if we have to change this function 
> to a Union[non-stdlib-type, stdlib-type], should we completely drop the type 
> annotation on that function?
> 
> Simon
> 
> Le vendredi 17 avril 2

Re: Removing url() ?

[Aymeric Augustin]

Hello,

When proposing deprecations that require numerous but simple code changes, 
providing an automated upgrade path could reduce frustrations about "useless 
churn".

Unfortunately, I don't know a great way to do that on Python code. Here are the 
options I'm aware of:

- Writing 2to3 fixers is painful, in my experience 
.
 Also, I don't think that 2to3 will survive the deprecation of lib2to3; removal 
is scheduled for Python 3.12.
- Bowler  seems designed to make it less painful, but is 
also written on top of lib2to3, so not a long term solution.
- codemod  gives up fully automated 
refactoring (which is at odds with Python's dynamic nature) and attempts 
"computer-aided refactoring".
- Most likely IDEs like PyCharm can rename an import throughout a project.

If I had to do something in this area, I'd try codemod, probably.

Of course, it will still be boring to perform the same change across N projects 
for no direct project-level upside, but at least it may require less time and 
less focus.

This is relevant for the thread about renaming request.GET/POST/etc. If we 
proceed with that change, we're bound to have the exact same discussion about 
removing the original names.

Best regards,

-- 
Aymeric.



> On 5 May 2020, at 17:41, Collin Anderson  wrote:
> 
> Hi All,
> 
> Are we _sure_ we want to completely get rid of url()?
> 
> Yesterday, url() was given a RemovedInDjango40Warning [PR]. The removal was 
> approved as part of the new path() syntax back in 2017 [DEP 0201].
> 
> Is this really worth it? It's only a few lines of code to keep backward 
> compatibility, and it seems to me it would take almost no work to maintain 
> that compatibility shim compared to the countless programmer hours needed to 
> upgrade their code, including many unpaid programmers working on open source 
> projects. I'll personally need to do a find/replace for url() to re_path() 
> across my ~20 projects, and update all of the imports. Isn't this useless 
> code churn?
> 
> Yes, there's an advantage to having one and only one way to do it, so I agree 
> we should encourage people to use re_path() instead of url(), but shouldn't 
> we also make it as easy as possible to upgrade django? Is getting rid of 
> url() really worth the cost?
> 
> Yes, the removal is still ~3+ years away, but that's a question of _when_ not 
> _if_.
> 
> If we want, we _could_ deprecate url() without giving it an actual removal 
> date [Compatibility Discussion] and leave the compatibility shim around 
> longer, if not indefinitely.
> 
> Thanks,
> Collin
> 
> 
> [PR] https://github.com/django/django/pull/12855
> 
> [DEP 0201] 
> https://github.com/django/deps/blob/master/final/0201-simplified-routing-syntax.rst
> 
> [Compatibility Discussion] 
> https://groups.google.com/d/topic/django-developers/asqnjcYPnms/discussion
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/17899c17-2c07-4a80-baa5-e0a348d9512c%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/236BB3BC-B228-4676-8955-4E302A38197F%40polytechnique.org.

Re: Rename request.GET and POST, and lowercase COOKIES, FILES, and META

[Aymeric Augustin]

Hello,

This is quite disruptive — says the guy who suggested to remove request.REQUEST 
a few years back — but I think we could go ahead and do it. The names proposed 
by Adam are good.

I read the concerns about  (e.g. for search forms) where 
form data ends up in query_params and not form_data. I'd call this a feature 
:-) Here's why:

- Users who don't know that GET queries mustn't have side effects and who don't 
want to bother will have an incentive to use a POST query to get the data in 
form_data; this is the safe option with regards to CSRF;
- Users who understand the safety model of browsers and who can determine that 
their view isn't vulnerable to CSRF will understand that the form data is in 
query_params, because that's where it is really.

Also, looking at various names between `form_data` and 
`form_data_decoded_from_x_www_form_urlencoded_post_body` (the technically 
correct name), I believe that `form_data` is the best compromise between 
clarity and terseness.

Best regards,

-- 
Aymeric.



> On 5 May 2020, at 23:26, Adam Johnson  wrote:
> 
> request.GET and request.POST are misleadingly named:
> GET contains the URL parameters and is therefore available whatever the 
> request method. This often confuses beginners and “returners” alike.
> POST contains form data on POST requests, but not other kinds of data from 
> POST requests. It can confuse users who are posting JSON or other formats.
> Additionally both names can lead users to think e.g. "if request.GET:" means 
> "if this is a GET request", which is not true.
> 
> I believe the CAPITALIZED naming style was inherited from PHP's global 
> variables $_GET, $_POST, $_FILES etc. ( 
> https://www.php.net/manual/en/reserved.variables.get.php 
>  ). It stands out 
> as unpythonic, since these are instance variables and not module-level 
> constants (as per PEP8 https://www.python.org/dev/peps/pep-0008/#constants 
>  ).
> 
> I therefore propose these renames:
> request.GET -> request.query_params (to match Django Rest Framework - see 
> below)
> request.POST -> request.form_data
> request.FILES -> request.files
> request.COOKIES -> request.cookies
> request.META -> request.meta
> Since these are very core attributes and the change would cause a huge amount 
> of churn, I propose not deprecating the old aliases immediately, but leaving 
> them in with a documentation note that they "may be deprecated." Perhaps they 
> can be like url() or ifequal which came up on the mailing list recently - put 
> through the normal deprecation path after a few releases with such a note.
> 
> Django Rest Framework already aliases GET as request.query_params in its 
> request wrapper: 
> https://www.django-rest-framework.org/api-guide/requests/#query_params 
>  . 
> Therefore the name request.query_params should not be surprising. DRF also 
> provides request.data which combines request.POST and request.FILES, and 
> flexibly parses from different content types, but I'm not sure that's 
> feasible to implement in Django core.
> 
> For reference, other Python web frameworks have more "Pythonic" naming:
> Bottle: request.url_args, request.forms, request.files, request.cookies, 
> request.environ
> Flask: request.args, request.form, request.files, request.cookies, 
> request.environ
> Starlette: request.query_params, request.form(), request.form()[field_name], 
> request.cookies, scope
> One more note for those who might think such core attributes should be left 
> alone: Django 2.2 added request.headers as a way of accessing headers by 
> name. This is convenient as it avoids the need to transform the header to the 
> WSGI environ name. makes the code more readable, and in the process reduces 
> the potential for bugs. I believe this proposal is in the same vein.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMyDDM3%2BUucViDezhkWrFsk6ZsKViWjOgtA5aBm9pnzozdc%2Beg%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dj

Re: New Merger nomination.

[Aymeric Augustin]

Hello,

I trust Claude to act as a Merger as described in DEP 10. I vote in favor.

With apologies for the late answer!

-- 
Aymeric.



> On 21 Apr 2020, at 20:11, Andrew Godwin  wrote:
> 
> I also vote in favour of Claude becoming a Merger!
> 
> Andrew
> 
> On Tuesday, April 21, 2020 at 4:28:41 AM UTC-6, Markus Holtermann wrote:
> I vote in favor of Claude becoming a MERGER. 
> 
> Cheers, 
> 
> Markus 
> 
> On Thu, Apr 16, 2020, at 10:31 PM, charettes wrote: 
> > I cast my vote in favor of Claude's nomination as well. 
> > 
> > Le jeudi 16 avril 2020 16:16:31 UTC-4, Adam Johnson a écrit : 
> > > This has fallen by the wayside, let's try restarting. 
> > > 
> > > As Carlton points out, Claude hasn't been merging in code without others 
> > > reviewing it. But as I understand it is useful to keep translations 
> > > moving that he can merge in his or others' (accepted) PR's. It gets us to 
> > > the minimum of three mergers, and Claude has stated he's willing to do 
> > > his best in the role of merger. And I hope Claude can help inform us what 
> > > we can do to make i18n more smooth. 
> > > 
> > > I nominate Claude as a merger. 
> > > 
> > > Technical board, please post your votes. 
> > > 
> > > On Thu, 19 Mar 2020 at 07:59, Carlton Gibson > 
> > > wrote: 
> > >> I don't know if this got blocked in the now-obsolete use of cognates of 
> > >> "commit"? 
> > >> 
> > >> > The Merger role is not just a new name for "committer". If Claude is 
> > >> > appointed a Merger, he will be forbidden to do what you are saying -- 
> > >> > a Merger cannot push their own contributions directly into Django! 
> > >> 
> > >> Claude hasn't, and wouldn't, "commit" directly to Django here. He's 
> > >> always opened a 
> > >> PR with the translations updates, which has then been reviewed, and 
> > >> which then, yes, he has 
> > >> merged, and forward/backported as necessary — which IS the Merger role. 
> > >> (IIUC) 
> > >> 
> > >> In the Merger role, Claude would also be able to approve and Merge other 
> > >> PRs. 
> > >> 
> > >> Beyond this important translations case, we do need a third Merger, and 
> > >> I cannot think of a better candidate. 
> > >> (The two most active contributors are Claude and Simon, and Simon cannot 
> > >> serve as a Merger because of his 
> > >> role on the Technical Board.) 
> > >> 
> > >> I would ask the Technical Board to proceed with the nomination, as, bar 
> > >> the loose language, I think it is in line with the DEP. 
> > >> 
> > >> Kind Regards, 
> > >> 
> > >> Carlton 
> > >> 
> > 
> > >>  -- 
> > >>  You received this message because you are subscribed to the Google 
> > >> Groups "Django developers (Contributions to Django itself)" group. 
> > >>  To unsubscribe from this group and stop receiving emails from it, send 
> > >> an email to django-d...@googlegroups.com <>. 
> > >>  To view this discussion on the web visit 
> > >> https://groups.google.com/d/msgid/django-developers/14af7b19-bcd3-4462-9e25-606f30ff6eda%40googlegroups.com
> > >>  
> > >> 
> > >>  
> > >>  > >>  
> > >> >.
> > >>  
> > > 
> > > 
> > > -- 
> > > Adam 
> > 
> >  -- 
> >  You received this message because you are subscribed to the Google 
> > Groups "Django developers (Contributions to Django itself)" group. 
> >  To unsubscribe from this group and stop receiving emails from it, send 
> > an email to django-d...@ <>googlegroups.com . 
> >  To view this discussion on the web visit 
> > https://groups.google.com/d/msgid/django-developers/c13e6529-0e75-4e04-8ea6-501982420504%40googlegroups.com
> >  
> > 
> >  
> >  >  
> > >.
> >  
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/15a11b73-d6dd-4ea4-af42-0919e75a5147%40googlegroups.com
>  
> 

Re: GSoC Proposal to extend the parallel test runner to Windows and macOS and to support Oracle parallel test running

[Aymeric Augustin]

Hello Ahmad,

I believe there's interest for supporting parallel test runs on Oracle, if only 
to make Django's own test suite faster.

I'm not very familiar with spawn vs. fork. As far as I understand, spawn starts 
a new process, so you'll have to redo some initialization in the child 
processes. In a regular application, this would mean calling `django.setup()`. 
In Django's own test suite, it might be different; I don't know. Try it and see 
what breaks, maybe?

Hope this helps :-)

-- 
Aymeric.



> On 23 Mar 2020, at 20:22, Ahmad A. Hussein  wrote:
> 
> Django's parallel test runner works through forking processes, making it 
> incompatible on Windows by default and incompatible in macOS due to a recent 
> update. Windows and macOS both support spawn and have it enabled by default. 
> Databases are cloned for each worker.
> 
> To switch from fork to spawn, state setup will be handled by spawned 
> processes instead of through inheritance via fork. Worker’s connections to 
> databases can still be done through get_test_db_clone_settings which changes 
> the names of databases assigned to a specific worker_id; however, SQLite's 
> cloning method is incompatible with spawn.  
> 
> 
> SQLite’s cloning method relies on it both being in-memory and fork as when we 
> fork the main process, we retain SQLite's in-memory database in the child 
> process. The solution is to get a SQL dump of the database and throw it into 
> the target cloned databases. This is also the established standard in 
> handling MySQL’s cloning process.
> 
> Both Postgresql's and MySQL's cloning methods are independent of fork or 
> spawn and won't require any modification.
> 
> Oracle has not been implemented in the parallel test runner originally even 
> on Linux and I propose to extend support to Oracle as well in my proposal. I 
> want to confirm if there is significant support behind this as a feature or 
> not before I commit to writing a specification, but as a summary it is 
> definitely possible as the work collaborated on by Aymeric Augustin and Shai 
> Berger show that cloning CAN be done through multiple ideas. The reason why 
> it's a headache is that Oracle does not support separate databases under a 
> single user- unlike our other supported databases, so we can't clone 
> databases without introducing another user. Some methods may also need to be 
> rewritten to accommodate for the Oracle backend, but that isn't an issue.
> 
> I've glossed over writing out a schedule or a more detailed abstract as I I'm 
> mainly posting here to see if there is indeed support for the Oracle proposal 
> and to make sure I am not missing any details in regards to adapting the 
> current parallel test runner to work through spawn. Let me know what you 
> think.
> 
> Regards,
> Ahmad
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/317f67c6-4b23-483f-ada5-9bdbb45d0997%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/django-developers/317f67c6-4b23-483f-ada5-9bdbb45d0997%40googlegroups.com?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/08E2379A-F09E-47C3-9464-42206ECCA44A%40polytechnique.org.

Re: Support for MongoDB

[Aymeric Augustin]

Hello Sanskar,

You can find discussions about this issue in archives of this mailing list. I 
don't think it came up recently; you'll have to go a few years back.

Support for MongoDB would mean support in the ORM. The R in ORM stands for 
Relational. MongoDB isn't a relational database. Therefore "support" for 
MongoDB would be deceptive. The consensus is for not adding it. I'm not aware 
of changes in MongoDB or in Django that could result in a different outcome if 
the discussion happens again.

If you'd like to push this proposal forwards anyway, writing a DEP would be the 
next step. Be aware that it will be difficult to get it accepted.

Best regards,

-- 
Aymeric.



> On 18 Mar 2020, at 06:21, Sanskar Jaiswal  
> wrote:
> 
> Hey everyone,
> 
> I would like to get feedback from the community, on whether providing support 
> for MongoDB out of the box would be nice idea. I know that Djongo, takes care 
> of that, but I believe having support in the framework itself, rather than 
> importing a library, would result in a more smooth experience, and would also 
> lead the industry to consider Django as a stronger candidate in their 
> respective tech stack.
> 
> Cheers
> Sanskar
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/330F6F6E-227E-4753-A210-C0A2587F718E%40gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/10CB170F-A607-44EA-BA5F-6866C4C4F339%40polytechnique.org.

Re: Proposal to deprecate NullBooleanField (and remove in Django 4.0)

[Aymeric Augustin]

Hello,

I'm inclined to Accept as well.

-- 
Aymeric.



> On 17 Mar 2020, at 08:49, Carlton Gibson  wrote:
> 
> Hi all. 
> 
> 
> https://code.djangoproject.com/ticket/31369
> Proposes to deprecate NullBooleanField. 
> 
> 
> https://code.djangoproject.com/ticket/29227 
> Allowed BooleanField to be null=True. 
> Introduced in Django 2.1. 
> 
> 
> I'm inclined to Accept. 
> Wonder if folks consider it a little too fast? 
> (We could defer for 5.0.) 
> 
> 
> Kind Regards,
> 
> Carlton
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/02a4b01d-e47f-4092-826e-c0cfa074901f%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6353F843-B61D-4C50-8A45-98DAD860301A%40polytechnique.org.

Re: Deprecating logout via GET

[Aymeric Augustin]

Hello,

Le dim. 1 mars 2020 à 11:04, Adam Johnson  a écrit :

> Yes, but then hovering on the link doesn't show the logout URL at the
>> bottom of the screen.
>
>
> I don't think this is a concern.
>

If it's just the link preview, yes, I think we can make the trade off.

The more general concern here is accessibility. It would be nice to
experiment and make sure the new implementation is usable with a screen
reader (for example).

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mX2vujJQmfeAd1DbS3D3kfj8fwSKTvZSPfvoNJCKRzQ0A%40mail.gmail.com.

Re: Proposal: Multiple-Inheritance in Templates

[Aymeric Augustin]

This suggestion is better than mine for the use case discussed here :-)

-- 
Aymeric.



> On 18 Feb 2020, at 21:57, Matthias Kestenholz  wrote:
> 
> Hi,
> 
> You could make the generic-list.html template extend a configurable base 
> template, e.g. "{% extends base_template|default:'base.html' %}", and add 
> context["base_template"] = "orders/base.html" in views belonging to the 
> orders app, and "products/base.html" in the products app.
> 
> I often use this pattern exactly for the case when some app needs to add its 
> own submenu.
> 
> Best,
> Matthias
> 
> 
> 
> On Tue, Feb 18, 2020 at 9:44 PM Yo-Yo Ma  > wrote:
> Please help us all understand, then:
> 
> How is it possible to inherit the blocks defined in both the orders/base.html 
> template AND the generic-list.html template while inheriting only from the 
> generic-list.html template?
> 
> On Tuesday, February 18, 2020 at 3:32:12 PM UTC-5, matthew.pava wrote:
> I did read your post, and I did come to that conclusion. I may have 
> ultimately misunderstood, and I’ll certainly take the blame for my own 
> misunderstanding. The example you provided can be done without multiple 
> template inheritance. So far, my thoughts on the design concept is that 
> multiple template inheritance is unnecessary.
> 
>  
> From: django-d...@googlegroups.com <> [mailto:django-d...@googlegroups.com 
> <>] On Behalf Of Yo-Yo Ma
> Sent: Tuesday, February 18, 2020 2:25 PM
> To: Django developers (Contributions to Django itself)
> Subject: Re: Proposal: Multiple-Inheritance in Templates
> 
>  
> @matthew.pava please read the entire OP again.
> 
>  
> You can't have actually read the post and come to the conclusion that 
> inheriting from generic-list.html would solve the problem.
> 
>  
> These kinds of "I didn't read it but it looks like you could just X" replies 
> are a big problem in mailing lists; they have the unintended consequence of 
> causing serious posts to like noobie "help me" questions by later passers by.
> 
>  
> I'm a 12-13 year Python veteran (11 years full-time Django), not a noob, and 
> I think this feature could be a tremendous benefit to Django templates, and 
> I'd like some real thoughts on the design concept.
> 
> 
> On Monday, February 17, 2020 at 10:55:44 AM UTC-5, matthew.pava wrote:
> 
> In your example, you could just inherit from generic-list.html since it 
> already extends base.html. You would be repeating yourself in your current 
> example.
> 
>  
> orders/list.html
> 
>  
> {% extends 'generic-list.html' %}
> 
>  
> {% block list-item %}
> 
> Order # {{ order.id  }} (total: {{ order.total 
> }})
> 
> {% endblock %}
> 
>  
>  
> From: django-d...@googlegroups.com <> [mailto:django-d...@googlegroups.com 
> <>] On Behalf Of Yo-Yo Ma
> Sent: Sunday, February 16, 2020 10:02 AM
> To: Django developers (Contributions to Django itself)
> Subject: Proposal: Multiple-Inheritance in Templates
> 
>  
> Please read this in full - you will not be disappointed by the end.
> 
>  
> We often talk about "multiple-inheritance" when referring to multiple levels 
> of {% extends ... %} and defining {% block ... %} many layers up in base 
> templates.
> 
>  
> But, something that comes up as a useful idea in my mind on a regular basis 
> when working in Django templates is the ability to actually inherit from 
> multiple base templates.
> 
>  
> The immediate reaction to this idea might be repulsion until you think 
> through the implications and weigh them against the benefits.
> 
>  
> An example (simplified) use case speaks louder than explanation:
> 
>  
> base.html
> 
>  
> Some top-level menu
> 
> {% block submenu %}{% endblock %}
> {% block content %}{% endblock %}
> 
>  
> --
> 
>  
> orders/base.html
> 
>  
> {% extends 'base.html' %}
> 
>  
> {% block submenu %}
> 
> View all Orders
> 
> Enter an Order
> {% endblock %}
> 
> 
> --
> 
>  
> products/base.html
> 
>  
> {% extends 'base.html' %}
> 
>  
> {% block submenu %}
> 
> View all Products
> 
> Create a Product
> {% endblock %}
> 
> 
> --
> 
>  
> So, we have a relatively common template setup, with some apps defining their 
> own base templates that extend the main base template, and then defining 
> their own submenus, etc.
> 
>  
> At this point, we know there will be a product list page and an order list 
> page, but we also know that the product list template will extend the product 
> base template and the order list template will extend the order base 
> template; same goes for the product create and order create, both also 
> extending their respective apps' base templates.
> 
>  
> This means duplication. The product list template will contain most of the 
> same list-related HTML, etc. as the order list, but the only way to avoid 
> that duplication will be to make a base list template and extend that 
> template from both the product list and order list.
> 
>  
> generic-list.html
> 
>  
> {% extends 'base.html' %}
> 
>  
> {%

Re: Proposal: Multiple-Inheritance in Templates

[Aymeric Augustin]

Hello Yo-Yo Ma,

Please assume good faith. You've been around for 11 years, so you know the way 
you address Matthew isn't how we behave on this mailing-list.

I believe that the most common way to achieve what you want is to include the 
"generic list" HTML with the {% include %} tag. This allows you to share 
generic components across multiple templates.

In theory, it's less powerful than the multiple inheritance mechanism you've 
been proposing (because it does a separate render and includes the result 
verbatim). In practice, I think it does the job.

Best regards,

-- 
Aymeric.



> On 18 Feb 2020, at 21:44, Yo-Yo Ma  wrote:
> 
> Please help us all understand, then:
> 
> How is it possible to inherit the blocks defined in both the orders/base.html 
> template AND the generic-list.html template while inheriting only from the 
> generic-list.html template?
> 
> On Tuesday, February 18, 2020 at 3:32:12 PM UTC-5, matthew.pava wrote:
> I did read your post, and I did come to that conclusion. I may have 
> ultimately misunderstood, and I’ll certainly take the blame for my own 
> misunderstanding. The example you provided can be done without multiple 
> template inheritance. So far, my thoughts on the design concept is that 
> multiple template inheritance is unnecessary.
> 
>  
> From: django-d...@googlegroups.com <> [mailto:django-d...@googlegroups.com 
> <>] On Behalf Of Yo-Yo Ma
> Sent: Tuesday, February 18, 2020 2:25 PM
> To: Django developers (Contributions to Django itself)
> Subject: Re: Proposal: Multiple-Inheritance in Templates
> 
>  
> @matthew.pava please read the entire OP again.
> 
>  
> You can't have actually read the post and come to the conclusion that 
> inheriting from generic-list.html would solve the problem.
> 
>  
> These kinds of "I didn't read it but it looks like you could just X" replies 
> are a big problem in mailing lists; they have the unintended consequence of 
> causing serious posts to like noobie "help me" questions by later passers by.
> 
>  
> I'm a 12-13 year Python veteran (11 years full-time Django), not a noob, and 
> I think this feature could be a tremendous benefit to Django templates, and 
> I'd like some real thoughts on the design concept.
> 
> 
> On Monday, February 17, 2020 at 10:55:44 AM UTC-5, matthew.pava wrote:
> 
> In your example, you could just inherit from generic-list.html since it 
> already extends base.html. You would be repeating yourself in your current 
> example.
> 
>  
> orders/list.html
> 
>  
> {% extends 'generic-list.html' %}
> 
>  
> {% block list-item %}
> 
> Order # {{ order.id  }} (total: {{ order.total 
> }})
> 
> {% endblock %}
> 
>  
>  
> From: django-d...@googlegroups.com <> [mailto:django-d...@googlegroups.com 
> <>] On Behalf Of Yo-Yo Ma
> Sent: Sunday, February 16, 2020 10:02 AM
> To: Django developers (Contributions to Django itself)
> Subject: Proposal: Multiple-Inheritance in Templates
> 
>  
> Please read this in full - you will not be disappointed by the end.
> 
>  
> We often talk about "multiple-inheritance" when referring to multiple levels 
> of {% extends ... %} and defining {% block ... %} many layers up in base 
> templates.
> 
>  
> But, something that comes up as a useful idea in my mind on a regular basis 
> when working in Django templates is the ability to actually inherit from 
> multiple base templates.
> 
>  
> The immediate reaction to this idea might be repulsion until you think 
> through the implications and weigh them against the benefits.
> 
>  
> An example (simplified) use case speaks louder than explanation:
> 
>  
> base.html
> 
>  
> Some top-level menu
> 
> {% block submenu %}{% endblock %}
> {% block content %}{% endblock %}
> 
>  
> --
> 
>  
> orders/base.html
> 
>  
> {% extends 'base.html' %}
> 
>  
> {% block submenu %}
> 
> View all Orders
> 
> Enter an Order
> {% endblock %}
> 
> 
> --
> 
>  
> products/base.html
> 
>  
> {% extends 'base.html' %}
> 
>  
> {% block submenu %}
> 
> View all Products
> 
> Create a Product
> {% endblock %}
> 
> 
> --
> 
>  
> So, we have a relatively common template setup, with some apps defining their 
> own base templates that extend the main base template, and then defining 
> their own submenus, etc.
> 
>  
> At this point, we know there will be a product list page and an order list 
> page, but we also know that the product list template will extend the product 
> base template and the order list template will extend the order base 
> template; same goes for the product create and order create, both also 
> extending their respective apps' base templates.
> 
>  
> This means duplication. The product list template will contain most of the 
> same list-related HTML, etc. as the order list, but the only way to avoid 
> that duplication will be to make a base list template and extend that 
> template from both the product list and order list.
> 
>  
> generic-list.html
> 
>  
> {% extends 'base.html' %}
> 
>  
> {% block content %}

Re: Use "raise from" where appropriate, all over the codebase

[Aymeric Augustin]

Hello Ram,

> On 6 Feb 2020, at 19:08, Ram Rachum  wrote:
> 
> In other words, "raise from" is the inevitable future, it's just that we're 
> not in a rush to get there.


I'm not sure how you came to this conclusion; I'm not seeing this in Carlton's 
and Mariusz' answers.

Carlton only said that `raise from` can be useful in specific cases where the 
default, implicit exception chaining doesn't do what you need.

> On 6 Feb 2020, at 15:37, Carlton Gibson  wrote:
> 
> Beyond that, the from syntax is simply more verbose. In the PR the changes 
> are 
> all adding a unneeded `as e`, followed by an equally unneeded `from e`. 


This wouldn't be an improvement.

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5AF3A99A-4C6E-4919-BACB-7FFAE7C34FEC%40polytechnique.org.

Re: Admin: Making raw_id fields navigable for readonly users

[Aymeric Augustin]

Hello Julien,

This would be a nice UX improvement. The link should only be rendered if the 
user has permission to view the target model.

Perhaps a stupid question, but — wouldn't this apply to non-raw_id, readonly 
foreign keys as well? I don't think they're rendered as links, are they?

Best regards,

-- 
Aymeric.



> On 18 Jan 2020, at 19:18, Julien Rebetez  wrote:
> 
> Hello,
> 
> When viewing a model with raw_id fields in the admin UI, if you only have 
> view permission, then those fields are not rendered as links (you cannot go 
> to the related model) - they are just plaintext.
> 
> I that something that you would agree to change ? That would be helpful for 
> us at work as we have quite a few raw_id fields and have readonly users as 
> well.
> 
> I have a working patch for this - just need to clean it up & add tests. So if 
> it's a welcome change, I'll go ahead and do a PR.
> 
> Thanks
> Julien
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/9c38e6ad-d663-429b-a10b-5d8ce2906810%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/FCDEAC08-D76E-4800-A8F4-1F537EC7D956%40polytechnique.org.

Automatically discovering AppConfig subclasses

[Aymeric Augustin]

Hello,

Since the app-loading refactor in Django 1.7, we recommend referencing 
AppConfig classes in INSTALLED_APPS e.g. 
"django.contrib.admin.apps.AdminConfig".

I don't think this style gained a lot of traction compared to the legacy, 
shorter style e.g. "django.contrib.admin". As a consequence, most apps define a 
`default_app_config` , which is an ugly hack for backwards-compatibility.

I think we should stop fighting the community, embrace the shorter style, and 
deprecate `default_app_config`. A  pull request doing all this is here: 
https://github.com/django/django/pull/12310 
. The rationale is explained with 
more details in the PR comments and the docs changes.

This was briefly discussed in the recent "GDAPS" thread. I'm starting a new 
thread to make sure this doesn't fly under the radar. Please let me know your 
thoughts!

Best regards,

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8F5547FD-0CE3-4B68-8492-7DDD70ADE9F0%40polytechnique.org.

Re: GDAPS

[Aymeric Augustin]

Hello Christian,

Regarding your first point, adding "default = True" to 1% of apps in a 
non-empty file seems better than adding default_app_config = 
"foo.apps.FooConfig" to 99% of apps in an otherwise empty file (usually). You 
can see the effect in 
https://github.com/django/django/pull/12310/commits/91cc52ed55dd1ec6cc05b9aa2d9a48aff3cdf87b.

Best regards,

-- 
Aymeric.



> On 13 Jan 2020, at 23:04, Christian González  
> wrote:
> 
> Oh, sorry, I just saw that this is discussed (andanswered) in the PR.
> 
> Am 13.01.20 um 23:03 schrieb Christian González:
>> Eh, sorry if I misunderstand - but adding a "default = true" line to an 
>> AppConfig is the same as addign default_app_config = "foo.apps.FooConfig".
>> 
>> A few things that came to my mind when reading the code:
>> 
>> * You check if there is only one AppConfig available 
>> (https://github.com/django/django/pull/12310/commits/6a4ee2d02477cac1fa67708ceefda2ffd56bbf96#diff-239ff7e40a4b9921b462471882a575f6R111
>>  
>> <https://github.com/django/django/pull/12310/commits/6a4ee2d02477cac1fa67708ceefda2ffd56bbf96#diff-239ff7e40a4b9921b462471882a575f6R111>)
>>  and if so, this one is used - that's perfect IMHO.
>> 
>> * But what if 2 AppConfigs have "default = true" ? Then line #139+ is run, 
>> and silently the default app config class is used - none of the two ones. 
>> Maybe an error message would be better. default=true should only be valid 
>> once!
>> 
>> Best regards,
>> 
>> Christian
>> 
>> 
>> Am 12.01.20 um 21:42 schrieb Aymeric Augustin:
>>> Hello,
>>> 
>>> I created a PR for this: https://github.com/django/django/pull/12310 
>>> <https://github.com/django/django/pull/12310>
>>> 
>>> I'd love to get some feedback on the design before I polish the patch.
>>> 
>>> Best regards,
>>> 
>>> -- 
>>> Aymeric.
>>> 
>>> 
>>> 
>>>> On 10 Jan 2020, at 14:18, Christian González 
>>>> mailto:christian.gonza...@nerdocs.at>> 
>>>> wrote:
>>>> 
>>>> 
>>>> 
>>>> Am 08.01.20 um 22:39 schrieb Aymeric Augustin:
>>>>> The original intent was to make configuration explicit by having 
>>>>> settings.py reference directly the target AppConfig class.
>>>>> 
>>>>> - When you write MIDDLEWARE = ["polls"], do you expect Django to enable 
>>>>> "polls.middleware.PollsMiddleware"?
>>>>> - When you write INSTALLED_APPS = ["polls"], do you expect Django to 
>>>>> enable "polls.apps.PollsAppConfig"?
>>>>> 
>>>>> Many readers of this mailing list will answer "no" and "yes" 
>>>>> respectively. I think the answer to both should be "no", according to 
>>>>> Django's design philosophy 
>>>>> <https://docs.djangoproject.com/en/3.0/misc/design-philosophies/#explicit-is-better-than-implicit>.
>>>>>  (If you're about to argue that an application can provide more than one 
>>>>> middleware — I'm aware of this and I know you understood my point anyway.)
>>>> Yes, that's exactly what I wanted to state.
>>>>> [...]
>>>>> 
>>>>> If we're reversing course, giving up on explicitness and going for magic, 
>>>>> we could embrace that decision fully: import the "apps" submodule inside 
>>>>> each application; if it exists and it defines exactly one AppConfig 
>>>>> subclass, use that as the config for this app. Then app authors can stop 
>>>>> littering their __init__.py files with default_app_config.
>>>> That would be perfect IMHO. One could consider restricting it to the 
>>>> same-named subclass like the app itself, so the app "foo" would need to 
>>>> have "FooConfig" as default.
>>>> 
>>>> I'm aware that this violates the 2nd "Zen of Python" principle "Explicit 
>>>> is better than implicit". and "Special cases aren't special enough to 
>>>> break the rules." - but the next one says: "Although practicality beats 
>>>> purity."
>>>> 
>>>> Yes, too much "magic" leads to a system where half of it uses magic, and 
>>>> the other half not - so it's not predictable for the developer what a 
>>>> certain l

Re: GDAPS

[Aymeric Augustin]

Hello,

I created a PR for this: https://github.com/django/django/pull/12310

I'd love to get some feedback on the design before I polish the patch.

Best regards,

-- 
Aymeric.



> On 10 Jan 2020, at 14:18, Christian González  
> wrote:
> 
> 
> 
> Am 08.01.20 um 22:39 schrieb Aymeric Augustin:
>> The original intent was to make configuration explicit by having settings.py 
>> reference directly the target AppConfig class.
>> 
>> - When you write MIDDLEWARE = ["polls"], do you expect Django to enable 
>> "polls.middleware.PollsMiddleware"?
>> - When you write INSTALLED_APPS = ["polls"], do you expect Django to enable 
>> "polls.apps.PollsAppConfig"?
>> 
>> Many readers of this mailing list will answer "no" and "yes" respectively. I 
>> think the answer to both should be "no", according to Django's design 
>> philosophy 
>> <https://docs.djangoproject.com/en/3.0/misc/design-philosophies/#explicit-is-better-than-implicit>.
>>  (If you're about to argue that an application can provide more than one 
>> middleware — I'm aware of this and I know you understood my point anyway.)
> Yes, that's exactly what I wanted to state.
>> [...]
>> 
>> If we're reversing course, giving up on explicitness and going for magic, we 
>> could embrace that decision fully: import the "apps" submodule inside each 
>> application; if it exists and it defines exactly one AppConfig subclass, use 
>> that as the config for this app. Then app authors can stop littering their 
>> __init__.py files with default_app_config.
> That would be perfect IMHO. One could consider restricting it to the 
> same-named subclass like the app itself, so the app "foo" would need to have 
> "FooConfig" as default.
> 
> I'm aware that this violates the 2nd "Zen of Python" principle "Explicit is 
> better than implicit". and "Special cases aren't special enough to break the 
> rules." - but the next one says: "Although practicality beats purity."
> 
> Yes, too much "magic" leads to a system where half of it uses magic, and the 
> other half not - so it's not predictable for the developer what a certain 
> line does.
> 
> But exactly in this part I think that having a "sane default" - in the sense 
> of app loading using by using the app name only - is ok.
> 
> I would bet that the vast majority of apps don't have two AppConfigs.
> And to urge all devs to use the more elaborative AppConfig line in 
> INSTALLED_APPS, just to make such special cases (and THESE are the special 
> cases!) are possible, is wrong IMHO.
> 
> Let it be easy - even the variable says INSTALLED_APPS, not 
> INSTALLED_CONFIGS. And let there be the possibility to declare other 
> AppConfigs on demand.
> 
> Christian
> 
> 
> 
> -- 
> Dr. Christian González
> https://nerdocs.at <https://nerdocs.at/>
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/b37e3655-4253-75a1-b750-6c81a0f6cf26%40nerdocs.at
>  
> <https://groups.google.com/d/msgid/django-developers/b37e3655-4253-75a1-b750-6c81a0f6cf26%40nerdocs.at?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A037EBE4-78A7-4037-9E8A-29DCC604CB85%40polytechnique.org.

Re: GDAPS

[Aymeric Augustin]

Hello Christian,

> On 5 Jan 2020, at 14:30, Christian González  
> wrote:
> 
> And to be honest - does this really make sense to urge people - writing more 
> code to satisfy the framework?
> What I mean is: Why don't you drop that recommendation - when noone is using 
> it - more than one AppConfigs still *are* possible, even when there is a 
> default
> 

The original intent was to make configuration explicit by having settings.py 
reference directly the target AppConfig class.

- When you write MIDDLEWARE = ["polls"], do you expect Django to enable 
"polls.middleware.PollsMiddleware"?
- When you write INSTALLED_APPS = ["polls"], do you expect Django to enable 
"polls.apps.PollsAppConfig"?

Many readers of this mailing list will answer "no" and "yes" respectively. I 
think the answer to both should be "no", according to Django's design 
philosophy 
.
 (If you're about to argue that an application can provide more than one 
middleware — I'm aware of this and I know you understood my point anyway.)

Even though the extra level of indirection via default_app_config exists only 
for backwards-compatibility, everyone seems to be happy with it. Perhaps it's 
time to be pragmatic and admit I made the wrong call.

If we're reversing course, giving up on explicitness and going for magic, we 
could embrace that decision fully: import the "apps" submodule inside each 
application; if it exists and it defines exactly one AppConfig subclass, use 
that as the config for this app. Then app authors can stop littering their 
__init__.py files with default_app_config.

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/18C7BEF5-7FF3-4324-91C0-6C4C8AC3174D%40polytechnique.org.

Re: app_get_config(app_label).path as only way to get an app's path

[Aymeric Augustin]

Hello Christian,

>  if there are more than one packages with the same label?

Two apps cannot have the same label. One of them must be relabeled as described 
in 
https://docs.djangoproject.com/en/2.2/ref/applications/#django.apps.AppConfig.label
 
.

If a pluggable app wants to access its own app config even if it was relabeled, 
there's no direct API to do so, but it's a one-liner:

from django.apps import apps
app_name = "gdaps.frontend"
app_config = [ac for ac in apps.get_app_configs() if ac.name == app_name][0]

This is pretty much how apps.is_installed(app_name) is implemented. There isn't 
a more efficient way to do this with the current implementation of 
django.apps.Apps.

Perhaps the API of django.apps.Apps could be improved to facilitate such use 
cases. At first sight, there are several options to consider:

- provide an apps.get_app_label(app_name) function that would do the 
translation from app_name to app_label
- accept either an app_label (e.g. "admin") or an app_name (e.g. 
"django.contrib.admin") in APIs that currently only accept an app_label
- etc.

It's been five years since Django 1.7 was released with new app registry. 
During this time, I haven't seen app_label conflicts or relabeling causing big 
problems in practice. (The only vaguely related case I've ever seen was someone 
who copy-pasted an app under a different name. This changed both the app_label 
and the app_name, so even relying on the app_name would fail in that case.)

I'd rather keep things simple here.

Best regards,

-- 
Aymeric.


> On 10 Nov 2019, at 22:12, Christian González  
> wrote:
> 
> Hi all,
> 
> I recently found a problem - maybe I didn't get everything, but the docs
> say that, if you want to get an app's path, you should use
> get_app_config(app_label).path.
> 
> Now, passing the app_label is handy if you have few packages installed,
> but it comes to name clashes if there are multiple packages with the
> same label (but different names).
> 
> I happen to start a bigger project, and there are some packages
> installed like "gdaps.frontend". Maybe one of my plugins later will be
> named "medux.frontend". These two have a different name, but the same label.
> 
> Ok, I can simply do a
> 
> from medux import frontend
> do_something_with(os.path.dirname(frontend.__file__))
> 
> But AppConfig.get_app_config() exactly should exist because I should NOT
> do that.
> 
> How do I get the path of an app in a deterministic way, if there are
> more than one packages with the same label? Is this a bug?
> 
> Thanks,
> 
> Christian
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/c8e6e354-7d02-835e-cd62-5d38102bd374%40nerdocs.at.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/BE8FAA8F-BE28-4DDB-B36C-3CD9C5DC6164%40polytechnique.org.

Re: Allowing numbers in the top level domain

[Aymeric Augustin]

Hello,

Also, as far as I know, the URLValidator is intended to catch common
mistakes of people typing URLs in text fields rather than to enforce
strictly a standard.

Best regards,

-- 
Aymeric.

Le mer. 30 oct. 2019 à 08:40, Claude Paroz  a écrit :

> Hi,
>
> Could you please tell us a bit more about what the specs say about numbers
> in the top-level domain?
>
> Claude
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/e2587b66-b398-4b74-a17a-a4dbb4bdec29%40googlegroups.com
> 
> .
>


-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mVF3FoQOJfubrbTfv-BY0oqRPwOr2pG%3DOxFeDH7c9%3Dv%2BQ%40mail.gmail.com.

Re: docs request: signals documentation warning

[Aymeric Augustin]

Hello,

Yes, I think the docs should be a better job at preventing people from getting 
signal-happy without grasping all the consequences. I've faced that issue in a 
big (> 1 million LoC) project as well.

Without getting too emotional, I think the docs could say that:

1. Signals are for executing custom logic when an event occurs in code that 
isn't under your control i.e. Django code or third-party code.

2. Dispatching a signal to a handler is an implicit function call. Whenever you 
can replace a signal by an explicit function call (because both the caller and 
callee are in you own code), you should —explicit is better than implicit.

Are these two ideas consensual enough to include in the docs?

(I haven't checked where they could be added and the wording requires some 
polish.)

Best regards,

-- 
Aymeric.



> On 21 Sep 2019, at 13:08, Adam Johnson  wrote:
> 
> Hi Cesar,
> 
> I think the community consensus is exactly where you're at: signals are only 
> useful in limited cases. We've even resisted adding more signals to Django 
> core in recent years.
> 
> Although the Django documentation tends to be quite neutral I think a single 
> paragraph listing the pros and cons of using them could be helpful.
> 
> Have you got a proposed wording? Feel free to open a PR.  
> https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/submitting-patches/
>  
> 
>  . Small documentation edits don't need a ticket, and you can refer back here.
> 
> Thanks,
> 
> Adam
> 
> On Sat, 21 Sep 2019 at 01:09, Cesar Canassa  > wrote:
> Hello,
> 
> I would like to propose a small change to the Django signals documentation:
> 
> https://docs.djangoproject.com/en/2.2/topics/signals/ 
> 
> 
> According to the documentation:
> 
> > Django includes a “signal dispatcher” which helps allow decoupled 
> > applications get notified when actions occur elsewhere in the framework.
> > In a nutshell, signals allow certain senders to notify a set of receivers 
> > that some action has taken place. They’re especially useful when many
> > pieces of code may be interested in the same events.
> 
> I have been using Django professionally for around 10 years and worked in 
> several big projects so far. One very common
> mistake that I see is people abusing the signal system as a way to "decouple" 
> their application. Developers usually try to design
> their apps to be uncoupled and independent or at least having a proper 
> tree-like dependency without circular links. But what usually
> happens is that somewhere in the middle of the road their initial design 
> proves to be flawed and they need to introduce an unwanted
> dependency to another app.
> 
> This is usually where people fire a signal to call the other app instead of 
> importing it.
> 
> These signals can make a big Django project a nightmare to work on, as they 
> are extremely hard to debug and to understand what's
> going on. There are very fell places where I can think of that the signal 
> system should be used instead of a simple function call. Reusable apps
> is a good example of a place where you might want to use signals.
> 
> I usually try to prevent my colleges from resorting to signals during our 
> sprint reviews and meetings. I explain my reasoning, send them
> blog posts, etc but we are humans and sometimes these discussions can get a 
> bit tiring and emotional. The Django documentation is
> the most authoritative source I can think of, so if the signals documentation 
> included a warning about the signal abuse and when it should
> be used and when it shouldn’t I could then refer people to the documentation. 
> That would help a lot as I believe fewer people would
> be more likely to abuse the signal system
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/c953406d-03bf-4918-9519-8247fd8af2a7%40googlegroups.com
>  
> .
> 
> 
> -- 
> Adam
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.

Re: Redis cache support in core

[Aymeric Augustin]

Hello,

This is an interesting point — and one we tend to forget too often.

The documentation for memcached says that it doesn't support Windows:
https://github.com/memcached/memcached/wiki/Install#installation

The documentation for redis says that it supports Windows, with some
limitations, and that there's an effort by Microsoft to lift these
limitations:
https://redislabs.com/ebook/appendix-a/a-3-installing-on-windows/a-3-1-drawbacks-of-redis-on-windows/

So the Windows situation seems slightly better with Redis than with
memcached, even if it isn't ideal.

Best regards,

-- 
Aymeric.

Le jeu. 5 sept. 2019 à 15:29, Matthew Pava  a écrit :

> I’d just like to point out that Redis support on Windows is limited at
> best. All other technologies that Django uses, as far as I can recall, do
> support Windows.
>
>
>
> *From:* django-developers@googlegroups.com [mailto:
> django-developers@googlegroups.com] *On Behalf Of *Jacob Rief
> *Sent:* Thursday, September 5, 2019 1:33 AM
> *To:* django-developers@googlegroups.com
> *Subject:* Re: Redis cache support in core
>
>
>
> I'm also in favor on having it as part of the core, since memcache is also
> supported.
>
>
>
> One of the nice features Redis provides, is the possibility to invalidate
> one or more cached object by using a wildcard key.
>
> It namely is the method delete_pattern() added by django-redis-cache to
> the given Django caching backend. That
>
> (or a similar method) then should be part of the other Django caching
> backends as well, such as the dummy cache or in-memory cache.
>
>
>
> Jacob
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJ_HXxo9jpr-%3DFmLXcYwwR6tdn-NVDfrEoo-qYnB-vQwKv9O%3DA%40mail.gmail.com
> 
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/b9357775f9684aa888091bce83f2852b%40iss2.ISS.LOCAL
> 
> .
>


-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mU9m23CWkGrvEETxOhyCFTsU-uP1dS0qnn%3DvPXfemTdqg%40mail.gmail.com.

Re: Creating a new "Triage & Review Team"

[Aymeric Augustin]

Yes, I'm in favor of trying this.

-- 
Aymeric.

Le mer. 14 août 2019 à 15:06, Josh Smeaton  a
écrit :

> I think that's sensible, including making the team public.
>
> > This struck me as a little absurd: if we weren't going to dissolve it,
> Nick should definitely be part of the old "Django Core".
>
> It's not done yet, and while we're in this limbo stage, we're potentially
> missing out on some good maintainers before a decision has even been made.
> You can still put forward names for the core team if you think they'd make
> good members,  though I'd understand your hesitation.
>
> On Tuesday, 13 August 2019 05:12:51 UTC+10, Carlton Gibson wrote:
>>
>> Oh, one more thing. I'd like to make this a "Public" team. I think there
>> should be some visibility and recognition for the folks who work so hard on
>> keeping Django strong.
>>
>> As part of that I'd add it to the Teams list on djangoproject.com, and
>> maintain a list of previous members, for those who had stepped down.
>>
>> C.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/8b5b205e-56ca-44dc-bfd8-8a22de90be60%40googlegroups.com
> 
> .
>


-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mW-b-o3GEt9D2DALC%2BVFeV-DKN1tSgVr1RBFuTfErQATw%40mail.gmail.com.

Re: Ideas for a new DEP on long-term Django API compatibility

[Aymeric Augustin]

Hello Pascal,

As we're getting to the end of useful arguments here, I'd like to ask you to 
step back for a minute and take a calm look at what you're writing. Imagine you 
were receiving such messages rather than sending them. Would you want to spend 
more time collaborating with the person addressing you that way and to receive 
more such messages? Perhaps this is another reason why you aren't getting as 
much support as you'd hope.

You did your best to provide nuanced feedback on a debatable 
backwards-compatibility issue — current behavior is obviously wrong but can't 
be fixed without a hard compatibility break:
> some wild guess / wishful thinking

You spent thousands of hours thinking about what would be best for users of the 
framework — getting occasional thanks and only occasional abuse, as we're lucky 
to have a positive community:
> what is practical for core developers is NOT the same as what is practical 
> for users

You don't need backwards compatibility because you're comfortable living on the 
edge but you always take the time to create deprecation paths according to the 
compatibility policy:
> even if it means creating a task force just for keeping a slight eye on 
> compatibility shims

You do your best to discuss positively with an agressive contributor who 
started with a strongly-worded blog post and doesn't seem to process the 
responses he's getting:
> All that to face red herrings


Here's literal belittling:
> *your* little version of Django


And the cherry of the cake, stating core devs are secret sadists:
> Punishing users


This isn't how we hold discussions here. You need to take a clue from how other 
people interact on this mailing list. Then you can put your energy and ideas to 
good use for improving Django.

>From a pragmatic perspective, putting people off isn't a good way to get them 
>to cooperate with you.

For example, Luke was open to amending the deprecation policy to better reflect 
reality. You should have suggested some concrete changes, erring on the side of 
caution. Instead you chose to repeat the same argument, except more 
agressively. This doesn't get us any closer to a documentation patch. In fact 
this reduces the likelihood that someone will choose to spend time writing that 
patch.

You last email sounds like you're throwing your toys and walking away because 
you didn't get full agreement with your position. However, that outcome was 
largely predictable when you were bringing up a position that diverges so much 
from the current practice. To create progress, it's up to you to create 
consensus around small steps and to create the corresponding pull requests, in 
that order.

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/E1632450-C0E8-410F-A3BB-03B5FAD94C13%40polytechnique.org.

Re: Django LTS support time

[Aymeric Augustin]

Hello,

Actually an individual attempted this for Django two years ago: 
https://web.archive.org/web/20170710090735/https://djangolts.com/ 


The website disappeared after a year. I don't know what happened. The quick 
death suggests there wasn't a ton of demand, though.

-- 
Aymeric.



> On 14 Aug 2019, at 15:13, Josh Smeaton  wrote:
> 
> I don't think the DSF has the capacity or the will to run a business offering 
> paid support contracts.
> 
> But nothing is stopping an enterprising individual or company from doing so. 
> All security patches are made public (eventually) and backporting fixes would 
> be fairly low effort. https://railslts.com/  is prior 
> art in this space.
> 
> I'm incredibly surprised no one has started up a business to do this for 
> Django.
> 
> On Tuesday, 13 August 2019 10:32:21 UTC+10, 1337 Shadow Hacker wrote:
> Actually I'm pretty sure it could be done even if DSF kept a profit, to 
> re-inject it into other developments for exemple. AFAIK the major difference 
> between non-profit and company is that you don't own it and as such you 
> cannot take dividends out of it personally. IMHO everybody would benefit if 
> DSF did more commerce and was stronger by that mean. I could sing an ode to 
> commerce but that would be much off-topic.
> אורי if you're looking for commercial extension you can certainly hire 
> someone for that for the time being, Patryk's company seems to have willing 
> developers, or hire someone to face the debt in your codebase, which Patryk's 
> company will probably recommend for the same budget. Sorry for being so 
> binary but unless another idea emerges from this topic...  it seems pretty 
> cornered at this point, sorry for not being more helpful.
> 
> Best of luck
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/ce923775-e684-43ad-ae16-e6db236e04d7%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/E8C7A3C7-B98A-4097-84C3-99B2DE03C045%40polytechnique.org.

Re: Proposing development discussion forums

[Aymeric Augustin]

Hello,

I'm in favor of trying Discourse.

-- 
Aymeric.



> On 10 Aug 2019, at 05:03, Andrew Godwin  wrote:
> 
> Hi everyone,
> 
> This might be slightly controversial, but I would like to propose that we 
> have a forum for discussing Django development (and potentially user 
> support), alongside the mailing list and maybe, eventually replacing it.
> 
> My full reasoning is below, but in short, it would be more accessible for new 
> users, have a better UI, give us the ability to moderate away problematic 
> posts, be better for privacy, and still allow email-based interaction.
> 
> At DjangoCon AU, the opening keynote was an invited speaker from the Rust 
> community (E. Dunham, https://twitter.com/QEDunham 
> ). I invite you to watch the full talk if you 
> are at all interested in how another language handles their community 
> (https://www.youtube.com/watch?v=TW7PxyrCBR0 
> ), but the takeaway for me was 
> their use of forums rather than mailing lists.
> 
> The Django mailing lists were an excellent choice when Django began, but I 
> feel they have aged out of the modern Web somewhat. The user interface for 
> accessing them is particularly poor, which makes it particularly bad for new 
> contributors.
> 
> In addition, when looking at how to organise the effort to help bring async 
> into Django, something more dynamic, and more segmented, than mailing lists 
> would be incredibly useful. I don't want to drown out the list in specific 
> discussions of how to port certain features, but we need to have those 
> discussions somewhere permanent and asynchronous (so not IRC).
> 
> The mutability of a forum is also not to be overlooked - as well as allowing 
> things like pinned posts and post edits for small issues (or a living header 
> on a long discussion topic), it also allows for permanent removal of things 
> that break the Code of Conduct. On the people front, it also allows people to 
> post without their email being public, allows for name changes, and provides 
> for someone's right to be forgotten via anonymisation of prior content.
> 
> Now, I'm not suggesting we kill the mailing list and switch over or anything 
> like that; instead, I suggest we run an instance of Discourse as a test, and 
> use it as the primary discussion area for async work, as well as anything 
> else that people want to discuss - with the expectation that anything 
> important still goes out to this mailing list.
> 
> Why Discourse? Apart from being a mature, open source forum project, it's 
> also very fully featured, and even supports subscribing and interacting with 
> the forum over email, so it can still fit into an email-based workflow. There 
> are also plenty of small niceties, like the option to have it hosted for us 
> via a paid service, or the ability to use GitHub for login rather than 
> requiring a separate username and password. It also helps that Rust seems 
> quite happy with it.
> 
> I'm mostly asking for the "temperature of the room" on this one - if we get 
> some small objections, I think a trial period is still worthwhile. If there 
> are major objections, then I'd like to ask people what their alternative 
> suggestions are for solving this sort of communication.
> 
> Do I think this would replace the mailing list? Not in the short term, but 
> maybe if it takes off and we all like it better. I personally would interact 
> with django-developers a whole lot more if I could just subscribe to certain 
> topics (rather than trying to emulate that with an email filter as I do 
> now!), and honestly the same thing for django-users. That said, I also 
> recognise that diluting the support/discussion pool is not exactly an 
> attractive idea, which is why I'm asking for input!
> 
> Thanks,
> Andrew
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAFwN1urJmpwTYqXSF_L57hDLL5-9T4Bkvj9Sb9p8na_-_YWEgA%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/36310CA0-594B-4EE0-81C1-1C854E5C8A57%40polytechnique.org.

Re: Make Development More Accessible

[Aymeric Augustin]

Hello John,

This was discussed before, when we moved from self-hosted svn to GitHub-hosted 
git, but I'm not sure there are public archives of all discussions.

As far as I remember, the main points to tackle are:

1. Does GitHub allow "anonymous triage" i.e. labelling, closing, and reopening 
issues by non-committers? I think there was a recent announcement in this area. 
I didn't check the details. Previously, bot-powered workarounds were suggested, 
but they wouldn't provide a good user experience. You want discoverable 
buttons, not a cheat sheet of magic comments.

2. Does the GitHub UI scale to thousands of issues? In theory, any 
classification system can be reproduced with namespaced labels e.g. 
"component:ORM", "status:ready-for-checkin", etc. In practice, it's unlikely to 
be as convenient as what currently exists on Trac.

Perhaps it's just me, but I always found GitHub issues hard to use when I had 
more than on page of issues. Indeed, at that point, I need a labelling system 
to filter issues. Then I need to keep all the rules of that system in my head 
instead of having the UI guide me — and prevent me from infringing the system...

3. How do we migrate issues history from Trac to GitHub? Preserving comment 
authorship doesn't seem obvious, especially for authors who don't have the same 
username on Trac and GitHub or authors who don't have a GitHub account.

Initially an effort was made to sync usernames of core devs between Trac and 
GitHub to prevent security problems but that's a small subset of contributors.

4. Are we still able to export everything from GitHub and move on to the next 
thing? Perhaps there's an obvious answer. I didn't look. Usually Django takes a 
pragmatic position: we won't reject GitHub outright because it isn't open 
source. However, we wouldn't want to lock ourselves into a platform we don't 
control.

Who would have bet, three years ago, that GitHub would be the property of 
Microsoft today? What if Microsoft sells it to Oracle in three years? It's nice 
to keep our options open :-)

We put the code there because we were confident that we could pull the git 
history. Then everyone started using pull requests, which was likely a good 
thing, but wasn't really planned or thought through, and I don't think we can 
export PR comments meaningfully. GitHub did some good vendor lock in there.

5. How do we preserve links to SVN commits? Currently, they're redirected on 
https://code.djangoproject.com/ with this nginx rule:

rewrite ^/changeset/(\d+)/?$ https://www.djangoproject.com/svntogit/$1/ 
permanent;

and then redirected again by this application:

https://github.com/django/djangoproject.com/tree/master/svntogit

It would be nice to preserve these links in issues copied from Trac to GitHub, 
which probably means pre-processing comments to rewrite links.

There may be more, but that's what comes to mind!

A process DEP 

 is the way to go to propose this change.

Best regards,

-- 
Aymeric.



> On 7 Aug 2019, at 08:24, John Gooding  wrote:
> 
> I'd like to propose moving Django issues to github and make a real decision 
> on it here in this thread. If there has been a recent discussion on this I 
> apologize, but searching for issue tracking / github links to about every 
> thread ever posted here.
> 
> I believe this would lower the barrier to entry and to help promote community 
> involvement. People are already there, people already use it, and we already 
> do pull requests there. Now I could be wrong here, but I also feel that it 
> would improve and promote discussion about changes and feature additions to 
> Django, because right now they are pretty hidden away in the current system. 
> 
> I'd also like to see the inclusion of a "discussion" label or similar for 
> issues. I think many of the conversations here on this forum would be much 
> better off as github issues. I see a lot of great stuff, and it's not clear 
> at all what the status is, has it moved forward, been officially denied? etc. 
> If they are github issues they will have definitive resolutions, whatever it 
> may be, and links to relevant code, PR's etc if needed.
> 
> I think there is a huge amount to gain by consolidating the ticket system and 
> many of the discussions on this forum into github's issue tracker. I don't 
> see any reason why it wouldn't be wroth the effort, and we only have much to 
> gain as a community from it. But that's just my 2 cents. I'd love to hear 
> what others think, for or against it.
> 
> John
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the

Re: Ignoring accept_language in localization middleware based on prefixed URL patterns

[Aymeric Augustin]

Hello,

Yes, I think this change makes sense, assuming no unexpected difficulties arise 
in the implementation.

Best regards,

-- 
Aymeric.



> On 6 Aug 2019, at 10:58, Janez Kranjc  wrote:
> 
> Hi guys! I’m Janez Kranjc, I’ve been using Django for a bit now - since 1.3 
> and I’ve recently come across and issue that’s been bothering me in multiple 
> projects that I’m working on.
> 
> Localization middleware ignores the accept language header if ANY url pattern 
> exists that is i18n prefixed regardless of the current URL of the request.
> 
> So the problem is the following: I have some URLs that are prefixed, and a 
> lot that are not (such as all of the API endpoints). I sometimes need to 
> return some translated strings in the API as well and for that I rely on the 
> accept-language header. However in the middleware it is ignored because an 
> unrelated part of the project has an i18n prefixed url pattern.
> 
> Another way to look at the problem is this:
> 
> Let’s say I have a SPA that uses i18n on its API endpoints and you rely on 
> accept-language to serve the responses in the correct locale. I then decide 
> to add a new app to your django project - a sales page. Instead of relying on 
> accept-language I wish to have i18n prefixed URLs (maybe for SEO reasons or 
> whatever). Suddenly the behaviour of the API changes even though I’ve made 
> changes to an entirely different part of the project.
> 
> Would it not make more sense for the middleware to check if the current URL 
> pattern (the one that the request URL resolves to) is prefixed or not.
> 
> The way I see it, this should be changed:
> 
> i18n_patterns_used, prefixed_default_language = 
> is_language_prefix_patterns_used(urlconf)
> 
> Instead of checking the entire urlconf it should only check the current 
> request URL and see if it resolves to a pattern that is i18n prefixed.
> 
> To get around this I need to use a custom localization firmware in a lot of 
> my projects. I would like to hear the devs’ opinion regarding this.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/1b315c0b-9200-417b-816a-dd3c3b90ec67%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/09EBC24C-868E-4197-AEAE-87CB82787DDE%40polytechnique.org.

Re: Translation templatetag aliases

[Aymeric Augustin]

Hello,

I'm in favor of adding support for {% translate %} and {% blocktranslate %} and 
switching the Django documentation to use these for two reasons:

- As stated by Mike, the mental associations that "block trans" creates for 
those who identify as trans are just bad — I can't believe it took us 12 years 
to notice :-/
- Even if we leave that aside, I'm not sure saving 4 characters (8 with the 
closing tag) is really worth the uncommon abbreviation of "translate".

Since this change brings mostly a social benefit rather than a technical 
benefit, we could keep the {% trans %} and {% blocktrans %} aliases forever. 
Also, this could minimize arguments between those who recognize the benefits of 
such changes and those who don't, as we've seen when we changed master / slave 
to primary / replica.

In my opinion, such debates mostly reflect the political rift that appeared in 
many Western countries in the recent years. It's completely predictable that 
they spill into our community. Since they aren't our main focus, we're trying 
to avoid spending too much time there. But we can't escape the fact that we're 
making Django first for people writing software, then for computers running 
that software.

As a community, we chose to state in our Code of Conduct that "we strive to be 
a community that welcomes and supports people of all backgrounds and identities 
[including] sexual orientation, gender identity and expression". I believe that 
the change proposed here is in line with this statement. I know that some 
community members will feel that we're doing too much here — and that others 
feel that we aren't doing enough in general — which is why I'm referencing 
something we already agreed upon.

Best regards,

-- 
Aymeric.



> On 26 Jul 2019, at 13:17, 'Mike Hansen' via Django developers (Contributions 
> to Django itself)  wrote:
> 
> Hello all,
> 
> Recently I had a member of my team bring up that it was uncomfortable
> for them to work in parts of our codebase where they regularly had to
> see "blocktrans" in the template files.  To make our work environment
> more inclusive, I wrote a Django package 
>  which adds "translate" 
> and 
> "blocktranslate" templatetag aliases so that we could update our own 
> internal templates to use these.
> 
> We felt that this change would be in line with the Django community
> so I made a ticket and pull request to Django at
> https://code.djangoproject.com/ticket/30585 
>  .  The ticket was closed as
> "wontfix", and it was mentioned that I should bring it to django-developers
> if I wanted to make further progress on the ticket.
> 
> Thanks,
> --Mike
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/a43ec0ab-f73a-4c11-8fc1-b05d081b24c3%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/EF4EE680-BBA1-40EB-979E-2671349186D6%40polytechnique.org.

Re: #30646 - close_if_unusable_or_obsolete fails to close unusable connections

[Aymeric Augustin]

Hello Daniel,

In the PR you propose to ping all database connections at the beginning and at 
the end of every HTTP request. I'm positive that this will have a significant 
performance impact. It's quite common to have secondary database connections 
that are rarely used, not scaled to be pinged twice per HTTP request, and 
sometimes slow to connect to.

I understand that you solved this problem in the context of Zalando. It's 
unclear to me that your solution easily generalizes to every Django-based 
system.

Surely, a serious, concrete proposal will be seriously considered. What we're 
missing at this point is a proposal that we can discuss!

As you know, currently, Django handles disconnects optimistically. It accepts 
to return one 500 error every time a worker loses its database connection. You 
can reduce this to one 500 error every time a worker loses its database 
connection while it's processing a request (rather than waiting for the next 
request) by disabling persistent connections with CONN_MAX_AGE = 0 but the 
performance hit is likely not worth the reliability improvement. On the ticket 
you say that you're seeing "a number of requests failing with 500s on every 
failover". Is that one request per thread or several requests in the same 
thread? The latter would be a bug.

You'd like to handle disconnects pessimistically. Since middleware runs outside 
of per-view transaction handling (when ATOMIC_REQUESTS is enabled), I think you 
can do anything you need in a middleware, which is likely what you have done 
(unless you had to patch Django?) A third-party package would be a good way to 
demonstrate the feature and also to document its pros and cons.

Best regards,

-- 
Aymeric.



> On 18 Jul 2019, at 19:03, Daniel Neuhaeuser  
> wrote:
> 
> Hello,
> 
> I'd like to start a discussion about the ticket I created 
> https://code.djangoproject.com/ticket/30646 
>  that was closed as won't fix.
> 
> Essentially the ticket is about how to handle database disconnects. These can 
> happen for a variety of reasons such as network problems or failover from a 
> master to a replica. It took us quite some time until we figured out how to 
> handle this reasonably and we now do this through a combination of:
> 
> * Checking whether connection is usable (what the ticket and the associated 
> PR do)
> * Exponential backoff and retry
> * A middleware that detects database errors that are likely transient and 
> turns them into 503 with Retry-After in situations where retrying would 
> introduce unreasonably high latency
> 
> Given that Django aims to be a full-stack framework for "perfectionists with 
> deadlines", I think this is complexity that should move into the framework. 
> At the very least it should be seriously considered.
> 
> SQLAlchemy, while not addressing the problem fully, covers this problem in 
> detail in their documentation 
> https://docs.sqlalchemy.org/en/13/core/pooling.html#dealing-with-disconnects 
> .
>  The "pre ping" approach mentioned there would correspond to what I've 
> proposed in the ticket and the associated PR. 
> 
> 
> Kind regards,
> Daniel
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/cc0bd0bb-38f9-4ce3-84f5-d45c415d390c%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/C65D803B-9A3E-454B-A609-8CB248C2B28C%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Pre-proposal: adopt dj-database-url as a DEP 7 official project

[Aymeric Augustin]

Hello,

In the short term, I'm in favor of moving dj-database-url to the Django 
organization.

I have only one concern. dj-database-url supports a few third-party database 
backends: 
https://github.com/jacobian/dj-database-url/blob/004150c17e1519be408f87e0a2d16a8633cad89f/dj_database_url.py#L34-L45
 

 I wouldn't want this to be interpreted as an official recommendation about 
which backend is best for MSSQL. This isn't enough to make dj-database-url 
unsuitable as an official project but it's worth keeping in mind. Perhaps 
"mssql" should be deprecated in favor of "mssql-X", "mssql-Y" and "mssql-Z" to 
put every alternative on an equal footing?

In the long term, I agree that dj-database-url could be deprecated if Django 
gets equivalent functionality built-in. This will require a bit of creativity 
to avoid hardcoding mappings for third-party database backends. Probably users 
of third-party backends will have to add a line to register a URL scheme with a 
third-party backend.

Best regards,

-- 
Aymeric.



> On 18 Jul 2019, at 18:45, Jacob Kaplan-Moss  wrote:
> 
> Hi folks -
> 
> I’d like to gauge interest in adopting dj-database-url
> (http://github.com/jacobian/dj-database-url 
> ) as an official project
> (https://github.com/django/deps/blob/master/final/0007-official-projects.rst 
> ).
> This is my pre-proposal. I think dj-database-url is very widely-used, and
> scratches a specific but annoying itch. Maintenance burden is minimal, but
> important.
> 
> What do other folks think? DEP 7 asks us to
> 
> > solicit feedback from the community and work out if there is rough agreement
> > that the project is a good thing for Django to adopt, particularly focusing 
> > on
> > any alternative approaches to the same problem and the relative merits of 
> > them,
> > including code design, scalability, alignment with existing Django design 
> > and
> > philosophy, and having an available development and maintenance team.
> 
> So - thoughts?
> 
> I'm volunteering to be the shepherd, but do intend to form a larger team.
> 
> I've also opened https://github.com/jacobian/dj-database-url/issues/120 
> 
> as another place for this discussion. Please feel free to comment either
> place.
> 
> [Backstory, and why this is living on my github right now. @kennethreitz was
> looking for a new maintainer, and reached out to me to see if Django might be
> interested. I thought perhaps so, and offered to take over maintainership in 
> the
> meantime while we work through the DEP 7 process. If it turns out that an
> Official Project is the wrong home for this, I'll either maintain it long-term
> or transfer it again.]
> 
> Jacob
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/c4044098-d97f-44be-8859-b05cb594a7be%40Spark
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1879FD6A-D91B-4759-8E32-E4E719BC2928%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Application specific middleware support

[Aymeric Augustin]

Oops! Scratch that, let's try again...

So Django doesn't create a relationship incoming HTTP requests and 
applications. You could try to create such a relationship, based on where 
URLconfs or views are defined, but you'll quickly discover tons of edge cases — 
starting with decorators — that make this unreliable in practice. You'd be 
going against Django's #1 design philosophy: loose coupling.

If you have control over the project and you know where the URLconf for the app 
is included, you can do: `if request.path.startswith("/your_app/"): 
do_stuff()`. If you're building a pluggable app and you don't have control over 
the project, you can write a decorator and apply it to every view.

Best regards,

-- 
Aymeric.



> On 7 Jul 2019, at 21:28, Adam Johnson  wrote:
> 
> Aymeric, are you sure about that? I can't find the code, and the docs for 
> current_app say it's noto set by Django itself: 
> https://docs.djangoproject.com/en/2.2/ref/request-response/#django.http.HttpRequest.current_app
>  
> <https://docs.djangoproject.com/en/2.2/ref/request-response/#django.http.HttpRequest.current_app>
> 
> Also middleware can't reliably make use of routing information except in 
> process_view, because if another middleware changes request.urlconf, it will 
> be "re-routed".
> 
> On Sun, 7 Jul 2019 at 19:51, Aymeric Augustin 
>  <mailto:aymeric.augus...@polytechnique.org>> wrote:
> Hello Kapil,
> 
> When you talk of a "URL served by an app", I suppose you're referring to URL 
> namespaces, which provide a request.currrent_app attribute associating an 
> HTTP request to an app. As far as I know, this is the only association of a 
> request and an app that Django defines.
> 
> Django's documentation encourages URL namespaces for all pluggable apps. 
> Unfortunately, adoption is limited because the benefits aren't sufficiently 
> clear for many users. That said, if you need to add a middleware to an app 
> that's under your control, you can give this app a URL namespace. Once you've 
> done that, in your middleware, you can run your logic only if the request is 
> routed to your app: `if request.current_app == "your_app": do_stuff()`.
> 
> As to why this isn't documented, most likely, that's because nobody ever had 
> this use case and took the time to submit a documentation patch. That's a 
> good contribution opportunity :-)
> 
> Best regards,
> 
> -- 
> Aymeric.
> 
> 
> 
>> On 7 Jul 2019, at 17:42, Kapil Garg > <mailto:kapilgarg1...@gmail.com>> wrote:
>> 
>> Hi, I am working on one django project and recently i had the requirement 
>> for a middleware. But i don't want this middleware to hook to every url 
>> served by the whole project but instead, only to one of the apps. I tried to 
>> look on the internet but everywhere there are hacks to implement it but not 
>> a in-built support from django. So i was curious why django doesn't provide 
>> support for app specific middleware or if it does then why there isn't any 
>> neat documentation about it. Thanks
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com 
>> <mailto:django-developers+unsubscr...@googlegroups.com>.
>> To post to this group, send email to django-developers@googlegroups.com 
>> <mailto:django-developers@googlegroups.com>.
>> Visit this group at https://groups.google.com/group/django-developers 
>> <https://groups.google.com/group/django-developers>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/1b8a2339-50fd-4b81-973f-9474787e3c6e%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/1b8a2339-50fd-4b81-973f-9474787e3c6e%40googlegroups.com?utm_medium=email&utm_source=footer>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To post to this group, send email to django-developers@googlegroups.com 
> <ma

Re: Application specific middleware support

[Aymeric Augustin]

Hello Kapil,

When you talk of a "URL served by an app", I suppose you're referring to URL 
namespaces, which provide a request.currrent_app attribute associating an HTTP 
request to an app. As far as I know, this is the only association of a request 
and an app that Django defines.

Django's documentation encourages URL namespaces for all pluggable apps. 
Unfortunately, adoption is limited because the benefits aren't sufficiently 
clear for many users. That said, if you need to add a middleware to an app 
that's under your control, you can give this app a URL namespace. Once you've 
done that, in your middleware, you can run your logic only if the request is 
routed to your app: `if request.current_app == "your_app": do_stuff()`.

As to why this isn't documented, most likely, that's because nobody ever had 
this use case and took the time to submit a documentation patch. That's a good 
contribution opportunity :-)

Best regards,

-- 
Aymeric.



> On 7 Jul 2019, at 17:42, Kapil Garg  wrote:
> 
> Hi, I am working on one django project and recently i had the requirement for 
> a middleware. But i don't want this middleware to hook to every url served by 
> the whole project but instead, only to one of the apps. I tried to look on 
> the internet but everywhere there are hacks to implement it but not a 
> in-built support from django. So i was curious why django doesn't provide 
> support for app specific middleware or if it does then why there isn't any 
> neat documentation about it. Thanks
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/1b8a2339-50fd-4b81-973f-9474787e3c6e%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/DEB7E81B-0043-4754-9D23-A6A234F9BCC6%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Discussing improvements of django.setup()

[Aymeric Augustin]

Hello Pakal,

Making django.setup() idempotent

In https://code.djangoproject.com/ticket/28752 
, you say:

> I've been bitten numerous times by the impredictable behaviour of django when 
> django.setup() was called numerous times.
> In the old days I had exceptions, now it's mainly subtle breakages of logging 
> configuration.

This is fuzzy for a bug report. Ideally, you should say:

- What you're doing (minimal reproduction instructions, within supported use of 
Django)
- What results you're expecting
- What results you're getting

Anyway it suggests that the problem is about logging. In this comment 
 I 
said that the only possible bug is that `configure_logging()` isn't idempotent. 
Your answer 
 confirms 
that the issue is really about logging. Let's fix configure_logging(), then!

Barring new arguments in favor of the proposed approach, which involves nested 
locks, I'm -1. Nested locks create a possibility of deadlocks. See my comment 
on the PR for details.

Making django.setup() tweakable

Again, the rationale is fuzzy. I'm seeing two use cases:

- pytest-django
- django-compat-patchers

One more or one less monkey patch isn't going to make a difference to these 
projects ;-)

I'm ready to accept that there's a need for projects that aren't big hacks but 
I need to see it explained clearly and specifically. I'd really like to see a 
feature request in the form of:

- I want to do X, which is a reasonable thing to do and stays within supported 
use of Django, except I need to override django.setup()
- Here's what the code looks like with a workaround (monkey-patching, probably)
- Here's what the code would look like with the improvement I'm asking for 
(hopefully much better)
- Here are the alternatives I considered and why I rejected them
- Other use cases also include Y and Z

I'm asking this because concrete use cases will put us in a better position to 
find the best solution.

Best regards,

-- 
Aymeric.



> On 27 Jun 2019, at 23:26, Pkl  wrote:
> 
> Hello everyone,
> 
> I'm bringing here, for broader review and discussion, the subject of making 
> django setup more "tweakable":
> https://code.djangoproject.com/ticket/30536
> https://github.com/django/django/pull/11435
> 
> There is indeed a need for doing pre/post operations at that crucial moment 
> of the framework lifetime, especially to apply compatibility shims 
> (greening-related, API stability-related...) and setup test scaffolding, 
> stubs etc.
> 
> A separate PR has been created to handle threadsafety/idempotence 
> (https://github.com/django/django/pull/11440) when multiple actors try to 
> setup django concurrently, but a discussion remains on how to potentially 
> allow users to specify an alternative setup 
> 
> I originally did a POC with a new django setting, but if we want to have full 
> control, even settings might not be normally loaded at that stage. So maybe, 
> as apollo13 suggested, an environment variable, like DJANGO_SETUP_CALLABLE, 
> complementing the DJANGO_SETTINGS_MODULE one?
> What are your thoughts about this ?
> 
> thanks,
> regards,
> Pakal
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/0206c98c-199f-49f9-9ff7-cba736e71224%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4673AD3D-4EDF-44FB-931F-B3E231D6311D%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Ideas for a new DEP on long-term Django API compatibility

[Aymeric Augustin]

Hello Pakal,

If I understand correctly, you're proposing that:

1. The Django project should maintain a higher level of backwards compatibility.
2. This would be easier to achieve outside the main codebase, in a submodule or 
in a separate repository.
3. This would reduce the amount of work required for maintaining Django 
(triaging tickets, fixing bugs, adding features).

Backwards-compatibility is a major concern for the development of Django, as 
you can see from in contribution documentation, in the archives of this mailing 
list, in DEPs, etc. Maintaining more compatibility with the same effort or less 
should be uncontroversial. Now the question is to get consensus that your 
proposal will indeed achieve the stated goals.

I'm quite skeptical of the "same effort or less". The more behaviors Django 
maintains — and you're proposing essentially to maintain all behaviors that 
existed at all previous releases — the more complexity accrues. You can't solve 
that just by shuffling deprecation warnings or compatibility imports in another 
module (and it seems to me that it would be harder to maintain, for the reason 
the Python ecosystem usually eschews monkey-patching: non local effects). 
Keeping old ways around forever increases the barrier to mastering Django for 
people who haven't been writing Django code for ten years like you and me. It 
also increases the scope on which compatibility must be maintained, and that 
can get in the way of making Django better.

One way to frame the debate is: does Django aim at being a fossilizing 
ecosystem or a living one? Up to this point, we've chosen to maintain a living 
ecosystem. At a given point in time, all maintained libraries support 2 or 3 
versions of Django (latest, previous, LTS). Things change a bit; unmaintained 
libraries that don't adapt fall out of favor; new libraries build upon the 
experience of previous ones and try to do better. One can see this as progress 
and a way to keep up with the moving web ecosystem. One can also see this as 
useless churn. As you call it "walk or die", it's pretty clear which camp 
you're in :-) I'd say there's truth in both!

As an experiment, let's just consider the first backwards-incompatible change 
from the latest release: 
https://docs.djangoproject.com/en/2.2/releases/2.2/#admin-actions-are-no-longer-collected-from-base-modeladmin-classes
 

 (I'm skipping the database backends changes because they're explicitly out of 
the backwards-compatibility policy, but we started documenting them for the 
maintainers of third-party database backends.) This is a good example. Django 
behaved in a non-Pythonic way; it didn't respect the Principle of Least 
Astonishment. Since we believe Django is alive and will have infinitely more 
future users than past users, we make the change. How can we maintain 
backwards-compatibility for this? We can try to detect if subclasses have all 
the actions of their parent classes and, if not, raise a warning. But then we 
raise inappropriate deprecations warnings in legitimate use cases where a 
subclass mustn't have some actions of its parent class, which is probably the 
use case for which this change was requested.

Experimenting with a third party module like you did is absolutely the way to 
go. You're already ahead of the usual advice :-) Keep in mind that the barrier 
for making a third-party project an official Django project or to merging it in 
Django is very high. With 1 fork and 2 stars, django-compat-patcher doesn't 
pass it yet.

Writing a DEP to formalize arguments on both sides is a valid idea. Don't 
forget the other side. In your freecodecamp article, you're spending two lines 
on the downsides of Compat Patchers, and this is not enough. How would Django 
communicate to users which backwards-incompatible changes are covered by Compat 
Patchers and which aren't? How would we respond to users asking for a Compat 
Patcher for a backwards-incompatible change for which it's impossible? 
Eventually, will every library document the list of Compat Patchers it requires 
or it's incompatible with?

As you can see, there's more than "blinding self-absorption" and "harmful 
psychological bias" here. Your freecodecamp article makes valid points. It also 
misses completely what's hard in maintaining backwards compatibility. I wish 
you hadn't found it useful to insult the work put into managing 
backwards-compatibility over the years and the people who did it.

Best regards,

-- 
Aymeric.



> On 29 Jun 2019, at 13:09, Pkl  wrote:
> 
> Hello everyone,
> 
> I'm planning on writing a PEP on long-term API stability for Django.
> Most of the rationale behind this kind of commitment is described here : 
> https://www.freecodecamp.org/news/api-stability-is-cheap-and-easy/
> 
> The idea is that the django ecosystem has suffered a lot, over the past 
> decade, from the 

Re: j'ai une erreur de la fonction def __str__(self):

[Aymeric Augustin]

Bonjour Randa,

Ce n'est pas la bonne liste de diffusion pour ce type de question. 
django-developers traite du développement de Django lui-même, pas de 
l'utilisation de Django.

Je vous encourage à écrire a django-users ou à passer par un site tel que Stack 
Overflow.

Par ailleurs, si vous posez une question en français, assurez-vous que vous 
êtes sur un forum francophone, ce qui n'est pas le cas ici.

Merci pour votre compréhension,

-- 
Aymeric.



> On 22 Jun 2019, at 16:06, randa maghraoui  wrote:
> 
> 
> C:\Users\randa\premier_projet\mysite>python manage.py shell
> Traceback (most recent call last):
>   File "manage.py", line 21, in 
> main()
>   File "manage.py", line 17, in main
> execute_from_command_line(sys.argv)
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\__init__.py",
>  line 381, in execute_from_command_line
> utility.execute()
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\__init__.py",
>  line 357, in execute
> django.setup()
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\__init__.py",
>  line 24, in setup
> apps.populate(settings.INSTALLED_APPS)
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\apps\registry.py",
>  line 114, in populate
> app_config.import_models()
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\apps\config.py",
>  line 211, in import_models
> self.models_module = import_module(models_module_name)
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\importlib\__init__.py",
>  line 127, in import_module
> return _bootstrap._gcd_import(name[level:], package, level)
>   File "", line 1006, in _gcd_import
>   File "", line 983, in _find_and_load
>   File "", line 967, in _find_and_load_unlocked
>   File "", line 677, in _load_unlocked
>   File "", line 724, in exec_module
>   File "", line 860, in get_code
>   File "", line 791, in source_to_code
>   File "", line 219, in _call_with_frames_removed
>   File "C:\Users\randa\premier_projet\mysite\polls\models.py", line 9
> def __unicode__(self):
>  ^
> IndentationError: unindent does not match any outer indentation level
> 
> C:\Users\randa\premier_projet\mysite> python manage.py shell
> Traceback (most recent call last):
>   File "manage.py", line 21, in 
> main()
>   File "manage.py", line 17, in main
> execute_from_command_line(sys.argv)
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\__init__.py",
>  line 381, in execute_from_command_line
> utility.execute()
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\__init__.py",
>  line 357, in execute
> django.setup()
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\__init__.py",
>  line 24, in setup
> apps.populate(settings.INSTALLED_APPS)
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\apps\registry.py",
>  line 114, in populate
> app_config.import_models()
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\apps\config.py",
>  line 211, in import_models
> self.models_module = import_module(models_module_name)
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\importlib\__init__.py",
>  line 127, in import_module
> return _bootstrap._gcd_import(name[level:], package, level)
>   File "", line 1006, in _gcd_import
>   File "", line 983, in _find_and_load
>   File "", line 967, in _find_and_load_unlocked
>   File "", line 677, in _load_unlocked
>   File "", line 724, in exec_module
>   File "", line 860, in get_code
>   File "", line 791, in source_to_code
>   File "", line 219, in _call_with_frames_removed
>   File "C:\Users\randa\premier_projet\mysite\polls\models.py", line 11
> def was_published_recently(self):
> ^
> IndentationError: unindent does not match any outer indentation level
> 
> C:\Users\randa\premier_projet\mysite> python manage.py shell
> Traceback (most recent call last):
>   File "manage.py", line 21, in 
> main()
>   File "manage.py", line 17, in main
> execute_from_command_line(sys.argv)
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\__init__.py",
>  line 381, in execute_from_command_line
> utility.execute()
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\core\management\__init__.py",
>  line 357, in execute
> django.setup()
>   File 
> "C:\Users\randa\AppData\Local\Programs\Python\Python37-32\lib\site-packages\django\__init__.py",
>  line 2

Re: Redis cache support in core

[Aymeric Augustin]

Hello,

Until now, this feature request was always declined. As a consequence, every 
user who wants Redis has to choose between django-redis and django-redis-cache. 
Considering that Redis must be the most popular cache backend these days, I'm 
in favor of providing an off-the-shelf solution in Django itself. It's unlikely 
to be a large maintenance burden. It will "just work".

I don't know if factoring out common functionality between the memcached and 
redis backends will really make them easier to maintain. We'll end up with 
three modules (key-value, memcached and redis), which will be more complicated 
than two. If we had three similar backends, that would be a strong argument for 
factoring out common functionality. With only two backends, it may not be worth 
the complexity.

To move this forwards, my suggestion would be to write a DEP, to flesh out the 
rationale for a built-in solution, and to focus on the breadth of functionality 
the built-in backend would support. My preference would be a basic set of 
features, like the other cache backends. I believe that's what 
django-redis-cache does. This will leave room for third-party packages like 
django-redis to provide more advanced features. To give a concrete example, 
like other cache and database backends, a redis backend could provide 
persistent connections but not implement a connection pool. Until now Django 
has left the management of connection pools to third-party packages.

Best regards,

-- 
Aymeric.



> On 17 Jun 2019, at 17:11, Dulmandakh Sukhbaatar  wrote:
> 
> Hello,
> 
> I would like to work on Redis support in core, and I would like to discuss 
> proper solution for that.
> 
> Redis is getting so popular and almost every modern backend stack uses it 
> someway, therefore I think that supporting it as a cache backend in core 
> would make Django more appealing. A solution I'm proposing is to extract base 
> KV backend from current Memcached and extend it for both Memcached and Redis, 
> and this won't add many new code to the core. Also we'll have base class for 
> KV storage backends.
> 
> Thanks.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3a506133-f08a-4d25-a9c5-099aae3722d8%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/F93AED93-32FC-4BF4-BDA7-48B4A76A8314%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Django ORM Internals

[Aymeric Augustin]

Hello,

I would suggest https://www.youtube.com/watch?v=bgV39DlmZ2U if you'd like
to understand how the ORM is structured in layers.

-- 
Aymeric.

Le lun. 13 mai 2019 à 12:16, Adam Johnson  a écrit :

> Yes there isn't much documented on the ORM internals, but there are other
> resources. The DUTH videos are a great start and inspired me to get started
> contributing to Django. Specifically:
>
>- https://www.youtube.com/watch?v=CGF-0csOjPw
>- https://www.youtube.com/watch?v=-4jhPRfCRSM
>- https://www.youtube.com/watch?v=-1tt0S__kQk
>
> This talk is also on my to-watch list, from James Bennett, he covers a lot
> of internals:
>
>- https://www.youtube.com/watch?v=tkwZ1jG3XgA
>
> As for looking through the code, try looking at django.contrib.postgres
> first - it's interesting as it's all *extensions* to the ORM (custom
> fields, migration operations, etc.) so an easier on ramp than diving into
> the inner code of Query etc. It's what insipred me to make my library
> django-mysql ( https://github.com/adamchainz/django-mysql ) that does the
> same kind of stuff for MySQL/MariaDB.
>
> Then maybe look itno tracing some simple queryset operations, such as
> Foo.objects.all(), and see all the steps that goes through. Understand what
> Query is and how SQLCompiler converts it to SQL that the database backend
> executes.
>
> Hope that helps,
>
> Adam
>
> On Mon, 13 May 2019 at 09:54, Mahdi Zareie  wrote:
>
>> Great, thanks
>>
>> On Monday, May 13, 2019 at 1:17:08 PM UTC+4:30, J. Pic wrote:
>>>
>>> Hi Mahdi,
>>>
>>> I would suggest reading the code and test code in the tests/ directory
>>> for the ORM.
>>>
>>> There might also be some videos from Django Under The Hood conferences,
>>> found some here:
>>>
>>> https://www.google.com/search?q=Django+Under+The+Hood+orm&tbm=vid
>>>
>>> Best
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/81554698-8fad-4fd0-a534-a566725058b3%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> Adam
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAMyDDM00cB5Mz%2Bsec3KpF%3DEF9Z4yXipzMwuag7q%2B5EFA8cch2g%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mUixaF76W9MwN_y4ufvT%3D-hMq4PkuwFZVva4UafktOA9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django Async DEP

[Aymeric Augustin]

> On 9 May 2019, at 22:06, Andrew Godwin  wrote:
> 
> Jannis suggested what I think might be a nicer approach on Twitter, which is 
> to add an async "proxy object" to access methods with, e.g.:
> 
> cache.get("foo")
> cache.async.get("foo")
> 
> This is still explicit but looks less ugly, and gives us a way in future to 
> make some modules transition to async-by-default (by e.g. supplying 
> cache.sync.get). What do you think?


I don't think it makes a significant difference from a readability perspective 
in this example. It does have some advantages, though:

- It could be easier to implement a sync version based on the async one, or 
vice-versa, if each one has its own class. It will probably be more DRY.
- It could also be a bit more usable by developers, especially in IDEs

Also it could provide an alternative solution for the async attribute access 
problem: `model_instance.async.related_field` could be a Future that, when 
awaited, resolves to the related field. So you could `await 
model_instance.async.related_field` instead of having to `select_related`. To 
be honest I didn't investigate all the consequences of this idea. It seemed 
worth mentioning, though, even if it turns out to be impractical :-)

We've always considered that implicit queries on attribute access were an 
intractable problem. I said it on stage an DjangoCon US 2013. I'm now wondering 
if I was wrong all along! In an async ORM context, every time we traverse a 
relation, we could create a Future that would execute the query and resolve to 
its result. This would require one await per SQL query so we'd still get the 
benefit of making queries explicit, although to a lesser extent than with an 
explicit select/prefetch_related.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/258CBCB2-2456-439F-82DF-E8760AE5EEF0%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Django Async DEP

[Aymeric Augustin]

Hello Andrew,

Thanks for your work putting together this plan. Within our constraints, it's a 
good plan.

Regarding templating, I would say it isn't a priority because a developer who 
knows how to parallelize I/O bound operations will prefer (or at least accept) 
to perform these operations in the view, not in the template.

I'm on the fence about the convention for async APIs. I'm not super excited by 
spraying async code with _async prefixes. The namespacing approach would allow 
for cleaner async code. Most Python modules should be either sync or async, not 
a mix of both styles. But I might be underestimating the importance of 
explicitness...

I don't have much else to say on the DEP. It makes a lot of sense. I'm happy to 
see this happening!

Cheers,

-- 
Aymeric.



> On 9 May 2019, at 18:10, Andrew Godwin  wrote:
> 
> Hello everyone,
> 
> While the ASGI patch (https://github.com/django/django/pull/11209 
> ) is maybe the first small step 
> in a long road to async, it's the only real one we could do without a DEP as 
> it purely pulls on existing specs and bugs.
> 
> To that end, I have drafted a DEP (provisionally #0009), "Async-capable 
> Django". It is a summary - and I use that word very lightly - of the rest of 
> the work to be done to make a version of Django that one could call "async 
> capable".
> 
> You can view it here: 
> https://github.com/andrewgodwin/deps/blob/async/draft/0009-async.rst 
> 
> 
> It is approximately 7,000 words long - while it could probably do with some 
> editing, the subject matter means it has to be quite in-depth. Even then, it 
> does not propose an exact plan to follow; instead, it proposes the overall 
> strategy and the high-level ideas about what needs to be done.
> 
> If you are short on time, please read the Foreword and the High-Level 
> Summary, and then jump around using the table of contents to the sections 
> that interest you.
> 
> Feedback on this is very much appreciated; you can either reply here or, if 
> you have comments that would benefit from precise location on the diff, use 
> the pull request here: https://github.com/django/deps/pull/56 
> 
> 
> This is a very complex topic, and I appreciate feedback might take a while; 
> at least a couple of weeks to get everyone's thoughts. If you want a more 
> private channel to discuss concerns or design questions you're not happy to 
> talk about publicly, feel free to email me directly.
> 
> This proposal does not quite fit the DEP template - we never really planned 
> for something of this scale - but I have done my best to make it work; I 
> think we still benefit from discussing and voting on it in the DEP format, 
> but suggestions on how to make it work better are welcome, provided they fit 
> within how Django operates as a community now.
> 
> Finally, if there is positive feedback on this, we have the separate question 
> of funding it that I will discuss separately, if and when we get that far. 
> The DEP covers some of the topics, but it's notable that the implementation 
> speed of this varies considerably; if we could get, say, me and someone else 
> on it full-time (which is not a cheap or likely prospect), we can probably be 
> done inside of a single release. If we rely just on volunteer time, it may 
> take years. There is hopefully somewhere between those two that works.
> 
> Thanks for taking the time to read through!
> 
> Andrew
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAFwN1uqav3JXQA3m%2Bajf0Bd15QQX3JTZTfmH3Hc3ECWyRR3CVg%40mail.gmail.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/

Re: injecting settings

[Aymeric Augustin]

> On 8 May 2019, at 11:38, Michal Petrucha  wrote:
> 
> On Tue, May 07, 2019 at 10:53:37PM +0200, Aymeric Augustin wrote:
>> If we're only talking about providing default values for settings, currently 
>> there are two straightforward solutions:
>> 
>> 1. Like Adam suggested, access settings like this: getattr(settings, 
>> 'MY_SETTING', 'my_default').
>> 
>> This works well when you access settings just once, probably at import time, 
>> and cache their value.
>> 
>> Here's an example: 
>> https://github.com/aaugustin/django-sesame/blob/070cdb3fcdfa6c7310d7461add328a8095148ff1/sesame/backends.py#L27-L34
>>  
>> <https://github.com/aaugustin/django-sesame/blob/070cdb3fcdfa6c7310d7461add328a8095148ff1/sesame/backends.py#L27-L34>
> 
> This approach, however, makes it impossible to use the decorators and
> context managers from django.test that override settings. Of course,
> there are other ways to tune those knobs in tests, but it takes away a
> standard solution provided by the framework.

I'm not sure I understand what you're referring to. The framework provides the 
setting_changed signal, which seems to work well here: 
https://github.com/aaugustin/django-sesame/blob/070cdb3fcdfa6c7310d7461add328a8095148ff1/sesame/test_signals.py#L8-L18

>> DRF relies on a single Django setting which is a dict containing all 
>> individual DRF settings. Then it converts that Django setting to an object 
>> and uses this abstraction to access individual settings as properties.
> 
> This seems to have become one of the current best practices, but it
> requires each reusable package that goes down this road to include a
> bunch of boilerplate just to deal with default values for settings. I
> suppose that boilerplate could be factored out into its own reusable
> package, but my gut tells me that there must be a better way to handle
> something as fundamental as providing defaults for settings.
> Definitely a very subjective claim, but I just feel like this is
> something that should be the responsibility of the framework itself.

I'm ambivalent about this. I did it myself (TEMPLATES). I'm not sure that was 
worth doing.

Grouping a bunch of related settings in a dict looks satisfying. However, it's 
less practical than giving them a common prefix, notably to override them in 
tests.

I suppose I could support a well-written proposal, though :-)

>> Here we're talking about something slightly different: formalizing how an 
>> application can declare default values for its own settings — essentially a 
>> per-app equivalent of Django's global_settings.py. To do this properly, we 
>> need two things:
>> 
>> - a good convention for declaring these settings: I would find it elegant to 
>> consider every uppercase class attribute of an AppConfig class as a setting, 
>> but that might be too magic (and perhaps backwards-incompatible);
> 
> I definitely agree that this feels too magic-y. Maybe introducing a
> class attribute such as “DEFAULT_SETTINGS” that would hold a
> dictionary could work, though?

Yes, it could work too. I considered it and didn't choose it because it 
increases the distance between what global_settings.py looks like and what app 
settings would look like. Not a very strong argument ;-)

Also there's already magic to include only uppercase variables currently: 
https://github.com/django/django/blob/ef9f2eb69c9396683cefa742bc7d0a0792090e8d/django/conf/__init__.py#L135-L137

>> - a way to insert them properly into the settings object: I tried to figure 
>> out how LazySettings and friends handle global_settings, unfortunately there 
>> are more use cases than I was willing to untangle tonight.
> 
> Another thing to possibly consider, what should happen if multiple
> packages try to provide different defaults for the same setting? I
> mean, of course, this has kind of been floated in this thread already,
> but it would add one more item to the list of things affected by the
> order of INSTALLED_APPS.

Yes. Since the semantic is "make this value the default if there isn't a value 
already" (i.e. setdefault), the most intuitive behavior should be "first app in 
INSTALLED_APPS wins".

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20D907CE-D92E-4EF5-8EF4-8AF87F646017%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: injecting settings

[Aymeric Augustin]

> On 7 May 2019, at 17:49, Christian González  
> wrote:
> Apps should have the possibility to add DEFAULT settings to the Django global 
> settings. As already mentioned in my plugin system question, I don't see a 
> problem with app/settings loading order. The loading order is done in 
> INSTALLED_APPS, immutable, and good as it is.
> 
> App A could e.g. have a WEBPACK_LOADER variable that is imported into the 
> global scope, and the settings.py then CAN override it.
> It's that Django's settings.py SHOULD remain the master of settings. It's 
> more about having defaults that could be supposed by modules.

I see — this is an easier question that what I assumed based in your initial 
email. I got sidetracked by the reference to app-loading.

If we're only talking about providing default values for settings, currently 
there are two straightforward solutions:

1. Like Adam suggested, access settings like this: getattr(settings, 
'MY_SETTING', 'my_default').

This works well when you access settings just once, probably at import time, 
and cache their value.

Here's an example: 
https://github.com/aaugustin/django-sesame/blob/070cdb3fcdfa6c7310d7461add328a8095148ff1/sesame/backends.py#L27-L34
 


2. Set a default value at import time:

# apps.py

from django.apps import AppConfig
from django.conf import settings

class MyAppConfig(AppConfig):
name = 'my_app'
verbose_name = "..."

def ready(self):
if not hasattr(settings, 'MY_SETTING'):
settings.MY_SETTING = 'my_default'

If you have many settings, you can define defaults in a DEFAULT_SETTINGS dict 
and do this:

def ready(self):
for setting_name, default_value in DEFAULT_SETTINGS:
if not hasattr(settings, setting_name):
setattr(settings, setting_name, default_value)

If I understand correctly, you'd like to standardize a best practice and 
perhaps provide some helpers to make this more convenient.

Historically, Django refrained from normalizing how to define settings because 
for lack of a consensus on a best practice. However, this decision was mostly 
about how to structure per-environments settings modules, how to handle 
secrets, etc.

Here we're talking about something slightly different: formalizing how an 
application can declare default values for its own settings — essentially a 
per-app equivalent of Django's global_settings.py. To do this properly, we need 
two things:

- a good convention for declaring these settings: I would find it elegant to 
consider every uppercase class attribute of an AppConfig class as a setting, 
but that might be too magic (and perhaps backwards-incompatible);
- a way to insert them properly into the settings object: I tried to figure out 
how LazySettings and friends handle global_settings, unfortunately there are 
more use cases than I was willing to untangle tonight.

This could be an interesting new feature, if you manage to build consensus 
around it. I'd like to hear what others think.


> On 7 May 2019, at 17:15, Christian González  
> wrote:
> 
> I myself am using an implementation like DRF does it:
> 
> https://github.com/encode/django-rest-framework/blob/e16273a6584f9657c1e5c388558e9c5c92e7ba38/rest_framework/settings.py
>  
> 
> They create a class "APISettings", and mimic the Django behaviour.
> 

This is yet another approach. DRF relies on a single Django setting which is a 
dict containing all individual DRF settings. Then it converts that Django 
setting to an object and uses this abstraction to access individual settings as 
properties.


Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/89CA0C27-41E2-495F-9BAA-4C658467D842%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: injecting settings

[Aymeric Augustin]

Hello Christian,

I'm not aware of any plans in this area.

When you say this is not possible, I assume you are referring to my work on
app-loading in Django 1.7. Here's what comes to mind.

There's a chicken'n'egg problems if an app modifies INSTALLED_APPS during
app loading. This is a common idea, typically to implement a modular system
or a plugin system. I don't think there's a good answer here.
INSTALLED_APPS is a special case among settings. I'm not sure what to make
of this limitation.

Users could get into complications if an app modifies a setting after
another app performed some setup based on the value of this setting. This
could make app ordering more tricky. You could solve that by letting users
deal with ordering problems. However, I'm wary of adding one more
ramification to the ordering of INSTALLED_APPS, which already controls 4 or
5 different things.

Furthermore, as I understand your proposal, settings overrides would occur
as a side effect of importing a conventionally-named per-app settings
module. A guiding principle in the design of app-design was to stop
depending on import order because it's Very Hard to control in Python —
developers do not consider that adding an import in a module should change
the behaviour of their application.

Finally, this would mean that pluggable apps cannot assume that settings
are immutable anymore. It would become a best practice for every pluggable
app to assume that any setting can change and to deal with the
consequences. I'm not enthusiastic at the prospect of adding this overhead
to the ecosystem.

All these problems are consequences of allowing settings to be mutable
during app-loading, while they're currently immutable in all circumstances.

On a positive note, while this isn't officially supported, from a technical
standpoint, nothing prevents you from modifying settings during
app-loading. You could implement a base AppConfig class that does what you
want. You "just" need to ensure that the settings you're changing are
taking effect on the fly or to reset any caches based on these settings.

I know there's demand for this feature; I don't know if it can be done
without creating too many potential issues. I think you'll need a DEP to
breaking the "settings are immutable" dogma — perhaps not a very long one
if you have working code and a clear stance on the points I raised.

Best regards

-- 
Aymeric.


Le mar. 7 mai 2019 à 17:02, Christian González <
christian.gonza...@nerdocs.at> a écrit :

> Hi,
>
> I know this is a bit of a question half development, half usage of django.
>
> I'd like to create an django app which has sub apps which inject
> automatic code into settings. This is not possible, as the docs say.
>
> Are there plans in development to enable that?
> It's not necessary to "change" settings at runtime. It's just necessary
> to load "once" settings from sub apps once they are installed, which is
> at server start anyway.
>
> There is the AppConfig.ready() method which can do things at server start.
>
> But my idea is: Wouldn't it be handy to have Django load settings from
> apps too, e.g. with a hook like having .settings.py? which are
> merged into the main settings? Is this considered as security risk?
>
> Thanks,
>
> Christian
>
> --
> Dr. Christian González
> https://nerdocs.at
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/11e20db3-fc69-c6a9-750c-4edfb86fcc44%40nerdocs.at
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mX4M09UX8fzgX6ia_nwXA%2BXCf0ShhkkWEVNHoadpFL_rA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to format Django using black

[Aymeric Augustin]

So this thread just passed the 100 message mark... I wish we could redirect 
some of that energy to things that actually matter to end users...

Anyway, I updated DEP 8 to account for all comments made so far. Thanks 
everyone who provided feedback!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0F32D990-E59B-4795-967A-5CF436CC6A7F%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to format Django using black

[Aymeric Augustin]

 file before the specified 
> commit, as opposed to the blame for the file in its current state, 
> disregarding the specified commit. Not bad, but not quite what you want. 
> git-hyper-blame has been raised several times, but it's non-standard, not 
> straightforward to install, and not supported by tooling. A git patch was 
> mentioned that incorporates hyper-blame's functionality into blame[2], which 
> looks like it'll be great, but won't help until it's merged, released, and 
> integrated into tooling.
> 
> Django is a mature project, which means that many, many lines have remained 
> unchanged for years, and will continue to be attributed to the black 
> mega-commit for many years to come.
> 
> I'm firmly -1 to globally applying black to the codebase until it can be done 
> without breaking blame.
> 
> Alex
> 
> [0] https://prettier.io/docs/en/options.html#range 
> <https://prettier.io/docs/en/options.html#range>
> [1] https://github.com/nrwl/precise-commits 
> <https://github.com/nrwl/precise-commits>
> [2] 
> https://public-inbox.org/git/CAJDYR9SL9JCJjdARejV=NCf9GYn72=bfszxx84idc416szm...@mail.gmail.com/T/
>  
> <https://public-inbox.org/git/CAJDYR9SL9JCJjdARejV=NCf9GYn72=bfszxx84idc416szm...@mail.gmail.com/T/>
> On Sun, Apr 28, 2019 at 10:51 PM Aymeric Augustin 
>  <mailto:aymeric.augus...@polytechnique.org>> wrote:
> Hello,
> 
> Here's my attempt at summarizing the conversation in a DEP: 
> https://github.com/django/deps/pull/55 
> <https://github.com/django/deps/pull/55>.
> 
> It's easier to read as a rich diff: 
> https://github.com/django/deps/pull/55/files?short_path=95a1a7b#diff-95a1a7b430e2608b84f5c834fd6c258c
>  
> <https://github.com/django/deps/pull/55/files?short_path=95a1a7b#diff-95a1a7b430e2608b84f5c834fd6c258c>
> 
> Please let me know if I missed or represented unfairly some ideas!
> 
> Best regards,
> 
> -- 
> Aymeric.
> 
> PS: while I'm eager to listen to feedback and iterate on this draft, I would 
> also prefer if this DEP didn't turn into a novel, so let's avoid going into 
> every detail and trust the implementation team to do a good job — thank you 
> :-)
> 
> 
>> On 13 Apr 2019, at 13:52, Herman S > <mailto:herman.schis...@gmail.com>> wrote:
>> 
>> Hi.
>> 
>> I propose that Django starts using 'black' [0] to auto-format all Python 
>> code.
>> For those unfamiliar with 'black' I recommend reading the the projects 
>> README.
>> The short version: it aims to reduce bike-shedding and non value-adding
>> discussions; saving time reviewing code; and making the barrier to entry 
>> lower
>> by taking some uncompromissing choices with regards to formatting.  This is
>> similar to tools such as 'gofmt' for Go and 'prettier' for Javascript.
>> 
>> Personally I first got involved contributing to Django couple of weeks back,
>> and from anecdotal experience I can testify to how 'formatting of code' 
>> creates
>> a huge barrier for entry. My PR at the time went multiple times back and 
>> forth
>> tweaking formatting. Before this, I had to research the style used by 
>> exploring
>> the docs at length and reading at least 10-20 different source – and even 
>> those
>> were not always consistent. At the end of the day I felt like almost 50% of 
>> the
>> time I used on the patch was not used on actually solving the issue at hand.
>> Thinking about code formatting in 2019 is a mental energy better used for 
>> other
>> things, and it feels unnecessary that core developers on Django spend their 
>> time
>> "nit-picking" on these things.
>> 
>> I recently led the efforts to make this change where I work. We have a 200K+
>> LOC Django code-base with more than 30K commits. Some key take-aways: it has
>> drastically changed the way we work with code across teams, new engineers are
>> easier on-boarded, PR are more focused on architectural choices and "naming
>> things", existing PRs before migration had surprisingly few conflicts and 
>> were
>> easy to fix, hot code paths are already "blameable" and it's easy to blame a
>> line of code and go past the "black-commit", and lastly the migration went
>> without any issues or down-time.
>> 
>> I had some really fruitful discussions at DjangoCon Europe this week on this
>> very topic, and it seems we are not alone in these experiences. I would love 
>> to
>> hear from all of you and hope that we can land on something that will enab

Re: Proposal to format Django using black

[Aymeric Augustin]

Hello,

Here's my attempt at summarizing the conversation in a DEP: 
https://github.com/django/deps/pull/55 .

It's easier to read as a rich diff: 
https://github.com/django/deps/pull/55/files?short_path=95a1a7b#diff-95a1a7b430e2608b84f5c834fd6c258c
 


Please let me know if I missed or represented unfairly some ideas!

Best regards,

-- 
Aymeric.

PS: while I'm eager to listen to feedback and iterate on this draft, I would 
also prefer if this DEP didn't turn into a novel, so let's avoid going into 
every detail and trust the implementation team to do a good job — thank you :-)


> On 13 Apr 2019, at 13:52, Herman S  wrote:
> 
> Hi.
> 
> I propose that Django starts using 'black' [0] to auto-format all Python code.
> For those unfamiliar with 'black' I recommend reading the the projects README.
> The short version: it aims to reduce bike-shedding and non value-adding
> discussions; saving time reviewing code; and making the barrier to entry lower
> by taking some uncompromissing choices with regards to formatting.  This is
> similar to tools such as 'gofmt' for Go and 'prettier' for Javascript.
> 
> Personally I first got involved contributing to Django couple of weeks back,
> and from anecdotal experience I can testify to how 'formatting of code' 
> creates
> a huge barrier for entry. My PR at the time went multiple times back and forth
> tweaking formatting. Before this, I had to research the style used by 
> exploring
> the docs at length and reading at least 10-20 different source – and even 
> those
> were not always consistent. At the end of the day I felt like almost 50% of 
> the
> time I used on the patch was not used on actually solving the issue at hand.
> Thinking about code formatting in 2019 is a mental energy better used for 
> other
> things, and it feels unnecessary that core developers on Django spend their 
> time
> "nit-picking" on these things.
> 
> I recently led the efforts to make this change where I work. We have a 200K+
> LOC Django code-base with more than 30K commits. Some key take-aways: it has
> drastically changed the way we work with code across teams, new engineers are
> easier on-boarded, PR are more focused on architectural choices and "naming
> things", existing PRs before migration had surprisingly few conflicts and were
> easy to fix, hot code paths are already "blameable" and it's easy to blame a
> line of code and go past the "black-commit", and lastly the migration went
> without any issues or down-time.
> 
> I had some really fruitful discussions at DjangoCon Europe this week on this
> very topic, and it seems we are not alone in these experiences. I would love 
> to
> hear from all of you and hope that we can land on something that will enable
> *more* people to easier contribute back to this project.
> 
> I've set up how this _could_ look depending on some configurables in Black:
> 
> * Default config: https://github.com/hermansc/django/pull/1
> * Line length kept at 119: https://github.com/hermansc/django/pull/3
> * Line length kept at 119, no string normalization:
> https://github.com/hermansc/django/pull/2
> 
> Please have a look at the Black documentation. It explains the benefits better
> than I possibly could do here.
> 
> With kind regards,
> Herman Schistad
> 
> [0]: https://github.com/ambv/black
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAN%3DnMTx0EE5WfXuccv_e3MBuCxp9u_pAV_ow5MxNST6MptTDBw%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ECC90690-D27E-40D3-B69E-765D988A1690%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to format Django using black

[Aymeric Augustin]

> On 14 Apr 2019, at 15:10, Josh Smeaton  wrote:
> 
> Finally, there are some tricks you pick up if black fights you on some 
> decisions. To use Berkers example:
> 
> TIME_INPUT_FORMATS = [ 
> '%H:%M:%S', # '14:30:59' 
> '%H:%M:%S.%f', # '14:30:59.000200' 
> '%H:%M', # '14:30' 
> ] 
> 
> TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M'] # '14:30:59' 
> # '14:30:59.000200' # '14:30' 
> 
> Break each individual format into its own variable, with appropriate comment, 
> and add the variables to the list.
> 
> HMS = "%H:%M:%S"  # 14:30:59
> HMSF = ".." 
> HM = ".."
> TIME_INPUT_FORMATS  = [HMS, HMSF, HM]
> 
> Obviously just an example, but something to keep in mind.

FWIW there's also this possibility:

TIME_INPUT_FORMATS = [ 
# '14:30:59' 
'%H:%M:%S',
# '14:30:59.000200'
'%H:%M:%S.%f',
# '14:30'
'%H:%M',
]

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/37026972-CFCF-45D7-8903-958C31719224%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to format Django using black

[Aymeric Augustin]

I'm writing a DEP.

-- 
Aymeric.



> On 25 Apr 2019, at 13:30, Josh Smeaton  wrote:
> 
> To answer the question about decision making...
> 
> Usually a decision is made if there’s reasonable consensus in the discussion 
> or after a vote. If no clear consensus can be reached with the core team, 
> then a decision can be escalated to the technical board for a vote. 
> 
> In the future, if the core team is dissolved in its primary capacity, 
> consensus in the discussion rather than consensus among the core team would 
> be the first step. 
> 
> I figure all reasonable arguments have probably been made at this stage. But 
> there are really two questions we should be proposing. 
> 
> 1. Should Django adopt an autoformatter?
> 2. If it should, should that autoformatter be black or something else? 
> 
> Considering the rather strong views in this thread, I’d probably recommend 
> that a DEP is drafted for each of the above, where opinions and comments can 
> be gathered more formally. 
> 
> On Thu, 25 Apr 2019 at 20:15, James Bennett  > wrote:
> I like Django's style guide.
> 
> I like the *idea* of an autoformatter.
> 
> I dislike the particular mostly-unconfigurable style Black enforces, and I 
> find that several of its rules negatively impact code readability.
> 
> So I would be -1 on applying it to Django.
> 
> -- 
> You received this message because you are subscribed to a topic in the Google 
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/django-developers/wK2PzdGNOpQ/unsubscribe 
> .
> To unsubscribe from this group and all its topics, send an email to 
> django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAL13Cg-MJqE%2BxmxpBhymxaUz__6V_htE1XA-XRxZH5Uxa3WY_A%40mail.gmail.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAPbDM0f24%3D62%3DBXD-OxgonjGpiJ13W4koaHg%2BbnwPdefzM2Baw%40mail.gmail.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1CAC7C71-01C6-4358-96FC-C41B8A1D2BE2%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: Allow ManyToMany using a intermediary table to be defined as symmetrical

[Aymeric Augustin]

Hello Nadège,

While I'm not an expert of this area, your proposal makes sense to me. It looks 
like a reasonable extension of previous work. You can go ahead with a ticket 
and a pull request.

Best regards,

-- 
Aymeric.



> On 23 Apr 2019, at 12:09, Nadège Michel  wrote:
> 
> Hello,
>  
> We use self.referencing ManyToMany relationships with intermediate tables in 
> our work project 
> and I was wondering why we had to create ourselves the reverse link when we 
> need the relationship to be symmetrical.
> I looked at the 'symmetrical' attribute documentation and though we should 
> just set it to True instead of False, 
> but you may know that it triggers the error "fields.E332 Many-to-many fields 
> with intermediate tables must not be symmetrical.".
> 
> I searched for a corresponding existing ticket a found this one which is kind 
> of related https://code.djangoproject.com/ticket/9475 
> 
> And you can see in the PR some discussion about the check 
> https://github.com/django/django/pull/8981#discussion_r247946460 
> 
> Thanks to the work of Collin Anderson in the previous PR I think we can now 
> remove that check.
> 
> Questions I had:
> retro compatibility?
> As the current behavior forces the user to set 'symmetrical=False', the 
> change is retro-compatible.
> use 'through_defaults' when creating both objects or define a new 
> 'through_reverse_defaults' to give different values for each row? 
> if you want symmetrical relationship, both objects should have the same 
> values 
> so I chose to use 'through_defaults' for both objects. Which was also what 
> was done in the #8981 
>  PR in the 
> first place.
> 
> I have a patch 
> 
>  with theses changes:
> removed the check 
> removed tests for that check
> added tests (tests/m2m_through, tests/m2m_recursive)
> updated documentation (may need a bit more work)
> added back the use of 'through_defaults' of #8981 
>  in the:
> if self.symmetrical:  
> self  ._add_items(...) 
> 
> Tests suite runs fine.
> 
> Any thoughts on this design change / new feature?  
> I'll be happy to create a ticket and submit my patch for reviews :) 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3e2f799d-3444-407b-bc81-523c3f55ec0b%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8B0DF00A-8241-4454-80C9-D1F6B91DAA14%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: django_chatterbot

[Aymeric Augustin]

Hello,

This mailing list is for the development of Django itself, not for support 
using Django or third-party apps.

Please use the support channels for the django-chatterbot project or a site 
like Stack Overflow.

-- 
Aymeric.



> On 18 Apr 2019, at 10:39, Meow  wrote:
> 
> Hi, Abhineet, could you show us the code ?
> On Apr 18, 2019, 06:47 +0800, Abhineet Baranwal 
> , wrote:
>> Hello there,
>> I am using django chatterbot but whenevere i typed any new sentence and hit 
>> enter then the bot returns always the current time.I do not know why it 
>> always respond with the current time .Any ideas?
>> 
>> And can anyone tell me how we can train my personal data with django 
>> chatterbot ?
>> 
>> 
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com 
>> .
>> To post to this group, send email to django-developers@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/02cefd26-feed-49be-9dc8-19cb80827a6c%40googlegroups.com
>>  
>> .
>> For more options, visit https://groups.google.com/d/optout 
>> .
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/e4060833-68bc-42b3-b86e-b33d7636fe41%40Spark
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/12D51EAB-AE27-4439-9EDD-0F414BCBEB0C%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Why does ModelForm do validation and not Model

[Aymeric Augustin]

Hi Curtis,

Le mer. 17 avr. 2019 à 22:32, Curtis Maloney  a écrit :

> It's mostly for performance reasons, since validation can be expensive.
>
> Really? My memory was that it was (a) backward compatibility [model
> validation was added later], and (b) practicality [try catching
> everywhere in your code you save a model, and enforce catching
> validation exceptions there].
>

These arguments are absolutely valid.

I can't say for sure if some concerns ranked higher than others.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mXP3P%2BBeKnU4Mu8WuC3Gh9PWh-2GDaC7H0Kzc6%2BEtcexg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why does ModelForm do validation and not Model

[Aymeric Augustin]

Hello Will,

It's mostly for performance reasons, since validation can be expensive. You can 
override save() to call full_clean() in a model if you'd like.

Cheers,

-- 
Aymeric.



> On 16 Apr 2019, at 20:47, Will Gordon  wrote:
> 
> I can't seem to find a good reason for this. And I could foresee this 
> preventing potential mistakes. I'm not proposing to actually change the 
> implementation, I guess I'm just looking for the reason of it.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/97ae2f12-bc27-403d-8b76-f456a63fc0d9%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/D7E2A8BE-C236-4000-8E05-A6D756AD4C46%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to format Django using black

[Aymeric Augustin]

Hello,

I'm strongly in favor of adopting black with the default options.

In my opinion, it's the first Python code formatter that passes the bar of "on 
average, does better than any single programmer". Trying to enforce something 
else manually is a waste of energy that produces worse results.

I like nicely formatted code and I hate some of what black produces, typically 
on Django model definitions. However, I've seen the benefits of an automated 
code formatter, even on projects where I write almost all of the code.

I'm positive the benefits will be great on Django, where you can tell when a 
module was (re)written just by looking at the style. Yes, there are some 
downsides, but it's clearly a good tradeoff.

Regarding the migration strategy, converting one package at a time sounds good, 
because then we can proof-read the Big Diff and perhaps tweak the code where 
the result really hurts the eyes.

Most of Django's style guide will still applies, as it's mostly conventions 
about naming things, ordering declarations, etc.

Best regards,

-- 
Aymeric.



> On 13 Apr 2019, at 13:52, Herman S  wrote:
> 
> Hi.
> 
> I propose that Django starts using 'black' [0] to auto-format all Python code.
> For those unfamiliar with 'black' I recommend reading the the projects README.
> The short version: it aims to reduce bike-shedding and non value-adding
> discussions; saving time reviewing code; and making the barrier to entry lower
> by taking some uncompromissing choices with regards to formatting.  This is
> similar to tools such as 'gofmt' for Go and 'prettier' for Javascript.
> 
> Personally I first got involved contributing to Django couple of weeks back,
> and from anecdotal experience I can testify to how 'formatting of code' 
> creates
> a huge barrier for entry. My PR at the time went multiple times back and forth
> tweaking formatting. Before this, I had to research the style used by 
> exploring
> the docs at length and reading at least 10-20 different source – and even 
> those
> were not always consistent. At the end of the day I felt like almost 50% of 
> the
> time I used on the patch was not used on actually solving the issue at hand.
> Thinking about code formatting in 2019 is a mental energy better used for 
> other
> things, and it feels unnecessary that core developers on Django spend their 
> time
> "nit-picking" on these things.
> 
> I recently led the efforts to make this change where I work. We have a 200K+
> LOC Django code-base with more than 30K commits. Some key take-aways: it has
> drastically changed the way we work with code across teams, new engineers are
> easier on-boarded, PR are more focused on architectural choices and "naming
> things", existing PRs before migration had surprisingly few conflicts and were
> easy to fix, hot code paths are already "blameable" and it's easy to blame a
> line of code and go past the "black-commit", and lastly the migration went
> without any issues or down-time.
> 
> I had some really fruitful discussions at DjangoCon Europe this week on this
> very topic, and it seems we are not alone in these experiences. I would love 
> to
> hear from all of you and hope that we can land on something that will enable
> *more* people to easier contribute back to this project.
> 
> I've set up how this _could_ look depending on some configurables in Black:
> 
> * Default config: https://github.com/hermansc/django/pull/1
> * Line length kept at 119: https://github.com/hermansc/django/pull/3
> * Line length kept at 119, no string normalization:
> https://github.com/hermansc/django/pull/2
> 
> Please have a look at the Black documentation. It explains the benefits better
> than I possibly could do here.
> 
> With kind regards,
> Herman Schistad
> 
> [0]: https://github.com/ambv/black
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAN%3DnMTx0EE5WfXuccv_e3MBuCxp9u_pAV_ow5MxNST6MptTDBw%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/DCD30C61-13DD-4EC2-B8B6

Re: Newcomer's questions

[Aymeric Augustin]

Hello Alexey,

Thanks for contributing!

1. You should make a PR against master and, if you're fixing a Trac ticket, 
start the commit message with "Fixed #x -- bla bla bla".

Version indicators such as [2.0.x] are added when patches are backported to 
stable branches. You don't need to care about this now. If your issue is 
eligible for backporting 
,
 then the person who merges the patch will likely do the backport as well.

2. Yes, your fork should be up to date with Django's master branch to ensure 
that the PR is clean.

See Working with git / After upstream has changed 

 for details (read the whole page, commands listed in that paragraph depend on 
other commands listed earlier).

3. As you wish :-)

I'm not aware of any guidelines about this relatively new GitHub feature when 
contributing to Django.

Best regards,

-- 
Aymeric.



> On 17 Mar 2019, at 10:34, Alexey Tsivunin  wrote:
> 
> I worked on issue and fixed a bug. So a have three commits (in the sequence 
> they were commited):
> - Added regression tests
> - Added bugfix
> - Added name to AUTHORS
> 
> This is my first PR and I have several questions:
> 
> 1. I don't quite understand about "headers" in commit messages. Which commit 
> should have header like "[2.0.x] Fixed #29471 -- "? Or each of them should 
> have it?
> 
> 2. Does my fork have to be up-to-date with original django repo before I make 
> the PR? If so, is there any guides how I can do this?
> 
> 3. Should I select "Allow edits from maintainers" when creating PR?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/e550ba3f-c6d4-49f5-94e2-29cfd0ee6798%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/87832FCE-FA46-4717-BB1C-E6F0F8795694%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: BitBounce Spam Replies From the Mailing List

[Aymeric Augustin]

They say they removed the offending user:  
https://twitter.com/stewart__dennis/status/1097296853551337472?s=21

This isn’t a long term fix — we’ll have the same issue if another user 
subscribes to BitBounce — but at least it solves our immediate problem.

-- 
Aymeric.

> Le 17 févr. 2019 à 21:35, Aymeric Augustin 
>  a écrit :
> 
> Since Twitter is the only place where BitBounce responded, I tried again: 
> https://twitter.com/aymericaugustin/status/1097231848973967362
> 
> I'm skeptical about their willingness to fight spam: they're using it as 
> their primary marketing channel. The more we're talking about them, the 
> happier they are...
> 
> -- 
> Aymeric.
> 
> 
> 
>> On 29 Jan 2019, at 11:30, Patryk Zawadzki  wrote:
>> 
>> Sorry for resurrecting but this is still very much a problem. Same person, 
>> same autoresponder.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/41722075-8e89-4777-8872-ee4660f14b26%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0E830FC6-B4C9-449E-A668-BB2DB965D58D%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: BitBounce Spam Replies From the Mailing List

[Aymeric Augustin]

Since Twitter is the only place where BitBounce responded, I tried again: 
https://twitter.com/aymericaugustin/status/1097231848973967362

I'm skeptical about their willingness to fight spam: they're using it as their 
primary marketing channel. The more we're talking about them, the happier they 
are...

-- 
Aymeric.



> On 29 Jan 2019, at 11:30, Patryk Zawadzki  wrote:
> 
> Sorry for resurrecting but this is still very much a problem. Same person, 
> same autoresponder.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/41722075-8e89-4777-8872-ee4660f14b26%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4C847606-C965-4C61-8851-63DA676381CF%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Extend FAQ with "How do I get Django and my JS framework to work together?"

[Aymeric Augustin]

Hello,

I wrote a three-part essay on this question last year:
1. https://fractalideas.com/blog/making-react-and-django-play-well-together/
2. 
https://fractalideas.com/blog/making-react-and-django-play-well-together-hybrid-app-model/
3. 
https://fractalideas.com/blog/making-react-and-django-play-well-together-single-page-app-model/

Even though I took a narrower view — I only considered React — I found enough 
decisions factors to write over 2000 words in the first post, which is too long 
for a FAQ :-)

On one hand, I'm not sure the Django docs should go into this level of detail 
and provide specific information about a particular JS framework. On the other 
hand, it's rather useless to talk about integrating Django with a JS frontend 
without discussing authentication and it's hard to discuss authentication in 
less than 2000 words — which were just for justifying my favorite solution, not 
for investigating every option!

I think we need a how-to guide rather than a FAQ entry. I would find it nice:

A. To describe the Singe Page App model — where Django only serves the API

We need to explain CORS and CSRF in the clearest terms possible. Many devs end 
up shotgun-debugging CORS or CSRF errors, which always results in insecure 
deployments.

My consulting experience suggests that we have a problem there: I never did an 
audit where the client got that right, even though they're all smart people 
trying to get things right.

Perhaps a condensed version of my third post could do the job?

Some may disagree with my recommendation against JWT, which may too opinionated 
for the Django docs. Again, in my experience, people tend to get security more 
wrong with JWT, which is why I prefer discouraging it and letting those who 
know what they're doing ignore my advice.

B. To say something about integrating a modern JS framework with 
django.contrib.staticfiles 

It's perfectly doable and provides all the benefits of 
django.contrib.staticfiles. However, it requires a bit of duct tape, as shown 
in my second post.

I'm a huge fan of this technique for simple website but I'm afraid I'm biased 
by my experience with Django. This is unlikely to be a popular option for those 
who are more familiar with a modern frontend framework than with 
django.contrib.staticfiles.

The docs should at least give the general idea of "compile your frontend to 
somewhere Django can find the files, then run collectstatic".

If someone starts writing documentation about this, I'm interested in reviewing 
it.

Best regards,

-- 
Aymeric.



> On 5 Feb 2019, at 11:17, Carlton Gibson  wrote:
> 
> I think this topic is very interesting. 
> 
> Two sides of it: 
> 
> * Static files handling
> * APIs
> 
> Curtis is right, there are other options but, Django REST Framework is 
> (whilst not perfect) pretty solid on the API front. I think Django has a good 
> story here. 
> It's pretty hard not to find DRF if you follow any guide, or any searching at 
> all. 
> 
> The static files story is a little different. It seems to me we don't tell 
> the best story there. 
> 
> Rails has two things which we could be envious of, even if we didn't want to 
> copy exactly:
> 
> * The frontend framework integration that's already been mentioned. 
> * The very easy "Ajax your form", with controllers (i.e. for us "generic 
> views") automatically handling ajax form submissions. 
> 
> Both these features get users further quicker in these aspects than we are 
> able to offer. 
> 
> We struggle to think of areas for improvements (re GSoC for example) but 
> maybe here is an area. 
> 
> This ties into Claude's proposal here: 
> https://groups.google.com/d/topic/django-developers/KYmNnvwXDUI/discussion
> 
> My own story is, I've had lots of success with, and still use, Django 
> Compressor.  
> At it's simplest you just map a content type to a shell command to run and 
> then include your Sass/Less/React/Elm/whatever files in your HTML (with 
> script or link tags, almost in the old-school way). 
> In development these are processed (& cached) per request. 
> For deployment you just run an offline compression task (management command) 
> and then upload the files. 
> That's it. 
> It's not a coverall approach — a frontend engineer will come along and 
> totally replace Compressor with whatever is this week's Top Javascript Build 
> System™ BUT it is a good 80:20: it lets me do something (that approximates) 
> respectable, without knowing hardly anything about the latest frontend 
> hotness. (GNU Make FTW! 🙂) 
> 
> I think if we were to offer something out-of-the-box that got as far as 
> Compressor, or further, we'd:
> satisfy most of our users, 
> allow yet more to get off the mark quickly, 
> and... well... those that need the full frontend toolchain would still be 
> free to use it. 
> I worry we'd never get anything like this into core... but I think it would 
> be good. (As I say, I think it's one area where we are lacking/behind the 
> competition.)
> 
> 

Re: django.utils.dateparse

[Aymeric Augustin]

Hello Guiseppe,

At this point I think we can agree on why we disagree :-)

First, I believe that the function responsible for converting datetimes
stored in ISO 8601 format in SQLite databases should parse ISO 8601 and not
do anything else. I'm -1 on changing it to accept localized datetimes. (A
third-party package could provide a model field supporting this.)

So we're discussing the addition of new functionality.

Second, I'm skeptical of functions accepting a variety of more or less well
specified inputs. I don't think such APIs are conducive to good coding
practices. Either you're getting data from an automated system, in which
case the format is known. Or you're getting data from humans, in which case
Django provides one solution: forms. Indeed, when interacting when humans,
the hard part isn't validating the data, it's providing good error messages.

I know that not everyone shares this preference for strict APIs. Some
languages try hard to make sense of poorly specified inputs, for example:

$ node -e 'console.log("1" + 1);'
11
$ php -r 'echo "1" + 1;'
2
$ python -c 'print("1" + 1)'
Traceback (most recent call last):
  File "", line 1, in 
TypeError: must be str, not int

I don't think I need to explain my opinion after showing this example :-)

It's a reasonable analogy for the issue we're discussing here. "1" and 1
are the same input provided in a slightly different format that doesn't
make a difference for humans but does make one for computers. So are
2019-02-04 and 04/02/19. (Or is it 02/04/19?)

Anyway, if you think this is generally useful, you can easily package it
into a third-party module. Widespread adoption would be a strong argument
for integrating it into a future version of Django.

Best regards,

Aymeric.

Le lun. 4 févr. 2019 à 10:22, Giuseppe De Marco 
a écrit :

> @Augustin
> Regarding your questions:
>
> - would this be useful?
> I think yes, for the following reasons:
> 1. We have an authentic regexp compiler based on DATE_FORMATS and
> DATETIME_FORMATS
> 3. We don't have to write datetime regexp anymore, this code will compile
> a regexp from a format, indipendently of its delimiter char (if -, / or
> whatever)
> 4. We get generalized function that returns datetime objects, no
> try/except and datetime.strptime, It's faster then other implementations!
> 5. It's settings.py focused, all we have to worry is a correct settings.py
> configuration. In other words We just have to collect all the possibile
> date/datetime formats that could be used in the project, even if they are
> used in forms or in model.fields
> 6. We don't need anymore to hardcode datetime regexp pattern in our code,
> the regexp compiler will work on top of date formats strings!
>
> - would a Form not be a better choice?
> Sure, I'm tring to generalize a method that could be a stop application
> for all the date and datetime approaches. It could be used for forms, in
> DATETIME_INPUT_FORMATS and DATE_INPUT_FORMAT. These could generate form
> specialized regexp compilations if this approach will be implemented.
>
> The main goal is to give a tool that will work well and in every
> conditions,and be funny too!
>
>
> Il giorno lun 4 feb 2019 alle ore 01:30 Tom Forbes  ha
> scritto:
>
>> I’m pretty sure 0.0241 usec per loop is either a typo or a mistake
>> during benchmarking. I’ve got no comment what you’re proposing but correct
>> and valid benchmarks are important, so I would double check that.
>>
>>
>>
>> On 3 February 2019 at 23:37:14, Giuseppe De Marco (
>> giuseppe.dema...@unical.it) wrote:
>>
>> Regarding the previous example,
>> better to read it here (my fault: I mistaken the format
>> '%Y-%m-%dT%H:%M:%S.%f'):
>>
>> https://github.com/peppelinux/Django-snippets/blob/master/datetime_heuristic_parser.py
>>
>> and also, it should came also with tzinfo regexp and other functions as
>> well, like parse_date time_duration... it's only an example to share our
>> experiences.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/CABms%2BYpcU3gbv7MqV1FkwU5mt9xhTRBoPeG6bPoYCnLvTJLGAw%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, s

Re: django.utils.dateparse

[Aymeric Augustin]

Hello Guiseppe,

In which circumstances:

- would this be useful?
- would a Form not be a better choice?

Best regards,

-- 
Aymeric.



> On 4 Feb 2019, at 00:10, Giuseppe De Marco  wrote:
> 
> Hi Aymeric,
> 
> Thank you for the answer and for the tests as well.
> I understand and also agree your vision on all the line, I got the specific 
> purpose of parse_date.
> I'd like to introduce a generalized way to parse date and datetime string 
> based on Django project configuration, in settings.py.
> It could be linked, as reference, to DATE_INPUT_FORMATS or DATE_FORMATS or 
> whatever better choice.
> 
> This is the code example, it works as standalone, I hope that it's easy to 
> read...
> 
> # datetime_euristic_parser.py
> import re
> import datetime
> 
> DATE_FORMATS = ['%Y-%m-%d', 
> '%d/%m/%Y', 
> '%d/%m/%y']
> DATETIME_FORMATS = ['%Y-%m-%d %H:%M:%S', 
> '%d/%m/%Y %H:%M:%S', 
> '%d/%m/%y %H:%M:%S',
> '%Y%m%d%H%M%SZ',
> '%Y%m%d%H%M%S.%fZ']
> # to be extended with all the matching patterns.
> DATETIME_ELEMENTS_REGEXP = {'%Y': '(?P\d{4})',
> '%y': '(?P\d{2})',
> '%m': '(?P\d{1,2})',
> '%d': '(?P\d{1,2})',
> '%H': '(?P\d{1,2})',
> '%M': '(?P\d{1,2})',
> '%S': '(?P\d{1,2})',
> '%f': '(?P\d{6})'} # ...
>  
> def datetime_regexp_builder(formats):
> """
> formats = DATE_FORMAT of DATETIME_FORMAT
> """
> regexp_dict = {}
> for df in formats:
> df_regexp = df
> for k,v in DATETIME_ELEMENTS_REGEXP.items():
> df_regexp = df_regexp.replace(k,v)
> regexp_dict[df] = df_regexp+'$'
> return regexp_dict
> 
> DATE_FORMATS_REGEXP = datetime_regexp_builder(DATE_FORMATS)
> DATETIME_FORMATS_REGEXP = datetime_regexp_builder(DATETIME_FORMATS)
> 
> def dformat_insp(date_str, format_regexp_dict, debug=False):
> """
> Takes a date string and returns a matching date regexp. 
> """
> insp_formats = []
> for f,p in format_regexp_dict.items():
> if debug: print(date_str, f, p)
> match = re.match(p, date_str)
> if match:
> res = (f, p, {k:int(v) for k,v in match.groupdict().items()})
> insp_formats.append(res)
> return insp_formats
> 
> def dateformat_insp(date_str):
> return dformat_insp(date_str, DATE_FORMATS_REGEXP)
> 
> def datetimeformat_insp(date_str):
> return dformat_insp(date_str, DATETIME_FORMATS_REGEXP)
> 
> def datetime_euristic_parser(value):
> """
> value can be a datestring or a datetimestring
> returns all the parsed date or datetime object
> """
> l = []
> res = dateformat_insp(value) or \
>   datetimeformat_insp(value)
> for i in res:
> l.append(datetime.datetime(**i[-1]))
> return l
> 
> # example
> if __name__ == '__main__':
> tests = ['04/12/2018',
>  '04/12/2018 3:2:1',
>  '2018-03-4 09:7:4',
>  '2018-03-04T09:7:4.645194',
>  '20180304121940.948000Z']
> 
> for i in tests: 
> res = dateformat_insp(i) or datetimeformat_insp(i)
> if res:
> print('Parsing succesfull on "{}": {}'.format(i, res))
> #print(datetime_euristic_parser(i))
> else:
> print('Parsing failed on "{}"'.format(i))
> print()
> 
> 
> I also checked these tests:
> python -m timeit -s "from django.utils.dateparse import parse_datetime" 
> "parse_datetime('2019-02-03T17:27:58.645194')"
> 1 loops, best of 3: 32.7 usec per loop
> 
> python -m timeit -s "import datetime" 
> "datetime.datetime.strptime('2019-02-03T17:27:58.645194', 
> '%Y-%m-%dT%H:%M:%S.%f')"
> 1 loops, best of 3: 53.5 usec per loop
> 
> python -m timeit -s "from datetime_euristic_parser. import 
> datetime_euristic_parser; 
> datetime_euristic_parser('2019-02-03T17:27:58.645194'

Re: Show applied datetime in showmigrations

[Aymeric Augustin]

Hello,

This sounds interesting.

You should think a bit about backwards compatibility because there's a good 
chance that some users are calling django-admin showmigrations in scripts or 
tools. I don't think this is a blocker for changing the output format. However 
you should plan to minimize disruption, perhaps by showing this information 
only a verbosity level higher than the default.

Best regards,

-- 
Aymeric.



> On 3 Feb 2019, at 18:28, Tim Schilling  wrote:
> 
> My idea is to add the applied datetime value to the showmigrations command.
> 
> I've run into the case where I'm working on a branch that involves a number 
> of migrations across various apps, but then have to switch to a different 
> branch which has different migrations. It can be troublesome to determine 
> which migrations are new and need to be rolled back. I've recently started 
> looking at the django_migrations table sorted on the applied column to 
> determine which I've run recently. This would make switching between branches 
> involving conflicting migrations easier.
> 
> Thanks,
> Tim
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/77e70de0-6418-4294-9135-a30a8a6b4b33%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/835B3F0F-88C5-42E7-9457-5028BA333D98%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: django.utils.dateparse

[Aymeric Augustin]

Hello Guiseppe,

django.utils.dateparse provides helpers needed by Django to implement datetime, 
date and time fields on SQLite. (SQLite doesn't have a native date time type.) 
Their job is to parse ISO 8601 fast. That's it.

A utility module should do exactly what Django needs and nothing more. 
django.utils.dateparse is documented so you can use it if it does what you 
want. If it doesn't, use something else ;-)

Django forms try various formats when they receive user input because there's 
uncertainty about the format. If you're parsing user input, then you should use 
a form and you'll get the behavior you want.

To clarify my point about performance, here's the function you're proposing, 
minus support for USE_L10N:

def parse_datetime_alternative(value):
for format in settings.DATETIME_INPUT_FORMATS:
try:
return datetime.datetime.strptime(format, value)
except (ValueError, TypeError):
continue

It's 10 times slower than the current implementation of parse_datetime:

$ python -m timeit -s "from django.conf import settings; settings.configured or 
settings.configure(); from django.utils.dateparse import 
parse_datetime_alternative as parse_datetime" 
"parse_datetime('2019-02-03T17:27:58.645194')"
5000 loops, best of 5: 54.2 usec per loop

$ python -m timeit -s "from django.utils.dateparse import parse_datetime" 
"parse_datetime('2019-02-03T17:27:58.645194')"
5 loops, best of 5: 5.48 usec per loop

I implemented parse_datetime with a regex because that's almost twice as fast 
as a single call to datetime.datetime.strptime:

$ python -m timeit -s "import datetime" 
"datetime.datetime.strptime('2019-02-03T17:27:58.645194', 
'%Y-%m-%dT%H:%M:%S.%f')"
2 loops, best of 5: 9.87 usec per loop

Best regards,

-- 
Aymeric.



> On 3 Feb 2019, at 17:10, Giuseppe De Marco  wrote:
> 
> Hi All, it's the first time for me in this ml,
> 
> I'd like to purpose a refactor of django.utils.dateparse functions.
> Currently a function in it, like parse_date for example, extract date time 
> string with a static regexp...
> 
> https://docs.djangoproject.com/pl/2.1/_modules/django/utils/dateparse/ 
> 
> 
> The first time I used It I thought that those functions parses date, time and 
> date time, according to settings.py definitions like DATETIME_INPUT_FORMATS, 
> but It was not this way... Then I read the code.
> 
> Wouldn't It better to use settings.py definitions to manage these formats and 
> django.utils.dateparse behaviour?
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABms%2BYpGtCQ1yzVfGtRKpDv60-4oH_SyGpPQYR6j1cR%2BL5LFRA%40mail.gmail.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/BEAAEDB4-D58E-4CD0-8F1A-11A5F60B556B%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Breaking change vs deprecation on Sitemaps `ping_google` command

[Aymeric Augustin]

Ditto.

I'd rather have a small, documented backwards incompatibility (and that's 
unlikely to cause significant problems) that another setting.

Cheers,

-- 
Aymeric.



> On 9 Jan 2019, at 16:09, charettes  wrote:
> 
> I agree with Tim, this is a minor change in a contrib app.
> 
> I don't think it warrants all the efforts required for a deprecation period 
> given the large adoption of https in the recent years.
> 
> Le mercredi 9 janvier 2019 06:41:25 UTC-5, Tim Graham a écrit :
> Adding temporary settings to the default project template sounds like a fair 
> bit of cruft for a contrib app that isn't enabled by default. I don't use 
> ping_google but I'm in favor of making the switch without a deprecation. 
> Users of ping_google please comment, but switching to https doesn't seem like 
> the sort of change that's going to cause massive pain.
> 
> On Wednesday, January 9, 2019 at 5:26:37 AM UTC-5, Markus Holtermann wrote:
> Hi all, 
> 
> We could introduce a settings variable `SITEMAPS_PING_GOOGLE_HTTPS` that's 
> part of newly created projects' settings (in 
> https://github.com/django/django/blob/master/django/conf/project_template/project_name/settings.py-tpl
>  
> )
>  and set to `True`. In global_settings.py it defaults to `False`. Users could 
> then turn it on for existing projects. Once the deprecation period is over we 
> would simply remove the variable and thus enforce HTTPS. We could then also 
> possibly issue a notice through the checks framework that the variable is now 
> obsolete. 
> 
> Cheers, 
> 
> Markus 
> 
> On Wed, Jan 9, 2019, at 10:59 AM, Carlton Gibson wrote: 
> > Hi all. 
> > 
> > There's PR to fix long-standing issue with HTTP-only URLs being 
> > generated by the `ping_google` sitemaps command. 
> > 
> > https://code.djangoproject.com/ticket/23829 
> >  
> > https://github.com/django/django/pull/10651 
> >  
> > 
> > (The PR also allows specifying the domain, so you don't need 
> > contrib.sites installed.) 
> > 
> > The idea is to generate HTTPS URLs by default, having a flag to switch 
> > (back) to HTTP if that's what you really need. (`--use_http`) 
> > 
> > This though is a breaking change potentially: you'd need to add the new 
> > flag to your deployment scripts, or wherever, to keep existing HTTP 
> > URLs working. 
> > 
> > Because of this Adam suggested putting it through the deprecation 
> > process: keep the HTTP default and switch to HTTPS as the default 
> > later. 
> > 
> > However, I can't see how we can offer the new preferred usage (default 
> > to HTTPS without additional flags to the command) from day-one, 
> > allowing a shim to fallback to the existing behaviour: the best we seem 
> > to be able to do is emit a warning saying there'll be a breaking change 
> > in the future. If that's the case I'd rather just bite the bullet on 
> > it: clearly state that the new flag will be needed to keep using HTTP 
> > URLs in the v2.2 release notes and move on. 
> > 
> > Thoughts please. Thanks. 
> > 
> > Carlton. 
> > 
> > 
> >   
> > 
> > 
> >  -- 
> >  You received this message because you are subscribed to the Google 
> > Groups "Django developers (Contributions to Django itself)" group. 
> >  To unsubscribe from this group and stop receiving emails from it, send 
> > an email to django-develop...@googlegroups.com <>. 
> >  To post to this group, send email to 
> > django-d...@googlegroups.com <>. 
> >  Visit this group at https://groups.google.com/group/django-developers 
> > . 
> >  To view this discussion on the web visit 
> > https://groups.google.com/d/msgid/django-developers/4188349d-a72a-4c2b-92d9-ee607c4c8b04%40googlegroups.com
> >  
> > 
> >  
> >  >  
> > >.
> >  
> >  For more options, visit https://groups.google.com/d/optout 
> > . 
> > 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> 

Re: Migration strategy for proxy model permissions to use their own content type

[Aymeric Augustin]

Hello Arthur,

I agree that option 1 is the way to go.

It would be nice to make the migration reversible. If someone upgrades to 
Django 2.2.x, migrates the database, and later discovers a blocking issue, they 
should have the option to migrate backwards and downgrade to 2.1.y.

Best regards,

-- 
Aymeric.



> On 26 Nov 2018, at 16:10, Arthur Rio  wrote:
> 
> Hi all,
> 
> I have been working on a 9 years old ticket that I'd like to close once and 
> for all. The outstanding question is about the migration path to choose in 
> order to update existing proxy model permissions. I have explained three 
> different approaches I can think of in the pull request: 
> https://github.com/django/django/pull/10381#issuecomment-435534644. 
> 
> I have implemented the first option and wrote the release notes explaining 
> what to expect for users upgrading: 
> https://github.com/django/django/pull/10381/files#diff-1f22a5c1d6164c6de8defb36f3829138.
> 
> Carlton, who has been the main reviewer, suggested to contact the mailing 
> list for further review in order to catch any red flag and make sure we have 
> general consensus on the chosen approach. 
> 
> Regards.
> 
> Arthur
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/f3298dbb-e072-4815-944c-2250d0b4d3a3%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/46CD821B-C17E-4FF8-B623-6B40B0FB0BBD%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Simplify authentication backend interface

[Aymeric Augustin]

Hello Tobias,

And sorry for the late answer!

Yes, this is a good idea.

https://docs.djangoproject.com/en/2.1/topics/auth/customizing/#handling-authorization-in-custom-backends
 

 reads more like "let's document the existing APIs to help users handle 
permissions in custom backends" than like "we spent a lot of time designing the 
most user friendly API for handling permissions in custom backends".

I think you're on the right track for defining:

A. which permission-related APIs custom authentication / authorization backends 
are expected to provide
B. which other public APIs have a reasonable implementation that usually 
doesn't need overriding

Regarding your questions:

> -   Either add get_user_permissions() or remove get_group_permissions()

The path of least resistance is to make `get_user_permissions()`, which already 
exists, a public API. I'm not finding a sufficiently good reason to remove 
`get_group_permissions()`.

> -   Add default implementations for get_all_permissions() and
> has_perm(), either in PermissionMixin or in a new BaseBackend class.


Did you mean has_perms()? Also, `PermissionsMixin` is a mixin for a user model 
class; it isn't the right target here. Moving generic parts of ModelBackend 
into a BaseBackend class would be good. This is where you start sorting APIs 
between my categories A and B.

Writing a good description of what changes you're planning to make and why in a 
Trac ticket should be enough — especially if you can divide changes in several 
steps, with one ticket per step. I don't think a DEP is necessary.

Cheers,

-- 
Aymeric.



> On 12 Nov 2018, at 17:57, Collin Anderson  wrote:
> 
> 
> > Add default implementations for get_all_permissions() and has_perm(), 
> > either in PermissionMixin or in a new BaseBackend class.
> On a first glance, I think that makes sense to me.
> 
> > Also note that the separation between user and group permissions may not be 
> > applicable with custom backends.
> That also makes sense to me. It does seems like kind of an implementation 
> detail as to where the permission is actually coming from, though I think 
> that information could be useful in some cases, so somehow making it optional 
> would be nice.
> 
> On Sat, Nov 10, 2018 at 3:07 AM Tobias Bengfort  > wrote:
> I feel like the interface for authentication backends is unnecessarily
> complex: Basically, you only need authenticate() and has_perm(), but
> currently the interface also includes get_group_permissions(),
> get_all_permissions(), and has_module_perms().
> 
> The architecture is like this: User inherits from PermissionMixin which
> implements the methods get_group_permissions(), get_all_permissions(),
> has_perm(), has_perms(), and has_module_perms().
> 
> All of these methods basically just call the corresponding methods on
> the authentication backends, if available. has_perms() is the only
> exception, as it is just a shortcut to call has_perm() multiple times.
> 
> I believe that has_perm() is vastly more important than
> get_*_permissions(). Still, I can understand that the latter exist. What
> I find confusing is that there is no get_user_permissions() in this
> interface. Whithout its counterpart, get_group_permissions() seems
> pretty much useless to me. Also note that the separation between user
> and group permissions may not be applicable with custom backends.
> 
> Another issue is that developers can easily end up with inconsistent
> backends: There is no guarantee that get_all_permissions() will return a
> superset of get_group_permissions(). There is also no guarantee that
> has_perm() will be equivalent to `perm in get_all_permissions()`.
> 
> Then there is the issue of has_module_perms(). As far as I understand,
> this should have been called has_app_perms(). It is used in
> contrib.admin where it makes a lot of sense, but I don't really see how
> it could be useful anywhere else.
> 
> I looked at some popular backends to see how they are implemented:
> 
> The default ModelBackend implements get_user_permissions() and
> get_group_permissions(). get_all_permissions() joins their results and
> has_perm() and has_module_perms() call get_all_permissions(). The
> results of get_all_permissions() are cached.
> 
> django-guardian works pretty much the same. However, even though the
> mechanism is the same, get_group_permissions() is not exposed in the
> interface. has_module_perms() is missing.
> 
> django-rules only implements has_perm(). get_*_permissions() are
> missing. has_module_perms() exists, but it is just an alias to
> has_perm().
> 
> I guess the benefits of simplifying the interface do not justify
> breaking changes for most of these issues. However, I think there are
> two changes that could significantly improve the situation:
> 
> -   Either add get_user_permissions() or remove get_group_permissi

Re: BitBounce Spam Replies From the Mailing List

[Aymeric Augustin]

I was pessimistic; it seems that a BitBounce employee might read this message.

So, BitBounce, if you’re reading this:

1. Please drop aphiso...@gmail.com from your service before they do more harm 
to your reputation — there are thousands of subscribers to this mailing list — 
and until they unsubscribe from all their mailing lists.

2. Please make a donation of $2000 to the Django Software Foundation ($1000 if 
you’re cheap) to offset the trouble you’ve caused up to this point: 
https://www.djangoproject.com/fundraising/.You’re the ones who started 
suggesting money was an appropriate compensation for unsolicited email after 
all :-) and you sent quite a lot to us :-(

Looking forwards to a prompt resolution,

-- 
Aymeric.

> Le 6 janv. 2019 à 10:35, Aymeric Augustin 
>  a écrit :
> 
> [[ I'm adding BitBounce support to this discussion, even though I expect I'll 
> just get one more of their spam — "we'll ignore the spam problem we cause to 
> you until you pay money to us". ]]
> 
>>> On 2 Jan 2019, at 01:39, Daniele Procida  wrote:
>>> 
>>>> On Wed, Jan 2, 2019, Daniele Procida  wrote:
>>>> 
>>>> On Tue, Jan 1, 2019, Daniele Procida  wrote:
>>>> 
>>>> If it continues to be an issue I will disable their receipt of email
>>>> temporarily. I assume I'll get the auto-replies myself to this message.
>>> 
>>> One user's email receipt disabled so far; the user has been informed.
>> 
>> ... and the other bounces from an email address that isn't even subscribed 
>> to the list!
>> 
>> That is extremely annoying.
> 
> It's aggravating.
> 
> Here's one possible solution:
> 
> - we export the individual email addresses of django-developers subscribers — 
> assuming Google Groups allows that
> - we write a script that sends an email to each of them with a unique sender 
> or subject — as far as I can tell these are the only parts of the original 
> email that make it into BitBounce's spam
> - when we get the BitBounce spam, we can map the sender or subject to the 
> original email address, then we unsubscribe it
> 
> We can make tests by writing to aphiso...@gmail.com — the address indirectly 
> subscribed to django-developers that we're trying to remove.
> 
> It seems that BitBounce's spam takes a while to arrive. Yesterday I emailed 
> the list at 23:24 and got the BitBounce spam at 00:00 (CET). Perhaps it's a 
> hourly cron, perhaps just a slow queue.
> 
> Best regards,
> 
> -- 
> Aymeric.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/96524150-766E-4DB8-9654-4EBB44CF3C89%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: BitBounce Spam Replies From the Mailing List

[Aymeric Augustin]

[[ I'm adding BitBounce support to this discussion, even though I expect I'll 
just get one more of their spam — "we'll ignore the spam problem we cause to 
you until you pay money to us". ]]

> On 2 Jan 2019, at 01:39, Daniele Procida  wrote:
> 
> On Wed, Jan 2, 2019, Daniele Procida  wrote:
> 
>> On Tue, Jan 1, 2019, Daniele Procida  wrote:
>> 
>>> If it continues to be an issue I will disable their receipt of email
>>> temporarily. I assume I'll get the auto-replies myself to this message.
>> 
>> One user's email receipt disabled so far; the user has been informed.
> 
> ... and the other bounces from an email address that isn't even subscribed to 
> the list!
> 
> That is extremely annoying.

It's aggravating.

Here's one possible solution:

- we export the individual email addresses of django-developers subscribers — 
assuming Google Groups allows that
- we write a script that sends an email to each of them with a unique sender or 
subject — as far as I can tell these are the only parts of the original email 
that make it into BitBounce's spam
- when we get the BitBounce spam, we can map the sender or subject to the 
original email address, then we unsubscribe it

We can make tests by writing to aphiso...@gmail.com 
 — the address indirectly subscribed to 
django-developers that we're trying to remove.

It seems that BitBounce's spam takes a while to arrive. Yesterday I emailed the 
list at 23:24 and got the BitBounce spam at 00:00 (CET). Perhaps it's a hourly 
cron, perhaps just a slow queue.

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/BEABF731-C4BC-417B-9E1B-A3BAE45E729C%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: New optional settings - disable bulk_create in all managers and call self.full_clean() before saving the models

[Aymeric Augustin]

Hello,

Your ValidateModelMixin and ManagerMixin look perfectly reasonable. You wanted 
to customize two aspects of Django's behavior. You managed to do so in four 
lines and three lines respectively. (I'm leaving out `id_generator` which is 
another concern.)

Preventing database writes without model validation would require further 
changes. You'd have to disable `update`, `bulk_update`, `raw` and perhaps 
others. Even then it would still be possible to use `django.db.connection` to 
make arbitrary queries, etc.

What I'm saying here is — I'm not sure there's a useful line that Django can 
draw and that would work for most projects. I wouldn't use the same constraints 
if I was selling theater tickets or managing access control for health records.

Generally speaking, developers write code that makes (hopefully correct) 
changes to the database. Each team devises best practices to guarantee a 
suitable level of quality. Models and managers mixins are a good technique for 
that.

Best regards,

-- 
Aymeric.



> On 5 Jan 2019, at 22:14, ⁨אורי⁩ <⁨u...@speedy.net⁩> wrote:
> 
> Hi,
> 
> I added this new ticket today and I would like to know what is your feedback 
> or comments?
> 
> https://code.djangoproject.com/ticket/30080 
> 
> 
> We are using Django for Speedy Net and Speedy Match 
>  (currently Django 1.11.18, we 
> can't upgrade to a newer version of Django because of one of our 
> requirements, django-modeltranslation). I want to validate each model before 
> saving it to the database, so I'm using class ValidateModelMixin as the base 
> of each model.
> 
> For this reason, I want to disable bulk_create in all managers in all of our 
> models. So I added this code:
> 
> class ValidateModelMixin(object):
> def save(self, *args, **kwargs):
> """Call `full_clean` before saving."""
> self.full_clean()
> return super().save(*args, **kwargs)
> 
> 
> class ManagerMixin(object):
> def bulk_create(self, *args, **kwargs):
> raise NotImplementedError("bulk_create is not implemented.")
> 
> 
> class BaseModel(ValidateModelMixin, models.Model):
> def save(self, *args, **kwargs):
> try:
> field = self._meta.get_field('id')
> if ((not (self.id)) and (hasattr(field, 'id_generator'))):
> self.id = field.id_generator()
> while (self._meta.model.objects.filter(id=self.id).exists()):
> self.id = field.id_generator()
> except FieldDoesNotExist:
> pass
> return super().save(*args, **kwargs)
> 
> class Meta:
> abstract = True
> 
> 
> class TimeStampedModel(BaseModel):
> date_created = models.DateTimeField(auto_now_add=True, db_index=True)
> date_updated = models.DateTimeField(auto_now=True, db_index=True)
> 
> class Meta:
> abstract = True
> 
> 
> class BaseManager(ManagerMixin, models.Manager):
> pass
> 
> 
> class BaseUserManager(ManagerMixin, DjangoBaseUserManager):
> pass
> 
> 
> I thought maybe it's good to add these settings - disable bulk_create in all 
> managers and call self.full_clean() before saving the models - as an optional 
> settings both in the project and also in each model (maybe in class Meta) so 
> it will be possible to override Django's default both per-project and also 
> for any specific model. I understand that the default is not to call 
> self.full_clean() before saving the models and to allow bulk_create in the 
> managers, but I suspect this may lead to invalid data in the database of the 
> projects.
> 
> The current code in the master is on 
> https://github.com/speedy-net/speedy-net/blob/master/speedy/core/base/models.py
>  
> ,
>  and the code in the branch I'm currently working on is on 
> https://github.com/speedy-net/speedy-net/blob/uri_merge_with_master_2019-01-05_a/speedy/core/base/models.py
>  
> .
> 
> Thanks,
> אורי (Uri)
> u...@speedy.net 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABD5YeHZNA94pOFcWbxbeP5%2BShE1XMZuqx-Rcfit%2BSYvHz-2qQ%40mail.gmail.com
>  

Re: Consensus about visual diagrams in the docs.

[Aymeric Augustin]

Hello,

A few years ago, I redid most illustrations and included them in the docs as 
SVG files. The primary driver was switching from PNG to SVG to better 
accommodate hi-dpi displays. I also committed the source files. Unfortunately, 
they're in a proprietary format and cannot be reused without a Mac and an 
OmniGraffle licence.

(Before someone starts listing the arguments against proprietary formats — yes, 
I learnt them 15 years ago, and yes, I still chose OmniGraffle's UX over 
Inkscape's. : You're welcome to redo the diagrams with Inkscape and replace 
mine as long as the result doesn't look significantly worse.)

While not ideal, this got the job done. Diagrams look all right on all kind of 
devices.

To sum up:
- I think SVG should be the preferred format for illustrations in the docs
- I'm worried about ASCII art on small devices: line wrapping would be a problem
- whatever you want to make the diagram as long as it exports to SVG and looks 
reasonable
- commit the source file so someone has a chance to reuse them, regardless of 
their format

Best regards,

-- 
Aymeric.



> On 30 Nov 2018, at 09:38, guettli  wrote:
> 
> I once created a simple ascii diagram to explain details of the Django ORM:
> 
> https://code.djangoproject.com/ticket/27936
> 
> Years ago it took some time for me to understand it. And since then it
> came up by new comers in our team several times.
> 
> There is agreement on the goal: yes, a visual diagrams in the docs would be 
> nice.
> 
> But there is no agreement on the strategy.
> 
> I think ascii art is fine. Some suggest to use a graphiz plugin in for sphinx.
> 
> There is no progress because there is no clear consensus up to now.
> 
> I do not care for the strategy, I care for the goal.
> 
> How to find clear consensus now?
> 
> Regards,
>Thomas Güttler
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/2f6f3c4e-44cf-4b35-bb09-751828cc4921%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/D58AA4E7-AC5B-425E-BC70-A63C3D11FC87%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Extending django.contrib.auth.models Group name to be 191 character max_length

[Aymeric Augustin]

That seems all right to me.

-- 
Aymeric.



> On 13 Nov 2018, at 16:56, joh...@alumni.upenn.edu wrote:
> 
> If the other fields are also 191, i see no reason not to make the change.  It 
> should be a non-breaking change.  And Yes i am that colleague...
> 
> On Tuesday, November 13, 2018 at 4:52:24 PM UTC+1, Tim Allen wrote:
> Greetings,
> 
> Over the past few releases, several CharFields in django.contrib.auth have 
> been increased to a max_length of 191: username, first_name, and last_name 
> immediately come to mind.
> 
> See: 
> https://groups.google.com/forum/#!msg/django-developers/h98-oEi7z7g/xzjtFMf1BwAJ
>  
> 
> 
> I've issued a PR to extend Group.name to the same length here: 
> https://github.com/django/django/pull/10627 
>  The original ticket this is 
> addressing was from a colleague who was bumping into the limit when syncing 
> groups to Django from LDAP: https://code.djangoproject.com/ticket/29939 
> 
> 
> The 191 character limit was chosen to support MySQL index limits.
> 
> Can anyone think of a reason not to change Group.name to be max_length=191 as 
> we have with the other fields?
> 
> Regards,
> 
> Tim
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/1f00582c-452c-4d56-9450-967a31818bea%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3105F97F-523E-49A8-A4F2-1947ACF0D736%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Pluggable secret key backend

[Aymeric Augustin]

Good point, I can think of at least two apps of mine that would break. 
Transitioning to a new setting makes more sense.

-- 
Aymeric.



> On 11 Nov 2018, at 18:58, Tom Forbes  wrote:
> 
> Is it going to be easy to adjust the semantics of SECRET_KEY to support 
> sequences like that? I’d imagine a lot of third party packages that expect 
> SECRET_KEY to be a string would break in weird ways (thanks to both strings 
> and tuples of strings being iterables that yield strings).
> 
> Here’s a quick search on GitHub: 
> https://github.com/search?q=%22settings.SECRET_KEY%22&type=Code
> 
> To ease the backward compatibility concerns we could use SECRET_KEYS, then 
> make SECRET_KEY (if it is not explicitly defined) map to SECRET_KEYS[0]? 
> Third party packages using would not necessarily work with the backwards 
> verification but they would at least not break and continue to work as 
> expected.
> 
> Tom
> 
> 
> 
> 
> 
> On 11 November 2018 at 07:38:15, Aymeric Augustin 
> (aymeric.augus...@polytechnique.org) wrote:
> 
>> Hello,
>> 
>> I think this is a great idea.
>> 
>> As suggested by others, an even better default implementation would be:
>> 
>> class SecretKeysBackend:
>> 
>> def get_signing_key(self):
>> if isinstance(settings.SECRET_KEY, (list, tuple)):
>> return settings.SECRET_KEY[0]
>> else:
>> return settings.SECRET_KEY
>> 
>> def get_verification_keys(self):
>> if isinstance(settings.SECRET_KEY, (list, tuple)):
>> return settings.SECRET_KEY
>> else:
>> return [settings.SECRET_KEY]
>> 
>> Once Django is updated to take advantage of this feature, hat would make key 
>> rotation practical for every Django user!
>> 
>> (And it seems easier to adjust the semantics of SECRET_KEY than to introduce 
>> a SECRET_KEYS settings.)
>> 
>> Best regards,
>> 
>> -- 
>> Aymeric.
>> 
>> 
>> 
>>> On 10 Nov 2018, at 11:12, Andreas Pelme  wrote:
>>> 
>>> Hi,
>>> 
>>> settings.SECRET_KEY can be used for sessions, password resets, form wizards 
>>> and
>>> other cryptographic signatures via the signing APIs. Changing SECRET_KEY 
>>> means
>>> that all of those will be invalidated and the users will be affected in 
>>> weird
>>> ways without really knowing what happened. (Why am I logged out? Where did 
>>> my
>>> form submission go? Why does not this password reset link work?). This is
>>> desirable in case the key is compromised and swift action must be taken.
>>> 
>>> There are other situations when it would be nice to change the SECRET_KEY 
>>> when
>>> this sudden invalidation is not desirable:
>>> 
>>> - When someone leaves a project/company that had access to the production
>>>  system. After SSH keys/login credentials is revoked the developer could
>>>  potentially have a copy of the secret key. It is essentially a backdoor 
>>> with
>>>  full remote access. It would be wise to rotate the key in those cases.
>>> 
>>> - Periodic and automatic rotations of keys to make it less useful in the
>>>  future.
>>> 
>>> The current situation of a single SECRET_KEY makes key rotation 
>>> impractical. If
>>> you run a busy site with active users 24/7, there is never a nice time to
>>> change the SECRET_KEY.
>>> 
>>> A solution for this problem would be sign new secrets with a new key while
>>> still allow signatures made with the old key to be considered valid at the 
>>> same
>>> time. Changing keys and having a couple of hours of overlap where signatures
>>> from both keys are accepted would mitigate most of the user facing problems
>>> with invalidating sessions, password reset links and form wizard progress.
>>> 
>>> You could do this today by implementing your own session backend, message
>>> storage backend and password reset token generator but that is cumbersome 
>>> and
>>> does not work across reusable apps that directly use low level Django 
>>> signing
>>> APIs unless they too provide hooks to provide your own secret.
>>> 
>>> I propose a pluggable project wide secret key backend
>>> (settings.SECRET_KEY_BACKEND maybe?) with an API something like:
>>> 
>>> class SecretKeyBackend:
>>>  def get_signing_key(self): …
>>>  def get_verification_keys(self): ...
>>> 
>&

Re: Pluggable secret key backend

[Aymeric Augustin]

Hello,

I think this is a great idea.

As suggested by others, an even better default implementation would be:

class SecretKeysBackend:

def get_signing_key(self):
if isinstance(settings.SECRET_KEY, (list, tuple)):
return settings.SECRET_KEY[0]
else:
return settings.SECRET_KEY

def get_verification_keys(self):
if isinstance(settings.SECRET_KEY, (list, tuple)):
return settings.SECRET_KEY
else:
return [settings.SECRET_KEY]

Once Django is updated to take advantage of this feature, hat would make key 
rotation practical for every Django user!

(And it seems easier to adjust the semantics of SECRET_KEY than to introduce a 
SECRET_KEYS settings.)

Best regards,

-- 
Aymeric.



> On 10 Nov 2018, at 11:12, Andreas Pelme  wrote:
> 
> Hi,
> 
> settings.SECRET_KEY can be used for sessions, password resets, form wizards 
> and
> other cryptographic signatures via the signing APIs. Changing SECRET_KEY means
> that all of those will be invalidated and the users will be affected in weird
> ways without really knowing what happened. (Why am I logged out? Where did my
> form submission go? Why does not this password reset link work?). This is
> desirable in case the key is compromised and swift action must be taken.
> 
> There are other situations when it would be nice to change the SECRET_KEY when
> this sudden invalidation is not desirable:
> 
> - When someone leaves a project/company that had access to the production
>  system. After SSH keys/login credentials is revoked the developer could
>  potentially have a copy of the secret key. It is essentially a backdoor with
>  full remote access. It would be wise to rotate the key in those cases.
> 
> - Periodic and automatic rotations of keys to make it less useful in the
>  future.
> 
> The current situation of a single SECRET_KEY makes key rotation impractical. 
> If
> you run a busy site with active users 24/7, there is never a nice time to
> change the SECRET_KEY.
> 
> A solution for this problem would be sign new secrets with a new key while
> still allow signatures made with the old key to be considered valid at the 
> same
> time. Changing keys and having a couple of hours of overlap where signatures
> from both keys are accepted would mitigate most of the user facing problems
> with invalidating sessions, password reset links and form wizard progress.
> 
> You could do this today by implementing your own session backend, message
> storage backend and password reset token generator but that is cumbersome and
> does not work across reusable apps that directly use low level Django signing
> APIs unless they too provide hooks to provide your own secret.
> 
> I propose a pluggable project wide secret key backend
> (settings.SECRET_KEY_BACKEND maybe?) with an API something like:
> 
> class SecretKeyBackend:
>  def get_signing_key(self): …
>  def get_verification_keys(self): ...
> 
> The default (and backward compatible) backend would then be implemented as
> something like:
> 
> class SecretKeySettingsBackend:
>  def get_signing_key(self):
>return settings.SECRET_KEY
>  def get_verification_keys(self):
>return [settings.SECRET_KEY]
> 
> django.core.signing.Signer.{sign,unsign} would need to be updated to use this
> backend instead of directly using settings.SECRET_KEY.
> 
> That would solve the problem project wide and work across any third party
> application that uses django.core.signing directly.
> 
> This would open the door for third party secrets backend packages that
> retrieves keys from systems such as Hashicorp Vault, AWS Secrets Manager,
> Google Cloud KMS, Docker Secrets etc.
> 
> Having a method that retrieves the key would allow changes to secret key 
> during
> run time instead of relying on a hard coded setting would allow the key to
> change without restarting the server process.
> 
> Would something like this be worth pursuing? Could it be designed in som other
> way? I could not find any previous discussion/tickets on this and thought it
> would be a good idea to discuss it here before opening a ticket or making an
> attempt at a PR. :)
> 
> Cheers,
> 
> Andreas
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from thi

Re: Make `raw_id_fields` the default functionality in the admin

[Aymeric Augustin]

On 6 Nov 2018, at 02:47, 'Ivan Anishchuk' via Django developers (Contributions 
to Django itself)  wrote:

> How about some field that intelligently guesses the table size and looks 
> either like default select or like raw id field depending on that?


Yes, that would be interesting.

I implemented a manual version of this in a project:
- I ran a script to count the number of instances of each model and put those 
with more than 100 or 1000 instances in a list
- I wrote a custom ModelAdmin class that automatically added ForeignKeys to 
models in that list to raw_id_fields

Best regards,

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/EF75EAC1-229E-47B0-ABBB-9C45598D933F%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: django.utils.timezone.make_aware confusion

[Aymeric Augustin]

Hello Dan,

If that assertion fails, I assume the default time zone (the TIME_ZONE setting) 
isn't Europe/Berlin.

See 
https://docs.djangoproject.com/en/2.1/ref/utils/#django.utils.timezone.make_aware
 


If you change it to dt2 = make_aware(datetime(2018, 10, 24), euberlin), it 
should work.

Best regards,

-- 
Aymeric.



> On 6 Nov 2018, at 06:21, Dan Davis  wrote:
> 
> Working on ticket #29984, I noticed that the following raises:
> 
> from datetime import datetime
> import pytz
> from django.utils.timezone import make_aware
> 
> euberlin = pytz.timezone('Europe/Berlin')s a
> dt1 = datetime(2018, 10, 24, tzinfo=euberlin)
> dt2 = make_aware(datetime(2018, 10, 24))
> assert dt1 == dt2
> 
> It is certainly necessary to have make_aware, because datetime may come in 
> without a timezone.  I'm familiar with the various solutions - user picks a 
> timezone in their profile, and/or JavaScript detects the timezone in the 
> browser and provides that to the backend at some point.  I'm even urging a 
> coworker to fix his public Django webapp to implement the latter solution.
> 
> However, it is disturbing that the datetimes are not equal.   Can someone 
> explain why not?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/46a382ef-2b19-4b81-87b8-babb1f93b9cc%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4EDAF400-ECC6-403C-85D4-8C9D9872BC90%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: django migrations on live database for large tables for postgresql

[Aymeric Augustin]

Hello Pavel,

Sorry for the delay in my answer.

Unfortunately I don't know django.db.migrations very well and I won't be able 
to help you with the details of a patch in this area.

I still think improving support for zero downtime migrations would be 
worthwhile. Writing a proposition in a ticket (or, if you're feeling really 
ambition, in a DEP) would be a good way to start.

Sorry I can't help more!

Best regards,

-- 
Aymeric.



> On 7 Oct 2018, at 20:53, pavel.tysla...@gmail.com wrote:
> 
> Hello, Aymeric
> 
> Thanks for your reply.
> 
> CREATE INDEX CONCURRENTLY is a big part for zero downtime migrations support, 
> but there are too many additional questions as should ALTER TABLE ADD COLUMN 
> UNIQUE splitted to CREATE INDEX and constraint creation or should be handled 
> FOREGN KEY, CHECK and NOT NULL constraints creation that can take ACCESS 
> EXCLUSIVE for long time on huge table or is it accessible to avoid 
> transactions in highlighted above cases?
> 
> Unfortunately I don't know what direction django planned to develop, so I can 
> go different ways without support.
> 
> Is it a good idea to create ticket about zero downtime migrations support 
> with detailed proposition of improvements and backward compatibility support?
> 
> Regards, Paveł
> 
> On Sunday, 7 October 2018 10:31:27 UTC+3, Aymeric Augustin wrote:
> Hello Pavel,
> 
> There's an accepted ticket about CREATE INDEX CONCURRENTLY: 
> https://code.djangoproject.com/ticket/21039 
> <https://code.djangoproject.com/ticket/21039>. A concrete proposal for adding 
> this feature to Django would be interesting.
> 
> I'm not really sure about your two other suggestions. If you think they're 
> improving the PostgreSQL backend, you're welcome to create a ticket and 
> propose a pull request.
> 
> Keep in mind backwards-compatibility if changes are improvements in your use 
> case but perhaps not for everyone else. For example, it may be better that 
> index creating remain an atomic operation (so, not concurrent) by default.
> 
> Best regards,
> 
> -- 
> Aymeric.
> 
> 
> 
>> On 26 Sep 2018, at 07:41, pavel.t...@ <>gmail.com <http://gmail.com/> wrote:
>> 
>> Hi,
>> 
>> I found that django migrations don't designed for live updates on large 
>> tables for postgres, eg. index creation doesn't use concurrently or alter 
>> table operation can take ACCESS EXCLUSIVE lock for long time.
>> 
>> Before I mostly wrote custom migrations, but after one of downtime I decided 
>> to avoid most common cases at all and wrote postgres backend that helps me 
>> in it: https://github.com/tbicr/django-pg-zero-downtime-migrations 
>> <https://github.com/tbicr/django-pg-zero-downtime-migrations>. When I wrote 
>> it I found that it's a pretty easy extend core postgres backend, but a few 
>> things required rewrite on my side:
>> 
>> - 
>> https://github.com/django/django/blob/master/django/db/backends/base/schema.py#L152
>>  
>> <https://github.com/django/django/blob/master/django/db/backends/base/schema.py#L152>
>>  used both for table columns creation and altering table add column, but not 
>> null constraint and index constraints can't be changed without whole method 
>> rewriting
>> - for one core sql statement I found more "safe" equivalent with two sql 
>> statements, but it's a bit hard to execute it, for example more save create 
>> invalid constraint and than validate it or create concurrently index and 
>> than create unique constraint
>> 
>> also I found that not all constraint names quoted in generated sql, for 
>> example add_unique_together got `ALTER TABLE "table_name" ADD CONSTRAINT 
>> table_name_f1_f2_hash_uniq UNIQUE ("f1", "f2")`, but expected `ALTER TABLE 
>> "table_name" ADD CONSTRAINT "table_name_f1_f2_hash_uniq" UNIQUE ("f1", "f2")`
>> 
>> So I just curious is anybody interested to make django migrations more 
>> reliable for large tables in prod or interested in some improvements 
>> highlighted above, that can simplify writing backends for this purpose?
>> 
>> I also will be appreciated if somebody can share another solutions for this 
>> purpose.
>> 
>> Paveł
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@ <>googlegroups.com 

Re: python-memcached seems to be unmaintained

[Aymeric Augustin]

Hello Adrian,

I think that's a good plan.

If there's opposition to deprecating the python-memcached backend now, it would 
still be possible to introduce and recommend the pymemcache backend.

Best regards,

-- 
Aymeric.



> On 29 Oct 2018, at 01:42, Adrian Turjak  wrote:
> 
> Through some of my last few projects using Django and Memcached I kept
> running into the problem that python-memcached appears to no longer be
> maintained[1], and even before that the release frequency was getting
> quite low.
> 
> For Django pylibmc is an alternative, but having to rely on an
> underlying C library is a bit annoying and the library does act a little
> differently.
> 
> Give the state of python-memcached I think Django should support
> pymemcache[2]. It is actively maintained by Pintrest, with appropriate
> opensource licensing, is fully python, and appears to be a great library.
> 
> There appear to have been attempts by thirdparties to make a Django
> cache backend for it, but none of those are maintained. Bringing the
> support into core, and replacing python-memcached with pymemcache may be
> a better good long term solution than continuing to support and
> recommend an unmaintained library when there is a suitable alternative.
> 
> I opened this ticket here to track that, and the recommendation was to
> bring it (rightly so) to the mailing list:
> https://code.djangoproject.com/ticket/29887
> 
> I'm also a member of the OpenStack community and I've talked to devs who
> were interested in taking over python-memcached as per the discussion in
> the linked issue, but it looks like OpenStack's oslo.cache is also
> switching to pymemcache at this stage because it offers them a great
> alternative.
> 
> Any thoughts?
> 
> 
> 1: https://github.com/linsomniac/python-memcached/issues/95
> 2: https://github.com/pinterest/pymemcache
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3ad5467c-8073-7273-ae5b-00f5dd713efa%40catalyst.net.nz.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3A4436B8-4102-42DE-9905-34F5DD9569F9%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.