RE: Can we move the activity on this list to the Forum now?

[Arthur Rio]

Yes please!


On May 3, 2023 at 11:19:12 PM, jure.erznoz...@gmail.com (
jure.erznoz...@gmail.com) wrote:

+1



*From:* django-developers@googlegroups.com <
django-developers@googlegroups.com> *On Behalf Of *natali...@gmail.com
*Sent:* sreda, 03. maj 2023 20:10
*To:* Django developers (Contributions to Django itself) <
django-developers@googlegroups.com>
*Subject:* Re: Can we move the activity on this list to the Forum now?



Hello everyone!



I was wondering if we could make a decision about this topic. On the one
hand, and as far as I understand, the forum is the preferred channel of
communication. On the other hand, having multiple channels of communication
can spread important discussions too thin, making it difficult to have
productive conversations in one place.



As a newcomer to the contributing community, I can attest that the current
situation causes some confusion. IMHO, the same applies to the chat
options: there is IRC and the Discord server (though perhaps I should start
a new forum topic about that in order to keep decisions separated).



In summary, I'm +1 to "move on-mass (all few of us :) over there"!



Thank you!

Natalia.

On Thursday, January 26, 2023 at 10:07:36 AM UTC-3 Alex Krupp wrote:

I just submitted a proposal to create a Discourse plugin to improve the
accuracy of their inbound email parsing. This would enable two things:

   - Folks who prefer to live in their inbox could continue to do so and
   contribute by just replying to emails. Discourse currently has
   reply-by-email, but it often mangles formatting and/or entirely deletes
   text. Once these issues are fixed, folks who like the current experience
   would be able to just pretend the forum doesn't exist and continue having
   the same experience as they currently have with Google Groups.
   - Right now importing the archives from Google Groups into Discourse
   isn't realistic for the same reasons; some messages will import correctly,
   but others will be mangled or missing text. This would also be made
   possible.

If this is something you care about (positively or negatively), here is the
exploratory proposal:



https://meta.discourse.org/t/proposed-plugin-to-improve-reply-by-email-accuracy/252944



Any feedback and/or testing would be much appreciated! Right now Discourse
recognizes that this is a problem and is interested in solving it, but
getting it prioritized will require folks to A) speak up saying they want
it done B) test the underlying API to verify that it actually solves the
problem.



Alex



On Fri, Jan 20, 2023 at 2:24 AM Carlton Gibson  wrote:

OK, I've updated the Trac wiki page to point to the forum too. (We use this
as a wiki-link in responses to guide folk onwards.)



The docs have a `|django-developers|` shortcut used in quite a few places
so that needs a moment more to review and adjust.

(I'll look at it soon-ish... — if no-one beats me to it :)



Thanks.

C



On Thu, 19 Jan 2023 at 18:15, Andrew Godwin  wrote:

We should at least update those Trac and Triage Workflow docs to point to
both, maybe with the Forum first?



Andrew



On Thu, Jan 19, 2023, at 12:30 AM, Carlton Gibson wrote:

I'm trying to begin new conversations there where I can.



The main issue is that we're still pointing people here from Trac and the
Triage Workflow docs — so if there's a rough "Yeah, let's do it" we can
adjust practice there.



I expect there will always be the "How do I start?" posts. The Forum at
least has a pinned post for that. ...



On Thu, 19 Jan 2023 at 01:04, 'Kye Russell' via Django developers
(Contributions to Django itself)  wrote:

Hi all,



I find that the signal-to-noise ratio on this mailing list is (by my
determination) quite bad around this time of year.



Is a move to the forum still on the cards?



Kye

On 6 Dec 2022 at 7:16 AM +0800, Andrew Godwin , wrote:



I did some investigation of moving django-users and django-developers to
the Forum right after DjangoCon; I wanted to see if we could import all the
old posts too, which we probably could, but I'm not entirely sure of the
utility of that.



I will say that the forum is a lot easier to moderate - the ability to
moderate after a post has gone out, rather than gating all posts behind
approval if they're untrusted, is a big step in itself, not to mention the
ability to remove sensitive or offensive content once it's posted.



Andrew



On Monday, November 28, 2022 at 10:01:17 PM UTC-7 m...@kye.id.au wrote:

IMO django-announce and django-updates serve a very different purpose and I
would be against moving them if it were suggested.



I am incredibly strongly in favour of moving django-developers and
django-users to the forums. IMO being able to more easily trap people
misusing this list as a tech support channel is itself reason enough to
move. Beyond that, I’d argue that the plentiful UX issues with Google
Groups, and mailing lists in general, certainly don’t do the community any
favours in terms of getting more 

Re: Content types shouldn't be created on post_migrate signal

[Arthur Rio]

Hi everyone 

After a long period of inactivity I'm back at it with the help of Taylor.

As a reminder: 
Here is the ticket: https://code.djangoproject.com/ticket/29843 
Here is the initial PR: https://github.com/django/django/pull/10540

After some experiments and discussions it felt like while the 
implementation might solve the initial problem, it's a bit under the hood 
and will be hard to debug if something goes wrong.
The idea was to inject operations at the time of running `migrate`.

So... we went back to the idea of having hooks during the `makemigrations` 
process instead, so that the operations would be written to the migration 
files, which would make it more explicit and less risky. Here is a first 
draft of how it would look like: https://github.com/django/django/pull/14229
.

1. We know that the `makemigrations` process is complicated, so before we 
invest more time down that path, is there something obvious we might be 
missing?
2. What do you think of this approach with hooks (pre and post 
"add_operation")?
3. Do you think it would be a useful feature for other third party apps as 
well (not just content types and permissions)?

Thank you for your input, stay safe

Arthur & Taylor

On Saturday, October 13, 2018 at 12:56:25 PM UTC-6 petter.s...@gmail.com 
wrote:

> I encountered a similar issue recently, but with auth permissions.
>
> It is described here: https://code.djangoproject.com/ticket/29843
>
> On Wednesday, October 3, 2018 at 1:01:37 PM UTC+2, Marcin Nowak wrote:
>>
>> Hello.
>>
>> There is a huge issue with content types framework. The data is loaded 
>> after every migration (post_migrate signal) automatically, and this 
>> approach is against db data consistency.
>> The biggest problem is with data migrations, which are applied at the 
>> first time (on clean database). Any data migration which depends on some 
>> content types will not insert the data.
>>
>> The sequence looks like this:
>>
>>1. create db
>>2. insert into auth_permission ...  inner join django_content_type ...
>>3. post migrate: insert into django_content_type ...
>>
>>
>> Two things are wrong:
>>
>>- automatic creation of content types
>>- creation of content types after running migrations
>>
>> Solution:
>>
>>- creation of new app should add very first migration, which will add 
>>entry to the content types
>>- create_contentypes handler should be removed from post_migrate 
>>signal
>>
>> Any data migration, which depends on some content types, should be 
>> declared with proper dependency. This will guarantee existence of the 
>> required content type entry.
>>
>>
>> BR,
>> Marcin
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9f6f9bf3-ad26-4874-939e-a7ad3e201a98n%40googlegroups.com.

Re: Moving rest_framework.status into core?

[Arthur Rio]

The one thing that I really prefer from DRF's statuses is their name,
containing the status_code. I used python's `http.HTTPStatus` in a flask
project and I had to have the python docs open
all the time to map between the status code and the name of the status. For
example:
- http.HTTPStatus.BAD_GATEWAY
- rest_framework.status.HTTP_502_BAD_GATEWAY

I don't know if it's enough of an argument to justify maintaining extra
code in Django, but as Carlton said, it's just a file with constants. If
you look at the file in DRF, there haven been very little changes to it
https://github.com/encode/django-rest-framework/commits/master/rest_framework/status.py
and
in my opinion, it does make the developer experience nicer.

Best,

Arthur


On March 23, 2021 at 9:24:32 AM, 'Adam Johnson' via Django developers
(Contributions to Django itself) (django-developers@googlegroups.com) wrote:

Python 3.5 added http.HTTPStatus:
https://docs.python.org/3.9/library/http.html#http.HTTPStatus , so I don't
see a need for such constants in Django, unless they add something extra.
Perhaps you could deprecate from DRF.

On Tue, 23 Mar 2021 at 15:18, Carlton Gibson 
wrote:

> Hiya.
>
> Maybe my search-foo is down but, surprisingly, I couldn't find a
> discussion on this.
>
> Should we maybe bring the HTTP status code constants that we find in
> rest_framework.status into core, as django.http.status?
>
> I find myself wanting them (not for the first time) in a project that
> doesn't really depend on DRF, and thinking (surely) these fit comfortably
> in Django's "I'm a web framework" remit. 樂
>
> Can I solicit opinions on that please? It's just a list of constants so...
> It seems to fit the "bring some of the more reusable bits of DRF into
> core" theme.
>
> Thanks.
>
> Kind regards,
> Carlton
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/541cf57b-f369-4dc9-a06e-e3ff8bf4485cn%40googlegroups.com
> 
> .
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAMyDDM0F9Ly_SMLnSvZRsO4%2BY_Zum9r%2BpvM1H4ZAAgF3mb%2BV3Q%40mail.gmail.com

.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADOBPEEfe_5bhizzaE74fmvcwpmYC5NOb5NA2yVqt_n%2BrDE5mA%40mail.gmail.com.

Re: Support for unittest -k option

[Arthur Rio]

I’m +1 on no short-hand. Also “kdb” is a little to close to “pdb” and
doesn’t really make sense to me.

—
Arthur

On March 11, 2019 at 3:33:50 PM, Dan Davis (dansm...@gmail.com) wrote:

I personally don't think a short-hand is needed.

On Mon, Mar 11, 2019 at 10:41 AM Tim Graham  wrote:

> -kdb could be a suitable short option.
>
> On Monday, March 11, 2019 at 9:20:37 AM UTC-4, Tobias McNulty wrote:
>>
>> Agreed it's probably better to make the switch now, and I'd be fine
>> without a replacement shorthand alternative for --keepdb.
>>
>> Cheers,
>>
>>
>> *Tobias McNulty*Chief Executive Officer
>>
>> tob...@caktusgroup.com
>> www.caktusgroup.com
>>
>>
>> On Mon, Mar 11, 2019 at 8:19 AM Carlton Gibson 
>> wrote:
>>
>>> Thanks François,
>>>
>>> Just on this, my thought is that if we don't follow `unittest` in
>>> changing `-k` for this, we have a steady trickle of confusion forever-more.
>>> I'd rather avoid that.
>>>
>>> C.
>>>
>>> On Monday, 11 March 2019 13:14:01 UTC+1, François Freitag wrote:

 Hi Django Devs,

 https://code.djangoproject.com/ticket/30245 suggests supporting Python
 unittest `-k` option, to selectively run tests matching a keyword.

 Currently, `-k` is the shorthand for `--keepdb` in Django.
 A `--filter` flag was suggested to preserve backward compatibility.
 Carlton suggested removing the `-k` option from `--keepdb` and reusing
 it for unittest `-k`. That would follow unittest more closely, reduce
 user confusion and ease maintenance.

 What do you think is best? Do you see other options?

 If re-taking the `-k` option for unittest `-k`, should a new shorthand
 be introduced for `--keepdb`?

 Thanks for your time,
 François

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/c54e52d2-012a-4852-9375-be37add55945%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/d0f1df05-238c-4ad6-97c2-a852bda939bc%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAFzonYYRhKbPbuUvZCb9hM3jM%3D90VPwv9Ug%2B0k%3DdXUuRyDMz_Q%40mail.gmail.com

.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADOBPEEJGbczwxmCax0ak0p3DqSJc9K_wApbbATo_q9Ywx%3D0ug%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Migration strategy for proxy model permissions to use their own content type

[Arthur Rio]

Thank you for your feedback Aymeric,

I have added the backward migration method + tests per your suggestion:
https://github.com/django/django/pull/10381/commits/d5c4a4b08ccee9239e5117df4c788a5a7a2f60a9

Regards


—
Arthur

On January 6, 2019 at 10:04:18 PM, Aymeric Augustin (
aymeric.augus...@polytechnique.org) wrote:

Hello Arthur,

I agree that option 1 is the way to go.

It would be nice to make the migration reversible. If someone upgrades to
Django 2.2.x, migrates the database, and later discovers a blocking issue,
they should have the option to migrate backwards and downgrade to 2.1.y.

Best regards,

-- 
Aymeric.



On 26 Nov 2018, at 16:10, Arthur Rio  wrote:

Hi all,

I have been working on a 9 years old ticket that I'd like to close once and
for all. The outstanding question is about the migration path to choose in
order to update existing proxy model permissions. I have explained three
different approaches I can think of in the pull request:
https://github.com/django/django/pull/10381#issuecomment-435534644.

I have implemented the first option and wrote the release notes explaining
what to expect for users upgrading:
https://github.com/django/django/pull/10381/files#diff-1f22a5c1d6164c6de8defb36f3829138
.

Carlton, who has been the main reviewer, suggested to contact the mailing
list for further review in order to catch any red flag and make sure we
have general consensus on the chosen approach.

Regards.

Arthur

--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/f3298dbb-e072-4815-944c-2250d0b4d3a3%40googlegroups.com
<https://groups.google.com/d/msgid/django-developers/f3298dbb-e072-4815-944c-2250d0b4d3a3%40googlegroups.com?utm_medium=email_source=footer>
.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/46CD821B-C17E-4FF8-B623-6B40B0FB0BBD%40polytechnique.org
<https://groups.google.com/d/msgid/django-developers/46CD821B-C17E-4FF8-B623-6B40B0FB0BBD%40polytechnique.org?utm_medium=email_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADOBPEFozUMDawj%3DEi2Ju5eFnTFVzKR%2BYWw5VA0edJ%3DF2neirw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: IRC Channel

[Arthur Rio]

Hi Avi,

There are #django (for usage) and #django-dev (for contributing), both on
irc.freenode.net, as mentioned here:
https://docs.djangoproject.com/en/dev/internals/contributing/

Regards

—
Arthur

On December 13, 2018 at 11:03:51 PM, Avi Garg (avi.grg4...@gmail.com) wrote:

Does Django have an IRC channel? I am unable to join it if so...
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/166e0cef-1f71-40c3-abbe-f0985d6ea7fb%40googlegroups.com

.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADOBPEHc%2BpGc%3D4E-x%2BK%3DX8DykTU6UT2sGV9VCGmdnf0%2Bcgzv3w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Migration strategy for proxy model permissions to use their own content type

[Arthur Rio]

Hi all,

I have been working on a 9 years old ticket that I'd like to close once and 
for all. The outstanding question is about the migration path to choose in 
order to update existing proxy model permissions. I have explained three 
different approaches I can think of in the pull 
request: https://github.com/django/django/pull/10381#issuecomment-435534644. 

I have implemented the first option and wrote the release notes explaining 
what to expect for users upgrading: 
https://github.com/django/django/pull/10381/files#diff-1f22a5c1d6164c6de8defb36f3829138.

Carlton, who has been the main reviewer, suggested to contact the mailing 
list for further review in order to catch any red flag and make sure we 
have general consensus on the chosen approach. 

Regards.

Arthur

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f3298dbb-e072-4815-944c-2250d0b4d3a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Content types shouldn't be created on post_migrate signal

[Arthur Rio]

Hey Marcin,

I hope you had a good week-end. After looking through the codebase to get 
more familiar with how `pre_migrate` and `post_migrate` are currently used, 
I thought we could simply have the same sort of signal for 
`post_makemigrations`,
where a third party could get a list of the changes and the plan and append 
operations to the generated file (probably adding itself as dependency 
too). However, it seems to be far from trivial given the related
threads I found related to the 
subject: 
https://groups.google.com/d/msg/django-developers/qRNkReCZiCk/I8dIxxhoBwAJ 
and https://groups.google.com/d/msg/django-developers/iClIpvwlbAU/4uX7q_7aAQAJ
I was surprised to see that the `RenameModel` was handled by `ContentType` 
using the `pre_migrate` hook rather than happening in the `makemigrations` 
phase, maybe someone can share some context?

I really think this is a real problem and something that could get improved 
but to do so it would require fixing the bigger issue.

To quote Simon Charette 
(https://groups.google.com/d/msg/django-developers/qRNkReCZiCk/Ah90crNFAAAJ):

"As for the makemigrations hooks I think it would require a lot of thought 
and
efforts to get right. Right now the auto-detector is a black box that deals 
with
dependencies and model state deltas resolution."

If now is the time to take a stab at it, I'd be happy to help as much as I 
can.

Regards

--
Arthur


On Friday, October 5, 2018 at 10:28:06 AM UTC-7, Arthur Rio wrote:
>
> For some reason the text is white… Here it is in black:
>
> Hey Marcin,
>
> The problem is that data migration based on app layer (python objects, ie. 
> Models and Managers here) will cause troubles after some time (when app is 
> changing).
> In the other words - you cannot rely on your app layer when doing database 
> changes. You should never do that, especially for projects requiring LTS.
>
> In theory I understand the idea, but in practice, migrations, the ORM and 
> the content type model are all part of Django and I don’t really see how 
> the app changing could cause troubles. Do you have an example in mind?
>
> Maybe "automatic" is not a proper word. They should be created 
> automatically, but should be applied "at the right time".
>
> Ok, that we agree on!
>
> CT opt-in should be connected to a signal related to the creation of 
> migration files (Autodetector?), not to a signal related to their 
> execution. I.e. pre/post_autodection signals should be introduced.
>
> I think we agree that the solution would be some sort of signal triggered 
> when a model creation/deletion is detected (in the `makemigrations` phase 
> as opposed to `migrate`) so that some code is added to the generated 
> migration. The use of a signal is important to keep things decoupled and 
> flexible.
>
> The “some code is added to the migration” part still needs to be 
> determined, either in the shape of insert/delete statements or a RunPython 
> leveraging the ORM. In both cases, the values to insert (Adding a model) or 
> to lookup for delete (Removing a model) are just 2 strings, the app label 
> and the model class name.
>
> After adding contrib.contenttypes, Django should check existence of 
> django_content_type table. If exists, Django should only check for data 
> changes and generate series of inserts/deletes. If not, Django should 
> generate inserts for all models. If CT is removed later, Django should 
> remove all CT data .
>
> It’s a good idea but I don’t know offhand how we can keep migrations and 
> content type decoupled to do that (especially the removal).
>
> Finally, I also think the concept could be extended to the permission 
> model which faces similar issues.
>
>
> Regards
>
> --
>
> Arthur
>
> On October 5, 2018 at 9:30:58 AM, Arthur Rio (arthur.ri...@gmail.com) 
> wrote:
>
> Hey Marcin,
>
> The problem is that data migration based on app layer (python objects, ie. 
> Models and Managers here) will cause troubles after some time (when app is 
> changing).
> In the other words - you cannot rely on your app layer when doing database 
> changes. You should never do that, especially for projects requiring LTS.
>
> In theory I understand the idea, but in practice, migrations, the ORM and 
> the content type model are all part of Django and I don’t really see how 
> the app changing could cause troubles. Do you have an example in mind?
>
> Maybe "automatic" is not a proper word. They should be created 
> automatically, but should be applied "at the right time".
>
> Ok, that we agree on!
>
> CT opt-in should be connected to a signal related to the creation of 
> migration files (Autodetector?), not to a signal related to their 
> execution. I.e. pre/post_autode

Re: Content types shouldn't be created on post_migrate signal

[Arthur Rio]

For some reason the text is white… Here it is in black:

Hey Marcin,

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You should never do that, especially for projects requiring LTS.

In theory I understand the idea, but in practice, migrations, the ORM and
the content type model are all part of Django and I don’t really see how
the app changing could cause troubles. Do you have an example in mind?

Maybe "automatic" is not a proper word. They should be created
automatically, but should be applied "at the right time".

Ok, that we agree on!

CT opt-in should be connected to a signal related to the creation of
migration files (Autodetector?), not to a signal related to their
execution. I.e. pre/post_autodection signals should be introduced.

I think we agree that the solution would be some sort of signal triggered
when a model creation/deletion is detected (in the `makemigrations` phase
as opposed to `migrate`) so that some code is added to the generated
migration. The use of a signal is important to keep things decoupled and
flexible.

The “some code is added to the migration” part still needs to be
determined, either in the shape of insert/delete statements or a RunPython
leveraging the ORM. In both cases, the values to insert (Adding a model) or
to lookup for delete (Removing a model) are just 2 strings, the app label
and the model class name.

After adding contrib.contenttypes, Django should check existence of
django_content_type table. If exists, Django should only check for data
changes and generate series of inserts/deletes. If not, Django should
generate inserts for all models. If CT is removed later, Django should
remove all CT data .

It’s a good idea but I don’t know offhand how we can keep migrations and
content type decoupled to do that (especially the removal).

Finally, I also think the concept could be extended to the permission model
which faces similar issues.


Regards

--

Arthur

On October 5, 2018 at 9:30:58 AM, Arthur Rio (arthur.ri...@gmail.com) wrote:

Hey Marcin,

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You should never do that, especially for projects requiring LTS.

In theory I understand the idea, but in practice, migrations, the ORM and
the content type model are all part of Django and I don’t really see how
the app changing could cause troubles. Do you have an example in mind?

Maybe "automatic" is not a proper word. They should be created
automatically, but should be applied "at the right time".

Ok, that we agree on!

CT opt-in should be connected to a signal related to the creation of
migration files (Autodetector?), not to a signal related to their
execution. I.e. pre/post_autodection signals should be introduced.

I think we agree that the solution would be some sort of signal triggered
when a model creation/deletion is detected (in the `makemigrations` phase
as opposed to `migrate`) so that some code is added to the generated
migration. The use of a signal is important to keep things decoupled and
flexible.

The “some code is added to the migration” part still needs to be
determined, either in the shape of insert/delete statements or a RunPython
leveraging the ORM. In both cases, the values to insert (Adding a model) or
to lookup for delete (Removing a model) are just 2 strings, the app label
and the model class name.

After adding contrib.contenttypes, Django should check existence of
django_content_type table. If exists, Django should only check for data
changes and generate series of inserts/deletes. If not, Django should
generate inserts for all models. If CT is removed later, Django should
remove all CT data .

It’s a good idea but I don’t know offhand how we can keep migrations and
content type decoupled to do that (especially the removal).

Finally, I also think the concept could be extended to the permission model
which faces similar issues.


Regards

--

Arthur



On October 4, 2018 at 4:47:19 PM, Marcin Nowak (marcin.j.no...@gmail.com)
wrote:


Hi Arthur.

BTW: RunPython() is another thing, which can break your migrations, and
should not be used (especially not by Django internally), because it relies
on the application layer.

How else can you do a data migration? There is no
> `migrations.InsertIntoTable`,
>

You're right. That's why "Insert" should be (in my opinion) introduced.
Together with "migrations.Delete".

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - 

Re: Content types shouldn't be created on post_migrate signal

[Arthur Rio]

Hey Marcin,

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You should never do that, especially for projects requiring LTS.

In theory I understand the idea, but in practice, migrations, the ORM and
the content type model are all part of Django and I don’t really see how
the app changing could cause troubles. Do you have an example in mind?

Maybe "automatic" is not a proper word. They should be created
automatically, but should be applied "at the right time".

Ok, that we agree on!

CT opt-in should be connected to a signal related to the creation of
migration files (Autodetector?), not to a signal related to their
execution. I.e. pre/post_autodection signals should be introduced.

I think we agree that the solution would be some sort of signal triggered
when a model creation/deletion is detected (in the `makemigrations` phase
as opposed to `migrate`) so that some code is added to the generated
migration. The use of a signal is important to keep things decoupled and
flexible.

The “some code is added to the migration” part still needs to be
determined, either in the shape of insert/delete statements or a RunPython
leveraging the ORM. In both cases, the values to insert (Adding a model) or
to lookup for delete (Removing a model) are just 2 strings, the app label
and the model class name.

After adding contrib.contenttypes, Django should check existence of
django_content_type table. If exists, Django should only check for data
changes and generate series of inserts/deletes. If not, Django should
generate inserts for all models. If CT is removed later, Django should
remove all CT data .

It’s a good idea but I don’t know offhand how we can keep migrations and
content type decoupled to do that (especially the removal).

Finally, I also think the concept could be extended to the permission model
which faces similar issues.


Regards

--

Arthur



On October 4, 2018 at 4:47:19 PM, Marcin Nowak (marcin.j.no...@gmail.com)
wrote:


Hi Arthur.

BTW: RunPython() is another thing, which can break your migrations, and
should not be used (especially not by Django internally), because it relies
on the application layer.

How else can you do a data migration? There is no
> `migrations.InsertIntoTable`,
>

You're right. That's why "Insert" should be (in my opinion) introduced.
Together with "migrations.Delete".

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You should never do that, especially for projects requiring LTS.

RunPython() should be used only when you cannot do anything else. In such
case you must accept all consequences. I'm not against RunPython(), but
against doing data migrations using it.

The problem with hypothetical "Insert" operation is with mapping field
types. Insert cannot use Models directly (app layer is changing over a
time), but should "know" how to map arguments (python types, values) to a
database literals. It can be achieved by introducing field mapping for a
every insert or per migration file (something like "model freeze", but only
for used fields).

Also Insert should not accept variables calculated in the app layer (within
a migration) - it should contain only plain/direct data. But using Python
as a language for migrations, will be hard to avoid such possibility. This
is important, because database management should not rely on app layer.
Using variables (i.e. from settings) would result in inconsistent data
changes between different environments.

the only other way currently would be to run a `migrations.RunSql` query
> which may look different based on the database used.
>

RunSQL is not the solution for db agnostic operations. It may be only used
within a project, because db engine used changes rarely (due to the nature
and importance of the data and relational databases, and systems dependent
on the db).
But RunSQL is a handful operation, because SQL is a natural language for db
management. I'm doing many raw sqls in migrations.


> Two things are wrong:
>
>- automatic creation of content types
>
> Why is it wrong to automatically create a content type?
>

Maybe "automatic" is not a proper word. They should be created
automatically, but should be applied "at the right time".


> Content type is an opt-in feature since you can remove it from
> `INSTALLED_APPS`.
>

I know, but it is required by contrib.auth. I saw no project depending on
something else, so CT is optional.. but theoretically ;)


>
>- creation of content types after running migrations
>
> That’s the only real problem for me. Maybe using a signal for
> `migrations.CreateModel` (e.g. post_create_model), instead of using

Re: Content types shouldn't be created on post_migrate signal

[Arthur Rio]

BTW: RunPython() is another thing, which can break your migrations, and
should not be used (especially not by Django internally), because it relies
on the application layer.

How else can you do a data migration? There is no
`migrations.InsertIntoTable`, the only other way currently would be to run
a `migrations.RunSql` query which may look different based on the database
used.

Two things are wrong:

   - automatic creation of content types

Why is it wrong to automatically create a content type? Content type is an
opt-in feature since you can remove it from `INSTALLED_APPS`.


   - creation of content types after running migrations

That’s the only real problem for me. Maybe using a signal for
`migrations.CreateModel` (e.g. post_create_model), instead of using
`post_migrate` would fix it, but there may be other scenarios I’m not
thinking about.


Solution:

   - creation of new app should add very first migration, which will add
   entry to the content types

How would you handle creating a model later on? Or if
`django.contrib.contenttypes` is only added later on to `INSTALLED_APPS`?


Regards,

—

Arthur


On October 4, 2018 at 1:36:39 PM, Marcin Nowak (marcin.j.no...@gmail.com)
wrote:

Hi Aymeric.

Thank you for your reply.

Unfortunately you wrote mostly about me or my writing style, not about the
issue.
I disagree with your opinion about my comments being passive or aggressive.
I'm always writing about a piece of code, functionality,
design/architecture or bug. I never criticised a person directly.



> Starting with "There is a huge issue with content types framework" isn't a
> good way to motivate them.
>
>
But there is an issue with content types framework (not with it's authors).



> Speaking for myself, I would be more eager to investigate if you skipped
> the hyperbole and remained neutral, for example "I'm facing an issue with
> the content types framework".
>
>
Sorry hearing that. I'm not native English speaker.

For me there is almost no difference with these two sentences, except that
first is about the affected thing ("there is an issue with") and second is
more about me ("I have a problem").
Both are valid, I think. I have a problem which is caused by CT framework's
design or issue (in fact it comes from a historical reason, before
migrations era).


> You'd have more success if you managed to write in a positive style.
>

I don't think that my style is unpleasant. If it is - sorry for that.
I'm always trying to describe the problem and give some proposals.
But I'll try to improve this.


> I think the issue itself is valid. I may have hit it before and worked
> around it, likely be executing a subset of migrations to trigger creation
> of content types, then executing the rest of the migrations. Django could
> probably do better.
>
>
I'll generate CTs in very first migration. This will be a workaround, of
course.
But because Django uses migrations internally, CT's should be added to the
database in a same way.

And RunPython() can be problematic. I did something similar with Liquibase'
executeCommand, which was calling management command. And of course it
caused problems after changing app layer.
I'm very conservative about databases and managing them, I think that there
must be complete separation between db and app layer (in the context of
managing dbs), because app layer is changing frequently (more often than
dbs). Mixing both worlds will cause troubles. Always.

Kind Regards,
Marcin
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/0e4464f7-2ed7-4e6b-9b7e-f98a385dffd5%40googlegroups.com

.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADOBPEGDbxpNnECnZMk-Ff51YX6T3drfm9MTBRS3gxmcEgQ1tg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.