Re: Can we remove FILE_CHARSET?

[Jon Dufresne]

> Is that always available these days? (I'd guess yes.)

I too would guess yes. I believe any reasonably modern text editor will
support
UTF-8 and even likely default to saving in that encoding. I know mine does.

> Is is something we want to impose? Not sure. Are there people doing
> otherwise? (No idea.)

For templates, it wouldn't be imposed. Users can still override the template
engine's encoding with the 'file_charset' option.

For static files, without imposing it, we're back to the third-party app
concern. Just like DEFAULT_CHARSET, it would difficult to change
FILE_CHARSET
_and_ integrate third party apps. The third party apps have likely encoded
their static files using UTF-8, so setting FILE_CHARSET to some other value
will break.

Cheers,
Jon


On Wed, Oct 3, 2018 at 12:14 PM Carlton Gibson 
wrote:

> Thanks for the follow-up Jon.
>
> I'll let Vasili follow-up on his use-case if possible/relevant.
>
> TBH I'm not at all sure about the SQL data files bit, which is in part why
> I asked here.
> (Encoding issues!)
>
> > Maybe that sentence should be rephrased to "template
> files, static files, and translation catalogs".
>
> OK, so IF it's just this, then I'm on Windows doing development in UTF-8
> no problem (and can't really envisage doing much different as it stands)
> but:
>
> * Is that always available these days? (I'd guess yes.)
> * Is is something we want to impose? Not sure. Are there people doing
> otherwise? (No idea.)
>
> (If we can drop a setting, that'd be )
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/13cc5f04-967b-4f53-92f6-cbe155014edc%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADhq2b4-7vc-hZgKP_%2BYOmUkbRC%2B8r7i2t74d0-9b0vGrVcmkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Content types shouldn't be created on post_migrate signal

[Arthur Rio]

For some reason the text is white… Here it is in black:

Hey Marcin,

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You should never do that, especially for projects requiring LTS.

In theory I understand the idea, but in practice, migrations, the ORM and
the content type model are all part of Django and I don’t really see how
the app changing could cause troubles. Do you have an example in mind?

Maybe "automatic" is not a proper word. They should be created
automatically, but should be applied "at the right time".

Ok, that we agree on!

CT opt-in should be connected to a signal related to the creation of
migration files (Autodetector?), not to a signal related to their
execution. I.e. pre/post_autodection signals should be introduced.

I think we agree that the solution would be some sort of signal triggered
when a model creation/deletion is detected (in the `makemigrations` phase
as opposed to `migrate`) so that some code is added to the generated
migration. The use of a signal is important to keep things decoupled and
flexible.

The “some code is added to the migration” part still needs to be
determined, either in the shape of insert/delete statements or a RunPython
leveraging the ORM. In both cases, the values to insert (Adding a model) or
to lookup for delete (Removing a model) are just 2 strings, the app label
and the model class name.

After adding contrib.contenttypes, Django should check existence of
django_content_type table. If exists, Django should only check for data
changes and generate series of inserts/deletes. If not, Django should
generate inserts for all models. If CT is removed later, Django should
remove all CT data .

It’s a good idea but I don’t know offhand how we can keep migrations and
content type decoupled to do that (especially the removal).

Finally, I also think the concept could be extended to the permission model
which faces similar issues.


Regards

--

Arthur

On October 5, 2018 at 9:30:58 AM, Arthur Rio (arthur.ri...@gmail.com) wrote:

Hey Marcin,

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You should never do that, especially for projects requiring LTS.

In theory I understand the idea, but in practice, migrations, the ORM and
the content type model are all part of Django and I don’t really see how
the app changing could cause troubles. Do you have an example in mind?

Maybe "automatic" is not a proper word. They should be created
automatically, but should be applied "at the right time".

Ok, that we agree on!

CT opt-in should be connected to a signal related to the creation of
migration files (Autodetector?), not to a signal related to their
execution. I.e. pre/post_autodection signals should be introduced.

I think we agree that the solution would be some sort of signal triggered
when a model creation/deletion is detected (in the `makemigrations` phase
as opposed to `migrate`) so that some code is added to the generated
migration. The use of a signal is important to keep things decoupled and
flexible.

The “some code is added to the migration” part still needs to be
determined, either in the shape of insert/delete statements or a RunPython
leveraging the ORM. In both cases, the values to insert (Adding a model) or
to lookup for delete (Removing a model) are just 2 strings, the app label
and the model class name.

After adding contrib.contenttypes, Django should check existence of
django_content_type table. If exists, Django should only check for data
changes and generate series of inserts/deletes. If not, Django should
generate inserts for all models. If CT is removed later, Django should
remove all CT data .

It’s a good idea but I don’t know offhand how we can keep migrations and
content type decoupled to do that (especially the removal).

Finally, I also think the concept could be extended to the permission model
which faces similar issues.


Regards

--

Arthur



On October 4, 2018 at 4:47:19 PM, Marcin Nowak (marcin.j.no...@gmail.com)
wrote:


Hi Arthur.

BTW: RunPython() is another thing, which can break your migrations, and
should not be used (especially not by Django internally), because it relies
on the application layer.

How else can you do a data migration? There is no
> `migrations.InsertIntoTable`,
>

You're right. That's why "Insert" should be (in my opinion) introduced.
Together with "migrations.Delete".

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You 

Re: Content types shouldn't be created on post_migrate signal

[Arthur Rio]

Hey Marcin,

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You should never do that, especially for projects requiring LTS.

In theory I understand the idea, but in practice, migrations, the ORM and
the content type model are all part of Django and I don’t really see how
the app changing could cause troubles. Do you have an example in mind?

Maybe "automatic" is not a proper word. They should be created
automatically, but should be applied "at the right time".

Ok, that we agree on!

CT opt-in should be connected to a signal related to the creation of
migration files (Autodetector?), not to a signal related to their
execution. I.e. pre/post_autodection signals should be introduced.

I think we agree that the solution would be some sort of signal triggered
when a model creation/deletion is detected (in the `makemigrations` phase
as opposed to `migrate`) so that some code is added to the generated
migration. The use of a signal is important to keep things decoupled and
flexible.

The “some code is added to the migration” part still needs to be
determined, either in the shape of insert/delete statements or a RunPython
leveraging the ORM. In both cases, the values to insert (Adding a model) or
to lookup for delete (Removing a model) are just 2 strings, the app label
and the model class name.

After adding contrib.contenttypes, Django should check existence of
django_content_type table. If exists, Django should only check for data
changes and generate series of inserts/deletes. If not, Django should
generate inserts for all models. If CT is removed later, Django should
remove all CT data .

It’s a good idea but I don’t know offhand how we can keep migrations and
content type decoupled to do that (especially the removal).

Finally, I also think the concept could be extended to the permission model
which faces similar issues.


Regards

--

Arthur



On October 4, 2018 at 4:47:19 PM, Marcin Nowak (marcin.j.no...@gmail.com)
wrote:


Hi Arthur.

BTW: RunPython() is another thing, which can break your migrations, and
should not be used (especially not by Django internally), because it relies
on the application layer.

How else can you do a data migration? There is no
> `migrations.InsertIntoTable`,
>

You're right. That's why "Insert" should be (in my opinion) introduced.
Together with "migrations.Delete".

The problem is that data migration based on app layer (python objects, ie.
Models and Managers here) will cause troubles after some time (when app is
changing).
In the other words - you cannot rely on your app layer when doing database
changes. You should never do that, especially for projects requiring LTS.

RunPython() should be used only when you cannot do anything else. In such
case you must accept all consequences. I'm not against RunPython(), but
against doing data migrations using it.

The problem with hypothetical "Insert" operation is with mapping field
types. Insert cannot use Models directly (app layer is changing over a
time), but should "know" how to map arguments (python types, values) to a
database literals. It can be achieved by introducing field mapping for a
every insert or per migration file (something like "model freeze", but only
for used fields).

Also Insert should not accept variables calculated in the app layer (within
a migration) - it should contain only plain/direct data. But using Python
as a language for migrations, will be hard to avoid such possibility. This
is important, because database management should not rely on app layer.
Using variables (i.e. from settings) would result in inconsistent data
changes between different environments.

the only other way currently would be to run a `migrations.RunSql` query
> which may look different based on the database used.
>

RunSQL is not the solution for db agnostic operations. It may be only used
within a project, because db engine used changes rarely (due to the nature
and importance of the data and relational databases, and systems dependent
on the db).
But RunSQL is a handful operation, because SQL is a natural language for db
management. I'm doing many raw sqls in migrations.


> Two things are wrong:
>
>- automatic creation of content types
>
> Why is it wrong to automatically create a content type?
>

Maybe "automatic" is not a proper word. They should be created
automatically, but should be applied "at the right time".


> Content type is an opt-in feature since you can remove it from
> `INSTALLED_APPS`.
>

I know, but it is required by contrib.auth. I saw no project depending on
something else, so CT is optional.. but theoretically ;)


>
>- creation of content types after running migrations
>
> That’s the only real problem for me. Maybe using a signal for
> `migrations.CreateModel` (e.g. post_create_model), instead of using

Re: Question (potential feature request) on dropping database column in migration

[vishvajit pathak]

Well this is what we call as the downtime.


On Tuesday, 2 October 2018 06:17:10 UTC+5:30, martin_x wrote:
>
> Hi there,
>
> Thanks for making Django a great framework for starters :D
>
> Recently my team have run into problems when trying to remove a database 
> column from the model without stopping our Django server, below is the 
> timeline:
>
> 1. Before migration happens, we have column_A we want to remove, and 
> running on old release_1
> 2. Now we run migration to remove column_A, our webserver is still running 
> on old release_1 and serving requests
> 3. After migration, we ship the new release_2
>
> However, between step 2 and 3, there are requests coming and referencing 
> column_A we want to remove, and throws a exception of course.
>
> So is there anyway I can mark a database column a special state 
> (deprecated, purged in memory, etc), so it has the following affect:
> 1. Won’t generate migration file, which means database wise, nothing has 
> changed
> 2. However, when Django loads the model into memory, it will ignore 
> column_A completely. So query, create, update, etc won’t try to access 
> column_A.
>
> The reason we want to do it that way is so we can achieve no downtime, 
> seamless migration for our application. I believe more people will benefit 
> from this.
>
> Please let me know if more information is needed. Looking forward to 
> hearing from you.
>
> Martin
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55d789ea-3f7c-4e4b-a682-3ff1745024c7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: django makemessages management command

[vishvajit pathak]

You could include the a python file where you can place all the strings 
which you manually wanted to add and then use the *makemessages *management 
command. I think it will serve the purpose. 

I think this behaviour of commenting out manually added strings by 
*makemessages *management command is correct.


On Monday, 24 September 2018 17:19:08 UTC+5:30, Osama AbuOmar wrote:
>
> About the django *makemessages *management command. I think there should 
> by an option to *NOT *let django comment out manually added translations 
> to the .po files.
> sometimes I need to manually add a translation definition (msgid, msgstr), 
> these are not in any of my template or python files. and when I run 
> makemessages they get commented out. and this is an issue.
> thanks 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/04357d3a-6033-4396-9d2c-791cca9cf742%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: local timezone-aware datetime fields in the admin site

[Aymeric Augustin]

Hello,

> Do you know what OS/browser combination you were referring to with "Windows 
> (which, I'm afraid, has its own time zone name)"?

At the system level, I'm not sure Windows is using (or was using) the timezone 
database (formerly known as the Olson database). I remember this being an issue 
but I have forgotten the details. (Most of my work in this area of Django was 7 
years ago.)

Your tests show that browsers normalize to the same timezone names, including 
Edge on Windows, which is good news. Furthermore the spec 

 says: [[timeZone]] is a String value with the IANA time zone name of the time 
zone used for formatting.

We can rely on this API. There's not need to test with another locale. Sorry 
for the noise!

Best regards,

-- 
Aymeric.



> On 5 Oct 2018, at 04:42, Paul Tiplady  wrote:
> 
> Thanks Aymeric,
> 
> A bit of digging gives:
> 
> MacOS Chrome: "America/Los_Angeles"
> MacOS Firefox: "America/Los_Angeles"
> MacOS Safari: "America/Los_Angeles"
> Windows Edge: ""America/Los_Angeles"
> 
> Do you know what OS/browser combination you were referring to with "Windows 
> (which, I'm afraid, has its own time zone name)"?
> 
> I haven't got around to testing locales, I'll report back once I do that.
> 
> On Wednesday, October 3, 2018 at 11:10:26 PM UTC-7, Aymeric Augustin wrote:
> Hello Paul,
> 
> Yes, if that works, it would be great.
> 
> The API you're pointing to returns a timezone name. This is better than an 
> offset because it avoids being off by one hour when editing dates or times on 
> the other side of the DST change.
> 
> You need to check what it does on Windows (which, I'm afraid, has its own 
> time zone name) and whether it's locale-dependent (that would be a mess).
> 
> If this API reliably returns a timezone name that pytz understands, then you 
> should be able to achieve your goal in the Widget for datetime fields and to 
> preserve backwards-compatibility.
> 
> Best regards,
> 
> -- 
> Aymeric.
> 
> 
> 
>> On 3 Oct 2018, at 23:04, Paul Tiplady qwil.com > 
>> wrote:
>> 
>> Timezone handling in the Django admin is quite confusing for users when the 
>> users are in multiple timezones, because everything in the admin site 
>> operates in the server's time.
>> 
>> Assuming USE_TZ=True, TIME_ZONE='UTC', USE_I18N, USE_L10N, when viewing a 
>> datetime field, users see the UTC time, with a note below saying "Note: You 
>> are 7 hours behind server time". This is better than nothing, but with a 
>> modern browser I believe it's possible to improve the situation. 
>> 
>> The ideal behaviour (I believe) would be localizing the timezones in the 
>> browser to the user's own time zone, and submitting back to the app using 
>> TZ-aware timestamp strings. It's possible to pull the TZ unambiguously from 
>> a modern browser: 
>> ​https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DateTimeFormat/resolvedOptions#Browser_compatibility#Description
>>  
>> .
>> 
>> ```
>> Intl.DateTimeFormat().resolvedOptions().timeZone
>> ```
>> 
>> Would this approach be acceptable? It seems that we'd want to default to the 
>> normal behaviour, and perhaps have this as config on the AdminSite object 
>> (since we want a way to enable this globally for sites that wish to use this 
>> feature).
>> 
>> (Originally raised here: https://code.djangoproject.com/ticket/29822 
>> , posting for discussion)
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@ <>googlegroups.com .
>> To post to this group, send email to django-d...@ <>googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/79896a0e-1338-4267-89bd-9904fca4f0d2%40googlegroups.com
>>  
>> .
>> For more options, visit https://groups.google.com/d/optout 
>> .
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .