Re: Question (potential feature request) on dropping database column in migration

[Todor Velichkov]

>
> Won’t generate migration file, which means database wise, nothing has 
> changed
>
You are the one generating migration files, Django is not forcing them.

The problem is not in Django, the problem is that this cannot be solved 
with a single deploy, there is not way for the old code to know whats 
coming in the future, which means you need to make a deploy just for this, 
and after that make a second deploy with the real changes. But if you are 
gonna make two deploys then you don't need Django to help you, you can 
solve this alone. Here is an example: Lets say we want to migrate from a 
full_name column to first_name and last_name columns. Here is what you can 
do:

1) Add first_name and last_name into the model and create a migration 
(which adds two columns and eventually populates them with data from 
full_name).
2) Change the code to make use of the new fields, remove full_name from the 
code, BUT do not make migration yet!
3) Deploy.
Now, during the deployment your old code will still use full_name which is 
still in the database, and when the new code gets live it will start using 
the new columns. 
Note: during the deployment you need to run migrations first, and reload 
the server after that, so your new code won't fail because new columns 
don't exist yet.
Note2: since you are running the migrations first, and restarting the 
server later, in theory its possible for a new data to appear into the 
database from the old code! Thus you wont have it migrated yet. e.g. A new 
user registered right after the migration ended and before the new code 
gets live, his name will be stored into the old full_name column! But you 
can fix this on the next step!
4) Now after the new code is live, and no one is using full_name anymore 
create a new migrations which will remove it from the database. In this 
migration you can double check that all data from full_name is properly 
migrated to the new columns.
5) Deploy again. (no code changes, just removing the old unused column from 
the database).



On Tuesday, October 2, 2018 at 3:47:10 AM UTC+3, martin_x wrote:
>
> Hi there,
>
> Thanks for making Django a great framework for starters :D
>
> Recently my team have run into problems when trying to remove a database 
> column from the model without stopping our Django server, below is the 
> timeline:
>
> 1. Before migration happens, we have column_A we want to remove, and 
> running on old release_1
> 2. Now we run migration to remove column_A, our webserver is still running 
> on old release_1 and serving requests
> 3. After migration, we ship the new release_2
>
> However, between step 2 and 3, there are requests coming and referencing 
> column_A we want to remove, and throws a exception of course.
>
> So is there anyway I can mark a database column a special state 
> (deprecated, purged in memory, etc), so it has the following affect:
> 1. Won’t generate migration file, which means database wise, nothing has 
> changed
> 2. However, when Django loads the model into memory, it will ignore 
> column_A completely. So query, create, update, etc won’t try to access 
> column_A.
>
> The reason we want to do it that way is so we can achieve no downtime, 
> seamless migration for our application. I believe more people will benefit 
> from this.
>
> Please let me know if more information is needed. Looking forward to 
> hearing from you.
>
> Martin
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a8bd0853-eb45-437d-ab87-f9586518e86f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Question (potential feature request) on dropping database column in migration

[Curtis Maloney]

On 10/02/2018 09:42 AM, martin_x wrote:

Hi there,

Thanks for making Django a great framework for starters :D

Recently my team have run into problems when trying to remove a database 
column from the model without stopping our Django server, below is the 
timeline:


There have been processes developed for addressing this, but they're 
quite involved.


You need to ensure any field the current running codebase knows about 
never disappears. So, the process of removing a field must be spread 
across multiple code deployments


The first setting it nullable [or with a "harmless" default] so when the 
new code inserts new rows the DBMS doesn't complain.


The next deploy can then drop the field, as Django no longer interacts 
with it.


Either way, you're still likely to encounter a full-table exclusive lock 
whilst the DBMS rewrites the table to no longer include that field.


Hope this helps some.

--
Curtis

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e8a32669-b566-17ee-132e-49e9dff33256%40tinbrain.net.
For more options, visit https://groups.google.com/d/optout.

Re: Question (potential feature request) on dropping database column in migration

[Tobias McNulty]

Hello,

I don't think this is something Django can or should handle. You need to
make your change in smaller increments so it doesn't break.

The proper forum for discussing this is the Django users list (or IRC or
Google...).

Good luck!

Tobias


On Mon, Oct 1, 2018, 8:47 PM martin_x  wrote:

> Hi there,
>
> Thanks for making Django a great framework for starters :D
>
> Recently my team have run into problems when trying to remove a database
> column from the model without stopping our Django server, below is the
> timeline:
>
> 1. Before migration happens, we have column_A we want to remove, and
> running on old release_1
> 2. Now we run migration to remove column_A, our webserver is still running
> on old release_1 and serving requests
> 3. After migration, we ship the new release_2
>
> However, between step 2 and 3, there are requests coming and referencing
> column_A we want to remove, and throws a exception of course.
>
> So is there anyway I can mark a database column a special state
> (deprecated, purged in memory, etc), so it has the following affect:
> 1. Won’t generate migration file, which means database wise, nothing has
> changed
> 2. However, when Django loads the model into memory, it will ignore
> column_A completely. So query, create, update, etc won’t try to access
> column_A.
>
> The reason we want to do it that way is so we can achieve no downtime,
> seamless migration for our application. I believe more people will benefit
> from this.
>
> Please let me know if more information is needed. Looking forward to
> hearing from you.
>
> Martin
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/50dbe737-0b6c-42d2-9f76-99c188c916e7%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMGFDKSXwhZ0Rhs1qH6FttP7rcgHoRoS5dLoHT7JFxn7iMdCzQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Question (potential feature request) on dropping database column in migration

[martin_x]

Hi there,

Thanks for making Django a great framework for starters :D

Recently my team have run into problems when trying to remove a database 
column from the model without stopping our Django server, below is the 
timeline:

1. Before migration happens, we have column_A we want to remove, and 
running on old release_1
2. Now we run migration to remove column_A, our webserver is still running 
on old release_1 and serving requests
3. After migration, we ship the new release_2

However, between step 2 and 3, there are requests coming and referencing 
column_A we want to remove, and throws a exception of course.

So is there anyway I can mark a database column a special state 
(deprecated, purged in memory, etc), so it has the following affect:
1. Won’t generate migration file, which means database wise, nothing has 
changed
2. However, when Django loads the model into memory, it will ignore 
column_A completely. So query, create, update, etc won’t try to access 
column_A.

The reason we want to do it that way is so we can achieve no downtime, 
seamless migration for our application. I believe more people will benefit 
from this.

Please let me know if more information is needed. Looking forward to 
hearing from you.

Martin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/50dbe737-0b6c-42d2-9f76-99c188c916e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fellow Reports - September 2018

[Tim Graham]

Week ending September 29, 2018

Triaged

---

https://code.djangoproject.com/ticket/29801 - Allow using fixtures in data 
migrations (accepted)

https://code.djangoproject.com/ticket/29804 - Add 'Did you mean' 
suggestions for unsupported lookup error (accepted)

Authored

--

https://github.com/django/django/pull/10453 - Tested showmigrations with 
apps without migrations.

https://github.com/django/django/commit/a7284cc0c3620030b43034cdf41216c0941bf411
 
- Fixed #29809 -- Fixed a crash when a "view only" user POSTs to the admin 
user change form.

Reviewed/committed

--

https://github.com/django/django/pull/10421 - Fixed #29630 -- Fixed crash 
of sliced queries with multiple columns with the same name on Oracle 12.1.

https://github.com/django/django/pull/10449 - Fixed #29683 -- Added view 
permission to docs.

https://github.com/django/django/pull/10416 - Fixed #29673 -- Reset the 
URLconf at the end of each request.

https://github.com/django/django/pull/10404 - Fixed #29768 -- Improved 
error message when an AppConfig has a typo in INSTALLED_APPS.

https://github.com/django/django/pull/10441 - Fixed #29796 -- Added system 
check for STATICFILES_DIRS prefix ending with a slash.

https://github.com/django/django/pull/10402 - Fixed #29767 -- Made 
date-related casts work on SQLite.
https://github.com/django/django/commit/bf39978a53f117ca02e9a0c78b76664a41a54745
 
- Fixed CVE-2018-16984 -- Fixed password hash disclosure to admin "view 
only" users.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/31ec91a4-7f19-4dfd-8bad-eb9e11e85c7f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fellow Reports -- September 2018

[Carlton Gibson]

Hi all,


Calendar Week 37 -- ending 16 September.


Triaged:

https://code.djangoproject.com/ticket/29758 -- Document testing custom 
error handlers (Accepted)
https://code.djangoproject.com/ticket/29750 -- Add a pre-dispatch() hook 
for class-based views (Accepted)


Reviewed:

https://code.djangoproject.com/ticket/11154 -- Inconsistency with 
permissions for proxy models
https://code.djangoproject.com/ticket/29711 -- Add system check to ensure 
uniqueness of admin actions' __name__.
https://code.djangoproject.com/ticket/29755 -- Infinite migrations created 
after removing Meta.default_related_name
https://code.djangoproject.com/ticket/14357 -- Prevent inappropriate 
order-based grouping on values+annotate queries
https://code.djangoproject.com/ticket/29538 -- Query Expression in ordering 
of a related object fails
https://code.djangoproject.com/ticket/29724 -- Admin date_hierarchy filter 
by month displays an extra day
https://code.djangoproject.com/ticket/29746 -- FlatpageForm help_text is 
misleading when APPEND_SLASH is False
https://code.djangoproject.com/ticket/29573 -- Add link from per-item 
aggregation topic discussion to `annotate()` ref.
https://code.djangoproject.com/ticket/16995 -- ModelFormSet initial data 
from initial parameter uses "extra" forms


Authored:

https://github.com/django/django/commit/bf39978a53f117ca02e9a0c78b76664a41a54745
-- Fixed CVE-2018-16984 -- Fixed password hash disclosure to admin 
"view only" users.




Calendar Week 38 -- ending 23 September.


Triaged:

https://code.djangoproject.com/ticket/29776 -- Pulling part of 
django.conf.settings from the database (wontfix)
https://code.djangoproject.com/ticket/29775 -- custom url converters are 
not picked up on reverse when part of included patterns with namespace 
(Accepted
)
https://code.djangoproject.com/ticket/29768 -- Confusing error when 
AppConfig subclass's name is misspelled (Accepted)


Reviewed:

https://code.djangoproject.com/ticket/29778 -- Unique index cannot be 
created for db table, whose name should be quoted
https://code.djangoproject.com/ticket/29765 -- Mention 
MIDDLEWARE/INSTALLED_APPS order is important in settings.py
https://code.djangoproject.com/ticket/29767 -- Django ORM: Cast between 
datetime and date.
https://code.djangoproject.com/ticket/11154 -- inconsistency with 
permissions for proxy models
https://code.djangoproject.com/ticket/20147 -- Provide an alternative to 
request.META for accessing HTTP headers
https://code.djangoproject.com/ticket/29761 -- GIS test failures



Calendar Week 39 -- ending 30 September.


Triaged:

https://code.djangoproject.com/ticket/29797 -- Allow the makemessages 
command to accept a directory to operate on like xgettext does (wontfix)
https://code.djangoproject.com/ticket/29798 -- truncatewords_html fails on 
some self-closing tags (Invalid)
https://code.djangoproject.com/ticket/29799 -- Allow registration and 
unregistration of lookups per field instances. (Accepted)
https://code.djangoproject.com/ticket/29782 -- Better error message when 
filtering users with AnonymousUser (Accepted)
https://code.djangoproject.com/ticket/29785 -- Using {{ csrf_token }} 
throws exception (AttributeError: 'dict' object has no attribute 'META') if 
no render context (Invalid)
https://code.djangoproject.com/ticket/29784 -- Review and update the 
external links in the settings reference documentation (Accepted)


Reviewed:

https://github.com/django/django/pull/10458 -- Updated naturaltime 
translation test.
https://github.com/django/django/pull/10448 -- Avoid starting a transaction 
when only a single (or atomic queries) are executed.
https://github.com/django/django/pull/10446 -- Refs #28909 -- Simplifed 
code using unpacking generalizations.
https://code.djangoproject.com/ticket/29758 -- Document testing custom 
error handlers
https://github.com/django/django/pull/10381 -- Fixed #11154 and #17904 -- 
Inconsistency with permissions for proxy models
https://github.com/django/django/pull/10430 -- Refs #29784 -- Switched to 
https:// links where available.



Kind Regards,

Carlton

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5b571a79-0bdd-48ce-a364-29c177f97f62%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Django security release issued: 2.1.2

[Carlton Gibson]

Today the Django team issued 2.1.2 as part of our security 
process. This release address a security issue, and we encourage all 
users to upgrade as soon as possible: 

https://www.djangoproject.com/weblog/2018/oct/01/security-release/ 


As a reminder, we ask that potential security issues be reported via 
private email to secur...@djangoproject.com  
and not via Django's Trac 
instance or the django-developers list. Please see 
https://www.djangoproject.com/security  
for further information. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/361FBF25-F849-47A9-AA24-3717B465F75F%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Django bugfix releases 1.11.16 and 2.0.9

[Carlton Gibson]

Details are available on the Django project weblog: 
https://www.djangoproject.com/weblog/2018/oct/01/bugfix-releases/ 


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/F0053153-780F-4D11-953F-FA5EFFC3D486%40gmail.com.
For more options, visit https://groups.google.com/d/optout.