Re: models.E026: The model cannot have more than one field with primary_key=True.

[Marcin Nowak]

Good point about concat. Thanks.
I think that introducing views in Django solve such issues (probably no pk 
in views will be allowed).

PS. I'll discuss this (and the 2nd topic) later, because I'm quite busy 
today. Thanks again for the hint.


W dniu środa, 1 kwietnia 2020 13:04:26 UTC+2 użytkownik Adam Johnson 
napisał:
>
> Hi Marcin
>
> Multiple column primary keys is a long-open feature request. There's a 
> wiki page: https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys 
> and the ticket, still open, is #373: 
> https://code.djangoproject.com/ticket/373 .
>
> I'm not sure your suggestion of removing the ".pk" mapping will either be 
> easy ("pk" is used a LOT throughout the ORM, not just for foreign keys), or 
> really help you very much (how would insertions really work, etc.)
>
> I've thought of a technique before for making composite PK's work, though 
> haven't tested it. It should be possible to put a database view in front of 
> your table that combines your primary key columns into one string. For 
> example:
>
> CREATE VIEW book_view AS SELECT CONCAT(catalogue_id, '-', book_id) AS 
> primary_key, author_id, ... FROM book;
>
> Then define your model as having a CharField as a primary key. PostgreSQL 
> and MariaDB/MySQL will both automatically make single-table views map 
> inserts and updates to the underlying table. Though you might need a 
> trigger to allow inserts (new object saves) to work, splitting the parts 
> back out.
>
> If you try this, let me know.
>
> Thanks,
>
> Adam
>
> On Wed, 1 Apr 2020 at 11:34, Marcin Nowak  > wrote:
>
>> Hi.
>>
>> I'd ask about  "models.E026: The model cannot have more than one field 
>> with primary_key=True".
>> It's not a true. The table or view *can* have more than one *column* as 
>> a part of a primary key.
>> It's pretty common, when you have views with cross joins or unpivots.
>>
>> I can add models.E026 to silenced system checks, but Django is mapping PK 
>> to the first field of the complex primary keys. This can introduce some 
>> issues.
>>
>> I know that supporting complex PKs is hard and generally unsupported, but 
>> how about removing auto-mapping ".pk" attribute for models with complex 
>> primary key
>> As a result there will be no possibility to declare foreign keys to such 
>> models, but there will be possibility to change E026to W0nn (just a 
>> warning).
>>
>> Regards,
>> Marcin
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-d...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/7b948901-e643-4bd0-917a-54ff7a18e9f9%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/7b948901-e643-4bd0-917a-54ff7a18e9f9%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/96d1bd22-1b82-4767-adbd-4be4aff1033d%40googlegroups.com.

models.E026: The model cannot have more than one field with primary_key=True.

[Marcin Nowak]

Hi.

I'd ask about  "models.E026: The model cannot have more than one field with 
primary_key=True".
It's not a true. The table or view *can* have more than one *column* as a 
part of a primary key.
It's pretty common, when you have views with cross joins or unpivots.

I can add models.E026 to silenced system checks, but Django is mapping PK 
to the first field of the complex primary keys. This can introduce some 
issues.

I know that supporting complex PKs is hard and generally unsupported, but 
how about removing auto-mapping ".pk" attribute for models with complex 
primary key
As a result there will be no possibility to declare foreign keys to such 
models, but there will be possibility to change E026to W0nn (just a 
warning).

Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7b948901-e643-4bd0-917a-54ff7a18e9f9%40googlegroups.com.

Migrations for dropping columns/tables must not have CASCADE keyword

[Marcin Nowak]

Hi.

My question is about these two:

https://github.com/django/django/blob/master/django/db/backends/base/schema.py#L55
https://github.com/django/django/blob/master/django/db/backends/base/schema.py#L64

Both are loose cannons. All dependent objects are silently destroyed during 
dropping table or column.
Please fix this.
 
Motivation:
I didn't checked what SQL is generated for dropping column.
As a result Django destroyed silently all my views dependent on the column, 
which was dropped.
The database I'm using is PostgreSQL.

Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b21b2791-539f-46e0-a60c-f90c208ca020%40googlegroups.com.

Re: Content types shouldn't be created on post_migrate signal

[Marcin Nowak]

Hi Arthur.

BTW: RunPython() is another thing, which can break your migrations, and 
should not be used (especially not by Django internally), because it relies 
on the application layer.

How else can you do a data migration? There is no 
> `migrations.InsertIntoTable`, 
>

You're right. That's why "Insert" should be (in my opinion) introduced. 
Together with "migrations.Delete".

The problem is that data migration based on app layer (python objects, ie. 
Models and Managers here) will cause troubles after some time (when app is 
changing).
In the other words - you cannot rely on your app layer when doing database 
changes. You should never do that, especially for projects requiring LTS.

RunPython() should be used only when you cannot do anything else. In such 
case you must accept all consequences. I'm not against RunPython(), but 
against doing data migrations using it.

The problem with hypothetical "Insert" operation is with mapping field 
types. Insert cannot use Models directly (app layer is changing over a 
time), but should "know" how to map arguments (python types, values) to a 
database literals. It can be achieved by introducing field mapping for a 
every insert or per migration file (something like "model freeze", but only 
for used fields). 

Also Insert should not accept variables calculated in the app layer (within 
a migration) - it should contain only plain/direct data. But using Python 
as a language for migrations, will be hard to avoid such possibility. This 
is important, because database management should not rely on app layer. 
Using variables (i.e. from settings) would result in inconsistent data 
changes between different environments.

the only other way currently would be to run a `migrations.RunSql` query 
> which may look different based on the database used.
>

RunSQL is not the solution for db agnostic operations. It may be only used 
within a project, because db engine used changes rarely (due to the nature 
and importance of the data and relational databases, and systems dependent 
on the db). 
But RunSQL is a handful operation, because SQL is a natural language for db 
management. I'm doing many raw sqls in migrations.
 

> Two things are wrong:
>
>- automatic creation of content types
>
> Why is it wrong to automatically create a content type? 
>

Maybe "automatic" is not a proper word. They should be created 
automatically, but should be applied "at the right time".
 

> Content type is an opt-in feature since you can remove it from 
> `INSTALLED_APPS`.
>

I know, but it is required by contrib.auth. I saw no project depending on 
something else, so CT is optional.. but theoretically ;)
  

>
>- creation of content types after running migrations
>
> That’s the only real problem for me. Maybe using a signal for 
> `migrations.CreateModel` (e.g. post_create_model), instead of using 
> `post_migrate` would fix it, but there may be other scenarios I’m not 
> thinking about.
>

Because signals are related to the app layer and  mixing app and db layers 
together is generally wrong (sorry for wording, please read as "not so 
good"), changing one signal to another is not a good solution.

Before I wrote about the issue, I was doing diagnosis and I was thinking 
about the issue for a while. So I think that the best place for applying 
data changes is a migration.
Django can detect model changes (adding and removals of models, too), so it 
can generate series of inserts or deletes in the right place, to run them *at 
the right time*.

Or if `django.contrib.contenttypes` is only added later on to 
> `INSTALLED_APPS`?


After adding contrib.contenttypes, Django should check existence of 
django_content_type table. If exists, Django should only check for data 
changes and generate series of inserts/deletes. If not, Django should 
generate inserts for all models. If CT is removed later, Django should 
remove all CT data .


CT opt-in should be connected to a signal related to the creation of 
migration files (Autodetector?), not to a signal related to their 
execution. I.e. pre/post_autodection signals should be introduced.


Kind Regards,

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/af22034d-3983-4b19-87b2-256adc1cb11a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Content types shouldn't be created on post_migrate signal

[Marcin Nowak]

Hi Aymeric.

Thank you for your reply. 

Unfortunately you wrote mostly about me or my writing style, not about the 
issue.
I disagree with your opinion about my comments being passive or aggressive. 
I'm always writing about a piece of code, functionality, 
design/architecture or bug. I never criticised a person directly.

 

> Starting with "There is a huge issue with content types framework" isn't a 
> good way to motivate them.
>
>
But there is an issue with content types framework (not with it's authors). 

 

> Speaking for myself, I would be more eager to investigate if you skipped 
> the hyperbole and remained neutral, for example "I'm facing an issue with 
> the content types framework".
>
>
Sorry hearing that. I'm not native English speaker. 

For me there is almost no difference with these two sentences, except that 
first is about the affected thing ("there is an issue with") and second is 
more about me ("I have a problem"). 
Both are valid, I think. I have a problem which is caused by CT framework's 
design or issue (in fact it comes from a historical reason, before 
migrations era).
 

> You'd have more success if you managed to write in a positive style.
>

I don't think that my style is unpleasant. If it is - sorry for that. 
I'm always trying to describe the problem and give some proposals. 
But I'll try to improve this.
  

> I think the issue itself is valid. I may have hit it before and worked 
> around it, likely be executing a subset of migrations to trigger creation 
> of content types, then executing the rest of the migrations. Django could 
> probably do better.
>
>
I'll generate CTs in very first migration. This will be a workaround, of 
course. 
But because Django uses migrations internally, CT's should be added to the 
database in a same way.

And RunPython() can be problematic. I did something similar with Liquibase' 
executeCommand, which was calling management command. And of course it 
caused problems after changing app layer. 
I'm very conservative about databases and managing them, I think that there 
must be complete separation between db and app layer (in the context of 
managing dbs), because app layer is changing frequently (more often than 
dbs). Mixing both worlds will cause troubles. Always.

Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0e4464f7-2ed7-4e6b-9b7e-f98a385dffd5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Content types shouldn't be created on post_migrate signal

[Marcin Nowak]

]
>
> Wouldn't a workaround be to call create_contenttypes() in a RunPython in 
> your app's migration before inserting the data which depends on the content 
> types?
>
>
Thanks, but as a workaround you can do almost everything. My post is about 
proposal of fixing the issue.

BTW: RunPython() is another thing, which can break your migrations, and 
should not be used (especially not by Django internally), because it relies 
on the application layer. I discussed about it some time ago.


BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/34a6dbc5-0a1c-4c77-be76-8a878a31d336%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: django makemessages management command

[Marcin Nowak]

> You can simply do that by including a gettext_noop('your custom string') 
> line anywhere in your code (or in a specific file you create for that 
> purpose).
>
>
This is an ugly workaround.  Makemessages has other issue - it grabs 
messages from external eggs/packages, which are found in current and nested 
dirs. So beware - it can make a mess with your translations. 

BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c733e5a3-44d1-45fa-9ef9-1596f95ebce2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Content types shouldn't be created on post_migrate signal

[Marcin Nowak]

Hello.

There is a huge issue with content types framework. The data is loaded 
after every migration (post_migrate signal) automatically, and this 
approach is against db data consistency.
The biggest problem is with data migrations, which are applied at the first 
time (on clean database). Any data migration which depends on some content 
types will not insert the data.

The sequence looks like this:

   1. create db
   2. insert into auth_permission ...  inner join django_content_type ...
   3. post migrate: insert into django_content_type ...


Two things are wrong:

   - automatic creation of content types
   - creation of content types after running migrations

Solution:

   - creation of new app should add very first migration, which will add 
   entry to the content types
   - create_contentypes handler should be removed from post_migrate signal

Any data migration, which depends on some content types, should be declared 
with proper dependency. This will guarantee existence of the required 
content type entry.


BR,
Marcin


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8869b1cb-2b92-4e05-823a-92e72308fc23%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Compilemessages and percent char in msgids

[Marcin Nowak]

BUMP

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/06006ec4-5b30-4b74-a756-6fa420655c08%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: #24483 fix breaks dynamic choices

[Marcin Nowak]

>
> Fix of #24483 breaks possibility to declare dynamic choices for model's 
> fields.
> I read the ticket and I understand the initial problem, but it was used 
> frequently to support dynamic choices.
>
> Now we have always consistent but static choices, tests are passing, 
> migrations have consistent choices list (never used, but they are there), 
> but we lost an important functionality. 
>
> It would be nice to bring back previous behaviour and fix the problem on 
> the migrations/keepdb side, not in models.
>
>
BUMP

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3347abe8-9474-4b0a-ab04-e952c4623808%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Compilemessages and percent char in msgids

[Marcin Nowak]

W dniu wtorek, 24 lipca 2018 14:43:51 UTC+2 użytkownik Ramiro Morales 
napisał:
>
> Marcin,
>
> You have upgraded recently to version 1.11.13?
>
>
About month ago. It was litlle painful process, BTW.

 

> IIRC handling of percent symbols in translatable literals has been the 
> subject of a few commits along the history trying to get it right for as 
> many use cases as possible. It's entirely possible it was broken in some 
> way during the 1.11 release cycle.
>

I saw these commits and some tickets (#24257 for example). I tried to match 
these changes, but I'm still getting bug reports about translations 
containing a percent char.


> Please examine the history for the compilemessages module source file. 
> Perhaps you can cherry pick one or more commit to your local copy.
>
>
I did that. I'm using older version, copied into my project/app.

 

> If memory doesn't fail me there is some complexity related to the fact 
> that xgettext (used by makemessages) auto detects some strings that contain 
> % symbols as literals using 'traditional' Python string interpolation and 
> there is no way to control such detection.
>
>
But the problem is that translations are inconsistent. It would be better 
to replace/fix/workaround xgettext instead of making translation process 
invalid...

BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/304b4aa5-e036-47bc-88b7-0d0e54818ee3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Compilemessages and percent char in msgids

[Marcin Nowak]

Hi.

I'm not sure what is going on, but I've updated recently Django to 1.11.13 
and two things seems to be broken (from my perspective):

   - behaviour of translation messages containing % is inconsistent, for 
   example:
   
   - prevously used msgis are staying untranslated with trans/blocktrans:
  
  msgid "Source %"
  msgstr "Translated %"
  
  ugettext("Source %") -> "Translated %"  [OK]
  trans "Source %" -> "Source %"  [FAIL, expected "Translated %"]
  blocktrans "Source %" -> "Source %"  [FAIL, expected "Translated %"]
  
  - msgids with double percent chars:
  
  msgid "Source2 %%"
  msgstr "Translated2 %"
  
  ugettext("Source2 %") -> "Source2 %"  [FAIL, should fail probably?]
  
  ugettext("Source2 %%") -> "Translated2 %"  [OK]
  trans "Source2 %" -> "Translated2 %"  [OK]
  blocktrans "Source2 %" -> "Source2 %" [FAIL, expected "Translated2 %"]
  blocktrans "Source2 %%" -> "Source2 %%"  [FAIL, expected "Source2 %"]
  
  activate(original) /* no msgid/msgstrs defined */
  
  ugettext("Source2 %%") -> "Source2 %%"  [FAIL, expected "Source2 %"]
  
  - msgids and msgstr with double percent chars:
  
  msgid "Source3 %%"
  msgstr "Translated3 %%"
  
  ugettext("Source3 %%") -> "Translated3 %%" [FAIL, expected 
  "Translated3 %"]
  ugettext("Source3 %") -> "Source3 %" [FAIL]
  trans "Source3 %" -> "Translated3 %" [OK]
  blocktrans "Source3 %" -> "Translated3 %" [OK]
  blocktrans "Source3 %%" -> "Source3 %%" [FAIL]
  
  activate(original) /* no msgid/msgstrs defined */
  
  ugettext("Source3 %%") -> "Source3 %%" [FAIL, expected "Source3 %"]
  ugettext("Source3 %")-> "Source3 %" [OK]
  trans "Source3 %" -> "Source3 %" [OK]
  trans "Source3 %%" -> "Source3 %%" [OK]
  blocktrans "Source3 %" -> "Source3 %" [OK]
  blocktrans "Source3 %%" -> "Source3 %%" [OK]
  
  - and second is wrong compilemessages behaviour, because it scans and 
   compiles all files starting from cwd. Eariler only django apps were 
   scanned. Now this command is trying to compile everything, also unused or 
   already compiled files from 3rd party packages.
   

First issue must be fixed. Translations behaviour isn't consistent. I 
cannot make any reliable translations having % in msgids anymore. Assuming 
that double percent in msgid is required (one for escape, one for percent 
sign), this makes wrong translations with "p/n/u/gettext" functions if 
msgstrs with single % were not defined. Please imagine switching to not 
fully translated language, and user will get double % in messages (excl. 
templates with trans, but not blocktrans).

Second should be probably fixed by reverting a patch which adds a tree scan 
of files. Currently I did  workaround by copying older version of 
compilemessages into my project's app. But this is not how Django should 
work. I'm using buldout where all external eggs,parts and other projects 
lies in cwd.

Any thoughts?

Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5918116a-319d-4d5b-b0dd-dcfdbf7b3928%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: default values on database level

[Marcin Nowak]

W dniu czwartek, 29 marca 2018 19:28:39 UTC+2 użytkownik Michael Grijalva 
napisał:
>
> Yeah seems like `db_default` was the felling, at least last time I scanned 
> through this long-going discussion :D 
>
> I've had to tweak Django's migration handling a bit to handle our 
> zero-downtime requirements.
>

This is not the only one issue with Django migrations. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/286135f8-8407-4964-8ebe-047cdddcf0c3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#24483 fix breaks dynamic choices

[Marcin Nowak]

Hi.

Fix of #24483 breaks possibility to declare dynamic choices for model's 
fields.
I read the ticket and I understand the initial problem, but it was used 
frequently to support dynamic choices.

Now we have always consistent but static choices, tests are passing, 
migrations have consistent choices list (never used, but they are there), 
but we lost an important functionality. 

It would be nice to bring back previous behaviour and fix the problem on 
the migrations/keepdb side, not in models.

BR,
M

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/00b9fee3-d3e1-4ee8-811e-ac38ef7b8992%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About migrations

[Marcin Nowak]

>
>
> Anyway, I don't want anyone to think that I complain as I don't have the 
> resources to write yet another migration tool and both South and Django 
> migrations beat writing SQL by hand.
>

Have you tried Liquibase ever? It is very reliable, unfortunatelly it is 
missing automatic changesets generation (because models aren't tracked) and 
you must rewrite Django's and 3rd party apps migrations by yourself. 
I started this topic to talk about an improvement to the Django migrations, 
to provide some advantages known from Liquibase.
Knowing LB may help everyone to understand my proposals.

So the most important for me is a separation from an application layer to 
improve stability.
The 2nd one is having a possibility to direct use of Django and 3rd party 
app migrations, to help everyone making upgrades.
The 3rd one is a "dependency hell", where project-wide and flat files of 
sequences are easier to maintain, especially when project dependencies 
(i.e. 3rd party apps) are changing.
They are very important for a long-term projects.

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cf0e4d55-2df8-41cf-8779-c7281be54733%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About migrations

[Marcin Nowak]

>
> Have each Field class deconstruct to a field name and params  [...]


 
Thanks, @patrys. A field deconstruction is a key to achieve what I tried to 
describe earlier.
We can discuss the details about implementation, but this is not important 
now.

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/62f0734e-5519-4935-a6fe-aace4b31aa76%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About migrations

[Marcin Nowak]

*About database agnostic migrations.*

Liquibase is a tool which is decoupled from the app layer and gives a 
possibility to write agnostic changesets.
To be independent you must use builtin operations and narrow used field 
types to "well known" subset 
. 
Anything else is passed through, same as plain SQLs are passed througd.

So as a DB architect I can decide what is most important to me - 
portability or db-specific features.

This concept may be easily adopted to the Django migrations, because the 
operations are currently implemented.


*About custom fields*

Liquibase is extensible. The concept is about extending a "well-know" 
fields subset by registering extensions.
For example - django.contrib.postgres may provide an extension to the 
migration subsystem by registering new types and new operations.

For 3rd party apps there will be requirement to deliver extension to a 
migration system, not to the project itself.
This would be real "game changer". Until the extension is installed, your 
migrations will work. 
You may recreate you app from scratch but leave extensions - and your db 
refactorings will be still safe.  


*About files separation*

Liquibase has the include operation. The similar may be implemented for the 
Django migrations. 
The only problem is that the Python code is rather unordered, and some 
explicit ordering should be introduced.

This would be quite irrelevant for Python-based changesets (migration 
files), but after dropping direct application layer dependency there would 
be possibility to use other language than Python to describe operations to 
aplly. 


*Application layer agnostic migrations*

The migrations subsystem must match their field types to the registered 
one, and render a changeset (migration file) using these registered *names 
*instead 
of direct field classes.
Anything unmatched must be passed as a CUSTOM type or just passed through 
by name (may introduce incompatibility, when someone will register a field 
of the same name in the future).
How to mark a custom type requires discussion, of course.


*Backward compatibility*

You can prevent backward compatibilty by allowing usage of directly 
imported fields, same as nowdays. In that case the field-mapping layer will 
be bypassed.


*Automatic changes detection*

Should work same as nowdays. But I must dig into internals to confirm that.


Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dca84944-a35b-47ea-9463-a64ea2c48343%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About migrations

[Marcin Nowak]

>
> Understand when I say that what you are proposing is a very, very big 
> change. Django's ORM is heavily coupled to runtime information and the app 
> layer, and I tried for many years to decouple them and ran into all sorts 
> of issues as a result. Importing the fields from the source code ended up 
> being the easiest, safest method that also happens to produce very 
> easy-to-understand errors when it breaks (rather than using old definitions 
> or silently failing).
>

Understood. 

>From my perspective, from few years work with Liquibase (started in 2008 
maybe), I had no failures except migration logic errors (due to ordering) 
or executing binaries.
This solution is completely separated from app logic, it is hell-stable, 
and supports databases longer than any app lifetime.

The problem with external solution lies in changesets creation, which is a 
completely manual process, where changes from 3rd party apps or Django 
itself cannot be automatically applied. 
The only one what is really problematic with Django migrations is "heavy 
coupling to app layer", as you said, which may cause migrations system 
failure when app layer is changing.

I'm trying to find a solution to achieve both - stability and automation 
(or semi-automation).

I am impressed that your thousands of migrations only take a few minutes to 
> run; 


Believe me or not, I was thinking about hundreds. Sorry for a mistake. It 
may be related to my poor english skills.
Currently there are 1480 changesets, to be precise. They're applying within 
2-3 minutes on two databases (sequentially) on a poor jenkins machine.

BR,
Marcin

>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/22fc3ae1-379e-4bca-aadf-5b9dc07a4bb0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About migrations

[Marcin Nowak]

>
>
> Some of these are problems, yes, but you have to understand they are 
> tradeoffs and the alternative is, in my opinion, worse.
>
>
Yes, I understand. But maybe there is another / better alternative.
Let's simplify a little and talk about a python deps first.
 

> > I believe that there exist some solution, which drops app layer 
> dependency ands allow using custom fields
>
> I would love to see what this would be. I spent a decade trying not to 
> import custom fields from the code to allow this and never succeeded; the 
> only way that I could make it work was to import them. Because Django 
> allows fields to change their type based on the database being talked to, 
> you can't bake the type into the migration at creation time
>

The advantages comes from db type independency, this is true, but in the 
other side you're including the app layer dependency. 

Let's imagine that one of builtin field will change it's definition. 
Running migrations on two different Django versions will produce two 
different outputs.
My perspective is more database-like than app-like, so I'm expecting same 
db schema as a result (for both cases).

So the first thing that comes into my mind sounds: a complete definiton 
should be baked in migration file. Then, when app layer changes (i.e. 
upgrading framework or changing custom field definition), the migration 
system should identify the change and produce new migration with baked in 
definition. If it is possible to develop, you'll achieve less dependencies. 
The definition (a meta-description of the field) will be baked in, instead 
of depending on the field itself. And you'll preserve database type 
independency.

This is a just first concept that comes to my mind, now.

> Python code in migrations
>
> This has always been optional; Django's makemigrations will never put in 
> RunPython for you, so if you choose to use it, then it's your choice and 
> you get the problems along with it (more tied to application code). 
>
>
In that case I'd like to avoid RunPython in Django's contrib apps builtin 
migrations, not to remove the possibility of running any executable.
I'd just like to add comment to it  "do it at your own risk" ;)  
 

> > Separate files for changesets
>
> This is needed if you are going to have migrations per-app and thus let 
> apps ship migrations. If you want to not have per-app migrations, then you 
> can have just the one file, but then you make third-party apps have to ship 
> migration snippets you need to remember to include when you make the next 
> migration.
>
>
Leave it as is for a while. It isn't so important.
 

> Squashing is a just a workaround tool for migrations design issues
>
> Yes, specifically it is designed to help solve the 
> custom-fields-app-dependency issue. It's also just nice to have anyway when 
> you get to 100 migrations.
>
>
The second isn't an issue, in practice. After x years I have thousands of 
migrations in Liquibase, and the only one downside of this lies in time 
required to run them all in CI build. But this is automated, so nobody 
cares about minute or two.
 

> > A discussion about removing app layer dependency and removing or 
> limiting RunPython usage, mostly.
>
> OK, so what is your proposed alternative for custom fields? It is no good 
> to propose to remove something without proposing what to do in its place; 
> we can't drop custom field support in migrations.
>

To be precise - I don't want to remove anything and break compatibility. 
I'd like to improve some things. 
The (first) proposal is about decoupling migrations from the app layer. I 
wrote the example few lines above.  

I don't want to write all possible proposals at this moment, because I 
suppose that you discussed the topic a very long time.
We can focus now on the sepearation of a field definitions, to achieve 
consistency in time.
 
BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ed5f17fe-3967-494d-8423-7d0143282ac9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

About migrations

[Marcin Nowak]

Hi.

At the begining please forgive my engilsh - i'm not a native speaker.

I wrote here and in other places my thoughts about db migrations few times, 
and probably Tim remembers me so well.
My opinion was not changed, but I realized that I cannot leave Django 
ecosystem for a long time.
In that case I'd like to talk about migrations, their advantages and 
disadvantages, and about possible solutions.

Advantages:

   - fast (automatic) creation
   - database independent (model-centric)
   - a standard for django itself and reusable apps
   - possibilty to create migrations manually

Disadvantages:

   - dependent on application layer (a python code - field, model classes, 
   etc)
   - allowing python code within migrations
   - separate files for changesets makes ugly conflicts when merging 
   branches
   - squashing required

I'd like to focus on disadvantages, because they're a casue of using 
alternative solutions by me (Liquibase in that case).


*Application layer dependency*

This is something whch causes fails of whole migration system. 
References to the application layer are included within migration files, 
because of saving a "model state".
Any significant code change will broke migrations.  We must avoid such 
situations by squashing migrations at "the right time".

In my opinion migrations should be application independent.  Unfortunatelly 
whole system is based on models written in Python, which may includue 
custom solutions (i.e. model fields).

I understand that it is hard to cut-out this feature, but I believe that 
there exist some solution, which drops app layer dependency ands allow 
using custom fields.


*Python code in migrations*

This is a generally bad idea. Any python code is strictly related to the 
time. When code changes, a "pythonic" migration may fail. 
And you will never know about failure  until you setup CI properly.

There are very rare cases, when something from app layer must be called 
between releases. 
In that cases I'm using management commands, but Liquibase allows me to 
execute any binary.
It is not so straightforward, so as a developer you feel that you're doing 
something strange/non-standard, and you should be careful.

The general problem is that some migrations stops working properly in the 
time. 
This should never happen. Well... I'm used to this and I adhere to this 
principle.

I'm using Liquibase for years and my migrations broke few times mostly on 
changesets based on binaries execution (a python code through manage 
commands).

 
*Separate files for changesets*

I like Liquibase because of possibility of using files containing more 
changesets.
When I need to split migration files, I can do that and use "include" 
directive.
There are advantages: easy conflict resolution, just one file per database 
and per current major release, and full changes history in older files.

Django produces spearate migraiton files. This causes conflicts and 
reordering migrations by declarations ("depends"). 
And because squashing is almost required for long project, you're loosing a 
changesets history.


*Squashing required*

For long-live projects this is a required operation. Squashing deletes 
changes history and removes obsolete python code from migrations.
Squashing is a just a workaround tool for migrations design issues.



*What I need?*

Well, I'd like to simplify my work. That's obvious. Maintaining a Liquibase 
changes outside Django is harder and requires more time, especially when 
I'm adding new apps or upgrading existing ones.
But I don't want to switch back to the Django migrations, because of the 
design issues described above (mostly beacuse of the app layer dependency 
and squashing requirement).

I started prototyping a tool which translates Django migrations into pure 
SQL, which can be embedded in a Liquibase changesets.
But Python migrations can't be translated to SQL, of course. And the worst 
thing is that Django provides Python migrations even for contrib apps.

In that case I realised, that building such tool without changing a concept 
of the Django migrations (by you - Django Developers), is a little 
unreasonable, until Python code execution is accepted and used internally 
(contenttypes, 0002).


*What I would like to achieve by this post?*

A discussion about removing app layer dependency and removing or limiting 
RunPython usage, mostly.
This should eliminate requirement of squashing and increase migrations 
stability.

Separate files vs big file is not important now. This is just unhandy, but 
does not produce failures.


Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at 

(contrib.admin) Improving object deletion

[Marcin Nowak]

Hi all.

Sometimes there is huge amount of related objects. In the effect there is 
no possibility to delete a model because of http timeout error.
Displaying objects to be deleted is quite nice feature, but just unusable 
in cases like that. 

Another thing is that nobody will read 74k related objects. Believe me. It 
is better to display a summary in that case. 
To display the summary, a related objects must be counted and this is time 
consuming process, of course, but we can count them similar way to 
Collector's fast deletion (using querysets instead of counting models one 
by one).

There is also other design flaw in ModelAdmin.delete_view() - related 
objects are counted twice: one time for display, and second time after 
confirming deletion (for request.method==POST). For the second case django 
is checking only permissions and collected objects aren't used for anything 
else. Until Django provides no row-level permissions checking those can be 
done without instantiating all related objects again.

I'd like to suggest:

   - separation of permissions checking from related objects collector,
   - adding configration option for ModelAdmin.delete_view (to display 
   related objects, display summary, or finally display nothing).

Thank you.

Marcin.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a38bd871-459d-4783-a435-c42de02310d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Resolved: wontfix is not productive

[Marcin Nowak]

On Tuesday, July 26, 2016 at 11:16:18 PM UTC+2, Aymeric Augustin wrote:
>
>
> Specifically, please explain why this has to live in Django.

Because Django has an implementation which can be extended without 
rewritting a whole thing from scratch?
Well... it is your decision about "including batteries" into the framework, 
making it full-stack...

I like quite opposite design, personally, but here is a guy who did useful 
improvement and his work was dropped because of... including another useful 
battery. 
I know that every line of code needs to be maintained and you're very 
careful with adding new features. 
But maybe there is a good time to think about decoupling Django?

BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/387c7fc9-bf01-4654-9e71-2fc01be5b352%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: UTF-8 name of a staticfiles test is breaking Django installation

[Marcin Nowak]

>
>
>> I'm using Buildout and virtualenv with Python 2.7.9.
>>
>>
> and setuptools 25.1.2 
>
>
It works with setuptools 17.0, but displays warning:

Getting distribution for 'Django==1.8.14'.
'tests/staticfiles_tests/apps/test/static/test/⊗.txt' not ANSI_X3.4-1968 
encodable -- skipping
warning: no previously-included files matching '__pycache__' found under 
directory '*'
warning: no previously-included files matching '*.py[co]' found under 
directory '*'
'tests/staticfiles_tests/apps/test/static/test/⊗.txt' not ANSI_X3.4-1968 
encodable -- skipping
Got Django 1.8.14.
 

M.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9914693c-56cd-43f2-ac19-b763628ce740%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

UTF-8 name of a staticfiles test is breaking Django installation

[Marcin Nowak]

Hi all,

Here is a file which breaks installation on fresh Debian system:
 
https://github.com/django/django/blob/master/tests/staticfiles_tests/apps/test/static/test/%E2%8A%97.txt

I'm using Buildout and virtualenv with Python 2.7.9.

The full traceback is:

Traceback (most recent call last):
  File "", line 1, in 
  File 
"/home/project/env/lib/python2.7/site-packages/setuptools/command/easy_install.py",
 
line 2271, in main
**kw
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
  File 
"/home/webcar/env/lib/python2.7/site-packages/setuptools/command/easy_install.py",
 
line 409, in run
self.easy_install(spec, not self.no_deps)
  File 
"/home/webcar/env/lib/python2.7/site-packages/setuptools/command/easy_install.py",
 
line 645, in easy_install
return self.install_item(None, spec, tmpdir, deps, True)
  File 
"/home/webcar/env/lib/python2.7/site-packages/setuptools/command/easy_install.py",
 
line 694, in install_item
dists = self.install_eggs(spec, download, tmpdir)
  File 
"/home/webcar/env/lib/python2.7/site-packages/setuptools/command/easy_install.py",
 
line 845, in install_eggs
unpack_archive(dist_filename, tmpdir, self.unpack_progress)
  File 
"/home/webcar/env/lib/python2.7/site-packages/setuptools/archive_util.py", 
line 52, in unpack_archive
driver(filename, extract_dir, progress_filter)
  File 
"/home/webcar/env/lib/python2.7/site-packages/setuptools/archive_util.py", 
line 148, in unpack_tarfile
prelim_dst = os.path.join(extract_dir, *name.split('/'))
  File "/home/webcar/env/lib/python2.7/posixpath.py", line 80, in join
path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: 
ordinal not in range(128)
An error occurred when trying to install Django 1.8.14. Look above this 
message for any errors that were output by easy_install.

The path to a file which fails is:
/tmp/easy_install-A3XRR9 
Django-1.8.14/tests/staticfiles_tests/apps/test/static/test/⊗.txt

Is this file really needed in Django dist?

Marcin
 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/967785a0-3df7-4454-a2dd-7c5e6875991d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

SelectMultiple widget value_from_datadict enhancement suggestion

[Marcin Nowak]

Dear all.

I would just say that implementation of SelectMultiple.value_from_datadict 
is not based on duck typing, and has some disadvantages.
Source: 
https://github.com/django/django/blob/master/django/forms/widgets.py#L606

def value_from_datadict(self, data, files, name):
if isinstance(data, MultiValueDict):
return data.getlist(name)
return data.get(name)

First, always checking the instance type has worse efficiency than 
try-except block, because most hits may be returned without any additional 
checking.
Second, as an author of library which also works with Django, I'm providing 
multivalue-like dict, but own implemented - it is not an instance of 
MultiValueDict and never be, but it has similar interface (i.e. `get()` and 
`getlist()` methods) 

Duck typing will work for both cases. The implementation might look like:

def value_from_datadict(self, data, files, name):
try:
return data.getlist(name)
except AttributeError:
return data.get(name)

For most cases a MultiValueDict / QueryDict are passed to forms, so the 
function will immediately return a proper value, and will also work for 
objects that "looks like MultiValueDict". On other cases it assumes that 
the `data` object is dict-like.

What do you think?

Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b5fe2add-05d7-4def-9863-ceecf4d0ecdc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

On Thursday, June 23, 2016 at 12:12:51 AM UTC+2, Shai Berger wrote:
>
>
> Thanks for this thoughtful clarification. I think I understand your 
> position 
> much better now. If


Thanks for reading, Shai. I know my English is far from perfect, so I 
appreciate your involvement.

I understand correctly, there are two issues you find with 
> migrations: 
>
> - They are designed to deal with schema changes, while you'd rather have a 
> tool for one-time schema generation; 
>
>
In most cases, yes. But I see now new possibilities (I'll wrote about my 
thoughts another time)
 

> - Migration files are Python code with potential dependencies, while you 
> prefer 
> your schema (and even data) changes to be expressed in pure SQL. 
>
>
This is just one of all reasons, but yes.
  

> > It was very handy to prototype the app layer and generate DDL. Then DDL 
> was 
> > used directly in db schema management system. 
>
> - Isn't it, then, possible to generate a schema by evolving it while 
> prototyping on your development machine, and when done, use the db schema 
> management tools to extract the DDL directly from the database? Isn't it 
> actually better than generating DDL in Python, according to your views? 
>
>
It depends on what "better" means. Django is very helpful for generating 
initial sqls, especially for indexes and foreign keys.  Further changes we 
make using db mgmt tool. This is part of our workflow for years - prototype 
python code, make some tests, generate sql for new models, make additional 
sqls (views, triggers, etc), finish feature, send to qa, deliver. 

Writing Django models in Python is faster than writing plain SQL or using 
db GUI tools. And we are writing them just only once. Using other DDL tools 
and making diffs between databases will slow down our workflow. We're 
accepting Django's table/columns/m2m/constraints naming conventions, and 
we're accepting all limitiations. The db schema is compatible with Django. 
We didn't found faster method yet (except Django's changes autodetection).


> - But Andrew's patch[1] completely ignores any existing migration files, 
> bypassing all problems related to migration files as well as the "journey 
> through winding road". 


If it will not use migration files when they already exists, it sounds ok.
I know too little about migations internals, so I can't be sure for now. 
 

> What he suggests is to use the migrations 
> infrastructure to generate just the SQL for the difference from "nothing" 
> to 
> "current models" (so, no "model state changes" either). 


I thought that diff from nothing to current will be achieved by applying 
migrations sequentially.
It sound ok if this will be ommitted.  
 

> Even the issue of 
> SchemaEditor's deferred_sql is handled by "collect_sql" mode, as you've 
> already noted yourself (and by the way, the deferring mechanism is 
> required in 
> order to resolve circular dependencies between models). Do you have other 
> objections to this patch's approach? 
>

Well.. yes and no.

I just have almost finished patch which brings back sql* commads based 
directly on SchemaEditor. The difference is in ommitting 
django.db.migrations completely, so there is straightforward path to get 
current db state (without using whole complex migrations abstraction 
layer), less dependencies and less risk of failure due to some 
db.migrations issues/regression. The drawback is that output sql may little 
differ comparing to output from migrations-based alternative, even if both 
methods will use same SchemaEditor. There also will be new association to 
SE, so further changes of SE will require attention not only to migrations 
subsystem. Please share your thougths because I can miss something.

I don't how important is to use django.db.migrations for every sql-related 
use case. It is about your preferences and design decisions. For me it is 
not a problem until both (sql* and migrations) depends on same sql factory 
(SchemaEditor here). For me, the sql* world is quite different from the 
migrations, and it is ok to not mix up these two things. The cost of 
maintaining additional association seems to be low, but I might be wrong.   
  

The "dead end" term used by me in the context of Andrew's patch was mostly 
related to the migration files issues. Having already better knoweldge I 
will look at his pull rq again.

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2c4cbc51-d9e3-401e-a8ab-fdb3e35d9c9a%40googlegroups.com.
For more options, visit 

Re: Extending JSONField serialization

[Marcin Nowak]

>
>
>
> It's not resolved - the ticket is open, and Accepted. 
>
>
Thanks. I've seen comment with status set to closed and I didn't notice the 
current.
My fault.

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fd3b2d11-cb89-4db9-97c9-fec1f9ee4934%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Extending JSONField serialization

[Marcin Nowak]

On Tuesday, January 5, 2016 at 5:29:14 PM UTC+1, Tim Graham wrote:
>
> This came up in a ticket a couple days ago: 
> https://code.djangoproject.com/ticket/25995
>
>>
>>

May I ask why this ticket was resolved just by adding some information to 
the documentation?
I do not understand why some 3rd party package has some significance here.

Let's add encoder_class as an optional kwarg of JSONField and pass it to 
loads/dumps calls.
Bad idea?

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2c74fc7e-a550-45f3-943f-67876cc79a17%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

I can give you many reasons and many details, but I know you will not agree 
with my conclusions. I remember what you (or someone from the team) said 
about my ideas. I've shared them already and someone told me why you've 
decided to do it that way, and why you wouldn't change it. I'm respecting 
your visions and ideas, but they are becoming more and more distant from 
mine. And it is OK until you're introducing obstacles (from my perspective, 
just a regular user). Removing daily used tool is an obstacle, believe me. 

Let's back to the migrations. They are very sensitive to Python imports, 
dependencies, and used objects.  Some time ago I've got a project to 
rescue, where deps were poorly managed and the code was trashy. After 
cleaning up, some apps couldn't migrate and Django was creating initial 
migration instead of adding migration with differences. If I remember the 
problem was related to some import error or some missing object. But it 
does not matter here. The worst thing was that Django skipped all 
migrations and tried to do somenthing unplanned, something that could 
damage the database, and something that damaged initial migration file. 
Just because some old and already applied migration has an some python 
error. And this is practically uncontrollable and unreliable behavior. 

What output will generate `sql` command based on schema migrations in such 
case? Should it really be dependent on the complicated system, where it is 
created to translate current state of model's metadata to DDL? I don't 
think so. I agree with you that there should be one way to do something. 
But I'm not sure that the results and use cases of "sql*" commands are same 
as for migrations.

Schema changes are about adding, removing or altering something in the 
database. Application layer issues shouldn't affect the sequence of DDL 
changes. Migration system can be written in Python (as a part of Django), 
but changesets should not be dependent on any part of an application layer. 
There is a one fundamental reason for this approach - the application layer 
is chagning more often than the database (both for schema and data). That's 
why it is so important for long-term projects. I don't like saying that, 
but for my 16 yr pro experience I saw this rule almost in every system. 
Then look at Django which is forcing you to use it's migrations, highly 
coupled with application layer, sometimes failing, and "owning" my 
old-and-very-important database... Do you understand now why I'm suggesting 
Django (from 1.7+) for short-term and smaller (simpler in terms of a 
complexity) systems? I'm not saying that they are completely wrong. They 
have advantages and the most I like is autodetection of model changes.

`sql*` commands were decent tools for translating model metadata to DDL. It 
was very handy to prototype the app layer and generate DDL. Then DDL was 
used directly in db schema management system. In this scenario we aren't 
talking about creating Django' migrations, especially not about tracking 
model changes. "sql*" commands were always releated to the current schema, 
so there is no necessity to change this. So whenever I'm talking about 
"sql*" command, I'm thinking about model's current schema and similar use 
cases.  I'm asking myself again: Should it be dependent on the migration 
system? What benefits it will give? Is it worth implementing simple 
function based on such complicated migration engine? How much it will cost? 
Who will use it? 

The reason of removing two differenct ways of managing db schema is clear 
for me. I just do not understand removing the tool without providing decent 
replacement for it.  For example, CBV generics were replacement for FBV 
generics. There was an incompatibility in the interface, but the generic 
views functionality was saved. For "sql*" commands there is no replacement. 
 I think that you're assuming that builtin migrations covers all use cases 
and you aren't worrying about projects, where migrations aren't used. I 
understand this, but it narrows the flexibility of Django.

Believe me or not... Whenever fellows asks me "what instead of Django?", 
I'm answering - Django.  I'm pointing (not disparaging) weakness just 
because I care and because I'm using Django for a very long time.  Django 
is good, but may be better. I would like to contribute and make it better, 
but I'm afraid that my ideas would be dropped due to some kind of "Django 
ethos". The most counter productive for me is rejecting the idea, then 
implementing it after X years. It was happened few times for me and I don't 
fully understand that. 
 
I'm with Django from v0.96. The things went little wrong starting from 1.7, 
IMO. Some guys on this forum told me, that I'm using Django bad way and it 
is possible that Django does not fit my needs. I can agree, but older 
versions were fitting my needs pretty well. My needs are still same, so 
what's happened? As a result I'm just giving advices like "do 

Re: Fate of sql* management commands

[Marcin Nowak]

>
>
> So add another step in the end to remove the temporary migrations folder 
> created just in order to enable sqlmigrate. If you're a real purist, add 
> yet 
> another one to make sure you don't have a django_migrations table left 
> over. 
>
>
Sorry, Shai, I doubt you understand the problem.


Really, this is django-users turf. 


Do what you want. IMO it isn't.

I'm talking here about ressurecting killed tools. And currently I'm working 
on a patch.
I think that redirecting discussion about Django internals to django-users 
list is more counter productive than my earlier thoughts. 

PS. I'm not a purist, but I don't like django_migrations table because it 
is created automatically in foreign database, which Django should not touch 
without direct command from dba architect.
   
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ed060117-164f-4814-b587-031c7dbba2e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

>
>
> You can write the "sqlall" you want as a shell script -- remove any 
> existing 
> migrations,  use "makemigrations" to create an initial migratgion, 


Not exactly. I'm not using migrations [.. cut ..]  
 

I repeat Aymeric's advice that spreading your frustrations over this 
> mailing 
> list is counter-productive. 
>
>
Same as removing useful tools. Let's remove `runserver` command for a year 
or two, ok?
In the other thread I just gave advise to guy, who is going through path 
similar to mine. 

[.. cut ..]  

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e6eeb4ae-9b76-48fc-ae87-49a8e1730cf2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

On Tuesday, June 21, 2016 at 10:55:13 PM UTC+2, Marcin Nowak wrote:
>
>
> Somebody wrote eariler that `sql*` commands can be written as an external 
> app/package. They can't. Django internals must be fixed before this, or new 
> app must provide complete solution including sql generator like SE (but 
> this is a second nonsense, because SE is strongly coupled with db backends).
>
>
Errata - I've missed "collect_sql" mode, where sql is not executed but 
collected. The patching should not be neccessary. The name of "execute()" 
method is misleading in such context. 

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d811b807-e60f-4fb1-bf10-3d6ae77dc7a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

On Tuesday, June 21, 2016 at 5:49:08 PM UTC+2, Marcin Nowak wrote:
>
>
>
> On Tuesday, June 21, 2016 at 4:00:51 PM UTC+2, Tim Graham wrote:
>>
>> No, it's not as simple as reverting the removal commit [0]. All the 
>> legacy schema generation methods are removed too [1]. 
>>
> We don't want to add them back 
>>
>
> I know.  
>  
>
>> but rather use the SQL generation from the migrations system 
>> (SchemaEditor classes). Please read this thread closely and look at 
>> Andrew's pull request [2].
>>
>>
> It is complicated for such simple job (dumping model schema to sql) but 
> I'll give a try, just later. 
>
>
No, Tim. I will not go through the Andrew's path. It is a "dead end".  
What the `sql` command should do is just simple dump model meta to desired 
sql dialect.

Migrations are designed to manage and apply model changes, so it is quite 
different use case. 

`sql` command should not depend on migrations, because:

   - they may be corrupted  (common case with south/django migrations; they 
   are unreliable)
   - achieving final state is a long journey through winding road,
   - app may not include migrations itself,
   - it is about SQL but not about managing model state/changes.


The only way is to fix SchemaEditor by extractnig sql from create_model(), 
without executing it.  But it's not so easy, because SE requires to be run 
as a context manager and executes deffered sqls at __exit__().  There are 
other SE`s methods which depends on defferend sql execution at 
`__exit__()`. Sorry, but it looks like code written in big hurry.

If SchemaEditor will not be removed in near future, I can try to patch it 
and use as a base of new `sql` command. I think there is no other 
reasonable way to do that. 

Somebody wrote eariler that `sql*` commands can be written as an external 
app/package. They can't. Django internals must be fixed before this, or new 
app must provide complete solution including sql generator like SE (but 
this is a second nonsense, because SE is strongly coupled with db backends).

And finally I would like to remind you, that `sql*` commands were removed 
by you. This was a mistake. Not the first, not the last.  But when you're 
asking me what I'm hoping writing such mails, just please stand in my (and 
others) shoes and think a little about it.

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5bd5ef38-83b2-470e-b574-9a533860115a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

On Tuesday, June 21, 2016 at 4:00:51 PM UTC+2, Tim Graham wrote:
>
> No, it's not as simple as reverting the removal commit [0]. All the legacy 
> schema generation methods are removed too [1]. 
>
We don't want to add them back 
>

I know.  
 

> but rather use the SQL generation from the migrations system (SchemaEditor 
> classes). Please read this thread closely and look at Andrew's pull request 
> [2].
>
>
It is complicated for such simple job (dumping model schema to sql) but 
I'll give a try, just later. 
Currently I must backport JSONField from 1.9 to 1.8, because I can't 
upgrade 1.8.6 to 1.9, because it has removed `sql*` commands.  
 
Marcin.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/81ceed6e-b3c2-41ae-89dd-b41fa4c23ed2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

On Tuesday, June 21, 2016 at 3:28:26 PM UTC+2, Tim Graham wrote:
>
> Marcin, what are you hoping to accomplish with your latest mail? As 
> Aymeric said in another thread, repeated complaining is not going to help. 
>

Sorry for that. I'm just overhelmed and tired of "fighting" with missing 
things which were good, then removed, and now someone should reimplement 
them again.
 

> There's already an accepted ticket for this feature. If you want to be 
> productive and helpful, pick up Andrew's patch and try to complete it. 
> Alternatively, if the business needs justify it, you could offer to pay 
> someone to implement it if that suits you better. Tim
>

Well, I'll do something with pleasure. And please don't worry about money. 
But I will not give any patch until you're rejecting change at a 
concept/design stage.

The old sql* commands were good and helpful. To bring them back someone 
must "revert" commits with removals, so there will be an old code 
generating SQL directly from models (ommiting builitin migrations). Do you 
agree?

 
Marcin.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1c2d2b7e-05dd-45e9-88e6-0515945295ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Threads and db connection handling question

[Marcin Nowak]

On Tuesday, June 21, 2016 at 2:34:11 PM UTC+2, Aymeric Augustin wrote:
 

> Regarding database connections, if you weren’t using Django, you’d have to 
> create a database connection with something like `psycopg2.connect()` and 
> dispose of it with `connection.close()` when you no longer need it. When 
> you’re using the Django ORM, database connections are opened automatically 
> when needed. You still have to close them manually if you aren’t working 
> within the traditional request-response cycle.
>

Yes, I'm using ORM. The connections are opening silently in main 
process/thread.
I've tried to close them manually, but I've had some kind of problems 
(don't remember what kind of), probably with wrapped `close()`.  


> If the implicit connection establishment is causing trouble, it shouldn’t 
> be hard to write a custom database backend that doesn’t have this behavior. 
>

Maybe this will help. It's worth trying someday.

 

> Spreading FUD about Django’s supposed unsuitability for non-trivial 
> projects isn’t going to help with your issues and isn’t going to create a 
> context where people will look forward to helping you, so please consider 
> avoiding gratuitous attacks if you need further help.
>

This not a FUD but the honest advice. And not an attack but statement of a 
fact. It's nothing personal.

There are much more problems with Django used in some kind of projects, 
especially in long-term projects. I wrote about those many times and almost 
always I've got two answers: 1) do not upgrade  2) this will not be 
changed. Oh, and third - "write yourself", and I'm writing and rewriting 
old good parts of Django and not talking about this (this is only what I 
can do in that case). 

But I belive this is a only matter of time.  I remember how big was 
resistance of implementing `model.reload()`. 7 years of talking about that, 
right? And here we go - it is available from v1.8 as "refresh_from_db()" :) 
 So let's hope for the best :)
 
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/559e8fdc-d49c-49c9-b71d-f2d9973ae085%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

On Monday, June 1, 2015 at 5:11:27 PM UTC+2, Andrew Godwin wrote:
>
> OK, so I've just gone ahead and done the initial work on this: 
> https://github.com/django/django/pull/4729
>
>
They shot not only in my head - 
http://stackoverflow.com/questions/35455706/equivalent-of-sqlall-in-django-1-9

Generating sql from migrations is useful for translating python migrations 
into SQL (the only one good way to handle database schema).
But old `sql` command generates SQL directly from models, so an application 
must not have new migrations defined.

The old-style commands are still required. I heard from my fellows that 
they are downgrading their projects from Django 1.9 to 1.8 due to missing 
sql commands. Not so funny.

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8f21d96f-c309-46c7-8ec2-19af7189bc12%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Threads and db connection handling question

[Marcin Nowak]

 

> From version to version it turns more into handy tool for blog 
> creators/webdesigners
>

Don't get me wrong. For smal/short-term projects (up to 1-2yr of operation) 
I'm still using Django and I would recommend it everyone for that kind of 
job. 
For long-term projects the first thing that will kill you is a db migration 
system (due to unnecessary python dependencies inside each migration file).

Marcin 
  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/52d52fa9-56f7-47f1-ae8f-68ef632d5d9b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Threads and db connection handling question

[Marcin Nowak]

On Saturday, June 4, 2016 at 12:12:42 AM UTC+2, Cristiano Coelho wrote:
>
> Aymeric, I have never said anything about connection pool, I'm talking 
> about thread pooling to perform async work
>

I have similar requirements and issues with Django. It looks like it is 
completely unreliable in async environment, and the worst thing is that I 
can't force Django to create new connection within the thread/process. I'm 
fighting with this almost day by day, enssuring that no db operation is 
performed in asyncs callbacks, and I'm using ram to return results to main 
thread/process and then write them to db. 

Well, they'll say that Django is not designed for that cases. Maybe. My 
advise - don't use Django for medium and big projects. Using Django for 
that job was my biggest mistake.   From version to version it turns more 
into handy tool for blog creators/webdesigners, forcing everyone else to 
(for example) do nasty things with your database ("I'm a Django and I own 
your database, ha ha!").   

Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3e6c1b86-fbb8-466f-9fee-c54363bf8ef1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Marcin Nowak]

Hi all,

On Wednesday, April 13, 2016 at 9:58:18 PM UTC+2, Carl Meyer wrote:
>
> Hi James et al, 
>
> In general, I like the idea of adding a helper to Django to read 
> settings from the environment. I think this helper should be kept as 
> simple and non-magical as is reasonable. Thus: 
>
> This sounds great. Django should not do anything magical and just in one 
way. 
Just give the tool to the developers.

But I would like to say my thoughts about "settings" itself.
They were good (simple) before incuding nested dictionaries.
After switching to dicts the settings handling went harder way.

As a example I can tell about reconfiguring LOGGING for dev/live settings.
We're tuning them via direct changes like 
LOGGING['loggers']['some-logger']['level']='...'
which is very error prone. The flat settings are easier to maintain.

Nested settings are good for framework internals and should stay in that 
form, 
but maybe solution for end users and ENV integration is flattening settings 
again, at the high-level?

Let's imagine something like INI file which is parsed somehow and converted 
to internal settings object.
The INI file has a shallow structure (options grouped in sections), 
can be easily extended / overrided / included (look at Buidout as an 
example),
and ENV keys will be related to flat INI instead of nested settings.py.

Let's consider an example:

[databases]
default.url = postgres+psycopg2://user:pwd@localhost:5432/exampledb
default.engine = django.db.backends.postgres_psycopg2


which can be simply reconfigured by ENV vars like:

DJANGO_DATABASES_DEFAULT_URL="postgres+psycopg2://user:pwd@localhost:5432/exampledb"
DJANGO_DATABASES_DEFAULT_ENGINE="django.db.backends.postgres_psycopg2"

OR add some preprocessing and flexibility:



[databases]
default.url = ${env:SOME-USER-KEY1}
default.engine = ${env:SOME-USER-KEY2}


[env]
SOME_USER_KEY1=a default value for default database URL
SOME_USER_KEY2=a default value for default database engine


where [env] section will by filled automatically by Django from os.environ.

This also gives a possibility to extend settings.ini via development.ini, 
where [databases] can be overridden with developer settings (without env 
access)


Thanks for reading and sorry for my English. I hope you understood the idea.
Marcin

 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6dbaaaca-8de7-49d3-a7b8-df011b94f147%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: __ne, #5763

[Marcin Nowak]

On Friday, November 20, 2015 at 8:37:02 PM UTC+1, Carsten Fuchs wrote:
>
> sorry if this is a stupid question, but after having read 
> https://code.djangoproject.com/ticket/5763 and the discussions linked 
> from it, why should __ne not be implemented?
>
>
Because Django ORM __ne SQLAlchemy. 

BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5cb90d5f-8683-4a34-9c15-f06bb23e6eee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: SQL ALchemy support on django

[Marcin Nowak]

On Tuesday, November 3, 2015 at 5:17:32 PM UTC+1, Asif Saifuddin wrote:
>
> I would like to create an experiemental repo on my github for 
> experiementing the sqla support to django orm, hence some useful resource 
> indicator would be great. sqla have core engine and orm on top of it, so 
> the idea way to start experiment should be based on core?
>
>
I would like to contribute. But I'm afraid about all things dependent on 
DjangoORM, like migrations,admin,(model) forms and so on.
I like Django, most of it`s features and simplicity, but not DB layer 
(incl. migrations), html-related forms nor system checks.

There are several ways I think:

   - fork of Django (a new project based on good things), with some 
   compatibility layer
   - brand new framework based on great Werkzeug and SqlAlchemy, with 
   good-things picked from Django
   - switching to Flask (there is SqlAlchemy ext)

The last is the simplest :)

Cheers,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/95130c0c-d113-4e7f-9c8d-56459fd796b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: should manage.py test run system checks?

[Marcin Nowak]

As I said some time ago - there should be a possibility to disable system 
checks (generally), and run them only when needed (i.e. by adding 
--system-check argument to mgmt command(s)).

Kind Regards,
Marcin

On Tuesday, October 20, 2015 at 12:02:31 PM UTC+2, Shai Berger wrote:
>
> Just brainstorming : by all of the above, it would seem that the best 
> default is to run checks unless specific tests are selected. Of course, 
> this could be perceived as inconsistent, and that would be a problem. 
>
> On 20 באוקטובר 2015 10:04:17 GMT+03:00, Aymeric Augustin <
> aymeric@polytechnique.org > wrote:
>>
>> 2015-10-20 2:48 GMT+02:00 Tim Graham :
>>
>>> me: I'm of the opinion that running the system checks as part of the 
>>> manage.py 
>>> test command should be opt-in (for example, by writing a test that 
>>> asserts the call_command('check') output is empty. For example, when 
>>> debugging a single test, it doesn't seem necessary to have the overhead of 
>>> running check. 
>>>
>> I share this opinion. Single-test run time is a very important use case, 
>> not only for TDD practitioners, but also for everyone who tries to fix a 
>> stubborn failing test.
>>
>>
>>
>> -- 
>> Aymeric.
>>
>>
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4abd21c6-5499-4a68-9ea6-2a662d622409%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django SQL templates

[Marcin Nowak]

On Thursday, August 27, 2015 at 9:43:40 PM UTC+2, Alexey Zankevich wrote:
>
> For such cases it's great to have DSL to build SQL queries, but it's a big 
> question how low- or high-level language should be.
>

Take a look at SQLAlchemy`s expressions API. The ORM is built top of it.
I think that Django` developers will continue extending expressions API 
(F(), Q(), Func(), and so on) to make query language more flexible. 

Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f782f5ef-2c83-4d60-b4a9-51a407f195f7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

On Thursday, August 27, 2015 at 3:09:06 PM UTC+2, Tim Graham wrote:
>
> I created a ticket and pull request for the silencing of error/critical 
> messages:
> https://code.djangoproject.com/ticket/25318#ticket
> https://github.com/django/django/pull/5197
>
>>
>>
Thanks, Tim!

Marcin.
 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9ecf68e4-35a7-483b-983e-c5ef9f9d3533%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

On Wednesday, August 26, 2015 at 9:08:16 AM UTC+2, Florian Apolloner wrote:
>
> Ups, not enough coffee yet, the admin.ready() just registers the checks, 
> it does not execute them, so order should not matter -- I shall close my 
> browser now and get some coffee :D
>

I don't know how exactly order of ready() calls is changing. After 
switching to SimpleAdminApp the order was changed. Why? I don't know, but 
it works. 

This disables admin autodiscovery though, I doubt that is what you want… 


I have a call for admin.autodiscover() in my urls, so there is no 
difference (?). I don't see any...

BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/96451ce3-e982-40b4-88e6-e2fb4f8528e9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

>
>
>> I think likely the root problem here is that your monkeypatch is in a 
>> module which is not imported when validation runs. If I were you, I'd 
>> make sure that your `admin.py` imports whatever module does the 
>> monkeypatching, and then I'd expect this error to go away. 
>>
>
> Yes, Django does not "know" about patched user in a validation stage.
> I will try to change it, but this is an example only.
>  
>

Thanks, Carl. This specific error can solved by changing 
'django.contrib.admin' to 'django.contrib.admin.apps.SimpleAdminConfig' in 
INSTALLED_APPS.

BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7c0cac6d-728e-401d-93a4-20ce5ffb8e03%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

On Tuesday, August 25, 2015 at 4:51:51 PM UTC+2, Carl Meyer wrote:
>
>
> I think likely the root problem here is that your monkeypatch is in a 
> module which is not imported when validation runs. If I were you, I'd 
> make sure that your `admin.py` imports whatever module does the 
> monkeypatching, and then I'd expect this error to go away. 
>

Yes, Django does not "know" about patched user in a validation stage.
I will try to change it, but this is an example only.
 
PS. I would like to avoid any monkey patches, hacks, and just omit 
obstacles. :)
Django is a good (the best?) web framework for rapid development, but 
sometimes is a part of a bigger stack. 
In that case some unnecessary Django`s limitations/forcings may be 
problematic to maintain whole projects.   

BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/94d8fbf8-7e0e-49d5-81e4-2d1372ac3fca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

On Tuesday, August 25, 2015 at 3:59:20 PM UTC+2, Aymeric Augustin wrote:
>
>
> There’s an obvious bug here: silenced checks are still printed out — even 
> though they don’t prevent Django from starting.
> I hadn’t understood that from your earlier messages. It shouldn’t be very 
> hard to fix.
>
>
Don't worry, it may come from my poor english skills.
 

> If you want to make sure we don’t lose track of this bug, you should file 
> a ticket.
>
>
Well, I'm not sure now, because Tim wrote that it was a design decision and 
it is well documented.
I would like to turn off system checker, live without it, and rely on my 
automated tests only. This should be simplest solution for uncommon cases, 
and should be simple to implement without design change.

Also please note that silencing specific errors is not the best solution. 
In the first case E116 may be untrue and may be silenced, but for other 
models this error may be true and shouldn't be silenced at all.
Silencing all E116 will give us messy information about state.


BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7c9a3567-ba85-4184-83ac-3bd6e9a99b69%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

Thanks for replies, Carl, Adam, Shai.

But I think you're worrying too much for too many things. You're trying to 
control everything, trying to replace automated tests with system check 
framework, trying to block "invalid" operations, etc, etc. This is bad way, 
IMO.
 
Let's see a small example how system check framework works:

1. I'm adding custom field to user (by patching User models, don't ask why 
- other ways does not work)

class MyAppConfig(AppConfig):
name = 'my app'

def ready(self):
from django.contrib.auth.models import User
from django.db import models

User.add_to_class('is_verified', models.BooleanField(default=False))

2. I'm adding new field to admin interface

@admin.register(User)
class MyUserAdmin(UserAdmin):
[...]
list_filter = ('is_superuser', 'is_active', 'is_verified')
[...]

3. Django raises an error:

ERRORS:
: (admin.E116) The value of 
'list_filter[2]' refers to 'is_verified', which does not refer to a Field.

Which is not true!

4. I'm silencing E116:

SILENCED_SYSTEM_CHECKS = ['admin.E116']

5. Django starts, admin works  (and new filter works also), but runserver 
still prints out a false message:

Performing system checks...

System check identified some issues:

ERRORS:
: (admin.E116) The value of 
'list_filter[2]' refers to 'is_verified', which does not refer to a Field.

I know that this is unbelievable, Carl, but the message *is displayed!*
And prints out untrue message at all...

Sorry, but this is not helpful...

BR,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7fef0c25-2706-4072-86d9-ca3e0dbc03c6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

Tim,

They aren't really silenced, but skipped. I would like to silence them for 
real. They are skipped, but still printed out.

Let's assume that I am a guy who wants to silence every system check. I 
need to list ALL errors and warnings, which is:
  * unhandy and it would require maintain a big list (especially after 
upgrading Django to newer version)
  * messages are still visible (I don't need them and I don't want to read 
them, and any other developer don't want to read them)
  * django bootstraping is still slower due to executing system checks.

As an old-and-experienced-guy I want to disable (completely) system checks 
on my responsibility. 
Just belive me - I know what I'm doing. And I don't want to hack Django to 
do any non-typical things.  

Why you can't add just one setting to bypass something which is currently 
built-in? Is that a really big problem?


On Friday, August 7, 2015 at 2:32:08 PM UTC+2, Tim Graham wrote:
>
> Could you elaborate on why disabling system checks entirely is better than 
> selectively disabling checks that are problematic for you? Is it really 
> never helpful? If there are more people clamoring for this feature, I think 
> I'd be more open to it.
>
> SILENCED_SYSTEM_CHECKS = ['admin.E108', 'admin.E304', 'fields.E305', 
> 'admin.E116', 'fields.E306', 'fields.W342', 'fields'W340']
>
> On Friday, August 7, 2015 at 4:35:37 AM UTC-4, Marcin Nowak wrote:
>>
>>
>>
>> On Thursday, August 6, 2015 at 5:36:09 PM UTC+2, Tim Graham wrote:
>>>
>>> Perhaps your monkeypatches should be moved to `AppConfig.ready()` 
>>> methods so they are applied before the system check framework is invoked? 
>>>
>>
>> I'll try that, thanks. To be honest I should remove monkeypatching, I 
>> don't like it, but I don't want do it now.
>>  
>>
>>>  
>>>
>> How many problematic checks are you running into?
>>>
>>
>> Above 60 overall.
>>
>> Can you describe them a bit more? If anything, SILENCED_SYSTEM_CHECKS = 
>>> ['*'] to disable checks seems like the best option that Josh proposed, but 
>>> I'd like more justification before we spend time doing that.
>>>
>>
>> E108, E304, E305, E116, E306, many W342, W340.
>> I'm using many mappings for read-only and they don't need to be valid as 
>> regular models.
>>
>> Cheers,
>> Marcin
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/558031ee-4ebd-4e39-89c6-4ddf84b0ea04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

Thanks for the snippet, Aymeric! I've found similar on Gist, but yours 
seems to be better. Thanks.

Marcin

On Wednesday, August 5, 2015 at 11:10:25 PM UTC+2, Aymeric Augustin wrote:
>
> Hi Marcin,
>
> Thanks for the clarification, that's helpful.
>
> The backwards compatibility wrapper looks like this in this case:
>
> @contextlib.contextmanager
> def commit_manually(using=None):
> transaction.set_autocommit(False, using)
> try:
> yield
> finally:
> transaction.set_autocommit(True, using)
>
> (typed on a phone and not tested)
>
> We recently decided against shipping compatibility wrappers with Django 
> because the interaction with the deprecation policy is messy. However, like 
> Tim said, we're trying to make it easier to have such wrappers.
>
> -- 
> Aymeric.
>
> Le 5 août 2015 à 15:59, Marcin Nowak <marcin@gmail.com > 
> a écrit :
>
> Hi Aymeric,
>
> This project is complex, as I said before, and works well with Django 1.4. 
> But we need to upgrade it to newer version because of lack of support, 
> security issues, and simplifcation of maintenance and growth (we're 
> currently using three different Django versions, from 1.4 to 1.7, and we've 
> decided to upgrade all subprojects to 1.8)
>
> commit_manually() is used within long-live processes running as a mgmt 
> commands, where is used multiprocessing too.
> Whole process is complicated and hard to maintain. I would like to avoid 
> logical / flow changes, whose are required to replace commit_manually() 
> with atomic().
> Second use case is a extract-transform-load subsytem, which also requires 
> precise development of queries, commits and rollbacks. I'm afraid that 
> upgrade of it may be also hard.
>
> I think that there are rare cases, when manual transaction handling is 
> required and should be available for punctilious developer.
> *BUT* I can't say that atomic() does not fit my needs. I know how it work 
> (I belive), but it may not cover current cases (implementation).
>
> I must try to do replacement or bring old behaviour of commit_manually() 
> with my custom wrapper. I'm trying to change code as less as possible.
>
> You're asking what could be done better.. Generally speaking - an API 
> compatibility layer. As a maintainer of big and old project I would like to 
> import "commit_manually" from special module delivered with Django, which 
> will guarantee stability and compatibility. This may be even undocumented. 
> Manual commit() and rollback() functions are provided in transaction module 
> (thanks so much for that!), so making compat layer should be possible, I 
> think.
>
> Kind Regards,
> Marcin
>
> On Wednesday, August 5, 2015 at 1:16:38 PM UTC+2, Aymeric Augustin wrote:
>>
>> On 5 août 2015, at 11:09, Marcin Nowak <marcin@gmail.com> wrote: 
>>
>> > I'm using commit_manually() frequently, but this decorator was 
>> removed... 
>>
>> Hi Marcin, 
>>
>> Sorry for derailing the discussion a bit, but could you clarify why you 
>> were 
>> using this API? I removed it after failing to find a use case where it 
>> would 
>> be preferrable to commit_on_success(). I’d love to know if I missed 
>> something 
>> in order to avoid making similar mistakes again. 
>>
>> Using commit_manually() properly required great care -- and a try/finally 
>> construct in practice -- to ensure that no code path escaped without a 
>> commit 
>> or a rollback. It also required a massive amount of boilerplate to avoid 
>> masking exceptions: 
>>
>> - 
>> https://groups.google.com/d/msg/django-developers/S3a5UAqAsmg/7C27UGl6G_0J 
>> - https://code.djangoproject.com/ticket/6623 
>>
>> If you replace it with transaction.atomic() and remove the boilerplate, 
>> you 
>> get the same behavior with ten times less code. 
>>
>> In case removing the boilerplate creates too much code churn, I 
>> documented a 
>> real drop-in replacement that doesn't have the drawbacks of 
>> commit_manually: 
>> https://docs.djangoproject.com/en/1.6/topics/db/transactions/#id5 
>>
>> What could I have done better? 
>>
>> If you don't want any changes in Django -- your posts to this list are 
>> often 
>> after-the-fact complains about new features -- perhaps you could stick 
>> with 
>> whatever version of Django you're currently running? 
>>
>> -- 
>> Aymeric. 
>>
>>
>>
>>
>> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django i

Re: How to disable system check framework?

[Marcin Nowak]

On Thursday, August 6, 2015 at 5:36:09 PM UTC+2, Tim Graham wrote:
>
> Perhaps your monkeypatches should be moved to `AppConfig.ready()` methods 
> so they are applied before the system check framework is invoked? 
>

I'll try that, thanks. To be honest I should remove monkeypatching, I don't 
like it, but I don't want do it now.
 

>  
>
How many problematic checks are you running into?
>

Above 60 overall.

Can you describe them a bit more? If anything, SILENCED_SYSTEM_CHECKS = 
> ['*'] to disable checks seems like the best option that Josh proposed, but 
> I'd like more justification before we spend time doing that.
>

E108, E304, E305, E116, E306, many W342, W340.
I'm using many mappings for read-only and they don't need to be valid as 
regular models.

Cheers,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f85884dc-8c08-4d04-a0fd-f55e0329c8e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [multidb] emit_pre_migrate_signal / emit_post_migrate_signal in Django 1.8

[Marcin Nowak]

On Wednesday, August 5, 2015 at 7:42:16 PM UTC+2, Tim Graham wrote:
>
> There is a check a few lines above the snippet you linked for 
> "allow_migrate_model('other', Permission)". Is that returning the correct 
> result? I don't think you can create permissions in the 'other' db without 
> content types since they are linked by a foreign key.
>
>
Probably you've got the point! I didn't pay enough attention to this method 
and didn't saw this check.
 

> p.s. "Is it a bug" questions should go to django-users, thanks!
>
>>
>>
Yes, sir! Just thought that there is a better place to ask about internals, 
but I was wrong I suppose.
 

Thanks, Tim!
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c8c2bac2-831d-46a1-a1dd-fb573974cf43%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[multidb] emit_pre_migrate_signal / emit_post_migrate_signal in Django 1.8

[Marcin Nowak]

Hi, it's me again.

I'm managing migrations myself without using builtin migrations.

My project is using many databases and I'm calling schema migration 
separately for each database, providing database alias as a command 
argument. 
I'm also emitting pre- and post- migrate signals.

The problem is with new Django (teste with 1.8.3). 

When updating schema for 'default' database (which contains all 
Django-related tables, like auth, contenttypes, and so on), evertyhing 
works well.

But when updating other databases (without auth, contenttypes, etc), and 
when emitting post migrate signal with database alias other than 'default', 
Django is reading all registered apps and calls (for example) 
"django.auth.management.create_permissions", even if Permission model (or 
ContentType) is routed to 'default' alias. As a result Django tries to 
insert data into tables whose does not exists in specified database.

I can bypass emitting signals for databases other than 'default', but the 
problem still remains in Django itself. 
I think that Django should use database routers to check whether model is 
managed/writeable.

Let's assume that we have two databases: 'default' an 'other'.
Signal is emitting with db alias 'other'.
Django calls 
https://github.com/django/django/blob/1.8.3/django/contrib/auth/management/__init__.py#L82
 
with alias 'other'.
Then tries to get ContentType using='other'. But the 'other' database has 
no such table.

Is that feature or bug?
Should I create contrib.* tables for each database?

Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8547aa2b-ae44-4cc7-b000-0683f4533789%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

No, there is not problem in this code only, but yeah - it is a big problem. 
Real problem is that newer versions of Django are pushing developers to do 
things by the only one way, which is good for blogs or cms, or maybe new 
and small projects. As the result it is getting harder to maintain complex 
project based on Django unless you stay with old-and-unsupported version. 
And I'm not talking about atomic(), which is good and simple.

> And defer upgrading to a proper transaction management for another few 
years?

No. Just provide old-style wrappers (at the python-level api compatibility) 
for old guys maintaining old projects ;)

I can say something similar about generics views based on CBV - function 
based wrappers can be built top of CBV and can be provided in django.compat 
or django.legacy package. Just suggesting.
 
Cheers,
Marcin

On Wednesday, August 5, 2015 at 5:28:39 PM UTC+2, Florian Apolloner wrote:
>
> On Wednesday, August 5, 2015 at 3:59:39 PM UTC+2, Marcin Nowak wrote:
>>
>> 
>>
> I must try to do replacement or bring old behaviour of commit_manually() 
>> with my custom wrapper. I'm trying to change code as less as possible.
>>
>
> To be honest the "issues" you outlined here seem to suggest a bigger 
> problem within your own code. I am aware that it is really hard and 
> annoying to change from old style transactions to the new ones in big 
> projects, but given the fact that atomic() just works and I have yet to see 
> a project which used old style transactions properly (see Aymerics 
> presentation on transactions), I think it is worth it.
>  
>
>> You're asking what could be done better.. Generally speaking - an API 
>> compatibility layer. As a maintainer of big and old project I would like to 
>> import "commit_manually" from special module delivered with Django, which 
>> will guarantee stability and compatibility.
>>
>
> And defer upgrading to a proper transaction management for another few 
> years? Seriously, if your project is to big to upgrade stick with 1.4 and 
> use a distribution which provides LTS support for it (but yes, ultimately 
> you will have to switch, but from what I hear Redhat is going to support 
> 1.4 for another while :D)=…
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c2fc9c5a-dc87-49e2-aed3-064a3e2135b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

It's ok. I think that the thread can be closed, except the suggestion of 
making compatibility layer, which is rather a kind of Django`s philosophy 
and may be continued in a new thread (if needed).



On Wednesday, August 5, 2015 at 4:08:53 PM UTC+2, Carlton wrote:
>
> Sorry to be a pain but, could this thread be continued (if it's going to 
> be) on Django Users?
>
> (sorry)
>
> —
>
>
>
> On Wed, Aug 5, 2015 at 3:59 PM, Marcin Nowak <marcin@gmail.com 
> > wrote:
>
>> Hi Aymeric,
>>
>> This project is complex, as I said before, and works well with Django 
>> 1.4. But we need to upgrade it to newer version because of lack of support, 
>> security issues, and simplifcation of maintenance and growth (we're 
>> currently using three different Django versions, from 1.4 to 1.7, and we've 
>> decided to upgrade all subprojects to 1.8)
>>
>> commit_manually() is used within long-live processes running as a mgmt 
>> commands, where is used multiprocessing too.
>> Whole process is complicated and hard to maintain. I would like to avoid 
>> logical / flow changes, whose are required to replace commit_manually() 
>> with atomic().
>> Second use case is a extract-transform-load subsytem, which also requires 
>> precise development of queries, commits and rollbacks. I'm afraid that 
>> upgrade of it may be also hard.
>>
>> I think that there are rare cases, when manual transaction handling is 
>> required and should be available for punctilious developer.
>> *BUT* I can't say that atomic() does not fit my needs. I know how it 
>> work (I belive), but it may not cover current cases (implementation).
>>
>> I must try to do replacement or bring old behaviour of commit_manually() 
>> with my custom wrapper. I'm trying to change code as less as possible.
>>
>> You're asking what could be done better.. Generally speaking - an API 
>> compatibility layer. As a maintainer of big and old project I would like to 
>> import "commit_manually" from special module delivered with Django, which 
>> will guarantee stability and compatibility. This may be even undocumented. 
>> Manual commit() and rollback() functions are provided in transaction module 
>> (thanks so much for that!), so making compat layer should be possible, I 
>> think.
>>
>> Kind Regards,
>> Marcin
>>
>> On Wednesday, August 5, 2015 at 1:16:38 PM UTC+2, Aymeric Augustin wrote:
>>>
>>> On 5 août 2015, at 11:09, Marcin Nowak <marcin@gmail.com> wrote: 
>>>
>>> > I'm using commit_manually() frequently, but this decorator was 
>>> removed... 
>>>
>>> Hi Marcin, 
>>>
>>> Sorry for derailing the discussion a bit, but could you clarify why you 
>>> were 
>>> using this API? I removed it after failing to find a use case where it 
>>> would 
>>> be preferrable to commit_on_success(). I’d love to know if I missed 
>>> something 
>>> in order to avoid making similar mistakes again. 
>>>
>>> Using commit_manually() properly required great care -- and a 
>>> try/finally 
>>> construct in practice -- to ensure that no code path escaped without a 
>>> commit 
>>> or a rollback. It also required a massive amount of boilerplate to avoid 
>>> masking exceptions: 
>>>
>>> - 
>>> https://groups.google.com/d/msg/django-developers/S3a5UAqAsmg/7C27UGl6G_0J 
>>> - https://code.djangoproject.com/ticket/6623 
>>>
>>> If you replace it with transaction.atomic() and remove the boilerplate, 
>>> you 
>>> get the same behavior with ten times less code. 
>>>
>>> In case removing the boilerplate creates too much code churn, I 
>>> documented a 
>>> real drop-in replacement that doesn't have the drawbacks of 
>>> commit_manually: 
>>> https://docs.djangoproject.com/en/1.6/topics/db/transactions/#id5 
>>>
>>> What could I have done better? 
>>>
>>> If you don't want any changes in Django -- your posts to this list are 
>>> often 
>>> after-the-fact complains about new features -- perhaps you could stick 
>>> with 
>>> whatever version of Django you're currently running? 
>>>
>>> -- 
>>> Aymeric. 
>>>
>>>
>>>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails fro

Re: How to disable system check framework?

[Marcin Nowak]

Hi Aymeric,

This project is complex, as I said before, and works well with Django 1.4. 
But we need to upgrade it to newer version because of lack of support, 
security issues, and simplifcation of maintenance and growth (we're 
currently using three different Django versions, from 1.4 to 1.7, and we've 
decided to upgrade all subprojects to 1.8)

commit_manually() is used within long-live processes running as a mgmt 
commands, where is used multiprocessing too.
Whole process is complicated and hard to maintain. I would like to avoid 
logical / flow changes, whose are required to replace commit_manually() 
with atomic().
Second use case is a extract-transform-load subsytem, which also requires 
precise development of queries, commits and rollbacks. I'm afraid that 
upgrade of it may be also hard.

I think that there are rare cases, when manual transaction handling is 
required and should be available for punctilious developer.
*BUT* I can't say that atomic() does not fit my needs. I know how it work 
(I belive), but it may not cover current cases (implementation).

I must try to do replacement or bring old behaviour of commit_manually() 
with my custom wrapper. I'm trying to change code as less as possible.

You're asking what could be done better.. Generally speaking - an API 
compatibility layer. As a maintainer of big and old project I would like to 
import "commit_manually" from special module delivered with Django, which 
will guarantee stability and compatibility. This may be even undocumented. 
Manual commit() and rollback() functions are provided in transaction module 
(thanks so much for that!), so making compat layer should be possible, I 
think.

Kind Regards,
Marcin

On Wednesday, August 5, 2015 at 1:16:38 PM UTC+2, Aymeric Augustin wrote:
>
> On 5 août 2015, at 11:09, Marcin Nowak <marcin@gmail.com > 
> wrote: 
>
> > I'm using commit_manually() frequently, but this decorator was 
> removed... 
>
> Hi Marcin, 
>
> Sorry for derailing the discussion a bit, but could you clarify why you 
> were 
> using this API? I removed it after failing to find a use case where it 
> would 
> be preferrable to commit_on_success(). I’d love to know if I missed 
> something 
> in order to avoid making similar mistakes again. 
>
> Using commit_manually() properly required great care -- and a try/finally 
> construct in practice -- to ensure that no code path escaped without a 
> commit 
> or a rollback. It also required a massive amount of boilerplate to avoid 
> masking exceptions: 
>
> - 
> https://groups.google.com/d/msg/django-developers/S3a5UAqAsmg/7C27UGl6G_0J 
> - https://code.djangoproject.com/ticket/6623 
>
> If you replace it with transaction.atomic() and remove the boilerplate, 
> you 
> get the same behavior with ten times less code. 
>
> In case removing the boilerplate creates too much code churn, I documented 
> a 
> real drop-in replacement that doesn't have the drawbacks of 
> commit_manually: 
> https://docs.djangoproject.com/en/1.6/topics/db/transactions/#id5 
>
> What could I have done better? 
>
> If you don't want any changes in Django -- your posts to this list are 
> often 
> after-the-fact complains about new features -- perhaps you could stick 
> with 
> whatever version of Django you're currently running? 
>
> -- 
> Aymeric. 
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3257c139-f52e-422f-8eb4-5a97771ca367%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

Thank, Josh.

I'm upgrading mature and complex project, which have some tricky parts 
implemented like a monkey-patched user model, which is applied after system 
chceck execution. So Django complains about missing fields in ModelAdmin, 
but patch works fine. There were some complains about doubled mappings to 
fields, but I've removed them. Generally speaking I don't need System 
Check, because I can find soultion myself other way.

Migrations are similar - they are too simple/limited for my needs. They 
operate on models, but I'm working mainly with database and I'm creating 
mappings for tables and views for easy querying. So I don't need builtin 
migrations feature.

I'm using commit_manually() frequently, but this decorator was removed... 

And so on...

That's why I feel that working with Django is harder. It's more strict, 
useful tools/helpers are missing, there is more features which should be 
bypassed/omitted. I feel that is harder to do simple things... Or maybe I'm 
getting old. ;)

Cheers,
Marcin


On Wednesday, August 5, 2015 at 3:08:26 AM UTC+2, Josh Smeaton wrote:
>
> Your "scare quotes" for "feature" are really disappointing. Especially 
> considering that the checks were hardcoded and couldn't be silenced prior 
> to the "system check framework".
>
> > Working with Django is getting harder with every new version.
>
> If you want to turn off new features or extension points, then I'd imagine 
> there'd be more work than for the regular user. Otherwise, I couldn't 
> disagree more.
>
> Anyhow, you want to turn off system checks for some reason. Here are some 
> potential ways forward:
>
> - Target the checks that are incorrectly spitting out errors or warnings, 
> and fix those.
> - Provide another setting "DISABLE_CHECKS" or something similar that will 
> prevent checks running altogether (probably not ideal, new settings are 
> hard to get into core, especially when the benefit here is still 
> questionable).
> - Provide a special value for the existing SILENCED_SYSTEM_CHECKS setting 
> that causes all checks to be silenced.
>
> You haven't really said much about the issue other than it bothers you. 
> The main purpose of checks are for development - so it doesn't have any 
> runtime or deployment time penalties. Perhaps if you took some time to 
> justify why users other than yourself might want to disable the check 
> framework, it'd be easier to start looking at solutions to problems.
>
> Cheers
>
>
> On Wednesday, 5 August 2015 01:16:21 UTC+10, Marcin Nowak wrote:
>>
>> Thanks, Carl. I know about silencing but I want to disable unwanted 
>> "feature". Working with Django is getting harder with every new version. 
>>
>>
>> On Tuesday, August 4, 2015 at 3:25:22 PM UTC+2, Carl Meyer wrote:
>>>
>>>
>>> > On Aug 4, 2015, at 9:18 AM, Marcin Nowak <marcin@gmail.com> 
>>> wrote: 
>>> > 
>>> > I need to upgrade project to Django 1.8 but the SystemCheck Framework 
>>> bothers me. 
>>> > It complains about thigs that should work, even if they are a little 
>>> tricky. 
>>> > 
>>> > I need to disable system checks designed for blogs and other simple 
>>> sites. How can I do that? 
>>>
>>> The third paragraph of the system checks documentation [1] links to [2]. 
>>>
>>> Also, this is a question about using Django, not developing Django, so 
>>> it belongs on the django-users mailing list, not here. 
>>>
>>> Carl 
>>>
>>> [1] https://docs.djangoproject.com/en/1.8/topics/checks/ 
>>> [2] 
>>> https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SILENCED_SYSTEM_CHECKS
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/47132c06-4617-4119-b8f0-b0f166df2103%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

Thanks, Carl. I know about silencing but I want to disable unwanted 
"feature". Working with Django is getting harder with every new version. 


On Tuesday, August 4, 2015 at 3:25:22 PM UTC+2, Carl Meyer wrote:
>
>
> > On Aug 4, 2015, at 9:18 AM, Marcin Nowak <marcin@gmail.com 
> > wrote: 
> > 
> > I need to upgrade project to Django 1.8 but the SystemCheck Framework 
> bothers me. 
> > It complains about thigs that should work, even if they are a little 
> tricky. 
> > 
> > I need to disable system checks designed for blogs and other simple 
> sites. How can I do that? 
>
> The third paragraph of the system checks documentation [1] links to [2]. 
>
> Also, this is a question about using Django, not developing Django, so it 
> belongs on the django-users mailing list, not here. 
>
> Carl 
>
> [1] https://docs.djangoproject.com/en/1.8/topics/checks/ 
> [2] 
> https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SILENCED_SYSTEM_CHECKS

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6bf13f68-a2ca-4c1d-95d7-314a10d95c10%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How to disable system check framework?

[Marcin Nowak]

Hi,

I need to upgrade project to Django 1.8 but the SystemCheck Framework 
bothers me.
It complains about thigs that should work, even if they are a little tricky.

I need to disable system checks designed for blogs and other simple sites. 
How can I do that?

Best Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/66320218-db2c-4861-afdd-0592b0ba3861%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why Django is making migrations for external apps?

[Marcin Nowak]

Sounds enough good. One important thing for me is that I have many of
external apps and few in project managed by me. Reconfiguring most of apps
just to tell Django "leave them alone" sounds like an overhead. That's why
I would like to tell Django "here are my apps, manage them please".

BR
Marcin

On 17 July 2015 at 18:30, Andrew Godwin  wrote:

>
>
> On Fri, Jul 17, 2015 at 11:19 AM, James Bennett 
> wrote:
>
>> One option for declaring an app "unmanaged" could be via AppConfig.
>> That's fairly clean, mirrors the way you can already set a model as
>> unmanaged in its Meta declaration, and allows for the end user of the app
>> to override by supplying an AppConfig which will generate migrations
>> (should they, for some use-case reason, want that).
>>
>
> Now this I like.
>
> Andrew
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/WgFacUFvgfs/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFwN1uq8bhij2WABvrRW7QPuMcCwVg6d7qKiuMREh2Udb8vNpA%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAB2m7Cw82JSu8o%3DLyCG%3DinggyYT7%2BNhUcFBJqBE_pYYUBJmqfg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why Django is making migrations for external apps?

[Marcin Nowak]

I would like to set something like MIGRATION_MANAGED_APPS = (...) in my
project`s settings.
Every not listed app should not be inspected and should be excluded when
calling `makemigrations` command.
Setting MIGRATION_MANAGED_APPS to None (default) should enable backward
compatibility.


On 17 July 2015 at 12:42, Shai Berger  wrote:

> On Thursday 16 July 2015 02:10:12 Andrew Godwin wrote:
> >
> > Also, yes, Django doesn't see unpacked .egg files as any different than
> > anything else (they're just directories after all) - we have no way of
> > telling which apps are yours and which are external, really.
>
> In my experience, most projects differentiate "our" apps from "external or
> 3rd
> party" apps based on directory structure -- "our" apps go in the main
> project
> directory (and version control) and "external" ones go into virtualenv
> and/or
> system folders.
>
> The issues of choices and other setting-dependent-model-attributes
> notwithstanding, can we solve the "external apps" issue by adding a
> parameter
> to the makemigrations command (and maybe even a setting) to tell it to add
> migrations only within a given directory?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAB2m7CxFSR8sg7kVxPYihrPrsrZx3sa9650k_mhfLoFgGNGgBA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why Django is making migrations for external apps?

[Marcin Nowak]

Please also review that 
case: 
https://github.com/pennersr/django-allauth/blob/master/allauth/socialaccount/models.py#L40

On Thursday, July 16, 2015 at 2:03:07 AM UTC+2, Tim Graham wrote:
>
> Andrew, do you think having the autodetector not "resolve" settings to 
> their values and instead include that as references in auto-generated 
> migrations is something we should try to implement?
>
> If so, let's suggest that solution on 
> https://code.djangoproject.com/ticket/24648
>
> On Wednesday, July 15, 2015 at 7:58:02 PM UTC-4, Andrew Godwin wrote:
>>
>> Hi Marcin,
>>
>> Having migrations per-app is kind of a key part of the way the design 
>> works, but as for the rest of it, you're free to just write migrations 
>> yourself and ignore makemigrations - the format is intended to be 
>> human-writable.
>>
>> We also deliberately separated the schema editing backends from the 
>> migration code itself, so you're free to develop your own migration 
>> solution that overrides some of the logic of how Django works without 
>> having to re-implement all of the SQL abstraction, but I will tell you that 
>> in my experience the solution we have now is the one that works the best 
>> for most people (South did indeed use to have a form of semi-manual 
>> picking).
>>
>> Dependencies also make things more complex - my attempts to try and 
>> communicate interdependent changes to human in textual/graphic form were 
>> unsuccessful, hence the current method of just doing everything for you as 
>> best we can.
>>
>> As for a short-term fix, Collin's idea of importing the choices into the 
>> migration to stop it detecting more changes sounds like a decent temporary 
>> fix.
>>
>> Andrew
>>
>> On Wed, Jul 15, 2015 at 4:46 PM, Marcin Nowak <marcin@gmail.com> 
>> wrote:
>>
>>> Thanks for clarification, Andrew. I understand it. 
>>>
>>> The real solution (for me) is not to do anything "automagically". In my 
>>> "dream" I see migrations whose can be "picked" manually (or semi-manually 
>>> via "pickmigrations ") into my project, whenever I want. No 
>>> autochecks, no automigration for anything, only bunch of helpers (i.e. 
>>> change-detector. picker). Real/valid/selected/picked migrations should be 
>>> "imported" as a project-wide changesets without splitting by apps (no 
>>> dependency problem, easier conflicts resolve), and migrations in reusable 
>>> apps should be provided as a resource(s) to pick/import. 
>>>
>>> I would like to spent some time while creating stable and solid 
>>> migrations in my project instead of searching workarounds for runtime 
>>> issues.
>>> But this means a big architecture change and I know that is improssible 
>>> to do for years...
>>>
>>> Thanks again.
>>> Marcin
>>>
>>>
>>> On Thursday, July 16, 2015 at 1:10:39 AM UTC+2, Andrew Godwin wrote:
>>>>
>>>> Hi Marcin,
>>>>
>>>> Django persists every single field attribute change, no matter what it 
>>>> is - this was a deliberate decision as we didn't want to declare certain 
>>>> attributes as "not important for schema" (for example, a subclass of 
>>>> ChoiceField might use an enum column type for the choices and so need that 
>>>> change).
>>>>
>>>> That said, this is definitely a common problem and I'm starting to 
>>>> think we should revisit it - specifically, have field classes themselves 
>>>> say which attributes are not worth making changes for (perhaps combined 
>>>> with greater control by fields over how their schema gets made). This 
>>>> isn't 
>>>> currently possible though.
>>>>
>>>> Also, yes, Django doesn't see unpacked .egg files as any different than 
>>>> anything else (they're just directories after all) - we have no way of 
>>>> telling which apps are yours and which are external, really. This should 
>>>> only happen if the external app already has a migrations directory, so the 
>>>> app in question should probably try and present a consistent choices list 
>>>> to the migration system in the meantime if you want to avoid this. There's 
>>>> no other real fix for this right now.
>>>>
>>>> Andrew
>>>>
>>>> On Wed, Jul 15, 2015 at 5:53 PM, Marcin Nowak <marcin@gmail.com> 
>>>> wrote

Re: Why Django is making migrations for external apps?

[Marcin Nowak]

Thanks for clarification, Andrew. I understand it. 

The real solution (for me) is not to do anything "automagically". In my 
"dream" I see migrations whose can be "picked" manually (or semi-manually 
via "pickmigrations ") into my project, whenever I want. No 
autochecks, no automigration for anything, only bunch of helpers (i.e. 
change-detector. picker). Real/valid/selected/picked migrations should be 
"imported" as a project-wide changesets without splitting by apps (no 
dependency problem, easier conflicts resolve), and migrations in reusable 
apps should be provided as a resource(s) to pick/import. 

I would like to spent some time while creating stable and solid migrations 
in my project instead of searching workarounds for runtime issues.
But this means a big architecture change and I know that is improssible to 
do for years...

Thanks again.
Marcin


On Thursday, July 16, 2015 at 1:10:39 AM UTC+2, Andrew Godwin wrote:
>
> Hi Marcin,
>
> Django persists every single field attribute change, no matter what it is 
> - this was a deliberate decision as we didn't want to declare certain 
> attributes as "not important for schema" (for example, a subclass of 
> ChoiceField might use an enum column type for the choices and so need that 
> change).
>
> That said, this is definitely a common problem and I'm starting to think 
> we should revisit it - specifically, have field classes themselves say 
> which attributes are not worth making changes for (perhaps combined with 
> greater control by fields over how their schema gets made). This isn't 
> currently possible though.
>
> Also, yes, Django doesn't see unpacked .egg files as any different than 
> anything else (they're just directories after all) - we have no way of 
> telling which apps are yours and which are external, really. This should 
> only happen if the external app already has a migrations directory, so the 
> app in question should probably try and present a consistent choices list 
> to the migration system in the meantime if you want to avoid this. There's 
> no other real fix for this right now.
>
> Andrew
>
> On Wed, Jul 15, 2015 at 5:53 PM, Marcin Nowak <marcin@gmail.com 
> > wrote:
>
>> Hi Andrew,
>>
>> I think I've found it. This reusable app uses a kind of dynamic choices 
>> (based on settings or somethin' else) and Django detetcs field definition 
>> change (choices attr).
>>
>> So the question is - should Django detect change of choices attribute 
>> even if database schema is not really altered?
>> And the original question modified - should Django generate migration 
>> file inside of an egg when runtime configuration or settings are changing?
>>
>> BR,
>> Marcin
>>
>>
>>
>> On Thursday, July 16, 2015 at 12:07:52 AM UTC+2, Andrew Godwin wrote:
>>>
>>> Hi Marcin,
>>>
>>> Django will only make migrations for external apps that already have 
>>> "migrations" directories and have model changes - it shouldn't make new 
>>> migrations unless you've actually changed the models in those reuseable 
>>> apps. Are you sure you're not specifying the names of those apps on the 
>>> makemigrations command line? Are you using something like --noinput?
>>>
>>> Andrew
>>>
>>> On Wed, Jul 15, 2015 at 3:16 PM, Marcin Nowak <marcin@gmail.com> 
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I'm working on a project which is based on Django`s internal migrations.
>>>> Sometimes when switching between branches (not sure for 100%), Django 
>>>> complains about changes in my models.
>>>>
>>>> When `makemigrations` command is executed Django generates migrations 
>>>> in external (reusable) app placed in egg. And this is *strange*, I think.
>>>> I don't want to modify any external app, and I don't want to depend on 
>>>> temporary migrations created only on my local machine.
>>>>
>>>> Why Django is creating migrations for external apps, especially placed 
>>>> in eggs? 
>>>> Migrations should be handled and created only for apps within the 
>>>> project, and read-only for others. 
>>>>
>>>> What do you think?
>>>> How I can workaround this issue?
>>>>
>>>> (Django 1.7.8)
>>>>
>>>>
>>>> Best Regards,
>>>> Marcin
>>>>
>>>> -- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "Django developers (

Re: Why Django is making migrations for external apps?

[Marcin Nowak]

Hi Andrew,

I think I've found it. This reusable app uses a kind of dynamic choices 
(based on settings or somethin' else) and Django detetcs field definition 
change (choices attr).

So the question is - should Django detect change of choices attribute even 
if database schema is not really altered?
And the original question modified - should Django generate migration file 
inside of an egg when runtime configuration or settings are changing?

BR,
Marcin



On Thursday, July 16, 2015 at 12:07:52 AM UTC+2, Andrew Godwin wrote:
>
> Hi Marcin,
>
> Django will only make migrations for external apps that already have 
> "migrations" directories and have model changes - it shouldn't make new 
> migrations unless you've actually changed the models in those reuseable 
> apps. Are you sure you're not specifying the names of those apps on the 
> makemigrations command line? Are you using something like --noinput?
>
> Andrew
>
> On Wed, Jul 15, 2015 at 3:16 PM, Marcin Nowak <marcin@gmail.com 
> > wrote:
>
>> Hello,
>>
>> I'm working on a project which is based on Django`s internal migrations.
>> Sometimes when switching between branches (not sure for 100%), Django 
>> complains about changes in my models.
>>
>> When `makemigrations` command is executed Django generates migrations in 
>> external (reusable) app placed in egg. And this is *strange*, I think.
>> I don't want to modify any external app, and I don't want to depend on 
>> temporary migrations created only on my local machine.
>>
>> Why Django is creating migrations for external apps, especially placed in 
>> eggs? 
>> Migrations should be handled and created only for apps within the 
>> project, and read-only for others. 
>>
>> What do you think?
>> How I can workaround this issue?
>>
>> (Django 1.7.8)
>>
>>
>> Best Regards,
>> Marcin
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/77b46084-ffa1-43a0-ae2d-206f868cbaa5%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/77b46084-ffa1-43a0-ae2d-206f868cbaa5%40googlegroups.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/81e00a02-95fa-451d-aa24-1e4bd592f511%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Why Django is making migrations for external apps?

[Marcin Nowak]

Hello,

I'm working on a project which is based on Django`s internal migrations.
Sometimes when switching between branches (not sure for 100%), Django 
complains about changes in my models.

When `makemigrations` command is executed Django generates migrations in 
external (reusable) app placed in egg. And this is *strange*, I think.
I don't want to modify any external app, and I don't want to depend on 
temporary migrations created only on my local machine.

Why Django is creating migrations for external apps, especially placed in 
eggs? 
Migrations should be handled and created only for apps within the project, 
and read-only for others. 

What do you think?
How I can workaround this issue?

(Django 1.7.8)


Best Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/77b46084-ffa1-43a0-ae2d-206f868cbaa5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

On Sunday, May 31, 2015 at 1:47:32 AM UTC+2, Tim Graham wrote:
>
> They were dropped as part of the removal of the old code that supported 
> syncing apps without migrations.
>

But you removed a feature, not just old code. 
In v1.8 (1.7 maybe?) this feature was broken (issue #24876) and it was 
completely removed in master (issue #24481).
This is not good.
 

>
> By the way, I suspect this could be implemented as a third-party app if 
> there is opposition to including it in Django itself. 
>
 
I know I can reimplement it myself, but core feature should be nearest to 
core source code, because it highly depends on internals.
This can be provided as a contrib app, for example. 

Why contrib.gis is supported for years? This app is used rarely. In 
contrast to gis - we generate/create SQLs every day. 
I don't know what you're doing with Django, but latests changes looks 
fearfull.. :(

/BR
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6ee9daaa-a619-4507-ae7f-0ed72d41a867%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

Tim,

I've just realised that there is no sql* commands in the newest Django.
Generating SQL from models was very useful. Why do you decided to drop that 
functionality? Because of builit-in migrations? 
Can you just bring them back? 

Thanks.
Marcin.


On Friday, May 29, 2015 at 7:26:26 PM UTC+2, Marcin Nowak wrote:
>
> Thanks, I didn't saw it.
>
> But I've found quick ad-hoc solution using monkey patching.
> Just include this snippet in project`s __init__:
>
> from django.core.management import sql
>
> def check_for_migrations_bypassed(*args, **kw):
> pass
>
> sql.check_for_migrations = check_for_migration_bypassed
>
> (Tested with v1.8)
>
> /BR
> Marcin
>
> On Friday, May 29, 2015 at 6:38:11 PM UTC+2, Tim Graham wrote:
>>
>> There's a ticket waiting someone to implement what has been discussed:
>>
>> https://code.djangoproject.com/ticket/24481
>>
>> On Friday, May 29, 2015 at 12:05:48 PM UTC-4, Marcin Nowak wrote:
>>>
>>>
>>>
>>> On Monday, March 30, 2015 at 1:58:02 AM UTC+2, Russell Keith-Magee wrote:
>>>>
>>>>
>>>>>
>>>>> *What is the new way to dump the sql schema of currently installed 
>>>>> django appz ?* It'd maybe be worth that I provide a doc patch to 
>>>>> inform users about it.
>>>>> *If there is none, is there an agreement to restore at least sqlall 
>>>>> and sqlclear ?*
>>>>>
>>>>  
>>>> No, there isn't :-)
>>>>
>>>
>>> We need old sql /sqlall behaviour. Currently we can't generate sql for 
>>> contrib apps. 
>>> Calling `sql auth` raises error:
>>>
>>> CommandError: App 'auth' has migrations. Only the sqlmigrate and 
>>> sqlflush commands can be used when an app has migrations.
>>>
>>> We're expecting SQL output instead.
>>>
>>> *We need to bypass migration mechanism for some specific projects*. 
>>> Please make life easier but not harder. Django is unusable from v1.7 for 
>>> some of us! 
>>>
>>> Or please give us hint how to achieve old behaviour.
>>>
>>> /BR
>>> Marcin
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ee82af74-ee72-4dbc-aa21-79070bcc4f79%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

Thanks, I didn't saw it.

But I've found quick ad-hoc solution using monkey patching.
Just include this snippet in project`s __init__:

from django.core.management import sql

def check_for_migrations_bypassed(*args, **kw):
pass

sql.check_for_migrations = check_for_migration_bypassed

(Tested with v1.8)

/BR
Marcin

On Friday, May 29, 2015 at 6:38:11 PM UTC+2, Tim Graham wrote:
>
> There's a ticket waiting someone to implement what has been discussed:
>
> https://code.djangoproject.com/ticket/24481
>
> On Friday, May 29, 2015 at 12:05:48 PM UTC-4, Marcin Nowak wrote:
>>
>>
>>
>> On Monday, March 30, 2015 at 1:58:02 AM UTC+2, Russell Keith-Magee wrote:
>>>
>>>
>>>>
>>>> *What is the new way to dump the sql schema of currently installed 
>>>> django appz ?* It'd maybe be worth that I provide a doc patch to 
>>>> inform users about it.
>>>> *If there is none, is there an agreement to restore at least sqlall and 
>>>> sqlclear ?*
>>>>
>>>  
>>> No, there isn't :-)
>>>
>>
>> We need old sql /sqlall behaviour. Currently we can't generate sql for 
>> contrib apps. 
>> Calling `sql auth` raises error:
>>
>> CommandError: App 'auth' has migrations. Only the sqlmigrate and sqlflush 
>> commands can be used when an app has migrations.
>>
>> We're expecting SQL output instead.
>>
>> *We need to bypass migration mechanism for some specific projects*. 
>> Please make life easier but not harder. Django is unusable from v1.7 for 
>> some of us! 
>>
>> Or please give us hint how to achieve old behaviour.
>>
>> /BR
>> Marcin
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fedc2fb6-e828-4670-bd40-d02f30b859cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Marcin Nowak]

On Monday, March 30, 2015 at 1:58:02 AM UTC+2, Russell Keith-Magee wrote:
>
>
>>
>> *What is the new way to dump the sql schema of currently installed django 
>> appz ?* It'd maybe be worth that I provide a doc patch to inform users 
>> about it.
>> *If there is none, is there an agreement to restore at least sqlall and 
>> sqlclear ?*
>>
>  
> No, there isn't :-)
>

We need old sql /sqlall behaviour. Currently we can't generate sql for 
contrib apps. 
Calling `sql auth` raises error:

CommandError: App 'auth' has migrations. Only the sqlmigrate and sqlflush 
commands can be used when an app has migrations.

We're expecting SQL output instead.

*We need to bypass migration mechanism for some specific projects*. 
Please make life easier but not harder. Django is unusable from v1.7 for 
some of us! 

Or please give us hint how to achieve old behaviour.

/BR
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/429b576a-d951-4e89-b36c-18062b647840%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Must a Django Database Support Migrations?

[Marcin Nowak]

Hi. 

Before v1.7 there were no migrations and Django was quite useful (for 
example with South as an 3rd party app). What happened then?

I think that migrations can be decoupled from core and you just don't want 
to do this for some unknown reason. 
I'll prepare a pull request if it will not be dropped because of some kind 
of ethos.

BR
Marcin

On Saturday, April 18, 2015 at 5:03:39 AM UTC+2, Markus Holtermann wrote:
>
> Hey Marvin, 
>
> Unfortunately this won't work as a few migration components are tightly 
> integrated into remaining parts of Django, e.g. the field deconstruction. 
>
> /Markus 
>
> On April 18, 2015 3:35:09 AM GMT+02:00, Marcin Nowak <marcin@gmail.com 
> > wrote: 
> >Hi. 
> >Just give people possibility to disable migrations functionality 
> >(remove it 
> >from core and put as a contrib app, if needed). 
> > 
> >/BR 
> >Marcin 
> > 
> > 
> >On Thursday, January 22, 2015 at 8:09:01 PM UTC+1, Andrew Godwin wrote: 
> >> 
> >> Aha! Then, I would suggest redesigning MigrationRecorder to only make 
> >the 
> >> table when an actual recording operation is done, and have it swallow 
> >the 
> >> table not existing as "no migrations applied" the rest of the time, 
> >if 
> >> people think that seems sensible. 
> >> 
> >> Andrew 
> >> 
> >> On Thu, Jan 22, 2015 at 10:44 AM, Markus Holtermann < 
> >> in...@markusholtermann.eu > wrote: 
> >> 
> >>> Hey, 
> >>> 
> >>> as soon as the MigrationRecorder is used there is a call to 
> >>> "ensure_schema" that forces the creation of the migration table. The 
> > 
> >>> runserver command (among others?) checks for unapplied migrations 
> >and thus 
> >>> creates the migration table. 
> >>> 
> >>> /Markus 
> >>> 
> >>> On Wednesday, January 21, 2015 at 12:36:47 AM UTC+1, Andrew Godwin 
> >wrote: 
> >>>> 
> >>>> Hi Ivan, 
> >>>> 
> >>>> I'm not sure what you're asking here - are you asking to have a way 
> >to 
> >>>> not have Django create the migrations recording table? I was under 
> >the 
> >>>> impression that it was only created when migrate was run (at least, 
> >that 
> >>>> was my original intention) so if you're managing your own schema 
> >just don't 
> >>>> run migrate. Is there something else that's not working right, or 
> >is that 
> >>>> being made too optimistically? 
> >>>> 
> >>>> Andrew 
> >>>> 
> >>>> On Tue, Jan 20, 2015 at 2:44 PM, Ivan VenOsdel <imi...@gmail.com> 
> >wrote: 
> >>>> 
> >>>>> From Andrew: "The only extra thing it would achieve is not having 
> >>>>> Django record migrations in the django_migrations table" 
> >>>>> 
> >>>>> The Django Users thread on how to keep this table from being 
> >created 
> >>>>> seemed to result in the 'solution' being either to stay with 1.6 
> >or comment 
> >>>>> the relevant lines in the Django code base. Is there really no 
> >other way? 
> >>>>> 
> >>>>> I love the new migrations facilities in Django 1.7 and was a 
> >>>>> contributor to the kickstarter but please keep in mind that many 
> >legacy DB 
> >>>>> projects are not avoiding migrations because they want to. IMO 
> >it's usually 
> >>>>> because they are forced to for some (usually political) reason 
> >where they 
> >>>>> don't have control over the schema. Forcing a perpetually empty 
> >>>>> django_migrations table pretty much locks out those users. 
> >>>>> 
> >>>>> I see that people are worried about encouraging the non-use of 
> >>>>> migrations but might I suggest following the Zen of Python and 
> >making 
> >>>>> migrations the "one obvious way to do it" and turning them off the 
> >not 
> >>>>> obvious way. 
> >>>>> 
> >>>>> Ivan 
> >>>>> 
> >>>>> On Wednesday, January 22, 2014 at 5:40:35 AM UTC-6, Andrew Godwin 
> >wrote: 
> >>>>>> 
> >>>>>> My thoughts in brief on this: 
> >>>>>> 
> >>>>>>  - Database backends don't support migrations - they support 

Re: Setting database default values in migrations (postgres)

[Marcin Nowak]

Simply do not use Django builtin migrations. Consider using Liquibase 
(maybe with Liquimigrate wrapper for Django) and feel freedom.

BR
Marcin

On Thursday, October 30, 2014 at 4:47:18 PM UTC+1, Peter Coles wrote:
>
> Hey all,
>
> I'm interested in getting database migrations to support setting 
> database-level default values. I've heard talk of this being a hotly 
> contested issue in the past, but I haven't been able to find the actual 
> conversations, so far this 
> 
>  
> is the best thread I've found and I've already asked about it in 
> django-users 
> .
>
> Instead of just asking for this feature, I've gone ahead and created a new 
> db backend that is almost exactly 
> the `django.db.backends.postgresql_psycopg2` module, but with two key code 
> blocks removed (that appear to be the logic to remove default values from 
> the database).
>
> Here's the repo: https://github.com/ringly/django-postgres-dbdefaults
>
> and here it is in pypi: 
> https://pypi.python.org/pypi/django-postgres-dbdefaults/0.0.1
>
> I'd love to hear feedback/thoughts/concerns.
>
> -Peter
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7601d0bc-bb97-48e6-95fb-a0228a81679c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Must a Django Database Support Migrations?

[Marcin Nowak]

Hi. 
Just give people possibility to disable migrations functionality (remove it 
from core and put as a contrib app, if needed).

/BR
Marcin


On Thursday, January 22, 2015 at 8:09:01 PM UTC+1, Andrew Godwin wrote:
>
> Aha! Then, I would suggest redesigning MigrationRecorder to only make the 
> table when an actual recording operation is done, and have it swallow the 
> table not existing as "no migrations applied" the rest of the time, if 
> people think that seems sensible.
>
> Andrew
>
> On Thu, Jan 22, 2015 at 10:44 AM, Markus Holtermann <
> in...@markusholtermann.eu > wrote:
>
>> Hey,
>>
>> as soon as the MigrationRecorder is used there is a call to 
>> "ensure_schema" that forces the creation of the migration table. The 
>> runserver command (among others?) checks for unapplied migrations and thus 
>> creates the migration table.
>>
>> /Markus
>>
>> On Wednesday, January 21, 2015 at 12:36:47 AM UTC+1, Andrew Godwin wrote:
>>>
>>> Hi Ivan,
>>>
>>> I'm not sure what you're asking here - are you asking to have a way to 
>>> not have Django create the migrations recording table? I was under the 
>>> impression that it was only created when migrate was run (at least, that 
>>> was my original intention) so if you're managing your own schema just don't 
>>> run migrate. Is there something else that's not working right, or is that 
>>> being made too optimistically?
>>>
>>> Andrew
>>>
>>> On Tue, Jan 20, 2015 at 2:44 PM, Ivan VenOsdel  wrote:
>>>
 From Andrew: "The only extra thing it would achieve is not having 
 Django record migrations in the django_migrations table"

 The Django Users thread on how to keep this table from being created 
 seemed to result in the 'solution' being either to stay with 1.6 or 
 comment 
 the relevant lines in the Django code base. Is there really no other way?

 I love the new migrations facilities in Django 1.7 and was a 
 contributor to the kickstarter but please keep in mind that many legacy DB 
 projects are not avoiding migrations because they want to. IMO it's 
 usually 
 because they are forced to for some (usually political) reason where they 
 don't have control over the schema. Forcing a perpetually empty 
 django_migrations table pretty much locks out those users.

 I see that people are worried about encouraging the non-use of 
 migrations but might I suggest following the Zen of Python and making 
 migrations the "one obvious way to do it" and turning them off the not 
 obvious way.

 Ivan

 On Wednesday, January 22, 2014 at 5:40:35 AM UTC-6, Andrew Godwin wrote:
>
> My thoughts in brief on this:
>
>  - Database backends don't support migrations - they support schema 
> alteration via SchemaEditor. This could be used separately from 
> migrations 
> if something wants it, and is meant to be an API on its own, so the 
> backend 
> is not the place to say if you want migrations or not.
>
>  - There is some merit to the ability to turn off migrations on a 
> per-backend basis, via a DATABASES setting, but bear in mind that routers 
> already let you do this (with the allow_migrate method). The only extra 
> thing it would achieve is not having Django record migrations in the 
> django_migrations table, but it also looks like it would be useful for 
> tests if you hadn't written that part yet.
>
>  - I feel like a DB backend should at least implement SchemaEditor 
> even if it returns NotImplementedError for most of the endpoints; even in 
> the weirdest relational system, you can still manage create/delete model 
> without too much difficulty.
>
>  - Bear in mind that the plan is to remove DatabaseCreation entirely 
> in favour of SchemaEditor in a few releases' time (and backends are more 
> than welcome to make DatabaseCreation use SchemaEditor behind the scenes 
> if 
> they want), so at that point if you don't implement it the backend is 
> only 
> ever good for querying, which to me feels like an incomplete backend.
>
>  - I'm not sure what the future of ./manage.py sqlall is, but it's 
> going to have to be ported to SchemaEditor in future anyway, so it's only 
> useful in the transition.
>
> Looking at the discussion, I think the best thing to do is:
>
>  - Make the schema and migrations test skip if 
> connection.schema_editor() raises a NotImplementedError (not too hard, we 
> can implement connection.has_schema_editor as a boolean to switch on)
>  - Provide some way to skip the "creating models" part of test setup, 
> so SchemaEditor is never needed during it
>
> I still think all the current third-party backends should be able to 
> provide a partial SchemaEditor implementation though, as at minimum they 
> all have the DatabaseCreation code in place to make new 

Re: Bypassing 1.7+ migrations

[Marcin Nowak]

On Friday, December 19, 2014 12:19:11 AM UTC+1, Shai Berger wrote:
>
>
> As far as I understood, Marcin wants to treat *all* models as unmanaged; 
> there 
> are no managed models to reference the unmanaged ones. 
>

I would like to disable/remove/kick-off built-in migration mechanism. :)

If may I suggest something - the migration subsystem should be delivered in 
the same way as other "contrib" apps. 
It should be an extension similar to South, but delivered as a part and of 
framework.
 

/Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d55fb217-2158-4b7a-8c5c-af4acb723d96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Bypassing 1.7+ migrations

[Marcin Nowak]

On Tuesday, December 16, 2014 1:49:53 PM UTC+1, Andrew Godwin wrote:
>
>
> Also bear in mind that, by doing your own schema management for the 
> contrib apps, you're kind of implicitly forking the schema; Django 1.8 will 
> ship with some migrations for these apps, and you're going to have to make 
> sure you manage those yourself and keep the database in sync with the code 
> (though of course the release notes will mention this kind of thing).
>

Yes, I'm aware of that, and I want to control every schema change (this is 
specific requirement for this project) 
 

>
> If you want to stub out migration operations, you can just make a custom 
> database backend that inherits from the one you use and define a new 
> SchemaEditor subclass with an execute() function that does nothing; that 
> should accomplish what you want. 
>

Thanks! I will try this approach. 
 

> Alternatively, you could wrap all migration operations into the State side 
> of SeparateDatabaseAndState, but that's a bit cumbersome.
>

For this I need to do some research. 
Thanks for tips!

 As far as I can tell your only issue is the runserver warning that you 
> have unmigrated changes, which there's currently no plan to be able to 
> disable.


I would like to bypass migrations to prevent accidental execution of 
`migrate` command, too.   
Generally speaking - I don't need builtin migrations (because executing 
them in my environment may be dangerous), but I need newer versions of 
Django.

I will try your tips.
 
Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/39e390e6-16a3-4591-8831-e73f281e615f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Bypassing 1.7+ migrations

[Marcin Nowak]

Hi Andrew,

Thanks for a reply. But as I've mentioned above I can't set managed=False 
on models delivered from external apps like django.contrib.auth, 
django.contrib.sessions, and many more. Forking them is not an option.

I think that bypassing migrations should be possible via some setting. 
That's why I wrote to developers forum directly.
Please think about that.

Marcin

On Tuesday, December 16, 2014 1:34:17 PM UTC+1, Andrew Godwin wrote:
>
> Hi Marcin,
>
> If you're using an external tool to manage schemas of models, just set 
> managed=False on the models and Django will stop trying to change their 
> schemas (including stopping making migrations for them).
>
> Andrew
>
> On Tue, Dec 16, 2014 at 1:11 AM, Marcin Nowak <marcin@gmail.com 
> > wrote:
>>
>> Hello!
>>
>> I'm using Django as a part of bigger stack where I'm using external tool 
>> to manage database migrations.
>> After migration to Django 1.7 I'm encouraged at every runserver command 
>> to execute some migrations, which may blow my databases.
>>
>> I want to bypass Django migrations to prevent any schema modification. I 
>> can bypass migrate*  commands by overriding them in my custom app, but this 
>> is a little tricky and does not cover all cases. Using managed=False is not 
>> a solution, because none of reusable apps have models defined as unmanaged 
>> (same for django.contrib, especially auth and sessions).
>> That's why I need to bypass migrations globally.
>>
>> Have you ever considered possibility to allow disabling it? Maybe via 
>> some setting to use kind a DummyMigrationExecutor instead of 
>> MigrationExecutor? 
>>
>> Cheers,
>> Marcin 
>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/a27293a1-4212-4a2a-ae44-4720fc674162%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/a27293a1-4212-4a2a-ae44-4720fc674162%40googlegroups.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d8226a81-9dfe-4019-94ce-98beb6e1bf04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Bypassing 1.7+ migrations

[Marcin Nowak]

Hello!

I'm using Django as a part of bigger stack where I'm using external tool to 
manage database migrations.
After migration to Django 1.7 I'm encouraged at every runserver command to 
execute some migrations, which may blow my databases.

I want to bypass Django migrations to prevent any schema modification. I 
can bypass migrate*  commands by overriding them in my custom app, but this 
is a little tricky and does not cover all cases. Using managed=False is not 
a solution, because none of reusable apps have models defined as unmanaged 
(same for django.contrib, especially auth and sessions).
That's why I need to bypass migrations globally.

Have you ever considered possibility to allow disabling it? Maybe via some 
setting to use kind a DummyMigrationExecutor instead of MigrationExecutor? 

Cheers,
Marcin 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a27293a1-4212-4a2a-ae44-4720fc674162%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 1.7: inconsistent behaviour of the fail_silently option when using SMTP backend

[Marcin Nowak]

Ok, I'll do it tomorrow.

On Sunday, September 28, 2014 5:35:31 PM UTC+2, Florian Apolloner wrote:
>
> Hi Marcin,
>
> can you please open a ticket for this? If you are up to it, a patch with 
> tests would be superb!
>
> Thanks,
> Florian
>
> On Sunday, September 28, 2014 1:26:42 PM UTC+2, Marcin Nowak wrote:
>>
>> Hello,
>>
>> After sereral minutes while using new Django (1.7) in DEBUG mode, I've 
>> got enigmatic error on the screen: "A server error occurred. Please 
>> contact the administrator."
>> This message is shown when another exception is raised within 
>> django.core.handlers.base.handle_uncaught_exception.
>> What exception was raised inside uncaught exception handler? 
>> socket.error. Why? Django tried to send me an email using nonexistent smtp 
>> (localhost:25, the default). 
>>
>> AdminEmailHandler is sending emails with fail_silently set to True, but 
>> the django.core.mail.smtp backend does not respect this setting as 
>> expected. Any connection problem will generate "A server error 
>> occured[...]" screen instead of traceback or error 500 page. This issue was 
>> created by resolving https://code.djangoproject.com/ticket/21189 
>>
>> "Fail silently" mode with SMTP backend is not 100% silent anymore. I saw 
>> that docs were changed, but the behaviour is inconsistent now. As a 
>> developer I'm expecting to silence ANY email error, including connection 
>> errors (not only SMTP errors). As an application developer I have no idea 
>> which email backend will be used, so I can't wrap every "mail.send()" with 
>> additional try-except block catching all possible exceptions. When I need 
>> to silent sending email fail, the email backend should do it for me (as 
>> before v1.7).
>>
>> Line 
>> https://github.com/django/django/blob/stable/1.7.x/django/core/mail/backends/smtp.py#L61
>>  
>> should catch socket.error too, or should be reverted to previous state.
>>
>> Kind Regards,
>> Marcin
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4208cf1b-8a36-478a-9e67-d12211d8cc07%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Django 1.7: inconsistent behaviour of the fail_silently option when using SMTP backend

[Marcin Nowak]

Hello,

After sereral minutes while using new Django (1.7) in DEBUG mode, I've got 
enigmatic error on the screen: "A server error occurred. Please contact the 
administrator."
This message is shown when another exception is raised within 
django.core.handlers.base.handle_uncaught_exception.
What exception was raised inside uncaught exception handler? socket.error. 
Why? Django tried to send me an email using nonexistent smtp (localhost:25, 
the default). 

AdminEmailHandler is sending emails with fail_silently set to True, but the 
django.core.mail.smtp backend does not respect this setting as expected. 
Any connection problem will generate "A server error occured[...]" screen 
instead of traceback or error 500 page. This issue was created by resolving 
https://code.djangoproject.com/ticket/21189 

"Fail silently" mode with SMTP backend is not 100% silent anymore. I saw 
that docs were changed, but the behaviour is inconsistent now. As a 
developer I'm expecting to silence ANY email error, including connection 
errors (not only SMTP errors). As an application developer I have no idea 
which email backend will be used, so I can't wrap every "mail.send()" with 
additional try-except block catching all possible exceptions. When I need 
to silent sending email fail, the email backend should do it for me (as 
before v1.7).

Line 
https://github.com/django/django/blob/stable/1.7.x/django/core/mail/backends/smtp.py#L61
 
should catch socket.error too, or should be reverted to previous state.

Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/88a73d00-68d0-4406-8cee-c8a7d87f3d9b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

flush (sqlflush) command and multiple databases

[Marcin Nowak]

Hey!

I think I've found a bug in flush command, but I'm not sure (maybe I don't 
understand Django ethos?) so I'm writing here.

I have multiple databases (PostgreSQL) and multiple apps in my projects. 
I'm using Django v1.5.1. 
Models are routed by my specific database router, so some apps are 
connected to database A, some to db B, and so on...

I'm trying to flush all databases for tests (calling flush for each 
database separately) and every sqlflush fails. Why? Because 
django.core.management.sql.sql_flush is called with only_django with 
only_existing=True that produces truncate sql for really existing tables in 
specified database (via --database argument), which is ok, BUT next step is 
a resetting ALL sequences for ALL models fetched from 
connection.introspection.sequence_list(). And that SQL simply fails. 
 Generated SQL for database A contains sequences for ALL tables and ALL 
defined databases, which simply fails.

I've tried to filter tables in sequence_list(), which is not good solution, 
but enough to ensure. And guess what? - flush command passed without errors.

Have you ever found that issue? 
Does anyone is using multiple databases with postgres backend (probably 
there is no issue with mysql)?
Or maybe I'm doing something wrong?

Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7f52c7ef-013b-4390-bf55-a57ad96e1c9d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.