Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

Thanks Melvyn, good to know. I'll keep that in mind



On Thursday, August 31, 2017 at 9:19:57 AM UTC-6, Melvyn Sopacua wrote:
>
> There are two main principles to remember about Django migrations in early 
> development stage:
>
> 1) Make migrations often and coherent.
>Typically, if you change something about a model, make a migration.
>When you want to cover multiple models in one migration, there should be
>a good reason for it. If you can't think of it, don't.
>
> 2) When the design is flawed, don't add to the problem by creating another
>migration. Instead roll back, remove the migration file(s), alter the
>design and make a new migration.
>
> The second principle is often overlooked and by applying the first, the
> "damage" is minimized.
>
> Rolling back is done by adding a migration number to the `migrate` 
> management
> command and it will roll back to the point including that migration.
>
> Once you freeze the model design, feel free to squash some or all 
> migrations
> into one file if things got a little big.
>
>
> On Wed, Aug 30, 2017 at 12:16 AM, Alexander Joseph  > wrote:
>
>> Thanks James,
>> I'm actually starting over from scratch instead of actually refactoring 
>> so the accounts app is the only real app I have so far. I actually did 
>> delete the database and just apply migrations to the newly created database 
>> and it worked ok. I'd like to get better at fixing these migration problems 
>> though since I'll probably run into a lot of them and wont be able to 
>> delete the database once I have data.
>>
>> Do you think it would have been better to run migrations for the specific 
>> accounts app? I thought it wouldnt matter much since accounts is my only 
>> app I have so far.
>>
>> My new structure so far looks like this
>>
>> business_management/
>> business_management/
>> accounts/
>> migrations/
>> __init__.py
>> admin.py
>> forms.py
>> models.py
>> tests.py
>> views.py
>> static/
>> templates/
>> config/
>> settings/
>> __init__.py
>> base.py
>> local.py
>> production.py
>> __init__.py
>> docs/
>> (developer documentation)
>> requirements/
>> (requirements.txt files)
>> utility/
>> (shell scripts)
>> manage.py
>> .gitignore
>>
>>
>>
>>
>> On Tuesday, August 29, 2017 at 3:35:21 PM UTC-6, James Schneider wrote:
>>>
>>>
>>>
>>> On Tue, Aug 29, 2017 at 7:13 AM, Alexander Joseph <
>>> alexander...@gmail.com> wrote:
>>>
 I'm not specifying the app level, I'm just running "python manage.py 
 makemigrations --settings=config.settings.local"  and  "python 
 manage.py migrate --settings=config.settings.local"
 I'm not modifying the migrations files and I dont have an app with the 
 label of admin, thats just the built-in admin. Thanks

>>>
>>>
>>> I know you were doing a lot of refactoring. Did you rename the accounts 
>>> app after performing an initial migration (applying an old migration with 
>>> the old app name structure)? It sounds like your DB and your migration 
>>> files might be out of sync. If you don't have any production data, I would 
>>> recommend deleting and recreating the entire database, and deleting and 
>>> recreating your migrations so that all the migrations are run in the 
>>> correct order.
>>>
>>> If you do have data you want to keep, then very likely you now have a 
>>> ton of work to do writing custom migrations to sync everything back up 
>>> manually. 
>>>
>>> -James
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/0686084f-1d5a-4f61-823e-7d84c58f317c%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Melvyn Sopacua
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/7b2eba67-bb19-4ba0-9d3d-7277da32f8be%40googlegroups.com.
For more options, visit 

Re: A lot of Problems with Migrating (conceptual)

[Melvyn Sopacua]

There are two main principles to remember about Django migrations in early
development stage:

1) Make migrations often and coherent.
   Typically, if you change something about a model, make a migration.
   When you want to cover multiple models in one migration, there should be
   a good reason for it. If you can't think of it, don't.

2) When the design is flawed, don't add to the problem by creating another
   migration. Instead roll back, remove the migration file(s), alter the
   design and make a new migration.

The second principle is often overlooked and by applying the first, the
"damage" is minimized.

Rolling back is done by adding a migration number to the `migrate`
management
command and it will roll back to the point including that migration.

Once you freeze the model design, feel free to squash some or all migrations
into one file if things got a little big.


On Wed, Aug 30, 2017 at 12:16 AM, Alexander Joseph <
alexander.v.jos...@gmail.com> wrote:

> Thanks James,
> I'm actually starting over from scratch instead of actually refactoring so
> the accounts app is the only real app I have so far. I actually did delete
> the database and just apply migrations to the newly created database and it
> worked ok. I'd like to get better at fixing these migration problems though
> since I'll probably run into a lot of them and wont be able to delete the
> database once I have data.
>
> Do you think it would have been better to run migrations for the specific
> accounts app? I thought it wouldnt matter much since accounts is my only
> app I have so far.
>
> My new structure so far looks like this
>
> business_management/
> business_management/
> accounts/
> migrations/
> __init__.py
> admin.py
> forms.py
> models.py
> tests.py
> views.py
> static/
> templates/
> config/
> settings/
> __init__.py
> base.py
> local.py
> production.py
> __init__.py
> docs/
> (developer documentation)
> requirements/
> (requirements.txt files)
> utility/
> (shell scripts)
> manage.py
> .gitignore
>
>
>
>
> On Tuesday, August 29, 2017 at 3:35:21 PM UTC-6, James Schneider wrote:
>>
>>
>>
>> On Tue, Aug 29, 2017 at 7:13 AM, Alexander Joseph > > wrote:
>>
>>> I'm not specifying the app level, I'm just running "python manage.py
>>> makemigrations --settings=config.settings.local"  and  "python
>>> manage.py migrate --settings=config.settings.local"
>>> I'm not modifying the migrations files and I dont have an app with the
>>> label of admin, thats just the built-in admin. Thanks
>>>
>>
>>
>> I know you were doing a lot of refactoring. Did you rename the accounts
>> app after performing an initial migration (applying an old migration with
>> the old app name structure)? It sounds like your DB and your migration
>> files might be out of sync. If you don't have any production data, I would
>> recommend deleting and recreating the entire database, and deleting and
>> recreating your migrations so that all the migrations are run in the
>> correct order.
>>
>> If you do have data you want to keep, then very likely you now have a ton
>> of work to do writing custom migrations to sync everything back up
>> manually.
>>
>> -James
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/0686084f-1d5a-4f61-823e-7d84c58f317c%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Melvyn Sopacua

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2Bgw1GVEqNVRDvZuUn_3o02ZvL_vZJctBqbVNdfFCcmfMwPMuw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

Thanks James,
I'm actually starting over from scratch instead of actually refactoring so 
the accounts app is the only real app I have so far. I actually did delete 
the database and just apply migrations to the newly created database and it 
worked ok. I'd like to get better at fixing these migration problems though 
since I'll probably run into a lot of them and wont be able to delete the 
database once I have data.

Do you think it would have been better to run migrations for the specific 
accounts app? I thought it wouldnt matter much since accounts is my only 
app I have so far.

My new structure so far looks like this

business_management/
business_management/
accounts/
migrations/
__init__.py
admin.py
forms.py
models.py
tests.py
views.py
static/
templates/
config/
settings/
__init__.py
base.py
local.py
production.py
__init__.py
docs/
(developer documentation)
requirements/
(requirements.txt files)
utility/
(shell scripts)
manage.py
.gitignore




On Tuesday, August 29, 2017 at 3:35:21 PM UTC-6, James Schneider wrote:
>
>
>
> On Tue, Aug 29, 2017 at 7:13 AM, Alexander Joseph  > wrote:
>
>> I'm not specifying the app level, I'm just running "python manage.py 
>> makemigrations --settings=config.settings.local"  and  "python manage.py 
>> migrate --settings=config.settings.local"
>> I'm not modifying the migrations files and I dont have an app with the 
>> label of admin, thats just the built-in admin. Thanks
>>
>
>
> I know you were doing a lot of refactoring. Did you rename the accounts 
> app after performing an initial migration (applying an old migration with 
> the old app name structure)? It sounds like your DB and your migration 
> files might be out of sync. If you don't have any production data, I would 
> recommend deleting and recreating the entire database, and deleting and 
> recreating your migrations so that all the migrations are run in the 
> correct order.
>
> If you do have data you want to keep, then very likely you now have a ton 
> of work to do writing custom migrations to sync everything back up 
> manually. 
>
> -James
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/0686084f-1d5a-4f61-823e-7d84c58f317c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[James Schneider]

On Tue, Aug 29, 2017 at 7:13 AM, Alexander Joseph <
alexander.v.jos...@gmail.com> wrote:

> I'm not specifying the app level, I'm just running "python manage.py
> makemigrations --settings=config.settings.local"  and  "python manage.py
> migrate --settings=config.settings.local"
> I'm not modifying the migrations files and I dont have an app with the
> label of admin, thats just the built-in admin. Thanks
>


I know you were doing a lot of refactoring. Did you rename the accounts app
after performing an initial migration (applying an old migration with the
old app name structure)? It sounds like your DB and your migration files
might be out of sync. If you don't have any production data, I would
recommend deleting and recreating the entire database, and deleting and
recreating your migrations so that all the migrations are run in the
correct order.

If you do have data you want to keep, then very likely you now have a ton
of work to do writing custom migrations to sync everything back up
manually.

-James

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2Be%2BciUQMcW4e-cqM7w7viK7s6H4oK_F78L8SxmkHJjSmquZzQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

I'm not specifying the app level, I'm just running "python manage.py
makemigrations --settings=config.settings.local"  and  "python manage.py
migrate --settings=config.settings.local"
I'm not modifying the migrations files and I dont have an app with the
label of admin, thats just the built-in admin. Thanks

On Tue, Aug 29, 2017 at 2:47 AM, James Schneider 
wrote:

>
> 
>
>   File "C:\Users\Alexander\Envs\business_management\lib\site-packag
> es\django\core\management\commands\migrate.py", line 86, in handle
> executor.loader.check_consistent_history(connection)
>   File "C:\Users\Alexander\Envs\business_management\lib\site-packag
> es\django\db\migrations\loader.py", line 298, in check_consistent_history
> connection.alias,
> django.db.migrations.exceptions.InconsistentMigrationHistory: Migration
> admin.0001_initial is applied before its dependency accounts.0001_initial
> on database 'default'.
>
>
>
> let me know if you need anymore info to help me out with this. Thanks in
> advance
>
>
>
> How are you generating your migration files?
>
> Are you specifying the app_label after the makemigrations command?
>
> Are you modifying the migration files after they've been generated?
>
> Do you have an app with a label of admin? Or is that a reference to the
> built-in admin?
>
> -James
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/django-users/kzNqB9lEIGA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/CA%2Be%2BciUKFGrv62awj%2BuuSSuVaS9uYqVs2%2Brv%
> 3DJPsaxvmJGNTfg%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Best Regards,

Alexander Joseph

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAGJrYjZjua%3DYrA70m_aodhDHrS2A-eJ%3DQ6LR_3oEK%3D9NKuD8JQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[James Schneider]

  File "C:\Users\Alexander\Envs\business_management\lib\site-packag
es\django\core\management\commands\migrate.py", line 86, in handle
executor.loader.check_consistent_history(connection)
  File "C:\Users\Alexander\Envs\business_management\lib\site-packag
es\django\db\migrations\loader.py", line 298, in check_consistent_history
connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration
admin.0001_initial is applied before its dependency accounts.0001_initial
on database 'default'.



let me know if you need anymore info to help me out with this. Thanks in
advance



How are you generating your migration files?

Are you specifying the app_label after the makemigrations command?

Are you modifying the migration files after they've been generated?

Do you have an app with a label of admin? Or is that a reference to the
built-in admin?

-James

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2Be%2BciUKFGrv62awj%2BuuSSuVaS9uYqVs2%2Brv%3DJPsaxvmJGNTfg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

OK, so I started over using postrgresql and so far I have one app named 
'accounts' - to hold user models, registration, etc. I tried to migrate and 
got the following error which looks a lot like some of the errors I was 
getting before..


(business_management) C:\Python36\Projects\business_management>python 
manage.py migrate --settings=config.settings.local
Traceback (most recent call last):
  File "manage.py", line 22, in 
execute_from_command_line(sys.argv)
  File 
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\__init__.py",
 
line 363, in execute_from_command_line
utility.execute()
  File 
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\__init__.py",
 
line 355, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
  File 
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\base.py",
 
line 283, in run_from_argv
self.execute(*args, **cmd_options)
  File 
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\base.py",
 
line 330, in execute
output = self.handle(*args, **options)
  File 
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\commands\migrate.py",
 
line 86, in handle
executor.loader.check_consistent_history(connection)
  File 
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\db\migrations\loader.py",
 
line 298, in check_consistent_history
connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration 
admin.0001_initial is applied before its dependency accounts.0001_initial 
on database 'default'.



let me know if you need anymore info to help me out with this. Thanks in 
advance


On Friday, August 25, 2017 at 12:25:00 PM UTC-6, Alexander Joseph wrote:
>
> Awesome, thanks James, thats exactly what I'm looking for. I'll try layout 
> 1 first as you suggest
>
>
>
> On Wednesday, August 23, 2017 at 7:49:05 PM UTC-6, James Schneider wrote:
>>
>>
>>
>> On Wed, Aug 23, 2017 at 4:40 PM, Alexander Joseph > > wrote:
>>
>>> One more question - is there a way to put apps in folders/sub-folders 
>>> instead of creating sub-apps/modules? I just want to keep things easier to 
>>> navigate on the development side. I will eventually have about 20 sub-apps 
>>> in the 'Engineering' app alone.
>>>
>>> Even if I could just group all the engineering sub-apps i have now under 
>>> an engineering folder without any further hierarchy that would help as 
>>> there will also be HR, financial apps, administration apps, etc.
>>>
>>> Thanks again
>>>
>>
>> Technically you can go as deep as you'd like. You can use the module 
>> strategy to emulate a Django 'app' without all the registration fuss. A 
>> couple abbreviated examples:
>>
>>
>> # 1. Group resources and references by object type
>> project/
>> engineering/
>> __init__.py
>> models/
>> __init__.py
>>  widgets.py
>>  gadgets.py
>>  product1.py
>>  product2.py
>> views/
>> __init__.py
>> product1.py
>> product2.py
>>  # etc.
>>
>> All of the models for engineering would be grouped in a single 'models' 
>> module. Imports would look like 'from engineering.models.widgets import 
>> WonderWidget'
>>
>> The engineering/models/__init__.py file would contain lines like 'from 
>> .widgets import *' for all of your files that contain models.
>>
>>
>> # 2. Group by business segment or workflow
>> project/
>> engineering/
>> __init__.py
>> models.py
>> product1/
>> __init__.py
>> models.py
>> views.py
>> urls.py
>> product2/
>> __init__.py
>> models.py
>> views.py
>> urls.py
>>  # etc.
>> 
>> In this case, you're replicating the structure of an 'app' without 
>> creating one by Django's definition. The engineering/models.py file would 
>> then contain imports like 'from .product1.models import *' in order to get 
>> the model auto-discovery to work correctly. 
>>
>> I'm under the impression that most developers use layout #1, but a large 
>> project might work better with #2 if there aren't a lot of 
>> cross-dependencies. 
>>
>> Note that using Python modules rather than real Django apps will also 
>> keep your settings.INSTALLED_APPS list minimized. With as many 'apps' as 
>> you were originally talking about, that list could be dozens or hundreds of 
>> items long.
>>
>> IMHO I would start with #1 and see how it works for you. It could be the 
>> generalized term 'product' that you're using, but to me, a product would 
>> not trigger a new 'app' for me, just extra model/view/url/etc. files, 
>> especially given how often products change or are added/removed. In that 
>> case, #2 might work better 

Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

Awesome, thanks James, thats exactly what I'm looking for. I'll try layout 
1 first as you suggest



On Wednesday, August 23, 2017 at 7:49:05 PM UTC-6, James Schneider wrote:
>
>
>
> On Wed, Aug 23, 2017 at 4:40 PM, Alexander Joseph  > wrote:
>
>> One more question - is there a way to put apps in folders/sub-folders 
>> instead of creating sub-apps/modules? I just want to keep things easier to 
>> navigate on the development side. I will eventually have about 20 sub-apps 
>> in the 'Engineering' app alone.
>>
>> Even if I could just group all the engineering sub-apps i have now under 
>> an engineering folder without any further hierarchy that would help as 
>> there will also be HR, financial apps, administration apps, etc.
>>
>> Thanks again
>>
>
> Technically you can go as deep as you'd like. You can use the module 
> strategy to emulate a Django 'app' without all the registration fuss. A 
> couple abbreviated examples:
>
>
> # 1. Group resources and references by object type
> project/
> engineering/
> __init__.py
> models/
> __init__.py
>  widgets.py
>  gadgets.py
>  product1.py
>  product2.py
> views/
> __init__.py
> product1.py
> product2.py
>  # etc.
>
> All of the models for engineering would be grouped in a single 'models' 
> module. Imports would look like 'from engineering.models.widgets import 
> WonderWidget'
>
> The engineering/models/__init__.py file would contain lines like 'from 
> .widgets import *' for all of your files that contain models.
>
>
> # 2. Group by business segment or workflow
> project/
> engineering/
> __init__.py
> models.py
> product1/
> __init__.py
> models.py
> views.py
> urls.py
> product2/
> __init__.py
> models.py
> views.py
> urls.py
>  # etc.
> 
> In this case, you're replicating the structure of an 'app' without 
> creating one by Django's definition. The engineering/models.py file would 
> then contain imports like 'from .product1.models import *' in order to get 
> the model auto-discovery to work correctly. 
>
> I'm under the impression that most developers use layout #1, but a large 
> project might work better with #2 if there aren't a lot of 
> cross-dependencies. 
>
> Note that using Python modules rather than real Django apps will also keep 
> your settings.INSTALLED_APPS list minimized. With as many 'apps' as you 
> were originally talking about, that list could be dozens or hundreds of 
> items long.
>
> IMHO I would start with #1 and see how it works for you. It could be the 
> generalized term 'product' that you're using, but to me, a product would 
> not trigger a new 'app' for me, just extra model/view/url/etc. files, 
> especially given how often products change or are added/removed. In that 
> case, #2 might work better since all of the related code is within a single 
> sub-folder (but all the references to that code aren't, so there is still 
> work to be done). YMMV
>
> -James
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/73339417-ce65-4ab1-8836-d521b39c62c7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[James Schneider]

On Wed, Aug 23, 2017 at 4:40 PM, Alexander Joseph <
alexander.v.jos...@gmail.com> wrote:

> One more question - is there a way to put apps in folders/sub-folders
> instead of creating sub-apps/modules? I just want to keep things easier to
> navigate on the development side. I will eventually have about 20 sub-apps
> in the 'Engineering' app alone.
>
> Even if I could just group all the engineering sub-apps i have now under
> an engineering folder without any further hierarchy that would help as
> there will also be HR, financial apps, administration apps, etc.
>
> Thanks again
>

Technically you can go as deep as you'd like. You can use the module
strategy to emulate a Django 'app' without all the registration fuss. A
couple abbreviated examples:


# 1. Group resources and references by object type
project/
engineering/
__init__.py
models/
__init__.py
 widgets.py
 gadgets.py
 product1.py
 product2.py
views/
__init__.py
product1.py
product2.py
 # etc.

All of the models for engineering would be grouped in a single 'models'
module. Imports would look like 'from engineering.models.widgets import
WonderWidget'

The engineering/models/__init__.py file would contain lines like 'from
.widgets import *' for all of your files that contain models.


# 2. Group by business segment or workflow
project/
engineering/
__init__.py
models.py
product1/
__init__.py
models.py
views.py
urls.py
product2/
__init__.py
models.py
views.py
urls.py
 # etc.

In this case, you're replicating the structure of an 'app' without creating
one by Django's definition. The engineering/models.py file would then
contain imports like 'from .product1.models import *' in order to get the
model auto-discovery to work correctly.

I'm under the impression that most developers use layout #1, but a large
project might work better with #2 if there aren't a lot of
cross-dependencies.

Note that using Python modules rather than real Django apps will also keep
your settings.INSTALLED_APPS list minimized. With as many 'apps' as you
were originally talking about, that list could be dozens or hundreds of
items long.

IMHO I would start with #1 and see how it works for you. It could be the
generalized term 'product' that you're using, but to me, a product would
not trigger a new 'app' for me, just extra model/view/url/etc. files,
especially given how often products change or are added/removed. In that
case, #2 might work better since all of the related code is within a single
sub-folder (but all the references to that code aren't, so there is still
work to be done). YMMV

-James

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2Be%2BciWr9GGYERPYDAYHvYdryRMq5NkWN3ws1zaKw8MxusoGJw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

One more question - is there a way to put apps in folders/sub-folders
instead of creating sub-apps/modules? I just want to keep things easier to
navigate on the development side. I will eventually have about 20 sub-apps
in the 'Engineering' app alone.

Even if I could just group all the engineering sub-apps i have now under an
engineering folder without any further hierarchy that would help as there
will also be HR, financial apps, administration apps, etc.

Thanks again



On Wed, Aug 23, 2017 at 5:28 PM, Alexander Joseph <
alexander.v.jos...@gmail.com> wrote:

> Thanks James, and thanks for your post on my other thread about structure.
> I agree I need to get more into conceptual introductory material. I think I
> will restructure my project and merge a lot of the apps/subapps I have now
> as you suggested. From what Tom said it sounds like that could be a lot of
> where my migration issues are coming from. I've seen a lot of dependency
> related errors when trying to run migrations.
>
> Thanks all for the ongoing advice. I'll try to keep this thread updated on
> how issues are resolved as I restructure my project according to your
> suggestions
>
> On Tue, Aug 22, 2017 at 3:57 PM, James Schneider 
> wrote:
>
>>
>>
>> On Tue, Aug 22, 2017 at 12:37 PM, Alexander Joseph <
>> alexander.v.jos...@gmail.com> wrote:
>>
>>> Thanks for all the advice.
>>>
>>> One more question - could project structure be causing issues with
>>> migrations? I'm working on a large project and many apps in my project have
>>> several layers of "sub-apps". My structure looks something like this...
>>>
>>
>> Possibly, especially if you're introducing any import loops. It's easy to
>> do. Trying to get the autodiscover mechanism for models to run through
>> nested apps can also be fun.
>>
>> 
>>
>> A few things about my structure - I read in "2 Scoops of Django" that, in
>>> general, if you have more than 5 models per model file then you should
>>> think about splitting the model up into different apps, rather than having
>>> long models files. And I structured it this way so that it would be a
>>> little easier to manage - instead of having all apps under the project
>>> folder, engineering apps would belong in the engineering folder, etc.
>>>
>>> Thanks again
>>>
>>>
>>
>> It depends. Your structure will likely change throughout the life of your
>> project. Start with what you have now and mold accordingly. Loose coupling
>> is key, though, to make it easier to move things around.
>>
>> Like I mentioned in your other thread, you need to redefine what you are
>> considering an 'app'. Your current definition is way too specific/narrow,
>> which is why you are ending up with so many. Almost certainly each product
>> does not need its own app. That's not to say you shouldn't break them
>> apart. I'm saying keep your 'apps' generalized (ie engineering, accounting,
>> etc.) and break up the models into importable Python modules with a
>> sensible structure. Using modules will alleviate much of the pain of the
>> app and model discovery process using the pattern I described in the other
>> thread. It also provides a mechanism for loose coupling, and keeps your
>> models in separate files/folders using any structure you'd like. Reducing
>> the number of apps also reduces the number of files present in your
>> project, since you only need a single apps.py for a larger app, rather than
>> managing 20 apps.py files for 20 smaller apps.
>>
>> 2SoD is an excellent resource, and in no way to do I claim to have a
>> modicum of the knowledge and experience that the authors do. I don't even
>> program for a living. I have a copy for 1.8. You will notice in the book
>> that they do not have separate apps for an ice cream cone vs. ice cream
>> sandwiches vs. toppings, even though these may be analogous to your
>> product_1, product_2, etc. It's hard to tell though, because I'm assuming
>> you generalized your structure, so I don't know if you are talking about
>> different ice cream-based treat types, or cars vs. rocket ships. The latter
>> would likely use separate apps because there would be little overlapping
>> workflows and logic, along with multiple models associated with each.
>>
>> I'd recommend looking in to some introductory material on database
>> normalization, not so much for the mechanics, but rather more conceptually.
>> The way you would break things apart for normalization often coincides
>> neatly with the way you should break apart apps (and builds your model list
>> for you), somewhere around the 2nd normal form. Again, your structure will
>> likely change as you go, so use the loose coupling via the __init__.py
>> module imports as I've shown, and that should help with the transition.
>>
>> -James
>>
>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Django users" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> 

Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

Thanks James, and thanks for your post on my other thread about structure.
I agree I need to get more into conceptual introductory material. I think I
will restructure my project and merge a lot of the apps/subapps I have now
as you suggested. From what Tom said it sounds like that could be a lot of
where my migration issues are coming from. I've seen a lot of dependency
related errors when trying to run migrations.

Thanks all for the ongoing advice. I'll try to keep this thread updated on
how issues are resolved as I restructure my project according to your
suggestions

On Tue, Aug 22, 2017 at 3:57 PM, James Schneider 
wrote:

>
>
> On Tue, Aug 22, 2017 at 12:37 PM, Alexander Joseph <
> alexander.v.jos...@gmail.com> wrote:
>
>> Thanks for all the advice.
>>
>> One more question - could project structure be causing issues with
>> migrations? I'm working on a large project and many apps in my project have
>> several layers of "sub-apps". My structure looks something like this...
>>
>
> Possibly, especially if you're introducing any import loops. It's easy to
> do. Trying to get the autodiscover mechanism for models to run through
> nested apps can also be fun.
>
> 
>
> A few things about my structure - I read in "2 Scoops of Django" that, in
>> general, if you have more than 5 models per model file then you should
>> think about splitting the model up into different apps, rather than having
>> long models files. And I structured it this way so that it would be a
>> little easier to manage - instead of having all apps under the project
>> folder, engineering apps would belong in the engineering folder, etc.
>>
>> Thanks again
>>
>>
>
> It depends. Your structure will likely change throughout the life of your
> project. Start with what you have now and mold accordingly. Loose coupling
> is key, though, to make it easier to move things around.
>
> Like I mentioned in your other thread, you need to redefine what you are
> considering an 'app'. Your current definition is way too specific/narrow,
> which is why you are ending up with so many. Almost certainly each product
> does not need its own app. That's not to say you shouldn't break them
> apart. I'm saying keep your 'apps' generalized (ie engineering, accounting,
> etc.) and break up the models into importable Python modules with a
> sensible structure. Using modules will alleviate much of the pain of the
> app and model discovery process using the pattern I described in the other
> thread. It also provides a mechanism for loose coupling, and keeps your
> models in separate files/folders using any structure you'd like. Reducing
> the number of apps also reduces the number of files present in your
> project, since you only need a single apps.py for a larger app, rather than
> managing 20 apps.py files for 20 smaller apps.
>
> 2SoD is an excellent resource, and in no way to do I claim to have a
> modicum of the knowledge and experience that the authors do. I don't even
> program for a living. I have a copy for 1.8. You will notice in the book
> that they do not have separate apps for an ice cream cone vs. ice cream
> sandwiches vs. toppings, even though these may be analogous to your
> product_1, product_2, etc. It's hard to tell though, because I'm assuming
> you generalized your structure, so I don't know if you are talking about
> different ice cream-based treat types, or cars vs. rocket ships. The latter
> would likely use separate apps because there would be little overlapping
> workflows and logic, along with multiple models associated with each.
>
> I'd recommend looking in to some introductory material on database
> normalization, not so much for the mechanics, but rather more conceptually.
> The way you would break things apart for normalization often coincides
> neatly with the way you should break apart apps (and builds your model list
> for you), somewhere around the 2nd normal form. Again, your structure will
> likely change as you go, so use the loose coupling via the __init__.py
> module imports as I've shown, and that should help with the transition.
>
> -James
>
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/django-users/kzNqB9lEIGA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/CA%2Be%2BciV7tZS8WJeFyn1p5MQrctM6pymp
> BPK2btm38FshX%3DSX1Q%40mail.gmail.com
> 
> .
>
> For more options, visit 

Re: A lot of Problems with Migrating (conceptual)

['Tom Evans' via Django users]

On Tue, Aug 22, 2017 at 8:37 PM, Alexander Joseph
 wrote:
> Thanks for all the advice.
>
> One more question - could project structure be causing issues with
> migrations? I'm working on a large project and many apps in my project have
> several layers of "sub-apps". My structure looks something like this...

This approach seems appealing, but personally I think it is a bad idea
because the ease of introducing dependencies between apps is extremely
high. Even when the apps are all tightly related to a particular
project (to the extent their package name is
"fooproject-some-function"), I think its better to simply develop each
one as a standalone app, separate from the other apps and not
requiring the other apps in order to test/develop.

With this approach, you will not get unnecessarily thin apps simply
because you cannot easily couple two apps together, and everything
within the app should have high cohesion. Remember (or discover :)
that in software engineering, high quality code is loosely coupled (it
doesn't depend on many external things) and highly cohesive (the
things within the module are related to each other).

I would also avoid the use of subapps. First off, Django only uses the
final part of the dotted import path as the app name -
'engineering.products.product1' has the same name as
'accounting.invoices.product1' (I know that doesn't exist, but...). I
have a (unsubstantiated) theory that your migration woes are down to
subapps.

Long model files are not good, but I'd also disagree with 2SoD that
getting to 5 models means a new app because of long files - what
ridiculousness! If your model file is getting long, replace it with a
model directory, each model in a separate module, and all of them
imported in the __init__. You should split your app when it is no
longer cohesive (or more accurately, when the level of cohesion is
upsetting and/or slows development). 2SoD is good, but its not a holy
text!

Cheers

Tom

PS

Make sure you have a sensible workflow for migrations - I'm assuming
you are using some sort of version control. While you are developing
your changes, you might generate a lot of temporary migrations. You do
not want to apply these migrations to live, so you should squash them
- either manually, using squashmigrations or by migrating the DB back
to before they existed, remove them and recreate them. If you do the
last one, remember to preserve any custom logic you have placed in the
migration.

At the end of the day, the migrations you commit are what will run on
your production database, so you should ensure they do exactly what
you are expecting them to do and are running in the correct order.
Particularly if you have a lot of small apps, getting the dependencies
right is essential.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFHbX1J-Oxam2oHB37ap3rY5Sio2%2BR0WGMKKpmPtT-5JMaGDHA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Antonis Christofides]

Hi,

> I read in "2 Scoops of Django" that, in general, if you have more than 5
> models per model file then you should think about splitting the model up into
> different apps, rather than having long models files.
I don't claim to be more experienced than the authors—on the contrary—and I have
learnt loads from this book. However I disagree with this assertion. Five models
is very few. In addition, I've found early splitting to usually be YAGNI
. It causes more
problems than it solves. Unless there is an obvious logical distinction of
functionality right from the start, put all your models in the same app. When it
grows and you see an obvious way to improve its structure, go on and refactor.
Don't "refactor" from the start before you even understand your app's structure.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com


On 2017-08-22 22:37, Alexander Joseph wrote:
> Thanks for all the advice.
>
> One more question - could project structure be causing issues with migrations?
> I'm working on a large project and many apps in my project have several layers
> of "sub-apps". My structure looks something like this...
>
>
> |
>
> my_project/
>     config/ 
>     docs/
>     accounting/
>         invoices/
>             __init__.py
>             admin.py
>             models.py
>             urls.py
>             views.py
>         purchase_orders/
>             __init__.py
>             admin.py
>             models.py
>             urls.py
>             views.py 
>         __init__.py
>         admin.py
>         models.py
>         urls.py
>         views.py
>     engineering/
>         products/
>             product1/
>                 sub_product1/
>                     __init__.py
>                     admin.py
>                     models.py
>                     urls.py
>                     views.py
>                 __init__.py
>                 admin.py
>                 models.py
>                 urls.py
>                 views.py
>             product2/
>                 __init__.py
>                 admin.py
>                 models.py
>                 urls.py
>                 views.py
>             __init__.py
>             admin.py
>             models.py
>             urls.py
>             views.py
>     requirements/
>     utility/
>
> |
>
> A few things about my structure - I read in "2 Scoops of Django" that, in
> general, if you have more than 5 models per model file then you should think
> about splitting the model up into different apps, rather than having long
> models files. And I structured it this way so that it would be a little easier
> to manage - instead of having all apps under the project folder, engineering
> apps would belong in the engineering folder, etc.
>
> Thanks again
>
>
>
>
> On Thursday, August 17, 2017 at 11:10:01 AM UTC-6, Alexander Joseph wrote:
>
> I'm pretty new to django and I've been having problems with
> makemigrations/migrate and I wonder if Im doing things right.
>
> I'm using MySQL backend and after reading in the documentation a little it
> sounds like Postgresql might make migrating more painless. Usually my
> problems stem from changing table fields, adding new tables/models, and
> migrating both on my development server and my production server
> (PythonAnywhere)
>
> More than once I've had to drop my database, delete all my migrations, and
> start over with initial migrations on the development server. This wont
> fly for long on production though of course .. once I actually have users
> and data.
>
> I'm wondering if I should switch to postgresql or if theres any further
> resources that you might know of that would help me out? Thanks
>
> -- 
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/d4cc32e5-c9c1-48d7-bb19-a6646800ce3d%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this 

Re: A lot of Problems with Migrating (conceptual)

[James Schneider]

On Tue, Aug 22, 2017 at 12:37 PM, Alexander Joseph <
alexander.v.jos...@gmail.com> wrote:

> Thanks for all the advice.
>
> One more question - could project structure be causing issues with
> migrations? I'm working on a large project and many apps in my project have
> several layers of "sub-apps". My structure looks something like this...
>

Possibly, especially if you're introducing any import loops. It's easy to
do. Trying to get the autodiscover mechanism for models to run through
nested apps can also be fun.



A few things about my structure - I read in "2 Scoops of Django" that, in
> general, if you have more than 5 models per model file then you should
> think about splitting the model up into different apps, rather than having
> long models files. And I structured it this way so that it would be a
> little easier to manage - instead of having all apps under the project
> folder, engineering apps would belong in the engineering folder, etc.
>
> Thanks again
>
>

It depends. Your structure will likely change throughout the life of your
project. Start with what you have now and mold accordingly. Loose coupling
is key, though, to make it easier to move things around.

Like I mentioned in your other thread, you need to redefine what you are
considering an 'app'. Your current definition is way too specific/narrow,
which is why you are ending up with so many. Almost certainly each product
does not need its own app. That's not to say you shouldn't break them
apart. I'm saying keep your 'apps' generalized (ie engineering, accounting,
etc.) and break up the models into importable Python modules with a
sensible structure. Using modules will alleviate much of the pain of the
app and model discovery process using the pattern I described in the other
thread. It also provides a mechanism for loose coupling, and keeps your
models in separate files/folders using any structure you'd like. Reducing
the number of apps also reduces the number of files present in your
project, since you only need a single apps.py for a larger app, rather than
managing 20 apps.py files for 20 smaller apps.

2SoD is an excellent resource, and in no way to do I claim to have a
modicum of the knowledge and experience that the authors do. I don't even
program for a living. I have a copy for 1.8. You will notice in the book
that they do not have separate apps for an ice cream cone vs. ice cream
sandwiches vs. toppings, even though these may be analogous to your
product_1, product_2, etc. It's hard to tell though, because I'm assuming
you generalized your structure, so I don't know if you are talking about
different ice cream-based treat types, or cars vs. rocket ships. The latter
would likely use separate apps because there would be little overlapping
workflows and logic, along with multiple models associated with each.

I'd recommend looking in to some introductory material on database
normalization, not so much for the mechanics, but rather more conceptually.
The way you would break things apart for normalization often coincides
neatly with the way you should break apart apps (and builds your model list
for you), somewhere around the 2nd normal form. Again, your structure will
likely change as you go, so use the loose coupling via the __init__.py
module imports as I've shown, and that should help with the transition.

-James

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2Be%2BciV7tZS8WJeFyn1p5MQrctM6pympBPK2btm38FshX%3DSX1Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

Thanks for all the advice.

One more question - could project structure be causing issues with 
migrations? I'm working on a large project and many apps in my project have 
several layers of "sub-apps". My structure looks something like this...



my_project/
config/ 
docs/
accounting/
invoices/
__init__.py
admin.py
models.py
urls.py
views.py
purchase_orders/
__init__.py
admin.py
models.py
urls.py
views.py 
__init__.py
admin.py
models.py
urls.py
views.py
engineering/
products/
product1/
sub_product1/
__init__.py
admin.py
models.py
urls.py
views.py
__init__.py
admin.py
models.py
urls.py
views.py
product2/
__init__.py
admin.py
models.py
urls.py
views.py
__init__.py
admin.py
models.py
urls.py
views.py
requirements/
utility/


A few things about my structure - I read in "2 Scoops of Django" that, in 
general, if you have more than 5 models per model file then you should 
think about splitting the model up into different apps, rather than having 
long models files. And I structured it this way so that it would be a 
little easier to manage - instead of having all apps under the project 
folder, engineering apps would belong in the engineering folder, etc.

Thanks again




On Thursday, August 17, 2017 at 11:10:01 AM UTC-6, Alexander Joseph wrote:
>
> I'm pretty new to django and I've been having problems with 
> makemigrations/migrate and I wonder if Im doing things right.
>
> I'm using MySQL backend and after reading in the documentation a little it 
> sounds like Postgresql might make migrating more painless. Usually my 
> problems stem from changing table fields, adding new tables/models, and 
> migrating both on my development server and my production server 
> (PythonAnywhere)
>
> More than once I've had to drop my database, delete all my migrations, and 
> start over with initial migrations on the development server. This wont fly 
> for long on production though of course .. once I actually have users and 
> data.
>
> I'm wondering if I should switch to postgresql or if theres any further 
> resources that you might know of that would help me out? Thanks
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/d4cc32e5-c9c1-48d7-bb19-a6646800ce3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[James Schneider]

# later in forms.py
category = forms.TypedChoiceField(required=False, choices=get_categories)

This has a similar effect to the ModelChoiceField, with the query only
running when the form is instantiated, but gives you the flexibility to use
whatever logic you'd like to come up with the choices for the field.


Forgot the doc link:


https://docs.djangoproject.com/en/1.11/ref/forms/fields/#django.forms.ChoiceField.choices

-James

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2Be%2BciVq8Dsxz2uw6NOJJcnAniDuiEY5qU6NKKDAYmOhWk%3DMVw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[James Schneider]

On Aug 18, 2017 3:14 PM, "Jim Illback"  wrote:

Here’s a “problem” with migrations that I’ve had. It only occurs on a first
time installation (like a first-time upload to Heroku, for example).

In forms.py, I have a field dependent upon a column in another table
(Category):
category = forms.TypedChoiceField(required=False, choices=[(category.pk,
category.name) for category in Category.objects.all()])


While the ModelChoiceField is more applicable to this issue, an alternative
for other scenarios is to use a callback function as the value for
choices=, for example:

#early in forms.py
def get_categories():
return [(category.pk, category.name) for category in
Category.objects.all()]

# later in forms.py
category = forms.TypedChoiceField(required=False, choices=get_categories)

This has a similar effect to the ModelChoiceField, with the query only
running when the form is instantiated, but gives you the flexibility to use
whatever logic you'd like to come up with the choices for the field.

-James

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2Be%2BciUAVuRFAc9Z6PvuNSh5p4jh_ZrPbvyPjWQfW2ccMPnTOg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Jim Illback]

Excellent - thanks much.

On Aug 18, 2017, at 4:02 PM, knbk 
> wrote:

There are various reasons why you should never execute queries at startup. One 
you've seen: it makes it impossible to migrate a new database. Other reasons 
include that the choices are cached and are only renewed when the server 
restarts, and that when running tests, the query is executed before the test 
database is set up and will run on the main database.

The solution in this case is to use a ModelChoiceField[1], i.e.:

category = forms.ModelChoiceField(queryset=Category.objects.all(), 
required=False)

This allows to pass in a queryset which is evaluated whenever the field is used 
to make sure the values are up-to-date. Since querysets are lazy, this won't 
execute any queries on startup.

[1] https://docs.djangoproject.com/en/1.11/ref/forms/fields/#modelchoicefield

On Saturday, August 19, 2017 at 12:14:48 AM UTC+2, subaru_87 wrote:
Here’s a “problem” with migrations that I’ve had. It only occurs on a first 
time installation (like a first-time upload to Heroku, for example).

In forms.py, I have a field dependent upon a column in another table (Category):
category = forms.TypedChoiceField(required=False, 
choices=[(category.pk, 
category.name) for category in Category.objects.all()])

Running makemigrations or migrate if there’s no pre-existing database with a 
table named “Category” gives an error saying as much. However, I’m expecting 
migrate to create the tables required. Therefore, it’s a catch-22.

Of course, the “easy” answer is to comment out the above code, add in:
 category = forms.CharField(max_length=12)
And run the initial migrations. Then, turn around and comment out that same 
code, uncomment the real code shown above, and run migrations again. It works, 
but it seems a bit awkward.

Has anyone else had this issue? Or am I doing something wrong?

Thanks much,
Jim Illback


On Aug 18, 2017, at 1:21 AM, James Bennett  
wrote:

On Thu, Aug 17, 2017 at 1:03 PM, Antonis Christofides 
 wrote:

Second, just to make things clear, the word "migration" has two meanings. The 
original meaning of migration is to switch to another software system (e.g. 
migrate from MySQL to PostgreSQL, or migrate a repository from subversion to 
git). In Django, the term "migration" means something else: to update the 
database to the new schema. In my opinion this terminology is wrong and 
confusing (apparently it comes from Ruby on Rails, but I'm not certain), and a 
better one would have been "dbupdate" or something, but since it's migration 
we'll keep it and you'll have to understand which meaning is meant in each case.

An important thing worth knowing is that Django's migration framework does 
*not* only track the database schema. It also tracks the state of the Python 
classes which define the models, including options which do not affect the 
underlying DB schema, and making such changes to a model will create the need 
for a migration.

This is absolutely necessary, though, because the migration framework generates 
a frozen snapshot of the state at each migration, so that you can access the 
ORM to do things like data updates via the RunPython migration operator. If you 
were to change, say, the default ordering of a model without also generating a 
migration to record that change (even though it has no effect on the database 
schema), you could run into trouble if you later removed a field referenced by 
the old ordering, since one of the intermediate ORM snapshots would attempt to 
order queries by a now-nonexistent column.

--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users...@googlegroups.com.
To post to this group, send email to djang...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
django-users+unsubscr...@googlegroups.com.
To post to this group, send email to 
django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 

Re: A lot of Problems with Migrating (conceptual)

[knbk]

There are various reasons why you should never execute queries at startup. 
One you've seen: it makes it impossible to migrate a new database. Other 
reasons include that the choices are cached and are only renewed when the 
server restarts, and that when running tests, the query is executed before 
the test database is set up and will run on the main database. 

The solution in this case is to use a ModelChoiceField[1], i.e.:

category = forms.ModelChoiceField(queryset=Category.objects.all(), 
required=False)

This allows to pass in a queryset which is evaluated whenever the field is 
used to make sure the values are up-to-date. Since querysets are lazy, this 
won't execute any queries on startup. 

[1] 
https://docs.djangoproject.com/en/1.11/ref/forms/fields/#modelchoicefield

On Saturday, August 19, 2017 at 12:14:48 AM UTC+2, subaru_87 wrote:
>
> Here’s a “problem” with migrations that I’ve had. It only occurs on a 
> first time installation (like a first-time upload to Heroku, for example).
>
> In forms.py, I have a field dependent upon a column in another table 
> (Category):
> category = forms.TypedChoiceField(required=False, choices=[(
> category.pk, category.name) for category in Category.objects.all()])
>
> Running makemigrations or migrate if there’s no pre-existing database with 
> a table named “Category” gives an error saying as much. However, I’m 
> expecting migrate to create the tables required. Therefore, it’s a 
> catch-22. 
>
> Of course, the “easy” answer is to comment out the above code, add in:
>  category = forms.CharField(max_length=12) 
> And run the initial migrations. Then, turn around and comment out that 
> same code, uncomment the real code shown above, and run migrations again. 
> It works, but it seems a bit awkward.
>
> Has anyone else had this issue? Or am I doing something wrong?
>
> Thanks much,
> Jim Illback
>
>
> On Aug 18, 2017, at 1:21 AM, James Bennett  > wrote:
>
> On Thu, Aug 17, 2017 at 1:03 PM, Antonis Christofides <
> ant...@djangodeployment.com > wrote:
>
>> Second, just to make things clear, the word "migration" has two meanings. 
>> The original meaning of migration is to switch to another software system 
>> (e.g. migrate from MySQL to PostgreSQL, or migrate a repository from 
>> subversion to git). In Django, the term "migration" means something else: 
>> to update the database to the new schema. In my opinion this terminology is 
>> wrong and confusing (apparently it comes from Ruby on Rails, but I'm not 
>> certain), and a better one would have been "dbupdate" or something, but 
>> since it's migration we'll keep it and you'll have to understand which 
>> meaning is meant in each case.
>>
>
> An important thing worth knowing is that Django's migration framework does 
> *not* only track the database schema. It also tracks the state of the 
> Python classes which define the models, including options which do not 
> affect the underlying DB schema, and making such changes to a model will 
> create the need for a migration.
>
> This is absolutely necessary, though, because the migration framework 
> generates a frozen snapshot of the state at each migration, so that you can 
> access the ORM to do things like data updates via the RunPython migration 
> operator. If you were to change, say, the default ordering of a model 
> without also generating a migration to record that change (even though it 
> has no effect on the database schema), you could run into trouble if you 
> later removed a field referenced by the old ordering, since one of the 
> intermediate ORM snapshots would attempt to order queries by a 
> now-nonexistent column.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users...@googlegroups.com .
> To post to this group, send email to djang...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/d878688d-00a3-452d-8cb8-956dcf41a52e%40googlegroups.com.
For more options, visit 

Re: A lot of Problems with Migrating (conceptual)

[Jim Illback]

Here’s a “problem” with migrations that I’ve had. It only occurs on a first 
time installation (like a first-time upload to Heroku, for example).

In forms.py, I have a field dependent upon a column in another table (Category):
category = forms.TypedChoiceField(required=False, choices=[(category.pk, 
category.name) for category in Category.objects.all()])

Running makemigrations or migrate if there’s no pre-existing database with a 
table named “Category” gives an error saying as much. However, I’m expecting 
migrate to create the tables required. Therefore, it’s a catch-22.

Of course, the “easy” answer is to comment out the above code, add in:
 category = forms.CharField(max_length=12)
And run the initial migrations. Then, turn around and comment out that same 
code, uncomment the real code shown above, and run migrations again. It works, 
but it seems a bit awkward.

Has anyone else had this issue? Or am I doing something wrong?

Thanks much,
Jim Illback


On Aug 18, 2017, at 1:21 AM, James Bennett 
> wrote:

On Thu, Aug 17, 2017 at 1:03 PM, Antonis Christofides 
> wrote:

Second, just to make things clear, the word "migration" has two meanings. The 
original meaning of migration is to switch to another software system (e.g. 
migrate from MySQL to PostgreSQL, or migrate a repository from subversion to 
git). In Django, the term "migration" means something else: to update the 
database to the new schema. In my opinion this terminology is wrong and 
confusing (apparently it comes from Ruby on Rails, but I'm not certain), and a 
better one would have been "dbupdate" or something, but since it's migration 
we'll keep it and you'll have to understand which meaning is meant in each case.

An important thing worth knowing is that Django's migration framework does 
*not* only track the database schema. It also tracks the state of the Python 
classes which define the models, including options which do not affect the 
underlying DB schema, and making such changes to a model will create the need 
for a migration.

This is absolutely necessary, though, because the migration framework generates 
a frozen snapshot of the state at each migration, so that you can access the 
ORM to do things like data updates via the RunPython migration operator. If you 
were to change, say, the default ordering of a model without also generating a 
migration to record that change (even though it has no effect on the database 
schema), you could run into trouble if you later removed a field referenced by 
the old ordering, since one of the intermediate ORM snapshots would attempt to 
order queries by a now-nonexistent column.

--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
django-users+unsubscr...@googlegroups.com.
To post to this group, send email to 
django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/A0879DE1-7EED-434F-8D71-3FFE1FB2755D%40hotmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[James Bennett]

On Thu, Aug 17, 2017 at 1:03 PM, Antonis Christofides <
anto...@djangodeployment.com> wrote:

> Second, just to make things clear, the word "migration" has two meanings.
> The original meaning of migration is to switch to another software system
> (e.g. migrate from MySQL to PostgreSQL, or migrate a repository from
> subversion to git). In Django, the term "migration" means something else:
> to update the database to the new schema. In my opinion this terminology is
> wrong and confusing (apparently it comes from Ruby on Rails, but I'm not
> certain), and a better one would have been "dbupdate" or something, but
> since it's migration we'll keep it and you'll have to understand which
> meaning is meant in each case.
>

An important thing worth knowing is that Django's migration framework does
*not* only track the database schema. It also tracks the state of the
Python classes which define the models, including options which do not
affect the underlying DB schema, and making such changes to a model will
create the need for a migration.

This is absolutely necessary, though, because the migration framework
generates a frozen snapshot of the state at each migration, so that you can
access the ORM to do things like data updates via the RunPython migration
operator. If you were to change, say, the default ordering of a model
without also generating a migration to record that change (even though it
has no effect on the database schema), you could run into trouble if you
later removed a field referenced by the old ordering, since one of the
intermediate ORM snapshots would attempt to order queries by a
now-nonexistent column.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[James Schneider]

On Aug 17, 2017 10:10 AM, "Alexander Joseph" 
wrote:

I'm pretty new to django and I've been having problems with
makemigrations/migrate and I wonder if Im doing things right.

I'm using MySQL backend and after reading in the documentation a little it
sounds like Postgresql might make migrating more painless. Usually my
problems stem from changing table fields, adding new tables/models, and
migrating both on my development server and my production server
(PythonAnywhere)

More than once I've had to drop my database, delete all my migrations, and
start over with initial migrations on the development server. This wont fly
for long on production though of course .. once I actually have users and
data.

I'm wondering if I should switch to postgresql or if theres any further
resources that you might know of that would help me out? Thanks


Switching to a different DBMS will probably not alleviate migration issues,
especially if your project is still in its infancy and your migrations are
all still auto-generated. The migration system was designed to abstract
away the differences between the supported DBMS' so that you as the
developer didn't need to worry about it. Ideally you would be able to
deploy the same set of migrations on any of the supported databases.

Many Django devs have a bias towards Postgres, but much effort is put
toward ensuring that MySQL is supported since that is the only option for
many developers on their hosting platform. Some operations require extra
work in the migrations, and are challenging to implement regardless of the
DB platform. This is likely what you are running into.

I highly doubt MySQL itself is the cause, but it's possible. I'd recommend
posting up some of the errors that you are getting so that the folks here
can help. Might be more beneficial than setting up Postgres and having the
same issues.

-James

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2Be%2BciXA-iNG5cf-q7P9T-cVCjsw44zTgbmBJFM_qBe2c0zdQw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

Thanks Antonis, and great post on production databases. I'll definately be 
switching to postgresql. I come from php/xamp and I liked the MySQL GUI 
with phpMyAdmin so I've gravitated towards MySQL since.



On Thursday, August 17, 2017 at 2:05:44 PM UTC-6, Antonis Christofides 
wrote:
>
> Hello,
>
> My opinion is that you should definitely switch to PostgreSQL, although 
> I'm not aware whether this would make migrating easier. I've written a 
> relatively 
> long post 
> 
>  
> about which database to choose in production.
>
> Second, just to make things clear, the word "migration" has two meanings. 
> The original meaning of migration is to switch to another software system 
> (e.g. migrate from MySQL to PostgreSQL, or migrate a repository from 
> subversion to git). In Django, the term "migration" means something else: 
> to update the database to the new schema. In my opinion this terminology is 
> wrong and confusing (apparently it comes from Ruby on Rails, but I'm not 
> certain), and a better one would have been "dbupdate" or something, but 
> since it's migration we'll keep it and you'll have to understand which 
> meaning is meant in each case.
>
> What your problem is cannot be understood with your general exposition, 
> however after you gain a little bit of experience with migrations you 
> shouldn't have any such problems. Migrations are tricky to understand but 
> once you do they work flawlessly. The next time you have a problem give us 
> some information with the exact error message so that we can explain what's 
> wrong.
>
> Regards,
>
> Antonis
>
> Antonis Christofideshttp://djangodeployment.com
>
> On 2017-08-17 20:10, Alexander Joseph wrote:
>
> I'm pretty new to django and I've been having problems with 
> makemigrations/migrate and I wonder if Im doing things right. 
>
> I'm using MySQL backend and after reading in the documentation a little it 
> sounds like Postgresql might make migrating more painless. Usually my 
> problems stem from changing table fields, adding new tables/models, and 
> migrating both on my development server and my production server 
> (PythonAnywhere)
>
> More than once I've had to drop my database, delete all my migrations, and 
> start over with initial migrations on the development server. This wont fly 
> for long on production though of course .. once I actually have users and 
> data.
>
> I'm wondering if I should switch to postgresql or if theres any further 
> resources that you might know of that would help me out? Thanks
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users...@googlegroups.com .
> To post to this group, send email to django...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/16d60ab0-51e8-42a3-8f42-f4753f0adda3%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ffe78757-1809-4b86-949c-a30e257163be%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: A lot of Problems with Migrating (conceptual)

[Antonis Christofides]

Hello,

My opinion is that you should definitely switch to PostgreSQL, although I'm not
aware whether this would make migrating easier. I've written a relatively long
post

about which database to choose in production.

Second, just to make things clear, the word "migration" has two meanings. The
original meaning of migration is to switch to another software system (e.g.
migrate from MySQL to PostgreSQL, or migrate a repository from subversion to
git). In Django, the term "migration" means something else: to update the
database to the new schema. In my opinion this terminology is wrong and
confusing (apparently it comes from Ruby on Rails, but I'm not certain), and a
better one would have been "dbupdate" or something, but since it's migration
we'll keep it and you'll have to understand which meaning is meant in each case.

What your problem is cannot be understood with your general exposition, however
after you gain a little bit of experience with migrations you shouldn't have any
such problems. Migrations are tricky to understand but once you do they work
flawlessly. The next time you have a problem give us some information with the
exact error message so that we can explain what's wrong.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

On 2017-08-17 20:10, Alexander Joseph wrote:
> I'm pretty new to django and I've been having problems with
> makemigrations/migrate and I wonder if Im doing things right.
>
> I'm using MySQL backend and after reading in the documentation a little it
> sounds like Postgresql might make migrating more painless. Usually my problems
> stem from changing table fields, adding new tables/models, and migrating both
> on my development server and my production server (PythonAnywhere)
>
> More than once I've had to drop my database, delete all my migrations, and
> start over with initial migrations on the development server. This wont fly
> for long on production though of course .. once I actually have users and 
> data.
>
> I'm wondering if I should switch to postgresql or if theres any further
> resources that you might know of that would help me out? Thanks
> -- 
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/16d60ab0-51e8-42a3-8f42-f4753f0adda3%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5c99b11c-31e0-08de-6921-71b016261dc8%40djangodeployment.com.
For more options, visit https://groups.google.com/d/optout.