Re: [openstack-dev] [neutron] how to create initial db migration script to sub-project

[Henry Gessau]

Moshe Levi <mosh...@mellanox.com> wrote:
> Hi Henry,
> 
> Thank for the reply. 
> 
> I tried to do the following with your commit [2]:

Note I just updated my patch for a minor problem.

> 1. I create models.py in networking_mlnx/db.

Nit: I would recommend creating all model under networking_mlnx/db/models/.

> 2. mysql -e "drop database neutron; create database neutron;"
> 3. neutron-db-manage --subproject=neutron  upgrade heads

You should upgrade all heads here (leave out the --subproject option).

> 4. neutron-db-manage --subproject=networking-mlnx  revision -m "test" --expend

Hopefully you spelled it correctly: --expand. :)

> 
> Now I don't see the errors as before, but the migration script that was 
> generated 
> Looks like [3] and doesn't contain the new table.

Yes, I can reproduce this problem. We need to set up a head.py file that
includes all mlnx models. I need to remember how to correctly set up this
inclusion and I will get back to you.

> 
> Am I doing it wrong?  
> 
> [3] - 
> from alembic import op
> import sqlalchemy as sa
> 
> 
> # revision identifiers, used by Alembic.
> revision = 'cbb661581082'
> down_revision = '65b6db113427b9_initial_contract'
> 
> 
> def upgrade():
> pass
> 
> -Original Message-
> From: Henry Gessau [mailto:hen...@gessau.net] 
> Sent: Sunday, July 24, 2016 9:21 PM
> To: OpenStack Development Mailing List (not for usage questions) 
> <openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [neutron] how to create initial db migration 
> script to sub-project
> 
> Hi Moshe,
> 
> It has been a while since a neutron sub-project initialized new alembic 
> migrations, so the devref is out of date, sorry. Let me help you get this 
> sorted out and then I can update the devref with the latest info.
> 
> First you need to create the initial migration for each branch (expand and 
> contract). This is done by submitting a patch -- the contents of which used 
> to be crafted by copying a similar patch from another sub-project. The 
> patches to copy from are now out of date, so I have created a patch [2] for 
> you and we can use this as the new "template patch for initial alembic 
> migrations."
> 
> Some more comments inline ...
> 
> Moshe Levi <mosh...@mellanox.com> wrote:
>> Hi,
>>
>> I am trying to create initial db for the networking-mlnx project. 
>> I am following this neutron alembic documentation [1].
>> I added a entrypoint to networking-mlnx setup.cfg 
>> neutron.db.alembic_migrations =
>> networking-mlnx = networking_mlnx.db.migration:alembic_migrations
>> I added model.py file to networking-mlnx/networking_mlnx/db
>> And I run python setup.py develop to regenerate the entrypoint
>>
>> I am running the following commands:
>> 1. mysql -e "drop database neutron; create database neutron;"
>> 2. neutron-db-manage --subproject=neutron  upgrade heads
> 
> So far so good.
> 
>> 3. neutron-db-manage --subproject=networking-mlnx  revision -m "test" 
>> --autogenerate
> 
> Because we now require split (expand and contract) branches, you must now 
> specify --expand or --contract *instead of* --autogenerate. I will update the 
> devref about this.
> 
>> I am getting the following errors:
>> INFO  [alembic.runtime.migration] Context impl MySQLImpl.
>>
>> INFO  [alembic.runtime.migration] Will assume non-transactional DDL. 
>>
>> INFO  [alembic.autogenerate.compare] Detected removed table 
>> u'alembic_version'  
>> INFO  [alembic.autogenerate.compare] Detected removed index 
>> 'idx_autoinc_vr_id' on 'ha_router_vrid_allocations' 
>>   Generating 
>> /.autodirect/mtrswgwork/moshele/openstack/networking-mlnx/networking_mlnx/db/migration/alembic_migrations/versions/120e7e350bb1_test.py
>>  ... done
>>   OK 
>>  
>>
>> Traceback (most recent call last):   
>>  
>>
>>   File "/usr/bin/neutron-db-manage", line 10, in 
>>  
>>
>> sys.exit(main()) 
>>  

Re: [openstack-dev] [neutron] how to create initial db migration script to sub-project

[Moshe Levi]

Hi Henry,

Thank for the reply. 

I tried to do the following with your commit [2]:
1. I create models.py in networking_mlnx/db.
2. mysql -e "drop database neutron; create database neutron;"
3. neutron-db-manage --subproject=neutron  upgrade heads
4. neutron-db-manage --subproject=networking-mlnx  revision -m "test" --expend

Now I don't see the errors as before, but the migration script that was 
generated 
Looks like [3] and doesn't contain the new table.

Am I doing it wrong?  

[3] - 
from alembic import op
import sqlalchemy as sa


# revision identifiers, used by Alembic.
revision = 'cbb661581082'
down_revision = '65b6db113427b9_initial_contract'


def upgrade():
pass

-Original Message-
From: Henry Gessau [mailto:hen...@gessau.net] 
Sent: Sunday, July 24, 2016 9:21 PM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Subject: Re: [openstack-dev] [neutron] how to create initial db migration 
script to sub-project

Hi Moshe,

It has been a while since a neutron sub-project initialized new alembic 
migrations, so the devref is out of date, sorry. Let me help you get this 
sorted out and then I can update the devref with the latest info.

First you need to create the initial migration for each branch (expand and 
contract). This is done by submitting a patch -- the contents of which used to 
be crafted by copying a similar patch from another sub-project. The patches to 
copy from are now out of date, so I have created a patch [2] for you and we can 
use this as the new "template patch for initial alembic migrations."

Some more comments inline ...

Moshe Levi <mosh...@mellanox.com> wrote:
> Hi,
> 
> I am trying to create initial db for the networking-mlnx project. 
> I am following this neutron alembic documentation [1].
> I added a entrypoint to networking-mlnx setup.cfg 
> neutron.db.alembic_migrations =
> networking-mlnx = networking_mlnx.db.migration:alembic_migrations
> I added model.py file to networking-mlnx/networking_mlnx/db
> And I run python setup.py develop to regenerate the entrypoint
> 
> I am running the following commands:
> 1. mysql -e "drop database neutron; create database neutron;"
> 2. neutron-db-manage --subproject=neutron  upgrade heads

So far so good.

> 3. neutron-db-manage --subproject=networking-mlnx  revision -m "test" 
> --autogenerate

Because we now require split (expand and contract) branches, you must now 
specify --expand or --contract *instead of* --autogenerate. I will update the 
devref about this.

> I am getting the following errors:
> INFO  [alembic.runtime.migration] Context impl MySQLImpl. 
>   
> INFO  [alembic.runtime.migration] Will assume non-transactional DDL.  
>   
> INFO  [alembic.autogenerate.compare] Detected removed table 
> u'alembic_version'  
> INFO  [alembic.autogenerate.compare] Detected removed index 
> 'idx_autoinc_vr_id' on 'ha_router_vrid_allocations' 
>   Generating 
> /.autodirect/mtrswgwork/moshele/openstack/networking-mlnx/networking_mlnx/db/migration/alembic_migrations/versions/120e7e350bb1_test.py
>  ... done
>   OK  
>   
>  
> Traceback (most recent call last):
>   
>  
>   File "/usr/bin/neutron-db-manage", line 10, in  
>   
>  
> sys.exit(main())  
>   
>  
>   File 
> "/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
>  line 689, in main
> return_val |= bool(CONF.command.func(config, CONF.command.name))  
>   
>  
>   File 
> "/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
>  line 275, in do_revision 
> update_head_files(config) 
>   
>  
>   File 
> "/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
>  line 398, in update_head_files   
&

Re: [openstack-dev] [neutron] how to create initial db migration script to sub-project

[Henry Gessau]

Hi Moshe,

It has been a while since a neutron sub-project initialized new alembic
migrations, so the devref is out of date, sorry. Let me help you get this
sorted out and then I can update the devref with the latest info.

First you need to create the initial migration for each branch (expand and
contract). This is done by submitting a patch -- the contents of which used to
be crafted by copying a similar patch from another sub-project. The patches to
copy from are now out of date, so I have created a patch [2] for you and we
can use this as the new "template patch for initial alembic migrations."

Some more comments inline ...

Moshe Levi  wrote:
> Hi,
> 
> I am trying to create initial db for the networking-mlnx project. 
> I am following this neutron alembic documentation [1].
> I added a entrypoint to networking-mlnx setup.cfg
> neutron.db.alembic_migrations =
> networking-mlnx = networking_mlnx.db.migration:alembic_migrations
> I added model.py file to networking-mlnx/networking_mlnx/db
> And I run python setup.py develop to regenerate the entrypoint 
> 
> I am running the following commands:
> 1. mysql -e "drop database neutron; create database neutron;"
> 2. neutron-db-manage --subproject=neutron  upgrade heads

So far so good.

> 3. neutron-db-manage --subproject=networking-mlnx  revision -m "test" 
> --autogenerate

Because we now require split (expand and contract) branches, you must now
specify --expand or --contract *instead of* --autogenerate. I will update the
devref about this.

> I am getting the following errors:
> INFO  [alembic.runtime.migration] Context impl MySQLImpl. 
>   
> INFO  [alembic.runtime.migration] Will assume non-transactional DDL.  
>   
> INFO  [alembic.autogenerate.compare] Detected removed table 
> u'alembic_version'  
> INFO  [alembic.autogenerate.compare] Detected removed index 
> 'idx_autoinc_vr_id' on 'ha_router_vrid_allocations' 
>   Generating 
> /.autodirect/mtrswgwork/moshele/openstack/networking-mlnx/networking_mlnx/db/migration/alembic_migrations/versions/120e7e350bb1_test.py
>  ... done
>   OK  
>   
>  
> Traceback (most recent call last):
>   
>  
>   File "/usr/bin/neutron-db-manage", line 10, in  
>   
>  
> sys.exit(main())  
>   
>  
>   File 
> "/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
>  line 689, in main
> return_val |= bool(CONF.command.func(config, CONF.command.name))  
>   
>  
>   File 
> "/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
>  line 275, in do_revision 
> update_head_files(config) 
>   
>  
>   File 
> "/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
>  line 398, in update_head_files   
> f.write(head_map[CONTRACT_BRANCH] + '\n') 
>   
>  
> KeyError: 'contract'
> 
> 
> And the migration script just dropping unrelated tables 

This should be fixed by env.py in [2].

> [moshele@r-dcs84 networking-mlnx]$ cat 
> /.autodirect/mtrswgwork/moshele/openstack/networking-mlnx/networking_mlnx/db/migration/alembic_migrations/versions/120e7e350bb1_test.py
> # revision identifiers, used by Alembic.
> revision = '120e7e350bb1'
> down_revision = None
> 
> from alembic import op
> import sqlalchemy as sa
> from sqlalchemy.dialects import mysql
> 
> def upgrade():
> ### commands auto generated by Alembic - please adjust! ###
> op.drop_table('alembic_version')
> op.drop_index('idx_autoinc_vr_id', 
> table_name='ha_router_vrid_allocations')
> ### end Alembic commands ###
> 
> 
> 
> It seem that the neutron alembic migration documentation [1] is out of date
> ( I don't see the --core_plugin flag in the neutron-db-manage man, but I do
> see the --subproject flag)
> 
> Does anyone know how to make it works? 

Yes, --core_plugin is defunct and I will remove it from the devref.

If you rebase on top of my patch [2] and run the revision 

[openstack-dev] [neutron] how to create initial db migration script to sub-project

[Moshe Levi]

Hi,

I am trying to create initial db for the networking-mlnx project. 
I am following this neutron alembic documentation [1].
I added a entrypoint to networking-mlnx setup.cfg
neutron.db.alembic_migrations =
networking-mlnx = networking_mlnx.db.migration:alembic_migrations
I added model.py file to networking-mlnx/networking_mlnx/db
And I run python setup.py develop to regenerate the entrypoint 

I am running the following commands:
1. mysql -e "drop database neutron; create database neutron;"
2. neutron-db-manage --subproject=neutron  upgrade heads
3. neutron-db-manage --subproject=networking-mlnx  revision -m "test" 
--autogenerate

I am getting the following errors:
INFO  [alembic.runtime.migration] Context impl MySQLImpl.   

INFO  [alembic.runtime.migration] Will assume non-transactional DDL.

INFO  [alembic.autogenerate.compare] Detected removed table u'alembic_version'  

INFO  [alembic.autogenerate.compare] Detected removed index 'idx_autoinc_vr_id' 
on 'ha_router_vrid_allocations' 
  Generating 
/.autodirect/mtrswgwork/moshele/openstack/networking-mlnx/networking_mlnx/db/migration/alembic_migrations/versions/120e7e350bb1_test.py
 ... done
  OK
 
Traceback (most recent call last):  
 
  File "/usr/bin/neutron-db-manage", line 10, in
 
sys.exit(main())
 
  File 
"/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
 line 689, in main
return_val |= bool(CONF.command.func(config, CONF.command.name))
 
  File 
"/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
 line 275, in do_revision 
update_head_files(config)   
 
  File 
"/.autodirect/mtrswgwork/moshele/openstack/neutron/neutron/db/migration/cli.py",
 line 398, in update_head_files   
f.write(head_map[CONTRACT_BRANCH] + '\n')   
 
KeyError: 'contract'


And the migration script just dropping unrelated tables 

[moshele@r-dcs84 networking-mlnx]$ cat 
/.autodirect/mtrswgwork/moshele/openstack/networking-mlnx/networking_mlnx/db/migration/alembic_migrations/versions/120e7e350bb1_test.py
# revision identifiers, used by Alembic.
revision = '120e7e350bb1'
down_revision = None

from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import mysql

def upgrade():
### commands auto generated by Alembic - please adjust! ###
op.drop_table('alembic_version')
op.drop_index('idx_autoinc_vr_id', table_name='ha_router_vrid_allocations')
### end Alembic commands ###



It seem that the neutron alembic migration documentation [1] is out of date ( I 
don't see the --core_plugin flag in the neutron-db-manage man, but I do see the 
--subproject flag)

Does anyone know how to make it works? 


[1] - http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev