Re: [openstack-dev] [Neutron] Run migrations for service plugins every time
Salvatore, Thanks for your answer. I was on holidays, so I clearly read your response just now. As I understand from your comments there is no reason for me to continue working on making migrations run unconditionally for service plugins. Making all migrations run unconditionally based on current state of database will be rather long process, because I'll have to find out which tables are in database for each plugin and then create migration that will recreate them, if they won't exist. While I was working on this change https://review.openstack.org/87935 I researched different opportunities how to check database state without breaking offline migrations. The only suggestion is to use op.execute() and SQL statement 'CREATE TABLE IF NOT EXIST'. The problem with that is that if table uses enum type it can't be done for PostgreSQL. I described this problem in earlier mail with topic 'Fix migration that breaks Grenade jobs' If we won't care about offline migrations this could be done by rerunning some migrations or recreating some tables, if running the whole migration is not reliable. Regards, Ann On Wed, Apr 30, 2014 at 6:03 PM, Salvatore Orlando sorla...@nicira.comwrote: Anna, It's good to see progress being made on this blueprint. I have some comments inline. Also, I would recommend keeping in mind the comments Mark had regarding migration generation and plugin configuration in hist post on the email thread I started. Salvatore On 30 April 2014 14:16, Anna Kamyshnikova akamyshnik...@mirantis.comwrote: Hi everyone! I'm working on blueprint https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations. This topic was discussed earlier by Salvatore in ML thread Migrations, service plugins and Grenade jobs. I'm researching how to make migrations for service plugins run unconditionally. In fact it is enough to change should_run method for migrations - to make it return True if this migration is for service plugin https://review.openstack.org/#/c/91323/1/neutron/db/migration/__init__.py . I think running migrations unconditionally for service plugins is an advancement but not yet a final solution. I would insist on the path you've been pursuing of running unconditionally all migrations. We should strive to solve the issues you found there so far It is almost working except for conflicts of VNPaaS service plugin with Brocade and Nuage plugins http://paste.openstack.org/show/77946/. 1) Migration for Brocade plugin fails, because 2c4af419145b_l3_support doesn't run (it adds necessary table 'routers') It can be fixed by adding Brocade plugin to list migration_for_plugins in 2c4af419145b_l3_support. 2) Migration for Nuage plugin fails, because e766b19a3bb_nuage_initial runs after 52ff27f7567a_support_for_vpnaas, and there is no necessary table 'routers'. It can be fixed by adding Nuage plugin to list migration_for_plugins in 2c4af419145b_l3_support and removing addition of these tables in e766b19a3bb_nuage_initial. I noticed that too in the past. However there are two aspects to this fix. The one you're mentioning is for fixing migrations in new deployments; on the other hand migrations should be fixed also for existing deployments. This kind of problem seems to me more concerning the work for removing schema auto-generation. Indeed the root problem here is probably that l3 schemas for these two plugins are being created only because of schema auto-generation. Also I researched opportunity to make all migrations run unconditionally, but this could not be done as there is going to be a lot of conflicts http://paste.openstack.org/show/77902/. Mostly this conflicts take place in initial migrations for core plugins as they create basical tables that was created before. For example, mlnx_initial create tables securitygroups, securitygroupportbindings, networkdhcpagentbindings, portbindingports that were already created in 3cb5d900c5de_securitygroups, 4692d074d587_agent_scheduler, 176a85fc7d79_add_portbindings_db migrations. Besides I was thinking about 'making migrations smart' - to make them check whether all necessary migrations has been run or not, but the main problem about that is we need to store information about applied migrations, so this should have been planned from the very beginning. I agree that just changing migrations to run unconditionally is not a viable solution. Rather than changing the existing migration path, I was thinking more about adding corrective unconditional migrations to make the database state the same regardless of the plugin configuration. The difficult part here is that these migrations would need to be smart enough to understand whether a previous migration was executed or skipped; this might also break offline migrations (but perhaps this might be tolerable). I look forward for any suggestions or thoughts on this topic. Regards, Ann ___
[openstack-dev] [Neutron] Run migrations for service plugins every time
Hi everyone! I'm working on blueprint https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations. This topic was discussed earlier by Salvatore in ML thread Migrations, service plugins and Grenade jobs. I'm researching how to make migrations for service plugins run unconditionally. In fact it is enough to change should_run method for migrations - to make it return True if this migration is for service plugin https://review.openstack.org/#/c/91323/1/neutron/db/migration/__init__.py. It is almost working except for conflicts of VNPaaS service plugin with Brocade and Nuage plugins http://paste.openstack.org/show/77946/. 1) Migration for Brocade plugin fails, because 2c4af419145b_l3_support doesn't run (it adds necessary table 'routers') It can be fixed by adding Brocade plugin to list migration_for_plugins in 2c4af419145b_l3_support. 2) Migration for Nuage plugin fails, because e766b19a3bb_nuage_initial runs after 52ff27f7567a_support_for_vpnaas, and there is no necessary table 'routers'. It can be fixed by adding Nuage plugin to list migration_for_plugins in 2c4af419145b_l3_support and removing addition of these tables in e766b19a3bb_nuage_initial. Also I researched opportunity to make all migrations run unconditionally, but this could not be done as there is going to be a lot of conflicts http://paste.openstack.org/show/77902/. Mostly this conflicts take place in initial migrations for core plugins as they create basical tables that was created before. For example, mlnx_initial create tables securitygroups, securitygroupportbindings, networkdhcpagentbindings, portbindingports that were already created in 3cb5d900c5de_securitygroups, 4692d074d587_agent_scheduler, 176a85fc7d79_add_portbindings_db migrations. Besides I was thinking about 'making migrations smart' - to make them check whether all necessary migrations has been run or not, but the main problem about that is we need to store information about applied migrations, so this should have been planned from the very beginning. I look forward for any suggestions or thoughts on this topic. Regards, Ann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Run migrations for service plugins every time
Anna, It's good to see progress being made on this blueprint. I have some comments inline. Also, I would recommend keeping in mind the comments Mark had regarding migration generation and plugin configuration in hist post on the email thread I started. Salvatore On 30 April 2014 14:16, Anna Kamyshnikova akamyshnik...@mirantis.comwrote: Hi everyone! I'm working on blueprint https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations. This topic was discussed earlier by Salvatore in ML thread Migrations, service plugins and Grenade jobs. I'm researching how to make migrations for service plugins run unconditionally. In fact it is enough to change should_run method for migrations - to make it return True if this migration is for service plugin https://review.openstack.org/#/c/91323/1/neutron/db/migration/__init__.py . I think running migrations unconditionally for service plugins is an advancement but not yet a final solution. I would insist on the path you've been pursuing of running unconditionally all migrations. We should strive to solve the issues you found there so far It is almost working except for conflicts of VNPaaS service plugin with Brocade and Nuage plugins http://paste.openstack.org/show/77946/. 1) Migration for Brocade plugin fails, because 2c4af419145b_l3_support doesn't run (it adds necessary table 'routers') It can be fixed by adding Brocade plugin to list migration_for_plugins in 2c4af419145b_l3_support. 2) Migration for Nuage plugin fails, because e766b19a3bb_nuage_initial runs after 52ff27f7567a_support_for_vpnaas, and there is no necessary table 'routers'. It can be fixed by adding Nuage plugin to list migration_for_plugins in 2c4af419145b_l3_support and removing addition of these tables in e766b19a3bb_nuage_initial. I noticed that too in the past. However there are two aspects to this fix. The one you're mentioning is for fixing migrations in new deployments; on the other hand migrations should be fixed also for existing deployments. This kind of problem seems to me more concerning the work for removing schema auto-generation. Indeed the root problem here is probably that l3 schemas for these two plugins are being created only because of schema auto-generation. Also I researched opportunity to make all migrations run unconditionally, but this could not be done as there is going to be a lot of conflicts http://paste.openstack.org/show/77902/. Mostly this conflicts take place in initial migrations for core plugins as they create basical tables that was created before. For example, mlnx_initial create tables securitygroups, securitygroupportbindings, networkdhcpagentbindings, portbindingports that were already created in 3cb5d900c5de_securitygroups, 4692d074d587_agent_scheduler, 176a85fc7d79_add_portbindings_db migrations. Besides I was thinking about 'making migrations smart' - to make them check whether all necessary migrations has been run or not, but the main problem about that is we need to store information about applied migrations, so this should have been planned from the very beginning. I agree that just changing migrations to run unconditionally is not a viable solution. Rather than changing the existing migration path, I was thinking more about adding corrective unconditional migrations to make the database state the same regardless of the plugin configuration. The difficult part here is that these migrations would need to be smart enough to understand whether a previous migration was executed or skipped; this might also break offline migrations (but perhaps this might be tolerable). I look forward for any suggestions or thoughts on this topic. Regards, Ann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
