Public bug reported: NOTE: This is a low priority issue, mostly because it eventually gets fixed by maintenance task
CheckRevisionNumberCommand relies in finding a corresponding entry in OVN's NAT table in order to update the OVN_REV_NUM_EXT_ID_KEY to keep ovn and neutron databases in sync. Ref: http://lucasgom.es/posts/neutron_ovn_database_consistency.html Trouble is that unless the floating ip is associated, there will be no entries in OVN's NAT table, causing the call to db_rev.bump_revision(context, floatingip, ovn_const.TYPE_FLOATINGIPS) to not take place. Steps to reproduce it: # create a floating ip but do not associate it with anything so router_id is None FIP=172.24.4.8 openstack floating ip create --floating-ip-address ${FIP} public FIP_UUID=$(openstack floating ip show ${FIP} -f value -c id) ; echo $FIP_UUID # Mess with its name, which will bump revision on fip object openstack floating ip set --description foo ${FIP_UUID} Code when there is no NAT for a given FIP makes line 1044 skip line 1045 https://github.com/openstack/neutron/blob/15088b39bab715e40d8161a85c95ca400708c83f/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L1044 check_rev_cmd.result is None The dbs are now the inconsistent state mysql> use neutron; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select * from standardattributes where resource_type="floatingips"; +----+---------------+---------------------+---------------------+-------------+-----------------+ | id | resource_type | created_at | updated_at | description | revision_number | +----+---------------+---------------------+---------------------+-------------+-----------------+ | 49 | floatingips | 2020-05-18 20:56:51 | 2020-05-18 20:58:58 | foo2 | 2 | +----+---------------+---------------------+---------------------+-------------+-----------------+ 1 row in set (0.01 sec) mysql> select * from ovn_revision_numbers where resource_type="floatingips"; +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ | standard_attr_id | resource_uuid | resource_type | revision_number | created_at | updated_at | +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ | 49 | 5a1e1ffa-0312-4e78-b7a0-551c396bcf6b | floatingips | 0 | 2020-05-18 20:56:51 | 2020-05-18 20:57:08 | +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ 1 row in set (0.00 sec) Maintenance task fixes it up later May 18 21:50:29 stack neutron-server[909]: DEBUG futurist.periodics [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBIn\ consistenciesPeriodics.check_for_inconsistencies' {{(pid=3186) _process_scheduled /usr/local/lib/python3.6/dist-packages/futurist/periodics.py:642}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Synchronizing Neutron and OVN datab\ ases {{(pid=3186) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:347}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Number of inconsistencies found at \ create/update: floatingips=1 {{(pid=3186) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:325}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Fixing resource 6b876a35-d286-4407-\ b538-9ce07ab1a281 (type: floatingips) at create/update {{(pid=3186) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:359}} May 18 21:50:29 stack neutron-server[909]: INFO neutron.db.ovn_revision_numbers_db [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Successfully bumped revision number for resource 6b876a35-d286-4407-b538-9ce07ab1\ a281 (type: floatingips) to 1 May 18 21:50:29 stack neutron-server[909]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Synchronization finished (took 0.08 \ seconds) ** Affects: neutron Importance: Undecided Status: New ** Description changed: NOTE: This is a low priority issue, mostly because it eventually gets fixed by maintenance task CheckRevisionNumberCommand relies in finding a corresponding entry in OVN's NAT table in order to update the OVN_REV_NUM_EXT_ID_KEY to keep ovn and neutron databases in sync. Ref: http://lucasgom.es/posts/neutron_ovn_database_consistency.html - Trouble is that unless the floating ip is associated, there will be no entries in OVN's NAT - table, causing the call to + Trouble is that unless the floating ip is associated, there will be no + entries in OVN's NAT table, causing the call to - db_rev.bump_revision(context, floatingip, - ovn_const.TYPE_FLOATINGIPS) + db_rev.bump_revision(context, floatingip, ovn_const.TYPE_FLOATINGIPS) to not take place. - Steps to reproduce it: # create a floating ip but do not associate it with anything so router_id is None FIP=172.24.4.8 openstack floating ip create --floating-ip-address ${FIP} public FIP_UUID=$(openstack floating ip show ${FIP} -f value -c id) ; echo $FIP_UUID # Mess with its name, which will bump revision on fip object openstack floating ip set --description foo ${FIP_UUID} - Code when there is no NAT for a given FIP makes line 1044 skip line 1045 https://github.com/openstack/neutron/blob/15088b39bab715e40d8161a85c95ca400708c83f/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L1044 check_rev_cmd.result is None - The dbs are now the inconsistent state mysql> use neutron; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select * from standardattributes where resource_type="floatingips"; +----+---------------+---------------------+---------------------+-------------+-----------------+ | id | resource_type | created_at | updated_at | description | revision_number | +----+---------------+---------------------+---------------------+-------------+-----------------+ | 49 | floatingips | 2020-05-18 20:56:51 | 2020-05-18 20:58:58 | foo2 | 2 | +----+---------------+---------------------+---------------------+-------------+-----------------+ 1 row in set (0.01 sec) mysql> select * from ovn_revision_numbers where resource_type="floatingips"; +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ | standard_attr_id | resource_uuid | resource_type | revision_number | created_at | updated_at | +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ | 49 | 5a1e1ffa-0312-4e78-b7a0-551c396bcf6b | floatingips | 0 | 2020-05-18 20:56:51 | 2020-05-18 20:57:08 | +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ 1 row in set (0.00 sec) - - Maintenance task fixes it up later May 18 21:50:29 stack neutron-server[909]: DEBUG futurist.periodics [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBIn\ consistenciesPeriodics.check_for_inconsistencies' {{(pid=3186) _process_scheduled /usr/local/lib/python3.6/dist-packages/futurist/periodics.py:642}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Synchronizing Neutron and OVN datab\ ases {{(pid=3186) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:347}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Number of inconsistencies found at \ create/update: floatingips=1 {{(pid=3186) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:325}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Fixing resource 6b876a35-d286-4407-\ b538-9ce07ab1a281 (type: floatingips) at create/update {{(pid=3186) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:359}} May 18 21:50:29 stack neutron-server[909]: INFO neutron.db.ovn_revision_numbers_db [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Successfully bumped revision number for resource 6b876a35-d286-4407-b538-9ce07ab1\ a281 (type: floatingips) to 1 May 18 21:50:29 stack neutron-server[909]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Synchronization finished (took 0.08 \ seconds) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1879407 Title: [OVN] Modifying FIP that is no associated causes ovn_revision_numbers to go stale Status in neutron: New Bug description: NOTE: This is a low priority issue, mostly because it eventually gets fixed by maintenance task CheckRevisionNumberCommand relies in finding a corresponding entry in OVN's NAT table in order to update the OVN_REV_NUM_EXT_ID_KEY to keep ovn and neutron databases in sync. Ref: http://lucasgom.es/posts/neutron_ovn_database_consistency.html Trouble is that unless the floating ip is associated, there will be no entries in OVN's NAT table, causing the call to db_rev.bump_revision(context, floatingip, ovn_const.TYPE_FLOATINGIPS) to not take place. Steps to reproduce it: # create a floating ip but do not associate it with anything so router_id is None FIP=172.24.4.8 openstack floating ip create --floating-ip-address ${FIP} public FIP_UUID=$(openstack floating ip show ${FIP} -f value -c id) ; echo $FIP_UUID # Mess with its name, which will bump revision on fip object openstack floating ip set --description foo ${FIP_UUID} Code when there is no NAT for a given FIP makes line 1044 skip line 1045 https://github.com/openstack/neutron/blob/15088b39bab715e40d8161a85c95ca400708c83f/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L1044 check_rev_cmd.result is None The dbs are now the inconsistent state mysql> use neutron; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select * from standardattributes where resource_type="floatingips"; +----+---------------+---------------------+---------------------+-------------+-----------------+ | id | resource_type | created_at | updated_at | description | revision_number | +----+---------------+---------------------+---------------------+-------------+-----------------+ | 49 | floatingips | 2020-05-18 20:56:51 | 2020-05-18 20:58:58 | foo2 | 2 | +----+---------------+---------------------+---------------------+-------------+-----------------+ 1 row in set (0.01 sec) mysql> select * from ovn_revision_numbers where resource_type="floatingips"; +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ | standard_attr_id | resource_uuid | resource_type | revision_number | created_at | updated_at | +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ | 49 | 5a1e1ffa-0312-4e78-b7a0-551c396bcf6b | floatingips | 0 | 2020-05-18 20:56:51 | 2020-05-18 20:57:08 | +------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+ 1 row in set (0.00 sec) Maintenance task fixes it up later May 18 21:50:29 stack neutron-server[909]: DEBUG futurist.periodics [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBIn\ consistenciesPeriodics.check_for_inconsistencies' {{(pid=3186) _process_scheduled /usr/local/lib/python3.6/dist-packages/futurist/periodics.py:642}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Synchronizing Neutron and OVN datab\ ases {{(pid=3186) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:347}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Number of inconsistencies found at \ create/update: floatingips=1 {{(pid=3186) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:325}} May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Fixing resource 6b876a35-d286-4407-\ b538-9ce07ab1a281 (type: floatingips) at create/update {{(pid=3186) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:359}} May 18 21:50:29 stack neutron-server[909]: INFO neutron.db.ovn_revision_numbers_db [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Successfully bumped revision number for resource 6b876a35-d286-4407-b538-9ce07ab1\ a281 (type: floatingips) to 1 May 18 21:50:29 stack neutron-server[909]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Synchronization finished (took 0.08 \ seconds) To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1879407/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
