Hi Volodymyr, This is a known issue with the neutron (neutron-lbaas) database getting out of sync and/or not properly handling driver errors. This is one reason we are moving to deprecate neutron-lbaas. If you can, we recommend you move to exclusively using Octavia without neutron-lbaas.
The fundamental issue is that neutron-lbaas does not use database transactions properly and does not lock records and the API correctly so multiple requests/threads can modify neutron database records for the load balancers. This is particularly an issue if you are doing a high rate of changes against the neutron-lbaas API. To correct all of these would require rewriting a significant amount of the neutron-lbaas API code which doesn't make much sense given we are moving towards deprecating it in favor of the Octavia API that does not have this issue. To mitigate this to some degree you can enable the multiple synchronization features that push data back to the neutron database correcting the inaccuracies in the neutron database. event_streamer_driver and sync_provisioning_status can be enabled in the octavia.conf [health_manager] section. https://docs.openstack.org/octavia/latest/configuration/configref.html#health_manager.event_streamer_driver This tells octavia to be aggressive at pushing status information back up to the neutron database getting it back in sync with reality. Michael On Wed, Nov 29, 2017 at 4:41 AM, Volodymyr Litovka <[email protected]> wrote: > Hi colleagues, > > just to inform you: openstack CLI extension output don't match corresponsing > 'neutron lbaas-*' CLI output, e.g. > > doka@lagavulin(admin@bush):~/heat$ neutron lbaas-loadbalancer-show > db8ae876-b6eb-4c45-95d1-33e0ca6193de > neutron CLI is deprecated and will be removed in the future. Use openstack > CLI instead. > +---------------------+------------------------------------------------+ > | Field | Value | > +---------------------+------------------------------------------------+ > | admin_state_up | True | > | description | | > | id | db8ae876-b6eb-4c45-95d1-33e0ca6193de | > | listeners | {"id": "3ce73276-9c80-4645-aa0c-263fae736ef5"} | > | name | nbt-balancer | > | operating_status | ONLINE | > | pools | {"id": "e106e039-af27-4cfa-baa2-7238acd3078e"} | > | provider | octavia | > | provisioning_status | ACTIVE | > | tenant_id | c1114776e144400da17d8e060856be8c | > | vip_address | 10.1.1.13 | > | vip_port_id | 6898f149-9d6c-4fa7-b16c-af582cd18efa | > | vip_subnet_id | ecb891c1-b7e5-45e0-8815-8675381d70d2 | > +---------------------+------------------------------------------------+ > doka@lagavulin(admin@bush):~/heat$ openstack loadbalancer show > db8ae876-b6eb-4c45-95d1-33e0ca6193de > +---------------------+--------------------------------------+ > | Field | Value | > +---------------------+--------------------------------------+ > | admin_state_up | True | > | created_at | 2017-11-29T12:02:55 | > | description | | > | flavor | | > | id | db8ae876-b6eb-4c45-95d1-33e0ca6193de | > | listeners | 3ce73276-9c80-4645-aa0c-263fae736ef5 | > | name | nbt-balancer | > | operating_status | OFFLINE | > | pools | e106e039-af27-4cfa-baa2-7238acd3078e | > | project_id | c1114776e144400da17d8e060856be8c | > | provider | octavia | > | provisioning_status | ACTIVE | > | updated_at | 2017-11-29T12:04:40 | > | vip_Address | 10.1.1.13 | > | vip_network_id | 141f1af9-c309-4263-b7f9-dab3922957c3 | > | vip_port_id | 6898f149-9d6c-4fa7-b16c-af582cd18efa | > | vip_subnet_id | ecb891c1-b7e5-45e0-8815-8675381d70d2 | > +---------------------+--------------------------------------+ > > more difference between corresponding command pairs for listeners and pools. > > Also, member list shows NO_MONITOR status for members > > doka@lagavulin(admin@bush):~/heat$ openstack loadbalancer member list > nbt-pool > +--------------------------------------+------+----------------------------------+---------------------+-----------+---------------+------------------+--------+ > | id | name | project_id > | provisioning_status | address | protocol_port | operating_status | > weight | > +--------------------------------------+------+----------------------------------+---------------------+-----------+---------------+------------------+--------+ > | f7f0d598-7e23-4f42-bb8e-6c5818b1b2cb | | > c1114776e144400da17d8e060856be8c | ACTIVE | 10.1.1.14 | > 8080 | NO_MONITOR | 1 | > | a369460a-6b0d-4c3d-8c84-9c76460719d0 | | > c1114776e144400da17d8e060856be8c | ACTIVE | 10.1.1.11 | > 8080 | NO_MONITOR | 1 | > +--------------------------------------+------+----------------------------------+---------------------+-----------+---------------+------------------+--------+ > > while healthmonitor exists > > doka@lagavulin(admin@bush):~/heat$ neutron lbaas-healthmonitor-list > neutron CLI is deprecated and will be removed in the future. Use openstack > CLI instead. > +--------------------------------------+------+----------------------------------+------+----------------+ > | id | name | tenant_id > | type | admin_state_up | > +--------------------------------------+------+----------------------------------+------+----------------+ > | 71cc7956-7c54-448a-96a7-709905c2bf4f | | > c1114776e144400da17d8e060856be8c | PING | True | > +--------------------------------------+------+----------------------------------+------+----------------+ > doka@lagavulin(admin@bush):~/heat$ neutron lbaas-healthmonitor-show > 71cc7956-7c54-448a-96a7-709905c2bf4f > neutron CLI is deprecated and will be removed in the future. Use openstack > CLI instead. > +------------------+------------------------------------------------+ > | Field | Value | > +------------------+------------------------------------------------+ > | admin_state_up | True | > | delay | 5 | > | id | 71cc7956-7c54-448a-96a7-709905c2bf4f | > | max_retries | 3 | > | max_retries_down | 3 | > | name | | > | pools | {"id": "e106e039-af27-4cfa-baa2-7238acd3078e"} | > | tenant_id | c1114776e144400da17d8e060856be8c | > | timeout | 1 | > | type | PING | > +------------------+------------------------------------------------+ > > but openstack cli extension thinks different > > doka@lagavulin(admin@bush):~/heat$ openstack loadbalancer healthmonitor list > > doka@lagavulin(admin@bush):~/heat$ openstack loadbalancer healthmonitor show > 71cc7956-7c54-448a-96a7-709905c2bf4f > Unable to locate 71cc7956-7c54-448a-96a7-709905c2bf4f in healthmonitors > > Be informed and can this impact something else? > > Thank you. > > -- > Volodymyr Litovka > "Vision without Execution is Hallucination." -- Thomas Edison > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
