Public bug reported: Description ===========
In case of for example DB errors `openstack server create --availability-zone ...` command returns "The requested availability zone is not available (HTTP 400)" instead of actual 50x error. This considered as client error and e.g. stops Heat to retry requests. For example: root@dev01:~# openstack server create --image cirros-0.5.1-x86_64-disk --flavor c1 vm5 --net public --availability-zone nova The requested availability zone is not available (HTTP 400) (Request-ID: req-50190228-51ac-4303-ad3e-d8b920bb7ad8) Logs: Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 00000000-0000-0000-0000-000000000000: ValueError: Cannot get service list (artificial error to help reproduce a bug) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context Traceback (most recent call last): Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/opt/stack/nova/nova/context.py", line 426, in gather_result Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context result = fn(*args, **kwargs) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context result = fn(cls, context, *args, **kwargs) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context raise ValueError('Cannot get service list (artificial error to help reproduce a bug)') Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context Jan 19 12:37:32 dev01 [email protected][4869]: DEBUG nova.objects.service [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] ! service get_all {{(pid=4872) get_all /opt/stack/nova/nova/objects/service.py:632}} Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 558d4e82-cdc0-4020-a56b-6835326f58ec: ValueError: Cannot get service list (artificial error to help reproduce a bug) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context Traceback (most recent call last): Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/opt/stack/nova/nova/context.py", line 426, in gather_result Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context result = fn(*args, **kwargs) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context result = fn(cls, context, *args, **kwargs) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context raise ValueError('Cannot get service list (artificial error to help reproduce a bug)') Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context Jan 19 12:37:32 dev01 [email protected][4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 00000000-0000-0000-0000-000000000000 is not responding and hence skipped from the results. Jan 19 12:37:32 dev01 [email protected][4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 558d4e82-cdc0-4020-a56b-6835326f58ec is not responding and hence skipped from the results. Jan 19 12:37:32 dev01 [email protected][4869]: INFO nova.api.openstack.wsgi [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] HTTP exception thrown: The requested availability zone is not available Under the same conditions, `openstack compute service list` returns an empty success result. Steps to reproduce ================== To emulate runtime errors something like that can be used: @@ -629,6 +629,10 @@ class ServiceList(base.ObjectListBase, base.NovaObject): @base.remotable_classmethod def get_all(cls, context, disabled=None, set_zones=False): + import datetime + if datetime.datetime.now() > datetime.datetime(2021, 1, 19, 12, 7, 0, 976979): + raise ValueError('Cannot get service list (artificial error to help reproduce a bug)') db_services = db.service_get_all(context, disabled=disabled) if set_zones: db_services = availability_zones.set_availability_zones( ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1912343 Title: Nova returns 'The requested availability zone is not available (HTTP 400)' on any cell errors Status in OpenStack Compute (nova): New Bug description: Description =========== In case of for example DB errors `openstack server create --availability-zone ...` command returns "The requested availability zone is not available (HTTP 400)" instead of actual 50x error. This considered as client error and e.g. stops Heat to retry requests. For example: root@dev01:~# openstack server create --image cirros-0.5.1-x86_64-disk --flavor c1 vm5 --net public --availability-zone nova The requested availability zone is not available (HTTP 400) (Request-ID: req-50190228-51ac-4303-ad3e-d8b920bb7ad8) Logs: Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 00000000-0000-0000-0000-000000000000: ValueError: Cannot get service list (artificial error to help reproduce a bug) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context Traceback (most recent call last): Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/opt/stack/nova/nova/context.py", line 426, in gather_result Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context result = fn(*args, **kwargs) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context result = fn(cls, context, *args, **kwargs) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context raise ValueError('Cannot get service list (artificial error to help reproduce a bug)') Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context Jan 19 12:37:32 dev01 [email protected][4869]: DEBUG nova.objects.service [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] ! service get_all {{(pid=4872) get_all /opt/stack/nova/nova/objects/service.py:632}} Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 558d4e82-cdc0-4020-a56b-6835326f58ec: ValueError: Cannot get service list (artificial error to help reproduce a bug) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context Traceback (most recent call last): Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/opt/stack/nova/nova/context.py", line 426, in gather_result Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context result = fn(*args, **kwargs) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context result = fn(cls, context, *args, **kwargs) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context raise ValueError('Cannot get service list (artificial error to help reproduce a bug)') Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug) Jan 19 12:37:32 dev01 [email protected][4869]: ERROR nova.context Jan 19 12:37:32 dev01 [email protected][4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 00000000-0000-0000-0000-000000000000 is not responding and hence skipped from the results. Jan 19 12:37:32 dev01 [email protected][4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 558d4e82-cdc0-4020-a56b-6835326f58ec is not responding and hence skipped from the results. Jan 19 12:37:32 dev01 [email protected][4869]: INFO nova.api.openstack.wsgi [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] HTTP exception thrown: The requested availability zone is not available Under the same conditions, `openstack compute service list` returns an empty success result. Steps to reproduce ================== To emulate runtime errors something like that can be used: @@ -629,6 +629,10 @@ class ServiceList(base.ObjectListBase, base.NovaObject): @base.remotable_classmethod def get_all(cls, context, disabled=None, set_zones=False): + import datetime + if datetime.datetime.now() > datetime.datetime(2021, 1, 19, 12, 7, 0, 976979): + raise ValueError('Cannot get service list (artificial error to help reproduce a bug)') db_services = db.service_get_all(context, disabled=disabled) if set_zones: db_services = availability_zones.set_availability_zones( To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1912343/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
