** Description changed: + [Impact] + + Loadbalacer stuck in status PENDING_DELETE if TLS cert unavailable + + [Test Case] + + Pls refer to [Test steps] section below. + + [Regression Potential] + + The fix is already in the upstream main, stable/2024.1, stable/2023.2, + stable/2023.1 branches, so it is a clean backport and might be helpful + for deployments using octavia. + + I also test this fix, it works well - + https://paste.ubuntu.com/p/s4MsMjV6mP/ + + [Others] + + Original Bug Description Below + =========== + Loadbalacer stuck in status PENDING_DELETE if TLS cert unavailable 1. Create load balancer with TERMINATED_HTTPS listener 2. Disable your TLS storage, or delete cert from storage 3. Try to delete loadbalancer with cascade flag Error on logs: ``` Unable to retrieve certificate(s) due to Could not retrieve certificate: <some id> Exception during message handling ``` ``` Traceback (most recent call last): File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming, res = self.dispatcher.dispatch(message), - File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch, return self._do_dispatch(endpoint, method, ctxt, args), - File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch, result = func(ctxt, **new_args), - File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/queue/v2/endpoints.py", line 56, in delete_load_balancer, self.worker.delete_load_balancer(loadbalancer, cascade), - File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/worker/v2/controller_worker.py", line 387, in delete_load_balancer, listeners = flow_utils.get_listeners_on_lb(db_lb), - File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/worker/v2/flows/flow_utils.py", line 52, in get_listeners_on_lb, prov_listener = provider_utils.db_listener_to_provider_listener(), - File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 182, in db_listener_to_provider_listener, new_listener_dict = listener_dict_to_provider_dict(), - File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 261, in listener_dict_to_provider_dict, with excutils.save_and_reraise_exception() as ctxt:, - File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__, self.force_reraise(), - File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise, raise self.value, - File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 258, in listener_dict_to_provider_dict, cert_dict = cert_parser.load_certificates_data(cert_manager) + File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch, return self._do_dispatch(endpoint, method, ctxt, args), + File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch, result = func(ctxt, **new_args), + File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/queue/v2/endpoints.py", line 56, in delete_load_balancer, self.worker.delete_load_balancer(loadbalancer, cascade), + File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/worker/v2/controller_worker.py", line 387, in delete_load_balancer, listeners = flow_utils.get_listeners_on_lb(db_lb), + File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/worker/v2/flows/flow_utils.py", line 52, in get_listeners_on_lb, prov_listener = provider_utils.db_listener_to_provider_listener(), + File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 182, in db_listener_to_provider_listener, new_listener_dict = listener_dict_to_provider_dict(), + File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 261, in listener_dict_to_provider_dict, with excutils.save_and_reraise_exception() as ctxt:, + File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__, self.force_reraise(), + File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise, raise self.value, + File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 258, in listener_dict_to_provider_dict, cert_dict = cert_parser.load_certificates_data(cert_manager) File "/var/lib/openstack/lib/python3.10/site-packages/octavia/common/tls_utils/cert_parser.py", line 381, in load_certificates_data, raise exceptions.CertificateRetrievalException(, octavia.common.exceptions.CertificateRetrievalException: Could not retrieve certificate: ] ``` + + [Test steps] + + 1. Create load balancer with TERMINATED_HTTPS listener, eg: + + secret1_id=$(openstack secret store --name='lb_tls_secret_1' -t 'application/octet-stream' -e 'base64' --payload="$(base64 < www.server1.com.p12)" -f value -c "Secret href") + octavia_user_id=$(openstack user show octavia --domain service_domain -f value -c id); echo $octavia_user_id; + openstack acl user add -u $octavia_user_id $secret1_id + subnetid=$(openstack subnet show private_subnet -f value -c id); echo $subnetid + lb_id=$(openstack loadbalancer create --name lb1 --vip-subnet-id $subnetid -f value -c id); echo $lb_id + listener_id=$(openstack loadbalancer listener create $lb_id --name https_listener --protocol-port 80 --protocol TERMINATED_HTTPS --default-tls-container=$secret1_id --sni-container-refs $secret1_id $secret2_id -f value -c id); echo $listener_id + + 2. Disable your TLS storage, or delete cert from storage, eg: + + openstack secret delete $secret1_id + + 3. Try to delete loadbalancer with cascade flag + + openstack loadbalancer delete lb1 --cascade
** Summary changed: - Loadbalacer stuck in status PENDING_DELETE if TLS storage unavailable in cascade deletion + [SRU] Loadbalacer stuck in status PENDING_DELETE if TLS storage unavailable in cascade deletion ** Patch added: "noble.debdiff" https://bugs.launchpad.net/cloud-archive/bobcat/+bug/2077348/+attachment/5845687/+files/noble.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2077348 Title: [SRU] Loadbalacer stuck in status PENDING_DELETE if TLS storage unavailable in cascade deletion To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2077348/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
