[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
** Changed in: cloud-archive/ussuri Status: Triaged => Fix Released ** Changed in: cloud-archive/train Status: Triaged => Fix Released ** Changed in: octavia (Ubuntu Focal) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
Marked as fix released since the point releases are released. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
^^ links don't work, these do: https://bugs.launchpad.net/cloud-archive/+bug/1900476 https://bugs.launchpad.net/cloud-archive/+bug/1900477 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
octavia 6.1.0 and 5.0.3 will be included in the following SRUs for Ubuntu: https:pad.lv/1900477 https:pad.lv/1900476 ** Changed in: octavia (Ubuntu Focal) Status: New => Triaged ** Changed in: cloud-archive/ussuri Status: New => Triaged ** Changed in: cloud-archive/train Status: New => Triaged ** Changed in: cloud-archive/train Importance: Undecided => High ** Changed in: cloud-archive/ussuri Importance: Undecided => High ** Changed in: octavia (Ubuntu Focal) Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
I'd like to see if we can pick these up in an upstream stable release so I've proposed new upstream releases at https://review.opendev.org/#/c/758606/. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
** Tags added: sts-sru-needed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
** Description changed: [Impact] (from storyboard desciption) Currently if taskflow process is interrupted (during create/update/failover - node is rebooted or service is restarted) - loadbalancer will stuck in PENDING state. Taskflow provides persistence module which allows to save flows state for recovery https://docs.openstack.org/taskflow/latest/user/persistence.html Otherwise partially created/updated/deleted resources should be moved to ERROR state when service is up again. (like it is done in Cinder) [Test Case] + + * deploy Openstack with Octavia and 2 compute hosts e.g. ./generate-bundle.sh --use-stable-charms --release train --octavia --num-compute 3 + * juju config octavia loadbalancer-topology=ACTIVE_STANDBY + * create ubuntu vm and install apache2 (i.e. listen port 80) + * create loadbalancer with vm as member and floating ip for LB vip + * test connection with: nc -vz LB_FIP 80 + * openstack loadbalancer amphora list + * get amphora master vm uuid: openstack loadbalancer amphora show -c compute_id -f value + * openstack server show -c "OS-EXT-SRV-ATTR:host" -f value + * poweroff compute host from previous step + * openstack loadbalancer failover LB_UUID + * wait a few seconds + * openstack loadbalancer amphora list + * Wait until you have one BACKUP and one MASTER + * Test connection with: nc -vz LB_FIP 80 + [Regression Potential] Tried to failover a loadbalancer that has missing entries of amphora master. The loadbalancer went to ERROR state. OpenStack version: Train The fix is available in upstream as part of the Octavia Failover refactor patches in Train https://review.opendev.org/#/q/status:merged+project:openstack/octavia+branch:stable/train+topic:failover-refactor Verified with the upstream patches and it worked. ** Description changed: [Impact] (from storyboard desciption) Currently if taskflow process is interrupted (during create/update/failover - node is rebooted or service is restarted) - loadbalancer will stuck in PENDING state. Taskflow provides persistence module which allows to save flows state for recovery https://docs.openstack.org/taskflow/latest/user/persistence.html Otherwise partially created/updated/deleted resources should be moved to ERROR state when service is up again. (like it is done in Cinder) [Test Case] * deploy Openstack with Octavia and 2 compute hosts e.g. ./generate-bundle.sh --use-stable-charms --release train --octavia --num-compute 3 * juju config octavia loadbalancer-topology=ACTIVE_STANDBY * create ubuntu vm and install apache2 (i.e. listen port 80) * create loadbalancer with vm as member and floating ip for LB vip * test connection with: nc -vz LB_FIP 80 * openstack loadbalancer amphora list * get amphora master vm uuid: openstack loadbalancer amphora show -c compute_id -f value * openstack server show -c "OS-EXT-SRV-ATTR:host" -f value * poweroff compute host from previous step * openstack loadbalancer failover LB_UUID * wait a few seconds * openstack loadbalancer amphora list * Wait until you have one BACKUP and one MASTER * Test connection with: nc -vz LB_FIP 80 - [Regression Potential] + While new failovers have been proven to work properly, this will resolve existing failed failovers which will require setting the LB state from PENDING_UPDATE to ERROR in the database prior to triggering a new failover. Tried to failover a loadbalancer that has missing entries of amphora master. The loadbalancer went to ERROR state. OpenStack version: Train The fix is available in upstream as part of the Octavia Failover refactor patches in Train https://review.opendev.org/#/q/status:merged+project:openstack/octavia+branch:stable/train+topic:failover-refactor Verified with the upstream patches and it worked. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
Verified with the following steps: 1. Create Train deployment with Octavia ./generate-bundle.sh --defaults --use-stable-charms --release train --octavia --num-compute 2 2. Configure Octavia juju config octavia loadbalancer-topology=ACTIVE_STANDBY juju config octavia spare-pool-size=2 2. Create Cirros VMs ./tools/instance_launch.sh 2 cirros 3. Create fake webserver on VMs (See https://code.launchpad.net/~nicolasbock/stsstack-bundles/+git/stsstack- bundles/+merge/392344) ./tools/run_fake_webserver.sh 4. Create load balancer ./tools/create_octavia_lb.sh 5. Test load balancer with curl LB_FIP 6. Shut down a nova-compute that is hosting one of the amphorae. This will break the load balancer due to https://storyboard.openstack.org/#!/story/2003084 7. Install SRU in octavia unit 8. Check load balancer with openstack loadbalancer list 9. Verify operation with step 5. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
** Patch added: "lp1899964-train.debdiff" https://bugs.launchpad.net/ubuntu/+source/octavia/+bug/1899964/+attachment/5422697/+files/lp1899964-train.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
** Patch added: "lp1899964-ussuri.debdiff" https://bugs.launchpad.net/ubuntu/+source/octavia/+bug/1899964/+attachment/5422695/+files/lp1899964-ussuri.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
Task originally tracked in storyboard: https://storyboard.openstack.org/#!/story/2005072 https://storyboard.openstack.org/#!/story/2007847 Victoria patches are already included in Ubuntu packages for 7.0.0 Ussuri patches to be SRUd: https://review.opendev.org/#/q/status:merged+project:openstack/octavia+branch:stable/ussuri+topic :failover-refactor Train patches to be SRUd: https://review.opendev.org/#/q/status:merged+project:openstack/octavia+branch:stable/train+topic :failover-refactor ** Description changed: + [Impact] + (from storyboard desciption) Currently if taskflow process is interrupted (during create/update/failover - node is rebooted or service is restarted) - loadbalancer will stuck in PENDING state. + Taskflow provides persistence module which allows to save flows state for recovery https://docs.openstack.org/taskflow/latest/user/persistence.html + Otherwise partially created/updated/deleted resources should be moved to ERROR state when service is up again. (like it is done in Cinder) + + [Test Case] + + [Regression Potential] + + + Tried to failover a loadbalancer that has missing entries of amphora master. The loadbalancer went to ERROR state. OpenStack version: Train The fix is available in upstream as part of the Octavia Failover refactor patches in Train https://review.opendev.org/#/q/status:merged+project:openstack/octavia+branch:stable/train+topic:failover-refactor Verified with the upstream patches and it worked. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
** Also affects: octavia (Ubuntu Groovy) Importance: Undecided Status: Fix Released ** Also affects: octavia (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: cloud-archive/victoria Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899964] Re: Failover of loadbalancer fails when Amphora master is missing
** Also affects: cloud-archive Importance: Undecided Status: New ** Also affects: cloud-archive/ussuri Importance: Undecided Status: New ** Also affects: cloud-archive/train Importance: Undecided Status: New ** Also affects: cloud-archive/victoria Importance: Undecided Status: New ** Changed in: cloud-archive/victoria Status: New => Fix Committed ** Changed in: octavia (Ubuntu) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899964 Title: Failover of loadbalancer fails when Amphora master is missing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1899964/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
