[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-22 Thread Jared Baker
@Rodolfo, We still aren't 100% certain that this particular change is the culprit but it's highly suspect because of the nature of the change compared to other commits. In this ticket you have at least 4 instances of this bug being reproducible by different people including Ubuntu maintainers and

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-22 Thread Jared Baker
@Christian, sorry if it wasn't clear but I don't believe this is database migration related. @Rodolfo, At the very least the code that was written with timeout=3 is likely problematic and should have been at least made user configurable. It's quite possible this change is the culprit and breaks

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-21 Thread Jared Baker
Our order of operations went as such: - All packages get updated to latest available Ussuri (16.3.1) for Bionic via apt-get dist-upgrade - Stop services (systemctl stop neutron-dhcp-agent; systemctl stop neutron-metadata-agent; systemctl stop neutron-ovs-cleanup; systemctl stop

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-18 Thread Jared Baker
Ante: Thank you for the link to the 16.3.0 packages. We managed to find 16.2.0 before your post and are currently using them and it seems to be working fine for us. Should the opportunity arise we will install 16.3.0 since we've downloaded the deb files just in case. Billy: I guess the hunt

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-16 Thread Jared Baker
Hey Ubuntu team, Is there any way for you to provide the Neutron 16.3.0 packages for Bionic? We are in emergency mode here and the current 16.0.0 we are running, although working better than 16.3.1 is causing major control plane instability. We are in desperate need of 16.3.0 packages. -- You

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-16 Thread Jared Baker
OK, I did find evidence that this issue did begin to happen as we upgraded our Train packages just before we upgraded to Ussuri (I believe its best practice to update all packages, THEN upgrade to a new version of OpenStack) Upgraded Neutron 15.3.0 to 15.3.3 at 16:40 EDT 2021-06-10 16:40:40

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-16 Thread Jared Baker
keepalived: 1:1.3.9-1ubuntu0.18.04.2 from: http://archive.ubuntu.com/ubuntu bionic-updates/main neutron-l3-agent.log filled with the following log lines for every single router in our cluster: 2021-06-10 19:59:53.602 328191 ERROR neutron.agent.l3.ha_router [-] Gateway interface for router

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-15 Thread Jared Baker
590551d [L3][HA] Retry when setting HA router GW status. I'm not a developer but this change does reflect the log error messages myself and the OP are seeing. https://github.com/openstack/neutron/commit/8f5a801270f81bd9fe3559fee9c1714c97849b3e -- You received this bug notification because you

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-14 Thread Jared Baker
I was not able to roll back to 16.3.0 because they are not on any repositories that I could find so I had to manually download neutron debian files for focal and thankfully they seem to kind of work on Bionic Ussuri. We are now running 16.0.0~b3~git2020041516.5f42488a9a- 0ubuntu2 which is not

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-14 Thread Jared Baker
I don't think so. We don't even use DVR. Our observations were basically identical to OP's which was a telltale log entry 'Gateway interface for router 02945b59-639b-41be-8237-3b7933b4e32d was not set up; router will not work properly' and also that any Layer-3 agents for a router all remained in

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-13 Thread Jared Baker
This bug appears to be present in 16.3.2 in the proposed repository as well. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1927868 Title: vRouter not working after update to 16.3.1 To manage

[Bug 1927868] Re: vRouter not working after update to 16.3.1

2021-06-10 Thread Jared Baker
This is a terrible bug. 16.3.1 needs to be pulled from the repositories immediately. This will cause all Ussuri OpenStack deployments on Bionic to be completely down. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1845246] Re: Failed to import plugin clustering

2019-10-22 Thread Jared Baker
Installed python3-senlinclient from stein-proposed root@lab-controller:~# apt-cache policy python3-senlinclient python3-senlinclient: Installed: 1.9.0-0ubuntu1~cloud0 Candidate: 1.9.0-0ubuntu1~cloud0 Version table: *** 1.9.0-0ubuntu1~cloud0 500 500

[Bug 1845246] Re: Failed to import plugin clustering

2019-10-17 Thread Jared Baker
Hi Corey, Looks like python3-senlinclient is already installed (but we dont use senlin). I would remove it but it wants to pull out all the openstack heat packages, which we use. root@lab-controller:~# dpkg -l | grep -i python3 | grep -i client ii python3-amqp

[Bug 1845246] [NEW] Failed to import plugin clustering

2019-09-24 Thread Jared Baker
Public bug reported: When running any openstack client command I am provided with the following error, including the correct output of the command. Example: openstack image list WARNING: Failed to import plugin clustering. +--+-++ | ID

[Bug 1659620] Re: Panels get broken with COMPRESS_OFFLINE = True

2019-08-07 Thread Jared Baker
Running Ubuntu 18.04 with openstack-dashboard 14.0.3-0ubuntu1~cloud0 and having issues with certain modals not working (create object container, create keypair, upload keypair, create server group) all popping up as blank modals. This has been a chronic issue for us for Ubuntu 16.04 and also 18.04

[Bug 1666853] Re: grub-probe fail when fs is larger than 2tb

2019-01-18 Thread Jared Baker
Possibly related bugs: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1812411 https://bugs.launchpad.net/cloud-utils/+bug/1762748 We run an OpenStack cloud and provide very large instances to our users and noticed that there was possibly a breaking change made in 1762748 which resulted

[Bug 1812411] Re: cloud-init corrupting grub

2019-01-18 Thread Jared Baker
Hi Scott, Yes probably related bugs for sure. Again, I'd be happy to help test a fix if one is developed as I'm able to easily reproduce this problem. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1812411] [NEW] cloud-init corrupting grub

2019-01-18 Thread Jared Baker
Public bug reported: I believe I have discovered problems in the *recent builds* of Ubuntu 14.04 and 16.04 (NO ISSUE with 18.04 or OLDER 14.04/16.04 builds) cloud images (https://cloud-images.ubuntu.com/) for instances with very large local disks. We are using OpenStack Queens on Ubuntu 16.04 and

[Bug 1775170] Re: [SRU] Fixing empty create swift container dialog after upgrading horizon from newton to ocata

2018-10-10 Thread Jared Baker
I did a bit more investigating and it seems to me like the contents of the defined memcached server can influence the generation of the javascript in '/var/lib/openstack-dashboard/static/dashboard/js' Is that assumption correct? If so, maybe the collect/compress should flush the memcached

[Bug 1775170] Re: [SRU] Fixing empty create swift container dialog after upgrading horizon from newton to ocata

2018-10-10 Thread Jared Baker
Hey Corey, I believe I have found the source of my problem with bad javascript being generated at the compress and collect static stage and it has to do with whats in my /etc/openstack-dashboard/local_settings.py, specifically my memcached section. We run 3 controllers and 3 instances of

[Bug 1775170] Re: [SRU] Fixing empty create swift container dialog after upgrading horizon from newton to ocata

2018-10-04 Thread Jared Baker
I don't think it's necessary to go through the process of a pike -> queens upgrade to reproduce the issue. Reason I say that is because we can take our current install and purge & install the openstack-dashboard & related packages and without our customizations we still experience the problem.

[Bug 1775170] Re: [SRU] Fixing empty create swift container dialog after upgrading horizon from newton to ocata

2018-10-03 Thread Jared Baker
Just to be clear, I've tried removing the following directories rm -rf /usr/share/openstack-dashboard/ rm -rf /var/lib/openstack-dashboard/static/ followed by an apt-get install --reinstall openstack-dashboard python- django-horizon Leaving the dashboard uncustomized, I am still getting a blank

[Bug 1775170] Re: [SRU] Fixing empty create swift container dialog after upgrading horizon from newton to ocata

2018-10-03 Thread Jared Baker
We implement the following customizations of the openstack-horizon dashboard: - /usr/share/openstack-dashboard/openstack_dashboard/templates/_login_footer.html - /usr/share/openstack-dashboard/openstack_dashboard/templates/_footer.html -

[Bug 1775170] Re: [SRU] Fixing empty create swift container dialog after upgrading horizon from newton to ocata

2018-10-03 Thread Jared Baker
This particular bug of the container creation modal being blank has plagued us for several OpenStack releases now. We are running Queens on Ubuntu 16.04 currently (http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/queens/main amd64 Packages) python-django-horizon

[Bug 1775862] Re: Missing Magnum-UI package for Ubuntu 16.04

2018-07-05 Thread Jared Baker
Are there any plans to release magnum-ui as part of the Ubuntu cloud archive repository? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1775862 Title: Missing Magnum-UI package for Ubuntu 16.04 To

[Bug 1762748] Re: Larger than 2 TB disks not possible

2018-04-30 Thread Jared Baker
I have tested cloud-guest-utils version 0.27-0ubuntu25.1 and it appears to allow instances with disks larger than 2TB to be deployed properly. My testing was to deploy an instance with a small root disk, then install the patched cloud-guest-utils on to the instance. I would then take a snapshot

[Bug 1573272] Re: default gateway route not installed for bond interfaces through reboot

2017-05-18 Thread Jared Baker
@Gael from what I can tell the fix has not been released based on this bug status is still 'in progress'. I'm running Ubuntu 16.04 with vlan 1.9-3.2ubuntu1 and it still contains the 'ip link set up dev $IF_VLAN_RAW_DEVICE' on line 63 of /etc/network /if-pre-up.d/vlan . You can modify the file