[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Hi Robie, apologies for the delay, I've been waiting for the customer to respond with a clarification of the problem for quite a few weeks. Though they have not confirmed the functional issue, the cpu_util metric was deprecated in Rocky and fully removed in Victoria and as a result within Watcher the metric instance_cpu_usage, which is obtained from Ceilometer's cpu_util, is reported as "None" rather than a percentage. There are various Watcher strategies (eg. the workload_balancing audit which recommends vm host migrations to balance the cluster's cpu usage) which rely on instance_cpu_usage to determine action plans. Since cpu_util was reporting None, these strategies were rendered ineffective. This would be a better description of the impact, though as I said I'm still in the process of verifying with the customer that this is what they are experiencing. The purpose of this patch is to calculate in- place what cpu_util would be reporting were it to exist. This is what my verification was based on but admittedly it was not clear in my SRU template - I thought the metric's deprecation would be sufficient justification for an SRU but now understand that the impact needs to be the functional issue(s). Once I hear back from the customer I'll rewrite the template. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Note that this wasn't a complete review - I'm struggling to do that without the context I would get from understanding the problem that you're wanting to fix. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
I see a technical analysis of the change that you're wanting to make under "Impact", but no actual explanation of an issue with an impact to users. For example, why is it that watcher's use of the deprecated ceilometer metric, cpu_util, which reported cpu utilization as a percentage, is a problem for users? Why would they want to change this behaviour in a stable release? Generally, whether some behaviour is "deprecated" or not in a stable Ubuntu release is not relevant, because we commit to maintaining that behaviour until the release EOLs. So why does it matter here? And assuming I'm correct in understanding that you want to change user behaviour, how will that affect users relying on the previous behaviour, and why is that OK? Or if that is not possible, then please explain how it is not possible. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
I've updated the description based on the SRU bug template (my apologies I was not aware of the guideline). Also, thank you for informing me of the procedure when the most recent changelog entry is UNRELEASED. Let me know if any further changes are required or if you have any questions. Thank you -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
** Description changed: - Watcher uses cpu_util wich was deprecated. - The `cpu_util` metric was deprecated and replaced with raw CPU time measurements stored in Gnocchi. + [ Impact ] - There is an upstream fix: - https://review.opendev.org/c/openstack/watcher/+/898791 + * The watcher releases targeted by this SRU are using a deprecated + ceilometer metric, cpu_util, which reported cpu utilization as a + percentage. This metric was deprecated in Openstack Rocky in favor of + the Gnocchi rate calculation equivalent [1]. + + * Upstream Watcher continued to use cpu_util until the commit at [2] + landed on master for 2024.1. This commit correctly performs the cpu + calculation and removes the deprecated metric. The calculation is + summarized in the next bullet point and there is an example calculation + in the original commit + + * The gnocchi calculation uses the cumulative cpu time in ns (reported + by the cpu metric), taken as a rate (the difference in cumulative time + over the last two sampling intervals) to find the total cpu time during + the previous sampling period. Dividing the cpu time in one interval by + the duration of the interval multiplied by the number of vcpus provides + the cpu utilization as a percentage: cpu_usage = [cpu_time / (period * + 10^9 * nvcpus)] * 100%. A sample calculation is provided in the original + commit message. + + * I cherry-picked to stable/2023.2 [3], but the other branches have + gone unmaintained + + [ Test Plan ] + + * Deploy openstack yoga on jammy with watcher and gnocchi services + + * Launch a server and take note of it's resource id. Then find the + gnocchi cpu metric associated with the instance + + * Create a watcher audit based on a goal that previously depended on instance cpu utilization. For example the workload_balance goal [4] + Ex. openstack optimize audit create -t CONTINUOUS -i 60 -g workload_balancing -s workload_balance --auto-trigger + Without the patch instance_cpu_usage appears as None in the audits. With the patch you can observe the correct cpu utilization percentage in the watcher-decision-engine.log + + * Wait for at least one sampling period to elapse and check + /var/log/watcher/watcher-decision-engine.log for entries showing + "instance_cpu_usage" - this is the cpu utilization as a percentage. + + * To verify the percentage with a manual calculation, run gnocchi + measure show --aggregation "rate:mean" and perform the + calculation instance_cpu_usage = 100*[ / (period * 10^9 * nvcpus) + using the cpu time from the corresponding sampling period + + [ What can go wrong ] + + * While this is replacing a deprecated methodology and metric and + should lead to improvements, any custom strategies relying on cpu_util + may be affected. + + [1] https://docs.openstack.org/releasenotes/ceilometer/rocky.html + [2] https://review.opendev.org/c/openstack/watcher/+/898791 + [3] https://review.opendev.org/c/openstack/watcher/+/934181 + [4] https://docs.openstack.org/watcher/rocky/strategies/workload_balance.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
missing sru information -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Hi Bryan Thanks for updating the patches - I've merged into the Git packaging repo and made the following adjustments to all of the changelog entries (yoga for example): diff --git a/debian/changelog b/debian/changelog index 8c5e82b..0722081 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,7 +1,12 @@ watcher (2:8.0.0-0ubuntu1.1) UNRELEASED; urgency=medium + [ Corey Bryant ] * d/gbp.conf: Create stable/yoga branch. + [ Bryan Fraschetti ] + * d/p/handle-deprecated-cpu-util.patch: Apply upstream patch to replace +usage of the deprecated "cpu_util" metric (LP: #2088620). + -- Corey Bryant Mon, 25 Apr 2022 10:10:44 -0400 When the most recent changelog entry is UNRELEASED you should add you changelog update as part of the existing entry rather than creating a new one. Use of 'dch' should do this for you. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
** Changed in: cloud-archive/zed Status: New => Won't Fix ** Changed in: watcher (Ubuntu) Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Hi @james-page Thanks for taking a look at the patch. I added headers to the patch file, with the upstream commit message as the description, and referenced the bug in the changelog. I also removed the tox constraints patch from the bobcat diff. Let me know if any further changes are required -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Yoga debdiff with DEP-3 headers added to the patch file, and Launchpad bug referenced in the changelog ** Patch added: "lp2088620_yoga.debdiff" https://bugs.launchpad.net/ubuntu/+source/watcher/+bug/2088620/+attachment/5843326/+files/lp2088620_yoga.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Zed debdiff with DEP-3 headers added to the patch file, and Launchpad bug referenced in the changelog ** Patch added: "lp2088620_zed.debdiff" https://bugs.launchpad.net/ubuntu/+source/watcher/+bug/2088620/+attachment/5843299/+files/lp2088620_zed.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Antelope patch with DEP-3 headers added to the patch file, and Launchpad bug referenced in the changelog ** Patch added: "lp2088620_antelope.debdiff" https://bugs.launchpad.net/ubuntu/+source/watcher/+bug/2088620/+attachment/5843298/+files/lp2088620_antelope.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
** Patch added: "Bobcat debdiff with the tox constraints patch removed, DEP-3 headers added to the patch file, and Launchpad bug referenced in the changelog" https://bugs.launchpad.net/ubuntu/+source/watcher/+bug/2088620/+attachment/5843297/+files/lp2088620_bobcat.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
@bryanfrashetti Please can you either use the commit from upstream including the patch headers or add you own headers according to https://dep- team.pages.debian.net/deps/dep3/ Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
A few more comments: - update-tox-ini-constraints.patch is not needed for the distro so please drop. - please reference this bug in the changelog entry using (LP: #2088620). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: watcher (Ubuntu Jammy) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: watcher (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: watcher (Ubuntu Focal) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
** No longer affects: watcher (Ubuntu Noble) ** Also affects: watcher (Ubuntu Noble) Importance: Undecided Status: New ** Changed in: watcher (Ubuntu Noble) Status: New => Fix Released ** Changed in: watcher (Ubuntu Oracular) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
The attachment "lp2088620_bobcat.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team. [This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.] ** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Unsigned debdiff for bobcat that resolves the issue. It contains two patches The first patch corresponds to the commit here: https://opendev.org/openstack/watcher/commit/c3640ed64e6e11438c62e412c17f8b0e174c4db4, which updates the url from which tox pulls its constraints. This is necessary to ensure the passing of the tests which are required by debuild. The second directly resolves the issue and is cherry-picked from https://opendev.org/openstack/watcher/commit/40e93407c77c227b5c9159462d75b7015116aa9f ** Patch added: "lp2088620_bobcat.debdiff" https://bugs.launchpad.net/ubuntu/+source/watcher/+bug/2088620/+attachment/5838253/+files/lp2088620_bobcat.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Unsigned debdiff for yoga that resolves the issue. It contains one patch which can be seen upstream: https://opendev.org/openstack/watcher/commit/40e93407c77c227b5c9159462d75b7015116aa9f ** Patch added: "lp2088620_yoga.debdiff" https://bugs.launchpad.net/ubuntu/+source/watcher/+bug/2088620/+attachment/5838265/+files/lp2088620_yoga.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Unsigned debdiff for antelope that resolves the issue. It contains one patch which can be seen upstream: https://opendev.org/openstack/watcher/commit/40e93407c77c227b5c9159462d75b7015116aa9f ** Patch added: "lp2088620_antelope.debdiff" https://bugs.launchpad.net/ubuntu/+source/watcher/+bug/2088620/+attachment/5838254/+files/lp2088620_antelope.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
Unsigned debdiff for zed that resolves the issue. It contains one patch which can be seen upstream: https://opendev.org/openstack/watcher/commit/40e93407c77c227b5c9159462d75b7015116aa9f ** Patch added: "lp2088620_zed.debdiff" https://bugs.launchpad.net/ubuntu/+source/watcher/+bug/2088620/+attachment/5838264/+files/lp2088620_zed.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
** Tags added: sts -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2088620] Re: [SRU] Deprecated usage of cpu_util
** Summary changed: - Deprecated usage of cpu_util + [SRU] Deprecated usage of cpu_util -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2088620 Title: [SRU] Deprecated usage of cpu_util To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
