[Bug 2099283] Re: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter
Test plan now looks much better, and good to see a well thought out section on where problems could occur. I don't see a merge for questing, and I note questing has the same version as plucky, so all that's missing at this point is for this change to make it into questing (the current devel) so that SRUs can start being sponsored. I note the change is merged in GitHub. Is the plan to simply upload a new release in questing? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2099283 Title: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter To manage notifications about this bug go to: https://bugs.launchpad.net/landscape-client/+bug/2099283/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2099283] Re: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter
** Also affects: landscape-client (Ubuntu Questing) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2099283 Title: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter To manage notifications about this bug go to: https://bugs.launchpad.net/landscape-client/+bug/2099283/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2099283] Re: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter
** Also affects: landscape-client (Ubuntu Oracular) Importance: Undecided Status: New ** Also affects: landscape-client (Ubuntu Noble) Importance: Undecided Status: New ** Also affects: landscape-client (Ubuntu Jammy) Importance: Undecided Status: New ** Also affects: landscape-client (Ubuntu Plucky) Importance: Undecided Status: New ** Also affects: landscape-client (Ubuntu Focal) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2099283 Title: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter To manage notifications about this bug go to: https://bugs.launchpad.net/landscape-client/+bug/2099283/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2099283] Re: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter
Targeted to Focal..Devel, as per bug title. The tests mentioned in comment #4 landed in Plucky and are therefore unblocked. But this is not yet included in "ubuntu/devel" (Plucky), which is now in Final Freeze. So SRUs are blocked on getting it landed in devel first. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2099283 Title: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter To manage notifications about this bug go to: https://bugs.launchpad.net/landscape-client/+bug/2099283/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2099283] Re: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter
** Description changed:
[ Impact ]
* The current implementation of `landscape-package-reporter` will use a
lot more CPU% than needed by creating `Origin` objects from the Python
`apt` package when computing changes in packages. This has been a
complaint from users as Landscape Client can use 100% of the CPU when
package-reporter is computing the changes to report back to Landscape
Server.
[ Test Plan ]
* Install `python3-psutil` and the proposed version of Landscape
Client.
* Replace this line in `/usr/lib/python3/dist-
packages/landscape/client/package/reporter.py`...:
* def _compute_packages_changes(self): # noqa: max-complexity: 13
* ...with the following to apply the benchmarking to the function which
computes changes in packages to report back to Landscape Server:
def _compute_packages_changes(self): # noqa: max-complexity: 13
import cProfile
import pstats
from datetime import datetime
import psutil
profile = cProfile.Profile()
process = psutil.Process()
start_cpu_times = process.cpu_times()
profile.enable()
result = self.compute_packages_change_inner()
end_cpu_times = process.cpu_times()
profile.disable()
user_time = end_cpu_times.user - start_cpu_times.user
system_time = end_cpu_times.system - start_cpu_times.system
total_cpu_time = user_time + system_time
output_path = "/var/lib/landscape/client/result.txt"
with open(output_path, "a") as fp:
now = datetime.now()
fp.write(f"\n- Run on: {now.strftime('%Y-%m-%d
%H:%M:%S')} -\n\n")
stats = pstats.Stats(profile, stream=fp)
stats.strip_dirs().sort_stats("cumulative").print_stats(10)
fp.write(f"CPU Time: {total_cpu_time}s\n")
return result
def compute_packages_change_inner(self):
* Now register the client with Landscape Server and wait for package
reporter to run and check the benchmarking results for the proposed
version:
* less /var/lib/landscape/client/result.txt
* Make sure that package-reporter can report package information to the
server properly:
* Install a package on the client machine manually and ensure it
shows up under the instance packages in the Landscape Server UI.
* Search for the package in Landscape Server and mark it for
removal. Wait for the activity to run and ensure the package was
removed.
* Search for a package that should be available to install for the
- instance (ex. devscripts) in the UI and install it to the client
- instance via Landscape Server.
+ instance (ex. hello) in the UI and install it to the client instance via
+ Landscape Server.
* Update all possible packages on the client instance using the old
Landscape Server UI ("Advanced options" - "You can request that all
possible upgrades be applied to the selected computers."). You can also
manually upgrade all packages that have an update.
* In a fresh instance, install the current version of Landscape Client
from the main Ubuntu repo for that series and add the benchmarking like
before.
* sudo apt update && sudo apt install -y landscape-client
python3-psutil
* Connect this client to the same server and wait for package reporter
to run, and check the benchmarking results for the current (unmodified)
version of Landscape Client:
* less /var/lib/landscape/client/result.txt
* The cpu times reported by the proposed version should be
significantly less than those of the current version. With these
changes, this benchmark ran about 11x faster on Jammy and 15x faster on
Focal in my testing. See my full observations in the Other Info section.
- * Repeat the steps under "Make sure that package-reporter can report
+ * Repeat the steps under "Make sure that package-reporter can report
package information to the server properly" for the unmodified client.
[ Where problems could occur ]
* This approach tweaks how package changes are computed, the
accuracy/reliability could be compromised. Further, this could lead to
issues with Landscape Client reporting package information too
frequently or at all.
* This proposed fix gets the archives directly from each package
versions rather than getting thing information from the Origin class of
the Python apt package. However, the point of this is to check if the
given package version matches the format of the security or backport
archives, so it's possible that this change could lead to package
version candidates being incorrectly skipped/not skipped.
[ Other Info ]
* Benchmarking observations:
https://docs.google.com/spreadsheets/d/1AOGhskitsj5XbgJGg0Ab-
EcfG4GxcB8wjRnWX0wMefE
* Raw paste for the resul
[Bug 2099283] Re: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter
** Changed in: landscape-client (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2099283 Title: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter To manage notifications about this bug go to: https://bugs.launchpad.net/landscape-client/+bug/2099283/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2099283] Re: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter
I've created the MPs for all the series :) However as a note, the unit tests for Landscape Client need to be fixed by https://code.launchpad.net/~mitchburton/ubuntu/+source/landscape- client/+git/landscape-client/+merge/483505 so the Plucky MP for this fix must wait for that to be merged. ** Summary changed: - Update Focal, Jammy, Noble, Oracular to reduce CPU usage of landscape-package-reporter + Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter ** Merge proposal linked: https://code.launchpad.net/~jansdhillon/ubuntu/+source/landscape-client/+git/landscape-client/+merge/483695 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2099283 Title: Update Focal, Jammy, Noble, Oracular, Plucky to reduce CPU usage of landscape-package-reporter To manage notifications about this bug go to: https://bugs.launchpad.net/landscape-client/+bug/2099283/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
