[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-10-10 Thread Philip Roche
https://bugs.launchpad.net/cloud-images/+bug/2038894 is a related bug to track specifically the introduction of listening port 5353 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2032933

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-09-21 Thread Philip Roche
There is a related bug @ https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2036968 which might have affected boot speed. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2032933 Title:

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-09-21 Thread Philip Roche
@paelzer agreed. Good plan. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2032933 Title: Mantic (23.10) minimal images increase in memory consumption, port usage and processes

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-09-12 Thread Christian Ehrhardt 
@phil As I said above for me it mostly was "I'm sure some change was expected, but so much?". It has various benefits as gladly outlined by Dimitri, fixing many issues, but coming at a price tag. Seeing how big the price tag is for small size, high density cases I consider it potentially too

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-09-04 Thread Philip Roche
@paelzer given the above findings and discussion, I would like to mark this as Invalid for cloud-images project and continue the conversation in the context of kernel only. +1 / -1 ? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-09-04 Thread Thomas Bechtold
Ah. I see. KSM_ENABLED=AUTO from /etc/default/qemu-kvm is used through /usr/share/qemu/init/qemu-kvm-init via the qemu-kvm.service systemd service. thanks for the hint! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-09-04 Thread Christian Ehrhardt 
> it only gets activated when you install ksmtuned (which is not installed by default). No, installing qemu-system-... will also enable it. So Seth gladly filed bug 2033565 to discuss and change this now or at least towards 24.04. -- You received this bug notification because you are a member

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-30 Thread Philip Roche
** Description changed: - The Mantic (Ubuntu 23.10) images are undergoing some big changes prior - to 23.10 release in October. + The Mantic (Ubuntu 23.10) download/qcow2 images available @ https://cloud-images.ubuntu.com/minimal/ + are undergoing some big changes prior to 23.10 release in

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-30 Thread Thomas Bechtold
@Seth, ksm is disabled by default so it's still opt-in: # cat /sys/kernel/mm/ksm/run 0 it only gets activated when you install ksmtuned (which is not installed by default). So I think that's fine. -- You received this bug notification because you are a member of Kernel Packages, which is

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-30 Thread Christian Ehrhardt 
> @paelzer Are you happy to adjust your regression testing/metrics gathering to > increase the > memory required knowing that it was a conscious decision to switch kernel and > incur the > performance hit for the benefit of using a kernel with more support and less > reported bugs? I am. In

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-29 Thread Philip Roche
I have uploaded further data now to https://people.canonical.com/~philroche/20230824-manticl-minimal- LP2032933/server-metrics/ with kernelmodules, kernelconfig, services, timers etc. for each of the three images being inspected. This additional data was gathered with a modified fork of the

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-29 Thread Dimitri John Ledkov
Is lxd / zfs now installed by default in the minimal images? It seems like some zfs things are loaded by default. Also counting processes alone, may or may not increase memory usage. Example: having N agetty, doesn't actually have memory cost of each individual agetty, does it? -- You received

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-29 Thread Dimitri John Ledkov
Additionally if something is built into the kernel that is optional at runtime, and resource intensive, we can make it modularization. But not sure that will win us anything if it gets autoloaded by default. Thus maybe some of the things listed should also be blacklisted to prevent auto-loading on

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-29 Thread Dimitri John Ledkov
Separately, please do note that switch from kvm kernel to generic is resolving the hundreds of bug reports of "works with generic / every other cloud, doesn't work with kvm". Also note that probably the baseline comparison shouldn't be the kvm kernel => as we know kvm kernel has never provided

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-28 Thread Seth Arnold
> +ksmd I'm concerned about this change. Historically, the page-merging code has allowed cross-VM snooping, including even recovery of GnuPG private keys: https://eprint.iacr.org/2013/448.pdf Unless something has changed to mitigate the cross-domain privacy leaks in ksmd, it ought to be opt-in

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-28 Thread Philip Roche
@paelzer > The change of the image build sadly combined it all See the description noting https://people.canonical.com/~philroche/20230824-manticl-minimal- LP2032933/ which should help in determining where the changes were introduced as I have provided three images across the various stages of

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-28 Thread Philip Roche
The diff in process count from kernel change image -> kernel change + seed change image is actually a reduction in processes - see diff @ https://pastebin.ubuntu.com/p/PXtQM9gB2K/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-28 Thread Simon Déziel
In the above tests from Christian, it's interesting to note the MemTotal shrink by ~22MiB. Is this due to the higher NR_CPUS alone? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2032933

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-28 Thread Christian Ehrhardt 
IMHO: If it turns out unfixable, this at least needs to be a release notes entry -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2032933 Title: Mantic (23.10) minimal images increase in

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-28 Thread Christian Ehrhardt 
One thing that came up when discussing with SMB as an obvious "uses mem more in a kernel" are the structures needed per POSSIBLE cpu. Comparing the system that I downgraded I found from [1]: With the -kvm kernel before: kernel: setup_percpu: NR_CPUS:64 nr_cpumask_bits:1 nr_cpu_ids:1

[Kernel-packages] [Bug 2032933] Re: Mantic (23.10) minimal images increase in memory consumption, port usage and processes running

2023-08-28 Thread Christian Ehrhardt 
Checking kernels: 1. This already uses linux-image-virtual, it is not the even bigger linux-image-generic. 2. The change of the image build sadly combined it all a) new image build/seeding by CPC b) different kernel type -kvm -> -virtual that uses generic c) switch of kernel versions We