By removing kernel modules one by one I identified the culprit in
"irdma". By removing this module 5.2GiB of RAM is freed

This is the output of "free" without the irdma Kernel module:

free -h
              total        used        free      shared  buff/cache   available
Mem:           15Gi       289Mi        14Gi       1.0Mi       651Mi        14Gi

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1990292

Title:
  Mistery memory usage in linux Kernel >= 5.14

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  My hardware is SuperMicro X11SDV-8C-TP8F with Intel D-2146NT.
  I've tested this issue with:

  * Ubuntu 22.04 Kernel 5.15
  * Ubuntu 22.04 Kernel 5.17
  * Ubuntu 20.04 Kernel 5.15
  * Ubuntu 20.04 Kernel 5.14

  Using Ubuntu 20.04 with kernel 5.13 or lower will NOT trigger this
  issue.

  The issue is that I have high RAM usage from the server startup I
  cannot explain.

  At startup without any services active I have nearly 6GiB of RAM used:
  ```
  # free -m
                 total        used        free      shared  buff/cache   
available
  Mem:           15663        5998        8936         155         728        
9213
  Swap:              0           0           0
  ```

  Using ps the sum of RSS memory does not adds up: it shows less than 1GiB used:
  ```
  # ps aux | awk '{sum+=$6} END {print sum / 1024}'
  944.879
  ```

  This is the meminfo output
  ```
  cat /proc/meminfo 
  MemTotal:       16039044 kB
  MemFree:         8486372 kB
  MemAvailable:    9607120 kB
  Buffers:           79440 kB
  Cached:          1300092 kB
  SwapCached:            0 kB
  Active:           575856 kB
  Inactive:        1141004 kB
  Active(anon):      24184 kB
  Inactive(anon):   369940 kB
  Active(file):     551672 kB
  Inactive(file):   771064 kB
  Unevictable:       27884 kB
  Mlocked:           27884 kB
  SwapTotal:       2097148 kB
  SwapFree:        2097148 kB
  Dirty:                36 kB
  Writeback:             0 kB
  AnonPages:        365256 kB
  Mapped:           304856 kB
  Shmem:             50208 kB
  KReclaimable:     110308 kB
  Slab:             455680 kB
  SReclaimable:     110308 kB
  SUnreclaim:       345372 kB
  KernelStack:        7664 kB
  PageTables:         6352 kB
  NFS_Unstable:          0 kB
  Bounce:                0 kB
  WritebackTmp:          0 kB
  CommitLimit:    10116668 kB
  Committed_AS:    2497484 kB
  VmallocTotal:   34359738367 kB
  VmallocUsed:       31940 kB
  VmallocChunk:          0 kB
  Percpu:            12800 kB
  HardwareCorrupted:     0 kB
  AnonHugePages:      4096 kB
  ShmemHugePages:        0 kB
  ShmemPmdMapped:        0 kB
  FileHugePages:         0 kB
  FilePmdMapped:         0 kB
  HugePages_Total:       0
  HugePages_Free:        0
  HugePages_Rsvd:        0
  HugePages_Surp:        0
  Hugepagesize:       2048 kB
  Hugetlb:               0 kB
  DirectMap4k:      315700 kB
  DirectMap2M:    11919360 kB
  DirectMap1G:     6291456 kB
  ```
  And this is the `slabtop` header:
  ```
   Active / Total Objects (% used)    : 1095487 / 1172800 (93.4%)
   Active / Total Slabs (% used)      : 28258 / 28258 (100.0%)
   Active / Total Caches (% used)     : 121 / 176 (68.8%)
   Active / Total Size (% used)       : 401911.16K / 415423.19K (96.7%)
   Minimum / Average / Maximum Object : 0.01K / 0.35K / 10.62K
  ```

  This is the `df -h` output:
  ```
  df -h
  Filesystem                         Size  Used Avail Use% Mounted on
  tmpfs                              1.6G   50M  1.5G   4% /run
  /dev/mapper/ubuntu--vg-ubuntu--lv   14G  3.8G  9.0G  30% /
  tmpfs                              7.7G   20K  7.7G   1% /dev/shm
  tmpfs                              5.0M     0  5.0M   0% /run/lock
  /dev/sda2                          2.0G  127M  1.7G   7% /boot
  /dev/mapper/vg0-var_log            9.8G  127M  9.2G   2% /var/log
  /dev/mapper/vg0-tmp                9.8G   68K  9.3G   1% /tmp
  /dev/mapper/vg0-var_tmp            9.8G   96K  9.3G   1% /var/tmp
  /dev/mapper/vg0-var_lib_docker     1.7T   11G  1.6T   1% /var/lib/docker
  tmpfs                              1.6G     0  1.6G   0% /run/user/2010
  tmpfs                              1.6G     0  1.6G   0% /run/user/2023
  ```
  --- 
  ProblemType: Bug
  AlsaDevices:
   total 0
   crw-rw---- 1 root audio 116,  1 Sep 20 14:54 seq
   crw-rw---- 1 root audio 116, 33 Sep 20 14:54 timer
  AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
  ApportVersion: 2.20.11-0ubuntu82.1
  Architecture: amd64
  ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', 
'/dev/snd/timer'] failed with exit code 1:
  CRDA: N/A
  CasperMD5CheckResult: unknown
  DistroRelease: Ubuntu 22.04
  IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
  MachineType: Supermicro SYS-5019D-FN8TP
  Package: linux (not installed)
  PciMultimedia:
   
  ProcEnviron:
   TERM=xterm-256color
   PATH=(custom, no user)
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  ProcFB: 0 astdrmfb
  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.15.0-47-generic 
root=/dev/mapper/ubuntu--vg-ubuntu--lv ro fsck.repair=yes 
debian-installer/language=it keyboard-configuration/layoutcode?=it
  ProcVersionSignature: Ubuntu 5.15.0-47.51-generic 5.15.46
  RelatedPackageVersions:
   linux-restricted-modules-5.15.0-47-generic N/A
   linux-backports-modules-5.15.0-47-generic  N/A
   linux-firmware                             20220329.git681281e4-0ubuntu3.5
  RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
  Tags:  jammy
  Uname: Linux 5.15.0-47-generic x86_64
  UnreportableReason: This report is about a package that is not installed.
  UpgradeStatus: No upgrade log present (probably fresh install)
  UserGroups: N/A
  _MarkForUpload: False
  dmi.bios.date: 05/25/2021
  dmi.bios.release: 5.14
  dmi.bios.vendor: American Megatrends Inc.
  dmi.bios.version: 1.5
  dmi.board.asset.tag: To be filled by O.E.M.
  dmi.board.name: X11SDV-8C-TP8F
  dmi.board.vendor: Supermicro
  dmi.board.version: 1.10
  dmi.chassis.asset.tag: To be filled by O.E.M.
  dmi.chassis.type: 1
  dmi.chassis.vendor: Supermicro
  dmi.chassis.version: 0123456789
  dmi.modalias: 
dmi:bvnAmericanMegatrendsInc.:bvr1.5:bd05/25/2021:br5.14:svnSupermicro:pnSYS-5019D-FN8TP:pvr0123456789:rvnSupermicro:rnX11SDV-8C-TP8F:rvr1.10:cvnSupermicro:ct1:cvr0123456789:skuTobefilledbyO.E.M.:
  dmi.product.family: To be filled by O.E.M.
  dmi.product.name: SYS-5019D-FN8TP
  dmi.product.sku: To be filled by O.E.M.
  dmi.product.version: 0123456789
  dmi.sys.vendor: Supermicro

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990292/+subscriptions


-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to