This bug is missing log files that will aid in diagnosing the problem.
While running an Ubuntu kernel (not a mainline or third-party kernel)
please enter the following command in a terminal window:
apport-collect 1875884
and then change the status of the bug to 'Confirmed'.
If, due to the nature of the issue you have encountered, you are unable
to run this command, please add a comment stating that fact and change
the bug status to 'Confirmed'.
This change has been made by an automated script, maintained by the
Ubuntu Kernel Team.
** Changed in: linux (Ubuntu)
Status: New => Incomplete
** Changed in: linux (Ubuntu Bionic)
Status: New => Incomplete
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1875884
Title:
Kernel log flood "ceph: Failed to find inode for 1"
Status in linux package in Ubuntu:
Incomplete
Status in linux-azure-4.15 package in Ubuntu:
New
Status in linux source package in Bionic:
Incomplete
Status in linux-azure-4.15 source package in Bionic:
New
Bug description:
OS provided by AKS is currently Ubuntu 16.04.6 LTS, kernel
4.15.0-1077-azure.
Every block written by a k8s pod to a ceph CSI volume generates 2 warning
lines in the node's system logs (kern.log, syslog, messages, warn):
"Apr 24 09:37:46 aks-<nodename> kernel: [242123.654538] ceph: Failed to find
inode for 1"
Under production load, eventually the node succumbs to DiskPressure as
the drive fills up. Also performance is noticeably degraded.
Background here: https://tracker.ceph.com/issues/45283
Luis Hernandez indicates 4 commits relating to this issue, just 2 of which
have been backported to Ubuntu 16.
d557c48db730 ("ceph: quota: add counter for snaprealms with quota") <==
e3161f17d926 ("ceph: quota: cache inode pointer in ceph_snap_realm")
0eb6bbe4d9cf ("ceph: fix root quota realm check") <==
2596366907f8 ("ceph: don't check quota for snap inode")
Quoth Luis:
"I've done a quick test and, after compiling the bionic kernel 4.15.0-96.97
(the latest released), I can reproduce the issue. Cherry-picking the 2 missing
commits (2596366907f8 and e3161f17d926) fixes it."
In my testing Ubuntu 18 does not exhibit the bug, but Azure support
tells me it will be months before they make it GA in AKS.
Can we get those commits backported to Ubuntu 16?
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-azure (not installed)
ProcVersionSignature: Ubuntu 4.15.0-1077.82-azure 4.15.18
Uname: Linux 4.15.0-1077-azure x86_64
ApportVersion: 2.20.1-0ubuntu2.23
Architecture: amd64
Date: Wed Apr 29 12:45:12 2020
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
SourcePackage: linux-meta-azure
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1875884/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
