The verification for makedumpfile used the vmcore file provided
by another user instead of /proc/vmcore (which is identical, as
it's a simple 'cp' copy of /proc/vmcore, per makedumpfile error.)
$ ls -lh /home/ubuntu/201909170743/vmcore.201909170743
-r-------- 1 ubuntu ubuntu 32G Sep 17 2019
/home/ubuntu/201909170743/vmcore.201909170743
$ file /home/ubuntu/201909170743/vmcore.201909170743
/home/ubuntu/201909170743/vmcore.201909170743: ELF 64-bit LSB core file
ARM aarch64, version 1 (SYSV), SVR4-style
The reproducer system is an arm64 guest in our internal openstack
cloud with the same kernel version as the user (4.15.0-76-generic.)
$ grep -ao -m1 'Linux version .* '
/home/ubuntu/201909170743/vmcore.201909170743
Linux version 4.15.0-76-generic (buildd@bos02-arm64-060) (gcc version
7.4.0 (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1)) #86-Ubuntu SMP Fri Jan 17
17:25:58 UTC 2020 (Ubuntu 4.15.0-76.86-generic
$ uname -mrv
4.15.0-76-generic #86-Ubuntu SMP Fri Jan 17 17:25:58 UTC 2020 aarch64
When using the original package version, makedumpfile fails with
error messages about a particular address, then kdump-tools (the
caller of makedumpfile) falls back to 'cp', as reported.
This takes a long time since it's a 32 GB file.
So, since the second step / invocation of makedumpfile, to store
the dmesg output on vmcore (makedumpfile --dump-dmesg), fails in
the same way (and for the very same particular address), that is,
an equivalent failure / symptom of the same root cause, then use
only that step, which runs fast regardless of failure or success.
These are the changes done to /usr/sbin/kdump-config (shell script
independent of makedumpfile binary/executable code.)
# Constants
#vmcore_file=/proc/vmcore
vmcore_file=/home/ubuntu/201909170743/vmcore.201909170743
...
function kdump_save_core()
...
log_action_msg "running makedumpfile $MAKEDUMP_ARGS
$vmcore_file $KDUMP_CORETEMP"
#makedumpfile $MAKEDUMP_ARGS $vmcore_file $KDUMP_CORETEMP
#ERROR=$?
ERROR=0
if [ $ERROR -ne 0 ] ; then
log_failure_msg "$NAME: makedumpfile failed, falling
back to 'cp'"
...
For documentation purposes,
With the user's vmcore, and still collecting the crashdump (i.e.,
first invocation of makedumpfile), the exact error reproduces:
$ dpkg -s makedumpfile | grep -i version
Version: 1:1.6.5-1ubuntu1~18.04.4
$ echo 1 | sudo tee /proc/sys/kernel/sysrq && echo c | sudo tee
/proc/sysrq-trigger
...
[ 222.162389] sysrq: SysRq : Trigger a crash
...
[ 222.185756] Call trace:
[ 222.186091] sysrq_handle_crash+0x24/0x30
[ 222.186628] __handle_sysrq+0xbc/0x1c0
[ 222.187128] write_sysrq_trigger+0xb8/0x120
[ 222.187690] proc_reg_write+0x80/0xc0
[ 222.188182] __vfs_write+0x48/0x80
[ 222.188639] vfs_write+0xac/0x1b0
[ 222.189148] SyS_write+0x74/0xf0
[ 222.189585] el0_svc_naked+0x30/0x34
[ 222.190073] Code: 52800020 b90ca020 d5033e9f d2800001 (39000020)
[ 222.190892] SMP: stopping secondary CPUs
[ 222.193873] Starting crashdump kernel...
[ 222.194414] Bye!
...
[ 8.168635] kdump-tools[516]: Starting kdump-tools: * running
makedumpfile -c -d 31 /home/ubuntu/201909170743/vmcore.201909170743
/var/crash/202006171229/dump-incomplete
[ 8.185786] kdump-tools[516]: readmem: Can't convert a virtual
address(6a2) to physical address.
[ 8.187727] kdump-tools[516]: readmem: type_addr: 0, addr:6a2,
size:1032
[ 8.191012] kdump-tools[516]: validate_mem_section: Can't read
mem_section array.
[ 8.195497] kdump-tools[516]: get_mem_section: Could not validate
mem_section.
[ 8.197581] kdump-tools[516]: get_mm_sparsemem: Can't get the
address of mem_section.
[ 8.199601] kdump-tools[516]: makedumpfile Failed.
[ 8.204346] kdump-tools[516]: * kdump-tools: makedumpfile failed,
falling back to 'cp'
And when skipping that, and just collecting the dmesg output,
the same error reproduces:
[ 8.369266] kdump-tools[513]: Starting kdump-tools: * running
makedumpfile -c -d 31 /home/ubuntu/201909170743/vmcore.201909170743
/var/crash/202006171242/dump-incomplete
[ 8.382379] kdump-tools[513]: mv: cannot stat
'/var/crash/202006171242/dump-incomplete': No such file or directory
[ 8.385529] kdump-tools[513]: * kdump-tools: saved vmcore in
/var/crash/202006171242
[ 8.405479] kdump-tools[513]: * running makedumpfile --dump-dmesg
/home/ubuntu/201909170743/vmcore.201909170743
/var/crash/202006171242/dmesg.202006171242
[ 8.422223] kdump-tools[513]: readmem: Can't convert a virtual
address(6a2) to physical address.
[ 8.424872] kdump-tools[513]: readmem: type_addr: 0, addr:6a2,
size:1032
[ 8.428291] kdump-tools[513]: validate_mem_section: Can't read
mem_section array.
[ 8.429779] kdump-tools[513]: get_mem_section: Could not validate
mem_section.
[ 8.432257] kdump-tools[513]: get_mm_sparsemem: Can't get the
address of mem_section.
[ 8.436297] kdump-tools[513]: makedumpfile Failed.
[ 8.439351] kdump-tools[513]: * kdump-tools: makedumpfile
--dump-dmesg failed. dmesg content will be unavailable
[ 8.442197] kdump-tools[513]: * kdump-tools: failed to save dmesg
content in /var/crash/202006171242
[ 8.448570] kdump-tools[513]: Wed, 17 Jun 2020 12:42:16 +0000
[ 8.455630] kdump-tools[513]: Rebooting.
[ 8.514560] reboot: Restarting system
So, this shows the failures seen on the first and second
invocations of makedumpfile are equivalent, and we can
use just the second (--dump-dmesg) which runs faster,
for the purposes of verifying this SRU.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1869465
Title:
Kdump-Tools: Makedumpfile Failed, Falling Back To 'Cp'
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/makedumpfile/+bug/1869465/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
