Re: [Crash-utility] [PATCH][v2] Remove __exception_text_start and __exception_text_end for ARM64

2020-01-06 Thread Dave Anderson


- Original Message -
> > Hi Prabhakar,
> >
> > On Mon, Dec 30, 2019 at 9:55 AM Prabhakar Kushwaha
> >  wrote:
> >>
> >> Dear Bhupesh,
> >>
> >> On Tue, Dec 24, 2019 at 9:39 AM Prabhakar Kushwaha
> >>  wrote:
> >> >
> >> > On Tue, Dec 24, 2019 at 1:09 AM Bhupesh Sharma 
> >> wrote:
> >> > >
> >> > > On Mon, Dec 23, 2019 at 7:49 PM Prabhakar Kushwaha
> >> > >  wrote:
> >> > > >
> >> > > > On Mon, Dec 23, 2019 at 7:32 PM Dave Anderson
> >>  wrote:
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > - Original Message -
> >> > > > > > __exception_text_start and __exception_text_end is used to
> >> group functions
> >> > > > > > and place according to linker script in such a way to achieve
> >> > > > > > kprobe blacklist. Linux commit b6e43c0e3129 ("arm64: remove
> >> __exception
> >> > > > > > annotations") has removed __exception_text_start and
> >> > > > > > __exception_text_end and uses NOKPROBE_SYMBOL() for blacklist
> >> kprobes.
> >> > > > > >
> >> > > > > > So removing references of __exception_text_start and
> >> __exception_text_end
> >> > > > > > for ARM64.
> >> > > > >
> >> > > > > NAK for a couple of reasons...
> >> > > > >
> >> > > > > First, they cannot be removed for backward-compatibility
> >> purposes, and secondly
> >> > > > > an alternative method is required for arm64_back_trace_cmd() for
> >> handling
> >> > > > > exception frames.
> >> > > > >
> >> > > >
> >> > > > We are getting following error with crash tool with latest kernel.
> >> > > > crash: cannot resolve "__exception_text_start" error
> >> > > >
> >> > > > it is because of Linux commit b6e43c0e3129 which removes
> >> > > > __exception_text_start and __exception_text_end.
> >> > > > We need to find alternate way of fixing it.
> >> > >
> >> > > I cannot seem to remember reproducing this issue with crash with
> >> > > latest kernel on my arm64 boards.
> >> > > Can you be more specific on the exact kernel version and the crash
> >> > > command you used, just to be sure we are able to reproduce the same?
> >> > >
> >> >
> >> > We are seeing this issue on Thuder X platform (arm64).
> >> >
> >> > Please find details of commit id.
> >> > A) crash tool: https://github.com/crash-utility/crash.git
> >> > commit:  af7f78dc501b8acf7fee3f924f69e93513d0a74b (Fix for the "log
> >> -a" option.)
> >> >
> >> > B) Linux:
> >> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> >> > commit:  46cf053efec6a3a5f343fead83efe8252a46 (Linux 5.5-rc3)
> >> >
> >> > C) Kexec:
> >> https://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git
> >> > commit: bd077966e2b9041c24df5b689e67670f02be7b0d (kexec-tools: Fix
> >> > conversion overflow when compiling on 32-bit platforms)
> >> >
> >> >
> >> > Logs
> >> > root@ubuntu$ ./crash /proc/vmcore /usr/src/tovards/linux/vmlinux
> >> >
> >> > crash 7.2.7++
> >> > Copyright (C) 2002-2019  Red Hat, Inc.
> >> > Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
> >> > Copyright (C) 1999-2006  Hewlett-Packard Co
> >> > Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
> >> > Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
> >> > Copyright (C) 2005, 2011  NEC Corporation
> >> > Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
> >> > Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
> >> > This program is free software, covered by the GNU General Public
> >> License,
> >> > and you are welcome to change it and/or distribute copies of it under
> >> > certain conditions.  Enter "help copying" to see the conditions.
> >> > This program has absolutely no warranty.  Enter "help warranty" for
> >> details.
> >> >
> >> > vmcoreinfo : vabits_actual: 48
> >> > GNU gdb (GDB) 7.6
> >> > Copyright (C) 2013 Free Software Foundation, Inc.
> >> > License GPLv3+: GNU GPL version 3 or later
> >> 
> >> > This is free software: you are free to change and redistribute it.
> >> > There is NO WARRANTY, to the extent permitted by law.  Type "show
> >> copying"
> >> > and "show warranty" for details.
> >> > This GDB was configured as "aarch64-unknown-linux-gnu".
> >> > For bug reporting instructions, please see:
> >> > ...
> >> >
> >> > crash: cannot resolve "__exception_text_start"
> >> >
> >>
> >> i hope you got sometime to reproduce this issue.
> >> It can also be seen on Linux 5.5-rc4 tag
> >> (fd6988496e79a6a4bdb514a4655d2920209eb85d).
> >>
> >> crash-util  commit 5e975dd8c817ea6aea35e1e15b83c378aee9c136 (HEAD ->
> >> master, origin/master, origin/HEAD)
> >
> > I was able to reproduce the issue with latest Linus's tree. I am
> > working on a patch (as per Dave's suggestion) to fix backward
> > compatibility as well as 'arm64_back_trace_cmd()' (bt command support)
> > and will post it out soon (I am hoping to post out a version after the
> > new year break).
> >
> > Thanks  and a Happy New Year,
> > Bhupesh
> >
> 
> Thanks Bhupesh, I'm also out until next week.
> 
> And yeah, for 5.5 and later 

Re: [Crash-utility] crash on arm64

2020-01-06 Thread Bhupesh Sharma
Hello Poonam,

I see Dave is already helping you with the issue. I saw a couple of
interesting things and have a suggestion regarding the same (see
in-line):

On Mon, Jan 6, 2020 at 6:45 PM Poonam Aggrwal  wrote:
>
> Hello
>
> I am getting the following error  when I use crash on arm64 platform (NXP 
> LS1043A, with 4 A53 cores).
>   crash: read error: kernel virtual address: 7b616100  type: "memory 
> section root table"
>
> Additional information is that when I boot the kernel with nokaslr in 
> bootargs, the crash seems to work well.
>
> Have any one else seen similar issue?
>
> The kernel version is top of mainline with the below patch applied:
> - arm64/crash_core: Export TCR_EL1.T1SZ in vmcoreinfo 
> (http://lists.infradead.org/pipermail/kexec/2019-November/023962.html)

I am not sure you need this patch, since you are using the 48-bit and
4K page size configuration, see below..

> And the crash-utils is from here: https://github.com/crash-utility/crash.git
> commit 5e975dd8c817ea6aea35e1e15b83c378aee9c136
> Author: Dave Anderson 
> Date:   Tue Dec 24 08:43:52 2019 -0500
>
> When determining the ARM64 kernel's "vabits_actual" value by reading
> the new TCR_EL1.T1SZ vmcoreinfo entry, display its value during
> session initialization only when invoking crash with "-d1" or larger
> -d debug value.
> (ander...@redhat.com)
>
> Logs of crash -d1 are below:
> nxa19049@lsv03080:~/data/ups/crash$ ./crash -d1 ../linux/vmlinux 
> ../linux/vmcore
>
> crash 7.2.7++
> Copyright (C) 2002-2019  Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
> Copyright (C) 1999-2006  Hewlett-Packard Co
> Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
> Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
> Copyright (C) 2005, 2011  NEC Corporation
> Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions.  Enter "help copying" to see the conditions.
> This program has absolutely no warranty.  Enter "help warranty" for details.
>
> vmcore_data:
>   flags: c0 (KDUMP_LOCAL|KDUMP_ELF64)
>ndfd: 3
> ofp: 7fa9998cf620
> header_size: 8192
>num_pt_load_segments: 4
>  pt_load_segment[0]:
> file_offset: 2000
>  phys_start: 8208
>phys_end: 839bc000
>   zero_fill: 0
>  pt_load_segment[1]:
> file_offset: 193e000
>  phys_start: 8000
>phys_end: d800
>   zero_fill: 0
>  pt_load_segment[2]:
> file_offset: 5993e000
>  phys_start: fb00
>phys_end: fb80
>   zero_fill: 0
>  pt_load_segment[3]:
> file_offset: 5a13e000
>  phys_start: fbc0
>phys_end: fbe0
>   zero_fill: 0
>  elf_header: 2724410
>   elf32: 0
> notes32: 0
>  load32: 0
>   elf64: 2724410
> notes64: 2724450
>  load64: 2724488
>sect0_64: 0
> nt_prstatus: 2725410
> nt_prpsinfo: 0
>   nt_taskstruct: 0
> task_struct: 0
>  arch_data1: (unused)
>  arch_data2: (unused)
>switch_stack: 0
>   page_size: 4096
>  xen_kdump_data: (unused)
>  num_prstatus_notes: 4
>  num_qemu_notes: 0
>  vmcoreinfo: 2725a98
> size_vmcoreinfo: 1939
>  nt_prstatus_percpu:
> 02725410 027255ac 02725748 027258e4
>  nt_qemu_percpu:
>backup_src_start: 0
> backup_src_size: 0
>   backup_offset: 0
>
> Elf64_Ehdr:
> e_ident: \177ELF
>   e_ident[EI_CLASS]: 2 (ELFCLASS64)
>e_ident[EI_DATA]: 1 (ELFDATA2LSB)
> e_ident[EI_VERSION]: 1 (EV_CURRENT)
>   e_ident[EI_OSABI]: 0 (ELFOSABI_SYSV)
> e_ident[EI_ABIVERSION]: 0
>  e_type: 4 (ET_CORE)
>   e_machine: 183 (EM_AARCH64)
>   e_version: 1 (EV_CURRENT)
> e_entry: 0
> e_phoff: 40
> e_shoff: 0
> e_flags: 0
>e_ehsize: 40
> e_phentsize: 38
> e_phnum: 5
> e_shentsize: 0
> e_shnum: 0
>  e_shstrndx: 0
> Elf64_Phdr:
>  p_type: 4 (PT_NOTE)
>p_offset: 4096 (1000)
> p_vaddr: 0
> p_paddr: 0
>p_filesz: 3612 (e1c)
> p_memsz: 3612 (e1c)
> p_flags: 0 ()
> p_align: 0
> Elf64_Phdr:
>  p_type: 1 (PT_LOAD)
>  

[Crash-utility] crash on arm64

2020-01-06 Thread Poonam Aggrwal
Hello 

I am getting the following error  when I use crash on arm64 platform (NXP 
LS1043A, with 4 A53 cores).
  crash: read error: kernel virtual address: 7b616100  type: "memory 
section root table"

Additional information is that when I boot the kernel with nokaslr in bootargs, 
the crash seems to work well.

Have any one else seen similar issue?

The kernel version is top of mainline with the below patch applied:
- arm64/crash_core: Export TCR_EL1.T1SZ in vmcoreinfo 
(http://lists.infradead.org/pipermail/kexec/2019-November/023962.html)

And the crash-utils is from here: https://github.com/crash-utility/crash.git
commit 5e975dd8c817ea6aea35e1e15b83c378aee9c136
Author: Dave Anderson 
Date:   Tue Dec 24 08:43:52 2019 -0500

When determining the ARM64 kernel's "vabits_actual" value by reading
the new TCR_EL1.T1SZ vmcoreinfo entry, display its value during
session initialization only when invoking crash with "-d1" or larger
-d debug value.
(ander...@redhat.com)

Logs of crash -d1 are below:
nxa19049@lsv03080:~/data/ups/crash$ ./crash -d1 ../linux/vmlinux ../linux/vmcore

crash 7.2.7++
Copyright (C) 2002-2019  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

vmcore_data:
  flags: c0 (KDUMP_LOCAL|KDUMP_ELF64)
   ndfd: 3
    ofp: 7fa9998cf620
    header_size: 8192
   num_pt_load_segments: 4
 pt_load_segment[0]:
    file_offset: 2000
 phys_start: 8208
   phys_end: 839bc000
  zero_fill: 0
 pt_load_segment[1]:
    file_offset: 193e000
 phys_start: 8000
   phys_end: d800
  zero_fill: 0
 pt_load_segment[2]:
    file_offset: 5993e000
 phys_start: fb00
   phys_end: fb80
  zero_fill: 0
 pt_load_segment[3]:
    file_offset: 5a13e000
 phys_start: fbc0
   phys_end: fbe0
  zero_fill: 0
 elf_header: 2724410
  elf32: 0
    notes32: 0
 load32: 0
  elf64: 2724410
    notes64: 2724450
 load64: 2724488
   sect0_64: 0
    nt_prstatus: 2725410
    nt_prpsinfo: 0
  nt_taskstruct: 0
    task_struct: 0
 arch_data1: (unused)
 arch_data2: (unused)
   switch_stack: 0
  page_size: 4096
 xen_kdump_data: (unused)
 num_prstatus_notes: 4
 num_qemu_notes: 0
 vmcoreinfo: 2725a98
    size_vmcoreinfo: 1939
 nt_prstatus_percpu:
    02725410 027255ac 02725748 027258e4
 nt_qemu_percpu:
   backup_src_start: 0
    backup_src_size: 0
  backup_offset: 0

Elf64_Ehdr:
    e_ident: \177ELF
  e_ident[EI_CLASS]: 2 (ELFCLASS64)
   e_ident[EI_DATA]: 1 (ELFDATA2LSB)
    e_ident[EI_VERSION]: 1 (EV_CURRENT)
  e_ident[EI_OSABI]: 0 (ELFOSABI_SYSV)
e_ident[EI_ABIVERSION]: 0
 e_type: 4 (ET_CORE)
  e_machine: 183 (EM_AARCH64)
  e_version: 1 (EV_CURRENT)
    e_entry: 0
    e_phoff: 40
    e_shoff: 0
    e_flags: 0
   e_ehsize: 40
    e_phentsize: 38
    e_phnum: 5
    e_shentsize: 0
    e_shnum: 0
 e_shstrndx: 0
Elf64_Phdr:
 p_type: 4 (PT_NOTE)
   p_offset: 4096 (1000)
    p_vaddr: 0
    p_paddr: 0
   p_filesz: 3612 (e1c)
    p_memsz: 3612 (e1c)
    p_flags: 0 ()
    p_align: 0
Elf64_Phdr:
 p_type: 1 (PT_LOAD)
   p_offset: 8192 (2000)
    p_vaddr: d3ebfa48
    p_paddr: 8208
   p_filesz: 26460160 (193c000)
    p_memsz: 26460160 (193c000)
    p_flags: 7 (PF_X|PF_W|PF_R)
    p_align: 0
Elf64_Phdr:
 p_type: 1 (PT_LOAD)
   p_offset: 26468352 (193e000)
    p_vaddr: 
    p_paddr: 8000
   p_filesz: 1476395008 (5800)
    p_memsz: 1476395008 (5800)
    p_flags: 7 (PF_X|PF_W|PF_R)
   

Re: [Crash-utility] crash on arm64

2020-01-06 Thread Poonam Aggrwal
Thank you Dave

> -Original Message-
> From: Dave Anderson 
> Sent: Sunday, January 5, 2020 10:16 PM
> To: Poonam Aggrwal 
> Cc: crash-utility@redhat.com
> Subject: Re: crash on arm64
> 
> 
> 
> - Original Message -
> >
> >
> > - Original Message -
> > > Hello
> > >
> > > I am getting the following error  when I use crash on arm64 platform
> > > (NXP LS1043A, with 4 A53 cores).
> > >   crash: read error: kernel virtual address: 7b616100  type:
> > >   "memory section root table"
> > >
> > > Additional information is that when I boot the kernel with nokaslr
> > > in bootargs, the crash seems to work well.
> > >
> > > Have any one else seen similar issue?
> > >
> > > The kernel version is top of mainline with the below patch applied:
> > > - arm64/crash_core: Export TCR_EL1.T1SZ in vmcoreinfo
> > >
> (https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fli
> > > sts.infradead.org%2Fpipermail%2Fkexec%2F2019-
> November%2F023962.html&
> > >
> amp;data=02%7C01%7Cpoonam.aggrwal%40nxp.com%7C70d5eb3b36634
> e3b307008
> > >
> d791fece02%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6371
> 38395873
> > >
> 999176sdata=zWz%2Bzm%2BGRgCvhX%2FsmXiprBU7de6N3rriivVzn
> g72d4Y%3
> > > Dreserved=0)
> > >
> > > And the crash-utils is from here:
> > >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
> > > thub.com%2Fcrash-
> utility%2Fcrash.gitdata=02%7C01%7Cpoonam.aggrw
> > >
> al%40nxp.com%7C70d5eb3b36634e3b307008d791fece02%7C686ea1d3bc
> 2b4c6fa9
> > >
> 2cd99c5c301635%7C0%7C0%7C637138395873999176sdata=qlngXi
> kARralK%
> > > 2B9fTP5NiikXuq%2FrVEG5vGkMQibSKrI%3Dreserved=0
> > > commit 5e975dd8c817ea6aea35e1e15b83c378aee9c136
> > > Author: Dave Anderson 
> > > Date:   Tue Dec 24 08:43:52 2019 -0500
> > >
> > > When determining the ARM64 kernel's "vabits_actual" value by
> reading
> > > the new TCR_EL1.T1SZ vmcoreinfo entry, display its value during
> > > session initialization only when invoking crash with "-d1" or larger
> > > -d debug value.
> > > (ander...@redhat.com)
> >
> > What version of kexec-tools are you using?
It is 2.0.20
git://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git
commit bd077966e2b9041c24df5b689e67670f02be7b0d
Author: Helge Deller 
Date:   Fri Oct 4 13:01:09 2019 +0200

kexec-tools: Fix conversion overflow when compiling on 32-bit platforms

When compiling kexec-tools on a 32-bit platform, assigning an
(unsigned long long) value to an (unsigned long) variable creates
this warning:

> 
> Also -- how do things work on the live system, both with and without
> KASLR?
I have not tried this, I can give it a try.

Regards
Poonam
> 
> Dave
> 
> 
> >
> > Dave
> >
> > >
> > > Logs of crash -d1 are below:
> > > nxa19049@lsv03080:~/data/ups/crash$ ./crash -d1 ../linux/vmlinux
> > > ../linux/vmcore
> > >
> > > crash 7.2.7++
> > > Copyright (C) 2002-2019  Red Hat, Inc.
> > > Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation Copyright (C)
> > > 1999-2006  Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012
> > > Fujitsu Limited Copyright (C) 2006, 2007  VA Linux Systems Japan
> > > K.K.
> > > Copyright (C) 2005, 2011  NEC Corporation Copyright (C) 1999, 2002,
> > > 2007  Silicon Graphics, Inc.
> > > Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
> > > This program is free software, covered by the GNU General Public
> > > License, and you are welcome to change it and/or distribute copies
> > > of it under certain conditions.  Enter "help copying" to see the
> conditions.
> > > This program has absolutely no warranty.  Enter "help warranty" for
> > > details.
> > >
> > > vmcore_data:
> > >   flags: c0 (KDUMP_LOCAL|KDUMP_ELF64)
> > >    ndfd: 3
> > >     ofp: 7fa9998cf620
> > >     header_size: 8192
> > >    num_pt_load_segments: 4
> > >  pt_load_segment[0]:
> > >     file_offset: 2000
> > >  phys_start: 8208
> > >    phys_end: 839bc000
> > >   zero_fill: 0
> > >  pt_load_segment[1]:
> > >     file_offset: 193e000
> > >  phys_start: 8000
> > >    phys_end: d800
> > >   zero_fill: 0
> > >  pt_load_segment[2]:
> > >     file_offset: 5993e000
> > >  phys_start: fb00
> > >    phys_end: fb80
> > >   zero_fill: 0
> > >  pt_load_segment[3]:
> > >     file_offset: 5a13e000
> > >  phys_start: fbc0
> > >    phys_end: fbe0
> > >   zero_fill: 0
> > >  elf_header: 2724410
> > >   elf32: 0
> > >     notes32: 0
> > >  load32: 0
> > >   elf64: 2724410
> > >     notes64: 2724450
> > >  load64: 2724488
> > >    sect0_64: 0
> > >     nt_prstatus: 2725410
> > >     nt_prpsinfo: 0
> > >