Re: [Crash-utility] [PATCH] arm64: exclude mapping symbols in modules

[AKASHI Takahiro]

On Tue, Oct 11, 2016 at 10:57:37AM +0900, AKASHI Takahiro wrote:
> Dave,
> 
> On Fri, Oct 07, 2016 at 11:02:01AM -0400, Dave Anderson wrote:
> > 
> > - Original Message -
> > > Dave,
> > > 
> > > > 
> > > > Now, this sample patch doesn't deal with branch instructions other than 
> > > > "bl",
> > > > so perhaps it could just check whether the last argument in the 
> > > > instruction
> > > > line is a translatable address.
> > > > 
> > > > On the other hand, for the PLT veneer issue, it would have to:
> > > > 
> > > >  (1) make sure it's a "bl", and
> > > 
> > > and other variants of "bl"
> > 
> > Specifically what other variants?  Do you mean any instruction that begins
> > with "b."?
> 
> I double-checked and found out that R_AARCH64_CALL26 and R_AARCH64_JUMP are
> the only elf relocation types for which PLT veneers will be generated at
> module loading.
> That is, "b" and "bl," but "b" is unlikely.
> 
> > > 
> > > >  (2) instead of blindly doing a translation of the PLT veneer label 
> > > > address,
> > > >  it would first have to check whether it points to a 12-byte chunk 
> > > > of
> > > >  kernel address construction, and if so, translate the reconstructed
> > > >  address.
> > > 
> > > Actually, a veneer always consists of 4 instructions:
> > > mov  x16, #imm16
> > > movk x16, #imm16, lsl #16
> > > movk x16, #imm16, lsl #32
> > > br   x16
> > 
> > Right, I meant that the target address is constructed in the first 12 bytes.
> > 
> > I'm not at all familiar with arm64 assembly.  It seems that each of the
> > instructions consume 4 bytes, but unlike the other architectures, I cannot
> > find any documentation as to how the instruction type, the target register,
> > the immediate value, etc., actually get encoded into the 32-bit instruction.
> > The documentation shows the assembly mnemonics themselves, but not how the
> > instruction is actually laid out it in memory.  Maybe I'm looking in the 
> > wrong
> > place.  
> 
> Well, formally, you should consult, what is called, "ARM ARM(Architecture
> Reference Manual," but practically, you can find all the information that
> you need in arch/arm64/kernel/module-plts.c.
> 
> > Taking the simplest of examples, here's a mov immediate instruction:
> > 
> >   crash> dis 0xfe0fbc84 2
> >   0xfe0fbc84 :   mov x7, 
> > #0x // #-1
> >   0xfe0fbc88 :   add x0, x0, x26
> >   crash>
> > 
> > And here's the encoding:
> > 
> >   crash> rd -32 0xfe0fbc84
> >   fe0fbc84:  9287  
> >   crash> 
> > 
> > Presumably the 7 is the register field, but how does it get -1 out of the 
> > rest
> > of the instruction?
> 
> Haha, mov is not mov, but movn (inverted immediate).
> the inverse of bit[20:5] will be stored in x7.
> 
> > Anwyay, without some basic understanding, I'm not touching this.  I was 
> > kind 
> > of hoping you could whip up the function...  ;-)
> 
> I hope so if I have time this week.

Please take a look at my RFC. Disassembled code looks like:
crash> mod -S ./
 MODULE   NAME  SIZE  OBJECT FILE
04d78f4b8000  testmod  16384  ./testmod.ko 
crash> dis testmod_init
0x04d78f4b6000 :   stp x29, x30, [sp,#-16]!
0x04d78f4b6004 :mov x29, sp
0x04d78f4b6008 :ldr x0, 0x04d78f4b6018
0x04d78f4b600c :   bl  0x04d78f4b6090 
0x04d78f4b6010 :   ldr x0, 0x04d78f4b6020
0x04d78f4b6014 :   bl  0x04d78f4b6080 

Thanks,
-Takahiro AKASHI

> > > It would be safe to identify any veneers with this type of sequence,
> > > but I'm wondering if there is any other trick of directly checking
> > > if the label address is fit in PLT section of a module.
> > 
> > I have no idea.
> > 
> > > (On arm64, this section is dynamically allocated on module loading,
> > > and so it's not trivial.)
> > > 
> > > >  
> > > > So I'm thinking something along these lines, say, where "value" may or 
> > > > may
> > > > not be modified by your new function:
> > > > 
> > > >if (IS_MODULE_VADDR(vaddr)) {
> > > >p1 = [strlen(inbuf)-1];
> > > >strcpy(buf1, inbuf);
> > > >argc = parse_line(buf1, argv);
> > > >if (STREQ(argv[argc-2], "bl") &&
> > > >extract_hex(argv[argc-1], , NULLCHAR, TRUE)) {
> > > > +  value = PLT_veneer_to_kvaddr(value);
> > > >sprintf(p1, " <%s>\n",
> > > >value_to_symstr(value, buf2, 
> > > > output_radix));
> > > >}
> > > >}
> > > 
> > > Looks nice.
> > > 
> > > > However, another thing to consider is what "dis" shows if the "mod" 
> > > > command
> > > > has already loaded the debuginfo data.  In that case, I'm guessing that 
> > > > gdb
> > > 

Re: [Crash-utility] [PATCH] arm64: exclude mapping symbols in modules

[AKASHI Takahiro]

Dave,

On Fri, Oct 07, 2016 at 11:02:01AM -0400, Dave Anderson wrote:
> 
> - Original Message -
> > Dave,
> > 
> > > 
> > > Now, this sample patch doesn't deal with branch instructions other than 
> > > "bl",
> > > so perhaps it could just check whether the last argument in the 
> > > instruction
> > > line is a translatable address.
> > > 
> > > On the other hand, for the PLT veneer issue, it would have to:
> > > 
> > >  (1) make sure it's a "bl", and
> > 
> > and other variants of "bl"
> 
> Specifically what other variants?  Do you mean any instruction that begins
> with "b."?

I double-checked and found out that R_AARCH64_CALL26 and R_AARCH64_JUMP are
the only elf relocation types for which PLT veneers will be generated at
module loading.
That is, "b" and "bl," but "b" is unlikely.

> > 
> > >  (2) instead of blindly doing a translation of the PLT veneer label 
> > > address,
> > >  it would first have to check whether it points to a 12-byte chunk of
> > >  kernel address construction, and if so, translate the reconstructed
> > >  address.
> > 
> > Actually, a veneer always consists of 4 instructions:
> > mov  x16, #imm16
> > movk x16, #imm16, lsl #16
> > movk x16, #imm16, lsl #32
> > br   x16
> 
> Right, I meant that the target address is constructed in the first 12 bytes.
> 
> I'm not at all familiar with arm64 assembly.  It seems that each of the
> instructions consume 4 bytes, but unlike the other architectures, I cannot
> find any documentation as to how the instruction type, the target register,
> the immediate value, etc., actually get encoded into the 32-bit instruction.
> The documentation shows the assembly mnemonics themselves, but not how the
> instruction is actually laid out it in memory.  Maybe I'm looking in the wrong
> place.  

Well, formally, you should consult, what is called, "ARM ARM(Architecture
Reference Manual," but practically, you can find all the information that
you need in arch/arm64/kernel/module-plts.c.

> Taking the simplest of examples, here's a mov immediate instruction:
> 
>   crash> dis 0xfe0fbc84 2
>   0xfe0fbc84 :   mov x7, 
> #0x // #-1
>   0xfe0fbc88 :   add x0, x0, x26
>   crash>
> 
> And here's the encoding:
> 
>   crash> rd -32 0xfe0fbc84
>   fe0fbc84:  9287  
>   crash> 
> 
> Presumably the 7 is the register field, but how does it get -1 out of the rest
> of the instruction?

Haha, mov is not mov, but movn (inverted immediate).
the inverse of bit[20:5] will be stored in x7.

> Anwyay, without some basic understanding, I'm not touching this.  I was kind 
> of hoping you could whip up the function...  ;-)

I hope so if I have time this week.

> > It would be safe to identify any veneers with this type of sequence,
> > but I'm wondering if there is any other trick of directly checking
> > if the label address is fit in PLT section of a module.
> 
> I have no idea.
> 
> > (On arm64, this section is dynamically allocated on module loading,
> > and so it's not trivial.)
> > 
> > >  
> > > So I'm thinking something along these lines, say, where "value" may or may
> > > not be modified by your new function:
> > > 
> > >if (IS_MODULE_VADDR(vaddr)) {
> > >p1 = [strlen(inbuf)-1];
> > >strcpy(buf1, inbuf);
> > >argc = parse_line(buf1, argv);
> > >if (STREQ(argv[argc-2], "bl") &&
> > >extract_hex(argv[argc-1], , NULLCHAR, TRUE)) {
> > > +  value = PLT_veneer_to_kvaddr(value);
> > >sprintf(p1, " <%s>\n",
> > >value_to_symstr(value, buf2, 
> > > output_radix));
> > >  }
> > >}
> > 
> > Looks nice.
> > 
> > > However, another thing to consider is what "dis" shows if the "mod" 
> > > command
> > > has already loaded the debuginfo data.  In that case, I'm guessing that 
> > > gdb
> > > would translate the address of the PLT veneer location?
> > 
> > Give that the output from "bt" command shows "testmod_init" which is
> > a module_init function of my sample module, I assume that the debug
> > data have already been loaded in my case.
> 
> No, definitely not.  When a crash session is initiated, it kicks off the
> gdb session with "gdb vmlinux", and so the embedded gdb has no clue about
> the existence of any kernel modules.  The kernel data itself may contain
> basic symbol information that was exported by the modules if the kernel was
> configured with CONFIG_KALLSYMS, and if so, the "bt" command can translate
> module symbols.  On the other hand, the "dis" command issues a disassembly
> request to the embedded gdb module, which has no clue about module symbols
> unless the debuginfo data of the modules is added.  To do that, you have to
> enter either "mod -S" to load the debuginfo of all modules, or "mod 

Re: [Crash-utility] [PATCH] arm64: exclude mapping symbols in modules

[Dave Anderson]

- Original Message -
> Dave,
> 
> > 
> > Now, this sample patch doesn't deal with branch instructions other than 
> > "bl",
> > so perhaps it could just check whether the last argument in the instruction
> > line is a translatable address.
> > 
> > On the other hand, for the PLT veneer issue, it would have to:
> > 
> >  (1) make sure it's a "bl", and
> 
> and other variants of "bl"

Specifically what other variants?  Do you mean any instruction that begins
with "b."?

> 
> >  (2) instead of blindly doing a translation of the PLT veneer label address,
> >  it would first have to check whether it points to a 12-byte chunk of
> >  kernel address construction, and if so, translate the reconstructed
> >  address.
> 
> Actually, a veneer always consists of 4 instructions:
> mov  x16, #imm16
> movk x16, #imm16, lsl #16
> movk x16, #imm16, lsl #32
> br   x16

Right, I meant that the target address is constructed in the first 12 bytes.

I'm not at all familiar with arm64 assembly.  It seems that each of the
instructions consume 4 bytes, but unlike the other architectures, I cannot
find any documentation as to how the instruction type, the target register,
the immediate value, etc., actually get encoded into the 32-bit instruction.
The documentation shows the assembly mnemonics themselves, but not how the
instruction is actually laid out it in memory.  Maybe I'm looking in the wrong
place.  

Taking the simplest of examples, here's a mov immediate instruction:

  crash> dis 0xfe0fbc84 2
  0xfe0fbc84 :   mov x7, 
#0x // #-1
  0xfe0fbc88 :   add x0, x0, x26
  crash>

And here's the encoding:

  crash> rd -32 0xfe0fbc84
  fe0fbc84:  9287  
  crash> 

Presumably the 7 is the register field, but how does it get -1 out of the rest
of the instruction?

Anwyay, without some basic understanding, I'm not touching this.  I was kind 
of hoping you could whip up the function...  ;-)
 
> It would be safe to identify any veneers with this type of sequence,
> but I'm wondering if there is any other trick of directly checking
> if the label address is fit in PLT section of a module.

I have no idea.

> (On arm64, this section is dynamically allocated on module loading,
> and so it's not trivial.)
> 
> >  
> > So I'm thinking something along these lines, say, where "value" may or may
> > not be modified by your new function:
> > 
> >if (IS_MODULE_VADDR(vaddr)) {
> >p1 = [strlen(inbuf)-1];
> >strcpy(buf1, inbuf);
> >argc = parse_line(buf1, argv);
> >if (STREQ(argv[argc-2], "bl") &&
> >extract_hex(argv[argc-1], , NULLCHAR, TRUE)) {
> > +  value = PLT_veneer_to_kvaddr(value);
> >sprintf(p1, " <%s>\n",
> >value_to_symstr(value, buf2, output_radix));
> >}
> >}
> 
> Looks nice.
> 
> > However, another thing to consider is what "dis" shows if the "mod" command
> > has already loaded the debuginfo data.  In that case, I'm guessing that gdb
> > would translate the address of the PLT veneer location?
> 
> Give that the output from "bt" command shows "testmod_init" which is
> a module_init function of my sample module, I assume that the debug
> data have already been loaded in my case.

No, definitely not.  When a crash session is initiated, it kicks off the
gdb session with "gdb vmlinux", and so the embedded gdb has no clue about
the existence of any kernel modules.  The kernel data itself may contain
basic symbol information that was exported by the modules if the kernel was
configured with CONFIG_KALLSYMS, and if so, the "bt" command can translate
module symbols.  On the other hand, the "dis" command issues a disassembly
request to the embedded gdb module, which has no clue about module symbols
unless the debuginfo data of the modules is added.  To do that, you have to
enter either "mod -S" to load the debuginfo of all modules, or "mod -s "
to load the debuginfo data of an individual module.  The "mod [-sS]" command
runs a gdb "add-symbol-file" command behind the scenes for each module, and
therefore requires that the kernel's debuginfo package is available on the 
host system.  

Anyway, that being the case, I'm still wondering whether the gdb output would
simply show the veneer address after the debuginfo data is loaded with the mod
command.  I presume that it would do so, I mean that's what it's supposed 
to do.  This veneer translation would simply be a nice-to-have feature. 
 
> > The sample KASLR vmcore you gave me doesn't have any modules, so I don't 
> > know.
> 
> I can give you my sample vmcore.
> Please tell me a location where I can push the iamge.

Do you have debuginfo objects for the modules?  I really need to see the
before-and-after-mod-command 

Re: [Crash-utility] [PATCH] arm64: exclude mapping symbols in modules

[AKASHI Takahiro]

Dave,

On Thu, Oct 06, 2016 at 04:35:42PM -0400, Dave Anderson wrote:
> 
> Hi Akashi,
> 
> I was playing around with this, and noted that if a module's debuginfo data 
> is not
> loaded into a crash session with the "mod" command, branch instruction 
> targets 
> that are within the module space are not translated.  For example, note the 
> handful
> of "bl" instructions with module address targets are empty:
>   
>   crash> dis dm_create | grep bl
>   0xfdfffc003814 :  bl  0xfe226ce0 
> 
>   0xfdfffc003828 :  bl  0xfe143754 
> 
>   0xfdfffc003850 : bl  0xfe3c4f08 
>   0xfdfffc00385c : bl  0xfe78241c 
> <_raw_spin_lock>
>   0xfdfffc003874 : bl  0xfe3c5d90 
>   0xfdfffc00388c : bl  0xfe123044 
> 
>   0xfdfffc0038b0 : bl  0xfe10f9d0 
> <__mutex_init>
>   0xfdfffc0038c0 : bl  0xfe10f9d0 
> <__mutex_init>
>   0xfdfffc0038d4 : bl  0xfe10f9d0 
> <__mutex_init>
>   0xfdfffc003918 : bl  0xfe3968c4 
> 
>   0xfdfffc003948 : bl  0xfe3ab298 
>   0xfdfffc003968 : bl  0xfe10a878 
> <__init_waitqueue_head>
>   0xfdfffc003994 : bl  0xfe10a878 
> <__init_waitqueue_head>
>   0xfdfffc0039a8 : bl  0xfe10a878 
> <__init_waitqueue_head>
>   0xfdfffc0039f8 : bl  0xfe3d1224 
>   0xfdfffc003a00 : bl  0xfe3aab08 
>   0xfdfffc003a1c : bl  0xfe3d1224 
>   0xfdfffc003a34 : bl  0xfe0e3acc 
> <__alloc_workqueue_key>
>   0xfdfffc003a48 : bl  0xfe3a9a8c 
>   0xfdfffc003a58 : bl  0xfe38ea84 
>   0xfdfffc003a70 : bl  0xfdfffc00e418
>   0xfdfffc003a78 : bl  0xfe78241c 
> <_raw_spin_lock>
>   0xfdfffc003a88 : bl  0xfe3c4e54 
>   0xfdfffc003aa4 : bl  0xfdfffc00d270
>   0xfdfffc003ad8 : bl  0xfe143860 
>   0xfdfffc003ae0 : bl  0xfe228464 
>   0xfdfffc003b08 : bl  0xfe3c4f08 
>   0xfdfffc003b14 : bl  0xfe78241c 
> <_raw_spin_lock>
>   0xfdfffc003b2c : bl  0xfe3c5d90 
>   0xfdfffc003b4c : bl  0xfdfffc001220
>   0xfdfffc003b54 : bl  0xfdfffc0012e0
>   0xfdfffc003b60 : bl  0xfe1c4828 
>   crash> 
>   
> With this patch: 
> 
>   --- a/arm64.c
>   +++ b/arm64.c
>   @@ -2977,6 +2977,16 @@ arm64_dis_filter(ulong vaddr, char *inbuf, unsigned 
> int output_radix)
>   sprintf(p1, "%s", buf1);
>   }
>
>   +   if (IS_MODULE_VADDR(vaddr)) {
>   +   p1 = [strlen(inbuf)-1];
>   +   strcpy(buf1, inbuf);
>   +   argc = parse_line(buf1, argv);
>   +   if (STREQ(argv[argc-2], "bl") &&
>   +   extract_hex(argv[argc-1], , NULLCHAR, TRUE))
>   +   sprintf(p1, " <%s>\n",
>   +   value_to_symstr(value, buf2, output_radix));
>   +   }
>   +
>   console("%s", inbuf);
>
>   return TRUE;
> 
> module addresses are translated without having to load the module's debuginfo:
> 
>   crash> dis dm_create | grep bl
>   0xfdfffc003814 :  bl  0xfe226ce0 
> 
>   0xfdfffc003828 :  bl  0xfe143754 
> 
>   0xfdfffc003850 : bl  0xfe3c4f08 
>   0xfdfffc00385c : bl  0xfe78241c 
> <_raw_spin_lock>
>   0xfdfffc003874 : bl  0xfe3c5d90 
>   0xfdfffc00388c : bl  0xfe123044 
> 
>   0xfdfffc0038b0 : bl  0xfe10f9d0 
> <__mutex_init>
>   0xfdfffc0038c0 : bl  0xfe10f9d0 
> <__mutex_init>
>   0xfdfffc0038d4 : bl  0xfe10f9d0 
> <__mutex_init>
>   0xfdfffc003918 : bl  0xfe3968c4 
> 
>   0xfdfffc003948 : bl  0xfe3ab298 
>   0xfdfffc003968 : bl  0xfe10a878 
> <__init_waitqueue_head>
>   0xfdfffc003994 : bl  0xfe10a878 
> <__init_waitqueue_head>
>   0xfdfffc0039a8 : bl  0xfe10a878 
> <__init_waitqueue_head>
>   0xfdfffc0039f8 : bl  0xfe3d1224 
>   0xfdfffc003a00 : bl  0xfe3aab08 
>   

Re: [Crash-utility] [PATCH] arm64: exclude mapping symbols in modules

[Dave Anderson]

Hi Akashi,

I was playing around with this, and noted that if a module's debuginfo data is 
not
loaded into a crash session with the "mod" command, branch instruction targets 
that are within the module space are not translated.  For example, note the 
handful
of "bl" instructions with module address targets are empty:
  
  crash> dis dm_create | grep bl
  0xfdfffc003814 :bl  0xfe226ce0 

  0xfdfffc003828 :bl  0xfe143754 

  0xfdfffc003850 :   bl  0xfe3c4f08 
  0xfdfffc00385c :   bl  0xfe78241c 
<_raw_spin_lock>
  0xfdfffc003874 :   bl  0xfe3c5d90 
  0xfdfffc00388c :   bl  0xfe123044 

  0xfdfffc0038b0 :   bl  0xfe10f9d0 
<__mutex_init>
  0xfdfffc0038c0 :   bl  0xfe10f9d0 
<__mutex_init>
  0xfdfffc0038d4 :   bl  0xfe10f9d0 
<__mutex_init>
  0xfdfffc003918 :   bl  0xfe3968c4 

  0xfdfffc003948 :   bl  0xfe3ab298 
  0xfdfffc003968 :   bl  0xfe10a878 
<__init_waitqueue_head>
  0xfdfffc003994 :   bl  0xfe10a878 
<__init_waitqueue_head>
  0xfdfffc0039a8 :   bl  0xfe10a878 
<__init_waitqueue_head>
  0xfdfffc0039f8 :   bl  0xfe3d1224 
  0xfdfffc003a00 :   bl  0xfe3aab08 
  0xfdfffc003a1c :   bl  0xfe3d1224 
  0xfdfffc003a34 :   bl  0xfe0e3acc 
<__alloc_workqueue_key>
  0xfdfffc003a48 :   bl  0xfe3a9a8c 
  0xfdfffc003a58 :   bl  0xfe38ea84 
  0xfdfffc003a70 :   bl  0xfdfffc00e418
  0xfdfffc003a78 :   bl  0xfe78241c 
<_raw_spin_lock>
  0xfdfffc003a88 :   bl  0xfe3c4e54 
  0xfdfffc003aa4 :   bl  0xfdfffc00d270
  0xfdfffc003ad8 :   bl  0xfe143860 
  0xfdfffc003ae0 :   bl  0xfe228464 
  0xfdfffc003b08 :   bl  0xfe3c4f08 
  0xfdfffc003b14 :   bl  0xfe78241c 
<_raw_spin_lock>
  0xfdfffc003b2c :   bl  0xfe3c5d90 
  0xfdfffc003b4c :   bl  0xfdfffc001220
  0xfdfffc003b54 :   bl  0xfdfffc0012e0
  0xfdfffc003b60 :   bl  0xfe1c4828 
  crash> 
  
With this patch: 

  --- a/arm64.c
  +++ b/arm64.c
  @@ -2977,6 +2977,16 @@ arm64_dis_filter(ulong vaddr, char *inbuf, unsigned 
int output_radix)
  sprintf(p1, "%s", buf1);
  }
   
  +   if (IS_MODULE_VADDR(vaddr)) {
  +   p1 = [strlen(inbuf)-1];
  +   strcpy(buf1, inbuf);
  +   argc = parse_line(buf1, argv);
  +   if (STREQ(argv[argc-2], "bl") &&
  +   extract_hex(argv[argc-1], , NULLCHAR, TRUE))
  +   sprintf(p1, " <%s>\n",
  +   value_to_symstr(value, buf2, output_radix));
  +   }
  +
  console("%s", inbuf);
   
  return TRUE;

module addresses are translated without having to load the module's debuginfo:

  crash> dis dm_create | grep bl
  0xfdfffc003814 :bl  0xfe226ce0 

  0xfdfffc003828 :bl  0xfe143754 

  0xfdfffc003850 :   bl  0xfe3c4f08 
  0xfdfffc00385c :   bl  0xfe78241c 
<_raw_spin_lock>
  0xfdfffc003874 :   bl  0xfe3c5d90 
  0xfdfffc00388c :   bl  0xfe123044 

  0xfdfffc0038b0 :   bl  0xfe10f9d0 
<__mutex_init>
  0xfdfffc0038c0 :   bl  0xfe10f9d0 
<__mutex_init>
  0xfdfffc0038d4 :   bl  0xfe10f9d0 
<__mutex_init>
  0xfdfffc003918 :   bl  0xfe3968c4 

  0xfdfffc003948 :   bl  0xfe3ab298 
  0xfdfffc003968 :   bl  0xfe10a878 
<__init_waitqueue_head>
  0xfdfffc003994 :   bl  0xfe10a878 
<__init_waitqueue_head>
  0xfdfffc0039a8 :   bl  0xfe10a878 
<__init_waitqueue_head>
  0xfdfffc0039f8 :   bl  0xfe3d1224 
  0xfdfffc003a00 :   bl  0xfe3aab08 
  0xfdfffc003a1c :   bl  0xfe3d1224 
  0xfdfffc003a34 :   bl  0xfe0e3acc 
<__alloc_workqueue_key>
  0xfdfffc003a48 :   bl  

Re: [Crash-utility] [PATCH] arm64: exclude mapping symbols in modules

[Dave Anderson]

- Original Message -
> Dave,
> 
> One question.
>
... 
>
> On arm64, when KASLR is enabled, a function call between a module and
> the kernel will be done via a veneer (PLT) if the displacement is more
> than +/-128MB. So disassembled code looks a bit useless:
> 
> ===8<===
> crash> mod -S
>  MODULE   NAME  SIZE  OBJECT FILE
> 04d78f4b8000  testmod  16384  
> /opt/buildroot/15.11_64/root/kexec/testmod.ko
> crash> bt
> PID: 1102   TASK: b4da8e91  CPU: 0   COMMAND: "insmod"
>  #0 [b4da8e9afa30] __crash_kexec at 0e0045020a54
>  #1 [b4da8e9afb90] panic at 0e004505523c
>  #2 [b4da8e9afc50] testmod_init at 04d78f4b6014 [testmod]
>  #3 [b4da8e9afb40] do_one_initcall at 0e0044f7333c
> ---  ---
>  PC: 000a  LR:   SP: 04d78f4b6000  PSTATE: 
> 7669726420656c75
> X12: b4da8e9ac000 X11: 04d78f4b6018 X10: b4da8e9afc50  X9: 
> 20676e6973756143
>  X8:   X7: 0e0045e5ce00  X6: 0e0045e5c000  X5: 61c5
>  X4: 0e0045020a58X3: b4da8e9afa30  X2: 
> 0e004502098c  X1: b4da8e9afa30
>  X0: 0124
> crash> dis testmod_init
> 0x04d78f4b6000 :   stp x29, x30, [sp,#-16]!
> 0x04d78f4b6004 :mov x29, sp
> 0x04d78f4b6008 :ldr x0, 0x04d78f4b6018
> 0x04d78f4b600c :   bl  0x04d78f4b6090
> 0x04d78f4b6010 :   ldr x0, 0x04d78f4b6020
> 0x04d78f4b6014 :   bl  0x04d78f4b6080
> 
>   => branch to a veneer
> crash> dis 0x04d78f4b6080 4
> 
> 0x04d78f4b6080 :   mov x16, #0x5120  // 
> #-44768
> 0x04d78f4b6084 :   movkx16, #0x4505, lsl #16
> 0x04d78f4b6088 :   movkx16, #0xe00, lsl #32
> 0x04d78f4b608c :  br  x16
> 
>   => branch to 0x0e0045055120
>(= panic())
> ===>8===
> 
> Is there any method to resolve such kind of indirect addressing
> to a symbolic name at dis command?
> (It may be difficult to discriminate PLT from normal branches, though).

Maybe something could be kludged up by the machdep->dis_filter() call?

In arm64_dis_filter(), whenever there is a "bl " instruction,
the 12 bytes at the PLT target address could be read, parsed, and the
real target address reconstructed.  If the reconstructed address resolves
to a kernel text address, it could be appended to the line.

Dave


 

> Thanks,
> -Takahiro AKASHI
>

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Re: [Crash-utility] [PATCH] arm64: exclude mapping symbols in modules

[AKASHI Takahiro]

Dave,

One question.

On Tue, Oct 04, 2016 at 11:00:16AM -0400, Dave Anderson wrote:
> 
> 
> - Original Message -
> > If some module has been inserted, crash util complains about
> > module symbols:
> > ...
> > please wait... (gathering module symbol data)
> > crash: store_module_symbols_v2: total: 15 mcnt: 16
> > and end up with stopping.
> > 
> > This patch excludes mapping symbols, like $d and $x, as arm does.

On arm64, when KASLR is enabled, a function call between a module and
the kernel will be done via a veneer (PLT) if the displacement is more
than +/-128MB. So disassembled code looks a bit useless:

===8<===
crash> mod -S
 MODULE   NAME  SIZE  OBJECT FILE
04d78f4b8000  testmod  16384  /opt/buildroot/15.11_64/root/kexec/testmod.ko 
crash> bt  
PID: 1102   TASK: b4da8e91  CPU: 0   COMMAND: "insmod"
 #0 [b4da8e9afa30] __crash_kexec at 0e0045020a54
 #1 [b4da8e9afb90] panic at 0e004505523c
 #2 [b4da8e9afc50] testmod_init at 04d78f4b6014 [testmod]
 #3 [b4da8e9afb40] do_one_initcall at 0e0044f7333c
---  ---
 PC: 000a  LR:   SP: 04d78f4b6000  PSTATE: 7669726420656c75
X12: b4da8e9ac000 X11: 04d78f4b6018 X10: b4da8e9afc50  X9: 
20676e6973756143
 X8:   X7: 0e0045e5ce00  X6: 0e0045e5c000  X5: 61c5
 X4: 0e0045020a58  X3: b4da8e9afa30  X2: 0e004502098c  X1: 
b4da8e9afa30
 X0: 0124
crash> dis testmod_init
0x04d78f4b6000 :   stp x29, x30, [sp,#-16]!
0x04d78f4b6004 :mov x29, sp
0x04d78f4b6008 :ldr x0, 0x04d78f4b6018
0x04d78f4b600c :   bl  0x04d78f4b6090
0x04d78f4b6010 :   ldr x0, 0x04d78f4b6020
0x04d78f4b6014 :   bl  0x04d78f4b6080

=> branch to a veneer
crash> dis 0x04d78f4b6080 4

0x04d78f4b6080 :   mov x16, #0x5120
// #-44768
0x04d78f4b6084 :   movkx16, #0x4505, lsl #16
0x04d78f4b6088 :   movkx16, #0xe00, lsl #32
0x04d78f4b608c :  br  x16

=> branch to 0x0e0045055120
 (= panic())
===>8===

Is there any method to resolve such kind of indirect addressing
to a symbolic name at dis command?
(It may be difficult to discriminate PLT from normal branches, though).

Thanks,
-Takahiro AKASHI


> > Signed-off-by: AKASHI Takahiro 
> 
> Queued for crash-7.1.6:
> 
>   
> https://github.com/crash-utility/crash/commit/9a5cbfe998060a8f7b1e463353abcfcf1fce
> 
> Thanks,
>   Dave
> 
>   
> > ---
> >  symbols.c | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/symbols.c b/symbols.c
> > index 99c8b8b..a657ead 100644
> > --- a/symbols.c
> > +++ b/symbols.c
> > @@ -2251,10 +2251,11 @@ store_module_kallsyms_v2(struct load_module *lm, int
> > start, int curr,
> > continue;
> >  
> > /*
> > -* On ARM we have linker mapping symbols like '$a' and '$d'.
> > +* On ARM/ARM64 we have linker mapping symbols like '$a'
> > +* or '$x' for ARM64, and '$d'.
> >  * Make sure that these don't end up into our symbol list.
> >  */
> > -   if (machine_type("ARM") &&
> > +   if ((machine_type("ARM") || machine_type("ARM64")) &&
> > !machdep->verify_symbol(nameptr, ec->st_value, ec->st_info))
> > continue;
> >  
> > --
> > 2.10.0
> > 
> > --
> > Crash-utility mailing list
> > Crash-utility@redhat.com
> > https://www.redhat.com/mailman/listinfo/crash-utility
> > 
> 
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Re: [Crash-utility] [PATCH] arm64: exclude mapping symbols in modules

[Dave Anderson]

- Original Message -
> If some module has been inserted, crash util complains about
> module symbols:
> ...
> please wait... (gathering module symbol data)
> crash: store_module_symbols_v2: total: 15 mcnt: 16
> and end up with stopping.
> 
> This patch excludes mapping symbols, like $d and $x, as arm does.
> 
> Signed-off-by: AKASHI Takahiro 

Queued for crash-7.1.6:

  
https://github.com/crash-utility/crash/commit/9a5cbfe998060a8f7b1e463353abcfcf1fce

Thanks,
  Dave

  
> ---
>  symbols.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/symbols.c b/symbols.c
> index 99c8b8b..a657ead 100644
> --- a/symbols.c
> +++ b/symbols.c
> @@ -2251,10 +2251,11 @@ store_module_kallsyms_v2(struct load_module *lm, int
> start, int curr,
>   continue;
>  
>   /*
> -  * On ARM we have linker mapping symbols like '$a' and '$d'.
> +  * On ARM/ARM64 we have linker mapping symbols like '$a'
> +  * or '$x' for ARM64, and '$d'.
>* Make sure that these don't end up into our symbol list.
>*/
> - if (machine_type("ARM") &&
> + if ((machine_type("ARM") || machine_type("ARM64")) &&
>   !machdep->verify_symbol(nameptr, ec->st_value, ec->st_info))
>   continue;
>  
> --
> 2.10.0
> 
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
> 

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility