Re: [Crash-utility] [营销邮件] Re: [营销邮件] Re: [External Mail][????] Re: ramdump support for va_bits_actual

2020-04-19 Thread 赵乾利 
Hi
vmemmap and VMEMMAP_START are different,vmemmap begin from physical address 0x0 
of page,but VMEMMAP_START start with memstart_addr.

[mm/init.c]
arm64_memblock_init
348 vmemmap = ((struct page *)VMEMMAP_START - (memstart_addr >> 
PAGE_SHIFT));

We can look at address_markers,this symbol indicates the entire memory map 
allocation
check vmemmap and address_markers, we can see vmemmap has out of vmemmap range
crash> p -x vmemmap
vmemmap = $1 = (struct page *) 0xfffefde0

address_markers = $2 =
 {{
  start_address = 0xff80,
name = 0xffd44b1fed8f "Linear Mapping start"
...
  }, {
start_address = 0xfffeffe0,
name = 0xffd44b2c5beb "vmemmap start"
  }, {
start_address = 0xffe0,
name = 0xffd44b1eb00c "vmemmap end"
  }, {
start_address = 0x,
name = 0x0
  }}


> When the readmem() of symbol_value("physvirt_offset") is made, arm64_VTOP() 
> will
> be called with its virtual address, right?
Yes,arm64_VTOP to get physvirt_offset needs virtual address,and physvirt_offset 
is a kimage symbol,so just need kimage_voffset can translate to physical 
address,then get value of physvirt_offset.
physvirt_offset just two value in different kernel 
version,"phys_offset-page_offset" or physvirt_offset,so after initialize 
ms->physvirt_offset to different value,we can use ms->physvirt_offset for 
vtop/ptov for Linner mapping.


From: crash-utility-boun...@redhat.com  on 
behalf of Dave Anderson 
Sent: Sunday, April 19, 2020 23:30
To: Discussion list for crash utility usage,maintenance and development
Subject: [营销邮件] Re: [Crash-utility] [营销邮件] Re:  [External Mail][] Re: 
ramdump support for   va_bits_actual

- Original Message -
> Hi,Dave
>
> I don't quite understand how you said to read vmemmap,VMEMMAP_START is a
> macro definition,or it's a constant,in crash-utility vmemmap_start  is a
> wrong value,this may cause arm64_IS_VMALLOC_ADDR return wrong status

I mean the kernel symbol "vmemmap", here in "arch/arm64/mm/init.c":

  struct page *vmemmap __ro_after_init;
  EXPORT_SYMBOL(vmemmap);

Doesn't it contain the resolved starting address?

> We can get physvirt_offset  earlier,in my patch,after calling
> arm64_calc_phys_offset we can initialize physvirt_offset,in this
> time,kimage_offset,page_offset and phys_offset are ready,such as:
>
> @@ -391,6 +391,13 @@ arm64_init(int when)
>
> ms = machdep->machspec;
>
> +   if (kernel_symbol_exists("physvirt_offset") &&
> +   readmem(symbol_value("physvirt_offset"), KVADDR,
> +   &value, sizeof(ulong), "physvirt_offset", 
> QUIET|RETURN_ON_ERROR))
> +   ms->physvirt_offset = value;
> +   else
> +   ms->physvirt_offset = ms->phys_offset -> 
> ms->page_offset;

When the readmem() of symbol_value("physvirt_offset") is made, arm64_VTOP() will
be called with its virtual address, right?

Dave



> 
> From: crash-utility-boun...@redhat.com  on
> behalf of Dave Anderson 
> Sent: Sunday, April 19, 2020 1:03
> To: Discussion list for crash utility usage,maintenance and development
> Subject: [营销邮件] Re: [Crash-utility] [External Mail][] Re: ramdump support
> for   va_bits_actual
>
> - Original Message -
> > Hi
> > I made almost the same patch to fix the problem with arm64 in version
> > 5.4...
> >
> > One very small change can merged together,vmemmap_start has a little error:
> > [arch/arm64/include/asm/memory.h]
> >  56 #define VMEMMAP_START   (-VMEMMAP_SIZE - SZ_2M)
> > in crash arm64.c
> > -   vmemmap_start = (-vmemmap_size);
> > +   vmemmap_start = (-vmemmap_size - MEGABYTES(2));
>
> Can't we just read the value of "vmemmap"?  If not, what is the difference
> between the calculated value above and the value of vmemmap?
>
> >
> >
> > BTW,in arm64_VTOP,it's easier to use the physvirt_offset directly
> > @@ -1148,8 +1155,7 @@ arm64_VTOP(ulong addr)
> >
> > }
> >
> > if (addr >= machdep->machspec->page_offset)
> > -   return machdep->machspec->phys_offset
> > -   + (addr - machdep->machspec->page_offset);
> > +   return (addr + machdep->machspec->physvirt_offset);
>
> Unfortunately that's not possible, because there is at least one arm64_VTOP()
> call *before* the new machdep->machspec->physvirt_offset gets initialized,
> which I presume is why Vinayak's patch checks whether it's non-zero first.
>
> Dave
>
>
>
> > 
> > From: crash-utility-boun...@redhat.com 
> > on
> > behalf of Dave Anderson 
> > Sent: Saturday, April 18, 2020 0:32
> > To: Discussion list for crash utility usage,maintenance and development
> > Subject: [External Mail][] Re: [Crash-utility] ramdump support for
> > va_bit

Re: [Crash-utility] [PATCH] add log -T option to display the message text with human readable timestamp

2020-04-19 Thread Wang Long 




On 20/4/2020 3:48 am, Dave Anderson wrote:


FWIW, I tried it on another RHEL7 machine running live,
but then also on a RHEL8 kernel dumpfile, and they all hang:


I apply this patch on RHEL7 virtual machine(VirtualBbox) and it work ok.
and on a RHEL7 kernel dumpfile, I found it hang after I send the patch.


and I debug it and found the  machdep->hz == 0 on the following:

  get_uptime(NULL, &uptime_jiffies);
  uptime_sec = (uptime_jiffies)/(ulonglong)machdep->hz;
  kt->boot_date.tv_sec = kt->date.tv_sec - uptime_sec;
  kt->boot_date.tv_nsec = 0;

because machdep-> hz has not been initialized here.  divide by zero make
the cpu spinning at 100%.

I thought two solutions:

(1) add misc_init function after machdep_init(POST_INIT) call, and 
calculate the value of kt-> boot_date in it.

read_in_kernel_config(IKCFG_INIT);
kernel_init();
machdep_init(POST_GDB);
vm_init();
machdep_init(POST_VM);
module_init();
help_init();
task_init();
vfs_init();
net_init();
dev_init();
machdep_init(POST_INIT);
+   misc_init();

(2) calculate the value of kt-> boot_date on cmd_log function, when we 
call log command.



Dave, Which one do you like?




   $ crash vmlinux vmcore

   crash 7.2.9rc13
   Copyright (C) 2002-2020  Red Hat, Inc.
   Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
   Copyright (C) 1999-2006  Hewlett-Packard Co
   Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
   Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
   Copyright (C) 2005, 2011  NEC Corporation
   Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
   Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
   This program is free software, covered by the GNU General Public License,
   and you are welcome to change it and/or distribute copies of it under
   certain conditions.  Enter "help copying" to see the conditions.
   This program has absolutely no warranty.  Enter "help warranty" for details.
  
   GNU gdb (GDB) 7.6

   Copyright (C) 2013 Free Software Foundation, Inc.
   License GPLv3+: GNU GPL version 3 or later 
   This is free software: you are free to change and redistribute it.
   There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
   and "show warranty" for details.
   This GDB was configured as "x86_64-unknown-linux-gnu"...

   WARNING: kernel relocated [950MB]: patching 94929 gdb minimal_symbol values

   crash: start patch...



- Original Message -



- Original Message -

Sometimes, we need to know the accurate time of the log, which
helps us analyze the problem.

add -T option(like dmesg -T command) for log command to display
the message text with human readable timestamp.

Signed-off-by: Wang Long 


Did you attempt this patch on a live system?  Because your patch to
kernel_init() hangs the session.  I didn't bother to investigate beyond
adding these two debug statements around your addition to kernel_init():

   error(INFO, "start patch...\n");
   get_uptime(NULL, &uptime_jiffies);
   uptime_sec = (uptime_jiffies)/(ulonglong)machdep->hz;
   kt->boot_date.tv_sec = kt->date.tv_sec - uptime_sec;
   kt->boot_date.tv_nsec = 0;
   error(INFO, "end patch...\n");
   
And that's where it hangs:
   
   $ ./crash
   
   crash 7.2.9rc13

   Copyright (C) 2002-2020  Red Hat, Inc.
   Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
   Copyright (C) 1999-2006  Hewlett-Packard Co
   Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
   Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
   Copyright (C) 2005, 2011  NEC Corporation
   Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
   Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
   This program is free software, covered by the GNU General Public License,
   and you are welcome to change it and/or distribute copies of it under
   certain conditions.  Enter "help copying" to see the conditions.
   This program has absolutely no warranty.  Enter "help warranty" for
   details.

   GNU gdb (GDB) 7.6

   Copyright (C) 2013 Free Software Foundation, Inc.
   License GPLv3+: GNU GPL version 3 or later
   
   This is free software: you are free to change and redistribute it.
   There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
   and "show warranty" for details.
   This GDB was configured as "x86_64-unknown-linux-gnu"...
   
   WARNING: kernel relocated [796MB]: patching 85687 gdb minimal_symbol values
   
   crash: start patch...
   
   


And it shows a cpu spinning at 100%:

   $ top
   top - 15:26:43 up 38 days,  3:41,  5

Re: [Crash-utility] Crash-utility Digest, Vol 175, Issue 28

2020-04-19 Thread Jon Deacon 
Stop sending please 

Sent from my iPhone

> On Apr 19, 2020, at 7:11 PM, crash-utility-requ...@redhat.com wrote:
> 
> Send Crash-utility mailing list submissions to
>crash-utility@redhat.com
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>https://www.redhat.com/mailman/listinfo/crash-utility
> or, via email, send a message with subject or body 'help' to
>crash-utility-requ...@redhat.com
> 
> You can reach the person managing the list at
>crash-utility-ow...@redhat.com
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Crash-utility digest..."
> 
> 
> Today's Topics:
> 
>   1. Re: Crash-utility Digest, Vol 175, Issue 27 (Jon Deacon)
> 
> 
> --
> 
> Message: 1
> Date: Sun, 19 Apr 2020 19:10:49 -0400
> From: Jon Deacon 
> To: crash-utility@redhat.com
> Subject: Re: [Crash-utility] Crash-utility Digest, Vol 175, Issue 27
> Message-ID: 
> Content-Type: text/plain; charset=utf-8
> 
> Would you please take me off your list totally ridiculous  
> 
> Sent from my iPhone
> 
>> On Apr 19, 2020, at 4:01 PM, crash-utility-requ...@redhat.com wrote:
>> 
>> ?Send Crash-utility mailing list submissions to
>>   crash-utility@redhat.com
>> 
>> To subscribe or unsubscribe via the World Wide Web, visit
>>   https://www.redhat.com/mailman/listinfo/crash-utility
>> or, via email, send a message with subject or body 'help' to
>>   crash-utility-requ...@redhat.com
>> 
>> You can reach the person managing the list at
>>   crash-utility-ow...@redhat.com
>> 
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Crash-utility digest..."
>> 
>> 
>> Today's Topics:
>> 
>>  1. Re: [PATCH] add log -T option to display the message text
>> with human readable timestamp (Dave Anderson)
>>  2. Re: [PATCH] add log -T option to display the message text
>> with human readable timestamp (Dave Anderson)
>>  3. Re: [PATCH] add log -T option to display themessagetext
>> with human readable timestamp (Moore, Martin (Linux ERT))
>> 
>> 
>> --
>> 
>> Message: 1
>> Date: Sun, 19 Apr 2020 15:34:46 -0400 (EDT)
>> From: Dave Anderson 
>> To: Wang Long 
>> Cc: crash-utility@redhat.com
>> Subject: Re: [Crash-utility] [PATCH] add log -T option to display the
>>   message text with human readable timestamp
>> Message-ID:
>>   <116726603.23908736.1587324886732.javamail.zim...@redhat.com>
>> Content-Type: text/plain; charset=utf-8
>> 
>> 
>> 
>> - Original Message -
>>> Sometimes, we need to know the accurate time of the log, which
>>> helps us analyze the problem.
>>> 
>>> add -T option(like dmesg -T command) for log command to display
>>> the message text with human readable timestamp.
>>> 
>>> Signed-off-by: Wang Long 
>> 
>> Did you attempt this patch on a live system?  Because your patch to 
>> kernel_init() hangs the session.  I didn't bother to investigate beyond
>> adding these two debug statements around your addition to kernel_init():
>> 
>> error(INFO, "start patch...\n");
>> get_uptime(NULL, &uptime_jiffies);
>> uptime_sec = (uptime_jiffies)/(ulonglong)machdep->hz;
>> kt->boot_date.tv_sec = kt->date.tv_sec - uptime_sec;
>> kt->boot_date.tv_nsec = 0;
>> error(INFO, "end patch...\n");
>> 
>> And that's where it hangs:
>> 
>> $ ./crash
>> 
>> crash 7.2.9rc13
>> Copyright (C) 2002-2020  Red Hat, Inc.
>> Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
>> Copyright (C) 1999-2006  Hewlett-Packard Co
>> Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
>> Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
>> Copyright (C) 2005, 2011  NEC Corporation
>> Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
>> Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
>> This program is free software, covered by the GNU General Public License,
>> and you are welcome to change it and/or distribute copies of it under
>> certain conditions.  Enter "help copying" to see the conditions.
>> This program has absolutely no warranty.  Enter "help warranty" for details.
>> 
>> GNU gdb (GDB) 7.6
>> Copyright (C) 2013 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later 
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-unknown-linux-gnu"...
>> 
>> WARNING: kernel relocated [796MB]: patching 85687 gdb minimal_symbol values
>> 
>> crash: start patch...  
>> 
>> 
>> 
>> And it shows a cpu spinning at 100%:
>> 
>> $ top
>> top - 15:26:43 up 38 days,  3:41,  5 users,  load average: 1.00, 0.89, 0.65
>> Tasks: 280 total,   2 running, 278 sleeping,   0 stopped,   0 zombie

Re: [Crash-utility] Crash-utility Digest, Vol 175, Issue 27

2020-04-19 Thread Jon Deacon 
Would you please take me off your list totally ridiculous  

Sent from my iPhone

> On Apr 19, 2020, at 4:01 PM, crash-utility-requ...@redhat.com wrote:
> 
> Send Crash-utility mailing list submissions to
>crash-utility@redhat.com
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>https://www.redhat.com/mailman/listinfo/crash-utility
> or, via email, send a message with subject or body 'help' to
>crash-utility-requ...@redhat.com
> 
> You can reach the person managing the list at
>crash-utility-ow...@redhat.com
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Crash-utility digest..."
> 
> 
> Today's Topics:
> 
>   1. Re: [PATCH] add log -T option to display the message text
>  with human readable timestamp (Dave Anderson)
>   2. Re: [PATCH] add log -T option to display the message text
>  with human readable timestamp (Dave Anderson)
>   3. Re: [PATCH] add log -T option to display themessagetext
>  with human readable timestamp (Moore, Martin (Linux ERT))
> 
> 
> --
> 
> Message: 1
> Date: Sun, 19 Apr 2020 15:34:46 -0400 (EDT)
> From: Dave Anderson 
> To: Wang Long 
> Cc: crash-utility@redhat.com
> Subject: Re: [Crash-utility] [PATCH] add log -T option to display the
>message text with human readable timestamp
> Message-ID:
><116726603.23908736.1587324886732.javamail.zim...@redhat.com>
> Content-Type: text/plain; charset=utf-8
> 
> 
> 
> - Original Message -
>> Sometimes, we need to know the accurate time of the log, which
>> helps us analyze the problem.
>> 
>> add -T option(like dmesg -T command) for log command to display
>> the message text with human readable timestamp.
>> 
>> Signed-off-by: Wang Long 
> 
> Did you attempt this patch on a live system?  Because your patch to 
> kernel_init() hangs the session.  I didn't bother to investigate beyond
> adding these two debug statements around your addition to kernel_init():
> 
>  error(INFO, "start patch...\n");
>  get_uptime(NULL, &uptime_jiffies);
>  uptime_sec = (uptime_jiffies)/(ulonglong)machdep->hz;
>  kt->boot_date.tv_sec = kt->date.tv_sec - uptime_sec;
>  kt->boot_date.tv_nsec = 0;
>  error(INFO, "end patch...\n");
> 
> And that's where it hangs:
> 
>  $ ./crash
> 
>  crash 7.2.9rc13
>  Copyright (C) 2002-2020  Red Hat, Inc.
>  Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
>  Copyright (C) 1999-2006  Hewlett-Packard Co
>  Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
>  Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
>  Copyright (C) 2005, 2011  NEC Corporation
>  Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
>  Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
>  This program is free software, covered by the GNU General Public License,
>  and you are welcome to change it and/or distribute copies of it under
>  certain conditions.  Enter "help copying" to see the conditions.
>  This program has absolutely no warranty.  Enter "help warranty" for details.
> 
>  GNU gdb (GDB) 7.6
>  Copyright (C) 2013 Free Software Foundation, Inc.
>  License GPLv3+: GNU GPL version 3 or later 
>  This is free software: you are free to change and redistribute it.
>  There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>  and "show warranty" for details.
>  This GDB was configured as "x86_64-unknown-linux-gnu"...
> 
>  WARNING: kernel relocated [796MB]: patching 85687 gdb minimal_symbol values
> 
>  crash: start patch...  
> 
>  
> 
> And it shows a cpu spinning at 100%:
> 
>  $ top
>  top - 15:26:43 up 38 days,  3:41,  5 users,  load average: 1.00, 0.89, 0.65
>  Tasks: 280 total,   2 running, 278 sleeping,   0 stopped,   0 zombie
>  %Cpu(s):  3.9 us,  8.7 sy,  0.0 ni, 87.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 
> st
>  KiB Mem : 15907600 total,   455876 free,  1232832 used, 14218892 buff/cache
>  KiB Swap:  8060924 total,  7395580 free,   665344 used. 14176220 avail Mem 
>PID USER  PR  NIVIRTRESSHR S  %CPU %MEM TIME+ COMMAND
>  26668 root  20   0  350268 213688   5680 R 100.0  1.3   5:42.70 crash
>   1707 root  20   0  115692   1184688 S   0.3  0.0   2:16.52 ksmtuned
>  12852 anderson  20   0 4235240 274608  20320 S   0.3  1.7 601:46.85 
> gnome-shell
>  13060 anderson  20   0  804924  14100   3744 S   0.3  0.1 118:44.59 gsd-color
>  27045 anderson  20   0  172452   2532   1648 R   0.3  0.0   0:00.08 top
>  1 root  20   0  210504   5592   3224 S   0.0  0.0  18:14.19 systemd
>  ...
> 
> I'll let you figure it out...
> 
> Dave
> 
> 
> 
> 
> 
> 
>> ---
>> defs.h   |  2 ++
>> help.c   | 28 +++-
>> kernel.c | 22 --
>> 3 files changed, 49 insertions(+), 3 deletions(-)
>> 
>> diff --git a/defs.h b/defs.h
>> index d8eda5e..1644dbd

Re: [Crash-utility] [PATCH] add log -T option to display the message text with human readable timestamp

2020-04-19 Thread Moore, Martin (Linux ERT) 
Just to mention an alternative possibility, the pykdump extension to crash 
(https://sourceforge.net/projects/pykdump/) includes the 'tslog' command to 
accomplish this.

Martin Moore
Linux Engineering Resolution Team
HPE Pointnext Services
Hewlett Packard Enterprise

martin.mo...@hpe.com 
8AM-5PM EDT (GMT-4) Monday-Friday
Manager: Becky McBride (becky.mcbr...@hpe.com)

-Original Message-
From: crash-utility-boun...@redhat.com  On 
Behalf Of Wang Long
Sent: Saturday, April 18, 2020 10:21 PM
To: ander...@redhat.com; crash-utility@redhat.com
Subject: [Crash-utility] [PATCH] add log -T option to display the message text 
with human readable timestamp

Sometimes, we need to know the accurate time of the log, which helps us analyze 
the problem.

add -T option(like dmesg -T command) for log command to display the message 
text with human readable timestamp.

Signed-off-by: Wang Long 
---
 defs.h   |  2 ++
 help.c   | 28 +++-
 kernel.c | 22 --
 3 files changed, 49 insertions(+), 3 deletions(-)

diff --git a/defs.h b/defs.h
index d8eda5e..1644dbd 100644
--- a/defs.h
+++ b/defs.h
@@ -689,6 +689,7 @@ struct kernel_table {   /* kernel data */
 ulong kernel_module;
int mods_installed;
struct timespec date;
+   struct timespec boot_date;
char proc_version[BUFSIZE];
struct new_utsname utsname;
uint kernel_version[3];
@@ -5577,6 +5578,7 @@ void dump_log(int);  #define SHOW_LOG_DICT  (0x2)  
#define SHOW_LOG_TEXT  (0x4)  #define SHOW_LOG_AUDIT (0x8)
+#define SHOW_LOG_CTIME (0x10)
 void set_cpu(int);
 void clear_machdep_cache(void);
 struct stack_hook *gather_text_list(struct bt_info *); diff --git a/help.c 
b/help.c index c443cad..1ee70f7 100644
--- a/help.c
+++ b/help.c
@@ -3892,12 +3892,13 @@ NULL
 char *help_log[] = {
 "log",
 "dump system message buffer",
-"[-tdma]",
+"[-Ttdma]",
 "  This command dumps the kernel log_buf contents in chronological order.  
The",  "  command supports the older log_buf formats, which may or may not 
contain a",  "  timestamp inserted prior to each message, as well as the newer 
variable-length",  "  record format, where the timestamp is contained in each 
log entry's header.",  "  ",
+"-T  Display the message text with human readable timestamp.",
 "-t  Display the message text without the timestamp; only applicable to 
the",
 "variable-length record format.",
 "-d  Display the dictionary of key/value pair properties that are 
optionally",
@@ -4031,6 +4032,31 @@ char *help_log[] = {
 "type=1307 audit(1489384479.809:4346):  cwd=\"/proc\"",
 "...",
 " ",
+"  Display the message text with human readable timestamp.\n"
+"%s> log -T",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0x-0x0009fbff] usable",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0x0009fc00-0x0009] reserved",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0x000f-0x000f] reserved",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0x0010-0xdffe] usable",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0xdfff-0xdfff] ACPI data",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0xfec0-0xfec00fff] reserved",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0xfee0-0xfee00fff] reserved",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0xfffc-0x] reserved",
+"[Sat Apr  4 07:41:09 2020] BIOS-e820: [mem 
0x0001-0x00011fff] usable",
+"[Sat Apr  4 07:41:09 2020] NX (Execute Disable) protection: active",
+"[Sat Apr  4 07:41:09 2020] SMBIOS 2.5 present.",
+"[Sat Apr  4 07:41:09 2020] DMI: innotek GmbH VirtualBox/VirtualBox, BIOS 
VirtualBox 12/01/2006",
+"[Sat Apr  4 07:41:09 2020] Hypervisor detected: KVM",
+"[Sat Apr  4 07:41:09 2020] kvm-clock: Using msrs 4b564d01 and 4b564d00",
+"[Sat Apr  4 07:41:09 2020] kvm-clock: cpu 0, msr 6de01001, primary cpu 
clock",
+"[Sat Apr  4 07:41:09 2020] kvm-clock: using sched offset of 11838753697 
cycles",
+"[Sat Apr  4 07:41:09 2020] clocksource: kvm-clock: mask: 
0x max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns",
+"[Sat Apr  4 07:41:09 2020] e820: update [mem 0x-0x0fff] 
usable ==> reserved",
+"[Sat Apr  4 07:41:09 2020] e820: remove [mem 0x000a-0x000f] 
usable",
+"[Sat Apr  4 07:41:09 2020] last_pfn = 0x12 max_arch_pfn = 
0x4",
+"[Sat Apr  4 07:41:09 2020] MTRR default type: uncachable",
+"[Sat Apr  4 07:41:09 2020] MTRR variable ranges disabled:",
+"...",
 NULL   
 };
 
diff --git a/kernel.c b/kernel.c
index 7604fac..016d8d9 100644
--- a/kernel.c
+++ b/kernel.c
@@ -111,6 +111,8 @@ kernel_init()
char *irq_desc_type_name;   
ulong pv_init_ops;
struct gnu_request req;
+   u

Re: [Crash-utility] [PATCH] add log -T option to display the message text with human readable timestamp

2020-04-19 Thread Dave Anderson 


FWIW, I tried it on another RHEL7 machine running live,
but then also on a RHEL8 kernel dumpfile, and they all hang:

  $ crash vmlinux vmcore

  crash 7.2.9rc13
  Copyright (C) 2002-2020  Red Hat, Inc.
  Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
  Copyright (C) 1999-2006  Hewlett-Packard Co
  Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
  Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
  Copyright (C) 2005, 2011  NEC Corporation
  Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
  Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
  This program is free software, covered by the GNU General Public License,
  and you are welcome to change it and/or distribute copies of it under
  certain conditions.  Enter "help copying" to see the conditions.
  This program has absolutely no warranty.  Enter "help warranty" for details.
 
  GNU gdb (GDB) 7.6 
  Copyright (C) 2013 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later 
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
  and "show warranty" for details.
  This GDB was configured as "x86_64-unknown-linux-gnu"...

  WARNING: kernel relocated [950MB]: patching 94929 gdb minimal_symbol values

  crash: start patch...  



- Original Message -
> 
> 
> - Original Message -
> > Sometimes, we need to know the accurate time of the log, which
> > helps us analyze the problem.
> > 
> > add -T option(like dmesg -T command) for log command to display
> > the message text with human readable timestamp.
> > 
> > Signed-off-by: Wang Long 
> 
> Did you attempt this patch on a live system?  Because your patch to
> kernel_init() hangs the session.  I didn't bother to investigate beyond
> adding these two debug statements around your addition to kernel_init():
> 
>   error(INFO, "start patch...\n");
>   get_uptime(NULL, &uptime_jiffies);
>   uptime_sec = (uptime_jiffies)/(ulonglong)machdep->hz;
>   kt->boot_date.tv_sec = kt->date.tv_sec - uptime_sec;
>   kt->boot_date.tv_nsec = 0;
>   error(INFO, "end patch...\n");
>   
> And that's where it hangs:
>   
>   $ ./crash
>   
>   crash 7.2.9rc13
>   Copyright (C) 2002-2020  Red Hat, Inc.
>   Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
>   Copyright (C) 1999-2006  Hewlett-Packard Co
>   Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
>   Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
>   Copyright (C) 2005, 2011  NEC Corporation
>   Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
>   Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
>   This program is free software, covered by the GNU General Public License,
>   and you are welcome to change it and/or distribute copies of it under
>   certain conditions.  Enter "help copying" to see the conditions.
>   This program has absolutely no warranty.  Enter "help warranty" for
>   details.
>
>   GNU gdb (GDB) 7.6
>   Copyright (C) 2013 Free Software Foundation, Inc.
>   License GPLv3+: GNU GPL version 3 or later
>   
>   This is free software: you are free to change and redistribute it.
>   There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>   and "show warranty" for details.
>   This GDB was configured as "x86_64-unknown-linux-gnu"...
>   
>   WARNING: kernel relocated [796MB]: patching 85687 gdb minimal_symbol values
>   
>   crash: start patch...
>   
>   
> 
> And it shows a cpu spinning at 100%:
> 
>   $ top
>   top - 15:26:43 up 38 days,  3:41,  5 users,  load average: 1.00, 0.89, 0.65
>   Tasks: 280 total,   2 running, 278 sleeping,   0 stopped,   0 zombie
>   %Cpu(s):  3.9 us,  8.7 sy,  0.0 ni, 87.3 id,  0.0 wa,  0.0 hi,  0.0 si,
>   0.0 st
>   KiB Mem : 15907600 total,   455876 free,  1232832 used, 14218892 buff/cache
>   KiB Swap:  8060924 total,  7395580 free,   665344 used. 14176220 avail Mem
> PID USER  PR  NIVIRTRESSHR S  %CPU %MEM TIME+ COMMAND
>   26668 root  20   0  350268 213688   5680 R 100.0  1.3   5:42.70 crash
>1707 root  20   0  115692   1184688 S   0.3  0.0   2:16.52
>ksmtuned
>   12852 anderson  20   0 4235240 274608  20320 S   0.3  1.7 601:46.85
>   gnome-shell
>   13060 anderson  20   0  804924  14100   3744 S   0.3  0.1 118:44.59
>   gsd-color
>   27045 anderson  20   0  172452   2532   1648 R   0.3  0.0   0:00.08 top
>   1 root  20   0  210504   5592   3224 S   0.0  0.0  18:14.19 systemd
>   ...
>  
> I'll let you figure it out...
> 
> Dave
> 
> 
> 
> 
> 
> 
> > ---
> >  defs.h   |  2 ++
> >  help.c   | 28 +++-
> >  kernel.c | 22 --
> >  3 files changed, 49 insertions(+), 3 deletions(-)
> > 
> > diff --git a/defs.h b/defs.h
> > index d8eda5e..

Re: [Crash-utility] [PATCH] add log -T option to display the message text with human readable timestamp

2020-04-19 Thread Dave Anderson 



- Original Message -
> Sometimes, we need to know the accurate time of the log, which
> helps us analyze the problem.
> 
> add -T option(like dmesg -T command) for log command to display
> the message text with human readable timestamp.
> 
> Signed-off-by: Wang Long 

Did you attempt this patch on a live system?  Because your patch to 
kernel_init() hangs the session.  I didn't bother to investigate beyond
adding these two debug statements around your addition to kernel_init():

  error(INFO, "start patch...\n");
  get_uptime(NULL, &uptime_jiffies);
  uptime_sec = (uptime_jiffies)/(ulonglong)machdep->hz;
  kt->boot_date.tv_sec = kt->date.tv_sec - uptime_sec;
  kt->boot_date.tv_nsec = 0;
  error(INFO, "end patch...\n");
  
And that's where it hangs:
  
  $ ./crash
  
  crash 7.2.9rc13
  Copyright (C) 2002-2020  Red Hat, Inc.
  Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
  Copyright (C) 1999-2006  Hewlett-Packard Co
  Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
  Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
  Copyright (C) 2005, 2011  NEC Corporation
  Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
  Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
  This program is free software, covered by the GNU General Public License,
  and you are welcome to change it and/or distribute copies of it under
  certain conditions.  Enter "help copying" to see the conditions.
  This program has absolutely no warranty.  Enter "help warranty" for details.
   
  GNU gdb (GDB) 7.6
  Copyright (C) 2013 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later 
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
  and "show warranty" for details.
  This GDB was configured as "x86_64-unknown-linux-gnu"...
  
  WARNING: kernel relocated [796MB]: patching 85687 gdb minimal_symbol values
  
  crash: start patch...  
  
  

And it shows a cpu spinning at 100%:

  $ top
  top - 15:26:43 up 38 days,  3:41,  5 users,  load average: 1.00, 0.89, 0.65
  Tasks: 280 total,   2 running, 278 sleeping,   0 stopped,   0 zombie
  %Cpu(s):  3.9 us,  8.7 sy,  0.0 ni, 87.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 
st
  KiB Mem : 15907600 total,   455876 free,  1232832 used, 14218892 buff/cache
  KiB Swap:  8060924 total,  7395580 free,   665344 used. 14176220 avail Mem 
PID USER  PR  NIVIRTRESSHR S  %CPU %MEM TIME+ COMMAND
  26668 root  20   0  350268 213688   5680 R 100.0  1.3   5:42.70 crash
   1707 root  20   0  115692   1184688 S   0.3  0.0   2:16.52 ksmtuned
  12852 anderson  20   0 4235240 274608  20320 S   0.3  1.7 601:46.85 
gnome-shell
  13060 anderson  20   0  804924  14100   3744 S   0.3  0.1 118:44.59 gsd-color
  27045 anderson  20   0  172452   2532   1648 R   0.3  0.0   0:00.08 top
  1 root  20   0  210504   5592   3224 S   0.0  0.0  18:14.19 systemd
  ...
 
I'll let you figure it out...

Dave






> ---
>  defs.h   |  2 ++
>  help.c   | 28 +++-
>  kernel.c | 22 --
>  3 files changed, 49 insertions(+), 3 deletions(-)
> 
> diff --git a/defs.h b/defs.h
> index d8eda5e..1644dbd 100644
> --- a/defs.h/
> +++ b/defs.h
> @@ -689,6 +689,7 @@ struct kernel_table {   /* kernel data */
>  ulong kernel_module;
>   int mods_installed;
>   struct timespec date;
> + struct timespec boot_date;
>   char proc_version[BUFSIZE];
>   struct new_utsname utsname;
>   uint kernel_version[3];
> @@ -5577,6 +5578,7 @@ void dump_log(int);
>  #define SHOW_LOG_DICT  (0x2)
>  #define SHOW_LOG_TEXT  (0x4)
>  #define SHOW_LOG_AUDIT (0x8)
> +#define SHOW_LOG_CTIME (0x10)
>  void set_cpu(int);
>  void clear_machdep_cache(void);
>  struct stack_hook *gather_text_list(struct bt_info *);
> diff --git a/help.c b/help.c
> index c443cad..1ee70f7 100644
> --- a/help.c
> +++ b/help.c
> @@ -3892,12 +3892,13 @@ NULL
>  char *help_log[] = {
>  "log",
>  "dump system message buffer",
> -"[-tdma]",
> +"[-Ttdma]",
>  "  This command dumps the kernel log_buf contents in chronological order.
>  The",
>  "  command supports the older log_buf formats, which may or may not contain
>  a",
>  "  timestamp inserted prior to each message, as well as the newer
>  variable-length",
>  "  record format, where the timestamp is contained in each log entry's
>  header.",
>  "  ",/
> +"-T  Display the message text with human readable timestamp.",
>  "-t  Display the message text without the timestamp; only applicable to
>  the",
>  "variable-length record format.",
>  "-d  Display the dictionary of key/value pair properties that are
>  optionally",
> @@ -4031,6 +4032,31 @@ char *help_log[] = {
>  "type=1307 audit(1489384479.809:4346):  cwd=\"/proc\"",
>  "..

Re: [Crash-utility] [营销邮件] Re: [External Mail][????] Re: ramdump support for va_bits_actual

2020-04-19 Thread Dave Anderson 


- Original Message -
> Hi,Dave
> 
> I don't quite understand how you said to read vmemmap,VMEMMAP_START is a
> macro definition,or it's a constant,in crash-utility vmemmap_start  is a
> wrong value,this may cause arm64_IS_VMALLOC_ADDR return wrong status

I mean the kernel symbol "vmemmap", here in "arch/arm64/mm/init.c":

  struct page *vmemmap __ro_after_init;
  EXPORT_SYMBOL(vmemmap);

Doesn't it contain the resolved starting address?

> We can get physvirt_offset  earlier,in my patch,after calling
> arm64_calc_phys_offset we can initialize physvirt_offset,in this
> time,kimage_offset,page_offset and phys_offset are ready,such as:
> 
> @@ -391,6 +391,13 @@ arm64_init(int when)
> 
> ms = machdep->machspec;
> 
> +   if (kernel_symbol_exists("physvirt_offset") &&
> +   readmem(symbol_value("physvirt_offset"), KVADDR,
> +   &value, sizeof(ulong), "physvirt_offset", 
> QUIET|RETURN_ON_ERROR))
> +   ms->physvirt_offset = value;
> +   else
> +   ms->physvirt_offset = ms->phys_offset -> 
> ms->page_offset;

When the readmem() of symbol_value("physvirt_offset") is made, arm64_VTOP() will
be called with its virtual address, right?

Dave
  

 
> 
> From: crash-utility-boun...@redhat.com  on
> behalf of Dave Anderson 
> Sent: Sunday, April 19, 2020 1:03
> To: Discussion list for crash utility usage,maintenance and development
> Subject: [营销邮件] Re: [Crash-utility] [External Mail][] Re: ramdump support
> for   va_bits_actual
> 
> - Original Message -
> > Hi
> > I made almost the same patch to fix the problem with arm64 in version
> > 5.4...
> >
> > One very small change can merged together,vmemmap_start has a little error:
> > [arch/arm64/include/asm/memory.h]
> >  56 #define VMEMMAP_START   (-VMEMMAP_SIZE - SZ_2M)
> > in crash arm64.c
> > -   vmemmap_start = (-vmemmap_size);
> > +   vmemmap_start = (-vmemmap_size - MEGABYTES(2));
> 
> Can't we just read the value of "vmemmap"?  If not, what is the difference
> between the calculated value above and the value of vmemmap?
> 
> >
> >
> > BTW,in arm64_VTOP,it's easier to use the physvirt_offset directly
> > @@ -1148,8 +1155,7 @@ arm64_VTOP(ulong addr)
> >
> > }
> >
> > if (addr >= machdep->machspec->page_offset)
> > -   return machdep->machspec->phys_offset
> > -   + (addr - machdep->machspec->page_offset);
> > +   return (addr + machdep->machspec->physvirt_offset);
> 
> Unfortunately that's not possible, because there is at least one arm64_VTOP()
> call *before* the new machdep->machspec->physvirt_offset gets initialized,
> which I presume is why Vinayak's patch checks whether it's non-zero first.
> 
> Dave
> 
> 
> 
> > 
> > From: crash-utility-boun...@redhat.com 
> > on
> > behalf of Dave Anderson 
> > Sent: Saturday, April 18, 2020 0:32
> > To: Discussion list for crash utility usage,maintenance and development
> > Subject: [External Mail][] Re: [Crash-utility] ramdump support for
> > va_bits_actual
> >
> > - Original Message -
> > > Hi Dave,
> > >
> > > Noticed that raw ramdumps of 5.4 kernel aren't working with crash tip.
> > > With the patches attached, I could get it working. Please take a look.
> > >
> > > Thanks,
> > > Vinayak
> > >
> >
> > Hi Vinayak,
> >
> > A couple quick questions come to mind...
> >
> > First, I haven't checked all possible READMEM plugins, but for example, if
> > this
> > function is run on a live system, the -1 file descriptor would cause the
> > READMEM()
> > call to fail:
> >
> >  static void
> > +arm64_calc_physvirt_offset(void)
> > +{
> > +   struct machine_specific *ms = machdep->machspec;
> > +   ulong physvirt_offset;
> > +   struct syment *sp;
> > +
> > +   ms->physvirt_offset = ms->phys_offset - ms->page_offset;
> > +
> > +   if ((sp = kernel_symbol_search("physvirt_offset")) &&
> > +   machdep->machspec->kimage_voffset) {
> > +   if (READMEM(-1, &physvirt_offset, sizeof(physvirt_offset),
> > +   sp->value, sp->value -
> > +   machdep->machspec->kimage_voffset) > 0) {
> > +   ms->physvirt_offset = physvirt_offset;
> > +   }
> > +   }
> > +
> > +   if (CRASHDEBUG(1))
> > +   fprintf(fp, "using %lx as physvirt_offset\n",
> > ms->physvirt_offset);
> > +}
> >
> > And here -- are you missing some brackets?  (run "make warn")
> >
> > But regardless of that, why are you setting it back to 48 if it's greater
> > than 48?
> >
> > diff --git a/arm64.c b/arm64.c
> > index 31d6e90..04efc13 100644
> > --- a/arm64.c
> > +++ b/arm64.c
> > @@ -4011,6 +4011,7 @@ arm64_calc_virtual_memory_ranges(void)
> > struct machine_sp

Re: [Crash-utility] [营销邮件] Re: [External Mail][????] Re: ramdump support for va_bits_actual

2020-04-19 Thread 赵乾利 
Hi,Dave

I don't quite understand how you said to read vmemmap,VMEMMAP_START is a macro 
definition,or it's a constant,in crash-utility vmemmap_start  is a wrong 
value,this may cause arm64_IS_VMALLOC_ADDR return wrong status

We can get physvirt_offset  earlier,in my patch,after calling  
arm64_calc_phys_offset we can initialize physvirt_offset,in this 
time,kimage_offset,page_offset and phys_offset are ready,such as:

@@ -391,6 +391,13 @@ arm64_init(int when)

ms = machdep->machspec;

+   if (kernel_symbol_exists("physvirt_offset") &&
+   readmem(symbol_value("physvirt_offset"), KVADDR,
+   &value, sizeof(ulong), "physvirt_offset", 
QUIET|RETURN_ON_ERROR))
+   ms->physvirt_offset = value;
+   else
+   ms->physvirt_offset = ms->phys_offset - ms->page_offset;


From: crash-utility-boun...@redhat.com  on 
behalf of Dave Anderson 
Sent: Sunday, April 19, 2020 1:03
To: Discussion list for crash utility usage,maintenance and development
Subject: [营销邮件] Re: [Crash-utility] [External Mail][] Re: ramdump support 
for   va_bits_actual

- Original Message -
> Hi
> I made almost the same patch to fix the problem with arm64 in version 5.4...
>
> One very small change can merged together,vmemmap_start has a little error:
> [arch/arm64/include/asm/memory.h]
>  56 #define VMEMMAP_START   (-VMEMMAP_SIZE - SZ_2M)
> in crash arm64.c
> -   vmemmap_start = (-vmemmap_size);
> +   vmemmap_start = (-vmemmap_size - MEGABYTES(2));

Can't we just read the value of "vmemmap"?  If not, what is the difference
between the calculated value above and the value of vmemmap?

>
>
> BTW,in arm64_VTOP,it's easier to use the physvirt_offset directly
> @@ -1148,8 +1155,7 @@ arm64_VTOP(ulong addr)
>
> }
>
> if (addr >= machdep->machspec->page_offset)
> -   return machdep->machspec->phys_offset
> -   + (addr - machdep->machspec->page_offset);
> +   return (addr + machdep->machspec->physvirt_offset);

Unfortunately that's not possible, because there is at least one arm64_VTOP()
call *before* the new machdep->machspec->physvirt_offset gets initialized,
which I presume is why Vinayak's patch checks whether it's non-zero first.

Dave



> 
> From: crash-utility-boun...@redhat.com  on
> behalf of Dave Anderson 
> Sent: Saturday, April 18, 2020 0:32
> To: Discussion list for crash utility usage,maintenance and development
> Subject: [External Mail][] Re: [Crash-utility] ramdump support for
> va_bits_actual
>
> - Original Message -
> > Hi Dave,
> >
> > Noticed that raw ramdumps of 5.4 kernel aren't working with crash tip.
> > With the patches attached, I could get it working. Please take a look.
> >
> > Thanks,
> > Vinayak
> >
>
> Hi Vinayak,
>
> A couple quick questions come to mind...
>
> First, I haven't checked all possible READMEM plugins, but for example, if
> this
> function is run on a live system, the -1 file descriptor would cause the
> READMEM()
> call to fail:
>
>  static void
> +arm64_calc_physvirt_offset(void)
> +{
> +   struct machine_specific *ms = machdep->machspec;
> +   ulong physvirt_offset;
> +   struct syment *sp;
> +
> +   ms->physvirt_offset = ms->phys_offset - ms->page_offset;
> +
> +   if ((sp = kernel_symbol_search("physvirt_offset")) &&
> +   machdep->machspec->kimage_voffset) {
> +   if (READMEM(-1, &physvirt_offset, sizeof(physvirt_offset),
> +   sp->value, sp->value -
> +   machdep->machspec->kimage_voffset) > 0) {
> +   ms->physvirt_offset = physvirt_offset;
> +   }
> +   }
> +
> +   if (CRASHDEBUG(1))
> +   fprintf(fp, "using %lx as physvirt_offset\n",
> ms->physvirt_offset);
> +}
>
> And here -- are you missing some brackets?  (run "make warn")
>
> But regardless of that, why are you setting it back to 48 if it's greater
> than 48?
>
> diff --git a/arm64.c b/arm64.c
> index 31d6e90..04efc13 100644
> --- a/arm64.c
> +++ b/arm64.c
> @@ -4011,6 +4011,7 @@ arm64_calc_virtual_memory_ranges(void)
> struct machine_specific *ms = machdep->machspec;
> ulong value, vmemmap_start, vmemmap_end, vmemmap_size, vmalloc_end;
> char *string;
> +   int ret;
> ulong PUD_SIZE = UNINITIALIZED;
>
> if (!machdep->machspec->CONFIG_ARM64_VA_BITS) {
> @@ -4018,6 +4019,12 @@ arm64_calc_virtual_memory_ranges(void)
> value = atol(string);
> free(string);
> machdep->machspec->CONFIG_ARM64_VA_BITS = value;
> +   } else if (kt->ikconfig_flags & IKCONFIG_AVAIL) {
> +   if ((ret = get_kernel_config("C