RE: [makedumpfile PATCH v2] Wipe excluded pages that are written into ELF dump file
Hello Eric, >On 08/07/2017 07:25 PM, Atsushi Kumagai wrote: >> Hello Eric, >> >> Thanks for your work, this version looks good to me. >> I'll merge this into v1.6.3. >> >> Regards, >> Atsushi Kumagai > >Thank you! >eric I noticed that wiping zero pages seems to cause different behavior of crash like: [kmem -i] -SWAP USED1 4 KB0% of TOTAL SWAP -SWAP FREE 489979 1.9 GB 99% of TOTAL SWAP +SWAP USED 229886 898 MB 46% of TOTAL SWAP // wiped page is counted as SWAP USED ? +SWAP FREE 260094 1016 MB 53% of TOTAL SWAP I don't investigate the details yet, but I decided to slip this feature to v1.6.4. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
makedumpfile-1.6.3: Extend supported kernel to 4.14.8
Hello, makedumpfile version 1.6.3 is released. Your comments/patches are welcome. This is my last work for makedumpfile, thank you so much for everything. My co-workers will take over the maintainer role, so there is no need to change your way of contribution. (See "BUG REPORT" in README for new maintainers) Main new feature: o Support new kernels - The supported kernel is updated to 4.14.8 in this version. 4.14.9 and later can't be filtered correctly due to inappropriate VMCOREINFO, it will be resolved in kernel 4.15. Changelog: o New feature - [PATCH v2] x86_64: Take care of init_level4_pgt rename in kernel (by Pratyush Anand) 64bd5db - [PATCH v2] Fix SECTION_MAP_MASK for kernel >= v.13 (by Pratyush Anand) 4bf4f2b - [PATCH v2] book3s/ppc64: Lower the max real address to 53 bits for kernels >= v4.11 (by Bhupesh Sharma) 6c83e74 - [PATCH v3 1/4] Support symbol __cpu_online_mask (by Takao Indoh) 3b11c00 - [PATCH] ppc64: update hash page table geometry (by Hari Bathini) 3c39f13 - [PATCH] handle renamed init_level4_pgt -> init_top_pgt (by Jeff Mahoney) 5e54d53 o Bugfix - [PATCH v2] arm64: Fix page table walk of 1GB section (by Bradley Bolen) 27508f1 - [PATCH v2 1/2] ppc64: set page_offset in get_versiondep_info_ppc64() (by Pingfan Liu) 52319d2 - [PATCH v2 2/2] ppc64: get the info of mem reserved for crashkernel (by Pingfan Liu) c7fcbbc - [PATCH v3 3/4] sadump: Fix a KASLR problem of sadump (by Takao Indoh) b4f7d95 - [PATCH v3 4/4] sadump: Fix a KASLR problem of sadump while kdump is working (by Takao Indoh) 13d3059 - [PATCH 2/2] Fix 'kernel_version' variable being uninitialized & introduce minor reorganization (by Bhupesh Sharma) d1ffe82 - [PATCH 1/2] Fix off-by-one errors in exclude_segment() (by Petr Tesarik) 590f35e - [PATCH 2/2] Fix physical-to-virtual conversion in exclude_segment() (by Petr Tesarik) 6c1bf2a o Cleanup - [PATCH] Fix formatting problems in header file (by Eric DeVolder) cefea9e - [PATCH v3 2/4] Introduce vtop4_x86_64_pagetable (by Takao Indoh) 6de5d37 - [PATCH 1/2] Fix compilation warnings on ppc64/ppc64le platforms (by Bhupesh Sharma) 0df157a - [PATCH] Make good use of is_cyclic_region() (by Atsushi Kumagai) 6bfd7a3 - [PATCH] Fix the regression about getting kernel version (by Atsushi Kumagai) 254c116 - [PATCH] Update maintainers (by Atsushi Kumagai) 9d3147e Explanation of makedumpfile: To shorten the size of the dumpfile and the time of creating the dumpfile, makedumpfile copies only the necessary pages for analysis to the dumpfile from /proc/vmcore. You can specify the kind of unnecessary pages with dump_level. If you want to shorten the size further, enable the compression of the page data. Download: You can download the latest makedumpfile from the following URL. Details of the change are written on the git page of the following site. https://sourceforge.net/projects/makedumpfile/ Method of installation: You can compile the makedumpfile command as follows; 1. "tar -zxvf makedumpfile-x.y.z.tar.gz" 2. "cd makedumpfile-x.y.z" 3. "make; make install" Usage: makedumpfile [-c|-l|-p] [-E] [-d dump_level] [-x vmlinux] dump_mem dump_file Example: If you want to exclude pages filled by zero, cache pages, user pages and free pages and to enable compression, please execute the following command. # makedumpfile -c -d 31 /proc/vmcore dumpfile Thanks Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH] sadump: Fix a problem of PTI enabled kernel
>> This patch fixes a problme that a dumpfile of sadump cannot be handled by
>> makedumpfile when Page Table Isolation(PTI) is enabled.
>>
>> When PTI is enabled, bit 12 of CR3 register is used to split user space and
>> kernel space. Also bit 11:0 is used for Process Context IDentifiers(PCID).
>> To
>> open a dump file of sadump, a value of CR3 is used to calculate KASLR offset
>> and
>> phys_base, therefore this patch fixes to mask CR3 register value collectly
>> for
>> PTI enabled kernel.
>>
>> Signed-off-by: Takao Indoh <indou.ta...@jp.fujitsu.com>
>> ---
>> makedumpfile.c | 2 ++
>> makedumpfile.h | 2 ++
>> sadump_info.c | 9 -
>> 3 files changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index 64b404a..247a056 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -1557,6 +1557,8 @@ get_symbol_info(void)
>> SYMBOL_INIT(divide_error, "divide_error");
>> SYMBOL_INIT(idt_table, "idt_table");
>> SYMBOL_INIT(saved_command_line, "saved_command_line");
>> +SYMBOL_INIT(pti_init, "pti_init");
>> +SYMBOL_INIT(kaiser_init, "kaiser_init");
>>
>> return TRUE;
>> }
>> diff --git a/makedumpfile.h b/makedumpfile.h
>> index 57cf4d9..8ee4d29 100644
>> --- a/makedumpfile.h
>> +++ b/makedumpfile.h
>> @@ -1608,6 +1608,8 @@ struct symbol_table {
>> unsigned long long divide_error;
>> unsigned long long idt_table;
>> unsigned long long saved_command_line;
>> +unsigned long long pti_init;
>> +unsigned long long kaiser_init;
>>
>> /*
>> * symbols on ppc64 arch
>> diff --git a/sadump_info.c b/sadump_info.c
>> index 148d4ba..dd50d48 100644
>> --- a/sadump_info.c
>> +++ b/sadump_info.c
>> @@ -1362,6 +1362,9 @@ finish:
>> *kernel. Retrieve vmcoreinfo from address of "elfcorehdr=" and
>> *get kaslr_offset and phys_base from vmcoreinfo.
>> */
>> +#define PTI_USER_PGTABLE_BIT(info->page_shift)
>> +#define PTI_USER_PGTABLE_MASK (1 << PTI_USER_PGTABLE_BIT)
>> +#define CR3_PCID_MASK 0xFFFull
>> int
>> calc_kaslr_offset(void)
>> {
>> @@ -1389,7 +1392,11 @@ calc_kaslr_offset(void)
>> }
>>
>> idtr = ((uint64_t)smram.IdtUpper)<<32 | (uint64_t)smram.IdtLower;
>> -cr3 = smram.Cr3;
>> +if ((SYMBOL(pti_init) != NOT_FOUND_SYMBOL) ||
>> +(SYMBOL(kaiser_init) != NOT_FOUND_SYMBOL))
>> +cr3 = smram.Cr3 & ~(CR3_PCID_MASK|PTI_USER_PGTABLE_MASK);
>> +else
>> +cr3 = smram.Cr3 & ~CR3_PCID_MASK;
>>
>> /* Convert virtual address of IDT table to physical address */
>> if ((idtr_paddr = vtop4_x86_64_pagetable(idtr, cr3)) == NOT_PADDR)
>
>Looks good to me.
>
>Thanks for your work.
>
>Kumagai-san, could you merge this patch?
Sure, it will be merged into v1.6.4 since v1.6.3 is almost ready for release.
Thanks,
Atsushi Kumagai
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH 0/2] Fix calculations in exclude_segment()
>On 01/19/18 at 12:45pm, Petr Tesarik wrote: >> I have observed that makedumpfile --mem-usage shows bogus numbers >> on a kaslr-enabled kernel. This boils down to incorrect calculations >> in exclude_segment(). Consequently, data beyond a split LOAD segment >> are read from the wrong file offset. >> >> Petr Tesarik (2): >> Fix off-by-one errors in exclude_segment() >> Fix physical-to-virtual conversion in exclude_segment() >> >> elf_info.c | 22 +++--- >> 1 file changed, 11 insertions(+), 11 deletions(-) > >Ack the series, thanks! Thanks guys, I'll merge them into v1.6.3. Regards, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y
>Hm, this fix means that the vmlinux symbol table and vmcoreinfo have
>different values for mem_section. That seems... odd. I had to patch
>makedumpfile to fix the case of an explicit vmlinux being passed on the
>command line (which I realized I don't need to do, but it should still
>work):
Looks good to me, I'll merge this into makedumpfile-1.6.4.
Thanks,
Atsushi Kumagai
>From 542a11a8f28b0f0a989abc3adff89da22f44c719 Mon Sep 17 00:00:00 2001
>Message-Id:
><542a11a8f28b0f0a989abc3adff89da22f44c719.1515995400.git.osan...@fb.com>
>From: Omar Sandoval <osan...@fb.com>
>Date: Sun, 14 Jan 2018 17:10:30 -0800
>Subject: [PATCH] Fix SPARSEMEM_EXTREME support on Linux v4.15 when passing
> vmlinux
>
>Since kernel commit 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at
>runtime for CONFIG_SPARSEMEM_EXTREME=y"), mem_section is a dynamically
>allocated array of pointers to mem_section instead of a static one
>(i.e., struct mem_section ** instead of struct mem_section * []). This
>adds an extra layer of indirection that breaks makedumpfile, which will
>end up with a bunch of bogus mem_maps.
>
>Since kernel commit a0b1280368d1 ("kdump: write correct address of
>mem_section into vmcoreinfo"), the mem_section symbol in vmcoreinfo
>contains the address of the actual struct mem_section * array instead of
>the address of the pointer in .bss, which gets rid of the extra
>indirection. However, makedumpfile still uses the debugging symbol from
>the vmlinux image. Fix this by allowing symbols from the vmcore to
>override symbols from the vmlinux image. As the comment in initial()
>says, "vmcoreinfo in /proc/vmcore is more reliable than -x/-i option".
>
>Signed-off-by: Omar Sandoval <osan...@fb.com>
>---
> makedumpfile.h | 6 --
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 57cf4d9..d68c798 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -274,8 +274,10 @@ do { \
> } while (0)
> #define READ_SYMBOL(str_symbol, symbol) \
> do { \
>- if (SYMBOL(symbol) == NOT_FOUND_SYMBOL) { \
>- SYMBOL(symbol) =
>read_vmcoreinfo_symbol(STR_SYMBOL(str_symbol)); \
>+ unsigned long _tmp_symbol; \
>+ _tmp_symbol = read_vmcoreinfo_symbol(STR_SYMBOL(str_symbol)); \
>+ if (_tmp_symbol != NOT_FOUND_SYMBOL) { \
>+ SYMBOL(symbol) = _tmp_symbol; \
> if (SYMBOL(symbol) == INVALID_SYMBOL_DATA) \
> return FALSE; \
> } \
>--
>2.9.5
>
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: makedumpfile saving vmcore fails with dynamically allocated mem_section (was: Re: [PATCH] handle renamed init_level4_pgt -> init_top_pgt)
>> >> > > The root cause is this commit makes mem_section as a pointer instead
>> >> > > of
>> >> > > the static array.
>> >> > >
>> >> > > VMCOREINFO_SYMBOL() expand it as _section, this is not correct in
>> >> > > the test case any more.
>> >> > >
>> >> > > This hack code works for me:
>> >> > >
>> >> > > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>> >> > > index b3663896278e..f5fe6068ae39 100644
>> >> > > --- a/kernel/crash_core.c
>> >> > > +++ b/kernel/crash_core.c
>> >> > > @@ -376,6 +376,8 @@ phys_addr_t __weak paddr_vmcoreinfo_note(void)
>> >> > > {
>> >> > > return __pa(vmcoreinfo_note);
>> >> > > }
>> >> > > +#define VMCOREINFO_SYMBOL_HACK(name) \
>> >> > > + vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned
>> >> > > long)name)
>> >> >
>> >> > Oh, you made a new one. We may use vmcoreinfo_append_str directly since
>> >> > there's an existing one in crash_save_vmcoreinfo():
>> >>
>> >> Yes, it should be something like this instead, this should ensure
>> >> makedumpfile maybe crash tool works without any modifications,
>> >> waiting for feedback from Atsushi, also ccing Dave for crash utility
>> >> potential issue.
>>
>> makedumpfile needs some fix anyway since some logic depend on the array
>> length
>> of mem_section. I'm trying to fix it referring to the crash's fix.
>
>Hmm, a makedumpfile only fix is enough for this issue?
Sorry, it was thoughtless of me. The crash's fix uses dwarf info,
the kernel side fix you thought is necessary for makedumpfile to work without
vmlinux.
Thanks,
Atsushi Kumagai
>>
>> > I'll give priority to release v1.6.3 since the commit will not be included
>> > in the supported kernel(4.14).
>>
>> The kernel change was merged also into kernel 4.14.9, I'm working on this
>> issue
>> for the next release(v1.6.3).
>>
>> Thanks,
>> Atsushi Kumagai
>>
>> >> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>> >> index b3663896278e..122494364179 100644
>> >> --- a/kernel/crash_core.c
>> >> +++ b/kernel/crash_core.c
>> >> @@ -410,10 +410,16 @@ static int __init crash_save_vmcoreinfo_init(void)
>> >> VMCOREINFO_SYMBOL(contig_page_data);
>> >> #endif
>> >> #ifdef CONFIG_SPARSEMEM
>> >> +#ifdef CONFIG_SPARSEMEM_EXTREME
>> >> + vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
>> >> + (unsigned long)mem_section);
>> >> +#else
>> >> VMCOREINFO_SYMBOL(mem_section);
>> >> +#endif
>> >
>> >vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
>> >(unsigned long)mem_section);
>> >
>> >This is enough to cover both cases. mem_section is array name in non
>> >SPARSEMEM_EXTREME case, so value of _section == value of
>> >mem_section.
>> >
>> >> VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>> >> VMCOREINFO_STRUCT_SIZE(mem_section);
>> >> VMCOREINFO_OFFSET(mem_section, section_mem_map);
>> >> +
>> >> #endif
>> >> VMCOREINFO_STRUCT_SIZE(page);
>> >> VMCOREINFO_STRUCT_SIZE(pglist_data);
>> >>
>> >> >
>> >> > vmcoreinfo_append_str("CRASHTIME=%ld\n", get_seconds());
>> >> >
>> >> > The thing is that VMCOREINFO_SYMBOL will reference symbol. While
>> >> > checking all exported symbol, all of them are array names. So here
>> >> > the value of _name is equal to the address of the first element of
>> >> > array. Now the first exception appeared, mem_section, which could be not
>> >> > an array name, but a pointer pointing at the allocated memory.
>> >> >
>> >> > For this issue, we either change as Dave mentioned two options, or can
>> >> > we adjust VMCOREINFO_SYMBOL(name)?
>> >> >
>> >> > >
>> >> > > static int __init crash_save_vmcoreinfo_init(void)
>> >> > > {
>> >> > > @@ -410,10 +412,11 @@ static int __init
>> >> > > crash_save_vmcoreinfo_init(void)
>> >> > >
RE: makedumpfile saving vmcore fails with dynamically allocated mem_section (was: Re: [PATCH] handle renamed init_level4_pgt -> init_top_pgt)
>On 01/03/18 at 10:30am, Dave Young wrote:
>> On 01/02/18 at 05:08pm, Baoquan He wrote:
>> > On 01/02/18 at 04:57pm, Dave Young wrote:
>> > > The root cause is this commit makes mem_section as a pointer instead of
>> > > the static array.
>> > >
>> > > VMCOREINFO_SYMBOL() expand it as _section, this is not correct in
>> > > the test case any more.
>> > >
>> > > This hack code works for me:
>> > >
>> > > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>> > > index b3663896278e..f5fe6068ae39 100644
>> > > --- a/kernel/crash_core.c
>> > > +++ b/kernel/crash_core.c
>> > > @@ -376,6 +376,8 @@ phys_addr_t __weak paddr_vmcoreinfo_note(void)
>> > > {
>> > > return __pa(vmcoreinfo_note);
>> > > }
>> > > +#define VMCOREINFO_SYMBOL_HACK(name) \
>> > > +vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned
>> > > long)name)
>> >
>> > Oh, you made a new one. We may use vmcoreinfo_append_str directly since
>> > there's an existing one in crash_save_vmcoreinfo():
>>
>> Yes, it should be something like this instead, this should ensure
>> makedumpfile maybe crash tool works without any modifications,
>> waiting for feedback from Atsushi, also ccing Dave for crash utility
>> potential issue.
makedumpfile needs some fix anyway since some logic depend on the array length
of mem_section. I'm trying to fix it referring to the crash's fix.
> I'll give priority to release v1.6.3 since the commit will not be included
> in the supported kernel(4.14).
The kernel change was merged also into kernel 4.14.9, I'm working on this issue
for the next release(v1.6.3).
Thanks,
Atsushi Kumagai
>> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>> index b3663896278e..122494364179 100644
>> --- a/kernel/crash_core.c
>> +++ b/kernel/crash_core.c
>> @@ -410,10 +410,16 @@ static int __init crash_save_vmcoreinfo_init(void)
>> VMCOREINFO_SYMBOL(contig_page_data);
>> #endif
>> #ifdef CONFIG_SPARSEMEM
>> +#ifdef CONFIG_SPARSEMEM_EXTREME
>> +vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
>> + (unsigned long)mem_section);
>> +#else
>> VMCOREINFO_SYMBOL(mem_section);
>> +#endif
>
> vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
> (unsigned long)mem_section);
>
>This is enough to cover both cases. mem_section is array name in non
>SPARSEMEM_EXTREME case, so value of _section == value of
>mem_section.
>
>> VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>> VMCOREINFO_STRUCT_SIZE(mem_section);
>> VMCOREINFO_OFFSET(mem_section, section_mem_map);
>> +
>> #endif
>> VMCOREINFO_STRUCT_SIZE(page);
>> VMCOREINFO_STRUCT_SIZE(pglist_data);
>>
>> >
>> > vmcoreinfo_append_str("CRASHTIME=%ld\n", get_seconds());
>> >
>> > The thing is that VMCOREINFO_SYMBOL will reference symbol. While
>> > checking all exported symbol, all of them are array names. So here
>> > the value of _name is equal to the address of the first element of
>> > array. Now the first exception appeared, mem_section, which could be not
>> > an array name, but a pointer pointing at the allocated memory.
>> >
>> > For this issue, we either change as Dave mentioned two options, or can
>> > we adjust VMCOREINFO_SYMBOL(name)?
>> >
>> > >
>> > > static int __init crash_save_vmcoreinfo_init(void)
>> > > {
>> > > @@ -410,10 +412,11 @@ static int __init crash_save_vmcoreinfo_init(void)
>> > > VMCOREINFO_SYMBOL(contig_page_data);
>> > > #endif
>> > > #ifdef CONFIG_SPARSEMEM
>> > > -VMCOREINFO_SYMBOL(mem_section);
>> > > +VMCOREINFO_SYMBOL_HACK(mem_section);
>> > > VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>> > > VMCOREINFO_STRUCT_SIZE(mem_section);
>> > > VMCOREINFO_OFFSET(mem_section, section_mem_map);
>> > > +
>> > > #endif
>> > > VMCOREINFO_STRUCT_SIZE(page);
>> > > VMCOREINFO_STRUCT_SIZE(pglist_data);
>> > >
>> > >
>> > > But probably we need this instead, but I can not test it because I do
>> > > not know how to fix makedumpfile to use a _NUMBER instead of a SYMBOL.
>> > > Thus bring up the issue, seeking for thought
RE: [PATCH] handle renamed init_level4_pgt -> init_top_pgt
>On 12/22/17 at 12:54pm, Dave Young wrote:
>> Hi Atsushi,
>> On 12/21/17 at 08:48am, Atsushi Kumagai wrote:
>> > Hello Dave,
>> >
>> > >[dyoung@dhcp-*-* makedumpfile]$ sudo ./makedumpfile -l -d 31
>> > >/mnt/vmcore/vmcore /tmp/vmcore.1
>> > >The kernel version is not supported.
>> > >The makedumpfile operation may be incomplete.
>> > >Checking for memory holes : [100.0 %] |
>> > > __vtop4_x86_64: Can't get a valid
>pte.
>> > >readmem: Can't convert a virtual address(88007ebb1000) to physical
>> > >address.
>> > >readmem: type_addr: 0, addr:88007ebb1000, size:32768
>> > >__exclude_unnecessary_pages: Can't read the buffer of struct page.
>> > >create_2nd_bitmap: Can't exclude unnecessary pages.
>> > >
>> > >makedumpfile Failed.
>> > >
>> > >If you need the vmcore for debugging please let me know, my test is just
>> > >a normal test in kvm guest.
>> >
>> > Thanks for your report.
>> > I can't reproduce that in my environment, could you give me the vmcore ?
>> > It's helpful if you append the vmlinux and the .config.
>>
>> I'm bisecting it, nearly finished, will post the results and share
>> vmcore if needed soon.
>
>vmcore/vmlinux (gzipped) and .config:
>http://people.redhat.com/ruyang/rhbz1528542/
Thanks.
>Bisect result is below:
I'm not sure why this commit affects makedumpfile yet, but
I'll give priority to release v1.6.3 since the commit will not be included
in the supported kernel(4.14).
Regards,
Atsushi Kumagai
>83e3c48729d9ebb7af5a31a504f3fd6aff0348c4 is the first bad commit
>commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
>Author: Kirill A. Shutemov <kirill.shute...@linux.intel.com>
>Date: Fri Sep 29 17:08:16 2017 +0300
>
>mm/sparsemem: Allocate mem_section at runtime for
> CONFIG_SPARSEMEM_EXTREME=y
>
>Size of the mem_section[] array depends on the size of the physical
> address space.
>
>In preparation for boot-time switching between paging modes on x86-64
>we need to make the allocation of mem_section[] dynamic, because otherwise
>we waste a lot of RAM: with CONFIG_NODE_SHIFT=10, mem_section[] size is
> 32kB
>for 4-level paging and 2MB for 5-level paging mode.
>
>The patch allocates the array on the first call to
> sparse_memory_present_with_active_regions().
>
>Signed-off-by: Kirill A. Shutemov <kirill.shute...@linux.intel.com>
>Cc: Andrew Morton <a...@linux-foundation.org>
>Cc: Andy Lutomirski <l...@amacapital.net>
>Cc: Borislav Petkov <b...@suse.de>
>Cc: Cyrill Gorcunov <gorcu...@openvz.org>
>Cc: Linus Torvalds <torva...@linux-foundation.org>
>Cc: Peter Zijlstra <pet...@infradead.org>
>Cc: Thomas Gleixner <t...@linutronix.de>
>Cc: linux...@kvack.org
>Link:
> http://lkml.kernel.org/r/20170929140821.37654-2-kirill.shute...@linux.intel.com
>Signed-off-by: Ingo Molnar <mi...@kernel.org>
>
>:04 04 68b7ff3eedc2c9ff56e31108f7e982eacbb233fc
>0014ee63bebe14efb0e36e0028e2cbe718fd6c30 M include
>:04 04 78adbc296527c802400b1f68e0fbd716920726fa
>a4eea8117cb318527c0d5a6281d68f312f644831 M mm
>
>>
>> >
>> > Thanks,
>> > Atsushi Kumagai
>> >
>> > >> >>Signed-off-by: Jeff Mahoney <je...@suse.com>
>> > >> >>---
>> > >> >> arch/x86_64.c | 24 +---
>> > >> >> makedumpfile.c | 6 ++
>> > >> >> makedumpfile.h | 2 ++
>> > >> >> 3 files changed, 29 insertions(+), 3 deletions(-)
>> > >> >>
>> > >> >>diff --git a/arch/x86_64.c b/arch/x86_64.c
>> > >> >>index 08dd6b2..9b09035 100644
>> > >> >>--- a/arch/x86_64.c
>> > >> >>+++ b/arch/x86_64.c
>> > >> >>@@ -259,16 +259,26 @@ vtop4_x86_64(unsigned long vaddr)
>> > >> >> {
>> > >> >> unsigned long page_dir, pml4, pgd_paddr, pgd_pte, pmd_paddr,
>> > >> >> pmd_pte;
>> > >> >> unsigned long pte_paddr, pte;
>> > >> >>+ unsigned long init_level4_pgt;
>> > >> >>
>> > >> >>- if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL) {
>> > >> >>+ if (SYMBOL(init_level4_pgt) != NOT_FOUND_SYMBOL)
>> > >> >>+ init_level4_pg
RE: [PATCH] handle renamed init_level4_pgt -> init_top_pgt
Hello Dave,
>[dyoung@dhcp-*-* makedumpfile]$ sudo ./makedumpfile -l -d 31
>/mnt/vmcore/vmcore /tmp/vmcore.1
>The kernel version is not supported.
>The makedumpfile operation may be incomplete.
>Checking for memory holes : [100.0 %] |
> __vtop4_x86_64: Can't get a valid pte.
>readmem: Can't convert a virtual address(88007ebb1000) to physical address.
>readmem: type_addr: 0, addr:88007ebb1000, size:32768
>__exclude_unnecessary_pages: Can't read the buffer of struct page.
>create_2nd_bitmap: Can't exclude unnecessary pages.
>
>makedumpfile Failed.
>
>If you need the vmcore for debugging please let me know, my test is just
>a normal test in kvm guest.
Thanks for your report.
I can't reproduce that in my environment, could you give me the vmcore ?
It's helpful if you append the vmlinux and the .config.
Thanks,
Atsushi Kumagai
>> >>Signed-off-by: Jeff Mahoney <je...@suse.com>
>> >>---
>> >> arch/x86_64.c | 24 +---
>> >> makedumpfile.c | 6 ++
>> >> makedumpfile.h | 2 ++
>> >> 3 files changed, 29 insertions(+), 3 deletions(-)
>> >>
>> >>diff --git a/arch/x86_64.c b/arch/x86_64.c
>> >>index 08dd6b2..9b09035 100644
>> >>--- a/arch/x86_64.c
>> >>+++ b/arch/x86_64.c
>> >>@@ -259,16 +259,26 @@ vtop4_x86_64(unsigned long vaddr)
>> >> {
>> >> unsigned long page_dir, pml4, pgd_paddr, pgd_pte, pmd_paddr, pmd_pte;
>> >> unsigned long pte_paddr, pte;
>> >>+ unsigned long init_level4_pgt;
>> >>
>> >>- if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL) {
>> >>+ if (SYMBOL(init_level4_pgt) != NOT_FOUND_SYMBOL)
>> >>+ init_level4_pgt = SYMBOL(init_level4_pgt);
>> >>+ else if (SYMBOL(init_top_pgt) != NOT_FOUND_SYMBOL)
>> >>+ init_level4_pgt = SYMBOL(init_top_pgt);
>> >>+ else {
>> >> ERRMSG("Can't get the symbol of init_level4_pgt.\n");
>> >> return NOT_PADDR;
>> >> }
>> >>
>> >>+ if (SYMBOL(level4_kernel_pgt) != NOT_FOUND_SYMBOL) {
>> >>+ ERRMSG("Kernel is built with 5-level page tables\n");
>> >>+ return NOT_PADDR;
>> >>+ }
>> >>+
>> >> /*
>> >>* Get PGD.
>> >>*/
>> >>- page_dir = SYMBOL(init_level4_pgt) - __START_KERNEL_map +
>> >>info->phys_base;
>> >>+ page_dir = init_level4_pgt - __START_KERNEL_map + info->phys_base;
>> >> if (is_xen_memory()) {
>> >> page_dir = ptom_xen(page_dir);
>> >> if (page_dir == NOT_PADDR)
>> >>@@ -549,8 +559,16 @@ find_vmemmap_x86_64()
>> >> struct vmap_pfns *vmapp, *vmaphead = NULL, *cur, *tail;
>> >>
>> >> init_level4_pgt = SYMBOL(init_level4_pgt);
>> >>+ if (init_level4_pgt == NOT_FOUND_SYMBOL)
>> >>+ init_level4_pgt = SYMBOL(init_top_pgt);
>> >>+
>> >> if (init_level4_pgt == NOT_FOUND_SYMBOL) {
>> >>- ERRMSG("init_level4_pgt not found\n");
>> >>+ ERRMSG("init_level4_pgt/init_top_pgt not found\n");
>> >>+ return FAILED;
>> >>+ }
>> >>+
>> >>+ if (SYMBOL(level4_kernel_pgt) != NOT_FOUND_SYMBOL) {
>> >>+ ERRMSG("kernel is configured for 5-level page tables\n");
>> >> return FAILED;
>> >> }
>> >> pagestructsize = size_table.page;
>> >>diff --git a/makedumpfile.c b/makedumpfile.c
>> >>index f85003a..6e5ec34 100644
>> >>--- a/makedumpfile.c
>> >>+++ b/makedumpfile.c
>> >>@@ -1486,6 +1486,8 @@ get_symbol_info(void)
>> >> SYMBOL_INIT(_stext, "_stext");
>> >> SYMBOL_INIT(swapper_pg_dir, "swapper_pg_dir");
>> >> SYMBOL_INIT(init_level4_pgt, "init_level4_pgt");
>> >>+ SYMBOL_INIT(level4_kernel_pgt, "level4_kernel_pgt");
>> >>+ SYMBOL_INIT(init_top_pgt, "init_top_pgt");
>> >> SYMBOL_INIT(vmlist, "vmlist");
>> >> SYMBOL_INIT(vmap_area_list, "vmap_area_list");
>> >> SYMBOL_INIT(node_online_map, "node_online_map");
>> >>@@ -2105,6 +2107,8 @@ write_vmcoreinfo_data(void)
>> >> WRITE_SYMBOL("_stext", _stext);
>> >> WRITE_SYMBOL("swapper_pg_dir",
RE: [PATCH] handle renamed init_level4_pgt -> init_top_pgt
>>Linux 4.13 renamed init_level4_pgt to init_top_pgt in preparation for
>>introducing 5-level page tables. This patch follows the rename if
>>the lookup for init_level4_pgt fails. It also checks to see if
>>5-level page tables are enabled and bails if it discovers they are.
>
>Thanks Jeff, but could you rebase it on the current devel branch ?
>vtop4 for x86_64 was modified in the commit below:
>
>commit 8c89727155f4994b4e75a659e28e5eff16ff6cbc
>Author: Takao Indoh <indou.ta...@jp.fujitsu.com>
>Date: Thu Oct 26 20:32:54 2017 +0900
>
>[PATCH v3 2/4] Introduce vtop4_x86_64_pagetable
>
>Regards,
>Atsushi Kumagai
I rebased that and updated the devel branch:
https://sourceforge.net/p/makedumpfile/code/ci/97e156b493c46fabfc4b136ab834a6495166ac66/
I'll merge it into v1.6.3 if you have no comment.
Thanks,
Atsushi Kumagai
>>Signed-off-by: Jeff Mahoney <je...@suse.com>
>>---
>> arch/x86_64.c | 24 +---
>> makedumpfile.c | 6 ++
>> makedumpfile.h | 2 ++
>> 3 files changed, 29 insertions(+), 3 deletions(-)
>>
>>diff --git a/arch/x86_64.c b/arch/x86_64.c
>>index 08dd6b2..9b09035 100644
>>--- a/arch/x86_64.c
>>+++ b/arch/x86_64.c
>>@@ -259,16 +259,26 @@ vtop4_x86_64(unsigned long vaddr)
>> {
>> unsigned long page_dir, pml4, pgd_paddr, pgd_pte, pmd_paddr, pmd_pte;
>> unsigned long pte_paddr, pte;
>>+ unsigned long init_level4_pgt;
>>
>>- if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL) {
>>+ if (SYMBOL(init_level4_pgt) != NOT_FOUND_SYMBOL)
>>+ init_level4_pgt = SYMBOL(init_level4_pgt);
>>+ else if (SYMBOL(init_top_pgt) != NOT_FOUND_SYMBOL)
>>+ init_level4_pgt = SYMBOL(init_top_pgt);
>>+ else {
>> ERRMSG("Can't get the symbol of init_level4_pgt.\n");
>> return NOT_PADDR;
>> }
>>
>>+ if (SYMBOL(level4_kernel_pgt) != NOT_FOUND_SYMBOL) {
>>+ ERRMSG("Kernel is built with 5-level page tables\n");
>>+ return NOT_PADDR;
>>+ }
>>+
>> /*
>> * Get PGD.
>> */
>>- page_dir = SYMBOL(init_level4_pgt) - __START_KERNEL_map +
>>info->phys_base;
>>+ page_dir = init_level4_pgt - __START_KERNEL_map + info->phys_base;
>> if (is_xen_memory()) {
>> page_dir = ptom_xen(page_dir);
>> if (page_dir == NOT_PADDR)
>>@@ -549,8 +559,16 @@ find_vmemmap_x86_64()
>> struct vmap_pfns *vmapp, *vmaphead = NULL, *cur, *tail;
>>
>> init_level4_pgt = SYMBOL(init_level4_pgt);
>>+ if (init_level4_pgt == NOT_FOUND_SYMBOL)
>>+ init_level4_pgt = SYMBOL(init_top_pgt);
>>+
>> if (init_level4_pgt == NOT_FOUND_SYMBOL) {
>>- ERRMSG("init_level4_pgt not found\n");
>>+ ERRMSG("init_level4_pgt/init_top_pgt not found\n");
>>+ return FAILED;
>>+ }
>>+
>>+ if (SYMBOL(level4_kernel_pgt) != NOT_FOUND_SYMBOL) {
>>+ ERRMSG("kernel is configured for 5-level page tables\n");
>> return FAILED;
>> }
>> pagestructsize = size_table.page;
>>diff --git a/makedumpfile.c b/makedumpfile.c
>>index f85003a..6e5ec34 100644
>>--- a/makedumpfile.c
>>+++ b/makedumpfile.c
>>@@ -1486,6 +1486,8 @@ get_symbol_info(void)
>> SYMBOL_INIT(_stext, "_stext");
>> SYMBOL_INIT(swapper_pg_dir, "swapper_pg_dir");
>> SYMBOL_INIT(init_level4_pgt, "init_level4_pgt");
>>+ SYMBOL_INIT(level4_kernel_pgt, "level4_kernel_pgt");
>>+ SYMBOL_INIT(init_top_pgt, "init_top_pgt");
>> SYMBOL_INIT(vmlist, "vmlist");
>> SYMBOL_INIT(vmap_area_list, "vmap_area_list");
>> SYMBOL_INIT(node_online_map, "node_online_map");
>>@@ -2105,6 +2107,8 @@ write_vmcoreinfo_data(void)
>> WRITE_SYMBOL("_stext", _stext);
>> WRITE_SYMBOL("swapper_pg_dir", swapper_pg_dir);
>> WRITE_SYMBOL("init_level4_pgt", init_level4_pgt);
>>+ WRITE_SYMBOL("level4_kernel_pgt", level4_kernel_pgt);
>>+ WRITE_SYMBOL("init_top_pgt", init_top_pgt);
>> WRITE_SYMBOL("vmlist", vmlist);
>> WRITE_SYMBOL("vmap_area_list", vmap_area_list);
>> WRITE_SYMBOL("node_online_map", node_online_map);
>>@@ -2500,6 +2504,8 @@ read_vmcoreinfo(void)
>> READ_SYMBOL("_stext", _stext);
>> READ_SYM
RE: [makedumpfile PATCH 1/2] Fix compilation warnings on ppc64/ppc64le platforms
Hello Bhupesh,
Thanks for your work, I'll merge these patches into v1.6.3.
Regards
Atsushi Kumagai
>I ran into the following compilation warnings while compiling
>makedumpfile on ppc64/ppc64le platforms:
>
>-D_LARGEFILE64_SOURCE -DVERSION='"1.6.2"' -DRELEASE_DATE='"27 Jul 2017"'
>-D__powerpc64le__ -DUSELZO -DUSESNAPPY print_info.o dwarf_info.o
>elf_info.o erase_info.o sadump_info.o cache.o arch/arm.o arch/arm64.o
>arch/x86.o arch/x86_64.o arch/ia64.o arch/ppc64.o arch/s390x.o
>arch/ppc.o arch/sparc64.o -rdynamic -o makedumpfile makedumpfile.c
>-lpthread -lsnappy -llzo2 -ldw -lbz2 -lebl -ldl -lelf -lz
>
>makedumpfile.c: In function ‘initial_xen’:
>makedumpfile.c:9442:16: warning: unused variable ‘size’ [-Wunused-variable]
> unsigned long size;
> ^
>makedumpfile.c:9441:8: warning: unused variable ‘offset’ [-Wunused-variable]
> off_t offset;
>^
>makedumpfile.c:9440:6: warning: unused variable ‘xen_info_required’
>[-Wunused-variable]
> int xen_info_required = TRUE;
>^
>
>This is caused because while we don't support Xen on powerpc,
>we still have the above variables declared outside the
>'#if defined(__powerpc64__) || defined(__powerpc32__)'
>check.
>
>This patch fixes the same by ensuring that these variables are only
>declared for archs != powerpc.
>
>Signed-off-by: Bhupesh Sharma <bhsha...@redhat.com>
>---
> makedumpfile.c | 8
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 6d5fc8b95415..7ce0c6d648aa 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -9437,15 +9437,15 @@ exclude_xen_user_domain(void)
> int
> initial_xen(void)
> {
>- int xen_info_required = TRUE;
>- off_t offset;
>- unsigned long size;
>-
> #if defined(__powerpc64__) || defined(__powerpc32__)
> MSG("\n");
> MSG("Xen is not supported on powerpc.\n");
> return FALSE;
> #else
>+ int xen_info_required = TRUE;
>+ off_t offset;
>+ unsigned long size;
>+
> #ifndef __x86_64__
> if (DL_EXCLUDE_ZERO < info->max_dump_level) {
> MSG("Dump_level is invalid. It should be 0 or 1.\n");
>--
>2.7.4
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH v3 0/4] Fix KASLR problem on sadump
Hello Indoh-san, >Hi Kumagai-san, Hatayama-san, > >These patch series fix a problem that makedumpfile cannot handle a dumpfile >which is captured by sadump on KASLR enabled kernel. > >When KASLR feature is enabled, a kernel is placed on the memory randomly and >therefore makedumpfile cannot handle a dumpfile because addresses of kernel >symbols in vmlinux are different from actual addresses. In the case of kdump, >information to get actual address is included in the vmcoreinfo, but dumpfile >of >sadump does not have such a information. > >These patches calculate kaslr offset and phys_base to solve this problem. The >basic idea is getting register (IDTR and CR3) from dump header, and calculate >kaslr_offset/phys_base using them. > >changelog: >v3: >- Split patch 3/3 into two parts > - core part to calculate kaslr_offset and phys_base > - Additional part to fix this problem during kdump Thanks for your work, I'll merge this version into v1.6.3. Regards, Atsushi Kumagai >v2: >http://lists.infradead.org/pipermail/kexec/2017-October/019554.html >- Change get_vec0_addr style >- Some tiny fixes > >v1: >http://lists.infradead.org/pipermail/kexec/2017-October/019530.html > >Takao Indoh (4): > Support symbol __cpu_online_mask > Introduce vtop4_x86_64_pagetable > sadump: Fix a KASLR problem of sadump > sadump: Fix a KASLR problem of sadump while kdump is working > > arch/x86_64.c | 30 - > makedumpfile.c | 20 ++- > makedumpfile.h | 8 +- > sadump_info.c | 417 - > 4 files changed, 462 insertions(+), 13 deletions(-) > >-- >2.9.5 > > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH v2 3/3] sadump: Fix a KASLR problem of sadump
quot;)) {
>+ nhdr_offset = offset;
>+ break;
>+ }
>+ }
>+
>+ if (!nhdr_offset)
>+ return FALSE;
>+
>+ *addr = nhdr_offset +
>+ divideup(nhdr.n_namesz, sizeof(Elf64_Word))*
>+ sizeof(Elf64_Word);
>+ *len = nhdr.n_descsz;
>+
>+ DEBUG_MSG("sadump: vmcoreinfo addr: %lx\n", *addr);
>+ DEBUG_MSG("sadump: vmcoreinfo len: %d\n", *len);
>+
>+ return TRUE;
>+}
>+
>+/*
>+ * Check if current kaslr_offset/phys_base is for 1st kernel or 2nd kernel.
>+ * If we are in 2nd kernel, get kaslr_offset/phys_base from vmcoreinfo.
>+ *
>+ * 1. Get command line and try to retrieve "elfcorehdr=" boot parameter
>+ * 2. If "elfcorehdr=" is not found in command line, we are in 1st kernel.
>+ *There is nothing to do.
>+ * 3. If "elfcorehdr=" is found, we are in 2nd kernel. Find vmcoreinfo
>+ *using "elfcorehdr=" and retrieve kaslr_offset/phys_base from vmcoreinfo.
>+ */
>+int
>+get_kaslr_offset_from_vmcoreinfo(ulong cr3, ulong *kaslr_offset,
>+ ulong *phys_base)
>+{
>+ ulong elfcorehdr_addr = 0;
>+ ulong vmcoreinfo_addr;
>+ int vmcoreinfo_len;
>+ char *buf, *pos;
>+ int ret = FALSE;
>+
>+ elfcorehdr_addr = get_elfcorehdr(cr3);
>+ if (!elfcorehdr_addr)
>+ return FALSE;
>+
>+ if (!get_vmcoreinfo_in_kdump_kernel(elfcorehdr_addr, _addr,
>+ _len))
>+ return FALSE;
>+
>+ if (!vmcoreinfo_len)
>+ return FALSE;
>+
>+ DEBUG_MSG("sadump: Find vmcoreinfo in kdump memory\n");
>+
>+ if (!(buf = malloc(vmcoreinfo_len))) {
>+ ERRMSG("Can't allocate vmcoreinfo buffer.\n");
>+ return FALSE;
>+ }
>+
>+ if (!readmem(PADDR, vmcoreinfo_addr, buf, vmcoreinfo_len))
>+ goto finish;
>+
>+ pos = strstr(buf, STR_NUMBER("phys_base"));
>+ if (!pos)
>+ goto finish;
>+ *phys_base = strtoull(pos + strlen(STR_NUMBER("phys_base")), NULL, 0);
>+
>+ pos = strstr(buf, STR_KERNELOFFSET);
>+ if (!pos)
>+ goto finish;
>+ *kaslr_offset = strtoull(pos + strlen(STR_KERNELOFFSET), NULL, 16);
>+ ret = TRUE;
>+
>+finish:
>+ free(buf);
>+ return ret;
>+}
>+
>+/*
>+ * Calculate kaslr_offset and phys_base
>+ *
>+ * kaslr_offset:
>+ * The difference between original address in vmlinux and actual address
>+ * placed randomly by kaslr feature. To be more accurate,
>+ * kaslr_offset = actual address - original address
>+ *
>+ * phys_base:
>+ * Physical address where the kerenel is placed. In other words, it's a
>+ * physical address of __START_KERNEL_map. This is also decided randomly by
>+ * kaslr.
>+ *
>+ * kaslr offset and phys_base are calculated as follows:
>+ *
>+ * kaslr_offset:
>+ * 1) Get IDTR and CR3 value from the dump header.
>+ * 2) Get a virtual address of IDT from IDTR value
>+ *--- (A)
>+ * 3) Translate (A) to physical address using CR3, which points a top of
>+ *page table.
>+ *--- (B)
>+ * 4) Get an address of vector0 (Devide Error) interrupt handler from
>+ *IDT, which are pointed by (B).
>+ *--- (C)
>+ * 5) Get an address of symbol "divide_error" form vmlinux
>+ *--- (D)
>+ *
>+ * Now we have two addresses:
>+ * (C)-> Actual address of "divide_error"
>+ * (D)-> Original address of "divide_error" in the vmlinux
>+ *
>+ * kaslr_offset can be calculated by the difference between these two
>+ * value.
>+ *
>+ * phys_base;
>+ * 1) Get IDT virtual address from vmlinux
>+ *--- (E)
>+ *
>+ * So phys_base can be calculated using relationship of directly mapped
>+ * address.
>+ *
>+ * phys_base =
>+ * Physical address(B) -
>+ * (Virtual address(E) + kaslr_offset - __START_KERNEL_map)
>+ *
>+ * Note that the address (A) cannot be used instead of (E) because (A) is
>+ * not direct map address, it's a fixed map address.
>+ *
>+ * This solution works in most every case, but does not work in the
>+ * following case.
>+ *
>+ * 1) If the dump is captured on early stage of kernel boot, IDTR points
>+ *early IDT table(early_idts) instead of normal IDT(idt_table).
>+ * 2) If the dump is captured whle kdump is working, IDTR points
^i
>+ *IDT table of 2nd kernel, not 1st kernel.
These cases sound like only for outside dump mechanisms like sadump, right
RE: [Makedumpfile PATCH v2] book3s/ppc64: Lower the max real address to 53 bits for kernels >= v4.11
Hello,
>On Wednesday 13 September 2017 01:33 AM, Bhupesh Sharma wrote:
>
>
> Kernel commit 2f18d533757da3899f4bedab0b2c051b080079dc lowered the
> max real address on ppc64 to 53 bits.
>
> Make similar changes in makedumpfile (on basis of the underlying kernel
> version), without which the makedumpfile will fail to create a dumpfile
> and instead throw a SEGV fault as shown below on kernels >= v4.11:
>
># makedumpfile --split -d 31 -x vmlinux vmcore dumpfile_{1,2,3} 2>&1
>
>The kernel version is not supported.
>The makedumpfile operation may be incomplete.
>[ 1196.252094] makedumpfile[2367]: unhandled signal 11 at
>0100f7011ca8 nip 1001eecc lr 1001f3c0 code 30001
>Segmentation fault
>
> Signed-off-by: Bhupesh Sharma <bhsha...@redhat.com>
> <mailto:bhsha...@redhat.com>
>
>
>Looks good...
>Acked-by: Hari Bathini <hbath...@linux.vnet.ibm.com>
><mailto:hbath...@linux.vnet.ibm.com>
Thanks, I'll merge this version into v1.6.3.
Regards,
Atsushi Kumagai
>
> ---
> Changes since v1:
> - As per Atsushi's comments introduced macros for 4_11 directly
> in v2 and use them in arch/ppc64.c
>
>arch/ppc64.c | 8 +++-
>makedumpfile.h | 5 +
>2 files changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/arch/ppc64.c b/arch/ppc64.c
> index bacac778f73c..2f5a0daa16b2 100644
> --- a/arch/ppc64.c
> +++ b/arch/ppc64.c
> @@ -307,11 +307,17 @@ ppc64_vmalloc_init(void)
> }
>
> info->pte_rpn_mask = PTE_RPN_MASK_DEFAULT;
> - if (info->kernel_version >= KERNEL_VERSION(4, 6, 0)) {
> + if ((info->kernel_version >= KERNEL_VERSION(4, 6, 0)) &&
> + (info->kernel_version < KERNEL_VERSION(4, 11, 0))) {
> info->pte_rpn_mask = PTE_RPN_MASK_L4_4_6;
> info->pte_rpn_shift = PTE_RPN_SHIFT_L4_4_6;
> }
>
> + if (info->kernel_version >= KERNEL_VERSION(4, 11, 0)) {
> + info->pte_rpn_mask = PTE_RPN_MASK_L4_4_11;
> + info->pte_rpn_shift = PTE_RPN_SHIFT_L4_4_11;
> + }
> +
> /*
>* Compute ptrs per each level
>*/
> diff --git a/makedumpfile.h b/makedumpfile.h
> index 7d81bbcf2234..f4ba02d11f09 100644
> --- a/makedumpfile.h
> +++ b/makedumpfile.h
> @@ -692,6 +692,11 @@ unsigned long get_kvbase_arm64(void);
>#define PUD_MASKED_BITS_4_7 0xc0ffUL
>#define PMD_MASKED_BITS_4_7 0xc0ffUL
>
> +#define PTE_RPN_SIZE_L4_4_11 53
> +#define PTE_RPN_MASK_L4_4_11 \
> + (((1UL << PTE_RPN_SIZE_L4_4_11) - 1) & ~((1UL <<
> info->page_shift) - 1))
> +#define PTE_RPN_SHIFT_L4_4_11 info->page_shift
> +
>/*
> * Supported MMU types
> */
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH] book3s/ppc64: Lower the max real address to 53 bits for kernels >= v4.11
Hello Bhupesh,
>Kernel commit 2f18d533757da3899f4bedab0b2c051b080079dc lowered the
>max real address on ppc64 to 53 bits.
>
>Make similar changes in makedumpfile (on basis of the underlying kernel
>version), without which the makedumpfile will fail to create a dumpfile
>and instead throw a SEGV fault as shown below on kernels >= v4.11:
>
> # makedumpfile --split -d 31 -x vmlinux vmcore dumpfile_{1,2,3} 2>&1
>
> The kernel version is not supported.
> The makedumpfile operation may be incomplete.
> [ 1196.252094] makedumpfile[2367]: unhandled signal 11 at
> 0100f7011ca8 nip 1001eecc lr 1001f3c0 code 30001
> Segmentation fault
>
>Signed-off-by: Bhupesh Sharma <bhsha...@redhat.com>
>---
> makedumpfile.h | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 7d81bbcf2234..142753d84e8d 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -684,8 +684,9 @@ unsigned long get_kvbase_arm64(void);
> #define PMD_MASKED_BITS_64K_4_6 0xc0ffUL
>
> #define PTE_RPN_MASK_DEFAULT 0xUL
>-#define PTE_RPN_SIZE_L4_4_6 (info->page_size == 65536 ? 41 : 45)
>-#define PTE_RPN_MASK_L4_4_6 (((1UL << PTE_RPN_SIZE_L4_4_6) - 1) <<
>info->page_shift)
>+#define PAGE_PA_MAX_L4_4_6(info->kernel_version >= KERNEL_VERSION(4,11,0)
>? 53 : 57)
>+#define PTE_RPN_MASK_L4_4_6 \
>+ (((1UL << PAGE_PA_MAX_L4_4_6) - 1) & ~((1UL << info->page_shift) - 1))
This fix is a bit complicated, please introduce *_L4_4_11 simply.
Thanks
Atsushi Kumagai
> #define PTE_RPN_SHIFT_L4_4_6 info->page_shift
>
> #define PGD_MASKED_BITS_4_7 0xc0ffUL
>--
>2.7.4
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH v2 0/2] makedumpfile/ppc64: enable mem-usage option
Hello Pingfan, >v1->v2: >follow the scheme for arch dependent code This is what I expected, thanks! I'll merge this into v1.6.3. Regards, Atsushi Kumagai >Pingfan Liu (2): > makedumpfile/ppc64: set page_offset in get_versiondep_info_ppc64() > makedumpfile/ppc64: get the info of mem reserved for crashkernel > > arch/ppc64.c | 37 + > makedumpfile.c | 3 +++ > makedumpfile.h | 11 +++ > 3 files changed, 51 insertions(+) > >-- >2.7.4 > > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH v2] Fix SECTION_MAP_MASK for kernel >= v.13
>commit 2d070eab2e82 "mm: consider zone which is not fully populated to >have holes" added a new flag SECTION_IS_ONLINE and therefore >SECTION_MAP_MASK has been changed. We are not able to find correct >mem_map in makedumpfile for kernel version v4.13-rc1 and onward because >of the above kernel change. > >This patch fixes the MASK value keeping the code backward compatible > >Signed-off-by: Pratyush Anand <pan...@redhat.com> >--- >v1->v2: Improved kernel_version comparison to take care of stable kernel >versions as well. Thanks, I'll merge this into v1.6.3. Regards, Atsushi Kumagai > makedumpfile.c | 5 - > makedumpfile.h | 4 +++- > 2 files changed, 7 insertions(+), 2 deletions(-) > >diff --git a/makedumpfile.c b/makedumpfile.c >index 30230a15a2e7..c975651ca357 100644 >--- a/makedumpfile.c >+++ b/makedumpfile.c >@@ -3304,7 +3304,10 @@ section_mem_map_addr(unsigned long addr) > return NOT_KV_ADDR; > } > map = ULONG(mem_section + OFFSET(mem_section.section_mem_map)); >- map &= SECTION_MAP_MASK; >+ if (info->kernel_version < KERNEL_VERSION(4, 13, 0)) >+ map &= SECTION_MAP_MASK_4_12; >+ else >+ map &= SECTION_MAP_MASK; > free(mem_section); > > return map; >diff --git a/makedumpfile.h b/makedumpfile.h >index 8a05794843fb..322f28c632b0 100644 >--- a/makedumpfile.h >+++ b/makedumpfile.h >@@ -183,7 +183,9 @@ isAnon(unsigned long mapping) > #define SECTIONS_PER_ROOT() (info->sections_per_root) > #define SECTION_ROOT_MASK() (SECTIONS_PER_ROOT() - 1) > #define SECTION_NR_TO_ROOT(sec) ((sec) / SECTIONS_PER_ROOT()) >-#define SECTION_MAP_LAST_BIT (1UL<<2) >+#define SECTION_IS_ONLINE (1UL<<2) >+#define SECTION_MAP_LAST_BIT (1UL<<3) >+#define SECTION_MAP_MASK_4_12 (~(SECTION_IS_ONLINE-1)) > #define SECTION_MAP_MASK (~(SECTION_MAP_LAST_BIT-1)) > #define NR_SECTION_ROOTS()divideup(num_section, SECTIONS_PER_ROOT()) > #define SECTION_NR_TO_PFN(sec)((sec) << PFN_SECTION_SHIFT()) >-- >2.9.4 > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 2/2] makedumpfile/ppc64: get the info of mem reserved for crashkernel
Hello Pingfan,
>In kernel, ppc64 does not export the mem layout by ioresource. So we
>need to get the mem info for crashkernel from device tree.
>
>Signed-off-by: Pingfan Liu <pi...@redhat.com>
>---
> arch/ppc64.c | 36
> makedumpfile.c | 10 ++
> makedumpfile.h | 4
> 3 files changed, 50 insertions(+)
>
>diff --git a/arch/ppc64.c b/arch/ppc64.c
>index 3fd6002..360590e 100644
>--- a/arch/ppc64.c
>+++ b/arch/ppc64.c
>@@ -617,4 +617,40 @@ vaddr_to_paddr_ppc64(unsigned long vaddr)
> return ppc64_vtop_level4(vaddr);
> }
>
>+int arch_crashkernel_mem_size()
>+{
>+ const char f_crashsize[] =
>"/proc/device-tree/chosen/linux,crashkernel-size";
>+ const char f_crashbase[] =
>"/proc/device-tree/chosen/linux,crashkernel-base";
>+ unsigned long crashk_sz_be, crashk_sz;
>+ unsigned long crashk_base_be, crashk_base;
>+ uint swap;
>+ FILE *fp, *fpb;
>+
>+ fp = fopen(f_crashsize, "r");
>+ if (!fp) {
>+ ERRMSG("Cannot open %s\n", f_crashsize);
>+ return FALSE;
>+ }
>+ fpb = fopen(f_crashbase, "r");
>+ if (!fp) {
>+ ERRMSG("Cannot open %s\n", f_crashbase);
>+ fclose(fp);
>+ return FALSE;
>+ }
>+
>+ fread(_sz_be, sizeof(crashk_sz_be), 1, fp);
>+ fread(_base_be, sizeof(crashk_base_be), 1, fpb);
>+ fclose(fp);
>+ fclose(fpb);
>+ /* dev tree is always big endian */
>+ swap = !is_bigendian();
>+ crashk_sz = swap64(crashk_sz_be, swap);
>+ crashk_base = swap64(crashk_base_be, swap);
>+ crash_reserved_mem_nr = 1;
>+ crash_reserved_mem[0].start = crashk_base;
>+ crash_reserved_mem[0].end = crashk_base + crashk_sz - 1;
>+
>+ return TRUE;
>+}
>+
> #endif /* powerpc64 */
>diff --git a/makedumpfile.c b/makedumpfile.c
>index f85003a..c599b91 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -10921,10 +10921,20 @@ static int crashkernel_mem_callback(void *data, int
>nr,
> return 0;
> }
>
>+#if !defined(HAVE_ARCH_CRASHKERNEL_MEM_SIZE)
>+int arch_crashkernel_mem_size()
>+{
>+ return FALSE;
>+}
>+#endif
>+
I hope you follow the scheme for arch dependent code like get_phys_base()
to get rid of such ifdef. Please see makedumpfile.h for details,
my idea is like below:
#ifdef __powerpc64__
#define arch_crashkernel_mem_size() arch_crashkernel_mem_size_ppc64()
#ifdef
#define arch_crashkernel_mem_size() stub_false()
Thanks,
Atsushi Kumagai
> int is_crashkernel_mem_reserved(void)
> {
> int ret;
>
>+ if (arch_crashkernel_mem_size())
>+ return TRUE;
>+
> ret = iomem_for_each_line("Crash kernel\n",
> crashkernel_mem_callback, NULL);
> crash_reserved_mem_nr = ret;
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 8a05794..48c1423 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -987,6 +987,8 @@ unsigned long long vaddr_to_paddr_ppc64(unsigned long
>vaddr);
> #define get_kaslr_offset(X) stub_false()
> #define vaddr_to_paddr(X) vaddr_to_paddr_ppc64(X)
> #define is_phys_addr(X) stub_true_ul(X)
>+
>+#define HAVE_ARCH_CRASHKERNEL_MEM_SIZE
> #endif /* powerpc64 */
>
> #ifdef __powerpc32__ /* powerpc32 */
>@@ -1939,6 +1941,8 @@ int iomem_for_each_line(char *match, int
>(*callback)(void *data, int nr,
>unsigned long base,
>unsigned long length),
> void *data);
>+int is_bigendian(void);
>+int arch_crashkernel_mem_size(void);
>
>
> /*
>--
>2.7.4
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH v2] makedumpfile: arm64: Fix page table walk of 1GB section
>On Thursday 17 August 2017 05:31 AM, Bradley Bolen wrote:
>> From: Bradley Bolen <bbo...@lexmark.com>
>>
>> makedumpfile was generating large (> 500MB) vmcore files for an arm64
>> board with 2GB of DRAM. It was not excluding any pages because the
>> mem_map address was not being converted correctly.
>>
>> readmem: Can't convert a virtual address(ffc07fff6000) to physical
>> address.
>> readmem: type_addr: 0, addr:ffc07fff6000, size:16
>> section_mem_map_addr: Can't get a struct mem_section(ffc07fff6000).
>> mem_map (0)
>> mem_map : 0
>>
>> makedumpfile was not handling 1GB sections in the PGD and was trying to
>> drill down to a PTE in which it was trying to dereference invalid
>> memory. This patch adds code to check the PGD for a section type and
>> handle it instead of treating it as a table entry.
>>
>> Signed-off-by: Bradley Bolen <bradleybo...@gmail.com>
I'll merge this patch into v1.6.3, thanks,
>Reviewed-by: Pratyush Anand <pan...@redhat.com>
Thanks for your comment for the comments, actually I was just wondering
about those comments.
Regards,
Atsushi Kumagai
>> ---
>> Changes in v2:
>> - updated comments
>>
>> arch/arm64.c | 15 ++-
>> 1 file changed, 14 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm64.c b/arch/arm64.c
>> index 958f57f..25d7a1f 100644
>> --- a/arch/arm64.c
>> +++ b/arch/arm64.c
>> @@ -57,6 +57,8 @@ static unsigned long kimage_voffset;
>> #define PGDIR_SHIFT((PAGESHIFT() - 3) * pgtable_level + 3)
>> #define PTRS_PER_PGD (1 << (va_bits - PGDIR_SHIFT))
>> #define PUD_SHIFT get_pud_shift_arm64()
>> +#define PUD_SIZE(1UL << PUD_SHIFT)
>> +#define PUD_MASK(~(PUD_SIZE - 1))
>> #define PTRS_PER_PTE (1 << (PAGESHIFT() - 3))
>> #define PTRS_PER_PUD PTRS_PER_PTE
>> #define PMD_SHIFT ((PAGESHIFT() - 3) * 2 + 3)
>> @@ -79,6 +81,10 @@ static unsigned long kimage_voffset;
>> #define PMD_TYPE_SECT 1
>> #define PMD_TYPE_TABLE 3
>>
>> +#define PUD_TYPE_MASK 3
>> +#define PUD_TYPE_SECT 1
>> +#define PUD_TYPE_TABLE 3
>> +
>> #define pgd_index(vaddr) (((vaddr) >> PGDIR_SHIFT) &
>> (PTRS_PER_PGD - 1))
>> #define pgd_offset(pgdir, vaddr) ((pgd_t *)(pgdir) + pgd_index(vaddr))
>>
>> @@ -253,6 +259,13 @@ vaddr_to_paddr_arm64(unsigned long vaddr)
>> return NOT_PADDR;
>> }
>>
>> +if ((pud_val(pudv) & PUD_TYPE_MASK) == PUD_TYPE_SECT) {
>> +/* 1GB section for Page Table level = 4 and Page Size = 4KB */
>> +paddr = (pud_val(pudv) & (PUD_MASK & PMD_SECTION_MASK))
>> ++ (vaddr & (PUD_SIZE - 1));
>> +return paddr;
>> +}
>> +
>> pmda = pmd_offset(puda, , vaddr);
>> if (!readmem(PADDR, (unsigned long long)pmda, , sizeof(pmdv))) {
>> ERRMSG("Can't read pmd\n");
>> @@ -278,7 +291,7 @@ vaddr_to_paddr_arm64(unsigned long vaddr)
>> }
>> break;
>> case PMD_TYPE_SECT:
>> -/* 1GB section */
>> +/* 512MB section for Page Table level = 3 and Page Size = 64KB*/
>> paddr = (pmd_val(pmdv) & (PMD_MASK & PMD_SECTION_MASK))
>> + (vaddr & (PMD_SIZE - 1));
>> break;
>>
>
>--
>Regards
>Pratyush
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH] makedumpfile: arm64: Fix page table walk of 1GB section
Hello Bradley,
Sorry for my late reply, I'll merge this patch into v1.6.3.
Thanks for your work.
Regards,
Atsushi Kumagai
>makedumpfile was generating large (> 500MB) vmcore files for an arm64
>board with 2GB of DRAM. It was not excluding any pages because the
>mem_map address was not being converted correctly.
>
>readmem: Can't convert a virtual address(ffc07fff6000) to physical
>address.
>readmem: type_addr: 0, addr:ffc07fff6000, size:16
>section_mem_map_addr: Can't get a struct mem_section(ffc07fff6000).
>mem_map (0)
>mem_map : 0
>
>makedumpfile was not handling 1GB sections in the PGD and was trying to
>drill down to a PTE in which it was trying to dereference invalid
>memory. This patch adds code to check the PGD for a section type and
>handle it instead of treating it as a table entry.
>
>Signed-off-by: Bradley Bolen <bradleybo...@gmail.com>
>---
> arch/arm64.c | 13 +
> 1 file changed, 13 insertions(+)
>
>diff --git a/arch/arm64.c b/arch/arm64.c
>index 958f57f..cae4b70 100644
>--- a/arch/arm64.c
>+++ b/arch/arm64.c
>@@ -57,6 +57,8 @@ static unsigned long kimage_voffset;
> #define PGDIR_SHIFT ((PAGESHIFT() - 3) * pgtable_level + 3)
> #define PTRS_PER_PGD (1 << (va_bits - PGDIR_SHIFT))
> #define PUD_SHIFT get_pud_shift_arm64()
>+#define PUD_SIZE (1UL << PUD_SHIFT)
>+#define PUD_MASK (~(PUD_SIZE - 1))
> #define PTRS_PER_PTE (1 << (PAGESHIFT() - 3))
> #define PTRS_PER_PUD PTRS_PER_PTE
> #define PMD_SHIFT ((PAGESHIFT() - 3) * 2 + 3)
>@@ -79,6 +81,10 @@ static unsigned long kimage_voffset;
> #define PMD_TYPE_SECT 1
> #define PMD_TYPE_TABLE3
>
>+#define PUD_TYPE_MASK 3
>+#define PUD_TYPE_SECT 1
>+#define PUD_TYPE_TABLE3
>+
> #define pgd_index(vaddr) (((vaddr) >> PGDIR_SHIFT) &
> (PTRS_PER_PGD - 1))
> #define pgd_offset(pgdir, vaddr) ((pgd_t *)(pgdir) + pgd_index(vaddr))
>
>@@ -253,6 +259,13 @@ vaddr_to_paddr_arm64(unsigned long vaddr)
> return NOT_PADDR;
> }
>
>+ if ((pud_val(pudv) & PUD_TYPE_MASK) == PUD_TYPE_SECT) {
>+ /* 1GB section */
>+ paddr = (pud_val(pudv) & (PUD_MASK & PMD_SECTION_MASK))
>+ + (vaddr & (PUD_SIZE - 1));
>+ return paddr;
>+ }
>+
> pmda = pmd_offset(puda, , vaddr);
> if (!readmem(PADDR, (unsigned long long)pmda, , sizeof(pmdv))) {
> ERRMSG("Can't read pmd\n");
>--
>1.9.3
>
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH] Fix SECTION_MAP_MASK for kernel >= v.13
Hello Pratyush, >commit 2d070eab2e82 "mm: consider zone which is not fully populated to >have holes" added a new flag SECTION_IS_ONLINE and therefore >SECTION_MAP_MASK has been changed. We are not able to find correct >mem_map in makedumpfile for kernel version v4.13-rc1 and onward because >of the above kernel change. > >This patch fixes the MASK value keeping the code backward compatible > >Signed-off-by: Pratyush Anand <pan...@redhat.com> >--- > makedumpfile.c | 5 - > makedumpfile.h | 4 +++- > 2 files changed, 7 insertions(+), 2 deletions(-) > >diff --git a/makedumpfile.c b/makedumpfile.c >index 30230a15a2e7..7350e5b5af95 100644 >--- a/makedumpfile.c >+++ b/makedumpfile.c >@@ -3304,7 +3304,10 @@ section_mem_map_addr(unsigned long addr) > return NOT_KV_ADDR; > } > map = ULONG(mem_section + OFFSET(mem_section.section_mem_map)); >- map &= SECTION_MAP_MASK; >+ if (info->kernel_version <= KERNEL_VERSION(4, 12, 0)) I think this condition should be "< KERNEL_VERSION(4, 13, 0)", actually v4.12.7 doesn't have SECTION_IS_ONLINE. Thanks, Atsushi Kumagai >+ map &= SECTION_MAP_MASK_4_12; >+ else >+ map &= SECTION_MAP_MASK; > free(mem_section); > > return map; >diff --git a/makedumpfile.h b/makedumpfile.h >index 8a05794843fb..322f28c632b0 100644 >--- a/makedumpfile.h >+++ b/makedumpfile.h >@@ -183,7 +183,9 @@ isAnon(unsigned long mapping) > #define SECTIONS_PER_ROOT() (info->sections_per_root) > #define SECTION_ROOT_MASK() (SECTIONS_PER_ROOT() - 1) > #define SECTION_NR_TO_ROOT(sec) ((sec) / SECTIONS_PER_ROOT()) >-#define SECTION_MAP_LAST_BIT (1UL<<2) >+#define SECTION_IS_ONLINE (1UL<<2) >+#define SECTION_MAP_LAST_BIT (1UL<<3) >+#define SECTION_MAP_MASK_4_12 (~(SECTION_IS_ONLINE-1)) > #define SECTION_MAP_MASK (~(SECTION_MAP_LAST_BIT-1)) > #define NR_SECTION_ROOTS()divideup(num_section, SECTIONS_PER_ROOT()) > #define SECTION_NR_TO_PFN(sec)((sec) << PFN_SECTION_SHIFT()) >-- >2.9.4 ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH] Fix formatting problems in header file
Hello,
>The list of #defines uses spaces to separate the value from the
>name. A few used tabs, so those tabs have been converted to
>spaces so that the values align again.
>
>Signed-off-by: Eric DeVolder <eric.devol...@oracle.com>
Thanks, I'll merge this into v1.6.3.
Regards,
Atsushi Kumagai
>---
> makedumpfile.h | 8
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 8a05794..34dd8d7 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -2244,7 +2244,7 @@ struct elf_prstatus {
> #define OPT_DEBUG 'D'
> #define OPT_DUMP_LEVEL 'd'
> #define OPT_ELF_DUMPFILE'E'
>-#define OPT_EXCLUDE_UNUSED_VM 'e'
>+#define OPT_EXCLUDE_UNUSED_VM 'e'
> #define OPT_FLATTEN 'F'
> #define OPT_FORCE 'f'
> #define OPT_GENERATE_VMCOREINFO 'g'
>@@ -2271,10 +2271,10 @@ struct elf_prstatus {
> #define OPT_EPPIC OPT_START+11
> #define OPT_NON_MMAPOPT_START+12
> #define OPT_MEM_USAGE OPT_START+13
>-#define OPT_SPLITBLOCK_SIZE OPT_START+14
>+#define OPT_SPLITBLOCK_SIZE OPT_START+14
> #define OPT_WORKING_DIR OPT_START+15
>-#define OPT_NUM_THREADS OPT_START+16
>-#define OPT_PARTIAL_DMESG OPT_START+17
>+#define OPT_NUM_THREADS OPT_START+16
>+#define OPT_PARTIAL_DMESG OPT_START+17
>
> /*
> * Function Prototype.
>--
>2.7.4
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH v2] Wipe excluded pages that are written into ELF dump file
Hello Eric,
>When a page is excluded by any of the existing dump levels,
>that page may still be written to the ELF dump file, depending
>upon the PFN_EXCLUDED mechanism.
>
>The PFN_EXCLUDED mechanism looks for N consecutive "not
>dumpable" pages, and if found, the current ELF segment is
>closed out and a new ELF segment started, at the next dumpable
>page. Otherwise, if the PFN_EXCLUDED criteria is not meet (that
>is, there is a mix of dumpable and not dumpable pages, but not
>N consecutive not dumpable pages) all pages are written to the
>dump file.
>
>This patch implements a mechanism for those "not dumpable" pages
>that are written to the ELF dump file to fill those pages with
>constant data, rather than the original data. In other words,
>the dump file still contains the page, but its data is wiped.
>The data is wiped with the value 0xDEAD9A6EDEAD9A6EUL (an
>attempt at DEADPAGE in hex, which works for 32-bit targets as
>well).
>
>The motivation for doing this is to protect real user (customer)
>data from "leaking" through to a dump file when that data was
>intended to be omitted.
>
>Signed-off-by: Eric DeVolder <eric.devol...@oracle.com>
>---
>v2: posted 04aug2017 to mailing list
> - Incorporate feedback from Daniel Kiper (wipe value)
> - Incorporate feedback from Atsushi Kumagai (eliminate the
> option and make as default/builtin behavior)
Thanks for your work, this version looks good to me.
I'll merge this into v1.6.3.
Regards,
Atsushi Kumagai
>v1: posted 31jul2017 to mailing list
>---
> makedumpfile.c | 27 ---
> makedumpfile.h | 1 +
> 2 files changed, 21 insertions(+), 7 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index f85003a..66c3105 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -7139,7 +7139,7 @@ out:
>
> int
> write_elf_load_segment(struct cache_data *cd_page, unsigned long long paddr,
>- off_t off_memory, long long size)
>+ off_t off_memory, long long size, struct cycle *cycle)
> {
> long page_size = info->page_size;
> long long bufsz_write;
>@@ -7163,10 +7163,23 @@ write_elf_load_segment(struct cache_data *cd_page,
>unsigned long long paddr,
> else
> bufsz_write = size;
>
>- if (read(info->fd_memory, buf, bufsz_write) != bufsz_write) {
>- ERRMSG("Can't read the dump memory(%s). %s\n",
>- info->name_memory, strerror(errno));
>- return FALSE;
>+ if (!is_dumpable(info->bitmap2, paddr_to_pfn(paddr), cycle)) {
>+ unsigned k;
>+ unsigned long *p = (unsigned long *)buf;
>+ for (k = 0; k < info->page_size; k += sizeof(unsigned
>long)) {
>+ *p++ = FILL_EXCLUDED_PAGES_VALUE;
>+ }
>+ if (lseek(info->fd_memory, bufsz_write, SEEK_CUR) < 0) {
>+ ERRMSG("Can't seek the dump memory(%s). %s\n",
>+ info->name_memory, strerror(errno));
>+ return FALSE;
>+ }
>+ } else {
>+ if (read(info->fd_memory, buf, bufsz_write) !=
>bufsz_write) {
>+ ERRMSG("Can't read the dump memory(%s). %s\n",
>+ info->name_memory, strerror(errno));
>+ return FALSE;
>+ }
> }
> filter_data_buffer((unsigned char *)buf, paddr, bufsz_write);
> paddr += bufsz_write;
>@@ -7431,7 +7444,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header,
>struct cache_data *cd_page)
>*/
> if (load.p_filesz)
> if (!write_elf_load_segment(cd_page,
> paddr,
>- off_memory,
>load.p_filesz))
>+ off_memory,
>load.p_filesz, ))
> return FALSE;
>
> load.p_paddr += load.p_memsz;
>@@ -7473,7 +7486,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header,
>struct cache_data *cd_page)
>*/
> if (load.p_filesz)
> if (!write_elf_load_segment(cd_page, paddr,
>- off_memory, load.p_filesz))
>+
RE: [Makedumpfile PATCH v2] x86_64: Take care of init_level4_pgt rename in kernel
Hello Pratyush,
>Following commit renamed init_level4_pgt to init_top_pgt in kernel.
>
>commit 65ade2f872b474fa8a04c2d397783350326634e6
>Author: Kirill A. Shutemov <kirill.shute...@linux.intel.com>
>Date: Tue Jun 6 14:31:27 2017 +0300
>
>x86/boot/64: Rename init_level4_pgt and early_level4_pgt
>
>This patch takes care of above kernel modification in makedumpfile.
>
>Signed-off-by: Pratyush Anand <pan...@redhat.com>
>---
>v2 -> v1
>Removed redundant 'if condition' for WRITE_SYMBOL().
Thanks, I'll merge this version into v1.6.3.
Regards,
Atsushi Kumagai
> makedumpfile.c | 4
> 1 file changed, 4 insertions(+)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index f85003a33551..30230a15a2e7 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -1486,6 +1486,8 @@ get_symbol_info(void)
> SYMBOL_INIT(_stext, "_stext");
> SYMBOL_INIT(swapper_pg_dir, "swapper_pg_dir");
> SYMBOL_INIT(init_level4_pgt, "init_level4_pgt");
>+ if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL)
>+ SYMBOL_INIT(init_level4_pgt, "init_top_pgt");
> SYMBOL_INIT(vmlist, "vmlist");
> SYMBOL_INIT(vmap_area_list, "vmap_area_list");
> SYMBOL_INIT(node_online_map, "node_online_map");
>@@ -2500,6 +2502,8 @@ read_vmcoreinfo(void)
> READ_SYMBOL("_stext", _stext);
> READ_SYMBOL("swapper_pg_dir", swapper_pg_dir);
> READ_SYMBOL("init_level4_pgt", init_level4_pgt);
>+ if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL)
>+ READ_SYMBOL("init_top_pgt", init_level4_pgt);
> READ_SYMBOL("vmlist", vmlist);
> READ_SYMBOL("vmap_area_list", vmap_area_list);
> READ_SYMBOL("node_online_map", node_online_map);
>--
>2.9.4
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH] Wipe excluded pages that are written into ELF dump file
Hello,
>On Mon, Jul 31, 2017 at 06:35:30AM -0700, Eric DeVolder wrote:
>> When a page is excluded by any of the existing dump levels,
>> that page may still be written to the ELF dump file, depending
>> upon the PFN_EXCLUDED mechanism.
>>
>> The PFN_EXCLUDED mechanism looks for N consecutive "not
>> dumpable" pages, and if found, the current ELF segment is
>> closed out and a new ELF segment started, at the next dumpable
>> page. Otherwise, if the PFN_EXCLUDED criteria is not meet (that
>> is, there is a mix of dumpable and not dumpable pages, but not
>> N consecutive not dumpable pages) all pages are written to the
>> dump file.
>>
>> This patch implements a mechanism for those "not dumpable" pages
>> that are written to the ELF dump file to fill those pages with
>> constant data, rather than the original data. In other words,
>> the dump file still contains the page, but its data is wiped.
>>
>> The motivation for doing this is to protect real user (customer)
>> data from "leaking" through to a dump file when that data was
>> intended to be omitted.
>>
>> Signed-off-by: Eric DeVolder <eric.devol...@oracle.com>
>> ---
>> makedumpfile.8 | 8
>> makedumpfile.c | 32 +---
>> makedumpfile.h | 2 ++
>> 3 files changed, 35 insertions(+), 7 deletions(-)
>>
>> diff --git a/makedumpfile.8 b/makedumpfile.8
>> index f94f2d7..64af0cf 100644
>> --- a/makedumpfile.8
>> +++ b/makedumpfile.8
>> @@ -621,6 +621,14 @@ order from left to right. \fIVMCORE\fRs are assembled
>> into a single
>> # makedumpfile \-x vmlinux \-\-diskset=vmcore1 \-\-diskset=vmcore2 dumpfile
>>
>> .TP
>> +\fB\-\-fill-excluded-pages FILL_VALUE\fR
>
>I am OK with --fill-excluded-pages to change default value but it is not needed
>in my opinion. However, I think that we should have --no-fill-excluded-pages
>variant. Just in case if somebody wish to disable default behavior
Umm, I can't think of any cases where a user expects "not dumpable pages" to
remain while he specifies a dump level to exclude those pages.
That's why I suggested that this feature should be default.
BTW, could you tell me the benefits of making FILL_VALUE changeable ?
>> +For the ELF dump file format, excluded pages may be written into the dump
>> +file, but the page contents are wiped. This option allows the wipe value
>> +to be changed from the default 0x5041474557495045UL "PAGEWIPE", or on
>> +32-bit systems "WIPE".
>
>0xDEAD9A6E == DEADPAGE where P == 9 (do you have better figure for P?) and G
>== 6
>
>This seems better for me because you do not need to convert it to ASCII
>to see what it is. And fills exactly 32-bits.
I'll leave it up to you :-)
Thanks,
Atsushi Kumagai
>> +
>> +
>> +.TP
>> \fB\-D\fR
>> Print debugging message.
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index f85003a..cee0ab0 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -7139,7 +7139,7 @@ out:
>>
>> int
>> write_elf_load_segment(struct cache_data *cd_page, unsigned long long paddr,
>> - off_t off_memory, long long size)
>> + off_t off_memory, long long size, struct cycle *cycle)
>> {
>> long page_size = info->page_size;
>> long long bufsz_write;
>> @@ -7163,10 +7163,23 @@ write_elf_load_segment(struct cache_data *cd_page,
>> unsigned long long paddr,
>> else
>> bufsz_write = size;
>>
>> -if (read(info->fd_memory, buf, bufsz_write) != bufsz_write) {
>> -ERRMSG("Can't read the dump memory(%s). %s\n",
>> -info->name_memory, strerror(errno));
>> -return FALSE;
>> +if (!is_dumpable(info->bitmap2, paddr_to_pfn(paddr), cycle)) {
>> +unsigned k;
>> +unsigned long *p = (unsigned long *)buf;
>> +for (k = 0; k < info->page_size; k += sizeof(unsigned
>> long)) {
>> +*p++ = info->fill_excluded_pages_value;
>> +}
>> +if (lseek(info->fd_memory, bufsz_write, SEEK_CUR) < 0) {
>> +ERRMSG("Can't seek the dump memory(%s). %s\n",
>> +info->name_memory, strerror(errno));
>> +return FALSE;
>> +}
>> +} els
RE: [PATCH] makedumpfile: x86: Take care of init_level4_pgt rename in kernel
>Hi Atsushi,
>
>Thanks for the review.
>
>On Tuesday 01 August 2017 01:11 PM, Atsushi Kumagai wrote:
>> Hello Pratyush,
>>
>> Thanks for your work.
>>
>>> On 07/30/17 at 10:11pm, Pratyush Anand wrote:
>>>> Following commit renamed init_level4_pgt to init_top_pgt in kernel.
>>>>
>>>> commit 65ade2f872b474fa8a04c2d397783350326634e6
>>>> Author: Kirill A. Shutemov <kirill.shute...@linux.intel.com>
>>>> Date: Tue Jun 6 14:31:27 2017 +0300
>>>>
>>>> x86/boot/64: Rename init_level4_pgt and early_level4_pgt
>>>>
>>>> This patch takes care of above kernel modification in makedumpfile.
>>>>
>>>> Signed-off-by: Pratyush Anand <pan...@redhat.com>
>>>> ---
>>>> makedumpfile.c | 9 -
>>>> 1 file changed, 8 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/makedumpfile.c b/makedumpfile.c
>>>> index f85003a33551..e875f4bb539e 100644
>>>> --- a/makedumpfile.c
>>>> +++ b/makedumpfile.c
>>>> @@ -1486,6 +1486,8 @@ get_symbol_info(void)
>>>>SYMBOL_INIT(_stext, "_stext");
>>>>SYMBOL_INIT(swapper_pg_dir, "swapper_pg_dir");
>>>>SYMBOL_INIT(init_level4_pgt, "init_level4_pgt");
>>>> + if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL)
>>>> + SYMBOL_INIT(init_level4_pgt, "init_top_pgt");
>>>>SYMBOL_INIT(vmlist, "vmlist");
>>>>SYMBOL_INIT(vmap_area_list, "vmap_area_list");
>>>>SYMBOL_INIT(node_online_map, "node_online_map");
>>>> @@ -2104,7 +2106,10 @@ write_vmcoreinfo_data(void)
>>>>WRITE_SYMBOL("init_uts_ns", init_uts_ns);
>>>>WRITE_SYMBOL("_stext", _stext);
>>>>WRITE_SYMBOL("swapper_pg_dir", swapper_pg_dir);
>>>> - WRITE_SYMBOL("init_level4_pgt", init_level4_pgt);
>>>> + if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL)
>>>> + WRITE_SYMBOL("init_top_pgt", init_level4_pgt);
>>>> + else
>>>> + WRITE_SYMBOL("init_level4_pgt", init_level4_pgt);
>>
>> Whether the source is init_level4_pgt or init_top_pgt, the value will
>> be kept as SYMBOL(init_level4_pgt), so this conditional judgment is
>> meaningless.
>
>IIUC,we will have above if condition true for v4.13 kernel and false for older
>kernel, right? Now, if keep the name "init_level4_pgt" in a vmcore generated
>for v4.13, would that be good, even though that would work conceptually?
SYMBOL(init_level4_pgt) should always have a valid address since it will be
initialized by the code you write as below:
SYMBOL_INIT(init_level4_pgt, "init_level4_pgt");
if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL)
SYMBOL_INIT(init_level4_pgt, "init_top_pgt");
SYMBOL(init_level4_pgt) will have the address of init_top_pgt in v4.13 and
it will have the address of init_level4_pgt in older kernel.
>I thought that generated vmcoreinfo from vmlinux should look like as being
>encoded into kernel, so same name of symbol as in kernel.
Yes, it's polite if the vmcoreinfo file has the original symbol name,
but it's not necessary since the file is only for makedumpfile itself.
In v4.13, the address of init_top_pgt will be saved into the vmcoreinfo file
as SYMBOL(init_level4_pg), but it shouldn't be a problem.
>>>> I think what you want to do is like commit: ed46b6ad9, but I don't want to
>>>> introduce a new flag every time a symbol name is changed. So how about
>>>> unification into "init_level4_pgt" for backward compatibility ?
>>>> The old name can be read in all versions of makedumpfile.
>>>
>>
>> Still not sure, it it will be a good idea to have "init_level4_pgt" in newer
>> generated vmcoreinfo ...
>
>So, what is the suggestion?
>Introduce a new variable like commit: ed46b6ad9 or keep symbol of
>"init_level4_pgt" in generated vmcoreinfo for all kernels?
My opinion is the latter. I can't find the reason to keep the original
symbol name in the vmcoreinfo file even at the cost of adding extra code.
Thanks,
Atsushi Kumagai
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH] makedumpfile: x86: Take care of init_level4_pgt rename in kernel
Hello Pratyush,
Thanks for your work.
>On 07/30/17 at 10:11pm, Pratyush Anand wrote:
>> Following commit renamed init_level4_pgt to init_top_pgt in kernel.
>>
>> commit 65ade2f872b474fa8a04c2d397783350326634e6
>> Author: Kirill A. Shutemov <kirill.shute...@linux.intel.com>
>> Date: Tue Jun 6 14:31:27 2017 +0300
>>
>> x86/boot/64: Rename init_level4_pgt and early_level4_pgt
>>
>> This patch takes care of above kernel modification in makedumpfile.
>>
>> Signed-off-by: Pratyush Anand <pan...@redhat.com>
>> ---
>> makedumpfile.c | 9 -
>> 1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index f85003a33551..e875f4bb539e 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -1486,6 +1486,8 @@ get_symbol_info(void)
>> SYMBOL_INIT(_stext, "_stext");
>> SYMBOL_INIT(swapper_pg_dir, "swapper_pg_dir");
>> SYMBOL_INIT(init_level4_pgt, "init_level4_pgt");
>> +if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL)
>> +SYMBOL_INIT(init_level4_pgt, "init_top_pgt");
>> SYMBOL_INIT(vmlist, "vmlist");
>> SYMBOL_INIT(vmap_area_list, "vmap_area_list");
>> SYMBOL_INIT(node_online_map, "node_online_map");
>> @@ -2104,7 +2106,10 @@ write_vmcoreinfo_data(void)
>> WRITE_SYMBOL("init_uts_ns", init_uts_ns);
>> WRITE_SYMBOL("_stext", _stext);
>> WRITE_SYMBOL("swapper_pg_dir", swapper_pg_dir);
>> -WRITE_SYMBOL("init_level4_pgt", init_level4_pgt);
>> +if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL)
>> +WRITE_SYMBOL("init_top_pgt", init_level4_pgt);
>> +else
>> +WRITE_SYMBOL("init_level4_pgt", init_level4_pgt);
Whether the source is init_level4_pgt or init_top_pgt, the value will
be kept as SYMBOL(init_level4_pgt), so this conditional judgment is meaningless.
I think what you want to do is like commit: ed46b6ad9, but I don't want to
introduce a new flag every time a symbol name is changed. So how about
unification into "init_level4_pgt" for backward compatibility ?
The old name can be read in all versions of makedumpfile.
Thanks,
Atsushi Kumagai
>> WRITE_SYMBOL("vmlist", vmlist);
>> WRITE_SYMBOL("vmap_area_list", vmap_area_list);
>> WRITE_SYMBOL("node_online_map", node_online_map);
>> @@ -2500,6 +2505,8 @@ read_vmcoreinfo(void)
>> READ_SYMBOL("_stext", _stext);
>> READ_SYMBOL("swapper_pg_dir", swapper_pg_dir);
>> READ_SYMBOL("init_level4_pgt", init_level4_pgt);
>> +if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL)
>> +READ_SYMBOL("init_top_pgt", init_level4_pgt);
>> READ_SYMBOL("vmlist", vmlist);
>> READ_SYMBOL("vmap_area_list", vmap_area_list);
>> READ_SYMBOL("node_online_map", node_online_map);
>> --
>> 2.9.4
>>
>>
>> ___
>> kexec mailing list
>> kexec@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec
>
>Tested-by: Dave Young <dyo...@redhat.com>
>
>Thanks
>Dave
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
makedumpfile-1.6.2: Add support for sparc64
Hello, makedumpfile version 1.6.2 is released. Your comments/patches are welcome. Main new feature: o Support for sparc64 - sparc64 is supported from this version, it's tested on kernel-4.1.12. o Add new option for --dump-dmesg - Introduce the new option "--partial-dmesg", it can exclude the buffers cleared by "dmesg --clear" while the current --dump-dmesg always extracts the whole ring buffer. o Support new kernels - The supported kernel is updated to 4.11 in this version. Changelog: o New feature - [PATCH] Add initial sparc64 support (by Tom Hromatka) 001d9b7 - [PATCH v5 1/2] Add runtime kaslr offset if it exists (by Pratyush Anand) 2bfb7f1 - [PATCH v5 2/2] x86_64: Calculate page_offset in case of re-filtering/sadump/virsh dump (by Pratyush Anand) 4944f93 - [PATCH v2] Add the --partial-dmesg option to dump only non-cleared dmesg (by Ivan Delalande) ec5da81 - [PATCH v3 1/2] Fold the calc of time delta into a func (by Pingfan Liu) 1192446 - [PATCH v3 2/2] print_info: Show the remaining time of dump progress (by Pingfan Liu) 3438e8d o Bugfix - [PATCH] elf_info.c: fix memory leak in get_kcore_dump_loads() (by Pingfan Liu) f10d1e2 - [PATCH] Error on re-filtering the dump file with no free pages (by Mikhail Zaslonko) deb8e1f - [PATCH] Fix get_kcore_dump_loads() error case (by Pratyush Anand) a569d63 - [PATCH] sadump: set info->page_size before cache_init() (by Hatayama, Daisuke) b98e6e0 - [PATCH] Fix the use of Xen physical and machine addresses (by Petr Tesarik) e765785 - [PATCH V3] elf_info: Fix file_size if segment is excluded (by Pratyush Anand) 88731c7 - [PATCH v3] Prevent data loss in last page of ELF core dumpfile (by Eric DeVolder) 622f0d7 - [PATCH] Consider not page-size aligned phys_end for paddr_to_pfn() (by Atsushi Kumagai) 25259e6 - [PATCH] Fix the regression caused by moving cache_init() earlier (by Atsushi Kumagai) be0461f o Cleanup - [PATCH 1/4] eppic: Rename scripts to reflect validity of kernel version (by Kamalesh Babulal) b55b29f - [PATCH 2/4] eppic/vhost_net_buffers: Introduce changes for kernel 3.19 (by Kamalesh Babulal) 0a1d8ee - [PATCH 3/4] eppic/dir_names: Introduce changes for kernel 3.14 (by Kamalesh Babulal) 5c822a7 - [PATCH 4/4] eppic/keyring: Introduce changes for kernel 4.4 (by Kamalesh Babulal) 19fe701 - [PATCH v3 1/7] show_mem_usage(): calculate page offset after elf load (by Pratyush Anand) 4b0bed3 - [PATCH v3 2/7] initial(): call cache_init() a bit early (by Pratyush Anand) 8e2834b - [PATCH v3 3/7] x86_64: check physical address in PT_LOAD for none direct mapped regions (by Pratyush Anand) f136302 - [PATCH v3 4/7] elf_info: kcore: check for invalid physical address (by Pratyush Anand) f4ab689 - [PATCH v3 5/7] makedumpfile: Correct the calculation of kvaddr in set_kcore_vmcoreinfo (by Baoquan He) 4c53423 - [PATCH v3 6/7] makedumpfile: Discard process_dump_load (by Baoquan He) f3ff8c6 - [PATCH v3 7/7] mem-usage: allow to work only with -f for kernel version < 4.11 (by Pratyush Anand) 78cbb40 - [PATCH] Fix typo in the ERRMSG of function show_mem_usage (by Eric Desrochers) ade8512 - [PATCH] Fix other typos in messages (by Atsushi Kumagai) b171374 - [PATCH] Fix compiler warnings (by Eric DeVolder) 04dab93 Explanation of makedumpfile: To shorten the size of the dumpfile and the time of creating the dumpfile, makedumpfile copies only the necessary pages for analysis to the dumpfile from /proc/vmcore. You can specify the kind of unnecessary pages with dump_level. If you want to shorten the size further, enable the compression of the page data. Download: You can download the latest makedumpfile from the following URL. Details of the change are written on the git page of the following site. https://sourceforge.net/projects/makedumpfile/ Method of installation: You can compile the makedumpfile command as follows; 1. "tar -zxvf makedumpfile-x.y.z.tar.gz" 2. "cd makedumpfile-x.y.z" 3. "make; make install" Usage: makedumpfile [-c|-l|-p] [-E] [-d dump_level] [-x vmlinux] dump_mem dump_file Example: If you want to exclude pages filled by zero, cache pages, user pages and free pages and to enable compression, please execute the following command. # makedumpfile -c -d 31 /proc/vmcore dumpfile Thanks Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch v3 2/7] initial(): call cach_init() a bit early
>Hi Atsushi,
>
>On Wednesday 19 July 2017 01:16 PM, Atsushi Kumagai wrote:
>> Hello Pratyush,
>>
>>> Call cach_init() before get_kcore_dump_loads(), because latter uses
>>> cache_search().
>>>
>>> Call path is like this :
>>> get_kcore_dump_loads() -> process_dump_load() -> vaddr_to_paddr() ->
>>> vtop4_x86_64() -> readmem() -> cache_search()
>>
>> I found that the current cach_init() is too early, cache_init() should be
>> called after fallback_to_current_page_size() for older kernel
>> which doesn't have VMCOREINFO.
>> Could you confirm that the patch below is fine with you ?
>
>Its fine and works.
Thanks, I'll merge this fix into v1.6.2.
Regards,
Atsushi Kumagai
>>
>> Thanks,
>> Atsushi Kumagai
>>
>> From: Atsushi Kumagai <ats-kuma...@wm.jp.nec.com>
>> Date: Wed, 19 Jul 2017 15:05:58 +0900
>> Subject: [PATCH] Fix the regression caused by moving cache_init() earlier
>>
>> info->page_size should be set before calling cache_init() since it
>> allocates some buffers based on info->page_size. However, cache_init()
>> was moved before fallback_to_current_page_size() while it set
>> info->page_size for older kernel which doesn't have VMCOREINFO.
>>
>> This patch moves cache_init() after fallback_to_current_page_size()
>> and reverts the commit: b98e6e0d2a(sadump: set info->page_size before
>> cache_init())
>>
>> Signed-off-by: Atsushi Kumagai <ats-kuma...@wm.jp.nec.com>
>> ---
>> makedumpfile.c | 28
>> 1 file changed, 16 insertions(+), 12 deletions(-)
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index a3f0697..f85003a 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -3905,15 +3905,6 @@ initial(void)
>> if (!get_value_for_old_linux())
>> return FALSE;
>>
>> -if (info->flag_sadump && !set_page_size(sadump_page_size()))
>> -return FALSE;
>> -
>> -if (!is_xen_memory() && !cache_init())
>> -return FALSE;
>> -
>> -if (info->flag_mem_usage && !get_kcore_dump_loads())
>> -return FALSE;
>> -
>> if (info->flag_refiltering) {
>> if (info->flag_elf_dumpfile) {
>> MSG("'-E' option is disable, ");
>> @@ -3936,6 +3927,9 @@ initial(void)
>> return FALSE;
>> }
>>
>> +if (!set_page_size(sadump_page_size()))
>> +return FALSE;
>> +
>> if (!sadump_initialize_bitmap_memory())
>> return FALSE;
>>
>> @@ -3949,9 +3943,7 @@ initial(void)
>> * postponed until debug information becomes
>> * available.
>> */
>> -
>> -} else if (!get_phys_base())
>> -return FALSE;
>> +}
>>
>> out:
>> if (!info->page_size) {
>> @@ -3962,6 +3954,18 @@ out:
>> if (!fallback_to_current_page_size())
>> return FALSE;
>> }
>> +
>> +if (!is_xen_memory() && !cache_init())
>> +return FALSE;
>> +
>> +if (info->flag_mem_usage && !get_kcore_dump_loads())
>> +return FALSE;
>> +
>> +if (!info->flag_refiltering && !info->flag_sadump) {
>> +if (!get_phys_base())
>> +return FALSE;
>> +}
>> +
>> if (!get_max_mapnr())
>> return FALSE;
>>
>>
>
>--
>Regards
>Pratyush
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch v3 2/7] initial(): call cach_init() a bit early
Hello Pratyush,
>Call cach_init() before get_kcore_dump_loads(), because latter uses
>cache_search().
>
>Call path is like this :
>get_kcore_dump_loads() -> process_dump_load() -> vaddr_to_paddr() ->
>vtop4_x86_64() -> readmem() -> cache_search()
I found that the current cach_init() is too early, cache_init() should be
called after fallback_to_current_page_size() for older kernel
which doesn't have VMCOREINFO.
Could you confirm that the patch below is fine with you ?
Thanks,
Atsushi Kumagai
From: Atsushi Kumagai <ats-kuma...@wm.jp.nec.com>
Date: Wed, 19 Jul 2017 15:05:58 +0900
Subject: [PATCH] Fix the regression caused by moving cache_init() earlier
info->page_size should be set before calling cache_init() since it
allocates some buffers based on info->page_size. However, cache_init()
was moved before fallback_to_current_page_size() while it set
info->page_size for older kernel which doesn't have VMCOREINFO.
This patch moves cache_init() after fallback_to_current_page_size()
and reverts the commit: b98e6e0d2a(sadump: set info->page_size before
cache_init())
Signed-off-by: Atsushi Kumagai <ats-kuma...@wm.jp.nec.com>
---
makedumpfile.c | 28
1 file changed, 16 insertions(+), 12 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index a3f0697..f85003a 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -3905,15 +3905,6 @@ initial(void)
if (!get_value_for_old_linux())
return FALSE;
- if (info->flag_sadump && !set_page_size(sadump_page_size()))
- return FALSE;
-
- if (!is_xen_memory() && !cache_init())
- return FALSE;
-
- if (info->flag_mem_usage && !get_kcore_dump_loads())
- return FALSE;
-
if (info->flag_refiltering) {
if (info->flag_elf_dumpfile) {
MSG("'-E' option is disable, ");
@@ -3936,6 +3927,9 @@ initial(void)
return FALSE;
}
+ if (!set_page_size(sadump_page_size()))
+ return FALSE;
+
if (!sadump_initialize_bitmap_memory())
return FALSE;
@@ -3949,9 +3943,7 @@ initial(void)
* postponed until debug information becomes
* available.
*/
-
- } else if (!get_phys_base())
- return FALSE;
+ }
out:
if (!info->page_size) {
@@ -3962,6 +3954,18 @@ out:
if (!fallback_to_current_page_size())
return FALSE;
}
+
+ if (!is_xen_memory() && !cache_init())
+ return FALSE;
+
+ if (info->flag_mem_usage && !get_kcore_dump_loads())
+ return FALSE;
+
+ if (!info->flag_refiltering && !info->flag_sadump) {
+ if (!get_phys_base())
+ return FALSE;
+ }
+
if (!get_max_mapnr())
return FALSE;
--
2.7.2
>Signed-off-by: Pratyush Anand <pan...@redhat.com>
>---
> makedumpfile.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 6942047199de..3b8e9810468d 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -3878,6 +3878,9 @@ initial(void)
> if (!get_value_for_old_linux())
> return FALSE;
>
>+ if (!is_xen_memory() && !cache_init())
>+ return FALSE;
>+
> if (info->flag_mem_usage && !get_kcore_dump_loads())
> return FALSE;
>
>@@ -4000,9 +4003,6 @@ out:
> }
> }
>
>- if (!is_xen_memory() && !cache_init())
>- return FALSE;
>-
> if (debug_info && !get_machdep_info())
> return FALSE;
>
>--
>2.9.3
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH] fix compiler warnings
Hello Eric,
Thanks, I'll merge this into v1.6.2.
Regards,
Atsushi Kumagai
>When building makedumpfile with GCC 5.4, I observe a few warnings:
>
>% make LINKTYPE=dynamic
>
>sadump_info.c: In function ‘sadump_is_dumpable’:
>sadump_info.c:145:3: warning: ignoring return value of ‘read’, declared with
>attribute warn_unused_result
>[-Wunused-result]
> read(bitmap->fd, bitmap->buf, BUFSIZE_BITMAP);
> ^
>In file included from sadump_info.c:21:0:
>makedumpfile.h: In function ‘is_dumpable_file’:
>makedumpfile.h:1992:3: warning: ignoring return value of ‘read’, declared with
>attribute warn_unused_result
>[-Wunused-result]
> read(bitmap->fd, bitmap->buf, BUFSIZE_BITMAP);
> ^
>makedumpfile.c: In function ‘write_eraseinfo’:
>makedumpfile.c:8273:4: warning: format not a string literal and no format
>arguments [-Wformat-security]
>DEBUG_MSG(obuf);
>^
>makedumpfile.c:8273:4: warning: format not a string literal and no format
>arguments [-Wformat-security]
>In file included from makedumpfile.c:16:0:
>makedumpfile.h: In function ‘is_dumpable_file’:
>makedumpfile.h:1992:3: warning: ignoring return value of ‘read’, declared with
>attribute warn_unused_result
>[-Wunused-result]
> read(bitmap->fd, bitmap->buf, BUFSIZE_BITMAP);
> ^
>
>This patch eliminates the warnings.
>
>Signed-off-by: Eric DeVolder <eric.devol...@oracle.com>
>---
>v1: 12jul2017 posted to kexec-tools mailing list
>---
> makedumpfile.c | 2 +-
> makedumpfile.h | 6 +-
> sadump_info.c | 6 +-
> 3 files changed, 11 insertions(+), 3 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index e69b6df..8b8a6b0 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -8270,7 +8270,7 @@ write_eraseinfo(struct cache_data *cd_page, unsigned
>long *size_out)
> }
> sprintf(obuf, "erase %s %s", erase_info[i].symbol_expr,
> size_str);
>- DEBUG_MSG(obuf);
>+ DEBUG_MSG("%s", obuf);
> if (!write_cache(cd_page, obuf, strlen(obuf)))
> goto out;
> size_eraseinfo += strlen(obuf);
>diff --git a/makedumpfile.h b/makedumpfile.h
>index e32e567..38ea673 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -1981,6 +1981,7 @@ static inline int
> is_dumpable_file(struct dump_bitmap *bitmap, mdf_pfn_t pfn)
> {
> off_t offset;
>+ ssize_t rcode;
> if (pfn == 0 || bitmap->no_block != pfn/PFN_BUFBITMAP) {
> offset = bitmap->offset + BUFSIZE_BITMAP*(pfn/PFN_BUFBITMAP);
> if (lseek(bitmap->fd, offset, SEEK_SET) < 0 ) {
>@@ -1989,7 +1990,10 @@ is_dumpable_file(struct dump_bitmap *bitmap, mdf_pfn_t
>pfn)
> return FALSE;
> }
>
>- read(bitmap->fd, bitmap->buf, BUFSIZE_BITMAP);
>+ rcode = read(bitmap->fd, bitmap->buf, BUFSIZE_BITMAP);
>+ if (rcode != BUFSIZE_BITMAP)
>+ ERRMSG("Can't read the bitmap(%s). %s\n",
>+ bitmap->file_name, strerror(errno));
> if (pfn == 0)
> bitmap->no_block = 0;
> else
>diff --git a/sadump_info.c b/sadump_info.c
>index f77a020..9a6b977 100644
>--- a/sadump_info.c
>+++ b/sadump_info.c
>@@ -138,11 +138,15 @@ static inline int
> sadump_is_dumpable(struct dump_bitmap *bitmap, mdf_pfn_t pfn)
> {
> off_t offset;
>+ ssize_t rcode;
>
> if (pfn == 0 || bitmap->no_block != pfn/PFN_BUFBITMAP) {
> offset = bitmap->offset + BUFSIZE_BITMAP*(pfn/PFN_BUFBITMAP);
> lseek(bitmap->fd, offset, SEEK_SET);
>- read(bitmap->fd, bitmap->buf, BUFSIZE_BITMAP);
>+ rcode = read(bitmap->fd, bitmap->buf, BUFSIZE_BITMAP);
>+ if (rcode != BUFSIZE_BITMAP)
>+ ERRMSG("Can't read the bitmap(%s). %s\n",
>+ bitmap->file_name, strerror(errno));
> if (pfn == 0)
> bitmap->no_block = 0;
> else
>--
>2.7.4
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH] Allow PFN_EXCLUDED to be tunable via command line option --exclude-threshold
Hello Eric, >> On 07/07/2017 04:09 AM, Atsushi Kumagai wrote: >>>> The PFN_EXCLUDED value is used to control at which point a run of >>>> zeros in the bitmap (zeros denote excluded pages) is large enough >>>> to warrant truncating the current output segment and to create a >>>> new output segment (containing non-excluded pages), in an ELF dump. >>>> >>>> If the run is smaller than PFN_EXCLUDED, then those excluded pages >>>> are still output in the ELF dump, for the current output segment. >>>> >>>> By using smaller values of PFN_EXCLUDED, the resulting dump file >>>> size can be made smaller by actually removing more excluded pages >>>> from the resulting dump file. >>>> >>>> This patch adds the command line option --exclude-threshold= >>>> to indicate the threshold. The default is 256, the legacy value >>>> of PFN_EXCLUDED. The smallest value permitted is 1. >>>> >>>> Using an existing vmcore, this was tested by the following: >>>> >>>> % makedumpfile -E -d31 --exclude-threshold=256 -x vmlinux vmcore >>>> newvmcore256 >>>> % makedumpfile -E -d31 --exclude-threshold=4 -x vmlinux vmcore >>>> newvmcore4 >>>> >>>> I utilize -d31 in order to exclude as many page types as possible, >>>> resulting in a [significantly] smaller file sizes than the original >>>> vmcore. >>>> >>>> -rwxrwx--- 1 edevolde edevolde 4034564096 Jun 27 10:24 vmcore >>>> -rw--- 1 edevolde edevolde 119808156 Jul 6 13:01 newvmcore256 >>>> -rw--- 1 edevolde edevolde 100811276 Jul 6 13:08 newvmcore4 >>>> >>>> The use of smaller value of PFN_EXCLUDED increases the number of >>>> output segments (the 'Number of program headers' in the readelf >>>> output) in the ELF dump file. >>> >>> How will you tune the value ? I'm not sure what is the benefit of the >>> tunable PFN_EXCLUDED. If there is no regression caused by too many >>> PT_LOAD >>> entries, I think we can decide a concrete PFN_EXCLUDED. >> >> Allow me note two things prior to addressing the question. >> >> Note that the value for PFN_EXCLUDED really should be in the range: >> >> 1 <= PFN_EXCLUDED <= NUM_PAGES(largest segment) >> >> but that values larger than NUM_PAGES(largest segment) behave the same >> as NUM_PAGES(largest segment) and simply prevent makedumpfile from ever >> omitting excluded pages from the dump file. >> >> Also note that the ELF header allows for a 16-bit e_phnum value for the >> number of segments in the dump file. As it stands today, I doubt that >> anybody has come close to reaching 65535 segments, but the combination >> of larger and larger memories as well as the work we (Oracle) are doing >> to further enhance the capabilities of makedumpfile, I believe we will >> start to challenge this 65535 number. I overlooked the limitation of the number of segments, so I considered only "The first benefit" you said below. >> The ability to tune PFN_EXCLUDED allows one to minimize file size while >> still staying within ELF boundaries. >> >> There are two ways in which have PFN_EXCLUDED as a tunable parameter >> benefits the user. >> >> The first benefit is, when making PFN_EXCLUDED smaller, makedumpfile has >> more opportunities to NOT write excluded pages to the resulting dump >> file, thus obtaining a smaller overall dump file size. And since a >> PT_LOAD header is smaller than a page, this penalty (of more segments) >> will always result in a smaller file size. (In the example I cite the >> dump file was 18MB smaller with a PFN_EXCLUDED value of 4 than default >> 256, in spite of increasing the number of segments from 6 to 244). >> >> The second benefit is, when enabling PFN_EXCLUDED to become larger, it >> allows makedumpfile to continue to generate valid ELF dump files in the >> presence of larger and larger memory systems. Generally speaking, the >> goal is to minimize the size of dump files via the exclusion of >> uninteresting pages (ie zero, free, etc), especially as the size of >> memory continues to grow and grow. As the memory increases, there are >> more and more of these uninteresting pages, and more opportunities for >> makedumpfile to omit them (even at the current PFN_EXCLUDED value of >> 256). Furthermore, we are working on additional page exclusion >> strategies that will drive more and more opportunities for makedumpfile >&
RE: [makedumpfile PATCH] Allow PFN_EXCLUDED to be tunable via command line option --exclude-threshold
ader: 64 (bytes)
> Size of program headers: 56 (bytes)
> Number of program headers: 244
> ^^^
> Size of section headers: 0 (bytes)
> Number of section headers: 0
> Section header string table index: 0
>
>The newvmcore4 has an even smaller file size than newvmcore256, with
>the small price being that there are now 244 rather than 18 segments
>in the dump file.
>
>And with a larger number of segments, loading both vmcore and newvmcore4
>into 'crash' resulted in identical outputs when run with the dmesg, ps,
>files, mount, and net sub-commands.
What about the processing speed of crash, is there no slow down ?
Thanks,
Atsushi Kumagai
>Signed-off-by: Eric DeVolder <eric.devol...@oracle.com>
>---
>v1: Posted 06jul2017 to kexec-tools mailing list
> - original
>---
> makedumpfile.c | 20 +---
> makedumpfile.h | 4 +++-
> 2 files changed, 20 insertions(+), 4 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index e69b6df..940f64c 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -7236,7 +7236,7 @@ get_loads_dumpfile_cyclic(void)
>
> /*
>* If the number of the contiguous pages to be
> excluded
>- * is 256 or more, those pages are excluded
>really.
>+ * is PFN_EXCLUDED or more, those pages are
>excluded really.
>* And a new PT_LOAD segment is created.
>*/
> if (num_excluded >= PFN_EXCLUDED) {
>@@ -7352,7 +7352,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header,
>struct cache_data *cd_page)
> continue;
> /*
>* If the number of the contiguous
> pages to be excluded
>- * is 255 or less, those pages are not
>excluded.
>+ * is less than PFN_EXCLUDED, those
>pages are not excluded.
>*/
> } else if (num_excluded < PFN_EXCLUDED) {
> if ((pfn == pfn_end - 1) && frac_tail) {
>@@ -7370,7 +7370,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header,
>struct cache_data *cd_page)
>
> /*
>* If the number of the contiguous pages to be
> excluded
>- * is 256 or more, those pages are excluded
>really.
>+ * is PFN_EXCLUDED or more, those pages are
>excluded really.
>* And a new PT_LOAD segment is created.
>*/
> load.p_memsz = memsz;
>@@ -11007,6 +11007,7 @@ static struct option longopts[] = {
> {"splitblock-size", required_argument, NULL, OPT_SPLITBLOCK_SIZE},
> {"work-dir", required_argument, NULL, OPT_WORKING_DIR},
> {"num-threads", required_argument, NULL, OPT_NUM_THREADS},
>+ {"exclude-threshold", required_argument, NULL,
>OPT_PFN_EXCLUDE_THRESHOLD},
> {0, 0, 0, 0}
> };
>
>@@ -11044,6 +11045,14 @@ main(int argc, char *argv[])
>*/
> info->flag_usemmap = MMAP_TRY;
>
>+ /*
>+ * A run of zeros in the bitmap (excluded pages) of less than
>+ * pfn_excluded_threshold in length will still be dumped. Runs greater
>+ * than or equal to pfn_excluded_threshold will result in the creation
>+ * of a new output segment, for ELF dumps.
>+ */
>+ info->pfn_exclude_threshold = 256;
>+
> info->block_order = DEFAULT_ORDER;
> message_level = DEFAULT_MSG_LEVEL;
> while ((opt = getopt_long(argc, argv, "b:cDd:eEFfg:hi:lpRvXx:",
> longopts,
>@@ -11163,6 +11172,11 @@ main(int argc, char *argv[])
> case OPT_NUM_THREADS:
> info->num_threads = MAX(atoi(optarg), 0);
> break;
>+ case OPT_PFN_EXCLUDE_THRESHOLD:
>+ info->pfn_exclude_threshold = strtoul(optarg, NULL, 0);
>+ if (0 == info->pfn_exclude_threshold)
>+ info->pfn_exclude_threshold = 1;
>+ break;
> case '?':
> MSG("Commandline parameter is invalid.\n");
> MSG("Try `makedumpfile --help' for more
> information.\n");
&
RE: makedumpfile: support for newer kernels [v4.9 onwards]
Hello Abhishek, Thanks for your investigation. >I was finally able to root cause the issue that I was facing in the >kdump compressed output >generated by makedumpfile. > >While debugging "write_kdump_pages_and_bitmap_cyclic"; I came to know that >somehow "info->max_mapnr" was modified/ reduced. > >Seeing further, I could see that the "get_mem_map" function at the end >[if (!is_xen_memory())] >is attempting to change "info->max_mapnr" value based on "mem=" option. >-- in particular commit ebe2fa3a5c1e2bfac3778c0fbeca4287a934cc58. >And in my kernel command line, "mem=2G" was present, as I intended to >take dump of 2G RAM, >resulting into reduced "info->max_mapnr". Basically, this adjusting info->max_mapnr must be harmless since it just intends to ignore unused memory regions. Did you find out why adjusting info->max_mapnr causes the problem ? >So, I commented out this portion and confirmed that "info->max_mapnr" >remains same >throughout. And I was able to get kdump compressed vmcore process-able >by crash utility. This sounds like there are some valid data beyond the adjusted info->max_mapnr, is that true ? Thanks, Atsushi Kumagai >Just to summarize: >Following was the problem statement: crash was giving "seek error" >while makedumpfile generated kdump-compressed vmcore was getting analyzed. >Details: >> The output generated by makedumpfile gets processed by crash utility >> if I use following command: >> makedumpfile -E /proc/vmcore vmcore >> >> But, the output generated by following command (which I used while >> reporting the issue) gives the "seek error" while being processed by >> crash. >> makedumpfile /proc/vmcore vmcore >> >> (As a result?), output from following command gives "seek error" as >> well while being processed by crash: >> makedumpfile -c /proc/vmcore vmcore > >Following is my setup details: >>>> Kernel config: >>>> ARM64_VA_BITS_48 [=y] >>>> ARM64_PAGE_SHIFT [=12] >>>> RANDOMIZE_BASE [=n] >>>> ARM64_4K_PAGES [=y] > >> kernel: 4.12- rc5 >> yocto based rootfs: Pyro[2.3] which has kexec 2.0.14 and makedumpfile 1.6.1. >> crash: 7.1.9++ [Head: 4d517ad28acd845fe6e91360e645cf0446a4757b] > > >Thanks, >Abhishek ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH v3] Prevent data loss in last page of ELF core dumpfile
Hello Eric, Thanks! I'll merge this version into v1.6.2. Regards, Atsushi Kumagai >When generating an ELF core dump file, if a segment size >is not an exact multiple of PAGE_SIZE, then the corresponding >generated segment is erroneously truncated to a PAGE_SIZE multiple. >Thus a small loss of data up to PAGE_SIZE-1 bytes can occur. > >The problem root is in the creation of the first bitmap, which >is the list of pages to dump as calculated from the vmcore >segments' information. (A second bitmap is created which is >a copy of the first bitmap with those bits corresponding to the >exclude/filter pages zero'd, and is the actual list of dumpable >pages). > >During creation of the first bitmap, each segment is processed >to determine the first and last page frame numbers corresponding >to the segment. The page dump loops are generally written as: > > for (pfn = pfn_start; pfn < pfn_end; ++pfn) > >meaning that the pfn_end needs to be one beyond the actual last >page frame number. > >The last page frame number is calculated via the paddr_to_pfn() >macro on the segment end physical address of p_addr + p_memsz. >The paddr_to_pfn() macro essentially performs a right shift of >the address to extract the pfn. Since p_memsz is typically a >multiple of PAGE_SIZE, the computed pfn_end is one beyond the >actual. For example, a segment which describes the first page of >memory would be p_paddr 0 + p_memsz 0x1000 = 0x1000, and when >right shifted yields pfn_end of 1, matching the loop semantics >above and resulting in one iteration of the loop. > >However, when the end physical address is not a multiple of >PAGE_SIZE, the paddr_to_pfn() macro truncates the address and >the need for one additional page for the remaining data is >unaccounted. For example, a segment which describes the 4097 >bytes (PAGE_SIZE + 1), results in p_addr 0 + p_memsz 0x1001 = >0x1001, and when right shifted yields pfn_end of 1. An additional >page is needed to account for the additional data, so pfn_end >needs to be 2 in this case. > >This patch detects this condition and accounts for the additional >needed page. > >This problem was observed by the test case described below. > >I have an existing ELF vmcore dumpfile and run it through >makedumpfile again, as such: > >% makedumpfile -E -x vmlinux vmcore newvmcore >% readelf -a vmcore > vmcore.txt >% readelf -a newvmcore > newvmcore.txt > >From crash, here is a description of the original vmcore: > > KERNEL: vmlinux >DUMPFILE: vmcore >CPUS: 4 >DATE: Thu Jan 7 07:49:10 2016 > UPTIME: 00:00:22 >LOAD AVERAGE: 0.00, 0.00, 0.00 > TASKS: 77 >NODENAME: mini-amd64 > RELEASE: 4.2.0-ns.gen.amd64.1 > VERSION: #1 SMP Wed Oct 28 16:32:12 CET 2015 > MACHINE: x86_64 (2194 Mhz) > MEMORY: 4 GB > PANIC: "sysrq: SysRq : Trigger a crash" > PID: 96 > COMMAND: "bash" >TASK: 88017a4c9e00 [THREAD_INFO: 88017a198000] > CPU: 3 > STATE: TASK_RUNNING (SYSRQ) > >In essence, no re-filtering has occured and I expect to see a very similar >ELF dump file to the original. And for the most part, the files are similar, >but I do observe some differences. > >The contents of vmcore.txt are: > >=== vmcore.txt === >ELF Header: > Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 > Class: ELF64 > Data: 2's complement, little endian > Version: 1 (current) > OS/ABI:UNIX - System V > ABI Version: 0 > Type: CORE (Core file) > Machine: Advanced Micro Devices X86-64 > Version: 0x1 > Entry point address: 0x0 > Start of program headers: 64 (bytes into file) > Start of section headers: 0 (bytes into file) > Flags: 0x0 > Size of this header: 64 (bytes) > Size of program headers: 56 (bytes) > Number of program headers: 6 > Size of section headers: 0 (bytes) > Number of section headers: 0 > Section header string table index: 0 > >There are no sections in this file. > >There are no sections to group in this file. > >Program Headers: > Type Offset VirtAddr PhysAddr > FileSizMemSiz Flags Align > NOTE 0x1000 0x 0x > 0x0c6c 0x0c6c 0 > LOAD 0x000
RE: [makedumpfile PATCH v2] Prevent data loss in last page of ELF core dumpfile
>> diff --git a/makedumpfile.c b/makedumpfile.c >> index e69b6df..26296f1 100644 >> --- a/makedumpfile.c >> +++ b/makedumpfile.c >> @@ -5410,7 +5410,8 @@ create_1st_bitmap_file(void) >> if (pfn_start > info->max_mapnr) >> continue; >> pfn_end = MIN(pfn_end, info->max_mapnr); >> - >> +/* Account for last page if it has less than page_size data in it */ >> +if (phys_end & (info->page_size-1)) ++pfn_end; > >Something is wrong with column alignment. Tabs instead of spaces? >And "++pfn_end;" should be in new line. > >Daniel I have no additional comments, thanks for your work. BTW, I found the same issues in places where paddr_to_pfn() expects that the end physical address is page-size aligned like get_max_mapnr(), I'll fix that as well. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH] Prevent data loss in last page of ELF core dumpfile
he_data *cd_page)
> for (pfn = MAX(pfn_start, cycle.start_pfn); pfn <
> cycle.end_pfn; pfn++) {
> if (!is_dumpable(info->bitmap2, pfn, )) {
> num_excluded++;
>- if ((pfn == pfn_end - 1) && frac_tail)
>+ if ((pfn == pfn_end - 1) && frac_tail) {
> memsz += frac_tail;
>- else
>+ filesz += frac_tail;
>+ } else {
> memsz += page_size;
>+ filesz += page_size;
>+}
As "!is_dumpable()" indicates, this block is for page filtering.
If a page is excluded, page_size shouldn't be added to filesz since
the page is not included in the file.
So I don't think this approach is proper, but I don't have a good
idea for now. It seems that a frac_tail page can be judged as
"not dumpable" erroneously even without filtering, I suspect it's
the root cause of your problem.
Thanks,
Atsushi Kumagai
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH V3] elf_info: fix file_size if segment is excluded
>Hi Atsushi, > >On Wednesday 31 May 2017 06:19 AM, Atsushi Kumagai wrote: >>> Hi Atsushi, >>> >>> On Wednesday 17 May 2017 01:15 PM, Atsushi Kumagai wrote: >>>> Hello Pratyush, >>>> >>>> Sorry for late reply. >>>> I have no further comment for now, but the current --mem-usage >>>> seems have some problems. I can't decide that this patch is insufficient or >>>> that's quite a different matter now, so please let me defer acknowledgment >>>> for a while. >>>> >>> >>> Any decision on this patch. >> >> The problem I met is still unresolved, but it occurs even without this patch. >> So I accept this patch for now. >> > >This patch is missing in your devel branch. > >Just a gentle reminder, so that it does not miss 1.6.2. Thank you for informing me, I pushed it. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: makedumpfile: support for newer kernels [v4.9 onwards]
Hello, I would like to inform you that I can't reply on this thread next week because of vacation, Thanks for your cooperation. BTW, I answer the first question: >Is there plan to support newer kernel? I'm going to release the next version in July, it will support kernel 4.11. Thanks, Atsushi Kumagai > >Hi Pratyush/Dave, > >On Tue, Jun 13, 2017 at 1:36 PM, Pratyush Anand <pan...@redhat.com> wrote: >> Hi Abhishek, >> >> On Thursday 08 June 2017 11:30 AM, Abhishek Shah wrote: >>> >>> Hi Pratyush, >>> >>> Kernel config: >>> ARM64_VA_BITS_48 [=y] >>> ARM64_PAGE_SHIFT [=12] >>> RANDOMIZE_BASE [=n] >>> ARM64_4K_PAGES [=y] >> >> These are the only variable configs on which arm64 makedumpfile depends. I >> tested with above configuration and things were working fine. >> >> I used 4.11 based kernel and crash-7.1.9. >> >> BTW, which version of crash utility do you use. If you are not at latest can >> you please try with latest. >> >I was using crash 7.1.9++ with head commit >5c52842a58a2602dba81de71831af98b2b53c6e0; >I have upgraded my kernel and built the crash utility again; and >following is my setup details now: > >kernel: 4.12- rc5 >yocto based rootfs: Pyro[2.3] which has kexec 2.0.14 and makedumpfile 1.6.1. >crash: 7.1.9++ [Head: 4d517ad28acd845fe6e91360e645cf0446a4757b] > >The output generated by makedumpfile gets processed by crash utility >if I use following command: >makedumpfile -E /proc/vmcore vmcore > >But, the output generated by following command (which I used while >reporting the issue) gives the "seek error" while being processed by >crash. >makedumpfile /proc/vmcore vmcore > >(As a result?), output from following command gives "seek error" as >well while being processed by crash: >makedumpfile -c /proc/vmcore vmcore > >makedumpfile Manual suggests: >"-E >Create DUMPFILE in the ELF format. >This option cannot be specified with -c option, because the ELF format >does not support compressed data." >So only elf formatted output from makedumpfile gets processed by crash. > >@Pratyush, Thanks for testing and verifying the kernel configuration >for arm64 makedumpfile. >When you say things were working fine for you, what exact >makedumpfile command did you use?? > >Regards, >Abhishek > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH v3 2/2] print_info: show the remaining time of dump progress
Hello Pingfan,
>On Wednesday 14 June 2017 03:11 PM, Pingfan Liu wrote:
>> If the dump progress is slow, it is better to show an estimate of
>> remaining time of the progress to the user. The estimator simply uses
>> the average speed to work.
>> Before this patch, the bar looks like:
>> Copying data : [ 1.8 %] /
>> After this patch:
>> Copying data : [ 1.8 %] /
>> eta: 39s
>>
>> Signed-off-by: Pingfan Liu <pi...@redhat.com>
>> ---
>> v2 -> v3: improve the commit log
>
>for the series
>
>Reviewed-by: Pratyush Anand <pan...@redhat.com>
Thanks, I'll merge v3 patches into v1.6.2.
Regards,
Atsushi Kumagai
>>
>> makedumpfile.c | 46 +++---
>> print_info.c | 41 +++--
>> print_info.h | 3 ++-
>> 3 files changed, 60 insertions(+), 30 deletions(-)
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index 301772a..35a36ce 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
>>
>> for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
>>
>> -print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
>> +print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes,
>> NULL);
>>
>> node_zones = pgdat + OFFSET(pglist_data.node_zones);
>>
>> @@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
>> for (i = 0; i < nr_zones; i++) {
>>
>> print_progress(PROGRESS_FREE_PAGES, i + nr_zones *
>> (num_nodes - 1),
>> -nr_zones * vt.numnodes);
>> +nr_zones * vt.numnodes, NULL);
>>
>> zone = node_zones + (i * SIZE(zone));
>> if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
>> @@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
>> /*
>> * print [100 %]
>> */
>> -print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
>> +print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
>> print_execution_time(PROGRESS_FREE_PAGES, _start);
>>
>> return TRUE;
>> @@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
>> for (i = 0; get_pt_load(i, _start, _end, NULL, NULL); i++) {
>>
>> if (!info->flag_mem_usage)
>> -print_progress(PROGRESS_HOLES, i, num_pt_loads);
>> +print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>>
>> pfn_start = paddr_to_pfn(phys_start);
>> pfn_end = paddr_to_pfn(phys_end);
>> @@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
>> * print 100 %
>> */
>> if (!info->flag_mem_usage) {
>> -print_progress(PROGRESS_HOLES, info->max_mapnr,
>> info->max_mapnr);
>> +print_progress(PROGRESS_HOLES, info->max_mapnr,
>> info->max_mapnr, NULL);
>> print_execution_time(PROGRESS_HOLES, _start);
>> }
>>
>> @@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct
>> dump_bitmap *bitmap, int
>> pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
>> pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
>>
>> -print_progress(PROGRESS_HOLES, i, num_pt_loads);
>> +print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>>
>> if (pfn_start >= pfn_end)
>> continue;
>> @@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct
>> dump_bitmap *bitmap, int
>> /*
>> * print 100 %
>> */
>> -print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
>> +print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>> print_execution_time(PROGRESS_HOLES, _start);
>>
>> return TRUE;
>> @@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>> for (mm = 0; mm < info->num_mem_map; mm++) {
>>
>> if (!info->flag_mem_usage)
>> -print_progress(PROGRESS_UNN_PAGES, mm,
>> info->num_mem_map);
>> +print_progress(PROGRESS_UNN_PAGES, mm,
>> info->num_mem_
RE: [makedumpfile PATCH v2] Add the --partial-dmesg option to dump only non-cleared dmesg
Hello Ivan,
>This option will make --dump-dmesg save only the part of the dmesg
>buffer since it was last cleared on the crashed kernel (with
>dmesg --clear and such) instead of the whole buffer since boot or
>wrap-around. It works on kernels with commit f468908bb55a ("printk:
>add clear_idx symbol to vmcoreinfo") merged in v4.6 and otherwise
>will default to the regular behavior.
>
>Signed-off-by: Ivan Delalande <col...@arista.com>
Thanks, I'll merge this patch into v1.6.2.
Regards,
Atsushi Kumagai
>---
> makedumpfile.8 | 8 +++-
> makedumpfile.c | 24 ++--
> makedumpfile.h | 3 +++
> 3 files changed, 32 insertions(+), 3 deletions(-)
>
>diff --git a/makedumpfile.8 b/makedumpfile.8
>index 9932364..76a8d6e 100644
>--- a/makedumpfile.8
>+++ b/makedumpfile.8
>@@ -20,7 +20,7 @@ makedumpfile \- make a small dumpfile of kdump
> .br
> \fBmakedumpfile\fR[\fIOPTION\fR] [\-\-xen-syms
> \fIXEN-SYMS\fR|\-\-xen-vmcoreinfo \fIVMCOREINFO\fR] \fIVMCORE\fR
>\fIDUMPFILE\fR
> .br
>-\fBmakedumpfile\fR \-\-dump-dmesg [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR]
>\fIVMCORE\fR \fILOGFILE\fR
>+\fBmakedumpfile\fR \-\-dump-dmesg [\-\-partial-dmesg] [\-x \fIVMLINUX\fR|\-i
>\fIVMCOREINFO\fR] \fIVMCORE\fR
>\fILOGFILE\fR
> .br
> \fBmakedumpfile\fR[\fIOPTION\fR] \-x \fIVMLINUX\fR
> \-\-diskset=\fIVMCORE1\fR \-\-diskset=\fIVMCORE2\fR
>[\-\-diskset=\fIVMCORE3\fR ..] \fIDUMPFILE\fR
> .br
>@@ -586,6 +586,12 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i
>\fIVMCOREINFO\fR].
>
>
> .TP
>+\fB\-\-partial-dmesg\fR
>+This option will make --dump-dmesg extract only dmesg logs since that buffer
>was
>+last cleared on the crashed kernel, through "dmesg --clear" for example.
>+
>+
>+.TP
> \fB\-\-mem-usage\fR
> This option is only for x86_64.
> This option is used to show the page numbers of current system in different
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 95cc5ef..6c5fa4c 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -1498,6 +1498,7 @@ get_symbol_info(void)
> SYMBOL_INIT(log_buf_len, "log_buf_len");
> SYMBOL_INIT(log_end, "log_end");
> SYMBOL_INIT(log_first_idx, "log_first_idx");
>+ SYMBOL_INIT(clear_idx, "clear_idx");
> SYMBOL_INIT(log_next_idx, "log_next_idx");
> SYMBOL_INIT(max_pfn, "max_pfn");
> SYMBOL_INIT(modules, "modules");
>@@ -2115,6 +2116,7 @@ write_vmcoreinfo_data(void)
> WRITE_SYMBOL("log_buf_len", log_buf_len);
> WRITE_SYMBOL("log_end", log_end);
> WRITE_SYMBOL("log_first_idx", log_first_idx);
>+ WRITE_SYMBOL("clear_idx", clear_idx);
> WRITE_SYMBOL("log_next_idx", log_next_idx);
> WRITE_SYMBOL("max_pfn", max_pfn);
> WRITE_SYMBOL("high_memory", high_memory);
>@@ -2509,6 +2511,7 @@ read_vmcoreinfo(void)
> READ_SYMBOL("log_buf_len", log_buf_len);
> READ_SYMBOL("log_end", log_end);
> READ_SYMBOL("log_first_idx", log_first_idx);
>+ READ_SYMBOL("clear_idx", clear_idx);
> READ_SYMBOL("log_next_idx", log_next_idx);
> READ_SYMBOL("max_pfn", max_pfn);
> READ_SYMBOL("high_memory", high_memory);
>@@ -5025,6 +5028,7 @@ dump_dmesg()
> int log_buf_len, length_log, length_oldlog, ret = FALSE;
> unsigned long index, log_buf, log_end;
> unsigned int idx, log_first_idx, log_next_idx;
>+ unsigned long long first_idx_sym;
> unsigned long log_end_2_6_24;
> unsigned log_end_2_6_25;
> char *log_buffer = NULL, *log_ptr = NULL;
>@@ -5058,7 +5062,13 @@ dump_dmesg()
> ERRMSG("Can't find variable-length record symbols");
> return FALSE;
> } else {
>- if (!readmem(VADDR, SYMBOL(log_first_idx),
>_first_idx,
>+ if (info->flag_partial_dmesg
>+ && SYMBOL(clear_idx) != NOT_FOUND_SYMBOL)
>+ first_idx_sym = SYMBOL(clear_idx);
>+ else
>+ first_idx_sym = SYMBOL(log_first_idx);
>+
>+ if (!readmem(VADDR, first_idx_sym, _first_idx,
> sizeof(log_first_idx))) {
> ERRMSG("Can't get log_first_idx.\n");
> return FALSE;
>@@ -5102,7 +5112,10 @@ dump_dmesg()
> DEBUG_MSG("log_buf : %lx\n", log_buf);
> DEBUG_MSG("log_end : %lx\n&
RE: [PATCH] Add the --partial-dmesg option to dump only non-cleared dmesg
Hello Ivan,
>This option will make --dump-dmesg save only the part of the dmesg
>buffer since it was last cleared on the crashed kernel (with
>dmesg --clear and such) instead of the whole buffer since boot or
>wrap-around. It works on kernels with commit f468908bb55a ("printk:
>add clear_idx symbol to vmcoreinfo") merged in v4.6 and otherwise
>will default to the regular behavior.
It works fine, thanks.
I think check_param_for_creating_dumpfile() should be modified
to reject a single --partial-dmesg, it should be always specified
with --dump-dmesg.
Thanks,
Atsushi Kumagai
>Signed-off-by: Ivan Delalande <col...@arista.com>
>---
> makedumpfile.8 | 8 +++-
> makedumpfile.c | 21 +++--
> makedumpfile.h | 3 +++
> 3 files changed, 29 insertions(+), 3 deletions(-)
>
>diff --git a/makedumpfile.8 b/makedumpfile.8
>index 9932364..76a8d6e 100644
>--- a/makedumpfile.8
>+++ b/makedumpfile.8
>@@ -20,7 +20,7 @@ makedumpfile \- make a small dumpfile of kdump
> .br
> \fBmakedumpfile\fR[\fIOPTION\fR] [\-\-xen-syms
> \fIXEN-SYMS\fR|\-\-xen-vmcoreinfo \fIVMCOREINFO\fR] \fIVMCORE\fR
>\fIDUMPFILE\fR
> .br
>-\fBmakedumpfile\fR \-\-dump-dmesg [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR]
>\fIVMCORE\fR \fILOGFILE\fR
>+\fBmakedumpfile\fR \-\-dump-dmesg [\-\-partial-dmesg] [\-x \fIVMLINUX\fR|\-i
>\fIVMCOREINFO\fR] \fIVMCORE\fR
>\fILOGFILE\fR
> .br
> \fBmakedumpfile\fR[\fIOPTION\fR] \-x \fIVMLINUX\fR
> \-\-diskset=\fIVMCORE1\fR \-\-diskset=\fIVMCORE2\fR
>[\-\-diskset=\fIVMCORE3\fR ..] \fIDUMPFILE\fR
> .br
>@@ -586,6 +586,12 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i
>\fIVMCOREINFO\fR].
>
>
> .TP
>+\fB\-\-partial-dmesg\fR
>+This option will make --dump-dmesg extract only dmesg logs since that buffer
>was
>+last cleared on the crashed kernel, through "dmesg --clear" for example.
>+
>+
>+.TP
> \fB\-\-mem-usage\fR
> This option is only for x86_64.
> This option is used to show the page numbers of current system in different
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 95cc5ef..7c88203 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -1498,6 +1498,7 @@ get_symbol_info(void)
> SYMBOL_INIT(log_buf_len, "log_buf_len");
> SYMBOL_INIT(log_end, "log_end");
> SYMBOL_INIT(log_first_idx, "log_first_idx");
>+ SYMBOL_INIT(clear_idx, "clear_idx");
> SYMBOL_INIT(log_next_idx, "log_next_idx");
> SYMBOL_INIT(max_pfn, "max_pfn");
> SYMBOL_INIT(modules, "modules");
>@@ -2115,6 +2116,7 @@ write_vmcoreinfo_data(void)
> WRITE_SYMBOL("log_buf_len", log_buf_len);
> WRITE_SYMBOL("log_end", log_end);
> WRITE_SYMBOL("log_first_idx", log_first_idx);
>+ WRITE_SYMBOL("clear_idx", clear_idx);
> WRITE_SYMBOL("log_next_idx", log_next_idx);
> WRITE_SYMBOL("max_pfn", max_pfn);
> WRITE_SYMBOL("high_memory", high_memory);
>@@ -2509,6 +2511,7 @@ read_vmcoreinfo(void)
> READ_SYMBOL("log_buf_len", log_buf_len);
> READ_SYMBOL("log_end", log_end);
> READ_SYMBOL("log_first_idx", log_first_idx);
>+ READ_SYMBOL("clear_idx", clear_idx);
> READ_SYMBOL("log_next_idx", log_next_idx);
> READ_SYMBOL("max_pfn", max_pfn);
> READ_SYMBOL("high_memory", high_memory);
>@@ -5025,6 +5028,7 @@ dump_dmesg()
> int log_buf_len, length_log, length_oldlog, ret = FALSE;
> unsigned long index, log_buf, log_end;
> unsigned int idx, log_first_idx, log_next_idx;
>+ unsigned long long first_idx_sym;
> unsigned long log_end_2_6_24;
> unsigned log_end_2_6_25;
> char *log_buffer = NULL, *log_ptr = NULL;
>@@ -5058,7 +5062,13 @@ dump_dmesg()
> ERRMSG("Can't find variable-length record symbols");
> return FALSE;
> } else {
>- if (!readmem(VADDR, SYMBOL(log_first_idx),
>_first_idx,
>+ if (info->flag_partial_dmesg
>+ && SYMBOL(clear_idx) != NOT_FOUND_SYMBOL)
>+ first_idx_sym = SYMBOL(clear_idx);
>+ else
>+ first_idx_sym = SYMBOL(log_first_idx);
>+
>+ if (!readmem(VADDR, first_idx_sym, _first_idx,
> sizeof(log_first_idx))) {
> ERRMSG("Can't get log_first_idx.\n");
> return FALSE;
>@@ -5102,7 +5112,10 @@ dump_dmesg()
RE: [makedumpfile PATCHv2 2/2] print_info: show the remaining time of dump progress
Hello Pingfan,
>On Wednesday 07 June 2017 01:39 PM, Pingfan Liu wrote:
>> If the dump progress is slow, it is better to show an estimate of
>> remaining time of the progress to the user. The estimator simply uses
>> the average speed to work.
>
>Nice improvement.
>
>It could have been better to write about how does log looks before and after
>this patch.
I have no additional comments on v2 patch, so I'll merge a next version
immediately if you update the description.
Thanks,
Atsushi Kumagai
>>
>> Signed-off-by: Pingfan Liu <pi...@redhat.com>
>> ---
>> v1 -> v2: fold eta string into eta_to_human_short()
>>
>>
>> makedumpfile.c | 46 +++---
>> print_info.c | 41 +++--
>> print_info.h | 3 ++-
>> 3 files changed, 60 insertions(+), 30 deletions(-)
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index 301772a..35a36ce 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
>>
>> for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
>>
>> -print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
>> +print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes,
>> NULL);
>>
>> node_zones = pgdat + OFFSET(pglist_data.node_zones);
>>
>> @@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
>> for (i = 0; i < nr_zones; i++) {
>>
>> print_progress(PROGRESS_FREE_PAGES, i + nr_zones *
>> (num_nodes - 1),
>> -nr_zones * vt.numnodes);
>> +nr_zones * vt.numnodes, NULL);
>>
>> zone = node_zones + (i * SIZE(zone));
>> if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
>> @@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
>> /*
>> * print [100 %]
>> */
>> -print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
>> +print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
>> print_execution_time(PROGRESS_FREE_PAGES, _start);
>>
>> return TRUE;
>> @@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
>> for (i = 0; get_pt_load(i, _start, _end, NULL, NULL); i++) {
>>
>> if (!info->flag_mem_usage)
>> -print_progress(PROGRESS_HOLES, i, num_pt_loads);
>> +print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>>
>> pfn_start = paddr_to_pfn(phys_start);
>> pfn_end = paddr_to_pfn(phys_end);
>> @@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
>> * print 100 %
>> */
>> if (!info->flag_mem_usage) {
>> -print_progress(PROGRESS_HOLES, info->max_mapnr,
>> info->max_mapnr);
>> +print_progress(PROGRESS_HOLES, info->max_mapnr,
>> info->max_mapnr, NULL);
>> print_execution_time(PROGRESS_HOLES, _start);
>> }
>>
>> @@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct
>> dump_bitmap *bitmap, int
>> pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
>> pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
>>
>> -print_progress(PROGRESS_HOLES, i, num_pt_loads);
>> +print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>>
>> if (pfn_start >= pfn_end)
>> continue;
>> @@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct
>> dump_bitmap *bitmap, int
>> /*
>> * print 100 %
>> */
>> -print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
>> +print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>> print_execution_time(PROGRESS_HOLES, _start);
>>
>> return TRUE;
>> @@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>> for (mm = 0; mm < info->num_mem_map; mm++) {
>>
>> if (!info->flag_mem_usage)
>> -print_progress(PROGRESS_UNN_PAGES, mm,
>> info->num_mem_map);
>> +print_progress(PROGRESS_UNN_PAGES, mm,
>> info->num_mem_map, NULL);
>>
>> mmd = >mem_map_data[mm];
>>
>> @@ -5860,7 +5860,7 @@ exclude_unnecess
RE: [Makedumpfile PATCH v4 1/2] makedumpfile: add runtime kaslr offset if it exists
>Hi Atsushi, > >On Friday 26 May 2017 08:36 AM, Pratyush Anand wrote: >> +#define get_kaslr_offset(X) FALSE >> #define get_xen_basic_info_arch(X) get_xen_basic_info_arm64(X) >> #define get_xen_info_arch(X) get_xen_info_arm64(X) >> #define is_phys_addr(X) stub_true_ul(X) >> @@ -851,6 +857,7 @@ unsigned long long vaddr_to_paddr_arm(unsigned long >> vaddr); >> #define get_phys_base() get_phys_base_arm() >> #define get_machdep_info() get_machdep_info_arm() >> #define get_versiondep_info() stub_true() >> +#define get_kaslr_offset(X) FALSE > > >It seems that using stub_false() would be better than FALSE. > >I see, its not yet in upstream/devel branch. >So, Either you can edit them before applying or may be I can send v5. > >Please let me know. I'll fix it and push it to the devel branch. Thanks for your pointing out. Regards, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [makedumpfile PATCH 2/2] print_info: show the remaining time of dump progress
Hello Pingfan,
>If the dump progress is slow, it is better to show an estimate of
>remaining time of the progress to the user. The estimator simply uses
>the average speed to work.
I like it, thanks for your work.
I have a minor comment.
[snip]
> void
>-print_progress(const char *msg, unsigned long current, unsigned long end)
>+print_progress(const char *msg, unsigned long current, unsigned long end,
>struct timeval *start)
> {
> float progress;
> time_t tm;
> static time_t last_time = 0;
> static unsigned int lapse = 0;
> static const char *spinner = "/|\\-";
>+ struct timeval delta;
>+ double eta;
>+ char eta_msg[32] = "eta: ";
I thinks this "eta: " should be set in eta_to_human_short(), because...
> if (current < end) {
> tm = time(NULL);
>@@ -368,13 +388,19 @@ print_progress(const char *msg, unsigned long current,
>unsigned long end)
> } else
> progress = 100;
>
>+ if (start != NULL) {
>+ calc_delta(start, );
>+ eta = delta.tv_sec + delta.tv_usec/1e6;
>+ eta = (100 - progress) * eta/progress;
>+ eta_to_human_short(eta, eta_msg+5);
the offset for "eta: " is here as a magic number.
It has low maintainability since the number is independent of
the actual string length.
Thanks,
Atsushi Kumagai
>+ }
> if (flag_ignore_r_char) {
>- PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
>- msg, progress, spinner[lapse % 4]);
>+ PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c %16s\n",
>+ msg, progress, spinner[lapse % 4], eta_msg);
> } else {
> PROGRESS_MSG("\r");
>- PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
>- msg, progress, spinner[lapse % 4]);
>+ PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c %16s",
>+ msg, progress, spinner[lapse % 4], eta_msg);
> }
> lapse++;
> }
>diff --git a/print_info.h b/print_info.h
>index 01e3706..1ce3593 100644
>--- a/print_info.h
>+++ b/print_info.h
>@@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
>
> void show_version(void);
> void print_usage(void);
>-void print_progress(const char *msg, unsigned long current, unsigned long
>end);
>+void print_progress(const char *msg, unsigned long current, unsigned long
>end, struct timeval *start);
>+
> void print_execution_time(char *step_name, struct timeval *tv_start);
>
> /*
>--
>2.7.4
>
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: makedumpfile PATCH] Fix the use of Xen physical and machine addresses
Thanks everyone, I'll merge this patch into v1.6.2 Regards, Atsushi Kumagai >On 06/01/2017 09:18 AM, Eric DeVolder wrote: >> >> >> On 05/31/2017 04:07 AM, Daniel Kiper wrote: >>> On Tue, May 30, 2017 at 11:28:15PM +0200, Petr Tesarik wrote: >>>> On Tue, 30 May 2017 14:30:43 -0500 >>>> Eric DeVolder <eric.devol...@oracle.com> wrote: >>>> >>>>> Hi, >>>>> Testing is underway. Generally working but I do have a couple of fails >>>>> I'm looking into: >>>>> >>>>> > FAIL xen-4.1.2-rc3-pre_domU-pv_xl_linux-2.6.39-x86_64.tar.bz2 >>>> ^^^ >>>> >>>> Is this a PV DomU xc_core style dump? If so, then it has never been >>>> supported by makedumpfile, because the file format is very different, >>>> with no program headers, but a special section called ".xen_pages". In >>>> fact, the file format itself does not cover this type of dump. >>> >>> Ahhh... IIRC, you are right. Thanks for pointing this out. Eric, please >>> double check it and if PV __GUESTS__ are not supported by makedumpfile >>> skip tests for them. Just test dom0 and Xen hypervisor stuff. >> >> I concur, this dump format is not supported by makedumpfile. >> >> I'm still working on validating 32-bit dump files. > >I have validated the 32-bit dump files with this patch. All is good. >eric > > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH V3] elf_info: fix file_size if segment is excluded
>Hi Atsushi, > >On Wednesday 17 May 2017 01:15 PM, Atsushi Kumagai wrote: >> Hello Pratyush, >> >> Sorry for late reply. >> I have no further comment for now, but the current --mem-usage >> seems have some problems. I can't decide that this patch is insufficient or >> that's quite a different matter now, so please let me defer acknowledgment >> for a while. >> > >Any decision on this patch. The problem I met is still unresolved, but it occurs even without this patch. So I accept this patch for now. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 1/2] sadump: set info->page_size before cache_init()
>On Tuesday 23 May 2017 08:22 AM, Hatayama, Daisuke wrote: >> Currently, makedumpfile results in Segmentation fault on sadump dump >> files as follows: >> >> # LANG=C makedumpfile -f --message-level=31 -ld31 -x vmlinux >> ./sadump_vmcore sadump_vmcore-ld31 >> sadump: read dump device as single partition >> sadump: single partition configuration >> page_size: 4096 >> sadump: timezone information is missing >> Segmentation fault >> >> By bisect, I found that this issue is caused by the following commit >> that moves invocation of cache_init() in initial() a bit early: >> >> # git bisect bad >> 8e2834bac4f62da3894da297f083068431be6d80 is the first bad commit >> commit 8e2834bac4f62da3894da297f083068431be6d80 >> Author: Pratyush Anand <pan...@redhat.com> >> Date: Thu Mar 2 17:37:11 2017 +0900 >> >> [PATCH v3 2/7] initial(): call cache_init() a bit early >> >> Call cache_init() before get_kcore_dump_loads(), because latter uses >> cache_search(). >> >> Call path is like this : >> get_kcore_dump_loads() -> process_dump_load() -> vaddr_to_paddr() -> >> vtop4_x86_64() -> readmem() -> cache_search() >> >> Signed-off-by: Pratyush Anand <pan...@redhat.com> >> >> :100644 100644 6942047199deb09dd1fff2121e264584dbb05587 >> 3b8e9810468de26b0d8b73d456f0bd4f3d3aa2fe M >makedumpfile.c >> >> In this timing, on sadump vmcores, info->page_size has not been >> initialized yet so has 0. So, malloc() in cache_init() returns a chunk >> of 0 size. A bit later, info->page_size is initialized with 4096. >> Later processing on cache.c behaves assuming the chunk size is 8 * >> 4096. This destroys objects allocated after the chunk, resulting in >> the above Segmentation fault. >> >> To fix this issue, this commit moves setting info->page_size before >> cache_init(). >> >> Signed-off-by: HATAYAMA Daisuke <d.hatay...@jp.fujitsu.com> >> Cc: Pratyush Anand <pan...@redhat.com> > >For 1/2 >Reviewed-by: Pratyush Anand <pan...@redhat.com> Thanks for your work, I'll merge this patch into v1.6.2. Regards, Atsushi Kumagai >> --- >> makedumpfile.c | 6 +++--- >> 1 file changed, 3 insertions(+), 3 deletions(-) >> >> diff --git a/makedumpfile.c b/makedumpfile.c >> index 301772a..f300b19 100644 >> --- a/makedumpfile.c >> +++ b/makedumpfile.c >> @@ -3878,6 +3878,9 @@ initial(void) >> if (!get_value_for_old_linux()) >> return FALSE; >> >> +if (info->flag_sadump && !set_page_size(sadump_page_size())) >> +return FALSE; >> + >> if (!is_xen_memory() && !cache_init()) >> return FALSE; >> >> @@ -3906,9 +3909,6 @@ initial(void) >> return FALSE; >> } >> >> -if (!set_page_size(sadump_page_size())) >> -return FALSE; >> - >> if (!sadump_initialize_bitmap_memory()) >> return FALSE; >> >> > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH v4 0/2] Fix refiltering when kaslr enabled
Hello Pratyush, I have nothing to comment anymore, thanks for your hard work. I'll merge v4 patches into v1.6.2. Regards, Atsushi Kumagai >Hi All, > >We came across another failure in makedumpfile when kaslr is enabled. This >failure occurs when we try re-filtering. We try to erase some symbol from a >dumpfile which was copied/compressed from /proc/vmcore using makedumpfile. > >We have very limited symbol information in vmcoreinfo. So symbols to be >erased may not be available in vmcoreinfo and we look for it in vmlinux. >However, symbol address from vmlinux is a static address which differs >from run time address with KASLR_OFFSET. Therefore, reading any "virtual >address of vmlinux" from vmcore is not possible. > >These patches finds runtime KASLR offset and then calculates run time >address of symbols read from vmlinux. > >Hatayama Daisuke also found some issue [1] when he was working with a >sadump and virsh dump of a none kaslr kernel. Patch 2/2 of this series has >been improved to take care of those issues as well. > >[1]http://lists.infradead.org/pipermail/kexec/2017-May/018833.html > >Thanks > >~Pratyush > >v1->v2: > - reading KERNELOFFSET from vmcoreinfo now instead of calculating it from > _stext >v2->v3: > - Fixed initialization of info->file_vmcoreinfo > - Improved page_offset calculation logic to take care of different dump > scenarios. >v3->v4: > - Removed info->kaslr_offset write to VMCOREINFO > > > >Pratyush Anand (2): > makedumpfile: add runtime kaslr offset if it exists > x86_64: calculate page_offset in case of re-filtering/sadump/virsh >dump > > arch/x86_64.c | 72 -- > erase_info.c | 1 + > makedumpfile.c | 46 + > makedumpfile.h | 16 + > 4 files changed, 128 insertions(+), 7 deletions(-) > >-- >2.9.3 ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch] Fix get_kcore_dump_loads() error case
>commit f10d1e2e94c50 introduced another bug while fixing memory leak.
>Use the braces with if condition.
Thanks, I'll merge this into v1.6.2
Atsushi Kumagai
>Signed-off-by: Pratyush Anand <pan...@redhat.com>
>---
> elf_info.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
>diff --git a/elf_info.c b/elf_info.c
>index 601d66e3f176..69b1719b020f 100644
>--- a/elf_info.c
>+++ b/elf_info.c
>@@ -893,9 +893,10 @@ int get_kcore_dump_loads(void)
> if (p->phys_start == NOT_PADDR
> || !is_phys_addr(p->virt_start))
> continue;
>- if (j >= loads)
>+ if (j >= loads) {
> free(pls);
> return FALSE;
>+ }
>
> if (j == 0) {
> offset_pt_load_memory = p->file_offset;
>--
>2.9.3
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH v3 1/2] makedumpfile: add runtime kaslr offset if it exists
Hello Pratyush,
>If we have to erase a symbol from vmcore whose address is not present in
>vmcoreinfo, then we need to pass vmlinux as well to get the symbol
>address.
>When kaslr is enabled, virtual address of all the kernel symbols are
>randomized with an offset. vmlinux always has a static address, but all
>the arch specific calculation are based on run time kernel address. So
>we need to find a way to translate symbol address from vmlinux to kernel
>run time address.
>
>without this patch:
> # cat > scrub.conf << EOF
> [vmlinux]
> erase jiffies
> erase init_task.utime
> for tsk in init_task.tasks.next within task_struct:tasks
> erase tsk.utime
> endfor
> EOF
>
># makedumpfile --split -d 5 -x vmlinux --config scrub.conf vmcore
> dumpfile_{1,2,3}
>
>readpage_kdump_compressed: pfn(f97ea) is excluded from vmcore.
>readmem: type_addr: 1, addr:f97eaff8, size:8
>vtop4_x86_64: Can't get pml4 (page_dir:f97eaff8).
>readmem: Can't convert a virtual address(819f1284) to physical
> address.
>readmem: type_addr: 0, addr:819f1284, size:390
>check_release: Can't get the address of system_utsname.
>
>After this patch check_release() is ok, and also we are able to erase
>symbol from vmcore.
>
>Signed-off-by: Pratyush Anand <pan...@redhat.com>
>---
> arch/x86_64.c | 36
> erase_info.c | 1 +
> makedumpfile.c | 53 +
> makedumpfile.h | 16
> 4 files changed, 106 insertions(+)
>
>diff --git a/arch/x86_64.c b/arch/x86_64.c
>index e978a36f8878..fd2e8ac154d6 100644
>--- a/arch/x86_64.c
>+++ b/arch/x86_64.c
>@@ -33,6 +33,42 @@ get_xen_p2m_mfn(void)
> return NOT_FOUND_LONG_VALUE;
> }
>
>+unsigned long
>+get_kaslr_offset_x86_64(unsigned long vaddr)
>+{
>+ unsigned int i;
>+ char buf[BUFSIZE_FGETS], *endp;
>+
>+ if (!info->kaslr_offset && info->file_vmcoreinfo) {
>+ if (fseek(info->file_vmcoreinfo, 0, SEEK_SET) < 0) {
>+ ERRMSG("Can't seek the vmcoreinfo file(%s). %s\n",
>+ info->name_vmcoreinfo, strerror(errno));
>+ return FALSE;
>+ }
>+
>+ while (fgets(buf, BUFSIZE_FGETS, info->file_vmcoreinfo)) {
>+ i = strlen(buf);
>+ if (!i)
>+ break;
>+ if (buf[i - 1] == '\n')
>+ buf[i - 1] = '\0';
>+ if (strncmp(buf, STR_KERNELOFFSET,
>+ strlen(STR_KERNELOFFSET)) == 0)
>+ info->kaslr_offset =
>+
>strtoul(buf+strlen(STR_KERNELOFFSET),,16);
>+ }
>+ }
>+ if (vaddr >= __START_KERNEL_map &&
>+ vaddr < __START_KERNEL_map + info->kaslr_offset)
>+ return info->kaslr_offset;
>+ else
>+ /*
>+ * TODO: we need to check if it is vmalloc/vmmemmap/module
>+ * address, we will have different offset
>+ */
>+ return 0;
>+}
>+
> static int
> get_page_offset_x86_64(void)
> {
>diff --git a/erase_info.c b/erase_info.c
>index f2ba9149e93e..60abfa1a1adf 100644
>--- a/erase_info.c
>+++ b/erase_info.c
>@@ -1088,6 +1088,7 @@ resolve_config_entry(struct config_entry *ce, unsigned
>long long base_vaddr,
> ce->line, ce->name);
> return FALSE;
> }
>+ ce->sym_addr += get_kaslr_offset(ce->sym_addr);
> ce->type_name = get_symbol_type_name(ce->name,
> DWARF_INFO_GET_SYMBOL_TYPE,
> >size, >type_flag);
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 301772a8820c..4986d098d69a 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -2099,6 +2099,13 @@ void
> write_vmcoreinfo_data(void)
> {
> /*
>+ * write 1st kernel's KERNELOFFSET
>+ */
>+ if (info->kaslr_offset)
>+ fprintf(info->file_vmcoreinfo, "%s%lx\n", STR_KERNELOFFSET,
>+ info->kaslr_offset);
When will this data written to VMCOREINFO file be used ?
info->kaslr_offset is necessary for vmlinux but -x and -i are exclusive.
Thanks,
Atsushi Kumagai
>+ /*
>* write 1st kernel's OSRELEASE
>*/
>
RE: [Makedumpfile PATCH 0/2] Fix refiltering when kaslr enabled
>On Tuesday 23 May 2017 08:39 AM, Atsushi Kumagai wrote:
>>>>>>>> Thanks for your report, I have received this.
>>>>>>>> I'm on vacation until Mar 8, I'll review it when I return from
>>>>>>>> vacation.
>>>>>>>
>>>>>>> Any further comment on it?
>>>>>>> Otherwise, I will send a v2 after accommodating concern from Xunlei.
>>>>>>
>>>>>> Unfortunately, it doesn't seem like I can make time anymore for review
>>>>>> this week,
>>>>>> but at least this patch doesn't seem to work in my environment (linux
>>>>>> 4.8 without kaslr).
>>>>>> Do you have any ideas ?
>>>>>
>>>>> I see, why it would have caused. I have not tested this case, but I hope
>>>>> my v2
>>>>> should not have this issue.
>>>>
>>>> Umm, v2 still doesn't work in my environment...
>>>> It seems that I have to investigate this deeper.
>>>
>>> Hummm, I thought we would see file_vmcoreinfo as NULL in
>>> get_kaslr_offset_x86_64() in your case. However, it's not true.
>>>
>>> I think, it will have to be initialized with NULL in main().
>>>
>>> Can you please try following fixup on top of this series:
>>
>> I found the cause, please see below:
>>
>> initial()
>> + find_kaslr_offsets()
>> + open_vmcoreinfo()
>> + get_kaslr_offset()// set info->kaslr_offset
>> + close_vmcoreinfo()
>> gather_filter_info()
>> (snip)
>> + resolve_config_entry()
>> + get_kaslr_offset()// occur SIGSEGV since info->file_vmcoreinfo
>> is closed
>
>Since info->file_vmcoreinfo is closed,therefore it should be NULL. But
>currently, close_vmcoreinfo() does not set it to NULL. We should also set
>info->file_vmcoreinfo to NULL in close_vmcoreinfo() apart from main().
Sure, uninitialized info->file_vmcoreinfo is a general issue,
I'll fix it in another patch.
>>
>>
>> The cause code is in [PATCH v2 1/2],
>>
>> diff --git a/erase_info.c b/erase_info.c
>> index f2ba914..60abfa1 100644
>> --- a/erase_info.c
>> +++ b/erase_info.c
>> @@ -1088,6 +1088,7 @@ resolve_config_entry(struct config_entry *ce, unsigned
>> long long base_vaddr,
>> ce->line, ce->name);
>> return FALSE;
>> }
>> + ce->sym_addr += get_kaslr_offset(ce->sym_addr);
>> ce->type_name = get_symbol_type_name(ce->name,
>> DWARF_INFO_GET_SYMBOL_TYPE,
>> >size, >type_flag);
>>
>>
>> I think we should use info->kaslr_offset instead of get_kaslr_offset(),
>> how about you ?
>
>Actually, we are not sure at this point that ce->sym_addr is a kernel linear
>address. It could be a module address and that can have different
>randomization offset.
I see, I misunderstood that the randomization offset is same in any case.
>My intention is to calculate all types of kaslr offsets in
>find_kaslr_offsets() very early and then store them in different "info"
>elements. Now, whenever we call get_kaslr_offset() we would return right
>offset as per the input address.
>I have very little idea about x86. So, I have left a TODO to calculate offset
>for non-linear addresses.
>
>>
>> BTW, I'm not sure why you didn't meet this issue...
>
>Because, I tested with kaslr kernel, so when get_kaslr_offset(ce->sym_addr)
>was called, I already had a valid info->kaslr_offset.
>
>So, what about following fixup
As mentioned at the beginning, the fix below is not your fault,
so I'll merge your patches as it is into v1.6.2.
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 57235690569e..4986d098d69a 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -8685,6 +8685,7 @@ close_vmcoreinfo(void)
> if(fclose(info->file_vmcoreinfo) < 0)
> ERRMSG("Can't close the vmcoreinfo file(%s). %s\n",
> info->name_vmcoreinfo, strerror(errno));
>+ info->file_vmcoreinfo = NULL;
> }
>
> void
>@@ -11076,6 +11077,7 @@ main(int argc, char *argv[])
> strerror(errno));
> goto out;
> }
>+ info->file_vmcoreinfo = NULL;
> info->fd_vmlinux = -1;
> info->fd_xen_syms = -1;
> info->fd_memory = -1;
However, I think [PATCH 2/2] should be dropped since the alternative patch
has appeared, is it ok with you ?
https://github.com/pratyushanand/makedumpfile/commit/16655ce1f4c2da8d4066072db2a03c84bf2553fe
Thanks,
Atsushi Kumagai
>
>~Pratyush
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH 0/2] Fix refiltering when kaslr enabled
>>>>>> Thanks for your report, I have received this.
>>>>>> I'm on vacation until Mar 8, I'll review it when I return from vacation.
>>>>>
>>>>> Any further comment on it?
>>>>> Otherwise, I will send a v2 after accommodating concern from Xunlei.
>>>>
>>>> Unfortunately, it doesn't seem like I can make time anymore for review
>>>> this week,
>>>> but at least this patch doesn't seem to work in my environment (linux 4.8
>>>> without kaslr).
>>>> Do you have any ideas ?
>>>
>>> I see, why it would have caused. I have not tested this case, but I hope my
>>> v2
>>> should not have this issue.
>>
>> Umm, v2 still doesn't work in my environment...
>> It seems that I have to investigate this deeper.
>
>Hummm, I thought we would see file_vmcoreinfo as NULL in
>get_kaslr_offset_x86_64() in your case. However, it's not true.
>
>I think, it will have to be initialized with NULL in main().
>
>Can you please try following fixup on top of this series:
I found the cause, please see below:
initial()
+ find_kaslr_offsets()
+ open_vmcoreinfo()
+ get_kaslr_offset()// set info->kaslr_offset
+ close_vmcoreinfo()
gather_filter_info()
(snip)
+ resolve_config_entry()
+ get_kaslr_offset()// occur SIGSEGV since info->file_vmcoreinfo is
closed
The cause code is in [PATCH v2 1/2],
diff --git a/erase_info.c b/erase_info.c
index f2ba914..60abfa1 100644
--- a/erase_info.c
+++ b/erase_info.c
@@ -1088,6 +1088,7 @@ resolve_config_entry(struct config_entry *ce, unsigned
long long base_vaddr,
ce->line, ce->name);
return FALSE;
}
+ ce->sym_addr += get_kaslr_offset(ce->sym_addr);
ce->type_name = get_symbol_type_name(ce->name,
DWARF_INFO_GET_SYMBOL_TYPE,
>size, >type_flag);
I think we should use info->kaslr_offset instead of get_kaslr_offset(),
how about you ?
BTW, I'm not sure why you didn't meet this issue...
Thanks,
Atsushi Kumagai
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 57235690569e..0fd485ccd45d 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -11076,6 +11076,7 @@ main(int argc, char *argv[])
> strerror(errno));
> goto out;
> }
>+ info->file_vmcoreinfo = NULL;
> info->fd_vmlinux = -1;
> info->fd_xen_syms = -1;
> info->fd_memory = -1;
>
>
>Thanks for testing it thoroughly.
>
>~Pratyush
>
>>
>> $ cat scrub.conf
>> [vmlinux]
>> erase modules size 50
>> $
>>
>> (gdb) r -cd31 -x vmlinux --config scrub.conf vmcore dumpfile.cd31
>> Starting program: /work/kdump_utils/makedumpfile/makedumpfile -cd31 -x
>> vmlinux --config scrub.conf vmcore
>dumpfile.cd31
>> warning: no loadable sections found in added symbol-file system-supplied
>> DSO at 0x77ffd000
>> [Thread debugging using libthread_db enabled]
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00308366ee0d in fseek () from /lib64/libc.so.6
>> Missing separate debuginfos, use: debuginfo-install
>> bzip2-libs-1.0.5-7.el6_0.x86_64
>elfutils-libelf-0.152-1.el6.x86_64 elfutils-libs-0.152-1.el6.x86_64
>glibc-2.12-1.132.el6.x86_64
>libgcc-4.4.7-4.el6.x86_64 libstdc++-4.4.7-4.el6.x86_64
>snappy-1.1.0-1.el6.x86_64
>xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 zlib-1.2.3-29.el6.x86_64
>> (gdb) bt
>> #0 0x00308366ee0d in fseek () from /lib64/libc.so.6
>> #1 0x00420937 in get_kaslr_offset_x86_64
>> (vaddr=18446744071589596288) at arch/x86_64.c:43
>> #2 0x00414310 in resolve_config_entry (ce=0x701370,
>> base_vaddr=, base_struct_name=0x0)
>at erase_info.c:1091
>> #3 0x00415a89 in get_config_symbol_addr (filter_symbol=0x701370,
>> size_symbol=0x701430) at erase_info.c:1264
>> #4 update_filter_info (filter_symbol=0x701370, size_symbol=0x701430) at
>> erase_info.c:1579
>> #5 0x00416543 in process_config (name_config=> out>) at erase_info.c:1789
>> #6 process_config_file (name_config=) at
>> erase_info.c:1862
>> #7 0x00417c57 in gather_filter_info () at erase_info.c:2356
>> #8 0x00443e5b in create_dumpfile () at makedumpfile.c:9870
>> #9 0x004457ae in main (argc=, argv=> optimized out>) at makedumpfile.c:11349
>> (gdb)
>>
>> Thanks,
>> Atsush
RE: [PATCH] makedumpfile: Error on re-filtering the dump file with no free pages
>> Hello Zaslonko,
>>
>> > When re-filtering the dump file after the free pages have already been
>> > stripped out we get an error "Can't get necessary symbols for excluding
>> > free pages" if newly specified dump level is below 16 (preserves free
>> > pages).
>> > According to the code, the check for the new dump level is done BEFORE
>> > the new dump level is actually set (based on the dump level specified in
>> > the command and the one from the input dump file).
>> > Moving the new_dump_level calculation ahead would fix the error.
>>
>> Well, I guess the real problem is different.
>> The error you said is printed by exclude_free_page():
>>
>> if ((SYMBOL(node_data) == NOT_FOUND_SYMBOL)
>> && (SYMBOL(pgdat_list) == NOT_FOUND_SYMBOL)
>> && (SYMBOL(contig_page_data) == NOT_FOUND_SYMBOL)) {
>> ERRMSG("Can't get necessary symbols for excluding free
>> pages.\n");
>> return FALSE;
>> }
>>
>> I think the availability of these symbols are not related to free pages.
>> exclude_free_page() is called if info->page_is_buddy is null, I estimate that
>> this situation occurs only if the kernel is older(2.6.16 or before).
>>
>> However, I don't think you use such too old kernel, so I suspect that
>> setup_page_is_buddy() should be updated for newer kernel.
>
>Mikhail is on vacation now - so I try to explain:
Thanks for your explanation, I understand this issue properly.
>The test case is as follows:
>
> 1) We have a -d31 filtered dump "dump.d31"
> 2) We want to compress the dump with "makedumpfile -c dump.31
> dump31.compressed"
>
>This fails with:
>
>makedumpfile -c dump.31 dump.31.compressed
>Excluding unnecessary pages: [100.0 %]
>exclude_free_page: Can't get necessary symbols for excluding free pages.
>dump_level is changed to 31, because dump.31 was created by dump_level(31).
>makedumpfile Failed.
>
>The problem is that setup_page_is_buddy() is not called in this case because
>info->dump_level is still 0 since it was not adjusted early enough:
>
>if (info->dump_level & DL_EXCLUDE_FREE)
>setup_page_is_buddy();
>
>Because it is not set info->page_is_buddy is NULL and therefore the following
>if condition gets true:
>
>if ((info->dump_level & DL_EXCLUDE_FREE) && !info->page_is_buddy)
>if (!exclude_free_page(cycle))
>return FALSE;
>
>Since we don't have the symbols in VMCOREINFO (and IMHO don't need it?) the
>exclude_free_page() functions fails with the described error message.
It seems that it's better if I update the condition check of exclude_free_page()
for recent kernels, but those symbols are unnecessary in this case as you
thought
anyway. exclude_free_page() shouldn't be called for recent kernels, I don't
think
this is an actual problem.
>So our fix is to adjust the info->level before setup_page_is_buddy() is called.
I'm sure this fix is reasonable, I'll merge this into v1.6.2.
Thanks,
Atsushi Kumagai
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: Help regarding renaming of vmcore-incomplete to vmcore
>I can't find this file in my system? Can you help me with it? In the first place, I don't know how you got core dump and what kind of environment you tested with, so it's difficult to give pointed advice. If you explain what you did in order from first in detail, I could be of your help. Thanks, Atsushi Kumagai >On 8 May 2017 at 07:11, Atsushi Kumagai <ats-kuma...@wm.jp.nec.com> wrote: >> Hello, >> >>>Hi I am a university student, and have made some changes to >>>makedumpfile to produce a gzipped output of core dump. The issue I'm >>>facing is that the output i get from makedumpfile (after it completes >>>successfully) is vmcore-incomplete.gzip. >>> >>>But this file is not being renamed to vmcore.gzip (even thought the >>>dump the complete as I verified it by loading into CRASH utility). Can >>>someone please point out to me the changes I should make to solve this >>>bug (or can point to the code where this renaming is done). >> >> makedumpfile doesn't have such renaming mechanism, it just name the dump >> as user specify. I guess "vmcore-incomplete" you said is came from >> kexec-tools. >> For example, kexec-tools of RHEL7 generate the code below for dump-capture >> kernel: >> >>110 echo "kdump: saving vmcore" >>111 $CORE_COLLECTOR /proc/vmcore >> $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore-incomplete || return 1 >>112 mv $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore-incomplete >> $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore >>113 sync >> >> $CORE_COLLECTOR is makedumpfile, so the code generate "vmcore-incomplete" >> by makedumpfile and rename it to vmcore by mv. >> If you want "vmcore.gzip", you should just specify vmcore.gzip for >> makedumpfile. >> >> >> Thanks, >> Atsushi Kumagai >> > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH] makedumpfile: Error on re-filtering the dump file with no free pages
Hello Zaslonko,
> When re-filtering the dump file after the free pages have already been
> stripped out we get an error "Can't get necessary symbols for excluding
> free pages" if newly specified dump level is below 16 (preserves free
> pages).
> According to the code, the check for the new dump level is done BEFORE
> the new dump level is actually set (based on the dump level specified in
> the command and the one from the input dump file).
> Moving the new_dump_level calculation ahead would fix the error.
Well, I guess the real problem is different.
The error you said is printed by exclude_free_page():
if ((SYMBOL(node_data) == NOT_FOUND_SYMBOL)
&& (SYMBOL(pgdat_list) == NOT_FOUND_SYMBOL)
&& (SYMBOL(contig_page_data) == NOT_FOUND_SYMBOL)) {
ERRMSG("Can't get necessary symbols for excluding free
pages.\n");
return FALSE;
}
I think the availability of these symbols are not related to free pages.
exclude_free_page() is called if info->page_is_buddy is null, I estimate that
this situation occurs only if the kernel is older(2.6.16 or before).
However, I don't think you use such too old kernel, so I suspect that
setup_page_is_buddy() should be updated for newer kernel.
Could you tell me your kernel version and how to reproduce this issue ?
Thanks,
Atsushi Kumagai
> Signed-off-by: Mikhail Zaslonko <zaslo...@linux.vnet.ibm.com>
> ---
> makedumpfile.c | 34 ++
> 1 file changed, 22 insertions(+), 12 deletions(-)
>
> diff --git a/makedumpfile.c b/makedumpfile.c
> index e69b6df..24f99fc 100644
> --- a/makedumpfile.c
> +++ b/makedumpfile.c
> @@ -9774,10 +9774,25 @@ writeout_multiple_dumpfiles(void)
> return ret;
> }
>
> +void
> +update_dump_level(void)
> +{
> + int new_level;
> +
> + new_level = info->dump_level | info->kh_memory->dump_level;
> + if (new_level != info->dump_level) {
> + info->dump_level = new_level;
> + MSG("dump_level is changed to %d, " \
> + "because %s was created by dump_level(%d).",
> + new_level, info->name_memory,
> + info->kh_memory->dump_level);
> + }
> +}
> +
> int
> create_dumpfile(void)
> {
> - int num_retry, status, new_level;
> + int num_retry, status;
>
> if (!open_files_for_creating_dumpfile())
> return FALSE;
> @@ -9786,6 +9801,10 @@ create_dumpfile(void)
> if (!get_elf_info(info->fd_memory, info->name_memory))
> return FALSE;
> }
> +
> + if (info->flag_refiltering)
> + update_dump_level();
> +
> if (!initial())
> return FALSE;
>
> @@ -9804,17 +9823,8 @@ create_dumpfile(void)
>
> num_retry = 0;
> retry:
> - if (info->flag_refiltering) {
> - /* Change dump level */
> - new_level = info->dump_level | info->kh_memory->dump_level;
> - if (new_level != info->dump_level) {
> - info->dump_level = new_level;
> - MSG("dump_level is changed to %d, " \
> - "because %s was created by dump_level(%d).",
> - new_level, info->name_memory,
> - info->kh_memory->dump_level);
> - }
> - }
> + if (info->flag_refiltering)
> + update_dump_level();
>
> if ((info->name_filterconfig || info->name_eppic_config)
> && !gather_filter_info())
> --
> 1.8.3.1
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH V3] elf_info: fix file_size if segment is excluded
Hello Pratyush,
Sorry for late reply.
I have no further comment for now, but the current --mem-usage
seems have some problems. I can't decide that this patch is insufficient or
that's quite a different matter now, so please let me defer acknowledgment
for a while.
Thanks,
Atsushi Kumagai
>I received following on a specific x86_64 hp virtual machine while
>executing `makedumpfile --mem-usage /proc/kcore`.
>
>vtop4_x86_64: Can't get a valid pte.
>readmem: Can't convert a virtual address(88115860) to physical address.
>readmem: type_addr: 0, addr:88115860, size:128
>get_nodes_online: Can't get the node online map.
>
>With some debug print in vtop4_x86_64() I noticed that pte value is read
>as 0, while crash reads the value correctly:
>
>from makedumpfile:
>vaddr=88115860
>page_dir=59eaff8
>pml4=59ed067
>pgd_paddr=59edff0
>pgd_pte=59ee063
>pmd_paddr=59ee200
>pmd_pte=3642f063
>pte_paddr=3642f8a8
>pte=0
>
>from crash
>crash> vtop 88115860
>VIRTUAL PHYSICAL
>88115860 5b15860
>
>PML4 DIRECTORY: 87fea000
>PAGE DIRECTORY: 59ed067
> PUD: 59edff0 => 59ee063
> PMD: 59ee200 => 3642f063
> PTE: 3642f8a8 => 5b15163
> PAGE: 5b15000
>
>With some more debug prints in elf_info.c
>
>Before calling exclude_segment()
>
>LOAD (2)
> phys_start : 10
> phys_end : dfffd000
> virt_start : 8a5a4010
> virt_end : 8a5b1fffd000
> file_offset: a5a40102000
> file_size : dfefd000
>
>exclude_segment() is called for Crash Kernel whose range is
>2b00-350f.
>
>We see following after exclude_segment()
>
>LOAD (2)
> phys_start : 10
> phys_end : 2aff
> virt_start : 8a5a4010
> virt_end : 8a5a6aff
> file_offset: a5a40102000
> file_size : dfefd000
>LOAD (3)
> phys_start : 3510
> phys_end : dfffd000
> virt_start : 8a5a7510
> virt_end : 8a5b1fffd000
> file_offset: a5a75102000
> file_size : 0
>
>Since file_size is calculated wrong therefore readpage_elf() does not
>behave correctly.
>
>This patch fixes above wrong behavior.
>
>Signed-off-by: Pratyush Anand <pan...@redhat.com>
>---
>v1->v2 : subtracted (end - start) from file_size as well
>v2->v3 : subtracted (end - start) for boundary cases as well.
> elf_info.c | 4
> 1 file changed, 4 insertions(+)
>
>diff --git a/elf_info.c b/elf_info.c
>index 8e2437622141..2c58359a438f 100644
>--- a/elf_info.c
>+++ b/elf_info.c
>@@ -826,9 +826,12 @@ static int exclude_segment(struct pt_load_segment
>**pt_loads,
> temp_seg.virt_end = vend;
> temp_seg.file_offset =
> (*pt_loads)[i].file_offset
> + temp_seg.virt_start -
> (*pt_loads)[i].virt_start;
>+ temp_seg.file_size = temp_seg.phys_end
>+ - temp_seg.phys_start;
>
> (*pt_loads)[i].virt_end = kvstart - 1;
> (*pt_loads)[i].phys_end = start - 1;
>+ (*pt_loads)[i].file_size -= temp_seg.file_size;
>
> tidx = i+1;
> } else if (kvstart != vstart) {
>@@ -838,6 +841,7 @@ static int exclude_segment(struct pt_load_segment
>**pt_loads,
> (*pt_loads)[i].phys_start = end + 1;
> (*pt_loads)[i].virt_start = kvend + 1;
> }
>+ (*pt_loads)[i].file_size -= (end -start);
> }
> }
> /* Insert split load segment, if any. */
>--
>2.9.3
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH 0/2] Fix refiltering when kaslr enabled
>Hi Atsushi, > >Thanks for the testing. > >On Wednesday 10 May 2017 01:37 PM, Atsushi Kumagai wrote: >>> Hi Atsushi, >>> >>> On Friday 28 April 2017 12:22 PM, Atsushi Kumagai wrote: >>>> Hello Pratyush, >>>> >>>> Thanks for your report, I have received this. >>>> I'm on vacation until Mar 8, I'll review it when I return from vacation. >>> >>> Any further comment on it? >>> Otherwise, I will send a v2 after accommodating concern from Xunlei. >> >> Unfortunately, it doesn't seem like I can make time anymore for review this >> week, >> but at least this patch doesn't seem to work in my environment (linux 4.8 >> without kaslr). >> Do you have any ideas ? > >I see, why it would have caused. I have not tested this case, but I hope my v2 >should not have this issue. Umm, v2 still doesn't work in my environment... It seems that I have to investigate this deeper. $ cat scrub.conf [vmlinux] erase modules size 50 $ (gdb) r -cd31 -x vmlinux --config scrub.conf vmcore dumpfile.cd31 Starting program: /work/kdump_utils/makedumpfile/makedumpfile -cd31 -x vmlinux --config scrub.conf vmcore dumpfile.cd31 warning: no loadable sections found in added symbol-file system-supplied DSO at 0x77ffd000 [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Segmentation fault. 0x00308366ee0d in fseek () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.5-7.el6_0.x86_64 elfutils-libelf-0.152-1.el6.x86_64 elfutils-libs-0.152-1.el6.x86_64 glibc-2.12-1.132.el6.x86_64 libgcc-4.4.7-4.el6.x86_64 libstdc++-4.4.7-4.el6.x86_64 snappy-1.1.0-1.el6.x86_64 xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x00308366ee0d in fseek () from /lib64/libc.so.6 #1 0x00420937 in get_kaslr_offset_x86_64 (vaddr=18446744071589596288) at arch/x86_64.c:43 #2 0x00414310 in resolve_config_entry (ce=0x701370, base_vaddr=, base_struct_name=0x0) at erase_info.c:1091 #3 0x00415a89 in get_config_symbol_addr (filter_symbol=0x701370, size_symbol=0x701430) at erase_info.c:1264 #4 update_filter_info (filter_symbol=0x701370, size_symbol=0x701430) at erase_info.c:1579 #5 0x00416543 in process_config (name_config=) at erase_info.c:1789 #6 process_config_file (name_config=) at erase_info.c:1862 #7 0x00417c57 in gather_filter_info () at erase_info.c:2356 #8 0x00443e5b in create_dumpfile () at makedumpfile.c:9870 #9 0x004457ae in main (argc=, argv=) at makedumpfile.c:11349 (gdb) Thanks, Atsushi Kuamgai >~Pratyush >> >> >> Program received signal SIGSEGV, Segmentation fault. >> 0x76be49f5 in fseek () from /lib64/libc.so.6 >> Missing separate debuginfos, use: debuginfo-install >> bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.163-3.el7.x86_64 >elfutils-libs-0.163-3.el7.x86_64 glibc-2.17-105.el7.x86_64 >libgcc-4.8.5-4.el7.x86_64 libstdc++-4.8.5-4.el7.x86_64 >snappy-1.1.0-3.el7.x86_64 xz-libs-5.1.2-12alpha.el7.x86_64 >zlib-1.2.7-15.el7.x86_64 >> (gdb) bt >> #0 0x76be49f5 in fseek () from /lib64/libc.so.6 >> #1 0x00429d38 in read_vmcoreinfo_symbol (str_symbol=0x44cb0c >> "SYMBOL(_stext)=") at makedumpfile.c:2384 >> #2 0x0042097a in get_kaslr_offset_x86_64 >> (vaddr=18446744071589596288) at arch/x86_64.c:45 >> #3 0x00414310 in resolve_config_entry (ce=0x701370, >> base_vaddr=, base_struct_name=0x0) >> at erase_info.c:1091 >> #4 0x00415a89 in get_config_symbol_addr (base_struct_name=0x0, >> base_vaddr=0, ce=0x701370) at erase_info.c:1264 >> #5 update_filter_info (filter_symbol=0x701370, size_symbol=0x701430) at >> erase_info.c:1579 >> #6 0x00416543 in process_config (config=) at >> erase_info.c:1789 >> #7 process_config_file (name_config=) at erase_info.c:1862 >> #8 0x00417c57 in gather_filter_info () at erase_info.c:2356 >> #9 0x00443ccb in create_dumpfile () at makedumpfile.c:9863 >> #10 0x0044561e in main (argc=, argv=) >> at makedumpfile.c:11342 >> (gdb) >> >> >> Thanks, >> Atsushi Kumagai >> >>> ~Pratyush >>> >>> >>>> >>>> Thanks, >>>> Atsushi Kumagai >>>> >>>>> Hi All, >>>>> >>>>> We came across another failure in makedumpfile when kaslr is enabled. This >>>>> failure occurs when we try re-filtering. We try to erase some symbol from >>>>> a >>>>> dumpfile which was copied/compressed from /proc/vmcor
RE: [PATCH] sparc64: Add initial sparc64 support
>On 05/14/2017 10:32 PM, Atsushi Kumagai wrote:
>> Hello Tom,
>>
>>> This commit adds support for sparc64. The changes were tested on a
>>> sparc64 T7-2 processor with 2 TB of RAM
>> Thanks for your work, what kernel version does this patch support ?
>
>I tested it on 4.1.12. I am trying to test it on 4.11.0 as well, but I
>am running into problems unrelated to kexec.
Sure, I note this test result.
>> I can't test for sparc64, so I hope you maintain this code.
>> Anyway, I'll merge this into v1.6.2.
>
>You can add me as the maintainer for sparc64. Thanks.
Thank you, too !
>> BTW, I have a trivial question below.
>>
>>> ---
>>> Makefile |6 +-
>>> arch/sparc64.c | 236
>>>
>>> makedumpfile.h | 110 ++
>>> 3 files changed, 350 insertions(+), 2 deletions(-)
>>> create mode 100644 arch/sparc64.c
>>>
>>> diff --git a/Makefile b/Makefile
>>> index 8b0fd24..4d96490 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -48,11 +48,13 @@ endif
>>> SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h
>>> sadump_info.h
>>> SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c
>>> cache.c
>>> OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART))
>>> -SRC_ARCH = arch/arm.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c
>>> arch/s390x.c arch/ppc.c
>>> -SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c
>>> arch/ppc64.c arch/s390x.c arch/ppc.c
>>> +SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c
>>> arch/ppc64.c arch/s390x.c arch/ppc.c
>>> arch/sparc64.c
>>> OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))
>>>
>>> LIBS = -ldw -lbz2 -lebl -ldl -lelf -lz
>>> +ifeq ($(ARCH), sparc64)
>>> +LIBS += -llzma
>>> +endif
>> Why lzma is required only on sparc64 ?
>
>I assumed it was a sparc-specific problem. On sparc64, I was seeing errors
>like the following:
>
>/usr/lib/gcc/sparc64-redhat-linux/4.4.7/../../../../lib64/libdw.a(lzma.o):
>In function `__libdw_unlzma':
>(.text+0x19c): undefined reference to `lzma_auto_decoder'
>
>But I tried building makedumpfile in an x86/64 VM, and also saw similar
>errors:
>
>/home/thromatka/git/elfutils-0.169/libdwfl/gzip.c:233: undefined
>reference to `lzma_auto_decoder'
>
>Perhaps it was the way I built elfutils? Thoughts?
Actually, elfutils requires liblzma as you said.
$ rpm -qf /usr/lib64/libdw.so.1
elfutils-libs-0.152-1.el6.x86_64
$ ldd /usr/lib64/libdw.so.1
linux-vdso.so.1 => (0x7fff0917d000)
libelf.so.1 => /usr/lib64/libelf.so.1 (0x00308520)
libdl.so.2 => /lib64/libdl.so.2 (0x003083e0)
liblzma.so.0 => /usr/lib64/liblzma.so.0 (0x00308ee0)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00309720)
libz.so.1 => /lib64/libz.so.1 (0x00308420)
libc.so.6 => /lib64/libc.so.6 (0x003083600000)
/lib64/ld-linux-x86-64.so.2 (0x563009dfc000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x003083a0)
$
It seems that -llzma is necessary if the library is static.
At least this is not a sparc64 issue, so I'll get rid of the code below
and fix it in another commit, is this fine with you ?
+ifeq ($(ARCH), sparc64)
+LIBS += -llzma
+endif
Thanks,
Atsushi Kumagai
>
>>
>> Thanks,
>> Atsushi Kumagai
>>
>>> ifneq ($(LINKTYPE), dynamic)
>>> LIBS := -static $(LIBS)
>>> endif
>>> diff --git a/arch/sparc64.c b/arch/sparc64.c
>>> new file mode 100644
>>> index 000..1cfaa85
>>> --- /dev/null
>>> +++ b/arch/sparc64.c
>>> @@ -0,0 +1,236 @@
>>> +/*
>>> + * Copyright (C) 2014, 2017 Oracle and/or its affiliates
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify
>>> + * it under the terms of the GNU General Public License as published by
>>> + * the Free Software Foundation (version 2 of the License).
>>> + *
>>> + * This program is distributed in the hope that it will be useful,
>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>> + * GNU General Public License for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> + * along with this program; if not, write to the Free Softw
RE: [PATCH] sparc64: Add initial sparc64 support
Hello Tom,
>This commit adds support for sparc64. The changes were tested on a
>sparc64 T7-2 processor with 2 TB of RAM
Thanks for your work, what kernel version does this patch support ?
I can't test for sparc64, so I hope you maintain this code.
Anyway, I'll merge this into v1.6.2.
BTW, I have a trivial question below.
>---
> Makefile |6 +-
> arch/sparc64.c | 236
> makedumpfile.h | 110 ++
> 3 files changed, 350 insertions(+), 2 deletions(-)
> create mode 100644 arch/sparc64.c
>
>diff --git a/Makefile b/Makefile
>index 8b0fd24..4d96490 100644
>--- a/Makefile
>+++ b/Makefile
>@@ -48,11 +48,13 @@ endif
> SRC_BASE = makedumpfile.c makedumpfile.h diskdump_mod.h sadump_mod.h
> sadump_info.h
> SRC_PART = print_info.c dwarf_info.c elf_info.c erase_info.c sadump_info.c
> cache.c
> OBJ_PART=$(patsubst %.c,%.o,$(SRC_PART))
>-SRC_ARCH = arch/arm.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c
>arch/s390x.c arch/ppc.c
>-SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c
>arch/ppc64.c arch/s390x.c arch/ppc.c
>+SRC_ARCH = arch/arm.c arch/arm64.c arch/x86.c arch/x86_64.c arch/ia64.c
>arch/ppc64.c arch/s390x.c arch/ppc.c
>arch/sparc64.c
> OBJ_ARCH=$(patsubst %.c,%.o,$(SRC_ARCH))
>
> LIBS = -ldw -lbz2 -lebl -ldl -lelf -lz
>+ifeq ($(ARCH), sparc64)
>+LIBS += -llzma
>+endif
Why lzma is required only on sparc64 ?
Thanks,
Atsushi Kumagai
> ifneq ($(LINKTYPE), dynamic)
> LIBS := -static $(LIBS)
> endif
>diff --git a/arch/sparc64.c b/arch/sparc64.c
>new file mode 100644
>index 000..1cfaa85
>--- /dev/null
>+++ b/arch/sparc64.c
>@@ -0,0 +1,236 @@
>+/*
>+ * Copyright (C) 2014, 2017 Oracle and/or its affiliates
>+ *
>+ * This program is free software; you can redistribute it and/or modify
>+ * it under the terms of the GNU General Public License as published by
>+ * the Free Software Foundation (version 2 of the License).
>+ *
>+ * This program is distributed in the hope that it will be useful,
>+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
>+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>+ * GNU General Public License for more details.
>+ *
>+ * You should have received a copy of the GNU General Public License
>+ * along with this program; if not, write to the Free Software
>+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
>+ */
>+
>+#ifdef __sparc64__
>+
>+#include "../elf_info.h"
>+#include "../makedumpfile.h"
>+#include "../print_info.h"
>+
>+int get_versiondep_info_sparc64(void)
>+{
>+ info->section_size_bits = _SECTION_SIZE_BITS;
>+
>+ if (info->kernel_version >= KERNEL_VERSION(3, 8, 13))
>+ info->max_physmem_bits = _MAX_PHYSMEM_BITS_L4;
>+ else {
>+ info->max_physmem_bits = _MAX_PHYSMEM_BITS_L3;
>+ info->flag_vmemmap = TRUE;
>+ info->vmemmap_start = VMEMMAP_BASE_SPARC64;
>+ info->vmemmap_end = VMEMMAP_BASE_SPARC64 +
>+ ((1UL << (info->max_physmem_bits - PAGE_SHIFT)) *
>+ SIZE(page));
>+ }
>+
>+ return TRUE;
>+}
>+
>+int get_phys_base_sparc64(void)
>+{
>+ /* Ideally we'd search the pt_load entries until we found one
>+ * containing KVBASE (_stext), but get_symbol_info hasn't been
>+ * called yet. We'll just go with the first entry.
>+ */
>+ unsigned long long phys_start;
>+ unsigned long long virt_start;
>+ unsigned long long virt_end;
>+
>+ if (get_pt_load(0, _start, NULL, _start, _end)) {
>+ info->phys_base = phys_start & ~KVBASE_MASK;
>+ return TRUE;
>+ }
>+ ERRMSG("Can't find kernel segment\n");
>+ return FALSE;
>+}
>+
>+int is_vmalloc_addr_sparc64(unsigned long vaddr)
>+{
>+ return (vaddr >= VMALLOC_START_SPARC64);
>+}
>+
>+int is_vmemmap_addr_sparc64(unsigned long vaddr)
>+{
>+ if (info->flag_vmemmap &&
>+ (vaddr >= info->vmemmap_start) && (vaddr < info->vmemmap_end))
>+ return TRUE;
>+
>+ return FALSE;
>+}
>+
>+unsigned long vmemmap_to_phys_sparc64(unsigned long vaddr)
>+{
>+ unsigned long vmemmap_table;
>+ unsigned long offset = vaddr - info->vmemmap_start;
>+ unsigned long chunk_offset = offset & ~VMEMMAP_CHUNK_MASK;
>+ unsigned long chunk;
>+ unsigned long index;
>+ unsigned long pte;
>+ unsigned long pte_paddr;
>+ unsigned long pte_offset;
>+
RE: [Makedumpfile PATCH 0/2] Fix refiltering when kaslr enabled
>Hi Atsushi, > >On Friday 28 April 2017 12:22 PM, Atsushi Kumagai wrote: >> Hello Pratyush, >> >> Thanks for your report, I have received this. >> I'm on vacation until Mar 8, I'll review it when I return from vacation. > >Any further comment on it? >Otherwise, I will send a v2 after accommodating concern from Xunlei. Unfortunately, it doesn't seem like I can make time anymore for review this week, but at least this patch doesn't seem to work in my environment (linux 4.8 without kaslr). Do you have any ideas ? Program received signal SIGSEGV, Segmentation fault. 0x76be49f5 in fseek () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.163-3.el7.x86_64 elfutils-libs-0.163-3.el7.x86_64 glibc-2.17-105.el7.x86_64 libgcc-4.8.5-4.el7.x86_64 libstdc++-4.8.5-4.el7.x86_64 snappy-1.1.0-3.el7.x86_64 xz-libs-5.1.2-12alpha.el7.x86_64 zlib-1.2.7-15.el7.x86_64 (gdb) bt #0 0x76be49f5 in fseek () from /lib64/libc.so.6 #1 0x00429d38 in read_vmcoreinfo_symbol (str_symbol=0x44cb0c "SYMBOL(_stext)=") at makedumpfile.c:2384 #2 0x0042097a in get_kaslr_offset_x86_64 (vaddr=18446744071589596288) at arch/x86_64.c:45 #3 0x00414310 in resolve_config_entry (ce=0x701370, base_vaddr=, base_struct_name=0x0) at erase_info.c:1091 #4 0x00415a89 in get_config_symbol_addr (base_struct_name=0x0, base_vaddr=0, ce=0x701370) at erase_info.c:1264 #5 update_filter_info (filter_symbol=0x701370, size_symbol=0x701430) at erase_info.c:1579 #6 0x00416543 in process_config (config=) at erase_info.c:1789 #7 process_config_file (name_config=) at erase_info.c:1862 #8 0x00417c57 in gather_filter_info () at erase_info.c:2356 #9 0x00443ccb in create_dumpfile () at makedumpfile.c:9863 #10 0x0044561e in main (argc=, argv=) at makedumpfile.c:11342 (gdb) Thanks, Atsushi Kumagai >~Pratyush > > >> >> Thanks, >> Atsushi Kumagai >> >>> Hi All, >>> >>> We came across another failure in makedumpfile when kaslr is enabled. This >>> failure occurs when we try re-filtering. We try to erase some symbol from a >>> dumpfile which was copied/compressed from /proc/vmcore using makedumpfile. >>> >>> We have very limited symbol information in vmcoreinfo. So symbols to be >>> erased may not be available in vmcoreinfo and we look for it in vmlinux. >>> However, symbol address from vmlinux is a static address which differs >>>from run time address with KASLR_OFFSET. Therefore, reading any "virtual >>> address of vmlinux" from vmcore is not possible. >>> >>> These patches finds runtime KASLR offset and then calculates run time >>> address of symbols read from vmlinux. >>> >>> Since, I am not an expert of x86, and these patches touch x86 part of >>> makedumpfile, therefore I have CCed x86 experts. Please, provide your >>> review comment and let me know if you think there could have been a better >>> way to resolve this issue. >>> >>> thanks >>> >>> ~Pratyush >>> >>> Pratyush Anand (2): >>> makedumpfile: add runtime kaslr offset if it exists >>> x86_64: calculate page_offset in case of re-filtering >>> >>> arch/x86_64.c | 45 +++-- >>> erase_info.c | 1 + >>> makedumpfile.c | 44 >>> makedumpfile.h | 15 +++ >>> 4 files changed, 103 insertions(+), 2 deletions(-) >>> >>> -- >>> 2.9.3 >> ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH] elf_info: fix file_size if segment is excluded
Hello Pratyush,
Thanks for your report, I have just one question.
>> exclude_segment() is called for Crash Kernel whose range is
>> 2b00-350f.
>>
>> We see following after exclude_segment()
>>
>> LOAD (2)
>> phys_start : 10
>> phys_end : 2aff
>> virt_start : 8a5a4010
>> virt_end : 8a5a6aff
>> file_offset: a5a40102000
>> file_size : dfefd000
>> LOAD (3)
>> phys_start : 3510
>> phys_end : dfffd000
>> virt_start : 8a5a7510
>> virt_end : 8a5b1fffd000
>> file_offset: a5a75102000
>> file_size : 0
>>
>> Since file_size is calculated wrong therefore readpage_elf() does not
>> behave correctly.
>>
>> This patch fixes above wrong behavior.
>>
>> Signed-off-by: Pratyush Anand <pan...@redhat.com>
>> ---
>> elf_info.c | 3 +++
>> 1 file changed, 3 insertions(+)
>>
>> diff --git a/elf_info.c b/elf_info.c
>> index 8e2437622141..6bf5e3373595 100644
>> --- a/elf_info.c
>> +++ b/elf_info.c
>> @@ -826,9 +826,12 @@ static int exclude_segment(struct pt_load_segment
>> **pt_loads,
>>temp_seg.virt_end = vend;
>>temp_seg.file_offset = (*pt_loads)[i].file_offset
>>+ temp_seg.virt_start - (*pt_loads)[i].virt_start;
>> +temp_seg.file_size = temp_seg.phys_end
>> +- temp_seg.phys_start;
>>
>>(*pt_loads)[i].virt_end = kvstart - 1;
>> (*pt_loads)[i].phys_end = start - 1;
>> +(*pt_loads)[i].file_size -= temp_seg.file_size;
I think we should additionally subtract (end - start), right ?
This code seems to leave Crash Kernel region for the former half of PT_LOAD.
Thanks,
Atsushi Kumagai
>>tidx = i+1;
>>} else if (kvstart != vstart) {
>> --
>> 2.9.3
>>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: Help regarding renaming of vmcore-incomplete to vmcore
Hello, >Hi I am a university student, and have made some changes to >makedumpfile to produce a gzipped output of core dump. The issue I'm >facing is that the output i get from makedumpfile (after it completes >successfully) is vmcore-incomplete.gzip. > >But this file is not being renamed to vmcore.gzip (even thought the >dump the complete as I verified it by loading into CRASH utility). Can >someone please point out to me the changes I should make to solve this >bug (or can point to the code where this renaming is done). makedumpfile doesn't have such renaming mechanism, it just name the dump as user specify. I guess "vmcore-incomplete" you said is came from kexec-tools. For example, kexec-tools of RHEL7 generate the code below for dump-capture kernel: 110 echo "kdump: saving vmcore" 111 $CORE_COLLECTOR /proc/vmcore $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore-incomplete || return 1 112 mv $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore-incomplete $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore 113 sync $CORE_COLLECTOR is makedumpfile, so the code generate "vmcore-incomplete" by makedumpfile and rename it to vmcore by mv. If you want "vmcore.gzip", you should just specify vmcore.gzip for makedumpfile. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH 0/2] Fix refiltering when kaslr enabled
Hello Pratyush, Thanks for your report, I have received this. I'm on vacation until Mar 8, I'll review it when I return from vacation. Thanks, Atsushi Kumagai >Hi All, > >We came across another failure in makedumpfile when kaslr is enabled. This >failure occurs when we try re-filtering. We try to erase some symbol from a >dumpfile which was copied/compressed from /proc/vmcore using makedumpfile. > >We have very limited symbol information in vmcoreinfo. So symbols to be >erased may not be available in vmcoreinfo and we look for it in vmlinux. >However, symbol address from vmlinux is a static address which differs >from run time address with KASLR_OFFSET. Therefore, reading any "virtual >address of vmlinux" from vmcore is not possible. > >These patches finds runtime KASLR offset and then calculates run time >address of symbols read from vmlinux. > >Since, I am not an expert of x86, and these patches touch x86 part of >makedumpfile, therefore I have CCed x86 experts. Please, provide your >review comment and let me know if you think there could have been a better >way to resolve this issue. > >thanks > >~Pratyush > >Pratyush Anand (2): > makedumpfile: add runtime kaslr offset if it exists > x86_64: calculate page_offset in case of re-filtering > > arch/x86_64.c | 45 +++-- > erase_info.c | 1 + > makedumpfile.c | 44 > makedumpfile.h | 15 +++ > 4 files changed, 103 insertions(+), 2 deletions(-) > >-- >2.9.3 ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH] [PATCH] Fix typo in the ERRMSG of function show_mem_usage. "kernel" instead of "kenrel".
Hello Eric,
Thanks, I'll merge this fix into v1.6.2.
P.S. I fixed other typos on this occasion.
Regards,
Atsushi Kumagai
>---
> makedumpfile.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index e3be1ab..18cf30f 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -10938,7 +10938,7 @@ int show_mem_usage(void)
> struct cycle cycle = {0};
>
> if (!is_crashkernel_mem_reserved()) {
>- ERRMSG("No memory is reserved for crashkenrel!\n");
>+ ERRMSG("No memory is reserved for crashkernel!\n");
> return FALSE;
> }
>
>--
>2.7.4
>
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch v3 7/7] mem-usage: allow to work only with -f for kernel version < 4.11
>> On Friday 03 March 2017 07:40 AM, Atsushi Kumagai wrote:
>> > > +if (info->kernel_version < KERNEL_VERSION(4, 11, 0) &&
>> > > +!info->flag_force) {
>> > > +MSG("mem-usage not supported for this
>> > > kernel.\n");
>> > > +MSG("You can try with -f if your kernel's kcore
>> > > has valid p_paddr\n");
>> > > +goto out;
>> > > +}
>> > You forgot to set COMPLETED to retcd before goto.
>> > The behavior will be different from the v2 patch.
>>
>>
>> I had thought about it. Should not an unsupported feature be a FAILED case?
>
>
>Judging from the result, it should be a failed case. People expect to
>get a dumped vmcore, it failed to collect at last because of unmatched
>kernel and tool.
When you change the design, please note it into changelog,
it's a review point of view.
Changes since v2:
- Fixed a memory leak issue and updated man page and usage info
As for the return value, I guess both is OK if there is a clear intention.
If this is what you intended, I have no objection.
(it actually sounds more reasonable)
So I'll merge the v3 patch into v1.6.2, thanks for your work.
Regards,
Atsushi Kumagai
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch v3 7/7] mem-usage: allow to work only with -f for kernel version < 4.11
>PT_LOAD of kcore does not have valid p_paddr values for kernel version
>less that v4.11. Therefore, older kernel will no long work for mem-usage
>with current makedumpfile code. They can only work when they are patched
>with fix to "update physical address for kcore ram and text".
>
>This patch fixes the makedumpfile so that it does not allow to work
>older kernel for --mem-usage until someone is sure that kernel is
>rightly patched and so uses -f in command line. It also updates man page
>and usage info accordingly.
>
>Signed-off-by: Pratyush Anand <pan...@redhat.com>
>---
> makedumpfile.8 | 9 -
> makedumpfile.c | 6 ++
> print_info.c | 3 ++-
> 3 files changed, 16 insertions(+), 2 deletions(-)
>
>diff --git a/makedumpfile.8 b/makedumpfile.8
>index 9069fb18cdb6..993236486e77 100644
>--- a/makedumpfile.8
>+++ b/makedumpfile.8
>@@ -235,13 +235,20 @@ the ELF format does not support compressed data.
>
> .TP
> \fB\-f\fR
>-Force existing DUMPFILE to be overwritten.
>+Force existing DUMPFILE to be overwritten and mem-usage to work with older
>+kernel as well.
> .br
> .B Example:
> .br
> # makedumpfile \-f \-d 31 \-x vmlinux /proc/vmcore dumpfile
> .br
> This command overwrites \fIDUMPFILE\fR even if it already exists.
>+.br
>+# makedumpfile \-f \-\-mem\-usage /proc/kcore
>+.br
>+Kernel version lesser than v4.11 will not work with \-\-mem\-usage
>+functionality until it has been patched with upstream commit 464920104bf7.
>+Therefore if you have patched your older kernel then use \-f.
>
> .TP
> \fB\-x\fR \fIVMLINUX\fR
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 3b8e9810468d..e3be1ab0a9ec 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -11269,6 +11269,12 @@ main(int argc, char *argv[])
> MSG("Try `makedumpfile --help' for more
> information.\n");
> goto out;
> }
>+ if (info->kernel_version < KERNEL_VERSION(4, 11, 0) &&
>+ !info->flag_force) {
>+ MSG("mem-usage not supported for this kernel.\n");
>+ MSG("You can try with -f if your kernel's kcore has
>valid p_paddr\n");
>+ goto out;
>+ }
You forgot to set COMPLETED to retcd before goto.
The behavior will be different from the v2 patch.
Thanks,
Atsushi Kumagai
>
> if (!show_mem_usage())
> goto out;
>diff --git a/print_info.c b/print_info.c
>index 392d863a4227..72ed8fa0c059 100644
>--- a/print_info.c
>+++ b/print_info.c
>@@ -309,7 +309,8 @@ print_usage(void)
> MSG(" Print debugging message.\n");
> MSG("\n");
> MSG(" [-f]:\n");
>- MSG(" Overwrite DUMPFILE even if it already exists.\n");
>+ MSG(" Overwrite DUMPFILE even if it already exists\n");
>+ MSG(" Force mem-usage to work with older kernel as well.\n");
> MSG("\n");
> MSG(" [-h, --help]:\n");
> MSG(" Show help message and LZO/snappy support status
> (enabled/disabled).\n");
>--
>2.9.3
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch v2 7/7] mem-usage: allow to work only with -f for kernel version < 4.11
>Hi Atsushi,
>
>On Thursday 02 March 2017 10:19 AM, Atsushi Kumagai wrote:
>> Hello Pratyush,
>>
>>> PT_LOAD of kcore does not have valid p_paddr values for kernel version
>>> less that v4.11. Therefore, older kernel will no long work for mem-usage
>>> with current makedumpfile code. They can only work when they are patched
>>> with fix to "update physical address for kcore ram and text".
>>>
>>> This patch fixes the makedumpfile so that it does not allow to work
>>> older kernel for --mem-usage until someone is sure that kernel is
>>> rightly patched and so uses -f in command line.
>>>
>>> Signed-off-by: Pratyush Anand <pan...@redhat.com>
>>> ---
>>> makedumpfile.c | 6 ++
>>> 1 file changed, 6 insertions(+)
>>>
>>> diff --git a/makedumpfile.c b/makedumpfile.c
>>> index 3b8e9810468d..bf006ea5dd5f 100644
>>> --- a/makedumpfile.c
>>> +++ b/makedumpfile.c
>>> @@ -11269,6 +11269,12 @@ main(int argc, char *argv[])
>>> MSG("Try `makedumpfile --help' for more
>>> information.\n");
>>> goto out;
>>> }
>>> + if (info->kernel_version < KERNEL_VERSION(4, 11, 0) &&
>>> + !info->flag_force) {
>>> + MSG("mem-usage not supported for this kernel.\n");
>>> + MSG("You can try with -f if your kernel's kcore has
>>> valid p_paddr\n");
>>> + return COMPLETED;
>>> + }
>>
>> Should use "goto out" to prevent memory leaks since some heap blocks are
>> allocated at the head of main().
>
>OK
>
>>
>> BTW, the descriptions of -f option in man and print_usage() don't mention
>> this usage:
>>
>>-f Force existing DUMPFILE to be overwritten.
>> Example:
>> # makedumpfile -f -d 31 -x vmlinux /proc/vmcore dumpfile
>> This command overwrites DUMPFILE even if it already exists.
>>
>> so they should be updated.
>
>Does following looks fine to you?
looks good to me.
I'll wait for the next version.
Thanks,
Atsushi Kumagai
>$ makedumpfile --help | grep -A3 "\-f"
> [-f]:
> Overwrite DUMPFILE even if it already exists
> Force mem-usage to work with older kernel as well.
>
>
>$ man makedumpfile | grep -w "\-f " -A6
>-f Force existing DUMPFILE to be overwritten and mem-usage
>to work with older kernel as well.
> Example:
> # makedumpfile -f -d 31 -x vmlinux /proc/vmcore dumpfile
> This command overwrites DUMPFILE even if it already exists.
> # makedumpfile -f --mem-usage /proc/kcore
> Kernel version lesser than v4.11 will not work with
>--mem-usage functionality until it has been patched with upstream commit
> 464920104bf7. Therefore if you have patched your older
>kernel then use -f.
>
>Thanks for the review.
>
>~Pratyush
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch v2 7/7] mem-usage: allow to work only with -f for kernel version < 4.11
Hello Pratyush,
>PT_LOAD of kcore does not have valid p_paddr values for kernel version
>less that v4.11. Therefore, older kernel will no long work for mem-usage
>with current makedumpfile code. They can only work when they are patched
>with fix to "update physical address for kcore ram and text".
>
>This patch fixes the makedumpfile so that it does not allow to work
>older kernel for --mem-usage until someone is sure that kernel is
>rightly patched and so uses -f in command line.
>
>Signed-off-by: Pratyush Anand <pan...@redhat.com>
>---
> makedumpfile.c | 6 ++
> 1 file changed, 6 insertions(+)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 3b8e9810468d..bf006ea5dd5f 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -11269,6 +11269,12 @@ main(int argc, char *argv[])
> MSG("Try `makedumpfile --help' for more
> information.\n");
> goto out;
> }
>+ if (info->kernel_version < KERNEL_VERSION(4, 11, 0) &&
>+ !info->flag_force) {
>+ MSG("mem-usage not supported for this kernel.\n");
>+ MSG("You can try with -f if your kernel's kcore has
>valid p_paddr\n");
>+ return COMPLETED;
>+ }
Should use "goto out" to prevent memory leaks since some heap blocks are
allocated at the head of main().
BTW, the descriptions of -f option in man and print_usage() don't mention this
usage:
-f Force existing DUMPFILE to be overwritten.
Example:
# makedumpfile -f -d 31 -x vmlinux /proc/vmcore dumpfile
This command overwrites DUMPFILE even if it already exists.
so they should be updated.
Thanks,
Atsushi Kumagai
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch 0/6] Fix --mem-usage /proc/kcore
>On Monday 13 February 2017 11:56 AM, Atsushi Kumagai wrote: >> Hello Pratyush, >> >>> `makedumpfile --mem-usage /proc/kcore` has been broken after kaslr specific >>> modifications. I have proposed a kernel patch [0] which has been ACKed by >>> Andrew Morton and is available in next-2017020 now. Hopefully, it should be >>> part of upstream in v4.10 release. This kernel patch helps to fix this >>> issue for both the case of kaslr enabled and disabled. >>> >>> To seek other's feedback, I am sending makedumpfile patches before kernel >>> patch hits upstream. >> >> Thanks for taking care of this feature. >> I understand that the problem is: >> >> - phys_base is necessary for VtoP, but: >>- SYMBOL(phys_base) is unavailable since /proc/kcore doesn't export >> VMCOREINFO. >>- get_phys_base() doesn't work well since current PT_LOAD doesn't have >> valid p_paddr. > >Yes, this is what the problem is. > >> >> The new makedumpfile with this patches can show unexpected behavior since it >> will refer to invalid p_paddr(always 0). Of course, also current >> makedumpfile cannot >> work if phys_base isn't 0. >> >> If we don't have any way to get phys_base in 1st kernel environment, >> I think we should disable this feature in older kernel (e.g. by checking >> kernel version). > >Hummm..I had thought that distros with old kernel will backport kernel >patch and then can use this feature. If we disable it for older kernel, >then they will not be able to do it. > >would it be fine if I cc kernel patch to stable mailing list? It seems >that patch can be easily applied all older kernel. Even if there are some "supported" older kernels, there is no guarantee that all of the kernels provide valid p_paddr. So makedumpfile have to know whether the target kernel supports --mem-usage or not in older kernel. Is it possible to add a new stuff into PT_NOTE of /proc/kcore to signify the kernel supports --mem-usage ? Otherwise, my only idea is --force option for older kernel, expert user can use it if he is sure that the kernel has valid p_paddr. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile Patch 0/6] Fix --mem-usage /proc/kcore
Hello Pratyush, >`makedumpfile --mem-usage /proc/kcore` has been broken after kaslr specific >modifications. I have proposed a kernel patch [0] which has been ACKed by >Andrew Morton and is available in next-2017020 now. Hopefully, it should be >part of upstream in v4.10 release. This kernel patch helps to fix this >issue for both the case of kaslr enabled and disabled. > >To seek other's feedback, I am sending makedumpfile patches before kernel >patch hits upstream. Thanks for taking care of this feature. I understand that the problem is: - phys_base is necessary for VtoP, but: - SYMBOL(phys_base) is unavailable since /proc/kcore doesn't export VMCOREINFO. - get_phys_base() doesn't work well since current PT_LOAD doesn't have valid p_paddr. The new makedumpfile with this patches can show unexpected behavior since it will refer to invalid p_paddr(always 0). Of course, also current makedumpfile cannot work if phys_base isn't 0. If we don't have any way to get phys_base in 1st kernel environment, I think we should disable this feature in older kernel (e.g. by checking kernel version). Thanks, Atsushi Kumagai >[0] >http://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/commit/?h=next-20170206=c9d4e5d7b7fd6c74e134ca44d >f8a5386efbc561c > >Baoquan He (2): > makedumpfile: Correct the calculation of kvaddr in >set_kcore_vmcoreinfo > makedumpfile: Discard process_dump_load > >Pratyush Anand (4): > show_mem_usage(): calculate page offset after elf load > initial(): call cach_init() a bit early > x86_64: check physical address in PT_LOAD for none direct mapped >regions > elf_info: kcore: check for invalid physical address > > arch/x86_64.c | 6 -- > elf_info.c | 25 + > makedumpfile.c | 12 ++-- > 3 files changed, 15 insertions(+), 28 deletions(-) > >-- >2.9.3 > > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
makedumpfile-1.6.1: Enhance the support for ppc64 and arm64
Hello, Thank you for waiting, at last makedumpfile version 1.6.1 is released. Your comments/patches are welcome. Main new feature: o Support new kernels - The supported kernel is updated to 4.8 in this version. Changelog: o New feature - [PATCH v2] Support _count -> _refcount rename in struct page (by Vitaly Kuznetsov) ed46b6a - [PATCH 1/8] ppc64: fix vtop page translation for 4K pages (by Hari Bathini) cf02f88 - [PATCH 2/8] ppc64: Use kernel terminology for each level in 4-level page table (by Hari Bathini) 37abbe1 - [PATCH 3/8] ppc64: address changes in kernel v4.5 (by Hari Bathini) bc86b61 - [PATCH 4/8] ppc64: address change in _PAGE_PRESNT flag for PowerISA v3.0 (by Hari Bathini) 3003fbe - [PATCH 5/8] ppc64: use physical addresses and unfold pud for 64K page size (by Hari Bathini) afe7f9f - [PATCH 6/8] ppc64: support big endian Linux page tables (by Hari Bathini) f3d7cc5 - [PATCH 7/8] ppc64: use the same masked bit values for 4K and 64K pagesizes (by Hari Bathini) 398746a - [PATCH 8/8] ppc64: enable address translation support for radix mmu (by Hari Bathini) ed65d60 - [PATCH 01/10] arm64: cleanup code, comment, blank space, blank lines etc (by Pratyush Anand) c42c582 - [PATCH 02/10] read_vmcoreinfo_long: Allow to read hex values as well (by Pratyush Anand) b01fa28 - [PATCH 03/10] Introduce read_vmcoreinfo_ulong() (by Pratyush Anand) edc314e - [PATCH 04/10] arm64: use already available PAGESIZE() and PAGESHIFT() macros (by Pratyush Anand) 48581a7 - [PATCH 05/10] arm64: fix page_offset definition (by Pratyush Anand) a4335c6 - [PATCH 06/10] arm64: fix re-filtering (by Pratyush Anand) 618b76c - [PATCH 07/10] arm64: use value of VA_BITS and PHYS_OFFSET embedded into vmcore (by Pratyush Anand) 74df822 - [PATCH 08/10] arm64: immunize identity mapped address finding w.r.t. kernel changes (by Pratyush Anand) a84f726 - [PATCH 09/10] arm64: Add support for 4level 4K page translations table (by Azriel Samson) 969ee30 - [PATCH 10/10] arm64: fix memory layout as per changes in v4.6 kernel (by Pratyush Anand) b6fe70c o Bugfix - [PATCH 1/2] sadump: fix segmentation fault on sadump-related formats (by HATAYAMA Daisuke) 0ba14bc - [PATCH 2/2] sadump: fix wrong progress report on sadump formats (by HATAYAMA Daisuke) d5d268b - [PATCH] Fix module init base and size offset for kernel v4.5 (by Guenther Hutzl) 32dd468 - [PATCH] Fix uninitialized file descriptors for parallel dump (by Atsushi Kumagai) 1474dd4 - [PATCH] x86_64: Set zero to phys_base for the kernel older than 2.6.22 (by Atsushi Kumagai) 114060b o Cleanup - [PATCH] Add more descriptions of multi-threads feature (by Zhou Wenjian) 2af8d6a - [PATCH v2] Cleanup: Use a negative number for uninitialized file descriptors (by Petr Tesarik) 8b8fa9f - [PATCH v3 1/3] open_dump_bitmap: open bitmap file in non-cyclic case (by Martin Wilck) a8d8657 - [PATCH v3 2/3] move call to open_dump_bitmap() to after call to initial() (by Martin Wilck) 3696c40 - [PATCH v3 3/3] close_dump_bitmap: simplify logic (by Martin Wilck) c1c53f8 - [PATCH v2 1/2] Adapt code to get value of phys_base (by Baoquan He) acdbb1d - [PATCH V2 1/4] x86_64: Calculate page_offset from pt_load (by Pratyush Anand) 0c9dd01 - [PATCH V2 2/4] x86_64: translate all VA to PA using page table values (by Pratyush Anand) c41e33c - [PATCH V2 3/4] x86_64: kill is_vmalloc_addr_x86_64() (by Pratyush Anand) 8419f4d - [PATCH V2 4/4] x86_64: kill some unused initialization (by Pratyush Anand) ecb3719 - [PATCH v2 2/2] Clean up unused KERNEL_IMAGE_SIZE (by Atsushi Kumagai) 0068010 Explanation of makedumpfile: To shorten the size of the dumpfile and the time of creating the dumpfile, makedumpfile copies only the necessary pages for analysis to the dumpfile from /proc/vmcore. You can specify the kind of unnecessary pages with dump_level. If you want to shorten the size further, enable the compression of the page data. Download: You can download the latest makedumpfile from the following URL. Details of the change are written on the git page of the following site. https://sourceforge.net/projects/makedumpfile/ Method of installation: You can compile the makedumpfile command as follows; 1. "tar -zxvf makedumpfile-x.y.z.tar.gz" 2. "cd makedumpfile-x.y.z" 3. "make; make install" Usage: makedumpfile [-c|-l|-p] [-E] [-d dump_level] [-x vmlinux] dump_mem dump_file Example: If you want to exclude pages filled by zero, cache pages, user pages and free pages and to enable compression, please execute the following command. # makedumpfile -c -d 31 /proc/vmcore dumpfile Thanks Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [RFC PATCH 0/4] eppic: Create kernel version compatible scripts
Hello Kamalesh, >This patch series creates eppic scripts a for range kernel versions >they are compatible with. Eppic scripts directory host sample >scripts to scrub sensitive information from the dump file generated >using makedumpfile. > >The initial version of these scripts was based on Fedora 19 kernel. >In brief, these scripts rely on hard coded kernel data structure >member offsets. Kernel data structures are bound to change in due course >and leading to failure when assumptions about the offsets differ. > >Atsushi-San suggested that, it's better to have different version of >eppic scripts, those will be valid for the range of kernel release and >their naming convention hinting, of the release they are valid across. > >The first patch renames the existing eppic script to reflect the kernel >release they are compatible with. Following the format: >-_to_.c > >Rest of three patches creates new eppic scripts to match with the kernel >data structures they work with. Thank you for reflecting my comments, looks good to me. v1.6.1 is ready now, these patches will be merged into v1.6.2. Thanks, Atsushi Kumagai >Kamalesh Babulal (4): > eppic: Rename scripts to reflect validity of kernel version > eppic/vhost_net_buffers: Introduce changes for kernel 3.19 > eppic/dir_names: Introduce changes for kernel 3.14 > eppic/keyring: Introduce changes for kernel 4.4 > > eppic_scripts/README | 26 +- > eppic_scripts/ap_messages.c| 82 -- > eppic_scripts/ap_messages_3_10_to_4_8.c| 82 ++ > eppic_scripts/dir_names.c | 78 - > eppic_scripts/dir_names_3_10_to_3_13.c | 78 + > eppic_scripts/dir_names_3_14_to_4_8.c | 82 ++ > eppic_scripts/keyring.c| 57 > eppic_scripts/keyring_3_10_to_4_3.c| 57 > eppic_scripts/keyring_4_4_to_4_8.c | 378 + > eppic_scripts/proc_names.c | 49 > eppic_scripts/proc_names_3_10_to_4_8.c | 49 > eppic_scripts/tcp_sk_buf.c | 82 -- > eppic_scripts/tcp_sk_buf_3_10_to_4_8.c | 82 ++ > eppic_scripts/udp_sk_buf.c | 83 -- > eppic_scripts/udp_sk_buf_3_10_to_4_8.c | 83 ++ > eppic_scripts/unix_sk_buff.c | 85 -- > eppic_scripts/unix_sk_buff_3_10_to_4_8.c | 85 ++ > eppic_scripts/vhost_net_buffers.c | 99 --- > eppic_scripts/vhost_net_buffers_3_10_to_3_18.c | 99 +++ > eppic_scripts/vhost_net_buffers_3_19_to_4_8.c | 104 +++ > eppic_scripts/vhost_scsi_buffers.c | 75 - > eppic_scripts/vhost_scsi_buffers_3_10_to_4_8.c | 75 + > 22 files changed, 1270 insertions(+), 700 deletions(-) > delete mode 100644 eppic_scripts/ap_messages.c > create mode 100644 eppic_scripts/ap_messages_3_10_to_4_8.c > delete mode 100644 eppic_scripts/dir_names.c > create mode 100644 eppic_scripts/dir_names_3_10_to_3_13.c > create mode 100644 eppic_scripts/dir_names_3_14_to_4_8.c > delete mode 100644 eppic_scripts/keyring.c > create mode 100644 eppic_scripts/keyring_3_10_to_4_3.c > create mode 100644 eppic_scripts/keyring_4_4_to_4_8.c > delete mode 100644 eppic_scripts/proc_names.c > create mode 100644 eppic_scripts/proc_names_3_10_to_4_8.c > delete mode 100644 eppic_scripts/tcp_sk_buf.c > create mode 100644 eppic_scripts/tcp_sk_buf_3_10_to_4_8.c > delete mode 100644 eppic_scripts/udp_sk_buf.c > create mode 100644 eppic_scripts/udp_sk_buf_3_10_to_4_8.c > delete mode 100644 eppic_scripts/unix_sk_buff.c > create mode 100644 eppic_scripts/unix_sk_buff_3_10_to_4_8.c > delete mode 100644 eppic_scripts/vhost_net_buffers.c > create mode 100644 eppic_scripts/vhost_net_buffers_3_10_to_3_18.c > create mode 100644 eppic_scripts/vhost_net_buffers_3_19_to_4_8.c > delete mode 100644 eppic_scripts/vhost_scsi_buffers.c > create mode 100644 eppic_scripts/vhost_scsi_buffers_3_10_to_4_8.c > >-- >2.7.4 > > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 1/2] eppic: vhost_net_buffers - Adopt to struct sk_buff changes
>On Monday 19 December 2016 01:04 PM, Atsushi Kumagai wrote: >> Hello Kamalesh, >> >> Thanks for taking care of the scripts. >> >> The current scripts are created based on the same kernel version: >> >> === >> Eppic scripts README >> == >> >> The eppic scripts are based on the fedora 19 kernel. >> >> So if you update a part of the scripts, the target kernel version of >> them will be complex and confusing. >> I don't think we need to update these scripts since they are just sample, >> the purpose is to tell users how to write a script. >> > >Thank you for the review. Agree, these scripts are guide on how to write >eppic >scripts. On the other hand, if the user tries to run these scripts on >the latest kernel + >supported makedumpfile for that release, they would fail. These scripts >are outdated, >when run on latest kernel due to data structure changes. I can take up the >responsibility of maintaining eppic scripts against kernel changes for >every makedumpfile >release. If you will maintain them after this, I have no reason to reject it. I don't think all of the kernel version should be covered by the samples, but it's better to keep existing scripts because users don't always use the latest kernel. For that reason, I want to clarify the correspondence between script file and kernel version at least for existing files like: vhost_net_buffers_2_6_18_to_2_6_32.c vhost_net_buffers_2_6_33_to_3_10.c vhost_net_buffers_3_11_to_4_8.c ... // just sample, not fact Of course writing the correspondence table into README is OK, any ideas are welcome. After this clean up, let's follow up newer kernel versions. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 1/2] eppic: vhost_net_buffers - Adopt to struct sk_buff changes
Hello Kamalesh,
Thanks for taking care of the scripts.
The current scripts are created based on the same kernel version:
===
Eppic scripts README
==
The eppic scripts are based on the fedora 19 kernel.
So if you update a part of the scripts, the target kernel version of
them will be complex and confusing.
I don't think we need to update these scripts since they are just sample,
the purpose is to tell users how to write a script.
Thanks,
Atsushi Kumagai
>Linux kernel commit 56b174256b69 ("net: add rbnode to struct sk_buff"),
>moves sk_buff->next into an union of sk_buff->{next/prev/tstamp/rb_node}.
>Introduce this structure member change, while traversing the socket
>buffer list.
>
>Signed-off-by: Kamalesh Babulal <kamal...@linux.vnet.ibm.com>
>---
> eppic_scripts/vhost_net_buffers.c | 11 ---
> 1 file changed, 8 insertions(+), 3 deletions(-)
>
>diff --git a/eppic_scripts/vhost_net_buffers.c
>b/eppic_scripts/vhost_net_buffers.c
>index 39ae595..1260acb 100644
>--- a/eppic_scripts/vhost_net_buffers.c
>+++ b/eppic_scripts/vhost_net_buffers.c
>@@ -45,7 +45,10 @@ vhost_net(struct vhost_net *net)
> memset((char *)&(buff->data_len), 'L', 0x4);
> }
>
>- next = buff->next;
>+ /*
>+ * .next is the first entry.
>+ */
>+ next = (struct sk_buff *)(unsigned long)*buff;
> }
>
> head = (struct sk_buff_head *)&(sk->sk_write_queue);
>@@ -60,8 +63,10 @@ vhost_net(struct vhost_net *net)
> memset((char *)&(buff->data_len), 'L', 0x4);
> }
>
>- next = buff->next;
>-
>+ /*
>+ * .next is the first entry.
>+ */
>+ next = (struct sk_buff *)(unsigned long)*buff;
> }
> }
> }
>--
>2.7.4
>
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH V2 2/4] x86_64: translate all VA to PA using page table values
>On 12/12/16 at 08:40am, Atsushi Kumagai wrote:
>> >On Saturday 10 December 2016 07:03 AM, b...@redhat.com wrote:
>> >> On 12/10/16 at 09:29am, Baoquan He wrote:
>> >>> On 12/09/16 at 10:25pm, Baoquan He wrote:
>> >>>> On 12/09/16 at 03:40pm, Pratyush Anand wrote:
>> >>>>>>> -page_dir = SYMBOL(init_level4_pgt);
>> >>>>>>> +page_dir = SYMBOL(init_level4_pgt) - __START_KERNEL_map +
>> >>>>>>> phys_base;
>> >>>>>>
>> >>>>>> I found that this change breaks the backward compatibility for
>> >>>>>> kernel 2.6.21 or older since phys_base was introduced in kernel 2.6.22
>> >>>>>> by the commit below:
>> >>>>>>
>> >>>>>> commit 1ab60e0f72f71ec54831e525a3e1154f1c092408
>> >>>>>> Author: Vivek Goyal <vgo...@in.ibm.com>
>> >>>>>> Date: Wed May 2 19:27:07 2007 +0200
>> >>>>>>
>> >>>>>> [PATCH] x86-64: Relocatable Kernel Support
>> >>>>>>
>> >>>>>> There is no problem if phys_base is always 0 in older kernel, but
>> >>>>>> get_phys_base_x86_64() calculates "phys_base = 0x10" from my
>> >>>>>> vmcore:
>> >>>>
>> >>>> This is really awkward. Checked code, found PAGE_OFFSET is
>> >>>> 0x8100 before 2.6.26, then changed to 0x8800
>> >>>> after that. Can we check the page_offset calculated from pt_load
>> >>>> segments, meanwhile check if has VMCOREINFO and osrelease after 2.6.21.
>> >>>>
>> >>>> With both of above condition, we could set phys_vase to 0. Not sure if
>> >>>> this can solve the existing problem.
>> >>>
>> >>> I meant making a judgement:
>> >>>
>> >>
>> >> Sorry, should be:
>> >> if (page_offset == 0x8100 && !info->kernel_version >
>> >> KERNEL_VERSION(2, 6, 21))
>> >> info->phys_base = 0;
>> >>
>> >
>> >
>> >But you can not read kernel_version because those version does not have
>> >VMCOREINFO. So, has_vmcoreinfo() still need to be used.
>>
>> Thanks for your comments, using has_vmcoreinfo() sounds like a good approach,
>> but not perfect way. VMCOREINFO has been introduced since 2.6.24,
>> 2.6.22 and 2.6.23 don't have VMCOREINFO but have phys_base.
>>
>> Conversely, 2.6.22 and 2.6.23 require vmlinux, so we can confirm the
>> existence of
>> phys_base with that. My idea is:
>>
>> diff --git a/arch/x86_64.c b/arch/x86_64.c
>> index 010ea10..893cd51 100644
>> --- a/arch/x86_64.c
>> +++ b/arch/x86_64.c
>> @@ -67,6 +67,14 @@ get_phys_base_x86_64(void)
>> return TRUE;
>> }
>>
>> + /* linux-2.6.21 or older don't have phys_base, should be set to 0. */
>> + if (!has_vmcoreinfo()) {
>> + SYMBOL_INIT(phys_base, "phys_base");
>> + if (SYMBOL(phys_base) == NOT_FOUND_SYMBOL) {
>> + return TRUE;
>> + }
>> + }
>> +
>> for (i = 0; get_pt_load(i, _start, NULL, _start, NULL);
>> i++) {
>> if (virt_start >= __START_KERNEL_map) {
>>
>>
>> This fix may resolve my issue, but now I have another question that
>> "Is the logic of get_phys_base_x86_64() correct in any kernel configuration
>> ?"
>> I mean I'm worried about the possibility that another offset gets mixed with
>> the result of get_phys_base_x86_64() like my 2.6.21 case.
>> After phys_base was introduced, does it always equal to the offset which can
>> be
>> calculated from PT_LOAD headers ?
>
>Don't worry. phys_base was introduced just after 2.6.21.
>
>commit 1ab60e0f72f71ec54831e525a3e1154f1c092408
>Author: Vivek Goyal <vgo...@in.ibm.com>
>Date: Wed May 2 19:27:07 2007 +0200
>
>[PATCH] x86-64: Relocatable Kernel Support
>
>[bhe@x1 linux]$ git describe 1ab60e0f72f71ec54831e525a3e1154f1c092408
>v2.6.21-1836-g1ab60e0
All right, thanks.
I'll release v1.6.1 soon if there is nothing wrong with the retest.
Regards,
Atsushi Kumagai
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH V2 2/4] x86_64: translate all VA to PA using page table values
Hello Pratyush,
>---
> arch/x86_64.c | 42 --
> makedumpfile.h | 4 ++--
> 2 files changed, 10 insertions(+), 36 deletions(-)
>
>diff --git a/arch/x86_64.c b/arch/x86_64.c
>index eba725e41aac..9afa38fd141a 100644
>--- a/arch/x86_64.c
>+++ b/arch/x86_64.c
>@@ -203,6 +203,12 @@ vtop4_x86_64(unsigned long vaddr)
> {
> unsigned long page_dir, pml4, pgd_paddr, pgd_pte, pmd_paddr, pmd_pte;
> unsigned long pte_paddr, pte;
>+ unsigned long phys_base;
>+
>+ if (SYMBOL(phys_base) != NOT_FOUND_SYMBOL)
>+ phys_base = info->phys_base;
>+ else
>+ phys_base = 0;
>
> if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL) {
> ERRMSG("Can't get the symbol of init_level4_pgt.\n");
>@@ -212,9 +218,9 @@ vtop4_x86_64(unsigned long vaddr)
> /*
>* Get PGD.
>*/
>- page_dir = SYMBOL(init_level4_pgt);
>+ page_dir = SYMBOL(init_level4_pgt) - __START_KERNEL_map + phys_base;
I found that this change breaks the backward compatibility for
kernel 2.6.21 or older since phys_base was introduced in kernel 2.6.22
by the commit below:
commit 1ab60e0f72f71ec54831e525a3e1154f1c092408
Author: Vivek Goyal <vgo...@in.ibm.com>
Date: Wed May 2 19:27:07 2007 +0200
[PATCH] x86-64: Relocatable Kernel Support
There is no problem if phys_base is always 0 in older kernel, but
get_phys_base_x86_64() calculates "phys_base = 0x10" from my vmcore:
Type Offset VirtAddr PhysAddr
FileSizMemSiz Flags Align
NOTE 0x0190 0x 0x
0x0590 0x0590 0
LOAD 0x0720 0x8000 0x0010//
CONFIG_PHYSICAL_START = 0x10
0x008b2000 0x008b2000 RWE0
LOAD 0x008b2720 0x8100 0x
0x000a 0x000a RWE0
LOAD 0x00952720 0x8110 0x0010
0x00f0 0x00f0 RWE0
LOAD 0x01852720 0x81000500 0x0500
0xcaf7 0xcaf7 RWE0
LOAD 0xcc7c2720 0x8101 0x0001
0x7000 0x7000 RWE0
Of course we shouldn't use that invalid phys_base:
crash> sym init_level4_pgt
80101000 (T) init_level4_pgt
crash> vtop 80101000
VIRTUAL PHYSICAL
80101000 101000 // just "VIRTUAL - __START_KERNEL_map"
PML4 DIRECTORY: 80101000
PAGE DIRECTORY: 103027
PUD: 103ff0 => 105027
PMD: 105000 => 1e3
PAGE: 0 (2MB)
PTE PHYSICAL FLAGS
1e3 0 (PRESENT|RW|ACCESSED|DIRTY|PSE|GLOBAL)
PAGEPHYSICAL MAPPING INDEX CNT FLAGS
81000500483810100000 1 400
crash>
At first I thought about setting 0 to phys_base if the kernel is
older than 2.6.22, but unfortunately we can't get the kernel version
before getting correct phys_base since VtoP is necessary to read
system_utsname.
(and 2.6.21 doesn't have VMCOREINFO, OSRELEASE can't be used too.)
Do you have any ideas for this issue ?
Thanks,
Atsushi Kumagai
> page_dir += pml4_index(vaddr) * sizeof(unsigned long);
>- if (!readmem(VADDR, page_dir, , sizeof pml4)) {
>+ if (!readmem(PADDR, page_dir, , sizeof pml4)) {
> ERRMSG("Can't get pml4 (page_dir:%lx).\n", page_dir);
> return NOT_PADDR;
> }
>@@ -285,38 +291,6 @@ vtop4_x86_64(unsigned long vaddr)
> return (pte & ENTRY_MASK) + PAGEOFFSET(vaddr);
> }
>
>-unsigned long long
>-vaddr_to_paddr_x86_64(unsigned long vaddr)
>-{
>- unsigned long phys_base;
>- unsigned long long paddr;
>-
>- /*
>- * Check the relocatable kernel.
>- */
>- if (SYMBOL(phys_base) != NOT_FOUND_SYMBOL)
>- phys_base = info->phys_base;
>- else
>- phys_base = 0;
>-
>- if (is_vmalloc_addr_x86_64(vaddr)) {
>- if ((paddr = vtop4_x86_64(vaddr)) == NOT_PADDR) {
>- ERRMSG("Can't convert a virtual address(%lx) to " \
>- "physical address.\n", vaddr);
>- return NOT_PADDR;
>- }
>- } else if (vaddr >= __START_KERNEL_map) {
>- paddr = vaddr - __START_KERNEL_map + phys_base;
>-
>- } else {
>- if (is_xen_memory())
>- paddr =
RE: [PATCH Makedumpfile 00/10] arm64 cleanup and kaslr support
Hello Pratyush, >On Tue, Nov 15, 2016 at 12:04 PM, Pratyush Anand <pan...@redhat.com> wrote: >> Hi Atsushi, >> >> There would be a conflict because of following patch while applying >> these patches. Other than that I also see a an issue with --config >> option. So I will fix that as well and repost the series soon. > >Sorry, for the noise.That was a bad compilation of the code. > >Anyway, I resolved the conflict because of below patch in upstream >devel branch and r5pushed the patches in my git tree: >https://github.com/pratyushanand/makedumpfile.git : arm64_devel > >There is no changes other than conflict resolution w.r.t. this patch series. > >~Pratyush > >> >> 0068010b9b83 [PATCH v2 2/2] Clean up unused KERNEL_IMAGE_SIZE I'm sorry ! I misunderstood that I already pushed this series in the devel branch. Should I wait for v2 series where the conflict is resolved ? Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH v2 1/2] makedumpfile: Adapt code to get value of phys_base
>Kernel code only exports virtual address of phys_base now and it's helpless
>for Crash and Makedumpfile. Below patch which changes code to export value
>of phys_base has been posted to upstream. So adapt code to get it.
>
>kexec: Change to export the value of phys_base instead of symbol address
>marc.info/?l=linux-kernel=147856863629624=2
>
>Signed-off-by: Baoquan He <b...@redhat.com>
>---
>v1->v2:
>Patch v1 is not compatible with the old kernel. Change code in
>get_phys_base_x86_64 and vtop4_x86_64 to avoid that according to
>Atsushi's suggestion.
Looks good to me, I'll merge this patch into v1.6.1.
For 2/2, I'll revert 56649f7b6bfe7 with your patch's comment.
Thanks,
Atsushi Kumagai
> arch/x86_64.c | 12 +---
> makedumpfile.c | 5 ++---
> makedumpfile.h | 2 +-
> 3 files changed, 8 insertions(+), 11 deletions(-)
>
>diff --git a/arch/x86_64.c b/arch/x86_64.c
>index 3ef33ae..010ea10 100644
>--- a/arch/x86_64.c
>+++ b/arch/x86_64.c
>@@ -62,6 +62,10 @@ get_phys_base_x86_64(void)
>* Get the relocatable offset
>*/
> info->phys_base = 0; /* default/traditional */
>+ if (NUMBER(phys_base) != NOT_FOUND_NUMBER) {
>+ info->phys_base = NUMBER(phys_base);
>+ return TRUE;
>+ }
>
> for (i = 0; get_pt_load(i, _start, NULL, _start, NULL); i++) {
> if (virt_start >= __START_KERNEL_map) {
>@@ -187,12 +191,6 @@ vtop4_x86_64(unsigned long vaddr)
> {
> unsigned long page_dir, pml4, pgd_paddr, pgd_pte, pmd_paddr, pmd_pte;
> unsigned long pte_paddr, pte;
>- unsigned long phys_base;
>-
>- if (SYMBOL(phys_base) != NOT_FOUND_SYMBOL)
>- phys_base = info->phys_base;
>- else
>- phys_base = 0;
>
> if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL) {
> ERRMSG("Can't get the symbol of init_level4_pgt.\n");
>@@ -202,7 +200,7 @@ vtop4_x86_64(unsigned long vaddr)
> /*
>* Get PGD.
>*/
>- page_dir = SYMBOL(init_level4_pgt) - __START_KERNEL_map + phys_base;
>+ page_dir = SYMBOL(init_level4_pgt) - __START_KERNEL_map +
>info->phys_base;
> page_dir += pml4_index(vaddr) * sizeof(unsigned long);
> if (!readmem(PADDR, page_dir, , sizeof pml4)) {
> ERRMSG("Can't get pml4 (page_dir:%lx).\n", page_dir);
>diff --git a/makedumpfile.c b/makedumpfile.c
>index b916dfb..a3f711e 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -1507,7 +1507,6 @@ get_symbol_info(void)
> SYMBOL_INIT(init_level4_pgt, "init_level4_pgt");
> SYMBOL_INIT(vmlist, "vmlist");
> SYMBOL_INIT(vmap_area_list, "vmap_area_list");
>- SYMBOL_INIT(phys_base, "phys_base");
> SYMBOL_INIT(node_online_map, "node_online_map");
> SYMBOL_INIT(node_states, "node_states");
> SYMBOL_INIT(node_memblk, "node_memblk");
>@@ -2134,7 +2133,6 @@ write_vmcoreinfo_data(void)
> WRITE_SYMBOL("init_level4_pgt", init_level4_pgt);
> WRITE_SYMBOL("vmlist", vmlist);
> WRITE_SYMBOL("vmap_area_list", vmap_area_list);
>- WRITE_SYMBOL("phys_base", phys_base);
> WRITE_SYMBOL("node_online_map", node_online_map);
> WRITE_SYMBOL("node_states", node_states);
> WRITE_SYMBOL("node_data", node_data);
>@@ -2261,6 +2259,7 @@ write_vmcoreinfo_data(void)
>
> WRITE_NUMBER("PAGE_BUDDY_MAPCOUNT_VALUE", PAGE_BUDDY_MAPCOUNT_VALUE);
> WRITE_NUMBER("KERNEL_IMAGE_SIZE", KERNEL_IMAGE_SIZE);
>+ WRITE_NUMBER("phys_base", phys_base);
>
> WRITE_NUMBER("HUGETLB_PAGE_DTOR", HUGETLB_PAGE_DTOR);
>
>@@ -2488,7 +2487,6 @@ read_vmcoreinfo(void)
> READ_SYMBOL("init_level4_pgt", init_level4_pgt);
> READ_SYMBOL("vmlist", vmlist);
> READ_SYMBOL("vmap_area_list", vmap_area_list);
>- READ_SYMBOL("phys_base", phys_base);
> READ_SYMBOL("node_online_map", node_online_map);
> READ_SYMBOL("node_states", node_states);
> READ_SYMBOL("node_data", node_data);
>@@ -2609,6 +2607,7 @@ read_vmcoreinfo(void)
>
> READ_NUMBER("PAGE_BUDDY_MAPCOUNT_VALUE", PAGE_BUDDY_MAPCOUNT_VALUE);
> READ_NUMBER("KERNEL_IMAGE_SIZE", KERNEL_IMAGE_SIZE);
>+ READ_NUMBER("phys_base", phys_base);
>
> READ_NUMBER("HUGETLB_PAGE_DTOR", HUGETLB_PAGE_DTOR);
>
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 338c651..4
RE: [PATCH 2/2] makedumpfile: Clean up unused KERNEL_IMAGE_SIZE
>On 11/10/16 at 03:58am, Atsushi Kumagai wrote: >> >> >> > The old MODULES_VADDR need be decided by KERNEL_IMAGE_SIZE when kaslr >> >> >> > enabled. Now MODULES_VADDR is not needed any more since Pratyush >> >> >> > makes >> >> >> > all VA to PA converting done by page table lookup. So remove its >> >> >> > related >> >> >> > code. >> >> >> >> >> >> Hi Bao, >> >> >> >> >> >> I'm not clear on this. The crash utility still reads/needs >> >> >> KERNEL_IMAGE_SIZE >> >> >> from the dumpfile's vmcoreinfo data. Is it being written to the >> >> >> vmcoreinfo >> >> >> section somewhere else in the code? >> >> > >> >> >Aha, seems makedumpfile will get the vmcoreinfo offset and size, then >> >> >write the whole vmcoreinfo block into dumped vmcore header. But if >> >> >specify '-g' for makedumpfile to only generate a vmcoreinfo file, it >> >> >won't contain KERNEL_IMAGE_SIZE. >> >> > >> >> >makedumpfile -g vmcoreinfo -x vmlinux >> >> > >> >> >So I am not sure if you care about vmcoreinfo file, but you are right, I >> >> >should not remove the vmcoreinfo reading and writing data. >> >> > >> >> >Thanks for pointing it out, will send v2 post. >> >> >> >> I understand the vmcoreinfo file generated by '-g' is used only for >> >> makedumpfile, so it's OK if it doesn't contain KERNEL_IMAGE_SIZE since >> >> crash doesn't refer to the file. >> >> This patch doesn't modify the original vmcoreinfo section in dumpfile, >> >> it sounds reasonable to me. >> > >> >Thanks for your comments, Atsushi! Then I am fine. >> >> Additionally, I think it would be better to remove all of the code >> you added in commit 56649f7b6bfe7. > >Check it again, seems patch 2/2 equals to reverting commit >56649f7b6bfe7. Did I miss anything? You are right, it will be just a revert commit. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 2/2] makedumpfile: Clean up unused KERNEL_IMAGE_SIZE
>> >> > The old MODULES_VADDR need be decided by KERNEL_IMAGE_SIZE when kaslr >> >> > enabled. Now MODULES_VADDR is not needed any more since Pratyush makes >> >> > all VA to PA converting done by page table lookup. So remove its related >> >> > code. >> >> >> >> Hi Bao, >> >> >> >> I'm not clear on this. The crash utility still reads/needs >> >> KERNEL_IMAGE_SIZE >> >> from the dumpfile's vmcoreinfo data. Is it being written to the >> >> vmcoreinfo >> >> section somewhere else in the code? >> > >> >Aha, seems makedumpfile will get the vmcoreinfo offset and size, then >> >write the whole vmcoreinfo block into dumped vmcore header. But if >> >specify '-g' for makedumpfile to only generate a vmcoreinfo file, it >> >won't contain KERNEL_IMAGE_SIZE. >> > >> >makedumpfile -g vmcoreinfo -x vmlinux >> > >> >So I am not sure if you care about vmcoreinfo file, but you are right, I >> >should not remove the vmcoreinfo reading and writing data. >> > >> >Thanks for pointing it out, will send v2 post. >> >> I understand the vmcoreinfo file generated by '-g' is used only for >> makedumpfile, so it's OK if it doesn't contain KERNEL_IMAGE_SIZE since >> crash doesn't refer to the file. >> This patch doesn't modify the original vmcoreinfo section in dumpfile, >> it sounds reasonable to me. > >Thanks for your comments, Atsushi! Then I am fine. Additionally, I think it would be better to remove all of the code you added in commit 56649f7b6bfe7. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 2/2] makedumpfile: Clean up unused KERNEL_IMAGE_SIZE
Hello Baoquan, >> > The old MODULES_VADDR need be decided by KERNEL_IMAGE_SIZE when kaslr >> > enabled. Now MODULES_VADDR is not needed any more since Pratyush makes >> > all VA to PA converting done by page table lookup. So remove its related >> > code. >> >> Hi Bao, >> >> I'm not clear on this. The crash utility still reads/needs KERNEL_IMAGE_SIZE >> from the dumpfile's vmcoreinfo data. Is it being written to the vmcoreinfo >> section somewhere else in the code? > >Aha, seems makedumpfile will get the vmcoreinfo offset and size, then >write the whole vmcoreinfo block into dumped vmcore header. But if >specify '-g' for makedumpfile to only generate a vmcoreinfo file, it >won't contain KERNEL_IMAGE_SIZE. > >makedumpfile -g vmcoreinfo -x vmlinux > >So I am not sure if you care about vmcoreinfo file, but you are right, I >should not remove the vmcoreinfo reading and writing data. > >Thanks for pointing it out, will send v2 post. I understand the vmcoreinfo file generated by '-g' is used only for makedumpfile, so it's OK if it doesn't contain KERNEL_IMAGE_SIZE since crash doesn't refer to the file. This patch doesn't modify the original vmcoreinfo section in dumpfile, it sounds reasonable to me. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [Makedumpfile PATCH V2 0/4] x86_64: Fix page_offset for randomized base enabled
Hello Pratyush, >Patch 1/4 fixes page_offset calculation, so that it is correctly calculated >on KASLR enabled kernel as well. >Patch 2/4 simplifies VA to PA translation. New code has been benchmarked >against old code on a 4T system. >Patch 3/4 and 4/4 is removal of (now) unnecessary code. > >I think, we should find a way to kill find_vememmap() as well, so that >VMEMMAP_START can be removed. I have very limited idea about x86, so unable >to do that as of now. > >Changes since V1: >* A bug in patch 1/4 fixed >* Patch log of 2/4 improved to include more number of trials The most of concern was the performance degradation, but there is no degradation in the additional benchmark. So I decide to merge this patch set into v1.6.1, thanks for your work. Regards, Atsushi Kumagai >Pratyush Anand (4): > x86_64: Calculate page_offset from pt_load > x86_64: translate all VA to PA using page table values > x86_64: kill is_vmalloc_addr_x86_64() > x86_64: kill some unused initialization > > arch/x86_64.c | 84 -- > makedumpfile.h | 9 +++ > 2 files changed, 32 insertions(+), 61 deletions(-) > >-- >2.7.4 > > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH Makedumpfile 1/4] x86_64: Calculate page_offset from pt_load
Hello,
>On 11/02/16 at 07:40am, Atsushi Kumagai wrote:
>> Hello Pratyush,
>>
>> >On Monday 24 October 2016 10:18 PM, Pratyush Anand wrote:
>> >> page_offset can always be calculated as 'virtual - physical' for a direct
>> >> mapping area on x86. Therefore, remove the version dependent calculation
>> >> and use this method.
>> >>
>> >> Signed-off-by: Pratyush Anand <pan...@redhat.com>
>> >> ---
>> >> arch/x86_64.c | 24
>> >> 1 file changed, 20 insertions(+), 4 deletions(-)
>> >>
>> >> diff --git a/arch/x86_64.c b/arch/x86_64.c
>> >> index ddf7be6bc57b..a96fd8ae00a1 100644
>> >> --- a/arch/x86_64.c
>> >> +++ b/arch/x86_64.c
>> >> @@ -44,6 +44,24 @@ get_xen_p2m_mfn(void)
>> >> return NOT_FOUND_LONG_VALUE;
>> >> }
>> >>
>> >> +static int
>> >> +get_page_offset_x86_64(void)
>> >> +{
>> >> + int i;
>> >> + unsigned long long phys_start;
>> >> + unsigned long long virt_start;
>> >> +
>> >> + for (i = 0; get_pt_load(i, _start, NULL, _start, NULL); i++) {
>> >> + if (virt_start >= __START_KERNEL_map) {
>> >
>> >OK..So, this is the problem. We should have
>> >
>> >if (virt_start < __START_KERNEL_map) {
>> >
>> >Kernel text region lies above __START_KERNEL_map, which is linearly
>> >mapped however not a direct mapping. Direct mapping region lies below it
>> >instead. So, page_offset can only be calculated with a region which is
>> >below __START_KERNEL_map.
>> >
>> >Thanks Baoquan for finding it.
>>
>> Could you explain the issue in more detail, what is the difference
>> between "linear map" and "direct map" ?
>
>Well, these could be easily misunderstood. Virtual address in kernel text
>region or within direct mapping which is from PAGE_OFFSET are all linear
>mapping. Because the related PA can be calculated by:
>PA= VA - PAGE_OFFSET; //y=x - a;
>PA= VA - __START_KERNEL_map + phys_base; //y=x-a+b
>
>In user-space kexec-tools we created pt_load program segments for kernel
>text region and crash memory regions. The first program header is kernel
>text region. So here if we want to get page_offset, just need to ignore
>the 1st one, kernel text region.
Thanks, I understand.
Now I have no concerns.
Regards,
Atsushi Kumagai
>> BTW, I confirmed that the difference of v2 patch is only this and
>> the result of benchmark looks great, thank you very much.
>>
>>
>> Regards,
>> Atsushi Kumagai
>>
>> >> + info->page_offset = virt_start - phys_start;
>> >> + return TRUE;
>> >> + }
>> >> + }
>> >> +
>> >> + ERRMSG("Can't get any pt_load to calculate page offset.\n");
>> >> + return FALSE;
>> >> +}
>> >> +
>> >> int
>> >> get_phys_base_x86_64(void)
>> >> {
>> >> @@ -159,10 +177,8 @@ get_versiondep_info_x86_64(void)
>> >> else
>> >> info->max_physmem_bits = _MAX_PHYSMEM_BITS_2_6_31;
>> >>
>> >> - if (info->kernel_version < KERNEL_VERSION(2, 6, 27))
>> >> - info->page_offset = __PAGE_OFFSET_ORIG;
>> >> - else
>> >> - info->page_offset = __PAGE_OFFSET_2_6_27;
>> >> + if (!get_page_offset_x86_64())
>> >> + return FALSE;
>> >>
>> >> if (info->kernel_version < KERNEL_VERSION(2, 6, 31)) {
>> >> info->vmalloc_start = VMALLOC_START_ORIG;
>> >>
>> >
>> >___
>> >kexec mailing list
>> >kexec@lists.infradead.org
>> >http://lists.infradead.org/mailman/listinfo/kexec
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH Makedumpfile 1/4] x86_64: Calculate page_offset from pt_load
Hello Pratyush,
>On Monday 24 October 2016 10:18 PM, Pratyush Anand wrote:
>> page_offset can always be calculated as 'virtual - physical' for a direct
>> mapping area on x86. Therefore, remove the version dependent calculation
>> and use this method.
>>
>> Signed-off-by: Pratyush Anand <pan...@redhat.com>
>> ---
>> arch/x86_64.c | 24
>> 1 file changed, 20 insertions(+), 4 deletions(-)
>>
>> diff --git a/arch/x86_64.c b/arch/x86_64.c
>> index ddf7be6bc57b..a96fd8ae00a1 100644
>> --- a/arch/x86_64.c
>> +++ b/arch/x86_64.c
>> @@ -44,6 +44,24 @@ get_xen_p2m_mfn(void)
>> return NOT_FOUND_LONG_VALUE;
>> }
>>
>> +static int
>> +get_page_offset_x86_64(void)
>> +{
>> +int i;
>> +unsigned long long phys_start;
>> +unsigned long long virt_start;
>> +
>> +for (i = 0; get_pt_load(i, _start, NULL, _start, NULL); i++) {
>> +if (virt_start >= __START_KERNEL_map) {
>
>OK..So, this is the problem. We should have
>
>if (virt_start < __START_KERNEL_map) {
>
>Kernel text region lies above __START_KERNEL_map, which is linearly
>mapped however not a direct mapping. Direct mapping region lies below it
>instead. So, page_offset can only be calculated with a region which is
>below __START_KERNEL_map.
>
>Thanks Baoquan for finding it.
Could you explain the issue in more detail, what is the difference
between "linear map" and "direct map" ?
BTW, I confirmed that the difference of v2 patch is only this and
the result of benchmark looks great, thank you very much.
Regards,
Atsushi Kumagai
>> +info->page_offset = virt_start - phys_start;
>> +return TRUE;
>> +}
>> +}
>> +
>> +ERRMSG("Can't get any pt_load to calculate page offset.\n");
>> +return FALSE;
>> +}
>> +
>> int
>> get_phys_base_x86_64(void)
>> {
>> @@ -159,10 +177,8 @@ get_versiondep_info_x86_64(void)
>> else
>> info->max_physmem_bits = _MAX_PHYSMEM_BITS_2_6_31;
>>
>> -if (info->kernel_version < KERNEL_VERSION(2, 6, 27))
>> -info->page_offset = __PAGE_OFFSET_ORIG;
>> -else
>> -info->page_offset = __PAGE_OFFSET_2_6_27;
>> +if (!get_page_offset_x86_64())
>> +return FALSE;
>>
>> if (info->kernel_version < KERNEL_VERSION(2, 6, 31)) {
>> info->vmalloc_start = VMALLOC_START_ORIG;
>>
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH Makedumpfile 00/10] arm64 cleanup and kaslr support
Hello Pratyush, >These patches were lying in my tree for quite some time now. VMCOREINFO >numbers/symbols used in these patches have been made part of kdump kernel >patches for last few versions including v26. So, there seems no contention >on embedding VA_BITS, kimage_voffset and PHYS_OFFSET into vmcore. > >These patches cleans up a lot arm64 code and also immunize it with many >kernel changes. Additionally,it adds 4 level 4K page support and support for >KASLR enabled kernel. > >Azriel Samson (1): > arm64: Add support for 4level 4K page translations table > >Pratyush Anand (9): > arm64: cleanup code, comment, blank space, blank lines etc > read_vmcoreinfo_long: Allow to read hex values as well > Introduce read_vmcoreinfo_ulong() > arm64: use already available PAGESIZE() and PAGESHIFT() macros > arm64: fix page_offset definition > arm64: fix re-filtering > arm64: use value of VA_BITS and PHYS_OFFSET embedded into vmcore > arm64: immunize identity mapped address finding w.r.t. kernel changes > arm64: fix memory layout as per changes in v4.6 kernel > > arch/arm64.c | 245 ++--- > makedumpfile.c | 46 +++ > makedumpfile.h | 28 +-- > 3 files changed, 162 insertions(+), 157 deletions(-) Thanks always for your work for arm64, I've reviewed this patch set. I'll merge them into v1.6.1. Regards, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH] Fix module module_init/init_size offsets for v4.5 kernel
>On Friday 28 October 2016 12:06 PM, Atsushi Kumagai wrote: >>> >On Friday 28 October 2016 10:37 AM, Kamalesh Babulal wrote: >>>> >>Kernel commit 7523e4dc50 (module: use a structure to encapsulate layout.) >>>> >>encapsulates core_layout and init_layout into module_layout structure. >>>> >> >>>> >>commit fa6a75a93 (Fix module core base and size offset for kernel v4.5) >>>> >>fixes offset value calculation for core layout's base and size, whereas >>>> >>Kernel v4.5 module changes also needs fixing of module init_size and >>>> >>module_init for the makedumpfile to read the correct module address. >>>> >> >>>> >>This patch fixes calculation of offsets values for module init_size and >>>> >>module_init. >>>> >> >>>> >>Signed-off-by: Kamalesh Babulal<kamal...@linux.vnet.ibm.com> >>>> >>Cc: Pratyush Anand<pan...@redhat.com> >>> > >>> >Reviewed-by: Pratyush Anand<pan...@redhat.com> >> Thanks, but I think Guenther posted the same fix which you can see >> in the devel branch: >> >> commit 32dd46803944959f78e01e7c4847c465efca99a6 >> Author: Guenther Hutzl<hu...@linux.vnet.ibm.com> >> Date: Wed Jul 6 20:00:54 2016 +0900 >> >> [PATCH] Fix module init base and size offset for kernel v4.5 > >Thank you for pointing out the commit. Should I rebase future patch >against devel branch. Yes, I recommend that. It's better also for me to review. Thanks, Atsushi Kumagai ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH] Fix module module_init/init_size offsets for v4.5 kernel
Hello Kamalesh and Pratyush,
>On Friday 28 October 2016 10:37 AM, Kamalesh Babulal wrote:
>> Kernel commit 7523e4dc50 (module: use a structure to encapsulate layout.)
>> encapsulates core_layout and init_layout into module_layout structure.
>>
>> commit fa6a75a93 (Fix module core base and size offset for kernel v4.5)
>> fixes offset value calculation for core layout's base and size, whereas
>> Kernel v4.5 module changes also needs fixing of module init_size and
>> module_init for the makedumpfile to read the correct module address.
>>
>> This patch fixes calculation of offsets values for module init_size and
>> module_init.
>>
>> Signed-off-by: Kamalesh Babulal <kamal...@linux.vnet.ibm.com>
>> Cc: Pratyush Anand <pan...@redhat.com>
>
>Reviewed-by: Pratyush Anand <pan...@redhat.com>
Thanks, but I think Guenther posted the same fix which you can see
in the devel branch:
commit 32dd46803944959f78e01e7c4847c465efca99a6
Author: Guenther Hutzl <hu...@linux.vnet.ibm.com>
Date: Wed Jul 6 20:00:54 2016 +0900
[PATCH] Fix module init base and size offset for kernel v4.5
This is a follow-up patch on the patch provided in post:
"[PATCH] makedumpfile: fix module core base and size offset for kernel v4.5"
by Pratyush Anand
Thanks,
Atsushi Kumagai
>> ---
>> makedumpfile.c | 18 ++
>> 1 file changed, 18 insertions(+)
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index 853b999..f33a90d 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -1689,7 +1689,25 @@ get_structure_info(void)
>> OFFSET(module.core_size) += core_layout;
>> }
>> OFFSET_INIT(module.module_init, "module", "module_init");
>> +if (OFFSET(module.module_init) == NOT_FOUND_STRUCTURE) {
>> +/* for kernel version 4.5 and above */
>> +long init_layout;
>> +
>> +OFFSET_INIT(module.module_init, "module", "init_layout");
>> +init_layout = OFFSET(module.module_init);
>> +OFFSET_INIT(module.module_init, "module_layout", "base");
>> +OFFSET(module.module_init) += init_layout;
>> +}
>> OFFSET_INIT(module.init_size, "module", "init_size");
>> +if (OFFSET(module.init_size) == NOT_FOUND_STRUCTURE) {
>> +/* for kernel version 4.5 and above */
>> +long init_layout;
>> +
>> +OFFSET_INIT(module.init_size, "module", "init_layout");
>> +init_layout = OFFSET(module.init_size);
>> +OFFSET_INIT(module.init_size, "module_layout", "size");
>> +OFFSET(module.init_size) += init_layout;
>> +}
>>
>> ENUM_NUMBER_INIT(NR_FREE_PAGES, "NR_FREE_PAGES");
>> ENUM_NUMBER_INIT(N_ONLINE, "N_ONLINE");
>>
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH Makedumpfile 2/4] x86_64: translate all VA to PA using page table values
>On 10/25/16 at 08:42pm, Pratyush Anand wrote:
>> > > With -d 1:
>> > > Trial 1: 2768.424565806 S
>> > > Trial 2: 2749.622115455 S
>> > > Trail 3: 2537.770359073 S
>> >
>> > Could you increase the number of trials ?
>>
>> OK, I can do that. Might take some time, as I will have to arrange that high
>> memory machine again.
>>
>> > If the average time is close to the results of Trial 1 (2768s) and 2
>> > (2749s),
>> > the regression rate is 8% and it sounds neither large nor small.
>> > If the average is a level of 2500s like Trial 3, it's ideal.
>> >
>> > > Signed-off-by: Pratyush Anand <pan...@redhat.com>
>> > > ---
>> > > arch/x86_64.c | 42 --
>> > > makedumpfile.h | 4 ++--
>> > > 2 files changed, 10 insertions(+), 36 deletions(-)
>> > >
>> > > diff --git a/arch/x86_64.c b/arch/x86_64.c
>> > > index a96fd8ae00a1..fe2764a8bec2 100644
>> > > --- a/arch/x86_64.c
>> > > +++ b/arch/x86_64.c
>> > > @@ -203,6 +203,12 @@ vtop4_x86_64(unsigned long vaddr)
>> > > {
>> > > unsigned long page_dir, pml4, pgd_paddr, pgd_pte, pmd_paddr, pmd_pte;
>> > > unsigned long pte_paddr, pte;
>> > > +unsigned long phys_base;
>> > > +
>> > > +if (SYMBOL(phys_base) != NOT_FOUND_SYMBOL)
>> > > +phys_base = info->phys_base;
>> > > +else
>> > > +phys_base = 0;
>> > >
>> > > if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL) {
>> > > ERRMSG("Can't get the symbol of init_level4_pgt.\n");
>> > > @@ -212,9 +218,9 @@ vtop4_x86_64(unsigned long vaddr)
>> > > /*
>> > > * Get PGD.
>> > > */
>> > > -page_dir = SYMBOL(init_level4_pgt);
>> > > +page_dir = SYMBOL(init_level4_pgt) - __START_KERNEL_map +
>> > > phys_base;
>> >
>> > I want to confirm that this VA to PA translation is always safe,
>> > otherwise we should do the condition check which was done in
>> > vaddr_to_paddr_x86_64(), isn't it ?
>> >
>>
>> I think this should be safe, however x86 expert can comment better. Baoquan
>> any comment here?
>
>Yes, I think this is safe. Below is the physical to virtual address
>translation function in x86 64. And init_level4_pgt is a global variable
>located in kernel text region.
>
>arch/x86/include/asm/page_64.h
>
>static inline unsigned long __phys_addr_nodebug(unsigned long x)
>{
>unsigned long y = x - __START_KERNEL_map;
>
>/* use the carry flag to determine if x was < __START_KERNEL_map */
>x = y + ((x > y) ? phys_base : (__START_KERNEL_map - PAGE_OFFSET));
>
>return x;
>}
Good, thanks for your comment.
I'll wait for the result of the benchmark.
Regards,
Atsushi Kumagai
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: makedumpfile issues many readpage_elf: Attempt to read non-existent page
Hello, >> I have now completed the kernel bisection between 4.7.8 and 4.8-rc1 and >> identified the kernel modification that triggers the errors cited above : >> >>> commit 021182e52fe01c1f7b126f97fd6ba048dc4234fd >>> Author: Thomas Garnier <thgar...@google.com> >>> Date: Tue Jun 21 17:47:03 2016 -0700 >>> >>> x86/mm: Enable KASLR for physical mapping memory regions >>> >>> Add the physical mapping in the list of randomized memory regions. >>> >>> The physical memory mapping holds most allocations from boot and heap >>> allocators. Knowing the base address and physical memory size, an >>> attacker >>> can deduce the PDE virtual address for the vDSO memory page. This attack >>> was demonstrated at CanSecWest 2016, in the following presentation: >>> >>> "Getting Physical: Extreme Abuse of Intel Based Paged Systems": >>> >https://github.com/n3k/CansecWest2016_Getting_Physical_Extreme_Abuse_of_Intel_Based_Paging_Systems/blob/master/Prese >ntation/CanSec2016_Presentation.pdf >>> >>> (See second part of the presentation). >>> >>> The exploits used against Linux worked successfully against 4.6+ but >>> fail with KASLR memory enabled: >>> >>> >https://github.com/n3k/CansecWest2016_Getting_Physical_Extreme_Abuse_of_Intel_Based_Paging_Systems/tree/master/Demos >/Linux/exploits >>> >>> Similar research was done at Google leading to this patch proposal. >>> >>> Variants exists to overwrite /proc or /sys objects ACLs leading to >>> elevation of privileges. These variants were tested against 4.6+. >>> >>> The page offset used by the compressed kernel retains the static value >>> since it is not yet randomized during this boot stage. >>> >>> Signed-off-by: Thomas Garnier <thgar...@google.com> >>> Signed-off-by: Kees Cook <keesc...@chromium.org> >>> Cc: Alexander Kuleshov <kuleshovm...@gmail.com> >> >> >> The interesting change seems to be : >> >>> -#define __PAGE_OFFSET _AC(0x8800, UL) >>> +#define __PAGE_OFFSET_BASE _AC(0x8800, UL) >>> +#ifdef CONFIG_RANDOMIZE_MEMORY >>> +#define __PAGE_OFFSET page_offset_base >>> +#else >>> +#define __PAGE_OFFSET __PAGE_OFFSET_BASE >>> +#endif /* CONFIG_RANDOMIZE_MEMORY */ >> >> I'll try to see if I can fix that. >> >> Kind regards, >> >> ...Louis >> >> >> >> > >Some more *important* information in this mostly monologue thread : Pratyush >Anand has pushed a patch to the list earlier today that apparently fixes this >issue : > >[PATCH Makedumpfile 1/4] x86_64: Calculate page_offset from pt_load[1] > >HTH, > >Kind regards, Yeah, It appears so. I'm reviewing the patches, please wait for that. I appreciate your investigation for this issue. Thanks, Atsushi Kumagai >...Louis > >[1] https://www.mail-archive.com/kexec@lists.infradead.org/msg16628.html >-- >Louis Bouchard >Software engineer, Cloud & Sustaining eng. >Canonical Ltd >Ubuntu developer Debian Maintainer >GPG : 429D 7A3B DD05 B6F8 AF63 B9C4 8B3D 867C 823E 7A61 ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH Makedumpfile 2/4] x86_64: translate all VA to PA using page table values
Hello Pratysh,
>Currently we translate some of the VA areas using linear mapping while some
>other(which can not be linearly mapped) using page table.
>
>However, we will have entry of a page in the page table irrespective of its
>virtual region. So, we can always look into page table for any VA to PA
>translation. This approach will solve lot of complexity in makedumpfile. It
>will in turn remove dependency over variables like VMALLOC_START,
>MODULES_VADDR etc whose definition keeps changing in newer kernel version.
>
>Moreover, I do not see any side effect of this approach in terms of
>execution timing. I tested with IBM x3950 X6 machine having 4136359 MB of
>memory. These are the results of makedumpfile execution time:
>
>Without this patch:
>===
>With -d 31:
>Trial 1: 237.59526248 S
>Trial 2: 235.236914962 S
>Trail 3: 237.678712045 S
>
>With -d 1:
>Trial 1: 2548.905296877 S
>Trial 2: 2549.759881756 S
>
>With this patch:
>===
>With -d 31:
>Trial 1: 232.713841516 S
>Trial 2: 228.45697177 S
>Trail 3: 232.942262441 S
>
>With -d 1:
>Trial 1: 2768.424565806 S
>Trial 2: 2749.622115455 S
>Trail 3: 2537.770359073 S
Could you increase the number of trials ?
If the average time is close to the results of Trial 1 (2768s) and 2 (2749s),
the regression rate is 8% and it sounds neither large nor small.
If the average is a level of 2500s like Trial 3, it's ideal.
>Signed-off-by: Pratyush Anand <pan...@redhat.com>
>---
> arch/x86_64.c | 42 --
> makedumpfile.h | 4 ++--
> 2 files changed, 10 insertions(+), 36 deletions(-)
>
>diff --git a/arch/x86_64.c b/arch/x86_64.c
>index a96fd8ae00a1..fe2764a8bec2 100644
>--- a/arch/x86_64.c
>+++ b/arch/x86_64.c
>@@ -203,6 +203,12 @@ vtop4_x86_64(unsigned long vaddr)
> {
> unsigned long page_dir, pml4, pgd_paddr, pgd_pte, pmd_paddr, pmd_pte;
> unsigned long pte_paddr, pte;
>+ unsigned long phys_base;
>+
>+ if (SYMBOL(phys_base) != NOT_FOUND_SYMBOL)
>+ phys_base = info->phys_base;
>+ else
>+ phys_base = 0;
>
> if (SYMBOL(init_level4_pgt) == NOT_FOUND_SYMBOL) {
> ERRMSG("Can't get the symbol of init_level4_pgt.\n");
>@@ -212,9 +218,9 @@ vtop4_x86_64(unsigned long vaddr)
> /*
>* Get PGD.
>*/
>- page_dir = SYMBOL(init_level4_pgt);
>+ page_dir = SYMBOL(init_level4_pgt) - __START_KERNEL_map + phys_base;
I want to confirm that this VA to PA translation is always safe,
otherwise we should do the condition check which was done in
vaddr_to_paddr_x86_64(), isn't it ?
Thanks,
Atsushi Kumagai
> page_dir += pml4_index(vaddr) * sizeof(unsigned long);
>- if (!readmem(VADDR, page_dir, , sizeof pml4)) {
>+ if (!readmem(PADDR, page_dir, , sizeof pml4)) {
> ERRMSG("Can't get pml4 (page_dir:%lx).\n", page_dir);
> return NOT_PADDR;
> }
>@@ -285,38 +291,6 @@ vtop4_x86_64(unsigned long vaddr)
> return (pte & ENTRY_MASK) + PAGEOFFSET(vaddr);
> }
>
>-unsigned long long
>-vaddr_to_paddr_x86_64(unsigned long vaddr)
>-{
>- unsigned long phys_base;
>- unsigned long long paddr;
>-
>- /*
>- * Check the relocatable kernel.
>- */
>- if (SYMBOL(phys_base) != NOT_FOUND_SYMBOL)
>- phys_base = info->phys_base;
>- else
>- phys_base = 0;
>-
>- if (is_vmalloc_addr_x86_64(vaddr)) {
>- if ((paddr = vtop4_x86_64(vaddr)) == NOT_PADDR) {
>- ERRMSG("Can't convert a virtual address(%lx) to " \
>- "physical address.\n", vaddr);
>- return NOT_PADDR;
>- }
>- } else if (vaddr >= __START_KERNEL_map) {
>- paddr = vaddr - __START_KERNEL_map + phys_base;
>-
>- } else {
>- if (is_xen_memory())
>- paddr = vaddr - PAGE_OFFSET_XEN_DOM0;
>- else
>- paddr = vaddr - PAGE_OFFSET;
>- }
>- return paddr;
>-}
>-
> /*
> * for Xen extraction
> */
>diff --git a/makedumpfile.h b/makedumpfile.h
>index a5955ff750e5..13559651feb6 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -863,12 +863,12 @@ int is_vmalloc_addr_x86_64(ulong vaddr);
> int get_phys_base_x86_64(void);
> int get_machdep_info_x86_64(void);
> int get_versiondep_info_x86_64(void);
>-unsigned long long vaddr_to_paddr_x86_64(unsigned long vaddr);
>+unsigned long long vtop4_x86_64(unsigned long vaddr);
> #define find_vmemmap()
RE: [RESEND PATCH 0/8] makedumpfile: Add support to understand Power ISA 3.0 based kernel
Hello Hari, >SA 3.0 based kernel > >With the introduction of radix MMU in Power ISA 3.0, there are changes >in kernel page table management accommodating that. This patch series >makes appropriate changes here to work for such kernels. This series >enables address translation support for radix MMU and also, fixes a >few bugs along the way. > >Resending the patches: >* by adding makedumpfile in subject to set this in context >* adding right mail id of Atsushi Kumagai > >--- > >Hari Bathini (8): > ppc64: fix vtop page translation for 4K pages > ppc64: Use kernel terminology for each level in 4-level page table > ppc64: address changes in kernel v4.5 > ppc64: address change in _PAGE_PRESNT flag for PowerISA v3.0 > ppc64: use physical addresses and unfold pud for 64K page size > ppc64: support big endian Linux page tables > ppc64: use the same masked bit values for 4K and 64K pagesizes > ppc64: enable address translation support for radix mmu I've reviewed them and they look good to me. I'll merge them into v1.6.1, thanks for your work. Regards, Atsushi Kumagai > > arch/ppc64.c | 231 > makedumpfile.c | 11 +++ > makedumpfile.h | 107 +- > 3 files changed, 280 insertions(+), 69 deletions(-) ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 3/3] makedumpfile: Add support for MM randomization
>> >/*
>> > * On linux-2.6.26, MAX_PHYSMEM_BITS is changed to 44 from 40.
>> >@@ -159,22 +160,13 @@ get_versiondep_info_x86_64(void)
>> >else
>> >info->max_physmem_bits = _MAX_PHYSMEM_BITS_2_6_31;
>> >
>> >- if (info->kernel_version < KERNEL_VERSION(2, 6, 27))
>> >- info->page_offset = __PAGE_OFFSET_ORIG;
>> >- else
>> >- info->page_offset = __PAGE_OFFSET_2_6_27;
>> >+ info->page_offset = NUMBER(page_offset);
>> >
>> >- if (info->kernel_version < KERNEL_VERSION(2, 6, 31)) {
>> >- info->vmalloc_start = VMALLOC_START_ORIG;
>> >- info->vmalloc_end = VMALLOC_END_ORIG;
>> >- info->vmemmap_start = VMEMMAP_START_ORIG;
>> >- info->vmemmap_end = VMEMMAP_END_ORIG;
>> >- } else {
>> >- info->vmalloc_start = VMALLOC_START_2_6_31;
>> >- info->vmalloc_end = VMALLOC_END_2_6_31;
>> >- info->vmemmap_start = VMEMMAP_START_2_6_31;
>> >- info->vmemmap_end = VMEMMAP_END_2_6_31;
>> >- }
>>
>> These *_END_* are no longer used, it's better to remove the definitions
>> of them.
>
>
>Seems is_vmalloc_addr_x86_64 still needs VMALLOC_END and VMEMMAP_END to
>make a judgement.
Yes, VMALLOC_END and VMEMMAP_END are necessary, but what I mentioned were
VMALLOC_END_ORIG, VMEMMAP_END_ORIG , VMALLOC_END_2_6_31 and VMEMMAP_END_2_6_31.
The symbols were used only to initialize info->vmalloc_end and
info->vmemmap_end,
so they will be unnecessary by this patch.
>> >+
>> >+ info->vmalloc_start = NUMBER(vmalloc_start);
>> >+ info->vmalloc_end = info->vmalloc_start + VMALLOC_SIZE - 1;
>> >+ info->vmemmap_start = NUMBER(vmemmap_start);
>> >+ info->vmemmap_end = info->vmemmap_start + VMEMMAP_SIZE - 1;
Thanks,
Atsushi Kumagai
>> >return TRUE;
>> > }
>> >diff --git a/makedumpfile.c b/makedumpfile.c
>> >index 2713f8a..6b0c6ab 100644
>> >--- a/makedumpfile.c
>> >+++ b/makedumpfile.c
>> >@@ -1985,6 +1985,7 @@ get_value_for_old_linux(void)
>> >NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE) =
>> >PAGE_BUDDY_MAPCOUNT_VALUE_v2_6_39_to_latest_version;
>> >}
>> >+ ERRMSG("info->kernel_version=%d\n", info->kernel_version);
>>
>> Is this just a debug message ?
>
>Yes, sorry for this. Will remove it.
>
>>
>> > #ifdef __x86_64__
>> >if (NUMBER(KERNEL_IMAGE_SIZE) == NOT_FOUND_NUMBER) {
>> >if (info->kernel_version < KERNEL_VERSION(2, 6, 26))
>> >@@ -1992,6 +1993,26 @@ get_value_for_old_linux(void)
>> >else
>> >NUMBER(KERNEL_IMAGE_SIZE) = KERNEL_IMAGE_SIZE_2_6_26;
>> >}
>> >+ if (NUMBER(page_offset) == NOT_FOUND_NUMBER) {
>> >+ if (info->kernel_version < KERNEL_VERSION(2, 6, 27))
>> >+ NUMBER(page_offset) = __PAGE_OFFSET_ORIG;
>> >+ else
>> >+ NUMBER(page_offset) = __PAGE_OFFSET_2_6_27;
>> >+ }
>> >+ if (NUMBER(vmalloc_start) == NOT_FOUND_NUMBER) {
>> >+ if (info->kernel_version < KERNEL_VERSION(2, 6, 31)) {
>> >+ NUMBER(vmalloc_start) = VMALLOC_START_ORIG;
>> >+ } else {
>> >+ NUMBER(vmalloc_start) = VMALLOC_START_2_6_31;
>> >+ }
>> >+ }
>> >+ if (NUMBER(vmemmap_start) == NOT_FOUND_NUMBER) {
>> >+ if (info->kernel_version < KERNEL_VERSION(2, 6, 31))
>> >+ NUMBER(vmemmap_start) = VMEMMAP_START_ORIG;
>> >+ else
>> >+ NUMBER(vmemmap_start) = VMEMMAP_START_2_6_31;
>> >+ }
>> >+
>>
>> (I should have said this when you post the early kaslr patch.)
>> This logic is only for x86_64, I don't like to take it out to
>> here(general pass) with #ifdef. Is there any necessity to write
>> this code here ?
>
>Yes, you are right. I plan to put them into get_versiondep_info_x86_64.
>>
>> > #endif
>> >if (SIZE(pageflags) == NOT_FOUND_STRUCTURE) {
>> >if (info->kernel_version >= KERNEL_VERSION(2, 6, 27))
>> >@@ -2249,6 +2270,9 @@ write_vmcoreinfo_data(void)
>> >
>> >WRITE_NUMBER("PAGE_BUDDY_MAPCOUNT_VALUE", PAGE_BUDDY_MAPCOUNT_VALUE);
>> >WRITE_NUMBER("KE
RE: makedumpfile issues many readpage_elf: Attempt to read non-existent page
>Hello,
>
>I am investigating an issue with makedumpfile and kernel 4.8 where makedumpfile
>(1.6.0) exits on error with the following message :
>
> get_mem_map: Can't distinguish the memory type.
>
>I found commit 2c21d4656e8d3c2af2b1e14809d076941ae69e96 in the upstream
>development branch that is supposed to fix this :
>
>[PATCH v2] Support _count -> _refcount rename in struct page
>
>_count member was renamed to _refcount in linux commit 0139aa7b7fa12
>("mm: rename _count, field of the struct page, to _refcount") and this
>broke makedumpfile. The reason for making the change was to find all users
>accessing it directly and not through the recommended API. I tried
>suggesting to revert the change but failed, I see no other choice than to
>start supporting both _count and _refcount in makedumpfile.
>
>Though, when I apply the patch and test on either Ubuntu's 4.8.0-11 kernel, or
>kernel.org's mainline 4.8.0-040800rc7 kernel, I get the following repeated
>multiple times :
>
>> makedumpfile -c -d 31 /proc/vmcore /var/crash/201609221517/dump-incomplete
>> [7.513337] kdump-tools[715]: readpage_elf: Attempt to read non-existent
>> page at 0x134dfff78000.
>> [7.524186] kdump-tools[715]: readmem: type_addr: 0,
>> addr:9b4dfff78000, size:16
>> [7.528440] kdump-tools[715]: section_mem_map_addr: Can't get a struct
>> mem_section(9b4dfff78000).
>> [7.536562] kdump-tools[715]: readpage_elf: Attempt to read non-existent
>> page at 0x134dfff78000.
>> [7.544356] kdump-tools[715]: readmem: type_addr: 0,
>> addr:9b4dfff78010, size:16
>> [7.552422] kdump-tools[715]: section_mem_map_addr: Can't get a struct
>> mem_section(9b4dfff78010).
>> [7.560317] kdump-tools[715]: readpage_elf: Attempt to read non-existent
>> page at 0x134dfff78000.
>> [7.568422] kdump-tools[715]: readmem: type_addr: 0,
>> addr:9b4dfff78020, size:16
>> [7.572296] kdump-tools[715]: section_mem_map_addr: Can't get a struct
>> mem_section(9b4dfff78020).
>
>I also tested with all the commits in the upstream/development branch with no
>luck, I still get the same behavior.
>
>Does someone have an idea of where this could come from ?
Thanks for your report, but I'm afraid that I'm going on
vacation till Oct 2.
I hope someone helps you until I'm back.
Regards,
Atsushi Kumagai
>TIA,
>
>Louis
>--
>Louis Bouchard
>Software engineer, Cloud & Sustaining eng.
>Canonical Ltd
>Ubuntu developer Debian Maintainer
>GPG : 429D 7A3B DD05 B6F8 AF63 B9C4 8B3D 867C 823E 7A61
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH v3 1/3] open_dump_bitmap: open bitmap file in non-cyclic case
Hello Martin,
>The logic of set_bitmap() requires that a bitmap fd exists in the
>non-cyclic case.
>
>Signed-off-by: Martin Wilck <mwi...@suse.de>
I'll merge this version into v1.6.1, thanks for your work.
Regards,
Atsushi Kumagai
>---
> makedumpfile.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index d168dfd..6164468 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -1365,7 +1365,7 @@ open_dump_bitmap(void)
>
> /* Unnecessary to open */
> if (!info->working_dir && !info->flag_reassemble &&
> !info->flag_refiltering
>- && !info->flag_sadump && !info->flag_mem_usage)
>+ && !info->flag_sadump && !info->flag_mem_usage && info->flag_cyclic)
> return TRUE;
>
> tmpname = getenv("TMPDIR");
>--
>2.9.3
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH v3 3/3] close_dump_bitmap: simplify logic
Hello Martin,
>The boolean expression replicates the logic of open_dump_bitmap().
>It's simpler and less error-prone to simply check if fd_bitmap is
>valid.
>
>(I forgot to apply the very change that Petr had asked for in V2 of
>this patch. I'm very sorry. Please apply V3).
This version looks good, so could you add your Signed-off-by ?
Thanks,
Atsushi Kumagai
>---
> makedumpfile.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 30e1fa8..0f5be7f 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -8615,8 +8615,7 @@ close_dump_file(void)
> void
> close_dump_bitmap(void)
> {
>- if (!info->working_dir && !info->flag_reassemble &&
>!info->flag_refiltering
>- && !info->flag_sadump && !info->flag_mem_usage)
>+ if (info->fd_bitmap < 0)
> return;
>
> if (close(info->fd_bitmap) < 0)
>--
>2.9.3
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH v2] Cleanup: Use a negative number for uninitialized file descriptors
Hello,
>Do not use zero to denote an invalid file descriptor.
>
>First, zero is a valid value, although quite unlikely to be used for
>anything except standard input.
>
>Second, open(2) returns a negative value on failure, so there are
>already checks for a negative value in some places.
>
>The purpose of this patch is not to allow running in an evil environment
>(with closed stdin), but to aid in debugging by using a consistent value
>for uninitialized file descriptors which is also regarded as invalid by
>the kernel. For example, attempts to close a negative FD will fail
>(unlike an attempt to close FD 0).
>
>Changes from v1:
> - Cleanup file descriptor usage in dwarf_info.c and sadump_info.c
Thanks for your quick response.
This fix isn't necessary but better to do it as you said.
Martin, could you rebase your three patches ?
I've updated the devel branch.
Thanks,
Atsushi Kumagai
>Signed-off-by: Petr Tesarik <ptesa...@suse.com>
>
>---
> dwarf_info.c | 6 --
> makedumpfile.c | 68 +-
> makedumpfile.h | 2 +-
> sadump_info.c | 3 ++-
> 4 files changed, 46 insertions(+), 33 deletions(-)
>
>diff --git a/dwarf_info.c b/dwarf_info.c
>index 8c491d3..4f9ad12 100644
>--- a/dwarf_info.c
>+++ b/dwarf_info.c
>@@ -53,7 +53,9 @@ struct dwarf_info {
> charsrc_name[LEN_SRCFILE]; /* OUT */
> Dwarf_Off die_offset; /* OUT */
> };
>-static struct dwarf_info dwarf_info;
>+static struct dwarf_info dwarf_info = {
>+ .fd_debuginfo = -1,
>+};
>
>
> /*
>@@ -1595,7 +1597,7 @@ set_dwarf_debuginfo(char *mod_name, char *os_release,
> if (dwarf_info.module_name
> && strcmp(dwarf_info.module_name, "vmlinux")
> && strcmp(dwarf_info.module_name, "xen-syms")) {
>- if (dwarf_info.fd_debuginfo > 0)
>+ if (dwarf_info.fd_debuginfo >= 0)
> close(dwarf_info.fd_debuginfo);
> if (dwarf_info.name_debuginfo)
> free(dwarf_info.name_debuginfo);
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 21784e8..d168dfd 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -3730,10 +3730,10 @@ free_for_parallel()
> return;
>
> for (i = 0; i < info->num_threads; i++) {
>- if (FD_MEMORY_PARALLEL(i) > 0)
>+ if (FD_MEMORY_PARALLEL(i) >= 0)
> close(FD_MEMORY_PARALLEL(i));
>
>- if (FD_BITMAP_MEMORY_PARALLEL(i) > 0)
>+ if (FD_BITMAP_MEMORY_PARALLEL(i) >= 0)
> close(FD_BITMAP_MEMORY_PARALLEL(i));
> }
> }
>@@ -4038,13 +4038,13 @@ out:
> void
> initialize_bitmap(struct dump_bitmap *bitmap)
> {
>- if (info->fd_bitmap) {
>+ if (info->fd_bitmap >= 0) {
> bitmap->fd= info->fd_bitmap;
> bitmap->file_name = info->name_bitmap;
> bitmap->no_block = -1;
> memset(bitmap->buf, 0, BUFSIZE_BITMAP);
> } else {
>- bitmap->fd= 0;
>+ bitmap->fd= -1;
> bitmap->file_name = NULL;
> bitmap->no_block = -1;
> memset(bitmap->buf, 0, info->bufsize_cyclic);
>@@ -4154,7 +4154,7 @@ set_bitmap_buffer(struct dump_bitmap *bitmap, mdf_pfn_t
>pfn, int val, struct cyc
> int
> set_bitmap(struct dump_bitmap *bitmap, mdf_pfn_t pfn, int val, struct cycle
> *cycle)
> {
>- if (bitmap->fd) {
>+ if (bitmap->fd >= 0) {
> return set_bitmap_file(bitmap, pfn, val);
> } else {
> return set_bitmap_buffer(bitmap, pfn, val, cycle);
>@@ -4170,7 +4170,7 @@ sync_bitmap(struct dump_bitmap *bitmap)
> /*
>* The bitmap doesn't have the fd, it's a on-memory bitmap.
>*/
>- if (bitmap->fd == 0)
>+ if (bitmap->fd < 0)
> return TRUE;
> /*
>* The bitmap buffer is not dirty, and it is not necessary
>@@ -5403,7 +5403,7 @@ create_1st_bitmap_buffer(struct cycle *cycle)
> int
> create_1st_bitmap(struct cycle *cycle)
> {
>- if (info->bitmap1->fd) {
>+ if (info->bitmap1->fd >= 0) {
> return create_1st_bitmap_file();
> } else {
> return create_1st_bitmap_buffer(cycle);
>@@ -5414,7 +5414,7 @@ static inline int
> is_in_segs(unsigned long long paddr)
> {
> if (info->flag_refiltering || info->flag_sadump) {
>- if (info->bitmap1->fd == 0) {
RE: [PATCH] Cleanup: Use a negative number for uninitialized file descriptors
Hello Petr,
>Do not use zero to denote an invalid file descriptor.
>
>First, zero is a valid value, although quite unlikely to be used for
>anything except standard input.
>
>Second, open(2) returns a negative value on failure, so there are
>already checks for a negative value in some places.
>
>The purpose of this patch is not to allow running in an evil environment
>(with closed stdin), but to aid in debugging by using a consistent value
>for uninitialized file descriptors which is also regarded as invalid by
>the kernel. For example, attempts to close a negative FD will fail
>(unlike an attempt to close FD 0).
>
>Signed-off-by: Petr Tesarik <ptesa...@suse.com>
Good, thanks for your work, but could you fix
more two points as below ?
dwarf_info.c::
1595 if (dwarf_info.module_name
1596 && strcmp(dwarf_info.module_name, "vmlinux")
1597 && strcmp(dwarf_info.module_name, "xen-syms")) {
1598 if (dwarf_info.fd_debuginfo > 0) //
should be '>= 0'
1599 close(dwarf_info.fd_debuginfo);
sadump_info.c::
1919 for (i = 1; i < si->num_disks; ++i) {
1920 if (si->diskset_info[i].fd_memory) //
should be 'fd_memory >=0'
1921 close(si->diskset_info[i].fd_memory);
Thanks,
Atsushi Kumagai
>---
> makedumpfile.c | 68 +-
> makedumpfile.h | 2 +-
> 2 files changed, 40 insertions(+), 30 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 21784e8..d168dfd 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -3730,10 +3730,10 @@ free_for_parallel()
> return;
>
> for (i = 0; i < info->num_threads; i++) {
>- if (FD_MEMORY_PARALLEL(i) > 0)
>+ if (FD_MEMORY_PARALLEL(i) >= 0)
> close(FD_MEMORY_PARALLEL(i));
>
>- if (FD_BITMAP_MEMORY_PARALLEL(i) > 0)
>+ if (FD_BITMAP_MEMORY_PARALLEL(i) >= 0)
> close(FD_BITMAP_MEMORY_PARALLEL(i));
> }
> }
>@@ -4038,13 +4038,13 @@ out:
> void
> initialize_bitmap(struct dump_bitmap *bitmap)
> {
>- if (info->fd_bitmap) {
>+ if (info->fd_bitmap >= 0) {
> bitmap->fd= info->fd_bitmap;
> bitmap->file_name = info->name_bitmap;
> bitmap->no_block = -1;
> memset(bitmap->buf, 0, BUFSIZE_BITMAP);
> } else {
>- bitmap->fd= 0;
>+ bitmap->fd= -1;
> bitmap->file_name = NULL;
> bitmap->no_block = -1;
> memset(bitmap->buf, 0, info->bufsize_cyclic);
>@@ -4154,7 +4154,7 @@ set_bitmap_buffer(struct dump_bitmap *bitmap, mdf_pfn_t
>pfn, int val, struct cyc
> int
> set_bitmap(struct dump_bitmap *bitmap, mdf_pfn_t pfn, int val, struct cycle
> *cycle)
> {
>- if (bitmap->fd) {
>+ if (bitmap->fd >= 0) {
> return set_bitmap_file(bitmap, pfn, val);
> } else {
> return set_bitmap_buffer(bitmap, pfn, val, cycle);
>@@ -4170,7 +4170,7 @@ sync_bitmap(struct dump_bitmap *bitmap)
> /*
>* The bitmap doesn't have the fd, it's a on-memory bitmap.
>*/
>- if (bitmap->fd == 0)
>+ if (bitmap->fd < 0)
> return TRUE;
> /*
>* The bitmap buffer is not dirty, and it is not necessary
>@@ -5403,7 +5403,7 @@ create_1st_bitmap_buffer(struct cycle *cycle)
> int
> create_1st_bitmap(struct cycle *cycle)
> {
>- if (info->bitmap1->fd) {
>+ if (info->bitmap1->fd >= 0) {
> return create_1st_bitmap_file();
> } else {
> return create_1st_bitmap_buffer(cycle);
>@@ -5414,7 +5414,7 @@ static inline int
> is_in_segs(unsigned long long paddr)
> {
> if (info->flag_refiltering || info->flag_sadump) {
>- if (info->bitmap1->fd == 0) {
>+ if (info->bitmap1->fd < 0) {
> initialize_1st_bitmap(info->bitmap1);
> create_1st_bitmap_file();
> }
>@@ -5872,7 +5872,7 @@ copy_bitmap_file(void)
> int
> copy_bitmap(void)
> {
>- if (info->fd_bitmap) {
>+ if (info->fd_bitmap >= 0) {
> return copy_bitmap_file();
> } else {
> return copy_bitmap_buffer();
>@@ -6313,7 +6313,7 @@ prepare_bitmap1_buffer(void)
> return FALSE;
>
RE: "kernel version not supported" message is causing concerns
Hello Louis, >Hello, > >Every so often, I get questions or bug report about the fact that makedumpfile >issues the following message when running : > > The kernel version is not supported. > The created dumpfile may be incomplete. > >While I understand the requirement for this message, it almost systematically >is >understood by users as "makedumpfile is broken". I don't think the message itself is bad, the situation for users has a problem, I guess. If a distribution provides a combination of a kernel and a makedumpfile which doesn't support it, it stands to reason that users send a bug report. >I would like to discuss the opportunity of changing the wording of this message >so it appears less confusing. Maybe something like : > >Warning: makedumpfile has not been tested on this version of the kernel. > If the created dumpfile is incomplete, you may need to upgrade to a > newer version of makedumpfile. I suspect such message doesn't make sense because even the latest makedumpfile doesn't support the kernel version in your most cases, right ? I know I have to work harder for following up the latest kernel ASAP, but I don't agree with your suggestion. Thanks, Atsushi Kumagai >I would be happy to propose a pull request for such a change if accepted. > >Kind regards, > >...Louis > > >-- >Louis Bouchard >Software engineer, Cloud & Sustaining eng. >Canonical Ltd >Ubuntu developer Debian Maintainer >GPG : 429D 7A3B DD05 B6F8 AF63 B9C4 8B3D 867C 823E 7A61 > >___ >kexec mailing list >kexec@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 3/3] makedumpfile: Add support for MM randomization
EL_IMAGE_SIZE);
>+ WRITE_NUMBER("PAGE_OFFSET", page_offset);
>+ WRITE_NUMBER("VMALLOC_START", vmalloc_start);
>+ WRITE_NUMBER("VMEMMAP_START", vmemmap_start);
>
> WRITE_NUMBER("HUGETLB_PAGE_DTOR", HUGETLB_PAGE_DTOR);
>
>@@ -2595,6 +2619,9 @@ read_vmcoreinfo(void)
>
> READ_NUMBER("PAGE_BUDDY_MAPCOUNT_VALUE", PAGE_BUDDY_MAPCOUNT_VALUE);
> READ_NUMBER("KERNEL_IMAGE_SIZE", KERNEL_IMAGE_SIZE);
>+ READ_NUMBER("PAGE_OFFSET", page_offset);
>+ READ_NUMBER("VMALLOC_START", vmalloc_start);
>+ READ_NUMBER("VMEMMAP_START", vmemmap_start);
>
> READ_NUMBER("HUGETLB_PAGE_DTOR", HUGETLB_PAGE_DTOR);
>
>@@ -3826,7 +3853,7 @@ initial(void)
> debug_info = TRUE;
> }
>
>- info->kernel_version = get_kernel_version(info.release);
>+ info->kernel_version = get_kernel_version(info->release);
Why don't you write "info->release" in [PATCH 1/3] ?
Thanks,
Atsushi Kumagai
> if (info->kernel_version == FALSE) {
> ERRMSG("Can't get the kernel version.\n");
> return FALSE;
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 533e5b8..0e34fae 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -1685,6 +1685,9 @@ struct number_table {
>
> longPAGE_BUDDY_MAPCOUNT_VALUE;
> longKERNEL_IMAGE_SIZE;
>+ longpage_offset;
>+ longvmalloc_start;
>+ longvmemmap_start;
> longSECTION_SIZE_BITS;
> longMAX_PHYSMEM_BITS;
> longHUGETLB_PAGE_DTOR;
>--
>2.5.5
>
>
>___
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
