[Bug general/30975] elfutils incorrectly reports core files with non-contiguous segments

2023-11-21 Thread amerey at redhat dot com
https://sourceware.org/bugzilla/show_bug.cgi?id=30975

--- Comment #5 from Aaron Merey  ---
A fix for this bug has been merged into elfutils upstream in commit
2f38fa57942f9. I'm going to leave this bug open for now while the patch makes
its way into Fedora and RHEL.

Pablo if you get a chance, please let us know whether this fixes things on your
end.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

Re: [PATCH v2] libdwfl: Correctly handle corefile non-contiguous segments

2023-11-21 Thread Aaron Merey
Hi Mark,

On Tue, Nov 21, 2023 at 7:48 AM Mark Wielaard  wrote:
> >
> > The size of the uncompressed testcore-noncontig has been reduced from
> > 736M to 54K.
>
> Uncompressed it is 580K. Still a bit large, but much more reasonable.
> We even have a couple of larger test files in the repo. Thanks.
> And in theory it can be replicated.

Sorry about that, I didn't mean to mix compressed and uncompressed sizes.

> > dwfl_addrsegment tests have been added to verify correct handling of
> > non-contiguous segments.
>
> BTW. Adding extra comments after the --- makes it easier to post a
> commit as you will apply it because comments after the --- will be
> ignored by git am.
>
> Please restore the original commit message before pushing.
> The description of the issue was really good.

Done.

> > +  /* First segment of non-contiguous module following the address space 
> > gap.  */
> > +  seg = dwfl_addrsegment (dwfl, 0x7f14e47ad000, NULL);
> > +  assert (seg == 40);
>
> OK. This does make sense if you have seen the eu-readelf -l testcore-
> noncontig output. Maybe that should be there in a comment?

Done.

> All looks good. Maybe add the suggested comment about looking at the
> program headers to undestand the testscase and please restore the
> original commit message before applying.

Pushed as commit 2f38fa57942f95.

Thanks,
Aaron



Re: [PATCH v2 2/2] tests: Add test for duplicate entries in archive

2023-11-21 Thread Mark Wielaard
Hi Aleksei,

On Mon, 2023-11-20 at 17:44 +, Aleksei Vetrov wrote:
> Test dwfl-report-offline-memory against an archive that contains
> non-relocatable ELFs with the same name and contents.
> 
> * tests/test-ar-duplicates.a.bz2: New test file.
> * tests/run-dwfl-report-offline-memory.sh: Test new
>   test-ar-duplicates.a.bz2.
> * tests/Makefile.am (EXTRA_DIST): Add test-ar-duplicates.a.bz2.

Thanks. Also tested it fails before your fix.

Pushed,

Mark


Re: [PATCH v2 1/2] libdwfl: handle duplicate ELFs when reporting archives

2023-11-21 Thread Mark Wielaard
Hi Aleksei,

On Mon, 2023-11-20 at 17:44 +, Aleksei Vetrov wrote:
> When archive is processed in process_archive (libdwfl/offline.c), it
> creates an Elf object for each archive member. Then in
> process_archive_member it calls process_file to create a Dwfl_Module
> through __libdwfl_report_elf.
> 
> The ownership of the Elf object is expected to be:
> 
> * either transfered to the Dwfl_Module, if __libdwfl_report_elf returns
>   not NULL;
> 
> * or handled at the end of process_archive_member by calling elf_end.
> 
> Moreover, Elf object is expected to be alive, if __libdwfl_report_elf
> returns not NULL, because at the end of process_archive_member it
> advances to the next member through the elf_next call.
> 
> The problem happens when __libdwfl_report_elf encounters Elf with the
> same name and content as it seen before. In that case dwfl_report_module
> will reuse existing Dwfl_Module object. This leads to a codepath that
> calls elf_end on the Elf object, while returning not NULL, breaking the
> elf_next call to the next member.
> 
> The fix is to destroy m->main.elf instead and put the new Elf object in
> the already existing Dwfl_Module.
> 
> * libdwfl/dwfl_report_elf.c (__libdwfl_report_elf): Replace Elf in
>   the Dwfl_Module in case of duplicate modules to prolong its
>   lifetime for subsequent processing.

Thanks, pushed.

Mark


Re: [PATCH v2] libdwfl: Correctly handle corefile non-contiguous segments

2023-11-21 Thread Mark Wielaard
Hi Aaron,

On Fri, 2023-11-17 at 21:48 -0500, Aaron Merey wrote:
> v1: https://sourceware.org/pipermail/elfutils-devel/2023q4/006644.html
> 
> v2 changes:
> 
> The size of the uncompressed testcore-noncontig has been reduced from
> 736M to 54K.

Uncompressed it is 580K. Still a bit large, but much more reasonable.
We even have a couple of larger test files in the repo. Thanks.
And in theory it can be replicated.

> dwfl_addrsegment tests have been added to verify correct handling of
> non-contiguous segments.

BTW. Adding extra comments after the --- makes it easier to post a
commit as you will apply it because comments after the --- will be
ignored by git am.

Please restore the original commit message before pushing.
The description of the issue was really good.

> ---
>  libdwfl/dwfl_segment_report_module.c |  37 ++-
>  tests/.gitignore |   1 +
>  tests/Makefile.am|   8 ++--
>  tests/dwfl-core-noncontig.c  |  67 +++
>  tests/run-dwfl-core-noncontig.sh |  63 +
>  tests/testcore-noncontig.bz2 | Bin 0 -> 54684 bytes
>  6 files changed, 162 insertions(+), 14 deletions(-)
>  create mode 100644 tests/dwfl-core-noncontig.c
>  create mode 100755 tests/run-dwfl-core-noncontig.sh
>  create mode 100644 tests/testcore-noncontig.bz2
> 
> diff --git a/libdwfl/dwfl_segment_report_module.c 
> b/libdwfl/dwfl_segment_report_module.c
> index 3ef62a7d..09ee37b3 100644
> --- a/libdwfl/dwfl_segment_report_module.c
> +++ b/libdwfl/dwfl_segment_report_module.c
> @@ -737,17 +737,34 @@ dwfl_segment_report_module (Dwfl *dwfl, int ndx, const 
> char *name,
>   && invalid_elf (module->elf, module->disk_file_has_build_id,
>   _id))
> {
> - elf_end (module->elf);
> - close (module->fd);
> - module->elf = NULL;
> - module->fd = -1;
> + /* If MODULE's build-id doesn't match the disk file's
> +build-id, close ELF only if MODULE and ELF refer to
> +different builds of files with the same name.  This
> +prevents premature closure of the correct ELF in cases
> +where segments of a module are non-contiguous in memory.  */
> + if (name != NULL && module->name[0] != '\0'
> + && strcmp (basename (module->name), basename (name)) == 0)
> +   {
> + elf_end (module->elf);
> + close (module->fd);
> + module->elf = NULL;
> + module->fd = -1;
> +   }
> }
> - if (module->elf != NULL)
> + else if (module->elf != NULL)
> {
> - /* Ignore this found module if it would conflict in address
> -space with any already existing module of DWFL.  */
> + /* This module has already been reported.  */
>   skip_this_module = true;
> }
> + else
> +   {
> + /* Only report this module if we haven't already done so.  */
> + for (Dwfl_Module *mod = dwfl->modulelist; mod != NULL;
> +  mod = mod->next)
> +   if (mod->low_addr == module_start
> +   && mod->high_addr == module_end)
> + skip_this_module = true;
> +   }
> }
>if (skip_this_module)
>   goto out;
> @@ -781,10 +798,6 @@ dwfl_segment_report_module (Dwfl *dwfl, int ndx, const 
> char *name,
>   }
>  }
>  
> -  /* Our return value now says to skip the segments contained
> - within the module.  */
> -  ndx = addr_segndx (dwfl, segment, module_end, true);
> -
>/* Examine its .dynamic section to get more interesting details.
>   If it has DT_SONAME, we'll use that as the module name.
>   If it has a DT_DEBUG, then it's actually a PIE rather than a DSO.
> @@ -929,6 +942,8 @@ dwfl_segment_report_module (Dwfl *dwfl, int ndx, const 
> char *name,
>ndx = -1;
>goto out;
>  }
> +  else
> +ndx++;
>  
>/* We have reported the module.  Now let the caller decide whether we
>   should read the whole thing in right now.  */

Code looks the same.

> diff --git a/tests/.gitignore b/tests/.gitignore
> index b9aa22ba..5bebb2c4 100644
> --- a/tests/.gitignore
> +++ b/tests/.gitignore
> @@ -49,6 +49,7 @@
>  /dwfl-report-elf-align
>  /dwfl-report-offline-memory
>  /dwfl-report-segment-contiguous
> +/dwfl-core-noncontig
>  /dwfllines
>  /dwflmodtest
>  /dwflsyms

Ack.

> diff --git a/tests/Makefile.am b/tests/Makefile.am
> index 7fb8efb1..9f8f7698 100644
> --- a/tests/Makefile.am
> +++ b/tests/Makefile.am
> @@ -42,7 +42,7 @@ check_PROGRAMS = arextract arsymtest newfile saridx 
> scnnames sectiondump \
> dwfl-bug-addr-overflow arls dwfl-bug-fd-leak \
> dwfl-addr-sect dwfl-bug-report early-offscn \
>