Re: [lldb-dev] Status of DWARF64 in LLDB

[Eric Christopher via lldb-dev]

On Tue, Mar 12, 2019 at 1:43 PM Jan Kratochvil via lldb-dev
 wrote:
>
> On Mon, 11 Mar 2019 21:25:05 +0100, Jan Kratochvil via lldb-dev wrote:
> > I think it is never needed in real world as long as one uses DWP and/or
> > -fdebug-types-section.  Red Hat is using neither (for DWZ postprocessing) 
> > and
> > so I did hit this limit of unsupported DWARF64 in GNU utilities [attached].
>
> FYI chromium-67.0.3396.87.rpm built with gcc-8.3.1-2.fc29.x86_64 DWARF-4
> -fdebug-types-section has 1.6GB .debug_info and 0.5GB .debug_types:
>
> Section Headers:
>   [Nr] Name   Type Address  Off  Size ES Flg Lk 
> Inf Al
>   [30] .debug_aranges PROGBITS    000360   7a7630 00  0   
> 0 16
>   [31] .debug_infoPROGBITS    7a7990 5e358d47 00  0   
> 0  1
>   [32] .debug_abbrev  PROGBITS  5eb006d7  716d053 00  0   
> 0  1
>   [33] .debug_linePROGBITS  65c6d72a  c686a97 00  0   
> 0  1
>   [34] .debug_str PROGBITS  722f41c1 21c96669 01  MS  0   
> 0  1
>   [35] .debug_loc PROGBITS  93f8a82a 24b3a638 00  0   
> 0  1
>   [36] .debug_ranges  PROGBITS  b8ac4e62  ec29fd0 00  0   
> 0  1
>   [37] .debug_types   PROGBITS  c76eee32 1cab0bd3 00  0   
> 0  1
>
FWIW I would consider Chromium a medium sized project...

I'll see if I can grab some sample numbers of a few internal applications.

-eric
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Jan Kratochvil via lldb-dev]

On Mon, 11 Mar 2019 21:25:05 +0100, Jan Kratochvil via lldb-dev wrote:
> I think it is never needed in real world as long as one uses DWP and/or
> -fdebug-types-section.  Red Hat is using neither (for DWZ postprocessing) and
> so I did hit this limit of unsupported DWARF64 in GNU utilities [attached].

FYI chromium-67.0.3396.87.rpm built with gcc-8.3.1-2.fc29.x86_64 DWARF-4
-fdebug-types-section has 1.6GB .debug_info and 0.5GB .debug_types:

Section Headers:
  [Nr] Name   Type Address  Off  Size ES Flg Lk Inf 
Al
  [30] .debug_aranges PROGBITS    000360   7a7630 00  0   0 
16
  [31] .debug_infoPROGBITS    7a7990 5e358d47 00  0   0 
 1
  [32] .debug_abbrev  PROGBITS  5eb006d7  716d053 00  0   0 
 1
  [33] .debug_linePROGBITS  65c6d72a  c686a97 00  0   0 
 1
  [34] .debug_str PROGBITS  722f41c1 21c96669 01  MS  0   0 
 1
  [35] .debug_loc PROGBITS  93f8a82a 24b3a638 00  0   0 
 1
  [36] .debug_ranges  PROGBITS  b8ac4e62  ec29fd0 00  0   0 
 1
  [37] .debug_types   PROGBITS  c76eee32 1cab0bd3 00  0   0 
 1


Jan
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Eric Christopher via lldb-dev]

On Mon, Mar 11, 2019 at 3:49 PM Adrian Prantl via lldb-dev
 wrote:
>
>
>
> On Mar 11, 2019, at 3:46 PM, Zachary Turner  wrote:
>
> Given that:
>
> 1) LLVM doesn't produce DWARF64
> 2) GCC has to be patched to produce DWARF64
> 3) LLDB's support is only partial but is untested and appears to be missing 
> major pieces in order for it to work
> 4) It's of questionable use as there are several viable alternatives
>
> Would it be reasonable to propose a patch removing the incomplete support 
> from LLDB?  We can always add it back later when someone is ready to really 
> support and test it properly, and the history in the repository will show 
> what code would need to be changed to get back to at least where the support 
> is today (which again, appears to not fully work).
>
> If we can go this route, it makes merging the two DWARF parsing 
> implementations quite a bit simpler
>
>
> I'm supportive of removing DWARF64 support from LLDB.
>

Sounds good to me as well. We may need to add it in the next few
years, but simplifying interfaces for now sounds entirely reasonable.

-eric

> -- adrian
>
>
> On Mon, Mar 11, 2019 at 3:33 PM Adrian Prantl  wrote:
>>
>>
>>
>> > On Mar 11, 2019, at 12:45 PM, Zachary Turner via lldb-dev 
>> >  wrote:
>> >
>> > I want to ask what the status of DWARF64 in LLDB is.  I can tell there's 
>> > some support for it by reading the code, but it seems to have zero test 
>> > coverage so it's not clear to me that anyone depends on it.  For example, 
>> > I know that clang and LLVM will not even generate DWARF64, so if anyone is 
>> > depending on it, they must be debugging programs built with some other 
>> > toolchain.
>>
>> AFAIR, Apple's tools only generate/support DWARF32. After implementing 
>> type-uniquing in dsymutil we didn't see any individual .dSYM bundles that 
>> came even close to the 4GB watermark.
>>
>> >
>> > I'm looking at unifying LLDB's DWARF parser with LLVM's, and this is the 
>> > biggest functional difference I can see.
>> >
>> > Certainly we can improve LLVM's support for consuming DWARF64, but it's a 
>> > question of priorities.  If nobody is actively depending on this, then 
>> > taking a regression here could be on the table and then re-prioritizing 
>> > adding back support in the future if / when we actually need it.
>>
>> -- adrian
>
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Adrian Prantl via lldb-dev]

> On Mar 11, 2019, at 3:46 PM, Zachary Turner  wrote:
> 
> Given that:
> 
> 1) LLVM doesn't produce DWARF64
> 2) GCC has to be patched to produce DWARF64
> 3) LLDB's support is only partial but is untested and appears to be missing 
> major pieces in order for it to work
> 4) It's of questionable use as there are several viable alternatives
> 
> Would it be reasonable to propose a patch removing the incomplete support 
> from LLDB?  We can always add it back later when someone is ready to really 
> support and test it properly, and the history in the repository will show 
> what code would need to be changed to get back to at least where the support 
> is today (which again, appears to not fully work).  
> 
> If we can go this route, it makes merging the two DWARF parsing 
> implementations quite a bit simpler

I'm supportive of removing DWARF64 support from LLDB.

-- adrian

> 
> On Mon, Mar 11, 2019 at 3:33 PM Adrian Prantl  > wrote:
> 
> 
> > On Mar 11, 2019, at 12:45 PM, Zachary Turner via lldb-dev 
> > mailto:lldb-dev@lists.llvm.org>> wrote:
> > 
> > I want to ask what the status of DWARF64 in LLDB is.  I can tell there's 
> > some support for it by reading the code, but it seems to have zero test 
> > coverage so it's not clear to me that anyone depends on it.  For example, I 
> > know that clang and LLVM will not even generate DWARF64, so if anyone is 
> > depending on it, they must be debugging programs built with some other 
> > toolchain.
> 
> AFAIR, Apple's tools only generate/support DWARF32. After implementing 
> type-uniquing in dsymutil we didn't see any individual .dSYM bundles that 
> came even close to the 4GB watermark.
> 
> > 
> > I'm looking at unifying LLDB's DWARF parser with LLVM's, and this is the 
> > biggest functional difference I can see.  
> > 
> > Certainly we can improve LLVM's support for consuming DWARF64, but it's a 
> > question of priorities.  If nobody is actively depending on this, then 
> > taking a regression here could be on the table and then re-prioritizing 
> > adding back support in the future if / when we actually need it.
> 
> -- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Zachary Turner via lldb-dev]

Given that:

1) LLVM doesn't produce DWARF64
2) GCC has to be patched to produce DWARF64
3) LLDB's support is only partial but is untested and appears to be missing
major pieces in order for it to work
4) It's of questionable use as there are several viable alternatives

Would it be reasonable to propose a patch removing the incomplete support
from LLDB?  We can always add it back later when someone is ready to really
support and test it properly, and the history in the repository will show
what code would need to be changed to get back to at least where the
support is today (which again, appears to not fully work).

If we can go this route, it makes merging the two DWARF parsing
implementations quite a bit simpler

On Mon, Mar 11, 2019 at 3:33 PM Adrian Prantl  wrote:

>
>
> > On Mar 11, 2019, at 12:45 PM, Zachary Turner via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > I want to ask what the status of DWARF64 in LLDB is.  I can tell there's
> some support for it by reading the code, but it seems to have zero test
> coverage so it's not clear to me that anyone depends on it.  For example, I
> know that clang and LLVM will not even generate DWARF64, so if anyone is
> depending on it, they must be debugging programs built with some other
> toolchain.
>
> AFAIR, Apple's tools only generate/support DWARF32. After implementing
> type-uniquing in dsymutil we didn't see any individual .dSYM bundles that
> came even close to the 4GB watermark.
>
> >
> > I'm looking at unifying LLDB's DWARF parser with LLVM's, and this is the
> biggest functional difference I can see.
> >
> > Certainly we can improve LLVM's support for consuming DWARF64, but it's
> a question of priorities.  If nobody is actively depending on this, then
> taking a regression here could be on the table and then re-prioritizing
> adding back support in the future if / when we actually need it.
>
> -- adrian
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Adrian Prantl via lldb-dev]

> On Mar 11, 2019, at 12:45 PM, Zachary Turner via lldb-dev 
>  wrote:
> 
> I want to ask what the status of DWARF64 in LLDB is.  I can tell there's some 
> support for it by reading the code, but it seems to have zero test coverage 
> so it's not clear to me that anyone depends on it.  For example, I know that 
> clang and LLVM will not even generate DWARF64, so if anyone is depending on 
> it, they must be debugging programs built with some other toolchain.

AFAIR, Apple's tools only generate/support DWARF32. After implementing 
type-uniquing in dsymutil we didn't see any individual .dSYM bundles that came 
even close to the 4GB watermark.

> 
> I'm looking at unifying LLDB's DWARF parser with LLVM's, and this is the 
> biggest functional difference I can see.  
> 
> Certainly we can improve LLVM's support for consuming DWARF64, but it's a 
> question of priorities.  If nobody is actively depending on this, then taking 
> a regression here could be on the table and then re-prioritizing adding back 
> support in the future if / when we actually need it.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Zachary Turner via lldb-dev]

Thanks Jan,

That was my suspicion as well.  If it's true that DWARF64 support is
currently non-functional, then I think the easiest path forward is to
remove any traces of it from LLDB as a way of bringing the two
implementations closer together.

I'll tinker around with this idea in a local branch while waiting to see if
anyone else has any input.

On Mon, Mar 11, 2019 at 1:25 PM Jan Kratochvil 
wrote:

> On Mon, 11 Mar 2019 20:45:48 +0100, Zachary Turner via lldb-dev wrote:
> > I want to ask what the status of DWARF64 in LLDB is.
>
> IMO there isn't as for example:
> lldb/source/Plugins/SymbolFile/DWARF/DIERef.cpp
> is using bits 32..63 for additional info (DWO file offset/index for
> example)
> while only bits 0..31 are used for DIE offset inside .debug_info section.
>
> lldb/include/lldb/Core/dwarf.h
> #ifdef DWARFUTILS_DWARF64
> but nobody ever defines DWARFUTILS_DWARF64 and so it uses:
> typedef uint32_t dw_offset_t; // Dwarf Debug Information Entry offset for
> any
>   // offset into the file
>
>
> > For example, I > know that clang and LLVM will not even generate DWARF64,
>
> Even GCC needs to be patched to generate DWARF64.
>
>
> > Certainly we can improve LLVM's support for consuming DWARF64, but it's a
> > question of priorities.
>
> I think it is never needed in real world as long as one uses DWP and/or
> -fdebug-types-section.  Red Hat is using neither (for DWZ postprocessing)
> and
> so I did hit this limit of unsupported DWARF64 in GNU utilities [attached].
>
>
> Jan
>
>
>
> -- Forwarded message --
> From: Jan Kratochvil 
> To: x...@redhat.com
> Cc:
> Bcc:
> Date: Wed, 8 Aug 2018 15:14:16 +0200
> Subject: DWARF64 for Chromium with 10GB of DWARF
> Hello,
>
> LLDB people were talking about 6GB Chromium binaries. So I checked Fedora
> Chromium but:
> # Debuginfo packages aren't very useful here. If you need to debug
> # you should do a proper debug build (not implemented in this spec
> yet)
> %global debug_package %{nil}
> and it uses no '-g' during compilation.
>
> After enabling Chromium debug info [attached, it has only -g2, not -g3] I
> got:
> obj/mojo/public/cpp/system/system/message_pipe.o:(.debug_loc+0x1bcc):
> relocation truncated to fit: R_X86_64_32 against `.debug_info'
> ...
> obj/third_party/blink/renderer/platform/platform/fe_convolve_matrix.o:(.debug_info+0x4b5dc):
> additional relocation overflows omitted from the output
> collect2: error: ld returned 1 exit status
>
> Which is logical as Chromium has 8GB of .debug_info section. I found no gcc
> option to enable 64-bit DWARF so I had to patch GCC for that [attached].
>
> But then the rpmbuild failed a different way:
>
> /usr/lib/rpm/debugedit:
> BUILDROOT/chromium-67.0.3396.87-2.fc28.x86_64/usr/lib64/chromium-browser/libEGL.so:
> 64-bit DWARF not supported
> ...
> eu-strip: elf32_updatefile.c:336: __elf64_updatemmap: Assertion
> `dl->data.d.d_size <= (shdr->sh_size - (GElf_Off) dl->data.d.d_off)' failed.
> /usr/lib/rpm/find-debuginfo.sh: line 231: 3998449 Aborted
>  eu-strip --remove-comment $r $g ${keep_remove_args} -f "$1" "$2"
> double free or corruption (out)
> /usr/lib/rpm/find-debuginfo.sh: line 231: 3998529 Aborted
>  eu-strip --remove-comment $r $g ${keep_remove_args} -f "$1" "$2"
> dwz:
> ./etc/chromium/native-messaging-hosts/remoting_user_session-67.0.3396.87-2.fc28.x86_64.debug:
> 64-bit DWARF not supported
>
> DWZ would be unable to handle it on x86_64 even if it did support DWARF64
> as
> rpmbuild limits it to 110e6 DIEs while this DWARF has 500e6 DIEs.
>
> Google AFAIK builds it with -gsplit-dwarf and then one can pack the *.dwo
> files into EXEC.dwp by /usr/bin/dwp (llvm-dwp in the Google case).
> DWZ-like optimization is then achieved by -fdebug-types-section.
> But then rpmbuild is not prepared to handle *.dwp (like current *.debug).
>
> The 10GB DWARF64 binary if anyone is interested:
> https://www.jankratochvil.net/t/chromium-headless_shell.xz
>
> Then it is questionable whether to deal with DWARF64/DWP just for Chromium.
> Customers really do not have binaries of this DWARF size?
>
>
> Jan
> ___
>
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Jan Kratochvil via lldb-dev]

On Mon, 11 Mar 2019 20:45:48 +0100, Zachary Turner via lldb-dev wrote:
> I want to ask what the status of DWARF64 in LLDB is.

IMO there isn't as for example:
lldb/source/Plugins/SymbolFile/DWARF/DIERef.cpp
is using bits 32..63 for additional info (DWO file offset/index for example)
while only bits 0..31 are used for DIE offset inside .debug_info section.

lldb/include/lldb/Core/dwarf.h
#ifdef DWARFUTILS_DWARF64
but nobody ever defines DWARFUTILS_DWARF64 and so it uses:
typedef uint32_t dw_offset_t; // Dwarf Debug Information Entry offset for any
  // offset into the file


> For example, I > know that clang and LLVM will not even generate DWARF64,

Even GCC needs to be patched to generate DWARF64.


> Certainly we can improve LLVM's support for consuming DWARF64, but it's a
> question of priorities.

I think it is never needed in real world as long as one uses DWP and/or
-fdebug-types-section.  Red Hat is using neither (for DWZ postprocessing) and
so I did hit this limit of unsupported DWARF64 in GNU utilities [attached].


Jan
--- Begin Message ---
Hello,

LLDB people were talking about 6GB Chromium binaries. So I checked Fedora
Chromium but:
# Debuginfo packages aren't very useful here. If you need to debug
# you should do a proper debug build (not implemented in this spec yet)
%global debug_package %{nil}
and it uses no '-g' during compilation.

After enabling Chromium debug info [attached, it has only -g2, not -g3] I got:
obj/mojo/public/cpp/system/system/message_pipe.o:(.debug_loc+0x1bcc): 
relocation truncated to fit: R_X86_64_32 against `.debug_info'
...
obj/third_party/blink/renderer/platform/platform/fe_convolve_matrix.o:(.debug_info+0x4b5dc):
 additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status

Which is logical as Chromium has 8GB of .debug_info section. I found no gcc
option to enable 64-bit DWARF so I had to patch GCC for that [attached].

But then the rpmbuild failed a different way:

/usr/lib/rpm/debugedit: 
BUILDROOT/chromium-67.0.3396.87-2.fc28.x86_64/usr/lib64/chromium-browser/libEGL.so:
 64-bit DWARF not supported
...
eu-strip: elf32_updatefile.c:336: __elf64_updatemmap: Assertion 
`dl->data.d.d_size <= (shdr->sh_size - (GElf_Off) dl->data.d.d_off)' failed.
/usr/lib/rpm/find-debuginfo.sh: line 231: 3998449 Aborted 
eu-strip --remove-comment $r $g ${keep_remove_args} -f "$1" "$2"
double free or corruption (out)
/usr/lib/rpm/find-debuginfo.sh: line 231: 3998529 Aborted 
eu-strip --remove-comment $r $g ${keep_remove_args} -f "$1" "$2"
dwz: 
./etc/chromium/native-messaging-hosts/remoting_user_session-67.0.3396.87-2.fc28.x86_64.debug:
 64-bit DWARF not supported

DWZ would be unable to handle it on x86_64 even if it did support DWARF64 as
rpmbuild limits it to 110e6 DIEs while this DWARF has 500e6 DIEs.

Google AFAIK builds it with -gsplit-dwarf and then one can pack the *.dwo
files into EXEC.dwp by /usr/bin/dwp (llvm-dwp in the Google case).
DWZ-like optimization is then achieved by -fdebug-types-section.
But then rpmbuild is not prepared to handle *.dwp (like current *.debug).

The 10GB DWARF64 binary if anyone is interested:
https://www.jankratochvil.net/t/chromium-headless_shell.xz

Then it is questionable whether to deal with DWARF64/DWP just for Chromium.
Customers really do not have binaries of this DWARF size?


Jan
diff --git a/gcc/defaults.h b/gcc/defaults.h
index 9035b333be8..ec4fce27395 100644
--- a/gcc/defaults.h
+++ b/gcc/defaults.h
@@ -458,7 +458,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  
If not, see
DWARF-2 specification.  The SGI/MIPS ABI defines it to be the same
as PTR_SIZE.  */
 #ifndef DWARF_OFFSET_SIZE
-#define DWARF_OFFSET_SIZE 4
+#define DWARF_OFFSET_SIZE PTR_SIZE
 #endif
 
 /* The size in bytes of a DWARF 4 type signature.  */
diff --git a/chromium.spec b/chromium.spec
index a2f2d93..a34b636 100644
--- a/chromium.spec
+++ b/chromium.spec
@@ -16,7 +16,7 @@
 
 # Debuginfo packages aren't very useful here. If you need to debug
 # you should do a proper debug build (not implemented in this spec yet)
-%global debug_package %{nil}
+#global debug_package %{nil}
 
 # %%{nil} for Stable; -beta for Beta; -dev for Devel
 # dash in -beta and -dev is intentional !
@@ -932,7 +932,7 @@ popd
 
 # Core defines are flags that are true for both the browser and headless.
 CHROMIUM_CORE_GN_DEFINES=""
-CHROMIUM_CORE_GN_DEFINES+=' is_debug=false'
+CHROMIUM_CORE_GN_DEFINES+=' is_debug=false symbol_level=2'
 %ifarch x86_64 aarch64
 CHROMIUM_CORE_GN_DEFINES+=' system_libdir="lib64"'
 %endif
@@ -960,7 +960,7 @@ CHROMIUM_BROWSER_GN_DEFINES+=' is_component_ffmpeg=true 
is_component_build=true'
 %else
 CHROMIUM_BROWSER_GN_DEFINES+=' is_component_ffmpeg=false 
is_component_build=false'
 %endif
-CHROMIUM_BROWSER_GN_DEFINES+=' remove_webcore_debug_symbols=true 
enable_hangout_services_extension=true'
+CHROMIUM_BROWSER_GN_DEFINES+='