Re: [lldb-dev] Resolving dynamic type based on RTTI fails in case of type names inequality in DWARF and mangled symbols

[Frédéric Riss via lldb-dev]

> On Dec 20, 2017, at 11:12 AM, Ted Woodward via lldb-dev 
>  wrote:
> 
> 
> 
>> -Original Message-
>> From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Greg
>> Clayton via lldb-dev
>> Sent: Wednesday, December 20, 2017 12:41 PM
>> To: Pavel Labath 
>> Cc: lldb-dev@lists.llvm.org
>> Subject: Re: [lldb-dev] Resolving dynamic type based on RTTI fails in case of
>> type names inequality in DWARF and mangled symbols
>> 
>> 
>> Modifying llvm-dsymutil to handle ELF so we can use "llvm-dsymutil --update
>> foo.elf" is the quickest way that doesn't involve modifying anything but 
>> llvm-
>> dsymutil. It will generate the accelerator tables manually and add/modify the
>> existing accelerator tables and write out the new elf file that is all fixed 
>> up. I
>> would suggest going this route at first to see what performance improvements
>> we will see with linux so that can drive how quickly we need to adopt this.
> 
> I like this. The question is - has everything we need in llvm-dsymutil been 
> upstreamed by Apple?

Unfortunately not. Accelerator tables is one big missing piece (not because we 
don’t want to, it was blocked on some reviews). We plan to work on this in the 
coming months though.

Fred

> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a 
> Linux Foundation Collaborative Project
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Resolving dynamic type based on RTTI fails in case of type names inequality in DWARF and mangled symbols

[Ted Woodward via lldb-dev]

> -Original Message-
> From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Greg
> Clayton via lldb-dev
> Sent: Wednesday, December 20, 2017 12:41 PM
> To: Pavel Labath 
> Cc: lldb-dev@lists.llvm.org
> Subject: Re: [lldb-dev] Resolving dynamic type based on RTTI fails in case of
> type names inequality in DWARF and mangled symbols
> 
> 
> Modifying llvm-dsymutil to handle ELF so we can use "llvm-dsymutil --update
> foo.elf" is the quickest way that doesn't involve modifying anything but llvm-
> dsymutil. It will generate the accelerator tables manually and add/modify the
> existing accelerator tables and write out the new elf file that is all fixed 
> up. I
> would suggest going this route at first to see what performance improvements
> we will see with linux so that can drive how quickly we need to adopt this.

I like this. The question is - has everything we need in llvm-dsymutil been 
upstreamed by Apple?

--
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux 
Foundation Collaborative Project

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Resolving dynamic type based on RTTI fails in case of type names inequality in DWARF and mangled symbols

[Greg Clayton via lldb-dev]

> On Dec 20, 2017, at 3:33 AM, Pavel Labath  wrote:
> 
> On 19 December 2017 at 17:39, Greg Clayton via lldb-dev
>  wrote:
>> The apple accelerator tables are only enabled for Darwin target, but there
>> is nothing to say we couldn't enable these for other targets in ELF files.
>> It would be a quick way to gauge the performance improvement that these
>> accelerator tables provide for linux.
> 
> I was actually experimenting with this last month. Unfortunately, I've
> learned that the situation is not as simple as flipping a switch in
> the compiler. In fact, there is no switch to flip as clang will
> already emit the apple tables if you pass -glldb. However, the
> resulting tables will be unusable due to the differences in how dwarf
> is linked on elf vs mach-o. In elf, we have the linker concatenate the
> debug info into the final executable/shared library, which it will
> also happily do for the .apple_*** sections.

That ruins the whole idea of the accelerator tables if they are concatenated...

> The problem comes from the then we are not able to pry these sections
> apart in the debugger, as they are not self-terminating and the linker
> will not insert any metadata to help us with that. So all we can do in
> the debugger is see that the .apple_names section is present, and
> index the first table in the section (which is quite useless). To get
> around this we would need to teach the linker(s) (elf targets use many
> linkers) to merge these accelerator tables in the same way that
> dsymutil does, or modify the build process to insert an additional
> dsymutil step.

agreed that is the only way to make this work.

> The second, more subtle problem I see is that these tables are an
> all-or-nothing event. If we see an accelerator table, we assume it is
> an index of the entire module, but that's not likely to be the case,
> especially in the early days of this feature's uptake. You will have
> people feeding the linkers with output from different compilers, some
> of which will produce these tables, and some not. Then the users will
> be surprised that the debugger is ignoring some of their symbols.

I think it is best to auto generate the tables from the DWARF directly after it 
has all been linked. Skip teaching the linker about merging it, just teach it 
to generate it.

> 
> The easiest way to make the apple tables work might be to use them in
> the split-dwarf scenario, as this is kinda similar to the mac
> non-dsym-bundle scenario, where the debug info remains in the original
> .o file and is not touched by the linker. Unfortunately, currently the
> combination of -glldb and -gsplit-dwarf makes the compilation fail.
> 
> However, I would actually advocate for pushing for the dwarf-5
> accelerator tables in the ELF case. The reason is that these solve the
> both problems I mention above:
> - they don't require any special support from the rest of the
> toolchain -- if a linker just concatenates the tables, the debugger
> will still be able to use them as indexes of the individual
> compilation units. Of course, if the linker knows about these tables,
> it can merge them and provide a single uber-table for indexing the
> full module, but this is optional. This makes incremental deployment
> much smoother, which leads me to the next item,
> - they support mixing indexed and non-indexed .o files -- the tables
> contain a list of compilation units they cover, so the debugger can
> match this against the actual list of CUs, and if it sees that one
> unit is not covered by any index, it can index that one manually.
> 
> This is probably a bit more work than just "flipping a switch", but I
> hope it will not be too much work. The layout and contents of the
> tables are generally the same, so I am hoping most of the compiler
> code for the apple tables can be reused for the dwarf5 tables. If
> things turn out they way I want them to, I'll be able to work on
> getting this done next year.

Modifying llvm-dsymutil to handle ELF so we can use "llvm-dsymutil --update 
foo.elf" is the quickest way that doesn't involve modifying anything but 
llvm-dsymutil. It will generate the accelerator tables manually and add/modify 
the existing accelerator tables and write out the new elf file that is all 
fixed up. I would suggest going this route at first to see what performance 
improvements we will see with linux so that can drive how quickly we need to 
adopt this. 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Resolving dynamic type based on RTTI fails in case of type names inequality in DWARF and mangled symbols

[Pavel Labath via lldb-dev]

On 19 December 2017 at 17:39, Greg Clayton via lldb-dev
 wrote:
> The apple accelerator tables are only enabled for Darwin target, but there
> is nothing to say we couldn't enable these for other targets in ELF files.
> It would be a quick way to gauge the performance improvement that these
> accelerator tables provide for linux.

I was actually experimenting with this last month. Unfortunately, I've
learned that the situation is not as simple as flipping a switch in
the compiler. In fact, there is no switch to flip as clang will
already emit the apple tables if you pass -glldb. However, the
resulting tables will be unusable due to the differences in how dwarf
is linked on elf vs mach-o. In elf, we have the linker concatenate the
debug info into the final executable/shared library, which it will
also happily do for the .apple_*** sections.

The problem comes from the then we are not able to pry these sections
apart in the debugger, as they are not self-terminating and the linker
will not insert any metadata to help us with that. So all we can do in
the debugger is see that the .apple_names section is present, and
index the first table in the section (which is quite useless). To get
around this we would need to teach the linker(s) (elf targets use many
linkers) to merge these accelerator tables in the same way that
dsymutil does, or modify the build process to insert an additional
dsymutil step.

The second, more subtle problem I see is that these tables are an
all-or-nothing event. If we see an accelerator table, we assume it is
an index of the entire module, but that's not likely to be the case,
especially in the early days of this feature's uptake. You will have
people feeding the linkers with output from different compilers, some
of which will produce these tables, and some not. Then the users will
be surprised that the debugger is ignoring some of their symbols.

The easiest way to make the apple tables work might be to use them in
the split-dwarf scenario, as this is kinda similar to the mac
non-dsym-bundle scenario, where the debug info remains in the original
.o file and is not touched by the linker. Unfortunately, currently the
combination of -glldb and -gsplit-dwarf makes the compilation fail.

However, I would actually advocate for pushing for the dwarf-5
accelerator tables in the ELF case. The reason is that these solve the
both problems I mention above:
- they don't require any special support from the rest of the
toolchain -- if a linker just concatenates the tables, the debugger
will still be able to use them as indexes of the individual
compilation units. Of course, if the linker knows about these tables,
it can merge them and provide a single uber-table for indexing the
full module, but this is optional. This makes incremental deployment
much smoother, which leads me to the next item,
- they support mixing indexed and non-indexed .o files -- the tables
contain a list of compilation units they cover, so the debugger can
match this against the actual list of CUs, and if it sees that one
unit is not covered by any index, it can index that one manually.

This is probably a bit more work than just "flipping a switch", but I
hope it will not be too much work. The layout and contents of the
tables are generally the same, so I am hoping most of the compiler
code for the apple tables can be reused for the dwarf5 tables. If
things turn out they way I want them to, I'll be able to work on
getting this done next year.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev