Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[David Blaikie via lldb-dev]

On Mon, Jun 18, 2018 at 9:54 AM  wrote:

> > Greg wrote:
> > > Pavel wrote:
> > > That said, having DWARF be able to represent the template member
> > > functions in an abstract way also sounds like nice thing to have from
> > > a debug info format.
> >
> > Yes, that would be great, but will require DWARF changes and is much more
> > long term.
>
> I'm curious what utility this has, other than tidying up the Clang AST
> interface part (because you know what templates exist inside the class).
> I mean, you can't instantiate new functions; and if you're trying to
> call an existing instance, you have to go find it anyway, in whichever
> CU it happens to have been instantiated.
>

A couple of questionable reasons:

1) name/overload resolution - having the names of functions you can't call
(because they've been inlined away, never instantiated, etc) means that if
a debugger is evaluating an expression it won't accidentally resolve a call
to a different function from the one that would've been used in the source
language. (eg: a class with foo(int) and foo(T) - if you call foo(true) -
but the debugger doesn't know any foo(T) exists, so it calls foo(int),
which could be varying degrees of unfortunate)

This could happen for any function though, and it'd certainly be
impractical to include all function declarations (especially for
non-members), all types, etc, to ensure that all names are available to
validate any ambiguities, etc.

2) Possible that there are libraries linked in that themselves don't have
debug info - but include specializations of a template (or definitions of
any declared function, really) - so having the debug info could be used to
know about those functions (given at least Itanium mangling, though - I'm
not sure the debug info would be necessary, maybe looking at the mangled
name would be sufficient for a debugger to know "oh, this function is a
member of this class and has these parameter types" - hmm, guess it
wouldn't know the return type without debug info, perhaps)


>
> Feel free to start a new thread if this is straying too far from the
> discussion that already strayed from the original topic. :-)
> Thanks,
> --paulr
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Eric Christopher via lldb-dev]

On Mon, Jun 18, 2018 at 9:57 AM Greg Clayton via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

>
>
> > On Jun 18, 2018, at 9:54 AM,  <
> paul.robin...@sony.com> wrote:
> >
> >> Greg wrote:
> >>> Pavel wrote:
> >>> That said, having DWARF be able to represent the template member
> >>> functions in an abstract way also sounds like nice thing to have from
> >>> a debug info format.
> >>
> >> Yes, that would be great, but will require DWARF changes and is much
> more
> >> long term.
> >
> > I'm curious what utility this has, other than tidying up the Clang AST
> > interface part (because you know what templates exist inside the class).
> > I mean, you can't instantiate new functions; and if you're trying to
> > call an existing instance, you have to go find it anyway, in whichever
> > CU it happens to have been instantiated.
> >
> > Feel free to start a new thread if this is straying too far from the
> > discussion that already strayed from the original topic. :-)
>
> I do agree. Probably no one else will want/need this in DWARF except us as
> I don't believe anyone else is re-creating compiler types with DWARF. Not
> that I don't think it is a good idea for debuggers to do as it allows the
> compiler to be used in the debugger. That being said, we should probably
> look for solutions that are better for all DWARF clients or just fix things
> in our debugger.
>
>
Oh, I've seen DWARF used outside of lldb's context to reconstruct types - I
also think it's a fairly legitimate use of debug info. If we can make it
easier to reconstruct the actual program...

-eric


>
> > Thanks,
> > --paulr
> >
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Jan Kratochvil via lldb-dev]

On Mon, 18 Jun 2018 18:57:07 +0200, Greg Clayton wrote:
> I do agree. Probably no one else will want/need this in DWARF except us as
> I don't believe anyone else is re-creating compiler types with DWARF.

GDB-GCC try to copy the LLDB approach:
https://sourceware.org/gdb/wiki/GCCCompileAndExecute


Jan
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Greg Clayton via lldb-dev]

> On Jun 18, 2018, at 9:54 AM,  
>  wrote:
> 
>> Greg wrote:
>>> Pavel wrote:
>>> That said, having DWARF be able to represent the template member
>>> functions in an abstract way also sounds like nice thing to have from
>>> a debug info format.
>> 
>> Yes, that would be great, but will require DWARF changes and is much more
>> long term.
> 
> I'm curious what utility this has, other than tidying up the Clang AST
> interface part (because you know what templates exist inside the class).
> I mean, you can't instantiate new functions; and if you're trying to
> call an existing instance, you have to go find it anyway, in whichever
> CU it happens to have been instantiated.
> 
> Feel free to start a new thread if this is straying too far from the
> discussion that already strayed from the original topic. :-)

I do agree. Probably no one else will want/need this in DWARF except us as I 
don't believe anyone else is re-creating compiler types with DWARF. Not that I 
don't think it is a good idea for debuggers to do as it allows the compiler to 
be used in the debugger. That being said, we should probably look for solutions 
that are better for all DWARF clients or just fix things in our debugger.


> Thanks,
> --paulr
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Greg Clayton via lldb-dev]

> On Jun 15, 2018, at 1:48 PM, Pavel Labath  wrote:
> 
> On Fri, 15 Jun 2018 at 20:14, David Blaikie  wrote:
>> 
>> How do you handle name lookup for nested classes? They have the same problem 
>> (they don't appear in all definitions) - don't appear in all descriptions of 
>> the outer/parent class. (in theory we could ensure there's always at least a 
>> declaration of the nested class - but we don't even do that if the nested 
>> class is unused)
>> 
>> Is it just the case that Clang doesn't mind you adding a new nested class 
>> but it does mind you adding a new member function template? If so, maybe we 
>> could change Clang to support adding new member function templates instead 
>> of extending DWARF?
> 
> 
> I was thinking about the same thing. It seems to me that this could be
> viewed as a deficiency of our implementation of dwarf parsing. (It's a
> pretty understandable deficiency, given that we are based on clang
> (compiler), and it thinks of the types in the same way as C++ does --
> incomplete; or complete with template members and all). However, these
> template member functions should not impact anything "important" in
> the class (data member layout, vtables, ...) so one could conceivably
> have an implementation which allows member addition on the fly. And in
> this case the existing accelerator tables would work perfectly -- we
> would get a query "does this class have method X", we would look at
> the accel table, and it would point us straight to X. However, I have
> no idea how hard would it be to fit this scheme into the existing
> clang/lldb design.

For a little background on how the DWARF parsing works. When anything has a 
type "class A" in LLDB (variable, method in class A, etc), we first create a 
forward declaration to class A in the clang AST. Each symbol file installs 
itself as an external AST source that can complete any types when requested. 
This uses the functionality that mimics each symbol file being a precompiled 
header. Since we have an external AST source, clang can ask for the type to be 
completed at any time (just like clang does with PCH files), which will call 
back into the external AST source and cause the type to be completed. 

In LLDB, we have 3 distinct states of a type: forward declaration to type, 
completed type, the type we need for layout. If a variable has a type "class 
A*", then when displaying this variable in the output of "frame var", we don't 
need to know the full definition of class A. If the user expands a type in say 
an IDE variable view or types an expression that results in a class A instance 
or reference, we will get the complete version of the type. When we are making 
other types that include class A through inheritance or as a member variable, 
we may or may not need get the full definition for class A. For example if we 
have a "class A* m_a_ptr;" member variable, we don't need to know about A since 
we only need to know what type we need in order to layout the type which is a 
pointer here. If we inherit from class A or have a "class A m_a;" then the 
layout type needed here would be the full type. So we are very careful to only 
expand the type when and if needed in LLDB. 

That being said we currently only have two versions of the type as far as clang 
know: forward and complete. I think this is how clang expects things to be as 
well. So the above approach could work as long as we can teach clang to ask the 
external AST source about addition names when doing method lookups, but that 
will take some clang modifications that will probably only need to be enabled 
in debugger mode. Also if we did add this ability, we will want to limit it to 
classes that have templated functions we don't want to always ask a class we 
have completed to continually look for things when the user types an expression 
wrong like "a.this_method_never_will_exist_in_a()". Maybe that is ok though.

If we are able to make the above approach work, then there will need to be an 
additional fix where we would need to teach AST importer to be able to compare 
two classes and ignore any template methods when doing the compare when in 
debugger mode only. We might have two shared libraries that each have a class 
A, one with one specialized function and one with another, and when one or both 
of those get imported into the expression or target AST, they would need to 
merge the type correctly when importing it. Right now if there are two class 
trying be to imported at the same decl context layer the import will fail if 
the class exists at the same level and isn't exactly the same. 

So quite a bit of clang work if we want to make this work.

Each symbol file installs itself as an external AST source so if we wanted to 
add the template member functions on the fly, hopefully we would get a request 
that says in the DeclContext "class A" what is "Foo" if you type something like:

A a;
(lldb) a.Foo()

But I am not sure what clang will do if it already 

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Pavel Labath via lldb-dev]

On Fri, 15 Jun 2018 at 20:14, David Blaikie  wrote:
>
> How do you handle name lookup for nested classes? They have the same problem 
> (they don't appear in all definitions) - don't appear in all descriptions of 
> the outer/parent class. (in theory we could ensure there's always at least a 
> declaration of the nested class - but we don't even do that if the nested 
> class is unused)
>
> Is it just the case that Clang doesn't mind you adding a new nested class but 
> it does mind you adding a new member function template? If so, maybe we could 
> change Clang to support adding new member function templates instead of 
> extending DWARF?


I was thinking about the same thing. It seems to me that this could be
viewed as a deficiency of our implementation of dwarf parsing. (It's a
pretty understandable deficiency, given that we are based on clang
(compiler), and it thinks of the types in the same way as C++ does --
incomplete; or complete with template members and all). However, these
template member functions should not impact anything "important" in
the class (data member layout, vtables, ...) so one could conceivably
have an implementation which allows member addition on the fly. And in
this case the existing accelerator tables would work perfectly -- we
would get a query "does this class have method X", we would look at
the accel table, and it would point us straight to X. However, I have
no idea how hard would it be to fit this scheme into the existing
clang/lldb design.



That said, having DWARF be able to represent the template member
functions in an abstract way also sounds like nice thing to have from
a debug info format.

pl
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[via lldb-dev]

gc> Solution #1 would cause us to dig through all definitions of all C++
gc> classes all the time when parsing DWARF to check if definitions of
gc> the classes had template methods. And we would need to find the class
gc> that has the most template methods. This would cause us to parse much
gc> more of the debug info all of the time and cause increased memory
gc> consumption and performance regressions.

pr> It would be cheap to put a flag on the class DIE that tells you there
pr> are template methods to go look for.  Then you incur the cost only
pr> when necessary.  And the accelerator table makes it fast to find the
pr> other class descriptions.

gc> That is a fine solution. But we still run into the problem where we don't
gc> know if the DWARF knows about that flag. If we do a flag, it would be nice
gc> if it were mandatory on all classes to indicate support for the flag. But
gc> this would be a fine solution and not hard to implement.

pr> So what you really want is not a flag, but a count, so you can tell when
pr> you've found all the different templates.  If the count is zero, there's
pr> nothing to look for.  If the count is two, you look around at all the
pr> various definitions of the class until you find two different templates,
pr> then you stop.  If there's no count attribute, your producer doesn't 
pr> know you want this information and you do it the hard way.  Or, we've
pr> invented a way to describe the templates directly in the class.
pr>
pr> How's that?

gc> that would work fine.

I filed PR37816 to track this idea.

The other ideas:

 - accelerator to point to the actual instantiations
 - emitting template definitions not just instantiations

would be trickier to define and harder to implement correctly.
I won't say they can't be done, but somebody else would have to do
the heavy lifting here, unless it turns out that our debugger folks
like the idea.

--paulr

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Greg Clayton via lldb-dev]

> On Jun 15, 2018, at 10:40 AM,  
>  wrote:
> 
> 
> 
>> -Original Message-
>> From: Greg Clayton [mailto:clayb...@gmail.com <mailto:clayb...@gmail.com>]
>> Sent: Friday, June 15, 2018 12:46 PM
>> To: Robinson, Paul
>> Cc: lab...@google.com <mailto:lab...@google.com>; dblai...@gmail.com 
>> <mailto:dblai...@gmail.com>; apra...@apple.com <mailto:apra...@apple.com>;
>> echri...@google.com <mailto:echri...@google.com>; jdevliegh...@apple.com 
>> <mailto:jdevliegh...@apple.com>; llvm-...@lists.llvm.org 
>> <mailto:llvm-...@lists.llvm.org>;
>> jan.kratoch...@redhat.com <mailto:jan.kratoch...@redhat.com>; 
>> lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org>; Jim Ingham
>> Subject: Re: [llvm-dev] [lldb-dev] Adding DWARF5 accelerator table support
>> to llvm
>> 
>> 
>> 
>>> On Jun 15, 2018, at 9:23 AM, 
>>  wrote:
>>> 
>>>> From: Greg Clayton [mailto:clayb...@gmail.com]
>>>> 
>>>> ...
>>>> If a class has templated functions, they will only be in the DWARF is a
>>>> specialization was created and used. If you have a class that looks
>> like:
>>>> 
>>>> class A {
>>>>   A();
>>>>void Foo(T t);
>>>> };
>>>> 
>>>> And then you have main.cpp that has a "double" and "int"
>> specialization,
>>>> the class definition in DWARF looks like:
>>>> 
>>>> class A {
>>>>   A();
>>>>void Foo(int t);
>>>>void Foo(double t);
>>>> };
>>>> 
>>>> In another source file, say foo.cpp, if its use of class A doesn't
>>>> specialize Foo, we have a class definition in DWARF that looks like:
>>>> 
>>>> class A {
>>>>   A();
>>>> };
>>>> 
>>> 
>>> I think it would be more instructive to think about a case where
>>> main.cpp had Foo and foo.cpp had Foo.
>> 
>> Any difference is the problem here for clang ASTs, so it just matters that
>> they are different. We make a CXXRecordDecl in the clang AST and if we see
>> even one specialization, then we add the generic version to the
>> CXXRecordDecl and we are good to go. So the main.cpp had Foo and
>> foo.cpp had Foo is actually fine. If I make a CXXRecordDecl from
>> either of these then the two definitions match since the clang AST
>> CXXRecordDecl just needs to have the templated function declaration.
> 
> Got it, good to know.
> You don't actually care what instantiations exist? That seems odd.

The compiler will use the generic type and ask where the specialized version 
lives. For the template method type, we will in the right stuff when we make 
the function type, but the class itself just has the generic definition. 
> 
>> 
>>> 
>>>> With the C++ ODR rules, we can pick any of the "class A" definitions
>>>> whose qualified name matches ("::A") and has the same decl file +
>>>> decl line. So when parsing "class A", the DWARF parser will use the
>>>> accelerator tables to find all instances of "class A", and it will
>>>> pick on and use it and this will become the one true definition for
>>>> "class A".
>>> 
>>> FTR, the Sony debugger finds them all and then merges them into the one
>>> true definition, because there's no promise that any one description is
>>> a superset of the rest.
>> 
>> At the expense of parsing every definition for a class within each file.
>>> 
>>>> This is because DWARF is only emitted for template functions when
>>>> there is a specialization, that mean any definition of "class A" might
>>>> or might not include any definition for " A::Foo(T t);".
>>> 
>>> It's not just template functions, you know; the implicit ctors/dtors
>>> might or might not be in any given description.  Those are also only
>>> instantiated and described in CUs that require them.
>> 
>> We don't care about those as those are marked as DW_AT_artificial and we
>> leave those out of the clang AST Context CXXRecordDecl because they are
>> implicit and the compiler can add those back in if needed since it knows
>> if a class can have the constructors implicitly created.
>> 
>>> 
>>>> When we copy types between ASTs, everything is fine if the classes
>>>> match (no copy needs to be made), but things go wro

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[via lldb-dev]

> -Original Message-
> From: Greg Clayton [mailto:clayb...@gmail.com]
> Sent: Friday, June 15, 2018 12:46 PM
> To: Robinson, Paul
> Cc: lab...@google.com; dblai...@gmail.com; apra...@apple.com;
> echri...@google.com; jdevliegh...@apple.com; llvm-...@lists.llvm.org;
> jan.kratoch...@redhat.com; lldb-dev@lists.llvm.org; Jim Ingham
> Subject: Re: [llvm-dev] [lldb-dev] Adding DWARF5 accelerator table support
> to llvm
> 
> 
> 
> > On Jun 15, 2018, at 9:23 AM, 
>  wrote:
> >
> >> From: Greg Clayton [mailto:clayb...@gmail.com]
> >>
> >> ...
> >> If a class has templated functions, they will only be in the DWARF is a
> >> specialization was created and used. If you have a class that looks
> like:
> >>
> >> class A {
> >>A();
> >> void Foo(T t);
> >> };
> >>
> >> And then you have main.cpp that has a "double" and "int"
> specialization,
> >> the class definition in DWARF looks like:
> >>
> >> class A {
> >>A();
> >> void Foo(int t);
> >> void Foo(double t);
> >> };
> >>
> >> In another source file, say foo.cpp, if its use of class A doesn't
> >> specialize Foo, we have a class definition in DWARF that looks like:
> >>
> >> class A {
> >>A();
> >> };
> >>
> >
> > I think it would be more instructive to think about a case where
> > main.cpp had Foo and foo.cpp had Foo.
> 
> Any difference is the problem here for clang ASTs, so it just matters that
> they are different. We make a CXXRecordDecl in the clang AST and if we see
> even one specialization, then we add the generic version to the
> CXXRecordDecl and we are good to go. So the main.cpp had Foo and
> foo.cpp had Foo is actually fine. If I make a CXXRecordDecl from
> either of these then the two definitions match since the clang AST
> CXXRecordDecl just needs to have the templated function declaration.

Got it, good to know.
You don't actually care what instantiations exist? That seems odd.

> 
> >
> >> With the C++ ODR rules, we can pick any of the "class A" definitions
> >> whose qualified name matches ("::A") and has the same decl file +
> >> decl line. So when parsing "class A", the DWARF parser will use the
> >> accelerator tables to find all instances of "class A", and it will
> >> pick on and use it and this will become the one true definition for
> >> "class A".
> >
> > FTR, the Sony debugger finds them all and then merges them into the one
> > true definition, because there's no promise that any one description is
> > a superset of the rest.
> 
> At the expense of parsing every definition for a class within each file.
> >
> >> This is because DWARF is only emitted for template functions when
> >> there is a specialization, that mean any definition of "class A" might
> >> or might not include any definition for " A::Foo(T t);".
> >
> > It's not just template functions, you know; the implicit ctors/dtors
> > might or might not be in any given description.  Those are also only
> > instantiated and described in CUs that require them.
> 
> We don't care about those as those are marked as DW_AT_artificial and we
> leave those out of the clang AST Context CXXRecordDecl because they are
> implicit and the compiler can add those back in if needed since it knows
> if a class can have the constructors implicitly created.
> 
> >
> >> When we copy types between ASTs, everything is fine if the classes
> >> match (no copy needs to be made), but things go wrong if things
> >> don't match and this causes expression errors.
> >>
> >> Some ways to fix this:
> >> 1 - anytime we need _any_ C++ class, we must dig up all definitions
> >> and check _all_ DW_TAG_subprogram DIEs within the class to check if
> >> any functions have templates and use the class with the most
> >> specializations
> >
> > This still fails in my "more instructive" case.  The accelerator table
> > does make it fast to find all the classes with the same name, and you
> > can then merge them.
> 
> That is a lot of DWARF parsing and logic to try and figure out what the
> full set of DW_TAG_subprograms are.
> 
> >
> >> 2 - have DWARF actually emit the template function info all the time
> >> as a type T, not a specialization, so we always have the full
> definition
> >
> > Hm.  You mean a subroutine_type with t

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Greg Clayton via lldb-dev]

> On Jun 15, 2018, at 9:23 AM,  
>  wrote:
> 
>> From: Greg Clayton [mailto:clayb...@gmail.com] 
>> 
>> ...
>> If a class has templated functions, they will only be in the DWARF is a
>> specialization was created and used. If you have a class that looks like:
>> 
>> class A {
>>A();
>> void Foo(T t);
>> };
>> 
>> And then you have main.cpp that has a "double" and "int" specialization,
>> the class definition in DWARF looks like:
>> 
>> class A {
>>A();
>> void Foo(int t);
>> void Foo(double t);
>> };
>> 
>> In another source file, say foo.cpp, if its use of class A doesn't
>> specialize Foo, we have a class definition in DWARF that looks like:
>> 
>> class A {
>>A();
>> };
>> 
> 
> I think it would be more instructive to think about a case where 
> main.cpp had Foo and foo.cpp had Foo.

Any difference is the problem here for clang ASTs, so it just matters that they 
are different. We make a CXXRecordDecl in the clang AST and if we see even one 
specialization, then we add the generic version to the CXXRecordDecl and we are 
good to go. So the main.cpp had Foo and foo.cpp had Foo is 
actually fine. If I make a CXXRecordDecl from either of these then the two 
definitions match since the clang AST CXXRecordDecl just needs to have the 
templated function declaration.

> 
>> With the C++ ODR rules, we can pick any of the "class A" definitions
>> whose qualified name matches ("::A") and has the same decl file + 
>> decl line. So when parsing "class A", the DWARF parser will use the
>> accelerator tables to find all instances of "class A", and it will
>> pick on and use it and this will become the one true definition for
>> "class A".
> 
> FTR, the Sony debugger finds them all and then merges them into the one
> true definition, because there's no promise that any one description is
> a superset of the rest.

At the expense of parsing every definition for a class within each file.
> 
>> This is because DWARF is only emitted for template functions when
>> there is a specialization, that mean any definition of "class A" might
>> or might not include any definition for " A::Foo(T t);".
> 
> It's not just template functions, you know; the implicit ctors/dtors
> might or might not be in any given description.  Those are also only
> instantiated and described in CUs that require them.

We don't care about those as those are marked as DW_AT_artificial and we leave 
those out of the clang AST Context CXXRecordDecl because they are implicit and 
the compiler can add those back in if needed since it knows if a class can have 
the constructors implicitly created.

> 
>> When we copy types between ASTs, everything is fine if the classes
>> match (no copy needs to be made), but things go wrong if things
>> don't match and this causes expression errors. 
>> 
>> Some ways to fix this:
>> 1 - anytime we need _any_ C++ class, we must dig up all definitions
>> and check _all_ DW_TAG_subprogram DIEs within the class to check if
>> any functions have templates and use the class with the most
>> specializations
> 
> This still fails in my "more instructive" case.  The accelerator table
> does make it fast to find all the classes with the same name, and you
> can then merge them.

That is a lot of DWARF parsing and logic to try and figure out what the full 
set of DW_TAG_subprograms are.

> 
>> 2 - have DWARF actually emit the template function info all the time
>> as a type T, not a specialization, so we always have the full definition
> 
> Hm.  You mean a subroutine_type with template_parameter children that
> don't have actual values?  That would give you a pattern, but not tell
> you what/where definitions exist.  I don't see how that can help?

Right now DWARF does only specializations, so DWARF would need to be extended 
to be able to specify the template details without requiring a specialization. 
Not easy for sure.

> 
>> 3 - have some accelerator table that explicitly points us to all
>> specializations of class methods given a class name
> 
> So you want an accelerator to tell you what bits you need to pick up
> from the various descriptions in order to construct the overall
> superset definition, which would be a little cheaper than parsing
> each entire class description (which you can already find through
> the existing accelerator tables).  I can see that.

Yes. This is the most appealing to me as well.

> 
>> Solution #1 would cause us to dig through all definitions of all C++
>> classes all the time when parsing DWARF to check if definitions of
>> the classes had template methods. And we would need to find the class
>> that has the most template methods. This would cause us to parse much
>> more of the debug info all of the time and cause increased memory
>> consumption and performance regressions.
> 
> It would be cheap to put a flag on the class DIE that tells you there
> are template methods to go look for.  Then you incur the cost only
> when necessary.  And the accelerator table makes 

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[via lldb-dev]

> From: Greg Clayton [mailto:clayb...@gmail.com] 
>
> ...
> If a class has templated functions, they will only be in the DWARF is a
> specialization was created and used. If you have a class that looks like:
>
> class A {
>   A();
>    void Foo(T t);
> };
>
> And then you have main.cpp that has a "double" and "int" specialization,
> the class definition in DWARF looks like:
>
> class A {
>   A();
>    void Foo(int t);
>    void Foo(double t);
> };
>
> In another source file, say foo.cpp, if its use of class A doesn't
> specialize Foo, we have a class definition in DWARF that looks like:
>
> class A {
>   A();
> };
>

I think it would be more instructive to think about a case where 
main.cpp had Foo and foo.cpp had Foo.

> With the C++ ODR rules, we can pick any of the "class A" definitions
> whose qualified name matches ("::A") and has the same decl file + 
> decl line. So when parsing "class A", the DWARF parser will use the
> accelerator tables to find all instances of "class A", and it will
> pick on and use it and this will become the one true definition for
> "class A".

FTR, the Sony debugger finds them all and then merges them into the one
true definition, because there's no promise that any one description is
a superset of the rest.

> This is because DWARF is only emitted for template functions when
> there is a specialization, that mean any definition of "class A" might
> or might not include any definition for " A::Foo(T t);".

It's not just template functions, you know; the implicit ctors/dtors
might or might not be in any given description.  Those are also only
instantiated and described in CUs that require them.

> When we copy types between ASTs, everything is fine if the classes
> match (no copy needs to be made), but things go wrong if things
> don't match and this causes expression errors. 
>
> Some ways to fix this:
> 1 - anytime we need _any_ C++ class, we must dig up all definitions
> and check _all_ DW_TAG_subprogram DIEs within the class to check if
> any functions have templates and use the class with the most
> specializations

This still fails in my "more instructive" case.  The accelerator table
does make it fast to find all the classes with the same name, and you
can then merge them.

> 2 - have DWARF actually emit the template function info all the time
> as a type T, not a specialization, so we always have the full definition

Hm.  You mean a subroutine_type with template_parameter children that
don't have actual values?  That would give you a pattern, but not tell
you what/where definitions exist.  I don't see how that can help?

> 3 - have some accelerator table that explicitly points us to all
> specializations of class methods given a class name

So you want an accelerator to tell you what bits you need to pick up
from the various descriptions in order to construct the overall
superset definition, which would be a little cheaper than parsing
each entire class description (which you can already find through
the existing accelerator tables).  I can see that.

> Solution #1 would cause us to dig through all definitions of all C++
> classes all the time when parsing DWARF to check if definitions of
> the classes had template methods. And we would need to find the class
> that has the most template methods. This would cause us to parse much
> more of the debug info all of the time and cause increased memory
> consumption and performance regressions.

It would be cheap to put a flag on the class DIE that tells you there
are template methods to go look for.  Then you incur the cost only
when necessary.  And the accelerator table makes it fast to find the
other class descriptions.

> Solution #2: not sure if DWARF even supports generic template
> definitions where the template isn't specialized. And, how would we
> be able to tell DWARF that emits only specialized templates vs one
> that has generic definitions...

Right, DWARF today doesn't do that, although as I mentioned earlier
conjuring up a subroutine_type with template parameters would not
appear to be any more helpful than a simple flag on the class.

>
> Solution #3 will require compiler changes.

Well, so does #2.

> So this is another vote to support the ability for a given class
> to be able to locate all of its functions, kind of like we need
> for Objective C where the class definition doesn't contain all of
> methods, so we have the .apple_objc section that provides this
> mapping for us. We would need something similar for C++.
>
> So maybe a possible solution is some sort of section that can
> specify all of the DIEs related to a class that are not contained
> in the class hierarchy itself. This would work for Objective C
> and for C++.
>
> Thoughts?
>
> Greg

So, would it be helpful to have a flag on the class that tells you
whether there are method instantiations to go look for?  My
impression is that templated class methods are unusual so it would
save the performance cost a lot of the time right there.

I 

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Greg Clayton via lldb-dev]

To elaborate a bit more on the issue that is detailed in 
https://reviews.llvm.org/rL260308:

There are many clang AST contexts that are used in LLDB:
- one for each lldb_private::Module that contains type definitions as we know 
them in the module and its symbol vendor
- one for each expression
- one for results of expressions in the lldb_private::Target

As we run expressions we end up copying classes around between the Module ASTs 
and the expression and Target ASTs. If a class has templated functions, they 
will only be in the DWARF is a specialization was created and used. If you have 
a class that looks like:

class A {
   A();
void Foo(T t);
};


And then you have main.cpp that has a "double" and "int" specialization, the 
class definition in DWARF looks like:

class A {
   A();
void Foo(int t);
void Foo(double t);
};

In another source file, say foo.cpp, if its use of class A doesn't specialize 
Foo, we have a class definition in DWARF that looks like:

class A {
   A();
};

With the C++ ODR rules, we can pick any of the "class A" definitions whose 
qualified name matches ("::A") and has the same decl file + decl line. So when 
parsing "class A", the DWARF parser will use the accelerator tables to find all 
instances of "class A", and it will pick on and use it and this will become the 
one true definition for "class A". This is because DWARF is only emitted for 
template functions when there is a specialization, that mean any definition of 
"class A" might or might not include any definition for " A::Foo(T 
t);". When we copy types between ASTs, everything is fine if the classes match 
(no copy needs to be made), but things go wrong if things don't match and this 
causes expression errors. 

Some ways to fix this:
1 - anytime we need _any_ C++ class, we must dig up all definitions and check 
_all_ DW_TAG_subprogram DIEs within the class to check if any functions have 
templates and use the class with the most specializations
2 - have DWARF actually emit the template function info all the time as a type 
T, not a specialization, so we always have the full definition
3 - have some accelerator table that explicitly points us to all 
specializations of class methods given a class name

Solution #1 would cause us to dig through all definitions of all C++ classes 
all the time when parsing DWARF to check if definitions of the classes had 
template methods. And we would need to find the class that has the most 
template methods. This would cause us to parse much more of the debug info all 
of the time and cause increased memory consumption and performance regressions.

Solution #2: not sure if DWARF even supports generic template definitions where 
the template isn't specialized. And, how would we be able to tell DWARF that 
emits only specialized templates vs one that has generic definitions...

Solution #3 will require compiler changes.

So this is another vote to support the ability for a given class to be able to 
locate all of its functions, kind of like we need for Objective C where the 
class definition doesn't contain all of methods, so we have the .apple_objc 
section that provides this mapping for us. We would need something similar for 
C++.

So maybe a possible solution is some sort of section that can specify all of 
the DIEs related to a class that are not contained in the class hierarchy 
itself. This would work for Objective C and for C++.

Thoughts?

Greg

> On Jun 15, 2018, at 3:34 AM, Pavel Labath  wrote:
> 
> I wasn't using type units (those don't work at all right now).
> 
> I've done a bit of digging, and i found this patch
>  which explicitly disables template
> member function parsing (though it seems it didn't really work before
> either). The patch contains a quite long explanation of why is this
> not working. I can't say I understand all of it (this is getting a bit
> out of my league), but the core of the issue seems to be that when we
> start to mix classes from two CU which have different sets of
> instantiations in a single expression, things quickly go south because
> the recycled clang ASTs from the two dwarf versions do not match.
> 
> For better or worse, it seems gdb is having similar issues as well, as
> I couldn't get it to grok my member template expressions either..
> 
> On Thu, 14 Jun 2018 at 19:47, David Blaikie  wrote:
>> 
>> oh, awesome.
>> 
>> Were you using type units? (I imagine that'd make the situation worse - 
>> since the way clang emits DWARF for a type with a member function template 
>> implicit specialization is to emit the type unit without any mention of 
>> this, and to emit the implicit specialization declaration into the stub type 
>> in the CU (that references the type unit)) Without type units I'd be pretty 
>> surprised if you couldn't call the implicit specialization at least from the 
>> CU in which it was instantiated.
>> 
>> On Thu, Jun 14, 2018 at 11:41 AM Pavel Labath  wrote:
>>> 
>>> On 

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Pavel Labath via lldb-dev]

I wasn't using type units (those don't work at all right now).

I've done a bit of digging, and i found this patch
 which explicitly disables template
member function parsing (though it seems it didn't really work before
either). The patch contains a quite long explanation of why is this
not working. I can't say I understand all of it (this is getting a bit
out of my league), but the core of the issue seems to be that when we
start to mix classes from two CU which have different sets of
instantiations in a single expression, things quickly go south because
the recycled clang ASTs from the two dwarf versions do not match.

For better or worse, it seems gdb is having similar issues as well, as
I couldn't get it to grok my member template expressions either..

On Thu, 14 Jun 2018 at 19:47, David Blaikie  wrote:
>
> oh, awesome.
>
> Were you using type units? (I imagine that'd make the situation worse - since 
> the way clang emits DWARF for a type with a member function template implicit 
> specialization is to emit the type unit without any mention of this, and to 
> emit the implicit specialization declaration into the stub type in the CU 
> (that references the type unit)) Without type units I'd be pretty surprised 
> if you couldn't call the implicit specialization at least from the CU in 
> which it was instantiated.
>
> On Thu, Jun 14, 2018 at 11:41 AM Pavel Labath  wrote:
>>
>> On Thu, 14 Jun 2018 at 19:29, Pavel Labath  wrote:
>> >
>> > On Thu, 14 Jun 2018 at 19:26, David Blaikie  wrote:
>> > >
>> > >
>> > >
>> > > On Thu, Jun 14, 2018 at 11:24 AM Pavel Labath  wrote:
>> > >>
>> > >> On Thu, 14 Jun 2018 at 17:58, Greg Clayton  wrote:
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Jun 14, 2018, at 9:36 AM, Adrian Prantl  wrote:
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev 
>> > >> >  wrote:
>> > >> >
>> > >> > Thank you all. I am going to try to reply to all comments in a single 
>> > >> > email.
>> > >> >
>> > >> > Regarding the  .apple_objc idea, I am afraid the situation is not as
>> > >> > simple as just flipping a switch.
>> > >> >
>> > >> >
>> > >> > Jonas is currently working on adding the support for DWARF5-style 
>> > >> > Objective-C accelerator tables to LLVM/LLDB/dsymutil. Based on the 
>> > >> > assumption that DWARF 4 and earlier are unaffected by any of this, I 
>> > >> > don't think it's necessary to spend any effort of making the 
>> > >> > transition smooth. I'm fine with having Objective-C on DWARF 5 broken 
>> > >> > on trunk for two weeks until Jonas is done adding Objective-C support 
>> > >> > to the DWARF 5 implementation.
>> > >>
>> > >> Ideally, I would like to enable the accelerator tables (possibly with
>> > >> a different version number or something) on DWARF 4 too (on non-apple
>> > >> targets only). The reason for this is that their absence if causing
>> > >> large slowdowns when debugging on non-apple platforms, and I wouldn't
>> > >> want to wait for dwarf 5 for that to go away (I mean no disrespect to
>> > >> Paul and DWARF 5 effort in general, but even if all of DWARF 5 in llvm
>> > >> was done tomorrow, there would still be lldb, which hasn't even begun
>> > >> to look at this version).
>> > >>
>> > >> That said, if you are working on the Objective C support right now,
>> > >> then I am happy to wait two weeks or so that we have a full
>> > >> implementation from the get-go.
>> > >>
>> > >> > But, other options may be possible as well. What's not clear to me is
>> > >> > whether these tables couldn't be replaced by extra information in the
>> > >> > .debug_info section. It seems to me that these tables are trying to
>> > >> > work around the issue that there is no straight way to go from a
>> > >> > DW_TAG_structure type DIE describing an ObjC class to it's methods. If
>> > >> > these methods (their forward declarations) were be present as children
>> > >> > of the type DIE (as they are for c++ classes), then these tables may
>> > >> > not be necessary. But maybe (probably) that has already been
>> > >> > considered and deemed infeasible for some reason. In any case this
>> > >> > seemed like a thing best left for people who actually work on ObjC
>> > >> > support to figure out.
>> > >> >
>> > >> >
>> > >> > That's really a question for Greg or Jim — I don't know why the 
>> > >> > current representation has the Objective-C methods outside of the 
>> > >> > structs. One reason might be that an interface's implementation can 
>> > >> > define more methods than are visible in its public interface in the 
>> > >> > header file, but we already seem to be aware of this and mark the 
>> > >> > implementation with DW_AT_APPLE_objc_complete_type. I also am not 
>> > >> > sure that this is the *only* reason for the objc accelerator table. 
>> > >> > But I'd like to learn.
>> > >>
>> > >> My observation was based on studying lldb code. The only place where
>> > >> the objc table is used is in the 

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Pavel Labath via lldb-dev]

On Thu, 14 Jun 2018 at 19:29, Pavel Labath  wrote:
>
> On Thu, 14 Jun 2018 at 19:26, David Blaikie  wrote:
> >
> >
> >
> > On Thu, Jun 14, 2018 at 11:24 AM Pavel Labath  wrote:
> >>
> >> On Thu, 14 Jun 2018 at 17:58, Greg Clayton  wrote:
> >> >
> >> >
> >> >
> >> > On Jun 14, 2018, at 9:36 AM, Adrian Prantl  wrote:
> >> >
> >> >
> >> >
> >> > On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev 
> >> >  wrote:
> >> >
> >> > Thank you all. I am going to try to reply to all comments in a single 
> >> > email.
> >> >
> >> > Regarding the  .apple_objc idea, I am afraid the situation is not as
> >> > simple as just flipping a switch.
> >> >
> >> >
> >> > Jonas is currently working on adding the support for DWARF5-style 
> >> > Objective-C accelerator tables to LLVM/LLDB/dsymutil. Based on the 
> >> > assumption that DWARF 4 and earlier are unaffected by any of this, I 
> >> > don't think it's necessary to spend any effort of making the transition 
> >> > smooth. I'm fine with having Objective-C on DWARF 5 broken on trunk for 
> >> > two weeks until Jonas is done adding Objective-C support to the DWARF 5 
> >> > implementation.
> >>
> >> Ideally, I would like to enable the accelerator tables (possibly with
> >> a different version number or something) on DWARF 4 too (on non-apple
> >> targets only). The reason for this is that their absence if causing
> >> large slowdowns when debugging on non-apple platforms, and I wouldn't
> >> want to wait for dwarf 5 for that to go away (I mean no disrespect to
> >> Paul and DWARF 5 effort in general, but even if all of DWARF 5 in llvm
> >> was done tomorrow, there would still be lldb, which hasn't even begun
> >> to look at this version).
> >>
> >> That said, if you are working on the Objective C support right now,
> >> then I am happy to wait two weeks or so that we have a full
> >> implementation from the get-go.
> >>
> >> > But, other options may be possible as well. What's not clear to me is
> >> > whether these tables couldn't be replaced by extra information in the
> >> > .debug_info section. It seems to me that these tables are trying to
> >> > work around the issue that there is no straight way to go from a
> >> > DW_TAG_structure type DIE describing an ObjC class to it's methods. If
> >> > these methods (their forward declarations) were be present as children
> >> > of the type DIE (as they are for c++ classes), then these tables may
> >> > not be necessary. But maybe (probably) that has already been
> >> > considered and deemed infeasible for some reason. In any case this
> >> > seemed like a thing best left for people who actually work on ObjC
> >> > support to figure out.
> >> >
> >> >
> >> > That's really a question for Greg or Jim — I don't know why the current 
> >> > representation has the Objective-C methods outside of the structs. One 
> >> > reason might be that an interface's implementation can define more 
> >> > methods than are visible in its public interface in the header file, but 
> >> > we already seem to be aware of this and mark the implementation with 
> >> > DW_AT_APPLE_objc_complete_type. I also am not sure that this is the 
> >> > *only* reason for the objc accelerator table. But I'd like to learn.
> >>
> >> My observation was based on studying lldb code. The only place where
> >> the objc table is used is in the AppleDWARFIndex::GetObjCMethods
> >> function, which is called from
> >> SymbolFileDWARF::GetObjCMethodDIEOffsets, whose only caller is
> >> DWARFASTParserClang::CompleteTypeFromDWARF, which seems to have a
> >> class DIE as an argument. However, if not all declarations of a
> >> class/interface have access to the full list of methods then this
> >> might be a problem for the approach I suggested.
> >
> >
> > Maybe, but the same is actually true for C++ classes too (see my comments 
> > in another reply about implicit specializations of class member templates 
> > (and there are a couple of other examples)) - so might be worth considering 
> > how those are handled/could be improved, and maybe in fixing those we could 
> > improve/normalize the ObjC representation and avoid the need for ObjC 
> > tables... maybe.
> >
>
> That's a good point! I need to check out how we handle that right now.

Apparently we handle that very poorly. :/ I wasn't even able to call
the instantiation which was present in the CU I was stopped in. I
didn't even get to the part about trying an instantiation from a
different CU.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Pavel Labath via lldb-dev]

On Thu, 14 Jun 2018 at 19:26, David Blaikie  wrote:
>
>
>
> On Thu, Jun 14, 2018 at 11:24 AM Pavel Labath  wrote:
>>
>> On Thu, 14 Jun 2018 at 17:58, Greg Clayton  wrote:
>> >
>> >
>> >
>> > On Jun 14, 2018, at 9:36 AM, Adrian Prantl  wrote:
>> >
>> >
>> >
>> > On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev 
>> >  wrote:
>> >
>> > Thank you all. I am going to try to reply to all comments in a single 
>> > email.
>> >
>> > Regarding the  .apple_objc idea, I am afraid the situation is not as
>> > simple as just flipping a switch.
>> >
>> >
>> > Jonas is currently working on adding the support for DWARF5-style 
>> > Objective-C accelerator tables to LLVM/LLDB/dsymutil. Based on the 
>> > assumption that DWARF 4 and earlier are unaffected by any of this, I don't 
>> > think it's necessary to spend any effort of making the transition smooth. 
>> > I'm fine with having Objective-C on DWARF 5 broken on trunk for two weeks 
>> > until Jonas is done adding Objective-C support to the DWARF 5 
>> > implementation.
>>
>> Ideally, I would like to enable the accelerator tables (possibly with
>> a different version number or something) on DWARF 4 too (on non-apple
>> targets only). The reason for this is that their absence if causing
>> large slowdowns when debugging on non-apple platforms, and I wouldn't
>> want to wait for dwarf 5 for that to go away (I mean no disrespect to
>> Paul and DWARF 5 effort in general, but even if all of DWARF 5 in llvm
>> was done tomorrow, there would still be lldb, which hasn't even begun
>> to look at this version).
>>
>> That said, if you are working on the Objective C support right now,
>> then I am happy to wait two weeks or so that we have a full
>> implementation from the get-go.
>>
>> > But, other options may be possible as well. What's not clear to me is
>> > whether these tables couldn't be replaced by extra information in the
>> > .debug_info section. It seems to me that these tables are trying to
>> > work around the issue that there is no straight way to go from a
>> > DW_TAG_structure type DIE describing an ObjC class to it's methods. If
>> > these methods (their forward declarations) were be present as children
>> > of the type DIE (as they are for c++ classes), then these tables may
>> > not be necessary. But maybe (probably) that has already been
>> > considered and deemed infeasible for some reason. In any case this
>> > seemed like a thing best left for people who actually work on ObjC
>> > support to figure out.
>> >
>> >
>> > That's really a question for Greg or Jim — I don't know why the current 
>> > representation has the Objective-C methods outside of the structs. One 
>> > reason might be that an interface's implementation can define more methods 
>> > than are visible in its public interface in the header file, but we 
>> > already seem to be aware of this and mark the implementation with 
>> > DW_AT_APPLE_objc_complete_type. I also am not sure that this is the *only* 
>> > reason for the objc accelerator table. But I'd like to learn.
>>
>> My observation was based on studying lldb code. The only place where
>> the objc table is used is in the AppleDWARFIndex::GetObjCMethods
>> function, which is called from
>> SymbolFileDWARF::GetObjCMethodDIEOffsets, whose only caller is
>> DWARFASTParserClang::CompleteTypeFromDWARF, which seems to have a
>> class DIE as an argument. However, if not all declarations of a
>> class/interface have access to the full list of methods then this
>> might be a problem for the approach I suggested.
>
>
> Maybe, but the same is actually true for C++ classes too (see my comments in 
> another reply about implicit specializations of class member templates (and 
> there are a couple of other examples)) - so might be worth considering how 
> those are handled/could be improved, and maybe in fixing those we could 
> improve/normalize the ObjC representation and avoid the need for ObjC 
> tables... maybe.
>

That's a good point! I need to check out how we handle that right now.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[David Blaikie via lldb-dev]

On Thu, Jun 14, 2018 at 11:24 AM Pavel Labath  wrote:

> On Thu, 14 Jun 2018 at 17:58, Greg Clayton  wrote:
> >
> >
> >
> > On Jun 14, 2018, at 9:36 AM, Adrian Prantl  wrote:
> >
> >
> >
> > On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
> >
> > Thank you all. I am going to try to reply to all comments in a single
> email.
> >
> > Regarding the  .apple_objc idea, I am afraid the situation is not as
> > simple as just flipping a switch.
> >
> >
> > Jonas is currently working on adding the support for DWARF5-style
> Objective-C accelerator tables to LLVM/LLDB/dsymutil. Based on the
> assumption that DWARF 4 and earlier are unaffected by any of this, I don't
> think it's necessary to spend any effort of making the transition smooth.
> I'm fine with having Objective-C on DWARF 5 broken on trunk for two weeks
> until Jonas is done adding Objective-C support to the DWARF 5
> implementation.
>
> Ideally, I would like to enable the accelerator tables (possibly with
> a different version number or something) on DWARF 4 too (on non-apple
> targets only). The reason for this is that their absence if causing
> large slowdowns when debugging on non-apple platforms, and I wouldn't
> want to wait for dwarf 5 for that to go away (I mean no disrespect to
> Paul and DWARF 5 effort in general, but even if all of DWARF 5 in llvm
> was done tomorrow, there would still be lldb, which hasn't even begun
> to look at this version).
>
> That said, if you are working on the Objective C support right now,
> then I am happy to wait two weeks or so that we have a full
> implementation from the get-go.
>
> > But, other options may be possible as well. What's not clear to me is
> > whether these tables couldn't be replaced by extra information in the
> > .debug_info section. It seems to me that these tables are trying to
> > work around the issue that there is no straight way to go from a
> > DW_TAG_structure type DIE describing an ObjC class to it's methods. If
> > these methods (their forward declarations) were be present as children
> > of the type DIE (as they are for c++ classes), then these tables may
> > not be necessary. But maybe (probably) that has already been
> > considered and deemed infeasible for some reason. In any case this
> > seemed like a thing best left for people who actually work on ObjC
> > support to figure out.
> >
> >
> > That's really a question for Greg or Jim — I don't know why the current
> representation has the Objective-C methods outside of the structs. One
> reason might be that an interface's implementation can define more methods
> than are visible in its public interface in the header file, but we already
> seem to be aware of this and mark the implementation with
> DW_AT_APPLE_objc_complete_type. I also am not sure that this is the *only*
> reason for the objc accelerator table. But I'd like to learn.
>
> My observation was based on studying lldb code. The only place where
> the objc table is used is in the AppleDWARFIndex::GetObjCMethods
> function, which is called from
> SymbolFileDWARF::GetObjCMethodDIEOffsets, whose only caller is
> DWARFASTParserClang::CompleteTypeFromDWARF, which seems to have a
> class DIE as an argument. However, if not all declarations of a
> class/interface have access to the full list of methods then this
> might be a problem for the approach I suggested.
>

Maybe, but the same is actually true for C++ classes too (see my comments
in another reply about implicit specializations of class member templates
(and there are a couple of other examples)) - so might be worth considering
how those are handled/could be improved, and maybe in fixing those we could
improve/normalize the ObjC representation and avoid the need for ObjC
tables... maybe.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[David Blaikie via lldb-dev]

If you end up revisiting the design/representation here - I'd be glad to be
involved. It reminds me of some of the tradeoffs/issues around how even
plain C++ types vary between translation units (eg: member function
template implicit specializations - necessarily different ones can appear
in different translation units (because they were instantiated in those
places/the set of implicit specializations isn't closed)). So maybe there's
some lessons to draw between these situations.

On Thu, Jun 14, 2018 at 9:36 AM Adrian Prantl  wrote:

>
>
> > On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
> >
> > Thank you all. I am going to try to reply to all comments in a single
> email.
> >
> > Regarding the  .apple_objc idea, I am afraid the situation is not as
> > simple as just flipping a switch.
>
> Jonas is currently working on adding the support for DWARF5-style
> Objective-C accelerator tables to LLVM/LLDB/dsymutil. Based on the
> assumption that DWARF 4 and earlier are unaffected by any of this, I don't
> think it's necessary to spend any effort of making the transition smooth.
> I'm fine with having Objective-C on DWARF 5 broken on trunk for two weeks
> until Jonas is done adding Objective-C support to the DWARF 5
> implementation.
>
>
> > (If it was, I don't think I would
> > have embarked on this adventure in the first place -- I would just
> > emit .apple_*** everywhere and call it done :)). The issue is that the
> > apple tables have assumptions about the macos debug info distribution
> > model hardcoded in them -- they assume they will either stay in the .o
> > file or be linked by a smart debug-info-aware linker (dsymutil). In
> > particular, this means they are not self-delimiting (no length field
> > as is typical for other dwarf artifacts), so if a linker which is not
> > aware of them would simply concatenate individual .o tables (which elf
> > linkers are really good at), the debugger would have no way to pry
> > them apart. And even if it somehow managed that, it still wouldn't
> > know if the indexes covered all of the compile units in the linked
> > file or only some of them (in case some of the object files were
> > compiled with the tables and some without).
> >
> > In light of that, I don't think it's worth trying to combine
> > .apple_objc with .debug_names in some way, and it would be much
> > simpler to just extend .debug_names with the necessary information. I
> > think the simplest way of achieving this (one which would require
> > least amount of standard-bending) is to take the index entry for the
> > objc class and add a special attribute to it (DW_IDX_method_list?)
> > with form DW_FORM_blockXXX and just have the references to the method
> > DIEs in the block data. This should make the implementation an almost
> > drop-in for the current .apple_objc functionality (we would still need
> > to figure out what to do with category methods, but it's not clear to
> > me whether lldb actually uses those anywhere).
> >
> > But, other options may be possible as well. What's not clear to me is
> > whether these tables couldn't be replaced by extra information in the
> > .debug_info section. It seems to me that these tables are trying to
> > work around the issue that there is no straight way to go from a
> > DW_TAG_structure type DIE describing an ObjC class to it's methods. If
> > these methods (their forward declarations) were be present as children
> > of the type DIE (as they are for c++ classes), then these tables may
> > not be necessary. But maybe (probably) that has already been
> > considered and deemed infeasible for some reason. In any case this
> > seemed like a thing best left for people who actually work on ObjC
> > support to figure out.
>
> That's really a question for Greg or Jim — I don't know why the current
> representation has the Objective-C methods outside of the structs. One
> reason might be that an interface's implementation can define more methods
> than are visible in its public interface in the header file, but we already
> seem to be aware of this and mark the implementation with
> DW_AT_APPLE_objc_complete_type. I also am not sure that this is the *only*
> reason for the objc accelerator table. But I'd like to learn.
>
> -- adrian
>
> > As far as the .debug_names size goes, I should also point out that the
> > binary in question was built with -fno-limit-debug-info, which isn't a
> > default setup on linux. I have tried measuring the sizes without that
> > flag and with fission enabled (-gsplit-dwarf) and the results are:
> > without compression:
> > - clang binary: 960 MB
> > - .debug_names: 130 MB (13%)
> > - debug_pubnames: 175 MB (18%)
> > - debug_pubtypes: 204 MB (21%)
> > - median time for setting a breakpoint on non-existent function
> > (variance +/- 2%):
> > real 0m3.526s
> > user 0m3.156s
> > sys 0m0.364s
> >
> > with -Wl,--compress-debug-sections=zlib:
> > - clang binary: 440 MB
> > - .debug_names: 80MB (18%)

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Pavel Labath via lldb-dev]

On Thu, 14 Jun 2018 at 17:58, Greg Clayton  wrote:
>
>
>
> On Jun 14, 2018, at 9:36 AM, Adrian Prantl  wrote:
>
>
>
> On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev 
>  wrote:
>
> Thank you all. I am going to try to reply to all comments in a single email.
>
> Regarding the  .apple_objc idea, I am afraid the situation is not as
> simple as just flipping a switch.
>
>
> Jonas is currently working on adding the support for DWARF5-style Objective-C 
> accelerator tables to LLVM/LLDB/dsymutil. Based on the assumption that DWARF 
> 4 and earlier are unaffected by any of this, I don't think it's necessary to 
> spend any effort of making the transition smooth. I'm fine with having 
> Objective-C on DWARF 5 broken on trunk for two weeks until Jonas is done 
> adding Objective-C support to the DWARF 5 implementation.

Ideally, I would like to enable the accelerator tables (possibly with
a different version number or something) on DWARF 4 too (on non-apple
targets only). The reason for this is that their absence if causing
large slowdowns when debugging on non-apple platforms, and I wouldn't
want to wait for dwarf 5 for that to go away (I mean no disrespect to
Paul and DWARF 5 effort in general, but even if all of DWARF 5 in llvm
was done tomorrow, there would still be lldb, which hasn't even begun
to look at this version).

That said, if you are working on the Objective C support right now,
then I am happy to wait two weeks or so that we have a full
implementation from the get-go.

> But, other options may be possible as well. What's not clear to me is
> whether these tables couldn't be replaced by extra information in the
> .debug_info section. It seems to me that these tables are trying to
> work around the issue that there is no straight way to go from a
> DW_TAG_structure type DIE describing an ObjC class to it's methods. If
> these methods (their forward declarations) were be present as children
> of the type DIE (as they are for c++ classes), then these tables may
> not be necessary. But maybe (probably) that has already been
> considered and deemed infeasible for some reason. In any case this
> seemed like a thing best left for people who actually work on ObjC
> support to figure out.
>
>
> That's really a question for Greg or Jim — I don't know why the current 
> representation has the Objective-C methods outside of the structs. One reason 
> might be that an interface's implementation can define more methods than are 
> visible in its public interface in the header file, but we already seem to be 
> aware of this and mark the implementation with 
> DW_AT_APPLE_objc_complete_type. I also am not sure that this is the *only* 
> reason for the objc accelerator table. But I'd like to learn.

My observation was based on studying lldb code. The only place where
the objc table is used is in the AppleDWARFIndex::GetObjCMethods
function, which is called from
SymbolFileDWARF::GetObjCMethodDIEOffsets, whose only caller is
DWARFASTParserClang::CompleteTypeFromDWARF, which seems to have a
class DIE as an argument. However, if not all declarations of a
class/interface have access to the full list of methods then this
might be a problem for the approach I suggested.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Greg Clayton via lldb-dev]

> On Jun 14, 2018, at 9:36 AM, Adrian Prantl  wrote:
> 
> 
> 
>> On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev 
>> mailto:llvm-...@lists.llvm.org>> wrote:
>> 
>> Thank you all. I am going to try to reply to all comments in a single email.
>> 
>> Regarding the  .apple_objc idea, I am afraid the situation is not as
>> simple as just flipping a switch.
> 
> Jonas is currently working on adding the support for DWARF5-style Objective-C 
> accelerator tables to LLVM/LLDB/dsymutil. Based on the assumption that DWARF 
> 4 and earlier are unaffected by any of this, I don't think it's necessary to 
> spend any effort of making the transition smooth. I'm fine with having 
> Objective-C on DWARF 5 broken on trunk for two weeks until Jonas is done 
> adding Objective-C support to the DWARF 5 implementation.
> 
> 
>> (If it was, I don't think I would
>> have embarked on this adventure in the first place -- I would just
>> emit .apple_*** everywhere and call it done :)). The issue is that the
>> apple tables have assumptions about the macos debug info distribution
>> model hardcoded in them -- they assume they will either stay in the .o
>> file or be linked by a smart debug-info-aware linker (dsymutil). In
>> particular, this means they are not self-delimiting (no length field
>> as is typical for other dwarf artifacts), so if a linker which is not
>> aware of them would simply concatenate individual .o tables (which elf
>> linkers are really good at), the debugger would have no way to pry
>> them apart. And even if it somehow managed that, it still wouldn't
>> know if the indexes covered all of the compile units in the linked
>> file or only some of them (in case some of the object files were
>> compiled with the tables and some without).
>> 
>> In light of that, I don't think it's worth trying to combine
>> .apple_objc with .debug_names in some way, and it would be much
>> simpler to just extend .debug_names with the necessary information. I
>> think the simplest way of achieving this (one which would require
>> least amount of standard-bending) is to take the index entry for the
>> objc class and add a special attribute to it (DW_IDX_method_list?)
>> with form DW_FORM_blockXXX and just have the references to the method
>> DIEs in the block data. This should make the implementation an almost
>> drop-in for the current .apple_objc functionality (we would still need
>> to figure out what to do with category methods, but it's not clear to
>> me whether lldb actually uses those anywhere).
>> 
>> But, other options may be possible as well. What's not clear to me is
>> whether these tables couldn't be replaced by extra information in the
>> .debug_info section. It seems to me that these tables are trying to
>> work around the issue that there is no straight way to go from a
>> DW_TAG_structure type DIE describing an ObjC class to it's methods. If
>> these methods (their forward declarations) were be present as children
>> of the type DIE (as they are for c++ classes), then these tables may
>> not be necessary. But maybe (probably) that has already been
>> considered and deemed infeasible for some reason. In any case this
>> seemed like a thing best left for people who actually work on ObjC
>> support to figure out.
> 
> That's really a question for Greg or Jim — I don't know why the current 
> representation has the Objective-C methods outside of the structs. One reason 
> might be that an interface's implementation can define more methods than are 
> visible in its public interface in the header file, but we already seem to be 
> aware of this and mark the implementation with 
> DW_AT_APPLE_objc_complete_type. I also am not sure that this is the *only* 
> reason for the objc accelerator table. But I'd like to learn.

New categories to objective C classes can be added and I believe that 
implementations can be spread across multiple files. Not sure why objective C 
doesn't contain its DW_TAG_subprogram inside of its DW_TAG_class_type, but that 
is the way things have been for a while.

> 
> -- adrian
> 
>> As far as the .debug_names size goes, I should also point out that the
>> binary in question was built with -fno-limit-debug-info, which isn't a
>> default setup on linux. I have tried measuring the sizes without that
>> flag and with fission enabled (-gsplit-dwarf) and the results are:
>> without compression:
>> - clang binary: 960 MB
>> - .debug_names: 130 MB (13%)
>> - debug_pubnames: 175 MB (18%)
>> - debug_pubtypes: 204 MB (21%)
>> - median time for setting a breakpoint on non-existent function
>> (variance +/- 2%):
>> real 0m3.526s
>> user 0m3.156s
>> sys 0m0.364s
>> 
>> with -Wl,--compress-debug-sections=zlib:
>> - clang binary: 440 MB
>> - .debug_names: 80MB (18%)
>> - .debug_pubnames: 31 MB (7.2%)
>> - .debug_pubtypes: 42MB (9.5%)
>> - median time for setting a breakpoint on non-existent function:
>> real 0m4.369s
>> user 0m3.948s
>> sys 0m0.416s
>> 
>> So, .debug_names indeed compresses worse than 

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Adrian Prantl via lldb-dev]

> On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev 
>  wrote:
> 
> Thank you all. I am going to try to reply to all comments in a single email.
> 
> Regarding the  .apple_objc idea, I am afraid the situation is not as
> simple as just flipping a switch.

Jonas is currently working on adding the support for DWARF5-style Objective-C 
accelerator tables to LLVM/LLDB/dsymutil. Based on the assumption that DWARF 4 
and earlier are unaffected by any of this, I don't think it's necessary to 
spend any effort of making the transition smooth. I'm fine with having 
Objective-C on DWARF 5 broken on trunk for two weeks until Jonas is done adding 
Objective-C support to the DWARF 5 implementation.


> (If it was, I don't think I would
> have embarked on this adventure in the first place -- I would just
> emit .apple_*** everywhere and call it done :)). The issue is that the
> apple tables have assumptions about the macos debug info distribution
> model hardcoded in them -- they assume they will either stay in the .o
> file or be linked by a smart debug-info-aware linker (dsymutil). In
> particular, this means they are not self-delimiting (no length field
> as is typical for other dwarf artifacts), so if a linker which is not
> aware of them would simply concatenate individual .o tables (which elf
> linkers are really good at), the debugger would have no way to pry
> them apart. And even if it somehow managed that, it still wouldn't
> know if the indexes covered all of the compile units in the linked
> file or only some of them (in case some of the object files were
> compiled with the tables and some without).
> 
> In light of that, I don't think it's worth trying to combine
> .apple_objc with .debug_names in some way, and it would be much
> simpler to just extend .debug_names with the necessary information. I
> think the simplest way of achieving this (one which would require
> least amount of standard-bending) is to take the index entry for the
> objc class and add a special attribute to it (DW_IDX_method_list?)
> with form DW_FORM_blockXXX and just have the references to the method
> DIEs in the block data. This should make the implementation an almost
> drop-in for the current .apple_objc functionality (we would still need
> to figure out what to do with category methods, but it's not clear to
> me whether lldb actually uses those anywhere).
> 
> But, other options may be possible as well. What's not clear to me is
> whether these tables couldn't be replaced by extra information in the
> .debug_info section. It seems to me that these tables are trying to
> work around the issue that there is no straight way to go from a
> DW_TAG_structure type DIE describing an ObjC class to it's methods. If
> these methods (their forward declarations) were be present as children
> of the type DIE (as they are for c++ classes), then these tables may
> not be necessary. But maybe (probably) that has already been
> considered and deemed infeasible for some reason. In any case this
> seemed like a thing best left for people who actually work on ObjC
> support to figure out.

That's really a question for Greg or Jim — I don't know why the current 
representation has the Objective-C methods outside of the structs. One reason 
might be that an interface's implementation can define more methods than are 
visible in its public interface in the header file, but we already seem to be 
aware of this and mark the implementation with DW_AT_APPLE_objc_complete_type. 
I also am not sure that this is the *only* reason for the objc accelerator 
table. But I'd like to learn.

-- adrian

> As far as the .debug_names size goes, I should also point out that the
> binary in question was built with -fno-limit-debug-info, which isn't a
> default setup on linux. I have tried measuring the sizes without that
> flag and with fission enabled (-gsplit-dwarf) and the results are:
> without compression:
> - clang binary: 960 MB
> - .debug_names: 130 MB (13%)
> - debug_pubnames: 175 MB (18%)
> - debug_pubtypes: 204 MB (21%)
> - median time for setting a breakpoint on non-existent function
> (variance +/- 2%):
> real 0m3.526s
> user 0m3.156s
> sys 0m0.364s
> 
> with -Wl,--compress-debug-sections=zlib:
> - clang binary: 440 MB
> - .debug_names: 80MB (18%)
> - .debug_pubnames: 31 MB (7.2%)
> - .debug_pubtypes: 42MB (9.5%)
> - median time for setting a breakpoint on non-existent function:
> real 0m4.369s
> user 0m3.948s
> sys 0m0.416s
> 
> So, .debug_names indeed compresses worse than .debug_pubnames/types,
> but that is not surprising as it has a more condensed encoding to
> begin with (no inline strings). However, even in it's compressed form
> its size is only slightly larger that the two other sections combined
> (while being infinitely more useful). As for the compression, my
> takeaway from this is that compression definitely has a measurable
> impact on startup time, but, on the grand scale of things, the impact
> is not actually that big. And if a 

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Adrian Prantl via lldb-dev]

> On Jan 30, 2018, at 7:49 AM, Pavel Labath  wrote:
> 
> On 30 January 2018 at 15:41, Adrian Prantl  wrote:
>> 
>> 
>>> On Jan 30, 2018, at 7:35 AM, Pavel Labath  wrote:
>>> 
>>> Hello all,
>>> 
>>> I am looking for feedback regarding implementation of the case folding
>>> algorithm for .debug_names hashes.
>>> 
>>> Unlike the apple tables, the .debug_names hashes are computed from
>>> case-folded names (to enable case-insensitive lookups for languages
>>> where that makes sense). The dwarf5 document specifies that the case
>>> folding should be done according the the "Caseless matching" Section
>>> of the Unicode standard (whose implementation is basically a long list
>>> of special cases). While certainly possible, implementing this would
>>> be much more complicated (and would probably make the code a bit
>>> slower) than a simple tolower(3) call. And the benefits of this are
>>> not really clear to me.
>> 
>> Assuming a UTF-8 encoding, will tolower(3) destroy any non-ASCII characters 
>> in the process? In Swift, for example, we allow a wide range of unicode 
>> characters in identifiers and I want to make sure that this doesn't cause 
>> any problems.
>> 
> 
> I'm not sure what it will do out-of-the-box, but I could certainly
> implement it such that it does not touch the fancy characters.
> 
> However, if we already have unicode characters in the input, then it
> may make sense to go all the way and implement the full folding
> algorithm. Because, once we start producing hashes like this, it will
> be hard to switch to being fully standard-compliant (as that would
> invalidate the existing hashes).
> 
> But the question then is: can I assume the input names will be unicode
> (w/utf8 encoding)?

We can make that happen and encode it explicitly in each compile unit:

> 3.1.1 Full and Partial Compilation Unit Entries
> ...
> A DW_AT_use_UTF8 attribute, which is a flag whose presence indicates that all 
> strings (such as the names of declared entities in the source program, or 
> filenames in the line number table) are represented using the UTF-8 
> representation. 

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Jonas Devlieghere via lldb-dev]

> On Jan 30, 2018, at 3:35 PM, Pavel Labath  wrote:
> 
> Hello all,
> 
> I am looking for feedback regarding implementation of the case folding
> algorithm for .debug_names hashes.
> 
> Unlike the apple tables, the .debug_names hashes are computed from
> case-folded names (to enable case-insensitive lookups for languages
> where that makes sense). The dwarf5 document specifies that the case
> folding should be done according the the "Caseless matching" Section
> of the Unicode standard (whose implementation is basically a long list
> of special cases). While certainly possible, implementing this would
> be much more complicated (and would probably make the code a bit
> slower) than a simple tolower(3) call.

Most of the characters in the CaseFolding.txt file seem to be contiguous, so I 
think we should be able to come up with am implementation that’s relatively 
efficient. 

> And the benefits of this are not really clear to me.

Adhering to the standard is a pretty big benefit IMHO, but I know what you 
mean. :-) 

> Do you know if we already make any promises or assumptions about the
> encoding and/or locale of the symbol names (and here I mainly mean the
> names in the debug info metadata, not llvm symbols).
> 
> If we don't already have a policy about this, then I propose to
> implement the case folding via tolower() (which is compatible with the
> full case folding algorithm, as long as one sticks to basic latin
> characters).
> 
> What do you think?

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Pavel Labath via lldb-dev]

On 30 January 2018 at 15:41, Adrian Prantl  wrote:
>
>
>> On Jan 30, 2018, at 7:35 AM, Pavel Labath  wrote:
>>
>> Hello all,
>>
>> I am looking for feedback regarding implementation of the case folding
>> algorithm for .debug_names hashes.
>>
>> Unlike the apple tables, the .debug_names hashes are computed from
>> case-folded names (to enable case-insensitive lookups for languages
>> where that makes sense). The dwarf5 document specifies that the case
>> folding should be done according the the "Caseless matching" Section
>> of the Unicode standard (whose implementation is basically a long list
>> of special cases). While certainly possible, implementing this would
>> be much more complicated (and would probably make the code a bit
>> slower) than a simple tolower(3) call. And the benefits of this are
>> not really clear to me.
>
> Assuming a UTF-8 encoding, will tolower(3) destroy any non-ASCII characters 
> in the process? In Swift, for example, we allow a wide range of unicode 
> characters in identifiers and I want to make sure that this doesn't cause any 
> problems.
>

I'm not sure what it will do out-of-the-box, but I could certainly
implement it such that it does not touch the fancy characters.

However, if we already have unicode characters in the input, then it
may make sense to go all the way and implement the full folding
algorithm. Because, once we start producing hashes like this, it will
be hard to switch to being fully standard-compliant (as that would
invalidate the existing hashes).

But the question then is: can I assume the input names will be unicode
(w/utf8 encoding)?
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Adrian Prantl via lldb-dev]

> On Jan 30, 2018, at 7:35 AM, Pavel Labath  wrote:
> 
> Hello all,
> 
> I am looking for feedback regarding implementation of the case folding
> algorithm for .debug_names hashes.
> 
> Unlike the apple tables, the .debug_names hashes are computed from
> case-folded names (to enable case-insensitive lookups for languages
> where that makes sense). The dwarf5 document specifies that the case
> folding should be done according the the "Caseless matching" Section
> of the Unicode standard (whose implementation is basically a long list
> of special cases). While certainly possible, implementing this would
> be much more complicated (and would probably make the code a bit
> slower) than a simple tolower(3) call. And the benefits of this are
> not really clear to me.

Assuming a UTF-8 encoding, will tolower(3) destroy any non-ASCII characters in 
the process? In Swift, for example, we allow a wide range of unicode characters 
in identifiers and I want to make sure that this doesn't cause any problems.

-- adrian
> 
> Do you know if we already make any promises or assumptions about the
> encoding and/or locale of the symbol names (and here I mainly mean the
> names in the debug info metadata, not llvm symbols).
> 
> If we don't already have a policy about this, then I propose to
> implement the case folding via tolower() (which is compatible with the
> full case folding algorithm, as long as one sticks to basic latin
> characters).
> 
> What do you think?

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Pavel Labath via lldb-dev]

Hello all,

I am looking for feedback regarding implementation of the case folding
algorithm for .debug_names hashes.

Unlike the apple tables, the .debug_names hashes are computed from
case-folded names (to enable case-insensitive lookups for languages
where that makes sense). The dwarf5 document specifies that the case
folding should be done according the the "Caseless matching" Section
of the Unicode standard (whose implementation is basically a long list
of special cases). While certainly possible, implementing this would
be much more complicated (and would probably make the code a bit
slower) than a simple tolower(3) call. And the benefits of this are
not really clear to me.

Do you know if we already make any promises or assumptions about the
encoding and/or locale of the symbol names (and here I mainly mean the
names in the debug info metadata, not llvm symbols).

If we don't already have a policy about this, then I propose to
implement the case folding via tolower() (which is compatible with the
full case folding algorithm, as long as one sticks to basic latin
characters).

What do you think?
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Frédéric Riss via lldb-dev]

> On Jan 18, 2018, at 7:52 AM, Pavel Labath via lldb-dev 
>  wrote:
> 
> Thank you for all the responses. Unfortunately I wasn't able to make
> any progress on creating the patches today. I'll be sure to add
> everyone who expressed interest here to the phabricator diff once I
> have them ready.
> 
> Jonas, do you have any dsymutil patches I can look at? I am interested
> in seeing what kind of interfaces are you using, particularly on the
> reading side. I think the current DWARFAcceleratorTable interface will
> need to be redesigned a bit if we want it to fit the dwarf5 tables as
> well.

Dsymutil never reads accelerator tables. It rebuilds them while walking the 
debug_info section.

Fred

> On 17 January 2018 at 20:52, Eric Christopher via lldb-dev
>  wrote:
>> FWIW I'm completely on board with everything Adrian has said in this thread
>> :)
>> 
>> -eric
>> 
>> On Wed, Jan 17, 2018 at 11:00 AM Adrian Prantl via llvm-dev
>>  wrote:
>>> 
>>> 
>>> 
 On Jan 17, 2018, at 9:20 AM, Jonas Devlieghere via llvm-dev
  wrote:
 
 As mentioned by Adrian in the comment you linked, I too am looking at
 DWARFv5
 accelerator tables in LLVM.
 
 To give you some background: my motivation is that I want to upstream
 support
 for (Apple style) accelerator tables in llvm-dsymutil,
>>> 
>>> Some background for the benefit of everyone who may not be aware of the
>>> genesis of the DWARF v5 accelerator tables:
>>> 
>>> DWARF v5 accelerator tables are a direct evolution of the "Apple"
>>> accelerator tables that are implemented in LLVM (after going through the
>>> standardization process and being reworked/generalized there), so we hope
>>> that the implementation can at least share some common interfaces with the
>>> existing Apple accelerator table implementation where this makes sense.
>>> 
>>> -- adrian
>>> ___
>>> LLVM Developers mailing list
>>> llvm-...@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Pavel Labath via lldb-dev]

Thank you for all the responses. Unfortunately I wasn't able to make
any progress on creating the patches today. I'll be sure to add
everyone who expressed interest here to the phabricator diff once I
have them ready.

Jonas, do you have any dsymutil patches I can look at? I am interested
in seeing what kind of interfaces are you using, particularly on the
reading side. I think the current DWARFAcceleratorTable interface will
need to be redesigned a bit if we want it to fit the dwarf5 tables as
well.

On 17 January 2018 at 20:52, Eric Christopher via lldb-dev
 wrote:
> FWIW I'm completely on board with everything Adrian has said in this thread
> :)
>
> -eric
>
> On Wed, Jan 17, 2018 at 11:00 AM Adrian Prantl via llvm-dev
>  wrote:
>>
>>
>>
>> > On Jan 17, 2018, at 9:20 AM, Jonas Devlieghere via llvm-dev
>> >  wrote:
>> >
>> > As mentioned by Adrian in the comment you linked, I too am looking at
>> > DWARFv5
>> > accelerator tables in LLVM.
>> >
>> > To give you some background: my motivation is that I want to upstream
>> > support
>> > for (Apple style) accelerator tables in llvm-dsymutil,
>>
>> Some background for the benefit of everyone who may not be aware of the
>> genesis of the DWARF v5 accelerator tables:
>>
>> DWARF v5 accelerator tables are a direct evolution of the "Apple"
>> accelerator tables that are implemented in LLVM (after going through the
>> standardization process and being reworked/generalized there), so we hope
>> that the implementation can at least share some common interfaces with the
>> existing Apple accelerator table implementation where this makes sense.
>>
>> -- adrian
>> ___
>> LLVM Developers mailing list
>> llvm-...@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Eric Christopher via lldb-dev]

FWIW I'm completely on board with everything Adrian has said in this thread
:)

-eric

On Wed, Jan 17, 2018 at 11:00 AM Adrian Prantl via llvm-dev <
llvm-...@lists.llvm.org> wrote:

>
>
> > On Jan 17, 2018, at 9:20 AM, Jonas Devlieghere via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
> >
> > As mentioned by Adrian in the comment you linked, I too am looking at
> DWARFv5
> > accelerator tables in LLVM.
> >
> > To give you some background: my motivation is that I want to upstream
> support
> > for (Apple style) accelerator tables in llvm-dsymutil,
>
> Some background for the benefit of everyone who may not be aware of the
> genesis of the DWARF v5 accelerator tables:
>
> DWARF v5 accelerator tables are a direct evolution of the "Apple"
> accelerator tables that are implemented in LLVM (after going through the
> standardization process and being reworked/generalized there), so we hope
> that the implementation can at least share some common interfaces with the
> existing Apple accelerator table implementation where this makes sense.
>
> -- adrian
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 9:20 AM, Jonas Devlieghere via llvm-dev 
>  wrote:
> 
> As mentioned by Adrian in the comment you linked, I too am looking at DWARFv5
> accelerator tables in LLVM. 
> 
> To give you some background: my motivation is that I want to upstream support
> for (Apple style) accelerator tables in llvm-dsymutil, 

Some background for the benefit of everyone who may not be aware of the genesis 
of the DWARF v5 accelerator tables:

DWARF v5 accelerator tables are a direct evolution of the "Apple" accelerator 
tables that are implemented in LLVM (after going through the standardization 
process and being reworked/generalized there), so we hope that the 
implementation can at least share some common interfaces with the existing 
Apple accelerator table implementation where this makes sense.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Jonas Devlieghere via lldb-dev]

Hi Pavel,

As mentioned by Adrian in the comment you linked, I too am looking at DWARFv5
accelerator tables in LLVM. 

To give you some background: my motivation is that I want to upstream support
for (Apple style) accelerator tables in llvm-dsymutil, which is currently
missing because the way they are generated is slightly different. As this
requires making changes the current code, I wanted to consider the impact on
the standard's tables. I have some old patches from Fred (Riss) that take 
different
approaches to this, but I have nothing concrete yet for the v5 part.

Ultimately my goal is to share as much logic as possible between the Apple and
DWARFv5 implementations, both on the reading and generation side of things.

Your timeline seems sensible. Adding support in llvm-dwarfdump will be very
useful. I look forward to reviewing this. 

How do you feel about creating a [WIP] diff for (2). I'm very excited about
bundling our efforts on this. Let me know what I can do if there's anything
else I can help with. 

Cheers,
Jonas

> On 17 Jan 2018, at 16:40, Victor Leschuk  wrote:
> 
> Hello, I hope I will have time to help you with that. I discussed
> dwarfv5 .debug_names implementation with involved party from RH. Anyway
> even if can't help much could you keep me in the loop please?
> 
> 
> On 01/17/2018 07:13 PM, Pavel Labath via llvm-dev wrote:
>> Hello all,
>> 
>> In  it was brought up that
>> there are at least two parties interested in having DWARF5 accelerator
>> tables implemented, so I'm writing this email to see if there's anyone
>> else interested in this topic, and to try to synchronize our efforts.
>> 
>> Our interest for this stems from a desire to make dwarf parsing fast
>> on non-apple targets (specifically android). As you may know, apple
>> targets already use a non-standard accelerator table format, which was
>> a precursor to the dwarf5 one. Due to other differences in debug info
>> deployment model, the apple tables are not directly applicable to
>> other targets, so instead of trying to make them fit, we chose to go
>> with the standard approach.
>> 
>> I personally will have some time to work on this this quarter or two,
>> and my plan is roughly the following:
>> 1. add .debug_names support to llvm-dwarfdump via the DebugInfo
>> library (to enable testing of the table generation)
>> 2. add .debug_names generation support (not enabled by default)
>> 3. add .debug_names support to lldb
>> 4. validate all three things work together
>> 5. hook up .debug_names to clang's -glldb flag.
>> 6. add .debug_names support to lld (accelerator table merging)
>> 
>> Right now I have (1) roughly implemented, and I think I'll be able to
>> put it up for review in a couple of days (although I expect the review
>> will go through several revisions before being accepted). I also have
>> a very basic implementation of (2), but this is still quite far from
>> being upstreamable.
>> 
>> So, my question is whether anyone is planning to work, or maybe
>> working already on dwarf5 accelerator tables? Help with reviewing
>> patches would also be greatly appreciated. If you have any questions
>> or concerns, let me know.
>> 
>> regards,
>> Pavel
>> ___
>> LLVM Developers mailing list
>> llvm-...@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> -- 
> Best Regards,
> 
> Victor Leschuk | Software Engineer | Access Softek
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Victor Leschuk via lldb-dev]

Hello, I hope I will have time to help you with that. I discussed
dwarfv5 .debug_names implementation with involved party from RH. Anyway
even if can't help much could you keep me in the loop please?


On 01/17/2018 07:13 PM, Pavel Labath via llvm-dev wrote:
> Hello all,
>
> In  it was brought up that
> there are at least two parties interested in having DWARF5 accelerator
> tables implemented, so I'm writing this email to see if there's anyone
> else interested in this topic, and to try to synchronize our efforts.
>
> Our interest for this stems from a desire to make dwarf parsing fast
> on non-apple targets (specifically android). As you may know, apple
> targets already use a non-standard accelerator table format, which was
> a precursor to the dwarf5 one. Due to other differences in debug info
> deployment model, the apple tables are not directly applicable to
> other targets, so instead of trying to make them fit, we chose to go
> with the standard approach.
>
> I personally will have some time to work on this this quarter or two,
> and my plan is roughly the following:
> 1. add .debug_names support to llvm-dwarfdump via the DebugInfo
> library (to enable testing of the table generation)
> 2. add .debug_names generation support (not enabled by default)
> 3. add .debug_names support to lldb
> 4. validate all three things work together
> 5. hook up .debug_names to clang's -glldb flag.
> 6. add .debug_names support to lld (accelerator table merging)
>
> Right now I have (1) roughly implemented, and I think I'll be able to
> put it up for review in a couple of days (although I expect the review
> will go through several revisions before being accepted). I also have
> a very basic implementation of (2), but this is still quite far from
> being upstreamable.
>
> So, my question is whether anyone is planning to work, or maybe
> working already on dwarf5 accelerator tables? Help with reviewing
> patches would also be greatly appreciated. If you have any questions
> or concerns, let me know.
>
> regards,
> Pavel
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 
Best Regards,

Victor Leschuk | Software Engineer | Access Softek

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev