Re: [Dwarf-Discuss] Using DWARF for C++ runtime reflection

2018-03-07 Thread Jason Nyberg 
Roman, For your given example:

  T   var1;
DW_AT_name(T)   var2;

  typeid(var1)  ==  typeid (var2) ?

I think the type "T" found in the Dwarf info would have to have come from
the same compile unit that the "normally declared" type "T" came from, i.e.
your "DW_AT_name(...)" function would have to guarantee that. There's
nothing stopping the same type name from being used in different ways in
different source files, as long as they don't collide in the same compile
unit.

Your project sounds very interesting to me because I've *also* been using
Dwarf for reflection; in my case I combine (all in C) a minimalistic
language, Dwarf comprehension, and libffi such that shared libraries become
"native" extensions to my simple language just by importing them.
Eventually I'll hoist the whole thing up into C++ too...

On Tue, Mar 6, 2018 at 2:55 PM Michael Eager  wrote:

> > Yes, assuming that the compiler generates a valid source type.
> > In some
> > cases, particularly with template classes, this may not be the
> case.
> >
> >
> >   So DWARF does not give any guarantees and it's up to compiler
> > vendor to decide about DW_AT_name?
> >
> >
> > DWARF doesn't give guarantees, just suggestions. Practically speaking
> > there's certainly benefit to compilers ensuring they produce a
> > consistent name across different translation units - and potentially
> > even a consistent name across compilers (so that code built with
> > different compilers and then debugged can behave consistently/well). (or
> > consistent enough for a debugger to cope with it - but I think as Daniel
> > pointed out, having the debugger have to decompose the name and do
> > various type equivalences is painful - which might lean one towards
> > trying to produce more consistent names across GCC and Clang - the few
> > cases I know of (enums, for example) could probably be improved just for
> > the compiler self-consistency aspect and incidentally improve the cross
> > compiler consistency anyway)
>
> Exactly.  In many areas, the DWARF Standard is permissive.  There are
> specific constraints which are mentioned, but the producer is given a
> lot of latitude whether to generate complete or incomplete, good or poor
> DWARF.  DWARF guarantees the organization and structure of the data.
>
> There is a comment about names on the DWARF Wiki under Best Practices:
> http://wiki.dwarfstd.org/index.php?title=Best_Practices
>
>
> --
> Michael Eagerea...@eagercon.com
> 1960 Park Blvd., Palo Alto, CA 94306
> ___
> Dwarf-Discuss mailing list
> Dwarf-Discuss@lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org
>
___
Dwarf-Discuss mailing list
Dwarf-Discuss@lists.dwarfstd.org
http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org

Re: [Dwarf-Discuss] Using DWARF for C++ runtime reflection

2018-03-06 Thread Michael Eager 
Yes, assuming that the compiler generates a valid source type. 
In some

cases, particularly with template classes, this may not be the case.


  So DWARF does not give any guarantees and it's up to compiler
vendor to decide about DW_AT_name?


DWARF doesn't give guarantees, just suggestions. Practically speaking 
there's certainly benefit to compilers ensuring they produce a 
consistent name across different translation units - and potentially 
even a consistent name across compilers (so that code built with 
different compilers and then debugged can behave consistently/well). (or 
consistent enough for a debugger to cope with it - but I think as Daniel 
pointed out, having the debugger have to decompose the name and do 
various type equivalences is painful - which might lean one towards 
trying to produce more consistent names across GCC and Clang - the few 
cases I know of (enums, for example) could probably be improved just for 
the compiler self-consistency aspect and incidentally improve the cross 
compiler consistency anyway)


Exactly.  In many areas, the DWARF Standard is permissive.  There are
specific constraints which are mentioned, but the producer is given a
lot of latitude whether to generate complete or incomplete, good or poor
DWARF.  DWARF guarantees the organization and structure of the data.

There is a comment about names on the DWARF Wiki under Best Practices:
http://wiki.dwarfstd.org/index.php?title=Best_Practices


--
Michael Eagerea...@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306
___
Dwarf-Discuss mailing list
Dwarf-Discuss@lists.dwarfstd.org
http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org

Re: [Dwarf-Discuss] Using DWARF for C++ runtime reflection

2018-03-06 Thread David Blaikie 
On Tue, Mar 6, 2018 at 11:21 AM Roman Popov  wrote:

> It is certainly an interesting use case.  I haven't see this done before,
>> but since DWARF gives descriptions of classes, it seems reasonable.
>
> I've found some attempts in this direction on github. So certainly I'm not
> the first one.  My application is hardware modeling, I make something
> similar to Chisel (https://chisel.eecs.berkeley.edu/) but in C++, instead
> of Scala.
>
>
>> The AT_name is the declared name of the type in the source, not the name
>>
> of an underlying type.  For example, if your source is ...
>
>
> Yes, I forgot about aliases. But since DWARF allows to go from alias to
> underlying type, it's not an issue for my application.
>
> Yes, assuming that the compiler generates a valid source type.  In some
>> cases, particularly with template classes, this may not be the case.
>
>
>  So DWARF does not give any guarantees and it's up to compiler vendor to
> decide about DW_AT_name?
>

DWARF doesn't give guarantees, just suggestions. Practically speaking
there's certainly benefit to compilers ensuring they produce a consistent
name across different translation units - and potentially even a consistent
name across compilers (so that code built with different compilers and then
debugged can behave consistently/well). (or consistent enough for a
debugger to cope with it - but I think as Daniel pointed out, having the
debugger have to decompose the name and do various type equivalences is
painful - which might lean one towards trying to produce more consistent
names across GCC and Clang - the few cases I know of (enums, for example)
could probably be improved just for the compiler self-consistency aspect
and incidentally improve the cross compiler consistency anyway)


>
>
> 2018-03-06 11:08 GMT-08:00 Michael Eager :
>
>> On 03/06/2018 10:37 AM, Roman Popov wrote:
>>
>>> Hi everyone,
>>> I'm working on a dynamic analysis tool that needs runtime reflection in
>>> C++. Since C++ has no standardized runtime reflection, I'm using DWARF as a
>>> source of reflection metadata.
>>>
>>> Is it a legitimate use-case from DWARF standard point of view?
>>>
>>
>> It is certainly an interesting use case.  I haven't see this done
>> before, but since DWARF gives descriptions of classes, it seems
>> reasonable.
>>
>> It has been working great for me until I've upgraded to latest g++ and
>>> clang compilers. Those compilers produce ambiguous names for some template
>>> instantiations, so my tools no longer work. So I start to wonder: whether
>>> it me misusing DWARF, or it is compilers that are buggy.
>>>
>>> In particular, what is the DWARF answer for these questions:
>>>
>>> 1.  Does DWARF guarantees that:
>>>
>>>   typeid(T1) == typeid(T2)==> DW_AT_name(T1)  == DW_AT_name(T2)
>>>   typeid(T1) != typeid(T2) ==> DW_AT_name(T1)  != DW_AT_name(T2)
>>>
>>> i.e. Is DW_AT_name an unique identifier for type?
>>>
>>
>> The AT_name is the declared name of the type in the source, not the name
>> of an underlying type.  For example, if your source is
>>
>>   typedef int int_name_1;
>>   typedef int int_name_2;
>>
>>   int_name_1 name_1;
>>   int_name_2 name_2;
>>
>> The (edited) DWARF for this
>>
>>  <1d>: (DW_TAG_typedef)
>> <1e>   DW_AT_name: int_name_1
>> <24>   DW_AT_type: <0x28>
>>  <28>: (DW_TAG_base_type)
>> <2b>   DW_AT_name: int
>>  <2f>: (DW_TAG_typedef)
>> <30>   DW_AT_name: int_name_2
>> <36>   DW_AT_type: <0x28>
>>  <3a>: (DW_TAG_variable)
>> <3b>   DW_AT_name: name_1
>> <41>   DW_AT_type: <0x1d>
>>  <4f>: (DW_TAG_variable)
>> <50>   DW_AT_name: name_2
>> <56>   DW_AT_type: <0x2f>
>>
>> typeid(name_1) == typeid(name_2) == "int" which is different from
>> AT_name(name_1) or AT_name(name_2).
>>
>>
>> 2.  Does DWARF guarantees that DW_AT_name(T)  is a valid source language
>>> name?
>>>
>>
>> Yes, assuming that the compiler generates a valid source type.  In some
>> cases, particularly with template classes, this may not be the case.
>>
>> i.e. If I copy string from DW_AT_name to source code, and compile it with
>>> the same compiler that produced DWARF, will it produce the same type?
>>>
>>>  T   var1;
>>>  DW_AT_name(T)   var2;
>>>
>>>typeid(var1)  ==  typeid (var2) ?
>>>
>>
>> I expect that it would.
>>
>> --
>> Michael Eagerea...@eagercon.com
>> 1960 Park Blvd., Palo Alto, CA 94306
>>
>
> ___
> Dwarf-Discuss mailing list
> Dwarf-Discuss@lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org
>
___
Dwarf-Discuss mailing list
Dwarf-Discuss@lists.dwarfstd.org
http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org

Re: [Dwarf-Discuss] Using DWARF for C++ runtime reflection

2018-03-06 Thread Roman Popov 
>
> It is certainly an interesting use case.  I haven't see this done before,
> but since DWARF gives descriptions of classes, it seems reasonable.

I've found some attempts in this direction on github. So certainly I'm not
the first one.  My application is hardware modeling, I make something
similar to Chisel (https://chisel.eecs.berkeley.edu/) but in C++, instead
of Scala.


> The AT_name is the declared name of the type in the source, not the name
> of an underlying type.  For example, if your source is ...


Yes, I forgot about aliases. But since DWARF allows to go from alias to
underlying type, it's not an issue for my application.

Yes, assuming that the compiler generates a valid source type.  In some
> cases, particularly with template classes, this may not be the case.


 So DWARF does not give any guarantees and it's up to compiler vendor to
decide about DW_AT_name?


2018-03-06 11:08 GMT-08:00 Michael Eager :

> On 03/06/2018 10:37 AM, Roman Popov wrote:
>
>> Hi everyone,
>> I'm working on a dynamic analysis tool that needs runtime reflection in
>> C++. Since C++ has no standardized runtime reflection, I'm using DWARF as a
>> source of reflection metadata.
>>
>> Is it a legitimate use-case from DWARF standard point of view?
>>
>
> It is certainly an interesting use case.  I haven't see this done
> before, but since DWARF gives descriptions of classes, it seems
> reasonable.
>
> It has been working great for me until I've upgraded to latest g++ and
>> clang compilers. Those compilers produce ambiguous names for some template
>> instantiations, so my tools no longer work. So I start to wonder: whether
>> it me misusing DWARF, or it is compilers that are buggy.
>>
>> In particular, what is the DWARF answer for these questions:
>>
>> 1.  Does DWARF guarantees that:
>>
>>   typeid(T1) == typeid(T2)==> DW_AT_name(T1)  == DW_AT_name(T2)
>>   typeid(T1) != typeid(T2) ==> DW_AT_name(T1)  != DW_AT_name(T2)
>>
>> i.e. Is DW_AT_name an unique identifier for type?
>>
>
> The AT_name is the declared name of the type in the source, not the name
> of an underlying type.  For example, if your source is
>
>   typedef int int_name_1;
>   typedef int int_name_2;
>
>   int_name_1 name_1;
>   int_name_2 name_2;
>
> The (edited) DWARF for this
>
>  <1d>: (DW_TAG_typedef)
> <1e>   DW_AT_name: int_name_1
> <24>   DW_AT_type: <0x28>
>  <28>: (DW_TAG_base_type)
> <2b>   DW_AT_name: int
>  <2f>: (DW_TAG_typedef)
> <30>   DW_AT_name: int_name_2
> <36>   DW_AT_type: <0x28>
>  <3a>: (DW_TAG_variable)
> <3b>   DW_AT_name: name_1
> <41>   DW_AT_type: <0x1d>
>  <4f>: (DW_TAG_variable)
> <50>   DW_AT_name: name_2
> <56>   DW_AT_type: <0x2f>
>
> typeid(name_1) == typeid(name_2) == "int" which is different from
> AT_name(name_1) or AT_name(name_2).
>
>
> 2.  Does DWARF guarantees that DW_AT_name(T)  is a valid source language
>> name?
>>
>
> Yes, assuming that the compiler generates a valid source type.  In some
> cases, particularly with template classes, this may not be the case.
>
> i.e. If I copy string from DW_AT_name to source code, and compile it with
>> the same compiler that produced DWARF, will it produce the same type?
>>
>>  T   var1;
>>  DW_AT_name(T)   var2;
>>
>>typeid(var1)  ==  typeid (var2) ?
>>
>
> I expect that it would.
>
> --
> Michael Eagerea...@eagercon.com
> 1960 Park Blvd., Palo Alto, CA 94306
>
___
Dwarf-Discuss mailing list
Dwarf-Discuss@lists.dwarfstd.org
http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org

Re: [Dwarf-Discuss] Using DWARF for C++ runtime reflection

2018-03-06 Thread Michael Eager 

On 03/06/2018 10:37 AM, Roman Popov wrote:

Hi everyone,
I'm working on a dynamic analysis tool that needs runtime reflection in 
C++. Since C++ has no standardized runtime reflection, I'm using DWARF 
as a source of reflection metadata.


Is it a legitimate use-case from DWARF standard point of view?


It is certainly an interesting use case.  I haven't see this done
before, but since DWARF gives descriptions of classes, it seems
reasonable.

It has been working great for me until I've upgraded to latest g++ and 
clang compilers. Those compilers produce ambiguous names for some 
template instantiations, so my tools no longer work. So I start to 
wonder: whether it me misusing DWARF, or it is compilers that are buggy.


In particular, what is the DWARF answer for these questions:

1.  Does DWARF guarantees that:

      typeid(T1) == typeid(T2)    ==> DW_AT_name(T1)  == DW_AT_name(T2)
      typeid(T1) != typeid(T2)     ==> DW_AT_name(T1)  != DW_AT_name(T2)

i.e. Is DW_AT_name an unique identifier for type?


The AT_name is the declared name of the type in the source, not the name
of an underlying type.  For example, if your source is

  typedef int int_name_1;
  typedef int int_name_2;

  int_name_1 name_1;
  int_name_2 name_2;

The (edited) DWARF for this

 <1d>: (DW_TAG_typedef)
<1e>   DW_AT_name: int_name_1
<24>   DW_AT_type: <0x28>
 <28>: (DW_TAG_base_type)
<2b>   DW_AT_name: int
 <2f>: (DW_TAG_typedef)
<30>   DW_AT_name: int_name_2
<36>   DW_AT_type: <0x28>
 <3a>: (DW_TAG_variable)
<3b>   DW_AT_name: name_1
<41>   DW_AT_type: <0x1d>
 <4f>: (DW_TAG_variable)
<50>   DW_AT_name: name_2
<56>   DW_AT_type: <0x2f>

typeid(name_1) == typeid(name_2) == "int" which is different from 
AT_name(name_1) or AT_name(name_2).



2.  Does DWARF guarantees that DW_AT_name(T)  is a valid source language 
name?


Yes, assuming that the compiler generates a valid source type.  In some
cases, particularly with template classes, this may not be the case.

i.e. If I copy string from DW_AT_name to source code, and compile it 
with the same compiler that produced DWARF, will it produce the same type?


     T   var1;
     DW_AT_name(T)   var2;

   typeid(var1)  ==  typeid (var2) ?


I expect that it would.

--
Michael Eagerea...@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306
___
Dwarf-Discuss mailing list
Dwarf-Discuss@lists.dwarfstd.org
http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org