Re: [lldb-dev] [Lldb-commits] [lldb] r360757 - Group forward declarations in one namespace lldb_private {}

2019-05-16 Thread Jim Ingham via lldb-dev


> On May 16, 2019, at 2:24 AM, Pavel Labath  wrote:
> 
> On 16/05/2019 01:10, Jim Ingham via lldb-dev wrote:
>> When you add to them you are often adding some larger feature which would 
>> have required a rebuild anyway, and they go long times with no change...  I 
>> have never found the rebuild required when these files are touched to be a 
>> drag on my productivity.  And I really appreciate their convenience.
>> But thanks for your friendly advice.
>> Jim
> 
> I don't want to make a big deal out of it, but I'm also not a fan of the 
> lldb-forward header. My two main reasons are:
> - it's inconsistent with the rest of llvm, which does not have any headers of 
> such sort (LLVM.h, which we talked about last time, is the only thing 
> remotely similar, but that's still has a much more narrow scope)
> - it makes it easier to violate layering. Eg. right now I can type something 
> like:
> void do_stuff_with(Target *);
> in to a "Utility" header, and it will compile just fine because it will have 
> the forward-declaration of the Target class available even though nothing in 
> Utility should know about that class.
> 
> Neither of these is a big problem: this is not the most important thing we 
> differ from llvm, and also the layering violation will become obvious once 
> you start to implement the "do_stuff_with" function (because you will 
> hopefully need to include the right header to get the full definition).

For sure, lldb-forward.h should only have forward declarations (or SP 
declarations) - as is the case now.  So you can't do anything but pass an 
object along with only lldb-forward.  I like it because it encourages not 
including the actual definition header files in other header files that don't 
need to see implementations.  It provides the set of names you can refer to 
that lldb provides, so it is easy to make your interface declarations work w/o 
being tempted to grab too much.  The dictate of living off lldb-forward.h 
actually helps to remove unnecessary dependencies.  The common defines have the 
other benefit that if you know you should be including these you will more 
easily learn to look there and see LLDB_INVALID_ADDRESS rather than inventing 
your own local invalid address marker...

> However, for these reasons, I would prefer if we got rid of this header, or 
> at least moved towards a world where we have one forward-declaring header for 
> each top-level module (so "lldb/Utility/forward.h", would forward-declare 
> only Utility stuff, etc.).
> 

I'm not opposed to the latter option, though I'm not convinced it would be 
worth the churn.

Jim


> pl

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [Lldb-commits] [lldb] r360757 - Group forward declarations in one namespace lldb_private {}

2019-05-15 Thread Jim Ingham via lldb-dev
When you add to them you are often adding some larger feature which would have 
required a rebuild anyway, and they go long times with no change...  I have 
never found the rebuild required when these files are touched to be a drag on 
my productivity.  And I really appreciate their convenience.

But thanks for your friendly advice.

Jim

> On May 15, 2019, at 3:30 PM, Eric Christopher  wrote:
> 
> The dependency edges caused by the file means that any time anyone
> touches that file everything that depends on it is rebuilt. The way
> it's included basically means you're doing a full build any time you
> touch it. It's who is depending on this file or any file this file
> includes, not the other way around.
> 
> If no one is expected to touch it then it's probably ok. It looks like
> each of the files (lldb/include/lldb/lldb-*.h) is touched ~ once every
> few weeks. Mostly I was looking at the edit-compile-debug cycle of the
> person working on the patch. Means about once a month someone working
> on a patch has to do a much more significant set of rebuilds than
> they'd already have to do because of their patch rather than a much
> more minimal set of rebuilds.
> 
> As far as having a convenient place to put public headers I do agree,
> but I'd probably prefer to see that via autogenerated documentation
> rather than a set of forward declarations in a single header.
> 
> Anyhow, your mileage may vary. Just giving the thoughts around the
> rest of the llvm project :)
> 
> -eric
> 
> On Wed, May 15, 2019 at 3:13 PM Jim Ingham  wrote:
>> 
>> This is a common header file that specifies either common enums & #defines 
>> in lldb, or forward declarations of the common classes in lldb.  They don't 
>> depend on any other header files in the system, they just establish the 
>> common names you should use.  So the dependency analysis introduced by these 
>> files should be trivial - they really don't depend on anything else.  And 
>> the only time you change them is when you add a new public type or define to 
>> lldb, which doesn't happen very often.
>> 
>> But it is very convenient to have a common global set of names for the 
>> public entities in lldb, that can be looked up in one place, etc.  You can't 
>> actually USE any of the classes from these header files, so they don't 
>> obscure what other source files actually use - one problem with the big 
>> mongo "Cocoa.h" header file pattern.  If you want to get the class 
>> definition of anything, you have to include the specific .h file for that 
>> thing.  This is just a list of forward declarations.  Switching to
>> 
>> I haven't done your experiment, but if "I depend on a header that either 
>> includes no other files or straight includes another file" really is a 
>> significant burden on the dependency analysis, then I'm not sure this 
>> pattern is actually the problem.
>> 
>> Jim
>> 
>>> On May 15, 2019, at 1:55 PM, Eric Christopher  wrote:
>>> 
>>> A couple of perspectives FWIW:
>>> 
>>> a) Header file includes increase overall dependencies which affects
>>> incremental rebuild time whenever you touch something. Especially more
>>> when that header is included into multiple headers that are then
>>> included into an array of translation units
>>> b) Having each file contain the forward declarations it needs from the
>>> project and no more also will help both compile time and understanding
>>> what's used in any particular header.
>>> 
>>> As a related exercise:
>>> 
>>> https://twitter.com/echristo/status/1116609586004316160
>>> 
>>> basically has removing a couple of transitive dependencies saving in
>>> excess of 70% of the incremental rebuild time after touching a file.
>>> This seems significant. :)
>>> 
>>> -eric
>>> 
>>> On Wed, May 15, 2019 at 10:49 AM Jim Ingham via lldb-dev
>>>  wrote:
>>>> 
>>>> This commit makes things look a little cleaner, but do we need any of 
>>>> these forward declarations at all?
>>>> 
>>>> Most of these files either directly include lldb-forward.h or get it from 
>>>> lldb-types.h or lldb-defines.h (which includes lldb-types.h).  As a 
>>>> general practice we've been getting all the lldb_private type forward 
>>>> declarations this way rather than cluttering up the .h files with 
>>>> individual forward declarations.  If we are going to keep doing it that 
>>>> way - which I can't see any reason not to do, then we should just delete 
>>

Re: [lldb-dev] [Lldb-commits] [lldb] r360757 - Group forward declarations in one namespace lldb_private {}

2019-05-15 Thread Jim Ingham via lldb-dev


> On May 15, 2019, at 3:13 PM, Jim Ingham  wrote:
> 
> This is a common header file that specifies either common enums & #defines in 
> lldb, or forward declarations of the common classes in lldb.  They don't 
> depend on any other header files in the system, they just establish the 
> common names you should use.  So the dependency analysis introduced by these 
> files should be trivial - they really don't depend on anything else.  And the 
> only time you change them is when you add a new public type or define to 
> lldb, which doesn't happen very often.
> 
> But it is very convenient to have a common global set of names for the public 
> entities in lldb, that can be looked up in one place, etc.  You can't 
> actually USE any of the classes from these header files, so they don't 
> obscure what other source files actually use - one problem with the big mongo 
> "Cocoa.h" header file pattern.  If you want to get the class definition of 
> anything, you have to include the specific .h file for that thing.  This is 
> just a list of forward declarations.  Switching to 

individual forward declares uglifies the code - as these examples show.

> 
> I haven't done your experiment, but if "I depend on a header that either 
> includes no other files or straight includes another file" really is a 
> significant burden on the dependency analysis, then I'm not sure this pattern 
> is actually the problem.
> 
> Jim
> 
>> On May 15, 2019, at 1:55 PM, Eric Christopher  wrote:
>> 
>> A couple of perspectives FWIW:
>> 
>> a) Header file includes increase overall dependencies which affects
>> incremental rebuild time whenever you touch something. Especially more
>> when that header is included into multiple headers that are then
>> included into an array of translation units
>> b) Having each file contain the forward declarations it needs from the
>> project and no more also will help both compile time and understanding
>> what's used in any particular header.
>> 
>> As a related exercise:
>> 
>> https://twitter.com/echristo/status/1116609586004316160
>> 
>> basically has removing a couple of transitive dependencies saving in
>> excess of 70% of the incremental rebuild time after touching a file.
>> This seems significant. :)
>> 
>> -eric
>> 
>> On Wed, May 15, 2019 at 10:49 AM Jim Ingham via lldb-dev
>>  wrote:
>>> 
>>> This commit makes things look a little cleaner, but do we need any of these 
>>> forward declarations at all?
>>> 
>>> Most of these files either directly include lldb-forward.h or get it from 
>>> lldb-types.h or lldb-defines.h (which includes lldb-types.h).  As a general 
>>> practice we've been getting all the lldb_private type forward declarations 
>>> this way rather than cluttering up the .h files with individual forward 
>>> declarations.  If we are going to keep doing it that way - which I can't 
>>> see any reason not to do, then we should just delete all these forward 
>>> defines, right?
>>> 
>>> Jim
>>> 
>>> 
>>>> On May 15, 2019, at 2:15 AM, Fangrui Song via lldb-commits 
>>>>  wrote:
>>>> 
>>>> Author: maskray
>>>> Date: Wed May 15 02:15:13 2019
>>>> New Revision: 360757
>>>> 
>>>> URL: http://llvm.org/viewvc/llvm-project?rev=360757=rev
>>>> Log:
>>>> Group forward declarations in one namespace lldb_private {}
>>>> 
>>>> Modified:
>>>>  lldb/trunk/include/lldb/Core/Address.h
>>>>  lldb/trunk/include/lldb/Core/AddressRange.h
>>>>  lldb/trunk/include/lldb/Core/AddressResolver.h
>>>>  lldb/trunk/include/lldb/Core/AddressResolverFileLine.h
>>>>  lldb/trunk/include/lldb/Core/AddressResolverName.h
>>>>  lldb/trunk/include/lldb/Core/Communication.h
>>>>  lldb/trunk/include/lldb/Core/Debugger.h
>>>>  lldb/trunk/include/lldb/Core/Disassembler.h
>>>>  lldb/trunk/include/lldb/Core/EmulateInstruction.h
>>>>  lldb/trunk/include/lldb/Core/FileLineResolver.h
>>>>  lldb/trunk/include/lldb/Core/FileSpecList.h
>>>>  lldb/trunk/include/lldb/Core/FormatEntity.h
>>>>  lldb/trunk/include/lldb/Core/Module.h
>>>>  lldb/trunk/include/lldb/Core/ModuleList.h
>>>>  lldb/trunk/include/lldb/Core/Opcode.h
>>>>  lldb/trunk/include/lldb/Core/PluginManager.h
>>>>  lldb/trunk/include/lldb/Core/SearchFilter.h
>>>>  lldb/trunk/include/lldb/Core/Section.h
>>>>  lldb/trunk/inc

Re: [lldb-dev] [Lldb-commits] [lldb] r360757 - Group forward declarations in one namespace lldb_private {}

2019-05-15 Thread Jim Ingham via lldb-dev
This is a common header file that specifies either common enums & #defines in 
lldb, or forward declarations of the common classes in lldb.  They don't depend 
on any other header files in the system, they just establish the common names 
you should use.  So the dependency analysis introduced by these files should be 
trivial - they really don't depend on anything else.  And the only time you 
change them is when you add a new public type or define to lldb, which doesn't 
happen very often.

But it is very convenient to have a common global set of names for the public 
entities in lldb, that can be looked up in one place, etc.  You can't actually 
USE any of the classes from these header files, so they don't obscure what 
other source files actually use - one problem with the big mongo "Cocoa.h" 
header file pattern.  If you want to get the class definition of anything, you 
have to include the specific .h file for that thing.  This is just a list of 
forward declarations.  Switching to 

I haven't done your experiment, but if "I depend on a header that either 
includes no other files or straight includes another file" really is a 
significant burden on the dependency analysis, then I'm not sure this pattern 
is actually the problem.

Jim

> On May 15, 2019, at 1:55 PM, Eric Christopher  wrote:
> 
> A couple of perspectives FWIW:
> 
> a) Header file includes increase overall dependencies which affects
> incremental rebuild time whenever you touch something. Especially more
> when that header is included into multiple headers that are then
> included into an array of translation units
> b) Having each file contain the forward declarations it needs from the
> project and no more also will help both compile time and understanding
> what's used in any particular header.
> 
> As a related exercise:
> 
> https://twitter.com/echristo/status/1116609586004316160
> 
> basically has removing a couple of transitive dependencies saving in
> excess of 70% of the incremental rebuild time after touching a file.
> This seems significant. :)
> 
> -eric
> 
> On Wed, May 15, 2019 at 10:49 AM Jim Ingham via lldb-dev
>  wrote:
>> 
>> This commit makes things look a little cleaner, but do we need any of these 
>> forward declarations at all?
>> 
>> Most of these files either directly include lldb-forward.h or get it from 
>> lldb-types.h or lldb-defines.h (which includes lldb-types.h).  As a general 
>> practice we've been getting all the lldb_private type forward declarations 
>> this way rather than cluttering up the .h files with individual forward 
>> declarations.  If we are going to keep doing it that way - which I can't see 
>> any reason not to do, then we should just delete all these forward defines, 
>> right?
>> 
>> Jim
>> 
>> 
>>> On May 15, 2019, at 2:15 AM, Fangrui Song via lldb-commits 
>>>  wrote:
>>> 
>>> Author: maskray
>>> Date: Wed May 15 02:15:13 2019
>>> New Revision: 360757
>>> 
>>> URL: http://llvm.org/viewvc/llvm-project?rev=360757=rev
>>> Log:
>>> Group forward declarations in one namespace lldb_private {}
>>> 
>>> Modified:
>>>   lldb/trunk/include/lldb/Core/Address.h
>>>   lldb/trunk/include/lldb/Core/AddressRange.h
>>>   lldb/trunk/include/lldb/Core/AddressResolver.h
>>>   lldb/trunk/include/lldb/Core/AddressResolverFileLine.h
>>>   lldb/trunk/include/lldb/Core/AddressResolverName.h
>>>   lldb/trunk/include/lldb/Core/Communication.h
>>>   lldb/trunk/include/lldb/Core/Debugger.h
>>>   lldb/trunk/include/lldb/Core/Disassembler.h
>>>   lldb/trunk/include/lldb/Core/EmulateInstruction.h
>>>   lldb/trunk/include/lldb/Core/FileLineResolver.h
>>>   lldb/trunk/include/lldb/Core/FileSpecList.h
>>>   lldb/trunk/include/lldb/Core/FormatEntity.h
>>>   lldb/trunk/include/lldb/Core/Module.h
>>>   lldb/trunk/include/lldb/Core/ModuleList.h
>>>   lldb/trunk/include/lldb/Core/Opcode.h
>>>   lldb/trunk/include/lldb/Core/PluginManager.h
>>>   lldb/trunk/include/lldb/Core/SearchFilter.h
>>>   lldb/trunk/include/lldb/Core/Section.h
>>>   lldb/trunk/include/lldb/Core/SourceManager.h
>>>   lldb/trunk/include/lldb/Core/StreamAsynchronousIO.h
>>>   lldb/trunk/include/lldb/Core/UserSettingsController.h
>>>   lldb/trunk/include/lldb/Core/Value.h
>>>   lldb/trunk/include/lldb/Core/ValueObject.h
>>>   lldb/trunk/include/lldb/Core/ValueObjectCast.h
>>>   lldb/trunk/include/lldb/Core/ValueObjectConstResult.h
>>>   lldb/trunk/include/lldb/Core/ValueObjectConstResultCast.h
>>>   lldb/trunk/include/lldb

Re: [lldb-dev] [Lldb-commits] [lldb] r360757 - Group forward declarations in one namespace lldb_private {}

2019-05-15 Thread Jim Ingham via lldb-dev
This commit makes things look a little cleaner, but do we need any of these 
forward declarations at all?  

Most of these files either directly include lldb-forward.h or get it from 
lldb-types.h or lldb-defines.h (which includes lldb-types.h).  As a general 
practice we've been getting all the lldb_private type forward declarations this 
way rather than cluttering up the .h files with individual forward 
declarations.  If we are going to keep doing it that way - which I can't see 
any reason not to do, then we should just delete all these forward defines, 
right?

Jim


> On May 15, 2019, at 2:15 AM, Fangrui Song via lldb-commits 
>  wrote:
> 
> Author: maskray
> Date: Wed May 15 02:15:13 2019
> New Revision: 360757
> 
> URL: http://llvm.org/viewvc/llvm-project?rev=360757=rev
> Log:
> Group forward declarations in one namespace lldb_private {}
> 
> Modified:
>lldb/trunk/include/lldb/Core/Address.h
>lldb/trunk/include/lldb/Core/AddressRange.h
>lldb/trunk/include/lldb/Core/AddressResolver.h
>lldb/trunk/include/lldb/Core/AddressResolverFileLine.h
>lldb/trunk/include/lldb/Core/AddressResolverName.h
>lldb/trunk/include/lldb/Core/Communication.h
>lldb/trunk/include/lldb/Core/Debugger.h
>lldb/trunk/include/lldb/Core/Disassembler.h
>lldb/trunk/include/lldb/Core/EmulateInstruction.h
>lldb/trunk/include/lldb/Core/FileLineResolver.h
>lldb/trunk/include/lldb/Core/FileSpecList.h
>lldb/trunk/include/lldb/Core/FormatEntity.h
>lldb/trunk/include/lldb/Core/Module.h
>lldb/trunk/include/lldb/Core/ModuleList.h
>lldb/trunk/include/lldb/Core/Opcode.h
>lldb/trunk/include/lldb/Core/PluginManager.h
>lldb/trunk/include/lldb/Core/SearchFilter.h
>lldb/trunk/include/lldb/Core/Section.h
>lldb/trunk/include/lldb/Core/SourceManager.h
>lldb/trunk/include/lldb/Core/StreamAsynchronousIO.h
>lldb/trunk/include/lldb/Core/UserSettingsController.h
>lldb/trunk/include/lldb/Core/Value.h
>lldb/trunk/include/lldb/Core/ValueObject.h
>lldb/trunk/include/lldb/Core/ValueObjectCast.h
>lldb/trunk/include/lldb/Core/ValueObjectConstResult.h
>lldb/trunk/include/lldb/Core/ValueObjectConstResultCast.h
>lldb/trunk/include/lldb/Core/ValueObjectConstResultChild.h
>lldb/trunk/include/lldb/Core/ValueObjectConstResultImpl.h
>lldb/trunk/include/lldb/Core/ValueObjectDynamicValue.h
>lldb/trunk/include/lldb/Core/ValueObjectList.h
>lldb/trunk/include/lldb/Core/ValueObjectMemory.h
>lldb/trunk/include/lldb/Core/ValueObjectRegister.h
>lldb/trunk/include/lldb/Core/ValueObjectSyntheticFilter.h
>lldb/trunk/include/lldb/Core/ValueObjectVariable.h
>lldb/trunk/include/lldb/Target/DynamicLoader.h
>lldb/trunk/include/lldb/Utility/Broadcaster.h
>lldb/trunk/include/lldb/Utility/Connection.h
>lldb/trunk/include/lldb/Utility/DataExtractor.h
>lldb/trunk/include/lldb/Utility/Event.h
>lldb/trunk/include/lldb/Utility/JSON.h
>lldb/trunk/include/lldb/Utility/Listener.h
>lldb/trunk/include/lldb/Utility/StringList.h
>lldb/trunk/include/lldb/Utility/StructuredData.h
>lldb/trunk/include/lldb/Utility/UserID.h
> 
> Modified: lldb/trunk/include/lldb/Core/Address.h
> URL: 
> http://llvm.org/viewvc/llvm-project/lldb/trunk/include/lldb/Core/Address.h?rev=360757=360756=360757=diff
> ==
> --- lldb/trunk/include/lldb/Core/Address.h (original)
> +++ lldb/trunk/include/lldb/Core/Address.h Wed May 15 02:15:13 2019
> @@ -19,36 +19,15 @@
> 
> namespace lldb_private {
> class Block;
> -}
> -namespace lldb_private {
> class CompileUnit;
> -}
> -namespace lldb_private {
> class ExecutionContextScope;
> -}
> -namespace lldb_private {
> class Function;
> -}
> -namespace lldb_private {
> class SectionList;
> -}
> -namespace lldb_private {
> class Stream;
> -}
> -namespace lldb_private {
> class Symbol;
> -}
> -namespace lldb_private {
> class SymbolContext;
> -}
> -namespace lldb_private {
> class Target;
> -}
> -namespace lldb_private {
> struct LineEntry;
> -}
> -
> -namespace lldb_private {
> 
> /// \class Address Address.h "lldb/Core/Address.h"
> /// A section + offset based address class.
> 
> Modified: lldb/trunk/include/lldb/Core/AddressRange.h
> URL: 
> http://llvm.org/viewvc/llvm-project/lldb/trunk/include/lldb/Core/AddressRange.h?rev=360757=360756=360757=diff
> ==
> --- lldb/trunk/include/lldb/Core/AddressRange.h (original)
> +++ lldb/trunk/include/lldb/Core/AddressRange.h Wed May 15 02:15:13 2019
> @@ -17,15 +17,8 @@
> 
> namespace lldb_private {
> class SectionList;
> -}
> -namespace lldb_private {
> class Stream;
> -}
> -namespace lldb_private {
> class Target;
> -}
> -
> -namespace lldb_private {
> 
> /// \class AddressRange AddressRange.h "lldb/Core/AddressRange.h"
> /// A section + offset based address range class.
> 
> Modified: 

Re: [lldb-dev] Where did the python/c++ API documentation go?

2019-04-22 Thread Jim Ingham via lldb-dev
Well, you can probably look at the copy in the sources (www/index.html & so 
forth).  But we do need to get this fixed as external folks who don't have a 
checkout do rely on this.

Jim


> On Apr 22, 2019, at 11:37 AM, Ted Woodward  wrote:
> 
> Great, thanks Jim. Glad to see people are already on this.
> 
> But where do I go if I need to look at the python API right now?
> (besides web.archive.org, which is what I ended up doing)
> 
>> -Original Message-
>> From: jing...@apple.com 
>> Sent: Monday, April 22, 2019 1:35 PM
>> To: Ted Woodward 
>> Cc: LLDB 
>> Subject: [EXT] Re: [lldb-dev] Where did the python/c++ API documentation
>> go?
>> 
>> 
>> 
>>> On Apr 22, 2019, at 10:59 AM, Ted Woodward via lldb-dev > d...@lists.llvm.org> wrote:
>>> 
>>> The new LLDB website at http://lldb.llvm.org has an external resources
>> page:
>>> http://lldb.llvm.org/resources/external.html
>>> 
>>> It has 2 entries on it for Documentation:
>>> https://lldb.llvm.org/python_reference/index.html
>>> https://lldb.llvm.org/cpp_reference/html/index.html
>>> 
>>> Both of these lead to “404 Not Found”:
>>> The requested URL /python_reference/index.html was not found on this
>> server.
>>> The requested URL /cpp_reference/html/index.html was not found on this
>> server.
>>> 
>>> Where do I go to find the python/c++ API documentation now?
>>> 
>>> BTW, I don’t think LLDB’s API documentation is an “External Resource”.
>> Those links should be on the main page, along with a link to the llvm main
>> page.
>> 
>> Both of these are known issues.  Jonas is going to move the API docs to the 
>> top
>> level.  IIUC this also needs some website admin intervention, which Jonas is
>> waiting on.
>> 
>> Jim
>> 
>> 
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Where did the python/c++ API documentation go?

2019-04-22 Thread Jim Ingham via lldb-dev


> On Apr 22, 2019, at 10:59 AM, Ted Woodward via lldb-dev 
>  wrote:
> 
> The new LLDB website at http://lldb.llvm.org has an external resources page:
> http://lldb.llvm.org/resources/external.html
>  
> It has 2 entries on it for Documentation:
> https://lldb.llvm.org/python_reference/index.html
> https://lldb.llvm.org/cpp_reference/html/index.html
>  
> Both of these lead to “404 Not Found”:
> The requested URL /python_reference/index.html was not found on this server.
> The requested URL /cpp_reference/html/index.html was not found on this server.
>  
> Where do I go to find the python/c++ API documentation now?
>  
> BTW, I don’t think LLDB’s API documentation is an “External Resource”. Those 
> links should be on the main page, along with a link to the llvm main page.

Both of these are known issues.  Jonas is going to move the API docs to the top 
level.  IIUC this also needs some website admin intervention, which Jonas is 
waiting on.

Jim


> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC] Having common Python OS Plug-in

2019-04-17 Thread Jim Ingham via lldb-dev
The os plugin in the KDK's is just a packaged up version of what's in the link 
you referenced, so the KDK wouldn't add anything.

Jim


> On Apr 17, 2019, at 12:50 PM, Alexander Polyakov  
> wrote:
> 
> It seems that I need a Apple ID to create an account, can you please share 
> the code?
> 
> I want common OS plug-in to be similar to Apple's lldbmacros (link), but it 
> should be with support of multiple architectures and operating systems, 
> logging and so on. I've created such a plug-in for Zephyr OS, maybe I'll be 
> able to also share the code to show how common plug-in might look like.
> 
> On Wed, Apr 17, 2019 at 10:40 PM Jim Ingham  wrote:
> If you want to undertake such a thing, that would be great.  The other fairly 
> complex OS plugin that exists in the wild is the one that ships with the 
> Darwin Kernel Debug Kit.  You can get this from 
> http://developer.apple.com/download/more - you need a free ADC account to get 
> this, though if that's a problem I can also get you a copy of the Python code 
> for the OS plugin.  If you're going to make some common utilities you might 
> have a look at what that one does if you haven't already.
> 
> Jim
> 
> 
> > On Apr 17, 2019, at 11:37 AM, Alexander Polyakov via lldb-dev 
> >  wrote:
> > 
> > Hi lldb-dev,
> > 
> > The last few months, I was implementing a LLDB OS Python plug-in. It seems 
> > that there are functionality that is duplicated across many OS plug-ins. I 
> > think we can change that by, for example, upstreaming to LLDB an OS python 
> > plug-in which can be used by all clients and may contain a lot of utility 
> > functions and useful mechanisms like logging, type summaries and so on.
> > 
> > What is your opinion on this?
> > 
> > -- 
> > Alexander
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC] Having common Python OS Plug-in

2019-04-17 Thread Jim Ingham via lldb-dev
If you want to undertake such a thing, that would be great.  The other fairly 
complex OS plugin that exists in the wild is the one that ships with the Darwin 
Kernel Debug Kit.  You can get this from 
http://developer.apple.com/download/more - you need a free ADC account to get 
this, though if that's a problem I can also get you a copy of the Python code 
for the OS plugin.  If you're going to make some common utilities you might 
have a look at what that one does if you haven't already.

Jim


> On Apr 17, 2019, at 11:37 AM, Alexander Polyakov via lldb-dev 
>  wrote:
> 
> Hi lldb-dev,
> 
> The last few months, I was implementing a LLDB OS Python plug-in. It seems 
> that there are functionality that is duplicated across many OS plug-ins. I 
> think we can change that by, for example, upstreaming to LLDB an OS python 
> plug-in which can be used by all clients and may contain a lot of utility 
> functions and useful mechanisms like logging, type summaries and so on.
> 
> What is your opinion on this?
> 
> -- 
> Alexander
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SBValue::GetName returns wrong result?

2019-04-01 Thread Jim Ingham via lldb-dev
Independent of the naming issue, if you have an address and want to view its 
pointee as a given type, CreateValueFromAddress is much more efficient than 
CreateValueFromExpression.  After all, CreateValueFromAddress just reads some 
memory and presents it as the given type, rather than having to parse and 
evaluate an expression just to figure out the type and address, and then go 
read the memory and present it as the given type..

As a side note, if you only wanted the dereferenced values, and needed to use 
the expression, you could also just make the expression be:

*((uint32_t *) ADDR)

The resultant SBValue would have the name you wanted.  But again, for "typed 
addresses" CreateValueFromAddress is a more robust & efficient API.

Jim


> On Apr 1, 2019, at 3:36 PM, Alexander Polyakov  wrote:
> 
> I have an address of memory where the value of some register is. I do 
> following:
> 
> addr = 0x1234 (just for example)
> rbx = target.CreateValueFromExpression('(uint32_t *) ' + str(addr), 'rbx')
> rbx = rbx.Dereference()
> 
> Then I want to create a map:
> rbx.GetName() => rbx.GetValue()
> 
> In this case rbx.GetName() will return "*rbx". 
> 
> Maybe it'd be better to use SBTarget::CreateValueFromAddress() instead of 
> CreateValueFromExpression. The Value created this way will be dereferenced 
> initially, so its name will not contain *, I guess.
> 
> On Tue, Apr 2, 2019 at 1:23 AM Jim Ingham  wrote:
> What are you using the name for?  If the name of an SBValue is the name of a 
> variable, then it makes sense (at least in C languages) for the name of the 
> dereference Value to be "*VARNAME".  After all that's what it is.  If the 
> name is some other random string, I'm not sure anything would be better or 
> worse, except it would be confusing to dereference an SBValue and get back 
> another value with the same name, so we have to choose something else.
> 
> Jim
> 
> > On Apr 1, 2019, at 3:16 PM, Alexander Polyakov  
> > wrote:
> > 
> > I can't say that it's a problem, I just want to know what is the actual 
> > reason of such a behavior to find good workaround.
> > 
> > I have a SBValue with a pointer to some object, e.g. "(uint32_t *) sp", 
> > when I do dereference it, I get another SBValue - "(uint32_t) *sp". The 
> > only way to deal with it that I see is to check the first symbol of name 
> > and erase it if it's equal to *.
> > 
> > I'm facing with that situation when creating an object from a pointer via 
> > SBTarget::CreateValueFromExpression.
> > 
> > 
> > 
> > On Mon, Apr 1, 2019 at 9:35 PM Jim Ingham  wrote:
> > Dereference returns another SBValue distinct from the initial one, so it 
> > needs to make up a name for it.  I think it would be confusing for it to 
> > return the same name, and putting a * at the beginning of the initial 
> > SBValue seems as good a choice as any.
> > 
> > Is this causing you some concrete problem?
> > 
> > Jim
> > 
> > 
> > > On Mar 30, 2019, at 11:18 AM, Alexander Polyakov via lldb-dev 
> > >  wrote:
> > > 
> > > Hi lldb-dev,
> > > 
> > > I have a SBValue created via 
> > > SBTarget.CreateValueFromExpression('some_name', expr).
> > > If the expression looks like '(some_type *) addr', then GetName returns 
> > > 'some_name' as expected, but when I do Dereference this value, GetName 
> > > returns '*some_name'.
> > > 
> > > So, is it a conventional behavior of the GetName method applied to 
> > > dereferenced SBValue?
> > > 
> > > -- 
> > > Alexander
> > > ___
> > > lldb-dev mailing list
> > > lldb-dev@lists.llvm.org
> > > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> > 
> > 
> > 
> > -- 
> > Alexander
> 
> 
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SBValue::GetName returns wrong result?

2019-04-01 Thread Jim Ingham via lldb-dev
What are you using the name for?  If the name of an SBValue is the name of a 
variable, then it makes sense (at least in C languages) for the name of the 
dereference Value to be "*VARNAME".  After all that's what it is.  If the name 
is some other random string, I'm not sure anything would be better or worse, 
except it would be confusing to dereference an SBValue and get back another 
value with the same name, so we have to choose something else.

Jim

> On Apr 1, 2019, at 3:16 PM, Alexander Polyakov  wrote:
> 
> I can't say that it's a problem, I just want to know what is the actual 
> reason of such a behavior to find good workaround.
> 
> I have a SBValue with a pointer to some object, e.g. "(uint32_t *) sp", when 
> I do dereference it, I get another SBValue - "(uint32_t) *sp". The only way 
> to deal with it that I see is to check the first symbol of name and erase it 
> if it's equal to *.
> 
> I'm facing with that situation when creating an object from a pointer via 
> SBTarget::CreateValueFromExpression.
> 
> 
> 
> On Mon, Apr 1, 2019 at 9:35 PM Jim Ingham  wrote:
> Dereference returns another SBValue distinct from the initial one, so it 
> needs to make up a name for it.  I think it would be confusing for it to 
> return the same name, and putting a * at the beginning of the initial SBValue 
> seems as good a choice as any.
> 
> Is this causing you some concrete problem?
> 
> Jim
> 
> 
> > On Mar 30, 2019, at 11:18 AM, Alexander Polyakov via lldb-dev 
> >  wrote:
> > 
> > Hi lldb-dev,
> > 
> > I have a SBValue created via 
> > SBTarget.CreateValueFromExpression('some_name', expr).
> > If the expression looks like '(some_type *) addr', then GetName returns 
> > 'some_name' as expected, but when I do Dereference this value, GetName 
> > returns '*some_name'.
> > 
> > So, is it a conventional behavior of the GetName method applied to 
> > dereferenced SBValue?
> > 
> > -- 
> > Alexander
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SBValue::GetName returns wrong result?

2019-04-01 Thread Jim Ingham via lldb-dev
Dereference returns another SBValue distinct from the initial one, so it needs 
to make up a name for it.  I think it would be confusing for it to return the 
same name, and putting a * at the beginning of the initial SBValue seems as 
good a choice as any.

Is this causing you some concrete problem?

Jim


> On Mar 30, 2019, at 11:18 AM, Alexander Polyakov via lldb-dev 
>  wrote:
> 
> Hi lldb-dev,
> 
> I have a SBValue created via SBTarget.CreateValueFromExpression('some_name', 
> expr).
> If the expression looks like '(some_type *) addr', then GetName returns 
> 'some_name' as expected, but when I do Dereference this value, GetName 
> returns '*some_name'.
> 
> So, is it a conventional behavior of the GetName method applied to 
> dereferenced SBValue?
> 
> -- 
> Alexander
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] What is the correct behavior of SBModule::GetVersion with a version number of 0.0.0

2019-03-28 Thread Jim Ingham via lldb-dev


> On Mar 28, 2019, at 3:03 AM, Pavel Labath  wrote:
> 
> On 28/03/2019 01:36, Jim Ingham wrote:
>> If you have a library that has a version number of 0.0.0,
>> uint32_t SBModule::GetVersion(uint32_t *versions, uint32_t num_versions)
>> will return a result of 2 (which is the number of elements it put into 
>> num_versions) and the two elements it actually stuffed into the versions 
>> array will be {UINT32_MAX, 0}.
>> That seems like a weird result to me, but the code goes to some trouble to 
>> make this up.  I was wondering what the reason for this is?
> 
> I think that is a bug. I am pretty sure my intention when writing this code 
> was to fill in everything with UINT32_MAX in this case, but I did not 
> implement that correctly. The code for computing the "result" was probably 
> meant to be something like:
> ---
> result = 0;
> if (!version.empty()) {
>  ++result;
>  if(version.getMinor())
>++result;
>  if(version.getSubminor())
>++result;
> }
> ---
> which would give an all-UINT32_MAX array in this case (and avoid the swig 
> wrapper crash).
> 
> However, that doesn't mean we have to implement it that way..
> 
>> So I was wondering whether the code in GetVersion that fills the Major 
>> version with UINT32_MAX was done for some reason, or is just an accident I 
>> should fix.
> 
> The idea there was to have this signal that the Module has no version at all. 
> However, given that ObjectFileMachO is the only one who uses this field, and 
> the convention there seems to be to treat 0 as a valid version (i.e., the 
> binary format has no ability to express "I absolutely don't know my version), 
> changing that seems fine to me.

For dylibs, the current_version field in the LD_ID_DYLIB load command (which is 
what we report in GetVersion) is always present, so there will always be a 
version number for mach-o dylibs.  If you don't specify a version ld will fill 
it with zero.  However, you can also specify -current_version 0.0.0 and that 
will produce exactly the same load command.  So I don't think it is possible to 
meaningfully have "no version" for dylibs on mach-o.  OTOH executables have no 
version number - presumably because the version number is for checking 
capabilities in a loaded object - and in that case the VersionTuple returned 
from the MachO plugin is not only "empty()" but HasMinor and HasSubminor are 
false.  This properly produces an empty list return from the MachO object file 
plugin.

> 
>> BTW, this comes about because llvm::VersionTuple doesn't have a HasMajor 
>> flag, and the only way to test the validity of the major version is to call 
>> "VersionTuple::empty" and that ONLY checks the values of the version 
>> numbers.  So something that has HasMinor = true but all the version numbers 
>> are 0 will say it is empty - which also seems a little odd to me...
> 
> Yes there, seems to be some confusion around the usage of empty(). The 
> MachObjectWriter will serialize an empty VersionTuple as 0 
> ,
>  but then there are also plenty of places which treat an empty VersionTuple 
> as invalid 
> . 
> (Incidentally, the MachObjectWriter stuff has to be fairly new code, because 
> VersionTuple was present only in clang until recently.)
> 
> Changing empty() to check the value of HasMinor flag sounds ok to me -- after 
> all, if someone went to the trouble of spelling it out, he probably really 
> meant that value to be valid. However, it does create a situation where it is 
> possible to express the version "0.0", and "1", but not "0", which may or may 
> not be fine...
> 
> Overall, it might be better to add a HasMajor flag too, and have the 
> VersionTuple be invalid (empty) only when this flag is false. Or, we could 
> drop the empty() method altogether, and use Optional to for the 
> cases where the version is completely absent.
> 

Thanks for the explanation!  When I get back to this I'll fix the 
SBModule::GetVersion for the 0.0 case and see about fixing the underlying 
behavior.  Once we don't crash the odd printing is a minor issue, so I'm out of 
time for this right now.

Jim  

> pl

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


[lldb-dev] What is the correct behavior of SBModule::GetVersion with a version number of 0.0.0

2019-03-27 Thread Jim Ingham via lldb-dev
If you have a library that has a version number of 0.0.0, 

uint32_t SBModule::GetVersion(uint32_t *versions, uint32_t num_versions) 

will return a result of 2 (which is the number of elements it put into 
num_versions) and the two elements it actually stuffed into the versions array 
will be {UINT32_MAX, 0}.

That seems like a weird result to me, but the code goes to some trouble to make 
this up.  I was wondering what the reason for this is?

This came to my attention because the swig wrapper for GetVersion treats 
UINT32_MAX as the sign that it should stop filling elements in the list array 
(even if the element that is UINT32_MAX is earlier in the returned array than 
the result that GetVersion returns.)  That is, we do:

uint32_t count = result;
if (count >= arg3)
count = arg3;
PyObject* list = PyList_New(count);
for (uint32_t j = 0; j < count; j++)
{
  if (arg2[j] < UINT32_MAX)
  {
PyObject* item = PyInt_FromLong(arg2[j]);
int ok = PyList_SetItem(list,j,item);
if (ok != 0)
{
  resultobj = Py_None;
  break;
}
  }
  else
  break;
}

And so if the 0th element in the list in UINT32_MAX, we make a Python list that 
we say we are going to fill with two elements, but don't put anything in it, 
and so if you try to iterate over this list Python crashes.

The wrapper behavior seems clearly wrong.  If you say you are making an array 
with N elements, you need to put N elements in it or iterating over it will 
crash.  That's easy to fix.

But then if we have a library whose actual version (e.g. as printed by the llvm 
objdump) is 0.0.0, we will call it {4294967295, 0} which just seems weird.  

So I was wondering whether the code in GetVersion that fills the Major version 
with UINT32_MAX was done for some reason, or is just an accident I should fix.

BTW, this comes about because llvm::VersionTuple doesn't have a HasMajor flag, 
and the only way to test the validity of the major version is to call 
"VersionTuple::empty" and that ONLY checks the values of the version numbers.  
So something that has HasMinor = true but all the version numbers are 0 will 
say it is empty - which also seems a little odd to me...

Jim

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] DEBUG_PRINTF() macro

2019-03-13 Thread Jim Ingham via lldb-dev
Seems like there are two kinds of DEBUG_PRINTF.  

One is in SymbolFileDWARF, DWARFASTParserClang.cpp etc, where that is inserting 
code into lldb.  Those uses seem to me like they should be going to the dwarf 
log channel when verbose is on.  Many of these places also acquire the dwarf 
log, so they could do this right.  I have no objections to switching these over 
(I'll leave it to Greg and others who debug DWARF a lot to decide whether they 
can be removed altogether.)

The other flavor of DEBUG_PRINTF isn't used in lldb code, it is inserted into 
code that we are compiling and inserting into the target.  There's a bunch of 
this in the ObjCLanguageRuntime, for instance.  When we run the function we've 
inserted it prints info about the information it is gathering.  Generally, the 
output is triggered by some log setting, which then sets an argument we pass to 
the function we're executing.

This is a really handy way to debug the work that functions we insert into the 
target are doing.  We should not remove any of those uses.  Presumably that 
wasn't what you were referring to, but just making sure they didn't get cut 
down in the same charge.

Jim


> On Mar 13, 2019, at 9:39 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> Apparently we have a macro called DEBUG_PRINTF() which, if you compile LLDB 
> with a special pre-processor setting enabled, will cause certain messages to 
> be printed to stdout while running LLDB.
> 
> Does anyone use this?  This seems like a kind of hacky alternative to 
> tracepoints and/or pretty printers, and in some cases is causing otherwise 
> dead code to be compiled into the binary, so there's some benefit to removing 
> it.  
> 
> Is anyone opposed to removing all of this?
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] RFC: Eliminate LLDB_DISABLE_PYTHON

2019-03-07 Thread Jim Ingham via lldb-dev


> On Mar 7, 2019, at 11:37 AM, Zachary Turner  wrote:
> 
> 
> 
> On Thu, Mar 7, 2019 at 11:03 AM Jim Ingham via lldb-dev 
>  wrote:
> Even though you can just use debugserver/lldb-server and debug remotely, many 
> people find it handy to be able to run a debugger directly on the device they 
> are using.  But requiring that you port Python and bundle it with your 
> embedded platform just to get a debugger there is a pretty big ask.  So it is 
> still quite useful to be able to build lldb without Python.  This option 
> hasn't been broken all that frequently in our uses, but if it is being a 
> problem, the better solution would be to set up a bot to build with this 
> option, so we can make sure it keeps working.
> 
> That's true, but there's a maintenance burden associated with this handiness, 
> and if nobody uses it that much (or at all?), it's better IMO to optimize for 
> low maintenance.  Every time I've tried this configuration it has been 
> broken, which leads me to believe that in practice nobody is actually using 
> it.  If that really is the case, I'd rather it be gone.  I don't think we 
> should keep code around just in case, without specific evidence that it's 
> providing value.


It does get used, though we might be able to get away from that at some point.  
But I still think requiring any new platform that might want to run a debugger 
to get Python up first is unfortunate, and we shouldn't lightly require it. 

But also, this isn't just about Python in particular.  Everything in lldb that 
touches the script interpreter should be going through the abstract 
ScriptInterpreter class.  The only place where the fact that the script 
interpreter happens to be Python should show up is in the 
ScriptInterpreterPython.  So building without Python should be as simple as not 
building the ScriptInterpreterPython and not initializing that plugin.  The 
maintenance burden for this option should be trivial.  Something is broken that 
LLDB_DISABLE_PYTHON has gotten entangled deeper in lldb.  I'd much prefer we 
fix that.

It would be really cool, for instance, if lldb could support a Python2 and a 
Python3 script interpreter to ease the transition for folks with lots of legacy 
Python 2 code (such folks exist).  lldb was designed to support that sort of 
thing.

Or maybe at some point we should support some other new hotness language.  I'm 
not sure it is good to bind ourselves inextricably to Python.

Jim

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] RFC: Eliminate LLDB_DISABLE_PYTHON

2019-03-07 Thread Jim Ingham via lldb-dev
Even though you can just use debugserver/lldb-server and debug remotely, many 
people find it handy to be able to run a debugger directly on the device they 
are using.  But requiring that you port Python and bundle it with your embedded 
platform just to get a debugger there is a pretty big ask.  So it is still 
quite useful to be able to build lldb without Python.  This option hasn't been 
broken all that frequently in our uses, but if it is being a problem, the 
better solution would be to set up a bot to build with this option, so we can 
make sure it keeps working.

Jim

> On Mar 7, 2019, at 10:27 AM, Adrian McCarthy via lldb-dev 
>  wrote:
> 
> We have a build option to build LLDB without Python.  This option is 
> automatically set if Cmake can't find or "validate" your Python distribution.
> 
> Since LLDB is rarely built with this option, nobody discovers when this 
> configuration breaks.  For example, if you try it today, you'll get a handful 
> of missing symbol errors because their declarations are #if'ed out but other 
> code still references them.
> 
> I'm proposing eliminating this option and fixing the validation of the Python 
> distribution in LLDBConfig.cmake.
> 
> If you believe LLDB_DISABLE_PYTHON is still useful, please speak up, and I'll 
> instead look at fixing the existing build problems.
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-21 Thread Jim Ingham via lldb-dev
What's triggering one of the OS Plugin methods to get run on this separate 
thread?  I would expect SetPrivateState would just cause the private stop event 
to get broadcast to the private state thread, and then that would wake up and 
then it would be the one to call the OS Plugin to do it's job.  That's how the 
GDBRemote plugin works, for instance.

Jim


> On Feb 21, 2019, at 2:54 PM, Tatyana Krasnukha 
>  wrote:
> 
> That clarified things, thanks!
> 
> I think, this is the reason:
> 
> ProcessRunLock ::GetRunLock() {
>  if (m_private_state_thread.EqualsThread(Host::GetCurrentThread()))
>return m_private_run_lock;
>  else
>return m_public_run_lock;
> }
> 
> In our case the current thread is not m_private_state_thread. I create a 
> separate thread for waiting a processor to halt and then SetPrivateState. It 
> seems, that was an inappropriate approach.
> 
> -Original Message-
> From: jing...@apple.com  
> Sent: Thursday, February 21, 2019 11:58 PM
> To: Tatyana Krasnukha 
> Cc: Alexander Polyakov ; LLDB 
> 
> Subject: Re: [lldb-dev] SB API is not working properly with OSPython plugin
> 
> 
> 
>> On Feb 21, 2019, at 11:22 AM, Tatyana Krasnukha via lldb-dev 
>>  wrote:
>> 
>>> lldb Process::SetPrivateState (stopped) stop_id = 2
>>> error: error: process must be stopped.
>> 
>> These two lines are printed from different threads, you cannot be sure the 
>> real order of execution is the same.
>> 
>> The plugin should subscribe on public state change events and wait until one 
>> comes (correct me if I’m wrong about that).
> 
> That's not right.  When the process stops, but before lldb broadcasts a 
> public stop event, it will query the OS Plugin directly to build up the 
> thread list.  It needs to do that before it declares a public stop because it 
> uses the thread list to reason about whether to declare a public stop or not. 
>  So the OS Plugin (and BTW the Thread Step Plan plugin is in the same boat) 
> have to run after a private stop but before public one.  There isn't a 
> principled way to do that.  The best we have at present is a hack that says 
> "if you are running on the private state thread, then you get to do things 
> between private & public stop as if there were a public stop".  Cleaning that 
> up is item 5 on the lldb Projects page if anyone is interested...
> 
> Jim
> 
>> 
>> From: lldb-dev  On Behalf Of 
>> Alexander Polyakov via lldb-dev
>> Sent: Thursday, February 21, 2019 9:54 PM
>> To: Jim Ingham 
>> Cc: LLDB 
>> Subject: Re: [lldb-dev] SB API is not working properly with OSPython 
>> plugin
>> 
>> It seems that the process plugin uses the Process::SetPrivateState at the 
>> right time. If you look at the log, you will see that the process is already 
>> in the 'private stopped' state when the OS plugin is invoked.
>> 
>> (lldb) c
>> lldb Process::Resume -- locking run lock
>> lldb Process::PrivateResume() m_stop_id = 1, public state: 
>> stopped private state: stopped
>> lldb Process::SetPrivateState (running)
>> intern-state Process::ShouldBroadcastEvent (0x1abea90) => new state: 
>> running, last broadcast state: running - YES
>> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new 
>> state running (old state stopped) to public
>> intern-state Process::PushProcessIOHandler pushing IO handler
>> intern-state Process::HandlePrivateEvent updated m_iohandler_sync to 1
>> lldb Process thinks the process has resumed.
>> intern-state timeout = , event_sp)...
>> lldb waited from m_iohandler_sync to change from 0. New value is 
>> 1.
>> dbg.evt-handler  Process::SetPublicState (state = running, restarted = 
>> 0) Process 1 resuming
>> lldb Process::SetPrivateState (stopped)
>> lldb Process::SetPrivateState (stopped) stop_id = 2
>> error: error: process must be stopped.
>> intern-state Process::ShouldBroadcastEvent (0x7f3e9c007450) => new 
>> state: stopped, last broadcast state: stopped - YES
>> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new 
>> state stopped (old state running) to public
>> intern-state timeout = , event_sp)...
>> dbg.evt-handler  Process::SetPublicState (state = stopped, restarted = 
>> 0) dbg.evt-handler  Process::SetPublicState (stopped) -- unlocking run 
>> lock
>> 
>> 
>> On Fri, Feb 15, 2019 at 1:38 AM Jim Ingham  wrote:
>> Your plugin should have set the private state to stopped when it figures out 
>> however it does that the process has stopped.  The API is 
>> Process::SetPrivateState.  Is that happening at the right time?
>> 
>> Jim
>> 
>> 
>> 
>> On Feb 14, 2019, at 1:50 PM, Alexander Polyakov  
>> wrote:
>> 
>> I found out that the plugin works well with an x86 application, so I think 
>> that the problem is in my process plugin. Maybe you know a place where to 
>> start looking for an issue?
>> 
>> On Thu, Feb 14, 2019 at 10:56 PM Jim Ingham  wrote:
>> The simplest thing 

Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-21 Thread Jim Ingham via lldb-dev


> On Feb 21, 2019, at 11:22 AM, Tatyana Krasnukha via lldb-dev 
>  wrote:
> 
> > lldb Process::SetPrivateState (stopped) stop_id = 2
> > error: error: process must be stopped.
>  
> These two lines are printed from different threads, you cannot be sure the 
> real order of execution is the same.
>  
> The plugin should subscribe on public state change events and wait until one 
> comes (correct me if I’m wrong about that).

That's not right.  When the process stops, but before lldb broadcasts a public 
stop event, it will query the OS Plugin directly to build up the thread list.  
It needs to do that before it declares a public stop because it uses the thread 
list to reason about whether to declare a public stop or not.  So the OS Plugin 
(and BTW the Thread Step Plan plugin is in the same boat) have to run after a 
private stop but before public one.  There isn't a principled way to do that.  
The best we have at present is a hack that says "if you are running on the 
private state thread, then you get to do things between private & public stop 
as if there were a public stop".  Cleaning that up is item 5 on the lldb 
Projects page if anyone is interested...

Jim

>  
> From: lldb-dev  On Behalf Of Alexander 
> Polyakov via lldb-dev
> Sent: Thursday, February 21, 2019 9:54 PM
> To: Jim Ingham 
> Cc: LLDB 
> Subject: Re: [lldb-dev] SB API is not working properly with OSPython plugin
>  
> It seems that the process plugin uses the Process::SetPrivateState at the 
> right time. If you look at the log, you will see that the process is already 
> in the 'private stopped' state when the OS plugin is invoked.
> 
> (lldb) c
> lldb Process::Resume -- locking run lock
> lldb Process::PrivateResume() m_stop_id = 1, public state: 
> stopped private state: stopped
> lldb Process::SetPrivateState (running)
> intern-state Process::ShouldBroadcastEvent (0x1abea90) => new state: 
> running, last broadcast state: running - YES
> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new state 
> running (old state stopped) to public
> intern-state Process::PushProcessIOHandler pushing IO handler
> intern-state Process::HandlePrivateEvent updated m_iohandler_sync to 1
> lldb Process thinks the process has resumed.
> intern-state timeout = , event_sp)...
> lldb waited from m_iohandler_sync to change from 0. New value is 
> 1.
> dbg.evt-handler  Process::SetPublicState (state = running, restarted = 0)
> Process 1 resuming
> lldb Process::SetPrivateState (stopped)
> lldb Process::SetPrivateState (stopped) stop_id = 2
> error: error: process must be stopped.
> intern-state Process::ShouldBroadcastEvent (0x7f3e9c007450) => new state: 
> stopped, last broadcast state: stopped - YES
> intern-state Process::HandlePrivateEvent (pid = 1) broadcasting new state 
> stopped (old state running) to public
> intern-state timeout = , event_sp)...
> dbg.evt-handler  Process::SetPublicState (state = stopped, restarted = 0)
> dbg.evt-handler  Process::SetPublicState (stopped) -- unlocking run lock
> 
>  
> On Fri, Feb 15, 2019 at 1:38 AM Jim Ingham  wrote:
> Your plugin should have set the private state to stopped when it figures out 
> however it does that the process has stopped.  The API is 
> Process::SetPrivateState.  Is that happening at the right time?
>  
> Jim
>  
> 
> 
> On Feb 14, 2019, at 1:50 PM, Alexander Polyakov  
> wrote:
>  
> I found out that the plugin works well with an x86 application, so I think 
> that the problem is in my process plugin. Maybe you know a place where to 
> start looking for an issue?
>  
> On Thu, Feb 14, 2019 at 10:56 PM Jim Ingham  wrote:
> The simplest thing possible to reproduce the failure.  So some OS_Plugin 
> implementation which tries to look up a global like this and fails, and some 
> program source I can run it under that provides said global.  That way I can 
> easily watch it fails as you describe when the plugin gets activated, and see 
> why it isn’t allowing this call on private stop.
>  
> Jim
>  
> 
> 
> On Feb 14, 2019, at 11:50 AM, Alexander Polyakov  
> wrote:
>  
> Sure, could you describe in more detail which example may help you?
>  
> чт, 14 февр. 2019 г. в 22:36, Jim Ingham :
> That’s a little complicated…  
>  
> lldb has two levels of stop state - private stops and public stops.  When the 
> process gets notification from the underlying process plugin that the process 
> stopped, it raises a private stop event.  That gets handled by the ShouldStop 
> mechanism on the private state thread in lldb, and then if the stop is judged 
> interesting to the user, it gets rebroadcast as a public stop.
>  
> For instance, when you issue a “step” command, lldb will stop and start the 
> process multiple times as it walks through the source line.  But only the 
> last of those stops are relevant to the user of LLDB, so all the other ones 
> exist 

Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-14 Thread Jim Ingham via lldb-dev
Your plugin should have set the private state to stopped when it figures out 
however it does that the process has stopped.  The API is 
Process::SetPrivateState.  Is that happening at the right time?

Jim


> On Feb 14, 2019, at 1:50 PM, Alexander Polyakov  
> wrote:
> 
> I found out that the plugin works well with an x86 application, so I think 
> that the problem is in my process plugin. Maybe you know a place where to 
> start looking for an issue?
> 
> On Thu, Feb 14, 2019 at 10:56 PM Jim Ingham  > wrote:
> The simplest thing possible to reproduce the failure.  So some OS_Plugin 
> implementation which tries to look up a global like this and fails, and some 
> program source I can run it under that provides said global.  That way I can 
> easily watch it fails as you describe when the plugin gets activated, and see 
> why it isn’t allowing this call on private stop.
> 
> Jim
> 
> 
>> On Feb 14, 2019, at 11:50 AM, Alexander Polyakov > > wrote:
>> 
>> Sure, could you describe in more detail which example may help you?
>> 
>> чт, 14 февр. 2019 г. в 22:36, Jim Ingham > >:
>> That’s a little complicated…  
>> 
>> lldb has two levels of stop state - private stops and public stops.  When 
>> the process gets notification from the underlying process plugin that the 
>> process stopped, it raises a private stop event.  That gets handled by the 
>> ShouldStop mechanism on the private state thread in lldb, and then if the 
>> stop is judged interesting to the user, it gets rebroadcast as a public stop.
>> 
>> For instance, when you issue a “step” command, lldb will stop and start the 
>> process multiple times as it walks through the source line.  But only the 
>> last of those stops are relevant to the user of LLDB, so all the other ones 
>> exist only as private stops.
>> 
>> The SB API’s for the most part should only consider a “publicly stopped” 
>> process accessible.  After all, you wouldn’t want some API to succeed 
>> sometimes if you happen to catch it in the middle of a private stop.
>> 
>> But the OperatingSystem plugin needs to get called right after a private 
>> stop, so it can provide threads for the ShouldStop mechanism.  We should 
>> really have some formal mechanism whereby things like the OS plugin can 
>> request elevated rights in the SB API’s, so that they can run at private 
>> stop time.  IIRC, we instead have a hack where SB API calls that run on the 
>> private state thread are blanket allowed to run at private stop.  The xnu 
>> Operating System plugin successfully gets global values to look up its 
>> threads.  So I’m not sure why that isn’t working for you.
>> 
>> Can you cook up a simple example showing the failure and I’ll have a look?
>> 
>> Jim
>> 
>> 
>>> On Feb 14, 2019, at 11:10 AM, Alexander Polyakov >> > wrote:
>>> 
>>> It is, the error is: error: error: process must be stopped.
>>> 
>>> I thought that the plugin (get_thread_info in my case) is invoked when the 
>>> process is already stopped, but it's not. Is it ok?
>>> 
>>> On Thu, Feb 14, 2019 at 9:53 PM Jim Ingham >> > wrote:
>>> All SBValues have an error in them (SBValue.GetError).  Does that say 
>>> anything interesting?
>>> 
>>> Jim
>>>   
>>> 
>>> 
>>> 
 On Feb 14, 2019, at 10:08 AM, Alexander Polyakov via lldb-dev 
 mailto:lldb-dev@lists.llvm.org>> wrote:
 
 Hi lldb-dev,
 
 I work on a custom implementation of OperatingSystem plugin using Python 
 and SB API. I’m trying to fetch information about some variables from the 
 target into the plugin, to do that I’m using the following Python code:
 ready_tasks = self._target.FindGlobalVariables(‘pxReadyTasksLists’, 
 1).GetValueAtIndex(0)
  
 When I do `print(ready_tasks)` I get:
 No value
  
 At the same time, doing the same actions inside lldb embedded interpreter 
 follows to:
 ` 
 print(lldb.target.FindGlobalVariables('pxReadyTasksLists',1).GetValueAtIndex(0))`
  
 (List_t [5]) pxReadyTasksLists = {
   [0] = {
 uxNumberOfItems = 0
 pxIndex = 0x
 xListEnd = {
   xItemValue = 0
   pxNext = 0x
   pxPrevious = 0x
 }
   }
 …
  
 Does anybody know what may cause such a behavior?
 
 -- 
 Alexander
 ___
 lldb-dev mailing list
 lldb-dev@lists.llvm.org 
 https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
 
>>> 
>>> 
>>> 
>>> -- 
>>> Alexander
>> 
>> -- 
>> Alexander
> 
> 
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-14 Thread Jim Ingham via lldb-dev
The simplest thing possible to reproduce the failure.  So some OS_Plugin 
implementation which tries to look up a global like this and fails, and some 
program source I can run it under that provides said global.  That way I can 
easily watch it fails as you describe when the plugin gets activated, and see 
why it isn’t allowing this call on private stop.

Jim


> On Feb 14, 2019, at 11:50 AM, Alexander Polyakov  
> wrote:
> 
> Sure, could you describe in more detail which example may help you?
> 
> чт, 14 февр. 2019 г. в 22:36, Jim Ingham  >:
> That’s a little complicated…  
> 
> lldb has two levels of stop state - private stops and public stops.  When the 
> process gets notification from the underlying process plugin that the process 
> stopped, it raises a private stop event.  That gets handled by the ShouldStop 
> mechanism on the private state thread in lldb, and then if the stop is judged 
> interesting to the user, it gets rebroadcast as a public stop.
> 
> For instance, when you issue a “step” command, lldb will stop and start the 
> process multiple times as it walks through the source line.  But only the 
> last of those stops are relevant to the user of LLDB, so all the other ones 
> exist only as private stops.
> 
> The SB API’s for the most part should only consider a “publicly stopped” 
> process accessible.  After all, you wouldn’t want some API to succeed 
> sometimes if you happen to catch it in the middle of a private stop.
> 
> But the OperatingSystem plugin needs to get called right after a private 
> stop, so it can provide threads for the ShouldStop mechanism.  We should 
> really have some formal mechanism whereby things like the OS plugin can 
> request elevated rights in the SB API’s, so that they can run at private stop 
> time.  IIRC, we instead have a hack where SB API calls that run on the 
> private state thread are blanket allowed to run at private stop.  The xnu 
> Operating System plugin successfully gets global values to look up its 
> threads.  So I’m not sure why that isn’t working for you.
> 
> Can you cook up a simple example showing the failure and I’ll have a look?
> 
> Jim
> 
> 
>> On Feb 14, 2019, at 11:10 AM, Alexander Polyakov > > wrote:
>> 
>> It is, the error is: error: error: process must be stopped.
>> 
>> I thought that the plugin (get_thread_info in my case) is invoked when the 
>> process is already stopped, but it's not. Is it ok?
>> 
>> On Thu, Feb 14, 2019 at 9:53 PM Jim Ingham > > wrote:
>> All SBValues have an error in them (SBValue.GetError).  Does that say 
>> anything interesting?
>> 
>> Jim
>>   
>> 
>> 
>> 
>>> On Feb 14, 2019, at 10:08 AM, Alexander Polyakov via lldb-dev 
>>> mailto:lldb-dev@lists.llvm.org>> wrote:
>>> 
>>> Hi lldb-dev,
>>> 
>>> I work on a custom implementation of OperatingSystem plugin using Python 
>>> and SB API. I’m trying to fetch information about some variables from the 
>>> target into the plugin, to do that I’m using the following Python code:
>>> ready_tasks = self._target.FindGlobalVariables(‘pxReadyTasksLists’, 
>>> 1).GetValueAtIndex(0)
>>>  
>>> When I do `print(ready_tasks)` I get:
>>> No value
>>>  
>>> At the same time, doing the same actions inside lldb embedded interpreter 
>>> follows to:
>>> ` 
>>> print(lldb.target.FindGlobalVariables('pxReadyTasksLists',1).GetValueAtIndex(0))`
>>>  
>>> (List_t [5]) pxReadyTasksLists = {
>>>   [0] = {
>>> uxNumberOfItems = 0
>>> pxIndex = 0x
>>> xListEnd = {
>>>   xItemValue = 0
>>>   pxNext = 0x
>>>   pxPrevious = 0x
>>> }
>>>   }
>>> …
>>>  
>>> Does anybody know what may cause such a behavior?
>>> 
>>> -- 
>>> Alexander
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org 
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
>>> 
>> 
>> 
>> 
>> -- 
>> Alexander
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-14 Thread Jim Ingham via lldb-dev
That’s a little complicated…  

lldb has two levels of stop state - private stops and public stops.  When the 
process gets notification from the underlying process plugin that the process 
stopped, it raises a private stop event.  That gets handled by the ShouldStop 
mechanism on the private state thread in lldb, and then if the stop is judged 
interesting to the user, it gets rebroadcast as a public stop.

For instance, when you issue a “step” command, lldb will stop and start the 
process multiple times as it walks through the source line.  But only the last 
of those stops are relevant to the user of LLDB, so all the other ones exist 
only as private stops.

The SB API’s for the most part should only consider a “publicly stopped” 
process accessible.  After all, you wouldn’t want some API to succeed sometimes 
if you happen to catch it in the middle of a private stop.

But the OperatingSystem plugin needs to get called right after a private stop, 
so it can provide threads for the ShouldStop mechanism.  We should really have 
some formal mechanism whereby things like the OS plugin can request elevated 
rights in the SB API’s, so that they can run at private stop time.  IIRC, we 
instead have a hack where SB API calls that run on the private state thread are 
blanket allowed to run at private stop.  The xnu Operating System plugin 
successfully gets global values to look up its threads.  So I’m not sure why 
that isn’t working for you.

Can you cook up a simple example showing the failure and I’ll have a look?

Jim


> On Feb 14, 2019, at 11:10 AM, Alexander Polyakov  
> wrote:
> 
> It is, the error is: error: error: process must be stopped.
> 
> I thought that the plugin (get_thread_info in my case) is invoked when the 
> process is already stopped, but it's not. Is it ok?
> 
> On Thu, Feb 14, 2019 at 9:53 PM Jim Ingham  > wrote:
> All SBValues have an error in them (SBValue.GetError).  Does that say 
> anything interesting?
> 
> Jim
>   
> 
> 
> 
>> On Feb 14, 2019, at 10:08 AM, Alexander Polyakov via lldb-dev 
>> mailto:lldb-dev@lists.llvm.org>> wrote:
>> 
>> Hi lldb-dev,
>> 
>> I work on a custom implementation of OperatingSystem plugin using Python and 
>> SB API. I’m trying to fetch information about some variables from the target 
>> into the plugin, to do that I’m using the following Python code:
>> ready_tasks = self._target.FindGlobalVariables(‘pxReadyTasksLists’, 
>> 1).GetValueAtIndex(0)
>>  
>> When I do `print(ready_tasks)` I get:
>> No value
>>  
>> At the same time, doing the same actions inside lldb embedded interpreter 
>> follows to:
>> ` 
>> print(lldb.target.FindGlobalVariables('pxReadyTasksLists',1).GetValueAtIndex(0))`
>>  
>> (List_t [5]) pxReadyTasksLists = {
>>   [0] = {
>> uxNumberOfItems = 0
>> pxIndex = 0x
>> xListEnd = {
>>   xItemValue = 0
>>   pxNext = 0x
>>   pxPrevious = 0x
>> }
>>   }
>> …
>>  
>> Does anybody know what may cause such a behavior?
>> 
>> -- 
>> Alexander
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org 
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
>> 
> 
> 
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] SB API is not working properly with OSPython plugin

2019-02-14 Thread Jim Ingham via lldb-dev
All SBValues have an error in them (SBValue.GetError).  Does that say anything 
interesting?

Jim
  



> On Feb 14, 2019, at 10:08 AM, Alexander Polyakov via lldb-dev 
>  wrote:
> 
> Hi lldb-dev,
> 
> I work on a custom implementation of OperatingSystem plugin using Python and 
> SB API. I’m trying to fetch information about some variables from the target 
> into the plugin, to do that I’m using the following Python code:
> ready_tasks = self._target.FindGlobalVariables(‘pxReadyTasksLists’, 
> 1).GetValueAtIndex(0)
>  
> When I do `print(ready_tasks)` I get:
> No value
>  
> At the same time, doing the same actions inside lldb embedded interpreter 
> follows to:
> ` 
> print(lldb.target.FindGlobalVariables('pxReadyTasksLists',1).GetValueAtIndex(0))`
>  
> (List_t [5]) pxReadyTasksLists = {
>   [0] = {
> uxNumberOfItems = 0
> pxIndex = 0x
> xListEnd = {
>   xItemValue = 0
>   pxNext = 0x
>   pxPrevious = 0x
> }
>   }
> …
>  
> Does anybody know what may cause such a behavior?
> 
> -- 
> Alexander
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Jim Ingham via lldb-dev


> On Jan 31, 2019, at 10:42 AM, Pavel Labath via lldb-dev 
>  wrote:
> 
> On 31/01/2019 19:26, Zachary Turner wrote:
>> Was the test failing specifically in the keyboard handler for up arrow, or 
>> was it failing in the command history searching code?  Because if it's the 
>> latter, then we could have a command which searches the command history.
> 
> The patch is r351313, if you want to look at it in detail. But, I don't think 
> this one example matters too much, since we will always have some code which 
> deals with the interactivity of the terminal. That will need to be tested 
> somehow.
> 
> Another example: we have a fairly complex piece of code that makes sure our 
> (lldb) prompt comes out in color. How do we write a test for that?

All the traffic back and forth with the terminal happens in the 
IOHandlerEditLine.  We should be able to get our hands on the Debuggers 
IOHandler and feed characters directly to it, and read the results.  So we 
should be able to write this kind of test by driving the debugger to whatever 
state you need with SB API and then just run one command and get the output 
string directly from the IOHandler.  We should be able to then scan that output 
for color codes.  I don't think we need an external process inspection tool to 
do this sort of thing.

Jim


> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Jim Ingham via lldb-dev
I don't think anybody uses these tests.  They are all time based benchmarks, 
and in the end there was just too much variability for them to be really 
useful.  We really need to do more work tracking performance, but I think a 
better approach is to focus on how much work we do (how many DIE's did you have 
to parse to do X, how many lookups did it take to compile an expression, or how 
many memory requests did a task take, things like that.  Those seem to me 
likely to be ore stable.

That said, I'm unclear why any of the benchmark tests would need pexpect to 
function.  The Driver used to have a lot more functionality in it that has 
since been moved into the SB API's.  Maybe there was good reason for doing 
performance testing through the driver but I can't think of any good reason for 
doing that nowadays. 

Jim

> On Jan 31, 2019, at 10:23 AM, Davide Italiano via lldb-dev 
>  wrote:
> 
> On Thu, Jan 31, 2019 at 10:09 AM Pavel Labath  wrote:
>> 
>> On 31/01/2019 02:32, Davide Italiano via lldb-dev wrote:
>>> As you probably know (I didn’t), lldb embeds its own version of
>>> `pexpect-2.4`, which doesn’t support python3.
>>> This is the (relatively short) list of tests relying on pyexpect:
>>> 
>>> testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
>>> # 7 (EOF)
>>> testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
>>> testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
>>> testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
>>> testcases/lldbtest.py:import pexpect
>>> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
>>> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
>>> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
>>> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
>>> testcases/lldbpexpect.py:import pexpect
>>> testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
>>> testcases/darwin_log.py:import pexpect
>>> testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
>>> testcases/benchmarks/stepping/TestSteppingSpeed.py:import pexpect
>>> testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
>>>   import pexpect
>>> testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
>>>import pexpect
>>> testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
>>>import pexpect
>>> testcases/benchmarks/expression/TestExpressionCmd.py:import pexpect
>>> testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
>>> testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
>>> testcases/benchmarks/startup/TestStartupDelays.py:import pexpect
>>> testcases/functionalities/command_regex/TestCommandRegex.py:
>>> import pexpect
>>> testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
>>>import pexpect
>>> testcases/functionalities/format/TestFormats.py:import pexpect
>>> 
>>> (I count 14, but there might be something else).
>>> 
>>> I audited all of them and from what I see they’re almost all testing the 
>>> driver.
>>> I had a chat with my coworkers and we agreed it's reasonable to
>>> replace them with lit tests (as they're just running commands).
>>> This would allow us to get rid of an external dependency, which
>>> happened to be cause of trouble in the past.
>>> 
>>> Are there any objections?
>>> 
>>> Thanks,
>>> 
>> 
>> I'm not a fan of pexpect, and if these tests can be converted to lit,
>> then I'm all for it. But I do have a question.
>> 
>> There is a class of tests that cannot be written in the current lit
>> framework, but they can with pexpect. A couple of weeks ago we had a
>> patch fixing a bug where pressing up arrow while searching through the
>> command history caused a crash. In the end a test for this was not
>> included because it was hard for a reason unrelated to pexpect, but
>> without pexpect (or something equivalent) writing a test for this would
>> be impossible.
>> 
> 
> I don't know about this, to be honest. Maybe lit should grow an
> interactive mode somehow to accomodate for this functionality?
> I'm not an expert in how it's implemented so that could be hard to achieve.
> FWIW, I haven't seen anything that really requires interactivity, but
> I have to admit I haven't looked really deeply.
> 
>> What's our story for testing interactive command-line functionalities?
>> The way I see it, if we don't use pexpect, we'll either have to use some
>> other tool which simulates a realistic terminal, or write our own. (We
>> already have one attempt for this in
>> unittests/Editline/EditlineTest.cpp, but this would need more work to be
>> fully functional.)
>> 
>> pl
>> 
>> 
>> PS: Does anyone actually use the benchmark tests? Can we just delete them?
> 
> I don't know. Maybe Jim knows. I personally 

Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Jim Ingham via lldb-dev
Even if it was the keyboard handler, lldb feeds characters to edit line through 
the IO Handler, so it should be possible to emulate the up arrow as well.  If 
there are reasons why that's not feasible, we should be able to make it work.  
This seems a tractable problem to me, and to me seems a better place to put 
effort than something like pexpect.

Jim


> On Jan 31, 2019, at 10:26 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> Was the test failing specifically in the keyboard handler for up arrow, or 
> was it failing in the command history searching code?  Because if it's the 
> latter, then we could have a command which searches the command history.
> 
> On Thu, Jan 31, 2019 at 10:23 AM Davide Italiano via lldb-dev 
>  wrote:
> On Thu, Jan 31, 2019 at 10:09 AM Pavel Labath  wrote:
> >
> > On 31/01/2019 02:32, Davide Italiano via lldb-dev wrote:
> > > As you probably know (I didn’t), lldb embeds its own version of
> > > `pexpect-2.4`, which doesn’t support python3.
> > > This is the (relatively short) list of tests relying on pyexpect:
> > >
> > > testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
> > >  # 7 (EOF)
> > > testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
> > > testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> > > testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> > > testcases/lldbtest.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/lldbpexpect.py:import pexpect
> > > testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
> > > testcases/darwin_log.py:import pexpect
> > > testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
> > > testcases/benchmarks/stepping/TestSteppingSpeed.py:import pexpect
> > > testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
> > >import pexpect
> > > testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
> > > import pexpect
> > > testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
> > > import pexpect
> > > testcases/benchmarks/expression/TestExpressionCmd.py:import 
> > > pexpect
> > > testcases/benchmarks/expression/TestRepeatedExprs.py:import 
> > > pexpect
> > > testcases/benchmarks/expression/TestRepeatedExprs.py:import 
> > > pexpect
> > > testcases/benchmarks/startup/TestStartupDelays.py:import pexpect
> > > testcases/functionalities/command_regex/TestCommandRegex.py:
> > > import pexpect
> > > testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
> > > import pexpect
> > > testcases/functionalities/format/TestFormats.py:import pexpect
> > >
> > > (I count 14, but there might be something else).
> > >
> > > I audited all of them and from what I see they’re almost all testing the 
> > > driver.
> > > I had a chat with my coworkers and we agreed it's reasonable to
> > > replace them with lit tests (as they're just running commands).
> > > This would allow us to get rid of an external dependency, which
> > > happened to be cause of trouble in the past.
> > >
> > > Are there any objections?
> > >
> > > Thanks,
> > >
> >
> > I'm not a fan of pexpect, and if these tests can be converted to lit,
> > then I'm all for it. But I do have a question.
> >
> > There is a class of tests that cannot be written in the current lit
> > framework, but they can with pexpect. A couple of weeks ago we had a
> > patch fixing a bug where pressing up arrow while searching through the
> > command history caused a crash. In the end a test for this was not
> > included because it was hard for a reason unrelated to pexpect, but
> > without pexpect (or something equivalent) writing a test for this would
> > be impossible.
> >
> 
> I don't know about this, to be honest. Maybe lit should grow an
> interactive mode somehow to accomodate for this functionality?
> I'm not an expert in how it's implemented so that could be hard to achieve.
> FWIW, I haven't seen anything that really requires interactivity, but
> I have to admit I haven't looked really deeply.
> 
> > What's our story for testing interactive command-line functionalities?
> > The way I see it, if we don't use pexpect, we'll either have to use some
> > other tool which simulates a realistic terminal, or write our own. (We
> > already have one attempt for this in
> > unittests/Editline/EditlineTest.cpp, but this would need more work to be
> > fully functional.)
> >
> > pl
> >
> >
> > PS: Does anyone actually use the benchmark tests? Can we just delete them?
> 
> I don't know. Maybe Jim knows. I personally don't use them.
> 

Re: [lldb-dev] OperatingSystem plugin

2018-12-19 Thread Jim Ingham via lldb-dev
You would use an operating system plugin in cases where the underlying process 
interface doesn't tell the complete story for the OS threads.  For instance, a 
lot of kernel and bare board OS'es have a gdb-remote stub that just describes 
the state of the threads currently running on the cores of the machine.  Any 
swapped out threads are unknown to it.  So the Operating system plugin 
reconstructs the swapped out threads by looking at OS specific data structures.

We didn't think that lldb should own the support for every variant of every OS 
which might want to reconstruct threads from some data structures in the OS.  
After all, this isn't dealing with fairly stable API's - like the Platforms do. 
 The OS plugins deal with internal data structures, which tend to change fairly 
frequently.  So trying to package them with lldb doesn't seem supportable.  

By making the OS plugin something that comes from outside lldb, we allow the 
kernel developers to provide this knowledge on demand.  That is very 
convenient.  For instance, on macOS, the dSYM bundle for the mach kernel 
contains the Operating System plugin for that kernel.  When lldb is used for 
kernel debugging, it finds the dSYM associated with the kernel it is targeting 
and loads in the python support from the dSYM. That way it is ensured to get 
the correct plugin for that kernel.  It also means that lldb can support kernel 
versions that didn't even exist when it was made.

As for why this is done with Python and not the C++ SB API's, there's nothing 
to stop it from also being in C++, you can actually write loadable C++ plugins 
with the SB API's for data formatters, for instance.   But there's generally no 
significant performance advantage to doing that, and its less convenient, so 
there just hasn't been much demand for it.

Jim


> On Dec 19, 2018, at 12:44 PM, Alexander Polyakov via lldb-dev 
>  wrote:
> 
> Hi lldb-dev,
> 
> Could someone explain me why do we use python (OperatingSystemPython) to 
> describe OS objects like threads? What are the advantages of such an approach 
> in comparison to C++ used in Platform plugin for example? IMO, the 
> OperatingSystem plugin could be more like the Platform one, it could have a 
> separate directory for each OS with its own implementation inside.
> 
> -- 
> Alexander
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


[lldb-dev] ObjectFileBreakpad breaks TestFunctionStarts on the MacOS bot

2018-12-07 Thread Jim Ingham via lldb-dev
The ObjectFileBreakpad change (r348592) first got tested on the bot for:

http://lab.llvm.org:8080/green/view/LLDB/job/lldb-cmake/13712/

and immediately TestFunctionStarts started failing.

You probably didn't get a mail because the previous test run failed with one of 
these "concurrent signal" tests which seem to be unreliable.

Jim

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] When should ArchSpecs match?

2018-12-06 Thread Jim Ingham via lldb-dev
I agree with Davide.  Particularly if there’s code that is relying on the 
“IsExactMatch” not behaving like the function name makes clear it obviously 
should behave, we should straighten that out.  Otherwise reasoning about this 
will be too confusing.

Jim


> On Dec 6, 2018, at 3:26 PM, Davide Italiano via lldb-dev 
>  wrote:
> 
> On Thu, Dec 6, 2018 at 3:20 PM Adrian Prantl via lldb-dev
> mailto:lldb-dev@lists.llvm.org>> wrote:
>> 
>> I was puzzled by the behavior of ArchSpec::IsExactMatch() and 
>> IsCompatibleMatch() yesterday, so I created a couple of unit tests to 
>> document the current behavior. Most of the tests make perfect sense, but a 
>> few edge cases really don't behave like I would have expected them to.
>> 
>>>  {
>>>ArchSpec A("arm64-*-*");
>>>ArchSpec B("arm64-apple-ios");
>>>ASSERT_FALSE(A.IsExactMatch(B));
>>>// FIXME: This looks unintuitive and we should investigate whether
>>>// this is the desired behavior.
>>>ASSERT_FALSE(A.IsCompatibleMatch(B));
>>>  }
>>>  {
>>>ArchSpec A("x86_64-*-*");
>>>ArchSpec B("x86_64-apple-ios-simulator");
>>>ASSERT_FALSE(A.IsExactMatch(B));
>>>// FIXME: See above, though the extra environment complicates things.
>>>ASSERT_FALSE(A.IsCompatibleMatch(B));
>>>  }
>>>  {
>>>ArchSpec A("x86_64");
>>>ArchSpec B("x86_64-apple-macosx10.14");
>>>// FIXME: The exact match also looks unintuitive.
>>>ASSERT_TRUE(A.IsExactMatch(B));
>>>ASSERT_TRUE(A.IsCompatibleMatch(B));
>>>  }
>>> 
>> 
>> Particularly, I believe that:
>> - ArchSpec("x86_64-*-*") and ArchSpec("x86_64") should behave the same.
> 
> Yes.
> 
>> - ArchSpec("x86_64").IsExactMatch("x86_64-apple-macosx10.14") should be 
>> false.
> 
> I'm at a loss trying to understand how this could not be false.
> It would mean that when we instantiate a Triple and we don't have an
> arch we believe it's a Mach-O binary.
> This sounds really wrong, and if there's code in lldb living under the
> assumption this is true, we might consider getting rid of it (or
> fixing it).
> 
>> - ArchSpec("x86_64-*-*").IsCompatibleMath("x86_64-apple-macosx") should be 
>> true.
>> 
> Yes.
> 
> --
> Davide
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org 
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
> 
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] New target; where to start with thread plans

2018-11-29 Thread Jim Ingham via lldb-dev
Ah, that’s right, we did it in debugserver as well.  Apple used to have a 
disassembly framework that we used for the purpose (separate from llvm).  But 
that’s long gone.

Jim


> On Nov 29, 2018, at 1:22 AM, Pavel Labath  wrote:
> 
> On 28/11/2018 21:05, Jim Ingham via lldb-dev wrote:
>> The main complication would be if your new target didn't support 
>> hardware-single-step.  If that's the case you will have to have a 
>> Disassembler that can predict the target PC of all instructions (when 
>> stopped at that instruction) since you'll have to use software breakpoints 
>> to drive the lowest level of stepping.  There's support in there for 
>> handling that at a high level since the first couple of iPhone ARM chips 
>> didn't have single step support but we haven't actually had on any real 
>> systems we support that lack hardware single step in years, so it may need 
>> some reviving.
> 
> Linux still uses software single stepping on arm(32), but the code for that 
> is in lldb-server, so the client doesn't even know it's being used. Of course 
> that won't help you if you're not using lldb-server.
> 
> Was there ever software single-stepping support in the client?
> 
> pl

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] New target; where to start with thread plans

2018-11-28 Thread Jim Ingham via lldb-dev
You will need to add ABI support.  You'll also need to make an Unwinder that 
works for your platform.  But the thread plan code is pretty generic and relies 
on the Unwinder and generic features of the Disassembler (mostly "IsBranch") to 
do its job.  For instance, thread step-out just asks the Unwinder for the 
return address for the current frame, sets a breakpoint there and continues.  
The architecture specific bits of that are all hidden by the Unwinder.

The main complication would be if your new target didn't support 
hardware-single-step.  If that's the case you will have to have a Disassembler 
that can predict the target PC of all instructions (when stopped at that 
instruction) since you'll have to use software breakpoints to drive the lowest 
level of stepping.  There's support in there for handling that at a high level 
since the first couple of iPhone ARM chips didn't have single step support but 
we haven't actually had on any real systems we support that lack hardware 
single step in years, so it may need some reviving.

Jim


> On Nov 28, 2018, at 9:50 AM, Adrian Harris via lldb-dev 
>  wrote:
> 
> Hi Everyone,
> 
> I’ve added basic support for a new target (via gdb-remote) and now it is time 
> to add ABI support to lldb to improve the debug experience. My understanding 
> is that I need to add an ABI plugin (we have a very basic stub now) and 
> thread-plan support. As a first step I’d like to be able to support the 
> ‘thread step-out’ command. Some basic guidance would be very much appreciated.
> 
> Thanks!
> Adrian
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Debugging Python scripts (backtraces, variables) with LLDB

2018-11-26 Thread Jim Ingham via lldb-dev
lldb does have an affordance for synthetic threads, but at present, those have 
to be memory threads, and once you have the 0th frame, they are backtraced just 
like ordinary threads.  So they are a start but don't provide a way to make up 
synthetic frames within a thread.  So I don't think that they do what you want 
exactly.

Recently, Kuba added the notion of a "frame recognizer" that can detect certain 
frames and alter their presentation.  The first use is adding variables to a 
frame - mostly to provide argument values to known functions for which you 
don't have debug information.
You could use that directly, recognizing the python "push a new stack frame" 
function and adding the Python locals as new variables.  The problem you will 
run into here is that we don't have a Python type system, so you'd have to 
represent these variables as C types.

In my original notion of the frame recognizers, I thought we would have more 
the directives like: "hide the next X frames", and "redirect the source info 
for this frame" and "change the language of this frame".  Then the higher layer 
frame printing code would follow the recognizer directives to produce a mixed 
stack frame.  There isn't any support for this in what Kuba did, but I think 
the framework he put in place could be extended along these lines to do the 
sort of thing you are thinking of.

Jim



> On Nov 20, 2018, at 10:08 AM, Alexandru Croitor via lldb-dev 
>  wrote:
> 
> Thanks for the link. I'm aware that VS provides that functionality (although 
> I forgot that it was open-source).
> 
> My aim would be to implement something similar in LLDB though.
> 
> 
>> On 20. Nov 2018, at 18:49, Zachary Turner  wrote:
>> 
>> 
>> 
>> On Tue, Nov 20, 2018 at 8:51 AM Alexandru Croitor via lldb-dev 
>>  wrote:
>> 
>> I would appreciate, if someone could point me to some relevant code that 
>> does something similar to what I'm asking, so I could use it as a base point 
>> for exploration.
>> 
>> Many thanks.
>> 
>> Not sure how much it will help you, but on Windows if you're using MS Visual 
>> Studio, their debugger does this.  You can seamlessly step between managed 
>> and native code and see Python callstacks interspersed with  native 
>> callstacks.  It's all open source but it's quite a lot of code to dig 
>> through, and unless you have a Windows machien, you won't be able to play 
>> around with it anyway.
>> 
>> https://github.com/Microsoft/PTVS/tree/master/Python/Product
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC] OS Awareness in LLDB

2018-11-12 Thread Jim Ingham via lldb-dev


> On Nov 11, 2018, at 10:30 AM, Alexander Polyakov  
> wrote:
> 
> The data formatter in lldb is a subsystem that allows users to define custom 
> display options, the Platform is the agent that provides a way to work with a 
> specific platform. I agree that we need the Platform to provide the OS 
> specific data formatters, but also the Platform should have a functionality 
> to find all OS mutexes, am I right?

Yes, that seems right to me.

Jim


> 
> On Thu, Nov 1, 2018 at 7:37 PM Jim Ingham  wrote:
> I don’t think we are disagreeing, probably I was just being a little too 
> terse.  What I’m saying is that to PRESENT a mutex in some detail, we need an 
> OS specific data formatter, so in that case, we just need some agent, either 
> the Platform or the SystemRuntime (but the Platform seems better to me for 
> this) to provide the OS specific data formatters.  But we don’t need a new 
> facility for that, just a way to dial up the right OS-specific instances.  
> And if we can reuse more general features to present this sort of OS specific 
> info we are better off; it reduces lldb’s surface area and provides 
> facilities we might find other neat ways to use.
> 
> OTOH, for a runtime feature like “Gather all the mutexes currently in flight” 
> we would need some new code.  There’s nothing in lldb that does that sort of 
> job.  The SystemRuntime seems like the right place for that code to go.
> 
> Jim
> 
> 
>> On Nov 1, 2018, at 1:25 AM, Alexander Polyakov  
>> wrote:
>> 
>> I'm new in plugin ecosystem, so I have some misunderstanding. You wrote that 
>> to comprehend mutexes we just need a data formatter, but how can we get the 
>> raw data of all mutexes in our OS? I thought I was supposed to write a 
>> generic code that will use a user-defined (specific for each OS) way to 
>> collect all mutexes and then use some data formatter to show them.
>> 
>> On Thu, Nov 1, 2018 at 2:31 AM Jim Ingham  wrote:
>> As Jason pointed out, we also have the SystemRuntime Plugin.  That is 
>> intended to provide extra runtime available information based on the current 
>> system.  For instance, on Darwin we use it to present the libdispatch queue 
>> view of threads on the system, and to decorate threads that are doing work 
>> on behalf of some queue with the state of the thread that enqueued the work 
>> at the time the work  item was enqueued.
>> 
>> If for instance you had a way to gather all the locks in the process 
>> (something we've been asked to do but I don't know how to do it on 
>> Darwin...), that would be the place to put that functionality.
>> 
>> Jim
>> 
>> 
>> > On Oct 31, 2018, at 4:05 PM, Jim Ingham via lldb-dev 
>> >  wrote:
>> > 
>> > Right now, the OS plugin only supports the job of adding threads.  And 
>> > that makes more sense as a plugin, because for instance if you had a 
>> > cooperative threading scheme that you were laying on top of the system 
>> > threads in a User Space process, you can use the Operating System plugin 
>> > to show you the cooperative threads.  This is not an abstract example...  
>> > I think it should stay with just that job.
>> > 
>> > The place where lldb holds this sort of knowledge is supposed to be in the 
>> > Platform class.  So for instance, to comprehend mutexes you really just 
>> > need a data formatter.  The trick is that it has to be a data formatter 
>> > that is provided by the platform.  Similarly you want to have frame 
>> > recognizers for interesting lower-level calls in the system.  The 
>> > machinery will shortly be there to do that, but loading the particular 
>> > recognizers will either need to be done by hand, or coordinated by the 
>> > Platform.  In general, I think most of the kinds of re-presentation you 
>> > need to do to make OS objects and processes more comprehensible can be 
>> > built as general mechanisms like the above.  Then the Platform can 
>> > coordinate providing the set of more general transformations that are 
>> > appropriate to the Platform you are targeting.
>> > 
>> > Jim
>> > 
>> > 
>> > 
>> >> On Oct 31, 2018, at 3:44 PM, Alexander Polyakov  
>> >> wrote:
>> >> 
>> >> So, if I understand you write, I can look at OS plugin and add a support 
>> >> of mutexes or memory pages for example?
>> >> 
>> >> чт, 1 нояб. 2018 г. в 1:05, Jim Ingham :
>> >> lldb has one feature - the "Operating System Plugin" that is specifically 
>> >> designed

Re: [lldb-dev] problem resolving symbolic breakpoint on a remote target

2018-11-08 Thread Jim Ingham via lldb-dev


> On Nov 8, 2018, at 11:18 AM, Adrian Harris  wrote:
> 
> Thanks Jim - that makes sense for the types of targets that lldb interacts 
> with mostly.
> 
> In my particular case, nothing is getting 'launched' - rather I'm attaching 
> to an already running target. The elf that I'm pointing to is an exact 
> executable image as the target has no OS to speak of and is very primitive.
> 
> Would it make sense to write a simple process plugin for my target? I'm a 
> little fuzzy on exactly how the 'process' interacts with the gdb-remote 
> target in lldb however.
> 

What you need is a DynamicLoader for the OS you are targeting.  The 
ProcessGDBRemote queries all the loaded DynamicLoader plugins for the one that 
matches the current process, and installs the one that matches.  Then that 
plugin has the job of filling in the section load list.  In your case, that 
would just be to copy all the section addresses directly to the section load 
map.

> More generally, how does lldb figure out where symbols are in an arbitrary 
> target when the use mode is attach, as opposed to launching the process 
> (therefore learning the layout).

That's handled the same way as launch.  When you attach, lldb gets host info 
from the other side of the connection (you can also help this out by providing 
an appropriate triple when you create the target before attaching).  It uses 
that to figure out which DynamicLoader plugin to use.  And again, it's the 
DynamicLoader's job to look at the memory of the target process and figure out 
where the images got mapped.  How it does that is magic specific to each plugin.

JIm


> 
> Adrian
> 
> 
> 
>> On Nov 8, 2018, at 11:36 AM, Jim Ingham  wrote:
>> 
>> lldb finds the symbol you asked for in the elf file's symbols, and makes a 
>> "location" for that address in that binary (as a section-relative address).  
>> But that won't help it actually SET the breakpoint, since that doesn't tell 
>> us where that section will end up in the executable image when it runs.  It 
>> is the job of the DynamicLoader plugin for whatever platform you are 
>> debugging to observe a process as it is getting launched and register where 
>> all the sections land memory.  The section load map is the storage for this 
>> information.  If that isn't getting filled in then we won't be able to 
>> actually set the breakpoint in the target.  It sounds like the Dynamic 
>> Loader step is missing.
>> 
>> Jim
>> 
>> 
>>> On Nov 8, 2018, at 10:20 AM, Adrian Harris via lldb-dev 
>>>  wrote:
>>> 
>>> (lldb) file tile.elf
>>> Current executable set to 'tile.elf' (cs).
>>> (lldb) b main
>>> lldb Target::AddBreakpoint (internal = no) => break_id = 1: 
>>> name = 'main'
>>> 
>>> 
>>> lldb Added location: 1.1: 
>>> module = tile.elf
>>> compile unit = token_pass.c
>>> function = main
>>> location = token_pass.c:74
>>> address = tile.elf[0x0410]
>>> resolved = false
>>> hit count = 0   
>>> 
>>> 
>>> Breakpoint 1: where = tile.elf`main + 16 at token_pass.c:74, address = 
>>> 0x0410
>>> (lldb) gdb-remote server4:33722
>>> Process 1 stopped
>>> * thread #1, stop reason = signal SIGTRAP
>>>  frame #0: 0x0422
>>> tile.elf`main:
>>>  tile.elf[0x422] <+34>: eq16   r5, 0
>>>  tile.elf[0x424] <+36>: addc16 r0 = r7, 0
>>>  tile.elf[0x428] <+40>: eq16   r0, r7
>>>  tile.elf[0x42a] <+42>: jc 0xb
>>> (lldb) break list
>>> Current breakpoints:
>>> 1: name = 'main', locations = 1
>>> 1.1: where = tile.elf`main + 16 at token_pass.c:74, address = 
>>> tile.elf[0x0410], unresolved, hit count = 0 
>>> 
>>> So it is still not resolved.
>>> 
>>> Adrian
>>> 
>>> 
 On Nov 8, 2018, at 11:06 AM, ted.woodw...@codeaurora.org wrote:
 
 What happens if you do this:
 
 (lldb) file tile.elf
 (lldb) b main
 (lldb) gdb-remote 
 
 ?
 
 --
 Ted Woodward
 Qualcomm Innovation Center, Inc.
 Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux 
 Foundation Collaborative Project
 
 -Original Message-
 From: lldb-dev  On Behalf Of Adrian 
 Harris via lldb-dev
 Sent: Thursday, November 8, 2018 11:09 AM
 To: via lldb-dev 
 Subject: [lldb-dev] problem resolving symbolic breakpoint on a remote 
 target
 
 Hi Everyone,
 
 I'm unable to resolve *symbolic* breakpoints on a gdb-remote target. 
 Address breakpoints work fine. I suspect this is probably some form of 
 user-error, but I've had no luck figuring it out on my own.
 
 My target has llvm support and lldb has been patched to add a new target 
 as well.
 
 Debug information is correct in the image.
 
 My steps are as follows:
 
 (lldb) gdb-remote 
 ... connection happens
 (lldb) image add tile.elf
 (lldb) target modules list
 [  0] 89569B3D----tile.elf 
 (lldb) break main
 lldb Target::AddBreakpoint (internal = no) => break_id = 1: 
 name 

Re: [lldb-dev] problem resolving symbolic breakpoint on a remote target

2018-11-08 Thread Jim Ingham via lldb-dev
lldb finds the symbol you asked for in the elf file's symbols, and makes a 
"location" for that address in that binary (as a section-relative address).  
But that won't help it actually SET the breakpoint, since that doesn't tell us 
where that section will end up in the executable image when it runs.  It is the 
job of the DynamicLoader plugin for whatever platform you are debugging to 
observe a process as it is getting launched and register where all the sections 
land memory.  The section load map is the storage for this information.  If 
that isn't getting filled in then we won't be able to actually set the 
breakpoint in the target.  It sounds like the Dynamic Loader step is missing.

Jim


> On Nov 8, 2018, at 10:20 AM, Adrian Harris via lldb-dev 
>  wrote:
> 
> (lldb) file tile.elf
> Current executable set to 'tile.elf' (cs).
> (lldb) b main
> lldb Target::AddBreakpoint (internal = no) => break_id = 1: name 
> = 'main'
> 
> 
> lldb Added location: 1.1: 
>  module = tile.elf
>  compile unit = token_pass.c
>  function = main
>  location = token_pass.c:74
>  address = tile.elf[0x0410]
>  resolved = false
>  hit count = 0   
> 
> 
> Breakpoint 1: where = tile.elf`main + 16 at token_pass.c:74, address = 0x0410
> (lldb) gdb-remote server4:33722
> Process 1 stopped
> * thread #1, stop reason = signal SIGTRAP
>frame #0: 0x0422
> tile.elf`main:
>tile.elf[0x422] <+34>: eq16   r5, 0
>tile.elf[0x424] <+36>: addc16 r0 = r7, 0
>tile.elf[0x428] <+40>: eq16   r0, r7
>tile.elf[0x42a] <+42>: jc 0xb
> (lldb) break list
> Current breakpoints:
> 1: name = 'main', locations = 1
>  1.1: where = tile.elf`main + 16 at token_pass.c:74, address = 
> tile.elf[0x0410], unresolved, hit count = 0 
> 
> So it is still not resolved.
> 
> Adrian
> 
> 
>> On Nov 8, 2018, at 11:06 AM, ted.woodw...@codeaurora.org wrote:
>> 
>> What happens if you do this:
>> 
>> (lldb) file tile.elf
>> (lldb) b main
>> (lldb) gdb-remote 
>> 
>> ?
>> 
>> --
>> Ted Woodward
>> Qualcomm Innovation Center, Inc.
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux 
>> Foundation Collaborative Project
>> 
>> -Original Message-
>> From: lldb-dev  On Behalf Of Adrian Harris 
>> via lldb-dev
>> Sent: Thursday, November 8, 2018 11:09 AM
>> To: via lldb-dev 
>> Subject: [lldb-dev] problem resolving symbolic breakpoint on a remote target
>> 
>> Hi Everyone,
>> 
>> I'm unable to resolve *symbolic* breakpoints on a gdb-remote target. Address 
>> breakpoints work fine. I suspect this is probably some form of user-error, 
>> but I've had no luck figuring it out on my own.
>> 
>> My target has llvm support and lldb has been patched to add a new target as 
>> well.
>> 
>> Debug information is correct in the image.
>> 
>> My steps are as follows:
>> 
>> (lldb) gdb-remote 
>> ... connection happens
>> (lldb) image add tile.elf
>> (lldb) target modules list
>> [  0] 89569B3D----tile.elf 
>> (lldb) break main
>> lldb Target::AddBreakpoint (internal = no) => break_id = 1: name 
>> = 'main'
>> 
>> 
>> lldb warning: Tried to add breakpoint site at 0x 
>> but it was already present.
>> 
>> lldb Added location: 1.1: 
>> module = tile.elf
>> compile unit = token_pass.c
>> function = main
>> location = token_pass.c:74
>> address = tile.elf[0x0410]
>> resolved = false
>> hit count = 0   
>> 
>> 
>> Breakpoint 1: where = tile.elf`main + 16 at token_pass.c:74, address = 0x0410
>> 
>> I traced the breakpoint resolving path in lldb and it ultimately fails in 
>> this function:
>> addr_t
>> SectionLoadList::GetSectionLoadAddress(const lldb::SectionSP ) const 
>> {  // TODO: add support for the same section having multiple load addresses  
>> addr_t section_load_addr = LLDB_INVALID_ADDRESS;  if (section) {
>>  std::lock_guard guard(m_mutex);
>>  sect_to_addr_collection::const_iterator pos =
>>  m_sect_to_addr.find(section.get());
>> 
>>  if (pos != m_sect_to_addr.end())
>>section_load_addr = pos->second;
>> }
>> return section_load_addr;
>> }
>> 
>> ... because the m_sect_to_addr map is not populated. I think that should 
>> happen in 
>> 
>> bool SectionLoadList::SetSectionLoadAddress(const lldb::SectionSP ,
>>  addr_t load_addr,
>>  bool warn_multiple) {
>> 
>> .. but it is never called. This is what makes me think I'm leaving out a 
>> critical step.
>> 
>> Thanks for any help,
>> Adrian
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org

Re: [lldb-dev] [RFC] OS Awareness in LLDB

2018-11-01 Thread Jim Ingham via lldb-dev
I don’t think we are disagreeing, probably I was just being a little too terse. 
 What I’m saying is that to PRESENT a mutex in some detail, we need an OS 
specific data formatter, so in that case, we just need some agent, either the 
Platform or the SystemRuntime (but the Platform seems better to me for this) to 
provide the OS specific data formatters.  But we don’t need a new facility for 
that, just a way to dial up the right OS-specific instances.  And if we can 
reuse more general features to present this sort of OS specific info we are 
better off; it reduces lldb’s surface area and provides facilities we might 
find other neat ways to use.

OTOH, for a runtime feature like “Gather all the mutexes currently in flight” 
we would need some new code.  There’s nothing in lldb that does that sort of 
job.  The SystemRuntime seems like the right place for that code to go.

Jim


> On Nov 1, 2018, at 1:25 AM, Alexander Polyakov  wrote:
> 
> I'm new in plugin ecosystem, so I have some misunderstanding. You wrote that 
> to comprehend mutexes we just need a data formatter, but how can we get the 
> raw data of all mutexes in our OS? I thought I was supposed to write a 
> generic code that will use a user-defined (specific for each OS) way to 
> collect all mutexes and then use some data formatter to show them.
> 
> On Thu, Nov 1, 2018 at 2:31 AM Jim Ingham  <mailto:jing...@apple.com>> wrote:
> As Jason pointed out, we also have the SystemRuntime Plugin.  That is 
> intended to provide extra runtime available information based on the current 
> system.  For instance, on Darwin we use it to present the libdispatch queue 
> view of threads on the system, and to decorate threads that are doing work on 
> behalf of some queue with the state of the thread that enqueued the work at 
> the time the work  item was enqueued.
> 
> If for instance you had a way to gather all the locks in the process 
> (something we've been asked to do but I don't know how to do it on 
> Darwin...), that would be the place to put that functionality.
> 
> Jim
> 
> 
> > On Oct 31, 2018, at 4:05 PM, Jim Ingham via lldb-dev 
> > mailto:lldb-dev@lists.llvm.org>> wrote:
> > 
> > Right now, the OS plugin only supports the job of adding threads.  And that 
> > makes more sense as a plugin, because for instance if you had a cooperative 
> > threading scheme that you were laying on top of the system threads in a 
> > User Space process, you can use the Operating System plugin to show you the 
> > cooperative threads.  This is not an abstract example...  I think it should 
> > stay with just that job.
> > 
> > The place where lldb holds this sort of knowledge is supposed to be in the 
> > Platform class.  So for instance, to comprehend mutexes you really just 
> > need a data formatter.  The trick is that it has to be a data formatter 
> > that is provided by the platform.  Similarly you want to have frame 
> > recognizers for interesting lower-level calls in the system.  The machinery 
> > will shortly be there to do that, but loading the particular recognizers 
> > will either need to be done by hand, or coordinated by the Platform.  In 
> > general, I think most of the kinds of re-presentation you need to do to 
> > make OS objects and processes more comprehensible can be built as general 
> > mechanisms like the above.  Then the Platform can coordinate providing the 
> > set of more general transformations that are appropriate to the Platform 
> > you are targeting.
> > 
> > Jim
> > 
> > 
> > 
> >> On Oct 31, 2018, at 3:44 PM, Alexander Polyakov  >> <mailto:polyakov@gmail.com>> wrote:
> >> 
> >> So, if I understand you write, I can look at OS plugin and add a support 
> >> of mutexes or memory pages for example?
> >> 
> >> чт, 1 нояб. 2018 г. в 1:05, Jim Ingham  >> <mailto:jing...@apple.com>>:
> >> lldb has one feature - the "Operating System Plugin" that is specifically 
> >> designed for debugging threads in kernel contexts.  The OS plugin allows a 
> >> kernel to present it's notion of threads into lldb_private::Threads.  The 
> >> xnu kernel macros have an implementation of this, as do some other 
> >> embedded OS'es.  lldb actually gets used pretty extensively for debugging 
> >> xnu - the Darwin kernel.
> >> 
> >> Kuba is adding the notion of "Frame recognizers" which can give 
> >> significance to particular functions when they appear on the stack (for 
> >> instance displaying the first argument of read, etc. as a file handle even 
> >> if you have no debug information for it.)  That's a

Re: [lldb-dev] [RFC] OS Awareness in LLDB

2018-10-31 Thread Jim Ingham via lldb-dev
As Jason pointed out, we also have the SystemRuntime Plugin.  That is intended 
to provide extra runtime available information based on the current system.  
For instance, on Darwin we use it to present the libdispatch queue view of 
threads on the system, and to decorate threads that are doing work on behalf of 
some queue with the state of the thread that enqueued the work at the time the 
work  item was enqueued.

If for instance you had a way to gather all the locks in the process (something 
we've been asked to do but I don't know how to do it on Darwin...), that would 
be the place to put that functionality.

Jim


> On Oct 31, 2018, at 4:05 PM, Jim Ingham via lldb-dev 
>  wrote:
> 
> Right now, the OS plugin only supports the job of adding threads.  And that 
> makes more sense as a plugin, because for instance if you had a cooperative 
> threading scheme that you were laying on top of the system threads in a User 
> Space process, you can use the Operating System plugin to show you the 
> cooperative threads.  This is not an abstract example...  I think it should 
> stay with just that job.
> 
> The place where lldb holds this sort of knowledge is supposed to be in the 
> Platform class.  So for instance, to comprehend mutexes you really just need 
> a data formatter.  The trick is that it has to be a data formatter that is 
> provided by the platform.  Similarly you want to have frame recognizers for 
> interesting lower-level calls in the system.  The machinery will shortly be 
> there to do that, but loading the particular recognizers will either need to 
> be done by hand, or coordinated by the Platform.  In general, I think most of 
> the kinds of re-presentation you need to do to make OS objects and processes 
> more comprehensible can be built as general mechanisms like the above.  Then 
> the Platform can coordinate providing the set of more general transformations 
> that are appropriate to the Platform you are targeting.
> 
> Jim
> 
> 
> 
>> On Oct 31, 2018, at 3:44 PM, Alexander Polyakov  
>> wrote:
>> 
>> So, if I understand you write, I can look at OS plugin and add a support of 
>> mutexes or memory pages for example?
>> 
>> чт, 1 нояб. 2018 г. в 1:05, Jim Ingham :
>> lldb has one feature - the "Operating System Plugin" that is specifically 
>> designed for debugging threads in kernel contexts.  The OS plugin allows a 
>> kernel to present it's notion of threads into lldb_private::Threads.  The 
>> xnu kernel macros have an implementation of this, as do some other embedded 
>> OS'es.  lldb actually gets used pretty extensively for debugging xnu - the 
>> Darwin kernel.
>> 
>> Kuba is adding the notion of "Frame recognizers" which can give significance 
>> to particular functions when they appear on the stack (for instance 
>> displaying the first argument of read, etc. as a file handle even if you 
>> have no debug information for it.)  That's another way that you could 
>> express your understanding of the OS you are running on for debugger users.  
>> Greg wrote a data formatter for the Darwin implementation of pthread_mutex 
>> that shows the thread that has the lock and some other information like 
>> that.  So data formatters are also a way lldb can express knowledge of the 
>> host OS.
>> 
>> Jim
>> 
>>> On Oct 31, 2018, at 12:52 PM, Leonard Mosescu via lldb-dev 
>>>  wrote:
>>> 
>>> Conceptually it's different levels of abstraction: a user-mode debugger 
>>> handles processes, threads as first class concepts. In kernel-mode (or 
>>> kernel land), these are just data structures that the code (the kernel) is 
>>> managing. From a more pragmatic perspective, the difference is in where the 
>>> debugging hooks are implemented and what interfaces are exposed (for 
>>> example a kernel mode debugger can normally "poke" around any piece of 
>>> memory and it has to be aware of things like VA mappings, while a user-mode 
>>> debugger is only allowed to control a limited slice of the system - ex. 
>>> control a sub-process through something like ptrace)
>>> 
>>> Unless you're specifically looking at kernel debugging I'd stay away from 
>>> that. For one thing, LLDB is mostly used as an user-mode debugger so the 
>>> impact of any improvements would be bigger.
>>> 
>>> Regarding the value of OS-awareness for user-mode debugging, I agree with 
>>> Zach - for example windbg provides both kernel mode and user mode !locks 
>>> commands. The only suggestion I'd add is to consider an expanded view of 
>>> the "OS" to include runtime c

Re: [lldb-dev] [RFC] OS Awareness in LLDB

2018-10-31 Thread Jim Ingham via lldb-dev
Right now, the OS plugin only supports the job of adding threads.  And that 
makes more sense as a plugin, because for instance if you had a cooperative 
threading scheme that you were laying on top of the system threads in a User 
Space process, you can use the Operating System plugin to show you the 
cooperative threads.  This is not an abstract example...  I think it should 
stay with just that job.

The place where lldb holds this sort of knowledge is supposed to be in the 
Platform class.  So for instance, to comprehend mutexes you really just need a 
data formatter.  The trick is that it has to be a data formatter that is 
provided by the platform.  Similarly you want to have frame recognizers for 
interesting lower-level calls in the system.  The machinery will shortly be 
there to do that, but loading the particular recognizers will either need to be 
done by hand, or coordinated by the Platform.  In general, I think most of the 
kinds of re-presentation you need to do to make OS objects and processes more 
comprehensible can be built as general mechanisms like the above.  Then the 
Platform can coordinate providing the set of more general transformations that 
are appropriate to the Platform you are targeting.

Jim



> On Oct 31, 2018, at 3:44 PM, Alexander Polyakov  
> wrote:
> 
> So, if I understand you write, I can look at OS plugin and add a support of 
> mutexes or memory pages for example?
> 
> чт, 1 нояб. 2018 г. в 1:05, Jim Ingham :
> lldb has one feature - the "Operating System Plugin" that is specifically 
> designed for debugging threads in kernel contexts.  The OS plugin allows a 
> kernel to present it's notion of threads into lldb_private::Threads.  The xnu 
> kernel macros have an implementation of this, as do some other embedded 
> OS'es.  lldb actually gets used pretty extensively for debugging xnu - the 
> Darwin kernel.
> 
> Kuba is adding the notion of "Frame recognizers" which can give significance 
> to particular functions when they appear on the stack (for instance 
> displaying the first argument of read, etc. as a file handle even if you have 
> no debug information for it.)  That's another way that you could express your 
> understanding of the OS you are running on for debugger users.  Greg wrote a 
> data formatter for the Darwin implementation of pthread_mutex that shows the 
> thread that has the lock and some other information like that.  So data 
> formatters are also a way lldb can express knowledge of the host OS.
> 
> Jim
> 
> > On Oct 31, 2018, at 12:52 PM, Leonard Mosescu via lldb-dev 
> >  wrote:
> > 
> > Conceptually it's different levels of abstraction: a user-mode debugger 
> > handles processes, threads as first class concepts. In kernel-mode (or 
> > kernel land), these are just data structures that the code (the kernel) is 
> > managing. From a more pragmatic perspective, the difference is in where the 
> > debugging hooks are implemented and what interfaces are exposed (for 
> > example a kernel mode debugger can normally "poke" around any piece of 
> > memory and it has to be aware of things like VA mappings, while a user-mode 
> > debugger is only allowed to control a limited slice of the system - ex. 
> > control a sub-process through something like ptrace)
> > 
> > Unless you're specifically looking at kernel debugging I'd stay away from 
> > that. For one thing, LLDB is mostly used as an user-mode debugger so the 
> > impact of any improvements would be bigger.
> > 
> > Regarding the value of OS-awareness for user-mode debugging, I agree with 
> > Zach - for example windbg provides both kernel mode and user mode !locks 
> > commands. The only suggestion I'd add is to consider an expanded view of 
> > the "OS" to include runtime components which may not be technically part of 
> > what most people think of as the "OS": user-mode loaders and high level 
> > things like std::mutex, etc.
> > 
> > On Wed, Oct 31, 2018 at 12:29 PM, Alexander Polyakov 
> >  wrote:
> > Looks like I don't completely understand what is the difference between 
> > user-mode and kernel-mode from the debugger's point of view. Could you 
> > please explain me this?
> > 
> > On Wed, Oct 31, 2018 at 10:22 PM Zachary Turner  wrote:
> > I don’t totally agree with this. I think there are a lot of useful os 
> > awareness tasks in user mode. For example, you’re debugging a deadlock and 
> > want to understand the state of other mutexes, who owns them, etc. or you 
> > want to examine open file descriptors. In the case of a heap corruption you 
> > may wish to study the internal structures of your process’s heap, or even 
> > lower level, the os virtual memory page table structures.
> > 
> > There’s quite a lot you can still do in user mode, but definitely there is 
> > more in kernel mode. As Leonard said, try put WinDbg as a lot of this stuff 
> > already exists so it’s a good reference 
> > On Wed, Oct 31, 2018 at 12:08 PM Alexander Polyakov via lldb-dev 
> >  wrote:
> > Hi 

Re: [lldb-dev] `ClangASTSource::IgnoreName` C++ false positives

2018-10-31 Thread Jim Ingham via lldb-dev
We need to do more work to make sure the "language" setting for expressions 
gets propagated everywhere and obeyed.  We could also be smarter about setting 
the default.

It seems fine to me for lldb to assume - if we know nothing different - that we 
should use ObjC++ as the language to compile expressions.  That makes sense for 
most developers, since it only has a few collisions (Class and id) that seldom 
cause problems IRL and gives the widest coverage for of expressions.  BUT... 
the behavior when compiling an expression should be determined by the 
expression's language  rather than how we set up the target ScratchASTContext.

So for instance, if you do:

(lldb) expression --language c++ -- id

that should work, though without the --language you would get an error.  That 
doesn't currently work, so apparently we aren't always checking the language of 
the expression.

The situation is a little more complicated than that because sometimes we check 
the frame's language (e.g. we do that in AddLocalVariableDecls, w/o checking 
whether the expression has a specific language.  So this definitely needs 
cleaning up.

Then we should also let the Language Runtime's adjust the default language, so 
if you have a program that doesn't load the ObjC runtime, we set the default 
language to C++, not ObjC++.

But the first stage is to make sure we are paying attention to the explicit 
language of the expression throughout the expression parser code.

Jim


> On Oct 31, 2018, at 7:25 AM, Zachary Turner  wrote:
> 
> The first thing I would try is to see where the language is getting set to 
> objective c and force it c++. Just as an experiment. Depending where it 
> happens, it may be possible to initialize it from the debug info (or hardcode 
> it).
> 
> But since ObjC assumptions are baked into several places, this has potential 
> to break some things 
> On Wed, Oct 31, 2018 at 6:54 AM Aleksandr Urakov 
>  wrote:
> Sorry, I have somehow missed the discussion continuation there. Yes, it's a 
> very similar problem, thanks. But unfortunately no one of the workarounds 
> mentioned there doesn't work in this situation...
> 
> On Wed, Oct 31, 2018 at 4:32 PM Zachary Turner  wrote:
> It seems like we hit this issue in different contexts almost at the same time 
> (see my thread several days ago about “problem formatting value objects”). 
> That might at least give you some context about why things
> 
> I wish ObjC assumptions weren’t so deeply embedded, but alas it is the case.
> 
> Hopefully Jim or someone has ideas on how to fix this properly. 
> On Wed, Oct 31, 2018 at 5:08 AM Aleksandr Urakov 
>  wrote:
> Hello,
> 
> I've tried to use a check like `if (m_ast_context->getLangOpts().ObjC) ...`, 
> but it seems that it's always true. How can we else determine here if the 
> Objective-C case is used? Or if we can't, where can we move `if (name == 
> id_name || name == Class_name)` to make it Objective-C only? What regressions 
> Objective-C users would have if we would remove this check from here?
> 
> Regards,
> Alex
> 
> On Wed, Oct 24, 2018 at 7:14 PM Aleksandr Urakov 
>  wrote:
> Hi all!
> 
> There are two hardcoded names to ignore in the `ClangASTSource::IgnoreName` 
> function, "Class" and "id", they are valid names for C++. It seems that they 
> were added for the Objective-C case. But the problem is that when they are in 
> locals they are blocking expressions evaluation.
> 
> For example for the next code:
> int main() {
>   int x = 5;
>   int id = 7;
>   int y = 8;
>   return 0;
> }
> if you'll break on `return 0` and will try to `print x`, then you'll get a 
> error like `no member named 'id' in namespace '$__lldb_local_vars'`.
> 
> Do you have any ideas, how can we fix it?
> 
> Regards,
> Alex
> 
> 
> -- 
> Aleksandr Urakov
> Software Developer
> JetBrains
> http://www.jetbrains.com
> The Drive to Develop
> 
> 
> -- 
> Aleksandr Urakov
> Software Developer
> JetBrains
> http://www.jetbrains.com
> The Drive to Develop

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Problem formatting class types

2018-10-26 Thread Jim Ingham via lldb-dev
Most C++ classes and C structs don't have data formatters, particularly not 
classes that you write yourself.  

The way value printing works in lldb is that we start by making the ValueObject 
for the value from its Type, so at that stage it is just a direct view of the 
members of the object.  That is done without help of the data formatters, 
reading instead directly from the object's type.  Then we consult our type 
match -> summary/synthetic children registries and we construct a summary or a 
set of "synthetic children" (or both) for the object if we find any matches 
there.  Then the ValueObjectPrinter prints the object using the Type based 
ValueObject, the Summary and the Synthetic Children, and there's a print 
options object that says whether to use the raw view, the summary and/or the 
synthetic children.

But for a type lldb knows nothing about, there won't be any entries in the 
formatter maps, so you should just see the direct Type based children in that 
case.

--raw sets the right options in the print option object to get the printer to 
just use the strict Type based view of the object, with no formatters applied.

In your case, you used "Class" as your type name and  Class is a special name 
in ObjC and there happens to be a formatter for that.  You can always figure 
out what formatters apply to the result of an expression with the "type 
{summary/synthetic} info" command.  For your example, I see (my variable of 
type Class was called myClass):

(lldb) type summary info myClass
summary applied to (Class) myClass is:  (not cascading) (hide value) (skip 
pointers) (skip references) Class summary provider
(lldb) type synthetic info myClass
synthetic applied to (Class) myClass is:  Class synthetic children

On macOS those summary/synthetic child providers are in the objc category.  The 
info output should really print the category as well, that would be helpful.  
But you can do "type summary list" and then find the summary in that list and 
go from there to the category.  Ditto for "type synthetic".

What do you get from that?

Jim

> On Oct 26, 2018, at 3:34 PM, Zachary Turner  wrote:
> 
> So, the second command works, but the first one doesn't.  It doesn't give any 
> error, but on the other hand, it doesn't change the results of printing the 
> variable.  When I run type category list though, I get this:
> 
> (lldb) type category list
> Category: default (enabled)
> Category: VectorTypes (enabled, applicable for language(s): objective-c++)
> Category: system (enabled, applicable for language(s): objective-c++)
> 
> So it looks like the behavior I'm seeing is already with the default 
> category.  Does this sound right?  Which code path is supposed to get 
> executed to format it as a C++ class?
> 
> On Fri, Oct 26, 2018 at 10:25 AM Jim Ingham  wrote:
> Remove the "not"...
> 
> Jim
> 
> > On Oct 26, 2018, at 10:24 AM, Jim Ingham  wrote:
> > 
> > But at the minimum, not loading formatters for a language that we can 
> > determine isn't used in this program seems like something we should try to 
> > avoid.
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Problem formatting class types

2018-10-26 Thread Jim Ingham via lldb-dev
The data formatters don't currently have a  specified language.  We should 
maybe add that and then we could only load formatters for a language when its 
runtime gets loaded.  Then they could also know to turn themselves on and off 
based on the language of the current frame, or of the current expression.  We 
have to be careful, because UIKit and AppKit have a bunch of interesting 
globals that show the state of the UI, and it very common for folks to just 
interrupt the debugger anywhere and run ObjC expressions (even though the frame 
language is not ObjC) and expect them and the associated formatters to work.  
Breaking that expectation would be pretty disruptive.

But at the minimum, not loading formatters for a language that we can determine 
isn't used in this program seems like something we should try to avoid.

You can work around this in the short term by doing either:

(lldb) type category disable objc
(lldb) expr myClass
(Class) $5 = {
  x = 0
  y = 1
  z = 'z'
}

Or if you don't want to change global settings, use:

(lldb) expr --raw -- myClass
(Class) $3 = {
  x = 0
  y = 1
  z = 'z'
}

Jim


> On Oct 26, 2018, at 3:29 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> Note that I also tried this with a a linux / DWARF executable and had the 
> same result.
> 
> On Fri, Oct 26, 2018 at 3:21 AM Zachary Turner  wrote:
> Hello,
> 
> I've got this code:
> 
> class Class {
>   int x = 0;
>   short y = 1;
>   char z = 'z';
> } C;
> 
> int main(int argc, char **argv) {
>   __debugbreak();
>   return 0;
> }
> 
> and I run the following LLDB session:
> 
> lldb.exe -f foo.exe
> (lldb) target create "foo.exe"
> Current executable set to 'foo.exe' (x86_64).
> (lldb) run
> Process 24604 launched: 'foo.exe' (x86_64)
> Process 24604 stopped
> * thread #1, stop reason = Exception 0x8003 encountered at address 
> 0x7ff70a0b1017
> frame #0: 0x7ff70a0b1018 foo.exe`main(argc=-1123614720, 
> argv=0x7ff70a0b1000) at foo.cpp:19
>16
>17   int main(int argc, char **argv) {
>18 __debugbreak();
> -> 19 return 0;
>20   }
> (lldb) p C
> (Class) $0 =
> (lldb)
> 
> The issue is, of course, that it doesn't display the members of the class C.  
> The type support in PDB is fine, so it's not that.  For example:
> 
> (lldb) type lookup Class
> class Class {
> int x;
> short y;
> char z;
> }
> 
> And it can definitely find C in memory:
> 
> (lldb) p 
> (Class *) $1 = 0x7ff70a0b3000
> 
> Instead, the issue seems to be related to the value object formatter.  I 
> tried to track this down but this code is pretty complicated.  However, there 
> are two issues that I was able to discover:
> 
> 1) It's using the objective C class formatter.  Obviously I'm not using 
> objective C, so that seems wrong right off the bat.  Specifically, the 
> "Synthetic children front end" is the ObjCClassSyntheticChildrenFrontEnd.  
> 
> 2) Because of #1, when it calls CalculateNumChildren() in Cocoa.cpp, it 
> returns 0.  I would expect it to be calling some function somewhere that 
> returns 3, because there are 3 members of the class.
> 
> What's strange is that I don't see anything in the CPlusPlusLanguage plugin 
> that provides a SyntheticChildrenFrontEnd that examines the CxxRecordDecl and 
> looks for children, so I don't know how this is supposed to work anywhere.  
> But I know it must work somewhere, so I assume I'm just missing something and 
> I need to find out the right place to hook A up to B and things will just 
> work.
> 
> Any pointers on what the expected code path that this should be taking is, so 
> I can try to figure out where I might be going off path?
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in case of missing location at block begin

2018-10-10 Thread Jim Ingham via lldb-dev
Thanks for looking into this!  

When I was first working on inlined stepping, I found a bunch of cases where 
the line table info and the ranges for the inlined subroutines disagreed.  I 
remember seeing cases, for instance, where the address given for foo.h:xxx in 
the line table was contained in one of the debug_info's inlined subroutine 
blocks from a different file.  I tried for a little while to put in heuristics 
to try to work past the disagreement.  But that was too easy to get wrong, and 
when I got that wrong it often had the result of turning a step into a 
continue.  It is annoying to stop too early, but it is much worse to stop too 
late (or never).  So I ended up bagging my attempts at heroics and whenever I 
get to a place where the line table and the inlined_subroutine bits of info are 
out of sync, I just stop.

Jim
 

> On Oct 10, 2018, at 4:34 PM, Matthias Braun  wrote:
> 
> 1) So I went and figured out why the lldb testcase at hand fails.
> 
> - It seems the debugger stepping logic will follow the program along until it 
> finds a new source location. However if that new source location is part of a 
> new DW_AT_abstract_location it is ignored in the step over mode.
> - In the testcase at hand the .loc location of an inlined function was moved 
> to an earlier place without the DW_AT_abstract_location entry getting 
> extended. So the debugger mistook the inlined function as the same scope and 
> stepping incorrectly halted there.
> 
> On the LLVM side DW_AT_abstract_location is generated by LexicalScopes which 
> is created by lib/CodeGen/LexicalScopes.cpp / extractLexicalScopes() with 
> completely separate code from the line table generation code in 
> lib/CodeGen/AsmPrinter/DwarfDebug.cpp you have to be aware of that and keep 
> the two algorithms in sync :-/
> 
> I fixed LexicalScopes.cpp to be in sync and the lldb test works fine again 
> for me.
> 
> 2) However talking to Adrian earlier he also expressed having a bad feeling 
> about moving debug location upwards instead of emitting line-0 entries. So I 
> think consensus here is to rather live with some line table bloat instead.
> 
> - Unless there are objections I will not go with option 1) but instead revert 
> this commit tomorrow. Note that I will only revert r343874 (i.e. the behavior 
> change for what to emit when the first instructions of a basic block have no 
> location attached), but not r343895 (removing debuglocs from spill/reload 
> instructions) as I still consider this a sensible commit. So in the end we 
> may have bigger line tables than before my changes, but simpler code in the 
> spill/reload generation and occasionally can avoid random source location 
> when spill/reloads get rescheduled.
> 
> - Matthias
> 
> 
>> On Oct 10, 2018, at 1:17 PM, via llvm-commits  
>> wrote:
>> 
>> 
>> 
>>> -Original Message-
>>> From: Matthias Braun [mailto:ma...@braunis.de]
>>> Sent: Wednesday, October 10, 2018 3:50 PM
>>> To: Robinson, Paul
>>> Cc: jing...@apple.com; v...@apple.com; llvm-comm...@lists.llvm.org; lldb-
>>> d...@lists.llvm.org
>>> Subject: Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in
>>> case of missing location at block begin
>>> 
>>> 
>>> 
 On Oct 10, 2018, at 12:18 PM, via llvm-commits >> comm...@lists.llvm.org> wrote:
 
 
 
> -Original Message-
> From: jing...@apple.com [mailto:jing...@apple.com]
> Sent: Wednesday, October 10, 2018 2:20 PM
> To: Vedant Kumar
> Cc: Robinson, Paul; Vedant Kumar via llvm-commits; LLDB; Matthias Braun
> Subject: Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location
>>> in
> case of missing location at block begin
> 
> 
> 
>> On Oct 10, 2018, at 9:54 AM, Vedant Kumar  wrote:
>> 
>>> On Oct 10, 2018, at 9:16 AM, Matthias Braun  wrote:
>>> 
>>> So I haven't worked much on debug info, but here's the explanation
>>> for
> my patches:
>>> My original motivation was getting rid of code some code in the llvm
> codegen that for spills and reloads just picked the next debug location
> around. That just seemed wrong to me, as spills and reloads really are
> bookkeeping, we just move values around between registers and memory
>>> and
> none of it is related to anything the user wrote in the source program.
>>> So
> not assigning any debug information felt right for these instructions.
> Then people noticed line table bloat because of this and I guess we
> assumed that having exact line-0 annotations isn't that useful that it
> warrants bloating the debug information...
>> 
>> Right. This doesn't seem any more arbitrary than reusing the previous
> instruction location, which we do all the time. I think it's a
>>> reasonable
> tradeoff.
 
 For spills and reloads, the next valid source location might be
>>> reasonable.
 For top-of-block instructions, I really don't think so; we had this
>>> issue
 in 

Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in case of missing location at block begin

2018-10-10 Thread Jim Ingham via lldb-dev


> On Oct 10, 2018, at 9:54 AM, Vedant Kumar  wrote:
> 
>> On Oct 10, 2018, at 9:16 AM, Matthias Braun  wrote:
>> 
>> So I haven't worked much on debug info, but here's the explanation for my 
>> patches:
>> My original motivation was getting rid of code some code in the llvm codegen 
>> that for spills and reloads just picked the next debug location around. That 
>> just seemed wrong to me, as spills and reloads really are bookkeeping, we 
>> just move values around between registers and memory and none of it is 
>> related to anything the user wrote in the source program. So not assigning 
>> any debug information felt right for these instructions. Then people noticed 
>> line table bloat because of this and I guess we assumed that having exact 
>> line-0 annotations isn't that useful that it warrants bloating the debug 
>> information...
> 
> Right. This doesn't seem any more arbitrary than reusing the previous 
> instruction location, which we do all the time. I think it's a reasonable 
> tradeoff.
> 
> 
>> 
>> - Matthias
>> 
>>> On Oct 10, 2018, at 8:45 AM, via llvm-commits  
>>> wrote:
>>> 
>>> Sorry for being late to the party, I have been away.
>>> 
>>> I am not persuaded that this patch is functionally correct.
>>> It's lovely to reduce size, but not at the cost of telling
>>> un-truths about source locations of instructions.  If the
>>> instructions at the top of the block have no source location,
>>> and you don't like using line 0, the correct path forward is to
>>> understand why those instructions do not have a source location,
>>> not to just pick one from a nearby instruction.
> 
> I think we should do that regardless, but it's a much bigger project which 
> won't fix the bloat issues the hwasan folks ran into.
> 
> 
>>> My choice would be to revert the LLVM patch rather than make
>>> the LLDB test accommodate the un-truth.
> 
> As Jim pointed out, the LLDB test is doing a whole lot more than it needs to. 
> It should be rewritten to focus on data formatting, instead of stepping.
> 

I didn't quite say that.  I said this test inadvertently showed itself a good 
test for stepping, as well as being a test for data formatting.  So we should 
extract the part that proved its worth in catching stepping bugs into a 
stepping-only test, and then rewrite the data formatter test so it wasn't 
reliant on stepping behavior.

Jim


> vedant
> 
>>> --paulr
>>> 
>>>> -Original Message-
>>>> From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Jim
>>>> Ingham via lldb-dev
>>>> Sent: Friday, October 05, 2018 9:30 PM
>>>> To: Vedant Kumar
>>>> Cc: Matthias Braun; Vedant Kumar via llvm-commits; LLDB
>>>> Subject: Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in
>>>> case of missing location at block begin
>>>> 
>>>> So in the test scenario, we have this code:
>>>> 
>>>>printf("// Set second break point at this line.");
>>>>(text_list.push_back(std::string("!!!")));
>>>> 
>>>> and we have a breakpoint on the printf line.  We've just continued to hit
>>>> the breakpoint at printf.  Then we do next twice.  That should certainly
>>>> get us past the push_back.  If it does not that's either a bug in the line
>>>> tables or in lldb's handling of them.  You would not expect to have to
>>>> next three times to go from the start of the printf line to past the
>>>> push_back of !!!.
>>>> 
>>>> Considered as a test about stepping, we should applaud the test for having
>>>> caught a bug, and go figure out whether the line tables are bogus in this
>>>> instance or just innovative and lldb will have to cope...
>>>> 
>>>> Considered as a test about data formatters, it is a little silly to try to
>>>> drive it using next's since the push_backs are going to introduce a bunch
>>>> of inlining and the debug information for inlining is often a bit wonky...
>>>> 
>>>> Best course is to use the breakpoints in this test to drive from point to
>>>> point, and make a reduced stepping test case that just shows this bad
>>>> stepping behavior.  Then we can fix the latter test failure either in
>>>> clang or lldb as is appropriate.
>>>> 
>>>> Jim
>>>> 
>>>> 
>>>>> On Oct 5, 2018, at 4:18 PM, Vedant Kumar via lldb-dev >&

[lldb-dev] SIGPIPE handling

2018-10-08 Thread Jim Ingham via lldb-dev
I've had requests from several users now to change the default SIGPIPE handling 
to "stop = false, notify = false".  The argument is that most of the time these 
signals are generally ignored and folks directly handle the read and write 
errors.  So having lldb stop by default for SIGPIPE is just annoying.  And if 
SIGPIPE debugging is interesting for some project, it is easy to turn this back 
on.   I just copied the gdb defaults as they were 10 years or so ago, there 
wasn't a better motivation for the way it is in lldb now.

The change is easy, but I wanted to make sure nobody has a counter-argument for 
keeping the current default - which is stop = yes, notify = yes, suppress = no. 
I'm proposing to change that to stop = no, notify = no, suppress = no.

Jim



___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Parsing Line Table to determine function prologue?

2018-10-08 Thread Jim Ingham via lldb-dev
A single sequence in the line table needs to be run from beginning to end to 
make sense of it.  It doesn't really have addresses, it generally has a start 
address, then a sequence of "increment line, increment address" instructions.  
So you have to run the state machine to figure out what the addresses are.

However, the line table does not have to be one continuous sequence.  The DWARF 
docs state this explicitly, and there is an "end_sequence" instruction to 
implement this.  I can't see any reason why you couldn't get the compiler to 
emit line tables in per-function sequences, and have the debugger optimize 
reading the line table by first scanning for sequence ends to get the map of 
chunks -> addresses, and then reading the line table in those chunks.  I don't 
think anybody does this, however.  clang emitted the whole CU as one sequence 
in the few examples I had sitting around.

Jim


> On Oct 8, 2018, at 12:28 PM, Leonard Mosescu via lldb-dev 
>  wrote:
> 
> Even if we do need to parse the line table, could it be done just for the 
> function in question?  The debug info tells us the function's address range, 
> so is there some technical reason why it couldn't parse the line table only 
> for the given address range?
> 
> My understanding is that there's one DWARF .debug_line "program" per CU, and 
> normally you'd need to "execute" the whole line number program.
> 
> On Sat, Oct 6, 2018 at 8:05 PM, Zachary Turner via lldb-dev 
>  wrote:
> While implementing native PDB support I noticed that LLDB is asking to parse 
> an entire compile unit's line table in order to determine if 1 address is a 
> function prologue or epilogue.
> 
> Is this necessary in DWARF-land?  It would be nice if I could just pass the 
> prologue and epilogue byte size directly to the constructor of the 
> lldb_private::Function object when I construct it.
> 
> It seems unnecessary to parse the entire line table just to set a breakpoint 
> by function name, but this is what ends up happening.
> 
> Even if we do need to parse the line table, could it be done just for the 
> function in question?  The debug info tells us the function's address range, 
> so is there some technical reason why it couldn't parse the line table only 
> for the given address range?
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] Should we stop supporting building with Visual Studio?

2018-10-08 Thread Jim Ingham via lldb-dev


> On Oct 8, 2018, at 8:09 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> I haven't heard of anyone using the Xcode generated project.  In fact, LLDB 
> maintains its own hand-created Xcode project precisely because the CMake one 
> is considered "unusable".  That said, I don't personally use Xcode or a Mac, 
> so I can't speak for if anyone else might be using the Xcode generator.
> 

All of Xcode's smart code comprehension relies on the clang index built from 
the project.  Xcode uses the project's build rules to construct the index.  
Apparently the cmake generated Xcode project doesn't communicate the info on 
how to build the project and thus the index to Xcode in a way it can use.  I 
haven't tried it recently for lldb (I don't think anybody on the team here 
has), but people here were hacking on the generator to use it for developing 
clang & swift not too long ago. The last report I heard was that it still needs 
more work to be usable, and that that work was not entirely trivial.  

Without the index, Xcode is a pretty weak IDE, which is why we maintain a 
hand-built Xcode project.

Jim
 
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in case of missing location at block begin

2018-10-05 Thread Jim Ingham via lldb-dev
So in the test scenario, we have this code:

printf("// Set second break point at this line.");
(text_list.push_back(std::string("!!!")));

and we have a breakpoint on the printf line.  We've just continued to hit the 
breakpoint at printf.  Then we do next twice.  That should certainly get us 
past the push_back.  If it does not that's either a bug in the line tables or 
in lldb's handling of them.  You would not expect to have to next three times 
to go from the start of the printf line to past the push_back of !!!.

Considered as a test about stepping, we should applaud the test for having 
caught a bug, and go figure out whether the line tables are bogus in this 
instance or just innovative and lldb will have to cope...

Considered as a test about data formatters, it is a little silly to try to 
drive it using next's since the push_backs are going to introduce a bunch of 
inlining and the debug information for inlining is often a bit wonky...

Best course is to use the breakpoints in this test to drive from point to 
point, and make a reduced stepping test case that just shows this bad stepping 
behavior.  Then we can fix the latter test failure either in clang or lldb as 
is appropriate.

Jim


> On Oct 5, 2018, at 4:18 PM, Vedant Kumar via lldb-dev 
>  wrote:
> 
> No worries, I’ve relaxed the test for now in r343899 to get the bots going 
> again. Essentially, the test will just step one more time to make sure the 
> full effect of the push_back() is visible.
> 
> If folks on lldb-dev (cc’d) have a better idea we can try it out.
> 
> vedant
> 
>> On Oct 5, 2018, at 4:15 PM, Matthias Braun  wrote:
>> 
>> So what should we do? Revert the llvm commit, fix the LLDB test, xfail on 
>> lldb? I'd be fine with any but don't want to learn how lldb tests work at 
>> this moment...
>> 
>>> On Oct 5, 2018, at 4:11 PM, Vedant Kumar via llvm-commits 
>>>  wrote:
>>> 
>>> Sadly, after this commit TestDataFormatterLibcxxList.py started failing 
>>> with:
>>> 
>>> ```
>>> output: Process 67333 stopped
>>> * thread #1, queue = 'com.apple.main-thread', stop reason = step over
>>> frame #0: 0x00010eb0 a.out`main at main.cpp:33:16
>>>30   (text_list.push_back(std::string("smart")));
>>>31   
>>>32   printf("// Set second break point at this line.");
>>> -> 33   (text_list.push_back(std::string("!!!"))); 
>>>^
>>>34   
>>>35   std::list countingList = {3141, 3142, 3142,3142,3142, 
>>> 3142, 3142, 3141};
>>>36   countingList.sort();
>>> 
>>> 
>>> runCmd: frame variable text_list[0]
>>> output: (std::__1::basic_string, 
>>> std::__1::allocator >) [0] = "goofy"
>>> 
>>> Expecting sub string: goofy
>>> Matched
>>> 
>>> runCmd: frame variable text_list[3]
>>> output: None
>>> 
>>> Expecting sub string: !!!
>>> ```
>>> 
>>> URL: http://lab.llvm.org:8080/green/view/LLDB/job/lldb-cmake/10854
>>> 
>>> I confirmed that reverting this fixes the issue.
>>> 
>>> I think the problem is that we’re breaking on an instruction that looks 
>>> like it’s on line 33, but it’s actually “before” the full effect of the 
>>> push_back() is visible.
>>> 
>>> In which case the test is wrong, because it’s setting the breakpoint too 
>>> early…
>>> 
>>> vedant
>>> 
 On Oct 5, 2018, at 11:29 AM, Matthias Braun via llvm-commits 
  wrote:
 
 Author: matze
 Date: Fri Oct  5 11:29:24 2018
 New Revision: 343874
 
 URL: http://llvm.org/viewvc/llvm-project?rev=343874=rev
 Log:
 DwarfDebug: Pick next location in case of missing location at block begin
 
 Context: Compiler generated instructions do not have a debug location
 assigned to them. However emitting 0-line records for all of them bloats
 the line tables for very little benefit so we usually avoid doing that.
 
 Not emitting anything will lead to the previous debug location getting
 applied to the locationless instructions. This is not desirable for
 block begin and after labels. Previously we would emit simply emit
 line-0 records in this case, this patch changes the behavior to do a
 forward search for a debug location in these cases before emitting a
 line-0 record to further reduce line table bloat.
 
 Inspired by the discussion in https://reviews.llvm.org/D52862
 
 Added:
llvm/trunk/test/DebugInfo/X86/dwarf-no-source-loc.mir
 Modified:
llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.h
llvm/trunk/test/DebugInfo/AArch64/line-header.ll
llvm/trunk/test/DebugInfo/ARM/single-constant-use-preserves-dbgloc.ll
llvm/trunk/test/DebugInfo/Mips/delay-slot.ll
llvm/trunk/test/DebugInfo/NVPTX/debug-info.ll
llvm/trunk/test/DebugInfo/X86/dwarf-no-source-loc.ll
 
 Modified: llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
 URL: 
 

Re: [lldb-dev] How to use --tty= with lldb-mi

2018-10-02 Thread Jim Ingham via lldb-dev
The way to do this in the MI interface is with the command: inferior-tty-set.  
Unfortunately, that command isn't implemented in the lldb-mi:

bool CMICmdCmdInferiorTtySet::Execute() {
  // Do nothing

  return MIstatus::success;
}

This should set the target.{input/output/error}-path to the tty it was passed.

Jim


> On Oct 2, 2018, at 4:33 PM, Pierson Lee (PIE) via lldb-dev 
>  wrote:
> 
> Hi,
>  
> I'm using lldb-mi to debug on OS X and I'm trying to redirect the debuggee 
> output to a tty, but it seems like it wants to use the lldb-launcher 
> (Darwin-debug) add on and open a terminal instead.
>  
> Is there a way to have the debuggee stdin/stdout redirection be sent to a tty 
> of my specification?
>  
> Thanks
> Pierson
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC] LLDB Reproducers

2018-09-20 Thread Jim Ingham via lldb-dev


> On Sep 20, 2018, at 4:16 AM, Zachary Turner  wrote:
> 
> For the first, I think 99% of the time the bug is not caused by the sequence 
> of gdb remote packets.  The sequence of gdb remote packets just happens to be 
> the means by which the debugger was put into the state in which it failed.  
> If there is another, stable way of getting the debugger into the same state 
> this part is solvable.

Yes, I don't actually think we are in disagreement.  

When you are trying to provide a way to actually gather the information to 
reproduce a bug that is happening in the wild, you want to be able to ensure 
that you gather whatever you need to do so.  You are trying to avoid a bunch of 
round-trips with the reporter.  How would you know that the problem is all 
above the basic Process model, in which case gathering just higher level 
behaviors will be enough to reproduce the bug?  That seems like a hard problem 
to me.  So instead, you try to be conservative and gather the information at 
the level that you know drives the system, in this case for process level bugs 
this is the gdb-packet traffic.  And if you are gathering at that level, the 
trace you've gathered is somewhat fragile.

So if you wanted to generate tests, you would do a post-processing step that 
takes the states generated by the lower level events from the reproducer and 
converts that into an object model level replayer.  That should be doable if 
you had the ability to mock behavior at the level of lldb_private::Process, 
etc.  I'm only saying I think the needs of the Reproducer are such that a 
direct use of its provider's data is unlikely to be a good way to implement 
tests.

Making a converter from "reproducer trace to "test" is a separate piece of 
work, and not directly related to reproducers.  After all, if done right this 
would be able to observe a user-driven debug session and make a test out of it 
as well.  But I don't think this level or recording is right for the Reproducer.

Another problem with using the reproducer traces to directly produce tests is 
that for good tests you generally want to reduce them to the simplest set of 
steps possible to show just this bug.  What you are going to get from an 
in-the-wild reproducer is unlikely to be that.  That's actually part of the 
point of the reproducer, to relieve bug reporters of the necessity of reducing 
their problem before submitting.  So you'd have to have some way of figuring 
out when the world was still correct, just before going bad, and then only 
include the set of steps leading from good to bad (there may be lots of 
extraneous actions in the trace.)

> 
> The second issue you raised does seem like something that would require human 
> intervention to specify the expected state though as part of a test

I also think some automated way to gather "what did you expect" would be 
helpful.  I have very often had to go many rounds with reporters to figure out 
what they are actually reporting as wrong.  But that isn't part of Jonas' 
proposal, and maybe is more work than is justified by saving the few of us who 
work on lldb from the "many rounds" described above.

Jim


> 
> On Wed, Sep 19, 2018 at 11:17 AM Jim Ingham  wrote:
> There are a couple of problems with using these reproducers in the testsuite.
> 
> The first is that we make no commitments that the a future lldb will 
> implement the "same" session with the same sequence of gdb-remote packet 
> requests.  We often monkey around with lldb's sequences of requests to make 
> things go faster.  So some future lldb will end up making a request that 
> wasn't in the data from the reproducer, and at that point we won't really 
> know what to do.  The Provider for gdb-remote packets should record the 
> packets it receives - not just the answers it gives - so it can detect this 
> error and not go off the rails.  But I'm pretty sure it isn't worth the 
> effort to try to get lldb to maintain all the old sequences it used in the 
> past in order to support keeping the reproducers alive.  But this does mean 
> that this is an unreliable way to write tests.
> 
> The second is that the reproducers as described have no notion of "expected 
> state".  They are meant to go along with a bug report where the "x was wrong" 
> part is not contained in the reproducer.  That would be an interesting thing 
> to think about adding, but I think the problem space here is complicated 
> enough already...  You can't write a test if you don't know the correct end 
> state.
> 
> Jim
> 
> 
> > On Sep 19, 2018, at 10:59 AM, Zachary Turner via lldb-dev 
> >  wrote:
> > 
> > I assume that reproducing race conditions is out of scope?
> > 
> > Also, will it be possible to incorporate these reproducers into the test 
> > suite somehow?  It would be nice if we could create a tar file similar to a 
> > linkrepro, check in the tar file, and then have a test where you don't have 
> > to write any python code, any Makefile, any source code, or any 

Re: [lldb-dev] skip some tests with "check-lldb"

2018-09-20 Thread Jim Ingham via lldb-dev
When I am working on some changes and there are tests I know I've broken but 
don't want to fix just yet, I just edit the test file and skip it using the 
skip decorators.  You have to remember not to check in the skips when you check 
in whatever change you are making - but presumably by the time you are ready to 
submit something you will have fixed the tests anyway.  That's much simpler 
than trying to monkey with the test runner to do your bidding.

Jim

> On Sep 20, 2018, at 5:25 AM, Gábor Márton via lldb-dev 
>  wrote:
> 
> Hi Jonas,
> 
> Thanks for the clarification.
> Finally I could skip some of the tests by using a negative lookahead
> regex passed to --filter. I wouldn't say that this is so convenient
> and easy to overview, but works.
> For example, to skip the TestCalculatorMode.py and TestPrintArray.py
> tests I used the following argument (appended to the normal lit
> command line reported by "ninja check-lldb -v" ) :
> --filter '^((?!(TestCalculatorMode\.py|TestPrintArray\.py)).)*$'
> Would be beneficial though to have a --exclude besides --filter to
> avoid using a complex negative lookahead regex.
> 
> Cheers,
> Gabor
> On Thu, Sep 20, 2018 at 12:53 PM Jonas Devlieghere
>  wrote:
>> 
>> When using lit as the driver (which is the case for check-lldb), the test
>> granularity is at file-level: lit invokes dotest.py for every test file. You
>> should be able to specify files to skip with lit's --filter.
>> 
>> With regards to threading, lit will schedule one instance of dotest.py on 
>> every
>> thread, which processes one test file. Unless you passed specific options for
>> dotest.py, the latter will run the different tests within that one file on
>> different threads as well. (IIRC)
>> 
>> On Sep 19, 2018, at 7:20 PM, Gábor Márton via lldb-dev 
>>  wrote:
>> 
>> That's okay, but is it possible to skip a few tests, when using lit? I was 
>> thinking about moving the test files I want to skip, but that has obvious 
>> drawbacks. Also --filter does not seem so useful in this case.
>> 
>> On Wed, 19 Sep 2018, 18:46 ,  wrote:
>>> 
>>> Unless you pass  --no-multiprocess to dotest, it should detect how many 
>>> cores your system has and use them.
>>> 
>>> --
>>> Ted Woodward
>>> Qualcomm Innovation Center, Inc.
>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux 
>>> Foundation Collaborative Project
>>> 
>>> -Original Message-
>>> From: lldb-dev  On Behalf Of Gábor Márton 
>>> via lldb-dev
>>> Sent: Wednesday, September 19, 2018 11:04 AM
>>> To: lldb-dev@lists.llvm.org
>>> Subject: Re: [lldb-dev] skip some tests with "check-lldb"
>>> 
>>> I just realized that `dotest.py` has a --thread option. Is that the one 
>>> which is used during the lit test (`ninja check-lldb`) ?
>>> 
>>> On Wed, Sep 19, 2018 at 6:00 PM Gábor Márton  wrote:
 
 Hi,
 
 I'd like to skip some tests when I run "ninja check-lldb", because they 
 fail.
 I am on release_70 branch.
 I know I could use dotest.py directly, but that would exercise only one 
 thread.
 Is there a way to execute the tests parallel on all cores and in the
 same time skip some of the tests?
 
 Thanks,
 Gabor
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
>> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC] LLDB Reproducers

2018-09-19 Thread Jim Ingham via lldb-dev
Yes, I think that would be pretty cool.  It is along the same lines we've been 
talking about with using "ProcessMock", "ThreadMock" etc. plugins.  However, I 
think you need both.  For instance if we bobble a gdb-remote packet, you will 
see that in a bad state of one of these higher level state descriptions, but 
without the actual packet traffic you wouldn't have that much help figuring out 
what actually went wrong.  OTOH, things like packet level recording will likely 
be much less stable than capturing state at a higher level.

Jim


> On Sep 19, 2018, at 11:10 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> By the way, several weeks / months ago I had an idea for exposing a debugger 
> object model.  That would be one very powerful way to create reproducers, but 
> it would be a large effort.  The idea is that if every important part of your 
> debugger is represented by some component in a debugger object model, and all 
> interactions (including internal interactions) go through the object model, 
> then you can record every state change to the object model and replay it.
> 
> On Wed, Sep 19, 2018 at 10:59 AM Zachary Turner  wrote:
> I assume that reproducing race conditions is out of scope?
> 
> Also, will it be possible to incorporate these reproducers into the test 
> suite somehow?  It would be nice if we could create a tar file similar to a 
> linkrepro, check in the tar file, and then have a test where you don't have 
> to write any python code, any Makefile, any source code, or any anything for 
> that matter.  It just enumerates all of these repro tar files in a certain 
> location and runs that test.
> 
> On Wed, Sep 19, 2018 at 10:48 AM Leonard Mosescu via lldb-dev 
>  wrote:
> Great, thanks. This means that the lldb-server issues are not in scope for 
> this feature, right?
> 
> On Wed, Sep 19, 2018 at 10:09 AM, Jonas Devlieghere  
> wrote:
> 
> 
>> On Sep 19, 2018, at 6:49 PM, Leonard Mosescu  wrote:
>> 
>> Sounds like a fantastic idea. 
>> 
>> How would this work when the behavior of the debugee process is 
>> non-deterministic?
> 
> All the communication between the debugger and the inferior goes through the
> GDB remote protocol. Because we capture and replay this, we can reproduce
> without running the executable, which is particularly convenient when you were
> originally debugging something on a different device for example. 
> 
>> 
>> On Wed, Sep 19, 2018 at 6:50 AM, Jonas Devlieghere via lldb-dev 
>>  wrote:
>> Hi everyone,
>> 
>> We all know how hard it can be to reproduce an issue or crash in LLDB. There
>> are a lot of moving parts and subtle differences can easily add up. We want 
>> to
>> make this easier by generating reproducers in LLDB, similar to what clang 
>> does
>> today.
>> 
>> The core idea is as follows: during normal operation we capture whatever
>> information is needed to recreate the current state of the debugger. When
>> something goes wrong, this becomes available to the user. Someone else should
>> then be able to reproduce the same issue with only this data, for example on 
>> a
>> different machine.
>> 
>> It's important to note that we want to replay the debug session from the
>> reproducer, rather than just recreating the current state. This ensures that 
>> we
>> have access to all the events leading up to the problem, which are usually 
>> far
>> more important than the error state itself.
>> 
>> # High Level Design
>> 
>> Concretely we want to extend LLDB in two ways:
>> 
>> 1.  We need to add infrastructure to _generate_ the data necessary for
>> reproducing.
>> 2.  We need to add infrastructure to _use_ the data in the reproducer to 
>> replay
>> the debugging session.
>> 
>> Different parts of LLDB will have different definitions of what data they 
>> need
>> to reproduce their path to the issue. For example, capturing the commands
>> executed by the user is very different from tracking the dSYM bundles on 
>> disk.
>> Therefore, we propose to have each component deal with its needs in a 
>> localized
>> way. This has the advantage that the functionality can be developed and 
>> tested
>> independently.
>> 
>> ## Providers
>> 
>> We'll call a combination of (1) and (2) for a given component a `Provider`. 
>> For
>> example, we'd have an provider for user commands and a provider for dSYM 
>> files.
>> A provider will know how to keep track of its information, how to serialize 
>> it
>> as part of the reproducer as well as how to deserialize it again and use it 
>> to
>> recreate the state of the debugger.
>> 
>> With one exception, the lifetime of the provider coincides with that of the
>> `SBDebugger`, because that is the scope of what we consider here to be a 
>> single
>> debug session. The exception would be the provider for the global module 
>> cache,
>> because it is shared between multiple debuggers. Although it would be
>> conceptually straightforward to add a provider for the shared module cache,
>> this significantly increases 

Re: [lldb-dev] [RFC] LLDB Reproducers

2018-09-19 Thread Jim Ingham via lldb-dev
There are a couple of problems with using these reproducers in the testsuite.

The first is that we make no commitments that the a future lldb will implement 
the "same" session with the same sequence of gdb-remote packet requests.  We 
often monkey around with lldb's sequences of requests to make things go faster. 
 So some future lldb will end up making a request that wasn't in the data from 
the reproducer, and at that point we won't really know what to do.  The 
Provider for gdb-remote packets should record the packets it receives - not 
just the answers it gives - so it can detect this error and not go off the 
rails.  But I'm pretty sure it isn't worth the effort to try to get lldb to 
maintain all the old sequences it used in the past in order to support keeping 
the reproducers alive.  But this does mean that this is an unreliable way to 
write tests.

The second is that the reproducers as described have no notion of "expected 
state".  They are meant to go along with a bug report where the "x was wrong" 
part is not contained in the reproducer.  That would be an interesting thing to 
think about adding, but I think the problem space here is complicated enough 
already...  You can't write a test if you don't know the correct end state.

Jim
 

> On Sep 19, 2018, at 10:59 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> I assume that reproducing race conditions is out of scope?
> 
> Also, will it be possible to incorporate these reproducers into the test 
> suite somehow?  It would be nice if we could create a tar file similar to a 
> linkrepro, check in the tar file, and then have a test where you don't have 
> to write any python code, any Makefile, any source code, or any anything for 
> that matter.  It just enumerates all of these repro tar files in a certain 
> location and runs that test.
> 
> On Wed, Sep 19, 2018 at 10:48 AM Leonard Mosescu via lldb-dev 
>  wrote:
> Great, thanks. This means that the lldb-server issues are not in scope for 
> this feature, right?
> 
> On Wed, Sep 19, 2018 at 10:09 AM, Jonas Devlieghere  
> wrote:
> 
> 
>> On Sep 19, 2018, at 6:49 PM, Leonard Mosescu  wrote:
>> 
>> Sounds like a fantastic idea. 
>> 
>> How would this work when the behavior of the debugee process is 
>> non-deterministic?
> 
> All the communication between the debugger and the inferior goes through the
> GDB remote protocol. Because we capture and replay this, we can reproduce
> without running the executable, which is particularly convenient when you were
> originally debugging something on a different device for example. 
> 
>> 
>> On Wed, Sep 19, 2018 at 6:50 AM, Jonas Devlieghere via lldb-dev 
>>  wrote:
>> Hi everyone,
>> 
>> We all know how hard it can be to reproduce an issue or crash in LLDB. There
>> are a lot of moving parts and subtle differences can easily add up. We want 
>> to
>> make this easier by generating reproducers in LLDB, similar to what clang 
>> does
>> today.
>> 
>> The core idea is as follows: during normal operation we capture whatever
>> information is needed to recreate the current state of the debugger. When
>> something goes wrong, this becomes available to the user. Someone else should
>> then be able to reproduce the same issue with only this data, for example on 
>> a
>> different machine.
>> 
>> It's important to note that we want to replay the debug session from the
>> reproducer, rather than just recreating the current state. This ensures that 
>> we
>> have access to all the events leading up to the problem, which are usually 
>> far
>> more important than the error state itself.
>> 
>> # High Level Design
>> 
>> Concretely we want to extend LLDB in two ways:
>> 
>> 1.  We need to add infrastructure to _generate_ the data necessary for
>> reproducing.
>> 2.  We need to add infrastructure to _use_ the data in the reproducer to 
>> replay
>> the debugging session.
>> 
>> Different parts of LLDB will have different definitions of what data they 
>> need
>> to reproduce their path to the issue. For example, capturing the commands
>> executed by the user is very different from tracking the dSYM bundles on 
>> disk.
>> Therefore, we propose to have each component deal with its needs in a 
>> localized
>> way. This has the advantage that the functionality can be developed and 
>> tested
>> independently.
>> 
>> ## Providers
>> 
>> We'll call a combination of (1) and (2) for a given component a `Provider`. 
>> For
>> example, we'd have an provider for user commands and a provider for dSYM 
>> files.
>> A provider will know how to keep track of its information, how to serialize 
>> it
>> as part of the reproducer as well as how to deserialize it again and use it 
>> to
>> recreate the state of the debugger.
>> 
>> With one exception, the lifetime of the provider coincides with that of the
>> `SBDebugger`, because that is the scope of what we consider here to be a 
>> single
>> debug session. The exception would be the provider for the global module 
>> 

Re: [lldb-dev] Using FileCheck in lldb inline tests

2018-08-15 Thread Jim Ingham via lldb-dev


> On Aug 14, 2018, at 7:48 PM, Zachary Turner via lldb-dev 
>  wrote:
> 
> 
> On Tue, Aug 14, 2018 at 6:58 PM Jason Molenda  wrote:
> 
> 
> > On Aug 14, 2018, at 6:39 PM, Zachary Turner  wrote:
> > 
> > Having bugs also makes the debugger harder to innovate in the future 
> > because it’s, not having tests leads to having bugs, and sb api tests leads 
> > to not having te
> 
> Yes, lldb does not have these problems -- because we learned from our decades 
> working on gdb, and did not repeat that mistake.  To be honest, lldb is such 
> a young debugger - barely a decade old, depending on how you count it, that 
> ANY testsuite approach would be fine at this point.  Add a couple more 
> decades and we'd be back into the hole that gdb was in.  {I have not worked 
> on gdb in over a decade, so I don't know how their testing methodology may be 
> today}
> That doesn’t mean that the current approach is the final word.  As new people 
> come onto the project, new ideas come forth and we should entertain them 
> rather than deciding that all decisions are set in stone forever.
> 
> For example, the object model based approach I mentioned earlier would not 
> have any of the problems that you’ve described from gdb.  Just because one 
> set of problems has been solved doesn’t mean we should declare victory and 
> say there’s no point in trying to solve the remaining problems too.  And 
> right now, the problem is that we need to be coming up with a way to make 
> tests easier to write so that people will actually write them

Just historically it is not true that we didn't have problems with command 
output scraping vrs. the testsuite.  The major offender in this regard was 
breakpoint setting/checking.  But the first time I had to go adjust a whole 
bunch of tests when I wanted to change the breakpoint output, I added test 
routines to do all this business, which mitigated the need for these changes.  
It is true I haven't had to fiddle with this much then - though that is mostly 
because I was careful in how I checked output, and didn't check inessential 
aspects.  That's easy to do when you sit down and do it once, but if everybody 
just wings some regex every time they do a check you will end up with lots 
over-eager checks that cause problems down the road.

I also want to reiterate another point Jason made, which is that LLDB not only 
provides a command line interface, but it also provides an API.  Again, we've 
gone over this before but I'm pretty sure that at this point it's still true 
that in terms of number of users, far more people interact with lldb through 
the API (Xcode, VSCode through the gdb-mi or through Greg's new adaptor when 
that's done...) than interact with it through the command line.  And writing SB 
API tests when you add a new feature is a great way to ensure that you have 
thought about exposing your new feature to the SB API's correctly.  In that 
example, a GUI needs to have a way to indicate that these "tail call" frames 
are special and you shouldn't be worried that they have no local variables, 
even though we show you source for them that obviously has variables...

So we need ways to encourage lldb developers to be aware that the SB API's need 
to have (tested) access to new features in the debugger and not just the 
command line.  SB API tests are good way to encourage this.  We can also do it 
as part of reviews and the like, but I think it better done as part of your 
development than after the fact in reviews...  Again, that doesn't argue for 
their being the ONLY way to test things in lldb, but it does need to be part of 
an lldb developer's practice.

I like your idea of some kind of object model based output for test checking.  
I somehow had the impression that that was your intent for lldb-test once it 
was a real boy, but maybe I was just projecting...  Relying on ad hoc dumping 
routines which are mostly for debugging purposes seemed okay as a boot-strap 
method, but not a great design long-term.  Teaching either the SB or the 
lldb_private objects to describe themselves in a structured way and using that 
to do testing seems great.

Jim


> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Where "thread until " should set breakpoints?

2018-08-03 Thread Jim Ingham via lldb-dev
Thanks!  I look forward to the patch.

Jim


> On Aug 2, 2018, at 8:56 PM, Venkata Ramanaiah Nalamothu 
>  wrote:
> 
> Thanks Jim for the elaborate reply.
> 
> I knew what is happening in below piece of code and also has a patch ready 
> but now needs a bit of fine tuning based on your below comments. I just 
> wanted to hear from you folks that my understanding is correct and I am not 
> missing something.
> 
> Also, the code around it needs modification of error messages to refer to 
> 'thread->GetID()' instead of 'm_options.m_thread_idx'. I will create a review 
> on phabricator with all these changes.
> 
> - Ramana
> 
> On Thu, Aug 2, 2018 at 10:56 PM, Jim Ingham  wrote:
> That's a bug.  The code in CommandObjectThreadUntil needs to be a little more 
> careful about sliding the line number to the "nearest source line that has 
> code."  It currently does:
> 
> for (uint32_t line_number : line_numbers) {
>   uint32_t start_idx_ptr = index_ptr;
>   while (start_idx_ptr <= end_ptr) {
> LineEntry line_entry;
> const bool exact = false;
> start_idx_ptr = sc.comp_unit->FindLineEntry(
> start_idx_ptr, line_number, sc.comp_unit, exact, _entry);
> if (start_idx_ptr == UINT32_MAX)
>   break;
> 
> addr_t address =
> line_entry.range.GetBaseAddress().GetLoadAddress(target);
> if (address != LLDB_INVALID_ADDRESS) {
>   if (fun_addr_range.ContainsLoadAddress(address, target))
> address_list.push_back(address);
>   else
> all_in_function = false;
> }
> start_idx_ptr++;
>   }
> }
> 
> The problem is that in the while loop we set:
> 
> const bool exact = false;
> 
> Having exact == false through the whole loop means that all the other line 
> table entries after the last exact match will match because from the index 
> ptr on there are no more entries, so we slide it to the next one.
> 
> In general it's not always easy to tell which lines will actually contribute 
> code so lldb is lax about line number matching, and slides the breakpoint to 
> the nearest subsequent line that has code when there's no exact match.  That 
> seems a good principle to follow here as well.  So I don't want to just set 
> exact to "true".
> 
> I think the better behavior is to try FindLineEntry the first time with exact 
> = false.  If that picks up a different line from the one given, reset the 
> line we are looking for to the found line.  In either case, then go on to 
> search for all the other instances of that line with exact = false.  It might 
> actually be handy to have another function on CompUnit like 
> FindAllEntriesForLine that bundles up this behavior as it seems a generally 
> useful one.
> 
> If you want to give this a try, please do.  Otherwise, file a bug and I'll 
> get to it when I have a chance.
> 
> Jim
> 
> 
> 
> > On Aug 1, 2018, at 10:22 PM, Ramana  wrote:
> > 
> > 
> > On Thu, Aug 2, 2018 at 3:32 AM, Jim Ingham  wrote:
> > 
> > 
> > > On Jul 24, 2018, at 9:05 PM, Ramana via lldb-dev 
> > >  wrote:
> > > 
> > > On the subject line, the ToT lldb (see code around 
> > > CommandObjectThread.cpp:1230) sets the breakpoint on the first exact 
> > > matching line of 'line-number' or the closest line number > 'line-number' 
> > > i.e. the best match.
> > > 
> > > And along with that, starting from the above exact/best matching line 
> > > number index in the line table, the breakpoints are also being set on 
> > > every other line number available in the line table in the current 
> > > function scope. This latter part, I believe, is incorrect.
> > 
> > Why do you think this is incorrect?  
> > 
> > The requirements for "thread until " are:
> > 
> > a) If any code contributed by  is executed before leaving the 
> > function, stop
> > b) If you end up leaving the function w/o triggering (a), then stop
> > 
> > Understood and no concerns on this.
> >  
> > Correct or incorrect should be determined by how well the implementation 
> > fits those requirements.
> > 
> > There isn't currently a reliable indication from the debug information or 
> > line tables that "line N will always be entered starting with the block at 
> > 0x123".  So you can't tell without doing control flow analysis, which if 
> > any of the separate entries in the line table for the same line will get 
> > hit in the course of executing the function.  So the safest thing to do is 
> > to set breakpoints on them all.
> > 
> > From the above, I understand that we have to do this when the debug line 
> > table has more than one entry for a particular source line. And this is 
> > what I referred to as "machine code for one single source line is scattered 
> > across" in my previous mail. Thanks for sharing why we had to do that.
> > 
> > Besides setting a few more breakpoints - which should be pretty cheap - I 

Re: [lldb-dev] Where "thread until " should set breakpoints?

2018-08-02 Thread Jim Ingham via lldb-dev


> On Aug 2, 2018, at 10:26 AM, Jim Ingham via lldb-dev 
>  wrote:
> 
> I think the better behavior is to try FindLineEntry the first time with exact 
> = false.  If that picks up a different line from the one given, reset the 
> line we are looking for to the found line.  In either case, then go on to 
> search for all the other instances of that line with exact = false.  It might 
> actually be handy to have another function on CompUnit like 
> FindAllEntriesForLine that bundles up this behavior as it seems a generally 
> useful one.

That should be "search for all the other instances of that line with exact = 
true"...


Jim

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Where "thread until " should set breakpoints?

2018-08-02 Thread Jim Ingham via lldb-dev
That's a bug.  The code in CommandObjectThreadUntil needs to be a little more 
careful about sliding the line number to the "nearest source line that has 
code."  It currently does:

for (uint32_t line_number : line_numbers) {
  uint32_t start_idx_ptr = index_ptr;
  while (start_idx_ptr <= end_ptr) {
LineEntry line_entry;
const bool exact = false;
start_idx_ptr = sc.comp_unit->FindLineEntry(
start_idx_ptr, line_number, sc.comp_unit, exact, _entry);
if (start_idx_ptr == UINT32_MAX)
  break;

addr_t address =
line_entry.range.GetBaseAddress().GetLoadAddress(target);
if (address != LLDB_INVALID_ADDRESS) {
  if (fun_addr_range.ContainsLoadAddress(address, target))
address_list.push_back(address);
  else
all_in_function = false;
}
start_idx_ptr++;
  }
}

The problem is that in the while loop we set:

const bool exact = false;

Having exact == false through the whole loop means that all the other line 
table entries after the last exact match will match because from the index ptr 
on there are no more entries, so we slide it to the next one.

In general it's not always easy to tell which lines will actually contribute 
code so lldb is lax about line number matching, and slides the breakpoint to 
the nearest subsequent line that has code when there's no exact match.  That 
seems a good principle to follow here as well.  So I don't want to just set 
exact to "true".

I think the better behavior is to try FindLineEntry the first time with exact = 
false.  If that picks up a different line from the one given, reset the line we 
are looking for to the found line.  In either case, then go on to search for 
all the other instances of that line with exact = false.  It might actually be 
handy to have another function on CompUnit like FindAllEntriesForLine that 
bundles up this behavior as it seems a generally useful one.

If you want to give this a try, please do.  Otherwise, file a bug and I'll get 
to it when I have a chance.

Jim



> On Aug 1, 2018, at 10:22 PM, Ramana  wrote:
> 
> 
> On Thu, Aug 2, 2018 at 3:32 AM, Jim Ingham  wrote:
> 
> 
> > On Jul 24, 2018, at 9:05 PM, Ramana via lldb-dev  
> > wrote:
> > 
> > On the subject line, the ToT lldb (see code around 
> > CommandObjectThread.cpp:1230) sets the breakpoint on the first exact 
> > matching line of 'line-number' or the closest line number > 'line-number' 
> > i.e. the best match.
> > 
> > And along with that, starting from the above exact/best matching line 
> > number index in the line table, the breakpoints are also being set on every 
> > other line number available in the line table in the current function 
> > scope. This latter part, I believe, is incorrect.
> 
> Why do you think this is incorrect?  
> 
> The requirements for "thread until " are:
> 
> a) If any code contributed by  is executed before leaving the 
> function, stop
> b) If you end up leaving the function w/o triggering (a), then stop
> 
> Understood and no concerns on this.
>  
> Correct or incorrect should be determined by how well the implementation fits 
> those requirements.
> 
> There isn't currently a reliable indication from the debug information or 
> line tables that "line N will always be entered starting with the block at 
> 0x123".  So you can't tell without doing control flow analysis, which if any 
> of the separate entries in the line table for the same line will get hit in 
> the course of executing the function.  So the safest thing to do is to set 
> breakpoints on them all.
> 
> From the above, I understand that we have to do this when the debug line 
> table has more than one entry for a particular source line. And this is what 
> I referred to as "machine code for one single source line is scattered 
> across" in my previous mail. Thanks for sharing why we had to do that.
> 
> Besides setting a few more breakpoints - which should be pretty cheap - I 
> don't see much downside to the way it is currently implemented.
> 
> Anyway, why did this bother you?
> 
> Jim
> 
> However, I am concerned about the below 'thread until' behaviour. For the 
> attached test case (kernels.cpp - OpenCL code), following is the debug line 
> table generated by the compiler.
> 
> File nameLine numberStarting address
> ./kernels.cpp:[++]
> kernels.cpp9   0xacc74d00
> kernels.cpp   12  0xacc74d00 
> kernels.cpp   14  0xacc74d40
> kernels.cpp   13  0xacc74dc0
> kernels.cpp   14  0xacc74e00
> kernels.cpp   25  0xacc74e80
> kernels.cpp 

Re: [lldb-dev] Where "thread until " should set breakpoints?

2018-08-01 Thread Jim Ingham via lldb-dev


> On Jul 24, 2018, at 9:05 PM, Ramana via lldb-dev  
> wrote:
> 
> On the subject line, the ToT lldb (see code around 
> CommandObjectThread.cpp:1230) sets the breakpoint on the first exact matching 
> line of 'line-number' or the closest line number > 'line-number' i.e. the 
> best match.
> 
> And along with that, starting from the above exact/best matching line number 
> index in the line table, the breakpoints are also being set on every other 
> line number available in the line table in the current function scope. This 
> latter part, I believe, is incorrect.

Why do you think this is incorrect?  

The requirements for "thread until " are:

a) If any code contributed by  is executed before leaving the 
function, stop
b) If you end up leaving the function w/o triggering (a), then stop

Correct or incorrect should be determined by how well the implementation fits 
those requirements.

There isn't currently a reliable indication from the debug information or line 
tables that "line N will always be entered starting with the block at 0x123".  
So you can't tell without doing control flow analysis, which if any of the 
separate entries in the line table for the same line will get hit in the course 
of executing the function.  So the safest thing to do is to set breakpoints on 
them all.  

Besides setting a few more breakpoints - which should be pretty cheap - I don't 
see much downside to the way it is currently implemented.

Anyway, why did this bother you?

Jim


> 
> What, I think, should happen is we just set only one breakpoint on the first 
> exact/best match for the given 'line-number' and another on the return 
> address from the current frame. And this leaves us with one special case 
> where the machine code for one single source line is scattered across (aka 
> scheduled across) and I do not know what is the expected behaviour in this 
> case.
> 
> If I my above understanding is correct and after discussing here on how to 
> handle the scattered code scenario, I will submit a patch.
> 
> Regards,
> Venkata Ramanaiah
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Process launch view duplication

2018-08-01 Thread Jim Ingham via lldb-dev
Generally, the driver will print stop info every time a stop event is forwarded 
from the private event queue to the public event queue.  You can check the 
output of the event log (log enable lldb event) to see if you are indeed 
sending two events.  If you are sending two events, you can add the -S option 
to the log command to see the stack trace when the events were broadcast to the 
public event queue and see if there's an obvious way to suppress one of them.

Jim


> On Jul 23, 2018, at 8:51 AM, Dávid Bolvanský via lldb-dev 
>  wrote:
> 
> Hello,
> 
> Our slightly modified LLDB based on v6.0 with custom platform and process has 
> a strange problem during process launch. We see the debugger view twice. 
> After stepping over, we see it correctly - just once. Does anybody know where 
> the problem could be?
> 
> Process, platform, thread, event logs: https://pastebin.com/3Ecns4W5
> 
> (lldb) target create "/home/davidbolvansky/Plocha/file.xexe"
> Current executable set to '/home/davidbolvansky/Plocha/file.xexe'.
> (lldb) b main
> Breakpoint 1: where = file.xexe`main + 32 at bitcnt.c:76, address = 0x01ec
> (lldb) r
> info: dynamic port 45999
> info: Waiting for a client...
> Process 1 launched: '/home/davidbolvansky/Plocha/file.xexe'.
> Process 1 stopped
> * thread #1, stop reason = breakpoint 1.1
> frame #0: 0x01ec file.xexe`main(argc=1, argv=0x08c0) at 
> bitcnt.c:76
>73   long i, j;
>74   unsigned long count;
>75 
> -> 76   for (i = 0; i < BENCHMARK_RUNS; i++)
>77   {
>78 count = 0;
>79 for (j = 0; j < DATA_TAB_SIZE; j++)
> Process 1 stopped
> * thread #1, stop reason = breakpoint 1.1
> frame #0: 0x01ec file.xexe`main(argc=1, argv=0x08c0) at 
> bitcnt.c:76
>73   long i, j;
>74   unsigned long count;
>75 
> -> 76   for (i = 0; i < BENCHMARK_RUNS; i++)
>77   {
>78 count = 0;
>79 for (j = 0; j < DATA_TAB_SIZE; j++)
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] lldb-mi doesn't propagate host environment variables

2018-07-02 Thread Jim Ingham via lldb-dev
In lldb, this behavior is controlled by the setting:

target.inherit-env

Passing debugger environment is equal parts handy (getting a DISPLAY variable 
right for instance) and a cause for confusion.  Not sure which is the most 
useful default.  lldb defaults to true.  But I can see an argument that for a 
server type program like lldb-mi it is better to be explicit about it.

Jim


> On Jun 29, 2018, at 5:03 AM, k.baladurin via lldb-dev 
>  wrote:
> 
> Hello!
> 
> lldb-mi doesn't propagate host environment variables while lldb does it:
> 
> $ cat env.c
> #include 
> #include 
> 
> int main()
> {
> printf("VAR = %s\n", getenv("VAR"));
> return 0;
> }
> $ gcc env.c -o env
> $ /home/kbaladurin/Downloads/llvm-x64-7.0/bin/lldb -v
> lldb version 7.0.0
> $ VAR=1 /home/kbaladurin/Downloads/llvm-x64-7.0/bin/lldb env
> (lldb) target create "env"
> Current executable set to 'env' (x86_64).
> (lldb) r
> Process 13139 launched: '/home/kbaladurin/Desktop/tests/env' (x86_64)
> VAR = 1
> Process 13139 exited with status = 0 (0x)
> (lldb) ^D
> $ VAR=1 /home/kbaladurin/Downloads/llvm-x64-7.0/bin/lldb-mi
> (gdb)
> -file-exec-and-symbols env
> ^done
> (gdb)
> =library-loaded,id="/home/kbaladurin/Desktop/tests/env",target-name="/home/kbaladurin/Desktop/tests/env",host-name="/home/kbaladurin/Desktop/tests/env",symbols-loaded="0",loaded_addr="-",size="0"
> -exec-run
> ^running
> =thread-group-started,id="i1",pid="13176"
> (gdb)
> =thread-created,id="1",group-id="i1"
> =thread-selected,id="1"
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/ld-2.23.so",target-name="/lib/x86_64-linux-gnu/ld-2.23.so",host-name="/lib/x86_64-linux-gnu/ld-2.23.so",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/ld-2.23.so",loaded_addr="-",size="0"
> (gdb)
> =library-loaded,id="[vdso]",target-name="[vdso]",host-name="[vdso]",symbols-loaded="1",symbols-path="",loaded_addr="0x77ffa000",size="0"
> (gdb)
> =library-loaded,id="/home/kbaladurin/Desktop/tests/env",target-name="/home/kbaladurin/Desktop/tests/env",host-name="/home/kbaladurin/Desktop/tests/env",symbols-loaded="0",loaded_addr="-",size="0"
> (gdb)
> *running,thread-id="all"
> (gdb)
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/libc.so.6",target-name="/lib/x86_64-linux-gnu/libc.so.6",host-name="/lib/x86_64-linux-gnu/libc.so.6",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/libc-2.23.so",loaded_addr="-",size="0"
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/libc.so.6",target-name="/lib/x86_64-linux-gnu/libc.so.6",host-name="/lib/x86_64-linux-gnu/libc.so.6",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/libc-2.23.so",loaded_addr="-",size="0"
> @"VAR = (null)\r\n"
> (gdb)
> =thread-exited,id="1",group-id="i1"
> =thread-group-exited,id="i1",exit-code="0"
> *stopped,reason="exited-normally"
> (gdb)
> 
> Is it expected behavior?
> 
> Thank you!
> 
> BR,
> Konstantin Baladurin
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] building only lldb with debug symbols

2018-06-28 Thread Jim Ingham via lldb-dev
If you are on macOS, then the Xcode build has a "Debug" configuration which 
builds clang/llvm Release+asserts with no debug info, and lldb at -O0 with 
debug info.

Jim


> On Jun 28, 2018, at 2:58 AM, Pavel Labath via lldb-dev 
>  wrote:
> 
> On Thu, 28 Jun 2018 at 04:34, Adrian Harris via lldb-dev
>  wrote:
>> 
>> Hi Everyone,
>> 
>> I'm writing a gdb-server for a new architecture and need to be able to debug 
>> lldb to track down issues. Unfortunately disk space is tight here and the 
>> llvm debug build consumes north of 40Gb with debug symbols. Is there any way 
>> to build *only* lldb with debug symbols (and no optimization)?
> 
> Hi Adrian,
> 
> This is a somewhat non-standard configuration, but there are a couple
> of ways you could achieve that. A couple which come to mind are:
> - If you're fine with hacking the cmake files then you could just
> insert something like
> set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g0")
> into the root llvm CMakeLists.txt
> and then add
> set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g")
> to lldb's CMakeLists.txt
> 
> - If you want a solution which does not involve modifying the cmake
> files, you could build llvm and lldb separately. The llvm could be
> built with -g0 (or even in release mode if you really want) and then
> the stanalone lldb build would be -g.
> 
> - You can also try building with -gsplit-dwarf
> (LLVM_USE_SPLIT_DWARF=On) to see if that saves enough space for things
> to work for you.
> 
> pl
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] RFC: libtrace

2018-06-27 Thread Jim Ingham via lldb-dev

> On Jun 26, 2018, at 5:14 PM, Zachary Turner  wrote:
> 
> Yes that’s what I’ve been thinking about as well.
> 
> One thing I’ve been giving a lot of thought to is whether to serialize the 
> handling of trace events.  I want to balance the “this is a library and you 
> should be able to get it to work for you no matter what your use case is” 
> aspect with the “you really just don’t want to go there, we know what’s best 
> for you” aspect.  Then there’s the  fact that not all platforms behave the 
> same, but we’d like a consistent set of expectations that makes it easy to 
> use for everyone.
> 
> So I’m leaning towards having the library serialize all tace events, because 
> it’s a nice common denominator that every platform can implement.
> 
> To be clear though, I don’t mean that if 2 processes are being traced 
> simultaneously and A stops followed by B stopping, then the tool will 
> necessarily block before handling  B’s stop.  I just mean that A and B’s stop 
> handlers will be invoked on a single thread (not the threads which are 
> tracing  A or B).  
> 
> So A stops, posts its stop event on the blessed thread and waits.  Then B 
> stops and does the same thing.  A’s handler runs, for whatever reason decides 
> it will continue later, saves off the event somewhere, then processes B’s.  
> Later something happens, it decides to continue A, signals A’s thread which 
> wakes up.
> 
> I think this kind of design eliminates a large class of race conditions 
> without sacrificing any performance.  
> 
> LLDB doesn’t currently work like this, but it would be nice not to end up 
> with another split similar to the dwarf split, so I’m curious if you can 
> think of any fundamental assumptions of LLDB’s architecture that this would 
> violate.  This way we’d at least know that it’s possible to use the api in 
> lldb (assuming it does everything lldb needs obviously) 

What you describe is actually pretty much how the lldb driver works.  Every 
time the lower levels of the Process (e.g. ProcessGDBRemote) class notice 
something interesting happening to the process they are managing, they post an 
event to the Listener in charge of driving that process.  Then the process is 
allowed to continue on its way, either stopped or continued depending (the 
event records whether a restart has occurred.)  The upper levels only know 
about what happened to the process when they fetch an event off the event 
queue.  For a single process that serializes the reporting of process state.  

As to multiple processes, you can decide whether you want to serialize all the 
process events using the same mechanism or not, depending on your use case.

In the lldb driver, there's one Listener that waits on all processes (the 
Debugger's listener).  These events all get effectively serialized in its event 
loop.  So if you were just straight using lldb classes you could trivially 
implement what you want to achieve.

That being said, I don't think you want to use lldb's process event system for 
your ptracer.  It has a lot of complexity which supports handling reactions to 
events (breakpoint commands and conditions) that have to operate in the same 
context as user commands even though they happen before the user has regained 
control, and which might or might not restart the process out from under you.  
They also manage the task of concealing the vast majority of stops from the 
higher level clients - for instance to pretend that a single "source line step 
over" didn't actually require lots  of stops and starts.  I don't think 
anything you have described requires handling either of these tasks.

But you could use the general event system to achieve the serialization of 
reporting w/o hooking into the lldb private/public state thread system.

Jim

> 
> Thoughts?
> 
> On Tue, Jun 26, 2018 at 1:09 PM Jim Ingham  wrote:
> You'd probably need to pull the Unwinder in if you want backtraces, but that 
> part shouldn't be that hard to disentangle.  I don't think you'd need much 
> else?
> 
> Basing your work on NativeProcess rather than lldb proper would also cut the 
> number of observer processes in half and avoid the context switches between 
> the server and the debugger.  That seems more appropriate for a lightweight 
> tool.
> 
> Jim
> 
> 
> > On Jun 26, 2018, at 12:59 PM, Jim Ingham via lldb-dev 
> >  wrote:
> > 
> > So you aren't planning to print values at all, just stop points (i.e. you 
> > are only interested in the line table and function symbols part of DWARF)?
> > 
> > Given what you've described so far, I'm wondering if what you really want 
> > is the NativeProcess classes with some symbol-file reading pulled in? Is 
> > there anything that you couldn't do from there?
> > 
> > Jim
> 

Re: [lldb-dev] [llvm-dev] RFC: libtrace

2018-06-26 Thread Jim Ingham via lldb-dev
One important question, does this tool need to work remotely?  

I'm guessing the answer to this is no, since if you are working remotely you 
won't have a performant enough solution to really be an effective tracer.  And 
if the guts of the debugger are remote, you care a lot less about the 
complexity of the remote part.  If you can always debug with the Host platform 
- in lldb terms - then it really does seem like you want to start with the 
NativeProcess classes.  That won't get you macOS hosting, but OTOH this would 
be good reason to get macOS onto the NativeProcess classes/lldb-server and off 
of debugserver...

> On Jun 26, 2018, at 1:38 PM, Zachary Turner via lldb-dev 
>  wrote:
> 
> 
> 
> On Tue, Jun 26, 2018 at 1:28 PM Adrian Prantl  wrote:
> 
> 
> > On Jun 26, 2018, at 11:58 AM, Zachary Turner via llvm-dev 
> >  wrote:
> > 
> > Hi all,
> > 
> > We have been thinking internally about a lightweight llvm-based ptracer.  
> > To address one question up front: the primary way in which this differs 
> > from LLDB is that it targets a more narrow use case -- there is no 
> > scripting support, no clang integration, no dynamic extensibility, no 
> > support for running jitted code in the target, and no user interface.  We 
> > have several use cases internally that call for varying levels of 
> > functionality from such a utility, and being able to use as little as 
> > possible of the library as is necessary for the given task is important for 
> > the scale in which we wish to use it. 
> > 
> > We are still in early discussions and planning, but I think this would be a 
> > good addition to the LLVM upstream.  Since we’re approaching this as a set 
> > of small isolated components, my thinking is to work on this completely 
> > upstream, directly under the llvm project (as opposed to making a separate 
> > subproject), but I’m open to discussion if anyone feels differently.
> > 
> > LLDB has solved a lot of the difficult problems needed for such a tool.  So 
> > in the spirit of code reuse, we think it’s worth trying componentize LLDB 
> > by sinking pieces into LLVM and rebasing LLDB as well as these smaller 
> > tools on top of these components, so that smaller tools can reduce code 
> > duplication and contribute to the overall health of the code base.
> 
> Do you have a rough idea of what components specifically the new tool would 
> need to function?
> 
> * process & thread control
> * platform agnostic ptrace wrapper (not all platforms even have ptrace, and 
> those that do the usage and capabilities vary quite a bit)
> * install various kinds of traps
> * monitor cpu performance counters

This part is all the job of the NativeProcess classes.  That's not terribly 
surprising, since their whole reason for being was as a low-level abstraction 
for process control without any of the higher-level work that lldb does.

> * symbol file parsing
> * symbol resolution (name <-> addr and line <-> addr)

This will involve getting object-file readers and a symbol file reader into 
your trace tool.  These should be pretty easy to extract from lldb, though you 
probably don't need the plugin architecture. 

> * unwinding and backtrace generation

Jason says this will be somewhat tricky to pull out of lldb.  OTOH much of the 
complexity of unwind is reconstructing all the non-volatile registers, and if 
you don't care about values, you don't really need that.  So some kind of 
lightweight pc/sp only backtrace would be more appropriate, and probably faster 
for your needs.

Jim

> 
>  
> 
> >  At the same time we think that in doing so we can break things up into 
> > more granular pieces, ultimately exposing a larger testing surface and 
> > enabling us to create exhaustive tests, giving LLDB more fine grained 
> > testing of important subsystems.
> 
> Are you thinking of the new utility as something that would naturally live in 
> llvm/tools or as something that would live in the LLDB repository?
> I would rather put it under LLDB and then link LLDB against certain pieces in 
> cases where that makes sense.
>  
> 
> > 
> > A good example of this would be LLDB’s DWARF parsing code, which is more 
> > featureful than LLVM’s but has kind of evolved in parallel.  Sinking this 
> > into LLVM would be one early target of such an effort, although over time 
> > there would likely be more.
> 
> As you are undoubtedly aware we've been carefully rearchitecting LLVM's DWARF 
> parser over the last few years to eventually become featureful enough so that 
> LLDB could use it, so any help on that front would be most welcome. As long 
> as we are careful to not regress in performance/lazyness, features and 
> fault-tolerance, deduplicating the implementations can only be good for LLVM 
> and LLDB.
> 
> Yea, this is the general idea.   Has anyone actively been working on this 
> specific effort recently?  To my knowledge someone started and then never 
> finished, but the efforts also never made it upstream, so my 

Re: [lldb-dev] RFC: libtrace

2018-06-26 Thread Jim Ingham via lldb-dev
You'd probably need to pull the Unwinder in if you want backtraces, but that 
part shouldn't be that hard to disentangle.  I don't think you'd need much else?

Basing your work on NativeProcess rather than lldb proper would also cut the 
number of observer processes in half and avoid the context switches between the 
server and the debugger.  That seems more appropriate for a lightweight tool.

Jim


> On Jun 26, 2018, at 12:59 PM, Jim Ingham via lldb-dev 
>  wrote:
> 
> So you aren't planning to print values at all, just stop points (i.e. you are 
> only interested in the line table and function symbols part of DWARF)?
> 
> Given what you've described so far, I'm wondering if what you really want is 
> the NativeProcess classes with some symbol-file reading pulled in? Is there 
> anything that you couldn't do from there?
> 
> Jim
> 
> 
>> On Jun 26, 2018, at 12:48 PM, Zachary Turner  wrote:
>> 
>> no expression parser or knowledge of any specific programming language.
>> 
>> Basically I just mean that the parsing of the native DWARF format itself is 
>> in scope, but anything beyond that is out of scope.  For symbolication we 
>> have things like llvm-symbolizer that already just work and are built on top 
>> of LLVM's dwarf parsing code.  Similarly, LLDB's type system could be built 
>> on top of it as well.  Given that I think everyone mostly agrees that 
>> unifying on one DWARF parser is a good idea in principle, this would mean no 
>> functional change from LLDB's point of view, it would just continue to do 
>> exactly what it does regarding parsing C++ expressions and converting these 
>> into types that clang understands.
>> 
>> It will probably be useful someday to have an expression parser and language 
>> specific type system, but when that comes I don't think we'd want anything 
>> radically different than what LLDB already has.
>> 
>> On Tue, Jun 26, 2018 at 12:26 PM Jim Ingham  wrote:
>> Just to be clear, by "no clang integration" do you mean "no expression 
>> parser" or do you mean something more radical?  For instance, adding a 
>> TypeSystem and its DWARF parser for C family languages that uses a different 
>> underlying representation than Clang AST's to store the results would be a 
>> lot of work that wouldn't be terribly interesting to lldb.  I don't think 
>> that's what you meant, but wanted to be sure.
>> 
>> Jim
>> 
>>> On Jun 26, 2018, at 11:58 AM, Zachary Turner via lldb-dev 
>>>  wrote:
>>> 
>>> Hi all,
>>> 
>>> We have been thinking internally about a lightweight llvm-based ptracer.  
>>> To address one question up front: the primary way in which this differs 
>>> from LLDB is that it targets a more narrow use case -- there is no 
>>> scripting support, no clang integration, no dynamic extensibility, no 
>>> support for running jitted code in the target, and no user interface.  We 
>>> have several use cases internally that call for varying levels of 
>>> functionality from such a utility, and being able to use as little as 
>>> possible of the library as is necessary for the given task is important for 
>>> the scale in which we wish to use it. 
>>> 
>>> We are still in early discussions and planning, but I think this would be a 
>>> good addition to the LLVM upstream.  Since we’re approaching this as a set 
>>> of small isolated components, my thinking is to work on this completely 
>>> upstream, directly under the llvm project (as opposed to making a separate 
>>> subproject), but I’m open to discussion if anyone feels differently.
>>> 
>>> LLDB has solved a lot of the difficult problems needed for such a tool.  So 
>>> in the spirit of code reuse, we think it’s worth trying componentize LLDB 
>>> by sinking pieces into LLVM and rebasing LLDB as well as these smaller 
>>> tools on top of these components, so that smaller tools can reduce code 
>>> duplication and contribute to the overall health of the code base.  At the 
>>> same time we think that in doing so we can break things up into more 
>>> granular pieces, ultimately exposing a larger testing surface and enabling 
>>> us to create exhaustive tests, giving LLDB more fine grained testing of 
>>> important subsystems.
>>> 
>>> A good example of this would be LLDB’s DWARF parsing code, which is more 
>>> featureful than LLVM’s but has kind of evolved in parallel.  Sinking this 
>>> into LLVM would be one early target of such an effort, although over time 
>>> there would likely be more

Re: [lldb-dev] RFC: libtrace

2018-06-26 Thread Jim Ingham via lldb-dev
So you aren't planning to print values at all, just stop points (i.e. you are 
only interested in the line table and function symbols part of DWARF)?

Given what you've described so far, I'm wondering if what you really want is 
the NativeProcess classes with some symbol-file reading pulled in? Is there 
anything that you couldn't do from there?

Jim


> On Jun 26, 2018, at 12:48 PM, Zachary Turner  wrote:
> 
> no expression parser or knowledge of any specific programming language.
> 
> Basically I just mean that the parsing of the native DWARF format itself is 
> in scope, but anything beyond that is out of scope.  For symbolication we 
> have things like llvm-symbolizer that already just work and are built on top 
> of LLVM's dwarf parsing code.  Similarly, LLDB's type system could be built 
> on top of it as well.  Given that I think everyone mostly agrees that 
> unifying on one DWARF parser is a good idea in principle, this would mean no 
> functional change from LLDB's point of view, it would just continue to do 
> exactly what it does regarding parsing C++ expressions and converting these 
> into types that clang understands.
> 
> It will probably be useful someday to have an expression parser and language 
> specific type system, but when that comes I don't think we'd want anything 
> radically different than what LLDB already has.
> 
> On Tue, Jun 26, 2018 at 12:26 PM Jim Ingham  wrote:
> Just to be clear, by "no clang integration" do you mean "no expression 
> parser" or do you mean something more radical?  For instance, adding a 
> TypeSystem and its DWARF parser for C family languages that uses a different 
> underlying representation than Clang AST's to store the results would be a 
> lot of work that wouldn't be terribly interesting to lldb.  I don't think 
> that's what you meant, but wanted to be sure.
> 
> Jim
> 
> > On Jun 26, 2018, at 11:58 AM, Zachary Turner via lldb-dev 
> >  wrote:
> > 
> > Hi all,
> > 
> > We have been thinking internally about a lightweight llvm-based ptracer.  
> > To address one question up front: the primary way in which this differs 
> > from LLDB is that it targets a more narrow use case -- there is no 
> > scripting support, no clang integration, no dynamic extensibility, no 
> > support for running jitted code in the target, and no user interface.  We 
> > have several use cases internally that call for varying levels of 
> > functionality from such a utility, and being able to use as little as 
> > possible of the library as is necessary for the given task is important for 
> > the scale in which we wish to use it. 
> > 
> > We are still in early discussions and planning, but I think this would be a 
> > good addition to the LLVM upstream.  Since we’re approaching this as a set 
> > of small isolated components, my thinking is to work on this completely 
> > upstream, directly under the llvm project (as opposed to making a separate 
> > subproject), but I’m open to discussion if anyone feels differently.
> > 
> > LLDB has solved a lot of the difficult problems needed for such a tool.  So 
> > in the spirit of code reuse, we think it’s worth trying componentize LLDB 
> > by sinking pieces into LLVM and rebasing LLDB as well as these smaller 
> > tools on top of these components, so that smaller tools can reduce code 
> > duplication and contribute to the overall health of the code base.  At the 
> > same time we think that in doing so we can break things up into more 
> > granular pieces, ultimately exposing a larger testing surface and enabling 
> > us to create exhaustive tests, giving LLDB more fine grained testing of 
> > important subsystems.
> > 
> > A good example of this would be LLDB’s DWARF parsing code, which is more 
> > featureful than LLVM’s but has kind of evolved in parallel.  Sinking this 
> > into LLVM would be one early target of such an effort, although over time 
> > there would likely be more.
> > 
> > Anyone have any thoughts / strong opinions on this proposal, or where the 
> > code should live?  Also, does anyone have any suggestions on things they’d 
> > like to see come out of this?  Whether it’s a specific new tool, new 
> > functionality to an existing tool, an architectural or design change to 
> > some existing tool or library, or something else entirely, all feedback and 
> > ideas are welcome.
> > 
> > Thanks,
> > Zach
> > 
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] RFC: libtrace

2018-06-26 Thread Jim Ingham via lldb-dev
Just to be clear, by "no clang integration" do you mean "no expression parser" 
or do you mean something more radical?  For instance, adding a TypeSystem and 
its DWARF parser for C family languages that uses a different underlying 
representation than Clang AST's to store the results would be a lot of work 
that wouldn't be terribly interesting to lldb.  I don't think that's what you 
meant, but wanted to be sure.

Jim

> On Jun 26, 2018, at 11:58 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> Hi all,
> 
> We have been thinking internally about a lightweight llvm-based ptracer.  To 
> address one question up front: the primary way in which this differs from 
> LLDB is that it targets a more narrow use case -- there is no scripting 
> support, no clang integration, no dynamic extensibility, no support for 
> running jitted code in the target, and no user interface.  We have several 
> use cases internally that call for varying levels of functionality from such 
> a utility, and being able to use as little as possible of the library as is 
> necessary for the given task is important for the scale in which we wish to 
> use it. 
> 
> We are still in early discussions and planning, but I think this would be a 
> good addition to the LLVM upstream.  Since we’re approaching this as a set of 
> small isolated components, my thinking is to work on this completely 
> upstream, directly under the llvm project (as opposed to making a separate 
> subproject), but I’m open to discussion if anyone feels differently.
> 
> LLDB has solved a lot of the difficult problems needed for such a tool.  So 
> in the spirit of code reuse, we think it’s worth trying componentize LLDB by 
> sinking pieces into LLVM and rebasing LLDB as well as these smaller tools on 
> top of these components, so that smaller tools can reduce code duplication 
> and contribute to the overall health of the code base.  At the same time we 
> think that in doing so we can break things up into more granular pieces, 
> ultimately exposing a larger testing surface and enabling us to create 
> exhaustive tests, giving LLDB more fine grained testing of important 
> subsystems.
> 
> A good example of this would be LLDB’s DWARF parsing code, which is more 
> featureful than LLVM’s but has kind of evolved in parallel.  Sinking this 
> into LLVM would be one early target of such an effort, although over time 
> there would likely be more.
> 
> Anyone have any thoughts / strong opinions on this proposal, or where the 
> code should live?  Also, does anyone have any suggestions on things they’d 
> like to see come out of this?  Whether it’s a specific new tool, new 
> functionality to an existing tool, an architectural or design change to some 
> existing tool or library, or something else entirely, all feedback and ideas 
> are welcome.
> 
> Thanks,
> Zach
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Handling callable addresses

2018-06-21 Thread Jim Ingham via lldb-dev
It's actually a little lame that we have a switch over architectures in generic 
Target code.  This should be gotten from the architecture plugin.  
GetBreakableLoadAddress is even worse, there's ~150 lines of MIPS specific code 
in Target.cpp, which seems really wrong to me.

Not really answering your question except formally none of this code should be 
in Target.cpp...

Jim


> On Jun 21, 2018, at 11:52 AM, Tatyana Krasnukha via lldb-dev 
>  wrote:
> 
> Hi,
>  
> It seems Target::GetCallableLoadAddress does equivalent things for arm/thumb 
> and for mipses, just written in different forms. May I join these cases 
> together?
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Accessing only specific words in guest memory

2018-06-21 Thread Jim Ingham via lldb-dev
Ah, I see.  It looks like you have to set this before you create the process.  
If I put this setting in my .lldbinit, or on the command line before I do "run" 
it does limit the read size.  But apparently once we've created the memory 
cache we don't replace it when this value changes.

That's still worth a bug either to fix it or to make the help string for this 
setting tell you how to use it.

Jim


> On Jun 21, 2018, at 1:47 PM, Jim Ingham  wrote:
> 
> A quick scan of the code looks like this should be hooked up, but you are 
> right, it doesn't seem to actually limit the cache page size.  There's only 
> one test for this and that one just checks that the setting has some 
> particular value for android.  Can you file a bug for this with bugs.llvm.org?
> 
> Jim
> 
> 
>> On Jun 21, 2018, at 11:51 AM, Andrew Baumann  
>> wrote:
>> 
>> Thanks! Disabling the cache worked.
>> 
>> Curiously, setting the line size seems to have no effect – it defaults to 
>> 512 (words, bytes, ...?), and still tries to read a page of guest memory 
>> regardless of how small I set it.
>> 
>> Andrew
>> 
>> From: jing...@apple.com  on behalf of Jim Ingham 
>> 
>> Sent: Thursday, June 21, 2018 11:32:03 AM
>> To: Andrew Baumann
>> Cc: lldb-dev@lists.llvm.org
>> Subject: Re: [lldb-dev] Accessing only specific words in guest memory
>> 
>> By default, lldb uses a memory cache to avoid making lots of tiny memory 
>> requests, which tends to be inefficient especially when talking to a remote 
>> device.  You can turn this off with the 
>> "target.process.disable-memory-cache" setting.
>> 
>> You can also adjust the cache page size with 
>> "target.process.memory-cache-line-size".
>> 
>> We don't yet have a memory map command to tell us what regions to read and 
>> not to read.  So if you try to actually print the structure (with "frame 
>> var" or "expr") we will grab the whole thing.  But turning off the cache 
>> should allow you to avoid the disallowed regions manually.
>> 
>> Jim
>> 
>>> On Jun 21, 2018, at 11:20 AM, Andrew Baumann via lldb-dev 
>>>  wrote:
>>> 
>>> Hi,
>>> 
>>> I’m trying to use lldb 6.0 on Linux to debug a process that has an SGX 
>>> enclave inside it. One of the data structures there (the “TCS”) only 
>>> permits access to certain fields. The problem is that when I try to access 
>>> those fields in LLDB, it tries to read an entire page of memory from the 
>>> guest, even if I only wanted to read one word. It then fails the operation 
>>> when access to later parts of the page fails.
>>> 
>>> e.g., this LLDB command:
>>> 
>>> (lldb) x/1g 0x25000
>>> error: memory read failed for 0x25000
>>> 
>>> Triggered this series of failures in the lldb-server process:
>>> 
>>> read(7, "$x25000,200#ed", 8192) = 18
>>> gettid()= 18169
>>> process_vm_readv(18174, [{iov_base=0x556a00151cf0, iov_len=512}], 1, 
>>> [{iov_base=0x25000, iov_len=512}], 1, 0) = -1 EFAULT (Bad address)
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25000, [NULL]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25008, [NULL]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25010, [0xd000]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25018, [0x30001]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25020, [0x1d950]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25028, [0x3fffc00be750]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25030, [0xa000]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25038, [0xb000]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25040, [0x]) = 0
>>> ptrace(PTRACE_PEEKDATA, 18174, 0x25048, 0x7ffc96125858) = -1 EIO 
>>> (Input/output error)
>>> write(7, "$E08#ad", 7)  = 7
>>> 
>>> How can I convince lldb to access just the words I asked it for?
>>> 
>>> Thanks,
>>> Andrew
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Accessing only specific words in guest memory

2018-06-21 Thread Jim Ingham via lldb-dev
A quick scan of the code looks like this should be hooked up, but you are 
right, it doesn't seem to actually limit the cache page size.  There's only one 
test for this and that one just checks that the setting has some particular 
value for android.  Can you file a bug for this with bugs.llvm.org?

Jim


> On Jun 21, 2018, at 11:51 AM, Andrew Baumann  
> wrote:
> 
> Thanks! Disabling the cache worked.
>  
> Curiously, setting the line size seems to have no effect – it defaults to 512 
> (words, bytes, ...?), and still tries to read a page of guest memory 
> regardless of how small I set it.
>  
> Andrew
>  
> From: jing...@apple.com  on behalf of Jim Ingham 
> 
> Sent: Thursday, June 21, 2018 11:32:03 AM
> To: Andrew Baumann
> Cc: lldb-dev@lists.llvm.org
> Subject: Re: [lldb-dev] Accessing only specific words in guest memory
>  
> By default, lldb uses a memory cache to avoid making lots of tiny memory 
> requests, which tends to be inefficient especially when talking to a remote 
> device.  You can turn this off with the "target.process.disable-memory-cache" 
> setting.
> 
> You can also adjust the cache page size with 
> "target.process.memory-cache-line-size".
> 
> We don't yet have a memory map command to tell us what regions to read and 
> not to read.  So if you try to actually print the structure (with "frame var" 
> or "expr") we will grab the whole thing.  But turning off the cache should 
> allow you to avoid the disallowed regions manually.
> 
> Jim
> 
> > On Jun 21, 2018, at 11:20 AM, Andrew Baumann via lldb-dev 
> >  wrote:
> > 
> > Hi,
> >  
> > I’m trying to use lldb 6.0 on Linux to debug a process that has an SGX 
> > enclave inside it. One of the data structures there (the “TCS”) only 
> > permits access to certain fields. The problem is that when I try to access 
> > those fields in LLDB, it tries to read an entire page of memory from the 
> > guest, even if I only wanted to read one word. It then fails the operation 
> > when access to later parts of the page fails.
> >  
> > e.g., this LLDB command:
> >  
> > (lldb) x/1g 0x25000
> > error: memory read failed for 0x25000
> >  
> > Triggered this series of failures in the lldb-server process:
> >  
> > read(7, "$x25000,200#ed", 8192) = 18
> > gettid()= 18169
> > process_vm_readv(18174, [{iov_base=0x556a00151cf0, iov_len=512}], 1, 
> > [{iov_base=0x25000, iov_len=512}], 1, 0) = -1 EFAULT (Bad address)
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25000, [NULL]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25008, [NULL]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25010, [0xd000]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25018, [0x30001]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25020, [0x1d950]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25028, [0x3fffc00be750]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25030, [0xa000]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25038, [0xb000]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25040, [0x]) = 0
> > ptrace(PTRACE_PEEKDATA, 18174, 0x25048, 0x7ffc96125858) = -1 EIO 
> > (Input/output error)
> > write(7, "$E08#ad", 7)  = 7
> >  
> > How can I convince lldb to access just the words I asked it for?
> >  
> > Thanks,
> > Andrew
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Accessing only specific words in guest memory

2018-06-21 Thread Jim Ingham via lldb-dev
By default, lldb uses a memory cache to avoid making lots of tiny memory 
requests, which tends to be inefficient especially when talking to a remote 
device.  You can turn this off with the "target.process.disable-memory-cache" 
setting.

You can also adjust the cache page size with 
"target.process.memory-cache-line-size".

We don't yet have a memory map command to tell us what regions to read and not 
to read.  So if you try to actually print the structure (with "frame var" or 
"expr") we will grab the whole thing.  But turning off the cache should allow 
you to avoid the disallowed regions manually.

Jim

> On Jun 21, 2018, at 11:20 AM, Andrew Baumann via lldb-dev 
>  wrote:
> 
> Hi,
>  
> I’m trying to use lldb 6.0 on Linux to debug a process that has an SGX 
> enclave inside it. One of the data structures there (the “TCS”) only permits 
> access to certain fields. The problem is that when I try to access those 
> fields in LLDB, it tries to read an entire page of memory from the guest, 
> even if I only wanted to read one word. It then fails the operation when 
> access to later parts of the page fails.
>  
> e.g., this LLDB command:
>  
> (lldb) x/1g 0x25000
> error: memory read failed for 0x25000
>  
> Triggered this series of failures in the lldb-server process:
>  
> read(7, "$x25000,200#ed", 8192) = 18
> gettid()= 18169
> process_vm_readv(18174, [{iov_base=0x556a00151cf0, iov_len=512}], 1, 
> [{iov_base=0x25000, iov_len=512}], 1, 0) = -1 EFAULT (Bad address)
> ptrace(PTRACE_PEEKDATA, 18174, 0x25000, [NULL]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25008, [NULL]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25010, [0xd000]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25018, [0x30001]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25020, [0x1d950]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25028, [0x3fffc00be750]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25030, [0xa000]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25038, [0xb000]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25040, [0x]) = 0
> ptrace(PTRACE_PEEKDATA, 18174, 0x25048, 0x7ffc96125858) = -1 EIO 
> (Input/output error)
> write(7, "$E08#ad", 7)  = 7
>  
> How can I convince lldb to access just the words I asked it for?
>  
> Thanks,
> Andrew
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] clang::VersionTuple

2018-06-15 Thread Jim Ingham via lldb-dev


> On Jun 15, 2018, at 3:44 AM, Pavel Labath via lldb-dev 
>  wrote:
> 
> Hello again,
> 
> Just a quick update on the state of this.
> 
> I've managed to move VersionTuple from clang to llvm. I've also
> created  to switch over our version
> handling to that class.
> 
> Could I interest anyone in taking a quick look at the patch?


Somehow I can’t log into Phabricator from home so I can’t comment right now, 
but I took a look.  

In some of your changes in the SB files you do:

  if (PlatformSP platform_sp = GetSP())
version = platform_sp->GetOSVersion();

I don’t like putting initializers in if statements like this because I always 
have to think twice about whether you meant “==“.  Moreover, all of the 
surrounding code does it differently:

  PlatformSP platform_sp = GetSP()
  if (platform_sp)
version = platform_sp->GetOSVersion();

so switching to the other form in a couple of places only kinda forces the 
double-take.  But that’s a little nit.

Given that clang::VersionTuple::tryParse does what the hand-parsing in place 
(which I’m assuming it does or you wouldn’t have used it) this looks fine and a 
nice cleanup to me.

Jim


> 
> pl
> 
> On Wed, 9 May 2018 at 08:49, Pavel Labath  wrote:
>> 
>> Thank you all for the feedback. I'll get on with this when I find some
>> spare time. I will send a patch which will show the final look of the code,
>> before I start moving VersionTuple into llvm.
>> 
>> cheers,
>> pl
>> On Tue, 8 May 2018 at 19:46, Frédéric Riss via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>> 
>> 
>> 
>>> On May 8, 2018, at 11:37 AM, Greg Clayton  wrote:
>> 
>>> I was referring to the Swift and Apple internal branches. They tend to
>> lock down against older llvm and clang repositories so when we put changes
>> in llvm or clang that are required for LLDB, it makes merging a bit tougher
>> in those cases. Again, I am not affected by this, just trying to watch out
>> for you guys.
>> 
>> 
>>> I understand and I appreciate it, I was just worried that I’m missing
>> something obvious. We branch LLDB at the same time as LLVM so that’s not
>> actually an issue. Of course, it might cause merge conflicts or make it
>> harder to cherry-pick patches, but that's just living downstream.
>> 
>>> Fred
>> 
>>> Greg
>> 
>> 
>>> On May 8, 2018, at 11:35 AM, Greg Clayton  wrote:
>> 
>>> I'm good if Apple is good.
>> 
>>> On May 8, 2018, at 11:31 AM, Frédéric Riss  wrote:
>> 
>> 
>> 
>>> On May 8, 2018, at 10:04 AM, Greg Clayton via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>> 
>> 
>> 
>>> On May 8, 2018, at 9:47 AM, Zachary Turner  wrote:
>> 
>>> We don’t want the lowest levels of lldb to depend on clang. If this is
>> useful we should move it from clang to llvm and use llvm::VersionTuple
>> 
>> 
>>> I agree, though this move will cause merging issues for many that have
>> repositories that link against older llvm/clang. Doesn't affect me anymore,
>> but Apple will be affected.
>> 
>> 
>>> I’m not sure I understand what issues you’re referring to, we don’t link
>> new LLDBs to old clangs (and even if we did, it wouldn’t be something the
>> that drives community decisions).
>> 
>>> Fred
>> 
>>> Greg
>> 
>>> On Tue, May 8, 2018 at 9:26 AM Greg Clayton via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>> 
 No issues from me.
>> 
> On May 8, 2018, at 9:11 AM, Pavel Labath via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
> 
> While moving Args around, I noticed that we have a bunch of
> functions/classes that pass/store version numbers as a triplet of
>> integers
> (e.g. Platform::GetOSVersion). I got halfway into creating a wrapper
>> class
> for that when I noticed clang::VersionTuple, which is pretty much what
>> I
> wanted out of the box.
> 
> Now there are small differences between this class, and what we have
>> now:
> it has an extra fourth "build" field, and it uses only 31 bits to
>> represent
> the values. None of these seem to matter (particularly as we are
> converting our representation into this struct in some places) that
>> much,
> but before I go through the trouble of pulling this class into llvm
> (although technically possible, it seems wrong to pull a clang
>> dependency
> at such a low level), I wanted to make sure we are able to use it.
> 
> Do you see any reason why we could not replace our version triplets
>> with
> clang::VersionTuple ?
> 
> cheers,
> pl
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
 ___
 lldb-dev mailing list
 lldb-dev@lists.llvm.org
 http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
>> 
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> 

Re: [lldb-dev] Do we have any infrastructure for creating mini dump files from a loaded process or from a core file?

2018-06-13 Thread Jim Ingham via lldb-dev
Greg already wrote a "save_crashlog" Python command that writes the state of 
the program as a macOS flavor Crashlog file.  It's in 
examples/Python/crashlog.py.  My guess is he had something similar to that in 
mind, but writing a mini dump file instead.

Jim


> On Jun 13, 2018, at 3:20 PM, Leonard Mosescu via lldb-dev 
>  wrote:
> 
> What about the case where you already have a Unix core file and you aren't in 
> a debugger but just want to convert it?  
> 
> Just curious, would a small Python script using the LLDB SB API satisfy this 
> requirement? 
> 
>  We could move all the code for consuming and producing Windows minidumps and 
> Unix / Mach-O corefiles from LLDB down into LLVMCoreFile, write a library 
> like llvm-core that can manipulate or inspect them, then have LLDB use it.  
> Kill 2 birds with one stone that way IMO.
> 
> I like the idea of factoring out reusable subsystems, and I'd love to see 
> something along these lines. Just a word of caution though: the hard part may 
> not be the generation of a "structurally valid" minidump file, but "parsing" 
> and modeling the process state (figuring out the list of modules & memory 
> regions, etc. See the Crashpad implementation for details).
> 
> On Wed, Jun 13, 2018 at 3:01 PM, Zachary Turner via lldb-dev 
>  wrote:
> Yea, I think something like this would actually make a useful llvm utility.  
> Call it llvm-core or something, and it links against the library 
> LLVMCoreFile.  We could move all the code for consuming and producing Windows 
> minidumps and Unix / Mach-O corefiles from LLDB down into LLVMCoreFile, write 
> a library like llvm-core that can manipulate or inspect them, then have LLDB 
> use it.  Kill 2 birds with one stone that way IMO.
> 
> On Wed, Jun 13, 2018 at 2:56 PM Jason Molenda  wrote:
> fwiw I had to prototype a new LC_NOTE load command a year ago in Mach-O core 
> files, to specify where the kernel binary was located.  I wrote a utility to 
> add the data to an existing corefile - both load command and payload - and it 
> was only about five hundred lines of C++.  I didn't link against anything but 
> libc, it's such  a simple task I didn't sweat trying to find an 
> object-file-reader/writer library.  ELF may be more complicated though.  
> 
> > On Jun 13, 2018, at 2:51 PM, Zachary Turner via lldb-dev 
> >  wrote:
> > 
> > What about the case where you already have a Unix core file and you aren't 
> > in a debugger but just want to convert it?  It seems like we could have a 
> > standalone utility that did that (one could imagine doing the reverse too). 
> >  I'm wondering if it wouldn't be possible to do this as a library or 
> > something that didn't have any dependencies on LLDB, that way a standalone 
> > tool could link against this library, and so could LLDB.  I think this 
> > would improve its usefulness quite a bit.
> > 
> > On Wed, Jun 13, 2018 at 2:42 PM Greg Clayton  wrote:
> > The goal is to take a live process (regular process just stopped, or a core 
> > file) and run "save_minidump ..." as a command and export a minidump file 
> > that can be sent elsewhere. Unix core files are too large to always send 
> > and they are less useful if they are not examined in the machine that they 
> > were produced on. So LLDB gives us the connection to the live process, and 
> > we can then create a minidump file. I am going to create a python module 
> > that can do this for us.
> > 
> > Greg 
> > 
> > 
> >> On Jun 13, 2018, at 2:29 PM, Zachary Turner via lldb-dev 
> >>  wrote:
> >> 
> >> Also, if the goal is to have this upstream somewhere, it would be nice to 
> >> have a tool this be a standalone tool.  This seems like something that you 
> >> shouldn't be required to start up a debugger to do, and probably doesn't 
> >> have many (or any for that matters) on the rest of LLDB.
> >> 
> >> On Wed, Jun 13, 2018 at 1:58 PM Leonard Mosescu  wrote:
> >> That being said, it's not exactly trivial to produce a good minidump. 
> >> Crashpad has a native & cross-platform minidump writer, that's what I'd 
> >> start with. 
> >> 
> >> Addendum: I realized after sending the email that if the goal is to 
> >> convert core files -> LLDB -> minidump a lot of the complexity found in 
> >> Crashpad can be avoided, so perhaps writing an LLDB minidump writer from 
> >> scratch would not be too bad.
> >> 
> >> On Wed, Jun 13, 2018 at 1:50 PM, Leonard Mosescu  
> >> wrote:
> >> The minidump format is more or less documented in MSDN. 
> >> 
> >> That being said, it's not exactly trivial to produce a good minidump. 
> >> Crashpad has a native & cross-platform minidump writer, that's what I'd 
> >> start with.
> >> 
> >> On Wed, Jun 13, 2018 at 1:38 PM, Adrian McCarthy via lldb-dev 
> >>  wrote:
> >> Zach's right.  On Windows, lldb can produce a minidump, but it just calls 
> >> out to a Microsoft library to do so.  We don't have any platform-agnostic 
> >> code for producing a minidump.
> >> 
> >> I've also pinged another Googler who 

Re: [lldb-dev] Making changes to the SB API

2018-06-11 Thread Jim Ingham via lldb-dev
Overloading with different numbers of arguments works with no problems, and we 
use that in a bunch of places in the SB API.  I seem to remember that SWIG 
doesn't always get more subtle overloading (float vrs. double) right, but I 
can't bring a specific example to mind.  Anyway, Greg's solution should work 
just fine.

Jim

> On Jun 11, 2018, at 12:20 PM, Leonard Mosescu via lldb-dev 
>  wrote:
> 
> Thanks. I wasn't sure how well C++ overloading works with SWIG, that's 
> definitely a more ergonomic solution.
> 
> 
> On Fri, Jun 8, 2018 at 1:16 PM, Greg Clayton  wrote:
> 
> 
>> On Jun 8, 2018, at 12:54 PM, Leonard Mosescu via lldb-dev 
>>  wrote:
>> 
>> What is the governing philosophy around making changes to the SB API? The 
>> "SB API Coding Rules"
>> page establishes the practices on how to avoid introducing accidental 
>> incompatibility, but what 
>> about the cases where there's a case for intentionally making changes?
>> 
>> For example, I'd like to make a small change to SBTarget to allow surfacing 
>> errors during LoadCore():
>> 
>> SBProcess SBTarget::LoadCore(const char *core_file)
>> 
>> And add an explicit out error parameter (in line with SBTarget::Attach(), 
>> Launch(), ...):
>> 
>> SBProcess SBTarget::LoadCore(const char *core_file, SBError )
>> 
>> If the rule is to strictly avoid any kind of changes then I'd have to resort 
>> to
>> a COM-like versioning and introduce a new SBTarget::LoadCore2 (or 
>> LoadCoreEx, ... pick
>> your poison, I'm not set on any name) while also keeping the existing 
>> LoadCore().
>> 
>> Any guidance on this? Thanks!
>> 
> 
> Just add an extra overloaded version of LoadCore. We don't want people's code 
> to not link and since Apple uses a LLDBRPC.framework that sub-launches a 
> lldb-rpc-server which can connect to older LLDB.framework binaries, we can't 
> remove functions.
> 
> Greg
> 
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] LLDB Process Attach Failed When Waiting

2018-06-05 Thread Jim Ingham via lldb-dev
Except for Windows and FreeBSD, lldb uses a server program to do the actual 
debugging - either debugserver on Darwin or lldb-server elsewhere.  The 
interface to these servers (and to the in-process debugging in Windows & 
FreeBSD) is abstracted being Process Plugins, which the generic code uses.  
Target::Attach is in generic code, it won't know anything about the actual 
method used to attach, wait for attach, whatever.  That will be dispatched to 
the currently active ProcessPlugin.

ProcessGDBRemote is the plugin that is used on Linux & Darwin.  It is the code 
that actually talks to debugserver or lldb-rpc-server from within lldb.  And 
that's the code in lldb that sends the vAttachWait gdb-remote protocol packet 
to instruct the above-mentioned servers to implement attach-wait.  That request 
works on Darwin because debugserver handles the vAttachWait packet, but doesn't 
work on Linux because lldb-rpc-server doesn't know currently respond to the 
vAttachWait packet.  So all you should need to do is teach lldb-server to 
handle the vAttachWait packet.

Jim
  

> On Jun 5, 2018, at 5:44 PM, Ryan Lovelett via lldb-dev 
>  wrote:
> 
> I think I might be a little lost. I built a lldb in debug mode and I am 
> running lldb in an lldb debugger (very meta).
> 
> Earlier in the thread you said "we need a better error message when 
> vAttachWait returns unsupported" I have found where the error message, e.g., 
> "error: attach failed: lost connection" is constructed. The "attach failed: " 
> comes from here [1] and the "lost connection" comes from here [2].
> 
> What do you mean by "vAttachWait"? Am I missing something obvious?
> 
> It seems like you are expecting lldb-server to be the place where the fix 
> will be implemented. Though in the debugger I'm seeing the method 
> Target::Attach() [3] as the place where the connection attempt is made and 
> fails.
> 
> [1] 
> https://github.com/apple/swift-lldb/blob/a8c149f75a8cba674bead048cd9c80ddc8166a8a/source/Commands/CommandObjectProcess.cpp#L518
> [2] 
> https://github.com/apple/swift-lldb/blob/a8c149f75a8cba674bead048cd9c80ddc8166a8a/source/Target/Target.cpp#L3444-L3445
> [3] 
> https://github.com/apple/swift-lldb/blob/a8c149f75a8cba674bead048cd9c80ddc8166a8a/source/Target/Target.cpp#L3374
> 
> On Mon, Jun 4, 2018, at 7:04 PM, Greg Clayton wrote:
>> I will be too busy this week to get to this, so please do have a stab at it.
>> 
>> Basically the flow that debug server does is:
>> 1 - get a list of all processes whose basename matches and remember those 
>> pids so we don't try to attach to them since we are waiting for a new 
>> process to show up
>> 2 - poll the OS for the process list and wait for a new pid to show up with 
>> the basename and attach to the first one that matches whose pid isn't i the 
>> list from step #1
>> 
>> On MacOSX we don't have a way to be notified of new processes are spawned, 
>> not sure on other unix variants. If it is possible, we would want change to:
>> 1 - sign up to be notified about new processes
>> 2 - as each process gets launched, check for a match and attach as quickly 
>> as possible
>> 
>> Hope this helps and I look forward to seeing your patch!
>> 
>> Greg
>> 
>>> On Jun 4, 2018, at 5:56 AM, Ryan Lovelett  wrote:
>>> 
>>> Greg,
>>> 
>>> Is there anything I can do to help you implement or test this feature? 
>>> Obviously I'm willing to roll-up my sleeves and work on this myself too if 
>>> you've become more busy than you expected. That happens to us all and I 
>>> completely understand.
>>> 
>>> Not being able to debug in this manner is blocking me from adding Linux 
>>> support to a Swift program. As you might imagine, I'm a little antsy to get 
>>> past this so I can start writing some code. 
>>> 
>>> On Thu, May 31, 2018, at 5:08 PM, Greg Clayton wrote:
 
 
> On May 31, 2018, at 12:40 PM, Ryan Lovelett  wrote:
> 
> Well at least there is a little good news. I understood way more of those 
> logs than I thought I did.
> 
>> So two issues here:
>> 1 - we need a better error message when vAttachWait returns unsupported
>> 2 - fix lldb-server to support vAttachWait
> 
> A few questions.
> 
> What is the protocol here? Can/should I report the issues at 
> https://bugs.llvm.org/ ? Or is that something that a project member needs 
> to do? Assuming that I can, would this be two seperate issues or one 
> "large" issue?
> 
> Barring that this is not something someone else might implement 
> better/faster than me. Considering I have zero LLDB development 
> experience is this something that I could tackle?
 
 I didn't realize this functionality was missing in lldb-server. I can take 
 a stab at implementing it and see what I can do. Stay tuned.
 
> 
> On Thu, May 31, 2018, at 3:02 PM, Greg Clayton wrote:
>> lldb-server claims it doesn't support it:
>> 
>> lldb <  48> send packet: 
>> 

Re: [lldb-dev] error printed to stderr: "Could not find implementation lookup function...."

2018-06-05 Thread Jim Ingham via lldb-dev
Greg,

Something is going wrong here before you get to this point.  You can't get to 
the code in question (creating the AppleObjCTrampolineHandler) if you don't 
have an ObjCLanguageRuntime of some sort.  The only way the trampoline gets 
made is in ReadObjCLibrary, which is a pure virtual method on 
ObjCLanguageRuntime, so somebody had to successfully made an 
ObjCLanguageRuntime.

However, the CreateInstance for the concrete ObjCLanguageRuntimes calls 
AppleObjCRuntime::GetObjCVersion and then checks that the version is the right 
one for the V1 or V2 implementations.  That will only return something 
different from eObjC_VersionUnknown if a scan over the loaded modules finds one 
for which AppleObjCRuntime::AppleIsModuleObjCLibrary returns true.  But that 
just checks that libobjc.A.dylib is in the target list.

So I don't see how you can get to this error if you don't have libobjc.A.dylib 
loaded.  OTOH, I don't know how you are seeing a libobjc.A.dylib that doesn't 
have class_getMethodImplementation in it...

BTW:

(lldb) b s -p return
Breakpoint 1: where = hello`main + 13 at hello.c:6, address = 0x00010fad
(lldb) run
Process 56377 launched: '/tmp/hello' (x86_64)
Process 56377 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
frame #0: 0x00010fad hello`main at hello.c:6
   3int
   4main()
   5{
-> 6  return 0;
  ^
   7}
Target 0: (hello) stopped.
(lldb) image list libobjc.A.dylib
[  0] DD9E5EC5-B507-3249-B700-93433E2D5EDF 0x7fff5093f000 
/usr/lib/libobjc.A.dylib 

So I'm also not sure it is possible to make a darwin binary that doesn't load 
libobjc.A.dylib.

Jim


> On Jun 5, 2018, at 1:20 PM, Greg Clayton  wrote:
> 
> This error gets printed for all Darwin programs that are not objective C and 
> is quite annoying in AppleObjCTrampolineHandler.cpp:
> 
> 
> 
>  if (m_impl_fn_addr == LLDB_INVALID_ADDRESS) {
>// If we can't even find the ordinary get method implementation function,
>// then we aren't going to be able to
>// step through any method dispatches.  Warn to that effect and get out of
>// here.
>if (process_sp->CanJIT()) {
>  process_sp->GetTarget().GetDebugger().GetErrorFile()->Printf(
>  "Could not find implementation lookup function \"%s\""
>  " step in through ObjC method dispatch will not work.\n",
>  get_impl_name.AsCString());
>}
>return;
>  } ...
> 
> 
> Can we quiet this error when libobjc isn't loaded? Our IDE prints out 
> anything it gets on stderr and this error always has people worried there is 
> something wrong.

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Issue with expression parser finding the correct types

2018-04-25 Thread Jim Ingham via lldb-dev


> On Apr 25, 2018, at 7:49 PM, Jim Ingham via lldb-dev 
> <lldb-dev@lists.llvm.org> wrote:
> 
> NamespaceSearchContext

that should be NameSearchContext.

Jim

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Issue with expression parser finding the correct types

2018-04-25 Thread Jim Ingham via lldb-dev


> On Apr 25, 2018, at 5:52 PM, Greg Clayton via lldb-dev 
>  wrote:
> 
> I have identified an issue in the expression parse. If you compile this code:
> 
> struct Foo { typedef unsigned node; };
> struct Bar { typedef int node; };
> int main(int argc, const char **argv) {
> Foo::node foo_carp = 22;
> Bar::node bar_node = 33;
> return 0;
> }
> 
> 
> Then then do:
> 
> (lldb) file a.out
> (lldb) b main
> (lldb) run
> (lldb) p (node *)_node
> 
> This should fail because there is no "node" type in the global namespace. But 
> we don't fail, we pick the first type whose base name matches "node" and 
> return it and start using it regardless of where the type is located decl 
> context wise... 
> 
> The issue happens when we call:
> 
> void ClangASTSource::FindExternalVisibleDecls(NameSearchContext , 
> lldb::ModuleSP module_sp, CompilerDeclContext _decl, unsigned int 
> current_id);
> 
> In this code we end up doing:
> 
> TypeList types;
> SymbolContext null_sc;
> const bool exact_match = false;
> llvm::DenseSet searched_symbol_files;
> if (module_sp && namespace_decl)
>   module_sp->FindTypesInNamespace(null_sc, name, _decl, 1, 
> types);
> else
>   m_target->GetImages().FindTypes(null_sc, name, exact_match, 1,
>   searched_symbol_files, types);
> 
> if (size_t num_types = types.GetSize()) {
>   for (size_t ti = 0; ti < num_types; ++ti) {
> 
> So if we don't have a valid namespace_decl, which will be the case when we 
> are searching at the translation unit level, we end up searching the target 
> for the first match for the type whose _basename_ matches "name" regardless 
> if of the module we are currently running the expression in, and regardless 
> of the translation unit decl context being a requirement. So we find any type 
> whose basename matches "name" from _any_ decl context. That is really bad!

> A few things wrong here:
> 1 - we seem to assume if we have a namespace (which is a parameter whose type 
> is CompilerDeclContext &), that it must go with the module_sp. Why? Because a 
> CompilerDeclContext has a "TypeSystem *" inside of it, and each module has a 
> their own "TypeSystem *" for a give language. We then call 
> module_sp->FindTypesInNamespace(...) with the CompilerDeclContext * to 
> namespace_decl. This eventually makes its way down to 
> SymbolFileDWARF::FindTypes(...) which will do the check:
> 
>   if (!DeclContextMatchesThisSymbolFile(parent_decl_ctx))
> return 0;
> 
> So if the CompilerDeclContext's type system doesn't match the type system of 
> the DWARF file that is makes it into, it will ignore the search entirely. 
> This means if we have a namespace from an expression like "p std::foo", and 
> we find the namespace "foo" in one module, we will only search a specific 
> module for "foo" within "std", even though another module could contain 
> "std::foo". So if the first module that has debug info for namespace "std" 
> doesn't contain "foo", we don't find the type?

The code above what you cited looks to see if the CompilerDeclContext is in the 
module, and if it isn't it searches all the modules for that 
CompilerDeclContext and then adds the appropriate module_sp and 
CompilerDeclContext  to the NameSpaceSearch context object.  As an aside, that 
search should probably break when it finds the namespace, since it's not going 
to find that CompilerDeclContext in another module...

It seems like we should then use the NamespaceSearchContext that we went to the 
trouble of making up, or at least look in the module we actually already found 
containing that Namespace.

This search is gated on !HasMerger, and HasMerger says:

  //--
  /// Returns true if there is a merger.  This only occurs if the target is
  /// using modern lookup.
  //--

This "modern lookup" is a target property that is off by default.  So we are 
doing this search.

I wonder if there's just something that didn't get wired up correctly when Sean 
was adding the modern lookup.

> 
> 2 - If we don't have a namespace, we want things only at the translation unit 
> level, yet we search all modules in the target in the target image list 
> order. It seems we should be checking by starting with the current module, 
> then expanding to any module except the current module if the type lookup 
> fails in the current module. Then we need to search for more than 1 type so 
> we can weed any results out that don't have a decl context of a translation 
> unit level or anonymous namespace from the current translation unit.

Yes, we certainly should start with the current module.  This does seem wrong 
to me.  Finding only one match also seems wrong.  This is providing a 
convenience - not having to type in the full name of a scoped object.  But we 
certainly should not do that if 

Re: [lldb-dev] Welcome Alexander!

2018-04-24 Thread Jim Ingham via lldb-dev


> On Apr 24, 2018, at 5:24 PM, Александр Поляков <polyakov@gmail.com> wrote:
> 
> Thanks Jim, it helped me a lot.
> 
> Can we do something like this: 
>1) having empty dummy target;

Yes, you don't need to do anything here.  The Debugger object always makes a 
dummy target.

>2) setting breakpoints to this dummy target until getting real file 
> through -file-exec-and-symbols;

Right, the Debugger has an API: GetDummyTarget that will get this for you, and 
then you call GetDummyTarget().BreakpointCreate...

>3) creating new target and moving all breakpoints from dummy target to it;

The move will happen automatically.

>4) clearing all breakpoints from the dummy target;

This seems reasonable.  Note however that you might inherit some breakpoints 
without seeing MI commands to do so - for instance from the user's .lldbinit 
file - and you probably want to propagate them every time you get a new 
-file-exec-and-symbols.  So you should keep track of the breakpoints that were 
in the dummy target when you got started, and only delete the ones you made 
after that.

>5) back to 1);
> 

Jim

> 
> 
> 
> 
> 
> Regards,
> Alexander Polyakov
> 
> 2018-04-25 2:56 GMT+03:00 Jim Ingham <jing...@apple.com>:
> In lldb, one Debugger can debug multiple different processes at a time.  This 
> is one of the ways lldb differs from gdb (*)...  In lldb, the Target is the 
> entity that represents a thing that you could debug.  A Target might or might 
> not actually be debugging anything.  If it is, the Target will have a 
> Process.  You generally make a target by giving it a file and maybe an 
> architecture.  Note the "file" command in lldb is just an alias for "target 
> create".  It makes a target out of a file.  Then when you want to debug that 
> file, you would say Target::Launch.
> 
> But a Target need not have a file.  For instance, if you do:
> 
> $ lldb --pid 12345
> 
> lldb has to make an empty target, attach it to the process with pid 12345, 
> and only then will it actually know what the file is.
> 
> Note also, in both lldb and gdb, you can set breakpoints in the 
> .lldbinit/.gdbinit file.  But both these init files get read in BEFORE any of 
> the command line arguments (including the one with the file command) get 
> processed.
> 
> So there has to be a way to hold onto breakpoints before any target is 
> created.  This was simple in gdb since it only supports one target, so you 
> can just stuff the breakpoints into the global list of breakpoint you were 
> going to use.  But you can't do that in lldb, because we could have many 
> targets. That's what the lldb "dummy target" is for.  It holds onto 
> breakpoints that are made in the absence of any target, and then each time a 
> new target gets made, it gets seeded with breakpoints from the dummy target.
> 
> Greg was worried that you could do:
> 
> -break-set
> -file-exec-and-symbols
> 
> and he wanted to make sure that works.  I think that's what you understood as 
> well.  
> 
> Since the gdb-mi interface models the way gdb works, it really only supports 
> having one target.  So I was suggesting that the lldb-mi module keep track of 
> this one privileged Target, and to make sure that -break-set sets breakpoints 
> in the dummy target if that privileged Target is still empty.
> 
> Jim
> 
> (*) one lldb process can also support multiple Debuggers, but that's another 
> story...
> 
> Jim
> 
> 
> 
> > On Apr 24, 2018, at 4:41 PM, Александр Поляков <polyakov@gmail.com> 
> > wrote:
> > 
> > I don't completely understand how it possible to add breakpoint before 
> > choosing a file(did you mean -file-exec-and-symbols cmd?).
> > And another important thing: could you explain me what is target in terms 
> > of lldb?
> > 
> > Thanks in advance.
> > 
> > Regards,
> > Alexander Polyakov
> > 
> > 2018-04-25 1:32 GMT+03:00 Ted Woodward <ted.woodw...@codeaurora.org>:
> > 
> > You'll still need HandleCommand for pass through commands. lldb commands 
> > send to lldb-mi are handled normally, so something like "file a.out" would 
> > set up a target using a.out. "-interpreter exec console " does the 
> > same thing. Other than that, I'm all for cleaning up lldb-mi. There were 
> > some funky decisions made when it was first developed.
> > 
> > Ted
> > 
> > --
> > Qualcomm Innovation Center, Inc.
> > The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a 
> > Linux Foundation Collaborative Project
> > 
> > > -Original Message-
> > >

Re: [lldb-dev] Welcome Alexander!

2018-04-24 Thread Jim Ingham via lldb-dev
In lldb, one Debugger can debug multiple different processes at a time.  This 
is one of the ways lldb differs from gdb (*)...  In lldb, the Target is the 
entity that represents a thing that you could debug.  A Target might or might 
not actually be debugging anything.  If it is, the Target will have a Process.  
You generally make a target by giving it a file and maybe an architecture.  
Note the "file" command in lldb is just an alias for "target create".  It makes 
a target out of a file.  Then when you want to debug that file, you would say 
Target::Launch.

But a Target need not have a file.  For instance, if you do:

$ lldb --pid 12345

lldb has to make an empty target, attach it to the process with pid 12345, and 
only then will it actually know what the file is.

Note also, in both lldb and gdb, you can set breakpoints in the 
.lldbinit/.gdbinit file.  But both these init files get read in BEFORE any of 
the command line arguments (including the one with the file command) get 
processed.

So there has to be a way to hold onto breakpoints before any target is created. 
 This was simple in gdb since it only supports one target, so you can just 
stuff the breakpoints into the global list of breakpoint you were going to use. 
 But you can't do that in lldb, because we could have many targets. That's what 
the lldb "dummy target" is for.  It holds onto breakpoints that are made in the 
absence of any target, and then each time a new target gets made, it gets 
seeded with breakpoints from the dummy target.

Greg was worried that you could do:

-break-set
-file-exec-and-symbols

and he wanted to make sure that works.  I think that's what you understood as 
well.  

Since the gdb-mi interface models the way gdb works, it really only supports 
having one target.  So I was suggesting that the lldb-mi module keep track of 
this one privileged Target, and to make sure that -break-set sets breakpoints 
in the dummy target if that privileged Target is still empty.

Jim

(*) one lldb process can also support multiple Debuggers, but that's another 
story...

Jim



> On Apr 24, 2018, at 4:41 PM, Александр Поляков <polyakov@gmail.com> wrote:
> 
> I don't completely understand how it possible to add breakpoint before 
> choosing a file(did you mean -file-exec-and-symbols cmd?).
> And another important thing: could you explain me what is target in terms of 
> lldb?
> 
> Thanks in advance.
> 
> Regards,
> Alexander Polyakov
> 
> 2018-04-25 1:32 GMT+03:00 Ted Woodward <ted.woodw...@codeaurora.org>:
> 
> You'll still need HandleCommand for pass through commands. lldb commands send 
> to lldb-mi are handled normally, so something like "file a.out" would set up 
> a target using a.out. "-interpreter exec console " does the same thing. 
> Other than that, I'm all for cleaning up lldb-mi. There were some funky 
> decisions made when it was first developed.
> 
> Ted
> 
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a 
> Linux Foundation Collaborative Project
> 
> > -Original Message-
> > From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Jim
> > Ingham via lldb-dev
> > Sent: Tuesday, April 24, 2018 5:19 PM
> > To: Greg Clayton <clayb...@gmail.com>
> > Cc: LLDB <lldb-dev@lists.llvm.org>
> > Subject: Re: [lldb-dev] Welcome Alexander!
> > 
> > 
> > 
> > > On Apr 24, 2018, at 3:08 PM, Greg Clayton <clayb...@gmail.com> wrote:
> > >
> > >
> > >
> > >> On Apr 24, 2018, at 3:00 PM, Jim Ingham <jing...@apple.com> wrote:
> > >>
> > >>
> > >>
> > >>> On Apr 24, 2018, at 2:46 PM, Greg Clayton <clayb...@gmail.com> wrote:
> > >>>
> > >>>
> > >>>
> > >>>> On Apr 24, 2018, at 2:35 PM, Jim Ingham <jing...@apple.com> wrote:
> > >>>>
> > >>>>
> > >>>>
> > >>>>> On Apr 24, 2018, at 9:44 AM, Greg Clayton <clayb...@gmail.com>
> > wrote:
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> On Apr 24, 2018, at 9:37 AM, Jim Ingham <jing...@apple.com>
> > wrote:
> > >>>>>>
> > >>>>>>>
> > >>>>>>> On Apr 24, 2018, at 7:43 AM, Greg Clayton via lldb-dev  > d...@lists.llvm.org> wrote:
> > >>>>>>>
> > >>>>>>> Welcome Alexander!
> > >>>>>>
> > >>>>>> Yes, welcome!
> > >>>>>&

Re: [lldb-dev] Welcome Alexander!

2018-04-24 Thread Jim Ingham via lldb-dev


> On Apr 24, 2018, at 3:08 PM, Greg Clayton  wrote:
> 
> 
> 
>> On Apr 24, 2018, at 3:00 PM, Jim Ingham  wrote:
>> 
>> 
>> 
>>> On Apr 24, 2018, at 2:46 PM, Greg Clayton  wrote:
>>> 
>>> 
>>> 
 On Apr 24, 2018, at 2:35 PM, Jim Ingham  wrote:
 
 
 
> On Apr 24, 2018, at 9:44 AM, Greg Clayton  wrote:
> 
> 
> 
>> On Apr 24, 2018, at 9:37 AM, Jim Ingham  wrote:
>> 
>>> 
>>> On Apr 24, 2018, at 7:43 AM, Greg Clayton via lldb-dev 
>>>  wrote:
>>> 
>>> Welcome Alexander!
>> 
>> Yes, welcome!
>> 
>>> 
>>> The title might be more stated as "Reimplement lldb-mi to correctly use 
>>> the SB API instead of using HandleCommand and regular expressions to 
>>> parse the command results" as it is already using the SB API, just not 
>>> using it anywhere close to correctly! 
>>> 
>>> I look forward to seeing the changes. 
>>> 
>>> A few things I ran into when playing with lldb-mi:
>>> - file-exec or exec-file packet might come _after_ some breakpoints are 
>>> set. We should make sure we create a lldb::SBTarget right away and set 
>>> the breakpoints on the empty target so that we don't miss these 
>>> breakpoints if this is still an issue. Then the when we receive the 
>>> exec-file packet, we set the file on the target
>> 
>> Breakpoints set before any target is created are set on the dummy 
>> target.  Breakpoints on the dummy target are copied into any new 
>> targets.  So this should not be necessary.  If that wasn't working we 
>> should figure that out, but it's not the responsibility of the MI to get 
>> this right.
> 
> We are trying not to use the command line and the command line is what 
> uses the dummy target automatically. When using the SB API you use a 
> lldb::SBTarget to set the breakpoint on so you need a target. What do you 
> suggest we use for the target? I would rather the lldb-mi code not rely 
> on the currently selected target or the dummy target.
 
 lldb-MI models gdb's behavior, which is one debugger with one target.  
 There is no command to add or switch to targets, etc.  So it doesn't seem 
 unreasonable for MI to keep track of its one actual target and if that is 
 empty, use SBDebugger::GetDummyTarget.  The other option is to make a 
 blank target up front and then add files to it when you see the -file-exec 
 command. But that seems more error-prone than using the mechanism lldb 
 provides for doing things before you have a target.  Again, if we were 
 modeling an API that could switch targets we might want to do something 
 more clever.  But that isn't how the GDB-MI was set up to work.
 
>>> 
>>> lldb-mi code may or may not have a target when it needs one. If it doesn't 
>>> have a target, use the SB API to get the dummy target and use that. 
>>> 
>>> Jim: is the dummy target good for anything other than adding breakpoints 
>>> to? What all gets copied from a the dummy target to the new target when one 
>>> gets created?
>> 
>> At present it only does breakpoints and stop hooks (see 
>> Target::PrimeFromDummyTarget.)  I didn't do watchpoints since those are 
>> seldom things you want to set generically, but you could probably add that.  
>> Was there anything else you were thinking of?
>> 
> 
> No, just mostly trying to let Alexander know what he should use the Dummy 
> target for and also for my own knowledge. If there are MI clients that do 
> other things, we will need to know if we need to create an empty real target 
> if they aren't breakpoints or stop hooks.

I can't think of any other things you add to a target like this.  The settings 
get inherited, and once you've started adding modules, I think you should 
create a new target to hold them.  But for anything interesting that's missing, 
as long as they are copiable it would be easy to add them.  Just call 
GetSelectedOrDummyTarget when you go to set them, and then put the copy in 
PrimeFromDummyTarget.

> 
> Greg
> 
>> Jim
>> 
>>> 
>>> Alexander, feel free to ask questions if you didn't understand any of the 
>>> above information. 
>>> 
>>> 
>>> 
 Jim
 
 
> 
>> 
>>> - remove all uses of HandleCommand and use SB APIs where possible
>>> - Add any SB API that might be missing and require us to use 
>>> HandleCommand
>>> 
>> 
>> The rest of these seem good guidelines.
>> 
>> Jim
>> 
>> 
>>> Good luck and let us know if you have any questions,
>>> 
>>> Greg Clayton
>>> 
>>> 
 On Apr 23, 2018, at 3:19 PM, Adrian Prantl via lldb-dev 
  wrote:
 
 Please join me in welcoming Alexander Polyakov, who will be working on 
 cleaning up 

Re: [lldb-dev] Welcome Alexander!

2018-04-24 Thread Jim Ingham via lldb-dev


> On Apr 24, 2018, at 2:46 PM, Greg Clayton  wrote:
> 
> 
> 
>> On Apr 24, 2018, at 2:35 PM, Jim Ingham  wrote:
>> 
>> 
>> 
>>> On Apr 24, 2018, at 9:44 AM, Greg Clayton  wrote:
>>> 
>>> 
>>> 
 On Apr 24, 2018, at 9:37 AM, Jim Ingham  wrote:
 
> 
> On Apr 24, 2018, at 7:43 AM, Greg Clayton via lldb-dev 
>  wrote:
> 
> Welcome Alexander!
 
 Yes, welcome!
 
> 
> The title might be more stated as "Reimplement lldb-mi to correctly use 
> the SB API instead of using HandleCommand and regular expressions to 
> parse the command results" as it is already using the SB API, just not 
> using it anywhere close to correctly! 
> 
> I look forward to seeing the changes. 
> 
> A few things I ran into when playing with lldb-mi:
> - file-exec or exec-file packet might come _after_ some breakpoints are 
> set. We should make sure we create a lldb::SBTarget right away and set 
> the breakpoints on the empty target so that we don't miss these 
> breakpoints if this is still an issue. Then the when we receive the 
> exec-file packet, we set the file on the target
 
 Breakpoints set before any target is created are set on the dummy target.  
 Breakpoints on the dummy target are copied into any new targets.  So this 
 should not be necessary.  If that wasn't working we should figure that 
 out, but it's not the responsibility of the MI to get this right.
>>> 
>>> We are trying not to use the command line and the command line is what uses 
>>> the dummy target automatically. When using the SB API you use a 
>>> lldb::SBTarget to set the breakpoint on so you need a target. What do you 
>>> suggest we use for the target? I would rather the lldb-mi code not rely on 
>>> the currently selected target or the dummy target.
>> 
>> lldb-MI models gdb's behavior, which is one debugger with one target.  There 
>> is no command to add or switch to targets, etc.  So it doesn't seem 
>> unreasonable for MI to keep track of its one actual target and if that is 
>> empty, use SBDebugger::GetDummyTarget.  The other option is to make a blank 
>> target up front and then add files to it when you see the -file-exec 
>> command. But that seems more error-prone than using the mechanism lldb 
>> provides for doing things before you have a target.  Again, if we were 
>> modeling an API that could switch targets we might want to do something more 
>> clever.  But that isn't how the GDB-MI was set up to work.
>> 
> 
> lldb-mi code may or may not have a target when it needs one. If it doesn't 
> have a target, use the SB API to get the dummy target and use that. 
> 
> Jim: is the dummy target good for anything other than adding breakpoints to? 
> What all gets copied from a the dummy target to the new target when one gets 
> created?

At present it only does breakpoints and stop hooks (see 
Target::PrimeFromDummyTarget.)  I didn't do watchpoints since those are seldom 
things you want to set generically, but you could probably add that.  Was there 
anything else you were thinking of?

Jim

> 
> Alexander, feel free to ask questions if you didn't understand any of the 
> above information. 
> 
> 
> 
>> Jim
>> 
>> 
>>> 
 
> - remove all uses of HandleCommand and use SB APIs where possible
> - Add any SB API that might be missing and require us to use HandleCommand
> 
 
 The rest of these seem good guidelines.
 
 Jim
 
 
> Good luck and let us know if you have any questions,
> 
> Greg Clayton
> 
> 
>> On Apr 23, 2018, at 3:19 PM, Adrian Prantl via lldb-dev 
>>  wrote:
>> 
>> Please join me in welcoming Alexander Polyakov, who will be working on 
>> cleaning up and completing LLDB's lldb-mi fronted as part of his Google 
>> Summer Of Code project:
>> 
>> Reimplement lldb-mi on top of the LLDB public SB API
>> https://summerofcode.withgoogle.com/projects/#5427847301169152
>> 
>> -- adrian
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Welcome Alexander!

2018-04-24 Thread Jim Ingham via lldb-dev


> On Apr 24, 2018, at 9:44 AM, Greg Clayton  wrote:
> 
> 
> 
>> On Apr 24, 2018, at 9:37 AM, Jim Ingham  wrote:
>> 
>>> 
>>> On Apr 24, 2018, at 7:43 AM, Greg Clayton via lldb-dev 
>>>  wrote:
>>> 
>>> Welcome Alexander!
>> 
>> Yes, welcome!
>> 
>>> 
>>> The title might be more stated as "Reimplement lldb-mi to correctly use the 
>>> SB API instead of using HandleCommand and regular expressions to parse the 
>>> command results" as it is already using the SB API, just not using it 
>>> anywhere close to correctly! 
>>> 
>>> I look forward to seeing the changes. 
>>> 
>>> A few things I ran into when playing with lldb-mi:
>>> - file-exec or exec-file packet might come _after_ some breakpoints are 
>>> set. We should make sure we create a lldb::SBTarget right away and set the 
>>> breakpoints on the empty target so that we don't miss these breakpoints if 
>>> this is still an issue. Then the when we receive the exec-file packet, we 
>>> set the file on the target
>> 
>> Breakpoints set before any target is created are set on the dummy target.  
>> Breakpoints on the dummy target are copied into any new targets.  So this 
>> should not be necessary.  If that wasn't working we should figure that out, 
>> but it's not the responsibility of the MI to get this right.
> 
> We are trying not to use the command line and the command line is what uses 
> the dummy target automatically. When using the SB API you use a 
> lldb::SBTarget to set the breakpoint on so you need a target. What do you 
> suggest we use for the target? I would rather the lldb-mi code not rely on 
> the currently selected target or the dummy target.

lldb-MI models gdb's behavior, which is one debugger with one target.  There is 
no command to add or switch to targets, etc.  So it doesn't seem unreasonable 
for MI to keep track of its one actual target and if that is empty, use 
SBDebugger::GetDummyTarget.  The other option is to make a blank target up 
front and then add files to it when you see the -file-exec command.  But that 
seems more error-prone than using the mechanism lldb provides for doing things 
before you have a target.  Again, if we were modeling an API that could switch 
targets we might want to do something more clever.  But that isn't how the 
GDB-MI was set up to work.

Jim


> 
>> 
>>> - remove all uses of HandleCommand and use SB APIs where possible
>>> - Add any SB API that might be missing and require us to use HandleCommand
>>> 
>> 
>> The rest of these seem good guidelines.
>> 
>> Jim
>> 
>> 
>>> Good luck and let us know if you have any questions,
>>> 
>>> Greg Clayton
>>> 
>>> 
 On Apr 23, 2018, at 3:19 PM, Adrian Prantl via lldb-dev 
  wrote:
 
 Please join me in welcoming Alexander Polyakov, who will be working on 
 cleaning up and completing LLDB's lldb-mi fronted as part of his Google 
 Summer Of Code project:
 
 Reimplement lldb-mi on top of the LLDB public SB API
 https://summerofcode.withgoogle.com/projects/#5427847301169152
 
 -- adrian
 ___
 lldb-dev mailing list
 lldb-dev@lists.llvm.org
 http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>> 
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Welcome Alexander!

2018-04-24 Thread Jim Ingham via lldb-dev

> On Apr 24, 2018, at 7:43 AM, Greg Clayton via lldb-dev 
>  wrote:
> 
> Welcome Alexander!

Yes, welcome!

> 
> The title might be more stated as "Reimplement lldb-mi to correctly use the 
> SB API instead of using HandleCommand and regular expressions to parse the 
> command results" as it is already using the SB API, just not using it 
> anywhere close to correctly! 
> 
> I look forward to seeing the changes. 
> 
> A few things I ran into when playing with lldb-mi:
> - file-exec or exec-file packet might come _after_ some breakpoints are set. 
> We should make sure we create a lldb::SBTarget right away and set the 
> breakpoints on the empty target so that we don't miss these breakpoints if 
> this is still an issue. Then the when we receive the exec-file packet, we set 
> the file on the target

Breakpoints set before any target is created are set on the dummy target.  
Breakpoints on the dummy target are copied into any new targets.  So this 
should not be necessary.  If that wasn't working we should figure that out, but 
it's not the responsibility of the MI to get this right.

> - remove all uses of HandleCommand and use SB APIs where possible
> - Add any SB API that might be missing and require us to use HandleCommand
> 

The rest of these seem good guidelines.

Jim
 

> Good luck and let us know if you have any questions,
> 
> Greg Clayton
> 
> 
>> On Apr 23, 2018, at 3:19 PM, Adrian Prantl via lldb-dev 
>>  wrote:
>> 
>> Please join me in welcoming Alexander Polyakov, who will be working on 
>> cleaning up and completing LLDB's lldb-mi fronted as part of his Google 
>> Summer Of Code project:
>> 
>> Reimplement lldb-mi on top of the LLDB public SB API
>> https://summerofcode.withgoogle.com/projects/#5427847301169152
>> 
>> -- adrian
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] FileSpec and normalization questions

2018-04-19 Thread Jim Ingham via lldb-dev
I can't see any reason why we benefit from having these differently spelled 
equivalent paths floating around.  You want to be careful to preserve the path 
syntax, since it would be weird to be cross debugging to a Windows machine and 
have to type Posix paths.  But other than that, removing all the extraneous 
cruft seems goodness to me.

Were you proposing just that you get the DWARF parser to do this, or were you 
proposing that that become a requirement for SymbolFile parsers, so that we can 
drop all the code that normalizes paths from debug information before 
comparisons?  We still have to normalize paths from users, but it would be nice 
not to have to do it if we know the source is debug info.

Jim


> On Apr 19, 2018, at 11:14 AM, Greg Clayton via lldb-dev 
>  wrote:
> 
> We currently have DWARF that has a DW_AT_comp_dir that is set to "./" 
> followed by any number of extra '/' characters. I would like to have this 
> path normalized as we parse the DWARF so we don't end up with line tables 
> with a variety of ".//+" prefixes on each source file.
> 
> While looking to solve this issue, I took a look at the functionality that is 
> in FileSpec right now. In:
> 
> void FileSpec::SetFile(llvm::StringRef pathname, bool resolve, PathSyntax 
> syntax);
> 
> 
> This function always calls a cheaper normalize function:
> 
> namespace {
>   void Normalize(llvm::SmallVectorImpl , FileSpec::PathSyntax 
> syntax);
> }
> 
> This function does nothing for posix paths, but switches backward slashes to 
> forward slashes. 
> 
> We have a FileSpec function named FileSpec::GetNormalizedPath() which will do 
> much more path normalization on a path by removing redundant "." and ".." and 
> "//". 
> 
> I can fix my DWARF issue in a few ways:
> 1 - fix FileSpec::SetFile() to still call ::Normalize() but have it do more 
> work and have it normalize out the and redundant or relative path info
> 2 - call FileSpec::GetNormalizedPath() on each comp dir before using it to 
> actually normalize it
> 
> The main question is: do we want paths floating around LLDB that aren't 
> normalized? It seems like having a mixture of theses path will lead to issues 
> in LLDB so I would vote for solution #1.
> 
> Also, looking at the tests for normalizing paths I found the following pairs 
> of pre-normalized and post-normalization paths for posix:
> 
>   {"//", "//"},
>   {"//net", "//net"},
> 
> Why wouldn't we reduce "//" to just "/" for posix? And why wouldn't we reduce 
> "//net" to "/net"?
> 
> Also I found:
> 
>   {"./foo", "foo"},
> 
> Do we prefer to not have "./foo" to stay as "./foo"? Seems like if users know 
> that their debug info has line table entries that are "./foo/foo.c" that they 
> might type this in:
> 
> (lldb) b ./foo/foo.c:12
> 
> But this will fail since it might not match the "foo/foo.c:12" that might 
> result from path normalization. We don't normalize right now so it doesn't 
> matter and things would match, but part of my fix is normalizing a path in 
> the DWARF that is currently ".foo/foo.c" down to either "./foo/foo.c" 
> or "foo/foo.c" so it will matter depending on what we decide here. 
> 
> Any input would be appreciated.
> 
> Greg Clayton
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] FileSpec and normalization questions

2018-04-19 Thread Jim Ingham via lldb-dev

> On Apr 19, 2018, at 11:21 AM, Zachary Turner <ztur...@google.com> wrote:
> 
> 
> 
> On Thu, Apr 19, 2018 at 11:19 AM Jim Ingham via lldb-dev 
> <lldb-dev@lists.llvm.org> wrote:
> The last time I looked at the llvm functions they only support the path 
> syntax of the llvm host, which won't do for lldb.  But maybe they have gotten 
> more general recently?
> 
> 
> Yes in fact I was the one who updated them to make them more general.  You 
> can now specify an enumeration parameter which will cause the algorithm to 
> treat paths as either windows, posix, or whatever the host is. 

If I were guessing, I would have guessed that!

Since you probably know how the llvm functions work better than most, were 
there technical reasons why, having done the work on the llvm side, you didn't 
adopt them in lldb?  Or was it just a matter of time?

Jim


___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Advice on architectures with multiple address spaces

2018-04-19 Thread Jim Ingham via lldb-dev


> On Apr 19, 2018, at 10:54 AM, Greg Clayton  wrote:
> 
> 
> 
>> On Apr 19, 2018, at 10:35 AM, Jim Ingham  wrote:
>> 
>> 
>> 
>>> On Apr 19, 2018, at 9:44 AM, Greg Clayton via lldb-dev 
>>>  wrote:
>>> 
>>> 
>>> 
 On Apr 19, 2018, at 6:51 AM, Zdenek Prikryl via lldb-dev 
  wrote:
 
 Hi lldb developers,
 
 I've been researching using lldb + gdbserver stub that is based on Harvard 
 architecture with multiple address spaces (one program, multiple data). 
 The commonly adopted approach is that everything is mapped to a single 
 "virtual" address space. The stub reads/writes from/to the right memory 
 based on the "virtual" addresses. But I'd like to use real addresses with 
 address space id instead. So, I've started looking at what has to be 
 changed.
 
 I've enhanced read/write commands (e.g. memory read --as  ...) and RSP 
 protocol (new packet) so that the stub can read/write properly. That 
 wasn't that complicated.
>>> 
>>> It might be nice to add a new RSP protocol packet that asks for the address 
>>> space names/values:
>>> 
>>> qGetAddressSpaces
>>> 
>>> which would return something like:
>>> 
>>> 1:text;2:data1,3:data2
>>> 
>>> or it would return not supported. If we get a valid return value from 
>>> qGetAddressSpaces, then it enables the use of the new packet you added 
>>> above. Else it defaults to using the old memory read functions.
>>> 
>>> 
 
 Now I've hit an issue with expressions (LLVMUserExpression.cpp) and local 
 variables (DWARFExpressions.cpp). There is a lot of memory read/write 
 functions that take just an address argument. Is the only way to go to 
 patch all these calls? Has anybody solved it differently?
>>> 
>>> My quick take is that any APIs that take just a lldb::addr_t would need to 
>>> take something like:
>>> 
>>> struct SpaceAddress {
>>> static constexpr uint32_t kNoSpace = 0;
>>> lldb::addr_t addr;
>>> uint32_t space;
>>> };
>>> 
>> 
>> I'm curious why you are suggesting another kind of address, rather than 
>> adding this functionality to Address?  When you actually go to resolve an 
>> Address in a target with a process you should have everything you need to 
>> know to give it the proper space.  Then fixing the expression evaluator (and 
>> anything else that needs fixing) would be a matter of consistently using 
>> Address rather than lldb::addr_t.  That seems general goodness, since 
>> converting to an lldb::addr_t loses information.
> 
> If we accept lldb_private::Address in all APIs that take a lldb::addr_t 
> currently, then we need to always be able to get to the target in case we 
> need to add code to resolve the address everywhere. I am thinking of 
> SpaceAddress as an augmented lldb::addr_t instead of a section + offset style 
> address. Also, there will be addresses in the code and data that do not exist 
> in actual sections. Not saying that you couldn't use lldb_private::Address. I 
> am open to suggestions though. So your though it remove all API that take 
> lldb::addr_t and use lldb_private::Address everywhere all the time?

It has always bugged me that we have these two ways of specifying addresses.  
Are there many/any places that have to resolve an Address to a real address in 
a process that don't have a Target readily available?  That would surprise me.  
I would much rather centralize on one way than adding a third.

Jim


>> 
>> Jim
>> 
>> 
>>> We would need a default value for "space" (feel free to rename) that 
>>> indicates the default address space as most of our architectures would not 
>>> need this support. If we added a constructor like:
>>> 
>>> SpaceAddress(lldb::addr_t a) : addr(a), space(kNoSpace) {}
>>> 
>>> Then all usages of the APIs that used to take just a "lldb::addr_t" would 
>>> implicitly call this constructor and continue to act as needed. Then we 
>>> would need to allow lldb_private::Address objects to resolve to a 
>>> SpaceAddress:
>>> 
>>> SpaceAddress lldb_private::Address::GetSpaceAddress(Target *target) const;
>>> 
>>> Since each lldb_private::Address has a section and each section knows its 
>>> address space. Then the tricky part is finding all locations in the 
>>> expression parser and converting those to track and use SpaceAddress. We 
>>> would probably need to modify the allocate memory packets in the RSP 
>>> protocol to be able to allocate memory in any address space as well.
>>> 
>>> I didn't spend much time think about correct names above, so feel free to 
>>> suggest alternate naming. 
>>> 
>>> Best advice:
>>> - make things "just work" to keep changes to a minimum and allowing 
>>> lldb::addr_t to implicitly convert to a SpaceAddress easily
>>> - when modifying RSP, make sure to check for existence of new feature 
>>> before enabling it
>>> - query for address space names so when we dump SpaceAddress we can show 
>>> 

Re: [lldb-dev] FileSpec and normalization questions

2018-04-19 Thread Jim Ingham via lldb-dev
The last time I looked at the llvm functions they only support the path syntax 
of the llvm host, which won't do for lldb.  But maybe they have gotten more 
general recently?

Jim


> On Apr 19, 2018, at 11:16 AM, Davide Italiano via lldb-dev 
>  wrote:
> 
> IIRC We have path normalization functions in llvm, have you looked at them?
> 
> Thanks,
> 
> --
> Davide
> 
> On Thu, Apr 19, 2018 at 11:14 AM, Greg Clayton via lldb-dev
>  wrote:
>> We currently have DWARF that has a DW_AT_comp_dir that is set to "./"
>> followed by any number of extra '/' characters. I would like to have this
>> path normalized as we parse the DWARF so we don't end up with line tables
>> with a variety of ".//+" prefixes on each source file.
>> 
>> While looking to solve this issue, I took a look at the functionality that
>> is in FileSpec right now. In:
>> 
>> void FileSpec::SetFile(llvm::StringRef pathname, bool resolve, PathSyntax
>> syntax);
>> 
>> 
>> This function always calls a cheaper normalize function:
>> 
>> namespace {
>>  void Normalize(llvm::SmallVectorImpl , FileSpec::PathSyntax
>> syntax);
>> }
>> 
>> This function does nothing for posix paths, but switches backward slashes to
>> forward slashes.
>> 
>> We have a FileSpec function named FileSpec::GetNormalizedPath() which will
>> do much more path normalization on a path by removing redundant "." and ".."
>> and "//".
>> 
>> I can fix my DWARF issue in a few ways:
>> 1 - fix FileSpec::SetFile() to still call ::Normalize() but have it do more
>> work and have it normalize out the and redundant or relative path info
>> 2 - call FileSpec::GetNormalizedPath() on each comp dir before using it to
>> actually normalize it
>> 
>> The main question is: do we want paths floating around LLDB that aren't
>> normalized? It seems like having a mixture of theses path will lead to
>> issues in LLDB so I would vote for solution #1.
>> 
>> Also, looking at the tests for normalizing paths I found the following pairs
>> of pre-normalized and post-normalization paths for posix:
>> 
>>  {"//", "//"},
>>  {"//net", "//net"},
>> 
>> Why wouldn't we reduce "//" to just "/" for posix? And why wouldn't we
>> reduce "//net" to "/net"?
>> 
>> Also I found:
>> 
>>  {"./foo", "foo"},
>> 
>> Do we prefer to not have "./foo" to stay as "./foo"? Seems like if users
>> know that their debug info has line table entries that are "./foo/foo.c"
>> that they might type this in:
>> 
>> (lldb) b ./foo/foo.c:12
>> 
>> But this will fail since it might not match the "foo/foo.c:12" that might
>> result from path normalization. We don't normalize right now so it doesn't
>> matter and things would match, but part of my fix is normalizing a path in
>> the DWARF that is currently ".foo/foo.c" down to either
>> "./foo/foo.c" or "foo/foo.c" so it will matter depending on what we decide
>> here.
>> 
>> Any input would be appreciated.
>> 
>> Greg Clayton
>> 
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Advice on architectures with multiple address spaces

2018-04-19 Thread Jim Ingham via lldb-dev


> On Apr 19, 2018, at 9:44 AM, Greg Clayton via lldb-dev 
>  wrote:
> 
> 
> 
>> On Apr 19, 2018, at 6:51 AM, Zdenek Prikryl via lldb-dev 
>>  wrote:
>> 
>> Hi lldb developers,
>> 
>> I've been researching using lldb + gdbserver stub that is based on Harvard 
>> architecture with multiple address spaces (one program, multiple data). The 
>> commonly adopted approach is that everything is mapped to a single "virtual" 
>> address space. The stub reads/writes from/to the right memory based on the 
>> "virtual" addresses. But I'd like to use real addresses with address space 
>> id instead. So, I've started looking at what has to be changed.
>> 
>> I've enhanced read/write commands (e.g. memory read --as  ...) and RSP 
>> protocol (new packet) so that the stub can read/write properly. That wasn't 
>> that complicated.
> 
> It might be nice to add a new RSP protocol packet that asks for the address 
> space names/values:
> 
> qGetAddressSpaces
> 
> which would return something like:
> 
> 1:text;2:data1,3:data2
> 
> or it would return not supported. If we get a valid return value from 
> qGetAddressSpaces, then it enables the use of the new packet you added above. 
> Else it defaults to using the old memory read functions.
> 
> 
>> 
>> Now I've hit an issue with expressions (LLVMUserExpression.cpp) and local 
>> variables (DWARFExpressions.cpp). There is a lot of memory read/write 
>> functions that take just an address argument. Is the only way to go to patch 
>> all these calls? Has anybody solved it differently?
> 
> My quick take is that any APIs that take just a lldb::addr_t would need to 
> take something like:
> 
> struct SpaceAddress {
>  static constexpr uint32_t kNoSpace = 0;
>  lldb::addr_t addr;
>  uint32_t space;
> };
> 

I'm curious why you are suggesting another kind of address, rather than adding 
this functionality to Address?  When you actually go to resolve an Address in a 
target with a process you should have everything you need to know to give it 
the proper space.  Then fixing the expression evaluator (and anything else that 
needs fixing) would be a matter of consistently using Address rather than 
lldb::addr_t.  That seems general goodness, since converting to an lldb::addr_t 
loses information.

Jim


> We would need a default value for "space" (feel free to rename) that 
> indicates the default address space as most of our architectures would not 
> need this support. If we added a constructor like:
> 
> SpaceAddress(lldb::addr_t a) : addr(a), space(kNoSpace) {}
> 
> Then all usages of the APIs that used to take just a "lldb::addr_t" would 
> implicitly call this constructor and continue to act as needed. Then we would 
> need to allow lldb_private::Address objects to resolve to a SpaceAddress:
> 
>  SpaceAddress lldb_private::Address::GetSpaceAddress(Target *target) const;
> 
> Since each lldb_private::Address has a section and each section knows its 
> address space. Then the tricky part is finding all locations in the 
> expression parser and converting those to track and use SpaceAddress. We 
> would probably need to modify the allocate memory packets in the RSP protocol 
> to be able to allocate memory in any address space as well.
> 
> I didn't spend much time think about correct names above, so feel free to 
> suggest alternate naming. 
> 
> Best advice:
> - make things "just work" to keep changes to a minimum and allowing 
> lldb::addr_t to implicitly convert to a SpaceAddress easily
> - when modifying RSP, make sure to check for existence of new feature before 
> enabling it
> - query for address space names so when we dump SpaceAddress we can show 
> something that means something to the user. This means we would need to query 
> the address space names from the current lldb_private::Process for display.
> 
> Submitting might go easier if we break it down into chunks:
> 1 - add SpaceAddress and modify all needed APIs to use it
> 2 - add ProcessGDBRemote changes that enable this support
> 
> It will be great to support this as a first class citizen within LLDB. You 
> might ask the Hexagon folks if they have done anything in case they already 
> support this is some shape or form.
> 
> Greg Clayton
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] lldb stops on every call to dlopen

2018-04-17 Thread Jim Ingham via lldb-dev
It is interesting that the stop reason on the thread that stopped is "trace".  
That's what you would expect returning from the single-step to step over the 
breakpoint.  But it looks like we got a signal while single-stepping, but the 
stop reason was misreported by somebody.

Jim


> On Apr 17, 2018, at 6:00 AM, Pavel Labath via lldb-dev 
>  wrote:
> 
> [+lldb-dev]
> 
> Hello Steve,
> 
> thanks for the report.
> 
> The fact that you see the rendezvous breakpoint being hit many times is not
> surprising. We get those every time the library is loaded (we need that to
> load relevant debug info and set potential breakpoints). However, they
> should generally not be surfaced to the user (unless you have the
> stop-on-sharedlibrary-events setting set, which you don't).
> 
> The part that is suspicious to me is that __restore_rt shows up on the top
> of the backtrace. This is a trampoline used to return from signal handlers,
> and it would seem to indicate that you got some sort of a signal while
> loading the libraries. I don't know why this would happen, but it could be
> that this is confusing lldb's auto-resume logic.
> 
> The interesting part to see here is what lldb thinks are the stop reasons
> for individual threads in the process (is the process multi-threaded?) for
> the last couple of stops. The "lldb step" and "gdb-remote packets" log
> categories are the most interesting to observe here. If you are able to
> send me the log traces, I can help you interpret them.
> 
> regards,
> pavel
> 
> 
> 
> 
> 
> On Tue, 17 Apr 2018 at 02:27, Steve Ravet via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
> 
>> Hello lldb developers, I am running into a problem with lldb on Linux.  I
> am currently running llvm 6.0.0.
> 
>> I have an executable that dynamically loads a large number of shared
> libraries at runtime.  These are explicitly loaded via dlopen (they are
> specified in a configuration file), and after loading a few (typically a
> dozen or so, but the number varies) lldb will halt during dlopen.  If I
> continue, it will load a few more then halt again, which makes debugging
> from startup impractical since there are so many libraries to be loaded
> (more than a hundred of them).
> 
>> When I build and debug this same C++ on macOS, the debugger works fine.
> I have verified that target.process.stop-on-sharedlibrary-events is false.
> I turned on dyld logging and I see lots of log messages about
> RendezvousBreakpoint being hit, but I don’t see anything that sheds light
> on why some libraries load without stopping but others don’t.
> 
>> I have tried to recreate this in a trivial program that calls dlopen in a
> loop, but haven’t been able to reproduce.
> 
>> Can your offer any suggestions for further debugging this?  More
> supporting evidence follows.
> 
>> Here is the message when the debugger stops:
> 
>> Process 120004 stopped
>> * thread #1, name = ‘', stop reason = trace
>> frame #0: 0x2cfca6a0 libc.so.6`__restore_rt
>> libc.so.6`__restore_rt:
>> ->  0x2cfca6a0 <+0>: movq   $0xf, %rax
>> 0x2cfca6a7 <+7>: syscall
>> 0x2cfca6a9 <+9>: nopl   (%rax)
> 
>> libc.so.6`__libc_sigaction:
>> 0x2cfca6b0 <+0>: subq   $0xd0, %rsp
> 
>> I do not have the stop on shared library events setting enabled:
> 
>> (lldb) settings show target.process.stop-on-sharedlibrary-events
>> target.process.stop-on-sharedlibrary-events (boolean) = false
> 
> 
> 
>> The backtrace goes back to dlopen:
> 
>> (lldb) bt
>> * thread #1, name = ‘x', stop reason = trace
>>   * frame #0: 0x2cfca6a0 libc.so.6`__restore_rt
>> frame #1: 0x2aab9eb0 ld-linux-x86-64.so.2
>> frame #2: 0x2aabdc53 ld-linux-x86-64.so.2`dl_open_worker + 499
>> frame #3: 0x2aab9286 ld-linux-x86-64.so.2`_dl_catch_error +
> 102
>> frame #4: 0x2aabd63a ld-linux-x86-64.so.2`_dl_open + 186
>> frame #5: 0x2c39df66 libdl.so.2`dlopen_doit + 102
>> frame #6: 0x2aab9286 ld-linux-x86-64.so.2`_dl_catch_error +
> 102
>> frame #7: 0x2c39e29c libdl.so.2`_dlerror_run + 124
>> frame #8: 0x2c39dee1 libdl.so.2`__dlopen_check + 49
> 
>> the dyld debug log has a lot of this:
>> 209 intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid
> 153501 stop_when_images_change=false
>> 210 intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit
> called for pid 153501
>> 211 intern-state DYLDRendezvous::Resolve address size: 8, padding 4
>> 212 intern-state DYLDRendezvous::Resolve cursor = 0x2accc160
>> 213 intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid
> 153501 stop_when_images_change=false
>> 214 intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit
> called for pid 153501
>> 215 intern-state DYLDRendezvous::Resolve address size: 8, padding 4
>> 216 intern-state DYLDRendezvous::Resolve cursor = 0x2accc160
> 
> 
> 
>> thanks,
>> --steve
> 

Re: [lldb-dev] GDB RSPs non-stop mode capability in v5.0

2018-04-03 Thread Jim Ingham via lldb-dev


> On Apr 3, 2018, at 1:28 AM, Ramana  wrote:
> 
> 
> 
> On Thu, Mar 29, 2018 at 11:17 PM, Jim Ingham  > wrote:
> The breakpoints aren't a structural problem.  If you can figure out a 
> non-code modifying way to handle breakpoints, that would be a very surgical 
> change.  And as Fred points out, out of place execution in the target would 
> be really handy for other things, like offloading breakpoint conditions into 
> the target, and only stopping if the condition is true.  So this is a well 
> motivated project.
> 
> And our model for handling both expression evaluation and execution control 
> are already thread-centric.  It would be pretty straight-forward to treat 
> "still running" threads the same way as threads with no interesting stop 
> reasons, for instance.
> 
> I think the real difficulty will come at the higher layers.  First off, we 
> gate a lot of Command & SB API operations on "is the process running" and 
> that will have to get much more fine-grained.  Figuring out a good model for 
> this will be important.
> 
> Then you're going to have to figure out what exactly to do when somebody is 
> in the middle of say running a long expression on thread A when thread B 
> stops.  What's a useful way to present this information?  If lldb is sharing 
> the terminal with the process, you can't just dump output in the middle of 
> command output, but you don't want to delay too long...
> 
> Also, the IOHandlers are currently a stack, but that model won't work when 
> the process IOHandler is going to have to be live (at least the output part 
> of it) while the CommandInterpreter IOHandler is also live.  That's going to 
> take reworking.
> 
> On the event and operations side, I think the fact that we have the 
> separation between the private and public states will make this a lot easier. 
>  We can use the event transition from private to public state to serialize 
> the activity that's going on under the covers so that it appears coherent to 
> the user.  The fact that lldb goes through separate channels for process I/O 
> and command I/O and we very seldom just dump stuff to stdout will also make 
> solving the problem of competing demands for the user's attention more 
> possible.
> 
> Thanks Jim for the elaborate view on the non-stop mode support.
> 
> BTW my understanding on public vs private states is that the public state is 
> as known by the user and all the process state changes will be first tracked 
> with private state which then will be made public, i.e. public state will be 
> updated, should the user need to know about that process state change. Is 
> there anything else I am missing on public vs private states?

That’s exactly it.  Another detail worth noting is that all the user-supplied 
work - particularly for breakpoints, the commands, conditions etc - happens 
when the event is pulled off the public queue.  So we can postpone all this 
work for threads the notification of whose stop we’re suspending by holding 
back some events from the private->public transition, which will I think be 
helpful in making the non-stop mode behave nicely.

Jim

> 
> 
> And I think we can't do any of this till we have a robust "ProcessMock" 
> plugin that we can use to emulate end-to-end through the debugger all the 
> corner cases that non-stop debugging will bring up.  Otherwise there will be 
> no way to reliably test any of this stuff, and it won't ever be stable.
> 
> I don't think any of this will be impossible, but it's going to be a lot of 
> work.
> 
> Jim
> 
> 
> > On Mar 29, 2018, at 9:27 AM, Greg Clayton via lldb-dev 
> > > wrote:
> >
> >
> >
> >> On Mar 29, 2018, at 9:10 AM, Frédéric Riss  >> > wrote:
> >>
> >>
> >>
> >>> On Mar 29, 2018, at 7:32 AM, Greg Clayton via lldb-dev 
> >>> > wrote:
> >>>
> >>>
> >>>
>  On Mar 29, 2018, at 2:08 AM, Ramana via lldb-dev 
>  > wrote:
> 
>  Hi,
> 
>  It appears that the lldb-server, as of v5.0, did not implement the GDB 
>  RSPs non-stop mode 
>  (https://sourceware.org/gdb/onlinedocs/gdb/Remote-Non_002dStop.html#Remote-Non_002dStop
>   
>  ).
>   Am I wrong?
> 
>  If the support is actually not there, what needs to be changed to enable 
>  the same in lldb-server?
> >>>
> >>> As Pavel said, adding support into lldb-server will be easy. Adding 
> >>> support to LLDB will be harder. One downside of enabling this mode will 
> >>> be a performance loss in the GDB remote packet transfer. Why? IIRC this 
> >>> mode requires a read thread where one thread is always reading packets 
> >>> and putting them into a packet 

Re: [lldb-dev] GDB RSPs non-stop mode capability in v5.0

2018-03-29 Thread Jim Ingham via lldb-dev


> On Mar 29, 2018, at 10:40 AM, Greg Clayton via lldb-dev 
>  wrote:
> 
> 
> 
>> On Mar 29, 2018, at 10:36 AM, Frédéric Riss  wrote:
>> 
>> 
>> 
>>> On Mar 29, 2018, at 9:27 AM, Greg Clayton  wrote:
>>> 
>>> 
>>> 
 On Mar 29, 2018, at 9:10 AM, Frédéric Riss  wrote:
 
 
 
> On Mar 29, 2018, at 7:32 AM, Greg Clayton via lldb-dev 
>  wrote:
> 
> 
> 
>> On Mar 29, 2018, at 2:08 AM, Ramana via lldb-dev 
>>  wrote:
>> 
>> Hi,
>> 
>> It appears that the lldb-server, as of v5.0, did not implement the GDB 
>> RSPs non-stop mode 
>> (https://sourceware.org/gdb/onlinedocs/gdb/Remote-Non_002dStop.html#Remote-Non_002dStop).
>>  Am I wrong?
>> 
>> If the support is actually not there, what needs to be changed to enable 
>> the same in lldb-server?
> 
> As Pavel said, adding support into lldb-server will be easy. Adding 
> support to LLDB will be harder. One downside of enabling this mode will 
> be a performance loss in the GDB remote packet transfer. Why? IIRC this 
> mode requires a read thread where one thread is always reading packets 
> and putting them into a packet buffer. Threads that want to send a packet 
> an get a reply must not send the packet then use a condition variable + 
> mutex to wait for the response. This threading overhead really slows down 
> the packet transfers. Currently we have a mutex on the GDB remote 
> communication where each thread that needs to send a packet will take the 
> mutex and then send the packet and wait for the response on the same 
> thread. I know the performance differences are large on MacOS, not sure 
> how they are on other systems. If you do end up enabling this, please run 
> the "process plugin packet speed-test" command which is available only 
> when debugging with ProcessGDBRemote. It will send an receive various 
> packets of various sizes and report speed statistics back to you.
>> 
>> Also, in lldb at least I see some code relevant to non-stop mode, but is 
>> non-stop mode fully implemented in lldb or there is only partial support?
> 
> Everything in LLDB right now assumes a process centric debugging model 
> where when one thread stops all threads are stopped. There will be quite 
> a large amount of changes needed for a thread centric model. The biggest 
> issue I know about is breakpoints. Any time you need to step over a 
> breakpoint, you must stop all threads, disable the breakpoint, single 
> step the thread and re-enable the breakpoint, then start all threads 
> again. So even the thread centric model would need to start and stop all 
> threads many times. 
 
 If we work on this, that’s not the way we should approach breakpoints in 
 non-stop mode (and it’s not how GDB does it). I’m not sure why Ramana is 
 interested in it, but I think one of the main motivations to add it to GDB 
 was systems where stopping all some threads for even a small amount of 
 time would just break things. You want a way to step over breakpoints 
 without disrupting the other threads.
 
 Instead of removing the breakpoint, you can just teach the debugger to 
 execute the code that has been patched in a different context. You can 
 either move the code someplace else and execute it there or emulate it. 
 Sometimes you’ll need to patch it if it is PC-relative. IIRC, GDB calls 
 this displaced stepping. It’s relatively simple and works great.
>>> 
>>> This indeed is one of the changes we would need to do for non-stop mode. We 
>>> have the EmulateInstruction class in LLDB that is designed just for this 
>>> kind of thing. You can give the emulator function a read/write memory and 
>>> read/write register callbacks and a baton and it can execute the 
>>> instruction and read/write memory and regisrters as needed through the 
>>> context. It would be very easy to have the read register callback know to 
>>> take the PC of the original instruction and return it if the PC is 
>>> requested.
>>> 
>>> We always got push back in the past about adding full instruction emulation 
>>> support as Chris Lattner wanted it to exist in LLVM in the tablegen tables, 
>>> but no one ever got around to doing that part. So we added prologue 
>>> instruction parsing and any instructions that can modify the PC (for single 
>>> stepping) to the supported emulated instructions. 
>>> 
>>> So yes, emulating instructions without removing them from the code is one 
>>> of the things required for this feature. Not impossible, just very time 
>>> consuming to be able to emulate every instruction out of place. I would 
>>> _love_ to see that go in and would be happy to review patches for anyone 
>>> wanting to take this on. Though the 

Re: [lldb-dev] GDB RSPs non-stop mode capability in v5.0

2018-03-29 Thread Jim Ingham via lldb-dev
The breakpoints aren't a structural problem.  If you can figure out a non-code 
modifying way to handle breakpoints, that would be a very surgical change.  And 
as Fred points out, out of place execution in the target would be really handy 
for other things, like offloading breakpoint conditions into the target, and 
only stopping if the condition is true.  So this is a well motivated project.

And our model for handling both expression evaluation and execution control are 
already thread-centric.  It would be pretty straight-forward to treat "still 
running" threads the same way as threads with no interesting stop reasons, for 
instance.

I think the real difficulty will come at the higher layers.  First off, we gate 
a lot of Command & SB API operations on "is the process running" and that will 
have to get much more fine-grained.  Figuring out a good model for this will be 
important.

Then you're going to have to figure out what exactly to do when somebody is in 
the middle of say running a long expression on thread A when thread B stops.  
What's a useful way to present this information?  If lldb is sharing the 
terminal with the process, you can't just dump output in the middle of command 
output, but you don't want to delay too long...  

Also, the IOHandlers are currently a stack, but that model won't work when the 
process IOHandler is going to have to be live (at least the output part of it) 
while the CommandInterpreter IOHandler is also live.  That's going to take 
reworking.

On the event and operations side, I think the fact that we have the separation 
between the private and public states will make this a lot easier.  We can use 
the event transition from private to public state to serialize the activity 
that's going on under the covers so that it appears coherent to the user.  The 
fact that lldb goes through separate channels for process I/O and command I/O 
and we very seldom just dump stuff to stdout will also make solving the problem 
of competing demands for the user's attention more possible.

And I think we can't do any of this till we have a robust "ProcessMock" plugin 
that we can use to emulate end-to-end through the debugger all the corner cases 
that non-stop debugging will bring up.  Otherwise there will be no way to 
reliably test any of this stuff, and it won't ever be stable.

I don't think any of this will be impossible, but it's going to be a lot of 
work.

Jim
 

> On Mar 29, 2018, at 9:27 AM, Greg Clayton via lldb-dev 
>  wrote:
> 
> 
> 
>> On Mar 29, 2018, at 9:10 AM, Frédéric Riss  wrote:
>> 
>> 
>> 
>>> On Mar 29, 2018, at 7:32 AM, Greg Clayton via lldb-dev 
>>>  wrote:
>>> 
>>> 
>>> 
 On Mar 29, 2018, at 2:08 AM, Ramana via lldb-dev  
 wrote:
 
 Hi,
 
 It appears that the lldb-server, as of v5.0, did not implement the GDB 
 RSPs non-stop mode 
 (https://sourceware.org/gdb/onlinedocs/gdb/Remote-Non_002dStop.html#Remote-Non_002dStop).
  Am I wrong?
 
 If the support is actually not there, what needs to be changed to enable 
 the same in lldb-server?
>>> 
>>> As Pavel said, adding support into lldb-server will be easy. Adding support 
>>> to LLDB will be harder. One downside of enabling this mode will be a 
>>> performance loss in the GDB remote packet transfer. Why? IIRC this mode 
>>> requires a read thread where one thread is always reading packets and 
>>> putting them into a packet buffer. Threads that want to send a packet an 
>>> get a reply must not send the packet then use a condition variable + mutex 
>>> to wait for the response. This threading overhead really slows down the 
>>> packet transfers. Currently we have a mutex on the GDB remote communication 
>>> where each thread that needs to send a packet will take the mutex and then 
>>> send the packet and wait for the response on the same thread. I know the 
>>> performance differences are large on MacOS, not sure how they are on other 
>>> systems. If you do end up enabling this, please run the "process plugin 
>>> packet speed-test" command which is available only when debugging with 
>>> ProcessGDBRemote. It will send an receive various packets of various sizes 
>>> and report speed statistics back to you.
 
 Also, in lldb at least I see some code relevant to non-stop mode, but is 
 non-stop mode fully implemented in lldb or there is only partial support?
>>> 
>>> Everything in LLDB right now assumes a process centric debugging model 
>>> where when one thread stops all threads are stopped. There will be quite a 
>>> large amount of changes needed for a thread centric model. The biggest 
>>> issue I know about is breakpoints. Any time you need to step over a 
>>> breakpoint, you must stop all threads, disable the breakpoint, single step 
>>> the thread and re-enable the breakpoint, then start all threads again. So 
>>> even the thread centric model 

Re: [lldb-dev] increase timeout for tests?

2018-03-13 Thread Jim Ingham via lldb-dev
It is unfortunate that we have to set really long test timeouts because we are 
timing the total Test class run, not the individual tests.  It is really 
convenient to be able to group similar tests in one class, so they can reuse 
setup and common check methods etc.  But if we're going to have to push the 
timeouts to something really long because these tests get charged incorrectly, 
which makes this strategy artificially less desirable.

When we spawn a dotest.py to run each test file, the runner that is doing the 
timeout hasn't ingested the test class, so it can't do something reasonable 
like count the number of tests and multiply that into the timeout to get the 
full test timeout.  I tried to hack around this but I wasn't successful at 
getting hold of the test configuration in the runner so you could figure out 
the number of tests there.  If somebody more familiar with the test harness 
than I am can see a way to do that, that seems like a much better way to go.

But if we can't do that, then we can increase the overall timeout.  Though we 
might want to override that with LLDB_TEST_TIMEOUT and set it to something 
lower on the bots.

Jim

> On Mar 13, 2018, at 5:47 PM, Vedant Kumar  wrote:
> 
> As a first step, I think there's consensus on increasing the test timeout to 
> ~3x the length of the slowest test we know of. That test appears to be 
> TestDataFormatterObjC, which takes 388 seconds on Davide's machine. So I 
> propose 20 minutes as the timeout value.
> 
> Separately, regarding x-failed pexpect()-backed tests, I propose deleting 
> them if they've been x-failed for over a year. That seems like a long enough 
> time to wait for someone to step up and fix them given that they're a real 
> testing/maintenance burden. For any group of to-be-deleted tests, like the 22 
> lldb-mi tests x-failed in all configurations, I'd file a PR about potentially 
> bringing the tests back. Thoughts?
> 
> thanks,
> vedant
> 
>> On Mar 13, 2018, at 11:52 AM, Davide Italiano  wrote:
>> 
>> On Tue, Mar 13, 2018 at 11:26 AM, Jim Ingham  wrote:
>>> It sounds like we timing out based on the whole test class, not the 
>>> individual tests?  If you're worried about test failures not hanging up the 
>>> test suite the you really want to do the latter.
>>> 
>>> These are all tests that contain 5 or more independent tests.  That's 
>>> probably why they are taking so long to run.
>>> 
>>> I don't object to having fairly long backstop timeouts, though I agree with 
>>> Pavel that we should choose something reasonable based on the slowest 
>>> running tests just so some single error doesn't cause test runs to just 
>>> never complete, making analysis harder.
>>> 
>> 
>> Vedant (cc:ed) is going to take a look at this as he's babysitting the
>> bots for the week. I'll defer the call to him.
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] increase timeout for tests?

2018-03-13 Thread Jim Ingham via lldb-dev
It sounds like we timing out based on the whole test class, not the individual 
tests?  If you're worried about test failures not hanging up the test suite the 
you really want to do the latter.

These are all tests that contain 5 or more independent tests.  That's probably 
why they are taking so long to run.  

I don't object to having fairly long backstop timeouts, though I agree with 
Pavel that we should choose something reasonable based on the slowest running 
tests just so some single error doesn't cause test runs to just never complete, 
making analysis harder.

Jim

> On Mar 13, 2018, at 10:43 AM, Davide Italiano  wrote:
> 
> Few examples:
> 
> 360 out of 617 test suites processed - TestObjCMethods.py
> XX
> TestObjCMethods.py: 363.726592
> 381 out of 617 test suites processed - TestHiddenIvars.py
>XX
>  TestHiddenIvars.py: 363.887766
> 387 out of 617 test suites processed - TestObjCNewSyntax.py
>   XX
> TestObjCNewSyntax.py: 363.842807
> 600 out of 617 test suites processed - TestDataFormatterObjC.py
> XX
> TestDataFormatterObjC.py: 388.414883
> 617 out of 617 test suites processed - TestLoadUnload.py
>  XX
> TestLoadUnload.py: 363.372328
> 
> On Tue, Mar 13, 2018 at 9:49 AM, Davide Italiano  
> wrote:
>> I'll re-run the test and send you a list.
>> 
>> Thank you!
>> 
>> --
>> Davide
>> 
>> On Tue, Mar 13, 2018 at 9:02 AM, Pavel Labath  wrote:
>>> Aha, in that case, it definitely sounds like increasing the timeout is in
>>> order. I would still go for something less than 30 minutes, but I don't care
>>> about that much. I may just export LLDB_TEST_TIMEOUT locally to lower it if
>>> tests taking long to time out start bugging me.
>>> 
>>> BTW, do you know which test is that? Is it one of the tests I have listed in
>>> one of the previous emails?
>>> 
>>> On Tue, 13 Mar 2018 at 14:52, Davide Italiano  wrote:
 
 On Tue, Mar 13, 2018 at 3:30 AM, Pavel Labath  wrote:
> I think we should get some data before pulling numbers out of our
> sleeves.
> If we can get some numbers on the slowest machine that we have around,
> then
> we should be able to specify the timeout as some multiple of the slowest
> test. For example, for me the slowest test takes around 110 seconds. The
> timeout is 4 minutes (~ 2x) and I don't get tests timing out. How long
> does
> the slowest test take for you? If we set the timeout as 3x or 4x that
> number, we should create a sufficiently large buffer and still avoid
> half-hour waits.
> 
 
 The longest test takes over 300 seconds. This is a late 2013 Mac Pro,
 so, definitely not the newest machine out there (but also something
 fairly high spec).
 For the archives, my conf is something like; cmake -GNinja ../llvm &&
 ninja check-lldb
 
 Thanks!
 
 --
 Davide

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] increase timeout for tests?

2018-03-12 Thread Jim Ingham via lldb-dev
The problem with having no timeouts is that you have to then be fairly careful 
how you write tests.  You can't do:

while (1) {
   print("Set a breakpoint here and hit it a few times then stop the test");
}

because if the breakpoint setting fails the test can run forever.  And we wait 
forever to launch or attach to a process internally in lldb, so if you mess up 
attaching or launching in some situation, that again makes the test run 
forever.  The timeouts are a backstop so you will get useful results from the 
testsuite in the case of this sort of error.

Jim


> On Mar 12, 2018, at 6:13 PM, Davide Italiano via lldb-dev 
>  wrote:
> 
> On Fri, Mar 9, 2018 at 3:45 AM, Pavel Labath  wrote:
>> 
>> 
>> 
>> On Thu, 8 Mar 2018 at 18:40, Davide Italiano  wrote:
>>> 
>>> On Thu, Mar 8, 2018 at 10:29 AM, Greg Clayton  wrote:
 It would be great to look into these and see what is taking so long.
 Some tests are doing way too much work and should be split up. But it would
 be great to see if we have any tests that are not good tests and are just
 taking too much time for no reason (like the watchpoint tests were that
 Pavel fixed).
 
>>> 
>>> I tend to agree with you. This is important. Pavel, is there a
>>> systematic way you use to profile test cases?
>>> Maybe we should consider having a testsuite mode where we warn for
>>> slow tests (or revamp the profiling mechanism, if there's one).
>>> 
>>> Thanks!
>> 
>> 
>> I don't have any fancy mechanism for profiling this. I just hacked the
>> process_file function in dosep.py to measure the time each test takes and
>> report it if it exceeds a certain threshold. Then I just looked at the tests
>> that seemed like they are taking more time than they ought to.
>> 
>> This is the patch, in it's entirety:
>> --- a/packages/Python/lldbsuite/test/dosep.py
>> +++ b/packages/Python/lldbsuite/test/dosep.py
>> @@ -489,8 +489,13 @@ def process_file(test_file, dotest_argv,
>> inferior_pid_events):
>> timeout = (os.getenv("LLDB_%s_TIMEOUT" % timeout_name) or
>>getDefaultTimeout(dotest_options.lldb_platform_name))
>> 
>> +import time
>> +t = time.time()
>> results.append(call_with_timeout(
>> command, timeout, base_name, inferior_pid_events, test_file))
>> +t = time.time() - t
>> +if t > 20:
>> +print("XX   %s:
>> %f"%(base_name, t))
>> 
>> # result = (name, status, passes, failures, unexpected_successes)
>> timed_out = [name for name, status, _, _, _ in results
>> 
>> We could try to make this more fancy, but I'm not sure worth it. It just
>> occurred to me you can achieve a similar effect by running the test suite
>> with a reduced timeout threshold. Something like "LLDB_TEST_TIMEOUT=20s
>> ninja check-lldb" should do the trick. At one point Todd also implemented a
>> mechanism for recording what the test is doing before we time out and kill
>> it. The idea was that this would help us determine what the test was doing
>> when it was killed (and whether it was really stuck, or just slow). However,
>> I've never used this, so I have no idea what's the state of it.
>> 
>> I am not opposed to raising the timeout threshold, but before we do that, we
>> should check whether it will actually help. I.e., whether the test is just
>> slow, or it sometimes hangs (I think TestMultithreaded is a good candidate
>> for the latter). Can you check what is the time those tests take to run
>> individually? Current osx timeout (dosep.py:getDefaultTimeout) is set to 6
>> minutes, so if the tests normally take much shorter than this (less than 2
>> minutes?), I think there is something else going on, and increasing the
>> timeout will not help.
>> 
>> Another interesting thing may be to check the system load while running the
>> test suite. Nowadays, lldb will sometimes spawn N (where N is the number of
>> cpus) threads to do debug info parsing, so when we run N tests in parallel,
>> we may end up with N^2 threads competing for N cpus. I've never seen this to
>> be an issue as our tests generally don't have that much debug info to begin
>> with, but I don't think we've ever investigated this either...
>> 
>> BTW: This is my list of tests that take over 20 seconds (I ran it on linux
>> so it does not include any osx-specific ones):
>> 20.729438 TestWatchpointCommands.py
>> 21.158373 TestMiData.py
>> 22.018853 TestRecursiveInferior.py
>> 22.692744 TestInferiorCrashing.py
>> 22.986511 TestLinuxCore.py
>> 23.073790 TestInferiorAssert.py
>> 23.193351 TestAnonymous.py
>> 24.539430 TestMiStack.py
>> 25.232940 TestUnwindExpression.py
>> 26.608233 TestSetWatchlocation.py
>> 28.977390 TestWatchLocation.py
>> 29.057234 TestMultithreaded.py
>> 29.813142 TestCPP11EnumTypes.py
>> 30.830618 TestWatchLocationWithWatchSet.py
>> 32.979785 TestThreadStates.py
>> 

Re: [lldb-dev] lldb showing wrong type structure for virtual pointer type

2018-03-01 Thread Jim Ingham via lldb-dev
Yes, the code was assuming that "exact_match" would be obeyed so it explicitly 
specified returning only 1 match.  When exact doesn't work, that means you get 
the first one...  You will only have a problem with the one whose name matches 
both types, since the other way around the vtable name will have the namespace 
in it, and it won't match the bare one.  So your workaround should be good 
altogether.

Jim


> On Mar 1, 2018, at 2:11 AM, jonas echterhoff  wrote:
> 
> FWIW, I just out found that I can trivially work around this problem by 
> changing the order of object files passed to the linker.
> 
> It turns out that lldb will just pick up the first type named "Transform" in 
> the binary (regardless of namespace). Now, if I make sure that the object 
> file containing the "correct" Transform type comes first in the linker 
> command line, it will also be first in the binary, and it will be picked up 
> by lldb. Now, in theory, this might cause the same problem, just the "other 
> way around", when debugging code using the namespaced version of the type, 
> but in our case, that should not be a problem, because the latter is not a 
> virtual type, so lldb should not use dynamic lookup in that case.
> 
> jonas
> 
>> On Mar 1, 2018, at 3:59 AM, Jim Ingham  wrote:
>> 
>> 
>> 
>>> On Feb 28, 2018, at 1:09 PM, jonas echterhoff  wrote:
>>> 
>>> 
>>> 
 On Feb 28, 2018, at 9:27 PM, Jim Ingham  wrote:
 
 Interesting.  
 
 First off, you can turn off fetching dynamic values globally (including in 
 the Xcode Locals view) by putting:
 
 settings set target.prefer-dynamic-value no-dynamic-values
>>> 
 in your ~/.lldbinit file.  You can toggle this on and off in a session, 
 though Xcode won't notice you've changed the value till you cause it to 
 refresh the locals (step or whatever).
>>> 
>>> This will fix the output of "frame variable". But it does not seem to fix 
>>> the variable display in the UI.
>> 
>> They must be setting the dynamic value directly when they fetch the values.  
>> That of course overrides the general setting.  I'll go see why they do that, 
>> but that won't help you for now.
>> 
>>> 
 We do log the process of finding the dynamic type.  You can see this by 
 running the command:
 
 log enable -f /tmp/lldb-object-log.txt lldb object
 
 Probably easiest to put that in your .lldbinit.
 
 That channel also logs when we read in modules, and so it might be a 
 little chatty, but you should see:
 
 : static-type = '' has vtable symbol 'vtable 
 for '
 
 and then some more messages that trace our attempt to look up DYNAMIC 
 CLASS.  If you turn on those logs, what do you see for these classes?
>>> 
>>> 0x00010f62ecd0: static-type = 'Transform *' has vtable symbol 'vtable 
>>> for Transform'
>>> 
>>> 0x00010f62ecd0: static-type = 'Transform *' has dynamic type: 
>>> uid={0x100012d7a}, type-name='Transform'
>> 
>> Grr...  We go from the name in the vtable symbol to the class type using 
>> Module::FindTypes, passing the exact_match flag 'cause we know this is an 
>> exact match.  Turns out the Module::FindTypes only obeys its exact_match 
>> flag if either the name you pass in starts with :: or if you can dial up the 
>> exact type kind (struct, class, enum, etc...) you are looking for.  We can't 
>> do the latter here because we don't know whether the dynamic type is a class 
>> or a struct, and we were just passing the name we got from the vtable.
>> 
>> I have a small fix for the dynamic type issue in r326412.  Fixing the 
>> FindTypes behavior is more involved, and I don't know whether other places 
>> rely on this misbehavior. I filed:
>> 
>> https://bugs.llvm.org/show_bug.cgi?id=36556
>> 
>> to examine that issue further.
>> 
>> Thanks for reporting this.
>> 
>> Jim
>> 
>> 
>>> 
>>> jonas
>>> 
 
 Jim
 
 
> On Feb 28, 2018, at 12:03 PM, jonas echterhoff  wrote:
> 
> 
> 
>> On Feb 28, 2018, at 7:14 PM, Jim Ingham  wrote:
>> 
>> Jonas,
>> 
>> What are you using to inspect the this pointer?
> 
> Normally, the Xcode debugger UI.
> 
>> You can use "frame variable" (the equivalent of gdb's "info locals") 
>> which just relies on debug info or the expression evaluator e.g. 
>> "print".  Do both methods show the same problem?
> 
> (lldb) frame variable this
> (Scripting::UnityEngine::Transform *) this = 0x00010fe2eb20
> 
> That gives me the wrong namespace
> 
> (lldb) print this
> (Scripting::UnityEngine::Transform *) $4 = 0x00010fe2eb20
> 
> That also gives me the wrong namespace
> 
> But:
> 
> (lldb) print *this
> (Transform) $5 = {
> [...]
> 
> gives me the correct (global) namespace.
> 
> Also:
> 
> (lldb) 

Re: [lldb-dev] lldb showing wrong type structure for virtual pointer type

2018-02-28 Thread Jim Ingham via lldb-dev


> On Feb 28, 2018, at 1:09 PM, jonas echterhoff  wrote:
> 
> 
> 
>> On Feb 28, 2018, at 9:27 PM, Jim Ingham  wrote:
>> 
>> Interesting.  
>> 
>> First off, you can turn off fetching dynamic values globally (including in 
>> the Xcode Locals view) by putting:
>> 
>> settings set target.prefer-dynamic-value no-dynamic-values
> 
>> in your ~/.lldbinit file.  You can toggle this on and off in a session, 
>> though Xcode won't notice you've changed the value till you cause it to 
>> refresh the locals (step or whatever).
> 
> This will fix the output of "frame variable". But it does not seem to fix the 
> variable display in the UI.

They must be setting the dynamic value directly when they fetch the values.  
That of course overrides the general setting.  I'll go see why they do that, 
but that won't help you for now.

> 
>> We do log the process of finding the dynamic type.  You can see this by 
>> running the command:
>> 
>> log enable -f /tmp/lldb-object-log.txt lldb object
>> 
>> Probably easiest to put that in your .lldbinit.
>> 
>> That channel also logs when we read in modules, and so it might be a little 
>> chatty, but you should see:
>> 
>> : static-type = '' has vtable symbol 'vtable for 
>> '
>> 
>> and then some more messages that trace our attempt to look up DYNAMIC CLASS. 
>>  If you turn on those logs, what do you see for these classes?
> 
> 0x00010f62ecd0: static-type = 'Transform *' has vtable symbol 'vtable for 
> Transform'
> 
> 0x00010f62ecd0: static-type = 'Transform *' has dynamic type: 
> uid={0x100012d7a}, type-name='Transform'

Grr...  We go from the name in the vtable symbol to the class type using 
Module::FindTypes, passing the exact_match flag 'cause we know this is an exact 
match.  Turns out the Module::FindTypes only obeys its exact_match flag if 
either the name you pass in starts with :: or if you can dial up the exact type 
kind (struct, class, enum, etc...) you are looking for.  We can't do the latter 
here because we don't know whether the dynamic type is a class or a struct, and 
we were just passing the name we got from the vtable.

I have a small fix for the dynamic type issue in r326412.  Fixing the FindTypes 
behavior is more involved, and I don't know whether other places rely on this 
misbehavior. I filed:

https://bugs.llvm.org/show_bug.cgi?id=36556

to examine that issue further.

Thanks for reporting this.

Jim


> 
> jonas
> 
>> 
>> Jim
>> 
>> 
>>> On Feb 28, 2018, at 12:03 PM, jonas echterhoff  wrote:
>>> 
>>> 
>>> 
 On Feb 28, 2018, at 7:14 PM, Jim Ingham  wrote:
 
 Jonas,
 
 What are you using to inspect the this pointer?
>>> 
>>> Normally, the Xcode debugger UI.
>>> 
 You can use "frame variable" (the equivalent of gdb's "info locals") which 
 just relies on debug info or the expression evaluator e.g. "print".  Do 
 both methods show the same problem?
>>> 
>>> (lldb) frame variable this
>>> (Scripting::UnityEngine::Transform *) this = 0x00010fe2eb20
>>> 
>>> That gives me the wrong namespace
>>> 
>>> (lldb) print this
>>> (Scripting::UnityEngine::Transform *) $4 = 0x00010fe2eb20
>>> 
>>> That also gives me the wrong namespace
>>> 
>>> But:
>>> 
>>> (lldb) print *this
>>> (Transform) $5 = {
>>> [...]
>>> 
>>> gives me the correct (global) namespace.
>>> 
>>> Also:
>>> 
>>> (lldb) frame variable -d no-dynamic-values this
>>> (Transform *) this = 0x00010fe2eb20
>>> 
>>> gives me the correct namespace.
>>> 
 
 Also note that lldb by default will try to discern the full dynamic type 
 of the variables it prints.  You can disable this by doing:
 
 (lldb) expr -d no-dynamic-values -- this
 
 or equivalently:
 
 (lldb) frame variable -d no-dynamic-values this
 
 Is it the dynamic value resolution that's causing the incorrect printing?
>>> 
>>> Yes, both of those above give me the correct types!
>>> 
>>> Now, this is already very helpful - Thank you! 
>>> This means I can get correct values using the lldb console. If there was 
>>> some way to make the Xcode UI show the correct values, that would be even 
>>> better.
>>> 
>>> jonas
>>> 
 
 Jim
 
 
 
 
> On Feb 28, 2018, at 3:03 AM, jonas echterhoff via lldb-dev 
>  wrote:
> 
> 
>> On Feb 28, 2018, at 11:19 AM, Dmitry Antipov  wrote:
>> 
>> On 02/28/2018 11:31 AM, jonas echterhoff via lldb-dev wrote:
>> 
>>> I'm using lldb-900.0.64.
>>   ^^
>>   ??
>> Latest official release is 5.0.1; also there are 6.0.0 (at -rc3, the 
>> next release)
>> and 7.0.0 (a.k.a SVN trunk). What's the 'version' output of your LLDB 
>> prompt?
> 
> It is what I posted:
> 
> jechter$ lldb --version
> lldb-900.0.64
> Swift-4.0
> 
> Maybe Apple uses a different versioning scheme for 

Re: [lldb-dev] lldb showing wrong type structure for virtual pointer type

2018-02-28 Thread Jim Ingham via lldb-dev
Interesting.  

First off, you can turn off fetching dynamic values globally (including in the 
Xcode Locals view) by putting:

settings set target.prefer-dynamic-value no-dynamic-values

in your ~/.lldbinit file.  You can toggle this on and off in a session, though 
Xcode won't notice you've changed the value till you cause it to refresh the 
locals (step or whatever).

We do log the process of finding the dynamic type.  You can see this by running 
the command:

log enable -f /tmp/lldb-object-log.txt lldb object

Probably easiest to put that in your .lldbinit.

That channel also logs when we read in modules, and so it might be a little 
chatty, but you should see:

: static-type = '' has vtable symbol 'vtable for 
'

and then some more messages that trace our attempt to look up DYNAMIC CLASS.  
If you turn on those logs, what do you see for these classes?

Jim


> On Feb 28, 2018, at 12:03 PM, jonas echterhoff  wrote:
> 
> 
> 
>> On Feb 28, 2018, at 7:14 PM, Jim Ingham  wrote:
>> 
>> Jonas,
>> 
>> What are you using to inspect the this pointer?
> 
> Normally, the Xcode debugger UI.
> 
>> You can use "frame variable" (the equivalent of gdb's "info locals") which 
>> just relies on debug info or the expression evaluator e.g. "print".  Do both 
>> methods show the same problem?
> 
> (lldb) frame variable this
> (Scripting::UnityEngine::Transform *) this = 0x00010fe2eb20
> 
> That gives me the wrong namespace
> 
> (lldb) print this
> (Scripting::UnityEngine::Transform *) $4 = 0x00010fe2eb20
> 
> That also gives me the wrong namespace
> 
> But:
> 
> (lldb) print *this
> (Transform) $5 = {
> [...]
> 
> gives me the correct (global) namespace.
> 
> Also:
> 
> (lldb) frame variable -d no-dynamic-values this
> (Transform *) this = 0x00010fe2eb20
> 
> gives me the correct namespace.
> 
>> 
>> Also note that lldb by default will try to discern the full dynamic type of 
>> the variables it prints.  You can disable this by doing:
>> 
>> (lldb) expr -d no-dynamic-values -- this
>> 
>> or equivalently:
>> 
>> (lldb) frame variable -d no-dynamic-values this
>> 
>> Is it the dynamic value resolution that's causing the incorrect printing?
> 
> Yes, both of those above give me the correct types!
> 
> Now, this is already very helpful - Thank you! 
> This means I can get correct values using the lldb console. If there was some 
> way to make the Xcode UI show the correct values, that would be even better.
> 
> jonas
> 
>> 
>> Jim
>> 
>> 
>> 
>> 
>>> On Feb 28, 2018, at 3:03 AM, jonas echterhoff via lldb-dev 
>>>  wrote:
>>> 
>>> 
 On Feb 28, 2018, at 11:19 AM, Dmitry Antipov  wrote:
 
 On 02/28/2018 11:31 AM, jonas echterhoff via lldb-dev wrote:
 
> I'm using lldb-900.0.64.
 ^^
 ??
 Latest official release is 5.0.1; also there are 6.0.0 (at -rc3, the next 
 release)
 and 7.0.0 (a.k.a SVN trunk). What's the 'version' output of your LLDB 
 prompt?
>>> 
>>> It is what I posted:
>>> 
>>> jechter$ lldb --version
>>> lldb-900.0.64
>>> Swift-4.0
>>> 
>>> Maybe Apple uses a different versioning scheme for lldb distributed with 
>>> their toolchains?
 
> Unfortunately, I have not yet succeeded in coming up with a small, 
> independent repro case which shows this problem.
 
 IIUC this is it:
>>> 
>>> [...]
>>> 
 Here 'this' is different between calls to obj2.f () and obj2.g () 
 (0x7fffdb50 vs.
 0x7fffdb40), and objects are shown as different as well - {111, 
 222} vs. {333, 444}.
>>> 
>>> Thanks. What you are showing there seems very peculiar. 
>>> 
>>> But I don't think it's the same problem as I have (and also, using the same 
>>> steps on my machine does not repro the problem you showed - I get the same 
>>> value for "this" and it's members between the calls to S::B::f and S::B::g).
>>> 
>>> My problem was not about showing a wrong object (My "this" pointer value 
>>> was correct), but about showing a wrong type representation of the correct 
>>> object data.
>>> 
>>> jonas
>>> 
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] lldb showing wrong type structure for virtual pointer type

2018-02-28 Thread Jim Ingham via lldb-dev
Jonas,

What are you using to inspect the this pointer?  You can use "frame variable" 
(the equivalent of gdb's "info locals") which just relies on debug info or the 
expression evaluator e.g. "print".  Do both methods show the same problem?

Also note that lldb by default will try to discern the full dynamic type of the 
variables it prints.  You can disable this by doing:

(lldb) expr -d no-dynamic-values -- this

or equivalently:

(lldb) frame variable -d no-dynamic-values this

Is it the dynamic value resolution that's causing the incorrect printing?

Jim




> On Feb 28, 2018, at 3:03 AM, jonas echterhoff via lldb-dev 
>  wrote:
> 
> 
>> On Feb 28, 2018, at 11:19 AM, Dmitry Antipov  wrote:
>> 
>> On 02/28/2018 11:31 AM, jonas echterhoff via lldb-dev wrote:
>> 
>>> I'm using lldb-900.0.64.
>>   ^^
>>   ??
>> Latest official release is 5.0.1; also there are 6.0.0 (at -rc3, the next 
>> release)
>> and 7.0.0 (a.k.a SVN trunk). What's the 'version' output of your LLDB prompt?
> 
> It is what I posted:
> 
> jechter$ lldb --version
> lldb-900.0.64
>  Swift-4.0
> 
> Maybe Apple uses a different versioning scheme for lldb distributed with 
> their toolchains?
>> 
>>> Unfortunately, I have not yet succeeded in coming up with a small, 
>>> independent repro case which shows this problem.
>> 
>> IIUC this is it:
> 
> [...]
> 
>> Here 'this' is different between calls to obj2.f () and obj2.g () 
>> (0x7fffdb50 vs.
>> 0x7fffdb40), and objects are shown as different as well - {111, 222} 
>> vs. {333, 444}.
> 
> Thanks. What you are showing there seems very peculiar. 
> 
> But I don't think it's the same problem as I have (and also, using the same 
> steps on my machine does not repro the problem you showed - I get the same 
> value for "this" and it's members between the calls to S::B::f and S::B::g).
> 
> My problem was not about showing a wrong object (My "this" pointer value was 
> correct), but about showing a wrong type representation of the correct object 
> data.
> 
> jonas
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] how do i submit my patch for 'Support demangling for D symbols via dlopen'

2018-02-27 Thread Jim Ingham via lldb-dev
I only briefly scanned the patch, but I'm not sure treating D as another 
mangling variant of C++ is the best model - I think it will be easier if we 
gather all the separate language features up in the individual languages.  You 
might have a look at the swift-enabled lldb 
(https://github.com/apple/swift-lldb/) to see how it is handled there.  That's 
a little weak because we iterate over supported languages by hand.  When there 
were a few that wasn't an issue, but as we pick up more languages, we should 
introduce a "do over supported languages" feature which would clean up the 
logic there.

Jim


> On Feb 26, 2018, at 9:01 PM, Timothee Cour via lldb-dev 
>  wrote:
> 
> https://github.com/llvm-mirror/lldb/pull/3
> 
> it would be *really* nice if llvm or lldb accepted industry standard
> github PR's, at least as an option. Would make contributing so much
> easier for outsiders
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] How to test lldb-vscode?

2018-02-14 Thread Jim Ingham via lldb-dev
The SB API's were designed to be binary stable API's when used either from C++ 
or Python (the latter are just a SWIG-ified version of the former.)  So just 
like the Python wrapper, with the exception that you might call an API that 
some older version of liblldb doesn't provide, Greg's adaptor wouldn't be 
sensitive to lldb library version.

Jim


> On Feb 14, 2018, at 4:43 PM, Vadim Chugunov via lldb-dev 
>  wrote:
> 
> My tests are here.   I probably wouldn't move all that to Python for my 
> extension, but feel free to draw ideas on things to test from it.
> 
> What's going to be your deployment story?   The nice thing about Python-based 
> adapter is that it will run on a wide range on LLDB versions.  For a native 
> adapter you'll need to somehow make sure that you have the right liblldb 
> installed, won't you?
> 
> 
> On Wed, Feb 14, 2018 at 3:54 PM, Greg Clayton  wrote:
> 
> 
>> On Feb 14, 2018, at 3:45 PM, Vadim Chugunov  wrote:
>> 
>> FYI - I've been maintaining a similar VSCode extension, which does it all 
>> through LLDB's Python interface. 
> 
> Indeed, I saw you plug-in and since it worked so well I wanted to make a 
> native version of this plug-in to avoid taking over the python interpreter 
> and get native performance. Very nice work on that plug-in as it inspired 
> this one! I adopted many similar settings that you allow in the launch.json 
> file and added some new ones.
> 
>> 
>> As for testing, the protocol is JSON with HTTP-like headers for framing, so 
>> you could have a Python script emulating the VSCode side.
> 
> That is what I was planning to do. Might be nice to work together on this and 
> get the testing working as it could test both of our plug-ins. Would be great 
> to have a debug session with a simple program that sets a breakpoint by name, 
> by file and line and exception and run, hit the breakpoints, view variables, 
> etc.
> 
> Greg
> 
>> 
>> 
>> On Wed, Feb 14, 2018 at 2:28 PM, Greg Clayton via lldb-dev 
>>  wrote:
>> I have a patch I will be submitting soon that adds a tool called 
>> "lldb-vscode" that is much like "lldb-mi" where it sits on top of 
>> LLDB.framework on Darwin and liblldb.so on all other platforms, and it 
>> speaks the Visual Studio Code Debug adaptor language and can be used to 
>> create a VSCode extension that does native LLDB debugging. The plug-in works 
>> really well to debug native apps and is very responsive and quick.
>> 
>> One issue I am thinking about is how to test this. Some VS Code tests out 
>> there use Javascript + npm to run a mock VSCode IDE and send/receive 
>> packets. I am not sure we can require people to have this on all build bots. 
>> So I am writing to get everyone's opinion on this so I can test this and get 
>> a patch submitted.
>> 
>> Any ideas would be appreciated.
>> 
>> Greg Clayton
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
> 
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


  1   2   3   4   >