Re: [lldb-dev] lldb command like gdb's "set auto-solib-add off"

[Tamas Berghammer via lldb-dev]

Few more addition to the above:

How are you running lldb-server on your device? For remote debugging
running lldb-server in platform mode (and then using remote-linux or
similar as the selected platform in LLDB) will give you a significantly
better performance over running lldb-server in gdbserver mode only and then
selecting remote-gdbserver as your platform in LLDB. The following things
*only apply* to the case when you are running lldb-server in platform mode.

If the target side libraries are backed by files on the target system then
LLDB should download them only once (at first usage) and then cache it on
the host in a module cache (even between LLDB or machine restarts). It
means that the startup time is expected to be quite high the first time you
debug on a specific device but it should be much faster afterwards (as you
already have the libraries on the host). If this is not the case it would
be interesting to see why module cache isn't working for you.

By default LLDB uses the gdb-remote protocol to download the files from the
target device what is known to be very very slow for transferring large
amount of data in bulk. For Android we implemented a faster way to download
the files using ADB what gave us a large performance gain (multiple times
faster, but don't remember exact number). You can see the code at
https://github.com/llvm-mirror/lldb/blob/a4df8399803ba766d05ef7fcd5d04dc0342d2682/source/Plugins/Platform/Android/PlatformAndroid.cpp#L190
I
expect that you can achieve similar gains if you implement
Platform*::GetFile and Platform*::PutFile for your platform based on a
faster method (e.g. scp/rsync)

Tamas

On Tue, May 23, 2017 at 12:23 AM Ted Woodward via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> To expand on Jim's message, "target modules search-paths add" can be used
> to help lldb find  the host-side copies of shared libraries when they're
> not in the same directory as on the target system.
>
> For example, if you have libraries in /usr/lib on the target system, and
> have copies on the host system in /local/scratch/work/debuglibs , you can
> say
> target modules search-paths add /usr/lib /local/scratch/work/debuglibs
> and when lldb goes to load (for example) /usr/lib/libc.so, it will try to
> load /local/scratch/work/debuglibs/libc.so from the host machine before
> trying to load through the memory interface.
>
> I found this very helpful when trying to debug dynamic executables on
> Linux running on a Hexagon board, running lldb on x86 Linux or Windows.
>
> Ted
>
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
>
> > -Original Message-
> > From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Jim
> > Ingham via lldb-dev
> > Sent: Monday, May 22, 2017 5:02 PM
> > To: Chunseok Lee <chunseok...@gmail.com>
> > Cc: lldb-dev <lldb-dev@lists.llvm.org>
> > Subject: Re: [lldb-dev] lldb command like gdb's "set auto-solib-add off"
> >
> > In general, if lldb can find host-side copies of binaries that match the
> ones it
> > finds on the device, it will do all symbol reading against the host
> copies.  In
> > the case of an OS X host debugging iOS, lldb uses Spotlight and a few
> other
> > tricks to find the host-side binaries.  You can also use
> "add-symbol-file" to
> > manually point lldb at the host-side symbol files.  If you are reading
> symbols
> > from host-side files, then symbol loading doesn't slow down debugging
> > startup that much.
> >
> > Presumably, your symbol files are only on the device, so you are reading
> > them from memory.  "settings set target.memory-module-load-level" is
> > almost what you want, but it applies to ALL shared libraries read from
> > memory.  If you can copy the symbol file that contains the
> > __jit_debug_register_code to the host you are debugging from, and use
> > add-symbol-file to tell lldb about it, then that one should NOT have to
> be
> > read from memory anymore.  Then you could turn "memory-module-load-
> > level" to partial or even mininal, and that should get you starting
> faster.
> >
> > The other option would be to extend the setting, so you can say:
> >
> > set set target.memory-module-load-level [[lib-name level] [lib-name
> level]
> > ...]
> >
> > If there's just one argument, that's equivalent to "all ".
> >
> > Jim
> >
> > > On May 22, 2017, at 2:35 PM, Chunseok Lee <chunseok...@gmail.com>
> > wrote:
> > >
> > >
> > >
> > > Thank you for your help.
> > > It wou

Re: [lldb-dev] lldb command like gdb's "set auto-solib-add off"

[Ted Woodward via lldb-dev]

To expand on Jim's message, "target modules search-paths add" can be used to 
help lldb find  the host-side copies of shared libraries when they're not in 
the same directory as on the target system.

For example, if you have libraries in /usr/lib on the target system, and have 
copies on the host system in /local/scratch/work/debuglibs , you can say
target modules search-paths add /usr/lib /local/scratch/work/debuglibs
and when lldb goes to load (for example) /usr/lib/libc.so, it will try to load 
/local/scratch/work/debuglibs/libc.so from the host machine before trying to 
load through the memory interface.

I found this very helpful when trying to debug dynamic executables on Linux 
running on a Hexagon board, running lldb on x86 Linux or Windows.

Ted

--
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux 
Foundation Collaborative Project

> -Original Message-
> From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Jim
> Ingham via lldb-dev
> Sent: Monday, May 22, 2017 5:02 PM
> To: Chunseok Lee <chunseok...@gmail.com>
> Cc: lldb-dev <lldb-dev@lists.llvm.org>
> Subject: Re: [lldb-dev] lldb command like gdb's "set auto-solib-add off"
> 
> In general, if lldb can find host-side copies of binaries that match the ones 
> it
> finds on the device, it will do all symbol reading against the host copies.  
> In
> the case of an OS X host debugging iOS, lldb uses Spotlight and a few other
> tricks to find the host-side binaries.  You can also use "add-symbol-file" to
> manually point lldb at the host-side symbol files.  If you are reading symbols
> from host-side files, then symbol loading doesn't slow down debugging
> startup that much.
> 
> Presumably, your symbol files are only on the device, so you are reading
> them from memory.  "settings set target.memory-module-load-level" is
> almost what you want, but it applies to ALL shared libraries read from
> memory.  If you can copy the symbol file that contains the
> __jit_debug_register_code to the host you are debugging from, and use
> add-symbol-file to tell lldb about it, then that one should NOT have to be
> read from memory anymore.  Then you could turn "memory-module-load-
> level" to partial or even mininal, and that should get you starting faster.
> 
> The other option would be to extend the setting, so you can say:
> 
> set set target.memory-module-load-level [[lib-name level] [lib-name level]
> ...]
> 
> If there's just one argument, that's equivalent to "all ".
> 
> Jim
> 
> > On May 22, 2017, at 2:35 PM, Chunseok Lee <chunseok...@gmail.com>
> wrote:
> >
> >
> >
> > Thank you for your help.
> > It would be really helpful to me.
> >
> > The reason behind the question is exactly what you mentioned. I am
> > wokring on debugging in devices and it seems that shared library loading(I
> do not know lldb loads symbols lazyly) runs very slowly since my testing
> program depends on so many shared libs.  since I am debuggging with gdbjit
> feature, I do not need shared library loading except one shared lib(which
> contains __jit_debug_register_code symbol) Thus, I want to turn off shred
> lib loading except one shared lib. Is it possible?
> >
> > Thank you.
> >
> > BR,
> > Chunseok Lee
> >
> >
> >
> > 2017. 5. 23. 오전 3:24 Jim Ingham <jing...@apple.com> 작성:
> >
> >> We designed lldb to be as lazy as possible in loading symbol information
> from the shared libraries as they get loaded.  If this is causing you 
> problems,
> the first thing to do is to figure out why lldb is pulling in more information
> than you need.  For instance, it looks like on Linux the gdb JIT loading
> breakpoint is getting set with a global search which is pulling in all the 
> symbols
> from all the initially loaded shared libraries.  It would be better to fix 
> things so
> that only explicit user actions (which can be scoped to shared libraries to 
> limit
> the search) pull in symbols so we don't have to fiddle around with turning off
> and on the loading of shared libraries.  Back when we were supporting gdb
> for MacOS I did a lot of work to try to get this right (there are times you 
> really
> need the shared library info - e.g. when something shows up in a backtrace)
> so you have to judiciously re-introduce symbols, or the user experience is
> noticeably degraded.
> >>
> >> It also depends on how far you want to go turning this off.  There's "don't
> look at shared libraries at all" which is what the auto-solib-add variable 
> does
> IIRC.  We also had a fairly extensive

Re: [lldb-dev] lldb command like gdb's "set auto-solib-add off"

[Jim Ingham via lldb-dev]

In general, if lldb can find host-side copies of binaries that match the ones 
it finds on the device, it will do all symbol reading against the host copies.  
In the case of an OS X host debugging iOS, lldb uses Spotlight and a few other 
tricks to find the host-side binaries.  You can also use "add-symbol-file" to 
manually point lldb at the host-side symbol files.  If you are reading symbols 
from host-side files, then symbol loading doesn't slow down debugging startup 
that much.

Presumably, your symbol files are only on the device, so you are reading them 
from memory.  "settings set target.memory-module-load-level" is almost what you 
want, but it applies to ALL shared libraries read from memory.  If you can copy 
the symbol file that contains the __jit_debug_register_code to the host you are 
debugging from, and use add-symbol-file to tell lldb about it, then that one 
should NOT have to be read from memory anymore.  Then you could turn 
"memory-module-load-level" to partial or even mininal, and that should get you 
starting faster.

The other option would be to extend the setting, so you can say:

set set target.memory-module-load-level [[lib-name level] [lib-name level] ...] 

If there's just one argument, that's equivalent to "all ".  

Jim

> On May 22, 2017, at 2:35 PM, Chunseok Lee  wrote:
> 
> 
> 
> Thank you for your help. 
> It would be really helpful to me. 
> 
> The reason behind the question is exactly what you mentioned. I am wokring on 
> debugging in devices and it seems that shared library loading(I do not know 
> lldb loads symbols lazyly) runs very slowly since my testing program depends 
> on so many shared libs.  since I am debuggging with gdbjit feature, I do not 
> need shared library loading except one shared lib(which contains 
> __jit_debug_register_code symbol)
> Thus, I want to turn off shred lib loading except one shared lib. Is it 
> possible?
> 
> Thank you. 
> 
> BR,
> Chunseok Lee
> 
> 
> 
> 2017. 5. 23. 오전 3:24 Jim Ingham  작성:
> 
>> We designed lldb to be as lazy as possible in loading symbol information 
>> from the shared libraries as they get loaded.  If this is causing you 
>> problems, the first thing to do is to figure out why lldb is pulling in more 
>> information than you need.  For instance, it looks like on Linux the gdb JIT 
>> loading breakpoint is getting set with a global search which is pulling in 
>> all the symbols from all the initially loaded shared libraries.  It would be 
>> better to fix things so that only explicit user actions (which can be scoped 
>> to shared libraries to limit the search) pull in symbols so we don't have to 
>> fiddle around with turning off and on the loading of shared libraries.  Back 
>> when we were supporting gdb for MacOS I did a lot of work to try to get this 
>> right (there are times you really need the shared library info - e.g. when 
>> something shows up in a backtrace) so you have to judiciously re-introduce 
>> symbols, or the user experience is noticeably degraded.
>> 
>> It also depends on how far you want to go turning this off.  There's "don't 
>> look at shared libraries at all" which is what the auto-solib-add variable 
>> does IIRC.  We also had a fairly extensive mechanism to specify 
>> "load-levels" for various libraries, which  was more user-friendly but much 
>> more work to support.  Anyway, it would be easy to just turn this shared 
>> library notifications - just don't set the dyld load notification 
>> breakpoint.  That would be the easy part.  But as I said above, you're also 
>> going to have to make sure you turn it back on for users when some action 
>> they request requires it.
>> 
>> Note, there is a setting to determine how much symbol information to read in 
>> from libraries loaded from memory (since in device debugging this can be 
>> quite slow.)  That is the target.memory-module-load-level setting.  You 
>> might look at that for some hints as to how to proceed.
>> 
>> Jim
>> 
>>> On May 22, 2017, at 8:02 AM, Chunseok Lee via lldb-dev 
>>>  wrote:
>>> 
>>> Hello. 
>>> 
>>> In gdb, I can toggle auto symbol loading using "set auto-solib-add off" 
>>> command.
>>> I wonder that lldb has similar command for disabling auto shared library 
>>> symbol loading? 
>>> If not, where is the good starting point to review the source code to 
>>> implement this feature?
>>> 
>>> BR,
>>> Chunseok Lee
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] lldb command like gdb's "set auto-solib-add off"

[Chunseok Lee via lldb-dev]

Thank you for your help. 
It would be really helpful to me. 

The reason behind the question is exactly what you mentioned. I am wokring on 
debugging in devices and it seems that shared library loading(I do not know 
lldb loads symbols lazyly) runs very slowly since my testing program depends on 
so many shared libs.  since I am debuggging with gdbjit feature, I do not need 
shared library loading except one shared lib(which contains 
__jit_debug_register_code symbol)
Thus, I want to turn off shred lib loading except one shared lib. Is it 
possible?

Thank you. 

BR,
Chunseok Lee



2017. 5. 23. 오전 3:24 Jim Ingham  작성:

> We designed lldb to be as lazy as possible in loading symbol information from 
> the shared libraries as they get loaded.  If this is causing you problems, 
> the first thing to do is to figure out why lldb is pulling in more 
> information than you need.  For instance, it looks like on Linux the gdb JIT 
> loading breakpoint is getting set with a global search which is pulling in 
> all the symbols from all the initially loaded shared libraries.  It would be 
> better to fix things so that only explicit user actions (which can be scoped 
> to shared libraries to limit the search) pull in symbols so we don't have to 
> fiddle around with turning off and on the loading of shared libraries.  Back 
> when we were supporting gdb for MacOS I did a lot of work to try to get this 
> right (there are times you really need the shared library info - e.g. when 
> something shows up in a backtrace) so you have to judiciously re-introduce 
> symbols, or the user experience is noticeably degraded.
> 
> It also depends on how far you want to go turning this off.  There's "don't 
> look at shared libraries at all" which is what the auto-solib-add variable 
> does IIRC.  We also had a fairly extensive mechanism to specify "load-levels" 
> for various libraries, which  was more user-friendly but much more work to 
> support.  Anyway, it would be easy to just turn this shared library 
> notifications - just don't set the dyld load notification breakpoint.  That 
> would be the easy part.  But as I said above, you're also going to have to 
> make sure you turn it back on for users when some action they request 
> requires it.
> 
> Note, there is a setting to determine how much symbol information to read in 
> from libraries loaded from memory (since in device debugging this can be 
> quite slow.)  That is the target.memory-module-load-level setting.  You might 
> look at that for some hints as to how to proceed.
> 
> Jim
> 
>> On May 22, 2017, at 8:02 AM, Chunseok Lee via lldb-dev 
>>  wrote:
>> 
>> Hello. 
>> 
>> In gdb, I can toggle auto symbol loading using "set auto-solib-add off" 
>> command.
>> I wonder that lldb has similar command for disabling auto shared library 
>> symbol loading? 
>> If not, where is the good starting point to review the source code to 
>> implement this feature?
>> 
>> BR,
>> Chunseok Lee
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] lldb command like gdb's "set auto-solib-add off"

[Jim Ingham via lldb-dev]

We designed lldb to be as lazy as possible in loading symbol information from 
the shared libraries as they get loaded.  If this is causing you problems, the 
first thing to do is to figure out why lldb is pulling in more information than 
you need.  For instance, it looks like on Linux the gdb JIT loading breakpoint 
is getting set with a global search which is pulling in all the symbols from 
all the initially loaded shared libraries.  It would be better to fix things so 
that only explicit user actions (which can be scoped to shared libraries to 
limit the search) pull in symbols so we don't have to fiddle around with 
turning off and on the loading of shared libraries.  Back when we were 
supporting gdb for MacOS I did a lot of work to try to get this right (there 
are times you really need the shared library info - e.g. when something shows 
up in a backtrace) so you have to judiciously re-introduce symbols, or the user 
experience is noticeably degraded.

It also depends on how far you want to go turning this off.  There's "don't 
look at shared libraries at all" which is what the auto-solib-add variable does 
IIRC.  We also had a fairly extensive mechanism to specify "load-levels" for 
various libraries, which  was more user-friendly but much more work to support. 
 Anyway, it would be easy to just turn this shared library notifications - just 
don't set the dyld load notification breakpoint.  That would be the easy part.  
But as I said above, you're also going to have to make sure you turn it back on 
for users when some action they request requires it.

Note, there is a setting to determine how much symbol information to read in 
from libraries loaded from memory (since in device debugging this can be quite 
slow.)  That is the target.memory-module-load-level setting.  You might look at 
that for some hints as to how to proceed.

Jim

> On May 22, 2017, at 8:02 AM, Chunseok Lee via lldb-dev 
>  wrote:
> 
> Hello. 
> 
> In gdb, I can toggle auto symbol loading using "set auto-solib-add off" 
> command.
> I wonder that lldb has similar command for disabling auto shared library 
> symbol loading? 
> If not, where is the good starting point to review the source code to 
> implement this feature?
> 
> BR,
> Chunseok Lee
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] lldb command like gdb's "set auto-solib-add off"

[Chunseok Lee via lldb-dev]

Hello. 

In gdb, I can toggle auto symbol loading using "set auto-solib-add off" command.
I wonder that lldb has similar command for disabling auto shared library symbol 
loading? 
If not, where is the good starting point to review the source code to implement 
this feature?

BR,
Chunseok Lee___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev