| Issue |
114701
|
| Summary |
LLDB: `DYLDRendezvous::GetAction()` found two `eAdd` states in a row
|
| Labels |
new issue
|
| Assignees |
|
| Reporter |
georglauterbach
|
> [!NOTE]
>
> I am by no means an LLDB (or LLVM, for that matter) expert. Please assume that I may be missing something obvious...
### Prerequisites
I am debugging a program with LLDB that loads many shared objects, more than 100. While the overwhelming majority works flawlessly, I am stuck at one point, and I can't seem to make any progress for two weeks now. I have tried every configuration that seemed to have made sense to me, and I am now here because there may be the possibility that there is something wrong with LLDB itself - possibly.
The code that I am working on closed-source (which is a shame, I know - I cannot do anything about that). But I will share as many details as I can so we can find the problem.
While collecting the information for this report, I uncovered information that I wasn't aware of either; maybe some of the log and information down below are actually the problem's solution?
### Environment information
I am debugging inside a container that runs Linux, more specifically: Yocto Linux (Kirkstone). This Linux has `glibc` 2.45 (no `musl`) and debug symbols should be available. The `glibc` source code is not available under `/usr/src/debug/glibc/2.35-r0/git`, though.
```console
$ /lib/libc.so.6
GNU C Library (GNU libc) stable release version 2.35.
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 11.3.0.
libc ABIs: UNIQUE IFUNC ABSOLUTE
For bug reporting instructions, please see:
<https://www.gnu.org/software/libc/bugs.html>.
```
The program is written in C++ and compiled with `g++`:
```console
$ g++ --version
g++ (GCC) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
```
The C++ standard used is mostly C++17, but for some libraries it could still be C++14 or C++11. (I am not a C++ expert; maybe this is bad?)
I am using the command line `lldb` that ships with [`vadimcn/codelldb`](https://github.com/vadimcn/codelldb):
```console
$ ./lldb --version
lldb version 19.1.0-custom
```
I have checked the target's environment, the debugger's environment, but changing the environment does not seem to make a difference here.
ASLR does not seem to make a difference either.
### Examining the Problem
The application uses `dlopen` to load shared objects. The call to `dlopen` that does not work has a backtrace that looks like this:
```txt
(lldb) thread backtrace
error: <ANOTHER SHARED LIBRARY LOADED EARLIER 1>.so 0x027572fd: DW_TAG_member '_M_local_buf' refers to type 0x0000000002866a2e which extends beyond the bounds of 0x027572f3
error: <YET ANOTHER SHARED LIBRARY LOADED EARLIER 2>.so.so 0x003d9ca2: DW_TAG_member '_M_local_buf' refers to type 0x00000000004713cf which extends beyond the bounds of 0x003d9c98
* thread #42, name = 'libref_adv_ld_d', stop reason = signal SIGSTOP
* frame #0: 0x00007ffff7fca110 ld-linux-x86-64.so.2`__GI__dl_debug_state at dl-debug.c:116:1
frame #1: 0x00007ffff7fcfa6c ld-linux-x86-64.so.2`_dl_map_object_from_fd(name=<unavailable>, origname=<unavailable>, fd=-1, fbp=<unavailable>, realname=<unavailable>, loader=<unavailable>, l_type=<unavailable>, mode=<unavailable>, stack_endp=<unavailable>, nsid=<unavailable>) at dl-load.c:1501:7
frame #2: 0x00007ffff7fd0488 ld-linux-x86-64.so.2`_dl_map_object(loader=0x0000000000000000, name="<BUILD DIR>/<PATH TO LIBRARY THAT DOES NOT WORK>/<LIBRARY THAT DOES NOT WOTRK>.so", type=2, trace_mode=0, mode=<unavailable>, nsid=<unavailable>) at dl-load.c:2309:10
frame #3: 0x00007ffff7fd4299 ld-linux-x86-64.so.2`dl_open_worker_begin(a=0x00007fff815fe740) at dl-open.c:534:21
frame #4: 0x00007ffff7062631 libc.so.6`__GI__dl_catch_exception(exception=<unavailable>, operate=<unavailable>, args=<unavailable>) at dl-error-skeleton.c:208:8
frame #5: 0x00007ffff7fd3a3a ld-linux-x86-64.so.2`dl_open_worker(a=0x00007fff815fe740) at dl-open.c:782:15
frame #6: 0x00007ffff7062631 libc.so.6`__GI__dl_catch_exception(exception=<unavailable>, operate=<unavailable>, args=<unavailable>) at dl-error-skeleton.c:208:8
frame #7: 0x00007ffff7fd3e1f ld-linux-x86-64.so.2`_dl_open(file=<unavailable>, mode=-2147483646, caller_dlopen=0x00007fffa4e1f3a1, nsid=-2, argc=5, argv=0x00007fffffffd5e8, env=0x00007fffffffd618) at dl-open.c:886:17
frame #8: 0x00007ffff6f9a5ac libc.so.6`dlopen_doit(a=0x00007fff815fe9b0) at dlopen.c:56:15
frame #9: 0x00007ffff7062631 libc.so.6`__GI__dl_catch_exception(exception=0x00007fff815fe910, operate=<unavailable>, args=<unavailable>) at dl-error-skeleton.c:208:8
frame #10: 0x00007ffff70626e3 libc.so.6`__GI__dl_catch_error(objname=0x00007fff815fe970, errstring=0x00007fff815fe978, mallocedp=0x00007fff815fe96f, operate=<unavailable>, args=<unavailable>) at dl-error-skeleton.c:227:19
frame #11: 0x00007ffff6f9a0ee libc.so.6`_dlerror_run(operate=<unavailable>, args=<unavailable>) at dlerror.c:138:17
frame #12: 0x00007ffff6f9a641 libc.so.6`___dlopen [inlined] dlopen_implementation(dl_caller=<unavailable>, mode=<unavailable>, file=<unavailable>) at dlopen.c:71:10
frame #13: 0x00007ffff6f9a620 libc.so.6`___dlopen(file=<unavailable>, mode=<unavailable>) at dlopen.c:81:12
frame #14: 0x00007fffa4e1f3a1 libsystem_router_d.so`RsCMW::Rpm::CSystemRouter::load_dll(this=0x0000555555e4c030, iDllName="<BUILD DIR>/<PATH TO LIBRARY THAT DOES NOT WORK>/<LIBRARY THAT DOES NOT WOTRK>.so") at system_router.cpp:545:21
```
When enabling additional logs via `log enable lldb dyld`, I see this:
```txt
<LOG FROM APPLICATION>
<LOADING A BUNCH OF SHARED LIBRARIES (more than 100) CORRECTLY>
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 50587
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe0d8
intern-state m_previous Rendezvous: version = 1, map_addr = 0x00007ffff7ffe2a0, brk = 0x00007ffff7fca110, state = 1 (eAdd), ldbase = 0x00007ffff7fc7000
intern-state m_current Rendezvous: version = 1, map_addr = 0x00007ffff7ffe2a0, brk = 0x00007ffff7fca110, state = 0 (eConsistent), ldbase = 0x00007ffff7fc7000
intern-state bool DYLDRendezvous::UpdateSOEntriesFromRemote() action = ""
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 50587 stop_when_images_change=false
<LOG FROM THE APPLICATION>
Process 50587 stopped and restarted: thread 1 received signal: SIGCHLD
<LOG FROM THE APPLICATION>
Process 50587 stopped and restarted: thread 1 received signal: SIGCHLD
<LOG FROM THE APPLICATION>
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 50587
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe0d8
intern-state m_previous Rendezvous: version = 1, map_addr = 0x00007ffff7ffe2a0, brk = 0x00007ffff7fca110, state = 0 (eConsistent), ldbase = 0x00007ffff7fc7000
intern-state m_current Rendezvous: version = 1, map_addr = 0x00007ffff7ffe2a0, brk = 0x00007ffff7fca110, state = 1 (eAdd), ldbase = 0x00007ffff7fc7000
intern-state bool DYLDRendezvous::UpdateSOEntriesFromRemote() action = ""
intern-state bool DYLDRendezvous::UpdateSOEntries() action = ""
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 50587 stop_when_images_change=false
<AND THEN REPEAT FOREVER:>
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 50587
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe0d8
intern-state m_previous Rendezvous: version = 1, map_addr = 0x00007ffff7ffe2a0, brk = 0x00007ffff7fca110, state = 1 (eAdd), ldbase = 0x00007ffff7fc7000
intern-state m_current Rendezvous: version = 1, map_addr = 0x00007ffff7ffe2a0, brk = 0x00007ffff7fca110, state = 1 (eAdd), ldbase = 0x00007ffff7fc7000
intern-state DYLDRendezvous::GetAction() found two eAdd states in a row, check process for multiple "_r_debug" symbols. Returning eAddModules to ensure shared libraries get loaded correctly
intern-state bool DYLDRendezvous::UpdateSOEntriesFromRemote() action = ""
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 50587 stop_when_images_change=false
```
---
I am really out of ideas at this point. Any help is greatly appreciated!
![]()
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
