Re: [lldb-dev] Removing linux mips support

[Adrian Prantl via lldb-dev]

Sounds very reasonable to me.

thanks for bringing this up!
-- adrian

> On Mar 9, 2021, at 12:24 AM, Pavel Labath via lldb-dev 
>  wrote:
> 
> Hi all,
> 
> I propose to remove support for linux mips debugging. This basically amounts 
> to deleting 
> source/Plugins/Process/Linux/NativeRegisterContextLinux_mips64.{cpp,h}. My 
> reasons for doing that are:
> 
> - This code is unmaintained (last non-mechanical change was in 2017) and 
> untested (no public buildbots), so we don't know if even basic functionality 
> works, or if it indeed builds.
> 
> - At the same, it is carrying a lot of technical debt, which is leaking out 
> of the mips-specific files, and interfering with other development efforts. 
> The last instance of this is D96766, which is adding FreeBSD mips support, 
> but needs to work around linux specific knowledge leaking into supposedly 
> generic code. This one should be fixable relatively easily (these days we 
> already have precedents for similar things in x86 and arm code), but it needs 
> someone who is willing to do that.
> 
> But that is not all. To support mips, we introduced two new fields into the 
> RegisterInfo struct (dynamic_size_dwarf_{expr_bytes,len}). These are 
> introducing a lot of clutter in all our RegisterInfo definitions (which we 
> have **a lot** of) and are not really consistent with the long term vision of 
> the gdb-remote protocol usage in lldb. These days, we have a different 
> mechanism for this (added to support a similar feature in arm), it would be 
> better to implement this feature in terms of that. I would tout this (removal 
> of these fields) as the main benefit of dropping mips support.
> 
> So, unless someone willing to address these issues (I'm happy to provide 
> support where I can), I propose we drop mips support. Generic mips support 
> will remain (and hopefully be better tested) thanks to the FreeBSD mips port, 
> so re-adding mips support should be a matter of reimplementing the linux bits.
> 
> regards,
> Pavel
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] [Job Ad] Apple's debugging technologies team is hiring

[Adrian Prantl via lldb-dev]

The debugging technologies team at Apple is looking for engineers to work on 
LLDB and its testing and delivery. This is an opportunity to join a team that 
combines compiler technology and debuggers under one umbrella to build great 
tools. LLDB is a core part of Apple's developer tools, used internally to debug 
Apple's software stack and externally by millions of developers.


Debugger Engineer
=

We are looking for an engineer to work on the Swift support in the LLDB 
debugger. In this position, you will work closely with the Swift compiler team 
to improve the Swift debugging experience. Most of your work will be open 
source, on llvm.org and GitHub where you’ll collaborate with the LLDB and Swift 
communities to better integrate Swift support with the generic debugger code.

You can find a more detailed description at:
https://jobs.apple.com/en-us/details/200164500/debugger-engineer


Debugger Qualification Engineer
===

We are also looking for an engineer to work on the delivery and testing of the 
LLDB debugger. In this position, you will be in charge of the LLDB debugger’s 
testing and delivery for all Apple platforms.

For a more detailed description go to:
https://jobs.apple.com/en-us/details/200164501/debugger-qualification-engineer



If you are interested, please send your resume to 
llvm-job-ap...@group.apple.com. If you have any further questions, feel free to 
reach out to me at apra...@apple.com.

cheers,
Adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] LLDB Website is not being updated

[Adrian Prantl via lldb-dev]

Ah, that's great! We should probably just don't print a version on this variant 
and/or add a box like the one at the top of http://llvm.org/docs/ 
<http://llvm.org/docs/>.

thanks,
adrian

> On Mar 2, 2020, at 1:43 PM, Jonas Devlieghere  wrote:
> 
> Hey Adrian,
> 
> The version is hard-coded in lldb/docs/conf.py, we just need to update it 
> there. As far as I know the website is being updated nightly.
> 
> Cheers,
> Jonas
> 
> On Mon, Mar 2, 2020 at 1:41 PM Adrian Prantl  <mailto:apra...@apple.com>> wrote:
> Hello Tanya,
> 
> I just looked at the LLDB website and it still displays the out-of-date 
> "version 8" page. Did you get a chance to investigate this in the mean time?
> 
> thanks,
> adrian
> 
> > On Nov 24, 2019, at 8:30 AM, Tanya Lattner  > <mailto:tanyalatt...@llvm.org>> wrote:
> > 
> > I’ll have to check the status of moving the scripts over and what is going 
> > on. But yes, this is all related to moving to GitHub and modifying scripts 
> > that used to be either on a post-commit hook or nightly cron.
> > 
> > -Tanya
> > 
> >> On Nov 21, 2019, at 3:04 PM, Jonas Devlieghere  >> <mailto:jo...@devlieghere.com>> wrote:
> >> 
> >> I see a bunch of eros here:
> >> http://lists.llvm.org/pipermail/www-scripts/2019-November/thread.html 
> >> <http://lists.llvm.org/pipermail/www-scripts/2019-November/thread.html>
> >> 
> >> Is this possibly related to the Github/monorepo transition?
> >> 
> >> -- Jonas
> >> 
> >> On Thu, Nov 21, 2019 at 10:52 AM Adrian Prantl via lldb-dev
> >> mailto:lldb-dev@lists.llvm.org>> wrote:
> >>> 
> >>> Hello Tanya,
> >>> 
> >>> it looks like the cron job that is supposed to be updating the LLDB 
> >>> website isn't running or is otherwise blocked at the moment. You can see 
> >>> on https://lldb.llvm.org <https://lldb.llvm.org/> that it says "Welcome 
> >>> to the LLDB version 8 documentation!". We also recently removed the "Why 
> >>> a New Debugger?" headline and that change isn't showing up either.
> >>> 
> >>> Would you mind taking a look?
> >>> 
> >>> thanks for your help,
> >>> Adrian
> >>> ___
> >>> lldb-dev mailing list
> >>> lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org>
> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
> >>> <https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev>
> > 
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] LLDB Website is not being updated

[Adrian Prantl via lldb-dev]

Hello Tanya,

I just looked at the LLDB website and it still displays the out-of-date 
"version 8" page. Did you get a chance to investigate this in the mean time?

thanks,
adrian

> On Nov 24, 2019, at 8:30 AM, Tanya Lattner  wrote:
> 
> I’ll have to check the status of moving the scripts over and what is going 
> on. But yes, this is all related to moving to GitHub and modifying scripts 
> that used to be either on a post-commit hook or nightly cron.
> 
> -Tanya
> 
>> On Nov 21, 2019, at 3:04 PM, Jonas Devlieghere  wrote:
>> 
>> I see a bunch of eros here:
>> http://lists.llvm.org/pipermail/www-scripts/2019-November/thread.html
>> 
>> Is this possibly related to the Github/monorepo transition?
>> 
>> -- Jonas
>> 
>> On Thu, Nov 21, 2019 at 10:52 AM Adrian Prantl via lldb-dev
>>  wrote:
>>> 
>>> Hello Tanya,
>>> 
>>> it looks like the cron job that is supposed to be updating the LLDB website 
>>> isn't running or is otherwise blocked at the moment. You can see on 
>>> https://lldb.llvm.org that it says "Welcome to the LLDB version 8 
>>> documentation!". We also recently removed the "Why a New Debugger?" 
>>> headline and that change isn't showing up either.
>>> 
>>> Would you mind taking a look?
>>> 
>>> thanks for your help,
>>> Adrian
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Is bitcast breaking lldb a bug?

[Adrian Prantl via lldb-dev]

Does this still reproduce with lldb compiled from the current state of the git 
repository (ToT)?

How do you know that it is LLDB loosing the variable and not clang? Does clang 
produce a location for the variable when you look at the dwarfdump output?

-- adrian

> On Feb 26, 2020, at 3:31 AM, Levo DeLellis via lldb-dev 
>  wrote:
> 
> This feels like a bug to me. Yesterday I was asking what the rules were 
> because it felt like things change and break randomly. Now I have a good 
> example. (link to my email yesterday 
> http://lists.llvm.org/pipermail/lldb-dev/2020-February/015989.html 
> )
> 
> Take this example source file
> 
> int main() {
> int dummy = 25;
> short wtf[dummy];
> memset(wtf, 0, dummy*sizeof(*wtf));
> return 0;
> }
> 
> Now emit the llvm-ir so we can edit it 
> 
> clang -g test.c -S -emit-llvm
> 
> Before line 21 write this line
> 
> %z8 = bitcast i16* %8 to i16*
> 
> Change the `metadata i16* %8` to `metadata i16* %z8`. Compile it then debug 
> line 4 `clang -g wtf.ll` `lldb-9 ./a.out` `break set -f test.c -l 4` `r` 
> `frame variable`
> 
> You'll see the array doesn't show up. If you change %z8 back to %8 it will 
> reappear. Is this a bug or can I not use bitcast when I'm trying to do things 
> with llvm.dbg.declare/addr/value?
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] What are the 'rules' to play nice with lldb-9?

[Adrian Prantl via lldb-dev]

> On Feb 25, 2020, at 5:29 PM, Levo DeLellis via lldb-dev 
>  wrote:
> 
> I'm thoroughly confused so I may say incorrect thing. I don't expect any 
> replies to this entire post but bits would be helpful. I'll double check if 
> you need me to be specific since it's likely I remember or ran something 
> wrong. For my language I output llvm-ir directly so I may generate llvm-ir 
> that the llvm api does not and I may create very specific ir if I need to.
> 
> I noticed a number of things. When I declare something with llvm.dbg.declare 
> it doesn't happen where I declare it but at the start of the function? I put 
> my alloca's at the start of the function like llvm recommends so I have 
> invalid pointers and invalid array sizes which causes vscode to crash. Which 
> is fine I'm sure I can write null and say the size is 0 before the first line 
> of the function.

These are really LLVM questions. Basically:

- if you can see the variable with dwarfdump and it doesn't show up in LLDB, 
you should ask on lldb-dev
- if you can't see the variable with dwarfdump, you should ask on llvm-dev

A golden rule of thumb is to look at what clang is doing and copy that.

The location of a dbg.declare is mostly irrelevant, since there can only be one 
dbg.declare per variable and it is valid throughout the entire scope.

> I thought maybe I can use llvm.dbg.addr to work around the problem but I 
> never got it working in the way I hoped (Does it execute at the start of the 
> function? which explains why my unexpected results).

I don't understand the question here. It would be helpful to post a minimal 
example, and look at the output of -print-after-all to see how it is 
transformed by the various passes, so you can ask a more directed question.

> I manage to have some good results with llvm.dbg.value. I noticed if I put a 
> value in a local variable and use `llvm.dbg.value(i64 %abc` there's a high 
> likelihood it will have an incorrect value or will be wrong once I step 
> through more code. However i64 1234 always seem to work.
> 
> How should I tell the debugger about my variables? This page says you are 
> "transitioning away from" llvm.dbg.declare 
> https://llvm.org/docs/SourceLevelDebugging.html I see llvm.dbg.addr says to 
> only use it once and I had some luck with llvm.dbg.value. How do I decide 
> when to use llvm.dbg.value vs llvm.dbg.addr? May I use llvm.dbg.value on a 
> variable that I didn't specific with llvm.dbg.addr? (for example a const 
> value in a variable that has no address)? What about an array passed in from 
> a function? Do I need to store the pointer and or length to a variable to 
> give it an address before the debugger can understand it? 

If your variables "live" on the stack, such as in clang -O0 code, you should 
use one dbg.declare per variable. Otherwise you'll need at least one dbg.value 
per SSA value that carries a variable value.

> 
> Is it possible to say ignore my variable until I hit this specific?
> My language uses constructors so I often have to execute code before pointers 
> become valid.

Yes, dbg.value(undef, ...) followed by another dbg.value will do that 
explicitly.

You can not achieve this with dbg.declare because it is global.

> The language also cleans up so on a return statement as a 'hack' I execute 
> `br false, label %dummy123, label %dummy123, !dbg !123\ndummy123:\n` so the 
> user can see the variables before the cleanup happens/pointers get freed. I 
> mentioned earlier when using vscode having invalid array pointer and invalid 
> size (very large number) made vscode crash. I'm not sure what happened but I 
> had lldb-mi-9 take up many gb (30gb+) in some situations so it may not be a 
> literal crash in the process but locks up and do undesirable behavior
> 
> I don't expect to get all the answers but maybe a piece here and there will 
> help. Should I continue to use llvm.dbg.declare for now with lldb 9? or 
> should I use llvm.dbg.addr and llvm.dbg.value?

Depends on the kind of code you generate.

> Should I always initialize my variables to 0? (so I don't crash vscode when 
> it uses lldb-mi-9) Is there a way I can say ignore this variable until I hit 
> line X where I know the memory will be initialized fully? Do I need to worry 
> about the order I initialize variables or the order I call llvm.dbg.addr? (I 
> remember, call llvm.dbg.addr once per variable)
> 
> Thank you for reading and thank you for any help
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Have the debugger show an away with a dynamic size?

[Adrian Prantl via lldb-dev]

I added the VLA support to clang and lldb about a year ago, so you'll need 
fairly recent version of both for it to work.

-- adrian

> On Feb 17, 2020, at 12:25 PM, Levo DeLellis  wrote:
> 
> It looks like I wasn't careful and mixed version. I compiled with clang-9 but 
> used lldb-6. Surprisingly this was the only error I notice when mixing these 
> version. I could swear I tried compiling with clang-6. I'd double check but 
> it appears that installing lldb-9 removed lldb(-6) from my system
> Thanks for pointing me in the right direction
> 
> On Mon, Feb 17, 2020 at 11:18 AM Adrian Prantl  > wrote:
> That is interesting. According to LLDB's test/lang/c/vla/* frame variable for 
> a VLA is supposed to work. Frame variable is also supposed to hide the 
> __vla_expr0 artificial helper variable. Is this an older LLDB from your 
> system or an LLDB you built from source? If yes, would you mind filing a 
> bugreport about this?
> 
> thanks,
> adrian
> 
> > On Feb 15, 2020, at 8:17 AM, Levo DeLellis  > > wrote:
> > 
> > Thanks for the suggestions but it doesn't appear to be working correctly 
> > for me. I tried building the below after seeing the results with "clang -g 
> > -std=c99 test.c" and got the same result
> > 
> > LLDB thinks MyArray is 81 elements long even though 81 and 80 doesn't show 
> > up anywhere in the llvm-ir (I tried again using an llvm ir file made by 
> > clang -g -std=c99 test.c -S -emit-llvm and clang -g test.ll)
> > 
> > $ cat test.c
> > int foo(int s) {
> > int MyArray[s];
> > int i;
> > for (i = 0; i < s; ++i)
> > MyArray[i] = s;
> > return 0;
> > }
> > 
> > int main(){
> > foo(5);
> > return 0;
> > }
> > $ clang -g test.c 
> > $ lldb ./a.out 
> > (lldb) target create "./a.out"
> > Current executable set to './a.out' (x86_64).
> > (lldb) break set -f test.c -l 6
> > Breakpoint 1: where = a.out`foo + 101 at test.c:7, address = 
> > 0x00400505
> > (lldb) r
> > Process 3205 launched: './a.out' (x86_64)
> > Process 3205 stopped
> > * thread #1, name = 'a.out', stop reason = breakpoint 1.1
> > frame #0: 0x00400505 a.out`foo(s=5) at test.c:7
> >4  for (i = 0; i < s; ++i)
> >5  MyArray[i] = s;
> >6  return 0;
> > -> 7   }
> >8   
> >9   int main(){
> >10  foo(5);
> > (lldb) frame variable
> > (int) s = 5
> > (unsigned long) __vla_expr0 = 5
> > (int) i = 5
> > (int [81]) MyArray = {
> >   [0] = 5
> >   [1] = 5
> >   [2] = 5
> >   [3] = 5
> >   [4] = 5
> >   [5] = 0
> >   [6] = -136481184
> >   [7] = 32767
> >   [8] = -8408
> >   [9] = 32767
> >   [10] = -8544
> >   [11] = 32767
> >   [12] = 1
> >   [13] = 5
> >   [14] = 5
> >   [15] = 0
> >   [16] = -8512
> >   [17] = 32767
> >   [18] = 0
> >   [19] = 5
> >   [20] = -8432
> >   [21] = 32767
> >   [22] = 4195641
> >   [23] = 0
> >   [24] = -8208
> >   [25] = 32767
> >   [26] = 0
> >   [27] = 0
> >   [28] = 4195664
> >   [29] = 0
> >   [30] = -140485737
> >   [31] = 32767
> >   [32] = 0
> >   [33] = 32
> >   [34] = -8200
> >   [35] = 32767
> >   [36] = 0
> >   [37] = 1
> >   [38] = 4195616
> >   [39] = 0
> >   [40] = 0
> >   [41] = 0
> >   [42] = -1953144313
> >   [43] = 1284291557
> >   [44] = 4195248
> >   [45] = 0
> >   [46] = -8208
> >   [47] = 32767
> >   [48] = 0
> >   [49] = 0
> >   [50] = 0
> >   [51] = 0
> >   [52] = 1064657415
> >   [53] = -1284291430
> >   [54] = 933978631
> >   [55] = -1284287451
> >   [56] = 0
> >   [57] = 32767
> >   [58] = 0
> >   [59] = 0
> >   [60] = 0
> >   [61] = 0
> >   [62] = -136423629
> >   [63] = 32767
> >   [64] = -136530376
> >   [65] = 32767
> >   [66] = 386784
> >   [67] = 0
> >   [68] = 0
> >   [69] = 0
> >   [70] = 0
> >   [71] = 0
> >   [72] = 0
> >   [73] = 0
> >   [74] = 4195248
> >   [75] = 0
> >   [76] = -8208
> >   [77] = 32767
> >   [78] = 4195290
> >   [79] = 0
> >   [80] = -8216
> > }
> > 
> > 
> > On Thu, Feb 13, 2020 at 3:53 PM Adrian Prantl  > > wrote:
> > Take a look at the IR clang produces for C99 variable-length arrays.
> > 
> > -- adrian
> > 
> >> On Feb 13, 2020, at 10:03 AM, Levo DeLellis via llvm-dev 
> >> mailto:llvm-...@lists.llvm.org>> wrote:
> >> 
> >> Hi. I searched and the closest thing I could find was this 
> >> http://lists.llvm.org/pipermail/llvm-dev/2018-February/121348.html 
> >> 
> >> 
> >> Currently a known sized array looks and debugs as expected. I use 
> >> llvm.dbg.declare with DICompositeType tag: DW_TAG_array_type and the size 
> >> field. In my language arrays are always passed around with a pointer and 
> >> size pair. I'd like debugging to show up as nicely instead of a pointer 
> >> addr with no information about the elements. How would I do this? I don't 
> >> use the C API, I output llvm-ir directly. I was hoping I can call 
> >> llvm.dbg.declare/addr/value to specify the pointer, name and size of the 
> >> 

Re: [lldb-dev] [llvm-dev] Have the debugger show an away with a dynamic size?

[Adrian Prantl via lldb-dev]

That is interesting. According to LLDB's test/lang/c/vla/* frame variable for a 
VLA is supposed to work. Frame variable is also supposed to hide the 
__vla_expr0 artificial helper variable. Is this an older LLDB from your system 
or an LLDB you built from source? If yes, would you mind filing a bugreport 
about this?

thanks,
adrian

> On Feb 15, 2020, at 8:17 AM, Levo DeLellis  wrote:
> 
> Thanks for the suggestions but it doesn't appear to be working correctly for 
> me. I tried building the below after seeing the results with "clang -g 
> -std=c99 test.c" and got the same result
> 
> LLDB thinks MyArray is 81 elements long even though 81 and 80 doesn't show up 
> anywhere in the llvm-ir (I tried again using an llvm ir file made by clang -g 
> -std=c99 test.c -S -emit-llvm and clang -g test.ll)
> 
> $ cat test.c
> int foo(int s) {
> int MyArray[s];
> int i;
> for (i = 0; i < s; ++i)
> MyArray[i] = s;
> return 0;
> }
> 
> int main(){
> foo(5);
> return 0;
> }
> $ clang -g test.c 
> $ lldb ./a.out 
> (lldb) target create "./a.out"
> Current executable set to './a.out' (x86_64).
> (lldb) break set -f test.c -l 6
> Breakpoint 1: where = a.out`foo + 101 at test.c:7, address = 
> 0x00400505
> (lldb) r
> Process 3205 launched: './a.out' (x86_64)
> Process 3205 stopped
> * thread #1, name = 'a.out', stop reason = breakpoint 1.1
> frame #0: 0x00400505 a.out`foo(s=5) at test.c:7
>4  for (i = 0; i < s; ++i)
>5  MyArray[i] = s;
>6  return 0;
> -> 7   }
>8   
>9   int main(){
>10  foo(5);
> (lldb) frame variable
> (int) s = 5
> (unsigned long) __vla_expr0 = 5
> (int) i = 5
> (int [81]) MyArray = {
>   [0] = 5
>   [1] = 5
>   [2] = 5
>   [3] = 5
>   [4] = 5
>   [5] = 0
>   [6] = -136481184
>   [7] = 32767
>   [8] = -8408
>   [9] = 32767
>   [10] = -8544
>   [11] = 32767
>   [12] = 1
>   [13] = 5
>   [14] = 5
>   [15] = 0
>   [16] = -8512
>   [17] = 32767
>   [18] = 0
>   [19] = 5
>   [20] = -8432
>   [21] = 32767
>   [22] = 4195641
>   [23] = 0
>   [24] = -8208
>   [25] = 32767
>   [26] = 0
>   [27] = 0
>   [28] = 4195664
>   [29] = 0
>   [30] = -140485737
>   [31] = 32767
>   [32] = 0
>   [33] = 32
>   [34] = -8200
>   [35] = 32767
>   [36] = 0
>   [37] = 1
>   [38] = 4195616
>   [39] = 0
>   [40] = 0
>   [41] = 0
>   [42] = -1953144313
>   [43] = 1284291557
>   [44] = 4195248
>   [45] = 0
>   [46] = -8208
>   [47] = 32767
>   [48] = 0
>   [49] = 0
>   [50] = 0
>   [51] = 0
>   [52] = 1064657415
>   [53] = -1284291430
>   [54] = 933978631
>   [55] = -1284287451
>   [56] = 0
>   [57] = 32767
>   [58] = 0
>   [59] = 0
>   [60] = 0
>   [61] = 0
>   [62] = -136423629
>   [63] = 32767
>   [64] = -136530376
>   [65] = 32767
>   [66] = 386784
>   [67] = 0
>   [68] = 0
>   [69] = 0
>   [70] = 0
>   [71] = 0
>   [72] = 0
>   [73] = 0
>   [74] = 4195248
>   [75] = 0
>   [76] = -8208
>   [77] = 32767
>   [78] = 4195290
>   [79] = 0
>   [80] = -8216
> }
> 
> 
> On Thu, Feb 13, 2020 at 3:53 PM Adrian Prantl  wrote:
> Take a look at the IR clang produces for C99 variable-length arrays.
> 
> -- adrian
> 
>> On Feb 13, 2020, at 10:03 AM, Levo DeLellis via llvm-dev 
>>  wrote:
>> 
>> Hi. I searched and the closest thing I could find was this 
>> http://lists.llvm.org/pipermail/llvm-dev/2018-February/121348.html
>> 
>> Currently a known sized array looks and debugs as expected. I use 
>> llvm.dbg.declare with DICompositeType tag: DW_TAG_array_type and the size 
>> field. In my language arrays are always passed around with a pointer and 
>> size pair. I'd like debugging to show up as nicely instead of a pointer addr 
>> with no information about the elements. How would I do this? I don't use the 
>> C API, I output llvm-ir directly. I was hoping I can call 
>> llvm.dbg.declare/addr/value to specify the pointer, name and size of the 
>> variable but I really have no idea how to pass the size to the debugger.
>> 
>> ___
>> LLVM Developers mailing list
>> llvm-...@lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] How/where to add test cases in LLDB.

[Adrian Prantl via lldb-dev]

Please note that when you are testing for a DWARF 5 feature specifically, you 
should think about using either Assembler or yaml2obj as input to lock down the 
binary encoding. If you do this, it would be best to not require a living 
process in your test, so the test can run even on platforms that cannot run 
that code. In these cases lldb-test is often useful.

Alternatively/additionally, you can write an end-to-end test that compiles from 
source; these are the python-based tests found in packages/.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Optional Dependencies in LLDB

[Adrian Prantl via lldb-dev]

> On Jan 6, 2020, at 11:17 AM, Jonas Devlieghere via lldb-dev 
>  wrote:
> 
> Hey everyone,
> 
> I just wanted to let you know that most of the work is complete for
> auto-detecting optional dependencies in LLDB. Unless explicitly
> specified, optional dependencies like editline will be enabled when
> available and disabled otherwise. 

This "explicitly specified" mode makes it possible to declare that I want it to 
be hard error if an optional dependency is missing (e.g., to avoid silently 
dropping editline support by accident)?

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] LLDB Website is not being updated

[Adrian Prantl via lldb-dev]

Hello Tanya,

it looks like the cron job that is supposed to be updating the LLDB website 
isn't running or is otherwise blocked at the moment. You can see on 
https://lldb.llvm.org that it says "Welcome to the LLDB version 8 
documentation!". We also recently removed the "Why a New Debugger?" headline 
and that change isn't showing up either.

Would you mind taking a look?

thanks for your help,
Adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Hiding trampoline functions from the backtrace, is it possible ?

[Adrian Prantl via lldb-dev]

I think the best mechanism for this would be to ensure that the trampolines are 
marked up as DW_AT_artificial and/or DW_AT_trampoline by the compiler. I'm 
pretty sure LLDB then already knows how to hide artificial frames (somebody 
else can probably provide pointers for how that works).

-- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] lldb access in Emacs via realgud

[Adrian Prantl via lldb-dev]

> On Sep 12, 2019, at 4:45 PM, Rocky Bernstein  wrote:
> 
> 
> 
> On Thu, Sep 12, 2019 at 5:51 PM Adrian Prantl  > wrote:
> 
> 
> > On Sep 12, 2019, at 2:24 PM, Rocky Bernstein  > > wrote:
> > 
> > 
> > 
> > On Thu, Sep 12, 2019 at 3:37 PM Adrian Prantl  > > wrote:
> >> 
> >> > On Sep 11, 2019, at 1:08 AM, Rocky Bernstein via lldb-dev 
> >> > mailto:lldb-dev@lists.llvm.org>> wrote:
> >> > 
> >> > 
> >> > Hi - I just wanted to mention that if there are emacs users there is an 
> >> > interface to lldb via realgud. See 
> >> > https://github.com/realgud/realgud-lldb 
> >> > 
> >> > 
> >> > A MELPA package and ELPA packageElpa package are available too 
> >> 
> >> Nice. It's always exciting to see wider adoption through better editor 
> >> integration. Out of curiosity, how does this compare to regular gud.el? 
> >> (https://opensource.apple.com/source/lldb/lldb-69/utils/emacs/gud.el.auto.html
> >>  
> >> )
> > 
> > "Regular" gud? The most recent copyright on that link is 2008. I see a 
> > gud.el in 26.2 and in the GNU savannah git sources, but neither mentions 
> > lldb. Assuming that file is really from 2008, has lldb changed since then? 
> > (This is a rhetorical question).  But the broader question is really who is 
> > maintaining that file you link, clearly it is not the GNU Emacs community. 
> > And how easy is it to do so? I see an "arch" tag on the file, so I guess 
> > this in version control somewhere. But if there is a bug in this file, what 
> > does one do? (This is not a rhetorical question; if you know the answer, I 
> > am interested.)
> 
> I think this file is effectively abandoned as it is neither part of the LLDB 
> repository nor is it shipping with emacs in macOS any longer.
> 
> > Adapted from https://github.com/realgud/realgud/blob/master/realgud.el 
> > 
> > 
> >> Here we make use of more modern programming practices, more numerous and 
> >> smaller files, unit tests, and better use of Emacs primitives, e.g. buffer 
> >> marks, buffer-local variables, structures, rings, hash tables. Although 
> >> there is still much to be desired, this code is more scalable and suitable 
> >> as a common base for an Emacs front-end to modern debuggers.
> >> Oh, and because global variables are largely banned, we can support 
> >> several simultaneous debug sessions.
> 
> > gdb-mi has a nicer multi-frame display,  but you were linking to gud.el 
> > which doesn't have that as far as I know. realgud-lldb at this point 
> > probably knows more about lldb. But you guys can probably verify that, and 
> > if realgud-lldb is lacking, I'd be interested to learn what should be 
> > added. 
> > 
> > gud has always been a bit too monolithic - it contains every debugger 
> > (including some obsolete ones - does anyone really still use dbx, and if so 
> > inside Emacs?). All of this is in that one several-thousand-line file. I 
> > find this ironic because the principal author is wrote something about 
> > "Cathedral versus Bazaar" and this is clearly Cathedral style. 
> > 
> > It took me quite a while to be able to break realgud into several distinct 
> > files. In fact I had to write my own nodejs-like "require" package to be 
> > able to do internal or relative module linking. And then after that, more 
> > work was done to split off the debuggers from the core debugger module into 
> > separate github projects.
> 
> Thanks for explaining the differences!
> 
> Is realgud scraping lldb's console output like gud was or is it using the 
> LLDB scripting API? If it isn't already you might want to consider using the 
> scripting API since LLDB's console output is not necessarily stable, but the 
> scripting API is.
> 
> Yes, it scrapes console output and that is a problem. A big problem.
> 
> I looked at the LLDB API and that seems pretty extensive and seems to cover a 
> lot more of what interaction from Emacs would like and is currently missing.
> 
> However as is in its current state, this isn't going to cut it because Emacs 
> uses its own Lisp, not Python.
> 
> I have toyed with the idea of hooking into something more standard, and as I 
> said, there are so many to choose from. For example, the V8 debugger protcol 
> works over a websocket,  and speaking over a websocket is something you can 
> expect IDEs to grok.  Python LLDB's API to say Microsoft's Debug Adaptor 
> Protocol  makes sense, 
> and https://github.com/vadimcn/vscode-lldb/blob/v1.3.0/MANUAL.md 
>  seems to get 
> pretty close to that. 
> 

[Adding Greg to the conversation]

What's the relation between that project and 

Re: [lldb-dev] lldb access in Emacs via realgud

[Adrian Prantl via lldb-dev]

> On Sep 12, 2019, at 2:24 PM, Rocky Bernstein  wrote:
> 
> 
> 
> On Thu, Sep 12, 2019 at 3:37 PM Adrian Prantl  wrote:
>> 
>> > On Sep 11, 2019, at 1:08 AM, Rocky Bernstein via lldb-dev 
>> >  wrote:
>> > 
>> > 
>> > Hi - I just wanted to mention that if there are emacs users there is an 
>> > interface to lldb via realgud. See https://github.com/realgud/realgud-lldb
>> > 
>> > A MELPA package and ELPA packageElpa package are available too 
>> 
>> Nice. It's always exciting to see wider adoption through better editor 
>> integration. Out of curiosity, how does this compare to regular gud.el? 
>> (https://opensource.apple.com/source/lldb/lldb-69/utils/emacs/gud.el.auto.html)
> 
> "Regular" gud? The most recent copyright on that link is 2008. I see a gud.el 
> in 26.2 and in the GNU savannah git sources, but neither mentions lldb. 
> Assuming that file is really from 2008, has lldb changed since then? (This is 
> a rhetorical question).  But the broader question is really who is 
> maintaining that file you link, clearly it is not the GNU Emacs community. 
> And how easy is it to do so? I see an "arch" tag on the file, so I guess this 
> in version control somewhere. But if there is a bug in this file, what does 
> one do? (This is not a rhetorical question; if you know the answer, I am 
> interested.)

I think this file is effectively abandoned as it is neither part of the LLDB 
repository nor is it shipping with emacs in macOS any longer.

> Adapted from https://github.com/realgud/realgud/blob/master/realgud.el
> 
>> Here we make use of more modern programming practices, more numerous and 
>> smaller files, unit tests, and better use of Emacs primitives, e.g. buffer 
>> marks, buffer-local variables, structures, rings, hash tables. Although 
>> there is still much to be desired, this code is more scalable and suitable 
>> as a common base for an Emacs front-end to modern debuggers.
>> Oh, and because global variables are largely banned, we can support several 
>> simultaneous debug sessions.

> gdb-mi has a nicer multi-frame display,  but you were linking to gud.el which 
> doesn't have that as far as I know. realgud-lldb at this point probably knows 
> more about lldb. But you guys can probably verify that, and if realgud-lldb 
> is lacking, I'd be interested to learn what should be added. 
> 
> gud has always been a bit too monolithic - it contains every debugger 
> (including some obsolete ones - does anyone really still use dbx, and if so 
> inside Emacs?). All of this is in that one several-thousand-line file. I find 
> this ironic because the principal author is wrote something about "Cathedral 
> versus Bazaar" and this is clearly Cathedral style. 
> 
> It took me quite a while to be able to break realgud into several distinct 
> files. In fact I had to write my own nodejs-like "require" package to be able 
> to do internal or relative module linking. And then after that, more work was 
> done to split off the debuggers from the core debugger module into separate 
> github projects.

Thanks for explaining the differences!

Is realgud scraping lldb's console output like gud was or is it using the LLDB 
scripting API? If it isn't already you might want to consider using the 
scripting API since LLDB's console output is not necessarily stable, but the 
scripting API is.

>> > 
>> > A question: what ever became of the effort to port the Emacs gdb-mi to 
>> > lldb? 
>> 
>> We recently removed lldb-mi from the LLDB repository because nobody in the 
>> community was willing to maintain it. In particular the tests were so 
>> unreliable that most bots disabled them wholesale because they were so 
>> noisy. We had a GSoC student a year ago who was able to rewrite many of the 
>> tests in a more reliable fashion, but there were still a lot of issues 
>> outstanding after the project was completed. If you are interested in 
>> picking this up, it may be worthwhile to think about implementing lldb-mi 
>> 2.0 as thin python layer using the python SBAPI. Python may be a better 
>> choice for the kind of text-heavy glue-code that lldb-mi is. Alternatively 
>> it also shouldn't be hard at all to revive the existing C++ code. It's 
>> written in a different style than most of LLDB or LLVM (and IMO it should 
>> have never been accepted upstream in this form), but it shouldn't be hard to 
>> get building since it (thanks to the GSoC project!) is using only the stable 
>> public SBAPI.
> 
> 
> The great thing about standards is that there are so many to choose from!  

It sounds like you are very much invested in realgud, but if anyone else is 
reading this and interested in taking up maintainership for lldb-mi, I think 
we'd be happy to welcome it back in tree as long as it is 100% reliably(!) 
tested and maintained.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] lldb access in Emacs via realgud

[Adrian Prantl via lldb-dev]

> On Sep 11, 2019, at 1:08 AM, Rocky Bernstein via lldb-dev 
>  wrote:
> 
> 
> Hi - I just wanted to mention that if there are emacs users there is an 
> interface to lldb via realgud. See https://github.com/realgud/realgud-lldb
> 
> A MELPA package and ELPA packageElpa package are available too 

Nice. It's always exciting to see wider adoption through better editor 
integration. Out of curiosity, how does this compare to regular gud.el? 
(https://opensource.apple.com/source/lldb/lldb-69/utils/emacs/gud.el.auto.html)

> 
> A question: what ever became of the effort to port the Emacs gdb-mi to lldb? 

We recently removed lldb-mi from the LLDB repository because nobody in the 
community was willing to maintain it. In particular the tests were so 
unreliable that most bots disabled them wholesale because they were so noisy. 
We had a GSoC student a year ago who was able to rewrite many of the tests in a 
more reliable fashion, but there were still a lot of issues outstanding after 
the project was completed. If you are interested in picking this up, it may be 
worthwhile to think about implementing lldb-mi 2.0 as thin python layer using 
the python SBAPI. Python may be a better choice for the kind of text-heavy 
glue-code that lldb-mi is. Alternatively it also shouldn't be hard at all to 
revive the existing C++ code. It's written in a different style than most of 
LLDB or LLVM (and IMO it should have never been accepted upstream in this 
form), but it shouldn't be hard to get building since it (thanks to the GSoC 
project!) is using only the stable public SBAPI.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [RFC] Fast Conditional Breakpoints (FCB)

[Adrian Prantl via lldb-dev]

> On Aug 15, 2019, at 2:03 PM, Ismail Bennani via lldb-dev 
>  wrote:
> 
> I built Clang (and LLVM) in Release Mode with Debug Info (-O2),
> and got these results:
> 
> |   Dwarf Occurences   |Occurences   |
> |--|-|
> | DW\_OP\_deref|1,570|
> | DW\_OP\_const|3,791|
> | DW\_OP\_addr |9,528|
> | DW\_OP\_lit  |62,826   |
> | DW\_OP\_fbreg|205,382  |
> | DW\_OP\_piece|242,888  |
> | DW\_OP\_stack\_value |992,261  |
> | DW\_OP\_breg |1,006,070|
> | DW\_OP\_reg  |5,175,831|
> | **Total**|  **7,700,147**  |
> 
> 
> I could technically implement the logic to support DW_OP_reg, DW_OP_breg
> and DW_OP_stack_value fairly easily (which still represents 90% of all ops).
> 
> However, DW_OP_piece is a more complex operation since it combines
> several other operations, and would require more work.
> 
> This would also imply that there will 2 DWARF Expression Interpreter in
> LLDB, hence twice as much code to maintain … I’ll try to see if I can
> use the existing interpreter for this feature.

I strongly agree that unless the code can be shared, the JIT-ed DWARF 
expression interpreter should be kept as simple as possible and aim to support 
the lion's share of DWARF expressions encountered in a typical program, but 
making it support 100% is a lot of effort and maintenance burden with very 
diminishing returns.

-- adrian

> 
> Ismail
> 
>> On Aug 14, 2019, at 3:42 PM, Finkel, Hal J.  wrote:
>> 
>> 
>> On 8/14/19 3:52 PM, Ismail Bennani via lldb-dev wrote:
>>> Hi everyone,
>>> 
>>> I’m Ismail, a compiler engineer intern at Apple. As a part of my internship,
>>> I'm adding Fast Conditional Breakpoints to LLDB, using code patching.
>>> 
>>> ...
>>> 
>>> Since all the registers are already mapped to a structure, I should
>>> be able to support more __DWARF Operations__ in the future.
>>> 
>>> After collecting some metrics on the __Clang__ binary, built at __-O0__,
>>> the debug info shows that __99%__ of the most used DWARF Operations are :
>>> 
>>> |DWARF Operation| Occurrences   |
>>> |---|---|
>>> |DW\_OP_fbreg   | 2 114 612 |
>>> |DW\_OP_reg |   820 548 |
>>> |DW\_OP_constu  |   267 450 |
>>> |DW\_OP_addr|17 370 |
>>> 
>>> |   __Top 4__   | __3 219 980 Occurrences__ |
>>> |---|---|
>>> |   __Total__   | __3 236 859 Occurrences__ |
>>> 
>>> Those 4 operations are the one that I'll support for now.
>>> To support more complex expressions, we would need to JIT-compile
>>> a DWARF expression interpreter.
>> 
>> 
>> First, this all sounds really useful.
>> 
>> Out of curiosity, how do these statistics change if you compile Clang 
>> with -O1? Many of my users need to debug slightly-optimized code.
>> 
>> -Hal
>> 
>> 
>> -- 
>> Hal Finkel
>> Lead, Compiler Technology and Programming Languages
>> Leadership Computing Facility
>> Argonne National Laboratory
>> 
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Who sets the 10-minute timeouts?

[Adrian Prantl via lldb-dev]

> On Aug 14, 2019, at 11:26 AM, Adrian McCarthy via lldb-dev 
>  wrote:
> 
> A recent change is causing several LLDB tests on Windows to fail and several 
> more to time out, which I intend to look into.
> 
> It appears the timeout period is set to 600 seconds (10 minutes), which seems 
> excessive and causes the Windows build bot to spend lots of time waiting.  
> (e.g., 
> http://lab.llvm.org:8011/builders/lldb-x64-windows-ninja/builds/7819/steps/test/logs/stdio
>  
> )
> 
> Is there a reason why the timeouts are set that long?  What would be a 
> reasonable value?

I recently increased/unified several internal timeouts throughout LLDB 
(https://reviews.llvm.org/D60340 ) in reaction 
to bots failing randomly on heavily used machines, particularly when ASAN is 
enabled, which can cause surprisingly long delays.

Since the normal operation should be that no tests fail, waiting an extra 10 
minutes in the exceptional case that a test does fail seems more desirable than 
the chance of a working test failing because of too-small timeout. Therefore, 
I'd rather pick an excessively large per-test timeout to be safe.

-- adrian___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Adrian Prantl via lldb-dev]

> On Mar 11, 2019, at 3:46 PM, Zachary Turner  wrote:
> 
> Given that:
> 
> 1) LLVM doesn't produce DWARF64
> 2) GCC has to be patched to produce DWARF64
> 3) LLDB's support is only partial but is untested and appears to be missing 
> major pieces in order for it to work
> 4) It's of questionable use as there are several viable alternatives
> 
> Would it be reasonable to propose a patch removing the incomplete support 
> from LLDB?  We can always add it back later when someone is ready to really 
> support and test it properly, and the history in the repository will show 
> what code would need to be changed to get back to at least where the support 
> is today (which again, appears to not fully work).  
> 
> If we can go this route, it makes merging the two DWARF parsing 
> implementations quite a bit simpler

I'm supportive of removing DWARF64 support from LLDB.

-- adrian

> 
> On Mon, Mar 11, 2019 at 3:33 PM Adrian Prantl  > wrote:
> 
> 
> > On Mar 11, 2019, at 12:45 PM, Zachary Turner via lldb-dev 
> > mailto:lldb-dev@lists.llvm.org>> wrote:
> > 
> > I want to ask what the status of DWARF64 in LLDB is.  I can tell there's 
> > some support for it by reading the code, but it seems to have zero test 
> > coverage so it's not clear to me that anyone depends on it.  For example, I 
> > know that clang and LLVM will not even generate DWARF64, so if anyone is 
> > depending on it, they must be debugging programs built with some other 
> > toolchain.
> 
> AFAIR, Apple's tools only generate/support DWARF32. After implementing 
> type-uniquing in dsymutil we didn't see any individual .dSYM bundles that 
> came even close to the 4GB watermark.
> 
> > 
> > I'm looking at unifying LLDB's DWARF parser with LLVM's, and this is the 
> > biggest functional difference I can see.  
> > 
> > Certainly we can improve LLVM's support for consuming DWARF64, but it's a 
> > question of priorities.  If nobody is actively depending on this, then 
> > taking a regression here could be on the table and then re-prioritizing 
> > adding back support in the future if / when we actually need it.
> 
> -- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Status of DWARF64 in LLDB

[Adrian Prantl via lldb-dev]

> On Mar 11, 2019, at 12:45 PM, Zachary Turner via lldb-dev 
>  wrote:
> 
> I want to ask what the status of DWARF64 in LLDB is.  I can tell there's some 
> support for it by reading the code, but it seems to have zero test coverage 
> so it's not clear to me that anyone depends on it.  For example, I know that 
> clang and LLVM will not even generate DWARF64, so if anyone is depending on 
> it, they must be debugging programs built with some other toolchain.

AFAIR, Apple's tools only generate/support DWARF32. After implementing 
type-uniquing in dsymutil we didn't see any individual .dSYM bundles that came 
even close to the 4GB watermark.

> 
> I'm looking at unifying LLDB's DWARF parser with LLVM's, and this is the 
> biggest functional difference I can see.  
> 
> Certainly we can improve LLVM's support for consuming DWARF64, but it's a 
> question of priorities.  If nobody is actively depending on this, then taking 
> a regression here could be on the table and then re-prioritizing adding back 
> support in the future if / when we actually need it.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Host is now dependency free

[Adrian Prantl via lldb-dev]

That's great news! I'm looking forward to having more fine-grained LLDB C++ 
modules.

-- adrian

> On Mar 8, 2019, at 1:00 PM, Zachary Turner via lldb-dev 
>  wrote:
> 
> It's been a long time coming and a lot of work to get here, but Host is now 
> dependency free.  While this may not be enforced in the Xcode project (unless 
> someone changes it to not link against any other libraries / targets), as of 
> r355730 this is  enforced in the CMake build, so if new dependencies are 
> introduced, it will break most non-OSX build bots.  
> 
> Mostly just throwing this out there so people are aware.
> 
> The good news is that this gets us one step closer to a shared libraries 
> build as well as a real C++ modules build, as well as being able to create 
> small debugger-related tools that are not full blown debuggers.
> 
> For the curious, the remaining cycles in the build graph (as well as their 
> outgoing edge counts) are:
> 
> 4 deps to break: lldb/Commands [3->] lldb/Expression [1->] lldb/Commands
> 5 deps to break: lldb/Plugins/SymbolFile/DWARF [4->] lldb/Expression [1->] 
> lldb/Plugins/SymbolFile/DWARF
> 5 deps to break: lldb/Plugins/Language/ObjC [4->] 
> lldb/Plugins/LanguageRuntime/ObjC/AppleObjCRuntime [1->] 
> lldb/Plugins/Language/ObjC
> 6 deps to break: lldb/Interpreter [1->] lldb/Breakpoint [5->] lldb/Interpreter
> 6 deps to break: lldb/Plugins/ScriptInterpreter/Python [2->] lldb/API [4->] 
> lldb/Plugins/ScriptInterpreter/Python
> 13 deps to break: lldb/Plugins/Language/ObjC [12->] lldb/Symbol [1->] 
> lldb/Plugins/Language/ObjC
> 14 deps to break: lldb/Interpreter [10->] lldb/DataFormatters [4->] 
> lldb/Interpreter
> 22 deps to break: lldb/Plugins/SymbolFile/PDB [21->] lldb/Symbol [1->] 
> lldb/Plugins/SymbolFile/PDB
> 23 deps to break: lldb/Target [3->] lldb/DataFormatters [20->] lldb/Target
> 26 deps to break: lldb/Expression [1->] lldb/Plugins/ExpressionParser/Clang 
> [25->] lldb/Expression
> 29 deps to break: lldb/Plugins/Language/CPlusPlus [26->] lldb/Core [3->] 
> lldb/Plugins/Language/CPlusPlus
> 29 deps to break: lldb/Plugins/Language/ObjC [27->] lldb/Core [2->] 
> lldb/Plugins/Language/ObjC
> 29 deps to break: lldb/Core [14->] lldb/DataFormatters [15->] lldb/Core
> 33 deps to break: lldb/Expression [30->] lldb/Symbol [3->] lldb/Expression
> 37 deps to break: lldb/Expression [33->] lldb/Core [4->] lldb/Expression
> 38 deps to break: lldb/Target [1->] lldb/Plugins/Language/ObjC [37->] 
> lldb/Target
> 42 deps to break: lldb/Interpreter [19->] lldb/Target [23->] lldb/Interpreter
> 42 deps to break: lldb/Breakpoint [39->] lldb/Core [3->] lldb/Breakpoint
> 49 deps to break: lldb/Interpreter [25->] lldb/Core [24->] lldb/Interpreter
> 51 deps to break: lldb/Target [4->] lldb/Plugins/ExpressionParser/Clang 
> [47->] lldb/Target
> 55 deps to break: lldb/Plugins/SymbolFile/DWARF [54->] lldb/Symbol [1->] 
> lldb/Plugins/SymbolFile/DWARF
> 62 deps to break: lldb/Plugins/ExpressionParser/Clang [58->] lldb/Symbol 
> [4->] lldb/Plugins/ExpressionParser/Clang
> 69 deps to break: lldb/Target [38->] lldb/Breakpoint [31->] lldb/Target
> 72 deps to break: lldb/Target [13->] lldb/Expression [59->] lldb/Target
> 72 deps to break: lldb/Utility [71->] lldb [1->] lldb/Utility
> 104 deps to break: lldb/Target [64->] lldb/Symbol [40->] lldb/Target
> 128 deps to break: lldb/Target [7->] lldb/Plugins/Process/Utility [121->] 
> lldb/Target
> 201 deps to break: lldb/Core [110->] lldb/Symbol [91->] lldb/Core
> 227 deps to break: lldb/Target [127->] lldb/Core [100->] lldb/Target
> 
> Found by running scripts/analyze-project-deps.py
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] RFC: Moving debug info parsing out of process

[Adrian Prantl via lldb-dev]

> On Mar 6, 2019, at 9:43 AM, Zachary Turner  wrote:
> 
> 
> On Mon, Mar 4, 2019 at 10:32 AM Zachary Turner  > wrote:
> On Sat, Mar 2, 2019 at 2:56 PM Adrian Prantl  > wrote:
>> It becomes testable as an independent component, because you can just send 
>> requests to it and dump the results and see if they make sense.  Currently 
>> there is almost zero test coverage of this aspect of LLDB apart from what 
>> you can get after going through many levels of indirection via spinning up a 
>> full debug session and doing things that indirectly result in symbol queries.
> 
> You are right that the type system debug info ingestion and AST 
> reconstruction is primarily tested end-to-end.
> Do you consider this something worth addressing by testing the debug info 
> ingestion in isolation?
> 
>  Wanted to bump this thread for visibility.  If nothing else, I'm interested 
> in an answer to this question.  Because if people agree that it would be 
> valuable to test this going forward, we should work out a plan about what 
> such tests would look like and how to refactor the code appropriately to make 
> it possible.

I think it would help me a lot to have a better idea what level of abstraction 
you are imagining. Could perhaps come up with a mock-up example with some 
mad-up syntax / API for what such a test could look like? More testing is 
always desirable, of course, but I'm afraid that we might end up in a situation 
like we are with yaml2obj, where we can only test really trivial things nicely 
and all the interesting cases aren't representable at all.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] RFC: Moving debug info parsing out of process

[Adrian Prantl via lldb-dev]

> On Feb 25, 2019, at 10:21 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> Hi all,
> 
> We've got some internal efforts in progress, and one of those would benefit 
> from debug info parsing being out of process (independently of whether or not 
> the rest of LLDB is out of process).
> 
> There's a couple of advantages to this, which I'll enumerate here:
> It improves one source of instability in LLDB which has been known to be 
> problematic -- specifically, that debug info can be bad and handling this can 
> often be difficult and bring down the entire debug session.  While other 
> efforts have been made to address stability by moving things out of process, 
> they have not been upstreamed, and even if they had I think we would still 
> want this anyway, for reasons that follow.
Where do you draw the line between debug info and the in-process part of LLDB? 
I'm asking because I have never seen the mechanical parsing of DWARF to be a 
source of instability; most crashes in LLDB are when reconstructing Clang ASTs 
because we're breaking some subtle and badly enforced invariants in Clang's 
Sema. Perhaps parsing PDBs is less stable? If you do mean at the AST level then 
I agree with the sentiment that it is a common source of crashes, but I don't 
see a good way of moving that component out of process. Serializing ASTs or 
types in general is a hard problem, and I'd find the idea of inventing yet 
another serialization format for types that we would have to develop, test, and 
maintain quite scary.
> It becomes theoretically possible to move debug info parsing not just to 
> another process, but to another machine entirely.  In a broader sense, this 
> decouples the physical debug info location (and for that matter, 
> representation) from the debugger host.
I can see how that can be useful in some settings. You'd need a really low 
latency network connection to make interactive debugging work but I expect 
you've got that covered :-)
> It becomes testable as an independent component, because you can just send 
> requests to it and dump the results and see if they make sense.  Currently 
> there is almost zero test coverage of this aspect of LLDB apart from what you 
> can get after going through many levels of indirection via spinning up a full 
> debug session and doing things that indirectly result in symbol queries.
You are right that the type system debug info ingestion and AST reconstruction 
is primarily tested end-to-end.

> The big win here, at least from my point of view, is the second one.  
> Traditional symbol servers operate by copying entire symbol files (DSYM, DWP, 
> PDB) from some machine to the debugger host.  These can be very large -- 
> we've seen 12+ GB in some cases -- which ranges from "slow bandwidth hog" to 
> "complete non-starter" depending on the debugger host and network. 

12 GB sounds suspiciously large. Do you know how this breaks down between line 
table, types, and debug locations? If it's types, are you deduplicating them? 
For comparison, the debug info of LLDB (which contains two compilers and a 
debugger) compresses to under 500MB, but perhaps the binaries you are working 
with are really just that much larger.

> In this kind of scenario, one could theoretically run the debug info process 
> on the same NAS, cloud, or whatever as the symbol server.  Then, rather than 
> copying over an entire symbol file, it responds only to the query you issued 
> -- if you asked for a type, it just returns a packet describing the type you 
> requested.
> 
> The API itself would be stateless (so that you could make queries for 
> multiple targets in any order) as well as asynchronous (so that responses 
> might arrive out of order).  Blocking could be implemented in LLDB, but 
> having the server be asynchronous means multiple clients could connect to the 
> same server instance.  This raises interesting possibilities.  For example, 
> one can imagine thousands of developers connecting to an internal symbol 
> server on the network and being able to debug remote processes or core dumps 
> over slow network connections or on machines with very little storage (e.g. 
> chromebooks).

You *could* just run LLDB remotely ;-)

That sounds all cool, but in my opinion you are leaving out the really 
important part: what is the abstraction level of the API going to be?

To be blunt, I'm against inventing yet another serialization format for *types* 
not just because of the considerable engineering effort it will take to get 
this right, but also because of the maintenance burden it would impose. We 
already have to support loading types from DWARF, PDB, Clang modules, the 
Objective-C runtime, Swift modules, and probably more sources, all of these 
operate to some degree at different levels of abstraction. Adding another 
source or abstraction layer into the mix needs to be really well thought out 
and justified.

> On the LLDB side, all of this is hidden behind the SymbolFile interface, 

Re: [lldb-dev] [RFC]The future of pexpect

[Adrian Prantl via lldb-dev]

> On Feb 25, 2019, at 2:21 PM, Davide Italiano  wrote:
> 
> On Mon, Feb 25, 2019 at 1:57 PM Adrian Prantl  wrote:
>> 
>> 
>> 
>>> On Feb 25, 2019, at 1:40 PM, Davide Italiano  wrote:
>>> 
>>> On Mon, Feb 25, 2019 at 1:35 PM Adrian Prantl  wrote:
 
 
 
> On Feb 25, 2019, at 1:15 PM, Davide Italiano  
> wrote:
> 
> On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
>> 
>> On 21/02/2019 19:48, Ted Woodward wrote:
>>> 
>>> 
 -Original Message-
 From: lldb-dev  On Behalf Of Pavel 
 Labath
 via lldb-dev
 Sent: Thursday, February 21, 2019 8:35 AM
 To: Davide Italiano 
 Cc: LLDB 
 Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
 
 On 21/02/2019 00:03, Davide Italiano wrote:
> I found out that there are tests that effectively require
> interactivity. Some of the lldb-mi ones are an example.
> A common use-case is that of sending SIGTERM in a loop to make sure
> `lldb-mi` doesn't crash and handle the signal correctly.
> 
> This functionality is really hard to replicate in lit_as is_.
> Any ideas on how we could handle this case?
 
 How hard is it to import a new version of pexpect which supports 
 python3 and
 stuff?
 
 I'm not sure how the situation is on darwin, but I'd expect (:P) that 
 most linux
 systems either already have it installed, or have an easy way to do 
 so. So we
 may not even be able to get away with just using the system one and 
 skipping
 tests when it's not present.
 
 BTW, for lldb-mi I would actually argue that it should *not* use 
 pexpect :D.
 Interactivity is one thing, and I'm very much in favour of keeping 
 that ability,
 but pexpect is not a prerequisite for that. For me, the main advantage 
 of
 pexpect is that it emulates a real terminal. However, lldb-mi does not 
 need
 that stuff. It doesn't have any command line editing capabilities or 
 similar. It's
 expecting to communicate with an IDE over a pipe, and that's it.
 
 Given that, it should be fairly easy to rewrite the lldb-mi tests to 
 work on top
 of the standard python "subprocess" library. While we're doing that, 
 we might
 actually fix some of the issues that have been bugging everyone in the 
 lldb-mi
 tests. At least for me, the most annoying thing was that when lldb-mi 
 fails to
 produce the expected output, the test does not fail immediately, but 
 instead
 the implementation of self.expect("^whatever") waits until the timeout
 expires, optimistically hoping that it will find some output that 
 match the
 pattern.
 
> 
> Pavel, I think yours is a really nice idea.
> I'm no python expert, but I found out making the conversion is
> relatively simple.
> I propose a proof-of-concept API and implementation here:
> 
> https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
> 
> Comments appreciated! Once we agree on how this should look like, I do
> recommend to have a new lldbMITest base class and incrementally start
> moving the tests to it.
> Once we're done, we can delete the old class.
> 
> Does this sound reasonable?
 
 What you are saying is that we would write the tests as Python tests in a 
 way similar to how lldbtest.expect() look in the dotest.py tests, banking 
 on synchronous mode taking care of all the synchronization? Are you 
 thinking of doing this for all the remaining tests or only the ones where 
 a command input depends on the result of a previous command?
 
>>> 
>>> I'm thinking to do this for all the remaining tests. Do you have any
>>> concerns about this? (I'm aware your GSoC student introduced the `lit
>>> lldb-mi` tests for a reason, I just don't know exactly what the reason
>>> was).
>> 
>> I think the reason was that for tests that don't need synchronization and 
>> have a static command input, writing a lit/FileCheck test is straightforward 
>> and easy. At the time I thought we could just rewrite *all* lldb-mi tests as 
>> FileCheck tests. If we do need a python mechanism anyway, however, I don't 
>> actually think that having two ways of writing tests is better; I'd rather 
>> have all tests in one place. Thankfully there shouldn't be that many tests, 
>> so we should be able to just convert all of them over to whatever format we 
>> eventually settle on.
>> -- Adrian
>> 
> 
> Sounds good to me. Do you have a preference? I wanted to move away
> from pexpect, so my preference was that of using `lit`.
> I then realized that there are some functionalities that are not
> really that easy to 

Re: [lldb-dev] [RFC]The future of pexpect

[Adrian Prantl via lldb-dev]

> On Feb 25, 2019, at 1:40 PM, Davide Italiano  wrote:
> 
> On Mon, Feb 25, 2019 at 1:35 PM Adrian Prantl  wrote:
>> 
>> 
>> 
>>> On Feb 25, 2019, at 1:15 PM, Davide Italiano  wrote:
>>> 
>>> On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
 
 On 21/02/2019 19:48, Ted Woodward wrote:
> 
> 
>> -Original Message-
>> From: lldb-dev  On Behalf Of Pavel 
>> Labath
>> via lldb-dev
>> Sent: Thursday, February 21, 2019 8:35 AM
>> To: Davide Italiano 
>> Cc: LLDB 
>> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
>> 
>> On 21/02/2019 00:03, Davide Italiano wrote:
>>> I found out that there are tests that effectively require
>>> interactivity. Some of the lldb-mi ones are an example.
>>> A common use-case is that of sending SIGTERM in a loop to make sure
>>> `lldb-mi` doesn't crash and handle the signal correctly.
>>> 
>>> This functionality is really hard to replicate in lit_as is_.
>>> Any ideas on how we could handle this case?
>> 
>> How hard is it to import a new version of pexpect which supports python3 
>> and
>> stuff?
>> 
>> I'm not sure how the situation is on darwin, but I'd expect (:P) that 
>> most linux
>> systems either already have it installed, or have an easy way to do so. 
>> So we
>> may not even be able to get away with just using the system one and 
>> skipping
>> tests when it's not present.
>> 
>> BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
>> :D.
>> Interactivity is one thing, and I'm very much in favour of keeping that 
>> ability,
>> but pexpect is not a prerequisite for that. For me, the main advantage of
>> pexpect is that it emulates a real terminal. However, lldb-mi does not 
>> need
>> that stuff. It doesn't have any command line editing capabilities or 
>> similar. It's
>> expecting to communicate with an IDE over a pipe, and that's it.
>> 
>> Given that, it should be fairly easy to rewrite the lldb-mi tests to 
>> work on top
>> of the standard python "subprocess" library. While we're doing that, we 
>> might
>> actually fix some of the issues that have been bugging everyone in the 
>> lldb-mi
>> tests. At least for me, the most annoying thing was that when lldb-mi 
>> fails to
>> produce the expected output, the test does not fail immediately, but 
>> instead
>> the implementation of self.expect("^whatever") waits until the timeout
>> expires, optimistically hoping that it will find some output that match 
>> the
>> pattern.
>> 
>>> 
>>> Pavel, I think yours is a really nice idea.
>>> I'm no python expert, but I found out making the conversion is
>>> relatively simple.
>>> I propose a proof-of-concept API and implementation here:
>>> 
>>> https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
>>> 
>>> Comments appreciated! Once we agree on how this should look like, I do
>>> recommend to have a new lldbMITest base class and incrementally start
>>> moving the tests to it.
>>> Once we're done, we can delete the old class.
>>> 
>>> Does this sound reasonable?
>> 
>> What you are saying is that we would write the tests as Python tests in a 
>> way similar to how lldbtest.expect() look in the dotest.py tests, banking on 
>> synchronous mode taking care of all the synchronization? Are you thinking of 
>> doing this for all the remaining tests or only the ones where a command 
>> input depends on the result of a previous command?
>> 
> 
> I'm thinking to do this for all the remaining tests. Do you have any
> concerns about this? (I'm aware your GSoC student introduced the `lit
> lldb-mi` tests for a reason, I just don't know exactly what the reason
> was).

I think the reason was that for tests that don't need synchronization and have 
a static command input, writing a lit/FileCheck test is straightforward and 
easy. At the time I thought we could just rewrite *all* lldb-mi tests as 
FileCheck tests. If we do need a python mechanism anyway, however, I don't 
actually think that having two ways of writing tests is better; I'd rather have 
all tests in one place. Thankfully there shouldn't be that many tests, so we 
should be able to just convert all of them over to whatever format we 
eventually settle on.
-- adrian


___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [RFC]The future of pexpect

[Adrian Prantl via lldb-dev]

> On Feb 25, 2019, at 1:15 PM, Davide Italiano  wrote:
> 
> On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
>> 
>> On 21/02/2019 19:48, Ted Woodward wrote:
>>> 
>>> 
 -Original Message-
 From: lldb-dev  On Behalf Of Pavel Labath
 via lldb-dev
 Sent: Thursday, February 21, 2019 8:35 AM
 To: Davide Italiano 
 Cc: LLDB 
 Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
 
 On 21/02/2019 00:03, Davide Italiano wrote:
> I found out that there are tests that effectively require
> interactivity. Some of the lldb-mi ones are an example.
> A common use-case is that of sending SIGTERM in a loop to make sure
> `lldb-mi` doesn't crash and handle the signal correctly.
> 
> This functionality is really hard to replicate in lit_as is_.
> Any ideas on how we could handle this case?
 
 How hard is it to import a new version of pexpect which supports python3 
 and
 stuff?
 
 I'm not sure how the situation is on darwin, but I'd expect (:P) that most 
 linux
 systems either already have it installed, or have an easy way to do so. So 
 we
 may not even be able to get away with just using the system one and 
 skipping
 tests when it's not present.
 
 BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
 :D.
 Interactivity is one thing, and I'm very much in favour of keeping that 
 ability,
 but pexpect is not a prerequisite for that. For me, the main advantage of
 pexpect is that it emulates a real terminal. However, lldb-mi does not need
 that stuff. It doesn't have any command line editing capabilities or 
 similar. It's
 expecting to communicate with an IDE over a pipe, and that's it.
 
 Given that, it should be fairly easy to rewrite the lldb-mi tests to work 
 on top
 of the standard python "subprocess" library. While we're doing that, we 
 might
 actually fix some of the issues that have been bugging everyone in the 
 lldb-mi
 tests. At least for me, the most annoying thing was that when lldb-mi 
 fails to
 produce the expected output, the test does not fail immediately, but 
 instead
 the implementation of self.expect("^whatever") waits until the timeout
 expires, optimistically hoping that it will find some output that match the
 pattern.
 
> 
> Pavel, I think yours is a really nice idea.
> I'm no python expert, but I found out making the conversion is
> relatively simple.
> I propose a proof-of-concept API and implementation here:
> 
> https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
> 
> Comments appreciated! Once we agree on how this should look like, I do
> recommend to have a new lldbMITest base class and incrementally start
> moving the tests to it.
> Once we're done, we can delete the old class.
> 
> Does this sound reasonable?

What you are saying is that we would write the tests as Python tests in a way 
similar to how lldbtest.expect() look in the dotest.py tests, banking on 
synchronous mode taking care of all the synchronization? Are you thinking of 
doing this for all the remaining tests or only the ones where a command input 
depends on the result of a previous command?

-- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Missing module (debug symbols?) for Swift

[Adrian Prantl via lldb-dev]

> On Feb 3, 2019, at 6:48 PM, Tabor Kelly via lldb-dev 
>  wrote:
> 
> Hello,
> 
> I am new to LLDB and don't quite understand the relationship between
> llvm.org and Apple. However, I would like to fix a bug that is
> preventing me from debugging some code that I compiled with Swift 4.2
> on Ubuntu 18.04. Specifically, when I use the Apple/Swift LLDB with
> this code and LLDB_SWIFT_DUMP_DIAGS=1 enabled I get errors that say
> ":0: error: missing required module 'Clibavahiclient'".
> However, I went looking and to my novice eyes it looks like the module
> (symbols?) that LLDB is looking for are on the file system.
> Specifically, there is a file (Clibavahiclient-2J8WBHG3Q43SC.pcm)
> which is an ELF binary with lots of DWARF symbols, including the
> following:
> 
> 0x0026:   DW_TAG_module
>DW_AT_name ("Clibavahiclient")
>DW_AT_LLVM_config_macros ("\"-D__swift__=40150\"")
>DW_AT_LLVM_include_path
> ("/work/AvahiClient/.build/checkouts/Clibavahiclient.git--4733535768090925471")
>DW_AT_LLVM_isysroot ("/")
> 

If the code that you are debugging is in a Swift context, then this is actually 
not the place that LLDB looks for paths to Clang modules. Instead it will looks 
at the serialized search paths in the .swiftmodule. You can see what they are 
by running "log enable lldb types" at the beginning of your session and look at 
the various SwiftASTContext initializations. Also note that we've fixed a lot 
of search path problems in Swift 5.0, so I'd really recommend checking that out 
first as it may already work out of the box.

-- adrian

> I have posted a full write-up with source code and all the symbols here:
> https://github.com/TaborKelly/AvahiClient
> 
> Am I on the right track? Does Clibavahiclient-2J8WBHG3Q43SC.pcm have
> the information that LLDB is missing? If so, how would I go about
> getting that knowledge to LLDB?
> 
> Thank you,
> 
> Tabor
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] [Github] RFC: linear history vs merge commits

[Adrian Prantl via lldb-dev]

> On Jan 29, 2019, at 4:55 PM, Jeremy Lakeman via llvm-dev 
>  wrote:
> 
> 5. When a new feature is broken up into a patch series, the series should be 
> rebased then immediately merged to help identify the individual patches in 
> the history graph.

Typically the LLVM development model discourages merging big features in one go 
and instead gravitates towards breaking patches up into small, easy to reason 
about patches that gradually morph the code towards the goal. This makes it 
easier for reviewers and to track down regressions after each patch landed. For 
this reason we rarely have series of connected patches coming in at once.

-- adrian


___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] [Github] RFC: linear history vs merge commits

[Adrian Prantl via lldb-dev]

A linear history makes it much easier to git-bisect, since each commit can be 
rolled back individually. Easier cross-project bisection is one of the 
advantages of having the monorepo, and I wouldn't want to loose that.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] lldb print c++ variables from dylib loaded in Swift application

[Adrian Prantl via lldb-dev]

> On Jan 27, 2019, at 10:29 AM, David M. Lary  wrote:
> 
> On Sun, Jan 27, 2019 at 10:33 AM Adrian Prantl  > wrote:
>>> On Jan 26, 2019, at 6:41 PM, David M. Lary via lldb-dev 
>>>  wrote:
>>> 
>>> I'm writing a Swift application for MacOS that makes use of a C/C++
>>> dylib from a third party to interact with a film scanner.  The
>>> application uses dlopen() & dlsym() to discover and call the C wrapper
>>> function for the C++ implementation.  Overall this works, but I've run
>>> into the following wall while debugging an issue within the dylib
>>> itself.
>>> 
>>> While debugging I want to examine std::list  structures in memory
>>> by address.  I am stopped deep inside the C++ code, and from what I've
>>> googled, printing STL variables is possible through `frame variable`,
>>> however for all of the functions within the dylib, `frame variable`
>>> doesn't find any variables.
>>> 
>>> I've tried using `expr -l c++ --
>>> reinterpret_cast>($rdi)`, however this just returns the
>>> error: "use of undeclared identifier 'std'".
>>> 
>>> Is there a way to cast an arbitrary address as an STL type, and print
>>> it out in lldb?
>> 
>> Casting to a std::list is not a problem at all; however, the error that 
>> you are seeing is because LLDB did not find a type definition for 
>> std::list in the debug information. Did you build the C++ dylib with 
>> full debug information?
>> 
>> Long-term it should also become possible to just import the C++ "std" Clang 
>> module into LLDB to get access to all types regardless of whether they were 
>> used in the debugged program and available in debug info, but this work 
>> isn't quite finished yet.
>> 
>> -- adrian
> 
> Adrian,
> 
> I did not build the C++ dylib, and it was not built with full debug
> information; it comes from a third-party, so I don't control it.
> However, from what you said, would creating an unrelated C++ dylib,
> declaring the required types, and linking it into the project provide
> needed types to lldb?  Would I need to use the types to ensure they're
> not optimized out of the dylib?

I'm not perfectly sure about how exactly the type lookup operates, but I 
believe that this might work. You might want to try it out with just a simple 
hello-world example first where you cast a raw address to the list type. if 
that works, chances are good.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] lldb print c++ variables from dylib loaded in Swift application

[Adrian Prantl via lldb-dev]

> On Jan 26, 2019, at 6:41 PM, David M. Lary via lldb-dev 
>  wrote:
> 
> I'm writing a Swift application for MacOS that makes use of a C/C++
> dylib from a third party to interact with a film scanner.  The
> application uses dlopen() & dlsym() to discover and call the C wrapper
> function for the C++ implementation.  Overall this works, but I've run
> into the following wall while debugging an issue within the dylib
> itself.
> 
> While debugging I want to examine std::list  structures in memory
> by address.  I am stopped deep inside the C++ code, and from what I've
> googled, printing STL variables is possible through `frame variable`,
> however for all of the functions within the dylib, `frame variable`
> doesn't find any variables.
> 
> I've tried using `expr -l c++ --
> reinterpret_cast>($rdi)`, however this just returns the
> error: "use of undeclared identifier 'std'".
> 
> Is there a way to cast an arbitrary address as an STL type, and print
> it out in lldb?

Casting to a std::list is not a problem at all; however, the error that 
you are seeing is because LLDB did not find a type definition for 
std::list in the debug information. Did you build the C++ dylib with full 
debug information?

Long-term it should also become possible to just import the C++ "std" Clang 
module into LLDB to get access to all types regardless of whether they were 
used in the debugged program and available in debug info, but this work isn't 
quite finished yet.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] [lit] check-all hanging

[Adrian Prantl via lldb-dev]

> On Jan 7, 2019, at 8:28 AM, Joel E. Denny via lldb-dev 
>  wrote:
> 
> On Mon, Jan 7, 2019 at 11:15 AM Davide Italiano  > wrote:
> On Sat, Jan 5, 2019 at 9:48 AM Joel E. Denny via lldb-dev
> mailto:lldb-dev@lists.llvm.org>> wrote:
> >
> > On Fri, Jan 4, 2019 at 11:39 AM Frédéric Riss  > > wrote:
> >>
> >>
> >>
> >> > On Jan 4, 2019, at 7:30 AM, Joel E. Denny  >> > > wrote:
> >> >
> >> > On Thu, Jan 3, 2019 at 11:30 AM Frédéric Riss  >> > > wrote:
> >> > -llvm-dev + lldb-dev for the lldv test failures.
> >> >
> >> >> On Jan 3, 2019, at 7:33 AM, Joel E. Denny  >> >> > wrote:
> >> >>
> >> >> All,
> >> >>
> >> >> Thanks for the replies.  Kuba: For LLDB, when were things expected to 
> >> >> have improved?  It's possible things improved for me at some point, but 
> >> >> this isn't something I've found time to track carefully, and I still 
> >> >> see problems.
> >> >>
> >> >> I ran check-all a couple of times last night at r350238, which I pulled 
> >> >> yesterday.  Here are the results:
> >> >>
> >> >> ```
> >> >> 
> >> >> Testing Time: 5043.24s
> >> >> 
> >> >> Unexpected Passing Tests (2):
> >> >> lldb-Suite :: functionalities/asan/TestMemoryHistory.py
> >> >> lldb-Suite :: functionalities/asan/TestReportData.py
> >> >>
> >> >> 
> >> >> Failing Tests (54):
> >> >> Clang :: CXX/modules-ts/basic/basic.link/p2/module.cpp
> >> >> Clang :: Modules/ExtDebugInfo.cpp
> >> >> Clang :: Modules/using-directive-redecl.cpp
> >> >> Clang :: Modules/using-directive.cpp
> >> >> Clang :: PCH/chain-late-anonymous-namespace.cpp
> >> >> Clang :: PCH/cxx-namespaces.cpp
> >> >> Clang :: PCH/namespaces.cpp
> >> >> LLDB :: ExecControl/StopHook/stop-hook-threads.test
> >> >> LeakSanitizer-AddressSanitizer-x86_64 :: 
> >> >> TestCases/Linux/use_tls_dynamic.cc
> >> >> LeakSanitizer-Standalone-x86_64 :: 
> >> >> TestCases/Linux/use_tls_dynamic.cc
> >> >> MemorySanitizer-X86_64 :: dtls_test.c
> >> >> MemorySanitizer-lld-X86_64 :: dtls_test.c
> >> >> lldb-Suite :: 
> >> >> functionalities/register/register_command/TestRegisters.py
> >> >> lldb-Suite :: tools/lldb-server/TestGdbRemoteRegisterState.py
> >> >
> >> > It’s hard to diagnose dotest failures without the log.
> >> >
> >> > (My last reply to this was rejected by the list because I wasn't 
> >> > subscribed.  Trying again.)
> >> >
> >> > I have no experience debugging lldb.  Here's the lit output for the last 
> >> > fail (now at r350377), but let me know if you want something more:
> >> >
> >> > ```
> >> > FAIL: lldb-Suite :: tools/lldb-server/TestGdbRemoteRegisterState.py 
> >> > (59083 of 59736)
> >> >  TEST 'lldb-Suite :: 
> >> > tools/lldb-server/TestGdbRemoteRegisterState.py' FAILED 
> >> > 
> >> > lldb version 8.0.0
> >> > LLDB library dir: /home/jdenny/ornl/llvm-mono-git-build/bin
> >> > LLDB import library dir: /home/jdenny/ornl/llvm-mono-git-build/bin
> >> > Libc++ tests will not be run because: Unable to find libc++ installation
> >> > Skipping following debug info categories: ['dsym', 'gmodules']
> >> >
> >> > Session logs for test failures/errors/unexpected successes will go into 
> >> > directory '/home/jdenny/ornl/llvm-mono-git-build/lldb-test-traces'
> >> > Command invoked: /home/jdenny/ornl/llvm-mono-git/lldb/test/dotest.py -q 
> >> > --arch=x86_64 -s /home/jdenny/ornl/llvm-mono-git-build/lldb-test-traces 
> >> > --build-dir 
> >> > /home/jdenny/ornl/llvm-mono-git-build/lldb-test-build.noindex -S nm -u 
> >> > CXXFLAGS -u CFLAGS --executable 
> >> > /home/jdenny/ornl/llvm-mono-git-build/./bin/lldb --dsymutil 
> >> > /home/jdenny/ornl/llvm-mono-git-build/./bin/dsymutil --filecheck 
> >> > /home/jdenny/ornl/llvm-mono-git-build/./bin/FileCheck -C 
> >> > /home/jdenny/ornl/llvm-mono-git-build/./bin/clang --env 
> >> > ARCHIVER=/usr/bin/ar --env OBJCOPY=/usr/bin/objcopy 
> >> > /home/jdenny/ornl/llvm-mono-git/lldb/packages/Python/lldbsuite/test/tools/lldb-server
> >> >  -p TestGdbRemoteRegisterState.py
> >> > UNSUPPORTED: LLDB 
> >> > (/home/jdenny/ornl/llvm-mono-git-build/bin/clang-8-x86_64) :: 
> >> > test_grp_register_save_restore_works_no_suffix_debugserver 
> >> > (TestGdbRemoteRegisterState.TestGdbRemoteRegisterState) (debugserver 
> >> > tests)
> >> > FAIL: LLDB (/home/jdenny/ornl/llvm-mono-git-build/bin/clang-8-x86_64) :: 
> >> > test_grp_register_save_restore_works_no_suffix_llgs 
> >> > (TestGdbRemoteRegisterState.TestGdbRemoteRegisterState)
> >> > lldb-server exiting...
> >> > UNSUPPORTED: LLDB 
> >> > (/home/jdenny/ornl/llvm-mono-git-build/bin/clang-8-x86_64) :: 
> >> > test_grp_register_save_restore_works_with_suffix_debugserver 
> >> > (TestGdbRemoteRegisterState.TestGdbRemoteRegisterState) (debugserver 
> >> > tests)
> >> > FAIL: LLDB 

Re: [lldb-dev] [llvm-dev] [lit] check-all hanging

[Adrian Prantl via lldb-dev]

> On Jan 3, 2019, at 8:30 AM, Frédéric Riss  wrote:
> 
> -llvm-dev + lldb-dev for the lldv test failures.
> 
>> On Jan 3, 2019, at 7:33 AM, Joel E. Denny > > wrote:
> 
>>   Expected Passes: 57489
>>   Expected Failures  : 276
>>   Unsupported Tests  : 1883
>>   Unexpected Passes  : 2
>>   Unexpected Failures: 54
>> 
>> 14 warning(s) in tests.
>> FAILED: CMakeFiles/check-all
>> ```
>> 
>> I immediately ran it again and saw one new unexpected fail:
>> 
>> ```
>> lldb-Suite :: tools/lldb-mi/syntax/TestMiSyntax.py
>> ```
> 
> Adrian, do we have remaining flakiness in the MI tests? Is this one of the 
> GSoc tests?

That particular test still uses pexpect and needs to be rewritten as a 
LIT-based test; so, yes.

-- adrian
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] When should ArchSpecs match?

[Adrian Prantl via lldb-dev]

I was puzzled by the behavior of ArchSpec::IsExactMatch() and 
IsCompatibleMatch() yesterday, so I created a couple of unit tests to document 
the current behavior. Most of the tests make perfect sense, but a few edge 
cases really don't behave like I would have expected them to.

>   {
> ArchSpec A("arm64-*-*");
> ArchSpec B("arm64-apple-ios");
> ASSERT_FALSE(A.IsExactMatch(B));
> // FIXME: This looks unintuitive and we should investigate whether
> // this is the desired behavior.
> ASSERT_FALSE(A.IsCompatibleMatch(B));
>   }
>   {
> ArchSpec A("x86_64-*-*");
> ArchSpec B("x86_64-apple-ios-simulator");
> ASSERT_FALSE(A.IsExactMatch(B));
> // FIXME: See above, though the extra environment complicates things.
> ASSERT_FALSE(A.IsCompatibleMatch(B));
>   }
>   {
> ArchSpec A("x86_64");
> ArchSpec B("x86_64-apple-macosx10.14");
> // FIXME: The exact match also looks unintuitive.
> ASSERT_TRUE(A.IsExactMatch(B));
> ASSERT_TRUE(A.IsCompatibleMatch(B));
>   }
> 

Particularly, I believe that:
- ArchSpec("x86_64-*-*") and ArchSpec("x86_64") should behave the same.
- ArchSpec("x86_64").IsExactMatch("x86_64-apple-macosx10.14") should be false.
- ArchSpec("x86_64-*-*").IsCompatibleMath("x86_64-apple-macosx") should be true.

Does anyone disagree with any of these statements?

I fully understand that changing any of these behaviors will undoubtedly break 
one or the other edge case, but I think it would be important to build on a 
foundation that actually makes sense if we want to be able to reason about the 
architecture matching logic at all.

let me know what you think!
-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] DW_OP_deref handling

[Adrian Prantl via lldb-dev]

> On Oct 12, 2018, at 9:53 AM, Adrian Prantl via lldb-dev 
>  wrote:
> 
> 
> 
>> On Oct 11, 2018, at 11:16 AM, Greg Clayton via lldb-dev 
>>  wrote:
>> 
>> DWARF5 finally added the ability to track what each value means on the 
>> expression stack. Prior to DWARF 5, we had no idea what each entry on the 
>> expression value stack was (file address, load address 
>> (Value::eValueTypeLoadAddress), plain value (Value::eValueTypeScalar). We 
>> have tracked this info for a while now, but the DWARF5 spec is much more 
>> specific on how things should be treated.
> 
> Greg,
> 
> I'd like to summarize my own understanding of how this works — could you take 
> a look and correct me where I'm wrong?
> 
> - The only way to push a file address onto the DWARF stack is a DW_OP_addr.
> 
> The decision of whether a value pushed onto the DWARF stack is a scalar or a 
> load address depends on the location kind (cf. DWARF 5, section 2.6 "Location 
> Descriptions"):
> - A register location description (DW_OP_reg.*) reads the register contents 
> and pushes a scalar.
> - An implicit location description (.* (DW_OP_implicit_.*|DW_OP_stack_value) 
> yields a scalar after evaluating the expression.
> - A memory location description (anything else, such as DW_OP_breg) yields a 
> load address.
> (- composite locations, like DW_OP_piece are handled according to these rules 
> for each piece)
> 
> Practically speaking, I think this means that a DW_OP_(f)breg always turns 
> into a load address (as it can only appear in an implicit or a memory 
> location), and a DW_OP_reg always. turns into a scalar.
> 
> 
> Is that what LLDB is doing, and if not, could that explain at least some of 
> the failures that Davide is seeing?

To answer my own question: I think that the patch to set the address type to 
scalar after interpreting a DW_OP_deref is just wrong, because of the above :-)

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in case of missing location at block begin

[Adrian Prantl via lldb-dev]

> On Oct 11, 2018, at 4:42 PM, Matthias Braun  wrote:
> 
> I reverted things in r344318 now.
> 
>> On Oct 10, 2018, at 5:02 PM, Jim Ingham  wrote:
>> 
>> Thanks for looking into this!  
>> 
>> When I was first working on inlined stepping, I found a bunch of cases where 
>> the line table info and the ranges for the inlined subroutines disagreed.  I 
>> remember seeing cases, for instance, where the address given for foo.h:xxx 
>> in the line table was contained in one of the debug_info's inlined 
>> subroutine blocks from a different file.  I tried for a little while to put 
>> in heuristics to try to work past the disagreement.  But that was too easy 
>> to get wrong, and when I got that wrong it often had the result of turning a 
>> step into a continue.  It is annoying to stop too early, but it is much 
>> worse to stop too late (or never).  So I ended up bagging my attempts at 
>> heroics and whenever I get to a place where the line table and the 
>> inlined_subroutine bits of info are out of sync, I just stop.
> Makes total sense from lldbs point of view. If anything this is something a 
> verifier could catch during testing, but it's a small issue so I'm not sure 
> it's worth starting this now.

Thanks for getting to the bottom of this!

-- adrian
> 
> - Matthias
> 
>> 
>> Jim
>> 
>> 
>>> On Oct 10, 2018, at 4:34 PM, Matthias Braun  wrote:
>>> 
>>> 1) So I went and figured out why the lldb testcase at hand fails.
>>> 
>>> - It seems the debugger stepping logic will follow the program along until 
>>> it finds a new source location. However if that new source location is part 
>>> of a new DW_AT_abstract_location it is ignored in the step over mode.
>>> - In the testcase at hand the .loc location of an inlined function was 
>>> moved to an earlier place without the DW_AT_abstract_location entry getting 
>>> extended. So the debugger mistook the inlined function as the same scope 
>>> and stepping incorrectly halted there.
>>> 
>>> On the LLVM side DW_AT_abstract_location is generated by LexicalScopes 
>>> which is created by lib/CodeGen/LexicalScopes.cpp / extractLexicalScopes() 
>>> with completely separate code from the line table generation code in 
>>> lib/CodeGen/AsmPrinter/DwarfDebug.cpp you have to be aware of that and keep 
>>> the two algorithms in sync :-/
>>> 
>>> I fixed LexicalScopes.cpp to be in sync and the lldb test works fine again 
>>> for me.
>>> 
>>> 2) However talking to Adrian earlier he also expressed having a bad feeling 
>>> about moving debug location upwards instead of emitting line-0 entries. So 
>>> I think consensus here is to rather live with some line table bloat instead.
>>> 
>>> - Unless there are objections I will not go with option 1) but instead 
>>> revert this commit tomorrow. Note that I will only revert r343874 (i.e. the 
>>> behavior change for what to emit when the first instructions of a basic 
>>> block have no location attached), but not r343895 (removing debuglocs from 
>>> spill/reload instructions) as I still consider this a sensible commit. So 
>>> in the end we may have bigger line tables than before my changes, but 
>>> simpler code in the spill/reload generation and occasionally can avoid 
>>> random source location when spill/reloads get rescheduled.
>>> 
>>> - Matthias
>>> 
>>> 
 On Oct 10, 2018, at 1:17 PM, via llvm-commits 
  wrote:
 
 
 
> -Original Message-
> From: Matthias Braun [mailto:ma...@braunis.de]
> Sent: Wednesday, October 10, 2018 3:50 PM
> To: Robinson, Paul
> Cc: jing...@apple.com; v...@apple.com; llvm-comm...@lists.llvm.org; lldb-
> d...@lists.llvm.org
> Subject: Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in
> case of missing location at block begin
> 
> 
> 
>> On Oct 10, 2018, at 12:18 PM, via llvm-commits  comm...@lists.llvm.org> wrote:
>> 
>> 
>> 
>>> -Original Message-
>>> From: jing...@apple.com [mailto:jing...@apple.com]
>>> Sent: Wednesday, October 10, 2018 2:20 PM
>>> To: Vedant Kumar
>>> Cc: Robinson, Paul; Vedant Kumar via llvm-commits; LLDB; Matthias Braun
>>> Subject: Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location
> in
>>> case of missing location at block begin
>>> 
>>> 
>>> 
 On Oct 10, 2018, at 9:54 AM, Vedant Kumar  wrote:
 
> On Oct 10, 2018, at 9:16 AM, Matthias Braun  wrote:
> 
> So I haven't worked much on debug info, but here's the explanation
> for
>>> my patches:
> My original motivation was getting rid of code some code in the llvm
>>> codegen that for spills and reloads just picked the next debug location
>>> around. That just seemed wrong to me, as spills and reloads really are
>>> bookkeeping, we just move values around between registers and memory
> and
>>> none of it is related to anything the user wrote in the source program.
> So
>>> not 

[lldb-dev] Announcing some new bots on green-dragon

[Adrian Prantl via lldb-dev]

Some of you may have noticed that green dragon underwent some changes this last 
week.

On http://green.lab.llvm.org/green/view/LLDB/ we now have the following 
configurations that are being tested:

- http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/ [New configuration 
in zorg]
  Release+Asserts build of the most current llvm/clang/lldb.


- http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/ [New!]
  Release+Asserts build of the most current LLDB, running the testsuite against 
clang 5.0.2.


- http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-6.0.1/ [New!]
  Release+Asserts build of the most current LLDB, running the testsuite against 
clang 6.0.1.


- http://green.lab.llvm.org/green/view/LLDB/job/lldb-sanitized/ [New!]
  RelWithDebInfo build of the most current LLDB
  with address and undefined behavior sanitizer enabled,
  running the testsuite against the latest (non-sanitized) stage1 compiler.


- http://green.lab.llvm.org/green/view/LLDB/job/lldb-xcode/
  LLDB being built from the Xcode project. (This one hasn't changed).


Note that the sanitizer bot is currently failing because it actually detected 
some UB issues in the data formatters — so it already payed for itself.

I hope that especially the sanitizer bot will prove to be very useful for the 
community. All of the new bots are incremental; they produce a new result ~ 
every 15 min depending on how much changed.

Please let me know if you run into any issues with the new configurations!
-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 15, 2018, at 10:31 AM, Александр Поляков  
> wrote:
> 
> I built an lldb myself. As far as I remember, this error has been seen in 
> swift's CI on machines running Linux. Adrian, could you correct me if I'm 
> wrong?
> 

Yes, we have seen these test fail regularly, but I don't have any log files 
from the CI machines, so I can't confirm the the issue you are seeing was the 
same one that our bots encountered.

-- adrian___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 14, 2018, at 2:43 PM, Александр Поляков  wrote:
> 
> Here is what I got from gdb-remote packet log:
> 
> (gdb)
> lldb-mi  <   1> send packet: +
> lldb-mi  history[1] tid=0x784a <   1> send packet: +
> lldb-mi  <  19> send packet: $QStartNoAckMode#b0
> lldb-mi  <   1> read packet: +
> lldb-mi  <   6> read packet: $OK#9a
> lldb-mi  <   1> send packet: +
> lldb-mi  <  41> send packet: $qSupported:xmlRegisters=i386,arm,mips#12
> lldb-mi  < 124> read packet: 
> $PacketSize=2;QStartNoAckMode+;QThreadSuffixSupported+;QListThreadsInStopReply+;qEcho+;QPassSignals+;qXfer:auxv:read+#be
> lldb-mi  <  26> send packet: $QThreadSuffixSupported#e4
> lldb-mi  <   6> read packet: $OK#9a
> lldb-mi  <  27> send packet: $QListThreadsInStopReply#21
> lldb-mi  <   6> read packet: $OK#9a
> lldb-mi  <  13> send packet: $qHostInfo#9b
> lldb-mi  <  11> send packet: $qEcho:1#5b
> 
> Could somebody help me with understanding of what is happening here?
> 

Just to clarify: Is this from a session that failed with the symptoms you 
described earlier?
I'm not familiar with the protocol, but the fact that there are send and read 
log entries makes it sound like the communication itself is working.

-- adrian___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 14, 2018, at 2:13 PM, Александр Поляков  wrote:
> 
> After long-long debugging I found out that lldb-mi can't successfully launch 
> a process since sometimes it isn't connected with something(I don't know what 
> is it).
> I found out that it fails after `if (IsConnected())` from 
> `GDBRemoteCommunication::SendPacketNoLock(llvm::StringRef payload)`.
> 
> Do you have any thoughts about this?

You mean that the connection to the debugserver/gdbserver isn't working? How 
does lldb-mi negoiate the port for communicating with the debugger server? Does 
it use the same mechanism as lldb? I'm asking because I remember that a while 
ago I fixed a bug in lldb's random port selection code that could cause two 
concurrently running instances to pick the same port. Does it look like the 
connection never works, or is it being droppped?

+Jim, do you happen to have an idea what this could be a symptom of?

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 14, 2018, at 12:11 PM, Александр Поляков  
> wrote:
> 
> It seems that the real problem is that sometimes lldb-mi can't launch a 
> process:
> 
> build/bin/lldb-mi --synchronous a.out < 
> llvm/tools/lldb/lit/tools/lldb-mi/exec/exec-step-instruction.test 
> (gdb)
> -file-exec-and-symbols "a.out"
> ^done
> (gdb)
> ^done
> (gdb)
> =library-loaded,id="/home/alexander/workspace/gsoc/a.out",target-name="/home/alexander/workspace/gsoc/a.out",host-name="/home/alexander/workspace/gsoc/a.out",symbols-loaded="0",loaded_addr="-",size="0"
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x004004df",func="main",file="main.c",fullname="/home/alexander/workspace/gsoc/llvm/tools/lldb/lit/tools/lldb-mi/exec/inputs/main.c",line="6",times="0",original-location="main"}
> (gdb)
> =breakpoint-modified,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x004004df",func="main",file="main.c",fullname="/home/alexander/workspace/gsoc/llvm/tools/lldb/lit/tools/lldb-mi/exec/inputs/main.c",line="6",times="0",original-location="main"}
> (gdb)
> ^done
> (gdb)
> ^error,msg="process launch failed: 'A' packet returned an error: -1"

Do you think you might be able to improve the error message here and get at the 
underlying failure?

-- adrian

> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^error,msg="Command 'exec-step-instruction'. Thread ID invalid"
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^error,msg="Command 'exec-next-instruction'. Thread ID invalid"
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^error,msg="this SBThread object is invalid"
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> 
> exec-run executes fine in synchronous mode(the only bad thing here is that 
> lldb-mi doesn't react to any external events like SIGSTOP - ctrl+C, while 
> lldb does). All previous logs were made in asynchronous mode and are wrong.
> 
> On Tue, Aug 14, 2018 at 2:36 AM Adrian Prantl  > wrote:
> 
> 
>> On Aug 13, 2018, at 4:19 PM, Александр Поляков > > wrote:
>> 
>> Yes, I do, I'm able to pass any command to lldb-mi before the breakpoint is 
>> hit. I guess that making exec-run to block the control until the process 
>> stops might be the solution we are looking for.
> 
> Right. I believe that it should (in synchronous mode) behave like "run" in 
> the normal lldb command interpreter and block until the process has stopped. 
> You could look at differences in the implementation of -exec-run and "run" in 
> the command interpreter first.
> 
> -- adrian
> 
> 
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 13, 2018, at 4:19 PM, Александр Поляков  wrote:
> 
> Yes, I do, I'm able to pass any command to lldb-mi before the breakpoint is 
> hit. I guess that making exec-run to block the control until the process 
> stops might be the solution we are looking for.

Right. I believe that it should (in synchronous mode) behave like "run" in the 
normal lldb command interpreter and block until the process has stopped. You 
could look at differences in the implementation of -exec-run and "run" in the 
command interpreter first.

-- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 13, 2018, at 11:54 AM, Александр Поляков  
> wrote:
> 
> Oops, I made a mistake, I typed break-insert main. Here is the right output:
> 
> build/bin/lldb-mi a.out 
> (gdb)
> -file-exec-and-symbols "a.out"
> ^done
> (gdb)
> =library-loaded,id="/home/alexander/workspace/gsoc/a.out",target-name="/home/alexander/workspace/gsoc/a.out",host-name="/home/alexander/workspace/gsoc/a.out",symbols-loaded="0",loaded_addr="-",size="0"
> -exec-run
> ^running
> =thread-group-started,id="i1",pid="6406"
> (gdb)
> =thread-created,id="1",group-id="i1"
> =thread-selected,id="1"
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/ld-2.23.so 
> ",target-name="/lib/x86_64-linux-gnu/ld-2.23.so 
> ",host-name="/lib/x86_64-linux-gnu/ld-2.23.so 
> ",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/ld-2.23.so
>  ",loaded_addr="-",size="0"
> (gdb)
> =library-loaded,id="[vdso]",target-name="[vdso]",host-name="[vdso]",symbols-loaded="1",symbols-path="",loaded_addr="0x77ffa000",size="0"
> (gdb)
> =library-loaded,id="/home/alexander/workspace/gsoc/a.out",target-name="/home/alexander/workspace/gsoc/a.out",host-name="/home/alexander/workspace/gsoc/a.out",symbols-loaded="0",loaded_addr="-",size="0"
> (gdb)
> *running,thread-id="all"
> (gdb)
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/libc.so.6",target-name="/lib/x86_64-linux-gnu/libc.so.6",host-name="/lib/x86_64-linux-gnu/libc.so.6",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/libc-2.23.so
>  ",loaded_addr="-",size="0"
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/libc.so.6",target-name="/lib/x86_64-linux-gnu/libc.so.6",host-name="/lib/x86_64-linux-gnu/libc.so.6",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/libc-2.23.so
>  ",loaded_addr="-",size="0"
> -break-insert func
> ^done,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x00400514",func="func",file="test.c",fullname="/home/alexander/workspace/gsoc/llvm/tools/lldb/lit/tools/lldb-mi/exec/inputs/test.c",line="2",times="0",original-location="func"}
> (gdb)
> =breakpoint-modified,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x00400514",func="func",file="test.c",fullname="/home/alexander/workspace/gsoc/llvm/tools/lldb/lit/tools/lldb-mi/exec/inputs/test.c",line="2",times="0",original-location="func"}
> (gdb)
> =breakpoint-modified,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x00400514",func="func",file="test.c",fullname="/home/alexander/workspace/gsoc/llvm/tools/lldb/lit/tools/lldb-mi/exec/inputs/test.c",line="2",times="0",original-location="func"}
> (gdb)
> -exec-next
> ^error,msg="Resume timed out."
> (gdb)
> (gdb)
> *stopped,reason="breakpoint-hit",disp="del",bkptno="1",frame={level="0",addr="0x00400514",func="func",args=[],file="test.c",fullname="/home/alexander/workspace/gsoc/llvm/tools/lldb/lit/tools/lldb-mi/exec/inputs/test.c",line="2"},thread-id="1",stopped-threads="all"

Do you get control back before the breakpoint is hit? If yes, should -exec-run 
block until the process stops (in synchronous mode)?


> (gdb)
> (gdb)
> *running,thread-id="all"
> (gdb)
> (gdb)
> *stopped,reason="breakpoint-hit",disp="del",bkptno="1",frame={level="0",addr="0x00400514",func="func",args=[],file="test.c",fullname="/home/alexander/workspace/gsoc/llvm/tools/lldb/lit/tools/lldb-mi/exec/inputs/test.c",line="2"},thread-id="1",stopped-threads="all"
> (gdb)
> 
> On Mon, Aug 13, 2018 at 9:40 PM Александр Поляков  > wrote:
> Sure, this is the log with the typed commands:
> 
> build/bin/lldb-mi a.out 
> (gdb)
> -file-exec-and-symbols "a.out"
> ^done
> (gdb)
> =library-loaded,id="/home/alexander/workspace/gsoc/a.out",target-name="/home/alexander/workspace/gsoc/a.out",host-name="/home/alexander/workspace/gsoc/a.out",symbols-loaded="0",loaded_addr="-",size="0"
> -exec-run
> ^running
> =thread-group-started,id="i1",pid="4410"
> (gdb)
> =thread-created,id="1",group-id="i1"
> =thread-selected,id="1"
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/ld-2.23.so 
> ",target-name="/lib/x86_64-linux-gnu/ld-2.23.so 
> ",host-name="/lib/x86_64-linux-gnu/ld-2.23.so 
> ",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/ld-2.23.so
>  ",loaded_addr="-",size="0"
> (gdb)
> =library-loaded,id="[vdso]",target-name="[vdso]",host-name="[vdso]",symbols-loaded="1",symbols-path="",loaded_addr="0x77ffa000",size="0"
> (gdb)
> =library-loaded,id="/home/alexander/workspace/gsoc/a.out",target-name="/home/alexander/workspace/gsoc/a.out",host-name="/home/alexander/workspace/gsoc/a.out",symbols-loaded="0",loaded_addr="-",size="0"
> (gdb)
> *running,thread-id="all"
> (gdb)
> (gdb)
> 

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 11, 2018, at 3:58 AM, Александр Поляков  wrote:
> 
> I reproduced the test you suggested and got following output:
> 
> build/bin/lldb-mi --synchronous a.out < 
> llvm/tools/lldb/lit/tools/lldb-mi/exec/lldb-mi-fail.test 
> (gdb)
> -file-exec-and-symbols "a.out"
> ^done
> (gdb)
> ^done
> (gdb)
> =library-loaded,id="/home/alexander/workspace/gsoc/a.out",target-name="/home/alexander/workspace/gsoc/a.out",host-name="/home/alexander/workspace/gsoc/a.out",symbols-loaded="0",loaded_addr="-",size="0"
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^done,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x000f",func="??",file="??",fullname="??/??",line="0",times="0",original-location="f"}
> (gdb)
> =breakpoint-modified,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x000f",func="??",file="??",fullname="??/??",line="0",times="0",original-location="f"}
> (gdb)
> ^done
> (gdb)
> ^running
> =thread-group-started,id="i1",pid="5075"
> (gdb)
> =thread-created,id="1",group-id="i1"
> =thread-selected,id="1"
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/ld-2.23.so",target-name="/lib/x86_64-linux-gnu/ld-2.23.so",host-name="/lib/x86_64-linux-gnu/ld-2.23.so",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/ld-2.23.so",loaded_addr="-",size="0"
> (gdb)
> =library-loaded,id="[vdso]",target-name="[vdso]",host-name="[vdso]",symbols-loaded="1",symbols-path="",loaded_addr="0x77ffa000",size="0"
> (gdb)
> =library-loaded,id="/home/alexander/workspace/gsoc/a.out",target-name="/home/alexander/workspace/gsoc/a.out",host-name="/home/alexander/workspace/gsoc/a.out",symbols-loaded="0",loaded_addr="-",size="0"
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/libc.so.6",target-name="/lib/x86_64-linux-gnu/libc.so.6",host-name="/lib/x86_64-linux-gnu/libc.so.6",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/libc-2.23.so",loaded_addr="-",size="0"
> (gdb)
> =library-loaded,id="/lib/x86_64-linux-gnu/libc.so.6",target-name="/lib/x86_64-linux-gnu/libc.so.6",host-name="/lib/x86_64-linux-gnu/libc.so.6",symbols-loaded="1",symbols-path="/usr/lib/debug/lib/x86_64-linux-gnu/libc-2.23.so",loaded_addr="-",size="0"
> (gdb)
> =thread-exited,id="1",group-id="i1"
> =thread-group-exited,id="i1",exit-code="0"
> *stopped,reason="exited-normally"
> (gdb)
> ^done
> (gdb)
> ^done
> (gdb)
> ^error,msg="Resume timed out."
> (gdb)
> ^done
> (gdb)
> 
> As a command that needs a breakpoint to be hit I chose the -exec-next 
> --thread 1. It's the same error which is seen on the bots.

Can you help me understand the order in which events happened in that log? I 
don't see the -exec-next command in the log. If you type in the commands 
manually, does it become obvious which ones are blocking and which ones aren't 
(but should be)?

-- adrian 
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

That't interesting. Where you able to simulate the error seen on the bots by 
inserting a sleep() unto the debugged program?

-- adrian

> On Aug 10, 2018, at 5:06 PM, Александр Поляков  wrote:
> 
> It seems that lldb-mi in such a situation just hangs:
> 
> build/bin/lldb-mi --synchronous bash
> (gdb)
> -file-exec-and-symbols "bash"
> ^done
> (gdb)
> =library-loaded,id="/bin/bash",target-name="/bin/bash",host-name="/bin/bash",symbols-loaded="0",loaded_addr="-",size="0"
> -exec-run
> ^C^C
> exit
> 
> It doesn't react to Ctrl+C and exit command.
> 
> сб, 11 авг. 2018 г. в 2:54, Adrian Prantl  >:
> 
> 
>> On Aug 10, 2018, at 4:50 PM, Александр Поляков > > wrote:
>> 
>> One important question: what do you mean talking about "block"? Does it mean 
>> that SBTarget::Launch blocks the process and the user can't continue working 
>> with this process until it stops?
> 
> Pretty much. The same way as the interactive (lldb) command line interface 
> behaves: You enter "run" and you won't get a prompt again until the process 
> stops. I'm imagining (but haven't verified) that synchronous mode behaves 
> like that.
> 
> -- adrian
> 
> 
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 10, 2018, at 4:50 PM, Александр Поляков  wrote:
> 
> One important question: what do you mean talking about "block"? Does it mean 
> that SBTarget::Launch blocks the process and the user can't continue working 
> with this process until it stops?

Pretty much. The same way as the interactive (lldb) command line interface 
behaves: You enter "run" and you won't get a prompt again until the process 
stops. I'm imagining (but haven't verified) that synchronous mode behaves like 
that.

-- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

I was wondering how this worked in the regular SBAPI that we use for all the 
"normal" python-based LLDB tests. The implementation of SBProcess::Continue() 
for example calls Process::Resume() or Process::ResumeSynchronous() depending 
on whether synchronous mode is set or not.
It's not immediately obvious to me whether -exec-run should wait until the 
process stopped before returning or whether -exec-step should wait until the 
process stopped before executing.

Based on a cursory reading of the sources it seems like SBTarget::Launch should 
block until the process stopped when it is in synchronous mode. Can you confirm 
this? If that is the case, can you figure out why -exec-run does not inherit 
this behavior?

-- adrian

> On Aug 10, 2018, at 4:27 PM, Александр Поляков  wrote:
> 
> AFAIK, there is no mechanism in lldb-mi to distinguish a command that expects 
> a frame, so we need to modify each command manually. Am I right?
> If so, I found the Process::WaitForProcessToStop method which we can add to 
> SB API and use in lldb-mi.
> 
> сб, 11 авг. 2018 г. в 0:50, Adrian Prantl :
> [adding lldb-dev back to the conversation]
> 
> > On Aug 10, 2018, at 2:37 PM, Adrian Prantl  wrote:
> > 
> > 
> > 
> >> On Aug 10, 2018, at 2:25 PM, Александр Поляков  
> >> wrote:
> >> 
> >> I didn't check this yet. lldb-mi already runs LIT test in the 
> >> --synchronous mode and the tests keep failing.
> >> 
> > 
> > Yes, that's why I said this:
> > 
> > 
> >>> пт, 10 авг. 2018 г. в 23:57, Adrian Prantl :
> >>> 
> >>> Before we continue to discuss -wait-for-breakpoint; where you actually 
> >>> able to verify my suspicion that that is what is happening on the bots? 
> >>> Fred suggested to me offline today that in synchronous mode, perhaps 
> >>> -exec-* should be waiting for the process to be stopped, which would also 
> >>> sound like a reasonable and less invasive solution to the problem.
> >>> 
> >> 
> > 
> > Instead of adding a new command to wait for the process to be stopped we 
> > might be able to just wait for the process to be stopped if in synchronous 
> > mode and we are running any commands that expect a frame (such as -exec-*).
> > 
> > -- adrian
> 
> 
> 
> -- 
> Alexander

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

[adding lldb-dev back to the conversation]

> On Aug 10, 2018, at 2:37 PM, Adrian Prantl  wrote:
> 
> 
> 
>> On Aug 10, 2018, at 2:25 PM, Александр Поляков  
>> wrote:
>> 
>> I didn't check this yet. lldb-mi already runs LIT test in the --synchronous 
>> mode and the tests keep failing.
>> 
> 
> Yes, that's why I said this:
> 
> 
>>> пт, 10 авг. 2018 г. в 23:57, Adrian Prantl :
>>> 
>>> Before we continue to discuss -wait-for-breakpoint; where you actually able 
>>> to verify my suspicion that that is what is happening on the bots? Fred 
>>> suggested to me offline today that in synchronous mode, perhaps -exec-* 
>>> should be waiting for the process to be stopped, which would also sound 
>>> like a reasonable and less invasive solution to the problem.
>>> 
>> 
> 
> Instead of adding a new command to wait for the process to be stopped we 
> might be able to just wait for the process to be stopped if in synchronous 
> mode and we are running any commands that expect a frame (such as -exec-*).
> 
> -- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

> On Aug 10, 2018, at 11:13 AM, Adrian Prantl via lldb-dev 
>  wrote:
> 
> [+lldb-dev, since this affects all bots equally]
> 
> Alexander, it looks like there is a race condition in some of the LIT-based 
> lldb-mi tests.
> 
> For example, exec-next.test:
> 
>  # Test lldb-mi -exec-next command.
> 
>  # Check that we have a valid target created via '%lldbmi %t'.
>  # CHECK: ^done
> 
>  -break-insert main
>  # CHECK: ^done,bkpt={number="1"
> 
>  -exec-run
>  # CHECK: ^running
>  # CHECK: *stopped,reason="breakpoint-hit"
> 
>  -exec-next --thread 0
>  # Check that exec-next can process the case of invalid thread ID.
>  # CHECK: ^error,msg="Command 'exec-next'. Thread ID invalid"
> 
>  ...
> 
> Here we are not actually waiting for the breakpoint to be hit. Synchronous 
> mode ensures that the lldb-mi driver waits for each command to be completed, 
> but that only means that -exec-run only returns after the process has been 
> launched and does not include waiting for the program to actually hit a 
> breakpoint. So if the program runs very slowly (such as often happens on a 
> very busy bot), the -exec-next and subsequent commands are scheduled before 
> the breakpoint is hit. In order to fix this we either need to move any tests 
> that schedule commands after hitting breakpoints back to Python, or we need 
> to add a new -wait-for-breakpoint MI command for testing only to force a 
> synchronization point.

In order to test my hypothesis, you could set the breakpoint in a different 
function than main and insert a sleep() into the program to simulate the 
condition on the bots, i.e.:

  void foo() {
// actual test code below
...
  }

  int main(int argc, char ** argv) {
sleep(10);
foo();
  }

and in the test:

  # instead of main:
  -break-insert foo
  # and then :-)
  -some-command-that-would-fail-if-executed-before-breakpoint-is-hit

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] Failing LIT-based lldb-mi tests

[Adrian Prantl via lldb-dev]

[+lldb-dev, since this affects all bots equally]

Alexander, it looks like there is a race condition in some of the LIT-based 
lldb-mi tests.

For example, exec-next.test:

  # Test lldb-mi -exec-next command.

  # Check that we have a valid target created via '%lldbmi %t'.
  # CHECK: ^done

  -break-insert main
  # CHECK: ^done,bkpt={number="1"

  -exec-run
  # CHECK: ^running
  # CHECK: *stopped,reason="breakpoint-hit"

  -exec-next --thread 0
  # Check that exec-next can process the case of invalid thread ID.
  # CHECK: ^error,msg="Command 'exec-next'. Thread ID invalid"

  ...

Here we are not actually waiting for the breakpoint to be hit. Synchronous mode 
ensures that the lldb-mi driver waits for each command to be completed, but 
that only means that -exec-run only returns after the process has been launched 
and does not include waiting for the program to actually hit a breakpoint. So 
if the program runs very slowly (such as often happens on a very busy bot), the 
-exec-next and subsequent commands are scheduled before the breakpoint is hit. 
In order to fix this we either need to move any tests that schedule commands 
after hitting breakpoints back to Python, or we need to add a new 
-wait-for-breakpoint MI command for testing only to force a synchronization 
point.

Could you please take look?

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] RFC: libtrace

[Adrian Prantl via lldb-dev]

> On Jun 26, 2018, at 1:38 PM, Zachary Turner  wrote:
> 
>> On Tue, Jun 26, 2018 at 1:28 PM Adrian Prantl  wrote:
>> 
>>> > On Jun 26, 2018, at 11:58 AM, Zachary Turner via llvm-dev 
>>> >  wrote:
>>> > A good example of this would be LLDB’s DWARF parsing code, which is more 
>>> > featureful than LLVM’s but has kind of evolved in parallel.  Sinking this 
>>> > into LLVM would be one early target of such an effort, although over time 
>>> > there would likely be more.
>>> 
>>> As you are undoubtedly aware we've been carefully rearchitecting LLVM's 
>>> DWARF parser over the last few years to eventually become featureful enough 
>>> so that LLDB could use it, so any help on that front would be most welcome. 
>>> As long as we are careful to not regress in performance/lazyness, features 
>>> and fault-tolerance, deduplicating the implementations can only be good for 
>>> LLVM and LLDB.
>>> 
>> Yea, this is the general idea.   Has anyone actively been working on this 
>> specific effort recently?  To my knowledge someone started and then never 
>> finished, but the efforts also never made it upstream, so my understanding 
>> is that it's a goal, but one that nobody has made significant headway on.
> 
That's not true. Greg Clayton started the effort in 2016 and landed many of the 
ground-breaking changes. The design ideas fleshed out during that initial 
effort (thanks to David Blaikie who spent a lot of time reviewing the new 
interfaces!) such as improved error handling where then picked up the entire 
team of contributors who worked on DWARF 5 support in LLVM and we've continued 
down that path ever since. The greatly improved llvm-dwarfdump was also born 
out of this effort, for example. We also payed attention that every refactoring 
of LLDB DWARF parser code would bring it closer to the new LLVM parser 
interface to narrow the gaps between the implementations.

-- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] RFC: libtrace

[Adrian Prantl via lldb-dev]

> On Jun 26, 2018, at 11:58 AM, Zachary Turner via llvm-dev 
>  wrote:
> 
> Hi all,
> 
> We have been thinking internally about a lightweight llvm-based ptracer.  To 
> address one question up front: the primary way in which this differs from 
> LLDB is that it targets a more narrow use case -- there is no scripting 
> support, no clang integration, no dynamic extensibility, no support for 
> running jitted code in the target, and no user interface.  We have several 
> use cases internally that call for varying levels of functionality from such 
> a utility, and being able to use as little as possible of the library as is 
> necessary for the given task is important for the scale in which we wish to 
> use it. 
> 
> We are still in early discussions and planning, but I think this would be a 
> good addition to the LLVM upstream.  Since we’re approaching this as a set of 
> small isolated components, my thinking is to work on this completely 
> upstream, directly under the llvm project (as opposed to making a separate 
> subproject), but I’m open to discussion if anyone feels differently.
> 
> LLDB has solved a lot of the difficult problems needed for such a tool.  So 
> in the spirit of code reuse, we think it’s worth trying componentize LLDB by 
> sinking pieces into LLVM and rebasing LLDB as well as these smaller tools on 
> top of these components, so that smaller tools can reduce code duplication 
> and contribute to the overall health of the code base.

Do you have a rough idea of what components specifically the new tool would 
need to function?

>  At the same time we think that in doing so we can break things up into more 
> granular pieces, ultimately exposing a larger testing surface and enabling us 
> to create exhaustive tests, giving LLDB more fine grained testing of 
> important subsystems.

Are you thinking of the new utility as something that would naturally live in 
llvm/tools or as something that would live in the LLDB repository?

> 
> A good example of this would be LLDB’s DWARF parsing code, which is more 
> featureful than LLVM’s but has kind of evolved in parallel.  Sinking this 
> into LLVM would be one early target of such an effort, although over time 
> there would likely be more.

As you are undoubtedly aware we've been carefully rearchitecting LLVM's DWARF 
parser over the last few years to eventually become featureful enough so that 
LLDB could use it, so any help on that front would be most welcome. As long as 
we are careful to not regress in performance/lazyness, features and 
fault-tolerance, deduplicating the implementations can only be good for LLVM 
and LLDB.

-- adrian

> 
> Anyone have any thoughts / strong opinions on this proposal, or where the 
> code should live?  Also, does anyone have any suggestions on things they’d 
> like to see come out of this?  Whether it’s a specific new tool, new 
> functionality to an existing tool, an architectural or design change to some 
> existing tool or library, or something else entirely, all feedback and ideas 
> are welcome.
> 
> Thanks,
> Zach
> 
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Adrian Prantl via lldb-dev]

> On Jun 14, 2018, at 7:01 AM, Pavel Labath via llvm-dev 
>  wrote:
> 
> Thank you all. I am going to try to reply to all comments in a single email.
> 
> Regarding the  .apple_objc idea, I am afraid the situation is not as
> simple as just flipping a switch.

Jonas is currently working on adding the support for DWARF5-style Objective-C 
accelerator tables to LLVM/LLDB/dsymutil. Based on the assumption that DWARF 4 
and earlier are unaffected by any of this, I don't think it's necessary to 
spend any effort of making the transition smooth. I'm fine with having 
Objective-C on DWARF 5 broken on trunk for two weeks until Jonas is done adding 
Objective-C support to the DWARF 5 implementation.


> (If it was, I don't think I would
> have embarked on this adventure in the first place -- I would just
> emit .apple_*** everywhere and call it done :)). The issue is that the
> apple tables have assumptions about the macos debug info distribution
> model hardcoded in them -- they assume they will either stay in the .o
> file or be linked by a smart debug-info-aware linker (dsymutil). In
> particular, this means they are not self-delimiting (no length field
> as is typical for other dwarf artifacts), so if a linker which is not
> aware of them would simply concatenate individual .o tables (which elf
> linkers are really good at), the debugger would have no way to pry
> them apart. And even if it somehow managed that, it still wouldn't
> know if the indexes covered all of the compile units in the linked
> file or only some of them (in case some of the object files were
> compiled with the tables and some without).
> 
> In light of that, I don't think it's worth trying to combine
> .apple_objc with .debug_names in some way, and it would be much
> simpler to just extend .debug_names with the necessary information. I
> think the simplest way of achieving this (one which would require
> least amount of standard-bending) is to take the index entry for the
> objc class and add a special attribute to it (DW_IDX_method_list?)
> with form DW_FORM_blockXXX and just have the references to the method
> DIEs in the block data. This should make the implementation an almost
> drop-in for the current .apple_objc functionality (we would still need
> to figure out what to do with category methods, but it's not clear to
> me whether lldb actually uses those anywhere).
> 
> But, other options may be possible as well. What's not clear to me is
> whether these tables couldn't be replaced by extra information in the
> .debug_info section. It seems to me that these tables are trying to
> work around the issue that there is no straight way to go from a
> DW_TAG_structure type DIE describing an ObjC class to it's methods. If
> these methods (their forward declarations) were be present as children
> of the type DIE (as they are for c++ classes), then these tables may
> not be necessary. But maybe (probably) that has already been
> considered and deemed infeasible for some reason. In any case this
> seemed like a thing best left for people who actually work on ObjC
> support to figure out.

That's really a question for Greg or Jim — I don't know why the current 
representation has the Objective-C methods outside of the structs. One reason 
might be that an interface's implementation can define more methods than are 
visible in its public interface in the header file, but we already seem to be 
aware of this and mark the implementation with DW_AT_APPLE_objc_complete_type. 
I also am not sure that this is the *only* reason for the objc accelerator 
table. But I'd like to learn.

-- adrian

> As far as the .debug_names size goes, I should also point out that the
> binary in question was built with -fno-limit-debug-info, which isn't a
> default setup on linux. I have tried measuring the sizes without that
> flag and with fission enabled (-gsplit-dwarf) and the results are:
> without compression:
> - clang binary: 960 MB
> - .debug_names: 130 MB (13%)
> - debug_pubnames: 175 MB (18%)
> - debug_pubtypes: 204 MB (21%)
> - median time for setting a breakpoint on non-existent function
> (variance +/- 2%):
> real 0m3.526s
> user 0m3.156s
> sys 0m0.364s
> 
> with -Wl,--compress-debug-sections=zlib:
> - clang binary: 440 MB
> - .debug_names: 80MB (18%)
> - .debug_pubnames: 31 MB (7.2%)
> - .debug_pubtypes: 42MB (9.5%)
> - median time for setting a breakpoint on non-existent function:
> real 0m4.369s
> user 0m3.948s
> sys 0m0.416s
> 
> So, .debug_names indeed compresses worse than .debug_pubnames/types,
> but that is not surprising as it has a more condensed encoding to
> begin with (no inline strings). However, even in it's compressed form
> its size is only slightly larger that the two other sections combined
> (while being infinitely more useful). As for the compression, my
> takeaway from this is that compression definitely has a measurable
> impact on startup time, but, on the grand scale of things, the impact
> is not actually that big. And if a 

Re: [lldb-dev] [cfe-dev] [RFC] Improving import failure checking strategy inside ASTImporter

[Adrian Prantl via lldb-dev]

Adding lldb-dev to the thread, since LLDB is a primary user of ASTImporter.

> On Apr 27, 2018, at 8:54 AM, Aleksei Sidorin via cfe-dev 
>  wrote:
> 
> Hello Gabor,
> 
> Thank you for the reply!
> 
> 26.04.2018 12:39, Gábor Márton пишет:
>> Hi Aleksei,
>> 
>> It is a great and welcoming initiative to identify the weaknesses in
>> the ASTImporter.
>> 
>> About, handling the error case after a call:
>> I couldn't agree more that this is a burden to always have a check at
>> the call site, but this is the price we pay for not having exceptions.
>> By using macros I have a feeling that the code would be even less
>> readable because we would hide control flow modifications inside the
>> macro, also because finally we'd need to create so many different
>> kinds of macros.
>> As far as I know, there are cases where an `Import` can return a
>> `nullptr` or a default even in case of non-error situations, please
>> correct me if I am wrong. Thus, introducing an optional like return
>> type would require to discover all these cases. Still, I think this
>> would worth it, because the enforcement of error checking (as you
>> mentioned), plus it would make it more explicit which situation is
>> really an error.
> Yes, this is what I described as problem (2).
> 
>> Actually, I think this should be done with a few large commits, e.g.
>> once we change the return type of `ASTImporter::Import(Decl*)` to
>> `Optional` then we have to change all call sites.
>> By introducing an `ImportOrError` and then doing the changes gradually
>> would result the same final state as if we had done it in one larger
>> step where we changed all call sites, but until all call sites are
>> changed we have to maintain both `Import` and `ImportOrError`.
>> 
>> 
>> Beside the problem of handling the error case after an `Import` call
>> we have observed several other weaknesses, and would like to hear your
>> opinion about them as well.
>> So, the list of problems so far:
>> 
>> - Do the same things after and before a new AST node is created (::Create)
>> The original import logic should be really simple, we create the node,
>> then we mark that as imported, then we recursively import the parts
>> for that node and then set them on that node.
> That sounds pretty good, and this is what I was thinking about some time ago. 
> You mean something like CreateDeserialized?
> The problem here are Types: their creation requires all dependencies to be 
> imported before the Type is created. I don't know how to deal with it.
> 
>> However, the AST is actually a graph, so we have to handle circles.
>> If we mark something as imported (Imported()) then we return with the
>> corresponding To whenever we want to import that node again, this way
>> circles are handled.
>> In order to make this algorithm work we must ensure things, which are
>> not enforced in the current ASTImporter:
>> * There are no Import() calls in between any node creation (::Create)
>> and the Imported() call.
>> * There are no VisitXXX() calls in any other VisitYYY() function, only
>> call of Import() allowed.
>> * Before actually creating an AST node (::Create), we must check if
>> the Node had been imported already, if yes then return with that one.
>> One very important case for this is connected to templates: we may
>> start an import both from the templated decl of a template and from
>> the template itself.
>> There are good news, the first and third points can be enforced by
>> providing a variadic forwarding template for the creation, we are
>> currently preparing to upstream this change from our fork
>> (https://github.com/Ericsson/clang/pull/354/files 
>> ) but this is going
>> to be a huge change.
>> Still, I have no idea how to enforce the second point. (Maybe a checker?)
> If I remember correctly, this pattern is not widely used and such code can be 
> refactored easily.
> 
>> - Decl chains are not imported
>> Currently we import declarations only if there is no available definition.
>> If in the "From" TU there are both a declaration and a definition we
>> import only the definition.
>> Thus we do not import all information from the original AST.
>> One example:
>> struct B { virtual void f(); };
>> void B::f() {}
>> Here, when we import the definition, then the virtual flag is not
>> getting imported, so finally we import a function as a non-virtual
>> function.
>> An other problem is with recursive functions:
>> void f(); void f() { f(); }
>> When we import the prototype then we end up having two identical
>> definitions for f(), and no prototype.
>> We again preparing a patch for this, but this solves the problem only
>> in case of functions.
> Yes, I have met this problem before. I tried to import all redeclaration 
> chain but haven't completed the work. One of problems here is how to stick 
> the redeclaration chain to an existing declaration.
> 
>> By not importing the whole 

[lldb-dev] Welcome Alexander!

[Adrian Prantl via lldb-dev]

Please join me in welcoming Alexander Polyakov, who will be working on cleaning 
up and completing LLDB's lldb-mi fronted as part of his Google Summer Of Code 
project:

Reimplement lldb-mi on top of the LLDB public SB API
https://summerofcode.withgoogle.com/projects/#5427847301169152

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] lldb compiled from source 5X slower than homebrew or default lldb

[Adrian Prantl via lldb-dev]

If this is a Mac I would recommend to build both lldbs in RelWithDebInfo mode 
and then run them through Instruments and compare the traces. I'd be very 
interested in learning what the cause for the slowdown is.

-- adrian

> On Mar 12, 2018, at 2:57 PM, Timothee Cour  wrote:
> 
> after further inspection it turns out it's a performance regression
> between llvm 5 and llvm 6:
> 
> $homebrew_D/Cellar/llvm/5.0.1/lldb -s test_lldb.txt -- prog
> 8 seconds
> $homebrew_D/Cellar/llvm/6.0.0/bin/lldb -s test_lldb.txt -- prog
> 40 seconds
> 
> cat test_lldb.txt:
> ```
> b FOO
> r
> bt
> q
> ```
> 
> prog is some binary I'm debugging. It's unfortunately hard to reduce.
> What are good ways to investigate this?
> 
> 
> 
> On Mon, Mar 12, 2018 at 1:42 PM, Adrian Prantl  wrote:
>> Have you tried diff'ing your cmake invocation against the on used by 
>> homebrew?
>> 
>> -- adrian
>> 
>>> On Mar 9, 2018, at 12:59 AM, Timothee Cour via lldb-dev 
>>>  wrote:
>>> 
>>> while testing out https://github.com/llvm-mirror/lldb/pull/3 I noticed
>>> that when I compiled lldb from source (even un-modified from git HEAD)
>>> it was 5X slower than homebrew lldb or default lldb:
>>> 
>>> I'm compiling lldb as follows:
>>> ```
>>> git clone https://github.com/llvm-mirror/llvm.git
>>> cd llvm/tools
>>> git clone https://github.com/llvm-mirror/lldb
>>> ## also, patch in https://github.com/llvm-mirror/lldb/pull/3
>>> git clone https://github.com/llvm-mirror/clang
>>> cd ..
>>> mkdir build
>>> cd build
>>> ccmake .. -G Ninja
>>> # here I set: CMAKE_BUILD_TYPE  Release
>>> ninja all
>>> ```
>>> 
>>> I also tried setting `LLVM_ENABLE_ASSERTIONS OFF` but no difference
>>> 
>>> 
>>> commands.txt
>>> ```
>>> b FOO
>>> r
>>> bt
>>> q
>>> ```
>>> 
>>> ./test is some program i have.
>>> 
>>> ```
>>> time lldb -s commands.txt -- ./test
>>> 8 seconds
>>> time ./build/bin/lldb -s commands.txt -- ./test
>>> 40 seconds
>>> ```
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] lldb compiled from source 5X slower than homebrew or default lldb

[Adrian Prantl via lldb-dev]

Have you tried diff'ing your cmake invocation against the on used by homebrew?

-- adrian

> On Mar 9, 2018, at 12:59 AM, Timothee Cour via lldb-dev 
>  wrote:
> 
> while testing out https://github.com/llvm-mirror/lldb/pull/3 I noticed
> that when I compiled lldb from source (even un-modified from git HEAD)
> it was 5X slower than homebrew lldb or default lldb:
> 
> I'm compiling lldb as follows:
> ```
> git clone https://github.com/llvm-mirror/llvm.git
> cd llvm/tools
> git clone https://github.com/llvm-mirror/lldb
> ## also, patch in https://github.com/llvm-mirror/lldb/pull/3
> git clone https://github.com/llvm-mirror/clang
> cd ..
> mkdir build
> cd build
> ccmake .. -G Ninja
> # here I set: CMAKE_BUILD_TYPE  Release
> ninja all
> ```
> 
> I also tried setting `LLVM_ENABLE_ASSERTIONS OFF` but no difference
> 
> 
> commands.txt
> ```
> b FOO
> r
> bt
> q
> ```
> 
> ./test is some program i have.
> 
> ```
> time lldb -s commands.txt -- ./test
> 8 seconds
> time ./build/bin/lldb -s commands.txt -- ./test
> 40 seconds
> ```
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [RFC] Testsuite in lldb & possible future directions

[Adrian Prantl via lldb-dev]

> On Feb 6, 2018, at 9:29 AM, Davide Italiano via lldb-dev 
>  wrote:
> 
> On Tue, Feb 6, 2018 at 8:18 AM, Pavel Labath  wrote:
>> On 6 February 2018 at 15:41, Davide Italiano  wrote:
>>> On Tue, Feb 6, 2018 at 7:09 AM, Pavel Labath  wrote:
 On 6 February 2018 at 04:11, Davide Italiano via lldb-dev
 
 So, I guess my question is: are you guys looking into making sure that
 others are also able to reproduce the 0-fail+0-xpass state? I would
 love to run the mac test suite locally, as I tend to touch a lot of
 stuff that impacts all targets, but as it stands now, I have very
 little confidence that the test I am running reflect in any way the
 results you will get when you run the test on your end.
 
 I am ready to supply any test logs or information you need if you want
 to try to tackle this.
 
>>> 
>>> Yes, I'm definitely interested in making the testusuite
>>> working/reliable on any configuration.
>>> I was afraid there were a lot of latent issues, that's why I sent this
>>> mail in the first place.
>>> It's also the reason why I started thinking about `lldb-test` as a
>>> driver for testing, because I found out the testsuite being a little
>>> inconsistent/brittle depending on the environment it's run on (which,
>>> FWIW, doesn't happen when you run lit/FileCheck or even the unit tests
>>> in lldb). I'm not currently claiming switching to a different method
>>> would improve the situation, but it's worth a shot.
>>> 
>> 
>> Despite Zachary's claims, I do not believe this is caused by the test
>> driver (dotest). It's definitely not beautiful, but I haven't seen an
>> issue that would be caused by this in a long time. The issue is that
>> the tests are doing too much -- even the simplest involves compiling a
>> fully working executable, which pulls in a lot of stuff from the
>> environment (runtime libraries, dynamic linker, ...) that we have no
>> control of. And of course it makes it impossible to test the debugging
>> functionality of any other platform than what you currently have in
>> front of you.
>> 
>> In this sense, the current setup makes an excellent integration test
>> suite -- if you run the tests and they pass, you can be fairly
>> confident that the debugging on your system is setup correctly.
>> However, it makes a very bad regression test suite, as the tests will
>> be checking something different on each machine.
>> 
> 
> Yes, I didn't complain about "dotest" in general, but, as you say, the
> fact that it pull in lots of stuffs we don't really have control on.
> Also, most of the times I actually found out we've been sloppy watching
> bots for a while, or XFAILING tests instead of fixing them and that resulted 
> in
> issues piling up). This is a more general problem not necessarily tied to
> `dotest` as a driver.
> 
>> So I believe we need more lightweight tests, and lldb-test can provide
>> us with that. The main question for me (and that's something I don't
> 
> +1.
> 
>> really have an answer to) is how to make writing tests like that easy.
>> E.g. for these "foreign" language plugins, the only way to make a
>> self-contained regression test would be to check-in some dwarf which
>> mimics what the compiler in question would produce. But doing that is
>> extremely tedious as we don't have any tooling for that. Since debug
>> info is very central to what we do, having something like that would
>> go a long way towards improving the testing situation, and it would be
>> useful for C/C++ as well, as we generally need to make sure that we
>> work with a wide range of compiler versions, not just accept what ToT
>> clang happens to produce.
>> 
> 
> I think the plan here (and I'd love to spend some time on this once we
> have stability, which seems we're slowly getting) is that of enhancing
> `yaml2*` to do the work for us.
> I do agree is a major undertaken but even spending a month on it will
> go a long way IMHO. I will try to come up with a plan after discussing
> with folks in my team (I'd really love to get also inputs from DWARF
> people in llvm, e.g. Eric or David Blake).

The last time I looked into yaml2obj was to use it for testing llvm-dwarfdump 
and back then I concluded that it needs a lot of work to be useful even for 
testing dwarfdump. In the current state it is both too low-level (e.g., you 
need to manually specify all Mach-O load commands, you have to manually compute 
and specify the size of each debug info section) and too high-level (it can 
only auto-generate one exact version of .debug_info headers) to be useful.

If we could make a tool whose input roughly looks like the output of dwarfdump, 
then this might be a viable option. Note that I'm not talking about syntax but 
about the abstraction level of the contents.

In summary, I think this is an interesting direction to explore, but we 
shouldn't underestimate the amount of 

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Adrian Prantl via lldb-dev]

> On Jan 30, 2018, at 7:49 AM, Pavel Labath  wrote:
> 
> On 30 January 2018 at 15:41, Adrian Prantl  wrote:
>> 
>> 
>>> On Jan 30, 2018, at 7:35 AM, Pavel Labath  wrote:
>>> 
>>> Hello all,
>>> 
>>> I am looking for feedback regarding implementation of the case folding
>>> algorithm for .debug_names hashes.
>>> 
>>> Unlike the apple tables, the .debug_names hashes are computed from
>>> case-folded names (to enable case-insensitive lookups for languages
>>> where that makes sense). The dwarf5 document specifies that the case
>>> folding should be done according the the "Caseless matching" Section
>>> of the Unicode standard (whose implementation is basically a long list
>>> of special cases). While certainly possible, implementing this would
>>> be much more complicated (and would probably make the code a bit
>>> slower) than a simple tolower(3) call. And the benefits of this are
>>> not really clear to me.
>> 
>> Assuming a UTF-8 encoding, will tolower(3) destroy any non-ASCII characters 
>> in the process? In Swift, for example, we allow a wide range of unicode 
>> characters in identifiers and I want to make sure that this doesn't cause 
>> any problems.
>> 
> 
> I'm not sure what it will do out-of-the-box, but I could certainly
> implement it such that it does not touch the fancy characters.
> 
> However, if we already have unicode characters in the input, then it
> may make sense to go all the way and implement the full folding
> algorithm. Because, once we start producing hashes like this, it will
> be hard to switch to being fully standard-compliant (as that would
> invalidate the existing hashes).
> 
> But the question then is: can I assume the input names will be unicode
> (w/utf8 encoding)?

We can make that happen and encode it explicitly in each compile unit:

> 3.1.1 Full and Partial Compilation Unit Entries
> ...
> A DW_AT_use_UTF8 attribute, which is a flag whose presence indicates that all 
> strings (such as the names of declared entities in the source program, or 
> filenames in the line number table) are represented using the UTF-8 
> representation. 

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Adrian Prantl via lldb-dev]

> On Jan 30, 2018, at 7:35 AM, Pavel Labath  wrote:
> 
> Hello all,
> 
> I am looking for feedback regarding implementation of the case folding
> algorithm for .debug_names hashes.
> 
> Unlike the apple tables, the .debug_names hashes are computed from
> case-folded names (to enable case-insensitive lookups for languages
> where that makes sense). The dwarf5 document specifies that the case
> folding should be done according the the "Caseless matching" Section
> of the Unicode standard (whose implementation is basically a long list
> of special cases). While certainly possible, implementing this would
> be much more complicated (and would probably make the code a bit
> slower) than a simple tolower(3) call. And the benefits of this are
> not really clear to me.

Assuming a UTF-8 encoding, will tolower(3) destroy any non-ASCII characters in 
the process? In Swift, for example, we allow a wide range of unicode characters 
in identifiers and I want to make sure that this doesn't cause any problems.

-- adrian
> 
> Do you know if we already make any promises or assumptions about the
> encoding and/or locale of the symbol names (and here I mainly mean the
> names in the debug info metadata, not llvm symbols).
> 
> If we don't already have a policy about this, then I propose to
> implement the case folding via tolower() (which is compatible with the
> full case folding algorithm, as long as one sticks to basic latin
> characters).
> 
> What do you think?

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 18, 2018, at 5:07 AM, Pavel Labath  wrote:
> 
> Looks like I missed a party. :)
> 
> I'll try to give my thoughts on some of the things that were said here:
> 
>> make -C
> I don't think make -C does what you think it does. "make -C foo" is
> basically equivalent to "cd foo && make", which is what we are doing
> now already. Of course, you can make this work, but you would have to
> pass an extra OUTDIR=... argument to make and then modify the
> Makefiles to reference $(OUTDIR) for its outputs:
> $(OUTDIR)/a.out: main.cc
>  $(CC) -o $(OUTDIR)/a.out main.cc ...
> 
> The standard way of doing an out-of-tree build with make is to have
> the Makefile in the build-directory and to set the magic VPATH
> variable in the Makefile (or as a part of make invocation). VPATH
> alters make's search path, so when searching for a dependency foo and
> the foo is not present in the current (build) directory, it will go
> searching for it in the VPATH (source) directory. You still need to be
> careful about paths in the command line (generally this means using
> make variables like $@ and $< instead of bare file names), but our
> makefiles are generally pretty good at this already. We even have a
> couple of makefiles using VPATH already (see TestConcurrentEvents) --
> Todd added this to speed up the build by spreading out tests over
> different folders while sharing sources (the serial execution
> problem).
> 
> I still fully support being able to build the tests out of tree, I
> just think it may be a bit more involved than you realise.

I uploaded my first attempt of implementing something along these lines to 
https://reviews.llvm.org/D42281 . Feedback of all kinds is very welcome!

-- adrian


___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 18, 2018, at 5:07 AM, Pavel Labath  wrote:
> 
> Looks like I missed a party. :)
> 
> I'll try to give my thoughts on some of the things that were said here:
> 
>> make -C
> I don't think make -C does what you think it does. "make -C foo" is
> basically equivalent to "cd foo && make", which is what we are doing
> now already. Of course, you can make this work, but you would have to
> pass an extra OUTDIR=... argument to make and then modify the
> Makefiles to reference $(OUTDIR) for its outputs:
> $(OUTDIR)/a.out: main.cc
>  $(CC) -o $(OUTDIR)/a.out main.cc ...
> 
> The standard way of doing an out-of-tree build with make is to have
> the Makefile in the build-directory and to set the magic VPATH
> variable in the Makefile (or as a part of make invocation). VPATH
> alters make's search path, so when searching for a dependency foo and
> the foo is not present in the current (build) directory, it will go
> searching for it in the VPATH (source) directory. You still need to be
> careful about paths in the command line (generally this means using
> make variables like $@ and $< instead of bare file names), but our
> makefiles are generally pretty good at this already. We even have a
> couple of makefiles using VPATH already (see TestConcurrentEvents) --
> Todd added this to speed up the build by spreading out tests over
> different folders while sharing sources (the serial execution
> problem).
> 
This is of course correct. Thanks for pointing this out and for outlining all 
the necessary changes!

> I still fully support being able to build the tests out of tree, I
> just think it may be a bit more involved than you realise.

Sounds good.

> 
>> cmake
> I agree that using cmake for building tests would some things simpler.
> Building fully working executables is fairly tricky, especially when
> you're cross-compiling. Take test/testcases/Android.rules for example.
> This is basically a reimplementation of the android cmake toolchain
> file distributed with the android ndk. If we had cmake for building
> tests we could delete all of that and replace it with
> -DCMAKE_TOOLCHAIN_FILE=$(ANDROID_NDK_HOME)/android.toolchain.cmake.
> However, I only had to write Android.rules just once, so it's not that
> big of a deal for me.
> 
>> explicit RUN lines:
> Yes, it's true that all you need is custom CXXFLAGS (and LDFLAGS), but
> those CXX flags could be quite complex. I'm not really opposed to
> that, but I don't see any immediate benefit either (the only impact
> for me would be that I'd have to translate Android.rules to python).
> 
>> running clean after every test
> Currently we must run "make clean" after every test because make does
> not track the changes in it's arguments. So, if you first run "make
> MAKE_DWO=NO" and then "make MAKE_DWO=YES", make will happily declare
> that it has nothing to do without building the DWO binary. (This will
> go away if we run each test variant in a separate directory).
> 
> I don't have any data, but I would expect that running make upfront
> would make a significant performance improvement on windows (spinning
> up tons of processes and creating/deleting a bunch of files tends to
> be much slower there), but to have no noticable difference on other
> platforms.
> 

I have not thought of Windows being a possible bottleneck. That sounds 
plausible. I'm still wondering how useful incremental builds for the testsuite 
are going to be. When I'm looking at our bots, almost all incoming commits are 
not in LLDB, but in LLVM or Clang. If we are modeling dependencies correctly, 
then each of these commits that changes the compiler should still trigger a 
full rebuild of all testcases, even with CMake. The only situation where 
incremental builds would be useful are for a configuration where we are 
building the LLDB testsuite against a fixed compiler, such as a released/stable 
version of clang or gcc. That's not to say that that isn't an important 
use-case, too, of course, but it's not how the bots on green dragon are 
configured at the moment.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 3:26 PM, Greg Clayton via lldb-dev 
>  wrote:
> 
> Everything sounds good on this thread. My two cents:
> 
> We should add some post verification after each test that looks for file that 
> are left over after the "clean" phase. This can help us catch the tests that 
> aren't cleaning up after themselves. This will help us weed out the bad tests 
> and fix this ASAP. This can be done both for in tree and out of tree 
> solutions to verify there is no source polution.
> 
> We can easily move build artifacts out of the source tree. Running the test 
> suite remotely via "lldb-server platform" has code that creates directories 
> for each test in the platform working directory. If the test runs fine and 
> passes, it cleans up the entire numbered test directory, else it leaves the 
> numbered directory there so we can debug any issues. Shouldn't be hard to 
> enable this.

For completeness, I looked at this and it doesn't look like that is how it 
works. My understanding (and keep in mind that this is the first time I am 
looking at this code so I might be misunderstanding something here) is that the 
remote platform support works by building the test on the host in-tree and then 
_RemoteProcess.launch() ships over only the binary when called from 
Base.spawnSubprocess().

That's not a big deal, though. I will introduce the concept of a build 
directory (which has to be separate from the remote platform working directory) 
and find a way to pass the source directory to runBuildCommands().

-- adrian

> 
> I like the current default of having a new directory with the time and data 
> with results inside as it allows comparison of one test suite run to another.
> 
> Switching build systems to cmake is fine if someone has the energy to do it, 
> that would be great. I don't see why we would go with a lit based system that 
> manually specifies the arguments. Seems like a pain to get the right compiler 
> flags for creating shared libs on different systems (or executables, 
> frameworks, etc). Seems like cmake is good at that and very simple.
> 
> 
>> On Jan 17, 2018, at 3:18 PM, Jim Ingham via lldb-dev 
>>  wrote:
>> 
>> Yeah, w.r.t. the actual builder part, it seems to me any option is going to 
>> be sufficiently simple to use that it would be hard for the incremental 
>> benefits to lldb developers to ever amortize the cost of switching.  The 
>> only compelling reason to me is if one or the other tool made it much easier 
>> to get building the test cases out of tree working, but that seems unlikely.
>> 
>> Jim
>> 
>> 
>>> On Jan 17, 2018, at 3:07 PM, Zachary Turner  wrote:
>>> 
>>> 
>>> 
>>> On Wed, Jan 17, 2018 at 3:04 PM Adrian Prantl  wrote:
>>> 
>>> On the other hand:
>>> - everybody already knows make
>>> 
>>> I'm not sold on this particular reason.  Make is not the LLVM build system, 
>>> CMake is.  "I don't know the build system of the project I actually work 
>>> on, but I do know this other build system" is not a compelling argument. 
>>> 
>>> (As an aside, not every knows Make that well, but it doesn't actually 
>>> matter because the amount of actual Make code is negligibly small, i.e. 1-2 
>>> lines per test in a vast majority of cases)
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 3:25 PM, Zachary Turner  wrote:
> 
> I don't know what would be involved in getting the tests building out of tree 
> with Make.  But I do know it would be simple with CMake.  I'm sure it's 
> probably not terrible with Make either, I just don't know enough about it to 
> say.
> 
> One thing that I do like about CMake is that it can be integrated into the 
> existing LLDB build configuration step, which already uses CMake, to build 
> inferiors up front.  This has the potential to speed up the test suite by an 
> order of magnitude.

Since the tests in the LLDB testsuite are typically very small and don't import 
a lot of headers I'm not convinced that an incremental build of the tests will 
have a very big impact on the running time of the testsuite, but to be honest I 
also haven't benchmarked it to see where the time is really spent.

-- adrian

> 
> Can we get that same effect with a Make-based solution?
> 
> On Wed, Jan 17, 2018 at 3:18 PM Jim Ingham  > wrote:
> Yeah, w.r.t. the actual builder part, it seems to me any option is going to 
> be sufficiently simple to use that it would be hard for the incremental 
> benefits to lldb developers to ever amortize the cost of switching.  The only 
> compelling reason to me is if one or the other tool made it much easier to 
> get building the test cases out of tree working, but that seems unlikely.
> 
> Jim
> 
> 
> > On Jan 17, 2018, at 3:07 PM, Zachary Turner  > > wrote:
> >
> >
> >
> > On Wed, Jan 17, 2018 at 3:04 PM Adrian Prantl  > > wrote:
> >
> > On the other hand:
> > - everybody already knows make
> >
> > I'm not sold on this particular reason.  Make is not the LLVM build system, 
> > CMake is.  "I don't know the build system of the project I actually work 
> > on, but I do know this other build system" is not a compelling argument.
> >
> > (As an aside, not every knows Make that well, but it doesn't actually 
> > matter because the amount of actual Make code is negligibly small, i.e. 1-2 
> > lines per test in a vast majority of cases)
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 3:05 PM, Zachary Turner  wrote:
> 
> Note that we're going off topic from the original goal, and I just want to 
> re-iterate that I fully support smaller, incremental changes. 

Indeed. So just to close the loop on this, it sounds like everybody is in 
agreement that running the tests out-of-tree is a worthwhile goal. So I will 
focus on implementing this next. My rough idea is to generate a fresh directory 
for each test configuration in $builddir/path/to/testname.config and run "make 
-C $srcdir/path/to/testname" there. It looks like the right place to 
implemented this is probably dotest.py.

I like the idea of running dotest from lit, but I'll save this for later.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 2:56 PM, Jim Ingham  wrote:
> 
>> 
>> On Jan 17, 2018, at 2:55 PM, Adrian Prantl  wrote:
>> 
>> 
>> 
>>> On Jan 17, 2018, at 2:50 PM, Jim Ingham  wrote:
>>> 
>>> 
>>> 
 On Jan 17, 2018, at 2:49 PM, Adrian Prantl  wrote:
 
 Question: how would you feel about converting the Makefiles to LIT-style 
 .test files with very explicit RUN-lines?
>>> 
>>> I'm not sure what you mean by this.
>> 
>> Instead of using a build system at all to build the tests, we would 
>> hard-code the compiler and linker invocations without encoding any 
>> dependencies. Because we still need this to be configurable, it would 
>> probably look something like this:
>> 
>> RUN: %CXX test.cpp -O0 %CXXFLAGS -o test.exe
>> RUN: %test_driver test.exe mytest.py
> 
> I'm worried we'd back into building another make system over time.  What 
> advantage would we get from this.

(It's possible that this isn't the right trade-off, I'm just exploring ideas 
here)

Some advantages would be:
- remove the dependency on Make
- possibly easier top debug testcases because all actions are explicit
- potentially slightly faster build times because we don't need to spawn make, 
but note that it also means that these actions will always be unconditionally 
executed
- uniformity with other LLVM projects and thus a smaller cognitive burden for 
developers that touch both clang and lldb

On the other hand:
- everybody already knows make
- maybe we do want a full build system to allow incremental builds of testcases

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 2:50 PM, Jim Ingham  wrote:
> 
> 
> 
>> On Jan 17, 2018, at 2:49 PM, Adrian Prantl  wrote:
>> 
>> 
>> 
>>> On Jan 17, 2018, at 2:31 PM, Zachary Turner  wrote:
>>> 
>>> I don't think new test authors really need to add CMake any more so than 
>>> they currently need to understand Make.  Which is to say, not very much.  
>>> Most Makefiles are currently 1-2 lines of code that simply does nothing 
>>> other than include the common Makefile.
>>> 
>>> On the other hand, CMake defines a lot of constructs designed to support 
>>> portable builds, so actually writing and maintaining that common CMake 
>>> build file would be much easier.  The existing Makefile-based system 
>>> already doesn't require you to understand the specific compiler invocations 
>>> you want.  Here's 3 random Makefiles, which is hopefully representative 
>>> given that I pulled them completely at random.
>>> 
>>> breakpoint-commands/Makefile:
>>> LEVEL = ../../../make
>>> CXX_SOURCES := nested.cpp
>>> include $(LEVEL)/Makefile.rules
>>> 
>>> functionalities/inferior-assert:
>>> LEVEL = ../../make
>>> C_SOURCES := main.c
>>> include $(LEVEL)/Makefile.rules
>>> 
>>> 
>>> types:
>>> LEVEL = ../make
>>> # Example:
>>> #
>>> # CXX_SOURCES := int.cpp
>>> include $(LEVEL)/Makefile.rules
>>> 
>>> None of this is particularly interesting.  There are a very few tests that 
>>> need to do something weird.  I opened 10 other random Makefiles and still 
>>> didn't find any.  I don't believe it would be hard to support those cases.
>>> 
>>> So now instead of "understand Make" it becomes "understand CMake".  Whic is 
>>> already a requirement of building LLVM.
>> 
>> Fair point. I would suggest that I'll try to make LLDB's testsuite build 
>> out-of-tree using the existing Makefile system. That should be a generally 
>> useful first step. After doing this I will hopefully have a much better 
>> understanding of the requirements of the Makefiles and then we can revisit 
>> this idea with me actually knowing what I'm talking about :-)
>> 
>>> If our test suite was lit-based where you actually had to write compiler 
>>> invocations into the test files, I would feel differently, but that isn't 
>>> what we have today.  We have something that is almost a direct mapping to 
>>> using CMake.
>> 
>> Question: how would you feel about converting the Makefiles to LIT-style 
>> .test files with very explicit RUN-lines?
> 
> I'm not sure what you mean by this.

Instead of using a build system at all to build the tests, we would hard-code 
the compiler and linker invocations without encoding any dependencies. Because 
we still need this to be configurable, it would probably look something like 
this:

  RUN: %CXX test.cpp -O0 %CXXFLAGS -o test.exe
  RUN: %test_driver test.exe mytest.py

-- adrian

> 
> Jim
> 
> 
>> 
>> -- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 2:31 PM, Zachary Turner  wrote:
> 
> I don't think new test authors really need to add CMake any more so than they 
> currently need to understand Make.  Which is to say, not very much.  Most 
> Makefiles are currently 1-2 lines of code that simply does nothing other than 
> include the common Makefile.
> 
> On the other hand, CMake defines a lot of constructs designed to support 
> portable builds, so actually writing and maintaining that common CMake build 
> file would be much easier.  The existing Makefile-based system already 
> doesn't require you to understand the specific compiler invocations you want. 
>  Here's 3 random Makefiles, which is hopefully representative given that I 
> pulled them completely at random.
> 
> breakpoint-commands/Makefile:
> LEVEL = ../../../make
> CXX_SOURCES := nested.cpp
> include $(LEVEL)/Makefile.rules
> 
> functionalities/inferior-assert:
> LEVEL = ../../make
> C_SOURCES := main.c
> include $(LEVEL)/Makefile.rules
> 
> 
> types:
> LEVEL = ../make
> # Example:
> #
> # CXX_SOURCES := int.cpp
> include $(LEVEL)/Makefile.rules
> 
> None of this is particularly interesting.  There are a very few tests that 
> need to do something weird.  I opened 10 other random Makefiles and still 
> didn't find any.  I don't believe it would be hard to support those cases.
> 
> So now instead of "understand Make" it becomes "understand CMake".  Whic is 
> already a requirement of building LLVM.

Fair point. I would suggest that I'll try to make LLDB's testsuite build 
out-of-tree using the existing Makefile system. That should be a generally 
useful first step. After doing this I will hopefully have a much better 
understanding of the requirements of the Makefiles and then we can revisit this 
idea with me actually knowing what I'm talking about :-)

> If our test suite was lit-based where you actually had to write compiler 
> invocations into the test files, I would feel differently, but that isn't 
> what we have today.  We have something that is almost a direct mapping to 
> using CMake.

Question: how would you feel about converting the Makefiles to LIT-style .test 
files with very explicit RUN-lines?

-- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 1:45 PM, Vedant Kumar <v...@apple.com> wrote:
> 
> I would prefer having all of our test dependencies tracked by CMake for all 
> the reasons Zach brought up, but I think we should defer that undertaking 
> until after the bots are more stable. We have some immediate problems caused 
> by stale in-tree test artifacts. As a first milestone, it'd be great to not 
> have to run `git clean -fdx` anymore.

I'm pretty sure I do not want to go all the way to CMake right now, but I am 
curious about your motivation: Why do you think that using CMake to build the 
tests in the testsuite is a good idea? In my opinion this adds a layer of 
complexity to the tests that makes it harder to understand what exactly is 
happening and test authors now need to understand CMake *and* the compiler 
invocations they want to execute in their tests. Do you also share Zachary's 
point of view that the tests should be build artifacts that should be kept 
after an incremental build?

-- adrian

> 
> 
>> On Jan 17, 2018, at 1:13 PM, Davide Italiano via lldb-dev 
>> <lldb-dev@lists.llvm.org> wrote:
>> 
>> On Wed, Jan 17, 2018 at 1:02 PM, Davide Italiano <dccitali...@gmail.com> 
>> wrote:
>>> On Wed, Jan 17, 2018 at 12:32 PM, Adrian Prantl via lldb-dev
>>> <lldb-dev@lists.llvm.org> wrote:
>>>> Hi lldb-dev!
>>>> 
>>>> I've been investigating some spurious LLDB test suite failures on 
>>>> http://green.lab.llvm.org/green/ that had to do with build artifacts from 
>>>> previous runs lying around in the test directories and this prompted me to 
>>>> ask a couple of general noob questions about the LLDB testsuite.
>>>> 
>>>> My understanding is that all execution tests are compiled using using 
>>>> `make` in-tree. I.e.: the test driver (dotest.py) effectively executes 
>>>> something equivalent to `cd $srcdir/packages/.../mytest && make`. And it 
>>>> does this in a serial fashion for all configurations (dwarf, dSYM, dwo, 
>>>> ...) and relies on the `clean` target to be implemented correctly.
>>>> 
>>>> I don't understand all the design decisions that went into the LLDB 
>>>> testsuite, but my naive intuition tells me that this is sub-optimal 
>>>> (because of the serialization of the configurations) and dangerous 
>>>> (because it relies on make clean being implemented correctly). It seems to 
>>>> me that a better approach would be to create a separate build directory 
>>>> for each test variant and then invoke something like `cd 
>>>> $builddir/test/mytest.dwarf && make -C $srcdir/packages/.../mytest`. This 
>>>> way all configurations can build in parallel, and we can simply nuke the 
>>>> build directory afterwards and this way get rid of all custom 
>>>> implementations of the `clean` target.
> 
> This sgtm as a starting point.
> 
> vedant
> 
>>>> 
>>>> - Is this already possible, and/or am I misunderstanding how it works?
>>>> - Would this be a goal that is worthwhile to pursue?
>>>> - Is there a good reason why we are not already doing it this way?
>>>> 
>>> 
>>> As we're discussing lldb test suite changes, another detail that I
>>> find a little weird is that every time you execute the test suite you
>>> get a new build directory named after the time at which you run the
>>> test.
>>> It would be much much better IMHO to just have a `log/` generic
>>> directory where the failures are logged, and those who want to
>>> override this setting can just pass a flag.
>>> 
>> 
>> (The logs should also be moved out of tree, FWIW).
>> 
>> --
>> Davide
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 1:13 PM, Davide Italiano <dccitali...@gmail.com> wrote:
> 
> On Wed, Jan 17, 2018 at 1:02 PM, Davide Italiano <dccitali...@gmail.com> 
> wrote:
>> On Wed, Jan 17, 2018 at 12:32 PM, Adrian Prantl via lldb-dev
>> <lldb-dev@lists.llvm.org> wrote:
>>> Hi lldb-dev!
>>> 
>>> I've been investigating some spurious LLDB test suite failures on 
>>> http://green.lab.llvm.org/green/ that had to do with build artifacts from 
>>> previous runs lying around in the test directories and this prompted me to 
>>> ask a couple of general noob questions about the LLDB testsuite.
>>> 
>>> My understanding is that all execution tests are compiled using using 
>>> `make` in-tree. I.e.: the test driver (dotest.py) effectively executes 
>>> something equivalent to `cd $srcdir/packages/.../mytest && make`. And it 
>>> does this in a serial fashion for all configurations (dwarf, dSYM, dwo, 
>>> ...) and relies on the `clean` target to be implemented correctly.
>>> 
>>> I don't understand all the design decisions that went into the LLDB 
>>> testsuite, but my naive intuition tells me that this is sub-optimal 
>>> (because of the serialization of the configurations) and dangerous (because 
>>> it relies on make clean being implemented correctly). It seems to me that a 
>>> better approach would be to create a separate build directory for each test 
>>> variant and then invoke something like `cd $builddir/test/mytest.dwarf && 
>>> make -C $srcdir/packages/.../mytest`. This way all configurations can build 
>>> in parallel, and we can simply nuke the build directory afterwards and this 
>>> way get rid of all custom implementations of the `clean` target.
>>> 
>>> - Is this already possible, and/or am I misunderstanding how it works?
>>> - Would this be a goal that is worthwhile to pursue?
>>> - Is there a good reason why we are not already doing it this way?
>>> 
>> 
>> As we're discussing lldb test suite changes, another detail that I
>> find a little weird is that every time you execute the test suite you
>> get a new build directory named after the time at which you run the
>> test.
>> It would be much much better IMHO to just have a `log/` generic
>> directory where the failures are logged, and those who want to
>> override this setting can just pass a flag.
>> 
> 
> (The logs should also be moved out of tree, FWIW).

If I'm going to move the test build artifacts out-of-source-tree the logs would 
naturally end up there, too. Let's discuss whether creating a timestamped log 
directory should be the default or an option in a different thread to keep 
things simple. This is entirely orthogonal.

-- adrian

> 
> --
> Davide

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 12:41 PM, Zachary Turner  wrote:
> 
> If we're going to be making any significant changes to the way inferiors are 
> compiled, why not use cmake?  Make clean is already not implemented correctly 
> in many places, leading to lots of remnants left over in the source tree 
> after test runs.  Furthermore, make is run every single time currently, 
> leading to hundreds (if not thousands) of unnecessary compilations.  Seems to 
> me like all the inferiors should be compiled one time, up front, as part of 
> the configure step, and into the build directory.  This is nice because it 
> already integrates perfectly into the existing LLVM "way" of building things.

To be honest, I have not considered the the tests to be part of the build. 
Doing so is an interesting idea that I haven't thought about so far, here are 
some thoughts about this:
- CMake is more a replacement for autoconf than for make and I'm not sure if we 
need a better tool for the "configuration" part of the testsuite.
- Some of the tests purposefully do weird stuff, such as deleting or damaging 
one .o file to test LLDB's abilities to cope with incomplete debug info. To 
implement this we will need to micro-manage things at the "make" level that 
could be hard to express in CMake.
- The LLVM/CFE way of building *tests* is to hard-code the commands to 
recompile the testcases every time you run the testsuite.
- We probably want to be able to run the LLDB testsuite using many different 
compilers. It could be possible that CMake is helpful for this, but then again 
I doubt that we need much more than setting a custom CC/CXX/CFLAGs to support 
this.

I don't want to immediately shoot this idea down, but I think it is one step 
further than I would like to go at this time. Thanks for pointing it out though!

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

Hi lldb-dev!

I've been investigating some spurious LLDB test suite failures on 
http://green.lab.llvm.org/green/ that had to do with build artifacts from 
previous runs lying around in the test directories and this prompted me to ask 
a couple of general noob questions about the LLDB testsuite.

My understanding is that all execution tests are compiled using using `make` 
in-tree. I.e.: the test driver (dotest.py) effectively executes something 
equivalent to `cd $srcdir/packages/.../mytest && make`. And it does this in a 
serial fashion for all configurations (dwarf, dSYM, dwo, ...) and relies on the 
`clean` target to be implemented correctly.

I don't understand all the design decisions that went into the LLDB testsuite, 
but my naive intuition tells me that this is sub-optimal (because of the 
serialization of the configurations) and dangerous (because it relies on make 
clean being implemented correctly). It seems to me that a better approach would 
be to create a separate build directory for each test variant and then invoke 
something like `cd $builddir/test/mytest.dwarf && make -C 
$srcdir/packages/.../mytest`. This way all configurations can build in 
parallel, and we can simply nuke the build directory afterwards and this way 
get rid of all custom implementations of the `clean` target.

- Is this already possible, and/or am I misunderstanding how it works?
- Would this be a goal that is worthwhile to pursue?
- Is there a good reason why we are not already doing it this way?

thanks,
adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Adding DWARF5 accelerator table support to llvm

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 9:20 AM, Jonas Devlieghere via llvm-dev 
>  wrote:
> 
> As mentioned by Adrian in the comment you linked, I too am looking at DWARFv5
> accelerator tables in LLVM. 
> 
> To give you some background: my motivation is that I want to upstream support
> for (Apple style) accelerator tables in llvm-dsymutil, 

Some background for the benefit of everyone who may not be aware of the genesis 
of the DWARF v5 accelerator tables:

DWARF v5 accelerator tables are a direct evolution of the "Apple" accelerator 
tables that are implemented in LLVM (after going through the standardization 
process and being reworked/generalized there), so we hope that the 
implementation can at least share some common interfaces with the existing 
Apple accelerator table implementation where this makes sense.

-- adrian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Remotely launching a new process with LLDB

[Adrian Prantl via lldb-dev]

I would recommend asking this on the lldb mailing list instead.

-- adrian

> On Nov 26, 2017, at 8:30 AM, Lior Halphon via llvm-dev 
>  wrote:
> 
> I'm trying to remotely launch and debug a new process with lldb without much 
> success.
> 
> Attaching to an already launched process works well by running these commands:
> 
> process connect 
> process attach -P gdb-remote --pid 
> But if I want debugserver to launch the executable by itself I'm running into 
> troubles. Especially, I have no clue what arguments should I pass to target
>  create.
> 
> According to this page  LLDB "will 
> transparently take care of [..] downloading the executable in order to be 
> able to debug", yet target
>  create seem to always require a local file. If I specify the remote file via 
> -r I get either unable
>  to open target file or remote
>  --> local transfer without local path is not implemented yet errors. If I 
> set the target to a local file (such as a local copy of the remote's loader 
> executable) without using -r, then attempt to run process
>  launch -p gdb-remote -s  LLDB will attempt running the local 
> path on the remote machine and fail.
> 
> What are the correct commands I need to use in order to launch a remote 
> process?
> 
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Apple radar references in the LLDB test suite

[Adrian Prantl via lldb-dev]

This is a very reasonable request, but I think it is better directed at 
lldb-dev.

-- adrian

On Aug 14, 2017, at 8:27 AM, Ed Maste via llvm-dev  
wrote:
> 
> 
> I was recently looking at an unexpected pass test result on FreeBSD[1]
> which is decorated with expectedFailureAll referencing
> rdar://problem/24599697. I'm looking for more details on this specific
> radar bug report in order to determine if the test is not exercising
> the original bug on FreeBSD, or if it's really an Apple-only issue.
> 
> As it's rather unfortunate to have a bug report reference for which no
> further public details are available I looked at other radar
> references in the test suite, and found 118 of them. Many of these are
> in comments, in macosx-specific cases (e.g.
> @expectedFailureAll(oslist=["macosx"], bugnumber="rdar://28805064")),
> or in tests decorated with skipUnlessDarwin, and I've ignored those.
> 
> There are 10 tests with expected failure decorators, and one skipped,
> that apply beyond macosx and reference a radar (and do not
> additionally reference an LLVM PR). Can someone at Apple check these
> radar references and submit LLVM PRs, reference existing PRs, add a
> reasonable description in the test source, etc. as appropriate?
> 
> functionalities/watchpoint/variable_out_of_scope/TestWatchedVarHitWhenInScope.py
> unittest2.expectedFailure("rdar://problem/18685649")
> 
> functionalities/asan/TestReportData.py
> @expectedFailureAll(archs=['i386'], bugnumber="rdar://28658860")
> 
> api/multiple-debuggers/TestMultipleDebuggers.py
> @expectedFailureAll(bugnumber="rdar://30564102")
> 
> lang/objc/bitfield_ivars/TestBitfieldIvars.py
> decorators.expectedFailureAll(bugnumber="rdar://problem/17990991")])
> 
> lang/c/shared_lib/TestSharedLib.py
> @unittest2.expectedFailure("rdar://problem/10704639")
> 
> lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
> @unittest2.expectedFailure("rdar://problem/10381325")
> 
> lang/cpp/stl/TestSTL.py
> @expectedFailureAll(bugnumber="rdar://problem/10400981")
> 
> lang/cpp/dynamic-value/TestCppValueCast.py
> @unittest2.expectedFailure("rdar://problem/10808472 SBValue::Cast test
> case is failing (virtual inheritance)")
> 
> lang/cpp/printf/TestPrintf.py
> decorators.expectedFailureAll(bugnumber="rdar://problem/24599697")])
> 
> lang/cpp/function-template-parameter-pack/TestFunctionTemplateParameterPack.py
> decorators.expectedFailureAll(bugnumber="rdar://problem/32096064")])
> 
> lang/cpp/unique-types/TestUniqueTypes.py
> self.skipTest("rdar://problem/9173060 lldb hangs while running
> unique-types for clang version < 3")
> 
> [1] 
> http://lists.llvm.org/pipermail/lldb-commits/Week-of-Mon-20170807/036634.html
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] LLDB tests

[Adrian Prantl via lldb-dev]

> On Jul 24, 2017, at 10:48 AM, Sean Callanan via lldb-dev 
>  wrote:
> 
> Steve,
> 
> since you asked about failures, here are some public bots you can look at to 
> get a general sense of how we are doing:
> 
> http://lab.llvm.org:8011/builders  
> [various platforms]
> http://lab.llvm.org:8080/green/view/LLDB/job/lldb_build_test/ 
>  [OS X]
FYI, we also have a slightly nicer URL for green dragon that doesn't involve 
nonstandard ports:
http://green.lab.llvm.org/green/view/LLDB/job/lldb_build_test/ 


-- adrian
> 
> https://ci.swift.org/view/All/job/oss-lldb-incremental-osx/ 
>  [OS X]
> https://ci.swift.org/view/All/job/oss-lldb-incremental-linux-ubuntu-16_10/ 
>  
> [Linux]
> There are many more bots, as you'll discover browsing around, but these 
> should give you a good idea of the health of our testsuite at any given time.
> 
> Sean
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] RFC: Improving the performance of ItaniumDemangle

[Adrian Prantl via lldb-dev]

[+lldb-dev]

> On May 3, 2017, at 2:46 AM, Pavel Labath via llvm-dev 
>  wrote:
> 
> One of the things we need in lldb is to find a "basename" of a
> function. This is so that you can set a breakpoint on a function named
> "foo", even if "foo" is a method of a template class declared inside
> another function, which is inside 5 namespaces, a couple of them being
> anonymous... :)
> 
> Currently we do that by parsing the demangled name, which is neither
> fast nor easy, especially when function pointers come into the game
> (D31451). We could probably save some runtime if the demangler could
> provide us with a bit more structured information instead of just the
> final demangled string. I expect the demangler to be in a much better
> position to do that then us trying to reverse engineer the string.
> 
> I have no idea how this fits in with the rest of the goals of llvm's
> demangler, but I guess it doesn't hurt throwing the idea out there.
> 
> cheers,
> pl
> 
> On 30 April 2017 at 21:54, Saleem Abdulrasool via llvm-dev
>  wrote:
>> 
>> 
>> On Tue, Apr 25, 2017 at 1:37 PM, Asiri Rathnayake
>>  wrote:
>>> 
>>> 
>>> 
>>> On Tue, Apr 25, 2017 at 8:36 PM, Vedant Kumar via llvm-dev
>>>  wrote:
 
 
> On Apr 25, 2017, at 12:24 PM, Scott Smith 
> wrote:
> 
> well, top-of-branch lldb uses this code, that's how I found it.  Do you
> mean libc++'s demangler?
 
 Thanks for explaining, this is the first time I'm looking at the
 demangler situation. It looks like libcxxabi has an arena-based demangler,
 and that the one in llvm is different.
 
 I'm confused by this because the comment in llvm says that libcxxabi is
 supposed to reuse the llvm demangler. This doesn't seem to be happening,
 right?
>>> 
>>> 
>>> This seems correct. libcxxabi demangler [1] is different from the one used
>>> by llvm [2]. I'm hoping Saleem, Eric or Jon (copied) knows a bit of history
>>> as to why this is so (perhaps because the two projects evolved independently
>>> ?).
>> 
>> 
>> They didnt really evolve independently, the version in LLVM was imported
>> from libc++.  However, we simplified it to make it more portable.  The
>> simpifications naturally led to the ability to remove the arena allocation
>> routines.  The copy in libc++ needs to retain a certain amount of
>> flexibility due to the exporting of the interface into the user's address
>> space (via the __cxa_demangle interface).  However, making adjustments that
>> improve performance in the LLVM version should be acceptable.
>> 
 
 
 
> FYI when I said 14+% (and now it's 17%), I mean the overall performance
> of starting lldb, not just the demangler itself.  It's probably several
> times faster now with this change (https://reviews.llvm.org/D32500)
 
 Do you know what the llvm policy is on using TLS in library code? I can't
 find any mention of this in the programmer's manual, and my officemates
 don't know either.
>>> 
>>> 
>>> Both libcxx and libcxxabi use __libcpp_tls_*() functions of the threading
>>> API [2] (which call pthread functions on most platforms) for thread-local
>>> storage needs. IIRC thread_local is not implemented across all the platforms
>>> that llvm support.
>>> 
>>> If the idea is to improve libcxxabi's demangler, then it should be
>>> straightforward to use these functions instead of thread_local.
>>> 
>>> [1]
>>> https://github.com/llvm-mirror/libcxxabi/blob/master/src/cxa_demangle.cpp
>>> [2]
>>> https://github.com/llvm-mirror/llvm/blob/master/lib/Demangle/ItaniumDemangle.cpp
>>> [3]
>>> https://github.com/llvm-mirror/libcxx/blob/master/include/__threading_support
>>> 
>>> PS: Here's a particularly amusing bug of the current libcxxabi demangler:
>>> https://bugs.llvm.org//show_bug.cgi?id=31031
>>> 
>>> Cheers,
>>> 
>>> / Asiri
>>> 
 
 
 vedant
 
 
> On Tue, Apr 25, 2017 at 12:19 PM, Vedant Kumar  wrote:
> I thought the plan of record was (r280732):
> 
> '''
> Once the fast demangler in lldb can handle any names this
> implementation can be replaced with it and we will have the one true
> demangler.
> '''
> 
> What is the status of lldb's fast demangler? Is it available on Ubuntu
> 16.04?
> 
> vedant
> 
> 
>> On Apr 24, 2017, at 6:02 PM, Scott Smith via llvm-dev
>>  wrote:
>> 
>> (Again), while trying to improve the performance of lldb, I ran into
>> a bottleneck with the demangler.  This may be specific to my platform -
>> Ubuntu 16.04, probably using libstdc++, not libc++.  It makes extensive 
>> use
>> of std::string and std::vector, and I see memory allocation at the top.  
>> I
>> prototyped a version that uses an arena-style memory allocator (you can
>>