[lldb-dev] Allowing PlatformPOSIX::LoadImage to lazy bind symbols

[Vedant Kumar via lldb-dev]

Hello,

I'd like to allow PlatformPOSIX::LoadImage to lazy bind symbols (instead of 
failing to load a library if one of its exported symbols is in fact 
unavailable).

Essentially, I'd like for RTLD_LAZY to replace the current default of RTLD_NOW.

In general, it seems like the debugger should allow programs to load & run with 
libraries as far as possible, instead of defaulting to being super-picky about 
unavailable symbols.

This is critical on macOS/Darwin, as libswiftCore.dylib may 1) export a version 
symbol using @available markup and then 2) expect that other exported APIs are 
only dynamically used once the version symbol is checked. We can't open a 
version of the library built with a bleeding-edge SDK on an older OS without 
RTLD_LAXY (or pervasive/expensive @available markup added to dyld APIs).

I pitched this to Jim, who brought up the possibility of making the RTLD_ 
behavior configurable via some option. My thinking on that is, as approximately 
no users should notice the switch, it's not worth adding an option. (We can add 
one later if there's some actual demand.)

WDYT?

best,
vedant


___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Deleting lldb/utils/test/

[Vedant Kumar via lldb-dev]

Sounds good, deleted as of 158f3360.

> On Oct 27, 2020, at 10:46 AM, Jonas Devlieghere  wrote:
> 
> Last time I looked at these nothing seemed relevant (anymore) to me either. 
> I'm in favor of getting rid of the directory.
> 
> On Tue, Oct 27, 2020 at 10:26 AM Vedant Kumar via lldb-dev 
> mailto:lldb-dev@lists.llvm.org>> wrote:
> Hi,
> 
> I'm considering deleting the lldb/utils/test/ directory as a cleanup. Does 
> anyone has a reason to keep these scripts around?
> 
> Here are the files in the directory:
> 
> % ls lldb/utils/test
> README-disasm   README-run-until-faulted
> lldb-disasm.py  main.c  run-dis.py
> README-lldb-disasm  disasm.py   
> llvm-mc-shell.pyras.py  
> run-until-faulted.py
> 
> AFAICT:
> 
> - disasm.py would've been helpful before lldb gained a 'disassemble' command, 
> but it doesn't seem useful anymore
> - ditto for lldb-disasm.py; this one also seems quite Darwin-specific (I'm 
> pretty sure it doesn't work anymore)
> - llvm-mc-shell.py might be useful if you want to type bytes by hand and see 
> the disassembly, but even then, seems better to just do `echo "" | 
> llvm-mc`
> - ras.py isn't running the test suite properly, also (imo) seems like an 
> ersatz Jenkins replacement
> - main.c is just an example program
> - run-dis.py looks like a driver for stress-testing lldb's disassembly 
> command, but it looks very iOS/Darwin specific and has likely outlived its 
> usefulness
> - run-until-faulted.py runs a program up to 100 times to see if any of the 
> runs fail; I suspect most users would reach for a shell one-liner before 
> looking for something like this script
> 
> Thoughts?
> 
> thanks,
> vedant
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
> <https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev>

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] Deleting lldb/utils/test/

[Vedant Kumar via lldb-dev]

Hi,

I'm considering deleting the lldb/utils/test/ directory as a cleanup. Does 
anyone has a reason to keep these scripts around?

Here are the files in the directory:

% ls lldb/utils/test
README-disasm   README-run-until-faultedlldb-disasm.py  
main.c  run-dis.py
README-lldb-disasm  disasm.py   
llvm-mc-shell.pyras.py  
run-until-faulted.py

AFAICT:

- disasm.py would've been helpful before lldb gained a 'disassemble' command, 
but it doesn't seem useful anymore
- ditto for lldb-disasm.py; this one also seems quite Darwin-specific (I'm 
pretty sure it doesn't work anymore)
- llvm-mc-shell.py might be useful if you want to type bytes by hand and see 
the disassembly, but even then, seems better to just do `echo "" | 
llvm-mc`
- ras.py isn't running the test suite properly, also (imo) seems like an ersatz 
Jenkins replacement
- main.c is just an example program
- run-dis.py looks like a driver for stress-testing lldb's disassembly command, 
but it looks very iOS/Darwin specific and has likely outlived its usefulness
- run-until-faulted.py runs a program up to 100 times to see if any of the runs 
fail; I suspect most users would reach for a shell one-liner before looking for 
something like this script

Thoughts?

thanks,
vedant
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] RFC: Processor Trace Support in LLDB

[Vedant Kumar via lldb-dev]

Hi Walter & Greg,

Thanks for sharing this RFC, and for your work in this area.

> On Sep 17, 2020, at 5:28 PM, Walter via lldb-dev  
> wrote:
> 
> Hi all,
>  
> Here I propose, along with Greg Clayton, Processor Trace support for LLDB. 
> I’m attaching a link to the document that contains this proposal if that’s 
> easier to read for you: 
> https://docs.google.com/document/d/1cOVTGp1sL_HBXjP9eB7qjVtDNr5xnuZvUUtv43G5eVI/edit#heading=h.t5mblb9ugv8f
>  
> .
>  Please make any comments in this mail list.
>  
>  
> If you want to quickly know what Processor Trace can do, you can read this 
> https://easyperf.net/blog/2019/08/23/Intel-Processor-Trace 
> .
>  
> Any comments are appreciated, especially the ones regarding the commands the 
> user will interact with. 
>  
> Thanks,
> Walter Erquinigo.
>  
>  
> # RFC: Processor Trace Support in LLDB
>  
>  
> # What is processor tracing?
>  
> Processor tracing works by capturing information about the execution of a 
> process so that the control flow of the program can be reconstructed later. 
> Implementations of this are Intel Processor Trace for X86, x86_64 
> ([https://software.intel.com/content/www/us/en/develop/blogs/processor-tracing.html](https://software.intel.com/content/www/us/en/develop/blogs/processor-tracing.html)
>  
> )
>  and ARM CoreSight for some ARM devices 
> ([https://developer.arm.com/ip-products/system-ip/coresight-debug-and-trace](https://developer.arm.com/ip-products/system-ip/coresight-debug-and-trace)
>  
> ).
>  
>  
> As a clarifying example, with these technologies it’s possible to trace all 
> the threads of a process, and after the process has finished, reconstruct 
> every single instruction address each thread has executed. This could include 
> some additional information like timestamps, async CPU events, kernel 
> instructions, bus clock ratio changes, etc. On the other hand, memory and 
> registers are not traced as a way to limit the size of the trace.
>  
>  
> # Intel Processor Trace as the first implementation
>  
> We’ll focus on Intel Processor Trace (Intel PT), but in a generic way so that 
> in the future similar technologies can be onboarded in LLDB.
>  
> Intel PT has the following features:
>  
>  
>  
> *   Control flow tracing in a highly encoded format
> *   3% to 5% slowdown when capturing
> *   No memory nor registers captured
> *   Kernel tracing support
> *   Timestamps of branches are produced, which can be used for profiling
> *   Adjustable size of trace buffer
> *   Supported on most Intel CPUs since 2015
> *   X86 and x86_64 only
> *   Official support only on Linux
> *   Basic support on Windows
> *   Decoding/analysis can be done on any operating system
>  
> A very nice introduction to Intel PT can be found 
> [https://software.intel.com/content/www/us/en/develop/blogs/processor-tracing.html](https://software.intel.com/content/www/us/en/develop/blogs/processor-tracing.html)
>  
> 
>  and 
> [https://easyperf.net/blog/2019/08/23/Intel-Processor-Trace](https://easyperf.net/blog/2019/08/23/Intel-Processor-Trace)
>  
> .
>  Totally recommended to fully grasp the impact of this project. 
>  
> More technical details are in 
> [https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/perf-intel-pt.txt](https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/perf-intel-pt.txt)
>  
> .
>  
>  
> Even more technical details are in the processor manual 
> 

Re: [lldb-dev] [llvm-dev] Why is lldb telling me "variable not available"?

[Vedant Kumar via lldb-dev]

> On Feb 26, 2020, at 3:29 PM, Brian Gesiak via lldb-dev 
>  wrote:
> 
> Vedant, Jeremy,
> 
> Thanks a ton! I copied ASan's use of 'replaceDbgDeclare', think that worked!
> 
> https://github.com/modocache/llvm-project/commit/afbc04e1dcba has some
> extremely quick and dirty changes I made (with no tests!), and a link
> to a Gist with the LLVM IR and DWARF produced,
> https://gist.github.com/modocache/6f29093ba2827946011b422ed3bd2903.
> 
> There's only one kink: the spot I placed it has 'replaceDbgDeclare'
> called multiple times, and it reuses the result of a single
> getelementptr instruction. Because of this, only the last invocation
> results in correct debug info being produced. Put another way: when I
> place a breakpoint in my example on the last instance that %i is
> reloaded, repro.cpp:45 at
> https://gist.github.com/modocache/6f29093ba2827946011b422ed3bd2903#file-repro-cpp-L45,
> I get the correct debug info: i = 3, j = 2. In every other spot, I get
> 'error: Couldn't materialize: couldn't get the value of variable i:
> variable not available'. Still, is great progress for me.
> 
> I think to work around this error, I need to do 2 things:
> 
> 1. Call replaceDbgDeclare only once.
> 2. Call it with the correct arguments -- you may have noticed in
> https://github.com/modocache/llvm-project/commit/afbc04e1dcba#diff-f1204f42e8cb9e3718892e63059f8347R743
> that I'm applying an offset of 0, which doesn't make much sense. I
> need to figure out how to use this function such that it encodes in
> DWARF to load at an offset from the coroutine frame, just as a
> getelementptr would.

Haven't looked at this closely, but (2) sounds right, the goal is to let the 
compiler know that variables which used to live in one memory location have 
moved to another. If you have the precise address of the variable's new 
location within the 'coro' frame, you don't need to specify the offset 
parameter to replaceDbgDeclare ("NewAddress" is exact). More typically, 
instrumentation passes maintain a pointer to the new frame and a table of 
offsets for various variables, so the common pattern is 
'replaceDbgDeclare(oldAllocaOrGEPForVar, newFramePtr, ..., 
offsetIntoNewFramePtrForVar)'.

vedant

> 
> - Brian
> 
> On Wed, Feb 26, 2020 at 5:15 PM Brian Gesiak  wrote:
>> 
>> Vedant, thank you! I had meant to ask if any of this reminded you all
>> of something else that I could emulate. I'll look into uses of
>> 'replaceDbgDeclare' in SafeStack/ASan. - Brian
>> 
>> On Wed, Feb 26, 2020 at 5:08 PM Vedant Kumar  wrote:
>>> 
>>> I haven't fully parsed this thread (sorry!), but I wanted to briefly 
>>> mention that the SafeStack & ASan passes both do something similar (I 
>>> think): move local variables backed by allocas onto a separate stack. These 
>>> passes use replaceDbgDeclare to rewrite dbg.declares s.t. they point into 
>>> the new stack. After that, llvm presumably runs LowerDbgDeclare (usually 
>>> via InstCombine), but all the inserted dbg.values have useful locations 
>>> relative to the new stack.
>>> 
>>> It could be worth investigating whether replaceDbgDeclare is a good fit for 
>>> the coro-split pass.
>>> 
>>> vedant
>>> 
>>> On Feb 26, 2020, at 1:32 PM, Brian Gesiak via llvm-dev 
>>>  wrote:
>>> 
>>> Awesome, thanks Jeremy.
>>> 
>>> On Wed, Feb 26, 2020 at 11:02 AM Jeremy Morse
>>>  wrote:
>>> 
>>> 
>>> Hi Brian,
>>> 
>>> On Tue, Feb 25, 2020 at 7:43 PM Brian Gesiak  wrote:
>>> 
>>> In other words, the value of %i is stored on the frame object, on the
>>> heap, at an offset of 7 into the frame. I'm beginning to think a
>>> fundamental fix for this issue would be to stop replacing
>>> llvm.dbg.declare with llvm.dbg.value, and instead replace the
>>> llvm.dbg.declare with llvm.dbg.addr that points the debugger to the %i
>>> variable's new permanent location as an offset into the coroutine
>>> frame object. Does this approach make sense to people on this mailing
>>> list, who probably know more about how these intrinsics work than I
>>> do?
>>> 
>>> 
>>> This matches a few similar use cases that I'm aware of -- certain
>>> kinds of struct that are passed-by-value according to the language,
>>> but passed-by-reference according to ABI, are treated in that way. In
>>> general, the downside is that the debugger can only observe variable
>>> values when they get written to memory, not when they're computed, as
>>> dbg.values and dbg.declares aren't supposed to be mixed. Observing
>>> variable values slightly later might be an improvement over the
>>> current situation.
>>> 
>>> 
>>> This is very reassuring, thank you!
>>> 
>>> Although, I don't think this will work immediately, see below,
>>> 
>>> I tried multiple approaches to manually inserting an llvm.dbg.addr
>>> after the store instruction, as per your suggestion, Jeremy. I used
>>> llc to compile the IR into an object file that I then linked, and
>>> inspected the DWARF generated for the file. Unfortunately, inserting
>>> dbg.addr that operated on the reloaded 

Re: [lldb-dev] [llvm-dev] Why is lldb telling me "variable not available"?

[Vedant Kumar via lldb-dev]

I haven't fully parsed this thread (sorry!), but I wanted to briefly mention 
that the SafeStack & ASan passes both do something similar (I think): move 
local variables backed by allocas onto a separate stack. These passes use 
replaceDbgDeclare to rewrite dbg.declares s.t. they point into the new stack. 
After that, llvm presumably runs LowerDbgDeclare (usually via InstCombine), but 
all the inserted dbg.values have useful locations relative to the new stack.

It could be worth investigating whether replaceDbgDeclare is a good fit for the 
coro-split pass.

vedant

> On Feb 26, 2020, at 1:32 PM, Brian Gesiak via llvm-dev 
>  wrote:
> 
> Awesome, thanks Jeremy.
> 
> On Wed, Feb 26, 2020 at 11:02 AM Jeremy Morse
> mailto:jeremy.morse.l...@gmail.com>> wrote:
>> 
>> Hi Brian,
>> 
>> On Tue, Feb 25, 2020 at 7:43 PM Brian Gesiak  wrote:
>>> In other words, the value of %i is stored on the frame object, on the
>>> heap, at an offset of 7 into the frame. I'm beginning to think a
>>> fundamental fix for this issue would be to stop replacing
>>> llvm.dbg.declare with llvm.dbg.value, and instead replace the
>>> llvm.dbg.declare with llvm.dbg.addr that points the debugger to the %i
>>> variable's new permanent location as an offset into the coroutine
>>> frame object. Does this approach make sense to people on this mailing
>>> list, who probably know more about how these intrinsics work than I
>>> do?
>> 
>> This matches a few similar use cases that I'm aware of -- certain
>> kinds of struct that are passed-by-value according to the language,
>> but passed-by-reference according to ABI, are treated in that way. In
>> general, the downside is that the debugger can only observe variable
>> values when they get written to memory, not when they're computed, as
>> dbg.values and dbg.declares aren't supposed to be mixed. Observing
>> variable values slightly later might be an improvement over the
>> current situation.
> 
> This is very reassuring, thank you!
> 
>> Although, I don't think this will work immediately, see below,
>> 
>>> I tried multiple approaches to manually inserting an llvm.dbg.addr
>>> after the store instruction, as per your suggestion, Jeremy. I used
>>> llc to compile the IR into an object file that I then linked, and
>>> inspected the DWARF generated for the file. Unfortunately, inserting
>>> dbg.addr that operated on the reloaded values didn't lead to any
>>> change in the DWARF that was produced --  specifically, this didn't
>>> make a difference:
>>> 
>>> call void @llvm.dbg.addr(metadata i32* %i.reload.addr62, metadata
>>> !873, metadata !DIExpression()), !dbg !884
>> 
>> Ouch, I tried this myself, and ran into the same difficulty. I'd
>> missed that all your functions are marked "optnone" / -O0, which means
>> a different instruction-selection pass (FastISel) runs, and it turns
>> out FastISel isn't aware of dbg.addrs existence. Even better, FastISel
>> doesn't manage to lower any debug intrinsic (including dbg.declare)
>> that refers to a GEP, because it doesn't have a register location (the
>> GEP gets folded into a memory addressing mode).
>> 
>> I've hacked together some support in [0], that allows dbg.addr's of
>> GEPs to be handled. A single dbg.addr at the start of the function
>> (and no dbg.values) should get you the same behaviour as a
>> dbg.declare.
> 
> Maybe I'm running into the limitations of how far I can get here by
> hacking up the LLVM IR directly, but I've made a couple of tries here
> without any luck. I put the IR files and the DWARF dumps of
> executables built from that IR in another GitHub gist,
> https://gist.github.com/modocache/8a7b12eb42012990ba534787c4a47275 
> .
> Here's a summary of the attempts I've made -- all of them with your
> patch included in my llvm-project tree:
> 
> 1. Don't lower llvm.dbg.declare to llvm.dbg.value at all, no other
> changes -- doesn't work, in the way I expected it to not work
> 
> I tried removing the call to llvm::LowerDbgDeclare that's being made
> by the coroutine passes. The IR and DWARF dump after having done so
> are in the files 'repro.declare.ll' and 'repro.declare.dwarf.txt'.
> 
> Using an executable built from this IR, I can use lldb to break at the
> line where the declare exists,
> https://gist.github.com/modocache/8a7b12eb42012990ba534787c4a47275#file-repro-declare-ll-L167
>  
> ,
> using the command 'b repro.cpp:24'. Running 'e i' at this point prints
> an incorrect value, '(int) $0 = 24742', but at least 'i' is found.
> 
> Then, continuing past a coroutine "suspend point" (a point after which
> the value of 'i' is stored into an offset of the coroutine frame
> object, an outlined coroutine function 'foo.resume' is invoked, 'i' is
> loaded out of an offset on the coroutine frame object, and normally a
> llvm.dbg.value call would have been generated for that 

Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in case of missing location at block begin

[Vedant Kumar via lldb-dev]

> On Oct 10, 2018, at 9:16 AM, Matthias Braun  wrote:
> 
> So I haven't worked much on debug info, but here's the explanation for my 
> patches:
> My original motivation was getting rid of code some code in the llvm codegen 
> that for spills and reloads just picked the next debug location around. That 
> just seemed wrong to me, as spills and reloads really are bookkeeping, we 
> just move values around between registers and memory and none of it is 
> related to anything the user wrote in the source program. So not assigning 
> any debug information felt right for these instructions. Then people noticed 
> line table bloat because of this and I guess we assumed that having exact 
> line-0 annotations isn't that useful that it warrants bloating the debug 
> information...

Right. This doesn't seem any more arbitrary than reusing the previous 
instruction location, which we do all the time. I think it's a reasonable 
tradeoff.


> 
> - Matthias
> 
>> On Oct 10, 2018, at 8:45 AM, via llvm-commits > <mailto:llvm-comm...@lists.llvm.org>> wrote:
>> 
>> Sorry for being late to the party, I have been away.
>> 
>> I am not persuaded that this patch is functionally correct.
>> It's lovely to reduce size, but not at the cost of telling
>> un-truths about source locations of instructions.  If the
>> instructions at the top of the block have no source location,
>> and you don't like using line 0, the correct path forward is to
>> understand why those instructions do not have a source location,
>> not to just pick one from a nearby instruction.

I think we should do that regardless, but it's a much bigger project which 
won't fix the bloat issues the hwasan folks ran into.


>> My choice would be to revert the LLVM patch rather than make
>> the LLDB test accommodate the un-truth.

As Jim pointed out, the LLDB test is doing a whole lot more than it needs to. 
It should be rewritten to focus on data formatting, instead of stepping.

vedant

>> --paulr
>> 
>>> -Original Message-
>>> From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org 
>>> <mailto:lldb-dev-boun...@lists.llvm.org>] On Behalf Of Jim
>>> Ingham via lldb-dev
>>> Sent: Friday, October 05, 2018 9:30 PM
>>> To: Vedant Kumar
>>> Cc: Matthias Braun; Vedant Kumar via llvm-commits; LLDB
>>> Subject: Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in
>>> case of missing location at block begin
>>> 
>>> So in the test scenario, we have this code:
>>> 
>>>printf("// Set second break point at this line.");
>>>(text_list.push_back(std::string("!!!")));
>>> 
>>> and we have a breakpoint on the printf line.  We've just continued to hit
>>> the breakpoint at printf.  Then we do next twice.  That should certainly
>>> get us past the push_back.  If it does not that's either a bug in the line
>>> tables or in lldb's handling of them.  You would not expect to have to
>>> next three times to go from the start of the printf line to past the
>>> push_back of !!!.
>>> 
>>> Considered as a test about stepping, we should applaud the test for having
>>> caught a bug, and go figure out whether the line tables are bogus in this
>>> instance or just innovative and lldb will have to cope...
>>> 
>>> Considered as a test about data formatters, it is a little silly to try to
>>> drive it using next's since the push_backs are going to introduce a bunch
>>> of inlining and the debug information for inlining is often a bit wonky...
>>> 
>>> Best course is to use the breakpoints in this test to drive from point to
>>> point, and make a reduced stepping test case that just shows this bad
>>> stepping behavior.  Then we can fix the latter test failure either in
>>> clang or lldb as is appropriate.
>>> 
>>> Jim
>>> 
>>> 
>>>> On Oct 5, 2018, at 4:18 PM, Vedant Kumar via lldb-dev >> d...@lists.llvm.org <mailto:d...@lists.llvm.org>> wrote:
>>>> 
>>>> No worries, I’ve relaxed the test for now in r343899 to get the bots
>>> going again. Essentially, the test will just step one more time to make
>>> sure the full effect of the push_back() is visible.
>>>> 
>>>> If folks on lldb-dev (cc’d) have a better idea we can try it out.
>>>> 
>>>> vedant
>>>> 
>>>>> On Oct 5, 2018, at 4:15 PM, Matthias Braun >>>> <mailto:ma...@braunis.de>> wrote:
>>>>> 
>>>>> So what should we 

Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in case of missing location at block begin

[Vedant Kumar via lldb-dev]

No worries, I’ve relaxed the test for now in r343899 to get the bots going 
again. Essentially, the test will just step one more time to make sure the full 
effect of the push_back() is visible.

If folks on lldb-dev (cc’d) have a better idea we can try it out.

vedant

> On Oct 5, 2018, at 4:15 PM, Matthias Braun  wrote:
> 
> So what should we do? Revert the llvm commit, fix the LLDB test, xfail on 
> lldb? I'd be fine with any but don't want to learn how lldb tests work at 
> this moment...
> 
>> On Oct 5, 2018, at 4:11 PM, Vedant Kumar via llvm-commits 
>> mailto:llvm-comm...@lists.llvm.org>> wrote:
>> 
>> Sadly, after this commit TestDataFormatterLibcxxList.py started failing with:
>> 
>> ```
>> output: Process 67333 stopped
>> * thread #1, queue = 'com.apple.main-thread', stop reason = step over
>> frame #0: 0x00010eb0 a.out`main at main.cpp:33:16
>>30(text_list.push_back(std::string("smart")));
>>31
>>32printf("// Set second break point at this line.");
>> -> 33(text_list.push_back(std::string("!!!"))); 
>> ^
>>34
>>35std::list countingList = {3141, 3142, 3142,3142,3142, 
>> 3142, 3142, 3141};
>>36countingList.sort();
>> 
>> 
>> runCmd: frame variable text_list[0]
>> output: (std::__1::basic_string, 
>> std::__1::allocator >) [0] = "goofy"
>> 
>> Expecting sub string: goofy
>> Matched
>> 
>> runCmd: frame variable text_list[3]
>> output: None
>> 
>> Expecting sub string: !!!
>> ```
>> 
>> URL: http://lab.llvm.org:8080/green/view/LLDB/job/lldb-cmake/10854 
>> 
>> 
>> I confirmed that reverting this fixes the issue.
>> 
>> I think the problem is that we’re breaking on an instruction that looks like 
>> it’s on line 33, but it’s actually “before” the full effect of the 
>> push_back() is visible.
>> 
>> In which case the test is wrong, because it’s setting the breakpoint too 
>> early…
>> 
>> vedant
>> 
>>> On Oct 5, 2018, at 11:29 AM, Matthias Braun via llvm-commits 
>>> mailto:llvm-comm...@lists.llvm.org>> wrote:
>>> 
>>> Author: matze
>>> Date: Fri Oct  5 11:29:24 2018
>>> New Revision: 343874
>>> 
>>> URL: http://llvm.org/viewvc/llvm-project?rev=343874=rev 
>>> 
>>> Log:
>>> DwarfDebug: Pick next location in case of missing location at block begin
>>> 
>>> Context: Compiler generated instructions do not have a debug location
>>> assigned to them. However emitting 0-line records for all of them bloats
>>> the line tables for very little benefit so we usually avoid doing that.
>>> 
>>> Not emitting anything will lead to the previous debug location getting
>>> applied to the locationless instructions. This is not desirable for
>>> block begin and after labels. Previously we would emit simply emit
>>> line-0 records in this case, this patch changes the behavior to do a
>>> forward search for a debug location in these cases before emitting a
>>> line-0 record to further reduce line table bloat.
>>> 
>>> Inspired by the discussion in https://reviews.llvm.org/D52862 
>>> 
>>> 
>>> Added:
>>>llvm/trunk/test/DebugInfo/X86/dwarf-no-source-loc.mir
>>> Modified:
>>>llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
>>>llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.h
>>>llvm/trunk/test/DebugInfo/AArch64/line-header.ll
>>>llvm/trunk/test/DebugInfo/ARM/single-constant-use-preserves-dbgloc.ll
>>>llvm/trunk/test/DebugInfo/Mips/delay-slot.ll
>>>llvm/trunk/test/DebugInfo/NVPTX/debug-info.ll
>>>llvm/trunk/test/DebugInfo/X86/dwarf-no-source-loc.ll
>>> 
>>> Modified: llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
>>> URL: 
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp?rev=343874=343873=343874=diff
>>>  
>>> 
>>> ==
>>> --- llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp (original)
>>> +++ llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp Fri Oct  5 11:29:24 
>>> 2018
>>> @@ -1313,6 +1313,49 @@ void DwarfDebug::collectEntityInfo(Dwarf
>>>   }
>>> }
>>> 
>>> +static const DebugLoc &
>>> +findNextDebugLoc(MachineBasicBlock::const_iterator MBBI,
>>> + MachineBasicBlock::const_iterator MBBE) {
>>> +  static DebugLoc NoLocation;
>>> +  for ( ; MBBI != MBBE; ++MBBI) {
>>> +if (MBBI->isDebugInstr())
>>> +  continue;
>>> +const DebugLoc  = MBBI->getDebugLoc();
>>> +if (DL)
>>> +  return DL;
>>> +  }
>>> +  return NoLocation;
>>> +}
>>> +
>>> +void DwarfDebug::emitDebugLoc(const DebugLoc ) {
>>> +  unsigned LastAsmLine =
>>> +  Asm->OutStreamer->getContext().getCurrentDwarfLoc().getLine();
>>> +
>>> +  // We have an explicit location, different 

Re: [lldb-dev] [llvm] r343874 - DwarfDebug: Pick next location in case of missing location at block begin

[Vedant Kumar via lldb-dev]

Sadly, after this commit TestDataFormatterLibcxxList.py started failing with:

```
output: Process 67333 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = step over
frame #0: 0x00010eb0 a.out`main at main.cpp:33:16
   30   (text_list.push_back(std::string("smart")));
   31   
   32   printf("// Set second break point at this line.");
-> 33   (text_list.push_back(std::string("!!!"))); 
   ^
   34   
   35   std::list countingList = {3141, 3142, 3142,3142,3142, 3142, 
3142, 3141};
   36   countingList.sort();


runCmd: frame variable text_list[0]
output: (std::__1::basic_string, 
std::__1::allocator >) [0] = "goofy"

Expecting sub string: goofy
Matched

runCmd: frame variable text_list[3]
output: None

Expecting sub string: !!!
```

URL: http://lab.llvm.org:8080/green/view/LLDB/job/lldb-cmake/10854 


I confirmed that reverting this fixes the issue.

I think the problem is that we’re breaking on an instruction that looks like 
it’s on line 33, but it’s actually “before” the full effect of the push_back() 
is visible.

In which case the test is wrong, because it’s setting the breakpoint too early…

vedant

> On Oct 5, 2018, at 11:29 AM, Matthias Braun via llvm-commits 
>  wrote:
> 
> Author: matze
> Date: Fri Oct  5 11:29:24 2018
> New Revision: 343874
> 
> URL: http://llvm.org/viewvc/llvm-project?rev=343874=rev
> Log:
> DwarfDebug: Pick next location in case of missing location at block begin
> 
> Context: Compiler generated instructions do not have a debug location
> assigned to them. However emitting 0-line records for all of them bloats
> the line tables for very little benefit so we usually avoid doing that.
> 
> Not emitting anything will lead to the previous debug location getting
> applied to the locationless instructions. This is not desirable for
> block begin and after labels. Previously we would emit simply emit
> line-0 records in this case, this patch changes the behavior to do a
> forward search for a debug location in these cases before emitting a
> line-0 record to further reduce line table bloat.
> 
> Inspired by the discussion in https://reviews.llvm.org/D52862
> 
> Added:
>llvm/trunk/test/DebugInfo/X86/dwarf-no-source-loc.mir
> Modified:
>llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
>llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.h
>llvm/trunk/test/DebugInfo/AArch64/line-header.ll
>llvm/trunk/test/DebugInfo/ARM/single-constant-use-preserves-dbgloc.ll
>llvm/trunk/test/DebugInfo/Mips/delay-slot.ll
>llvm/trunk/test/DebugInfo/NVPTX/debug-info.ll
>llvm/trunk/test/DebugInfo/X86/dwarf-no-source-loc.ll
> 
> Modified: llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
> URL: 
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp?rev=343874=343873=343874=diff
> ==
> --- llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp (original)
> +++ llvm/trunk/lib/CodeGen/AsmPrinter/DwarfDebug.cpp Fri Oct  5 11:29:24 2018
> @@ -1313,6 +1313,49 @@ void DwarfDebug::collectEntityInfo(Dwarf
>   }
> }
> 
> +static const DebugLoc &
> +findNextDebugLoc(MachineBasicBlock::const_iterator MBBI,
> + MachineBasicBlock::const_iterator MBBE) {
> +  static DebugLoc NoLocation;
> +  for ( ; MBBI != MBBE; ++MBBI) {
> +if (MBBI->isDebugInstr())
> +  continue;
> +const DebugLoc  = MBBI->getDebugLoc();
> +if (DL)
> +  return DL;
> +  }
> +  return NoLocation;
> +}
> +
> +void DwarfDebug::emitDebugLoc(const DebugLoc ) {
> +  unsigned LastAsmLine =
> +  Asm->OutStreamer->getContext().getCurrentDwarfLoc().getLine();
> +
> +  // We have an explicit location, different from the previous location.
> +  // Don't repeat a line-0 record, but otherwise emit the new location.
> +  // (The new location might be an explicit line 0, which we do emit.)
> +  unsigned Line = DL.getLine();
> +  if (PrevInstLoc && Line == 0 && LastAsmLine == 0)
> +return;
> +  unsigned Flags = 0;
> +  if (DL == PrologEndLoc) {
> +Flags |= DWARF2_FLAG_PROLOGUE_END | DWARF2_FLAG_IS_STMT;
> +PrologEndLoc = DebugLoc();
> +  }
> +  // If the line changed, we call that a new statement; unless we went to
> +  // line 0 and came back, in which case it is not a new statement.
> +  unsigned OldLine = PrevInstLoc ? PrevInstLoc.getLine() : LastAsmLine;
> +  if (Line && Line != OldLine)
> +Flags |= DWARF2_FLAG_IS_STMT;
> +
> +  const MDNode *Scope = DL.getScope();
> +  recordSourceLine(Line, DL.getCol(), Scope, Flags);
> +
> +  // If we're not at line 0, remember this location.
> +  if (Line)
> +PrevInstLoc = DL;
> +}
> +
> // Process beginning of an instruction.
> void DwarfDebug::beginInstruction(const MachineInstr *MI) {
>   DebugHandlerBase::beginInstruction(MI);
> @@ -1352,54 +1395,41 @@ void DwarfDebug::beginInstruction(const
> // If 

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

 gdb. That said, I think there are reliable/maintainable ways to do this, 
>>> and proven examples we can learn from in llvm/clang/etc.
>>> 
>>> 
>>>> It's also helpful when adding new features to ensure you've exposed the 
>>>> feature through the API sufficiently.  The first thing I thought to try 
>>>> when writing the example below was SBFrame::IsArtificial() (see 
>>>> SBFrame::IsInlined()) which doesn't exist.  If a driver / IDE is going to 
>>>> visually indicate artificial frames, they'll need that.
>>> 
>>> Sure. That's true, we do need API exposure for new features, and again we 
>>> can enforce that during code review. The reason you didn't find 
>>> IsArtificial() is because it's sitting on my disk :). Haven't shared the 
>>> patch yet.
>>> 
>>> vedant
>>> 
>>>> 
>>>> J
>>>> 
>>>>> On Aug 14, 2018, at 5:56 PM, Vedant Kumar  wrote:
>>>>> 
>>>>> It'd be easy to update FileCheck tests when changing the debugger (this 
>>>>> happens all the time in clang/swift). OTOH, the verbosity of the python 
>>>>> API means that fewer tests get written. I see a real need to make 
>>>>> expressive tests easier to write.
>>>>> 
>>>>> vedant
>>>>> 
>>>>>> On Aug 14, 2018, at 5:38 PM, Jason Molenda  wrote:
>>>>>> 
>>>>>> I'd argue against this approach because it's exactly why the lit tests 
>>>>>> don't run against the lldb driver -- they're hardcoding the output of 
>>>>>> the lldb driver command into the testsuite and these will eventually 
>>>>>> make it much more difficult to change and improve the driver as we've 
>>>>>> accumulated this style of test.
>>>>>> 
>>>>>> This is a perfect test for a normal SB API.  Run to your breakpoints and 
>>>>>> check the stack frames.
>>>>>> 
>>>>>> f0 = thread.GetFrameAtIndex(0)
>>>>>> check that f0.GetFunctionName() == sink
>>>>>> check that f0.IsArtifical() == True
>>>>>> check that f0.GetLineEntry().GetLine() == expected line number
>>>>>> 
>>>>>> 
>>>>>> it's more verbose, but it's also much more explicit about what it's 
>>>>>> checking, and easy to see what has changed if there is a failure.
>>>>>> 
>>>>>> 
>>>>>> J
>>>>>> 
>>>>>>> On Aug 14, 2018, at 5:31 PM, Vedant Kumar via lldb-dev 
>>>>>>>  wrote:
>>>>>>> 
>>>>>>> Hello,
>>>>>>> 
>>>>>>> I'd like to make FileCheck available within lldb inline tests, in 
>>>>>>> addition to existing helpers like 'runCmd' and 'expect'.
>>>>>>> 
>>>>>>> My motivation is that several tests I'm working on can't be made as 
>>>>>>> rigorous as they need to be without FileCheck-style checks. In 
>>>>>>> particular, the 'matching', 'substrs', and 'patterns' arguments to 
>>>>>>> runCmd/expect don't allow me to verify the ordering of checked input, 
>>>>>>> to be stringent about line numbers, or to capture & reuse snippets of 
>>>>>>> text from the input stream.
>>>>>>> 
>>>>>>> I'd curious to know if anyone else is interested or would be willing to 
>>>>>>> review this (https://reviews.llvm.org/D50751).
>>>>>>> 
>>>>>>> Here's an example of an inline test which benefits from FileCheck-style 
>>>>>>> checking. This test is trying to check that certain frames appear in a 
>>>>>>> backtrace when stopped inside of the "sink" function. Notice that 
>>>>>>> without FileCheck, it's not possible to verify the order in which 
>>>>>>> frames are printed, and that dealing with line numbers would be 
>>>>>>> cumbersome.
>>>>>>> 
>>>>>>> ```
>>>>>>> --- 
>>>>>>> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>>>>>>> +++ 
>>>>>>> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>&

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

They add support for injecting artificial tail call frames into a 
StackFrameList. D50478 <https://reviews.llvm.org/D50478> is the first one -- 
there's no SB API extension in the initial patch, although that's planned.

For context, tail calling frames go missing in backtraces because the frame of 
the caller is reused by the callee. Call site information from D49887 
<https://reviews.llvm.org/D49887> allows a debugger to reconstruct a sequence 
of (tail) calls which led from one function to another (it essentially provides 
a call graph marked up with return-pc information). The motivation for doing 
this is to improve crashlog/backtrace quality for optimized ObjC/Swift apps.

Testing is a bit tricky because it's important to make sure that lldb doesn't 
create artificial frames in scenarios where the execution history can't be 
inferred from the call graph (e.g when there's tail recursion, etc).

vedant

> On Aug 15, 2018, at 2:01 PM, Zachary Turner  wrote:
> 
> What do your patches do, out of curiosity?
> 
> On Wed, Aug 15, 2018 at 12:45 PM Vedant Kumar  <mailto:v...@apple.com>> wrote:
> 
>> On Aug 15, 2018, at 12:27 PM, Zachary Turner > <mailto:ztur...@google.com>> wrote:
>> 
>> Back to the original proposal, my biggest concern is that a single inline 
>> test could generate many FileCheck invocations.  This could cause measurable 
>> performance impact on the test suite.  Have you considered this?
> 
> That's a good point. I hadn't considered that. My thoughts on that are;
> 
> - It's relatively cheap to create a FileCheck process. If the build is 
> (A|T)sanified, we can copy in a non-sanitized FileCheck to speed things up.
> 
> - Based on the time it takes to run check-{llvm,clang} locally, which have 
> ~56,000 FileCheck invocations, my intuition is that the overhead ought to be 
> manageable.
> 
> - The status quo is doing Python's re.search over a chunk of command output. 
> My (unverified) intuition is that FileCheck won't be slower than that. 
> Actually, FileCheck has an algorithmic advantage because it doesn't re-scan 
> the input text from the beginning of the text each time it tries to match a 
> substring. `self.expect` does.
> 
> 
>> 
>> Another possible solution is what i mentioned earlier, basically to expose a 
>> debugger object model.  This would allow you to accomplish what you want 
>> without FileCheck, while simultaneously being making many other types of 
>> tests easier to write at the same time.  On the other hand, it’s a larger 
>> effort to create this system, but I think long term it would pay back 
>> enormously (it’s even useful as a general purpose debugger feature, not 
>> limited to testing)
> 
> I'd volunteer to work on that. At the moment I really need to get some form 
> of testing put together for my patches soon.
> 
> vedant
> 
>> 
>> On Tue, Aug 14, 2018 at 5:31 PM Vedant Kumar via lldb-dev 
>> mailto:lldb-dev@lists.llvm.org>> wrote:
>> Hello,
>> 
>> I'd like to make FileCheck available within lldb inline tests, in addition 
>> to existing helpers like 'runCmd' and 'expect'.
>> 
>> My motivation is that several tests I'm working on can't be made as rigorous 
>> as they need to be without FileCheck-style checks. In particular, the 
>> 'matching', 'substrs', and 'patterns' arguments to runCmd/expect don't allow 
>> me to verify the ordering of checked input, to be stringent about line 
>> numbers, or to capture & reuse snippets of text from the input stream.
>> 
>> I'd curious to know if anyone else is interested or would be willing to 
>> review this (https://reviews.llvm.org/D50751 
>> <https://reviews.llvm.org/D50751>).
>> 
>> Here's an example of an inline test which benefits from FileCheck-style 
>> checking. This test is trying to check that certain frames appear in a 
>> backtrace when stopped inside of the "sink" function. Notice that without 
>> FileCheck, it's not possible to verify the order in which frames are 
>> printed, and that dealing with line numbers would be cumbersome.
>> 
>> ```
>> --- 
>> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>> +++ 
>> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>> @@ -9,16 +9,21 @@
>>  
>>  volatile int x;
>>  
>> +// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]
>>  void __attribute__((noinline)) sink() {
>> -  x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 'func3', 
>> 'sink'])
>> +  x++; //% self.filecheck("b

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

> On Aug 15, 2018, at 12:27 PM, Zachary Turner  wrote:
> 
> Back to the original proposal, my biggest concern is that a single inline 
> test could generate many FileCheck invocations.  This could cause measurable 
> performance impact on the test suite.  Have you considered this?

That's a good point. I hadn't considered that. My thoughts on that are;

- It's relatively cheap to create a FileCheck process. If the build is 
(A|T)sanified, we can copy in a non-sanitized FileCheck to speed things up.

- Based on the time it takes to run check-{llvm,clang} locally, which have 
~56,000 FileCheck invocations, my intuition is that the overhead ought to be 
manageable.

- The status quo is doing Python's re.search over a chunk of command output. My 
(unverified) intuition is that FileCheck won't be slower than that. Actually, 
FileCheck has an algorithmic advantage because it doesn't re-scan the input 
text from the beginning of the text each time it tries to match a substring. 
`self.expect` does.


> 
> Another possible solution is what i mentioned earlier, basically to expose a 
> debugger object model.  This would allow you to accomplish what you want 
> without FileCheck, while simultaneously being making many other types of 
> tests easier to write at the same time.  On the other hand, it’s a larger 
> effort to create this system, but I think long term it would pay back 
> enormously (it’s even useful as a general purpose debugger feature, not 
> limited to testing)

I'd volunteer to work on that. At the moment I really need to get some form of 
testing put together for my patches soon.

vedant

> 
> On Tue, Aug 14, 2018 at 5:31 PM Vedant Kumar via lldb-dev 
> mailto:lldb-dev@lists.llvm.org>> wrote:
> Hello,
> 
> I'd like to make FileCheck available within lldb inline tests, in addition to 
> existing helpers like 'runCmd' and 'expect'.
> 
> My motivation is that several tests I'm working on can't be made as rigorous 
> as they need to be without FileCheck-style checks. In particular, the 
> 'matching', 'substrs', and 'patterns' arguments to runCmd/expect don't allow 
> me to verify the ordering of checked input, to be stringent about line 
> numbers, or to capture & reuse snippets of text from the input stream.
> 
> I'd curious to know if anyone else is interested or would be willing to 
> review this (https://reviews.llvm.org/D50751 
> <https://reviews.llvm.org/D50751>).
> 
> Here's an example of an inline test which benefits from FileCheck-style 
> checking. This test is trying to check that certain frames appear in a 
> backtrace when stopped inside of the "sink" function. Notice that without 
> FileCheck, it's not possible to verify the order in which frames are printed, 
> and that dealing with line numbers would be cumbersome.
> 
> ```
> --- 
> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
> +++ 
> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
> @@ -9,16 +9,21 @@
>  
>  volatile int x;
>  
> +// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]
>  void __attribute__((noinline)) sink() {
> -  x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 'func3', 
> 'sink'])
> +  x++; //% self.filecheck("bt", "main.cpp")
>  }
>  
> +// CHECK-NEXT: frame #1: {{.*}}func3() {{.*}}[opt] [artificial]
>  void __attribute__((noinline)) func3() { sink(); /* tail */ }
>  
> +// CHECK-NEXT: frame #2: {{.*}}func2() at main.cpp:[[@LINE+1]] [opt]
>  void __attribute__((disable_tail_calls, noinline)) func2() { func3(); /* 
> regular */ }
>  
> +// CHECK-NEXT: frame #3: {{.*}}func1() {{.*}}[opt] [artificial]
>  void __attribute__((noinline)) func1() { func2(); /* tail */ }
>  
> +// CHECK-NEXT: frame #4: {{.*}}main at main.cpp:[[@LINE+2]] [opt]
>  int __attribute__((disable_tail_calls)) main() {
>func1(); /* regular */
>return 0;
> ```
> 
> For reference, here's the output of the "bt" command:
> 
> ```
> runCmd: bt
> output: * thread #1, queue = 'com.apple.main-thread', stop reason = 
> breakpoint 1.1
>   * frame #0: 0x00010c6a6f64 a.out`sink() at main.cpp:14 [opt]
> frame #1: 0x00010c6a6f70 a.out`func3() at main.cpp:15 [opt] 
> [artificial]
> frame #2: 0x00010c6a6f89 a.out`func2() at main.cpp:21 [opt]
> frame #3: 0x00010c6a6f90 a.out`func1() at main.cpp:21 [opt] 
> [artificial]
> frame #4: 0x00010c6a6fa9 a.out`main at main.cpp:28 [opt]
> ```
> 
> thanks,
> vedant
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
> <http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev>

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

Aug 14, 2018, at 5:56 PM, Vedant Kumar  wrote:
>>>> 
>>>> It'd be easy to update FileCheck tests when changing the debugger (this 
>>>> happens all the time in clang/swift). OTOH, the verbosity of the python 
>>>> API means that fewer tests get written. I see a real need to make 
>>>> expressive tests easier to write.
>>>> 
>>>> vedant
>>>> 
>>>>> On Aug 14, 2018, at 5:38 PM, Jason Molenda  wrote:
>>>>> 
>>>>> I'd argue against this approach because it's exactly why the lit tests 
>>>>> don't run against the lldb driver -- they're hardcoding the output of the 
>>>>> lldb driver command into the testsuite and these will eventually make it 
>>>>> much more difficult to change and improve the driver as we've accumulated 
>>>>> this style of test.
>>>>> 
>>>>> This is a perfect test for a normal SB API.  Run to your breakpoints and 
>>>>> check the stack frames.
>>>>> 
>>>>> f0 = thread.GetFrameAtIndex(0)
>>>>> check that f0.GetFunctionName() == sink
>>>>> check that f0.IsArtifical() == True
>>>>> check that f0.GetLineEntry().GetLine() == expected line number
>>>>> 
>>>>> 
>>>>> it's more verbose, but it's also much more explicit about what it's 
>>>>> checking, and easy to see what has changed if there is a failure.
>>>>> 
>>>>> 
>>>>> J
>>>>> 
>>>>>> On Aug 14, 2018, at 5:31 PM, Vedant Kumar via lldb-dev 
>>>>>>  wrote:
>>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> I'd like to make FileCheck available within lldb inline tests, in 
>>>>>> addition to existing helpers like 'runCmd' and 'expect'.
>>>>>> 
>>>>>> My motivation is that several tests I'm working on can't be made as 
>>>>>> rigorous as they need to be without FileCheck-style checks. In 
>>>>>> particular, the 'matching', 'substrs', and 'patterns' arguments to 
>>>>>> runCmd/expect don't allow me to verify the ordering of checked input, to 
>>>>>> be stringent about line numbers, or to capture & reuse snippets of text 
>>>>>> from the input stream.
>>>>>> 
>>>>>> I'd curious to know if anyone else is interested or would be willing to 
>>>>>> review this (https://reviews.llvm.org/D50751).
>>>>>> 
>>>>>> Here's an example of an inline test which benefits from FileCheck-style 
>>>>>> checking. This test is trying to check that certain frames appear in a 
>>>>>> backtrace when stopped inside of the "sink" function. Notice that 
>>>>>> without FileCheck, it's not possible to verify the order in which frames 
>>>>>> are printed, and that dealing with line numbers would be cumbersome.
>>>>>> 
>>>>>> ```
>>>>>> --- 
>>>>>> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>>>>>> +++ 
>>>>>> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>>>>>> @@ -9,16 +9,21 @@
>>>>>> 
>>>>>> volatile int x;
>>>>>> 
>>>>>> +// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]
>>>>>> void __attribute__((noinline)) sink() {
>>>>>> -  x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 
>>>>>> 'func3', 'sink'])
>>>>>> +  x++; //% self.filecheck("bt", "main.cpp")
>>>>>> }
>>>>>> 
>>>>>> +// CHECK-NEXT: frame #1: {{.*}}func3() {{.*}}[opt] [artificial]
>>>>>> void __attribute__((noinline)) func3() { sink(); /* tail */ }
>>>>>> 
>>>>>> +// CHECK-NEXT: frame #2: {{.*}}func2() at main.cpp:[[@LINE+1]] [opt]
>>>>>> void __attribute__((disable_tail_calls, noinline)) func2() { func3(); /* 
>>>>>> regular */ }
>>>>>> 
>>>>>> +// CHECK-NEXT: frame #3: {{.*}}func1() {{.*}}[opt] [artificial]
>>>>>> void __attribute__((noinline)) func1() { func2(); /* tail */ }
>>>>>> 
>>>>>> +// CHECK-NEXT: frame #4: {{.*}}main at main.cpp:[[@LINE+2]] [opt]
>>>>>> int __attribute__((disable_tail_calls)) main() {
>>>>>> func1(); /* regular */
>>>>>> return 0;
>>>>>> ```
>>>>>> 
>>>>>> For reference, here's the output of the "bt" command:
>>>>>> 
>>>>>> ```
>>>>>> runCmd: bt
>>>>>> output: * thread #1, queue = 'com.apple.main-thread', stop reason = 
>>>>>> breakpoint 1.1
>>>>>> * frame #0: 0x00010c6a6f64 a.out`sink() at main.cpp:14 [opt]
>>>>>> frame #1: 0x00010c6a6f70 a.out`func3() at main.cpp:15 [opt] 
>>>>>> [artificial]
>>>>>> frame #2: 0x00010c6a6f89 a.out`func2() at main.cpp:21 [opt]
>>>>>> frame #3: 0x00010c6a6f90 a.out`func1() at main.cpp:21 [opt] 
>>>>>> [artificial]
>>>>>> frame #4: 0x00010c6a6fa9 a.out`main at main.cpp:28 [opt]
>>>>>> ```
>>>>>> 
>>>>>> thanks,
>>>>>> vedant
>>>>>> ___
>>>>>> lldb-dev mailing list
>>>>>> lldb-dev@lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>> 
>>>> 
>>> 
>> 
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

> On Aug 15, 2018, at 10:15 AM, Jim Ingham via lldb-dev 
>  wrote:
> 
> 
> 
>> On Aug 14, 2018, at 7:48 PM, Zachary Turner via lldb-dev 
>>  wrote:
>> 
>> 
>> On Tue, Aug 14, 2018 at 6:58 PM Jason Molenda  wrote:
>> 
>> 
>>> On Aug 14, 2018, at 6:39 PM, Zachary Turner  wrote:
>>> 
>>> Having bugs also makes the debugger harder to innovate in the future 
>>> because it’s, not having tests leads to having bugs, and sb api tests leads 
>>> to not having te
>> 
>> Yes, lldb does not have these problems -- because we learned from our 
>> decades working on gdb, and did not repeat that mistake.  To be honest, lldb 
>> is such a young debugger - barely a decade old, depending on how you count 
>> it, that ANY testsuite approach would be fine at this point.  Add a couple 
>> more decades and we'd be back into the hole that gdb was in.  {I have not 
>> worked on gdb in over a decade, so I don't know how their testing 
>> methodology may be today}
>> That doesn’t mean that the current approach is the final word.  As new 
>> people come onto the project, new ideas come forth and we should entertain 
>> them rather than deciding that all decisions are set in stone forever.
>> 
>> For example, the object model based approach I mentioned earlier would not 
>> have any of the problems that you’ve described from gdb.  Just because one 
>> set of problems has been solved doesn’t mean we should declare victory and 
>> say there’s no point in trying to solve the remaining problems too.  And 
>> right now, the problem is that we need to be coming up with a way to make 
>> tests easier to write so that people will actually write them
> 
> Just historically it is not true that we didn't have problems with command 
> output scraping vrs. the testsuite.  The major offender in this regard was 
> breakpoint setting/checking.  But the first time I had to go adjust a whole 
> bunch of tests when I wanted to change the breakpoint output, I added test 
> routines to do all this business, which mitigated the need for these changes. 
>  It is true I haven't had to fiddle with this much then - though that is 
> mostly because I was careful in how I checked output, and didn't check 
> inessential aspects.  That's easy to do when you sit down and do it once, but 
> if everybody just wings some regex every time they do a check you will end up 
> with lots over-eager checks that cause problems down the road.

I definitely sympathize with this, but don't think the problems that arise in 
some tests which pattern-match against input are characteristic of all such 
tests. The lesson you're describing -- the need to be careful to only examine 
the interesting parts of the API/driver output -- applies generally.


> I also want to reiterate another point Jason made, which is that LLDB not 
> only provides a command line interface, but it also provides an API.  Again, 
> we've gone over this before but I'm pretty sure that at this point it's still 
> true that in terms of number of users, far more people interact with lldb 
> through the API (Xcode, VSCode through the gdb-mi or through Greg's new 
> adaptor when that's done...) than interact with it through the command line.  
> And writing SB API tests when you add a new feature is a great way to ensure 
> that you have thought about exposing your new feature to the SB API's 
> correctly.  In that example, a GUI needs to have a way to indicate that these 
> "tail call" frames are special and you shouldn't be worried that they have no 
> local variables, even though we show you source for them that obviously has 
> variables...

I agree with all of this. However, I don't take it to mean that there ought not 
be tests for the driver output.


> So we need ways to encourage lldb developers to be aware that the SB API's 
> need to have (tested) access to new features in the debugger and not just the 
> command line.  SB API tests are good way to encourage this.  We can also do 
> it as part of reviews and the like, but I think it better done as part of 
> your development than after the fact in reviews...  Again, that doesn't argue 
> for their being the ONLY way to test things in lldb, but it does need to be 
> part of an lldb developer's practice.

Totally agree.

vedant

> I like your idea of some kind of object model based output for test checking. 
>  I somehow had the impression that that was your intent for lldb-test once it 
> was a real boy, but maybe I was just projecting...  Relying on ad hoc dumping 
> routines which are mostly for debugging purposes seemed okay as a boot-strap 
> method, but not a great design long-term.  Teaching either the SB or the 
> lldb_private objects to describe themselves in a structured way and using 
> that to do testing seems great.
> 
> Jim
> 
> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

;> It's more verbose, and it does mean test writers need to learn the public 
>> API, but it's also much more stable and debuggable in the future.  It's a 
>> higher up front cost but we're paid back in being able to develop lldb more 
>> quickly in the future, where our published API behaviors are being tested 
>> directly, and the things that must not be broken.  The lldb driver's output 
>> isn't a contract, and treating it like one makes the debugger harder to 
>> innovate in the future.
>> 
>> It's also helpful when adding new features to ensure you've exposed the 
>> feature through the API sufficiently.  The first thing I thought to try when 
>> writing the example below was SBFrame::IsArtificial() (see 
>> SBFrame::IsInlined()) which doesn't exist.  If a driver / IDE is going to 
>> visually indicate artificial frames, they'll need that.
>> 
>> J
>> 
>>> On Aug 14, 2018, at 5:56 PM, Vedant Kumar  wrote:
>>> 
>>> It'd be easy to update FileCheck tests when changing the debugger (this 
>>> happens all the time in clang/swift). OTOH, the verbosity of the python API 
>>> means that fewer tests get written. I see a real need to make expressive 
>>> tests easier to write.
>>> 
>>> vedant
>>> 
>>>> On Aug 14, 2018, at 5:38 PM, Jason Molenda  wrote:
>>>> 
>>>> I'd argue against this approach because it's exactly why the lit tests 
>>>> don't run against the lldb driver -- they're hardcoding the output of the 
>>>> lldb driver command into the testsuite and these will eventually make it 
>>>> much more difficult to change and improve the driver as we've accumulated 
>>>> this style of test.
>>>> 
>>>> This is a perfect test for a normal SB API.  Run to your breakpoints and 
>>>> check the stack frames.
>>>> 
>>>> f0 = thread.GetFrameAtIndex(0)
>>>> check that f0.GetFunctionName() == sink
>>>> check that f0.IsArtifical() == True
>>>> check that f0.GetLineEntry().GetLine() == expected line number
>>>> 
>>>> 
>>>> it's more verbose, but it's also much more explicit about what it's 
>>>> checking, and easy to see what has changed if there is a failure.
>>>> 
>>>> 
>>>> J
>>>> 
>>>>> On Aug 14, 2018, at 5:31 PM, Vedant Kumar via lldb-dev 
>>>>>  wrote:
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> I'd like to make FileCheck available within lldb inline tests, in 
>>>>> addition to existing helpers like 'runCmd' and 'expect'.
>>>>> 
>>>>> My motivation is that several tests I'm working on can't be made as 
>>>>> rigorous as they need to be without FileCheck-style checks. In 
>>>>> particular, the 'matching', 'substrs', and 'patterns' arguments to 
>>>>> runCmd/expect don't allow me to verify the ordering of checked input, to 
>>>>> be stringent about line numbers, or to capture & reuse snippets of text 
>>>>> from the input stream.
>>>>> 
>>>>> I'd curious to know if anyone else is interested or would be willing to 
>>>>> review this (https://reviews.llvm.org/D50751).
>>>>> 
>>>>> Here's an example of an inline test which benefits from FileCheck-style 
>>>>> checking. This test is trying to check that certain frames appear in a 
>>>>> backtrace when stopped inside of the "sink" function. Notice that without 
>>>>> FileCheck, it's not possible to verify the order in which frames are 
>>>>> printed, and that dealing with line numbers would be cumbersome.
>>>>> 
>>>>> ```
>>>>> --- 
>>>>> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>>>>> +++ 
>>>>> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>>>>> @@ -9,16 +9,21 @@
>>>>> 
>>>>> volatile int x;
>>>>> 
>>>>> +// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]
>>>>> void __attribute__((noinline)) sink() {
>>>>> -  x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 
>>>>> 'func3', 'sink'])
>>>>> +  x++; //% self.filecheck("bt", "main.cpp")
>>>>> }
>>>>> 
>>>>>

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

> On Aug 14, 2018, at 6:19 PM, Jason Molenda  wrote:
> 
> It's more verbose, and it does mean test writers need to learn the public 
> API, but it's also much more stable and debuggable in the future.

I'm not sure about this. Having looked at failing sb api tests for a while now, 
I find them about as easy to navigate and fix as FileCheck tests in llvm.


> It's a higher up front cost but we're paid back in being able to develop lldb 
> more quickly in the future, where our published API behaviors are being 
> tested directly, and the things that must not be broken.

I think the right solution here is to require API tests when new functionality 
is introduced. We can enforce this during code review. Making it impossible to 
write tests against the driver's output doesn't seem like the best solution. It 
means that far fewer tests will be written (note that a test suite run of lldb 
gives less than 60% code coverage). It also means that the driver's output 
isn't tested as much as it should be.


>  The lldb driver's output isn't a contract, and treating it like one makes 
> the debugger harder to innovate in the future.

I appreciate your experience with this (pattern matching on driver input) in 
gdb. That said, I think there are reliable/maintainable ways to do this, and 
proven examples we can learn from in llvm/clang/etc.


> It's also helpful when adding new features to ensure you've exposed the 
> feature through the API sufficiently.  The first thing I thought to try when 
> writing the example below was SBFrame::IsArtificial() (see 
> SBFrame::IsInlined()) which doesn't exist.  If a driver / IDE is going to 
> visually indicate artificial frames, they'll need that.

Sure. That's true, we do need API exposure for new features, and again we can 
enforce that during code review. The reason you didn't find IsArtificial() is 
because it's sitting on my disk :). Haven't shared the patch yet.

vedant

> 
> J
> 
>> On Aug 14, 2018, at 5:56 PM, Vedant Kumar  wrote:
>> 
>> It'd be easy to update FileCheck tests when changing the debugger (this 
>> happens all the time in clang/swift). OTOH, the verbosity of the python API 
>> means that fewer tests get written. I see a real need to make expressive 
>> tests easier to write.
>> 
>> vedant
>> 
>>> On Aug 14, 2018, at 5:38 PM, Jason Molenda  wrote:
>>> 
>>> I'd argue against this approach because it's exactly why the lit tests 
>>> don't run against the lldb driver -- they're hardcoding the output of the 
>>> lldb driver command into the testsuite and these will eventually make it 
>>> much more difficult to change and improve the driver as we've accumulated 
>>> this style of test.
>>> 
>>> This is a perfect test for a normal SB API.  Run to your breakpoints and 
>>> check the stack frames.
>>> 
>>> f0 = thread.GetFrameAtIndex(0)
>>> check that f0.GetFunctionName() == sink
>>> check that f0.IsArtifical() == True
>>> check that f0.GetLineEntry().GetLine() == expected line number
>>> 
>>> 
>>> it's more verbose, but it's also much more explicit about what it's 
>>> checking, and easy to see what has changed if there is a failure.
>>> 
>>> 
>>> J
>>> 
>>>> On Aug 14, 2018, at 5:31 PM, Vedant Kumar via lldb-dev 
>>>>  wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> I'd like to make FileCheck available within lldb inline tests, in addition 
>>>> to existing helpers like 'runCmd' and 'expect'.
>>>> 
>>>> My motivation is that several tests I'm working on can't be made as 
>>>> rigorous as they need to be without FileCheck-style checks. In particular, 
>>>> the 'matching', 'substrs', and 'patterns' arguments to runCmd/expect don't 
>>>> allow me to verify the ordering of checked input, to be stringent about 
>>>> line numbers, or to capture & reuse snippets of text from the input stream.
>>>> 
>>>> I'd curious to know if anyone else is interested or would be willing to 
>>>> review this (https://reviews.llvm.org/D50751).
>>>> 
>>>> Here's an example of an inline test which benefits from FileCheck-style 
>>>> checking. This test is trying to check that certain frames appear in a 
>>>> backtrace when stopped inside of the "sink" function. Notice that without 
>>>> FileCheck, it's not possible to verify the order in which frames are 
>>>> printed, and that dealing with line numbers would be cumbersome.
>>>> 
>>>> ```
>>>> --- 
>>>> a/lldb/packages/Pyt

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

> On Aug 14, 2018, at 6:39 PM, Zachary Turner  wrote:
> 
> Having bugs also makes the debugger harder to innovate in the future because 
> it’s, not having tests leads to having bugs, and sb api tests leads to not 
> having tests.

Yeah, this might be a bit of an under-appreciated point.

If there's a high up front cost to writing tests, not as many of them will be 
written.

I'm having a hard time imagining writing sb api tests which can exercise every 
tricky path through a new patchset I'm working on. Without being able to write 
tests against more compact input (a la FileCheck), I really don't see a way 
forward.


> At the end of the day, it doesn’t matter how stable the tests are if there 
> arent enough of them. There should be about 10x-20x as many tests as there 
> are currently, and that will simply never happen under the current approach. 
> If it means we need to have multiple different styles of test, so be it. The 
> problem we face right now has nothing to do with command output changing, and 
> in fact I don’t that we’ve *ever* had this problem. So we should be focusing 
> on problems we have, not problems we don’t have.
> 
> Note that it is not strictly necessary for a test to check the debuggers 
> command output. There could be a different set of commands whose only purpose 
> is to print information for the purposes of debugging. One idea would be to 
> introduce the notion of a debugger object model, where you print various 
> aspects of the debuggers state with an object like syntax. For example,
> 
> (lldb) p debugger.targets
> ~/foo (running, pid: 123)
> 
> (lldb) p debugger.targets[0].threads[0].frames[1]
> int main(int argc=3, char **argv=0x12345678) + 0x72
> 
> (lldb) p debugger.targets[0].threads[0].frames[1].params[0]
> int argc=3
> 
> (lldb) p debugger.targets[0].breakpoints
> [1] main.cpp:72
> 
> Etc. you can get arbitrarily granular and dxpose every detail of the 
> debuggers internal state this way, and the output is so simple that you never 
> have to worry about it changing.
> 
> That said, I think history has shown that limiting ourselves to sb api tests, 
> despite all the theoretical benefits, leads to insufficient test coverage. So 
> while it has benefits, it also has problems for which we need a better 
> solution

Yeah.

vedant

> On Tue, Aug 14, 2018 at 6:19 PM Jason Molenda via lldb-dev 
> mailto:lldb-dev@lists.llvm.org>> wrote:
> It's more verbose, and it does mean test writers need to learn the public 
> API, but it's also much more stable and debuggable in the future.  It's a 
> higher up front cost but we're paid back in being able to develop lldb more 
> quickly in the future, where our published API behaviors are being tested 
> directly, and the things that must not be broken.  The lldb driver's output 
> isn't a contract, and treating it like one makes the debugger harder to 
> innovate in the future.
> 
> It's also helpful when adding new features to ensure you've exposed the 
> feature through the API sufficiently.  The first thing I thought to try when 
> writing the example below was SBFrame::IsArtificial() (see 
> SBFrame::IsInlined()) which doesn't exist.  If a driver / IDE is going to 
> visually indicate artificial frames, they'll need that.
> 
> J
> 
> > On Aug 14, 2018, at 5:56 PM, Vedant Kumar  > <mailto:v...@apple.com>> wrote:
> > 
> > It'd be easy to update FileCheck tests when changing the debugger (this 
> > happens all the time in clang/swift). OTOH, the verbosity of the python API 
> > means that fewer tests get written. I see a real need to make expressive 
> > tests easier to write.
> > 
> > vedant
> > 
> >> On Aug 14, 2018, at 5:38 PM, Jason Molenda  >> <mailto:jmole...@apple.com>> wrote:
> >> 
> >> I'd argue against this approach because it's exactly why the lit tests 
> >> don't run against the lldb driver -- they're hardcoding the output of the 
> >> lldb driver command into the testsuite and these will eventually make it 
> >> much more difficult to change and improve the driver as we've accumulated 
> >> this style of test.
> >> 
> >> This is a perfect test for a normal SB API.  Run to your breakpoints and 
> >> check the stack frames.
> >> 
> >> f0 = thread.GetFrameAtIndex(0)
> >> check that f0.GetFunctionName() == sink
> >> check that f0.IsArtifical() == True
> >> check that f0.GetLineEntry().GetLine() == expected line number
> >> 
> >> 
> >> it's more verbose, but it's also much more explicit about what it's 
> >> checking, and easy to see what has changed if there is a failure.
> >>

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

> On Aug 14, 2018, at 6:07 PM, Zachary Turner  wrote:
> 
> 
> 
> On Tue, Aug 14, 2018 at 5:56 PM Vedant Kumar  > wrote:
> 
> 
>> On Aug 14, 2018, at 5:34 PM, Zachary Turner > > wrote:
>> 
>> I’ve thought about this in the past but the conclusion I came to is that 
>> lldbinline tests are actually just filecheck tests in disguise. Why do we 
>> need both? I’d rather delete the lldbinline infrastructure entirely and make 
>> a new lit TestFormat that basically does what lldbinline already does 
> 
> An inline test does more than simply pattern-matching input. It builds a 
> program, sets breakpoints, etc. I'd rather make this existing infrastructure 
> easier to use than come up with something new.
> 
> vedant
> 
> Right, but only one specific type of lit test  depends on pattern matching, 
> and those are the SHTest format tests.  You can make an arbitrary test 
> format, including one that builds programs, set breakpoints etc.  the format 
> and structure of an lldbinline test need not even change at all (except that 
> I think we could eliminate the .py file).  The lldbinline tests are about as 
> close to a drop in fit for lit as we can get, the only thing that needs to 
> happen is the code in lldbinline.py needs to move to something called 
> InlineTestFormat.py and then be hooked into lit

Creating a new lit test format compatible with inline tests sounds like a nice 
follow-up step. The immediate problem I'm trying to address is that it's too 
cumbersome to write the tests I need without FileCheck-like checks in inline 
tests. I think introducing this functionality is a necessary prerequisite.

vedant

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

It'd be easy to update FileCheck tests when changing the debugger (this happens 
all the time in clang/swift). OTOH, the verbosity of the python API means that 
fewer tests get written. I see a real need to make expressive tests easier to 
write.

vedant

> On Aug 14, 2018, at 5:38 PM, Jason Molenda  wrote:
> 
> I'd argue against this approach because it's exactly why the lit tests don't 
> run against the lldb driver -- they're hardcoding the output of the lldb 
> driver command into the testsuite and these will eventually make it much more 
> difficult to change and improve the driver as we've accumulated this style of 
> test.
> 
> This is a perfect test for a normal SB API.  Run to your breakpoints and 
> check the stack frames.
> 
>  f0 = thread.GetFrameAtIndex(0)
>  check that f0.GetFunctionName() == sink
>  check that f0.IsArtifical() == True
>  check that f0.GetLineEntry().GetLine() == expected line number
> 
> 
> it's more verbose, but it's also much more explicit about what it's checking, 
> and easy to see what has changed if there is a failure.
> 
> 
> J
> 
>> On Aug 14, 2018, at 5:31 PM, Vedant Kumar via lldb-dev 
>>  wrote:
>> 
>> Hello,
>> 
>> I'd like to make FileCheck available within lldb inline tests, in addition 
>> to existing helpers like 'runCmd' and 'expect'.
>> 
>> My motivation is that several tests I'm working on can't be made as rigorous 
>> as they need to be without FileCheck-style checks. In particular, the 
>> 'matching', 'substrs', and 'patterns' arguments to runCmd/expect don't allow 
>> me to verify the ordering of checked input, to be stringent about line 
>> numbers, or to capture & reuse snippets of text from the input stream.
>> 
>> I'd curious to know if anyone else is interested or would be willing to 
>> review this (https://reviews.llvm.org/D50751).
>> 
>> Here's an example of an inline test which benefits from FileCheck-style 
>> checking. This test is trying to check that certain frames appear in a 
>> backtrace when stopped inside of the "sink" function. Notice that without 
>> FileCheck, it's not possible to verify the order in which frames are 
>> printed, and that dealing with line numbers would be cumbersome.
>> 
>> ```
>> --- 
>> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>> +++ 
>> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
>> @@ -9,16 +9,21 @@
>> 
>> volatile int x;
>> 
>> +// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]
>> void __attribute__((noinline)) sink() {
>> -  x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 'func3', 
>> 'sink'])
>> +  x++; //% self.filecheck("bt", "main.cpp")
>> }
>> 
>> +// CHECK-NEXT: frame #1: {{.*}}func3() {{.*}}[opt] [artificial]
>> void __attribute__((noinline)) func3() { sink(); /* tail */ }
>> 
>> +// CHECK-NEXT: frame #2: {{.*}}func2() at main.cpp:[[@LINE+1]] [opt]
>> void __attribute__((disable_tail_calls, noinline)) func2() { func3(); /* 
>> regular */ }
>> 
>> +// CHECK-NEXT: frame #3: {{.*}}func1() {{.*}}[opt] [artificial]
>> void __attribute__((noinline)) func1() { func2(); /* tail */ }
>> 
>> +// CHECK-NEXT: frame #4: {{.*}}main at main.cpp:[[@LINE+2]] [opt]
>> int __attribute__((disable_tail_calls)) main() {
>>   func1(); /* regular */
>>   return 0;
>> ```
>> 
>> For reference, here's the output of the "bt" command:
>> 
>> ```
>> runCmd: bt
>> output: * thread #1, queue = 'com.apple.main-thread', stop reason = 
>> breakpoint 1.1
>>  * frame #0: 0x00010c6a6f64 a.out`sink() at main.cpp:14 [opt]
>>frame #1: 0x00010c6a6f70 a.out`func3() at main.cpp:15 [opt] 
>> [artificial]
>>frame #2: 0x00010c6a6f89 a.out`func2() at main.cpp:21 [opt]
>>frame #3: 0x00010c6a6f90 a.out`func1() at main.cpp:21 [opt] 
>> [artificial]
>>frame #4: 0x00010c6a6fa9 a.out`main at main.cpp:28 [opt]
>> ```
>> 
>> thanks,
>> vedant
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

> On Aug 14, 2018, at 5:34 PM, Zachary Turner  wrote:
> 
> I’ve thought about this in the past but the conclusion I came to is that 
> lldbinline tests are actually just filecheck tests in disguise. Why do we 
> need both? I’d rather delete the lldbinline infrastructure entirely and make 
> a new lit TestFormat that basically does what lldbinline already does 

An inline test does more than simply pattern-matching input. It builds a 
program, sets breakpoints, etc. I'd rather make this existing infrastructure 
easier to use than come up with something new.

vedant


> On Tue, Aug 14, 2018 at 5:31 PM Vedant Kumar via lldb-dev 
> mailto:lldb-dev@lists.llvm.org>> wrote:
> Hello,
> 
> I'd like to make FileCheck available within lldb inline tests, in addition to 
> existing helpers like 'runCmd' and 'expect'.
> 
> My motivation is that several tests I'm working on can't be made as rigorous 
> as they need to be without FileCheck-style checks. In particular, the 
> 'matching', 'substrs', and 'patterns' arguments to runCmd/expect don't allow 
> me to verify the ordering of checked input, to be stringent about line 
> numbers, or to capture & reuse snippets of text from the input stream.
> 
> I'd curious to know if anyone else is interested or would be willing to 
> review this (https://reviews.llvm.org/D50751 
> <https://reviews.llvm.org/D50751>).
> 
> Here's an example of an inline test which benefits from FileCheck-style 
> checking. This test is trying to check that certain frames appear in a 
> backtrace when stopped inside of the "sink" function. Notice that without 
> FileCheck, it's not possible to verify the order in which frames are printed, 
> and that dealing with line numbers would be cumbersome.
> 
> ```
> --- 
> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
> +++ 
> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
> @@ -9,16 +9,21 @@
>  
>  volatile int x;
>  
> +// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]
>  void __attribute__((noinline)) sink() {
> -  x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 'func3', 
> 'sink'])
> +  x++; //% self.filecheck("bt", "main.cpp")
>  }
>  
> +// CHECK-NEXT: frame #1: {{.*}}func3() {{.*}}[opt] [artificial]
>  void __attribute__((noinline)) func3() { sink(); /* tail */ }
>  
> +// CHECK-NEXT: frame #2: {{.*}}func2() at main.cpp:[[@LINE+1]] [opt]
>  void __attribute__((disable_tail_calls, noinline)) func2() { func3(); /* 
> regular */ }
>  
> +// CHECK-NEXT: frame #3: {{.*}}func1() {{.*}}[opt] [artificial]
>  void __attribute__((noinline)) func1() { func2(); /* tail */ }
>  
> +// CHECK-NEXT: frame #4: {{.*}}main at main.cpp:[[@LINE+2]] [opt]
>  int __attribute__((disable_tail_calls)) main() {
>func1(); /* regular */
>return 0;
> ```
> 
> For reference, here's the output of the "bt" command:
> 
> ```
> runCmd: bt
> output: * thread #1, queue = 'com.apple.main-thread', stop reason = 
> breakpoint 1.1
>   * frame #0: 0x00010c6a6f64 a.out`sink() at main.cpp:14 [opt]
> frame #1: 0x00010c6a6f70 a.out`func3() at main.cpp:15 [opt] 
> [artificial]
> frame #2: 0x00010c6a6f89 a.out`func2() at main.cpp:21 [opt]
> frame #3: 0x00010c6a6f90 a.out`func1() at main.cpp:21 [opt] 
> [artificial]
> frame #4: 0x00010c6a6fa9 a.out`main at main.cpp:28 [opt]
> ```
> 
> thanks,
> vedant
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
> <http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev>

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] Using FileCheck in lldb inline tests

[Vedant Kumar via lldb-dev]

Hello,

I'd like to make FileCheck available within lldb inline tests, in addition to 
existing helpers like 'runCmd' and 'expect'.

My motivation is that several tests I'm working on can't be made as rigorous as 
they need to be without FileCheck-style checks. In particular, the 'matching', 
'substrs', and 'patterns' arguments to runCmd/expect don't allow me to verify 
the ordering of checked input, to be stringent about line numbers, or to 
capture & reuse snippets of text from the input stream.

I'd curious to know if anyone else is interested or would be willing to review 
this (https://reviews.llvm.org/D50751 ).

Here's an example of an inline test which benefits from FileCheck-style 
checking. This test is trying to check that certain frames appear in a 
backtrace when stopped inside of the "sink" function. Notice that without 
FileCheck, it's not possible to verify the order in which frames are printed, 
and that dealing with line numbers would be cumbersome.

```
--- 
a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
+++ 
b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp
@@ -9,16 +9,21 @@
 
 volatile int x;
 
+// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]
 void __attribute__((noinline)) sink() {
-  x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 'func3', 
'sink'])
+  x++; //% self.filecheck("bt", "main.cpp")
 }
 
+// CHECK-NEXT: frame #1: {{.*}}func3() {{.*}}[opt] [artificial]
 void __attribute__((noinline)) func3() { sink(); /* tail */ }
 
+// CHECK-NEXT: frame #2: {{.*}}func2() at main.cpp:[[@LINE+1]] [opt]
 void __attribute__((disable_tail_calls, noinline)) func2() { func3(); /* 
regular */ }
 
+// CHECK-NEXT: frame #3: {{.*}}func1() {{.*}}[opt] [artificial]
 void __attribute__((noinline)) func1() { func2(); /* tail */ }
 
+// CHECK-NEXT: frame #4: {{.*}}main at main.cpp:[[@LINE+2]] [opt]
 int __attribute__((disable_tail_calls)) main() {
   func1(); /* regular */
   return 0;
```

For reference, here's the output of the "bt" command:

```
runCmd: bt
output: * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 
1.1
  * frame #0: 0x00010c6a6f64 a.out`sink() at main.cpp:14 [opt]
frame #1: 0x00010c6a6f70 a.out`func3() at main.cpp:15 [opt] [artificial]
frame #2: 0x00010c6a6f89 a.out`func2() at main.cpp:21 [opt]
frame #3: 0x00010c6a6f90 a.out`func1() at main.cpp:21 [opt] [artificial]
frame #4: 0x00010c6a6fa9 a.out`main at main.cpp:28 [opt]
```

thanks,
vedant
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] run tests with ASAN (dotest.py)

[Vedant Kumar via lldb-dev]

Hi Gábor,

That symbol appears to be defined in compiler-rt/lib/asan/asan_rtl.cc 
, so it should be provided by the asan runtime. Are you 
sure that the runtime expected by the host compiler is being dynamically loaded 
here? You can check this using 'ldd' (IIRC) on Linux or 'otool -l' on Darwin. 
Also, did you take the extra step needed to preload the runtime 
(LD_PRELOAD/DYLD_INSERT_LIBRARIES)?

best,
vedant


> On Aug 2, 2018, at 12:24 PM, Gábor Márton via lldb-dev 
>  wrote:
> 
> Hi,
> 
> I'd like to run the tests, when LLVM, Clang and LLDB are all built with ASAN.
> I am using release_60 version of LLDB.
> The unittests just run fine, but with `dotest.py`, there is an error.
> Maybe the LLDB build does not respect the global -DLLVM_USE_SANITIZER
> flag of cmake ?
> 
> Any help would be appreciated,
> Thanks,
> Gabor
> 
> 
> ```
> /var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py
> --executable /var/jenkins_home/workspace/ctu_pipeline/build/bin/lldb
> /var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/
> -v --excluded ./lldb_test_exclude
> lldb version 6.0.0 (https://github.com/llvm-mirror/lldb.git revision
> b6df24ff1b258b18041161b8f32ac316a3b5d8d9)
>  clang revision 64eed461cdd3705e7bc1ccc95df9858f7fe216a8
>  llvm revision 089d4c0c490687db6c75f1d074e99c4d42936a50
> ['/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py',
> '--executable',
> '/var/jenkins_home/workspace/ctu_pipeline/build/bin/lldb',
> '/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/',
> '-v', '--excluded', './lldb_test_exclude']
> LLDB library dir: /var/jenkins_home/workspace/ctu_pipeline/build/bin
> LLDB import library dir: /var/jenkins_home/workspace/ctu_pipeline/build/bin
> The 'lldb-mi' executable cannot be located.  The lldb-mi tests can not
> be run as a result.
> Traceback (most recent call last):
>  File 
> "/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py",
> line 7, in 
>lldbsuite.test.run_suite()
>  File 
> "/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/dotest.py",
> line 1129, in run_suite
>import lldb
>  File 
> "/var/jenkins_home/workspace/ctu_pipeline/build/lib/python2.7/site-packages/lldb/__init__.py",
> line 53, in 
>_lldb = swig_import_helper()
>  File 
> "/var/jenkins_home/workspace/ctu_pipeline/build/lib/python2.7/site-packages/lldb/__init__.py",
> line 49, in swig_import_helper
>_mod = imp.load_module('_lldb', fp, pathname, description)
> ImportError: 
> /var/jenkins_home/workspace/ctu_pipeline/build/lib/libLLVMDemangle.so.6:
> undefined symbol: __asan_option_detect_stack_use_after_return
> 
> ```
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] increase timeout for tests?

[Vedant Kumar via lldb-dev]

I got the 22 number from a command which may have counted too much:
find . -name \*TestMi\*.py -exec grep -E "(unittest2\.)?expectedFailure(All)?" 
{} \;  | wc -l

Some of the 'expectedFailureAll' decorators actually specified an OS list. I'm 
not planning on touching those.

There were a handful of lldb-mi tests that didn't appear to work at all, and 
I've filed bugs / deleted those in r327552. If you see something you feel 
really should stay in tree, we can bring it back.

vedant

> On Mar 14, 2018, at 11:27 AM, Ted Woodward via lldb-dev 
> <lldb-dev@lists.llvm.org> wrote:
> 
> I don't see 22 lldb-mi tests xfailed everywhere. I see a lot of tests 
> skipped, but
> those are clearly marked as skip on Windows, FreeBSD, Darwin, Linux. I've got
> a good chunk of the lldb-mi tests running on Hexagon. I don’t want them 
> deleted,
> since I use them.
> 
> lldb-mi tests can be hard to debug, but I found that setting the lldb-mi log 
> to be
> stdout helps a lot. In lldbmi_testcase.py, in spawnLldbMi, add this line:
> 
>self.child.logfile = sys.stdout
> 
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a 
> Linux Foundation Collaborative Project
> 
>> -Original Message-----
>> From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Vedant
>> Kumar via lldb-dev
>> Sent: Tuesday, March 13, 2018 7:48 PM
>> To: Davide Italiano <dccitali...@gmail.com>
>> Cc: LLDB <lldb-dev@lists.llvm.org>
>> Subject: Re: [lldb-dev] increase timeout for tests?
>> 
>> As a first step, I think there's consensus on increasing the test timeout to 
>> ~3x
>> the length of the slowest test we know of. That test appears to be
>> TestDataFormatterObjC, which takes 388 seconds on Davide's machine. So I
>> propose 20 minutes as the timeout value.
>> 
>> Separately, regarding x-failed pexpect()-backed tests, I propose deleting 
>> them
>> if they've been x-failed for over a year. That seems like a long enough time 
>> to
>> wait for someone to step up and fix them given that they're a real
>> testing/maintenance burden. For any group of to-be-deleted tests, like the 22
>> lldb-mi tests x-failed in all configurations, I'd file a PR about potentially
>> bringing the tests back. Thoughts?
>> 
>> thanks,
>> vedant
>> 
>>> On Mar 13, 2018, at 11:52 AM, Davide Italiano <dccitali...@gmail.com>
>> wrote:
>>> 
>>> On Tue, Mar 13, 2018 at 11:26 AM, Jim Ingham <jing...@apple.com>
>> wrote:
>>>> It sounds like we timing out based on the whole test class, not the 
>>>> individual
>> tests?  If you're worried about test failures not hanging up the test suite 
>> the you
>> really want to do the latter.
>>>> 
>>>> These are all tests that contain 5 or more independent tests.  That's
>> probably why they are taking so long to run.
>>>> 
>>>> I don't object to having fairly long backstop timeouts, though I agree with
>> Pavel that we should choose something reasonable based on the slowest
>> running tests just so some single error doesn't cause test runs to just never
>> complete, making analysis harder.
>>>> 
>>> 
>>> Vedant (cc:ed) is going to take a look at this as he's babysitting the
>>> bots for the week. I'll defer the call to him.
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] increase timeout for tests?

[Vedant Kumar via lldb-dev]

As a first step, I think there's consensus on increasing the test timeout to 
~3x the length of the slowest test we know of. That test appears to be 
TestDataFormatterObjC, which takes 388 seconds on Davide's machine. So I 
propose 20 minutes as the timeout value.

Separately, regarding x-failed pexpect()-backed tests, I propose deleting them 
if they've been x-failed for over a year. That seems like a long enough time to 
wait for someone to step up and fix them given that they're a real 
testing/maintenance burden. For any group of to-be-deleted tests, like the 22 
lldb-mi tests x-failed in all configurations, I'd file a PR about potentially 
bringing the tests back. Thoughts?

thanks,
vedant

> On Mar 13, 2018, at 11:52 AM, Davide Italiano  wrote:
> 
> On Tue, Mar 13, 2018 at 11:26 AM, Jim Ingham  wrote:
>> It sounds like we timing out based on the whole test class, not the 
>> individual tests?  If you're worried about test failures not hanging up the 
>> test suite the you really want to do the latter.
>> 
>> These are all tests that contain 5 or more independent tests.  That's 
>> probably why they are taking so long to run.
>> 
>> I don't object to having fairly long backstop timeouts, though I agree with 
>> Pavel that we should choose something reasonable based on the slowest 
>> running tests just so some single error doesn't cause test runs to just 
>> never complete, making analysis harder.
>> 
> 
> Vedant (cc:ed) is going to take a look at this as he's babysitting the
> bots for the week. I'll defer the call to him.

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] tests xfailed globally with radar references

[Vedant Kumar via lldb-dev]

I took a first cut at this in r327463.

vedant

> On Mar 13, 2018, at 2:39 PM, Vedant Kumar  wrote:
> 
> Hi Ted,
> 
> Thanks for bringing this up. I'll volunteer to take a look at these tests and 
> file PRs where appropriate. I should get to this within a day or two.
> 
> vedant
> 
>> On Mar 13, 2018, at 10:31 AM, Ted Woodward via lldb-dev 
>>  wrote:
>> 
>> I was investigating TestWatchedVarHitWhenInScope.py, which tests that a
>> watchpoint is hit in scope and not hit when out of scope. It's xfailed due
>> to a radar:
>>   @unittest2.expectedFailure("rdar://problem/18685649")
>> 
>> This is an actual bug - watchpoints are hit when the address is modified but
>> the variable is out of scope. Which can happen a lot if the variable being
>> watched is a local on the stack. I manually ran the testcase on Linux, and
>> it failed when it hit the watchpoint out of scope:
>> Process 18652 stopped
>> * thread #1, name = 'a.out', stop reason = watchpoint 1
>>   frame #0: 0x7722d0d2 libc.so.6`__run_exit_handlers(status=0,
>> listp=0x775b36c8, run_list_atexit=true) at exit.c:35
>> 
>> A google search found this message, posted by Ed Maste to llvm-dev:
>> http://lists.llvm.org/pipermail/llvm-dev/2017-August/116471.html .
>> 
>> I echo Ed's request - could someone at Apple look at these radars, and open
>> llvm bugs for the ones that aren't Darwin-only?
>> 
>> 
>> Ed's message:
>> 
>> I was recently looking at an unexpected pass test result on FreeBSD[1]
>> which is decorated with expectedFailureAll referencing
>> rdar://problem/24599697. I'm looking for more details on this specific
>> radar bug report in order to determine if the test is not exercising
>> the original bug on FreeBSD, or if it's really an Apple-only issue.
>> 
>> As it's rather unfortunate to have a bug report reference for which no
>> further public details are available I looked at other radar
>> references in the test suite, and found 118 of them. Many of these are
>> in comments, in macosx-specific cases (e.g.
>> @expectedFailureAll(oslist=["macosx"], bugnumber="rdar://28805064")),
>> or in tests decorated with skipUnlessDarwin, and I've ignored those.
>> 
>> There are 10 tests with expected failure decorators, and one skipped,
>> that apply beyond macosx and reference a radar (and do not
>> additionally reference an LLVM PR). Can someone at Apple check these
>> radar references and submit LLVM PRs, reference existing PRs, add a
>> reasonable description in the test source, etc. as appropriate?
>> 
>> functionalities/watchpoint/variable_out_of_scope/TestWatchedVarHitWhenInScop
>> e.py
>> unittest2.expectedFailure("rdar://problem/18685649")
>> 
>> functionalities/asan/TestReportData.py
>> @expectedFailureAll(archs=['i386'], bugnumber="rdar://28658860")
>> 
>> api/multiple-debuggers/TestMultipleDebuggers.py
>> @expectedFailureAll(bugnumber="rdar://30564102")
>> 
>> lang/objc/bitfield_ivars/TestBitfieldIvars.py
>> decorators.expectedFailureAll(bugnumber="rdar://problem/17990991")])
>> 
>> lang/c/shared_lib/TestSharedLib.py
>> @unittest2.expectedFailure("rdar://problem/10704639")
>> 
>> lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
>> @unittest2.expectedFailure("rdar://problem/10381325")
>> 
>> lang/cpp/stl/TestSTL.py
>> @expectedFailureAll(bugnumber="rdar://problem/10400981")
>> 
>> lang/cpp/dynamic-value/TestCppValueCast.py
>> @unittest2.expectedFailure("rdar://problem/10808472 SBValue::Cast test
>> case is failing (virtual inheritance)")
>> 
>> lang/cpp/printf/TestPrintf.py
>> decorators.expectedFailureAll(bugnumber="rdar://problem/24599697")])
>> 
>> lang/cpp/function-template-parameter-pack/TestFunctionTemplateParameterPack.
>> py
>> decorators.expectedFailureAll(bugnumber="rdar://problem/32096064")])
>> 
>> lang/cpp/unique-types/TestUniqueTypes.py
>> self.skipTest("rdar://problem/9173060 lldb hangs while running
>> unique-types for clang version < 3")
>> 
>> [1]
>> http://lists.llvm.org/pipermail/lldb-commits/Week-of-Mon-20170807/036634.htm
>> l
>> 
>> --
>> Qualcomm Innovation Center, Inc.
>> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
>> Linux Foundation Collaborative Project
>> 
>> 
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] tests xfailed globally with radar references

[Vedant Kumar via lldb-dev]

Hi Ted,

Thanks for bringing this up. I'll volunteer to take a look at these tests and 
file PRs where appropriate. I should get to this within a day or two.

vedant
 
> On Mar 13, 2018, at 10:31 AM, Ted Woodward via lldb-dev 
>  wrote:
> 
> I was investigating TestWatchedVarHitWhenInScope.py, which tests that a
> watchpoint is hit in scope and not hit when out of scope. It's xfailed due
> to a radar:
>@unittest2.expectedFailure("rdar://problem/18685649")
> 
> This is an actual bug - watchpoints are hit when the address is modified but
> the variable is out of scope. Which can happen a lot if the variable being
> watched is a local on the stack. I manually ran the testcase on Linux, and
> it failed when it hit the watchpoint out of scope:
> Process 18652 stopped
> * thread #1, name = 'a.out', stop reason = watchpoint 1
>frame #0: 0x7722d0d2 libc.so.6`__run_exit_handlers(status=0,
> listp=0x775b36c8, run_list_atexit=true) at exit.c:35
> 
> A google search found this message, posted by Ed Maste to llvm-dev:
> http://lists.llvm.org/pipermail/llvm-dev/2017-August/116471.html .
> 
> I echo Ed's request - could someone at Apple look at these radars, and open
> llvm bugs for the ones that aren't Darwin-only?
> 
> 
> Ed's message:
> 
> I was recently looking at an unexpected pass test result on FreeBSD[1]
> which is decorated with expectedFailureAll referencing
> rdar://problem/24599697. I'm looking for more details on this specific
> radar bug report in order to determine if the test is not exercising
> the original bug on FreeBSD, or if it's really an Apple-only issue.
> 
> As it's rather unfortunate to have a bug report reference for which no
> further public details are available I looked at other radar
> references in the test suite, and found 118 of them. Many of these are
> in comments, in macosx-specific cases (e.g.
> @expectedFailureAll(oslist=["macosx"], bugnumber="rdar://28805064")),
> or in tests decorated with skipUnlessDarwin, and I've ignored those.
> 
> There are 10 tests with expected failure decorators, and one skipped,
> that apply beyond macosx and reference a radar (and do not
> additionally reference an LLVM PR). Can someone at Apple check these
> radar references and submit LLVM PRs, reference existing PRs, add a
> reasonable description in the test source, etc. as appropriate?
> 
> functionalities/watchpoint/variable_out_of_scope/TestWatchedVarHitWhenInScop
> e.py
> unittest2.expectedFailure("rdar://problem/18685649")
> 
> functionalities/asan/TestReportData.py
> @expectedFailureAll(archs=['i386'], bugnumber="rdar://28658860")
> 
> api/multiple-debuggers/TestMultipleDebuggers.py
> @expectedFailureAll(bugnumber="rdar://30564102")
> 
> lang/objc/bitfield_ivars/TestBitfieldIvars.py
> decorators.expectedFailureAll(bugnumber="rdar://problem/17990991")])
> 
> lang/c/shared_lib/TestSharedLib.py
> @unittest2.expectedFailure("rdar://problem/10704639")
> 
> lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
> @unittest2.expectedFailure("rdar://problem/10381325")
> 
> lang/cpp/stl/TestSTL.py
> @expectedFailureAll(bugnumber="rdar://problem/10400981")
> 
> lang/cpp/dynamic-value/TestCppValueCast.py
> @unittest2.expectedFailure("rdar://problem/10808472 SBValue::Cast test
> case is failing (virtual inheritance)")
> 
> lang/cpp/printf/TestPrintf.py
> decorators.expectedFailureAll(bugnumber="rdar://problem/24599697")])
> 
> lang/cpp/function-template-parameter-pack/TestFunctionTemplateParameterPack.
> py
> decorators.expectedFailureAll(bugnumber="rdar://problem/32096064")])
> 
> lang/cpp/unique-types/TestUniqueTypes.py
> self.skipTest("rdar://problem/9173060 lldb hangs while running
> unique-types for clang version < 3")
> 
> [1]
> http://lists.llvm.org/pipermail/lldb-commits/Week-of-Mon-20170807/036634.htm
> l
> 
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
> 
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Current state of the unit tests

[Vedant Kumar via lldb-dev]

> On Feb 23, 2018, at 3:17 PM, Vedant Kumar via lldb-dev 
> <lldb-dev@lists.llvm.org> wrote:
> 
> Hi,
> 
> At the moment, I'm seeing two issues with the unit tests on my machine.
> 
> First, TestBase.LaunchModePreservesEnvironment is failing:
> 
>> [ RUN  ] TestBase.LaunchModePreservesEnvironment
>> /Users/vsk/src/llvm.org-lldbsan/llvm/tools/lldb/unittests/tools/lldb-server/tests/LLGSTest.cpp:30:
>>  Failure
>> Value of: 
>> llvm::detail::TakeExpected(Client.GetLatestStopReplyAs())
>> Expected: succeeded with value (is an object whose given property is equal 
>> to 2-byte object <00-00>)
>>  Actual: succeeded with value 16-byte object <10-60 A7-04 01-00 00-00 01-00 
>> 00-00 00-00 00-30>, whose given property is 2-byte object <00-01>(is an 
>> object whose given property isn't equal to 2-byte object <00-00>)
>> [  FAILED  ] TestBase.LaunchModePreservesEnvironment (67 ms)
> 
> I filed https://bugs.llvm.org/show_bug.cgi?id=36494 to track the issue.

^ This issue is resolved now with a cmake fix.


> Second, TestClient::SendMessage is generating quite a lot of "INFO" output 
> which clutters up the terminal. Pavel, would you mind if I removed this 
> logging?

^ I'll let Pavel decide the best way to deal with this one.

vedant

> 
> thanks,
> vedant
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Current state of the unit tests

[Vedant Kumar via lldb-dev]

check-lldb-lit should just be a dependency of check-lldb, so the dotest.py 
tests should still run.

Are one of the lit tests failing? That might explain why subsequent tests 
aren't run.

vedant

> On Feb 23, 2018, at 4:13 PM, Adrian McCarthy <amcca...@google.com> wrote:
> 
> As of this afternoon, it seems ninja check-lldb runs only the lit tests and 
> not the dotest.py tests.  Was this an intentional change?
> 
> On Fri, Feb 23, 2018 at 3:36 PM, Vedant Kumar via lldb-dev 
> <lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org>> wrote:
> Cool, I'll work up a patch for this.
> 
> And thanks for commenting on PR36494, I'm testing a fix out right now :).
> 
> vedant
> 
>> On Feb 23, 2018, at 3:35 PM, Pavel Labath <lab...@google.com 
>> <mailto:lab...@google.com>> wrote:
>> 
>> On 23 February 2018 at 15:17, Vedant Kumar <v...@apple.com 
>> <mailto:v...@apple.com>> wrote:
>>> Second, TestClient::SendMessage is generating quite a lot of "INFO" output 
>>> which clutters up the terminal. Pavel, would you mind if I removed this 
>>> logging?
>>> 
>> 
>> Yeah, we should probably do that. The idea here was that the packet
>> log would provide you with additional context for the situation when
>> the test fails, but it *is* very verbose. I'll have to come up with a
>> better solution for error reporting here.
> 
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
> <http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev>
> 
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Current state of the unit tests

[Vedant Kumar via lldb-dev]

Cool, I'll work up a patch for this.

And thanks for commenting on PR36494, I'm testing a fix out right now :).

vedant

> On Feb 23, 2018, at 3:35 PM, Pavel Labath  wrote:
> 
> On 23 February 2018 at 15:17, Vedant Kumar  wrote:
>> Second, TestClient::SendMessage is generating quite a lot of "INFO" output 
>> which clutters up the terminal. Pavel, would you mind if I removed this 
>> logging?
>> 
> 
> Yeah, we should probably do that. The idea here was that the packet
> log would provide you with additional context for the situation when
> the test fails, but it *is* very verbose. I'll have to come up with a
> better solution for error reporting here.

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] Current state of the unit tests

[Vedant Kumar via lldb-dev]

Hi,

At the moment, I'm seeing two issues with the unit tests on my machine.

First, TestBase.LaunchModePreservesEnvironment is failing:

> [ RUN  ] TestBase.LaunchModePreservesEnvironment
> /Users/vsk/src/llvm.org-lldbsan/llvm/tools/lldb/unittests/tools/lldb-server/tests/LLGSTest.cpp:30:
>  Failure
> Value of: 
> llvm::detail::TakeExpected(Client.GetLatestStopReplyAs())
> Expected: succeeded with value (is an object whose given property is equal to 
> 2-byte object <00-00>)
>   Actual: succeeded with value 16-byte object <10-60 A7-04 01-00 00-00 01-00 
> 00-00 00-00 00-30>, whose given property is 2-byte object <00-01>(is an 
> object whose given property isn't equal to 2-byte object <00-00>)
> [  FAILED  ] TestBase.LaunchModePreservesEnvironment (67 ms)

I filed https://bugs.llvm.org/show_bug.cgi?id=36494 to track the issue.

Second, TestClient::SendMessage is generating quite a lot of "INFO" output 
which clutters up the terminal. Pavel, would you mind if I removed this logging?

thanks,
vedant
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Vedant Kumar via lldb-dev]

Lit provides some helpful utilities which make it easier to write portable 
tests. E.g %t, for temporary test directories, and portable replacements for 
things like `diff -r`. This is how compiler-rt's end-to-end tests are 
structured, and we haven't needed any build-system like functionality there.

vedant

> On Jan 17, 2018, at 2:56 PM, Jim Ingham  wrote:
> 
> 
>> On Jan 17, 2018, at 2:55 PM, Adrian Prantl  wrote:
>> 
>> 
>> 
>>> On Jan 17, 2018, at 2:50 PM, Jim Ingham  wrote:
>>> 
>>> 
>>> 
 On Jan 17, 2018, at 2:49 PM, Adrian Prantl  wrote:
 
 
 
> On Jan 17, 2018, at 2:31 PM, Zachary Turner  wrote:
> 
> I don't think new test authors really need to add CMake any more so than 
> they currently need to understand Make.  Which is to say, not very much.  
> Most Makefiles are currently 1-2 lines of code that simply does nothing 
> other than include the common Makefile.
> 
> On the other hand, CMake defines a lot of constructs designed to support 
> portable builds, so actually writing and maintaining that common CMake 
> build file would be much easier.  The existing Makefile-based system 
> already doesn't require you to understand the specific compiler 
> invocations you want.  Here's 3 random Makefiles, which is hopefully 
> representative given that I pulled them completely at random.
> 
> breakpoint-commands/Makefile:
> LEVEL = ../../../make
> CXX_SOURCES := nested.cpp
> include $(LEVEL)/Makefile.rules
> 
> functionalities/inferior-assert:
> LEVEL = ../../make
> C_SOURCES := main.c
> include $(LEVEL)/Makefile.rules
> 
> 
> types:
> LEVEL = ../make
> # Example:
> #
> # CXX_SOURCES := int.cpp
> include $(LEVEL)/Makefile.rules
> 
> None of this is particularly interesting.  There are a very few tests 
> that need to do something weird.  I opened 10 other random Makefiles and 
> still didn't find any.  I don't believe it would be hard to support those 
> cases.
> 
> So now instead of "understand Make" it becomes "understand CMake".  Whic 
> is already a requirement of building LLVM.
 
 Fair point. I would suggest that I'll try to make LLDB's testsuite build 
 out-of-tree using the existing Makefile system. That should be a generally 
 useful first step. After doing this I will hopefully have a much better 
 understanding of the requirements of the Makefiles and then we can revisit 
 this idea with me actually knowing what I'm talking about :-)
 
> If our test suite was lit-based where you actually had to write compiler 
> invocations into the test files, I would feel differently, but that isn't 
> what we have today.  We have something that is almost a direct mapping to 
> using CMake.
 
 Question: how would you feel about converting the Makefiles to LIT-style 
 .test files with very explicit RUN-lines?
>>> 
>>> I'm not sure what you mean by this.
>> 
>> Instead of using a build system at all to build the tests, we would 
>> hard-code the compiler and linker invocations without encoding any 
>> dependencies. Because we still need this to be configurable, it would 
>> probably look something like this:
>> 
>> RUN: %CXX test.cpp -O0 %CXXFLAGS -o test.exe
>> RUN: %test_driver test.exe mytest.py
> 
> I'm worried we'd back into building another make system over time.  What 
> advantage would we get from this.
> 
> Jim
> 
> 
> 
>> 
>> -- adrian
>> 
>>> 
>>> Jim
>>> 
>>> 
 
 -- adrian
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Vedant Kumar via lldb-dev]

I would prefer having all of our test dependencies tracked by CMake for all the 
reasons Zach brought up, but I think we should defer that undertaking until 
after the bots are more stable. We have some immediate problems caused by stale 
in-tree test artifacts. As a first milestone, it'd be great to not have to run 
`git clean -fdx` anymore.


> On Jan 17, 2018, at 1:13 PM, Davide Italiano via lldb-dev 
>  wrote:
> 
> On Wed, Jan 17, 2018 at 1:02 PM, Davide Italiano  
> wrote:
>> On Wed, Jan 17, 2018 at 12:32 PM, Adrian Prantl via lldb-dev
>>  wrote:
>>> Hi lldb-dev!
>>> 
>>> I've been investigating some spurious LLDB test suite failures on 
>>> http://green.lab.llvm.org/green/ that had to do with build artifacts from 
>>> previous runs lying around in the test directories and this prompted me to 
>>> ask a couple of general noob questions about the LLDB testsuite.
>>> 
>>> My understanding is that all execution tests are compiled using using 
>>> `make` in-tree. I.e.: the test driver (dotest.py) effectively executes 
>>> something equivalent to `cd $srcdir/packages/.../mytest && make`. And it 
>>> does this in a serial fashion for all configurations (dwarf, dSYM, dwo, 
>>> ...) and relies on the `clean` target to be implemented correctly.
>>> 
>>> I don't understand all the design decisions that went into the LLDB 
>>> testsuite, but my naive intuition tells me that this is sub-optimal 
>>> (because of the serialization of the configurations) and dangerous (because 
>>> it relies on make clean being implemented correctly). It seems to me that a 
>>> better approach would be to create a separate build directory for each test 
>>> variant and then invoke something like `cd $builddir/test/mytest.dwarf && 
>>> make -C $srcdir/packages/.../mytest`. This way all configurations can build 
>>> in parallel, and we can simply nuke the build directory afterwards and this 
>>> way get rid of all custom implementations of the `clean` target.

This sgtm as a starting point.

vedant

>>> 
>>> - Is this already possible, and/or am I misunderstanding how it works?
>>> - Would this be a goal that is worthwhile to pursue?
>>> - Is there a good reason why we are not already doing it this way?
>>> 
>> 
>> As we're discussing lldb test suite changes, another detail that I
>> find a little weird is that every time you execute the test suite you
>> get a new build directory named after the time at which you run the
>> test.
>> It would be much much better IMHO to just have a `log/` generic
>> directory where the failures are logged, and those who want to
>> override this setting can just pass a flag.
>> 
> 
> (The logs should also be moved out of tree, FWIW).
> 
> --
> Davide
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] [5.0.0 Release] Release Candidate 2 tagged

[Vedant Kumar via lldb-dev]

Hi,

There are no regressions from RC1 on Darwin.

I've uploaded clang+llvm-5.0.0-rc2-x86_64-apple-darwin.tar.xz to 
/data/testers-uploads.

SHA 1: 8ffaf7ab44e7e7fe582dc9573aad9745ee63e186
SHA 256: 578c18ceb6485fd2fcac329a7a5fe7e55d1c70669a942651d49cdc3ea626a5af

I've also resurrected the Darwin bot which tests the release branch (thanks to 
Mehdi for creating the original!):

  
http://lab.llvm.org:8080/green/view/All/job/clang-stage1-configure-RA-release-5/

thanks,
vedant

> On Aug 10, 2017, at 7:00 PM, Hans Wennborg via llvm-dev 
>  wrote:
> 
> Dear testers,
> 
> 5.0.0-rc2 was just tagged.
> 
> I know we still have a bunch of open release blockers, but there has
> been a lot of merged patches and I'd like to find out what the status
> is.
> 
> Please build, test and upload binaries to the sftp. When uploading,
> make sure to use the /data/testers-uploads/ directory on the server.
> Let me know if there are any issues.
> 
> I'll upload sources, docs and your binaries to the pre-release website
> once they're ready.
> 
> Cheers,
> Hans
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [Release-testers] [cfe-dev] [5.0.0 Release] Release Candidate 1 tagged

[Vedant Kumar via lldb-dev]

I've uploaded the x86_64/Darwin package:

SHA1(clang+llvm-5.0.0-rc1-x86_64-apple-darwin.tar.xz) = 
66cb113cb735b7f0d3b7eebe155969f069c8a684
SHA256(clang+llvm-5.0.0-rc1-x86_64-apple-darwin.tar.xz) = 
ca13407528536f0c4e9b5bf00115a2f3d65d73f512737d334f69ad53bd18101c

thanks!
vedant

> On Aug 3, 2017, at 4:51 PM, Hans Wennborg via Release-testers 
>  wrote:
> 
> On Tue, Aug 1, 2017 at 9:13 AM, Hans Wennborg  wrote:
>> On Tue, Aug 1, 2017 at 3:01 AM, Simon Dardis  wrote:
>>> Currently I'm getting "Couldn't write to remote file 
>>> "/home/testers/uploads/clang+llvm-5.0.0-rc1-mips-linux-gnu.tar.xz": 
>>> Failure",
>>> does that account need more space?
>> 
>> Yes, the partition is full. I've asked Anton to see if we can fix this.
> 
> Tanya created a new directory on the server: /data/testers-uploads/
> Can you try uploading to that instead?
> ___
> Release-testers mailing list
> release-test...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/release-testers

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Code Coverage with GCOV

[Vedant Kumar via lldb-dev]

Is it possible that the files are written to an unexpected directory? Have you
tried running the program under strace/dtruss to conclusively determine whether
or not the files are being written? Could you try running lldb-server in lldb
and setting a breakpoint on `llvm_gcda_start_file'?

vedant

> On Aug 17, 2016, at 3:50 AM, Ravitheja Addepally  
> wrote:
> 
> Hello,
>I added the following lines to the lldb-server and lldb CMakeLists.txt
>set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fprofile-arcs -ftest-coverage")
>set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fprofile-arcs 
> -ftest-coverage")
> 
> I am able to get the lldb-server gcov files when I start its instance and 
> then connect to it. But since lldb starts the lldb-server in different way, 
> the files are not written, because I do see that the gcov instrumentation 
> functions are present in the lldb-server.
> 
> Best Regards,
> A Ravi Theja
> 
> On Tue, Aug 16, 2016 at 7:27 PM, Vedant Kumar  wrote:
> Hi Ravitheja,
> 
> Could you show us the diff to your cmake configuration? You may want to take a
> look at how the LLVM_BUILD_INSTRUMENTED cmake option is implemented to
> double-check that you've got something reasonable (see: llvm/CMakeLists.txt 
> and
> cmake/modules/HandleLLVMOptions.cmake).
> 
> Also, you may want to double-check that the gcda files are being written to 
> the
> right directory.
> 
> best,
> vedant
> 
> 
> > On Aug 16, 2016, at 7:58 AM, Ravitheja Addepally via lldb-dev 
> >  wrote:
> >
> > Hello,
> >I was trying to get Code Coverage for LLDB with GCOV on Linux by 
> > adding the corresponding flags in the CMake files, but I am facing problem 
> > with lldb-server, which does not give seem to generate the gcda files. The 
> > host side does generate the gcov files although. The lldb-server does 
> > generate the gcov files if it is started in Standalone mode but it does not 
> > generate the gcov files if started by LLDB. Has anyone tried this already ? 
> > How can I get Code Coverage for lldb / lldb-server on Linux ?
> >
> >
> > Best Regards,
> > A Ravi Theja
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] LLDB Evolution

[Vedant Kumar via lldb-dev]

FWIW, as a happy lldb user, it's exciting to read about these planned
developments.

I have two follow-up questions about the section on 'Testing Strategy
Evaluation': (1) what is lldb's policy on including test updates with bug fix
commits and functional changes, and (2) how is this policy enforced?

AFAICT, it seems that lldb is not as strict about its test policy as other llvm
sub-projects. That could be to its detriment. Here are some very rough numbers
on the number of commits which include test updates [1]:

  - lldb: 287 of the past 1000 commits
  - llvm: 511 of the past 1000 commits
  - clang: 622 of the past 1000 commits
  - compiler-rt: 543 of the past 1000 commits

NFC commits make these numbers a bit noisy. But, unless lldb has a much higher
ratio of NFC commits to functional changes as compared to other llvm
sub-projects, this is a concerning statistic.

best,
vedant

[1] Based on ToT = r278069.

HAS_TEST=0
TOTAL=1000
for HASH in $(git log --oneline -n$TOTAL | cut -d' ' -f1); do
  git show --stat $HASH | grep "|" | grep -q test && HAS_TEST=$((HAS_TEST+1))
done
echo $HAS_TEST "/" $TOTAL

> On Aug 8, 2016, at 2:57 PM, Zachary Turner via lldb-dev 
>  wrote:
> 
> 
> 
> On Mon, Aug 8, 2016 at 2:40 PM Kate Stone via lldb-dev 
>  wrote:
> LLDB has come a long way since the project was first announced.  As a robust 
> debugger for C-family languages and Swift, LLDB is constantly in use by 
> millions of developers.  It has also become a foundation for bringing up 
> debugger support for other languages like Go and RenderScript.  In addition 
> to the original macOS implementation the Linux LLDB port is in active use and 
> Windows support has made significant strides.  IDE and editor integration via 
> both SB APIs and MI have made LLDB available to even more users.  It’s 
> definitely a project every contributor can be proud of and I’d like to take a 
> moment to thank everyone who has been involved in one way or another.
> 
> It’s also a project that shows some signs of strain due to its rapid growth.  
> We’ve accumulated some technical debt that must be paid off, and in general 
> it seems like a good time to reflect on where we'll be headed next.  We’ve 
> outlined a few goals for discussion below as well as one more short-term 
> action.  Discussion is very much encouraged.
> 
> Forward-Looking Goals
> 
>1. Testing Strategy Evaluation
> 
> Keeping our code base healthy is next to impossible without a robust testing 
> strategy.  Our existing suite of tests is straightforward to run locally, and 
> serves as a foundation for continuous integration.  That said, it is 
> definitely not exhaustive.  Obvious priorities for improvement include 
> gathering coverage information, investing in more conventional unit tests in 
> addition to the suite of end-to-end tests, and introducing tests in code 
> bases where we depend on debugger-specific behavior (e.g.: for expression 
> evaluation.)
> I know this is going to be controversial, but I think we should at least do a 
> serious evaluation of whether using the lit infrastructure would work for 
> LLDB.  Conventional wisdom is that it won't work because LLDB tests are 
> fundamentally different than LLVM tests.  I actually completely agree with 
> the latter part.  They are fundamentally different.
> 
> However, we've seen some effort to move towards lldb inline tests, and in a 
> sense that's conceptually exactly what lit tests are.  My concern is that 
> nobody with experience working on LLDB has a sufficient understanding of what 
> lit is capable of to really figure this out.
> 
> I know when I mentioned this some months ago Jonathan Roelofs chimed in and 
> said that he believes lit is extensible enough to support LLDB's use case.  
> The argument -- if I remember it correctly -- is that the traditional view of 
> what a lit test (i.e. a sequence of commands that checks the output of a 
> program against expected output) is one particular implementation of a 
> lit-style test.  But that you can make your own which do whatever you want.
> 
> This would not just be busy work either.  I think everyone involved with LLDB 
> has experienced flakiness in the test suite.  Sometimes it's flakiness in 
> LLDB itself, but sometimes it is flakiness in the test infrastructure.  It 
> would be nice to completely eliminate one source of flakiness.
> 
> I think it would be worth having some LLDB experts sit down in person with 
> some lit experts and brainstorm ways to make LLDB use lit.  
> 
> Certainly it's worth a serious look, even if nothing comes of it.
> 
>  
>4. Good Citizenship in the LLVM Community
> 
> Last, but definitely not least, LLDB should endeavor to be a good citizen of 
> the LLVM community.  We should encourage developers to think of the 
> technology stack as a coherent effort, where common code should be introduced 
> at an appropriate level in the stack.  Opportunities to factor