Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-11-04 Thread David Blaikie via lldb-dev 
I haven't made any further progress on it - I think the actual git diff I
posted, changing config.llvm_libs_dir wouldn't quite be shippable as-is,
because it's only correct to add the "/@LLVM_DEFAULT_TARGET_TRIPLE@" if the
runtimes were built with LLVM_ENABLE_PER_TARGET_RUNTIME_DIR ON (which is
the default on Linux, but not on MacOS) - so some extra conditionality is
needed, but I'm not sure of the best/right place to implement that.

On Thu, Nov 4, 2021 at 8:15 AM Raphael Isemann  wrote:

> Is someone currently working on fixing this? FWIW, I think David's
> change seems to go in the right direction (when I originally looked at
> this I also ended up on the wrong rpath but I thought it was some
> other code that set the wrong value. Didn't realize we have two places
> where this happens). I think David's diff is better than we currently
> have so maybe we should just turn this into a review?
>
> Am Di., 26. Okt. 2021 um 06:43 Uhr schrieb David Blaikie <
> dblai...@gmail.com>:
> >
> > On Mon, Oct 25, 2021 at 1:28 PM Louis Dionne  wrote:
> >>
> >> I believe the issue is probably not related so much to
> LLVM_ENABLE_PROJECTS vs LLVM_ENABLE_RUNTIMES, but rather to the fact that
> LLVM_ENABLE_RUNTIMES uses per-target runtime directories now (hasn't always
> been the case), which basically means that libc++ ends up in
> `/lib//libc++.so` instead of
> `/lib/libc++.so`.
> >
> >
> > Ish, yes. It's a bug in LLVM_ENABLE_RUNTIMES that isn't present in
> LLVM_ENABLE_PROJECTS, so for now if I want to run the lldb pretty printer
> tests for libc++ on Linux it seems the only way I can is by using the
> deprecated functionality of libc++ in LLVM_ENABLE_PROJECTS.
> >
> > Consider this a bug report (looks like a bug in the lldb CMake
> configuration, not in libc++'s build itself, but something to figure out if
> Linux lldb devs are going to use libc++ +.ENABLE_RUNTIMES path) on that
> deprecation?
> >
> >>
> >> I think you either want to specify the per-target library dir when
> running against libc++, or you want to disable that and use
> `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF` when configuring the runtimes. In
> all cases, you want to be using `LLVM_ENABLE_RUNTIMES` and not
> `LLVM_ENABLE_PROJECTS`, since the latter is deprecated now.
> >
> >
> > I didn't enable LLVM_ENABLE_PER_TARGET_RUNTIME_DIR myself/in the root
> cmake config. It looks like it's hardcoded(?) into the ENABLE_RUNTIMES
> sub-build?
> https://github.com/llvm/llvm-project/blob/e5fb79b31424267704e9d2d9674089fd7316453e/llvm/runtimes/CMakeLists.txt#L76
> I'm not sure there's any way to override that from the root? And in any
> case I'd have thought the defaults would need to/be intended to work
> correctly on supported platforms?
> >
> > So something in lldb's dir handling (maybe some general infrastructure
> in LLVM could use some improvement to provide an LLVM_RUNTIME_LIBS_DIR, or
> similar? that could then be used from other places - rather than libc++,
> for instance, creating that directory for itself based on LLVM_LIBS_DIR and
> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR, etc) needs some fixes to support the
> current defaults/hardcoded modes on Linux?
> >
> >>
> >> Cheers,
> >> Louis
> >>
> >> On Oct 25, 2021, at 13:57, David Blaikie  wrote:
> >>
> >> +Louis Dionne - perhaps the libcxx and lldb folks would be interesting
> in finding a suitable way to address this issue, since currently either
> option (using libcxx in ENABLE_PROJECTS or using it in ENABLE_RUNTIMES) is
> incomplete - if I use ENABLE_RUNTIMES I get the libcxx testing run against
> the just-built clang and generally this is the "supported" configuration,
> but then some lldb tests fail because they can't find libcxx.so.1 (on
> Linux) - and using ENABLE_PROJECTS means not using the just-built clang for
> libcxx tests (so missing the libcxx breakages caused by my array name
> change) but do use the just-built libcxx in lldb tests and find failures
> there...
> >>
> >> On Wed, Oct 20, 2021 at 1:57 PM David Blaikie 
> wrote:
> >>>
> >>> On Tue, Oct 19, 2021 at 4:55 PM David Blaikie 
> wrote:
> 
>  On Tue, Oct 19, 2021 at 9:08 AM Raphael Isemann 
> wrote:
> >
> > Actually the RPATH theory is wrong, but the LLVM_ENABLE_PROJECT
> > workaround *should* still work.
> 
> 
>  I'll give that a go (it's running at the moment) though I guess this
> is inconsistent with the direction libcxx is moving in for building, re:
> https://groups.google.com/g/llvm-dev/c/tpuLxk_ipLw
> >>>
> >>>
> >>> Yep, it does work with LLVM_ENABLE_PROJECT rather than
> LLVM_ENABLE_RUNTIME.
> >>>
> >>> Specifically the test binary is linked with an rpath to the just-built
> lib directory that ensures the just-built libc++.so is found:
> >>>
> >>> /usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang main.o
> -g -O0 -fno-builtin -m64
> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/../../../../../include
>

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-11-04 Thread Raphael Isemann via lldb-dev 
Is someone currently working on fixing this? FWIW, I think David's
change seems to go in the right direction (when I originally looked at
this I also ended up on the wrong rpath but I thought it was some
other code that set the wrong value. Didn't realize we have two places
where this happens). I think David's diff is better than we currently
have so maybe we should just turn this into a review?

Am Di., 26. Okt. 2021 um 06:43 Uhr schrieb David Blaikie :
>
> On Mon, Oct 25, 2021 at 1:28 PM Louis Dionne  wrote:
>>
>> I believe the issue is probably not related so much to LLVM_ENABLE_PROJECTS 
>> vs LLVM_ENABLE_RUNTIMES, but rather to the fact that LLVM_ENABLE_RUNTIMES 
>> uses per-target runtime directories now (hasn't always been the case), which 
>> basically means that libc++ ends up in 
>> `/lib//libc++.so` instead of `/lib/libc++.so`.
>
>
> Ish, yes. It's a bug in LLVM_ENABLE_RUNTIMES that isn't present in 
> LLVM_ENABLE_PROJECTS, so for now if I want to run the lldb pretty printer 
> tests for libc++ on Linux it seems the only way I can is by using the 
> deprecated functionality of libc++ in LLVM_ENABLE_PROJECTS.
>
> Consider this a bug report (looks like a bug in the lldb CMake configuration, 
> not in libc++'s build itself, but something to figure out if Linux lldb devs 
> are going to use libc++ +.ENABLE_RUNTIMES path) on that deprecation?
>
>>
>> I think you either want to specify the per-target library dir when running 
>> against libc++, or you want to disable that and use 
>> `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF` when configuring the runtimes. In 
>> all cases, you want to be using `LLVM_ENABLE_RUNTIMES` and not 
>> `LLVM_ENABLE_PROJECTS`, since the latter is deprecated now.
>
>
> I didn't enable LLVM_ENABLE_PER_TARGET_RUNTIME_DIR myself/in the root cmake 
> config. It looks like it's hardcoded(?) into the ENABLE_RUNTIMES sub-build? 
> https://github.com/llvm/llvm-project/blob/e5fb79b31424267704e9d2d9674089fd7316453e/llvm/runtimes/CMakeLists.txt#L76
>  I'm not sure there's any way to override that from the root? And in any case 
> I'd have thought the defaults would need to/be intended to work correctly on 
> supported platforms?
>
> So something in lldb's dir handling (maybe some general infrastructure in 
> LLVM could use some improvement to provide an LLVM_RUNTIME_LIBS_DIR, or 
> similar? that could then be used from other places - rather than libc++, for 
> instance, creating that directory for itself based on LLVM_LIBS_DIR and 
> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR, etc) needs some fixes to support the 
> current defaults/hardcoded modes on Linux?
>
>>
>> Cheers,
>> Louis
>>
>> On Oct 25, 2021, at 13:57, David Blaikie  wrote:
>>
>> +Louis Dionne - perhaps the libcxx and lldb folks would be interesting in 
>> finding a suitable way to address this issue, since currently either option 
>> (using libcxx in ENABLE_PROJECTS or using it in ENABLE_RUNTIMES) is 
>> incomplete - if I use ENABLE_RUNTIMES I get the libcxx testing run against 
>> the just-built clang and generally this is the "supported" configuration, 
>> but then some lldb tests fail because they can't find libcxx.so.1 (on Linux) 
>> - and using ENABLE_PROJECTS means not using the just-built clang for libcxx 
>> tests (so missing the libcxx breakages caused by my array name change) but 
>> do use the just-built libcxx in lldb tests and find failures there...
>>
>> On Wed, Oct 20, 2021 at 1:57 PM David Blaikie  wrote:
>>>
>>> On Tue, Oct 19, 2021 at 4:55 PM David Blaikie  wrote:

 On Tue, Oct 19, 2021 at 9:08 AM Raphael Isemann  
 wrote:
>
> Actually the RPATH theory is wrong, but the LLVM_ENABLE_PROJECT
> workaround *should* still work.


 I'll give that a go (it's running at the moment) though I guess this is 
 inconsistent with the direction libcxx is moving in for building, re: 
 https://groups.google.com/g/llvm-dev/c/tpuLxk_ipLw
>>>
>>>
>>> Yep, it does work with LLVM_ENABLE_PROJECT rather than LLVM_ENABLE_RUNTIME.
>>>
>>> Specifically the test binary is linked with an rpath to the just-built lib 
>>> directory that ensures the just-built libc++.so is found:
>>>
>>> /usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang main.o -g 
>>> -O0 -fno-builtin -m64  
>>> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/../../../../../include
>>>  
>>> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/list
>>>  
>>> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make
>>>  -include 
>>> /usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/test_common.h
>>>  -fno-limit-debug-info  -gsplit-dwarf-stdlib=libc++ 
>>> -Wl,-rpath,/usr/local/google/home/blaikie/dev/llvm/build/release/./lib 
>>> --driver-mode=g++ -o "a.out"
>>>
>>> Oh, actually it passes the same rpath when using LLVM_ENABLE_RUNTIME, but 
>>> the

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-25 Thread David Blaikie via lldb-dev 
On Mon, Oct 25, 2021 at 1:28 PM Louis Dionne  wrote:

> I believe the issue is probably not related so much to
> LLVM_ENABLE_PROJECTS vs LLVM_ENABLE_RUNTIMES, but rather to the fact that
> LLVM_ENABLE_RUNTIMES uses per-target runtime directories now (hasn't always
> been the case), which basically means that libc++ ends up in
> `/lib//libc++.so` instead of
> `/lib/libc++.so`.
>

Ish, yes. It's a bug in LLVM_ENABLE_RUNTIMES that isn't present in
LLVM_ENABLE_PROJECTS, so for now if I want to run the lldb pretty printer
tests for libc++ on Linux it seems the only way I can is by using the
deprecated functionality of libc++ in LLVM_ENABLE_PROJECTS.

Consider this a bug report (looks like a bug in the lldb CMake
configuration, not in libc++'s build itself, but something to figure out if
Linux lldb devs are going to use libc++ +.ENABLE_RUNTIMES path) on that
deprecation?


> I think you either want to specify the per-target library dir when running
> against libc++, or you want to disable that and use
> `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF` when configuring the runtimes. In
> all cases, you want to be using `LLVM_ENABLE_RUNTIMES` and not
> `LLVM_ENABLE_PROJECTS`, since the latter is deprecated now.
>

I didn't enable LLVM_ENABLE_PER_TARGET_RUNTIME_DIR myself/in the root cmake
config. It looks like it's hardcoded(?) into the ENABLE_RUNTIMES sub-build?
https://github.com/llvm/llvm-project/blob/e5fb79b31424267704e9d2d9674089fd7316453e/llvm/runtimes/CMakeLists.txt#L76
I'm not sure there's any way to override that from the root? And in any
case I'd have thought the defaults would need to/be intended to work
correctly on supported platforms?

So something in lldb's dir handling (maybe some general infrastructure in
LLVM could use some improvement to provide an LLVM_RUNTIME_LIBS_DIR, or
similar? that could then be used from other places - rather than libc++,
for instance, creating that directory for itself based on LLVM_LIBS_DIR and
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR, etc) needs some fixes to support the
current defaults/hardcoded modes on Linux?


> Cheers,
> Louis
>
> On Oct 25, 2021, at 13:57, David Blaikie  wrote:
>
> +Louis Dionne  - perhaps the libcxx and lldb folks
> would be interesting in finding a suitable way to address this issue, since
> currently either option (using libcxx in ENABLE_PROJECTS or using it in
> ENABLE_RUNTIMES) is incomplete - if I use ENABLE_RUNTIMES I get the libcxx
> testing run against the just-built clang and generally this is the
> "supported" configuration, but then some lldb tests fail because they can't
> find libcxx.so.1 (on Linux) - and using ENABLE_PROJECTS means not using the
> just-built clang for libcxx tests (so missing the libcxx breakages caused
> by my array name change) but do use the just-built libcxx in lldb tests and
> find failures there...
>
> On Wed, Oct 20, 2021 at 1:57 PM David Blaikie  wrote:
>
>> On Tue, Oct 19, 2021 at 4:55 PM David Blaikie  wrote:
>>
>>> On Tue, Oct 19, 2021 at 9:08 AM Raphael Isemann 
>>> wrote:
>>>
 Actually the RPATH theory is wrong, but the LLVM_ENABLE_PROJECT
 workaround *should* still work.

>>>
>>> I'll give that a go (it's running at the moment) though I guess this is
>>> inconsistent with the direction libcxx is moving in for building, re:
>>> https://groups.google.com/g/llvm-dev/c/tpuLxk_ipLw
>>>
>>
>> Yep, it does work with LLVM_ENABLE_PROJECT rather than
>> LLVM_ENABLE_RUNTIME.
>>
>> Specifically the test binary is linked with an rpath to the just-built
>> lib directory that ensures the just-built libc++.so is found:
>>
>> /usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang main.o -g
>> -O0 -fno-builtin -m64  
>> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/../../../../../include
>> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/list
>> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make
>> -include
>> /usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/test_common.h
>> -fno-limit-debug-info  -gsplit-dwarf-stdlib=libc++
>> -Wl,-rpath,/usr/local/google/home/blaikie/dev/llvm/build/release/./lib
>> --driver-mode=g++ -o "a.out"
>>
>> Oh, actually it passes the same rpath when using LLVM_ENABLE_RUNTIME, but
>> the libc++.so.1 is in a different place:
>> ./lib/x86_64-unknown-linux-gnu/libc++.so.1
>>
>> Looks like this rpath setting happens here: (changing this to a junk
>> argument causes the test to fail to build as expected)
>>
>> https://github.com/llvm/llvm-project/blob/618583565687f5a494066fc902a977f6057fc93e/lldb/packages/Python/lldbsuite/test/make/Makefile.rules#L400
>>
>> And it gets the LLVM_LIBS_DIR from here:
>> https://github.com/llvm/llvm-project/blob/207998c242c8c8a270ff22a5136da87338546725/lldb/test/API/lit.cfg.py#L163
>>
>> So maybe we need to pass down the default target triple too, since that
>> seems to be how

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-25 Thread Louis Dionne via lldb-dev 
I believe the issue is probably not related so much to LLVM_ENABLE_PROJECTS vs 
LLVM_ENABLE_RUNTIMES, but rather to the fact that LLVM_ENABLE_RUNTIMES uses 
per-target runtime directories now (hasn't always been the case), which 
basically means that libc++ ends up in `/lib//libc++.so` 
instead of `/lib/libc++.so`.

I think you either want to specify the per-target library dir when running 
against libc++, or you want to disable that and use 
`LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF` when configuring the runtimes. In all 
cases, you want to be using `LLVM_ENABLE_RUNTIMES` and not 
`LLVM_ENABLE_PROJECTS`, since the latter is deprecated now.

Cheers,
Louis

> On Oct 25, 2021, at 13:57, David Blaikie  wrote:
> 
> +Louis Dionne  - perhaps the libcxx and lldb folks 
> would be interesting in finding a suitable way to address this issue, since 
> currently either option (using libcxx in ENABLE_PROJECTS or using it in 
> ENABLE_RUNTIMES) is incomplete - if I use ENABLE_RUNTIMES I get the libcxx 
> testing run against the just-built clang and generally this is the 
> "supported" configuration, but then some lldb tests fail because they can't 
> find libcxx.so.1 (on Linux) - and using ENABLE_PROJECTS means not using the 
> just-built clang for libcxx tests (so missing the libcxx breakages caused by 
> my array name change) but do use the just-built libcxx in lldb tests and find 
> failures there... 
> 
> On Wed, Oct 20, 2021 at 1:57 PM David Blaikie  > wrote:
> On Tue, Oct 19, 2021 at 4:55 PM David Blaikie  > wrote:
> On Tue, Oct 19, 2021 at 9:08 AM Raphael Isemann  > wrote:
> Actually the RPATH theory is wrong, but the LLVM_ENABLE_PROJECT
> workaround *should* still work.
> 
> I'll give that a go (it's running at the moment) though I guess this is 
> inconsistent with the direction libcxx is moving in for building, re: 
> https://groups.google.com/g/llvm-dev/c/tpuLxk_ipLw 
> 
> 
> Yep, it does work with LLVM_ENABLE_PROJECT rather than LLVM_ENABLE_RUNTIME.
> 
> Specifically the test binary is linked with an rpath to the just-built lib 
> directory that ensures the just-built libc++.so is found:
> 
> /usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang main.o -g -O0 
> -fno-builtin -m64  
> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/../../../../../include
>  
> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/list
>  
> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make
>  -include 
> /usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/test_common.h
>  -fno-limit-debug-info  -gsplit-dwarf-stdlib=libc++ 
> -Wl,-rpath,/usr/local/google/home/blaikie/dev/llvm/build/release/./lib 
> --driver-mode=g++ -o "a.out"
>  
> Oh, actually it passes the same rpath when using LLVM_ENABLE_RUNTIME, but the 
> libc++.so.1 is in a different place: 
> ./lib/x86_64-unknown-linux-gnu/libc++.so.1
> 
> Looks like this rpath setting happens here: (changing this to a junk argument 
> causes the test to fail to build as expected)
> https://github.com/llvm/llvm-project/blob/618583565687f5a494066fc902a977f6057fc93e/lldb/packages/Python/lldbsuite/test/make/Makefile.rules#L400
>  
> 
> 
> And it gets the LLVM_LIBS_DIR from here: 
> https://github.com/llvm/llvm-project/blob/207998c242c8c8a270ff22a5136da87338546725/lldb/test/API/lit.cfg.py#L163
>  
> 
> 
> So maybe we need to pass down the default target triple too, since that seems 
> to be how libc++ is deciding where to put the library? ( 
> https://github.com/llvm/llvm-project/blob/207998c242c8c8a270ff22a5136da87338546725/libcxx/CMakeLists.txt#L424
>  
> 
>  ) at least on non-apple :/ (or maybe there's some way to make the connection 
> between the two less brittle - for libc++'s build to export some variable 
> that lldb can use, or for LLVM to provide something for both to use?)
> 
> Yeah, applying this change does work for me, but wouldn't work on Apple for 
> instance (where libcxx doesn't add the default target triple to the path):
> $ git diff
> diff --git lldb/test/API/lit.site.cfg.py.in  
> lldb/test/API/lit.site.cfg.py.in 
> index 987078a53edb..e327429b7ff9 100644
> --- lldb/test/API/lit.site.cfg.py.in 
> +++ lldb/test/API/lit.site.cfg.py.in 
> @@ -3,7 +3,7 @@
>

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-25 Thread David Blaikie via lldb-dev 
+Louis Dionne  - perhaps the libcxx and lldb folks would
be interesting in finding a suitable way to address this issue, since
currently either option (using libcxx in ENABLE_PROJECTS or using it in
ENABLE_RUNTIMES) is incomplete - if I use ENABLE_RUNTIMES I get the libcxx
testing run against the just-built clang and generally this is the
"supported" configuration, but then some lldb tests fail because they can't
find libcxx.so.1 (on Linux) - and using ENABLE_PROJECTS means not using the
just-built clang for libcxx tests (so missing the libcxx breakages caused
by my array name change) but do use the just-built libcxx in lldb tests and
find failures there...

On Wed, Oct 20, 2021 at 1:57 PM David Blaikie  wrote:

> On Tue, Oct 19, 2021 at 4:55 PM David Blaikie  wrote:
>
>> On Tue, Oct 19, 2021 at 9:08 AM Raphael Isemann 
>> wrote:
>>
>>> Actually the RPATH theory is wrong, but the LLVM_ENABLE_PROJECT
>>> workaround *should* still work.
>>>
>>
>> I'll give that a go (it's running at the moment) though I guess this is
>> inconsistent with the direction libcxx is moving in for building, re:
>> https://groups.google.com/g/llvm-dev/c/tpuLxk_ipLw
>>
>
> Yep, it does work with LLVM_ENABLE_PROJECT rather than LLVM_ENABLE_RUNTIME.
>
> Specifically the test binary is linked with an rpath to the just-built lib
> directory that ensures the just-built libc++.so is found:
>
> /usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang main.o -g
> -O0 -fno-builtin -m64  
> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/../../../../../include
> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/list
> -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make
> -include
> /usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/test_common.h
> -fno-limit-debug-info  -gsplit-dwarf-stdlib=libc++
> -Wl,-rpath,/usr/local/google/home/blaikie/dev/llvm/build/release/./lib
> --driver-mode=g++ -o "a.out"
>
> Oh, actually it passes the same rpath when using LLVM_ENABLE_RUNTIME, but
> the libc++.so.1 is in a different place:
> ./lib/x86_64-unknown-linux-gnu/libc++.so.1
>
> Looks like this rpath setting happens here: (changing this to a junk
> argument causes the test to fail to build as expected)
>
> https://github.com/llvm/llvm-project/blob/618583565687f5a494066fc902a977f6057fc93e/lldb/packages/Python/lldbsuite/test/make/Makefile.rules#L400
>
> And it gets the LLVM_LIBS_DIR from here:
> https://github.com/llvm/llvm-project/blob/207998c242c8c8a270ff22a5136da87338546725/lldb/test/API/lit.cfg.py#L163
>
> So maybe we need to pass down the default target triple too, since that
> seems to be how libc++ is deciding where to put the library? (
> https://github.com/llvm/llvm-project/blob/207998c242c8c8a270ff22a5136da87338546725/libcxx/CMakeLists.txt#L424
> ) at least on non-apple :/ (or maybe there's some way to make the
> connection between the two less brittle - for libc++'s build to export some
> variable that lldb can use, or for LLVM to provide something for both to
> use?)
>
> Yeah, applying this change does work for me, but wouldn't work on Apple
> for instance (where libcxx doesn't add the default target triple to the
> path):
>
> $ git diff
>
> *diff --git lldb/test/API/lit.site.cfg.py.in 
> lldb/test/API/lit.site.cfg.py.in *
>
> *index 987078a53edb..e327429b7ff9 100644*
>
> *--- lldb/test/API/lit.site.cfg.py.in *
>
> *+++ lldb/test/API/lit.site.cfg.py.in *
>
> @@ -3,7 +3,7 @@
>
>  config.llvm_src_root = "@LLVM_SOURCE_DIR@"
>
>  config.llvm_obj_root = "@LLVM_BINARY_DIR@"
>
>  config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
>
> -config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
>
> +config.llvm_libs_dir = "@LLVM_LIBS_DIR@/@LLVM_DEFAULT_TARGET_TRIPLE@"
>
>  config.llvm_shlib_dir = "@SHLIBDIR@"
>
>  config.llvm_build_mode = "@LLVM_BUILD_MODE@"
>
>  config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
>
> Thoughts?
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-20 Thread David Blaikie via lldb-dev 
On Tue, Oct 19, 2021 at 4:55 PM David Blaikie  wrote:

> On Tue, Oct 19, 2021 at 9:08 AM Raphael Isemann 
> wrote:
>
>> Actually the RPATH theory is wrong, but the LLVM_ENABLE_PROJECT
>> workaround *should* still work.
>>
>
> I'll give that a go (it's running at the moment) though I guess this is
> inconsistent with the direction libcxx is moving in for building, re:
> https://groups.google.com/g/llvm-dev/c/tpuLxk_ipLw
>

Yep, it does work with LLVM_ENABLE_PROJECT rather than LLVM_ENABLE_RUNTIME.

Specifically the test binary is linked with an rpath to the just-built lib
directory that ensures the just-built libc++.so is found:

/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang main.o -g
-O0 -fno-builtin -m64
-I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/../../../../../include
-I/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/list
-I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make
-include
/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/test_common.h
-fno-limit-debug-info  -gsplit-dwarf-stdlib=libc++
-Wl,-rpath,/usr/local/google/home/blaikie/dev/llvm/build/release/./lib
--driver-mode=g++ -o "a.out"

Oh, actually it passes the same rpath when using LLVM_ENABLE_RUNTIME, but
the libc++.so.1 is in a different place:
./lib/x86_64-unknown-linux-gnu/libc++.so.1

Looks like this rpath setting happens here: (changing this to a junk
argument causes the test to fail to build as expected)
https://github.com/llvm/llvm-project/blob/618583565687f5a494066fc902a977f6057fc93e/lldb/packages/Python/lldbsuite/test/make/Makefile.rules#L400

And it gets the LLVM_LIBS_DIR from here:
https://github.com/llvm/llvm-project/blob/207998c242c8c8a270ff22a5136da87338546725/lldb/test/API/lit.cfg.py#L163

So maybe we need to pass down the default target triple too, since that
seems to be how libc++ is deciding where to put the library? (
https://github.com/llvm/llvm-project/blob/207998c242c8c8a270ff22a5136da87338546725/libcxx/CMakeLists.txt#L424
) at least on non-apple :/ (or maybe there's some way to make the
connection between the two less brittle - for libc++'s build to export some
variable that lldb can use, or for LLVM to provide something for both to
use?)

Yeah, applying this change does work for me, but wouldn't work on Apple for
instance (where libcxx doesn't add the default target triple to the path):

$ git diff

*diff --git lldb/test/API/lit.site.cfg.py.in 
lldb/test/API/lit.site.cfg.py.in *

*index 987078a53edb..e327429b7ff9 100644*

*--- lldb/test/API/lit.site.cfg.py.in *

*+++ lldb/test/API/lit.site.cfg.py.in *

@@ -3,7 +3,7 @@

 config.llvm_src_root = "@LLVM_SOURCE_DIR@"

 config.llvm_obj_root = "@LLVM_BINARY_DIR@"

 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"

-config.llvm_libs_dir = "@LLVM_LIBS_DIR@"

+config.llvm_libs_dir = "@LLVM_LIBS_DIR@/@LLVM_DEFAULT_TARGET_TRIPLE@"

 config.llvm_shlib_dir = "@SHLIBDIR@"

 config.llvm_build_mode = "@LLVM_BUILD_MODE@"

 config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"

Thoughts?
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-19 Thread David Blaikie via lldb-dev 
On Tue, Oct 19, 2021 at 9:08 AM Raphael Isemann  wrote:

> Actually the RPATH theory is wrong, but the LLVM_ENABLE_PROJECT
> workaround *should* still work.
>

I'll give that a go (it's running at the moment) though I guess this is
inconsistent with the direction libcxx is moving in for building, re:
https://groups.google.com/g/llvm-dev/c/tpuLxk_ipLw


>
> Am Di., 19. Okt. 2021 um 18:02 Uhr schrieb Raphael Isemann
> :
> >
> > I just saw in your review comment that this is using
> > LLVM_ENABLE_RUNTIMES and not LLVM_ENABLE_PROJECTS for libcxx, so the
> > failure just comes from us setting the wrong RPATH due to the
> > different runtimes library directory (at least from what I can see).
> > Would it be possible to put libcxx in LLVM_ENABLE_PROJECTS for now? I
> > think this shouldn't be too hard to fix (famous last words?).
> >
> > Am Mo., 18. Okt. 2021 um 22:30 Uhr schrieb David Blaikie <
> dblai...@gmail.com>:
> > >
> > > On Mon, Oct 18, 2021 at 9:45 AM Raphael Isemann 
> wrote:
> > >>
> > >> I think https://reviews.llvm.org/D111978 ,
> > >> https://reviews.llvm.org/D111981 and the other patches Pavel & me put
> > >> up today should improve this situation IIUC.
> > >
> > >
> > > Thanks Raphael - really appreciate you & looking into this!
> > >
> > > With https://reviews.llvm.org/D111981 I still seem to not have that
> cxx dependency (building/running the test, then building libcxx, then
> running the test again goes from unsupported -> failing) - didn't seem to
> work for me?
> > >
> > > The diagnostic improvement sounds good to me (pity about whatever
> limitations it has re: remote or windows execution gathering the stdout). I
> guess gathering the logs in the remote execution case has the same problem
> (that the log is made on the remote machine/non-trivial to retrieve?)?
> > >
> > > & yeah, have you got any patches/ideas about how to make the libcxx
> tests use the just-built libcxx library (using LD_LIBRARY_PATH, rpath,
> etc)? For now, even with both these fixes I'll just be seeing (consistent,
> which is nice) failures, rather than being able to run these tests
> successfully. I'll either have to get used to ignoring certain failures, or
> disable the tests by not building libcxx in that build tree, which would
> also be unfortunate. (or maybe there's some other workarounds?) Any idea
> how this works for other folks?
> > >
> > > - Dave
> > >
> > >> - Raphael
> > >>
> > >> Am Mo., 18. Okt. 2021 um 05:54 Uhr schrieb David Blaikie via lldb-dev
> > >> :
> > >> >
> > >> > Wondering if anyone else has encountered/dealt with debugging lldb
> test failures like the one shown at the end of this email ("AssertionError:
> 10 != 5" in "test.assertEqual(process.GetState(), lldb.eStateStopped)"
> while checking that a breakpoint was reached)
> > >> >
> > >> > Is there anything that could be done to improve the debuggability
> of such failures? Logging standard output/error from the lldb process or
> any other logging it might have? At least for one of these I managed to
> figure it out by running lldb directly on the binary and finding that the
> binary couldn't find libc++.so (that's a story for another bug/email
> thread, looks like maybe lldb libc++ pretty printer tests are using the
> system installed libc++, not the just-built libc++ (& thus not running if
> there is no system installed libc++)). But my current failure like this
> seems a bit more inscrutible and I'm still looking into it.
> > >> >
> > >> > Looks like dotest.py has some sense of logging (it has a
> --log-success option which says preserves the logs even on failure - though
> the output of dotest.py, at least for me, has no mention of logs, log
> files, or where they might be located, and looking at the source points to
> some sort of ".log" files... ah, found some)
> > >> >
> > >> > So, yeah, there do seem to be some Failure.log, SkippedTest.log,
> etc - should dotest print something about the paths to those files when it
> exits with failure, maybe?
> > >> >
> > >> > 
> > >> >
> > >> > FAIL: lldb-api ::
> functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py
> (23 of 23)
> > >> >
> > >> >  TEST 'lldb-api ::
> functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py'
> FAILED 
> > >> >
> > >> > Script:
> > >> >
> > >> > --
> > >> >
> > >> > /usr/bin/python3
> /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/dotest.py -u
> CXXFLAGS -u CFLAGS --env ARCHIVER=/usr/bin/ar --env
> OBJCOPY=/usr/bin/objcopy --env
> LLVM_LIBS_DIR=/usr/local/google/home/blaikie/dev/llvm/build/release/./lib
> --arch x86_64 --build-dir
> /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex
> --lldb-module-cache-dir
> /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-lldb/lldb-api
> --clang-module-cache-dir
>

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-19 Thread Raphael Isemann via lldb-dev 
Actually the RPATH theory is wrong, but the LLVM_ENABLE_PROJECT
workaround *should* still work.

Am Di., 19. Okt. 2021 um 18:02 Uhr schrieb Raphael Isemann
:
>
> I just saw in your review comment that this is using
> LLVM_ENABLE_RUNTIMES and not LLVM_ENABLE_PROJECTS for libcxx, so the
> failure just comes from us setting the wrong RPATH due to the
> different runtimes library directory (at least from what I can see).
> Would it be possible to put libcxx in LLVM_ENABLE_PROJECTS for now? I
> think this shouldn't be too hard to fix (famous last words?).
>
> Am Mo., 18. Okt. 2021 um 22:30 Uhr schrieb David Blaikie :
> >
> > On Mon, Oct 18, 2021 at 9:45 AM Raphael Isemann  wrote:
> >>
> >> I think https://reviews.llvm.org/D111978 ,
> >> https://reviews.llvm.org/D111981 and the other patches Pavel & me put
> >> up today should improve this situation IIUC.
> >
> >
> > Thanks Raphael - really appreciate you & looking into this!
> >
> > With https://reviews.llvm.org/D111981 I still seem to not have that cxx 
> > dependency (building/running the test, then building libcxx, then running 
> > the test again goes from unsupported -> failing) - didn't seem to work for 
> > me?
> >
> > The diagnostic improvement sounds good to me (pity about whatever 
> > limitations it has re: remote or windows execution gathering the stdout). I 
> > guess gathering the logs in the remote execution case has the same problem 
> > (that the log is made on the remote machine/non-trivial to retrieve?)?
> >
> > & yeah, have you got any patches/ideas about how to make the libcxx tests 
> > use the just-built libcxx library (using LD_LIBRARY_PATH, rpath, etc)? For 
> > now, even with both these fixes I'll just be seeing (consistent, which is 
> > nice) failures, rather than being able to run these tests successfully. 
> > I'll either have to get used to ignoring certain failures, or disable the 
> > tests by not building libcxx in that build tree, which would also be 
> > unfortunate. (or maybe there's some other workarounds?) Any idea how this 
> > works for other folks?
> >
> > - Dave
> >
> >> - Raphael
> >>
> >> Am Mo., 18. Okt. 2021 um 05:54 Uhr schrieb David Blaikie via lldb-dev
> >> :
> >> >
> >> > Wondering if anyone else has encountered/dealt with debugging lldb test 
> >> > failures like the one shown at the end of this email ("AssertionError: 
> >> > 10 != 5" in "test.assertEqual(process.GetState(), lldb.eStateStopped)" 
> >> > while checking that a breakpoint was reached)
> >> >
> >> > Is there anything that could be done to improve the debuggability of 
> >> > such failures? Logging standard output/error from the lldb process or 
> >> > any other logging it might have? At least for one of these I managed to 
> >> > figure it out by running lldb directly on the binary and finding that 
> >> > the binary couldn't find libc++.so (that's a story for another bug/email 
> >> > thread, looks like maybe lldb libc++ pretty printer tests are using the 
> >> > system installed libc++, not the just-built libc++ (& thus not running 
> >> > if there is no system installed libc++)). But my current failure like 
> >> > this seems a bit more inscrutible and I'm still looking into it.
> >> >
> >> > Looks like dotest.py has some sense of logging (it has a --log-success 
> >> > option which says preserves the logs even on failure - though the output 
> >> > of dotest.py, at least for me, has no mention of logs, log files, or 
> >> > where they might be located, and looking at the source points to some 
> >> > sort of ".log" files... ah, found some)
> >> >
> >> > So, yeah, there do seem to be some Failure.log, SkippedTest.log, etc - 
> >> > should dotest print something about the paths to those files when it 
> >> > exits with failure, maybe?
> >> >
> >> > 
> >> >
> >> > FAIL: lldb-api :: 
> >> > functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py
> >> >  (23 of 23)
> >> >
> >> >  TEST 'lldb-api :: 
> >> > functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py'
> >> >  FAILED 
> >> >
> >> > Script:
> >> >
> >> > --
> >> >
> >> > /usr/bin/python3 
> >> > /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/dotest.py -u 
> >> > CXXFLAGS -u CFLAGS --env ARCHIVER=/usr/bin/ar --env 
> >> > OBJCOPY=/usr/bin/objcopy --env 
> >> > LLVM_LIBS_DIR=/usr/local/google/home/blaikie/dev/llvm/build/release/./lib
> >> >  --arch x86_64 --build-dir 
> >> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex
> >> >  --lldb-module-cache-dir 
> >> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-lldb/lldb-api
> >> >  --clang-module-cache-dir 
> >> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-clang/lldb-api
> >> >  --executable 
> >> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/lldb 
> >> > --compiler 
>

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-19 Thread Raphael Isemann via lldb-dev 
I just saw in your review comment that this is using
LLVM_ENABLE_RUNTIMES and not LLVM_ENABLE_PROJECTS for libcxx, so the
failure just comes from us setting the wrong RPATH due to the
different runtimes library directory (at least from what I can see).
Would it be possible to put libcxx in LLVM_ENABLE_PROJECTS for now? I
think this shouldn't be too hard to fix (famous last words?).

Am Mo., 18. Okt. 2021 um 22:30 Uhr schrieb David Blaikie :
>
> On Mon, Oct 18, 2021 at 9:45 AM Raphael Isemann  wrote:
>>
>> I think https://reviews.llvm.org/D111978 ,
>> https://reviews.llvm.org/D111981 and the other patches Pavel & me put
>> up today should improve this situation IIUC.
>
>
> Thanks Raphael - really appreciate you & looking into this!
>
> With https://reviews.llvm.org/D111981 I still seem to not have that cxx 
> dependency (building/running the test, then building libcxx, then running the 
> test again goes from unsupported -> failing) - didn't seem to work for me?
>
> The diagnostic improvement sounds good to me (pity about whatever limitations 
> it has re: remote or windows execution gathering the stdout). I guess 
> gathering the logs in the remote execution case has the same problem (that 
> the log is made on the remote machine/non-trivial to retrieve?)?
>
> & yeah, have you got any patches/ideas about how to make the libcxx tests use 
> the just-built libcxx library (using LD_LIBRARY_PATH, rpath, etc)? For now, 
> even with both these fixes I'll just be seeing (consistent, which is nice) 
> failures, rather than being able to run these tests successfully. I'll either 
> have to get used to ignoring certain failures, or disable the tests by not 
> building libcxx in that build tree, which would also be unfortunate. (or 
> maybe there's some other workarounds?) Any idea how this works for other 
> folks?
>
> - Dave
>
>> - Raphael
>>
>> Am Mo., 18. Okt. 2021 um 05:54 Uhr schrieb David Blaikie via lldb-dev
>> :
>> >
>> > Wondering if anyone else has encountered/dealt with debugging lldb test 
>> > failures like the one shown at the end of this email ("AssertionError: 10 
>> > != 5" in "test.assertEqual(process.GetState(), lldb.eStateStopped)" while 
>> > checking that a breakpoint was reached)
>> >
>> > Is there anything that could be done to improve the debuggability of such 
>> > failures? Logging standard output/error from the lldb process or any other 
>> > logging it might have? At least for one of these I managed to figure it 
>> > out by running lldb directly on the binary and finding that the binary 
>> > couldn't find libc++.so (that's a story for another bug/email thread, 
>> > looks like maybe lldb libc++ pretty printer tests are using the system 
>> > installed libc++, not the just-built libc++ (& thus not running if there 
>> > is no system installed libc++)). But my current failure like this seems a 
>> > bit more inscrutible and I'm still looking into it.
>> >
>> > Looks like dotest.py has some sense of logging (it has a --log-success 
>> > option which says preserves the logs even on failure - though the output 
>> > of dotest.py, at least for me, has no mention of logs, log files, or where 
>> > they might be located, and looking at the source points to some sort of 
>> > ".log" files... ah, found some)
>> >
>> > So, yeah, there do seem to be some Failure.log, SkippedTest.log, etc - 
>> > should dotest print something about the paths to those files when it exits 
>> > with failure, maybe?
>> >
>> > 
>> >
>> > FAIL: lldb-api :: 
>> > functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py
>> >  (23 of 23)
>> >
>> >  TEST 'lldb-api :: 
>> > functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py'
>> >  FAILED 
>> >
>> > Script:
>> >
>> > --
>> >
>> > /usr/bin/python3 
>> > /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/dotest.py -u 
>> > CXXFLAGS -u CFLAGS --env ARCHIVER=/usr/bin/ar --env 
>> > OBJCOPY=/usr/bin/objcopy --env 
>> > LLVM_LIBS_DIR=/usr/local/google/home/blaikie/dev/llvm/build/release/./lib 
>> > --arch x86_64 --build-dir 
>> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex
>> >  --lldb-module-cache-dir 
>> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-lldb/lldb-api
>> >  --clang-module-cache-dir 
>> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-clang/lldb-api
>> >  --executable 
>> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/lldb 
>> > --compiler 
>> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/clang 
>> > --dsymutil 
>> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/dsymutil 
>> > --llvm-tools-dir 
>> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin 
>> > --lldb-libs-dir 
>> > /usr/local/google/home/blaikie/dev/llvm/build/release/./lib 
>> >

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-18 Thread David Blaikie via lldb-dev 
On Mon, Oct 18, 2021 at 9:45 AM Raphael Isemann  wrote:

> I think https://reviews.llvm.org/D111978 ,
> https://reviews.llvm.org/D111981 and the other patches Pavel & me put
> up today should improve this situation IIUC.
>

Thanks Raphael - really appreciate you & looking into this!

With https://reviews.llvm.org/D111981 I still seem to not have that cxx
dependency (building/running the test, then building libcxx, then running
the test again goes from unsupported -> failing) - didn't seem to work for
me?

The diagnostic improvement sounds good to me (pity about whatever
limitations it has re: remote or windows execution gathering the stdout). I
guess gathering the logs in the remote execution case has the same problem
(that the log is made on the remote machine/non-trivial to retrieve?)?

& yeah, have you got any patches/ideas about how to make the libcxx tests
use the just-built libcxx library (using LD_LIBRARY_PATH, rpath, etc)? For
now, even with both these fixes I'll just be seeing (consistent, which is
nice) failures, rather than being able to run these tests successfully.
I'll either have to get used to ignoring certain failures, or disable the
tests by not building libcxx in that build tree, which would also be
unfortunate. (or maybe there's some other workarounds?) Any idea how this
works for other folks?

- Dave

- Raphael
>
> Am Mo., 18. Okt. 2021 um 05:54 Uhr schrieb David Blaikie via lldb-dev
> :
> >
> > Wondering if anyone else has encountered/dealt with debugging lldb test
> failures like the one shown at the end of this email ("AssertionError: 10
> != 5" in "test.assertEqual(process.GetState(), lldb.eStateStopped)" while
> checking that a breakpoint was reached)
> >
> > Is there anything that could be done to improve the debuggability of
> such failures? Logging standard output/error from the lldb process or any
> other logging it might have? At least for one of these I managed to figure
> it out by running lldb directly on the binary and finding that the binary
> couldn't find libc++.so (that's a story for another bug/email thread, looks
> like maybe lldb libc++ pretty printer tests are using the system installed
> libc++, not the just-built libc++ (& thus not running if there is no system
> installed libc++)). But my current failure like this seems a bit more
> inscrutible and I'm still looking into it.
> >
> > Looks like dotest.py has some sense of logging (it has a --log-success
> option which says preserves the logs even on failure - though the output of
> dotest.py, at least for me, has no mention of logs, log files, or where
> they might be located, and looking at the source points to some sort of
> ".log" files... ah, found some)
> >
> > So, yeah, there do seem to be some Failure.log, SkippedTest.log, etc -
> should dotest print something about the paths to those files when it exits
> with failure, maybe?
> >
> > 
> >
> > FAIL: lldb-api ::
> functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py
> (23 of 23)
> >
> >  TEST 'lldb-api ::
> functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py'
> FAILED 
> >
> > Script:
> >
> > --
> >
> > /usr/bin/python3
> /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/dotest.py -u
> CXXFLAGS -u CFLAGS --env ARCHIVER=/usr/bin/ar --env
> OBJCOPY=/usr/bin/objcopy --env
> LLVM_LIBS_DIR=/usr/local/google/home/blaikie/dev/llvm/build/release/./lib
> --arch x86_64 --build-dir
> /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex
> --lldb-module-cache-dir
> /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-lldb/lldb-api
> --clang-module-cache-dir
> /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-clang/lldb-api
> --executable
> /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/lldb --compiler
> /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/clang
> --dsymutil
> /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/dsymutil
> --llvm-tools-dir
> /usr/local/google/home/blaikie/dev/llvm/build/release/./bin --lldb-libs-dir
> /usr/local/google/home/blaikie/dev/llvm/build/release/./lib
> /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set
> -p TestDataFormatterLibcxxSet.py
> >
> > --
> >
> > Exit Code: 1
> >
> >
> > Command Output (stdout):
> >
> > --
> >
> > lldb version 14.0.0 (g...@github.com:llvm/llvm-project.git revision
> 6176fda3f992b5086302b3826aa0636135cc4cc0)
> >
> >   clang revision 6176fda3f992b5086302b3826aa0636135cc4cc0
> >
> >   llvm revision 6176fda3f992b5086302b3826aa0636135cc4cc0
> >
> > Skipping the following test categories: ['dsym', 'gmodules',
> 'debugserver', 'objc']
> >
> >
> > --
> >
> > Command Output (stderr):
> >
> > --
> >
> > UNSUPPORTED: LLDB
>

Re: [lldb-dev] Some API test failures are really opaque/could be improved

2021-10-18 Thread Raphael Isemann via lldb-dev 
I think https://reviews.llvm.org/D111978 ,
https://reviews.llvm.org/D111981 and the other patches Pavel & me put
up today should improve this situation IIUC.

- Raphael

Am Mo., 18. Okt. 2021 um 05:54 Uhr schrieb David Blaikie via lldb-dev
:
>
> Wondering if anyone else has encountered/dealt with debugging lldb test 
> failures like the one shown at the end of this email ("AssertionError: 10 != 
> 5" in "test.assertEqual(process.GetState(), lldb.eStateStopped)" while 
> checking that a breakpoint was reached)
>
> Is there anything that could be done to improve the debuggability of such 
> failures? Logging standard output/error from the lldb process or any other 
> logging it might have? At least for one of these I managed to figure it out 
> by running lldb directly on the binary and finding that the binary couldn't 
> find libc++.so (that's a story for another bug/email thread, looks like maybe 
> lldb libc++ pretty printer tests are using the system installed libc++, not 
> the just-built libc++ (& thus not running if there is no system installed 
> libc++)). But my current failure like this seems a bit more inscrutible and 
> I'm still looking into it.
>
> Looks like dotest.py has some sense of logging (it has a --log-success option 
> which says preserves the logs even on failure - though the output of 
> dotest.py, at least for me, has no mention of logs, log files, or where they 
> might be located, and looking at the source points to some sort of ".log" 
> files... ah, found some)
>
> So, yeah, there do seem to be some Failure.log, SkippedTest.log, etc - should 
> dotest print something about the paths to those files when it exits with 
> failure, maybe?
>
> 
>
> FAIL: lldb-api :: 
> functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py
>  (23 of 23)
>
>  TEST 'lldb-api :: 
> functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py'
>  FAILED 
>
> Script:
>
> --
>
> /usr/bin/python3 
> /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/dotest.py -u 
> CXXFLAGS -u CFLAGS --env ARCHIVER=/usr/bin/ar --env OBJCOPY=/usr/bin/objcopy 
> --env 
> LLVM_LIBS_DIR=/usr/local/google/home/blaikie/dev/llvm/build/release/./lib 
> --arch x86_64 --build-dir 
> /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex 
> --lldb-module-cache-dir 
> /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-lldb/lldb-api
>  --clang-module-cache-dir 
> /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-clang/lldb-api
>  --executable 
> /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/lldb --compiler 
> /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/clang --dsymutil 
> /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/dsymutil 
> --llvm-tools-dir /usr/local/google/home/blaikie/dev/llvm/build/release/./bin 
> --lldb-libs-dir /usr/local/google/home/blaikie/dev/llvm/build/release/./lib 
> /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set
>  -p TestDataFormatterLibcxxSet.py
>
> --
>
> Exit Code: 1
>
>
> Command Output (stdout):
>
> --
>
> lldb version 14.0.0 (g...@github.com:llvm/llvm-project.git revision 
> 6176fda3f992b5086302b3826aa0636135cc4cc0)
>
>   clang revision 6176fda3f992b5086302b3826aa0636135cc4cc0
>
>   llvm revision 6176fda3f992b5086302b3826aa0636135cc4cc0
>
> Skipping the following test categories: ['dsym', 'gmodules', 'debugserver', 
> 'objc']
>
>
> --
>
> Command Output (stderr):
>
> --
>
> UNSUPPORTED: LLDB 
> (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: 
> test_ref_and_ptr_dsym 
> (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does 
> not fall in any category of interest for this run)
>
> FAIL: LLDB 
> (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: 
> test_ref_and_ptr_dwarf 
> (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
>
> FAIL: LLDB 
> (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: 
> test_ref_and_ptr_dwo 
> (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
>
> UNSUPPORTED: LLDB 
> (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: 
> test_ref_and_ptr_gmodules 
> (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does 
> not fall in any category of interest for this run)
>
> UNSUPPORTED: LLDB 
> (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: 
> test_with_run_command_dsym 
> (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does 
> not fall in any category of interest for this run)
>
> FAIL: LLDB 
> (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: 
> test_with_run_command_dwarf 
>