Re: [lldb-dev] LLDB_PYTHON_HOME

[Adrian McCarthy via lldb-dev]

Setting extra Python3 strategy variables in the Cmake call didn't solve the
problem.  They may have been necessary, but they weren't sufficient.

In the end, I installed Python 3.8.2, and pointed everything at that.  For
reasons I haven't discovered, Cmake seems to choose that one over the
incomplete Python 3.7 distribution that's bundled with Visual Studio.  (The
obvious guess is that it's a higher version number, but the Cmake
documentation denies that.)

"Listen, strange women lying in ponds *Cmake* distributing swords *selecting
Python installations* is no basis for a system of government *cross-platform
build configuration.*"

On Mon, May 11, 2020 at 4:34 PM Adrian McCarthy  wrote:

> Ug.  After a rebase, this problem has again resurfaced for me.
>
> * PATH has only C:\Python36
> * cmake version 3.15.19101501-MSVC_2
> * cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON
> -DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1 
> -DPYTHON_HOME=C:\Python36
> -DLLDB_PYTHON_HOME=C:\Python36 -DPython3_ROOT_DIR=C:\Python36
> -DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja_dbg\bin\clang.exe
> ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
> -DLLVM_ENABLE_PROJECTS="clang;lld;lldb;debuginfo-tests"
>
> Yet when linking LLDB, it goes looking at the Python 3.7 installation that
> comes with Visual Studio.  That wouldn't be much of a problem unless you're
> trying to build debug, which I am.  The VS version, doesn't come with debug
> versions of the interpreter libraries, so the link fails:
>
> [1/7] Linking CXX shared library bin\liblldb.dll
> FAILED: bin/liblldb.dll lib/liblldb.lib
> cmd.exe /C "cd . && "C:\Program Files (x86)\Microsoft Visual
> Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"
> -E vs_link_dll --intdir=tools\lldb\source\API\CMakeFiles\liblldb.dir
> --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
> --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  --
> C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1424~1.283\bin\Hostx64\x64\link.exe
> /nologo @CMakeFiles\liblldb.rsp  /out:bin\liblldb.dll
> /implib:lib\liblldb.lib /pdb:bin\liblldb.pdb /dll /version:11.0
> /machine:x64 /debug /INCREMENTAL   && cd ."
> LINK Pass 1: command
> "C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1424~1.283\bin\Hostx64\x64\link.exe
> /nologo @CMakeFiles\liblldb.rsp /out:bin\liblldb.dll
> /implib:lib\liblldb.lib /pdb:bin\liblldb.pdb /dll /version:11.0
> /machine:x64 /debug /INCREMENTAL /MANIFEST
> /MANIFESTFILE:tools\lldb\source\API\CMakeFiles\liblldb.dir/intermediate.manifest
> tools\lldb\source\API\CMakeFiles\liblldb.dir/manifest.res" failed (exit
> code 1104) with the following output:
> LINK : fatal error LNK1104: cannot open file 'python37_d.lib'
> ninja: build stopped: subcommand failed.
>
> It looks like Cmake has added several more hints for finding Python
> .  I'm
> trying now with -DPython3_FIND_REGISTRY=LAST.
>
> I miss hermetic builds.
>
> On Thu, Feb 27, 2020 at 11:33 AM Adrian McCarthy 
> wrote:
>
>> >  This was all fixed in CMake 3.12.
>>
>> For some definitions of "all fixed." ;-)
>>
>> It seems weird to me that FindPython3 found the VS-distributed Python,
>> which isn't mentioned anywhere in the environment block, and that it chose
>> that one over the Python 3 installation that was in the path and that
>> included the interpreters and all of the corresponding libraries.
>>
>> >  With FindPython{2,3} you know you'll have a matching interpreter and
>> library.
>>
>> The build failure was that the Python distributed in VS does not have a
>> debug version of the library (python37_d.lib), so you don't actually get a
>> matching interpreter and library for debug builds.
>>
>> It's also not clear whether LLVM was consistently using the Python found
>> the old way.  My generated build.ninja file seemed to be using both
>> versions inconsistently to run lit tests and the like.
>>
>> Anyway, thanks for the explanations.  I've LGTMed your patch, which
>> should eliminate future surprises when something else smuggles yet another
>> version of Python onto a machine.
>>
>> On Thu, Feb 27, 2020 at 11:14 AM Jonas Devlieghere 
>> wrote:
>>
>>> So to make my previous explanation more concrete:
>>>
>>> On Thu, Feb 27, 2020 at 11:05 AM Jonas Devlieghere <
>>> jo...@devlieghere.com> wrote:
>>>


 On Thu, Feb 27, 2020 at 10:53 AM Adrian McCarthy 
 wrote:

> Thanks for the info.  Setting Python3_ROOT_DIR solves the problem.
>
> Looking at the cmake output from before setting Python3_ROOT_DIR,
> cmake looks for Python twice and finds it at the two different locations.
>
> Early on:
>
> -- Found PythonInterp: C:/Python36/python.exe (found version "3.6.8")
>

>>> ^ This is using the "old" (CMake < 3.12) way of finding the Python
>>> interpreter.
>>>
>>>

> Which looks good (modulo the incorrect slash direction).  But later:

Re: [lldb-dev] LLDB_PYTHON_HOME

[Adrian McCarthy via lldb-dev]

Ug.  After a rebase, this problem has again resurfaced for me.

* PATH has only C:\Python36
* cmake version 3.15.19101501-MSVC_2
* cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON
-DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1
-DPYTHON_HOME=C:\Python36
-DLLDB_PYTHON_HOME=C:\Python36 -DPython3_ROOT_DIR=C:\Python36
-DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja_dbg\bin\clang.exe
..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
-DLLVM_ENABLE_PROJECTS="clang;lld;lldb;debuginfo-tests"

Yet when linking LLDB, it goes looking at the Python 3.7 installation that
comes with Visual Studio.  That wouldn't be much of a problem unless you're
trying to build debug, which I am.  The VS version, doesn't come with debug
versions of the interpreter libraries, so the link fails:

[1/7] Linking CXX shared library bin\liblldb.dll
FAILED: bin/liblldb.dll lib/liblldb.lib
cmd.exe /C "cd . && "C:\Program Files (x86)\Microsoft Visual
Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"
-E vs_link_dll --intdir=tools\lldb\source\API\CMakeFiles\liblldb.dir
--rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
--mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  --
C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1424~1.283\bin\Hostx64\x64\link.exe
/nologo @CMakeFiles\liblldb.rsp  /out:bin\liblldb.dll
/implib:lib\liblldb.lib /pdb:bin\liblldb.pdb /dll /version:11.0
/machine:x64 /debug /INCREMENTAL   && cd ."
LINK Pass 1: command
"C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1424~1.283\bin\Hostx64\x64\link.exe
/nologo @CMakeFiles\liblldb.rsp /out:bin\liblldb.dll
/implib:lib\liblldb.lib /pdb:bin\liblldb.pdb /dll /version:11.0
/machine:x64 /debug /INCREMENTAL /MANIFEST
/MANIFESTFILE:tools\lldb\source\API\CMakeFiles\liblldb.dir/intermediate.manifest
tools\lldb\source\API\CMakeFiles\liblldb.dir/manifest.res" failed (exit
code 1104) with the following output:
LINK : fatal error LNK1104: cannot open file 'python37_d.lib'
ninja: build stopped: subcommand failed.

It looks like Cmake has added several more hints for finding Python
.  I'm
trying now with -DPython3_FIND_REGISTRY=LAST.

I miss hermetic builds.

On Thu, Feb 27, 2020 at 11:33 AM Adrian McCarthy 
wrote:

> >  This was all fixed in CMake 3.12.
>
> For some definitions of "all fixed." ;-)
>
> It seems weird to me that FindPython3 found the VS-distributed Python,
> which isn't mentioned anywhere in the environment block, and that it chose
> that one over the Python 3 installation that was in the path and that
> included the interpreters and all of the corresponding libraries.
>
> >  With FindPython{2,3} you know you'll have a matching interpreter and
> library.
>
> The build failure was that the Python distributed in VS does not have a
> debug version of the library (python37_d.lib), so you don't actually get a
> matching interpreter and library for debug builds.
>
> It's also not clear whether LLVM was consistently using the Python found
> the old way.  My generated build.ninja file seemed to be using both
> versions inconsistently to run lit tests and the like.
>
> Anyway, thanks for the explanations.  I've LGTMed your patch, which should
> eliminate future surprises when something else smuggles yet another version
> of Python onto a machine.
>
> On Thu, Feb 27, 2020 at 11:14 AM Jonas Devlieghere 
> wrote:
>
>> So to make my previous explanation more concrete:
>>
>> On Thu, Feb 27, 2020 at 11:05 AM Jonas Devlieghere 
>> wrote:
>>
>>>
>>>
>>> On Thu, Feb 27, 2020 at 10:53 AM Adrian McCarthy 
>>> wrote:
>>>
 Thanks for the info.  Setting Python3_ROOT_DIR solves the problem.

 Looking at the cmake output from before setting Python3_ROOT_DIR, cmake
 looks for Python twice and finds it at the two different locations.

 Early on:

 -- Found PythonInterp: C:/Python36/python.exe (found version "3.6.8")

>>>
>> ^ This is using the "old" (CMake < 3.12) way of finding the Python
>> interpreter.
>>
>>
>>>
 Which looks good (modulo the incorrect slash direction).  But later:

 -- Found Python3: C:/Program Files (x86)/Microsoft Visual
 Studio/Shared/Python37_64/python.exe (found version "3.7.5") found
 components:  Interpreter Development
 -- Found PythonInterpAndLibs: C:/Program Files (x86)/Microsoft Visual
 Studio/Shared/Python37_64/libs/python37.lib

>>>
>> ^ This is using the "new" (CMake > 3.12) way of finding the Python
>> interpreter and libraries.
>>
>>
>>>
 Which is where the discrepancy comes in.  Note that only C:\Python36 is
 in my PATH.

 It's frustrating that this keeps breaking.  Last time, I had to purge
 all but one Python installation from my machine to get it to make a
 consistent choice.  But I just upgraded to VS 2019, and it smuggled in its
 own version.

 So why are there two searches anyway?  And why do they have different
 

Re: [lldb-dev] LLDB_PYTHON_HOME

[Adrian McCarthy via lldb-dev]

>  This was all fixed in CMake 3.12.

For some definitions of "all fixed." ;-)

It seems weird to me that FindPython3 found the VS-distributed Python,
which isn't mentioned anywhere in the environment block, and that it chose
that one over the Python 3 installation that was in the path and that
included the interpreters and all of the corresponding libraries.

>  With FindPython{2,3} you know you'll have a matching interpreter and
library.

The build failure was that the Python distributed in VS does not have a
debug version of the library (python37_d.lib), so you don't actually get a
matching interpreter and library for debug builds.

It's also not clear whether LLVM was consistently using the Python found
the old way.  My generated build.ninja file seemed to be using both
versions inconsistently to run lit tests and the like.

Anyway, thanks for the explanations.  I've LGTMed your patch, which should
eliminate future surprises when something else smuggles yet another version
of Python onto a machine.

On Thu, Feb 27, 2020 at 11:14 AM Jonas Devlieghere 
wrote:

> So to make my previous explanation more concrete:
>
> On Thu, Feb 27, 2020 at 11:05 AM Jonas Devlieghere 
> wrote:
>
>>
>>
>> On Thu, Feb 27, 2020 at 10:53 AM Adrian McCarthy 
>> wrote:
>>
>>> Thanks for the info.  Setting Python3_ROOT_DIR solves the problem.
>>>
>>> Looking at the cmake output from before setting Python3_ROOT_DIR, cmake
>>> looks for Python twice and finds it at the two different locations.
>>>
>>> Early on:
>>>
>>> -- Found PythonInterp: C:/Python36/python.exe (found version "3.6.8")
>>>
>>
> ^ This is using the "old" (CMake < 3.12) way of finding the Python
> interpreter.
>
>
>>
>>> Which looks good (modulo the incorrect slash direction).  But later:
>>>
>>> -- Found Python3: C:/Program Files (x86)/Microsoft Visual
>>> Studio/Shared/Python37_64/python.exe (found version "3.7.5") found
>>> components:  Interpreter Development
>>> -- Found PythonInterpAndLibs: C:/Program Files (x86)/Microsoft Visual
>>> Studio/Shared/Python37_64/libs/python37.lib
>>>
>>
> ^ This is using the "new" (CMake > 3.12) way of finding the Python
> interpreter and libraries.
>
>
>>
>>> Which is where the discrepancy comes in.  Note that only C:\Python36 is
>>> in my PATH.
>>>
>>> It's frustrating that this keeps breaking.  Last time, I had to purge
>>> all but one Python installation from my machine to get it to make a
>>> consistent choice.  But I just upgraded to VS 2019, and it smuggled in its
>>> own version.
>>>
>>> So why are there two searches anyway?  And why do they have different
>>> algorithms that lead to different results?  (I'm not sure _how_ it ever
>>> found the Microsoft copy, since there's nothing in the process environment
>>> that points that way.)
>>>
>>
>> The reason there's two searches is because LLVM and LLDB have different
>> requirements. LLVM just needs a python interpreter to run some scripts.
>> LLDB on the other hand needs an interpreter and a matching Python library
>> to link against. Before CMake 3.12, finding the interpreter and the
>> libraries are two separate calls to find with no guarantees that they
>> match. This lead to all kinds of issues, where you're linking against one
>> version of Python and then trying to run the test suite with a totally
>> different interpreter. There were other problems on Windows, which meant
>> that we had our own hand-rolled implementation to find Python.
>>
>> This was all fixed in CMake 3.12. With FindPython{2,3} you know you'll
>> have a matching interpreter and library. It also fixed all the problems we
>> had to work around for Windows. Unfortunately, LLVM's minimum CMake version
>> is 3.4, so we can't use it yet. For LLDB on Windows we agreed that the
>> benefits of using FindPython3 are worth bumping the minimum required CMake
>> version (see lldb/CMakeLists.txt, line 2-4). Once LLVM moves to CMake 3.12
>> or later, all these problems should be fixed. We can then call FindPython3
>> once and rely on everything being consistent.
>>
>>
>>>
>>> On Thu, Feb 27, 2020 at 10:23 AM Jonas Devlieghere <
>>> jo...@devlieghere.com> wrote:
>>>
 Hey Adrian,

 Config.h gets generated by expanding the corresponding CMake variables.
 If you look at LLDBConfig.cmake, you can see that LLDB_PYTHON_HOME is
 computed from PYTHON_EXECUTABLE. The problem appears that somehow CMake
 ignored your specified PYTHON_HOME and decided to pick a different Python.
 I'm not sure why though, because I use a similar CMake invocation on
 Windows.

 > cmake ..\llvm-project\llvm -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo
 -DLLVM_ENABLE_PROJECTS="llvm;clang;lldb;lld" -DLLVM_ENABLE_ASSERTIONS=OFF
 -DLLVM_ENABLE_ZLIB=FALSE -DLLDB_ENABLE_PYTHON=TRUE
 -DPYTHON_HOME="C:/Program Files/Python36/"

 According to FindPython3 (
 https://cmake.org/cmake/help/v3.12/module/FindPython3.html), you can
 set Python3_ROOT_DIR as a hint. Can you give that a try? If 

Re: [lldb-dev] LLDB_PYTHON_HOME

[Jonas Devlieghere via lldb-dev]

So to make my previous explanation more concrete:

On Thu, Feb 27, 2020 at 11:05 AM Jonas Devlieghere 
wrote:

>
>
> On Thu, Feb 27, 2020 at 10:53 AM Adrian McCarthy 
> wrote:
>
>> Thanks for the info.  Setting Python3_ROOT_DIR solves the problem.
>>
>> Looking at the cmake output from before setting Python3_ROOT_DIR, cmake
>> looks for Python twice and finds it at the two different locations.
>>
>> Early on:
>>
>> -- Found PythonInterp: C:/Python36/python.exe (found version "3.6.8")
>>
>
^ This is using the "old" (CMake < 3.12) way of finding the Python
interpreter.


>
>> Which looks good (modulo the incorrect slash direction).  But later:
>>
>> -- Found Python3: C:/Program Files (x86)/Microsoft Visual
>> Studio/Shared/Python37_64/python.exe (found version "3.7.5") found
>> components:  Interpreter Development
>> -- Found PythonInterpAndLibs: C:/Program Files (x86)/Microsoft Visual
>> Studio/Shared/Python37_64/libs/python37.lib
>>
>
^ This is using the "new" (CMake > 3.12) way of finding the Python
interpreter and libraries.


>
>> Which is where the discrepancy comes in.  Note that only C:\Python36 is
>> in my PATH.
>>
>> It's frustrating that this keeps breaking.  Last time, I had to purge all
>> but one Python installation from my machine to get it to make a consistent
>> choice.  But I just upgraded to VS 2019, and it smuggled in its own version.
>>
>> So why are there two searches anyway?  And why do they have different
>> algorithms that lead to different results?  (I'm not sure _how_ it ever
>> found the Microsoft copy, since there's nothing in the process environment
>> that points that way.)
>>
>
> The reason there's two searches is because LLVM and LLDB have different
> requirements. LLVM just needs a python interpreter to run some scripts.
> LLDB on the other hand needs an interpreter and a matching Python library
> to link against. Before CMake 3.12, finding the interpreter and the
> libraries are two separate calls to find with no guarantees that they
> match. This lead to all kinds of issues, where you're linking against one
> version of Python and then trying to run the test suite with a totally
> different interpreter. There were other problems on Windows, which meant
> that we had our own hand-rolled implementation to find Python.
>
> This was all fixed in CMake 3.12. With FindPython{2,3} you know you'll
> have a matching interpreter and library. It also fixed all the problems we
> had to work around for Windows. Unfortunately, LLVM's minimum CMake version
> is 3.4, so we can't use it yet. For LLDB on Windows we agreed that the
> benefits of using FindPython3 are worth bumping the minimum required CMake
> version (see lldb/CMakeLists.txt, line 2-4). Once LLVM moves to CMake 3.12
> or later, all these problems should be fixed. We can then call FindPython3
> once and rely on everything being consistent.
>
>
>>
>> On Thu, Feb 27, 2020 at 10:23 AM Jonas Devlieghere 
>> wrote:
>>
>>> Hey Adrian,
>>>
>>> Config.h gets generated by expanding the corresponding CMake variables.
>>> If you look at LLDBConfig.cmake, you can see that LLDB_PYTHON_HOME is
>>> computed from PYTHON_EXECUTABLE. The problem appears that somehow CMake
>>> ignored your specified PYTHON_HOME and decided to pick a different Python.
>>> I'm not sure why though, because I use a similar CMake invocation on
>>> Windows.
>>>
>>> > cmake ..\llvm-project\llvm -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo
>>> -DLLVM_ENABLE_PROJECTS="llvm;clang;lldb;lld" -DLLVM_ENABLE_ASSERTIONS=OFF
>>> -DLLVM_ENABLE_ZLIB=FALSE -DLLDB_ENABLE_PYTHON=TRUE
>>> -DPYTHON_HOME="C:/Program Files/Python36/"
>>>
>>> According to FindPython3 (
>>> https://cmake.org/cmake/help/v3.12/module/FindPython3.html), you can
>>> set Python3_ROOT_DIR as a hint. Can you give that a try? If that works we
>>> should populate that variable from PYTHON_HOME in
>>> FindPythonInterpAndLibs.cmake.
>>>
>>> Cheers,
>>> Jonas
>>>
>>> On Thu, Feb 27, 2020 at 10:10 AM Adrian McCarthy via lldb-dev <
>>> lldb-dev@lists.llvm.org> wrote:
>>>
 Is there documentation on how lldb\include\lldb\host\config.h is
 generated?  I'm again having the problem of the config trying to point to
 the wrong Python installation.

 When I run cmake, I explicitly point PYTHON_HOME to C:\Python36 like
 this:

 cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON
 -DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1
 -DPYTHON_HOME=C:\Python36
 -DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja\bin\clang.exe
 ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
 -DLLVM_ENABLE_PROJECTS="clang;lld;lldb"

 But the generated Config.h contains:

 #define LLDB_PYTHON_HOME "C:/Program Files (x86)/Microsoft Visual
 Studio/Shared/Python37_64"


 And the mismatch causes my build to fail because it goes looking for
 python37_d.dll, which is apparently not part of the Microsoft distribution.
 

Re: [lldb-dev] LLDB_PYTHON_HOME

[Jonas Devlieghere via lldb-dev]

On Thu, Feb 27, 2020 at 10:53 AM Adrian McCarthy 
wrote:

> Thanks for the info.  Setting Python3_ROOT_DIR solves the problem.
>
> Looking at the cmake output from before setting Python3_ROOT_DIR, cmake
> looks for Python twice and finds it at the two different locations.
>
> Early on:
>
> -- Found PythonInterp: C:/Python36/python.exe (found version "3.6.8")
>
> Which looks good (modulo the incorrect slash direction).  But later:
>
> -- Found Python3: C:/Program Files (x86)/Microsoft Visual
> Studio/Shared/Python37_64/python.exe (found version "3.7.5") found
> components:  Interpreter Development
> -- Found PythonInterpAndLibs: C:/Program Files (x86)/Microsoft Visual
> Studio/Shared/Python37_64/libs/python37.lib
>
> Which is where the discrepancy comes in.  Note that only C:\Python36 is in
> my PATH.
>
> It's frustrating that this keeps breaking.  Last time, I had to purge all
> but one Python installation from my machine to get it to make a consistent
> choice.  But I just upgraded to VS 2019, and it smuggled in its own version.
>
> So why are there two searches anyway?  And why do they have different
> algorithms that lead to different results?  (I'm not sure _how_ it ever
> found the Microsoft copy, since there's nothing in the process environment
> that points that way.)
>

The reason there's two searches is because LLVM and LLDB have different
requirements. LLVM just needs a python interpreter to run some scripts.
LLDB on the other hand needs an interpreter and a matching Python library
to link against. Before CMake 3.12, finding the interpreter and the
libraries are two separate calls to find with no guarantees that they
match. This lead to all kinds of issues, where you're linking against one
version of Python and then trying to run the test suite with a totally
different interpreter. There were other problems on Windows, which meant
that we had our own hand-rolled implementation to find Python.

This was all fixed in CMake 3.12. With FindPython{2,3} you know you'll have
a matching interpreter and library. It also fixed all the problems we had
to work around for Windows. Unfortunately, LLVM's minimum CMake version is
3.4, so we can't use it yet. For LLDB on Windows we agreed that the
benefits of using FindPython3 are worth bumping the minimum required CMake
version (see lldb/CMakeLists.txt, line 2-4). Once LLVM moves to CMake 3.12
or later, all these problems should be fixed. We can then call FindPython3
once and rely on everything being consistent.


>
> On Thu, Feb 27, 2020 at 10:23 AM Jonas Devlieghere 
> wrote:
>
>> Hey Adrian,
>>
>> Config.h gets generated by expanding the corresponding CMake variables.
>> If you look at LLDBConfig.cmake, you can see that LLDB_PYTHON_HOME is
>> computed from PYTHON_EXECUTABLE. The problem appears that somehow CMake
>> ignored your specified PYTHON_HOME and decided to pick a different Python.
>> I'm not sure why though, because I use a similar CMake invocation on
>> Windows.
>>
>> > cmake ..\llvm-project\llvm -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo
>> -DLLVM_ENABLE_PROJECTS="llvm;clang;lldb;lld" -DLLVM_ENABLE_ASSERTIONS=OFF
>> -DLLVM_ENABLE_ZLIB=FALSE -DLLDB_ENABLE_PYTHON=TRUE
>> -DPYTHON_HOME="C:/Program Files/Python36/"
>>
>> According to FindPython3 (
>> https://cmake.org/cmake/help/v3.12/module/FindPython3.html), you can set
>> Python3_ROOT_DIR as a hint. Can you give that a try? If that works we
>> should populate that variable from PYTHON_HOME in
>> FindPythonInterpAndLibs.cmake.
>>
>> Cheers,
>> Jonas
>>
>> On Thu, Feb 27, 2020 at 10:10 AM Adrian McCarthy via lldb-dev <
>> lldb-dev@lists.llvm.org> wrote:
>>
>>> Is there documentation on how lldb\include\lldb\host\config.h is
>>> generated?  I'm again having the problem of the config trying to point to
>>> the wrong Python installation.
>>>
>>> When I run cmake, I explicitly point PYTHON_HOME to C:\Python36 like
>>> this:
>>>
>>> cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON
>>> -DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1
>>> -DPYTHON_HOME=C:\Python36
>>> -DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja\bin\clang.exe
>>> ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
>>> -DLLVM_ENABLE_PROJECTS="clang;lld;lldb"
>>>
>>> But the generated Config.h contains:
>>>
>>> #define LLDB_PYTHON_HOME "C:/Program Files (x86)/Microsoft Visual
>>> Studio/Shared/Python37_64"
>>>
>>>
>>> And the mismatch causes my build to fail because it goes looking for
>>> python37_d.dll, which is apparently not part of the Microsoft distribution.
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] LLDB_PYTHON_HOME

[Adrian McCarthy via lldb-dev]

Thanks for the info.  Setting Python3_ROOT_DIR solves the problem.

Looking at the cmake output from before setting Python3_ROOT_DIR, cmake
looks for Python twice and finds it at the two different locations.

Early on:

-- Found PythonInterp: C:/Python36/python.exe (found version "3.6.8")

Which looks good (modulo the incorrect slash direction).  But later:

-- Found Python3: C:/Program Files (x86)/Microsoft Visual
Studio/Shared/Python37_64/python.exe (found version "3.7.5") found
components:  Interpreter Development
-- Found PythonInterpAndLibs: C:/Program Files (x86)/Microsoft Visual
Studio/Shared/Python37_64/libs/python37.lib

Which is where the discrepancy comes in.  Note that only C:\Python36 is in
my PATH.

It's frustrating that this keeps breaking.  Last time, I had to purge all
but one Python installation from my machine to get it to make a consistent
choice.  But I just upgraded to VS 2019, and it smuggled in its own version.

So why are there two searches anyway?  And why do they have different
algorithms that lead to different results?  (I'm not sure _how_ it ever
found the Microsoft copy, since there's nothing in the process environment
that points that way.)

On Thu, Feb 27, 2020 at 10:23 AM Jonas Devlieghere 
wrote:

> Hey Adrian,
>
> Config.h gets generated by expanding the corresponding CMake variables. If
> you look at LLDBConfig.cmake, you can see that LLDB_PYTHON_HOME is computed
> from PYTHON_EXECUTABLE. The problem appears that somehow CMake ignored your
> specified PYTHON_HOME and decided to pick a different Python. I'm not sure
> why though, because I use a similar CMake invocation on Windows.
>
> > cmake ..\llvm-project\llvm -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo
> -DLLVM_ENABLE_PROJECTS="llvm;clang;lldb;lld" -DLLVM_ENABLE_ASSERTIONS=OFF
> -DLLVM_ENABLE_ZLIB=FALSE -DLLDB_ENABLE_PYTHON=TRUE
> -DPYTHON_HOME="C:/Program Files/Python36/"
>
> According to FindPython3 (
> https://cmake.org/cmake/help/v3.12/module/FindPython3.html), you can set
> Python3_ROOT_DIR as a hint. Can you give that a try? If that works we
> should populate that variable from PYTHON_HOME in
> FindPythonInterpAndLibs.cmake.
>
> Cheers,
> Jonas
>
> On Thu, Feb 27, 2020 at 10:10 AM Adrian McCarthy via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
>> Is there documentation on how lldb\include\lldb\host\config.h is
>> generated?  I'm again having the problem of the config trying to point to
>> the wrong Python installation.
>>
>> When I run cmake, I explicitly point PYTHON_HOME to C:\Python36 like this:
>>
>> cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON
>> -DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1
>> -DPYTHON_HOME=C:\Python36
>> -DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja\bin\clang.exe
>> ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
>> -DLLVM_ENABLE_PROJECTS="clang;lld;lldb"
>>
>> But the generated Config.h contains:
>>
>> #define LLDB_PYTHON_HOME "C:/Program Files (x86)/Microsoft Visual
>> Studio/Shared/Python37_64"
>>
>>
>> And the mismatch causes my build to fail because it goes looking for
>> python37_d.dll, which is apparently not part of the Microsoft distribution.
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] LLDB_PYTHON_HOME

[Jonas Devlieghere via lldb-dev]

Hey Adrian,

Config.h gets generated by expanding the corresponding CMake variables. If
you look at LLDBConfig.cmake, you can see that LLDB_PYTHON_HOME is computed
from PYTHON_EXECUTABLE. The problem appears that somehow CMake ignored your
specified PYTHON_HOME and decided to pick a different Python. I'm not sure
why though, because I use a similar CMake invocation on Windows.

> cmake ..\llvm-project\llvm -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo
-DLLVM_ENABLE_PROJECTS="llvm;clang;lldb;lld" -DLLVM_ENABLE_ASSERTIONS=OFF
-DLLVM_ENABLE_ZLIB=FALSE -DLLDB_ENABLE_PYTHON=TRUE
-DPYTHON_HOME="C:/Program Files/Python36/"

According to FindPython3 (
https://cmake.org/cmake/help/v3.12/module/FindPython3.html), you can set
Python3_ROOT_DIR as a hint. Can you give that a try? If that works we
should populate that variable from PYTHON_HOME in
FindPythonInterpAndLibs.cmake.

Cheers,
Jonas

On Thu, Feb 27, 2020 at 10:10 AM Adrian McCarthy via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> Is there documentation on how lldb\include\lldb\host\config.h is
> generated?  I'm again having the problem of the config trying to point to
> the wrong Python installation.
>
> When I run cmake, I explicitly point PYTHON_HOME to C:\Python36 like this:
>
> cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON
> -DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1
> -DPYTHON_HOME=C:\Python36
> -DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja\bin\clang.exe
> ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
> -DLLVM_ENABLE_PROJECTS="clang;lld;lldb"
>
> But the generated Config.h contains:
>
> #define LLDB_PYTHON_HOME "C:/Program Files (x86)/Microsoft Visual
> Studio/Shared/Python37_64"
>
>
> And the mismatch causes my build to fail because it goes looking for
> python37_d.dll, which is apparently not part of the Microsoft distribution.
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev