Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Vadim Chugunov via lldb-dev]

FWIW, Python provides stable ABI 
for a subset of their API.

I've actually managed to create a version of LLDB that is Python-optional
and Python version-agnostic (for versions 3.3 up).   Though given the
number of hoops I had to jump through to get there, I cannot recommend this
approach for mainline LLDB.


On Fri, Nov 22, 2019 at 9:39 AM Adrian McCarthy via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> Yes, I think it's pretty reasonable to expect a specific version of
> Python, especially if the pre-built Python DLLs for Windows are still built
> with versions as old as VS 2013.  Once you get to VS 2015 or 2017, I think
> the compatibility improves.
>
> Perhaps the best thing for the pre-built LLDB is to include the right
> Python DLL in the distro, assuming the licensing allows that.
>
> The more sophisticated solutions are probably more work than is justified
> by the value.
>
> On Fri, Nov 22, 2019 at 8:29 AM Ted Woodward  wrote:
>
>>
>>
>> > > * Dynamically load any supported Python DLL if/when needed
>> > That might be tricky since the different versions are not binary
>> compatible in
>> > general. But it is possible, as Apple folks have shown, though it
>> amounts to
>> > building multiple copies of ScriptInterpreterPython and then choosing
>> the
>> > right one at runtime.
>>
>> It's not just the python dll; it's the modules directory as well. My
>> experiments with different versions of Python on Linux led me to just ship
>> the right python with our distribution.
>>
>> I saw things like building with 2.7.6 but using the 2.7.3 library/modules
>> (and vice versa) would crash, and building with 2.7.6 but running with
>> 2.6.x seemed to be OK, mostly. On Windows, I had crashes when loading
>> Python 2.7.8 from python.org (built with VS 2008) in lldb built with VS
>> 2013, so you have to think about other library dependencies too.
>>
>> My conclusion - you MUST use the same python that lldb was built with.
>> Period.
>>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Adrian McCarthy via lldb-dev]

Yes, I think it's pretty reasonable to expect a specific version of Python,
especially if the pre-built Python DLLs for Windows are still built with
versions as old as VS 2013.  Once you get to VS 2015 or 2017, I think the
compatibility improves.

Perhaps the best thing for the pre-built LLDB is to include the right
Python DLL in the distro, assuming the licensing allows that.

The more sophisticated solutions are probably more work than is justified
by the value.

On Fri, Nov 22, 2019 at 8:29 AM Ted Woodward  wrote:

>
>
> > > * Dynamically load any supported Python DLL if/when needed
> > That might be tricky since the different versions are not binary
> compatible in
> > general. But it is possible, as Apple folks have shown, though it
> amounts to
> > building multiple copies of ScriptInterpreterPython and then choosing the
> > right one at runtime.
>
> It's not just the python dll; it's the modules directory as well. My
> experiments with different versions of Python on Linux led me to just ship
> the right python with our distribution.
>
> I saw things like building with 2.7.6 but using the 2.7.3 library/modules
> (and vice versa) would crash, and building with 2.7.6 but running with
> 2.6.x seemed to be OK, mostly. On Windows, I had crashes when loading
> Python 2.7.8 from python.org (built with VS 2008) in lldb built with VS
> 2013, so you have to think about other library dependencies too.
>
> My conclusion - you MUST use the same python that lldb was built with.
> Period.
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Ted Woodward via lldb-dev]

> > * Dynamically load any supported Python DLL if/when needed
> That might be tricky since the different versions are not binary compatible in
> general. But it is possible, as Apple folks have shown, though it amounts to
> building multiple copies of ScriptInterpreterPython and then choosing the
> right one at runtime.

It's not just the python dll; it's the modules directory as well. My 
experiments with different versions of Python on Linux led me to just ship the 
right python with our distribution.

I saw things like building with 2.7.6 but using the 2.7.3 library/modules (and 
vice versa) would crash, and building with 2.7.6 but running with 2.6.x seemed 
to be OK, mostly. On Windows, I had crashes when loading Python 2.7.8 from 
python.org (built with VS 2008) in lldb built with VS 2013, so you have to 
think about other library dependencies too.

My conclusion - you MUST use the same python that lldb was built with. Period.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Pavel Labath via lldb-dev]

On 22/11/2019 01:26, Adrian McCarthy wrote:
Yes, that sounds plausible, but I don't recall for sure.  I think 
there's a build-time option to say you don't want Python at all, but I 
can't remember if there was a load-as-needed option.


I'm pretty sure we have never had explicit support for anything like 
this. The only way I can see this happening is if this fell out 
"accidentally" out of our lazy python initialization and some windows 
dll behavior, but I don't think windows has anything like that. (At 
least on linux, I know that lazy binding can delay library mismatch 
errors until the first time you call some function, but they won't help 
you if the library is not there at all.)


I think the more likely scenario is that python was disabled in the 
previous "official" releases, and that some of the python changes 
enabled it.




In any event, the current situation is what it is.  What's feasible and 
worth doing for the future?


* Hard dependency (as we have right now)
I'm fine with that. We could add a note on the website that one needs to 
have python installed for this to work. Or we could disable python for 
the official releases.


* Dynamically load Python DLL on startup if it exists, or provide a 
better error message with instructions
* Dynamically load Python DLL on startup if it exists, otherwise disable 
Python-dependent features

* Dynamically load a specific version of the Python DLL if/when needed
All of these seem fine too, if anyone is willing to invest the time to 
make it work (it shouldn't be _that_ hard). Since python is pretty 
compartmentalized nowadays, it shouldn't relatively easy to disable 
python features at runtime instead of just exiting.


The main question I have here is should we dlopen python.dll, or some 
lldb wrapper of it (the entire "script interpreter" plugin).


I'd also like to note that this isn't the only external dependency of 
lldb. (Well, it might be on windows..) Lldb can use libcurses, libedit, 
libz, etc. Libedit is fairly likely to not be present on a random linux 
system. libcurses are almost certainly there, but it's not always a 
compatible version, etc.



* Dynamically load any supported Python DLL if/when needed
That might be tricky since the different versions are not binary 
compatible in general. But it is possible, as Apple folks have shown, 
though it amounts to building multiple copies of ScriptInterpreterPython 
and then choosing the right one at runtime.


pl
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Adrian McCarthy via lldb-dev]

Yes, that sounds plausible, but I don't recall for sure.  I think there's a
build-time option to say you don't want Python at all, but I can't remember
if there was a load-as-needed option.

In any event, the current situation is what it is.  What's feasible and
worth doing for the future?

* Hard dependency (as we have right now)
* Dynamically load Python DLL on startup if it exists, or provide a better
error message with instructions
* Dynamically load Python DLL on startup if it exists, otherwise disable
Python-dependent features
* Dynamically load a specific version of the Python DLL if/when needed
* Dynamically load any supported Python DLL if/when needed





On Thu, Nov 21, 2019 at 1:57 PM Leonard Mosescu  wrote:

> What kind of behavior did you expect?
>
> I could be wrong, but I thought that previous versions of LLDB would use
> LoadLibrary() instead of linking to the import library?
>
> --
> *From:* Pavel Labath 
> *Sent:* Wednesday, November 20, 2019 11:32 PM
> *To:* Adrian McCarthy ; Leonard Mosescu <
> lmose...@nvidia.com>
> *Cc:* lldb-dev@lists.llvm.org 
> *Subject:* Re: [lldb-dev] The pre-built Windows LLDB binary has a
> dependency on an external python36.dll?
>
> On 20/11/2019 23:53, Adrian McCarthy via lldb-dev wrote:
> > That said, I didn't expect an explicit dependency on python36.dll.
>
> What kind of behavior did you expect?
>
> pl
> --
> This email message is for the sole use of the intended recipient(s) and
> may contain confidential information.  Any unauthorized review, use,
> disclosure or distribution is prohibited.  If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies
> of the original message.
> --
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Leonard Mosescu via lldb-dev]

What kind of behavior did you expect?
I could be wrong, but I thought that previous versions of LLDB would use 
LoadLibrary() instead of linking to the import library?


From: Pavel Labath 
Sent: Wednesday, November 20, 2019 11:32 PM
To: Adrian McCarthy ; Leonard Mosescu 
Cc: lldb-dev@lists.llvm.org 
Subject: Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on 
an external python36.dll?

On 20/11/2019 23:53, Adrian McCarthy via lldb-dev wrote:
> That said, I didn't expect an explicit dependency on python36.dll.

What kind of behavior did you expect?

pl

---
This email message is for the sole use of the intended recipient(s) and may 
contain
confidential information.  Any unauthorized review, use, disclosure or 
distribution
is prohibited.  If you are not the intended recipient, please contact the 
sender by
reply email and destroy all copies of the original message.
---
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Pavel Labath via lldb-dev]

On 20/11/2019 23:53, Adrian McCarthy via lldb-dev wrote:

That said, I didn't expect an explicit dependency on python36.dll.


What kind of behavior did you expect?

pl
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Adrian McCarthy via lldb-dev]

Here are some possibly related reviews.  Note that some of these were
abandoned, but I'm including them because the comments might give some
context.

https://reviews.llvm.org/D69684 
https://reviews.llvm.org/D69931
https://reviews.llvm.org/D67942
https://reviews.llvm.org/D69535
https://reviews.llvm.org/D68613
https://reviews.llvm.org/D68442


https://reviews.llvm.org/D68613

On Wed, Nov 20, 2019 at 2:53 PM Adrian McCarthy  wrote:

> There has been a lot of churn in the build process for Python on Windows
> over the past couple months.
>
> Older versions included a pre-built Python DLL on Windows because of ABI
> compatibility.  That issue is resolved, though, and I thought that was
> already over by version 7 or earlier.
>
> Because other compatibility issues, it was decided that the Windows
> version could depend on 3.6+ while non-Windows versions might still rely on
> slightly older versions of Python.
>
> That said, I didn't expect an explicit dependency on python36.dll.  Please
> file a bug.  I'll look through the revision history and see if I can loop
> in the folks who've been making changes in this area.  I suspect it was an
> unintended side effect.
>
> On Wed, Nov 20, 2019 at 1:59 PM Leonard Mosescu via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
>> I just installed the pre-built LLVM9 binaries on a Windows machine and I
>> noticed that LLDB.exe imports from python36.dll. Was this an intentional
>> change from LLVM8? (which doesn't depend on external python DLLs)
>>
>> Trying to use the LLDB that comes with LLVM9, you'd get a pop-up
>> complaining that python36.dll was not found (unless you happen to have on
>> in your PATH) and LLDB fails to start.
>>
>> Thanks,
>> Leonard.
>>
>> --
>> This email message is for the sole use of the intended recipient(s) and
>> may contain confidential information.  Any unauthorized review, use,
>> disclosure or distribution is prohibited.  If you are not the intended
>> recipient, please contact the sender by reply email and destroy all copies
>> of the original message.
>> --
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Adrian McCarthy via lldb-dev]

There has been a lot of churn in the build process for Python on Windows
over the past couple months.

Older versions included a pre-built Python DLL on Windows because of ABI
compatibility.  That issue is resolved, though, and I thought that was
already over by version 7 or earlier.

Because other compatibility issues, it was decided that the Windows version
could depend on 3.6+ while non-Windows versions might still rely on
slightly older versions of Python.

That said, I didn't expect an explicit dependency on python36.dll.  Please
file a bug.  I'll look through the revision history and see if I can loop
in the folks who've been making changes in this area.  I suspect it was an
unintended side effect.

On Wed, Nov 20, 2019 at 1:59 PM Leonard Mosescu via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> I just installed the pre-built LLVM9 binaries on a Windows machine and I
> noticed that LLDB.exe imports from python36.dll. Was this an intentional
> change from LLVM8? (which doesn't depend on external python DLLs)
>
> Trying to use the LLDB that comes with LLVM9, you'd get a pop-up
> complaining that python36.dll was not found (unless you happen to have on
> in your PATH) and LLDB fails to start.
>
> Thanks,
> Leonard.
>
> --
> This email message is for the sole use of the intended recipient(s) and
> may contain confidential information.  Any unauthorized review, use,
> disclosure or distribution is prohibited.  If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies
> of the original message.
> --
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] The pre-built Windows LLDB binary has a dependency on an external python36.dll?

[Leonard Mosescu via lldb-dev]

I just installed the pre-built LLVM9 binaries on a Windows machine and I 
noticed that LLDB.exe imports from python36.dll. Was this an intentional change 
from LLVM8? (which doesn't depend on external python DLLs)

Trying to use the LLDB that comes with LLVM9, you'd get a pop-up complaining 
that python36.dll was not found (unless you happen to have on in your PATH) and 
LLDB fails to start.

Thanks,
Leonard.


---
This email message is for the sole use of the intended recipient(s) and may 
contain
confidential information.  Any unauthorized review, use, disclosure or 
distribution
is prohibited.  If you are not the intended recipient, please contact the 
sender by
reply email and destroy all copies of the original message.
---
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev