Sorry, hit send too soon. The big thing that will change is people who do:
#include "lldb-python.h" and #if !defined(LLDB_DISABLE_PYTHON) There are currently a lot of instances of this in the code. The path forward for this would be to gradually change #if defined(LLDB_DISABLE_PYTHON) to simply checking if GetScriptInterpreter() returns null. I scanned about 10% of the occurrences of this and there's nothing in any of them that are python specific. I don't know how much of this kind of thing is out of tree, but either way it's a purely mechanical change. If there are places that are actually relying on it being a ScriptInterpreterPython, then that's an error in the code, because as you said it was designed to be pluggable. On Wed, Feb 25, 2015 at 1:33 PM Zachary Turner <ztur...@google.com> wrote: > Also, I don't think the fixups necessary for out of tree code associated > with separating the library will be anything larger than what would need to > be done for someone adding another language anyway. Some header and source > files will move around, and the build dependency structure will change. > The big thing this will break is people who do > > #include "lldb/lldb-python.h" > > On Wed, Feb 25, 2015 at 11:47 AM Zachary Turner <ztur...@google.com> > wrote: > >> Most Python stuff is in ScriptInterpreterPython (see >> Include/Utility/PythonPointer.h for an example of code that isn't). But >> ScriptInterpreterPython is just compiled straight into Interpreter, instead >> of into its own library. That's the part that isn't working. If someone >> wants to add support for another language, it would be nice to do it in a >> way that's upstreamable. The way to do that in my opinion isn't to just >> keep sticking new implementations into Interpreter, it's to separate the >> python interpreter code into its own library that can be conditionally >> linked in or not. >> >> If we separate the library, then there is a little bit of upfront pain to >> fix out of tree code, and after that initial hurdle is overcome, everything >> is back to normal. If we fragment the testing environments into one >> subset of people that only cares about python 2.7 and another subset of >> people that only cares about 3.5, then there will be contiuous burden on >> everyone to not break the other person. So I don't think we should do the >> python 3.5 solution unless *everyone* is on board to start moving to >> 3.5. Anwyay, python 3.5 isn't even released yet, nor is the C++ compiler. >> The compiler is targeting summer of this year and python 3.5 is targeting >> september of this year (I believe). >> >> >> On Wed, Feb 25, 2015 at 11:31 AM Greg Clayton <gclay...@apple.com> wrote: >> >>> We had planned on python being able to be replaced by another language >>> and abstracted all python stuff into the pure virtual ScriptInterpreter and >>> the one and only scripting subclass ScriptInterpreterPython. What part of >>> this abstraction isn't working for people? I don't see the need for heroic >>> efforts that are proposed without a specific reason. All python specific >>> stuff is in ScriptInterpreterPython and a few swig class. Adding support >>> for another language should be as simple as configuring swig and making it >>> generate all of the classes and subclassing ScriptInterpreter. Python can >>> be disabled for people that don't want/need python support, as any language >>> that we add should be. >>> >>> So I like solution 2 where only Windows uses Python 3.5 and we allow the >>> test suite to be updated so that it can run with 3.5, yet it still runs on >>> 2.7. >>> >>> On Feb 24, 2015, at 8:56 PM, Zachary Turner <ztur...@google.com> wrote: >>> > >>> > A little background: The single biggest painpoint for working with >>> LLDB on Windows currently is Python. There is a long documented history of >>> the problems with python extension modules on Windows. Part of it is >>> Microsoft's fault, part of it is Python's fault (PEP 384 attempts to solve >>> this, but it appears stalled), but the end result is that it's really >>> terrible and there's nothing anyone can do about it. >>> > >>> > The implications of this for LLDB on Windows are the following: >>> > 1) Anyone building LLDB on Windows needs to compile Python from source >>> > 2) Even after doing so, configuring the LLDB build is difficult and >>> requires a lot of manual steps >>> > 3) You can't use any other extension modules in your custom built >>> version of python, including any of the over 50,000 from PyPI, unless you >>> also build them from source, which isn't even always possible. >>> > >>> > If you want to be compatible with the binary release of Python and >>> interoperate with the version of Python that probably 99% of Windows people >>> have on their machine, you must compile your extension module with a very >>> old version of the compiler. So old, in fact, that it's almost >>> end-of-lifed. If it weren't for Python actually, it would have been >>> end-of-lifed already, and Microsoft ships a free version of this toolchain >>> for the express purpose of compiling python extension modules. >>> > >>> > I've been thinking about this for many months, and I believe I finally >>> have a solution (2 solutions actually, although I think we should do both. >>> I'll get to that later). Both will probably be painful, but in the end for >>> the better on all platforms. >>> > >>> > Solution 1: Decouple embedded python code from LLDB >>> > Rationale: Currently calls to embedded python code live in a few >>> different places. Primarily in source/Interpreter (e.g. >>> ScriptInterpreterPython) and the generated SWIG code, but there's a few >>> utility headers scattered around. >>> > >>> > I'm proposing moving all of this code to a single shared library with >>> nothing else and creating some heavy restrictions on what kind of code can >>> go into this library, primarily to satisfy the requirement that it be >>> compilable with the old version of the compiler in question. These >>> restrictions would be: >>> > 1) Can't use fancy C++. It's hard to say exactly what subset of C++ >>> we could use here, but a good rough approximation is to say C++98. >>> > 2) Can't depend on any LLVM headers or even other LLDB libraries. >>> Other LLDB projects could depend on it, but it has to stand alone to >>> guarantee that it doesn't pick up C++ that won't compile under the old MSVC >>> compiler. >>> > >>> > I understand that the first restriction is very annoying, but it would >>> only be imposed upon a very small amount of source code. About 2 or 3 >>> source files. >>> > >>> > The second restriction, and the decoupling as a whole will probably >>> cause some pain and breakage for out-of-tree code, but I believe it's a >>> positive in the long run. In particular, it opens the door to replacing >>> the embedded interpreter with that of a different language. By separating >>> the code in this fashion, all one has to do is write a new module for their >>> own language and initialize it instead of ScriptInterpreterPython. I've >>> already seen people asking on the list about generating bindings for other >>> languages and replacing the interpreter, and i believe a separation of this >>> kind is a pre-requisite anyway. >>> > >>> > Solution 2: Upgrade to Python 3.5 >>> > Rationale: Hopefully I didn't send you running for the hills. This is >>> going to have to happen someday anyway. Python 2.7 end of life is set to >>> 2020. Seems like a long time, but it'll be here before we know it, and if >>> we aren't prepared, it's going to be a world of hurt. >>> > >>> > Why does this help us on Windows? Visual Studio 2015, which releases >>> sometime this year, is finally set to have a stable ABI for it's CRT. This >>> means that any application compiled against VS2015 or later will be forward >>> compatible with future versions of the CRT forever. Python 3.5 is not yet >>> released, but the current proposal is for Python 3.5 to ship its binary >>> release compiled with VC++ 2015, effectively killing this problem. >>> > >>> > I understand that there is a lot of out-of-tree code that is written >>> against Python 2.7. I would encourage the people who own this code to >>> begin thinking about migrating to Python 3 soon. In the meantime, I >>> believe we can begin to address this in-tree in 3 main phases. >>> > >>> > 1) Port the test suite to Python 3.5, using a subset of Python 3.5 >>> that is also compatible with 2.7. This ensures no out of tree code is >>> broken. >>> > 2) Upgrading ScriptInterpreterPython with preprocessor flags to select >>> whether you want to link against Python 2.7 and 3.5, and expose these >>> options from CMake and Xcode so you can choose what you link against. >>> > 3) Making multiple buildbots with different configurations and >>> different versions of Python to get better code coverage to ensure that >>> tests work under both language versions. >>> > >>> > I realize that the impact of both of these changes is high, but we >>> have a very strong desire to solve this problem on Windows and so we need >>> to push some kind of solution forward. I think both solutions actually >>> contribute to the long term benefit of LLDB as a whole, and so I think it's >>> worth seriously considering both of these and trying to come up with a path >>> forward. >>> > _______________________________________________ >>> > lldb-dev mailing list >>> > lldb-dev@cs.uiuc.edu >>> > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev >>> >>>
_______________________________________________ lldb-dev mailing list lldb-dev@cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
