On Sat, Mar 12, 2016 at 6:38 AM, Martin Panter <vadmium...@gmail.com> wrote:
> Hi Russell. Sorry for the minor ~1 month delay in replying :) > > I have been doing some experimenting to see what is involved in > cross-compiling Python (Native host = Linux, target = Windows via > mingw and some patches). So I have a slightly better understanding of > the problem than before. > > On 16 February 2016 at 01:41, Russell Keith-Magee > <russ...@keith-magee.com> wrote: > > In order to build for a host platform, you have to compile for a local > > platform first - for example, to compile an iOS ARM64 binary, you have to > > compile for OS X x86_64 first. This gives you a local platform version of > > Python you can use when building the iOS version. > > > > Early in the Makefile, the variable PYTHON_FOR_BUILD is set. This points > at > > the CPU-local version of Python that can be invoked, which is used for > > module builds, and for compiling the standard library source code. This > is > > set by —host and —build flags to configure, plus the use of CC and > LDFLAGS > > environment variables to point at the compiler and libraries for the > > platform you’re compiling for, and a PATH variable that provides the > local > > platform’s version of Python. > > So far I haven’t succeeded with my Min GW cross build and am > temporarily giving up due to incompatibilities. But my attempts looked > a bit like this: > > make clean # Work around confusion with existing in-source build > mkdir native > (cd native/ && ../configure) > make -C native/ Parser/pgen > mkdir mingw > (cd mingw/ && ../configure --host=i486-mingw32 --build=x86) > make -C mingw/ PGEN=../native/Parser/pgen > > Actually it was not as smooth as the above commands, because pgen > tends to get overwritten with a cross-compiled version. Perhaps we > could add a PGEN_FOR_BUILD override, like HOSTPGEN in the patch used > at < > https://wayback.archive.org/web/20160131224915/http://randomsplat.com/id5-cross-compiling-python-for-embedded-linux.html > >. > > That might fix the pgen problem, but _freeze_importlib still remains. I suppose the same thing might be possible for _freeze_importlib as well… > There are two places where special handling is required: the compilation > and > > execution of the parser generator, and _freeze_importlib. In both cases, > the > > tool needs to be compiled for the local platform, and then executed. > > Historically (i.e., Py3.4 and earlier), this has been done by spawning a > > child MAKE to compile the tool; this runs the compilation phase with the > > local CPU environment, before returning to the master makefile and > executing > > the tool. By spawning the child MAKE, you get a “clean” environment, so > the > > tool is built natively. However, as I understand it, it causes problems > with > > parallel builds due to race conditions on build rules. The change in > > Python3.5 simplified the rule so that child MAKE calls weren’t used, but > > that means that pgen and _freeze_importlib are compiled for ARM64, so > they > > won’t run on the local platform. > > You suggest that the child Make command happened to compile pgen etc > natively, rather than with the cross compiler. But my understanding is > that when you invoke $(MAKE), all the environment variables, configure > settings, etc, including the cross compiler, would be inherited by the > child. > > Would it be more correct to say instead that in 3.4 you did a separate > native build step, precompiling pgen and _freeze_importlib for the > native build host? Then you hoped that the child Make was _not_ > invoked in the cross-compilation stage and your precompiled > executables would not be rebuilt? > Yes - as far as I can make out (with my admittedly hazy understanding), that appears to be what is going on. Although it’s not that I “hoped” the build wouldn’t happen on the second pass - it was the behavior that was previously relied, and on was altered. > > As best as I can work out, the solution is to: > > > > (1) Include the parser generator and _freeze_importlib as part of the > > artefacts of local platform. That way, you could use the version of pgen > and > > _freeze_importlib that was compiled as part of the local platform build. > At > > present, pgen and _freeze_importlib are used during the build process, > but > > aren’t preserved at the end of the build; or > > I don’t understand. After I run Make, it looks like I get working > executables leftover at Programs/_freeze_importlib and Parser/pgen. Do > you mean to install these programs with “make install” or something? > Making them part of the installable artefacts would be one option, but they don’t have to be installed, just preserved. For example, as a nasty hack, I’ve been able to use this approach to get the build working for 3.5. After the native build, I copy _freeze_importlib to a “safe” location. I then copy it back into place prior to the target build. It works, but it’s in no way suitable for a final build. > > (2) Include some concept of the “local compiler” in the build process, > which > > can be used to compile pgen and _freeze_importlib; or > > On the surface solution (2) sounds like the ideal fix. But I guess the > local native compiler might also require a separate set of CPPFLAGS, > pyconfig.h settings etc. In other words it is sounding like a whole > separate “configure” run. I am thinking it might be simplest to just > require this native “configure” run to be done manually. > That run is going to happen anyway, since you have to compile and build for the native platform. Yours, Russ Magee %-)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
