[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Changes by Donald Stufft : -- pull_requests: +956 ___ Python tracker ___ ___

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: AFAIK many changes that have been made in Python 3 for the support of the cross compilation have not been backported to 2.7, including this one. -- ___ Python tracker

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

John Huang added the comment: The same issue in Python 2.7.12. Some extensions modules are not built if cross-compiling. -- nosy: +John Huang ___ Python tracker

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Changes by Xavier de Gaye : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Roundup Robot added the comment: New changeset 4b2679a06ace by Xavier de Gaye in branch '3.5': Issue #28444: Fix missing extensions modules when cross compiling. https://hg.python.org/cpython/rev/4b2679a06ace New changeset cddb7b2aba34 by Xavier de Gaye in branch '3.6': Issue #28444: Merge with

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Martin Panter added the comment: Looks good to me :) -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: New patch taking into account Martin last review and some updated comments. >> Why do you remove the code that loops over Modules/Setup? Maybe is it >> redundant with the other code for removing the already-built-in modules? > Yes because this is redundant,

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Changes by Xavier de Gaye : -- nosy: +akuchling ___ Python tracker ___ ___

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Martin Panter added the comment: Your second patch looks better, given my limited understanding of the scripts involved. :) I left one more suggestion though. -- ___ Python tracker

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: Thanks for reviewing the patch Martin. > Why do you remove the code that loops over Modules/Setup? Maybe is it > redundant with the other code for removing the already-built-in modules? Yes because this is redundant, maybe not the case when this was written

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Martin Panter added the comment: PS: I agree it would be good to add more documentation for cross-compiling. I tried to suggest something in an outdated patch once before; see the bottom of . --

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Martin Panter added the comment: Well, I am not really an expert on the setup.py stuff, but I will ask a question anyway that may help the review process: Why do you remove the code that loops over Modules/Setup? Maybe is it redundant with the other code for removing the already-built-in

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: Thanks for the reports. The next step is having the patch reviewed by one of the Python build machinery experts. Nosying Martin, Zachary and Matthias. -- nosy: +doko, martin.panter, zach.ware ___ Python tracker

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: The "removed_modules.patch" applied cleanly on the Python 3.6.0b2 sources downloaded from: https://www.python.org/ftp/python/3.6.0/Python-3.6.0b2.tar.xz and worked SUCCESSFULLY The environment is the same as in my last post Attached please find the build logs

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Changes by Benny K J : Added file: http://bugs.python.org/file45112/python_3_6_0b2_remove_modules.config.log ___ Python tracker ___

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Changes by Benny K J : Added file: http://bugs.python.org/file45110/python_3_5_2_remove_modules.config.log ___ Python tracker ___

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: Thank you for the response and really appreciate the quick patch. The "removed_modules.patch" applied cleanly on the Python sources downloaded from: https://www.python.org/ftp/python/3.5.2/Python-3.5.2.tar.xz and worked SUCCESSFULLY on HOST: Ubuntu 16.04 LTS

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: The attached patch fixes the problem and allows cross-building the extension modules independently of the configuration of the native interpreter that may have set some modules to be statically built. The patch also prints now the list of modules that are

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: I've confirmed that the suggested steps works fine on Ubuntu 14.04 as well as 16.04 versions standard procedure for cross-compilation: * Configure the python source, for example update Modules/Setup. * Build python natively out of the source tree [1]. *

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: Thank you for the support and the detailed explanation on why the native python is necessary. I'm now able to successfully compile all the extension modules on an Ubuntu Machine where there isn't any Python packages installed. I'll go ahead and compile the python

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: The cross-build uses a native python to run setup.py to build the extension modules, and to run 'python -S -m sysconfig --generate-posix-vars' and to byte compile the modules from the standard library. So you do need a native python. That is why you should

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: @Xavier de Gaye : Thanks a lot for helping. Unfortunately I didn't notice your last post (Date: 2016-10-15 11:13). Kindly ignore my last quires on the native Python. I'll follow your steps and report back the results asap. --

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: To eliminate the issues introduced by the native python on the Ubuntu 16.04 host, I've set up a new Ubuntu Host with all the native python packages removed. But now when I try to build the make fails saying: arm-linux-gnueabihf-gcc -shared

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Changes by Xavier de Gaye : -- versions: +Python 3.6, Python 3.7 ___ Python tracker ___

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: So the problem is that setup.py in build_extensions() does not build the extensions that have been already built statically into the native Ubuntu interpreter. The solution is to build first natively from source python3.5 and set the PATH environment

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: Thank you for your response Yes. There is a python 3.5 that is installed on the system (from apt-get) benny@whachamacallit:~$ which python3 /usr/bin/python3 benny@whachamacallit:~$ python3 Python 3.5.2 (default, Sep 10 2016, 08:21:44) [GCC 5.4.0 20160609] on

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: I assume from your logs that a native (not the cross-built one) python3.5 already exists and is on your PATH. What is the value of sys.builtin_module_names as given by this native python3.5 interpreter ? -- ___

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: please let me know if there is any additional steps I need to follow when building from the source downloaded from python.org -- ___ Python tracker

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: Thank you for the response. Yes I've downloaded the XZ compressed source from python.org Based on your comment I've double checked the MD5 with https://www.python.org/downloads/release/python-352/ benny@whachamacallit:~/Downloads$ md5sum Python-3.5.2.tar.xz

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: Your Setup file matches the one distributed with Python 3.5.2. Did you get the Python source from https://www.python.org/ ? -- ___ Python tracker

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: Attaching the modules/set-up for natively building on x86_64 -- Added file: http://bugs.python.org/file45100/setup_x86_x86 ___ Python tracker

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: Thank you for your response. Please note that I haven't touched the Modules/Setup.local or Modules/Setup or the global variable `disabled_module_list` in setup.py However I'm noticing that the math is commented in the Modules/Setup on both cross compiling on

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Chi Hsuan Yen added the comment: Wierd. Modules like 'math', 'select' or '_socket' are added unconditionally in detect_modules() of setup.py. Did you disable modules in Modules/Setup.local or Modules/Setup or via the global variable `disabled_module_list` in setup.py? -- nosy: +Chi

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: Could you please tell me if the decision to make the extension modules like math, socket, select etc taken in the configure step or is it left to the Makefile. Also could you please give me some hints on where to look for on how this decision is made (e.g file

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Benny K J added the comment: Thank you for the response. I'm not too sure how to set the root directory for arm headers and libraries in the configure command line. So I've tried the below, but still not all of the extensions modules was not build (e.g math, socket, select) export

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Xavier de Gaye added the comment: Please upload the output of make, this is where the error messages are printed when extension modules fail to build. Where is set the root directory for arm headers and libraries in the configure command line ? -- nosy: +xdegaye

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

Changes by Benny K J : -- type: -> compile error ___ Python tracker ___ ___

[issue28444] Missing extensions modules when cross compiling python 3.5.2 for arm on Linux

New submission from Benny K J: When cross compiling Python for ARM many of the extension modules are not build However when compiling for the native platform the extension modules are properly build. Cross Compilation Steps === CONFIG_SITE=config.site