Re: [OE-core] [CONSOLIDATED PULL 17/17] python: skip setup.py 'import check' when cross-compiling
Missing PR bump!
Op 8 nov. 2011, om 20:22 heeft Saul Wold het volgende geschreven:
From: Tom Zanussi tom.zanu...@intel.com
build_extension() in setup.py, as part of the build process, does an
'import check' on the built extension. The import check in turn
dlopen()'s the shared library associated with the extension, which
isn't something that makes sense if that library was cross-compiled
for a different architecture.
This was noticed with an x86_64 target that was compiled with avx
support, because it caused 'illegal instruction' exceptions:
| /bin/sh: line 1: 14575 Illegal instruction ... -E ./setup.py -q build
For other target architectures, it doesn't necessarily cause illegal
instruction exceptions, but still fails. For example, on arm, the
failure pathway causes this warning:
*** WARNING: renaming cmath since importing it failed: .../cmath.so:
wrong ELF class: ELFCLASS32
This patch to setup.py and the associated recipe changes allow the
whole 'import check' logic to be skipped when cross-compiling.
Signed-off-by: Tom Zanussi tom.zanu...@intel.com
---
.../python/setup_py_skip_cross_import_check.patch | 27
meta/recipes-devtools/python/python_2.7.2.bb |5 +++
2 files changed, 32 insertions(+), 0 deletions(-)
create mode 100644
meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
diff --git
a/meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
b/meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
new file mode 100644
index 000..6ccdb94
--- /dev/null
+++
b/meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
@@ -0,0 +1,27 @@
+This patch skips over the 'import check' setup.py does when building
+extensions. This generally won't work when cross-compiling.
+
+Upstream-Status: Inappropriate [embedded-specific]
+
+Signed-off-by: Tom Zanussi tom.zanu...@intel.com
+
+Index: Python-2.7.2/setup.py
+===
+--- Python-2.7.2.orig/setup.py 2011-11-04 16:46:34.553796410 -0500
Python-2.7.2/setup.py2011-11-04 16:59:49.692802313 -0500
+@@ -287,6 +287,15 @@
+ (ext.name, sys.exc_info()[1]))
+ self.failed.append(ext.name)
+ return
++
++# If we're cross-compiling, we want to skip the import check
++# i.e. we shouldn't be dynamically loading target shared libs
++if os.environ.get('CROSS_COMPILE') is not None:
++self.announce(
++'WARNING: skipping import check for cross-compiled %s' %
++ext.name)
++return
++
+ # Workaround for Mac OS X: The Carbon-based modules cannot be
+ # reliably imported into a command-line Python
+ if 'Carbon' in ext.extra_link_args:
diff --git a/meta/recipes-devtools/python/python_2.7.2.bb
b/meta/recipes-devtools/python/python_2.7.2.bb
index bb088a4..360e64e 100644
--- a/meta/recipes-devtools/python/python_2.7.2.bb
+++ b/meta/recipes-devtools/python/python_2.7.2.bb
@@ -18,6 +18,7 @@ SRC_URI += \
file://multilib.patch \
file://cgi_py.patch \
file://remove_sqlite_rpath.patch \
+ file://setup_py_skip_cross_import_check.patch \
S = ${WORKDIR}/Python-${PV}
@@ -57,6 +58,8 @@ do_compile() {
# then call do_install twice we get Makefile.orig == Makefile.sysroot
install -m 0644 Makefile Makefile.sysroot
+ export CROSS_COMPILE=${TARGET_PREFIX}
+
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
STAGING_LIBDIR=${STAGING_LIBDIR} \
@@ -78,6 +81,8 @@ do_install() {
# make install needs the original Makefile, or otherwise the inclues
would
# go to ${D}${STAGING...}/...
install -m 0644 Makefile.orig Makefile
+
+ export CROSS_COMPILE=${TARGET_PREFIX}
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
--
1.7.6.4
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [CONSOLIDATED PULL 17/17] python: skip setup.py 'import check' when cross-compiling
On Wed, 2011-11-09 at 02:34 -0800, Cui, Dexuan wrote:
How I wish I could notice the patch this morning so I could save 1 day!
??? Can you please explain what you mean?
Anyway, thanks TomZ for the patch! :-)
You're welcome, I guess!
Tom
Thanks,
-- Dexuan
-Original Message-
From: openembedded-core-boun...@lists.openembedded.org
[mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf
Of Saul Wold
Sent: Wednesday, November 09, 2011 3:23 AM
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [CONSOLIDATED PULL 17/17] python: skip setup.py
'import check' when cross-compiling
From: Tom Zanussi tom.zanu...@intel.com
build_extension() in setup.py, as part of the build process, does an
'import check' on the built extension. The import check in turn
dlopen()'s the shared library associated with the extension, which
isn't something that makes sense if that library was cross-compiled
for a different architecture.
This was noticed with an x86_64 target that was compiled with avx
support, because it caused 'illegal instruction' exceptions:
| /bin/sh: line 1: 14575 Illegal instruction ... -E ./setup.py -q build
For other target architectures, it doesn't necessarily cause illegal
instruction exceptions, but still fails. For example, on arm, the
failure pathway causes this warning:
*** WARNING: renaming cmath since importing it failed: .../cmath.so:
wrong ELF class: ELFCLASS32
This patch to setup.py and the associated recipe changes allow the
whole 'import check' logic to be skipped when cross-compiling.
Signed-off-by: Tom Zanussi tom.zanu...@intel.com
---
.../python/setup_py_skip_cross_import_check.patch | 27
meta/recipes-devtools/python/python_2.7.2.bb |5 +++
2 files changed, 32 insertions(+), 0 deletions(-)
create mode 100644
meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.
patch
diff --git
a/meta/recipes-devtools/python/python/setup_py_skip_cross_import_chec
k.patch
b/meta/recipes-devtools/python/python/setup_py_skip_cross_import_chec
k.patch
new file mode 100644
index 000..6ccdb94
--- /dev/null
+++
b/meta/recipes-devtools/python/python/setup_py_skip_cross_import_chec
k.patch
@@ -0,0 +1,27 @@
+This patch skips over the 'import check' setup.py does when building
+extensions. This generally won't work when cross-compiling.
+
+Upstream-Status: Inappropriate [embedded-specific]
+
+Signed-off-by: Tom Zanussi tom.zanu...@intel.com
+
+Index: Python-2.7.2/setup.py
+==
=
+--- Python-2.7.2.orig/setup.py 2011-11-04 16:46:34.553796410 -0500
Python-2.7.2/setup.py 2011-11-04 16:59:49.692802313 -0500
+@@ -287,6 +287,15 @@
+ (ext.name, sys.exc_info()[1]))
+ self.failed.append(ext.name)
+ return
++
++# If we're cross-compiling, we want to skip the import check
++# i.e. we shouldn't be dynamically loading target shared libs
++if os.environ.get('CROSS_COMPILE') is not None:
++self.announce(
++'WARNING: skipping import check for cross-compiled
%s' %
++ext.name)
++return
++
+ # Workaround for Mac OS X: The Carbon-based modules cannot
be
+ # reliably imported into a command-line Python
+ if 'Carbon' in ext.extra_link_args:
diff --git a/meta/recipes-devtools/python/python_2.7.2.bb
b/meta/recipes-devtools/python/python_2.7.2.bb
index bb088a4..360e64e 100644
--- a/meta/recipes-devtools/python/python_2.7.2.bb
+++ b/meta/recipes-devtools/python/python_2.7.2.bb
@@ -18,6 +18,7 @@ SRC_URI += \
file://multilib.patch \
file://cgi_py.patch \
file://remove_sqlite_rpath.patch \
+ file://setup_py_skip_cross_import_check.patch \
S = ${WORKDIR}/Python-${PV}
@@ -57,6 +58,8 @@ do_compile() {
# then call do_install twice we get Makefile.orig == Makefile.sysroot
install -m 0644 Makefile Makefile.sysroot
+ export CROSS_COMPILE=${TARGET_PREFIX}
+
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
STAGING_LIBDIR=${STAGING_LIBDIR} \
@@ -78,6 +81,8 @@ do_install() {
# make install needs the original Makefile, or otherwise the inclues
would
# go to ${D}${STAGING...}/...
install -m 0644 Makefile.orig Makefile
+
+ export CROSS_COMPILE=${TARGET_PREFIX}
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
--
1.7.6.4
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [CONSOLIDATED PULL 17/17] python: skip setup.py 'import check' when cross-compiling
On Wed, 2011-11-09 at 16:57 -0800, Cui, Dexuan wrote: Zanussi, Tom wrote on 2011-11-09: On Wed, 2011-11-09 at 02:34 -0800, Cui, Dexuan wrote: How I wish I could notice the patch this morning so I could save 1 day! ??? Can you please explain what you mean? I meant I spent 1 day on debugging the same issue and got the cause but wasn't sure how the patch should be made, so I wanted to ask for suggestion in the ML and before that I tried to find out if anybody reported the same issue and I saw your patch that was sent 1.5 days ago... :-) Yeah, sure - I'm just kind of curious since I'd never seen it before until I added the avx instruction support. Just out of curiosity, what were your host and target arches? Tom Anyway, thanks TomZ for the patch! :-) You're welcome, I guess! Tom Thanks, -- Dexuan ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] [CONSOLIDATED PULL 17/17] python: skip setup.py 'import check' when cross-compiling
Zanussi, Tom wrote on 2011-11-10: On Wed, 2011-11-09 at 16:57 -0800, Cui, Dexuan wrote: Zanussi, Tom wrote on 2011-11-09: On Wed, 2011-11-09 at 02:34 -0800, Cui, Dexuan wrote: How I wish I could notice the patch this morning so I could save 1 day! ??? Can you please explain what you mean? I meant I spent 1 day on debugging the same issue and got the cause but wasn't sure how the patch should be made, so I wanted to ask for suggestion in the ML and before that I tried to find out if anybody reported the same issue and I saw your patch that was sent 1.5 days ago... :-) Yeah, sure - I'm just kind of curious since I'd never seen it before until I added the avx instruction support. Actually I think the issue is always there(at least from when python was upgrade to 2.7.x). e.g., in target, run python and try import grp, we'll get an error saying the built-in module grp couldn't be found; in python's log.do_compile, we can see the obvious warnings. Just out of curiosity, what were your host and target arches? I'm trying the task self hosted image(https://wiki.yoctoproject.org/wiki/Build_Appliance_Design): in target, I found bitbake's do_package failed because import grp failed, so I had to look into the issue. Thanks, -- Dexuan ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
