Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Mark, I am continuing to look at this. I hope you don't mind if I keep
updating this thread with my investigation.

First, I figured out my confusion. I was associating the wrong Makefile
command with the illegal instruction. It was actually the command under the
recipe for 'sharedmods', but it was silenced in the Makefile so I assumed
it was another line. You're right that the path with the *.so's is the
offending path. I've scoured online patches and can't understand how this
mechanism works outside of Yocto. I figure people trying to compile for ARM
would have similar issues. There seems to be a fundamental problem with how
Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
However, other people seem to have successfully built python for other
architectures, so I'm still confused.

I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory, and
the python recipe successfully passed. Can you elaborate on the issues you
were having with your patch? The only difference is that I maintained the
other two directories in $PYTHONPATH,
e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
offending one.

On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi 
wrote:

> Mark,
>
> I ran the same command, including the offending PYTHONPATH, from the shell
> without error. Any reason why that would work, despite it not working from
> within the Makefile? I was thinking it was another environment variable
> that was causing the issue (one that's not set in my bash environment by
> default). My $PATH didn't have python-native so I called it using an
> absolute path, but otherwise it's the same command run from the Makefile
> (parsed, of course):
>
> bash$
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
> -S -m sysconfig --generate-posix-vars
> bash$ ls pybuilddir.txt
> pybuilddir.txt
>
> Maybe I'm missing something.
>
>
> On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle 
> wrote:
>
>> On 11/25/15 10:11 AM, Michael Habibi wrote:
>> > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing
>> to a
>> > native version of Python:
>> >
>> > *| which python2.7*
>> > *|
>> >
>> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
>> > |
>> >
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> > _PYTHON_HOST_PLATFORM=linux2-x86_64
>> >
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> > python2.7 -S -m sysconfig --generate-posix-vars ;\
>> > |   if test $? -ne 0 ; then \
>> > |   echo "generate-posix-vars failed" ; \
>> > |   rm -f ./pybuilddir.txt ; \
>> > |   exit 1 ; \
>> > |   fi
>> > | Illegal instruction (core dumped)
>> > | make: *** [sharedmods] Error 132
>> >
>> > I guess one of the environmental variables being passed in is screwing
>> up what
>> > it's doing. Unfortunately I don't know enough about the inner workings
>> of Python
>> > to be of much help moving forward. I gave it my best shot!
>>
>> The problem I tracked down was:
>>
>>
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>>
>> This is full of .so objects when loaded cause the illegal instruction.
>>
>> I came up with a patch that I thought was going to fix it, but it's
>> triggered
>> other failures.
>>
>>
>> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>>
>> Just to be clear.. this does NOT work properly in many cases.  But might
>> give
>> you or someone else a clue as to how to possibly fix it.
>>
>> --Mark
>>
>> >
>> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi > > > wrote:
>> >
>> > Ah sorry, I misspoke. I walked through the Makefile and configure
>> scripts a
>> > bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version
>> of
>> > Python. It seems it's not being configured correctly. Either the
>> path is
>> > setup wrong and python2.7 is not pointing to the right version, or
>> python2.7
>> > needs to be pointing to the absolute path instead of relying on
>> $PATH.
>> >
>> >
>> >
>> >
>> > On 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Mark Hatle]

This is the patch that we ended up using:

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113457.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113456.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113458.html

On 11/30/15 5:12 PM, Michael Habibi wrote:
> OK I've dug into this further today, and here are some findings:
> 
> With my patch (which is similar to yours), it fails do_compile/package_qa
> because of poisoned system directories during the compilation steps. This
> approach seems to be a bust. Originally when I tested it, I hacked up the
> Makefile and only re-ran do_install onward, and didn't re-run do_compile
> (because I had to create the patch). Once I created the patch and re-ran from
> clean, I got some errors.
> 
> It appears the only reason it is using that directory in PYTHONPATH at all 
> (the
> one generated from the shell command and pybuilddir.txt) is for a potential
> sysconfig file in there. I tracked it down to a comment in a bug on Python's 
> bug
> tracker:
> 
> http://bugs.python.org/issue15484#msg180577
> 
> If we change that directory, it will pull in the sysconfig settings from the
> native Python configure (probably defaults to native sysconfig settings 
> because
> the PATH doesn't include the overrides), and then do some really bad things. 
> The
> question then became: in the unmodified build, why does do_install fail but
> do_compile doesn't? They both seem to be using sharedmods recipe which uses
> $PYTHONPATH. The answer is that when we run do_compile the first time around,
> the offending *.so's are not generated yet, and thus are not included by
> python-native. The second time around (in do_install), the *.so's are in the
> path and so python-native will use them and override its own.
> 
> My latest experiment is to remove that location altogether from PYTHONPATH and
> see if it works (neither the shell-generated directory in python-cross, nor 
> the
> lib-dynload directory from python-native). The compile works the first time
> through despite there being no sysconfig data, so I'm wondering if the install
> step will also work without finding any data. python-native should continue to
> work with its internal settings to find modules and libraries.
> 
> To be continued tomorrow, I'm sure...
> 
> On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi  > wrote:
> 
> Mark, I am continuing to look at this. I hope you don't mind if I keep
> updating this thread with my investigation.
> 
> First, I figured out my confusion. I was associating the wrong Makefile
> command with the illegal instruction. It was actually the command under 
> the
> recipe for 'sharedmods', but it was silenced in the Makefile so I assumed 
> it
> was another line. You're right that the path with the *.so's is the
> offending path. I've scoured online patches and can't understand how this
> mechanism works outside of Yocto. I figure people trying to compile for 
> ARM
> would have similar issues. There seems to be a fundamental problem with 
> how
> Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
> However, other people seem to have successfully built python for other
> architectures, so I'm still confused.
> 
> I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory, 
> and
> the python recipe successfully passed. Can you elaborate on the issues you
> were having with your patch? The only difference is that I maintained the
> other two directories in $PYTHONPATH,
> e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
> offending one.
> 
> On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi  > wrote:
> 
> Mark,
> 
> I ran the same command, including the offending PYTHONPATH, from the
> shell without error. Any reason why that would work, despite it not
> working from within the Makefile? I was thinking it was another
> environment variable that was causing the issue (one that's not set in
> my bash environment by default). My $PATH didn't have python-native 
> so I
> called it using an absolute path, but otherwise it's the same command
> run from the Makefile (parsed, of course):
> 
> bash$
> 
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> 
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> 
> 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

OK I've dug into this further today, and here are some findings:

With my patch (which is similar to yours), it fails do_compile/package_qa
because of poisoned system directories during the compilation steps. This
approach seems to be a bust. Originally when I tested it, I hacked up the
Makefile and only re-ran do_install onward, and didn't re-run do_compile
(because I had to create the patch). Once I created the patch and re-ran
from clean, I got some errors.

It appears the only reason it is using that directory in PYTHONPATH at all
(the one generated from the shell command and pybuilddir.txt) is for a
potential sysconfig file in there. I tracked it down to a comment in a bug
on Python's bug tracker:

http://bugs.python.org/issue15484#msg180577

If we change that directory, it will pull in the sysconfig settings from
the native Python configure (probably defaults to native sysconfig settings
because the PATH doesn't include the overrides), and then do some really
bad things. The question then became: in the unmodified build, why does
do_install fail but do_compile doesn't? They both seem to be using
sharedmods recipe which uses $PYTHONPATH. The answer is that when we run
do_compile the first time around, the offending *.so's are not generated
yet, and thus are not included by python-native. The second time around (in
do_install), the *.so's are in the path and so python-native will use them
and override its own.

My latest experiment is to remove that location altogether from PYTHONPATH
and see if it works (neither the shell-generated directory in python-cross,
nor the lib-dynload directory from python-native). The compile works the
first time through despite there being no sysconfig data, so I'm wondering
if the install step will also work without finding any data. python-native
should continue to work with its internal settings to find modules and
libraries.

To be continued tomorrow, I'm sure...

On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi 
wrote:

> Mark, I am continuing to look at this. I hope you don't mind if I keep
> updating this thread with my investigation.
>
> First, I figured out my confusion. I was associating the wrong Makefile
> command with the illegal instruction. It was actually the command under the
> recipe for 'sharedmods', but it was silenced in the Makefile so I assumed
> it was another line. You're right that the path with the *.so's is the
> offending path. I've scoured online patches and can't understand how this
> mechanism works outside of Yocto. I figure people trying to compile for ARM
> would have similar issues. There seems to be a fundamental problem with how
> Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
> However, other people seem to have successfully built python for other
> architectures, so I'm still confused.
>
> I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory,
> and the python recipe successfully passed. Can you elaborate on the issues
> you were having with your patch? The only difference is that I maintained
> the other two directories in $PYTHONPATH,
> e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
> offending one.
>
> On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi 
> wrote:
>
>> Mark,
>>
>> I ran the same command, including the offending PYTHONPATH, from the
>> shell without error. Any reason why that would work, despite it not working
>> from within the Makefile? I was thinking it was another environment
>> variable that was causing the issue (one that's not set in my bash
>> environment by default). My $PATH didn't have python-native so I called it
>> using an absolute path, but otherwise it's the same command run from the
>> Makefile (parsed, of course):
>>
>> bash$
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
>> -S -m sysconfig --generate-posix-vars
>> bash$ ls pybuilddir.txt
>> pybuilddir.txt
>>
>> Maybe I'm missing something.
>>
>>
>> On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle 
>> wrote:
>>
>>> On 11/25/15 10:11 AM, Michael Habibi wrote:
>>> > Well, I'm wrong yet again. A 'which python2.7' shows that it's
>>> pointing to a
>>> > native version of Python:
>>> >
>>> > *| which python2.7*
>>> > *|
>>> >
>>> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
>>> > |
>>> >
>>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>>> > 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Mark, great! I was literally about to try double-stacking the PATH next
(since removing both didn't work for reasons that became obvious after
running the experiment), so that it would grab the *.so's from
python-native and then the sysconfig from the python-cross. Looks like I
may need to subscribe to that list so I'm aware of these patches!

I'm not sure what the streams look like (I can't visualize it yet), but is
this change in any Yocto branch yet, or does it need to get pulled by
someone? I can merge it locally, but just curious how the OE-Core and Yocto
repos are related.

On Mon, Nov 30, 2015 at 6:07 PM, Mark Hatle 
wrote:

> This is the patch that we ended up using:
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113457.html
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113456.html
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113458.html
>
> On 11/30/15 5:12 PM, Michael Habibi wrote:
> > OK I've dug into this further today, and here are some findings:
> >
> > With my patch (which is similar to yours), it fails do_compile/package_qa
> > because of poisoned system directories during the compilation steps. This
> > approach seems to be a bust. Originally when I tested it, I hacked up the
> > Makefile and only re-ran do_install onward, and didn't re-run do_compile
> > (because I had to create the patch). Once I created the patch and re-ran
> from
> > clean, I got some errors.
> >
> > It appears the only reason it is using that directory in PYTHONPATH at
> all (the
> > one generated from the shell command and pybuilddir.txt) is for a
> potential
> > sysconfig file in there. I tracked it down to a comment in a bug on
> Python's bug
> > tracker:
> >
> > http://bugs.python.org/issue15484#msg180577
> >
> > If we change that directory, it will pull in the sysconfig settings from
> the
> > native Python configure (probably defaults to native sysconfig settings
> because
> > the PATH doesn't include the overrides), and then do some really bad
> things. The
> > question then became: in the unmodified build, why does do_install fail
> but
> > do_compile doesn't? They both seem to be using sharedmods recipe which
> uses
> > $PYTHONPATH. The answer is that when we run do_compile the first time
> around,
> > the offending *.so's are not generated yet, and thus are not included by
> > python-native. The second time around (in do_install), the *.so's are in
> the
> > path and so python-native will use them and override its own.
> >
> > My latest experiment is to remove that location altogether from
> PYTHONPATH and
> > see if it works (neither the shell-generated directory in python-cross,
> nor the
> > lib-dynload directory from python-native). The compile works the first
> time
> > through despite there being no sysconfig data, so I'm wondering if the
> install
> > step will also work without finding any data. python-native should
> continue to
> > work with its internal settings to find modules and libraries.
> >
> > To be continued tomorrow, I'm sure...
> >
> > On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi  > > wrote:
> >
> > Mark, I am continuing to look at this. I hope you don't mind if I
> keep
> > updating this thread with my investigation.
> >
> > First, I figured out my confusion. I was associating the wrong
> Makefile
> > command with the illegal instruction. It was actually the command
> under the
> > recipe for 'sharedmods', but it was silenced in the Makefile so I
> assumed it
> > was another line. You're right that the path with the *.so's is the
> > offending path. I've scoured online patches and can't understand how
> this
> > mechanism works outside of Yocto. I figure people trying to compile
> for ARM
> > would have similar issues. There seems to be a fundamental problem
> with how
> > Python sets up PYTHONPATH using the pybuilddir file and
> $abs_builddir.
> > However, other people seem to have successfully built python for
> other
> > architectures, so I'm still confused.
> >
> > I hardcoded a fix for PYTHONPATH to use the host lib-dynload
> directory, and
> > the python recipe successfully passed. Can you elaborate on the
> issues you
> > were having with your patch? The only difference is that I
> maintained the
> > other two directories in $PYTHONPATH,
> > e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced
> the
> > offending one.
> >
> > On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi <
> mikehab...@gmail.com
> > > wrote:
> >
> > Mark,
> >
> > I ran the same command, including the offending PYTHONPATH, from
> the
> > shell without error. Any reason why that would work, despite it
> not
> > working from within the Makefile? I was thinking it was another
> >

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

For what it's worth, this is the offending portion of Makefile.pre:452:

# Create build directory and generate the sysconfig build-time data there.
# pybuilddir.txt contains the name of the build dir and is used for
# sys.path fixup -- see Modules/getpath.c.
# Since this step runs before shared modules are built, try to avoid
bootstrap
# problems by creating a dummy pybuilddir.txt just to allow interpreter
# initialization to succeed.  It will be overwritten by generate-posix-vars
# or removed in case of failure.
pybuilddir.txt: $(BUILDPYTHON)
→ @echo "none" > ./pybuilddir.txt
→ $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars ;\
→ if test $$? -ne 0 ; then \
→ → echo "generate-posix-vars failed" ; \
→ → rm -f ./pybuilddir.txt ; \
→ → exit 1 ; \
→ fi

I don't know enough about the Python build to understand what it's trying
to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
acceptable?

I'm surprised this seems to work for other people, since this should be
failing for anyone using Python on a target platform different from their
host.

On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle 
wrote:

> On 11/24/15 3:23 PM, Mark Hatle wrote:
> > My guess is that there is a bug in the python integration where it's not
> > realizing the host and target are different systems, so it's trying to
> run a
> > target program on your host.  Your host isn't haswell, so... Illegal
> Instruction
> > -- and the system stops.
>
> Just an FYI, I hit the same problem today.  I suspect it is the host
> trying to
> run target software and I'm looking into it.
>
> --Mark
>
> > The alternative would be something is running QEMU to execute a target
> binary
> > and QEMU doesn't have instruction support -- but that doesn't look like
> the case
> > here.
> >
> > --Mark
> >
> > On 11/24/15 3:06 PM, Michael Habibi wrote:
> >> All,
> >>
> >> I added a new machine definition with tuning parameters for haswell
> >> microarchitectures (basically just duplicated corei7 but tuned it for
> haswell).
> >> This seems to work correctly, but failed when running do_install for
> python
> >> toward the end of the build process. I am running with the Yocto 2.0
> framework,
> >> with very minimal changes to create a new distribution and machine for
> our
> >> custom embedded device. Note I had this distro configuration working
> before, and
> >> the only difference is I added a new machine with this tuning.
> >>
> >> I believe the issue is because, as part of the do_install step for
> Python, it
> >> attempts to run python on the local host, despite being cross-compiled.
> However,
> >> it is tuned for a processor that my host machine doesn't run, so I get
> an
> >> instruction exception.
> >>
> >> Did I do something weird? I figure python would be configured for
> >> cross-compiling and therefore wouldn't try to run the target version on
> the
> >> host, even for sanity tests. Here is the output of the failure:
> >>
> >> $ tail -20
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
> >>
> >> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
> >>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
> >>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
> >>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
> >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
> >> x86_64-diags-linux-ranlib libpython2.7.a
> >> Modules/posixmodule.o: In function `posix_tmpnam':
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
> >> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
> >> Modules/posixmodule.o: In function `posix_tempnam':
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
> >> warning: the use of `tempnam' is dangerous, better use `mkstemp'
> >> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse
> -mavx2
> >> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
> >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python
> \
> >> Modules/python.o \
> >> -L. -lpython2.7 -lpthread -ldl  -lpthread
> -lutil   -lm
> >>
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> >> _PYTHON_HOST_PLATFORM=linux2-x86_64
> >>
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> >> python2.7 -S -m sysconfig --generate-posix-vars ;\
> >> if test $? -ne 0 ; then \
> >> echo "generate-posix-vars failed" ; \
> >> rm 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to
a native version of Python:

*| which python2.7*
*|
/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
|
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
python2.7 -S -m sysconfig --generate-posix-vars ;\
|   if test $? -ne 0 ; then \
|   echo "generate-posix-vars failed" ; \
|   rm -f ./pybuilddir.txt ; \
|   exit 1 ; \
|   fi
| Illegal instruction (core dumped)
| make: *** [sharedmods] Error 132

I guess one of the environmental variables being passed in is screwing up
what it's doing. Unfortunately I don't know enough about the inner workings
of Python to be of much help moving forward. I gave it my best shot!


On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi 
wrote:

> Ah sorry, I misspoke. I walked through the Makefile and configure scripts
> a bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> Python. It seems it's not being configured correctly. Either the path is
> setup wrong and python2.7 is not pointing to the right version, or
> python2.7 needs to be pointing to the absolute path instead of relying on
> $PATH.
>
>
>
>
> On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi 
> wrote:
>
>> For what it's worth, this is the offending portion of Makefile.pre:452:
>>
>> # Create build directory and generate the sysconfig build-time data there.
>> # pybuilddir.txt contains the name of the build dir and is used for
>> # sys.path fixup -- see Modules/getpath.c.
>> # Since this step runs before shared modules are built, try to avoid
>> bootstrap
>> # problems by creating a dummy pybuilddir.txt just to allow interpreter
>> # initialization to succeed.  It will be overwritten by
>> generate-posix-vars
>> # or removed in case of failure.
>> pybuilddir.txt: $(BUILDPYTHON)
>> → @echo "none" > ./pybuilddir.txt
>> → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars
>> ;\
>> → if test $$? -ne 0 ; then \
>> → → echo "generate-posix-vars failed" ; \
>> → → rm -f ./pybuilddir.txt ; \
>> → → exit 1 ; \
>> → fi
>>
>> I don't know enough about the Python build to understand what it's trying
>> to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
>> acceptable?
>>
>> I'm surprised this seems to work for other people, since this should be
>> failing for anyone using Python on a target platform different from their
>> host.
>>
>> On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle 
>> wrote:
>>
>>> On 11/24/15 3:23 PM, Mark Hatle wrote:
>>> > My guess is that there is a bug in the python integration where it's
>>> not
>>> > realizing the host and target are different systems, so it's trying to
>>> run a
>>> > target program on your host.  Your host isn't haswell, so... Illegal
>>> Instruction
>>> > -- and the system stops.
>>>
>>> Just an FYI, I hit the same problem today.  I suspect it is the host
>>> trying to
>>> run target software and I'm looking into it.
>>>
>>> --Mark
>>>
>>> > The alternative would be something is running QEMU to execute a target
>>> binary
>>> > and QEMU doesn't have instruction support -- but that doesn't look
>>> like the case
>>> > here.
>>> >
>>> > --Mark
>>> >
>>> > On 11/24/15 3:06 PM, Michael Habibi wrote:
>>> >> All,
>>> >>
>>> >> I added a new machine definition with tuning parameters for haswell
>>> >> microarchitectures (basically just duplicated corei7 but tuned it for
>>> haswell).
>>> >> This seems to work correctly, but failed when running do_install for
>>> python
>>> >> toward the end of the build process. I am running with the Yocto 2.0
>>> framework,
>>> >> with very minimal changes to create a new distribution and machine
>>> for our
>>> >> custom embedded device. Note I had this distro configuration working
>>> before, and
>>> >> the only difference is I added a new machine with this tuning.
>>> >>
>>> >> I believe the issue is because, as part of the do_install step for
>>> Python, it
>>> >> attempts to run python on the local host, despite being
>>> cross-compiled. However,
>>> >> it is tuned for a processor that my host machine doesn't run, so I
>>> get an
>>> >> instruction exception.
>>> >>
>>> >> Did I do something weird? I figure python would be configured for
>>> >> cross-compiling and therefore wouldn't try to run the target version
>>> on the
>>> >> host, even for sanity tests. Here is the output of the failure:
>>> >>
>>> >> $ tail -20
>>> >>
>>> 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Mark,

I ran the same command, including the offending PYTHONPATH, from the shell
without error. Any reason why that would work, despite it not working from
within the Makefile? I was thinking it was another environment variable
that was causing the issue (one that's not set in my bash environment by
default). My $PATH didn't have python-native so I called it using an
absolute path, but otherwise it's the same command run from the Makefile
(parsed, of course):

bash$
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
-S -m sysconfig --generate-posix-vars
bash$ ls pybuilddir.txt
pybuilddir.txt

Maybe I'm missing something.


On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle 
wrote:

> On 11/25/15 10:11 AM, Michael Habibi wrote:
> > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing
> to a
> > native version of Python:
> >
> > *| which python2.7*
> > *|
> >
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> > |
> >
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> > _PYTHON_HOST_PLATFORM=linux2-x86_64
> >
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> > python2.7 -S -m sysconfig --generate-posix-vars ;\
> > |   if test $? -ne 0 ; then \
> > |   echo "generate-posix-vars failed" ; \
> > |   rm -f ./pybuilddir.txt ; \
> > |   exit 1 ; \
> > |   fi
> > | Illegal instruction (core dumped)
> > | make: *** [sharedmods] Error 132
> >
> > I guess one of the environmental variables being passed in is screwing
> up what
> > it's doing. Unfortunately I don't know enough about the inner workings
> of Python
> > to be of much help moving forward. I gave it my best shot!
>
> The problem I tracked down was:
>
>
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>
> This is full of .so objects when loaded cause the illegal instruction.
>
> I came up with a patch that I thought was going to fix it, but it's
> triggered
> other failures.
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>
> Just to be clear.. this does NOT work properly in many cases.  But might
> give
> you or someone else a clue as to how to possibly fix it.
>
> --Mark
>
> >
> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi  > > wrote:
> >
> > Ah sorry, I misspoke. I walked through the Makefile and configure
> scripts a
> > bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> > Python. It seems it's not being configured correctly. Either the
> path is
> > setup wrong and python2.7 is not pointing to the right version, or
> python2.7
> > needs to be pointing to the absolute path instead of relying on
> $PATH.
> >
> >
> >
> >
> > On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <
> mikehab...@gmail.com
> > > wrote:
> >
> > For what it's worth, this is the offending portion of
> Makefile.pre:452:
> >
> > # Create build directory and generate the sysconfig build-time
> data there.
> > # pybuilddir.txt contains the name of the build dir and is used
> for
> > # sys.path fixup -- see Modules/getpath.c.
> > # Since this step runs before shared modules are built, try to
> avoid
> > bootstrap
> > # problems by creating a dummy pybuilddir.txt just to allow
> interpreter
> > # initialization to succeed.  It will be overwritten by
> generate-posix-vars
> > # or removed in case of failure.
> > pybuilddir.txt: $(BUILDPYTHON)
> > → @echo "none" > ./pybuilddir.txt
> > → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig
> --generate-posix-vars ;\
> > → if test $$? -ne 0 ; then \
> > → → echo "generate-posix-vars failed" ; \
> > → → rm -f ./pybuilddir.txt ; \
> > → → exit 1 ; \
> > → fi
> >
> > I don't know enough about the Python build to understand what
> it's
> > trying to do, but perhaps replacing PYTHON_FOR_BUILD with
> HOSTPYTHON may
> > be acceptable?
> >
> > I'm surprised this 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Mark Hatle]

On 11/25/15 10:11 AM, Michael Habibi wrote:
> Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to a
> native version of Python:
> 
> *| which python2.7*
> *|
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> |
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> python2.7 -S -m sysconfig --generate-posix-vars ;\
> |   if test $? -ne 0 ; then \
> |   echo "generate-posix-vars failed" ; \
> |   rm -f ./pybuilddir.txt ; \
> |   exit 1 ; \
> |   fi
> | Illegal instruction (core dumped)
> | make: *** [sharedmods] Error 132
> 
> I guess one of the environmental variables being passed in is screwing up what
> it's doing. Unfortunately I don't know enough about the inner workings of 
> Python
> to be of much help moving forward. I gave it my best shot!

The problem I tracked down was:

PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7

This is full of .so objects when loaded cause the illegal instruction.

I came up with a patch that I thought was going to fix it, but it's triggered
other failures.

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html

Just to be clear.. this does NOT work properly in many cases.  But might give
you or someone else a clue as to how to possibly fix it.

--Mark

> 
> On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi  > wrote:
> 
> Ah sorry, I misspoke. I walked through the Makefile and configure scripts 
> a
> bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> Python. It seems it's not being configured correctly. Either the path is
> setup wrong and python2.7 is not pointing to the right version, or 
> python2.7
> needs to be pointing to the absolute path instead of relying on $PATH.
> 
> 
> 
> 
> On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi  > wrote:
> 
> For what it's worth, this is the offending portion of 
> Makefile.pre:452:
> 
> # Create build directory and generate the sysconfig build-time data 
> there.
> # pybuilddir.txt contains the name of the build dir and is used for
> # sys.path fixup -- see Modules/getpath.c.
> # Since this step runs before shared modules are built, try to avoid
> bootstrap
> # problems by creating a dummy pybuilddir.txt just to allow 
> interpreter
> # initialization to succeed.  It will be overwritten by 
> generate-posix-vars
> # or removed in case of failure.
> pybuilddir.txt: $(BUILDPYTHON)
> → @echo "none" > ./pybuilddir.txt
> → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig 
> --generate-posix-vars ;\
> → if test $$? -ne 0 ; then \
> → → echo "generate-posix-vars failed" ; \
> → → rm -f ./pybuilddir.txt ; \
> → → exit 1 ; \
> → fi
> 
> I don't know enough about the Python build to understand what it's
> trying to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON 
> may
> be acceptable?
> 
> I'm surprised this seems to work for other people, since this should 
> be
> failing for anyone using Python on a target platform different from
> their host.
> 
> On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle  > wrote:
> 
> On 11/24/15 3:23 PM, Mark Hatle wrote:
> > My guess is that there is a bug in the python integration where
> it's not
> > realizing the host and target are different systems, so it's
> trying to run a
> > target program on your host.  Your host isn't haswell, so...
> Illegal Instruction
> > -- and the system stops.
> 
> Just an FYI, I hit the same problem today.  I suspect it is the 
> host
> trying to
> run target software and I'm looking into it.
> 
> --Mark
> 
> > The alternative would be something is running QEMU to execute a
> target binary
> > and QEMU doesn't have instruction support -- but that doesn't 
> look
> like the case
> > here.
> >
> > --Mark
> >
> > On 11/24/15 3:06 PM, Michael Habibi wrote:
> >> All,
> >>
> >> I added a new machine 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Mark Hatle]

On 11/24/15 3:23 PM, Mark Hatle wrote:
> My guess is that there is a bug in the python integration where it's not
> realizing the host and target are different systems, so it's trying to run a
> target program on your host.  Your host isn't haswell, so... Illegal 
> Instruction
> -- and the system stops.

Just an FYI, I hit the same problem today.  I suspect it is the host trying to
run target software and I'm looking into it.

--Mark

> The alternative would be something is running QEMU to execute a target binary
> and QEMU doesn't have instruction support -- but that doesn't look like the 
> case
> here.
> 
> --Mark
> 
> On 11/24/15 3:06 PM, Michael Habibi wrote:
>> All,
>>
>> I added a new machine definition with tuning parameters for haswell
>> microarchitectures (basically just duplicated corei7 but tuned it for 
>> haswell).
>> This seems to work correctly, but failed when running do_install for python
>> toward the end of the build process. I am running with the Yocto 2.0 
>> framework,
>> with very minimal changes to create a new distribution and machine for our
>> custom embedded device. Note I had this distro configuration working before, 
>> and
>> the only difference is I added a new machine with this tuning.
>>
>> I believe the issue is because, as part of the do_install step for Python, it
>> attempts to run python on the local host, despite being cross-compiled. 
>> However,
>> it is tuned for a processor that my host machine doesn't run, so I get an
>> instruction exception.
>>
>> Did I do something weird? I figure python would be configured for
>> cross-compiling and therefore wouldn't try to run the target version on the
>> host, even for sanity tests. Here is the output of the failure:
>>
>> $ tail -20
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>>
>> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
>>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
>>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
>>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
>>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>> x86_64-diags-linux-ranlib libpython2.7.a
>> Modules/posixmodule.o: In function `posix_tmpnam':
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
>> Modules/posixmodule.o: In function `posix_tempnam':
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>> warning: the use of `tempnam' is dangerous, better use `mkstemp'
>> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse 
>> -mavx2
>> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python \
>> Modules/python.o \
>> -L. -lpython2.7 -lpthread -ldl  -lpthread -lutil   
>> -lm
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> python2.7 -S -m sysconfig --generate-posix-vars ;\
>> if test $? -ne 0 ; then \
>> echo "generate-posix-vars failed" ; \
>> rm -f ./pybuilddir.txt ; \
>> exit 1 ; \
>> fi
>> Illegal instruction (core dumped)
>> make: *** [sharedmods] Error 132
>> WARNING: exit code 1 from a shell command.
>> ERROR: oe_runmake failed
>> ERROR: Function failed: do_install (log file is located at
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>>
>> Here is my tune-haswell.inc (ignore some copy/paste comment issues right now 
>> =):
>>
>> # Settings for the GCC(1) cpu-type "haswell":
>> #
>> # Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE, SSE2, SSE3,·
>> # SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
>> # RDRND, FMA, BMI, BMI2 and F16C instruction set support.
>> #
>> # This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay Trail) 
>> CPUs
>> # (and beyond).
>> #
>> DEFAULTTUNE ?= "haswell"
>>
>> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
>> require conf/machine/include/tune-corei7.inc
>>
>> # Extra tune features
>> TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
>> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
>> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
>>
>> # Extra tune selections
>> AVAILTUNES += "haswell"
>>