[issue19142] Cross-compile fails trying to execute foreign pgen on build host
Trevor Bowen added the comment: I thought make touch was only for those trying to build from the Mecurial source, as opposed to building from the released tar-ball source. I thought my efforts laid on the other side of the need for that command. If I understood wrong, when would I use it and why? ... Thanks! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19142 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19142] Cross-compile fails trying to execute foreign pgen on build host
Trevor Bowen added the comment: Thanks, David! I have no interest in running pgen on the target/host. My only interest is building python and its various modules to run on my embedded host. I do not want to develop Python on the embedded host. Unfortunately, the build process requires Parser/pgen to build the grammar files, which are needed for several object files. Here's the relevant snippet from the Makefile.pre.in: $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGENSRCS) @$(MKDIR_P) Include $(MAKE) $(PGEN) $(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C) $(GRAMMAR_C): $(GRAMMAR_H) $(GRAMMAR_INPUT) $(PGENSRCS) $(MAKE) $(GRAMMAR_H) touch $(GRAMMAR_C) $(PGEN):$(PGENOBJS) $(CC) $(OPT) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN) Parser/grammar.o: $(srcdir)/Parser/grammar.c \ $(srcdir)/Include/token.h \ $(srcdir)/Include/grammar.h ... Python/compile.o Python/symtable.o Python/ast.o: $(GRAMMAR_H) $(AST_H) If there is a way to eliminate the need for Parser/pgen to run on the build system to cross-compile the default all target, that would be great. ... I'll experiment with make touch. ... Thanks! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19142 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19142] Cross-compile fails trying to execute foreign pgen on build host
Trevor Bowen added the comment: I executed make touch between configure and make, but the build process still created Parser/pgen and then tried to use it, which of course crashed the build, since pgen was compiled for the embedded host not the build system. :( Was that the wrong usage? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19142 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19142] Cross-compile fails trying to execute foreign pgen on build host
Trevor Bowen added the comment: In the vein of: http://randomsplat.com/id5-cross-compiling-python-for-embedded-linux.html I have created a patch and top-level build script, which builds the requisite python interpreter and Parser/pgen binary to run on the build system, which are both used during the cross-compile process. In some ways, this patch is simpler, because of the new cross-compile capabilities in 2.7.4 and 2.7.5. However, previous work is still insufficient, because the Makefile assumes the cross-compiled Parser/pgen can be executed on the build system during build. This patch primarily compensates for that, but it also works around some misplaced installation file issues. My top-level cross-compile script, which applies the patch *during* the build process is also attached. It depends on a VCS to restore the original version, but I could have just as easily made 2 copies of the patched files (before and after patching) and copied the original versions as needed. I hope this helps others. Hopefully, bits of this can be integrated to help further simplify future cross-compile efforts. -- keywords: +patch Added file: http://bugs.python.org/file31991/Python-2.7.5-xcompile.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19142 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19142] Cross-compile fails trying to execute foreign pgen on build host
Trevor Bowen added the comment:
Short version of cross-compile script without error checking:
#!/bin/bash
export RFS=/local/my_root_file_system
make distclean
rm -rf python_for_build Parser/pgen_for_build
git checkout -- Makefile.pre.in Modules/Setup.dist configure setup.py
./configure
make python Parser/pgen
mv python python_for_build
mv Parser/pgen Parser/pgen_for_build
patch -p3 Python-2.7.5-xcompile.patch
export
PATH=/opt/my_cross_compile_toolchain/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin:${PATH}
make distclean
./configure --host=powerpc-none-linux-gnuspe --build=i586-linux-gnu --prefix=/
\
--disable-ipv6 ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no
ac_cv_have_long_long_format=yes
make --jobs=4
sudo make install DESTDIR=${RFS} PATH=${PATH}
--
Added file: http://bugs.python.org/file31992/cross-compile.sh
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19142
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19142] Cross-compile fails trying to execute foreign pgen on build host
Trevor Bowen added the comment: FWIW, I also explored my original proposal, which essentially moved the above script and modifications into the Python configure.ac and Makefile.pre.in files, so that Python's internal build process would create the native build system versions of Python and Parser/pgen, if necessary. I made some progress in creating additional Make targets just for the native build system, which redefined the toolchain after completion; however, I encountered unexepected, overwhelming configure contamination in the auto-generated header files. Maybe someone can push this farther than me? I'm attaching my patch, in case it helps someone else and just as reference. As for me, I think it is much easier to use the above solution, which would require very little patching to the sources to make a 2-step compilation to be trivial. -- Added file: http://bugs.python.org/file31993/Python-2.7.5-xcompile_improvements.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19142 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19142] Cross-compile fails trying to execute foreign pgen on build host
Trevor Bowen added the comment: Ok, thanks for the tips. I'm new to developing on Python itself. I'll start simple by trying to develop a set of patches for the 2.7.5 source tar-ball, which I usually use to build Python. If that succeeds, I'll look into pushing it into the related source files. So far, I'm anticipating modifying these files: configure, Makefile.pre.in, and setup.py. ... Thanks! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19142 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19142] Cross-compile fails trying to execute foreign pgen on build host
New submission from Trevor Bowen:
FWIW, I'm using a Freescale cross-compile tool-chain on a Linux x86-64 build
host, although I have duplicated the cross-compile error on an x86 Ubunutu
10.04 build host.
Steps to reproduce:
$ wget http://www.python.org/ftp/python/2.7.5/Python-2.7.5.tar.bz2
$ tar -jxf Python-2.7.5.tar.bz2
$ cd Python-2.7.5
$ export
PATH=/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin:${PATH}
# For info only:
$ ls -l
/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin
total 10472
-rwxrwxrwx 1 root root 590485 Mar 5 2012 powerpc-none-linux-gnuspe-addr2line
-rwxrwxrwx 1 root root 614647 Mar 5 2012 powerpc-none-linux-gnuspe-ar
-rwxrwxrwx 1 root root 897161 Mar 5 2012 powerpc-none-linux-gnuspe-as
-rwxrwxrwx 1 root root 235382 Mar 5 2012 powerpc-none-linux-gnuspe-c++
-rwxrwxrwx 1 root root 589227 Mar 5 2012 powerpc-none-linux-gnuspe-c++filt
-rwxrwxrwx 1 root root 234277 Mar 5 2012 powerpc-none-linux-gnuspe-cpp
-rwxrwxrwx 1 root root8503 Mar 5 2012 powerpc-none-linux-gnuspe-embedspu
-rwxrwxrwx 1 root root 235382 Mar 5 2012 powerpc-none-linux-gnuspe-g++
-rwxrwxrwx 1 root root 233126 Mar 5 2012 powerpc-none-linux-gnuspe-gcc
-rwxrwxrwx 1 root root 233126 Mar 5 2012 powerpc-none-linux-gnuspe-gcc-4.3.2
-rwxrwxrwx 1 root root 16512 Mar 5 2012 powerpc-none-linux-gnuspe-gccbug
-rwxrwxrwx 1 root root 28017 Mar 5 2012 powerpc-none-linux-gnuspe-gcov
-rwxrwxrwx 1 root root 655127 Mar 5 2012 powerpc-none-linux-gnuspe-gprof
-rwxrwxrwx 1 root root 1036372 Mar 5 2012 powerpc-none-linux-gnuspe-ld
-rwxrwxrwx 1 root root 603678 Mar 5 2012 powerpc-none-linux-gnuspe-nm
-rwxrwxrwx 1 root root 750617 Mar 5 2012 powerpc-none-linux-gnuspe-objcopy
-rwxrwxrwx 1 root root 895336 Mar 5 2012 powerpc-none-linux-gnuspe-objdump
-rwxrwxrwx 1 root root 614647 Mar 5 2012 powerpc-none-linux-gnuspe-ranlib
-rwxrwxrwx 1 root root 264063 Mar 5 2012 powerpc-none-linux-gnuspe-readelf
-rwxrwxrwx 1 root root 593901 Mar 5 2012 powerpc-none-linux-gnuspe-size
-rwxrwxrwx 1 root root 591853 Mar 5 2012 powerpc-none-linux-gnuspe-strings
-rwxrwxrwx 1 root root 750617 Mar 5 2012 powerpc-none-linux-gnuspe-strip
$ ./configure --host=powerpc-none-linux-gnuspe \
--build=i586-linux-gnu --prefix=/ \
--disable-ipv6 ac_cv_file__dev_ptmx=no \
ac_cv_file__dev_ptc=no
$ make
Make fails because the foreign (powerpc, in this case) pgen binary is used to
build the $(GRAMMAR_H). Instead, a native version of the python interpreter
(preferrable) and pgen (essential) need to be built first using the host's
toolchain. Then, these tools can be used to build the full foreign suite.
FWIW, This 2 step process is well documented here:
http://randomsplat.com/id5-cross-compiling-python-for-embedded-linux.html
However, the cross-compile fixes from Issue17086 may have helped this process.
(It's hard for me to determine how to take advantage of these updates.)
Certainly, they changed the above process dramatically.
Incidentally, Python 3.3.2 exhibits almost the exact same build error, if not
the same.
Build log on an x86-64 host for Python-2.7.5 for a PowerPC-linux target is
attached.
Tail of log:
make Parser/pgen
make[1]: Entering directory `/home/user/Python-2.7.5'
powerpc-none-linux-gnuspe-gcc -c -fno-strict-aliasing -g -O2 -DNDEBUG -g
-fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include
-DPy_BUILD_CORE -o Python/mysnprintf.o Python/mysnprintf.c
powerpc-none-linux-gnuspe-gcc -c -fno-strict-aliasing -g -O2 -DNDEBUG -g
-fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include
-DPy_BUILD_CORE -o Python/pyctype.o Python/pyctype.c
powerpc-none-linux-gnuspe-gcc -c -fno-strict-aliasing -g -O2 -DNDEBUG -g
-fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include
-DPy_BUILD_CORE -o Parser/tokenizer_pgen.o Parser/tokenizer_pgen.c
powerpc-none-linux-gnuspe-gcc -c -fno-strict-aliasing -g -O2 -DNDEBUG -g
-fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include
-DPy_BUILD_CORE -o Parser/printgrammar.o Parser/printgrammar.c
powerpc-none-linux-gnuspe-gcc -c -fno-strict-aliasing -g -O2 -DNDEBUG -g
-fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include
-DPy_BUILD_CORE -o Parser/pgenmain.o Parser/pgenmain.c
powerpc-none-linux-gnuspe-gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o
Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o
Parser/firstsets.o Parser/grammar.o Parser/pgen.o Objects/obmalloc.o
Python/mysnprintf.o Python/pyctype.o Parser/tokenizer_pgen.o
Parser/printgrammar.o Parser/pgenmain.o -lpthread -ldl -lpthread -lutil -o
Parser/pgen
make[1]: Leaving directory `/home/user/Python-2.7.5'
Parser/pgen ./Grammar/Grammar Include/graminit.h Python/graminit.c
Parser/pgen: Parser/pgen: cannot execute binary file
make: *** [Include/graminit.h
[issue19142] Cross-compile fails trying to execute foreign pgen on build host
Trevor Bowen added the comment: Sorry, I do not mean to compound an already open and complex problem. I thought that the fixes in 2.7.4 were meant in part to help alleviate this problem. I had not found any feedback or tutorials, so I wanted to provide a status update of sorts. I'm sure there is some. I just have not been able to find it myself. If the configure script passed in the cross-compile tool-chain *and* the build-host tool-chain, could the Makefile be enhanced to build pgen and python first with the build-host and then use them to build the full cross-compiled versions? Or, would there be a better general approach? I am not familiar with the MINGW requirements. Would this satisfy their needs? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19142 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
