Laundry list of cross compilation issues
First off, thanks to everyone who tried to help me with my strange cross compilation endeavor. I am happy to say that I finally got it working. Although I cannot post proper patches to fix the cross issues, still I would like to provide a starting point, in case someone else wants to clean this up or just build cross. I built version 4.4.0.3 under ptxdist with the following configure environment ... $(CROSS_ENV) \ CFLAGS=-I$(SYSROOT)/usr/include \ CXXFLAGS=-I$(SYSROOT)/usr/include \ LDFLAGS=-L$(SYSROOT)/usr/lib -L$(SYSROOT)/lib -Wl,-rpath-link,$(SYSROOT)/usr/lib \ X_LIBS=-lxcb -lXau -lXdmcp -lX11 -lICE -lSM and these configure options. --host=x86_64-unknown-linux-gnu \ --build=x86_64-host-linux-gnu \ --disable-cups \ --disable-dbus \ --disable-firebird-sdbc \ --disable-gconf \ --disable-gtk \ --disable-odk \ --disable-pdfimport \ --disable-postgresql-sdbc \ --disable-randr \ --enable-python=no \ --with-lang=en-US de \ --with-system-cairo \ --with-system-zlib=no \ --without-help \ --without-java \ --without-system-dicts \ --x-includes=$(SYSROOT)/usr/include \ --x-libraries=$(SYSROOT)/usr/lib Now, to the trouble spots: * configure.ac This script has lots of issues, most of which I hacked around. The whole CONF-FOR-BUILD thing is, well, creative. The unset does not go far enough. In my case I added: unset \ AR AS LD NM CC CXX CPP RANLIB READELF OBJCOPY OBJDUMP STRIP DLLTOOL \ GNAT GNATBIND GNATCHOP GNATCLEAN GNATFIND GNATKR GNATLINK GNATLS \ GNATMAKE GNATNAME GNATPREP GNATXREF CC_FOR_BUILD CPP_FOR_BUILD \ LINK_FOR_BUILD SYSROOT PKG_CONFIG_PATH PKG_CONFIG_LIBDIR PKG_CONFIG \ enable_option_checking enable_maintainer_mode enable_static \ ac_cv_file__dev_random ac_cv_file__proc_self_exe \ ac_cv_file__proc_self_fd ac_cv_file__proc_self_maps \ ac_cv_func_dcgettext ac_cv_func_getpgrp_void ac_cv_func_getrlimit \ ac_cv_func_malloc_0_nonnull ac_cv_func_memcmp_clean \ ac_cv_func_memcmp_working ac_cv_func_nonposix_getgrgid_r \ ac_cv_func_nonposix_getpwuid_r ac_cv_func_posix_getgrgid_r \ ac_cv_func_posix_getpwnam_r ac_cv_func_posix_getpwuid_r \ ac_cv_func_printf_unix98 ac_cv_func_realloc_0_nonnull \ ac_cv_func_setgrent_void ac_cv_func_setpgrp_void \ ac_cv_func_setvbuf_reversed ac_cv_func_strstr ac_cv_func_strtod \ ac_cv_func_strtoul ac_cv_func_vsnprintf_c99 ac_cv_func_wait3_rusage \ ac_cv_have_abstract_sockets ac_cv_lib_c_inet_aton \ ac_cv_sizeof_long_double ac_cv_sizeof_long_long ac_cv_sysv_ipc \ ac_cv_type_uintptr_t bash_cv_func_ctype_nonascii \ bash_cv_func_sigsetjmp bash_cv_func_strcoll_broken \ bash_cv_must_reinstall_sighandlers glib_cv_long_long_format \ gt_cv_func_gettext_libintl ac_cv_func_fork ac_cv_func_fork_works \ ac_cv_func_iconv_open CPPFLAGS CFLAGS CXXFLAGS LDFLAGS X_LIBS This corresponds to the standard CROSS_ENV provided by ptxdist. Also, the PATH needs reseting in order to avoid the cross tools. I added --disable-gconf \ --disable-dbus \ --disable-gtk \ --disable-gstreamer-1-0 \ --enable-python=no \ to the configure options. Probably this list could be expanded. The tests for $cross_compiling are pretty messed up, especially WRT cppunit and python. The PATH from the enviroment is not getting through the script into the config_host.mk as it should. The same goes for CPPFLAGS. My work around was to fix the PATH after the fact, and to put the nedded CPPFLAGS into CFLAGS and CXXFLAGS. * icu 53 Some of the objects (sorry didn't keep notes which ones) are built using the system headers, which in my case where icu version 52. This happens even when using the external icu. If the system lacks the icu headers, then LO cannot build. I got tired of playing whack-the-mole and just reverted the LO patch that upgraded to 53. * external/glew ** ExternalProject_glew.mk - pass CFLAGS to CFLAGS.EXTRA - pass LDFLAGS to LDFLAGS.EXTRA * external/libgltf ** ExternalProject_libgltf.mk - configure with --build and --host * external/nss ** ExternalProject_nss.mk - needs --enable-64bit to avoid -m32 (which my cross gcc doesn't have) - needs to set NSINSTALL and NSDISTMODE on the make command line, not in the environment. - picks the wrong macros for atomics in nspr/pr/include/md/_linux.h - should not try to sign libraries using cross compiled binaries nss/cmd/shlibsign/sign.sh * external/redland ** ExternalProject_raptor.mk - pass CFLAGS, LIBXML_CFLAGS, and LDFLAGS ** ExternalProject_rasqal.mk - pass CFLAGS and LDFLAGS ** ExternalProject_redland.mk - pass CFLAGS and LDFLAGS Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: current fresh release not tagged in git?
On Mon, May 18, 2015 at 01:50:42PM +0200, Richard Cochran wrote: On Mon, May 18, 2015 at 12:21:24PM +0200, Rene Engelhard wrote: Forgot git fetch --tags? I did 'git remote update origin', and I thought that includes tags. Now I see. The tag 'libreoffice-4.4.3.2' is not the branch called 'libreoffice-4-4-3'. In fact, is isn't on any branch at all, AFAICT. Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
current fresh release not tagged in git?
Dear list, It appears that LO Fresh 4.4.3 is not tagged in git. git tag | grep libreoffice-4 ... libreoffice-4.4.0.1 libreoffice-4.4.0.2 libreoffice-4.4.0.3 libreoffice-4.4.1.1 libreoffice-4.4.1.2 libreoffice-4.4.2.1 git branch -a | grep libreoffice-4 ... remotes/origin/libreoffice-4-4 remotes/origin/libreoffice-4-4-0 remotes/origin/libreoffice-4-4-1 remotes/origin/libreoffice-4-4-2 remotes/origin/libreoffice-4-4-3 Or am I missing something? Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Mon, May 11, 2015 at 05:57:38PM +0200, Michael Stahl wrote: On 08.05.2015 16:53, Miklos Vajna wrote: In any case: if test $cross_compiling != yes; then libo_CHECK_SYSTEM_MODULE([cppunit],[CPPUNIT],[cppunit = 1.12.0]) fi in configure.ac where cppunit is disabled in case of cross-compiling. If I remove this test, then it works. the problem is that sal/Executable_cppunittester.mk had a different conditional, it was wrongly checking for DESKTOP so it's not built on Android/iOS but is built for any other cross-compilation target, fixed on master with 26ebcb023ef3d38956085cbb39416e6f148ab351 This does not work for me (cherry picked onto 4403). -- /mnt/storage/git/libreoffice/unotest/source/cpp/unoexceptionprotector/unoexceptionprotector.cxx:28:29: fatal error: cppunit/Message.h: No such file or directory #include cppunit/Message.h ^ compilation terminated. /mnt/storage/git/libreoffice/solenv/gbuild/LinkTarget.mk:189: recipe for target '/mnt/storage/git/libreoffice/workdir/CxxObject/unotest/source/cpp/unoexceptionprotector/unoexceptionprotector.o' failed make[1]: *** [/mnt/storage/git/libreoffice/workdir/CxxObject/unotest/source/cpp/unoexceptionprotector/unoexceptionprotector.o] Error 1 make[1]: *** Waiting for unfinished jobs Makefile:237: recipe for target 'build' failed make: *** [build] Error 2 Sorry, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: current fresh release not tagged in git?
On Mon, May 18, 2015 at 12:21:24PM +0200, Rene Engelhard wrote: Forgot git fetch --tags? I did 'git remote update origin', and I thought that includes tags. Hm. Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Mon, May 11, 2015 at 06:41:06AM +0200, Andras Timar wrote: On Fri, May 8, 2015 at 5:09 PM, Richard Cochran richardcoch...@gmail.com wrote: I specifically did 'make build' in order to avoid the tests, but the build system compiles the tests anyhow. The 'build' target includes tests. Try the 'build-nocheck' target. Thanks, that is very useful to know. BTW, the output of 'make help' says, AVAILABLE TARGETS all build product and run unit tests (default goal) buildbuild product ... which implies that 'build = all - unittests'. No mention is make of 'build-nocheck' at all. Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Mon, May 11, 2015 at 08:56:31AM +0200, Richard Cochran wrote: Thanks, that is very useful to know. Actually, that target fails in the same way as 'build' does: In file included from /mnt/storage/git/libreoffice/sal/cppunittester/cppunittester.cxx:28:0: /mnt/storage/git/libreoffice/include/cppunittester/protectorfactory.hxx:24:31: fatal error: cppunit/Protector.h: No such file or directory #include cppunit/Protector.h ^ compilation terminated. It appears that sal/cppunittester is always built. Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Fri, May 08, 2015 at 07:07:01PM +0200, Miklos Vajna wrote: What's your cross-compilation target? x86_64 http://tinderbox.libreoffice.org/MASTER/status.html says that the Android and iOS cross-compilation targets work fine (those columns are green), as far as I see. That doesn't surprise me. The makefiles for the externals are full of special cases for those two targets. What is lacking is any kind of general cross compilation concept. (I am not saying there should be one though, just stating a fact.) Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Fri, May 08, 2015 at 08:07:19PM +0300, Tor Lillqvist wrote: Well, we have tinderbox slaves that cross-compile for Android and iOS constantly, so it can't be totally broken. Those only work because the makefiles are full of special hacks for those two targets. For example: find external -name \*mk | xargs grep -E 'IOS|ANDROID' | wc -l 75 At the very least, you would need to be consistent about passing --host and --build to the externals: find external -name ExternalProject\*mk | wc -l 73 find external -name ExternalProject\*mk | xargs grep CROSS_COMPILING | wc -l 52 So you need to be more specific. Also whether something is considered broken or not depends somewhat on whether it is even promised anywhere to work, don't you think? From previous experience with large C++ projects, I didn't really expect LO to cross compile at all, and I am not saying that it aught to. As you should have noticed, LibreOffice is far from some typical small Open Source library using GNU auto everything that would be cross-compilable by simple passing --host and --build options to the configure script. LibreOffice's configury and build system is quite complex. (But then, so is the build system of most *large* Open Source software packages.) We don't promise anywhere that arbitrary cross-compilation would work. It doesn't work in general. That is not so bad, but it would be better to specifically limit the configuration choices to the supported (and tested!) targets. The same goes for the many --without-foo and --disable-bar options. Many of these don't work. You don't even tell what the host platform for which you are cross-compiling is. x86_64 Sure, we don't have anything testing cross-compilation from one Unix to another Unix, for instance (like from x86_64 Linux to MIPS Linux, etc), so if that is broken, that is not surprising. I think at some stage a few of us managed to cross-compile from x86 (or x86_64) Linux to Raspberry Pi ARM Linux, but that was last year and many things might have bit-rotted since. Yup, I can see the bit rot. I think it would be a lot of work to make the LO build system truly support cross. I can't do that myself, and so I am not complaining. I don't even think cross compiling is a good idea. In this day and age, most embedded ARM target can just use the (natively built) debian packages. But I will have to somehow get this working (with hacks) for one particular target, and I do appreciate the helpful suggestions! Once I am done I will have a better idea of the magnitude of the cross issues. I can at least report what I found, if that helps. Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Sat, May 09, 2015 at 02:31:16PM +0200, David Tardon wrote: At the very least, you would need to be consistent about passing --host and --build to the externals: Cross-compilation support for the bundled projects was added to these that needed to be cross-compiled. Doing it for all of them would be a lot of effort for no immediate gain. Instead, people interested in cross-compilation are expected to fix things themselves. All of the libraries used on the target machine need to be cross compiled. What is your motivation then? The customer wants this. The same goes for the many --without-foo and --disable-bar options. Many of these don't work. Examples, please? --enable-python=no There are others, too. Sorry, I've been through so many bad combinations, but I didn't keep notes about this. (I do have more specifics about configure.ac and the makefiles.) Like I said, at the end I'll try to give feedback about where the issues are, but I don't expect being able to really clean this all up. That would be a ton of work. x86_64 Why?! Well, the customer is always right, you know. ;^) Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Sat, May 09, 2015 at 06:59:47PM +0300, Tor Lillqvist wrote: I think you missed the meaning of host in GNU autoconf terminology;) (np, I confuse those constantly myself, too, unless I am ver careful) Yes, the gnu terminology is confusing to me, too. But no, this time I am not confused! Both the build system and the host system are x86_64, but the host is using the OSELAS compiler and the ptxdist BSP. I know that is very strange, and I don't recommend it at all, but most of the problems I am seeing will also appear when using an ARM cross toolchain. Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Build dependency on external cppunit
Dear list, I am trying to build Version 4.4.0.3 with a -j30 parallel build, but I am running into this error. In file included from /git/libreoffice/sal/cppunittester/cppunittester.cxx:28:0: /git/libreoffice/include/cppunittester/protectorfactory.hxx:24:31: fatal error: cppunit/Protector.h: No such file or directory #include cppunit/Protector.h I am not using '--with-system-cppunit'. Questions: 1. Where is the dependency from sal-cppunit specified? 2. How can I force a build of cppunit (and other externals)? Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Fri, May 08, 2015 at 02:48:53PM +0200, Miklos Vajna wrote: Hi Richard, On Fri, May 08, 2015 at 02:12:45PM +0200, Richard Cochran richardcoch...@gmail.com wrote: Can't I just force cppunit to build manually beforehand? make cppunit.all Tried that already, too. Here is what I got: PARALLELISM=1 make cppunit.all GMAKE_OPTIONS='VERBOSE=1' make[1]: Entering directory '/mnt/storage/git/libreoffice' ... mkdir -p /mnt/storage/git/libreoffice/workdir/Dep/CxxObject/UnoControls/source/base/ mkdir -p /mnt/storage/git/libreoffice/workdir/Dep/CxxObject/UnoControls/source/controls/ [build MOD] cppunit S=/mnt/storage/git/libreoffice I=$S/instdir W=$S/workdir mkdir -p $W/Module/ touch $W/Module/cppunit make[1]: Leaving directory '/mnt/storage/git/libreoffice' The 'touch' is the only result! I should probably add that I am trying to cross compile LO (I know, I know, don't ask why), and it is clear that building cross is not fully implemented, but in this case I think it should still work. Should do that, though you just hide the root cause with that. :-) Yep. Unfortunately I am having a hard time understanding how this build system is supposed to work, otherwise I would fix it and send a patch! Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Fri, May 08, 2015 at 04:53:49PM +0200, Miklos Vajna wrote: On Fri, May 08, 2015 at 03:55:35PM +0200, Richard Cochran richardcoch...@gmail.com wrote: Ok, it is somehow related to the cross compile configuration. The 'make cppunit.all' works as expected with a native build. Isn't the very point of cross-compilation that the resulting binary is not something you can run? If so, I find it a feature that unit tests are not built / executed when cross-compiling. :-) Yes, *I* would say so, but the LO build thinks otherwise! I specifically did 'make build' in order to avoid the tests, but the build system compiles the tests anyhow. There are LOTS of other ways in which cross compile is broken. It seems clear that it really isn't supported or tested, and so I don't expect too much. My goal is just to get it working, somehow, for a particular setup. In any case: if test $cross_compiling != yes; then libo_CHECK_SYSTEM_MODULE([cppunit],[CPPUNIT],[cppunit = 1.12.0]) fi in configure.ac where cppunit is disabled in case of cross-compiling. Ok, I'll take a look. Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Fri, May 08, 2015 at 03:31:39PM +0200, Richard Cochran wrote: I should probably add that I am trying to cross compile LO (I know, I know, don't ask why), and it is clear that building cross is not fully implemented, but in this case I think it should still work. Ok, it is somehow related to the cross compile configuration. The 'make cppunit.all' works as expected with a native build. Sigh, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Build dependency on external cppunit
On Fri, May 08, 2015 at 12:08:23PM +0200, Stephan Bergmann wrote: On 05/08/2015 10:45 AM, Richard Cochran wrote: 1. Where is the dependency from sal-cppunit specified? in sal/Executable_cppunittester.mk: $(eval $(call gb_Executable_use_externals,cppunittester,\ boost_headers \ cppunit \ )) Hm, I saw that, but still the build breaks. The include path would be correct, but the cppunit tar file never gets unpacked. Any ideas? Can't I just force cppunit to build manually beforehand? Thanks, Richard ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
