Hey Paul, Thanks for all the details!
On Fri, Jul 4, 2014 at 2:41 PM, Paul Osmialowski <[email protected]> wrote: > > Hi Todd, > > As I promissed: > > Host: Gentoo Linux x86_64 (a.k.a. amd64) > Target: FoundationV8 ARM64 emulator running minimal Linux rootfs image > Objective: cross-compile as much as possible of llvm + clang + lldb + > compiler_rt from master branch of their repositories for ARM64/AArch64 > architecture then try to execute lldb and lldb-gdbserver on the Target; try > to fulfil ncurses and Python dependency > > Okay, good to know what you're shooting for. I'll be working in this same area very shortly since I want to get Linux ARM64 support working, later for Android. I had worked briefly with ARM on getting the FoundationV8 simulator running a Linaro build back when they started packaging the ARMv8 simulator with DS-5. I had to jump away from it, but coming back to it very soon now that llgs is starting to get some traction. > I'm using CMake throughout all the process. Note that I'm building > everything in home directory and (with all required development tools > installed), I don't need superuser privileges at any stage. > > Great. > Gentoo specific issues: > I used to use Gentoo for years - on both x86 and PowerPC (old Mac box) setups. Glad you're testing things out there! > 1. Currently, default C/C++ compiler for Gentoo systems is gcc 4.7.x. > Latest llvm/clang/lldb code requires at least gcc 4.8.x for proper > compilation. Gcc 4.8.3 must be installed in a separate slot > (Gentoo-specific approach for holding multiple versions of the software); > CC and CXX shell variables must be set for CMake to indicate which compiler > should be used > We'll need to fiddle maybe a bit with this. I *think* we should be able to get gcc 4.7.1+ working, but might require slightly different flags. If we don't accept gcc 4.7, I think (depending on the work needed) we may want to get that working anyway to stay at parity with minimum llvm/clang toolchains. > 2. get_python_lib Python function from distutils.sysconfig returns path > which contains '/lib64/' part no matter how first two boolean parameters > are set (see revision D4058 for fix proposed by me; also patch attached to > bug report 20206 touches crucial lines of one of affected files); this is > important only when you plan to do 'make install' (and my approach relys on > this) > 3. Python 2.7 in Gentoo is compiled with unicode ucs4, while Python 2.7 > installed on the Target is compiled with unicode ucs2 - separate local > Python installation must be used during cross-compilation in order to make > swig happy (installation procedure will be described later). > 4. Python 2.7 is not the default Python interpreter in Gentoo (currently it > is Python 3.x) while lldb requires version 2.7; fortunately, many different > versions of Python interpreter can co-exist in one system - whenever > required, exact version can be specified to use. > > Ok. Realistically I'll be setting up my environment on Ubuntu 14.04 x86_64 (our current dev platform over here). However, I will see what happens when I use gcc 4.7.{latest}. If that's not too hard to fix up, I'll see if I can get some cycles on that. > Target specific issues: > 1. The target architecture is recognised as aarch64 not ARM64, I prepared > two revisions that try address this issue: D4381 and D4379. > Yeah, I think on the lldb side we're going to want to normalize those to arm64, and deal with ABI differences based on target triples. I'll have a look at your patches when I get there. > 2. Ncurses library installed on target is not complete, i.e. libpanel part > is missing; also two other required libraries aren't there: libedit and > libxml2. > Hmm. If all you want on the ARM64 target is the lldb-gdbserver, then this issue might go away when we do the static linking of lldb-gdbserver and lldb-platform in the near future (some cmake/configure/make changes). That's a little ways off but we're starting to work on it. The goal there is to remove the llgs <=> liblldb.so linkage, so we can do tight static linking and dead-code eliminate the majority of the core lldb code that isn't needed for lldb-gdbserver. We may also do the same for lldb-platform, although for our direct application this is not critical path. > 3. Python 2.7 installed on the Target is compiled with unicode ucs2. > > Two stages process: > 1. Building for PC > 2. Building for Target > > This process is split into two stages since the second stage requires > llvm-tblgen and clang-tblgen built at the first stage (and since we're > working with the most recent code, older versions of those two cannot be > used). > > Getting repositories > > In my home directory I'm having following directory structure: > > $HOME/llvm.git - llvm repository cloned > $HOME/llvm.git/projects/compiler-rt - compiler-rt repository cloned > $HOME/llvm.git/tools/clang - clang repository cloned > $HOME/llvm.git/tools/lldb - lldb repository cloned, that's where the magic > happens > > Building for PC > =============== > > I'm building the stuff for PC in $HOME/llvm/build directory (contrary to > ARM64 stuff which I'm building in $HOME/llvm-aarch64/build directory). > > Being in previously created $HOME/llvm/build directory I'm invoking what > follows: > > $ CC=gcc-4.8.3 \ > CXX=g++-4.8.3 \ > CFLAGS=-pipe \ > CXXFLAGS=-pipe \ > cmake \ > -DPYTHON_EXECUTABLE=/usr/bin/python2.7 \ > -DCMAKE_INSTALL_PREFIX=$HOME/llvm \ > ../../llvm.git > > Note that you can select different CMAKE_INSTALL_PREFIX, or you can skip > installation prefix selection at all so the outcome is to be installed > (with sudo) in /usr/local by default. > PYTHON_EXECUTABLE specifies which Python interpreter should be used. > > After running cmake (by invoking command above), now make can be invoked, > e.g.: > > $ VERBOSE=1 make -j8 > > (make sure you have a lot of RAM, 12GB or more, or use -j2 instead of -j8). > > Then, when build is successfully completed: > > $ make install > > And that's it. > > Building for ARM64 > ================== > > The Linaro toolchain with gcc 4.8.2 for AArch64 > (gcc-linaro-aarch64-linux-gnu-4.8-2013.08_linux.tar.xz) I'm having > unpacked into $HOME/aarch64/gcc-linaro-aarch64-linux-gnu directory, i.e. > gcc executable binary is: > > $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc-4.8.2 > > Due to target specific issues listed above, some dependencies must be > cross-compiled first. In my home directory I'm initially having this: > > $HOME/llvm-aarch64/deps/src/crosscompileconf.sh > $HOME/llvm-aarch64/deps/src/libedit-20130712-3.1.tar.gz > $HOME/llvm-aarch64/deps/src/libxml2-2.9.1.tar.gz > $HOME/llvm-aarch64/deps/libpython2.7.so.1.0 > $HOME/llvm-aarch64/deps/libpython2.7.so.1 > $HOME/llvm-aarch64/deps/libpython2.7.so > $HOME/llvm-aarch64/deps-x86_64/src/confpython.sh > $HOME/llvm-aarch64/deps-x86_64/src/Python-2.7.6.tar.xz > > The $HOME/llvm-aarch64/deps/libpython2.7.so.1.0 file was obtained form > Target's rootfs. $HOME/llvm-aarch64/deps/libpython2.7.so.1 and > $HOME/llvm-aarch64/deps/libpython2.7.so are just symlinks: > > libpython2.7.so -> libpython2.7.so.1.0 > libpython2.7.so.1 -> libpython2.7.so.1.0 > > $ file libpython2.7.so.1.0 > libpython2.7.so.1.0: ELF 64-bit LSB shared object, ARM aarch64, version 1 > (SYSV), dynamically linked, > BuildID[sha1]=c6b21fd1aa4225ed9cd979a0dd7b59d1cbd5f281, > stripped > $ $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objdump > -T libpython2.7.so |grep -i ucs | head -1 > 00000000000bbba0 g DF .text 0000000000000054 Base _PyUnicodeUCS2_ > AsDefaultEncodedString > > It's clear it was built with unicode ucs2. > > It turned out that in order to build ncurses for AArch64, the very latest > snapshot of ncurses from git is required. To have it, go to > $HOME/llvm-aarch64/deps/src and type: > > $ git clone https://github.com/chitranshi/ncurses.git ncurses.git > > This leaves you with following directory: > > $HOME/llvm-aarch64/deps/src/ncurses.git > > In my case git SHA was: > > $ cd ncurses.git > $ git rev-parse HEAD > 79bb76ec205843adec46225f64aaa1f8e730b466 > > My $HOME/llvm-aarch64/deps/src/crosscompileconf.sh file looks like this > (forgive me poor mail formatting, I can send you that file separately if > needed): > > PATH=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin:$PATH \ > PKG_CONFIG_PATH=$HOME/llvm-aarch64/deps/lib/pkgconfig:$PKG_CONFIG_PATH \ > CC=aarch64-linux-gnu-gcc-4.8.2 \ > CXX=aarch64-linux-gnu-g++ \ > CFLAGS="-fPIC -DPIC > --sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc > -isystem > $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include > -I$HOME/llvm-aarch64/deps/include/ncurses" \ > CXXFLAGS="-fPIC -DPIC --sysroot=$HOME/aarch64/gcc- > linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc -isystem > $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include > -I$HOME/llvm-aarch64/deps/include/ncurses" \ > LDFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64- > linux-gnu/aarch64-linux-gnu/libc -L$HOME/llvm-aarch64/deps/lib" \ > LIBS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc > -L$HOME/llvm-aarch64/deps/lib" \ > ./configure --prefix=$HOME/llvm-aarch64/deps --host=aarch64-linux-gnu > --without-python --without-ada --disable-shared --enable-static > --without-shared --without-cxx-shared --enable-safe-sprintf > > > Note that all the dependencies I'm having as static libraries. > > $HOME/llvm-aarch64/deps-x86_64/src/confpython.sh looks like this: > > ./configure --prefix=$HOME/llvm-aarch64/deps-x86_64 --enable-unicode=ucs2 > > > To build Python with unicode ucs2, unpack Python-2.7.6.tar.xz in > $HOME/llvm-aarch64/deps-x86_64/src and go to unpacked > $HOME/llvm-aarch64/deps-x86_64/src/Python-2.7.6 directory. > > Invoke configuration script: > > $ source ../confpython.sh > > When finished type: > > $ make > > and when it's done type: > > $ make install > > This should leave you with one more Python 2.7 interpreter: > $HOME/llvm-aarch64/deps-x86_64/bin/python2.7 and its companion files. > > > To build libmxl2, go to $HOME/llvm-aarch64/deps/src and unpack, configure, > build and install it in $HOME/llvm-aarch64/deps: > > $ tar -xzf libxml2-2.9.1.tar.gz > $ cd libxml2-2.9.1 > $ source ../crosscompileconf.sh > $ make > $ make install > > To build ncurses go to $HOME/llvm-aarch64/deps/src/ncurses.git and type: > > $ source ../crosscompileconf.sh > $ make > $ make install > > Now with ncurses ready, you can build libedit, go to > $HOME/llvm-aarch64/deps/src and do: > > $ tar -xzf libedit-20130712-3.1.tar.gz > $ cd libedit-20130712-3.1 > $ source ../crosscompileconf.sh > $ make > $ make install > > After this, build environment is ready to use. Examine > $HOME/llvm-aarch64/deps directory in order to make sure things are there in > bin, include, lib and share directories. > > Create and go to $HOME/llvm-aarch64/build directory and type: > > $ PATH=$HOME/llvm-aarch64/deps/bin:$HOME/aarch64/gcc-linaro- > aarch64-linux-gnu/bin:$HOME/llvm-aarch64/deps-x86_64/bin:$PATH \ > CC=aarch64-linux-gnu-gcc-4.8.2 \ > CXX=aarch64-linux-gnu-g++ \ > CFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64- > linux-gnu/aarch64-linux-gnu/libc -isystem $HOME/aarch64/gcc-linaro- > aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include > -I$HOME/llvm-aarch64/deps/include -I$HOME/llvm-aarch64/deps/include/ncurses > -I$HOME/llvm-aarch64/deps-x86_64/include/python2.7" \ > CXXFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64- > linux-gnu/aarch64-linux-gnu/libc -isystem $HOME/aarch64/gcc-linaro- > aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include > -I$HOME/llvm-aarch64/deps/include -I$HOME/llvm-aarch64/deps/include/ncurses > -I$HOME/llvm-aarch64/deps-x86_64/include/python2.7" \ > LDFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64- > linux-gnu/aarch64-linux-gnu/libc -L$HOME/llvm-aarch64/deps/lib > -L$HOME/llvm-aarch64/deps" \ > LIBS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc > -L$HOME/llvm-aarch64/deps/lib -L$HOME/llvm-aarch64/deps" \ > PKG_CONFIG_PATH=$HOME/llvm-aarch64/deps/lib/pkgconfig:$ > HOME/llvm-aarch64/deps-x86_64/lib/pkgconfig \ > cmake \ > -DCMAKE_CROSSCOMPILING=True \ > -DPYTHON_EXECUTABLE=$HOME/llvm-aarch64/deps-x86_64/bin/python2.7 \ > -DCMAKE_LINKER=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ld > \ > -DCMAKE_AR=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ar > \ > -DCMAKE_NM=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-nm > \ > -DCMAKE_OBJCOPY=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objcopy > \ > -DCMAKE_OBJDUMP=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objdump > \ > -DCMAKE_STRIP=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-strip > \ > -DCMAKE_RANLIB=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ranlib > \ > -DCMAKE_INSTALL_PREFIX=$HOME/llvm-aarch64 \ > -DLLVM_TABLEGEN=$HOME/llvm/bin/llvm-tblgen \ > -DCLANG_TABLEGEN=$HOME/llvm/build/bin/clang-tblgen \ > -DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-linux-gnu \ > -DLLVM_TARGET_ARCH=AArch64 \ > -DLLDB_DISABLE_CURSES=OFF \ > -DLLDB_DISABLE_PYTHON=OFF \ > -DPYTHON_LIBRARY=$HOME/llvm-aarch64/deps/libpython2.7.so.1.0 \ > ../../llvm.git > > Note that CLANG_TABLEGEN points to a binary file held in build directory. > I've already posted a bug 19908 about that. > > After running cmake (by invoking command above), now make can be invoked, > e.g.: > > $ VERBOSE=1 make -j8 > > (make sure you have a lot of RAM, 12GB or more, or use -j2 instead of -j8). > > Then, when build is successfully completed: > > $ make install > > And that's it. Copy $HOME/llvm-aarch64/lib/liblldb.so.3.5.0, > $HOME/llvm-aarch64/bin/lldb-3.5.0 (as lldb) and > $HOME/llvm-aarch64/bin/lldb-gdbserver-3.5.0 (as lldb-gdbserver or llgs > for short) to your Target. > Note that these binaries are huge. Don't repeat my mistake and execute > them from NFS exported directory. Copy to local filesystem before executing. > > # TERMINFO=/etc/terminfo LD_LIBRARY_PATH=. ./lldb > (lldb) gui > > The ncurses gui looks bit funny when started on serial terminal emulator. > > Have fun, > > Will do, thanks for the details! Sincerely, Todd Fiala > Paul > > On Thu, 3 Jul 2014, Todd Fiala wrote: > > Thanks, Paul! No rush, just would love to see what that looks like so I >> can replicate it over here at some point. >> >> >> On Thu, Jul 3, 2014 at 2:14 PM, Paul Osmialowski <[email protected]> >> wrote: >> Hi Todd, >> >> I'll describe it soon (I guess tomorrow). >> >> Cheers, >> Paul >> >> On Thu, 3 Jul 2014, Todd Fiala wrote: >> >> Hey Paul, >> I was wondering if you could post your cross compile Linaro >> setup. That would be great as a starting point as I look into >> testing/reworking some of the Linux constants and start >> gearing up on some alternate >> architecture support. >> >> Much appreciated! >> >> Sincerely, >> Todd Fiala >> >> >> >> >> >> -- >> -Todd >> >> -- -Todd
_______________________________________________ lldb-dev mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
