Hi guys, thanks so much for taking the time to assist me during this effort. Will try to outline the overall steps needed to successfully build aurora components and deploy the hello_world of docker.
1. Made the configurations needed as outlined in the *provision-dev-cluster.sh* - From the *prepare_sources *I just picked up the configuration needed for the mesos-slaves and add them as flags on my start-mesos-slave.sh script, because I have built it already from source ( I couldn't use the mesosphere ppa with the ARM processor) - In *prepare_extras *I just changed it to point to my directory - Same for *install_cluster_config* - Kept *install_ssh_config *as it is - Same for *start_services* - Disabled *enable_gradle_daemon configure_netrc sudoless_docker_setup *as I don't think I need them 2. Changed the configurations for the upstart jobs: - In *aurora-scheduler.conf *I changed the references to my directory and IP/Hostname values. Moreover I had to edit as follows: env JAVA_OPTS='-Djava.library.path=/usr/lib:/usr/local/lib Because my libmesos.so was created there - In *aurora-thermos-observer.conf *I had to just change the directory 3. I had to copy all the *.egg files in the third_party directory as described by Florian and Stephan 4. Modified *aurorabuild.sh *all the pointers to my directory. Moreover I had to edit as follows: export LD_LIBRARY_PATH=/usr/lib/jvm/java-7-openjdk-armhf/jre/lib/arm/server 5. Followed the instructions by John to fix the issue with pants: - ./build-support/thrift/thriftw 0.9.1 --version - modified pants.ini to add [binaries] baseurls: +['http://localhost:9999'] path_by_id: {('linux', 'armv7l'): ['local']} - mkdir -p build-support/thrift/serve/bin/thrift/local/0.9.1 - ln -s $PWD/build-support/thrift/thrift-0.9.1/compiler/cpp/thrift build-support/thrift/serve/bin/thrift/local/0.9.1/thrift - cd build-support/thrift/serve && (nohup python2 -m SimpleHTTPServer 9999 >&/tmp/thrift.serve &) && cd - 6. Executed ./*aurorabuild.sh all* to build the components 7. Created my docker container since the example doesn't work out of the box as the instructions found here https://issues.apache.org/jira/browse/AURORA-1487 - cd examples/jobs/docker - Create Dockerfile as: FROM armv7/armhf-ubuntu:14.04.3 RUN apt-get update && apt-get -y install \ libapr1-dev \ libcurl4-nss-dev \ libsvn-dev \ python-all-dev - docker build -t aurora/docker:2.7 . - - Edit *hello_docker.aurora *as follows: with image = 'aurora/docker:2.7' - aurora job create devcluster/docker-test/devel/hello_docker ./hello_docker.aurora I have one question regarding the architecture. If I need to add another node as a "worker" (to contain observer+executor) what are the components I need to build in that case? It's just the build_executor and build_observer functions need to be called from the build script? Thanks a lot, Yiannis On 19 July 2016 at 09:39, Erb, Stephan <[email protected]> wrote: > Minor addition to the explanation of Florian: the folder in question has > to be called `third_party’ (mind the underscore). > > > > *From: *Florian Pfeiffer <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Tuesday 19 July 2016 at 10:19 > > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Build thermos executor on ARM > > > > if this hasn't changed somewhen during the last versions, you need to > create a directory "thirdparty" (written out.. don't mix it up with the > already existing "3rdparty" directory) and copy the egg there. > > > > 2016-07-18 18:09 GMT+02:00 Yiannis Gkoufas <[email protected]>: > > great, so if I understood correctly, it just needs the *.egg files right? > > since I have built it myself, what is the directory in which I would have > to copy them? > > > > On 18 July 2016 at 16:52, Erb, Stephan <[email protected]> > wrote: > > The Thermos executor is bundling the Mesos driver necessary for > communication with to the agent. To make this work, you have to build > Python egg files containing the Mesos driver. This is the script we > normally use to achieve this: > https://github.com/apache/aurora/blob/master/build-support/python/make-mesos-native-egg > > > > > > *From: *Yiannis Gkoufas <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Monday 18 July 2016 at 17:35 > > > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Build thermos executor on ARM > > > > So the error I am getting now is: > > > > 15:33:32 00:25 [chroot]INFO] Attempting to fetch thrift binary from: > https://dl.bintray.com/pantsbuild/bin/build-support/bin/thrift/local/0.9.1/thrift > ... > > INFO] Attempting to fetch thrift binary from: > http://localhost:9999/bin/thrift/local/0.9.1/thrift ... > > 100% .................................................. 20270 KB 0.285s > > INFO] Fetched thrift binary from: > http://localhost:9999/bin/thrift/local/0.9.1/thrift . > > > > Waiting for background workers to finish. > > 15:33:39 00:32 [complete] > > FAILURE > > Exception caught: (<class 'pex.resolver.Unsatisfiable'>) > > > > Exception message: Could not satisfy all requirements for > mesos.native==0.28.2: > > mesos.native==0.28.2 > > > > Which is kind of strange, because I have actually already installed mesos > 0.28.2 > > > > On 18 July 2016 at 16:16, John Sirois <[email protected]> wrote: > > On Jul 18, 2016 9:04 AM, "Yiannis Gkoufas" <[email protected]> wrote: > > > > Hi John, > > > > just to confirm: this is how this part of pants.ini should look like: > > > > backend_packages: [ > > [binaries] > > baseurls: +['http://localhost:9999'] > > path_by_id: {('linux', 'armv7l'): ['local']} > > ] > > > > right? > > Not quite, kill the surrounding backend_packages, so: > > [binaries] > baseurls: +['http://localhost:9999'] > path_by_id: {('linux', 'armv7l'): ['local']} > > > > > > > On 17 July 2016 at 11:58, Yiannis Gkoufas <[email protected]> wrote: > >> > >> Wow, thanks a lot John! > >> Will give it a shot and get back to you! > >> > >> On 15 July 2016 at 18:05, John Sirois <[email protected]> wrote: > >>> > >>> Here's a workaround for you to try. If this works, I can modify the > Aurora pants build without modifying pants itself to do all this > automatically. > >>> > >>> The basic idea is that pants probes a list of URLs for pre-built > binaries. You can add a local HTTP server to the list and point that > server at the thrift binary Aurora builds for gradle via make. > >>> Below is how you do this: > >>> > >>> 0.) Ensure the Aurora gradle thrift is built. > >>> $ ./build-support/thrift/thriftw 0.9.1 --version > >>> Thrift version 0.9.1 > >>> > >>> 1.) Just for reference, the pants options in-play here > >>> $ ./pants options | grep binaries > >>> binaries.baseurls = [' > https://dl.bintray.com/pantsbuild/bin/build-support'] (from HARDCODED) > >>> binaries.fetch_timeout_secs = 30 (from HARDCODED) > >>> binaries.path_by_id = {} (from NONE) > >>> $ ./pants options | grep binary > >>> thrift-binary.supportdir = bin/thrift (from HARDCODED) > >>> thrift-binary.version = 0.9.1 (from CONFIG in pants.ini) > >>> > >>> 2.) Grab machine name pants will try to find your thrift binary under > (here is an example from my machine with result 'x86_64' - yours will be > different and arm-ey): > >>> $ python2 -c 'import os; sysname, _, _, _, machine = os.uname(); print > sysname.lower(), machine' > >>> linux x86_64 > >>> > >>> 3.) Edit pants.ini to tell pants where to look for binaries for your > machine: > >>> $ git diff -U1 > >>> diff --git a/pants.ini b/pants.ini > >>> index de0ab30..e2cd94e 100644 > >>> --- a/pants.ini > >>> +++ b/pants.ini > >>> @@ -32,2 +32,6 @@ backend_packages: [ > >>> > >>> +[binaries] > >>> +baseurls: +['http://localhost:9999'] > >>> +path_by_id: {('linux', 'x86_64'): ['local']} > >>> + > >>> [thrift-binary] > >>> > >>> 4.) Setup the URL you told pants about above: > >>> $ mkdir -p build-support/thrift/serve/bin/thrift/local/0.9.1 > >>> $ ln -s $PWD/build-support/thrift/thrift-0.9.1/compiler/cpp/thrift > build-support/thrift/serve/bin/thrift/local/0.9.1/thrift > >>> $ cd build-support/thrift/serve && (nohup python2 -m SimpleHTTPServer > 9999 >&/tmp/thrift.serve &) && cd - > >>> > >>> 5.) Run pants and confirm download of a thrift binary for your machine: > >>> $ ./pants binary src/main/python/:: > >>> > >>> 10:58:44 00:00 [main] > >>> (To run a reporting server: ./pants server) > >>> 10:58:44 00:00 [setup] > >>> 10:58:44 00:00 [parse] > >>> Executing tasks in goals: binary > >>> 10:58:44 00:00 [binary] > >>> 10:58:44 00:00 [binary-prep-command] > >>> 10:58:44 00:00 [python-binary-create] > >>> 10:58:44 00:00 [cache] > >>> No cached artifacts for 8 targets. > >>> Invalidated 8 targets. > >>> 10:58:44 00:00 [chroot]INFO] Attempting to fetch thrift binary > from: http://localhost:9999/bin/thrift/local/0.9.1/thrift ... > >>> 100% .................................................. 31661 KB 0.364s > >>> INFO] Fetched thrift binary from: > http://localhost:9999/bin/thrift/local/0.9.1/thrift . > >>> ... > >>> > >>> Let me know what you get! > >>> > >>> On Thu, Jul 14, 2016 at 6:24 AM, Yiannis Gkoufas <[email protected]> > wrote: > >>>> > >>>> Cool, can you imagine a way to work around this? > >>>> I have to customize pants itself right? > >>>> > >>>> On 14 July 2016 at 11:54, Erb, Stephan <[email protected]> > wrote: > >>>>> > >>>>> Yeah, that sounds like it could be the issue here. > >>>>> > >>>>> > >>>>> > >>>>> From: Yiannis Gkoufas <[email protected]> > >>>>> Reply-To: "[email protected]" <[email protected]> > >>>>> Date: Thursday 14 July 2016 at 12:28 > >>>>> To: "[email protected]" <[email protected]> > >>>>> Subject: Re: Build thermos executor on ARM > >>>>> > >>>>> > >>>>>> > >>>>>> Hi there, > >>>>>> > >>>>>> > >>>>>> > >>>>>> sure: > >>>>>> > >>>>>> > >>>>>> > >>>>>> 10:26:45 00:00 [binary] > >>>>>> > >>>>>> 10:26:45 00:00 [binary-jvm-prep-command] > >>>>>> > >>>>>> 10:26:46 00:01 [jvm_prep_command] > >>>>>> > >>>>>> 10:26:46 00:01 [binary-prep-command] > >>>>>> > >>>>>> 10:26:46 00:01 [python-binary-create] > >>>>>> > >>>>>> Detected interpreter /usr/bin/python2.7: > CPython-2.7.6 > >>>>>> > >>>>>> 10:26:46 00:01 [chroot] > >>>>>> > >>>>>> Building chroot for > [PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), > thermos_executor))]: > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), _executor)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(api/src/main/thrift/org/apache/aurora/gen/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), gen)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/common/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/config/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/config/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/common/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/core/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), core)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/monitoring/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), monitoring)) > >>>>>> > >>>>>> Dumping library: > PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), > thermos_executor)) > >>>>>> > >>>>>> FAILURE: Update --binaries-path-by-id to find binaries for Linux > armv7l 3.10.40 > >>>>>> > >>>>>> > >>>>>> > >>>>>> If I am not completely wrong, I think pants is looking one of the > architectures outlined here: > >>>>>> > >>>>>> > >>>>>> > >>>>>> > https://github.com/pantsbuild/pants/blob/master/src/python/pants/binaries/binary_util.py#L24 > >>>>>> > >>>>>> > >>>>>> > >>>>>> and probably that's why it fails > >>>>>> > >>>>>> > >>>>>> > >>>>>> On 14 July 2016 at 11:17, Erb, Stephan <[email protected]> > wrote: > >>>>>>> > >>>>>>> Cool, ARM J. Please try to run it like this, so that we get a > little bit more output: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> PEX_VERBOSE=1 ./pants -ldebug binary > src/main/python/apache/aurora/executor:thermos_executor > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> From: Yiannis Gkoufas <[email protected]> > >>>>>>> Reply-To: "[email protected]" <[email protected]> > >>>>>>> Date: Thursday 14 July 2016 at 10:44 > >>>>>>> To: "[email protected]" <[email protected]> > >>>>>>> Subject: Build thermos executor on ARM > >>>>>>> > >>>>>>> > >>>>>>>> > >>>>>>>> Hi guys, > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> I have been trying to get aurora working on ARM processors. > >>>>>>>> > >>>>>>>> I had no problem with the scheduler, but I didn't manage to > compile the thermos executor. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> When I give: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> ./pants binary > src/main/python/apache/aurora/executor:thermos_executor > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> I get back: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> FAILURE: Update --binaries-path-by-id to find binaries for Linux > armv7l 3.10.40 > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Let me know if you have any hints on how to workaround this. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Thanks! > >>>>>> > >>>>>> > >>>> > >>>> > >>> > >> > > > > > > > > > >
