great, so if I understood correctly, it just needs the *.egg files right? since I have built it myself, what is the directory in which I would have to copy them?
On 18 July 2016 at 16:52, Erb, Stephan <[email protected]> wrote: > The Thermos executor is bundling the Mesos driver necessary for > communication with to the agent. To make this work, you have to build > Python egg files containing the Mesos driver. This is the script we > normally use to achieve this: > https://github.com/apache/aurora/blob/master/build-support/python/make-mesos-native-egg > > > > > > *From: *Yiannis Gkoufas <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Monday 18 July 2016 at 17:35 > > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Build thermos executor on ARM > > > > So the error I am getting now is: > > > > 15:33:32 00:25 [chroot]INFO] Attempting to fetch thrift binary from: > https://dl.bintray.com/pantsbuild/bin/build-support/bin/thrift/local/0.9.1/thrift > ... > > INFO] Attempting to fetch thrift binary from: > http://localhost:9999/bin/thrift/local/0.9.1/thrift ... > > 100% .................................................. 20270 KB 0.285s > > INFO] Fetched thrift binary from: > http://localhost:9999/bin/thrift/local/0.9.1/thrift . > > > > Waiting for background workers to finish. > > 15:33:39 00:32 [complete] > > FAILURE > > Exception caught: (<class 'pex.resolver.Unsatisfiable'>) > > > > Exception message: Could not satisfy all requirements for > mesos.native==0.28.2: > > mesos.native==0.28.2 > > > > Which is kind of strange, because I have actually already installed mesos > 0.28.2 > > > > On 18 July 2016 at 16:16, John Sirois <[email protected]> wrote: > > On Jul 18, 2016 9:04 AM, "Yiannis Gkoufas" <[email protected]> wrote: > > > > Hi John, > > > > just to confirm: this is how this part of pants.ini should look like: > > > > backend_packages: [ > > [binaries] > > baseurls: +['http://localhost:9999'] > > path_by_id: {('linux', 'armv7l'): ['local']} > > ] > > > > right? > > Not quite, kill the surrounding backend_packages, so: > > [binaries] > baseurls: +['http://localhost:9999'] > path_by_id: {('linux', 'armv7l'): ['local']} > > > > > > > On 17 July 2016 at 11:58, Yiannis Gkoufas <[email protected]> wrote: > >> > >> Wow, thanks a lot John! > >> Will give it a shot and get back to you! > >> > >> On 15 July 2016 at 18:05, John Sirois <[email protected]> wrote: > >>> > >>> Here's a workaround for you to try. If this works, I can modify the > Aurora pants build without modifying pants itself to do all this > automatically. > >>> > >>> The basic idea is that pants probes a list of URLs for pre-built > binaries. You can add a local HTTP server to the list and point that > server at the thrift binary Aurora builds for gradle via make. > >>> Below is how you do this: > >>> > >>> 0.) Ensure the Aurora gradle thrift is built. > >>> $ ./build-support/thrift/thriftw 0.9.1 --version > >>> Thrift version 0.9.1 > >>> > >>> 1.) Just for reference, the pants options in-play here > >>> $ ./pants options | grep binaries > >>> binaries.baseurls = [' > https://dl.bintray.com/pantsbuild/bin/build-support'] (from HARDCODED) > >>> binaries.fetch_timeout_secs = 30 (from HARDCODED) > >>> binaries.path_by_id = {} (from NONE) > >>> $ ./pants options | grep binary > >>> thrift-binary.supportdir = bin/thrift (from HARDCODED) > >>> thrift-binary.version = 0.9.1 (from CONFIG in pants.ini) > >>> > >>> 2.) Grab machine name pants will try to find your thrift binary under > (here is an example from my machine with result 'x86_64' - yours will be > different and arm-ey): > >>> $ python2 -c 'import os; sysname, _, _, _, machine = os.uname(); print > sysname.lower(), machine' > >>> linux x86_64 > >>> > >>> 3.) Edit pants.ini to tell pants where to look for binaries for your > machine: > >>> $ git diff -U1 > >>> diff --git a/pants.ini b/pants.ini > >>> index de0ab30..e2cd94e 100644 > >>> --- a/pants.ini > >>> +++ b/pants.ini > >>> @@ -32,2 +32,6 @@ backend_packages: [ > >>> > >>> +[binaries] > >>> +baseurls: +['http://localhost:9999'] > >>> +path_by_id: {('linux', 'x86_64'): ['local']} > >>> + > >>> [thrift-binary] > >>> > >>> 4.) Setup the URL you told pants about above: > >>> $ mkdir -p build-support/thrift/serve/bin/thrift/local/0.9.1 > >>> $ ln -s $PWD/build-support/thrift/thrift-0.9.1/compiler/cpp/thrift > build-support/thrift/serve/bin/thrift/local/0.9.1/thrift > >>> $ cd build-support/thrift/serve && (nohup python2 -m SimpleHTTPServer > 9999 >&/tmp/thrift.serve &) && cd - > >>> > >>> 5.) Run pants and confirm download of a thrift binary for your machine: > >>> $ ./pants binary src/main/python/:: > >>> > >>> 10:58:44 00:00 [main] > >>> (To run a reporting server: ./pants server) > >>> 10:58:44 00:00 [setup] > >>> 10:58:44 00:00 [parse] > >>> Executing tasks in goals: binary > >>> 10:58:44 00:00 [binary] > >>> 10:58:44 00:00 [binary-prep-command] > >>> 10:58:44 00:00 [python-binary-create] > >>> 10:58:44 00:00 [cache] > >>> No cached artifacts for 8 targets. > >>> Invalidated 8 targets. > >>> 10:58:44 00:00 [chroot]INFO] Attempting to fetch thrift binary > from: http://localhost:9999/bin/thrift/local/0.9.1/thrift ... > >>> 100% .................................................. 31661 KB 0.364s > >>> INFO] Fetched thrift binary from: > http://localhost:9999/bin/thrift/local/0.9.1/thrift . > >>> ... > >>> > >>> Let me know what you get! > >>> > >>> On Thu, Jul 14, 2016 at 6:24 AM, Yiannis Gkoufas <[email protected]> > wrote: > >>>> > >>>> Cool, can you imagine a way to work around this? > >>>> I have to customize pants itself right? > >>>> > >>>> On 14 July 2016 at 11:54, Erb, Stephan <[email protected]> > wrote: > >>>>> > >>>>> Yeah, that sounds like it could be the issue here. > >>>>> > >>>>> > >>>>> > >>>>> From: Yiannis Gkoufas <[email protected]> > >>>>> Reply-To: "[email protected]" <[email protected]> > >>>>> Date: Thursday 14 July 2016 at 12:28 > >>>>> To: "[email protected]" <[email protected]> > >>>>> Subject: Re: Build thermos executor on ARM > >>>>> > >>>>> > >>>>>> > >>>>>> Hi there, > >>>>>> > >>>>>> > >>>>>> > >>>>>> sure: > >>>>>> > >>>>>> > >>>>>> > >>>>>> 10:26:45 00:00 [binary] > >>>>>> > >>>>>> 10:26:45 00:00 [binary-jvm-prep-command] > >>>>>> > >>>>>> 10:26:46 00:01 [jvm_prep_command] > >>>>>> > >>>>>> 10:26:46 00:01 [binary-prep-command] > >>>>>> > >>>>>> 10:26:46 00:01 [python-binary-create] > >>>>>> > >>>>>> Detected interpreter /usr/bin/python2.7: > CPython-2.7.6 > >>>>>> > >>>>>> 10:26:46 00:01 [chroot] > >>>>>> > >>>>>> Building chroot for > [PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), > thermos_executor))]: > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), _executor)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(api/src/main/thrift/org/apache/aurora/gen/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), gen)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/common/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/config/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/config/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/common/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/core/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), core)) > >>>>>> > >>>>>> Dumping library: > PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/monitoring/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), monitoring)) > >>>>>> > >>>>>> Dumping library: > PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, > FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), > thermos_executor)) > >>>>>> > >>>>>> FAILURE: Update --binaries-path-by-id to find binaries for Linux > armv7l 3.10.40 > >>>>>> > >>>>>> > >>>>>> > >>>>>> If I am not completely wrong, I think pants is looking one of the > architectures outlined here: > >>>>>> > >>>>>> > >>>>>> > >>>>>> > https://github.com/pantsbuild/pants/blob/master/src/python/pants/binaries/binary_util.py#L24 > >>>>>> > >>>>>> > >>>>>> > >>>>>> and probably that's why it fails > >>>>>> > >>>>>> > >>>>>> > >>>>>> On 14 July 2016 at 11:17, Erb, Stephan <[email protected]> > wrote: > >>>>>>> > >>>>>>> Cool, ARM J. Please try to run it like this, so that we get a > little bit more output: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> PEX_VERBOSE=1 ./pants -ldebug binary > src/main/python/apache/aurora/executor:thermos_executor > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> From: Yiannis Gkoufas <[email protected]> > >>>>>>> Reply-To: "[email protected]" <[email protected]> > >>>>>>> Date: Thursday 14 July 2016 at 10:44 > >>>>>>> To: "[email protected]" <[email protected]> > >>>>>>> Subject: Build thermos executor on ARM > >>>>>>> > >>>>>>> > >>>>>>>> > >>>>>>>> Hi guys, > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> I have been trying to get aurora working on ARM processors. > >>>>>>>> > >>>>>>>> I had no problem with the scheduler, but I didn't manage to > compile the thermos executor. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> When I give: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> ./pants binary > src/main/python/apache/aurora/executor:thermos_executor > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> I get back: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> FAILURE: Update --binaries-path-by-id to find binaries for Linux > armv7l 3.10.40 > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Let me know if you have any hints on how to workaround this. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Thanks! > >>>>>> > >>>>>> > >>>> > >>>> > >>> > >> > > > > > >
