Wow, thanks a lot John! Will give it a shot and get back to you! On 15 July 2016 at 18:05, John Sirois <[email protected]> wrote:
> Here's a workaround for you to try. If this works, I can modify the > Aurora pants build without modifying pants itself to do all this > automatically. > > The basic idea is that pants probes a list of URLs for pre-built > binaries. You can add a local HTTP server to the list and point that > server at the thrift binary Aurora builds for gradle via make. > Below is how you do this: > > 0.) Ensure the Aurora gradle thrift is built. > $ ./build-support/thrift/thriftw 0.9.1 --version > Thrift version 0.9.1 > > 1.) Just for reference, the pants options in-play here > $ ./pants options | grep binaries > binaries.baseurls = ['https://dl.bintray.com/pantsbuild/bin/build-support'] > (from HARDCODED) > binaries.fetch_timeout_secs = 30 (from HARDCODED) > binaries.path_by_id = {} (from NONE) > $ ./pants options | grep binary > thrift-binary.supportdir = bin/thrift (from HARDCODED) > thrift-binary.version = 0.9.1 (from CONFIG in pants.ini) > > 2.) Grab machine name pants will try to find your thrift binary under > (here is an example from my machine with result 'x86_64' - yours will be > different and arm-ey): > $ python2 -c 'import os; sysname, _, _, _, machine = os.uname(); print > sysname.lower(), machine' > linux x86_64 > > 3.) Edit pants.ini to tell pants where to look for binaries for your > machine: > $ git diff -U1 > diff --git a/pants.ini b/pants.ini > index de0ab30..e2cd94e 100644 > --- a/pants.ini > +++ b/pants.ini > @@ -32,2 +32,6 @@ backend_packages: [ > > +[binaries] > +baseurls: +['http://localhost:9999'] > +path_by_id: {('linux', 'x86_64'): ['local']} > + > [thrift-binary] > > 4.) Setup the URL you told pants about above: > $ mkdir -p build-support/thrift/serve/bin/thrift/local/0.9.1 > $ ln -s $PWD/build-support/thrift/thrift-0.9.1/compiler/cpp/thrift > build-support/thrift/serve/bin/thrift/local/0.9.1/thrift > $ cd build-support/thrift/serve && (nohup python2 -m SimpleHTTPServer 9999 > >&/tmp/thrift.serve &) && cd - > > 5.) Run pants and confirm download of a thrift binary for your machine: > $ ./pants binary src/main/python/:: > > 10:58:44 00:00 [main] > (To run a reporting server: ./pants server) > 10:58:44 00:00 [setup] > 10:58:44 00:00 [parse] > Executing tasks in goals: binary > 10:58:44 00:00 [binary] > 10:58:44 00:00 [binary-prep-command] > 10:58:44 00:00 [python-binary-create] > 10:58:44 00:00 [cache] > No cached artifacts for 8 targets. > Invalidated 8 targets. > 10:58:44 00:00 [chroot]INFO] Attempting to fetch thrift binary from: > http://localhost:9999/bin/thrift/local/0.9.1/thrift ... > 100% .................................................. 31661 KB 0.364s > INFO] Fetched thrift binary from: > http://localhost:9999/bin/thrift/local/0.9.1/thrift . > ... > > Let me know what you get! > > On Thu, Jul 14, 2016 at 6:24 AM, Yiannis Gkoufas <[email protected]> > wrote: > >> Cool, can you imagine a way to work around this? >> I have to customize pants itself right? >> >> On 14 July 2016 at 11:54, Erb, Stephan <[email protected]> >> wrote: >> >>> Yeah, that sounds like it could be the issue here. >>> >>> >>> >>> *From: *Yiannis Gkoufas <[email protected]> >>> *Reply-To: *"[email protected]" <[email protected]> >>> *Date: *Thursday 14 July 2016 at 12:28 >>> *To: *"[email protected]" <[email protected]> >>> *Subject: *Re: Build thermos executor on ARM >>> >>> >>> >>> Hi there, >>> >>> >>> >>> sure: >>> >>> >>> >>> 10:26:45 00:00 [binary] >>> >>> 10:26:45 00:00 [binary-jvm-prep-command] >>> >>> 10:26:46 00:01 [jvm_prep_command] >>> >>> 10:26:46 00:01 [binary-prep-command] >>> >>> 10:26:46 00:01 [python-binary-create] >>> >>> Detected interpreter /usr/bin/python2.7: CPython-2.7.6 >>> >>> 10:26:46 00:01 [chroot] >>> >>> Building chroot for >>> [PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), >>> thermos_executor))]: >>> >>> Dumping library: >>> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), _executor)) >>> >>> Dumping library: >>> PythonLibrary(BuildFileAddress(BuildFile(api/src/main/thrift/org/apache/aurora/gen/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), gen)) >>> >>> Dumping library: >>> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/common/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) >>> >>> Dumping library: >>> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/config/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) >>> >>> Dumping library: >>> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/config/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) >>> >>> Dumping library: >>> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/common/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) >>> >>> Dumping library: >>> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/core/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), core)) >>> >>> Dumping library: >>> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/monitoring/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), monitoring)) >>> >>> Dumping library: >>> PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, >>> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), >>> thermos_executor)) >>> >>> FAILURE: Update --binaries-path-by-id to find binaries for Linux armv7l >>> 3.10.40 >>> >>> >>> >>> If I am not completely wrong, I think pants is looking one of the >>> architectures outlined here: >>> >>> >>> >>> >>> https://github.com/pantsbuild/pants/blob/master/src/python/pants/binaries/binary_util.py#L24 >>> >>> >>> >>> and probably that's why it fails >>> >>> >>> >>> On 14 July 2016 at 11:17, Erb, Stephan <[email protected]> >>> wrote: >>> >>> Cool, ARM J. Please try to run it like this, so that we get a little >>> bit more output: >>> >>> >>> >>> PEX_VERBOSE=1 ./pants -ldebug binary >>> src/main/python/apache/aurora/executor:thermos_executor >>> >>> >>> >>> *From: *Yiannis Gkoufas <[email protected]> >>> *Reply-To: *"[email protected]" <[email protected]> >>> *Date: *Thursday 14 July 2016 at 10:44 >>> *To: *"[email protected]" <[email protected]> >>> *Subject: *Build thermos executor on ARM >>> >>> >>> >>> Hi guys, >>> >>> >>> >>> I have been trying to get aurora working on ARM processors. >>> >>> I had no problem with the scheduler, but I didn't manage to compile the >>> thermos executor. >>> >>> >>> >>> When I give: >>> >>> >>> >>> ./pants binary src/main/python/apache/aurora/executor:thermos_executor >>> >>> >>> >>> I get back: >>> >>> >>> >>> FAILURE: Update --binaries-path-by-id to find binaries for Linux armv7l >>> 3.10.40 >>> >>> >>> >>> Let me know if you have any hints on how to workaround this. >>> >>> >>> >>> Thanks! >>> >>> >>> >>> >> >
