Here's a workaround for you to try. If this works, I can modify the Aurora pants build without modifying pants itself to do all this automatically.
The basic idea is that pants probes a list of URLs for pre-built binaries. You can add a local HTTP server to the list and point that server at the thrift binary Aurora builds for gradle via make. Below is how you do this: 0.) Ensure the Aurora gradle thrift is built. $ ./build-support/thrift/thriftw 0.9.1 --version Thrift version 0.9.1 1.) Just for reference, the pants options in-play here $ ./pants options | grep binaries binaries.baseurls = ['https://dl.bintray.com/pantsbuild/bin/build-support'] (from HARDCODED) binaries.fetch_timeout_secs = 30 (from HARDCODED) binaries.path_by_id = {} (from NONE) $ ./pants options | grep binary thrift-binary.supportdir = bin/thrift (from HARDCODED) thrift-binary.version = 0.9.1 (from CONFIG in pants.ini) 2.) Grab machine name pants will try to find your thrift binary under (here is an example from my machine with result 'x86_64' - yours will be different and arm-ey): $ python2 -c 'import os; sysname, _, _, _, machine = os.uname(); print sysname.lower(), machine' linux x86_64 3.) Edit pants.ini to tell pants where to look for binaries for your machine: $ git diff -U1 diff --git a/pants.ini b/pants.ini index de0ab30..e2cd94e 100644 --- a/pants.ini +++ b/pants.ini @@ -32,2 +32,6 @@ backend_packages: [ +[binaries] +baseurls: +['http://localhost:9999'] +path_by_id: {('linux', 'x86_64'): ['local']} + [thrift-binary] 4.) Setup the URL you told pants about above: $ mkdir -p build-support/thrift/serve/bin/thrift/local/0.9.1 $ ln -s $PWD/build-support/thrift/thrift-0.9.1/compiler/cpp/thrift build-support/thrift/serve/bin/thrift/local/0.9.1/thrift $ cd build-support/thrift/serve && (nohup python2 -m SimpleHTTPServer 9999 >&/tmp/thrift.serve &) && cd - 5.) Run pants and confirm download of a thrift binary for your machine: $ ./pants binary src/main/python/:: 10:58:44 00:00 [main] (To run a reporting server: ./pants server) 10:58:44 00:00 [setup] 10:58:44 00:00 [parse] Executing tasks in goals: binary 10:58:44 00:00 [binary] 10:58:44 00:00 [binary-prep-command] 10:58:44 00:00 [python-binary-create] 10:58:44 00:00 [cache] No cached artifacts for 8 targets. Invalidated 8 targets. 10:58:44 00:00 [chroot]INFO] Attempting to fetch thrift binary from: http://localhost:9999/bin/thrift/local/0.9.1/thrift ... 100% .................................................. 31661 KB 0.364s INFO] Fetched thrift binary from: http://localhost:9999/bin/thrift/local/0.9.1/thrift . ... Let me know what you get! On Thu, Jul 14, 2016 at 6:24 AM, Yiannis Gkoufas <[email protected]> wrote: > Cool, can you imagine a way to work around this? > I have to customize pants itself right? > > On 14 July 2016 at 11:54, Erb, Stephan <[email protected]> > wrote: > >> Yeah, that sounds like it could be the issue here. >> >> >> >> *From: *Yiannis Gkoufas <[email protected]> >> *Reply-To: *"[email protected]" <[email protected]> >> *Date: *Thursday 14 July 2016 at 12:28 >> *To: *"[email protected]" <[email protected]> >> *Subject: *Re: Build thermos executor on ARM >> >> >> >> Hi there, >> >> >> >> sure: >> >> >> >> 10:26:45 00:00 [binary] >> >> 10:26:45 00:00 [binary-jvm-prep-command] >> >> 10:26:46 00:01 [jvm_prep_command] >> >> 10:26:46 00:01 [binary-prep-command] >> >> 10:26:46 00:01 [python-binary-create] >> >> Detected interpreter /usr/bin/python2.7: CPython-2.7.6 >> >> 10:26:46 00:01 [chroot] >> >> Building chroot for >> [PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), >> thermos_executor))]: >> >> Dumping library: >> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), _executor)) >> >> Dumping library: >> PythonLibrary(BuildFileAddress(BuildFile(api/src/main/thrift/org/apache/aurora/gen/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), gen)) >> >> Dumping library: >> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/common/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) >> >> Dumping library: >> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/config/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) >> >> Dumping library: >> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/config/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) >> >> Dumping library: >> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/common/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) >> >> Dumping library: >> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/core/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), core)) >> >> Dumping library: >> PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/monitoring/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), monitoring)) >> >> Dumping library: >> PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, >> FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), >> thermos_executor)) >> >> FAILURE: Update --binaries-path-by-id to find binaries for Linux armv7l >> 3.10.40 >> >> >> >> If I am not completely wrong, I think pants is looking one of the >> architectures outlined here: >> >> >> >> >> https://github.com/pantsbuild/pants/blob/master/src/python/pants/binaries/binary_util.py#L24 >> >> >> >> and probably that's why it fails >> >> >> >> On 14 July 2016 at 11:17, Erb, Stephan <[email protected]> >> wrote: >> >> Cool, ARM J. Please try to run it like this, so that we get a little bit >> more output: >> >> >> >> PEX_VERBOSE=1 ./pants -ldebug binary >> src/main/python/apache/aurora/executor:thermos_executor >> >> >> >> *From: *Yiannis Gkoufas <[email protected]> >> *Reply-To: *"[email protected]" <[email protected]> >> *Date: *Thursday 14 July 2016 at 10:44 >> *To: *"[email protected]" <[email protected]> >> *Subject: *Build thermos executor on ARM >> >> >> >> Hi guys, >> >> >> >> I have been trying to get aurora working on ARM processors. >> >> I had no problem with the scheduler, but I didn't manage to compile the >> thermos executor. >> >> >> >> When I give: >> >> >> >> ./pants binary src/main/python/apache/aurora/executor:thermos_executor >> >> >> >> I get back: >> >> >> >> FAILURE: Update --binaries-path-by-id to find binaries for Linux armv7l >> 3.10.40 >> >> >> >> Let me know if you have any hints on how to workaround this. >> >> >> >> Thanks! >> >> >> >> >
