On Jul 18, 2016 9:04 AM, "Yiannis Gkoufas" <johngou...@gmail.com> wrote: > > Hi John, > > just to confirm: this is how this part of pants.ini should look like: > > backend_packages: [ > [binaries] > baseurls: +['http://localhost:9999'] > path_by_id: {('linux', 'armv7l'): ['local']} > ] > > right?
Not quite, kill the surrounding backend_packages, so: [binaries] baseurls: +['http://localhost:9999'] path_by_id: {('linux', 'armv7l'): ['local']} > > On 17 July 2016 at 11:58, Yiannis Gkoufas <johngou...@gmail.com> wrote: >> >> Wow, thanks a lot John! >> Will give it a shot and get back to you! >> >> On 15 July 2016 at 18:05, John Sirois <john.sir...@gmail.com> wrote: >>> >>> Here's a workaround for you to try. If this works, I can modify the Aurora pants build without modifying pants itself to do all this automatically. >>> >>> The basic idea is that pants probes a list of URLs for pre-built binaries. You can add a local HTTP server to the list and point that server at the thrift binary Aurora builds for gradle via make. >>> Below is how you do this: >>> >>> 0.) Ensure the Aurora gradle thrift is built. >>> $ ./build-support/thrift/thriftw 0.9.1 --version >>> Thrift version 0.9.1 >>> >>> 1.) Just for reference, the pants options in-play here >>> $ ./pants options | grep binaries >>> binaries.baseurls = [' https://dl.bintray.com/pantsbuild/bin/build-support'] (from HARDCODED) >>> binaries.fetch_timeout_secs = 30 (from HARDCODED) >>> binaries.path_by_id = {} (from NONE) >>> $ ./pants options | grep binary >>> thrift-binary.supportdir = bin/thrift (from HARDCODED) >>> thrift-binary.version = 0.9.1 (from CONFIG in pants.ini) >>> >>> 2.) Grab machine name pants will try to find your thrift binary under (here is an example from my machine with result 'x86_64' - yours will be different and arm-ey): >>> $ python2 -c 'import os; sysname, _, _, _, machine = os.uname(); print sysname.lower(), machine' >>> linux x86_64 >>> >>> 3.) Edit pants.ini to tell pants where to look for binaries for your machine: >>> $ git diff -U1 >>> diff --git a/pants.ini b/pants.ini >>> index de0ab30..e2cd94e 100644 >>> --- a/pants.ini >>> +++ b/pants.ini >>> @@ -32,2 +32,6 @@ backend_packages: [ >>> >>> +[binaries] >>> +baseurls: +['http://localhost:9999'] >>> +path_by_id: {('linux', 'x86_64'): ['local']} >>> + >>> [thrift-binary] >>> >>> 4.) Setup the URL you told pants about above: >>> $ mkdir -p build-support/thrift/serve/bin/thrift/local/0.9.1 >>> $ ln -s $PWD/build-support/thrift/thrift-0.9.1/compiler/cpp/thrift build-support/thrift/serve/bin/thrift/local/0.9.1/thrift >>> $ cd build-support/thrift/serve && (nohup python2 -m SimpleHTTPServer 9999 >&/tmp/thrift.serve &) && cd - >>> >>> 5.) Run pants and confirm download of a thrift binary for your machine: >>> $ ./pants binary src/main/python/:: >>> >>> 10:58:44 00:00 [main] >>> (To run a reporting server: ./pants server) >>> 10:58:44 00:00 [setup] >>> 10:58:44 00:00 [parse] >>> Executing tasks in goals: binary >>> 10:58:44 00:00 [binary] >>> 10:58:44 00:00 [binary-prep-command] >>> 10:58:44 00:00 [python-binary-create] >>> 10:58:44 00:00 [cache] >>> No cached artifacts for 8 targets. >>> Invalidated 8 targets. >>> 10:58:44 00:00 [chroot]INFO] Attempting to fetch thrift binary from: http://localhost:9999/bin/thrift/local/0.9.1/thrift ... >>> 100% .................................................. 31661 KB 0.364s >>> INFO] Fetched thrift binary from: http://localhost:9999/bin/thrift/local/0.9.1/thrift . >>> ... >>> >>> Let me know what you get! >>> >>> On Thu, Jul 14, 2016 at 6:24 AM, Yiannis Gkoufas <johngou...@gmail.com> wrote: >>>> >>>> Cool, can you imagine a way to work around this? >>>> I have to customize pants itself right? >>>> >>>> On 14 July 2016 at 11:54, Erb, Stephan <stephan....@blue-yonder.com> wrote: >>>>> >>>>> Yeah, that sounds like it could be the issue here. >>>>> >>>>> >>>>> >>>>> From: Yiannis Gkoufas <johngou...@gmail.com> >>>>> Reply-To: "user@aurora.apache.org" <user@aurora.apache.org> >>>>> Date: Thursday 14 July 2016 at 12:28 >>>>> To: "user@aurora.apache.org" <user@aurora.apache.org> >>>>> Subject: Re: Build thermos executor on ARM >>>>> >>>>> >>>>>> >>>>>> Hi there, >>>>>> >>>>>> >>>>>> >>>>>> sure: >>>>>> >>>>>> >>>>>> >>>>>> 10:26:45 00:00 [binary] >>>>>> >>>>>> 10:26:45 00:00 [binary-jvm-prep-command] >>>>>> >>>>>> 10:26:46 00:01 [jvm_prep_command] >>>>>> >>>>>> 10:26:46 00:01 [binary-prep-command] >>>>>> >>>>>> 10:26:46 00:01 [python-binary-create] >>>>>> >>>>>> Detected interpreter /usr/bin/python2.7: CPython-2.7.6 >>>>>> >>>>>> 10:26:46 00:01 [chroot] >>>>>> >>>>>> Building chroot for [PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), thermos_executor))]: >>>>>> >>>>>> Dumping library: PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), _executor)) >>>>>> >>>>>> Dumping library: PythonLibrary(BuildFileAddress(BuildFile(api/src/main/thrift/org/apache/aurora/gen/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), gen)) >>>>>> >>>>>> Dumping library: PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/common/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) >>>>>> >>>>>> Dumping library: PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/config/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) >>>>>> >>>>>> Dumping library: PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/config/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), config)) >>>>>> >>>>>> Dumping library: PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/common/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), common)) >>>>>> >>>>>> Dumping library: PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/core/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), core)) >>>>>> >>>>>> Dumping library: PythonLibrary(BuildFileAddress(BuildFile(src/main/python/apache/thermos/monitoring/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), monitoring)) >>>>>> >>>>>> Dumping library: PythonBinary(BuildFileAddress(BuildFile(src/main/python/apache/aurora/executor/BUILD, FileSystemProjectTree(/home/ubuntu/apache-aurora-0.15.0)), thermos_executor)) >>>>>> >>>>>> FAILURE: Update --binaries-path-by-id to find binaries for Linux armv7l 3.10.40 >>>>>> >>>>>> >>>>>> >>>>>> If I am not completely wrong, I think pants is looking one of the architectures outlined here: >>>>>> >>>>>> >>>>>> >>>>>> https://github.com/pantsbuild/pants/blob/master/src/python/pants/binaries/binary_util.py#L24 >>>>>> >>>>>> >>>>>> >>>>>> and probably that's why it fails >>>>>> >>>>>> >>>>>> >>>>>> On 14 July 2016 at 11:17, Erb, Stephan <stephan....@blue-yonder.com> wrote: >>>>>>> >>>>>>> Cool, ARM J. Please try to run it like this, so that we get a little bit more output: >>>>>>> >>>>>>> >>>>>>> >>>>>>> PEX_VERBOSE=1 ./pants -ldebug binary src/main/python/apache/aurora/executor:thermos_executor >>>>>>> >>>>>>> >>>>>>> >>>>>>> From: Yiannis Gkoufas <johngou...@gmail.com> >>>>>>> Reply-To: "user@aurora.apache.org" <user@aurora.apache.org> >>>>>>> Date: Thursday 14 July 2016 at 10:44 >>>>>>> To: "user@aurora.apache.org" <user@aurora.apache.org> >>>>>>> Subject: Build thermos executor on ARM >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Hi guys, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I have been trying to get aurora working on ARM processors. >>>>>>>> >>>>>>>> I had no problem with the scheduler, but I didn't manage to compile the thermos executor. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> When I give: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ./pants binary src/main/python/apache/aurora/executor:thermos_executor >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I get back: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> FAILURE: Update --binaries-path-by-id to find binaries for Linux armv7l 3.10.40 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Let me know if you have any hints on how to workaround this. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks! >>>>>> >>>>>> >>>> >>>> >>> >> >
