Oh, just remembered... Here is a list of Bazel platform values. https://github.com/bazelbuild/platforms/blob/main/cpu/BUILD
Bazel auto populates this. Here is a full list. I believe the M1 is `armv6-m`. I think I have a theory as to what's happening, but I'd have to dig into this a bit. Short summary though, your logic in the Pex builder script is trying to run the Linux code, and not the Darwin alternative. We seem to set the `darwin` and `k8` `config_settings` closures in `tools/platform/BUILD`. We set each based on the CPU type. I wonder if we are mixing CPU type and OS type in our config. On Sat, Feb 26, 2022 at 11:09 PM Nicholas Nezis <[email protected]> wrote: > So I'm wondering if it's something special with the M1. I feel like I've > built on MacOS recently without issue. And when I run `mktemp` from the > CLI, I do see the MacOS specific arguments. > > In the `tools/platform/BUILD` file, we define a `darwin` and `k8` type > based on matching `cpu` flavor. Not sure what the M1 would register as. > > > On Sat, Feb 26, 2022 at 9:56 PM Windham Wong <[email protected]> > wrote: > >> It looks like happening in macOS only because macOS has the same command >> of tools but entirely different on the parameters or usage. I personally >> don't build Heron on my Macbook but I know the frustration of doing that >> if the project does not support macOS natively. >> >> I believe this is just a missing piece for the bazel configure for using >> special parameters on different distros, just like what I have >> encountered on Debian as well. I would vote for a yes to add a distro >> check before running `mktemp` here, if I have a voting right here. :P >> >> Regards J, >> *Windham Wong* >> OSWE, OSCP, GCIA, Specialist in Cybersecurity >> Co-Founder, Managing Partner of >> *Stormeye.io, Hong Kong Managed Security Operation Center Limited* >> Specialist in Cybersecurity, Log Management and SIEM System >> <https://www.stormeye.io> >> Email // [email protected] >> Phone // +852_3590_2212_|_+852_9832_0707 <tel:+85235902212> >> Fax // +852_3590_2202 <tel:+852_3590_2202> >> >> On 27/2/2022 3:18, Josh Fischer wrote: >> > Is that the suggested fix for this? Or is this just a short term >> solution? >> > >> > I thought we had Bazel conditionals that used the appropriate tools >> based >> > on the distro Heron is being built on. >> > >> > On Sat, Feb 26, 2022 at 1:50 AM Nicholas Nezis<[email protected] >> > >> > wrote: >> > >> >> It seems that `mktemp` on Darwin has different arguments compared to >> Linux. >> >> >> >> If you `brew install coreutils` and set `gnubin` on your `PATH`, you >> might >> >> be able to get matching `mktemp` as you'd have on linux. >> >> >> >> >> >> On Fri, Feb 25, 2022 at 9:06 PM Josh Fischer<[email protected]> >> wrote: >> >> >> >>> I tried to build off master expecting errors since I'm on an M1. >> Anyone >> >>> seen this "mktmp" error before? Python version is 3.8.x. You'll see >> the >> >>> full Python version at the bottom of the shell snippet. >> >>> >> >>> bazel build --config=darwin_nostyle scripts/packages:binpkgs >> >>> --host_javabase=@local_jdk//:jdk --verbose_failures >> >>> >> >>> INFO: Analyzed target //scripts/packages:binpkgs (0 packages loaded, 0 >> >>> targets configured). >> >>> >> >>> INFO: Found 1 target... >> >>> >> >>> ERROR: >> >>> >> >> >> /Users/joshfischer/Source/apache/incubator-heron/tools/rules/pex/BUILD:57:8: >> >>> Bootstrapping pex //tools/rules/pex:pex_wrapper [for host] failed: >> (Exit >> >>> 1): bash failed: error executing command >> >>> >> >>> (cd >> >>> >> >>> >> >> >> /private/var/tmp/_bazel_joshfischer/70e9bd6aaead2621044e8d790ac25401/execroot/org_apache_heron >> >>> && \ >> >>> >> >>> exec env - \ >> >>> >> >>> PATH=' left out for brevity >> >>> >> >>> /bin/bash -c 'source >> >> external/bazel_tools/tools/genrule/genrule-setup.sh; >> >>> OUTDIR=$(cd bazel-out/host/bin/tools/rules/pex && pwd) >> >>> >> >>> # Workaround really long shebang lines breaking on linux: >> >>> >> >>> # Use a /tmp path, but keep the actual venv inside the bazel outdir. >> >>> >> >>> # Avoids having to worry about cleanup, even if sandboxing is off. >> >>> >> >>> TMPF=$(mktemp -d -p /tmp pex.XXXXX) >> >>> >> >>> ln -sf "$OUTDIR" "$TMPF" >> >>> >> >>> VENV="${TMPF}/venv" >> >>> >> >>> python3 -m venv $VENV --clear >> >>> >> >>> VIRTUAL_ENV_DISABLE_PROMPT=1 source "$VENV/bin/activate" >> >>> >> >>> TEMP="bazel-out/host/bin/tools/rules/pex/pexbuild" >> >>> >> >>> pip install pex --quiet --no-cache-dir --no-index >> >>> --find-links >> >>> $(dirname external/pex_pkg/file/pex-2.1.62-py2.py3-none-any.whl) >> >>> --find-links $(dirname external/wheel_pkg/file/wheel-0.36.1.tar.gz) >> >>> --find-links $(dirname >> >>> external/setuptools_pkg/file/setuptools-51.0.0-py3-none-any.whl) >> >>> >> >>> # Work around setuptools insistance on writing to the source >> directory, >> >>> >> >>> # which is discouraged by Bazel (and annoying) >> >>> >> >>> cp -r $(dirname tools/rules/pex/wrapper/setup.py) >> >>> bazel-out/host/bin/tools/rules/pex/.pex_wrapper >> >>> >> >>> # Use the bootstrapped pex to build pex_wrapper.pex >> >>> >> >>> pex bazel-out/host/bin/tools/rules/pex/.pex_wrapper >> >>> --disable-cache >> >>> --no-index --entry-point=pex_wrapper >> >>> --output-file=bazel-out/host/bin/tools/rules/pex/pex_wrapper.pex >> >>> --find-links $(dirname >> >>> external/pex_pkg/file/pex-2.1.62-py2.py3-none-any.whl) >> >>> --find-links >> >>> $(dirname >> >> external/setuptools_pkg/file/setuptools-51.0.0-py3-none-any.whl) >> >>> --find-links $(dirname >> >>> external/requests_pkg/file/requests-2.27.1-py2.py3-none-any.whl) >> >>> --find-links $(dirname >> >>> external/charset_pkg/file/charset_normalizer-2.0.10-py3-none-any.whl) >> >>> --find-links $(dirname >> >>> external/idna_pkg/file/idna-3.3-py2.py3-none-any.whl) >> >>> --find-links >> >>> $(dirname >> external/urllib3_pkg/file/urllib3-1.26.8-py2.py3-none-any.whl) >> >>> --find-links $(dirname >> >>> external/certifi_pkg/file/certifi-2021.10.8-py2.py3-none-any.whl) >> >>> --find-links $(dirname >> external/wheel_pkg/file/wheel-0.36.1.tar.gz)') >> >>> >> >>> Execution platform: @local_config_platform//:host >> >>> >> >>> mktemp: illegal option -- p >> >>> >> >>> usage: mktemp [-d] [-q] [-t prefix] [-u] template ... >> >>> >> >>> mktemp [-d] [-q] [-u] -t prefix >> >>> >> >>> Target //scripts/packages:binpkgs failed to build >> >>> >> >>> INFO: Elapsed time: 0.245s, Critical Path: 0.03s >> >>> >> >>> INFO: 14 processes: 14 internal. >> >>> >> >>> FAILED: Build did NOT complete successfully >> >>> >> >>> MacBook-Pro:incubator-heron joshfischer$ python -V >> >>> >> >>> Python 3.8.12 >> >>> >> >>> On Fri, Feb 25, 2022 at 12:08 PM Saad Ur Rahman < >> [email protected] >> >>> >> >>> wrote: >> >>> >> >>>> I just checked the diff on the PR and it is in fact Python 3.8 - I >> >> stand >> >>>> corrected. I am not sure about the ASF infra issues but I believe it >> >> was >> >>>> related to PEX and Python. The recent PRs should, hopefully, fix the >> >>> issue. >> >>>> On Fri, Feb 25, 2022 at 12:09 PM Josh Fischer<[email protected]> >> >>> wrote: >> >>>>> Ahh, 3.9, even better. >> >>>>> >> >>>>> Random question: did we ever figure out what the build issue was >> >> when >> >>>>> building heron in containers on the Apache jenkins instance? >> >>>>> >> >>>>> On Fri, Feb 25, 2022 at 11:05 AM Saad Ur Rahman < >> >>> [email protected] >> >>>>> wrote: >> >>>>> >> >>>>>> It is a drastic improvement, but a minor correction on the base >> >>> version >> >>>>> of >> >>>>>> Python: it is currently 3.9. We really should run a test build on >> >> the >> >>>> ASF >> >>>>>> build infrastructure to see if it builds smoothly. >> >>>>>> >> >>>>>> On Fri, Feb 25, 2022 at 11:48 AM Josh Fischer <[email protected] >> >>>>> wrote: >> >>>>>>> Fantastic. This helps remove the pressure of the native Python >> >>> rules >> >>>>> for >> >>>>>>> the moment. >> >>>>>>> >> >>>>>>> On Fri, Feb 25, 2022 at 10:44 AM Saad Ur Rahman < >> >>>>> [email protected] >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>>> Correct, the base Python version was bumped up to 3.8. It was >> >> the >> >>>>> last >> >>>>>>>> major PR merged in #3646 ( >> >>>>>>>> https://github.com/apache/incubator-heron/pull/3646). I >> >> believe >> >>>> the >> >>>>>> base >> >>>>>>>> Ubuntu container version is now 20.04 LTS. >> >>>>>>>> >> >>>>>>>> On Fri, Feb 25, 2022 at 11:36 AM Josh Fischer < >> >>> [email protected] >> >>>>>>> wrote: >> >>>>>>>>> I remember reading somewhere that during a meet up that some >> >>> work >> >>>>> was >> >>>>>>>> done >> >>>>>>>>> to upgrade the current supported version of Python within the >> >>>> Heron >> >>>>>>> code >> >>>>>>>>> base. Am I remembering this correctly?-- >> >>>>>>>>> Sent from A Mobile Device >> >>>>>>>>> >> >>>>>>> -- >> >>>>>>> Sent from A Mobile Device >> >>>>>>> >> >>>>> -- >> >>>>> Sent from A Mobile Device >> >>>>> > >
