[
https://issues.apache.org/jira/browse/ARROW-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rok Mihevc updated ARROW-5222:
------------------------------
External issue URL: https://github.com/apache/arrow/issues/21696
> [Python] Issues with installing pyarrow for development on MacOS
> ----------------------------------------------------------------
>
> Key: ARROW-5222
> URL: https://issues.apache.org/jira/browse/ARROW-5222
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Documentation, Python
> Reporter: Neal Richardson
> Assignee: Neal Richardson
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.14.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> gI tried following the
> [instructions|https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst]
> for installing pyarrow for developers on macos, and I ran into quite a bit
> of difficulty. I'm hoping we can improve our documentation and/or tooling to
> make this a smoother process.
> I know we can't anticipate every quirk of everyone's dev environment, but in
> my case, I was getting set up on a new machine, so this was from a clean
> slate. I'm also new to contributing to the project, so I'm a "clean slate" in
> that regard too, so my ignorance may be exposing other assumptions in the
> docs.
> # The instructions recommend using conda, but as this [Stack Overflow
> question|https://stackoverflow.com/questions/55798166/cmake-fails-with-when-attempting-to-compile-simple-test-program]
> notes, cmake fails. Uwe helpfully suggested installing an older MacOS SDK
> from [here|https://github.com/phracker/MacOSX-SDKs/releases]. That may work,
> but I'm personally wary to install binaries from an unofficial github
> account, let alone record that in our docs as an official recommendation.
> Either way, we should update the docs either to note this necessity or to
> recommend against installing with conda on macos.
> # After that, I tried to go the Homebrew path. Ultimately this did succeed,
> but it was rough. It seemed that I had to `brew install` a lot of packages
> that weren't included in the arrow/python/Brewfile (i.e. try to cmake, see
> what missing dependency it failed on, `brew install` it, retry `cmake`, and
> repeat). Among the libs I installed this way were double-conversion snappy
> brotli protobuf gtest rapidjson flatbuffers lz4 zstd c-ares boost. It's not
> clear how many of these extra dependencies I had to install were because I'd
> only installed the xcode command-line tools and not the full xcode from the
> App Store; regardless, the Brewfile should be complete if we want to use it.
> # In searching Jira for the double-conversion issue (the first one I hit), I
> found [this issue/PR|https://github.com/apache/arrow/pull/4132/files], which
> added double-conversion to a different Brewfile, in c_glib. So I tried `brew
> bundle` installing that Brewfile. It would probably be good to have a common
> Brewfile for the C++ setup, which the python and glib ones could load and
> then add any other extra dependencies, if necessary. That way, there's one
> place to add common dependencies.
> # I got close here but still had issues with `BOOST_HOME` not being found,
> even though I had brew-installed it. From the console output, it appeared
> that even though I was not using conda and did not have an active conda
> environment (I'd even done `conda env remove --name pyarrow-dev`), the cmake
> configuration script detected that conda existed and decided to use conda to
> resolve dependencies. I tried setting lots of different environment variables
> to tell cmake not to use conda, but ultimately I was only able to get past
> this by deleting conda from my system entirely.
> # This let me get to the point of being able to `import pyarrow`. But then
> running tests failed because the `hypothesis` package was not installed. I
> see that it is included in requirements-test.txt and setup.py under
> tests_require, but I followed the installation instructions and this package
> did not end up in my virtualenv. `pip install hypothesis` resolved it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
