Thanks Uwe for doing this! It's amazing how well the arrow build system is working given it supports so many different configurations (which makes Uwe's refactor all the more impressive). Big thanks for everybody helping to maintain it.
On Fri, Mar 15, 2019 at 8:08 AM Wes McKinney <wesmck...@gmail.com> wrote: > Thanks Uwe for seeing through this massive improvement to our C++ > build system! This should help package managers build the project in > precisely the way that's needed to make the Arrow C++ libraries > available in more places. > > A couple notes: > > * I personally keep my C++ toolchain and conda environments separate; > if you also do this you can see how to configure to pick up your > toolchain directory even though $CONDA_PREFIX is set: > > -DARROW_DEPENDENCY_SOURCE=SYSTEM -DARROW_PACKAGE_PREFIX=$CPP_TOOLCHAIN > > (see my environment configuration at > > https://github.com/wesm/dev-toolchain/blob/master/toolchain/arrow-toolchain.sh#L142 > ) > > * One of the cool things about this is that you can control dependency > resolution in a more granular way: > > -D$PACKAGE_SOURCE=BUNDLED -> Use ExternalProject to build > -D$PACKAGE_SOURCE=SYSTEM -> Look in system paths for the package > -D$PACKAGE_ROOT=$SOMEWHERE -> Look in $SOMEWHERE for the package > > So in the event that you need to override one of your system > dependencies, you have the flexibility to do that. > > - Wes > > Thanks > Wes > > On Fri, Mar 15, 2019 at 9:48 AM Uwe L. Korn <m...@uwekorn.com> wrote: > > > > Hello fellow Arrow Devs, > > > > we have merged the CMake refactor yesterday > https://github.com/apache/arrow/pull/3688 and this means that the build > system behaves a bit different. The main differences are: > > > > * If you're in a conda environment, we automatically detect this using > the environment variable $CONDA_PREFIX and expect that all dependencies > (except jemalloc and ORC) are installed via conda. > > * Otherwise, we will look in the standard system paths for a dependency. > If it isn't found, we use CMake's ExternalProject mechanism to build it. > > * The *_HOME variables are not longer use and are replaced by *_ROOT > variables to use CMake's standard detection features. Be aware that > dependencies are no longer written in all caps but their preferred casing > as seen in > https://github.com/apache/arrow/blob/0d302125abb4b514dba210f496c574a77ce4cd1d/cpp/cmake_modules/ThirdpartyToolchain.cmake#L41-L59 > > * You can manually select the way we detect dependencies via > ARROW_DEPENDENCY_SOURCE > https://github.com/apache/arrow/blob/0d302125abb4b514dba210f496c574a77ce4cd1d/cpp/CMakeLists.txt#L189-L207 > The hope is that you as a developer should not normally need to change this > and as packager for distributions, you can use > `ARROW_DEPENDENCY_SOURCE=SYSTEM` to ensure that ExternalProject is not used > but only packages from the package manager. If your system is in a > non-default prefix, you can indicate this by setting ARROW_PACKAGE_PREFIX. > > > > Also, please clear your existing CMake directories and do a fresh built > to avoid any problems. As well when you're using conda packages, please > update them all using `conda update --all` as I have errors in the > packaging directly on conda-forge instead of doing workarounds in our CMake > code. A helpful information is here that conda-forge now provides a > `compilers` package that provides the whole build toolchain. > > > > Uwe >
