Note that there are already JIRA issues about introducing a dynamic linking option in several dependencies
* Thrift https://issues.apache.org/jira/browse/ARROW-4447 * re2 https://issues.apache.org/jira/browse/ARROW-3435 * Compression libs (zstd, snappy, bz2, brotli, lz4) https://issues.apache.org/jira/browse/ARROW-2217 So out of the libraries listed, the ones where dynamic linking would need to be introduced are * double-conversion * gflags * glog * grpc * orc gbenchmark is a test/benchmark-only dependency Maintaining all this toolchain stuff is a lot of work. We haven't added these knobs to turn yet because deploying on Linux package managers has not been a priority for any of the core developers. If someone would like to sponsor dev work on this I would be happy to discuss separately. Thanks Wes On Fri, Feb 8, 2019 at 3:51 PM Uwe L. Korn <[email protected]> wrote: > > > > On Fri, Feb 8, 2019, at 10:49 PM, Todd Rme wrote: > > On Fri, Feb 8, 2019 at 2:20 PM Uwe L. Korn <[email protected]> wrote: > > > > > > Hello all, > > > > > > On Fri, Feb 8, 2019, at 8:02 PM, Suvayu Ali wrote: > > > > Hello Todd, > > > > > > > > On Fri, Feb 08, 2019 at 11:41:05AM -0500, Todd Rme wrote: > > > > > On 2019/02/02 10:00:37, Suvayu Ali <[email protected]> wrote: > > > > > > > > > > > 1. Fedora doesn't allow including external dependencies. In my > > > > > > experience> > > > > > > building arrow on Fedora, the way external deps like Protobuf, > > > > > > Thrift, RE2,> > > > > > > etc are handled is unlikely to pass package review. There maybe > > > > > > exceptions,> > > > > > > but it has to be well justified (e.g. ROOT).> > > > > > > > > > > openSUSE has a similar policy, with an additional caveat: openSUSE > > > > > usually doesn't allow static libraries. So even if system libraries > > > > > were supported, we wouldn't be able to use them because arrow requires > > > > > them to be statically linked. So supported dynamically linked > > > > > libraries would be needed. > > > > > > > > I think on Fedora that is also true in general. But I believe there are > > > > static builds available for certain libraries (e.g. llvm, boost). That > > > > said, > > > > are you sure static linking is required by arrow? When I look at the > > > > output > > > > of ldd run against the arrow libraries, I see the dynamic system > > > > libraries I > > > > did manage to link against (boost and zlib). I also see the linking > > > > between > > > > the different arrow libraries like, libgandiva and libplasma linking to > > > > libarrow. > > > > > > > > If I understand correctly, static linking is used for the dependencies > > > > that are > > > > _not_ taken from the system, so in my case that would be double- > > > > precision, thrift, etc. > > > > > > We should be able to link all libraries dynamically. At the moment there > > > may be some that can only be linked statically in our build but that > > > should not be hard to change. It is important to open JIRAs when there > > > are problems. From the main Arrow (C++/Python) developer's perspective, > > > it probably looks all ok because we are quite a lot conda users. When you > > > don't report anything, we are not aware of it. With the necessary > > > information, we can help all to solve the necessary CMake things. > > > > > > Also feel free to collect the relevant JIRAs on a matching Wiki page at > > > https://cwiki.apache.org/confluence/display/ARROW to give an overview of > > > what has to be done to get into a certain OS. > > > > > > Uwe > > > > Would you prefer a single ticket for all static libraries or a single > > ticket per static library? > > One per library as it might be more difficult for some.
