Thanks Todd and Thomas for your detailed reply! I think it's doable to link the libkudu_client statically to avoid upgrade issues like mine, since not all the libraries in native-toolchain are linked dynamically (e.g. liborc is linked statically). It'd be better if we can provide build options to link them statically, so I just create a JIRA for this: IMPALA-7066.
Thanks, Quanlong At 2018-05-24 01:05:41, "Thomas Tauber-Marshall" <[email protected]> wrote: >There isn't a built in way in buildall to statically link the Kudu client, >and I'm not aware of any way to do it. > >We don't provide any guarantees or do any testing around compatibility of >Impala with Kudu clients of different versions other than the versions that >correspond, i.e. the value of IMPALA_KUDU_VERSION for the branch of Impala >that you want to run. If you upgraded Impala but were dynamically linking >against an older version of the Kudu client that may have been what caused >the crash. > >And Todd is right as well that you'll need to use a libkudu that was built >with Impala's toolchain, either by building the native-toolchain project >yourself, or by downloading the pre-built toolchain with >bin/bootstrap-toolchain.py, which will automatically grab the right libkudu >version for the branch of Impala that you're building. > >On Tue, May 22, 2018 at 10:14 PM Todd Lipcon <[email protected]> wrote: > >> Hi Quanlong, >> >> I think the comment you found must be out of date. I think Thomas may know >> the latest state of how the Kudu client is built as part of >> "native-toolchain". >> >> One thing to note is that, since Impala builds with its own toolchain, you >> need to make sure that it dynamic-links against the libkudu_client that >> comes from the toolchain. If you end up linking against one built using >> your system toolchain there is a chance that you'll get ABI mismatches and >> strange crashes. Perhaps that's what you saw when you first tried. >> >> -Todd >> >> On Tue, May 22, 2018 at 3:21 PM, Quanlong Huang <[email protected]> >> wrote: >> >>> Hi all, >>> >>> >>> Recently when I tried to upgrade our Impala cluster in CDH5.7.1 to >>> Impala-2.12, impalad crashed when inserting data to kudu tables. However, >>> it works when reading from kudu tables. >>> Finally, I found out that the kudu client >>> (/usr/lib/x86_64-linux-gnu/libkudu_client.so.0) is still linked >>> dynamically. Issues are resolved after I update libkudu_client.so.0. >>> >>> >>> In kudu-table-sink.cc, the comments said the kudu-client can be linked >>> statically: >>> https://github.com/apache/impala/blob/2.12.0/be/src/exec/kudu-table-sink.cc#L151 >>> However, when building with -static in ./buildall.sh, the kudu-client is >>> still linked dynamically (see `ldd be/build/latest/service/impalad`). Is >>> there a build option to link it statically? >>> >> >>> >>> Thanks, >>> Quanlong >> >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >>
