I merged the fix so hopefully this should now work for everyone - thanks Martin for helping us out here.
On Thu, Aug 25, 2016 at 9:03 AM, Martin Grund (Das Grundprinzip.de) < [email protected]> wrote: > What happened is that the Util library was linked to the error-utils-test > (which it is by default anyways because of the way how ADD_BE_TEST works). > Now, what happened is that if you dynamically link the order of the > libraries plays a role. The additional target-link-library would move the > libUtil.so to the end of the library list and if the linker picks this up > some of the symbols declared earlier where already dropped. > > On Wed, Aug 24, 2016 at 11:00 AM Henry Robinson <[email protected]> > wrote: > > > Seems great to me - but do we know why this fixes the issue? Not that > that > > should prevent us from merging the patch. > > > > On 24 August 2016 at 08:54, Tim Armstrong <[email protected]> > wrote: > > > > > The fix works for me too, thanks Martin! > > > > > > I gave the patch a +1 but will upgrade to +2 if it seems everyone is > > happy > > > with it. > > > > > > Cheers, > > > Tim > > > > > > On Wed, Aug 24, 2016 at 8:15 AM, Yonghyun Hwang <[email protected] > > > > > wrote: > > > > > > > Hello Martin, > > > > > > > > Your fix works great! :) As for the build failure, what I've found is > > > that > > > > cmake 3.2.3 causes a build failure w/o your fix while cmake 2.8.12 > > works > > > > good. > > > > > > > > After the commit for "IMPALA-3924: Ubuntu16 support", cmake 3.2.3 got > > > > deployed under "toolchain", which reveals this build issue. > > > > > > > > So, any one using the newer cmake would require your fix to properly > > > build > > > > impala w/ -so. > > > > > > > > Thank you, > > > > Yonghyun > > > > > > > > On Tue, Aug 23, 2016 at 11:40 PM, Daniel Hecht <[email protected]> > > > > wrote: > > > > > > > > > Hi Martin, > > > > > > > > > > Nice to hear from you. > > https://issues.cloudera.org/browse/IMPALA-3979 > > > > > > > > > > Thanks, > > > > > Dan > > > > > > > > > > On Tuesday, August 23, 2016, Martin Grund (Das Grundprinzip.de) < > > > > > [email protected]> wrote: > > > > > > > > > > > I've seen this in the past and propose a patch to fix it: > > > > > > https://gerrit.cloudera.org/#/c/4108/ > > > > > > > > > > > > Let me know if you have JIRA to track this to update the commit > > > > message. > > > > > > > > > > > > Martin > > > > > > > > > > > > On Mon, Aug 22, 2016 at 10:15 PM Laszlo Gaal < > > > [email protected] > > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > link.txt is generated by cmake based on the contents of > > > > CMakeLists.txt > > > > > > and > > > > > > > the project/environment settings. Most of the link-time > > > dependencies > > > > > seem > > > > > > > to be set up in $(IMPALA_HOME)/be/CMakeLists.txt, based on > > > settings > > > > > > > calculated one level up in $(IMPALA_HOME)/CMakeLists.txt. > > > > > > > $(IMPALA_HOME)/be/CMakeLists.txt looks complex; I'll look more > > > into > > > > it > > > > > > > tomorrow. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > - LaszloG > > > > > > > > > > > > > > On Mon, Aug 22, 2016 at 6:07 PM, Yonghyun Hwang < > > > > [email protected] > > > > > > <javascript:;>> > > > > > > > wrote: > > > > > > > > > > > > > > > I observed the same linking error and digged this into > further. > > > The > > > > > > below > > > > > > > > is what I've found. > > > > > > > > > > > > > > > > *1. how to reproduce* > > > > > > > > > > > > > > > > # assuming that impala-config.sh is sourced > > > > > > > > $ cd ${IMPALA_HOME} > > > > > > > > *$ make -j8 impalad* > > > > > > > > ... > > > > > > > > [100%] Built target CodeGen > > > > > > > > Linking CXX executable ../../build/debug/service/impalad > > > > > > > > ... > > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > > `google_breakpad::ExceptionHandler:: > WriteMinidump(std::string > > > > > const&, > > > > > > > bool > > > > > > > > (*)(google_breakpad::MinidumpDescriptor const&, void*, > bool), > > > > > void*)' > > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > > `snappy::MaxCompressedLength(unsigned long)' > > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > `my_strlen' > > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > `LZ4_compress' > > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > > `BZ2_bzDecompress' > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > > > > > *$ cd ${IMPALA_HOME}/be/src/service* > > > > > > > > *$ ${IMPALA_HOME}/toolchain/cmake-3.2.3-p1/bin/cmake -E * > > > > > > > > *cmake_link_script CMakeFiles/impalad.dir/link.txt > --verbose=* > > > > > > > > ... > > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > > ... > > > > > > > > *../../build/debug/util/libUtil.so: undefined reference to > > > > > > > `LZ4_compress'* > > > > > > > > *../../build/debug/util/libUtil.so: undefined reference to > > > > > > > > `BZ2_bzDecompress'* > > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > > *`snappy::RawCompress*(char const*, > > > > > > > > unsigned long, char*, unsigned long*)' > > > > > > > > ... > > > > > > > > boost::regex_traits<char, boost::cpp_regex_traits<char> > > > > > > > > >::do_assign(char > > > > > > > > const*, char > > > > > > > > const*, unsigned int)' > > > > > > > > ... > > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > *`snappy::* > > > > > > > > RawUncompress(char > > > > > > > > const*, unsigned long, char*)' > > > > > > > > > > > > > > > > *2. cause of the failure* > > > > > > > > > > > > > > > > $ cd ${IMPALA_HOME}/be/src/service > > > > > > > > *$ cat CMakeFiles/impalad.dir/link.txt* > > > > > > > > ${IMPALA_HOME}/toolchain/gcc-4.9.2/bin/g++ -Wall > > > > > -Wno-sign-compare > > > > > > > > -Wno-unknown-pragmas -pthread -fno-strict-aliasing -std=c++14 > > > > > > > > -Wno-deprecated ... > > > > > > > > *../../../toolchain/snappy-1.1.3/lib/libsnappy.a > > > > > > > > ../../../toolchain/lz4-svn/lib/liblz4.a* > > > > > > > > ... > > > > > > > > *../../../toolchain/boost-1.57.0/lib/libboost_regex.a* > > > > > > > > ... > > > > > > > > *../../build/debug/util/libUtil.so* > > > > > > > > -lpthread -ldl -lssl -lcrypto > > > > > > > > > > > > > > > > Apparently, libUtil.so __should__ show before libsnappy.a and > > > > > *regex.a. > > > > > > > > > > > > > > > > > > > > > > > > Next step is to find out who creates > > CMakeFiles/impalad.dir/link. > > > > txt > > > > > > and > > > > > > > > why libUtil.so is placed after libsnappy.a, liblz4.a, and > > > > > > > libboost_regex.a > > > > > > > > > > > > > > > > Thank you, > > > > > > > > Yonghyun > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Aug 17, 2016 at 10:13 AM, Henry Robinson < > > > > [email protected] > > > > > > <javascript:;>> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Only that it's IMPALA-3979. Does anyone have any theories > > yet? > > > > > > > > > > > > > > > > > > On 17 August 2016 at 10:10, Dimitris Tsirogiannis < > > > > > > > > > [email protected] <javascript:;>> wrote: > > > > > > > > > > > > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > > > > > When I build from the latest master using the > > > > -build_shared_libs > > > > > > > > option I > > > > > > > > > > get linking errors (see [1]). Building with static works > > > fine. > > > > > Any > > > > > > > > ideas? > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > Dimitris > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > Linking CXX executable ../../build/debug/service/impalad > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'BZ2_bzBuffToBuffCompress' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'snappy::MaxCompressedLength(unsigned long)' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'snappy::RawCompress(char const*, unsigned long, char*, > > > > unsigned > > > > > > > > long*)' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'LZ4_compress' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'BZ2_bzDecompressEnd' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'BZ2_bzDecompressInit' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'BZ2_bzBuffToBuffDecompress' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'BZ2_bzDecompress' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'snappy::GetUncompressedLength(char const*, unsigned > long, > > > > > > unsigned > > > > > > > > > > long*)' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'snappy::RawUncompress(char const*, unsigned long, > char*)' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'LZ4_uncompress' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'my_strlen' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'google_breakpad::ExceptionHandler:: > > > WriteMinidump(std::string > > > > > > const&, > > > > > > > > > bool > > > > > > > > > > (*)(google_breakpad::MinidumpDescriptor const&, void*, > > > bool), > > > > > > void*)' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > > > 'google_breakpad::ExceptionHandler::ExceptionHandler(google_ > > > > > > > > > > breakpad::MinidumpDescriptor > > > > > > > > > > const&, bool (*)(void*), bool (*)(google_breakpad:: > > > > > > MinidumpDescriptor > > > > > > > > > > const&, void*, bool), void*, bool, int)' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'ProfilerStart' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'ProfilerStop' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'boost::basic_regex<char, boost::regex_traits<char, > > > > > > > > > > boost::cpp_regex_traits<char> > >::do_assign(char const*, > > > char > > > > > > > const*, > > > > > > > > > > unsigned int)' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_ > > > > > iterator<char > > > > > > > > > const*, > > > > > > > > > > std::string>, > > > > > > > > > > std::allocator<boost::sub_match<__gnu_cxx::__normal_ > > > > > iterator<char > > > > > > > > > const*, > > > > > > > > > > std::string> > >, boost::regex_traits<char, > > > > > > > > > boost::cpp_regex_traits<char> > > > > > > > > > > > >::find()' > > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > > reference > > > > to > > > > > > > > > > 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_ > > > > > iterator<char > > > > > > > > > const*, > > > > > > > > > > std::string>, > > > > > > > > > > std::allocator<boost::sub_match<__gnu_cxx::__normal_ > > > > > iterator<char > > > > > > > > > const*, > > > > > > > > > > std::string> > >, boost::regex_traits<char, > > > > > > > > > boost::cpp_regex_traits<char> > > > > > > > > > > > >::construct_init(boost::basic_regex<char, > > > > > > boost::regex_traits<char, > > > > > > > > > > boost::cpp_regex_traits<char> > > const&, > > > > > > > > > > boost::regex_constants::_match_flags)' > > > > > > > > > > /opt/Impala-Toolchain/openldap-2.4.25/lib/libldap.a( > > > > > > > > > > os-ip.o):os-ip.c:function > > > > > > > > > > ldap_int_poll: warning: `sys_nerr' is deprecated; use > > > > `strerror' > > > > > or > > > > > > > > > > `strerror_r' instead > > > > > > > > > > /opt/Impala-Toolchain/openldap-2.4.25/lib/libldap.a( > > > > > > > > > > os-ip.o):os-ip.c:function > > > > > > > > > > ldap_int_poll: warning: `sys_errlist' is deprecated; use > > > > > `strerror' > > > > > > > or > > > > > > > > > > `strerror_r' instead > > > > > > > > > > collect2: error: ld returned 1 exit status > > > > > > > > > > be/src/service/CMakeFiles/impalad.dir/build.make:166: > > recipe > > > > for > > > > > > > > target > > > > > > > > > > 'be/build/debug/service/impalad' failed > > > > > > > > > > make[3]: *** [be/build/debug/service/impalad] Error 1 > > > > > > > > > > CMakeFiles/Makefile2:3682: recipe for target > > > > > > > > > > 'be/src/service/CMakeFiles/impalad.dir/all' failed > > > > > > > > > > make[2]: *** [be/src/service/CMakeFiles/impalad.dir/all] > > > > Error 2 > > > > > > > > > > CMakeFiles/Makefile2:3694: recipe for target > > > > > > > > > > 'be/src/service/CMakeFiles/impalad.dir/rule' failed > > > > > > > > > > make[1]: *** [be/src/service/CMakeFiles/ > impalad.dir/rule] > > > > Error > > > > > 2 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Henry Robinson > > > > > > > > > Software Engineer > > > > > > > > > Cloudera > > > > > > > > > 415-994-6679 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Henry Robinson > > Software Engineer > > Cloudera > > 415-994-6679 > > >
