Hi Arya,

The hash that we're interested in is 51659402847B9CA647AB6C35F403A7EB0
(which is the expected hash of the symbols for the impalad binary).

It is found in the "debug_identifier" field of your minidump.
So minidump_stackwalk will look in your ~/impala-syms  path for a directory
with this hash. Looks like it is not finding it. You can confirm by running:

find ~/impala-syms  -name 51659402847B9CA647AB6C35F403A7EB0

If it returns nothing, then the symbols you dumped from your previous build
are not the same ones that will be able to read this minidump.

Can you build this branch on a machine with the same Ubuntu version as the
machine that the MDMP came from, then dump the syms and try again?

On Mon, Feb 12, 2018 at 2:55 PM, Arya Goudarzi <gouda...@gmail.com> wrote:

> Thank you for your reply Vincent.
> We don't use Redhat. We are on Ubuntu and we use Cloudera Parcels for
> installation. Is there a way to build debug symbols from the parcel
> package?
> I just confirmed that the source I build if for the same git tag/hash, so
> I am still puzzled why the symbols don't show up:
> root@cloudera-worker-beef-eb891d7e:/opt/cloudera/
> parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/impala/sbin-retail# impalad
> --version
> impalad version 2.7.0-cdh5.10.0 RELEASE (build
> *785a073cd07e2540d521ecebb8b38161ccbd2aa2*)
> Built on Fri Jan 20 12:03:56 PST 2017
> On Thu, Feb 8, 2018 at 8:46 PM Vincent Tran <vtt...@cloudera.com> wrote:
>> Looks like the hash of the symbols from your build does not match the one
>> in the minidump.
>> Can you try to pull the symbols from the rpms and try again?
>> I've only used the symbols from local builds on minidumps generated by
>> the same minicluster. So I don't know if you can use that method for
>> minidumps written by remote clusters. If it is to work, the build machine
>> will at least need to share the same OS as the host that wrote the minidump.
>> On Feb 8, 2018 11:07 PM, "Arya Goudarzi" <gouda...@gmail.com> wrote:
>>> Hi Team,
>>> I am trying to troubleshoot a SIGSEGV crash we are getting with Impala
>>> 2.7 frequently. I followed the instructions here:
>>> https://cwiki.apache.org/confluence/display/IMPALA/
>>> Debugging+Impala+Minidumps
>>> However, it seems symboles are not resolving when I try to extract them
>>> with minidump_stackwalk.
>>> Here are the details:
>>> CDH5.10.0
>>> Impala 2.7 build 785a073cd07e2540d521ecebb8b38161ccbd2aa2
>>> What I have done is the following:
>>>    - clone Cloudera/Impala from github
>>>    - checkout 785a073cd07e2540d521ecebb8b38161ccbd2aa2
>>>    - ./configure
>>>    - make
>>>    - ./bin/dump_breakpad_symbols.py -b be/build/latest -d ~/impala-syms
>>>    --dump_syms ./toolchain/breakpad-20150612-p1/bin/dump_syms
>>>    - ./toolchain/breakpad-20150612-p1/bin/minidump_stackwalk
>>>    ~/034329c7-9088-0fa3-3fb76e38-2f2208f5.dmp ~/impala-syms >
>>>    ~/resolved.txt
>>> However, symbols aren't resolved. If I look at the end of resolved.txt I
>>> see this:
>>> Loaded modules:
>>> 0x00400000 - 0x02830fff  impalad  ???  (main)  (WARNING: No symbols,
>>> impalad, 51659402847B9CA647AB6C35F403A7EB0)
>>> 0x7f6071517000 - 0x7f6071725fff  libudfsample.64523.0.so  ???
>>> 0x7f60fd341000 - 0x7f60fd559fff  libresolv-2.19.so  ???
>>> 0x7f60fd55c000 - 0x7f60fd761fff  libnss_dns-2.19.so  ???
>>> ....
>>> I have repeated the process many time to confirm I am not making a
>>> mistake following the steps, but I might be missing something.
>>> Your advise is greatly appreciated.
>>> Cheers,
>>> -Arya

Vincent T. Tran
Customer Operations Engineer
Cloudera, Inc.

Reply via email to