[Apologies to the list if someone has already responded to Michael,
but I don't recall seeing it.]
Hi Michael,
By far the easiest way to debug Slurm problems is by doing your own,
local build, outside the context of RPM. You can find appropriate
./configure arguments (at least to start) in the slurm.spec file,
then do the standard
$ ./configure {args }
$ make
$ make install
(and, optionally,)
$ make contrib-install
Alternatively, if you really like the rpmbuild approach, there is
probably an argument in the spec file to either
[*]Generate a debuginfo rpm
[*]Skip stripping the debug info
HTH!
Andy
On 04/25/2016 12:20 PM, Michael Kit
Gilbert wrote:
using gdb to debug slurm-15.08?
Hello everyone,
I'm trying to troubleshoot a problem with a local patch I'm
writing for Slurm and can't seem to get gdb working properly.
I've built the rpms with -D '%_with_cflags CFLAGS="-O0 -g3"'
and slurmctld and slurmd appear to be starting normally.
However, when I try to attach gdb to the slurmd process, I get
an error "Missing separate
debuginfos, use: debuginfo-install
slurm-15.08.4-1.el6.x86_64". I haven't had any luck
finding debuginfo rpms for slurm-15.
Hoping it was just a minor issue, I continued with
debugging and here's what I got:
[root@head
slurm_uid_patch]# gdb slurmd 26127
GNU gdb (GDB) Red Hat
Enterprise Linux (7.2-60.el6_4.1)
Copyright (C) 2010 Free
Software Foundation, Inc.
License GPLv3+: GNU GPL
version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software:
you are free to change and redistribute it.
There is NO WARRANTY,
to the extent permitted by law.� Type "show copying"
and "show warranty" for
details.
This GDB was configured
as "x86_64-redhat-linux-gnu".
For bug reporting
instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from
/usr/sbin/slurmd...done.
Attaching to program:
/usr/sbin/slurmd, process 26127
Reading symbols from
/lib64/libdl.so.2...Reading symbols from
/usr/lib/debug/lib64/libdl-2.12.so.debug...done.
done.
Loaded symbols for
/lib64/libdl.so.2
Reading symbols from
/lib64/libpthread.so.0...Reading symbols from
/usr/lib/debug/lib64/libpthread-2.12.so.debug...done.
[Thread debugging using
libthread_db enabled]
done.
Loaded symbols for
/lib64/libpthread.so.0
Reading symbols from
/lib64/libc.so.6...Reading symbols from
/usr/lib/debug/lib64/libc-2.12.so.debug...done.
done.
Loaded symbols for
/lib64/libc.so.6
Reading symbols from
/lib64/ld-linux-x86-64.so.2...Reading symbols from
/usr/lib/debug/lib64/ld-2.12.so.debug...done.
done.
Loaded symbols for
/lib64/ld-linux-x86-64.so.2
Reading symbols from
/usr/lib64/slurm/select_cons_res.so...done.
Loaded symbols for
/usr/lib64/slurm/select_cons_res.so
Reading symbols from
/usr/lib64/slurm/gres_nic.so...done.
Loaded symbols for
/usr/lib64/slurm/gres_nic.so
Reading symbols from
/usr/lib64/slurm/topology_none.so...done.
Loaded symbols for
/usr/lib64/slurm/topology_none.so
Reading symbols from
/usr/lib64/slurm/route_default.so...done.
Loaded symbols for
/usr/lib64/slurm/route_default.so
Reading symbols from
/usr/lib64/slurm/proctrack_cgroup.so...done.
Loaded symbols for
/usr/lib64/slurm/proctrack_cgroup.so
Reading symbols from
/usr/lib64/slurm/task_cgroup.so...done.
Loaded symbols for
/usr/lib64/slurm/task_cgroup.so
Reading symbols from
/usr/lib64/slurm/auth_munge.so...done.
Loaded symbols for
/usr/lib64/slurm/auth_munge.so
Reading symbols from
/usr/lib64/libmunge.so.2...(no debugging symbols
found)...done.
Loaded symbols for
/usr/lib64/libmunge.so.2
Reading symbols from
/usr/lib64/slurm/crypto_munge.so...done.
Loaded symbols for
/usr/lib64/slurm/crypto_munge.so
Reading symbols from
/usr/lib64/slurm/jobacct_gather_none.so...done.
Loaded symbols for
/usr/lib64/slurm/jobacct_gather_none.so
Reading symbols from
/usr/lib64/slurm/job_container_none.so...done.
Loaded symbols for
/usr/lib64/slurm/job_container_none.so
Reading symbols from
/usr/lib64/slurm/core_spec_none.so...done.
Loaded symbols for
/usr/lib64/slurm/core_spec_none.so
Reading symbols from
/usr/lib64/slurm/switch_none.so...done.
Loaded symbols for
/usr/lib64/slurm/switch_none.so
Reading symbols from
/usr/lib64/slurm/acct_gather_energy_none.so...done.
Loaded symbols for
/usr/lib64/slurm/acct_gather_energy_none.so
Reading symbols from
/usr/lib64/slurm/acct_gather_profile_none.so...done.
Loaded symbols for
/usr/lib64/slurm/acct_gather_profile_none.so
Reading symbols from
/usr/lib64/slurm/acct_gather_infiniband_none.so...done.
Loaded symbols for
/usr/lib64/slurm/acct_gather_infiniband_none.so
Reading symbols from
/usr/lib64/slurm/acct_gather_filesystem_none.so...done.
Loaded symbols for
/usr/lib64/slurm/acct_gather_filesystem_none.so
0x0000003fd240eadd in
accept () at ../sysdeps/unix/syscall-template.S:82
82 � � �T_PSEUDO
(SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
Missing separate
debuginfos, use: debuginfo-install
slurm-15.08.4-1.el6.x86_64
(gdb) bt
#0 �0x0000003fd240eadd
in accept () at ../sysdeps/unix/syscall-template.S:82
#1 �0x00000000004df938
in slurm_accept_msg_conn ()
#2 �0x0000000000427f0c
in _msg_engine ()
#3 �0x0000000000427c61
in main ()
(gdb) *thread apply
all bt full*
Thread 1 (Thread
0x7f684747c700 (LWP 26127)):
#0 �0x0000003fd240eadd
in accept () at ../sysdeps/unix/syscall-template.S:82
No locals.
#1 �0x00000000004df938
in slurm_accept_msg_conn ()
No symbol table info
available.
#2 �0x0000000000427f0c
in _msg_engine ()
No symbol table info
available.
#3 �0x0000000000427c61
in main ()
No symbol table info
available.
I found a post
where someone had recommended to do "thread apply all bt
full", so I did that as well. This call is in bold text
above.
Does anyone have
any ideas as to how I can get debugging to work?
Thanks!
Mike
