Hi Sanju,
Thanks for the response.
I have resolved the issue, actually I have updated from 3.7.6 to 5.0,
in new version RPC is coming from libtirpb , but I forgot to enable
"--with-libtirpc" in configuration.
After enabling able to start glusterd.
Regards,
Abhishek
On Wed, Mar 6, 2019 at 12:58 PM Sanju Rakonde <srako...@redhat.com
<mailto:srako...@redhat.com>> wrote:
Abhishek,
We need below information on investigate this issue.
1. gluster --version
2. Please run glusterd in gdb, so that we can capture the
backtrace. I see some rpc errors in log, but backtrace will be
more helpful.
To run glusterd in gdb, you need start glusterd in gdb (i.e.
gdb glusterd, and then give the command "run -N"). when you see a
segmentation fault, please capture the backtrace and
paste it here.
On Wed, Mar 6, 2019 at 10:07 AM ABHISHEK PALIWAL
<abhishpali...@gmail.com <mailto:abhishpali...@gmail.com>> wrote:
Hi Team,
I am facing the issue where at the time of starting the
glusterd segmentation fault is reported.
Below are the logs
root@128:/usr/sbin# ./glusterd --debug
[1970-01-01 15:19:43.940386] I [MSGID: 100030]
[glusterfsd.c:2691:main] 0-./glusterd: Started running
./glusterd version 5.0 (args: ./glusterd --debug)
[1970-01-01 15:19:43.940855] D
[logging.c:1833:__gf_log_inject_timer_event] 0-logging-infra:
Starting timer now. Timeout = 120, current buf size = 5
[1970-01-01 15:19:43.941736] D [MSGID: 0]
[glusterfsd.c:747:get_volfp] 0-glusterfsd: loading volume file
/etc/glusterfs/glusterd.vol
[1970-01-01 15:19:43.945796] D [MSGID: 101097]
[xlator.c:341:xlator_dynload_newway] 0-xlator:
dlsym(xlator_api) on
/usr/lib64/glusterfs/5.0/xlator/mgmt/glusterd.so: undefined
symbol: xlator_api. Fall back to old symbols
[1970-01-01 15:19:43.946279] I [MSGID: 106478]
[glusterd.c:1435:init] 0-management: Maximum allowed open file
descriptors set to 65536
[1970-01-01 15:19:43.946419] I [MSGID: 106479]
[glusterd.c:1491:init] 0-management: Using /var/lib/glusterd
as working directory
[1970-01-01 15:19:43.946515] I [MSGID: 106479]
[glusterd.c:1497:init] 0-management: Using /var/run/gluster as
pid file working directory
[1970-01-01 15:19:43.946968] D [MSGID: 0]
[glusterd.c:458:glusterd_rpcsvc_options_build] 0-glusterd:
listen-backlog value: 10
[1970-01-01 15:19:43.947139] D [rpcsvc.c:2607:rpcsvc_init]
0-rpc-service: RPC service inited.
[1970-01-01 15:19:43.947241] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
[1970-01-01 15:19:43.947379] D
[rpc-transport.c:269:rpc_transport_load] 0-rpc-transport:
attempt to load file
/usr/lib64/glusterfs/5.0/rpc-transport/socket.so
[1970-01-01 15:19:43.955198] D [socket.c:4464:socket_init]
0-socket.management: Configued transport.tcp-user-timeout=0
[1970-01-01 15:19:43.955316] D [socket.c:4482:socket_init]
0-socket.management: Reconfigued transport.keepalivecnt=9
[1970-01-01 15:19:43.955415] D
[socket.c:4167:ssl_setup_connection_params]
0-socket.management: SSL support on the I/O path is NOT enabled
[1970-01-01 15:19:43.955504] D
[socket.c:4170:ssl_setup_connection_params]
0-socket.management: SSL support for glusterd is NOT enabled
[1970-01-01 15:19:43.955612] D
[name.c:572:server_fill_address_family] 0-socket.management:
option address-family not specified, defaulting to inet6
[1970-01-01 15:19:43.955928] D
[rpc-transport.c:269:rpc_transport_load] 0-rpc-transport:
attempt to load file
/usr/lib64/glusterfs/5.0/rpc-transport/rdma.so
[1970-01-01 15:19:43.956079] E
[rpc-transport.c:273:rpc_transport_load] 0-rpc-transport:
/usr/lib64/glusterfs/5.0/rpc-transport/rdma.so: cannot open
shared object file: No such file or directory
[1970-01-01 15:19:43.956177] W
[rpc-transport.c:277:rpc_transport_load] 0-rpc-transport:
volume 'rdma.management': transport-type 'rdma' is not valid
or not found on this machine
[1970-01-01 15:19:43.956270] W
[rpcsvc.c:1789:rpcsvc_create_listener] 0-rpc-service: cannot
create listener, initing the transport failed
[1970-01-01 15:19:43.956362] E [MSGID: 106244]
[glusterd.c:1798:init] 0-management: creation of 1 listeners
failed, continuing with succeeded transport
[1970-01-01 15:19:43.956459] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc peer, Num: 1238437, Ver: 2,
Port: 0
[1970-01-01 15:19:43.956561] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc cli read-only, Num: 1238463,
Ver: 2, Port: 0
[1970-01-01 15:19:43.956666] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc mgmt, Num: 1238433, Ver: 2,
Port: 0
[1970-01-01 15:19:43.956758] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc mgmt v3, Num: 1238433, Ver:
3, Port: 0
[1970-01-01 15:19:43.956853] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: Gluster Portmap, Num: 34123456, Ver: 1,
Port: 0
[1970-01-01 15:19:43.956946] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: Gluster Handshake, Num: 14398633, Ver: 2,
Port: 0
[1970-01-01 15:19:43.957062] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: Gluster MGMT Handshake, Num: 1239873, Ver:
1, Port: 0
[1970-01-01 15:19:43.957205] D [rpcsvc.c:2607:rpcsvc_init]
0-rpc-service: RPC service inited.
[1970-01-01 15:19:43.957303] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
[1970-01-01 15:19:43.957408] D
[rpc-transport.c:269:rpc_transport_load] 0-rpc-transport:
attempt to load file
/usr/lib64/glusterfs/5.0/rpc-transport/socket.so
[1970-01-01 15:19:43.957563] D [socket.c:4424:socket_init]
0-socket.management: disabling nodelay
[1970-01-01 15:19:43.957650] D [socket.c:4464:socket_init]
0-socket.management: Configued transport.tcp-user-timeout=0
[1970-01-01 15:19:43.957738] D [socket.c:4482:socket_init]
0-socket.management: Reconfigued transport.keepalivecnt=9
[1970-01-01 15:19:43.957830] D
[socket.c:4167:ssl_setup_connection_params]
0-socket.management: SSL support on the I/O path is NOT enabled
[1970-01-01 15:19:43.957922] D
[socket.c:4170:ssl_setup_connection_params]
0-socket.management: SSL support for glusterd is NOT enabled
[1970-01-01 15:19:43.958186] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc cli, Num: 1238463, Ver: 2,
Port: 0
[1970-01-01 15:19:43.958280] D
[rpcsvc.c:2146:rpcsvc_program_register] 0-rpc-service: New
program registered: Gluster Handshake (CLI Getspec), Num:
14398633, Ver: 2, Port: 0
[1970-01-01 15:19:43.958461] D [MSGID: 0]
[glusterd-utils.c:7878:glusterd_sm_tr_log_init] 0-glusterd:
returning 0
[1970-01-01 15:19:43.958557] D [MSGID: 0]
[glusterd.c:1875:init] 0-management: cannot get
run-with-valgrind value
[1970-01-01 15:19:43.960895] E [MSGID: 101032]
[store.c:447:gf_store_handle_retrieve] 0-: Path corresponding
to /var/lib/glusterd/glusterd.info <http://glusterd.info>. [No
such file or directory]
[1970-01-01 15:19:43.961016] D [MSGID: 0]
[store.c:452:gf_store_handle_retrieve] 0-: Returning -1
[1970-01-01 15:19:43.961108] D [MSGID: 0]
[glusterd-store.c:2169:glusterd_retrieve_op_version]
0-management: Unable to get store handle!
[1970-01-01 15:19:43.961216] E [MSGID: 101032]
[store.c:447:gf_store_handle_retrieve] 0-: Path corresponding
to /var/lib/glusterd/glusterd.info <http://glusterd.info>. [No
such file or directory]
[1970-01-01 15:19:43.961325] D [MSGID: 0]
[store.c:452:gf_store_handle_retrieve] 0-: Returning -1
[1970-01-01 15:19:43.961428] D [MSGID: 0]
[glusterd-store.c:2345:glusterd_retrieve_uuid] 0-management:
Unable to get storehandle!
[1970-01-01 15:19:43.961523] D [MSGID: 0]
[glusterd-store.c:2366:glusterd_retrieve_uuid] 0-management:
Returning -1
[1970-01-01 15:19:43.961617] I [MSGID: 106514]
[glusterd-store.c:2304:glusterd_restore_op_version]
0-management: Detected new install. Setting op-version to
maximum : 50000
[1970-01-01 15:19:43.962658] D [MSGID: 0]
[store.c:432:gf_store_handle_new] 0-: Returning 0
[1970-01-01 15:19:43.962750] D [MSGID: 0]
[store.c:452:gf_store_handle_retrieve] 0-: Returning 0
[1970-01-01 15:19:43.963047] D [MSGID: 0]
[store.c:515:gf_store_iter_new] 0-: Returning with 0
[1970-01-01 15:19:43.963194] D [MSGID: 0]
[store.c:632:gf_store_iter_get_next] 0-: Returning with 0
[1970-01-01 15:19:43.963318] D [MSGID: 0]
[store.c:632:gf_store_iter_get_next] 0-: Returning with -1
[1970-01-01 15:19:43.963455] D [MSGID: 0]
[store.c:473:gf_store_handle_destroy] 0-: Returning 0
[1970-01-01 15:19:43.963757] D [MSGID: 0]
[glusterd-store.c:3546:glusterd_store_retrieve_volumes]
0-management: Returning with 0
[1970-01-01 15:19:43.964159] D [MSGID: 0]
[glusterd-store.c:4662:glusterd_store_retrieve_peers]
0-management: Returning with 0
[1970-01-01 15:19:43.964471] I [MSGID: 106194]
[glusterd-store.c:3983:glusterd_store_retrieve_missed_snaps_list]
0-management: No missed snaps list.
[1970-01-01 15:19:43.964580] D [MSGID: 0]
[glusterd-store.c:4104:glusterd_store_retrieve_snaps]
0-management: Returning with 0
[1970-01-01 15:19:43.964680] D [MSGID: 0]
[glusterd-store.c:4894:glusterd_restore] 0-management: Returning 0
[1970-01-01 15:19:43.965060] D [MSGID: 0]
[options.c:1225:xlator_option_init_int32] 0-management: option
event-threads using set value 1
Final graph:
+------------------------------------------------------------------------------+
1: volume management
2: type mgmt/glusterd
3: option rpc-auth.auth-glusterfs on
4: option rpc-auth.auth-unix on
5: option rpc-auth.auth-null on
6: option rpc-auth-allow-insecure on
7: option transport.listen-backlog 10
8: option event-threads 1
9: option ping-timeout 0
10: option transport.socket.read-fail-log off
11: option transport.socket.keepalive-interval 2
12: option transport.socket.keepalive-time 10
13: option transport-type rdma
14: option working-directory /var/lib/glusterd
15: end-volume
16:
+------------------------------------------------------------------------------+
[1970-01-01 15:19:43.966808] I [MSGID: 101190]
[event-epoll.c:622:event_dispatch_epoll_worker] 0-epoll:
Started thread with index 1
[1970-01-01 15:19:44.840454] E
[rpcsvc.c:513:rpcsvc_request_create] 0-rpc-service: RPC
version not supported (XID: 0x0, Ver: 0, Program: 0, ProgVers:
0, Proc: 2) from trans (socket.management)
[1970-01-01 15:19:44.840884] D
[rpcsvc.c:1416:rpcsvc_error_reply] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xffbac)[0x3fffa12acbe4]
(--> /usr/lib64/libgfrpc.so.0(+0xc5f4)[0x3fffa12525f4] (-->
/usr/lib64/libgfrpc.so.0(+0xcf00)[0x3fffa1252f00] (-->
/usr/lib64/libgfrpc.so.0(+0xd224)[0x3fffa1253224] (-->
/usr/lib64/libgfrpc.so.0(+0xd84c)[0x3fffa125384c] ))))) 0-:
sending a RPC error reply
[1970-01-01 15:19:44.841055] D
[logging.c:1805:gf_log_flush_extra_msgs] 0-logging-infra: Log
buffer size reduced. About to flush 5 extra log messages
[1970-01-01 15:19:44.841156] D
[logging.c:1808:gf_log_flush_extra_msgs] 0-logging-infra: Just
flushed 5 extra log messages
pending frames:
patchset: git://git.gluster.org/glusterfs.git
<http://git.gluster.org/glusterfs.git>
signal received: 11
time of crash:
1970-01-01 15:19:44
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 5.0
/usr/lib64/libglusterfs.so.0(+0x422a4)[0x3fffa12ab2a4]
/usr/lib64/libglusterfs.so.0(gf_print_trace-0xf5080)[0x3fffa12b82e0]
./glusterd(glusterfsd_print_trace-0x22fa4)[0x100067ec]
linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0)[0x3fffa13f0478]
/lib64/libc.so.6(xdr_accepted_reply-0x72d3c)[0x3fffa11375cc]
/lib64/libc.so.6(xdr_accepted_reply-0x72d9c)[0x3fffa113756c]
/lib64/libc.so.6(xdr_union-0x63a94)[0x3fffa1147dd4]
/lib64/libc.so.6(xdr_replymsg-0x72c58)[0x3fffa11376e0]
/lib64/libc.so.6(xdr_sizeof-0x62a78)[0x3fffa1149120]
/usr/lib64/libgfrpc.so.0(+0x9b0c)[0x3fffa124fb0c]
/usr/lib64/libgfrpc.so.0(rpcsvc_submit_generic-0x149f4)[0x3fffa125228c]
/usr/lib64/libgfrpc.so.0(+0xc614)[0x3fffa1252614]
/usr/lib64/libgfrpc.so.0(+0xcf00)[0x3fffa1252f00]
/usr/lib64/libgfrpc.so.0(+0xd224)[0x3fffa1253224]
/usr/lib64/libgfrpc.so.0(+0xd84c)[0x3fffa125384c]
/usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x10eec)[0x3fffa125610c]
/usr/lib64/glusterfs/5.0/rpc-transport/socket.so(+0xc09c)[0x3fff9d51709c]
/usr/lib64/libglusterfs.so.0(+0xb84bc)[0x3fffa13214bc]
/lib64/libpthread.so.0(+0xbb30)[0x3fffa11bdb30]
/lib64/libc.so.6(clone-0x9e964)[0x3fffa110817c]
---------
Segmentation fault (core dumped)
Could you please help me, what actually the problem?
--
Regards
Abhishek Paliwal
_______________________________________________
Gluster-users mailing list
gluster-us...@gluster.org <mailto:gluster-us...@gluster.org>
https://lists.gluster.org/mailman/listinfo/gluster-users
--
Thanks,
Sanju
--
Regards
Abhishek Paliwal
_______________________________________________
Gluster-users mailing list
gluster-us...@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
