See <https://build.gluster.org/job/regression-test-burn-in/4484/display/redirect?page=changes>
Changes: [Vijay Bellur] libglusterfs: define macros needed for cloudsync [Xavi Hernandez] cluster/ec: Fix handling of heal info cases without locks [Xavi Hernandez] tests/bitrot: enable self-heal daemon before accessing the files [Pranith Kumar K] cluster/afr: Invalidate inode on change of split-brain-choice ------------------------------------------ [...truncated 1.10 MB...] Thread 7 (Thread 0x7f534d04d700 (LWP 19553)): #0 0x00007f535c87a965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f535d7e6bea in rpcsvc_request_handler (arg=0x7f534daea080) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-lib/src/rpcsvc.c>:2189 queue = 0x7f534daea080 program = 0x7f534daea010 req = 0x7f534d04c3a8 tmp_req = 0x7f534d04c3a8 actor = 0x7f534e1e11a0 <glusterfs4_0_fop_actors+64> done = false ret = 0 tmp_list = {next = 0x7f534d04ce80, prev = 0x7f534d04ce80} __FUNCTION__ = "rpcsvc_request_handler" #2 0x00007f535c876dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f535c13dead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 6 (Thread 0x7f5353011700 (LWP 19455)): #0 0x00007f535c87ad12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f535daa2358 in syncenv_task (proc=0x21fd0f0) at <https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c>:612 env = 0x21fcd30 task = 0x0 sleep_till = {tv_sec = 1554484653, tv_nsec = 0} ret = 0 #2 0x00007f535daa254d in syncenv_processor (thdata=0x21fd0f0) at <https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c>:679 env = 0x21fcd30 proc = 0x21fd0f0 task = 0x0 #3 0x00007f535c876dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x00007f535c13dead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 5 (Thread 0x7f5353812700 (LWP 19454)): #0 0x00007f535c87ad12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f535daa2358 in syncenv_task (proc=0x21fcd30) at <https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c>:612 env = 0x21fcd30 task = 0x0 sleep_till = {tv_sec = 1554484653, tv_nsec = 0} ret = 0 #2 0x00007f535daa254d in syncenv_processor (thdata=0x21fcd30) at <https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c>:679 env = 0x21fcd30 proc = 0x21fcd30 task = 0x0 #3 0x00007f535c876dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x00007f535c13dead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 4 (Thread 0x7f5354013700 (LWP 19453)): #0 0x00007f535c104e2d in nanosleep () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f535c104cc4 in sleep () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f535da893f2 in pool_sweeper (arg=0x0) at <https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/mem-pool.c>:466 state = {death_row = {next = 0x0, prev = 0x0}, cold_lists = {0x0 <repeats 1024 times>}, n_cold_lists = 0} pool_list = 0x0 next_pl = 0x0 pt_pool = 0x0 i = 0 poisoned = false #3 0x00007f535c876dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x00007f535c13dead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 3 (Thread 0x7f5355015700 (LWP 19451)): #0 0x00007f535c87de3d in nanosleep () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f535da6133e in gf_timer_proc (data=0x21fbc50) at <https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/timer.c>:194 now = 2514542538186228 now_ts = {tv_sec = 2514542, tv_nsec = 538186228} reg = 0x21fbc50 sleepts = {tv_sec = 1, tv_nsec = 0} event = 0x224d9f0 tmp = 0x7f5340029570 old_THIS = 0x0 #2 0x00007f535c876dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f535c13dead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 2 (Thread 0x7f534c1ec700 (LWP 19560)): #0 0x00007f535c87ad12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f534e443827 in iot_worker (data=0x7f533002b900) at <https://build.gluster.org/job/regression-test-burn-in/ws/xlators/performance/io-threads/src/io-threads.c>:197 conf = 0x7f533002b900 this = 0x7f5330025240 stub = 0x0 sleep_till = {tv_sec = 1554484174, tv_nsec = 2247669} ret = 0 pri = -1 bye = false __FUNCTION__ = "iot_worker" #2 0x00007f535c876dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f535c13dead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 1 (Thread 0x7f533d7f8700 (LWP 19558)): #0 0x00007f5330031cc0 in ?? () No symbol table info available. #1 0x00007f535c52797b in BIO_write () from /lib64/libcrypto.so.10 No symbol table info available. #2 0x00007f5351ba3692 in ssl3_write_pending () from /lib64/libssl.so.10 No symbol table info available. #3 0x00007f5351ba58f3 in ssl3_dispatch_alert () from /lib64/libssl.so.10 No symbol table info available. #4 0x00007f5351ba491a in ssl3_read_bytes () from /lib64/libssl.so.10 No symbol table info available. #5 0x00007f5351ba1644 in ssl3_read_internal () from /lib64/libssl.so.10 No symbol table info available. #6 0x00007f5351defe53 in ssl_do (this=0x7f5344041480, buf=0x7f534c84cf80, len=40, func=0x7f5351bbb5e0 <SSL_read>) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:270 r = -1 priv = 0x7f5344041b80 __FUNCTION__ = "ssl_do" __func__ = "ssl_do" #7 0x00007f5351df08ef in __socket_ssl_readv (this=0x7f5344041480, opvector=0x7f533d7f79b0, opcount=1) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:578 priv = 0x7f5344041b80 sock = 14 ret = -1 __FUNCTION__ = "__socket_ssl_readv" #8 0x00007f5351df09a9 in __socket_ssl_read (this=0x7f5344041480, buf=0x7f534c84cf80, count=40) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:598 iov = {iov_base = 0x7f534c84cf80, iov_len = 40} ret = -1 #9 0x00007f5351df0b0a in __socket_cached_read (this=0x7f5344041480, opvector=0x7f5344041d48, opcount=1) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:636 priv = 0x7f5344041b80 in = 0x7f5344041d18 req_len = 8 ret = -1 #10 0x00007f5351df0f32 in __socket_rwv (this=0x7f5344041480, vector=0x7f5344041d48, count=1, pending_vector=0x7f5344041db8, pending_count=0x7f5344041dc0, bytes=0x7f533d7f7b58, write=0) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:748 priv = 0x7f5344041b80 sock = 14 ret = -1 opvector = 0x7f5344041d48 opcount = 1 moved = 0 __FUNCTION__ = "__socket_rwv" #11 0x00007f5351df1313 in __socket_readv (this=0x7f5344041480, vector=0x7f5344041d48, count=1, pending_vector=0x7f5344041db8, pending_count=0x7f5344041dc0, bytes=0x7f533d7f7b58) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:838 ret = -1 #12 0x00007f5351df70fd in __socket_read_frag (this=0x7f5344041480) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:2291 bytes_read = 0 in = 0x7f5344041d18 priv = 0x7f5344041b80 ret = 0 buf = 0x0 remaining_size = 0 in = 0x7f5344041d18 frag = 0x7f5344041d38 __FUNCTION__ = "__socket_read_frag" #13 0x00007f5351df7892 in __socket_proto_state_machine (this=0x7f5344041480, pollin=0x7f533d7f7cd0) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:2458 ret = 0 priv = 0x7f5344041b80 iobuf = 0x7f5344019298 iobref = 0x0 vector = {{iov_base = 0xffffffff, iov_len = 0}, {iov_base = 0x7f533d7f7c60, iov_len = 0}} in = 0x7f5344041d18 frag = 0x7f5344041d38 __FUNCTION__ = "__socket_proto_state_machine" #14 0x00007f5351df7c1a in socket_proto_state_machine (this=0x7f5344041480, pollin=0x7f533d7f7cd0) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:2552 ret = -1 __FUNCTION__ = "socket_proto_state_machine" #15 0x00007f5351df7d46 in socket_event_poll_in (this=0x7f5344041480, notify_handled=true) at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:2595 ret = -1 pollin = 0x0 priv = 0x7f5344041b80 ctx = 0x7f5344001570 #16 0x00007f5351df8db0 in socket_event_handler (fd=14, idx=0, gen=1, data=0x7f5344041480, poll_in=1, poll_out=0, poll_err=0, event_thread_died=0 '\000') at <https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-transport/socket/src/socket.c>:3001 this = 0x7f5344041480 priv = 0x7f5344041b80 ret = 0 ctx = 0x7f5344001570 socket_closed = false notify_handled = false __FUNCTION__ = "socket_event_handler" #17 0x00007f535dacbdd7 in event_dispatch_epoll_handler (event_pool=0x7f5344034d60, event=0x7f533d7f7e80) at <https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/event-epoll.c>:648 ev_data = 0x7f533d7f7e84 slot = 0x7f53440468b0 handler = 0x7f5351df890b <socket_event_handler> data = 0x7f5344041480 idx = 0 gen = 1 ret = 0 fd = 14 handled_error_previously = false __FUNCTION__ = "event_dispatch_epoll_handler" #18 0x00007f535dacc2f0 in event_dispatch_epoll_worker (data=0x7f53340008e0) at <https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/event-epoll.c>:761 event = {events = 1, data = {ptr = 0x100000000, fd = 0, u32 = 0, u64 = 4294967296}} ret = 1 ev_data = 0x7f53340008e0 event_pool = 0x7f5344034d60 myindex = 1 timetodie = 0 gen = 0 poller_death_notify = {next = 0x0, prev = 0x0} slot = 0x0 tmp = 0x0 __FUNCTION__ = "event_dispatch_epoll_worker" #19 0x00007f535c876dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #20 0x00007f535c13dead in clone () from /lib64/libc.so.6 No symbol table info available. ========================================================= Finish backtrace program name : /build/install/sbin/glusterfsd corefile : /glfs_epoll000-19450.core ========================================================= + rm -f /build/install/cores/gdbout.txt + sort /build/install/cores/liblist.txt + uniq + cat /build/install/cores/liblist.txt.tmp + grep -v /build/install + tar -cf /archives/archived_builds/build-install-regression-test-burn-in-4484.tar /build/install/sbin /build/install/bin /build/install/lib /build/install/libexec /build/install/cores tar: Removing leading `/' from member names + tar -rhf /archives/archived_builds/build-install-regression-test-burn-in-4484.tar -T /build/install/cores/liblist.txt tar: Removing leading `/' from member names + bzip2 /archives/archived_builds/build-install-regression-test-burn-in-4484.tar + rm -f /build/install/cores/liblist.txt + rm -f /build/install/cores/liblist.txt.tmp + find /archives -size +1G -delete -type f + [[ builder207.int.aws.gluster.org == *\a\w\s* ]] + scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i '' /archives/archived_builds/build-install-regression-test-burn-in-4484.tar.bz2 [email protected]:/var/www/glusterfs-logs/regression-test-burn-in-4484.tgz Warning: Identity file not accessible: No such file or directory. Warning: Permanently added 'logs.aws.gluster.org,18.219.45.211' (ECDSA) to the list of known hosts. Permission denied (publickey,gssapi-keyex,gssapi-with-mic). lost connection + true + echo 'Cores and builds archived in https://logs.aws.gluster.org/regression-test-burn-in-4484.tgz' Cores and builds archived in https://logs.aws.gluster.org/regression-test-burn-in-4484.tgz + echo 'Open core using the following command to get a proper stack' Open core using the following command to get a proper stack + echo 'Example: From root of extracted tarball' Example: From root of extracted tarball + echo '\t\tgdb -ex '\''set sysroot ./'\'' -ex '\''core-file ./build/install/cores/xxx.core'\'' <target, say ./build/install/sbin/glusterd>' \t\tgdb -ex 'set sysroot ./' -ex 'core-file ./build/install/cores/xxx.core' <target, say ./build/install/sbin/glusterd> + RET=1 + '[' 1 -ne 0 ']' + tar -czf <https://build.gluster.org/job/regression-test-burn-in/4484/artifact/glusterfs-logs.tgz> /var/log/glusterfs /var/log/messages /var/log/messages-20190310 /var/log/messages-20190317 /var/log/messages-20190324 /var/log/messages-20190331 tar: Removing leading `/' from member names + case $(uname -s) in ++ uname -s + /sbin/sysctl -w kernel.core_pattern=/%e-%p.core kernel.core_pattern = /%e-%p.core + exit 1 Build step 'Execute shell' marked build as failure _______________________________________________ maintainers mailing list [email protected] https://lists.gluster.org/mailman/listinfo/maintainers
