Xiaoyu Yao created HDDS-507:
-------------------------------
Summary: RocksDB fails with SEGFAULT randomly during
PipelineCloseHandler#onMessage
Key: HDDS-507
URL: https://issues.apache.org/jira/browse/HDDS-507
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Xiaoyu Yao
This can be repro-ed by when TestNodeFailure multiple times. Jenkins sometimes
also hit this.
{code}
Current thread (0x00007fbe6f018800): JavaThread
"EventQueue-PipelineCloseForPipelineCloseHandler" daemon [_thread_in_native,
id=58639, stack(0x0000700018009000,0x0000700018109000)]
siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr:
0x000000040000001d
Stack: [0x0000700018009000,0x0000700018109000], sp=0x0000700018108128, free
space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [librocksdbjni6372054043595793813.jnilib+0x163ac8]
rocksdb::GetColumnFamilyID(rocksdb::ColumnFamilyHandle*)+0x8
C [librocksdbjni6372054043595793813.jnilib+0x228368]
rocksdb::DB::Put(rocksdb::WriteOptions const&, rocksdb::ColumnFamilyHandle*,
rocksdb::Slice const&, rocksdb::Slice const&)+0x58
C [librocksdbjni6372054043595793813.jnilib+0x2282fe]
rocksdb::DBImpl::Put(rocksdb::WriteOptions const&,
rocksdb::ColumnFamilyHandle*, rocksdb::Slice const&, rocksdb::Slice const&)+0xe
C [librocksdbjni6372054043595793813.jnilib+0x171c84]
rocksdb::CompactedDBImpl::Open(rocksdb::Options const&,
std::__1::basic_string<char, std::__1::char_traits<char>,
std::__1::allocator<char> > const&, rocksdb::DB**)+0x2a4
C [librocksdbjni6372054043595793813.jnilib+0x971f7]
rocksdb_put_helper(JNIEnv_*, rocksdb::DB*, rocksdb::WriteOptions const&,
rocksdb::ColumnFamilyHandle*, _jbyteArray*, int, int, _jbyteArray*, int,
int)+0x137
j org.rocksdb.RocksDB.put(JJ[BII[BII)V+0
j org.rocksdb.RocksDB.put(Lorg/rocksdb/WriteOptions;[B[B)V+17
j org.apache.hadoop.utils.RocksDBStore.put([B[B)V+10
j
org.apache.hadoop.hdds.scm.pipelines.PipelineSelector.updatePipelineState(Lorg/apache/hadoop/hdds/scm/container/common/helpers/Pipeline;Lorg/apache/hadoop/hdds/protocol/proto/HddsProtos$LifeCycleEvent;)V+222
j
org.apache.hadoop.hdds.scm.pipelines.PipelineSelector.finalizePipeline(Lorg/apache/hadoop/hdds/scm/container/common/helpers/Pipeline;)V+75
j
org.apache.hadoop.hdds.scm.container.ContainerMapping.handlePipelineClose(Lorg/apache/hadoop/hdds/scm/container/common/helpers/PipelineID;)V+18
j
org.apache.hadoop.hdds.scm.pipelines.PipelineCloseHandler.onMessage(Lorg/apache/hadoop/hdds/scm/container/common/helpers/PipelineID;Lorg/apache/hadoop/hdds/server/events/EventPublisher;)V+5
j
org.apache.hadoop.hdds.scm.pipelines.PipelineCloseHandler.onMessage(Ljava/lang/Object;Lorg/apache/hadoop/hdds/server/events/EventPublisher;)V+6
J 5844 C1
org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(Lorg/apache/hadoop/hdds/server/events/EventHandler;Ljava/lang/Object;Lorg/apache/hadoop/hdds/server/events/EventPublisher;)V
(41 bytes) @ 0x0000000115c80bc4 [0x0000000115c80aa0+0x124]
J 5670 C1
org.apache.hadoop.hdds.server.events.SingleThreadExecutor$$Lambda$143.run()V
(20 bytes) @ 0x00000001168f625c [0x00000001168f61c0+0x9c]
j
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
J 3226 C1 java.util.concurrent.ThreadPoolExecutor$Worker.run()V (9 bytes) @
0x0000000116356e44 [0x0000000116356d40+0x104]
J 3107 C1 java.lang.Thread.run()V (17 bytes) @ 0x0000000115d7b0c4
[0x0000000115d7af80+0x144]
v ~StubRoutines::call_stub
V [libjvm.dylib+0x2ef1f6] JavaCalls::call_helper(JavaValue*, methodHandle*,
JavaCallArguments*, Thread*)+0x6ae
V [libjvm.dylib+0x2ef99a] JavaCalls::call_virtual(JavaValue*, KlassHandle,
Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x164
V [libjvm.dylib+0x2efb46] JavaCalls::call_virtual(JavaValue*, Handle,
KlassHandle, Symbol*, Symbol*, Thread*)+0x4a
V [libjvm.dylib+0x34a46d] thread_entry(JavaThread*, Thread*)+0x7c
V [libjvm.dylib+0x56eb0f] JavaThread::thread_main_inner()+0x9b
V [libjvm.dylib+0x57020a] JavaThread::run()+0x1c2
V [libjvm.dylib+0x48d4a6] java_start(Thread*)+0xf6
C [libsystem_pthread.dylib+0x3661] _pthread_body+0x154
C [libsystem_pthread.dylib+0x350d] _pthread_body+0x0
C [libsystem_pthread.dylib+0x2bf9] thread_start+0xd
C 0x0000000000000000
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
