- **status**: unassigned --> assigned
- **assigned_to**: Anders Bjornerstedt
- **Milestone**: 4.3.3 --> 4.4.1
---
** [tickets:#1076] 2PBE: pbed aborts at pbeClosePrepareTrans**
**Status:** assigned
**Milestone:** 4.4.1
**Created:** Mon Sep 15, 2014 06:52 AM UTC by Sirisha Alla
**Last Updated:** Mon Sep 15, 2014 06:52 AM UTC
**Owner:** Anders Bjornerstedt
The issue is seen on SLES X86 with 2PBE and 50k objects. Opensaf is running on
changeset 5697 + #946 patches
Syslog on SC-2:
Sep 12 19:15:00 SLES-64BIT-SLOT2 osafamfnd[2409]: NO Assigned
'safSi=SC-2N,safApp=OpenSAF' ACTIVE to 'safSu=SC-2,safSg=2N,safApp=OpenSAF'
Sep 12 19:15:00 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer disconnected
618 <0, 2010f> (@OpenSafImmReplicatorA)
Sep 12 19:15:00 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:00 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer (applier)
connected: 640 (@OpenSafImmReplicatorA) <0, 2010f>
Sep 12 19:15:01 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:01 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:02 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:02 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:03 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:03 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:04 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:04 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:05 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTA update ccb:1000000f0
Sep 12 19:15:05 SLES-64BIT-SLOT2 osafimmpbed: NO Slave PBE time-out in waiting
on porepare for PRTA update ccb:1000000f0
dn:safNode=PL-3,safCluster=myClmCluster
Sep 12 19:15:06 SLES-64BIT-SLOT2 osafimmnd[2332]: WA Timeout on Persistent
runtime Object Mutation, waiting on PBE
Sep 12 19:15:06 SLES-64BIT-SLOT2 osafimmnd[2332]: WA Got error on non local rt
object update err: 6
Sep 12 19:15:07 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer disconnected
610 <0, 2010f> (safAmfService)
Sep 12 19:15:07 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer (applier)
connected: 641 (@safAmfService2010f) <0, 2010f>
Sep 12 19:15:07 SLES-64BIT-SLOT2 osafamfd[2396]: NO Switching StandBy -->
Active State
Sep 12 19:15:07 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer disconnected
623 <14, 2020f> (@safAmfService2020f)
Sep 12 19:15:07 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer connected: 642
(safAmfService) <14, 2020f>
Sep 12 19:15:07 SLES-64BIT-SLOT2 osafrded[2303]: NO RDE role set to ACTIVE
Sep 12 19:15:07 SLES-64BIT-SLOT2 osafclmd[2377]: NO ACTIVE request
Sep 12 19:15:07 SLES-64BIT-SLOT2 osafamfd[2396]: NO Controller switch over done
Sep 12 19:15:12 SLES-64BIT-SLOT2 osafimmnd[2332]: WA Timeout on Persistent
runtime Object Mutation, waiting on PBE
Sep 12 19:15:12 SLES-64BIT-SLOT2 osafimmnd[2332]: WA >>s_info->to_svc == 0<<
reply context destroyed before this reply could be made
Sep 12 19:15:12 SLES-64BIT-SLOT2 osafimmnd[2332]: ER Failed to send response to
agent/client over MDS rc:2
Sep 12 19:15:14 SLES-64BIT-SLOT2 osafimmpbed: NO 2PBE Error (21) in PRTA update
(ccbId:1000000f0)
Sep 12 19:15:14 SLES-64BIT-SLOT2 osafimmnd[2332]: WA update of PERSISTENT
runtime attributes in object 'safNode=PL-3,safCluster=myClmCluster' REVERTED.
PBE rc:21
Sep 12 19:15:15 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Create of class
testMA_verifyPrimNoResponseDelCallback_101 is PERSISTENT.
Sep 12 19:15:16 SLES-64BIT-SLOT2 osafimmpbed: IN Create of class
testMA_verifyPrimNoResponseDelCallback_101 committing with ccbId:1000000ee
Sep 12 19:15:16 SLES-64BIT-SLOT2 osafimmpbed: ER pbePrepareTrans was called
when sqliteTransLock(0)!=1
Sep 12 19:15:16 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer locally
disconnected. Marking it as doomed 625 <315, 2020f> (@OpenSafImmPBE)
Sep 12 19:15:16 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer locally
disconnected. Marking it as doomed 626 <316, 2020f> (OsafImmPbeRt_B)
Sep 12 19:15:16 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer disconnected
625 <315, 2020f> (@OpenSafImmPBE)
Sep 12 19:15:16 SLES-64BIT-SLOT2 osafimmnd[2332]: NO Implementer disconnected
626 <316, 2020f> (OsafImmPbeRt_B)
Sep 12 19:15:17 SLES-64BIT-SLOT2 osafimmnd[2332]: WA SLAVE PBE process has
apparently died at non coord
Program terminated with signal 6, Aborted.
#0 0x00007fd4af31fb55 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x00007fd4af31fb55 in raise () from /lib64/libc.so.6
#1 0x00007fd4af321131 in abort () from /lib64/libc.so.6
#2 0x00000000004104f7 in pbeClosePrepareTrans() () at immpbe_dump.cc:63
#3 0x0000000000408730 in saImmOiAdminOperationCallback(unsigned long long,
unsigned long long, SaNameT const*, unsigned long long,
SaImmAdminOperationParamsT_2 const**) ()
at immpbe_daemon.cc:559
#4 0x00007fd4b012c7b8 in imma_process_callback_info () from
/usr/lib/../lib64/libSaImmOi.so.0
#5 0x00007fd4b012d196 in imma_hdl_callbk_dispatch_all () from
/usr/lib/../lib64/libSaImmOi.so.0
#6 0x00007fd4b01200c1 in saImmOiDispatch () at imma_oi_api.c:561
#7 0x000000000040dcce in pbeRtObjThread(void*) () at immpbe_daemon.cc:2103
#8 0x00007fd4afaf77b6 in start_thread () from /lib64/libpthread.so.0
#9 0x00007fd4af3c49cd in clone () from /lib64/libc.so.6
#10 0x0000000000000000 in ?? ()
(gdb) bt full
#0 0x00007fd4af31fb55 in raise () from /lib64/libc.so.6
No symbol table info available.
#1 0x00007fd4af321131 in abort () from /lib64/libc.so.6
No symbol table info available.
#2 0x00000000004104f7 in pbeClosePrepareTrans() () at immpbe_dump.cc:63
sPbeFileName = {static npos = <optimized out>, _M_dataplus =
{<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data
fields>}, <No data fields>},
_M_p = 0x676e58 "/home/sirisha/immsv/immpbe//imm.db.2020f"}}
preparedSql = {0x422700 "INSERT INTO objects_int_multi (obj_id,
attr_name, int_val) VALUES (?, ?, ?)",
0x422750 "INSERT INTO objects_real_multi (obj_id, attr_name,
real_val) VALUES (?, ?, ?)",
0x4227a0 "INSERT INTO objects_text_multi (obj_id, attr_name,
text_val) VALUES (?, ?, ?)",
0x4227f0 "INSERT INTO classes (class_id, class_name, class_category)
VALUES (?, ?, ?)",
0x422840 "INSERT INTO attr_def (class_id, attr_name, attr_type,
attr_flags) VALUES (?, ?, ?, ?)",
0x422898 "INSERT INTO attr_dflt (class_id, attr_name, int_dflt)
VALUES (?, ?, ?)",
0x4228e0 "INSERT INTO attr_dflt (class_id, attr_name, real_dflt)
VALUES (?, ?, ?)",
0x422928 "INSERT INTO attr_dflt (class_id, attr_name, text_dflt)
VALUES (?, ?, ?)",
0x422970 "INSERT INTO objects (obj_id, class_id, dn, last_ccb) VALUES
(?, ?, ?, ?)",
0x4229c0 "INSERT INTO ccb_commits (ccb_id, epoch, commit_time) VALUES
(?, ?, ?)", 0x422a08 "SELECT obj_id FROM objects WHERE class_id = ?",
0x422a38 "SELECT obj_id,class_id FROM objects WHERE dn = ?", 0x422a70
"SELECT class_name FROM classes WHERE class_id = ?",
0x422aa8 "SELECT class_id FROM classes WHERE class_name = ?",
0x422ae0 "SELECT attr_type,attr_flags FROM attr_def WHERE class_id = ? AND
attr_name = ?",
0x422b30 "SELECT epoch, commit_time FROM ccb_commits WHERE ccb_id =
?", 0x422b70 "DELETE FROM classes WHERE class_id = ?",
0x422b98 "DELETE FROM objects WHERE obj_id = ?", 0x422bc0 "DELETE
FROM attr_def WHERE class_id = ?", 0x422be8 "DELETE FROM attr_dflt WHERE
class_id = ?",
0x422c18 "DELETE FROM objects_int_multi WHERE obj_id = ?", 0x422c48
"DELETE FROM objects_real_multi WHERE obj_id = ?",
0x422c78 "DELETE FROM objects_text_multi WHERE obj_id = ?", 0x422ca8
"DELETE FROM objects_int_multi WHERE obj_id = ? AND attr_name = ?",
0x422cf0 "DELETE FROM objects_real_multi WHERE obj_id = ? AND
attr_name = ?", 0x422d38 "DELETE FROM objects_text_multi WHERE obj_id = ? AND
attr_name = ?",
0x422d80 "DELETE FROM objects_int_multi WHERE obj_id = ? AND
attr_name = ? AND int_val = ?",
0x422dd8 "DELETE FROM objects_real_multi WHERE obj_id = ? AND
attr_name = ? AND real_val = ?",
0x422e30 "DELETE FROM objects_text_multi WHERE obj_id = ? AND
attr_name = ? AND text_val = ?", 0x422e88 "DELETE FROM ccb_commits WHERE epoch
<= ?",
0x422eb8 "UPDATE objects SET last_ccb = ? WHERE obj_id = ?"}
sqliteTransLock = 0
std::__ioinit = {static _S_refcount = <optimized out>, static
_S_synced_with_stdio = <optimized out>}
preparedStmt = {0x66d2d8, 0x676e98, 0x66d3f8, 0x66d518, 0x707ab8,
0x679c68, 0x67a4a8, 0x67ace8, 0x67b528, 0x67bcc8, 0x67c4f8, 0x67cb48, 0x67d338,
0x67d988, 0x67e0e8,
0x67ea58, 0x67f1b8, 0x67f6d8, 0x67fbf8, 0x680358, 0x680ab8, 0x681218,
0x681978, 0x6820d8, 0x682878, 0x683018, 0x6837b8, 0x684068, 0x684918, 0x6851c8,
0x6858f8}
#3 0x0000000000408730 in saImmOiAdminOperationCallback(unsigned long long,
unsigned long long, SaNameT const*, unsigned long long,
SaImmAdminOperationParamsT_2 const**) ()
at immpbe_daemon.cc:559
category_mask = 0
max_waiting_time_ms = 5000
ccb_id_string = 0x41fc32 "ccbId"
rtCallbacks = {
saImmOiAdminOperationCallback = 0x407a94
<saImmOiAdminOperationCallback(unsigned long long, unsigned long long, SaNameT
const*, unsigned long long, SaImmAdminOperationParamsT_2 const**)>,
saImmOiCcbAbortCallback = 0x40ba05 <saImmOiCcbAbortCallback(unsigned long long,
unsigned long long)>,
saImmOiCcbApplyCallback = 0x40b8f9 <saImmOiCcbApplyCallback(unsigned
long long, unsigned long long)>,
saImmOiCcbCompletedCallback = 0x40b447
<saImmOiCcbCompletedCallback(unsigned long long, unsigned long long)>,
---Type <return> to continue, or q <return> to quit---
saImmOiCcbObjectCreateCallback = 0x40bb06
<saImmOiCcbObjectCreateCallback(unsigned long long, unsigned long long, char*,
SaNameT const*, SaImmAttrValuesT_2 const**)>, saImmOiCcbObjectDeleteCallback =
0x40c88b <saImmOiCcbObjectDeleteCallback(unsigned long long, unsigned long
long, SaNameT const*)>,
saImmOiCcbObjectModifyCallback = 0x40ae41
<saImmOiCcbObjectModifyCallback(unsigned long long, unsigned long long, SaNameT
const*, SaImmAttrModificationT_2 const**)>, saImmOiRtAttrUpdateCallback =
0x40cb0b <saImmOiRtAttrUpdateCallback(unsigned long long, SaNameT const*, char*
const*)>}
sOwnerHandle = 1410526778086559854
s2PbeBCcbOpCountNowAtB = 0
immOiRtSelectionObject = 18
nrtfds = 1
pbeOiRtHandle = 1357209797135
sEpoch = 35
sNoStdFlags = 22
pbeOmHandle = 21474968079
rtfds = {{fd = 18, events = 1, revents = 1}}
s2PbeBCcbOpCountToExpectAtB = 0
sClassIdMap = 0x7fff5ef5a0c0
sDbHandle = 0x702c48
sLastCcbCommitTime = 1410526899000000000
s2PbeBCcbUtilCcbData = 0x0
immOiSelectionObject = 16
sLastCcbCommit = 22
sClassCount = 80
sPbe2 = true
sleep_delay_ms = 500
sPbe2B = true
sBufsize = 256
num_ops_string = 0x41fc38 "numOps"
s2PbeBCcbToCompleteAtB = 0
immOmSelectionObject = 13
sObjCount = 50374
pbeOiHandle = 1352914829839
callbacks = {
saImmOiAdminOperationCallback = 0x407a94
<saImmOiAdminOperationCallback(unsigned long long, unsigned long long, SaNameT
const*, unsigned long long, SaImmAdminOperationParamsT_2 const**)>,
saImmOiCcbAbortCallback = 0x40ba05 <saImmOiCcbAbortCallback(unsigned long long,
unsigned long long)>,
saImmOiCcbApplyCallback = 0x40b8f9 <saImmOiCcbApplyCallback(unsigned
long long, unsigned long long)>,
saImmOiCcbCompletedCallback = 0x40b447
<saImmOiCcbCompletedCallback(unsigned long long, unsigned long long)>,
saImmOiCcbObjectCreateCallback = 0x40bb06
<saImmOiCcbObjectCreateCallback(unsigned long long, unsigned long long, char*,
SaNameT const*, SaImmAttrValuesT_2 const**)>, saImmOiCcbObjectDeleteCallback =
0x40c88b <saImmOiCcbObjectDeleteCallback(unsigned long long, unsigned long
long, SaNameT const*)>,
saImmOiCcbObjectModifyCallback = 0x40ae41
<saImmOiCcbObjectModifyCallback(unsigned long long, unsigned long long, SaNameT
const*, SaImmAttrModificationT_2 const**)>, saImmOiRtAttrUpdateCallback =
0x40cb0b <saImmOiRtAttrUpdateCallback(unsigned long long, SaNameT const*, char*
const*)>}
fds = {{fd = 16, events = 1, revents = 0}, {fd = 13, events = 1,
revents = 0}, {fd = 20, events = 1, revents = 0}}
term_sel_obj = {raise_obj = 19, rmv_obj = 20}
nfds = 3
---Type <return> to continue, or q <return> to quit---
#4 0x00007fd4b012c7b8 in imma_process_callback_info () from
/usr/lib/../lib64/libSaImmOi.so.0
immd_evt_names = {0x7fd4b0144c7c "undefined", 0x7fd4b014534d
"IMMD_EVT_MDS_INFO", 0x7fd4b014535f "IMMD_EVT_ND2D_INTRO",
0x7fd4b01449b0 "IMMD_EVT_ND2D_ANNOUNCE_LOADING", 0x7fd4b0145373
"IMMD_EVT_ND2D_REQ_SYNC", 0x7fd4b014538a "IMMD_EVT_ND2D_ANNOUNCE_DUMP",
0x7fd4b01453a6 "IMMD_EVT_ND2D_SYNC_START", 0x7fd4b01453bf
"IMMD_EVT_ND2D_SYNC_ABORT", 0x7fd4b01453d8 "IMMD_EVT_ND2D_ADMINIT_REQ",
0x7fd4b01453f2 "IMMD_EVT_ND2D_ACTIVE_SET", 0x7fd4b014540b
"IMMD_EVT_ND2D_IMM_SYNC_INFO2", 0x7fd4b0145428 "IMMD_EVT_ND2D_FEVS_REQ",
0x7fd4b014543f "IMMD_EVT_ND2D_CCBINIT_REQ", 0x7fd4b0145459
"IMMD_EVT_ND2D_IMPLSET_REQ", 0x7fd4b0145473 "IMMD_EVT_ND2D_OI_OBJ_MODIFY",
0x7fd4b014548f "IMMD_EVT_ND2D_DISCARD_IMPL", 0x7fd4b01454aa
"IMMD_EVT_ND2D_ABORT_CCB", 0x7fd4b01449d0 "IMMD_EVT_ND2D_ADMO_HARD_FINALIZE",
0x7fd4b01454c2 "IMMD_EVT_CB_DUMP", 0x7fd4b01454d3
"IMMD_EVT_MDS_QUIESCED_ACK_RSP", 0x7fd4b01454f1 "IMMD_EVT_RDA_CB",
0x7fd4b01449f8 "IMMD_EVT_ND2D_PBE_PRTO_PURGE_MUTATIONS",
0x7fd4b0145501 "IMMD_EVT_ND2D_LOADING_FAILED", 0x7fd4b014551e
"IMMD_EVT_ND2D_SYNC_FEVS_BASE",
0x7fd4b014553b "IMMD_EVT_ND2D_FEVS_REQ_2", 0x7fd4b0144a20
"IMMD_EVT_ND2D_LOADING_COMPLETED", 0x7fd4b0145554 "IMMD_EVT_ND2D_2PBE_PRELOAD",
0x7fd4b014556f "IMMD_EVT_ND2D_IMPLSET_REQ_2", 0x7fd4b014533c
"undefined (high)"}
immnd_evt_names = {0x7fd4b0144c7c "undefined", 0x7fd4b0144c86
"IMMND_EVT_MDS_INFO", 0x7fd4b0144c99 "IMMND_EVT_TIME_OUT", 0x7fd4b0144cac
"IMMND_EVT_A2ND_IMM_INIT",
0x7fd4b0144cc4 "IMMND_EVT_A2ND_IMM_OI_INIT", 0x7fd4b0144cdf
"IMMND_EVT_A2ND_IMM_FINALIZE", 0x7fd4b0144510 "IMMND_EVT_A2ND_IMM_OI_FINALIZE",
0x7fd4b0144cfb "IMMND_EVT_A2ND_IMM_ADMINIT", 0x7fd4b0144d16
"IMMND_EVT_A2ND_ADMO_FINALIZE", 0x7fd4b0144d33 "IMMND_EVT_A2ND_ADMO_SET",
0x7fd4b0144d4b "IMMND_EVT_A2ND_ADMO_RELEASE", 0x7fd4b0144d67
"IMMND_EVT_A2ND_ADMO_CLEAR", 0x7fd4b0144d81 "IMMND_EVT_A2ND_IMM_ADMOP",
0x7fd4b0144530 "IMMND_EVT_A2ND_IMM_ADMOP_ASYNC", 0x7fd4b0144d9a
"IMMND_EVT_A2ND_IMM_FEVS", 0x7fd4b0144db2 "IMMND_EVT_A2ND_CCBINIT",
0x7fd4b0144dc9 "IMMND_EVT_A2ND_SEARCHINIT", 0x7fd4b0144de3
"IMMND_EVT_A2ND_SEARCHNEXT", 0x7fd4b0144dfd "IMMND_EVT_A2ND_SEARCHFINALIZE",
0x7fd4b0144e1b "IMMND_EVT_A2ND_SEARCH_REMOTE", 0x7fd4b0144550
"IMMND_EVT_A2ND_RT_ATT_UPPD_RSP", 0x7fd4b0144e38 "IMMND_EVT_A2ND_ADMOP_RSP",
0x7fd4b0144570 "IMMND_EVT_A2ND_ASYNC_ADMOP_RSP", 0x7fd4b0144590
"IMMND_EVT_A2ND_CCB_COMPLETED_RSP", 0x7fd4b01445b8
"IMMND_EVT_A2ND_CCB_OBJ_CREATE_RSP",
0x7fd4b01445e0 "IMMND_EVT_A2ND_CCB_OBJ_MODIFY_RSP", 0x7fd4b0144608
"IMMND_EVT_A2ND_CCB_OBJ_DELETE_RSP", 0x7fd4b0144e51
"IMMND_EVT_A2ND_CLASS_CREATE",
0x7fd4b0144630 "IMMND_EVT_A2ND_CLASS_DESCR_GET", 0x7fd4b0144e6d
"IMMND_EVT_A2ND_CLASS_DELETE", 0x7fd4b0144e89 "IMMND_EVT_A2ND_OBJ_CREATE",
0x7fd4b0144ea3 "IMMND_EVT_A2ND_OBJ_MODIFY", 0x7fd4b0144ebd
"IMMND_EVT_A2ND_OBJ_DELETE", 0x7fd4b0144ed7 "IMMND_EVT_A2ND_CCB_APPLY",
0x7fd4b0144ef0 "IMMND_EVT_A2ND_CCB_FINALIZE", 0x7fd4b0144f0c
"IMMND_EVT_A2ND_OBJ_SYNC", 0x7fd4b0144f24 "IMMND_EVT_A2ND_SYNC_FINALIZE",
0x7fd4b0144f41 "IMMND_EVT_A2ND_OI_OBJ_CREATE", 0x7fd4b0144f5e
"IMMND_EVT_A2ND_OI_OBJ_MODIFY", 0x7fd4b0144f7b "IMMND_EVT_A2ND_OI_OBJ_DELETE",
0x7fd4b0144f98 "IMMND_EVT_A2ND_OI_IMPL_SET", 0x7fd4b0144fb3
"IMMND_EVT_A2ND_OI_IMPL_CLR", 0x7fd4b0144fce "IMMND_EVT_A2ND_OI_CL_IMPL_SET",
0x7fd4b0144fec "IMMND_EVT_A2ND_OI_CL_IMPL_REL", 0x7fd4b0144650
"IMMND_EVT_A2ND_OI_OBJ_IMPL_SET", 0x7fd4b0144670
"IMMND_EVT_A2ND_OI_OBJ_IMPL_REL",
0x7fd4b014500a "IMMND_EVT_ND2ND_ADMOP_RSP", 0x7fd4b0144690
"IMMND_EVT_ND2ND_ASYNC_ADMOP_RSP", 0x7fd4b0145024
"IMMND_EVT_ND2ND_SYNC_FINALIZE",
0x7fd4b0145042 "IMMND_EVT_ND2ND_SEARCH_REMOTE", 0x7fd4b01446b0
"IMMND_EVT_ND2ND_SEARCH_REMOTE_RSP", 0x7fd4b0145060 "IMMND_EVT_D2ND_INTRO_RSP",
0x7fd4b0145079 "IMMND_EVT_D2ND_SYNC_REQ", 0x7fd4b0145091
"IMMND_EVT_D2ND_LOADING_OK", 0x7fd4b01450ab "IMMND_EVT_D2ND_SYNC_START",
0x7fd4b01450c5 "IMMND_EVT_D2ND_SYNC_ABORT", 0x7fd4b01450df
"IMMND_EVT_D2ND_DUMP_OK", 0x7fd4b01450f6 "IMMND_EVT_D2ND_GLOB_FEVS_REQ",
0x7fd4b0145113 "IMMND_EVT_D2ND_ADMINIT", 0x7fd4b014512a
"IMMND_EVT_D2ND_CCBINIT", 0x7fd4b0145141 "IMMND_EVT_D2ND_IMPLSET_RSP",
0x7fd4b014515c "IMMND_EVT_D2ND_DISCARD_IMPL", 0x7fd4b0145178
"IMMND_EVT_D2ND_ABORT_CCB", 0x7fd4b01446d8 "IMMND_EVT_D2ND_ADMO_HARD_FINALIZE",
0x7fd4b0145191 "IMMND_EVT_D2ND_DISCARD_NODE", 0x7fd4b01451ad
"IMMND_EVT_D2ND_RESET", 0x7fd4b01451c2 "IMMND_EVT_CB_DUMP",
0x7fd4b0144700 "IMMND_EVT_A2ND_IMM_OM_RESURRECT", 0x7fd4b0144720
"IMMND_EVT_A2ND_IMM_OI_RESURRECT", 0x7fd4b01451d4
"IMMND_EVT_A2ND_IMM_CLIENTHIGH",
0x7fd4b0144740 "IMMND_EVT_ND2ND_SYNC_FINALIZE_2", 0x7fd4b0144760
"IMMND_EVT_A2ND_RECOVER_CCB_OUTCOME", 0x7fd4b0144788
"IMMND_EVT_A2ND_PBE_PRT_OBJ_CREATE_RSP",
0x7fd4b01447b0 "IMMND_EVT_D2ND_PBE_PRTO_PURGE_MUTATIONS",
0x7fd4b01447d8 "IMMND_EVT_A2ND_PBE_PRTO_DELETES_COMPLETED_RSP",
0x7fd4b0144808 "IMMND_EVT_A2ND_PBE_PRT_ATTR_UPDATE_RSP",
0x7fd4b0144830 "IMMND_EVT_A2ND_IMM_OM_CLIENTHIGH", 0x7fd4b0144858
"IMMND_EVT_A2ND_IMM_OI_CLIENTHIGH",
0x7fd4b01451f2 "IMMND_EVT_A2ND_PBE_ADMOP_RSP", 0x7fd4b014520f
"IMMND_EVT_D2ND_SYNC_FEVS_BASE", 0x7fd4b014522d "IMMND_EVT_A2ND_OBJ_SYNC_2",
0x7fd4b0145247 "IMMND_EVT_A2ND_IMM_FEVS_2", 0x7fd4b0144880
"IMMND_EVT_D2ND_GLOB_FEVS_REQ_2", 0x7fd4b01448a0
"IMMND_EVT_A2ND_CCB_COMPLETED_RSP_2",
0x7fd4b01448c8 "IMMND_EVT_A2ND_CCB_OBJ_CREATE_RSP_2", 0x7fd4b01448f0
"IMMND_EVT_A2ND_CCB_OBJ_MODIFY_RSP_2", 0x7fd4b0144918
"IMMND_EVT_A2ND_CCB_OBJ_DELETE_RSP_2",
0x7fd4b0145261 "IMMND_EVT_A2ND_ADMOP_RSP_2", 0x7fd4b0144940
"IMMND_EVT_A2ND_ASYNC_ADMOP_RSP_2", 0x7fd4b014527c
"IMMND_EVT_ND2ND_ADMOP_RSP_2",
0x7fd4b0144968 "IMMND_EVT_ND2ND_ASYNC_ADMOP_RSP_2", 0x7fd4b0144990
"IMMND_EVT_A2ND_OI_CCB_AUG_INIT", 0x7fd4b0145298 "IMMND_EVT_A2ND_AUG_ADMO",
---Type <return> to continue, or q <return> to quit---
0x7fd4b01452b0 "IMMND_EVT_A2ND_CL_TIMEOUT", 0x7fd4b01452ca
"IMMND_EVT_A2ND_ACCESSOR_GET", 0x7fd4b01452e6 "IMMND_EVT_A2ND_CCB_VALIDATE",
0x7fd4b0145302 "IMMND_EVT_A2ND_OI_IMPL_SET_2", 0x7fd4b014531f
"IMMND_EVT_D2ND_IMPLSET_RSP_2", 0x7fd4b014533c "undefined (high)"}
#5 0x00007fd4b012d196 in imma_hdl_callbk_dispatch_all () from
/usr/lib/../lib64/libSaImmOi.so.0
immd_evt_names = {0x7fd4b0144c7c "undefined", 0x7fd4b014534d
"IMMD_EVT_MDS_INFO", 0x7fd4b014535f "IMMD_EVT_ND2D_INTRO",
0x7fd4b01449b0 "IMMD_EVT_ND2D_ANNOUNCE_LOADING", 0x7fd4b0145373
"IMMD_EVT_ND2D_REQ_SYNC", 0x7fd4b014538a "IMMD_EVT_ND2D_ANNOUNCE_DUMP",
0x7fd4b01453a6 "IMMD_EVT_ND2D_SYNC_START", 0x7fd4b01453bf
"IMMD_EVT_ND2D_SYNC_ABORT", 0x7fd4b01453d8 "IMMD_EVT_ND2D_ADMINIT_REQ",
0x7fd4b01453f2 "IMMD_EVT_ND2D_ACTIVE_SET", 0x7fd4b014540b
"IMMD_EVT_ND2D_IMM_SYNC_INFO2", 0x7fd4b0145428 "IMMD_EVT_ND2D_FEVS_REQ",
0x7fd4b014543f "IMMD_EVT_ND2D_CCBINIT_REQ", 0x7fd4b0145459
"IMMD_EVT_ND2D_IMPLSET_REQ", 0x7fd4b0145473 "IMMD_EVT_ND2D_OI_OBJ_MODIFY",
0x7fd4b014548f "IMMD_EVT_ND2D_DISCARD_IMPL", 0x7fd4b01454aa
"IMMD_EVT_ND2D_ABORT_CCB", 0x7fd4b01449d0 "IMMD_EVT_ND2D_ADMO_HARD_FINALIZE",
0x7fd4b01454c2 "IMMD_EVT_CB_DUMP", 0x7fd4b01454d3
"IMMD_EVT_MDS_QUIESCED_ACK_RSP", 0x7fd4b01454f1 "IMMD_EVT_RDA_CB",
0x7fd4b01449f8 "IMMD_EVT_ND2D_PBE_PRTO_PURGE_MUTATIONS",
0x7fd4b0145501 "IMMD_EVT_ND2D_LOADING_FAILED", 0x7fd4b014551e
"IMMD_EVT_ND2D_SYNC_FEVS_BASE",
0x7fd4b014553b "IMMD_EVT_ND2D_FEVS_REQ_2", 0x7fd4b0144a20
"IMMD_EVT_ND2D_LOADING_COMPLETED", 0x7fd4b0145554 "IMMD_EVT_ND2D_2PBE_PRELOAD",
0x7fd4b014556f "IMMD_EVT_ND2D_IMPLSET_REQ_2", 0x7fd4b014533c
"undefined (high)"}
immnd_evt_names = {0x7fd4b0144c7c "undefined", 0x7fd4b0144c86
"IMMND_EVT_MDS_INFO", 0x7fd4b0144c99 "IMMND_EVT_TIME_OUT", 0x7fd4b0144cac
"IMMND_EVT_A2ND_IMM_INIT",
0x7fd4b0144cc4 "IMMND_EVT_A2ND_IMM_OI_INIT", 0x7fd4b0144cdf
"IMMND_EVT_A2ND_IMM_FINALIZE", 0x7fd4b0144510 "IMMND_EVT_A2ND_IMM_OI_FINALIZE",
0x7fd4b0144cfb "IMMND_EVT_A2ND_IMM_ADMINIT", 0x7fd4b0144d16
"IMMND_EVT_A2ND_ADMO_FINALIZE", 0x7fd4b0144d33 "IMMND_EVT_A2ND_ADMO_SET",
0x7fd4b0144d4b "IMMND_EVT_A2ND_ADMO_RELEASE", 0x7fd4b0144d67
"IMMND_EVT_A2ND_ADMO_CLEAR", 0x7fd4b0144d81 "IMMND_EVT_A2ND_IMM_ADMOP",
0x7fd4b0144530 "IMMND_EVT_A2ND_IMM_ADMOP_ASYNC", 0x7fd4b0144d9a
"IMMND_EVT_A2ND_IMM_FEVS", 0x7fd4b0144db2 "IMMND_EVT_A2ND_CCBINIT",
0x7fd4b0144dc9 "IMMND_EVT_A2ND_SEARCHINIT", 0x7fd4b0144de3
"IMMND_EVT_A2ND_SEARCHNEXT", 0x7fd4b0144dfd "IMMND_EVT_A2ND_SEARCHFINALIZE",
0x7fd4b0144e1b "IMMND_EVT_A2ND_SEARCH_REMOTE", 0x7fd4b0144550
"IMMND_EVT_A2ND_RT_ATT_UPPD_RSP", 0x7fd4b0144e38 "IMMND_EVT_A2ND_ADMOP_RSP",
0x7fd4b0144570 "IMMND_EVT_A2ND_ASYNC_ADMOP_RSP", 0x7fd4b0144590
"IMMND_EVT_A2ND_CCB_COMPLETED_RSP", 0x7fd4b01445b8
"IMMND_EVT_A2ND_CCB_OBJ_CREATE_RSP",
0x7fd4b01445e0 "IMMND_EVT_A2ND_CCB_OBJ_MODIFY_RSP", 0x7fd4b0144608
"IMMND_EVT_A2ND_CCB_OBJ_DELETE_RSP", 0x7fd4b0144e51
"IMMND_EVT_A2ND_CLASS_CREATE",
0x7fd4b0144630 "IMMND_EVT_A2ND_CLASS_DESCR_GET", 0x7fd4b0144e6d
"IMMND_EVT_A2ND_CLASS_DELETE", 0x7fd4b0144e89 "IMMND_EVT_A2ND_OBJ_CREATE",
0x7fd4b0144ea3 "IMMND_EVT_A2ND_OBJ_MODIFY", 0x7fd4b0144ebd
"IMMND_EVT_A2ND_OBJ_DELETE", 0x7fd4b0144ed7 "IMMND_EVT_A2ND_CCB_APPLY",
0x7fd4b0144ef0 "IMMND_EVT_A2ND_CCB_FINALIZE", 0x7fd4b0144f0c
"IMMND_EVT_A2ND_OBJ_SYNC", 0x7fd4b0144f24 "IMMND_EVT_A2ND_SYNC_FINALIZE",
0x7fd4b0144f41 "IMMND_EVT_A2ND_OI_OBJ_CREATE", 0x7fd4b0144f5e
"IMMND_EVT_A2ND_OI_OBJ_MODIFY", 0x7fd4b0144f7b "IMMND_EVT_A2ND_OI_OBJ_DELETE",
0x7fd4b0144f98 "IMMND_EVT_A2ND_OI_IMPL_SET", 0x7fd4b0144fb3
"IMMND_EVT_A2ND_OI_IMPL_CLR", 0x7fd4b0144fce "IMMND_EVT_A2ND_OI_CL_IMPL_SET",
0x7fd4b0144fec "IMMND_EVT_A2ND_OI_CL_IMPL_REL", 0x7fd4b0144650
"IMMND_EVT_A2ND_OI_OBJ_IMPL_SET", 0x7fd4b0144670
"IMMND_EVT_A2ND_OI_OBJ_IMPL_REL",
0x7fd4b014500a "IMMND_EVT_ND2ND_ADMOP_RSP", 0x7fd4b0144690
"IMMND_EVT_ND2ND_ASYNC_ADMOP_RSP", 0x7fd4b0145024
"IMMND_EVT_ND2ND_SYNC_FINALIZE",
0x7fd4b0145042 "IMMND_EVT_ND2ND_SEARCH_REMOTE", 0x7fd4b01446b0
"IMMND_EVT_ND2ND_SEARCH_REMOTE_RSP", 0x7fd4b0145060 "IMMND_EVT_D2ND_INTRO_RSP",
0x7fd4b0145079 "IMMND_EVT_D2ND_SYNC_REQ", 0x7fd4b0145091
"IMMND_EVT_D2ND_LOADING_OK", 0x7fd4b01450ab "IMMND_EVT_D2ND_SYNC_START",
0x7fd4b01450c5 "IMMND_EVT_D2ND_SYNC_ABORT", 0x7fd4b01450df
"IMMND_EVT_D2ND_DUMP_OK", 0x7fd4b01450f6 "IMMND_EVT_D2ND_GLOB_FEVS_REQ",
0x7fd4b0145113 "IMMND_EVT_D2ND_ADMINIT", 0x7fd4b014512a
"IMMND_EVT_D2ND_CCBINIT", 0x7fd4b0145141 "IMMND_EVT_D2ND_IMPLSET_RSP",
0x7fd4b014515c "IMMND_EVT_D2ND_DISCARD_IMPL", 0x7fd4b0145178
"IMMND_EVT_D2ND_ABORT_CCB", 0x7fd4b01446d8 "IMMND_EVT_D2ND_ADMO_HARD_FINALIZE",
0x7fd4b0145191 "IMMND_EVT_D2ND_DISCARD_NODE", 0x7fd4b01451ad
"IMMND_EVT_D2ND_RESET", 0x7fd4b01451c2 "IMMND_EVT_CB_DUMP",
0x7fd4b0144700 "IMMND_EVT_A2ND_IMM_OM_RESURRECT", 0x7fd4b0144720
"IMMND_EVT_A2ND_IMM_OI_RESURRECT", 0x7fd4b01451d4
"IMMND_EVT_A2ND_IMM_CLIENTHIGH",
0x7fd4b0144740 "IMMND_EVT_ND2ND_SYNC_FINALIZE_2", 0x7fd4b0144760
"IMMND_EVT_A2ND_RECOVER_CCB_OUTCOME", 0x7fd4b0144788
"IMMND_EVT_A2ND_PBE_PRT_OBJ_CREATE_RSP",
0x7fd4b01447b0 "IMMND_EVT_D2ND_PBE_PRTO_PURGE_MUTATIONS",
0x7fd4b01447d8 "IMMND_EVT_A2ND_PBE_PRTO_DELETES_COMPLETED_RSP",
0x7fd4b0144808 "IMMND_EVT_A2ND_PBE_PRT_ATTR_UPDATE_RSP",
0x7fd4b0144830 "IMMND_EVT_A2ND_IMM_OM_CLIENTHIGH", 0x7fd4b0144858
"IMMND_EVT_A2ND_IMM_OI_CLIENTHIGH",
0x7fd4b01451f2 "IMMND_EVT_A2ND_PBE_ADMOP_RSP", 0x7fd4b014520f
"IMMND_EVT_D2ND_SYNC_FEVS_BASE", 0x7fd4b014522d "IMMND_EVT_A2ND_OBJ_SYNC_2",
0x7fd4b0145247 "IMMND_EVT_A2ND_IMM_FEVS_2", 0x7fd4b0144880
"IMMND_EVT_D2ND_GLOB_FEVS_REQ_2", 0x7fd4b01448a0
"IMMND_EVT_A2ND_CCB_COMPLETED_RSP_2",
0x7fd4b01448c8 "IMMND_EVT_A2ND_CCB_OBJ_CREATE_RSP_2", 0x7fd4b01448f0
"IMMND_EVT_A2ND_CCB_OBJ_MODIFY_RSP_2", 0x7fd4b0144918
"IMMND_EVT_A2ND_CCB_OBJ_DELETE_RSP_2",
---Type <return> to continue, or q <return> to quit---
0x7fd4b0145261 "IMMND_EVT_A2ND_ADMOP_RSP_2", 0x7fd4b0144940
"IMMND_EVT_A2ND_ASYNC_ADMOP_RSP_2", 0x7fd4b014527c
"IMMND_EVT_ND2ND_ADMOP_RSP_2",
0x7fd4b0144968 "IMMND_EVT_ND2ND_ASYNC_ADMOP_RSP_2", 0x7fd4b0144990
"IMMND_EVT_A2ND_OI_CCB_AUG_INIT", 0x7fd4b0145298 "IMMND_EVT_A2ND_AUG_ADMO",
0x7fd4b01452b0 "IMMND_EVT_A2ND_CL_TIMEOUT", 0x7fd4b01452ca
"IMMND_EVT_A2ND_ACCESSOR_GET", 0x7fd4b01452e6 "IMMND_EVT_A2ND_CCB_VALIDATE",
0x7fd4b0145302 "IMMND_EVT_A2ND_OI_IMPL_SET_2", 0x7fd4b014531f
"IMMND_EVT_D2ND_IMPLSET_RSP_2", 0x7fd4b014533c "undefined (high)"}
#6 0x00007fd4b01200c1 in saImmOiDispatch () at imma_oi_api.c:561
No locals.
#7 0x000000000040dcce in pbeRtObjThread(void*) () at immpbe_daemon.cc:2103
category_mask = 0
max_waiting_time_ms = 5000
ccb_id_string = 0x41fc32 "ccbId"
rtCallbacks = {
saImmOiAdminOperationCallback = 0x407a94
<saImmOiAdminOperationCallback(unsigned long long, unsigned long long, SaNameT
const*, unsigned long long, SaImmAdminOperationParamsT_2 const**)>,
saImmOiCcbAbortCallback = 0x40ba05 <saImmOiCcbAbortCallback(unsigned long long,
unsigned long long)>,
saImmOiCcbApplyCallback = 0x40b8f9 <saImmOiCcbApplyCallback(unsigned
long long, unsigned long long)>,
saImmOiCcbCompletedCallback = 0x40b447
<saImmOiCcbCompletedCallback(unsigned long long, unsigned long long)>,
saImmOiCcbObjectCreateCallback = 0x40bb06
<saImmOiCcbObjectCreateCallback(unsigned long long, unsigned long long, char*,
SaNameT const*, SaImmAttrValuesT_2 const**)>, saImmOiCcbObjectDeleteCallback =
0x40c88b <saImmOiCcbObjectDeleteCallback(unsigned long long, unsigned long
long, SaNameT const*)>,
saImmOiCcbObjectModifyCallback = 0x40ae41
<saImmOiCcbObjectModifyCallback(unsigned long long, unsigned long long, SaNameT
const*, SaImmAttrModificationT_2 const**)>, saImmOiRtAttrUpdateCallback =
0x40cb0b <saImmOiRtAttrUpdateCallback(unsigned long long, SaNameT const*, char*
const*)>}
sOwnerHandle = 1410526778086559854
s2PbeBCcbOpCountNowAtB = 0
immOiRtSelectionObject = 18
nrtfds = 1
pbeOiRtHandle = 1357209797135
sEpoch = 35
sNoStdFlags = 22
pbeOmHandle = 21474968079
rtfds = {{fd = 18, events = 1, revents = 1}}
s2PbeBCcbOpCountToExpectAtB = 0
sClassIdMap = 0x7fff5ef5a0c0
sDbHandle = 0x702c48
sLastCcbCommitTime = 1410526899000000000
s2PbeBCcbUtilCcbData = 0x0
immOiSelectionObject = 16
sLastCcbCommit = 22
sClassCount = 80
sPbe2 = true
sleep_delay_ms = 500
sPbe2B = true
sBufsize = 256
num_ops_string = 0x41fc38 "numOps"
s2PbeBCcbToCompleteAtB = 0
---Type <return> to continue, or q <return> to quit---
immOmSelectionObject = 13
sObjCount = 50374
pbeOiHandle = 1352914829839
callbacks = {
saImmOiAdminOperationCallback = 0x407a94
<saImmOiAdminOperationCallback(unsigned long long, unsigned long long, SaNameT
const*, unsigned long long, SaImmAdminOperationParamsT_2 const**)>,
saImmOiCcbAbortCallback = 0x40ba05 <saImmOiCcbAbortCallback(unsigned long long,
unsigned long long)>,
saImmOiCcbApplyCallback = 0x40b8f9 <saImmOiCcbApplyCallback(unsigned
long long, unsigned long long)>,
saImmOiCcbCompletedCallback = 0x40b447
<saImmOiCcbCompletedCallback(unsigned long long, unsigned long long)>,
saImmOiCcbObjectCreateCallback = 0x40bb06
<saImmOiCcbObjectCreateCallback(unsigned long long, unsigned long long, char*,
SaNameT const*, SaImmAttrValuesT_2 const**)>, saImmOiCcbObjectDeleteCallback =
0x40c88b <saImmOiCcbObjectDeleteCallback(unsigned long long, unsigned long
long, SaNameT const*)>,
saImmOiCcbObjectModifyCallback = 0x40ae41
<saImmOiCcbObjectModifyCallback(unsigned long long, unsigned long long, SaNameT
const*, SaImmAttrModificationT_2 const**)>, saImmOiRtAttrUpdateCallback =
0x40cb0b <saImmOiRtAttrUpdateCallback(unsigned long long, SaNameT const*, char*
const*)>}
fds = {{fd = 16, events = 1, revents = 0}, {fd = 13, events = 1,
revents = 0}, {fd = 20, events = 1, revents = 0}}
term_sel_obj = {raise_obj = 19, rmv_obj = 20}
nfds = 3
#8 0x00007fd4afaf77b6 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#9 0x00007fd4af3c49cd in clone () from /lib64/libc.so.6
No symbol table info available.
#10 0x0000000000000000 in ?? ()
No symbol table info available.
syslog and immnd traces are attached. If required SC-1 traces are available
attached to #1075
---
Sent from sourceforge.net because [email protected] is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.------------------------------------------------------------------------------
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk
_______________________________________________
Opensaf-tickets mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
