- **status**: review --> fixed
- **Comment**:
changeset: 6043:fa8f3226802f
tag: tip
user: Anders Bjornerstedt <[email protected]>
date: Fri Oct 10 16:07:50 2014 +0200
summary: IMM (PBE): Return NO_RESOURCES to avoid "validation error" from
PBE [#1057]
changeset: 6042:979244deb178
user: Anders Bjornerstedt <[email protected]>
date: Fri Oct 10 15:35:26 2014 +0200
summary: IMM (2PBE): Slave PBE regenerates file when restarted soon after
PRT error [#1057]
changeset: 6041:92d30d64f767
parent: 6034:52005f352062
user: Anders Bjornerstedt <[email protected]>
date: Fri Oct 10 14:23:34 2014 +0200
summary: IMM (2PBE): Fix incorrect transaction abort in slave PBE [#1057]
changeset: 6040:a7766e9cb310
branch: opensaf-4.5.x
user: Anders Bjornerstedt <[email protected]>
date: Fri Oct 10 16:07:50 2014 +0200
summary: IMM (PBE): Return NO_RESOURCES to avoid "validation error" from
PBE [#1057]
changeset: 6039:99c1079bd800
branch: opensaf-4.5.x
user: Anders Bjornerstedt <[email protected]>
date: Fri Oct 10 15:35:26 2014 +0200
summary: IMM (2PBE): Slave PBE regenerates file when restarted soon after
PRT error [#1057]
changeset: 6038:ee89bd106cf1
branch: opensaf-4.5.x
parent: 6033:050627d2024c
user: Anders Bjornerstedt <[email protected]>
date: Fri Oct 10 14:23:34 2014 +0200
summary: IMM (2PBE): Fix incorrect transaction abort in slave PBE [#1057]
changeset: 6037:fd8d91deec76
branch: opensaf-4.4.x
user: Anders Bjornerstedt <[email protected]>
date: Tue Oct 21 00:48:55 2014 +0200
summary: IMM (PBE): Return NO_RESOURCES to avoid "validation error" from
PBE [#1057]
changeset: 6036:506f5b68298d
branch: opensaf-4.4.x
user: Anders Bjornerstedt <[email protected]>
date: Fri Oct 10 15:35:26 2014 +0200
summary: IMM (2PBE): Slave PBE regenerates file when restarted soon after
PRT error [#1057]
changeset: 6035:dcba0f49afa4
branch: opensaf-4.4.x
parent: 6032:533f7a91f12e
user: Anders Bjornerstedt <[email protected]>
date: Tue Oct 21 00:30:24 2014 +0200
summary: IMM (2PBE): Fix incorrect transaction abort in slave PBE [#1057]
---
** [tickets:#1057] (2PBE) Slave PBE restarts multiple times**
**Status:** fixed
**Milestone:** 4.4.2
**Created:** Wed Sep 10, 2014 07:00 AM UTC by Sirisha Alla
**Last Updated:** Fri Oct 10, 2014 02:58 PM UTC
**Owner:** Anders Bjornerstedt
The issue is seen on SLES X86. Opensaf is up with changeset 5697 and 2pbe is
loaded with 50k objects.
When CCBs are in progress, prto object creation is attempted. Following is the
syslog of SLOT2 when slave pbe restarted
Sep 9 15:55:37 SLES-64BIT-SLOT2 osafimmnd[6344]: NO Ccb 644 COMMITTED (exowner)
Sep 9 15:55:37 SLES-64BIT-SLOT2 osafimmpbed: IN ccb-prepare received at PBE
slave ccbId:1000005a5/4294968741 numOps:1
Sep 9 15:55:37 SLES-64BIT-SLOT2 osafimmpbed: NO Prepare
ccb:1000005a5/4294968741 received at Pbe slave when Prior Ccb 644 still
processing
Sep 9 15:55:37 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005a5
Sep 9 15:55:38 SLES-64BIT-SLOT2 osafimmpbed: IN ccb-prepare received at PBE
slave ccbId:1000005a5/4294968741 numOps:1
Sep 9 15:55:38 SLES-64BIT-SLOT2 osafimmpbed: NO Prepare
ccb:1000005a5/4294968741 received at Pbe slave when Prior Ccb 644 still
processing
Sep 9 15:55:38 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005a5
Sep 9 15:55:38 SLES-64BIT-SLOT2 osafimmpbed: IN ccb-prepare received at PBE
slave ccbId:1000005a5/4294968741 numOps:1
Sep 9 15:55:38 SLES-64BIT-SLOT2 osafimmpbed: NO Prepare
ccb:1000005a5/4294968741 received at Pbe slave when Prior Ccb 644 still
processing
Sep 9 15:55:38 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005a5
Sep 9 15:55:39 SLES-64BIT-SLOT2 osafimmpbed: IN ccb-prepare received at PBE
slave ccbId:1000005a5/4294968741 numOps:1
Sep 9 15:55:39 SLES-64BIT-SLOT2 osafimmpbed: NO Prepare
ccb:1000005a5/4294968741 received at Pbe slave when Prior Ccb 644 still
processing
Sep 9 15:55:39 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005a5
Sep 9 15:55:39 SLES-64BIT-SLOT2 osafimmpbed: IN ccb-prepare received at PBE
slave ccbId:1000005a5/4294968741 numOps:1
Sep 9 15:55:39 SLES-64BIT-SLOT2 osafimmpbed: NO Prepare
ccb:1000005a5/4294968741 received at Pbe slave when Prior Ccb 644 still
processing
Sep 9 15:55:39 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005a5
Sep 9 15:55:40 SLES-64BIT-SLOT2 osafimmpbed: IN ccb-prepare received at PBE
slave ccbId:1000005a5/4294968741 numOps:1
Sep 9 15:55:40 SLES-64BIT-SLOT2 osafimmpbed: NO Prepare
ccb:1000005a5/4294968741 received at Pbe slave when Prior Ccb 644 still
processing
Sep 9 15:55:40 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005a5
Sep 9 15:55:40 SLES-64BIT-SLOT2 osafimmnd[6344]: WA Create of PERSISTENT
runtime object 'DistObj3=DistRunTime,DistObj1=DistRunTime' REVERTED. PBE rc:20
Sep 9 15:55:40 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005a5
Sep 9 15:55:41 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005a5
Sep 9 15:55:41 SLES-64BIT-SLOT2 osafimmnd[6344]: NO Implementer disconnected
579 <0, 2040f> (implementertestMA_verifyObjImplReleaseModCallbackNode_101_133)
Sep 9 15:55:41 SLES-64BIT-SLOT2 osafimmnd[6344]: NO Implementer disconnected
581 <0, 2040f> (@applier1testMA_verifyObjImplReleaseModCallbackNode_101_133)
Sep 9 15:55:41 SLES-64BIT-SLOT2 osafimmpbed: IN Delaying class delete at slave
PBE due to ongoing commit of ccb:284/644
Sep 9 15:55:41 SLES-64BIT-SLOT2 osafimmpbed: NO Slave PBE time-out in waiting
on porepare for PRTO create ccb:1000005a5
dn:DistObj3=DistRunTime,DistObj1=DistRunTime
Sep 9 15:55:41 SLES-64BIT-SLOT2 osafimmpbed: NO 2PBE Error (21) in PRTO create
(ccbId:1000005a5)
Sep 9 15:55:41 SLES-64BIT-SLOT2 osafimmpbed: ER PBE-B got completed callback
for Ccb:284/644 before prepare from PBE-A
Sep 9 15:55:41 SLES-64BIT-SLOT2 osafimmpbed: WA PBE slave exiting in prepare
for ccb 284/644, file should be regenerated.
Sep 9 15:55:42 SLES-64BIT-SLOT2 osafimmnd[6344]: WA SLAVE PBE process has
apparently died at non coord
Sep 9 15:55:42 SLES-64BIT-SLOT2 osafimmnd[6344]: NO Delete of class
testMA_verifyObjImplReleaseModCallbackNode_101_133 is PERSISTENT.
Sep 9 15:55:42 SLES-64BIT-SLOT2 osafimmnd[6344]: NO Implementer disconnected
580 <0, 2030f> (@applier2testMA_verifyObjImplReleaseModCallbackNode_101_133)
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmnd[6344]: NO STARTING SLAVE PBE process.
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmnd[6344]: NO
pbe-db-file-path:/home/sirisha/immsv/immpbe//imm.db.2020f VETERAN:1 B:1
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmpbed: logtrace: trace enabled to file
/var/log/opensaf/osafimmnd, mask=0xffffffff
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmpbed: IN arg[0] ==
'/usr/lib64/opensaf/osafimmpbed'
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmpbed: IN arg[1] == '--recover'
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmpbed: IN arg[2] == '--pbe2B'
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmpbed: IN arg[3] ==
'/home/sirisha/immsv/immpbe//imm.db.2020f'
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmpbed: NO Successfully opened
pre-existing sqlite pbe file /home/sirisha/immsv/immpbe//imm.db.2020f
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmpbed: IN saImmRepositoryInit:
SA_IMM_KEEP_REPOSITORY - attaching to repository
Sep 9 15:55:43 SLES-64BIT-SLOT2 osafimmpbed: NO pbeDaemon starting with
obj-count:50477
Sep 9 15:56:31 SLES-64BIT-SLOT2 osafimmnd[6344]: WA Create of PERSISTENT
runtime object 'DistObj3=DistRunTime,DistObj1=DistRunTime' REVERTED. PBE rc:20
Sep 9 15:56:31 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005be
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmpbed: IN PBE slave waiting for prepare
from primary on PRTO create ccb:1000005be
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmnd[6344]: NO implementer for class
'testMA_verifyObjApplRejDelCallbackNode_101_133' is released => class extent is
UNSAFE
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmpbed: NO Slave PBE time-out in waiting
on porepare for PRTO create ccb:1000005be
dn:DistObj3=DistRunTime,DistObj1=DistRunTime
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmpbed: NO 2PBE Error (21) in PRTO create
(ccbId:1000005be)
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmpbed: ER PBE-B got completed callback
for Ccb:28a/650 before prepare from PBE-A
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmpbed: WA PBE slave exiting in prepare
for ccb 28a/650, file should be regenerated.
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmnd[6344]: NO Implementer locally
disconnected. Marking it as doomed 583 <316, 2020f> (@OpenSafImmPBE)
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmnd[6344]: NO Implementer locally
disconnected. Marking it as doomed 584 <317, 2020f> (OsafImmPbeRt_B)
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmnd[6344]: NO Precheck of fevs message
of type <0> failed with ERROR:9
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmnd[6344]: ER Asyncronous FEVS message
failed verification - dropping message!
Sep 9 15:56:32 SLES-64BIT-SLOT2 osafimmnd[6344]: WA Error code 2 returned for
message type 14 - ignoring
After this slave pbe continuously restarted leading to failures in the
application.
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmnd[6344]: NO
pbe-db-file-path:/home/sirisha/immsv/immpbe//imm.db.2020f VETERAN:1 B:1
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmpbed: logtrace: trace enabled to file
/var/log/opensaf/osafimmnd, mask=0xffffffff
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmpbed: IN arg[0] ==
'/usr/lib64/opensaf/osafimmpbed'
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmpbed: IN arg[1] == '--recover'
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmpbed: IN arg[2] == '--pbe2B'
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmpbed: IN arg[3] ==
'/home/sirisha/immsv/immpbe//imm.db.2020f'
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmpbed: NO Successfully opened
pre-existing sqlite pbe file /home/sirisha/immsv/immpbe//imm.db.2020f
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmpbed: IN saImmRepositoryInit:
SA_IMM_KEEP_REPOSITORY - attaching to repository
Sep 9 15:56:33 SLES-64BIT-SLOT2 osafimmpbed: NO pbeDaemon starting with
obj-count:50481
Sep 9 15:56:38 SLES-64BIT-SLOT2 osafimmpbed: ER saImmOiImplementerSet for
@OpenSafImmPBE failed 6
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmnd[6344]: WA SLAVE PBE process has
apparently died at non coord
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmnd[6344]: NO STARTING SLAVE PBE process.
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmnd[6344]: NO
pbe-db-file-path:/home/sirisha/immsv/immpbe//imm.db.2020f VETERAN:1 B:0
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmpbed: logtrace: trace enabled to file
/var/log/opensaf/osafimmnd, mask=0xffffffff
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmpbed: IN arg[0] ==
'/usr/lib64/opensaf/osafimmpbed'
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmpbed: IN arg[1] == '--pbe2B'
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmpbed: IN arg[2] ==
'/home/sirisha/immsv/immpbe//imm.db.2020f'
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmpbed: IN Generating DB file from
current IMM state. DB file: /home/sirisha/immsv/immpbe//imm.db.2020f
Sep 9 15:56:39 SLES-64BIT-SLOT2 osafimmpbed: NO Successfully opened empty
local sqlite pbe file /tmp/imm.db.4zX3j7
Sep 9 15:56:40 SLES-64BIT-SLOT2 osafimmnd[6344]: WA ERR_NO_RESOURCES: Active
Ccbs still exist in the system
Sep 9 15:56:40 SLES-64BIT-SLOT2 osafimmpbed: ER Failed on
saImmOmSearchInitialize:18 - exiting
Sep 9 15:56:40 SLES-64BIT-SLOT2 osafimmpbed: ER immPbe.cc exiting (line:264)
Slave PBE successfully came up once the application is stopped. Syslog and
immnd traces attached.
---
Sent from sourceforge.net because [email protected] is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.
------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
Opensaf-tickets mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
