- **Milestone**: 5.19.01 --> 5.19.03
---
** [tickets:#2984] dtm: dtm close socket due to duplication of add node IP
info**
**Status:** review
**Milestone:** 5.19.03
**Created:** Thu Dec 13, 2018 05:05 AM UTC by Canh Truong
**Last Updated:** Wed Jan 09, 2019 09:23 PM UTC
**Owner:** Canh Truong
The situation of issue:
1/ Stanby SC2 is restart. When SC2 is up, SC2's dtm send dtm node discovery
msg broadcast to all nodes (dtmscb->multicast->Send(); ) . PL4 receive this
msg and start to connect to create communication sock with SC2 by
(dtm_process_connect()).
2/ SC2 listen and accept this connection then SC2's dtm store PL4 node IP
info to database (dtm_node_add()) and send the SC2 node info to PL4.
3/ PL4 get it and send PL4 info to SC2. (dtm_process_node_info()).
4/ SC2 get info from SC4 was sending. and add PL4 node ID info to database.
The time between (2) and (4), SC2 also receive msg that PL4 send dtm node
discovery msg broadcast to:
*while (1) {
if (dtmscb->contbcastint) {
mNCSTASKSLEEP(dtmscb->contbcastint);
/ periodically send a broadcast /
dtmscb->multicast->Send();
} else {
for (;;) pause();
}
}*
SC2 will also process the connection to connect to PL4. And SC2 add node IP
info of PL4 again to database. This is reject and SC2 close the comunication
socket. This cause PL4 mds receive 0 bytes and node PL4 reboot.
--------------------------------------------------------------------------------------------------------------------------------
SC2 dtm trace:
*143>1 2018-12-07T13:32:09.9422+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="302"] 155:dtm/dtmnd/dtmnodesockets.cc:522 >> dtmprocessconnect
<143>1 2018-12-07T13:32:09.942207+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="303"] 155:dtm/dtmnd/dtmnodesockets.cc:533 TR DTM: received the self
nodeid bcast message, dropping message clusterid: 1 nodeid: 131599
<143>1 2018-12-07T13:32:09.94221+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="304"] 155:dtm/dtmnd/dtmnodesockets.cc:539 << dtmprocessconnect
<143>1 2018-12-07T13:32:09.942213+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="305"] 155:dtm/dtmnd/multicast.cc:822 >> Receive
<143>1 2018-12-07T13:32:09.942239+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="306"] 155:dtm/dtmnd/multicast.cc:832 << Receive: rc :-1
<143>1 2018-12-07T13:32:09.942543+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="307"] 155:dtm/dtmnd/dtmnode.cc:418 TR DTM :Listening socket is
readable
<143>1 2018-12-07T13:32:09.942556+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="308"] 155:dtm/dtmnd/dtmnodesockets.cc:670 >> dtmprocessaccept
<143>1 2018-12-07T13:32:09.942575+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="309"] 155:dtm/dtmnd/dtmnodesockets.cc:73 >> setkeepalive
<143>1 2018-12-07T13:32:09.942582+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="310"] 155:dtm/dtmnd/dtmnodesockets.cc:143 << setkeepalive: rc :1
<143>1 2018-12-07T13:32:09.942587+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="311"] 155:dtm/dtmnd/dtmnodedb.cc:38 >> dtmnodenew
<143>1 2018-12-07T13:32:09.942629+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="312"] 155:dtm/dtmnd/dtmnodedb.cc:45 << dtmnodenew
<143>1 2018-12-07T13:32:09.942633+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="313"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.942636+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="314"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 2
<143>1 2018-12-07T13:32:09.942639+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="315"] 155:dtm/dtmnd/dtmnodedb.cc:219 TR DTM:Adding nodeip to the
database with nodeip :10.0.3.104 as key
<143>1 2018-12-07T13:32:09.942643+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="316"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 1
<143>1 2018-12-07T13:32:09.942646+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="317"] 155:dtm/dtmnd/dtmnodesockets.cc:760 << dtmprocessaccept: DTM:
newconnsd :31
<143>1 2018-12-07T13:32:09.942653+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="318"] 155:dtm/dtmnd/dtmnodesockets.cc:208 >> dtmcommsocketsend
<143>1 2018-12-07T13:32:09.942695+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="319"] 155:dtm/dtmnd/dtmnodesockets.cc:218 << dtmcommsocketsend: rc
:1
<143>1 2018-12-07T13:32:09.942712+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="320"] 155:dtm/dtmnd/dtmnode.cc:508 TR DTM: add New incoming
connection to fd: 31
<143>1 2018-12-07T13:32:09.942719+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="321"] 155:dtm/dtmnd/dtmnodesockets.cc:670 >> dtmprocessaccept
<143>1 2018-12-07T13:32:09.942741+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="322"] 155:dtm/dtmnd/dtmnodesockets.cc:73 >> setkeepalive
<143>1 2018-12-07T13:32:09.942747+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="323"] 155:dtm/dtmnd/dtmnodesockets.cc:143 << setkeepalive: rc :1
<143>1 2018-12-07T13:32:09.942751+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="324"] 155:dtm/dtmnd/dtmnodedb.cc:38 >> dtmnodenew
<143>1 2018-12-07T13:32:09.942792+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="325"] 155:dtm/dtmnd/dtmnodedb.cc:45 << dtmnodenew
<143>1 2018-12-07T13:32:09.942796+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="326"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.942799+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="327"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 2
<143>1 2018-12-07T13:32:09.942802+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="328"] 155:dtm/dtmnd/dtmnodedb.cc:219 TR DTM:Adding nodeip to the
database with nodeip :10.0.3.101 as key
<143>1 2018-12-07T13:32:09.942806+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="329"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 1
<143>1 2018-12-07T13:32:09.942809+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="330"] 155:dtm/dtmnd/dtmnodesockets.cc:760 << dtmprocessaccept: DTM:
newconnsd :32
<143>1 2018-12-07T13:32:09.942812+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="331"] 155:dtm/dtmnd/dtmnodesockets.cc:208 >> dtmcommsocketsend
<143>1 2018-12-07T13:32:09.94284+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="332"] 155:dtm/dtmnd/dtmnodesockets.cc:218 << dtmcommsocketsend: rc
:1
<143>1 2018-12-07T13:32:09.942845+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="333"] 155:dtm/dtmnd/dtmnode.cc:508 TR DTM: add New incoming
connection to fd: 32
<143>1 2018-12-07T13:32:09.942851+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="334"] 155:dtm/dtmnd/dtmnodesockets.cc:670 >> dtmprocessaccept
<143>1 2018-12-07T13:32:09.942862+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="335"] 155:dtm/dtmnd/dtmnodesockets.cc:73 >> setkeepalive
<143>1 2018-12-07T13:32:09.942869+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="336"] 155:dtm/dtmnd/dtmnodesockets.cc:143 << setkeepalive: rc :1
<143>1 2018-12-07T13:32:09.942872+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="337"] 155:dtm/dtmnd/dtmnodedb.cc:38 >> dtmnodenew
<143>1 2018-12-07T13:32:09.942928+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="338"] 155:dtm/dtmnd/dtmnodedb.cc:45 << dtmnodenew
<143>1 2018-12-07T13:32:09.942957+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="339"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.942961+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="340"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 2
<143>1 2018-12-07T13:32:09.942964+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="341"] 155:dtm/dtmnd/dtmnodedb.cc:219 TR DTM:Adding nodeip to the
database with nodeip :10.0.3.105 as key
<143>1 2018-12-07T13:32:09.942968+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="342"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 1
<143>1 2018-12-07T13:32:09.942971+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="343"] 155:dtm/dtmnd/dtmnodesockets.cc:760 << dtmprocessaccept: DTM:
newconnsd :33
<143>1 2018-12-07T13:32:09.942975+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="344"] 155:dtm/dtmnd/dtmnodesockets.cc:208 >> dtmcommsocketsend
<143>1 2018-12-07T13:32:09.94301+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="345"] 155:dtm/dtmnd/dtmnodesockets.cc:218 << dtmcommsocketsend: rc
:1
<143>1 2018-12-07T13:32:09.943016+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="346"] 155:dtm/dtmnd/dtmnode.cc:508 TR DTM: add New incoming
connection to fd: 33
<143>1 2018-12-07T13:32:09.943021+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="347"] 155:dtm/dtmnd/dtmnodesockets.cc:670 >> dtmprocessaccept
<143>1 2018-12-07T13:32:09.943035+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="348"] 155:dtm/dtmnd/dtmnodesockets.cc:0 << dtmprocessaccept
<143>1 2018-12-07T13:32:09.943041+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="349"] 155:dtm/dtmnd/multicast.cc:822 >> Receive
<143>1 2018-12-07T13:32:09.943047+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="350"] 155:dtm/dtmnd/multicast.cc:832 << Receive: rc :60
<143>1 2018-12-07T13:32:09.943054+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="351"] 155:dtm/dtmnd/dtmnodesockets.cc:522 >> dtmprocessconnect
<143>1 2018-12-07T13:32:09.94306+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="352"] 155:dtm/dtmnd/dtmnodesockets.cc:553 TR mcast flag: 0
<143>1 2018-12-07T13:32:09.943064+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="353"] 155:dtm/dtmnd/dtmnodedb.cc:135 >> dtmnodegetbyid
<143>1 2018-12-07T13:32:09.943067+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="354"] 155:dtm/dtmnd/dtmnodedb.cc:146 << dtmnodegetbyid
<143>1 2018-12-07T13:32:09.94307+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="355"] 155:dtm/dtmnd/dtmnodedb.cc:38 >> dtmnodenew
<143>1 2018-12-07T13:32:09.943137+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="356"] 155:dtm/dtmnd/dtmnodedb.cc:45 << dtmnodenew
<143>1 2018-12-07T13:32:09.943143+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="357"] 155:dtm/dtmnd/dtmnodesockets.cc:242 >> commsocketsetupnew
<143>1 2018-12-07T13:32:09.943149+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="358"] 155:dtm/dtmnd/dtmnodesockets.cc:245 TR DTM:dgramportrcvr :6900
<143>1 2018-12-07T13:32:09.943158+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="359"] 155:dtm/dtmnd/dtmnodesockets.cc:269 TR DTM:foreignaddress :
10.0.3.104 localportstr :6700
<143>1 2018-12-07T13:32:09.943165+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="360"] 155:dtm/dtmnd/dtmnodesockets.cc:289 TR DTM : family : 2,
socktype : 1, protocol :6
<143>1 2018-12-07T13:32:09.943179+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="361"] 155:dtm/dtmnd/dtmnodesockets.cc:73 >> setkeepalive
<143>1 2018-12-07T13:32:09.94319+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="362"] 155:dtm/dtmnd/dtmnodesockets.cc:143 << setkeepalive: rc :1
<143>1 2018-12-07T13:32:09.943288+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="363"] 155:dtm/dtmnd/dtmnodesockets.cc:344 << commsocketsetupnew:
sockdesc : 34
<143>1 2018-12-07T13:32:09.943298+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="364"] 155:dtm/dtmnd/dtmnodesockets.cc:623 TR DTM: dtmnodeadd
.nodeip: 10.0.3.104 nodeid: 132111, commsocket 34
<143>1 2018-12-07T13:32:09.943308+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="365"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.943314+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="366"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 0
<143>1 2018-12-07T13:32:09.94332+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="367"] 155:dtm/dtmnd/dtmnodedb.cc:203 TR DTM:Adding nodeid to the
database with nodeid :132111 as key
<143>1 2018-12-07T13:32:09.943326+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="368"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 1
<143>1 2018-12-07T13:32:09.943331+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="369"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.943339+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="370"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 2
<143>1 2018-12-07T13:32:09.943346+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="371"] 155:dtm/dtmnd/dtmnodedb.cc:219 TR DTM:Adding nodeip to the
database with nodeip :10.0.3.104 as key
<143>1 2018-12-07T13:32:09.943351+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="372"] 155:dtm/dtmnd/dtmnodedb.cc:225 TR DTM:ncspatriciatreeadd for
nodeip FAILED for :10.0.3.104 :2
<143>1 2018-12-07T13:32:09.943357+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="373"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 2
<139>1 2018-12-07T13:32:09.943408+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="374"] 155:dtm/dtmnd/dtmnodesockets.cc:634 ER DTM: dtmnodeadd failed
.nodeip: 10.0.3.104, nodeid: 132111
<143>1 2018-12-07T13:32:09.943415+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="375"] 155:dtm/dtmnd/dtmnodesockets.cc:157 >> dtmcommsocketclose
<143>1 2018-12-07T13:32:09.94342+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="376"] 155:dtm/dtmnd/dtmnodesockets.cc:161 TR DTM: node deleting
enty
<143>1 2018-12-07T13:32:09.943425+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="377"] 155:dtm/dtmnd/dtmnodedb.cc:254 >> dtmnodedelete: DTM:value of
i 0
<143>1 2018-12-07T13:32:09.94344+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="378"] 155:dtm/dtmnd/dtmnodedb.cc:262 TR DTM:Deleting nodeid from
the database with nodeid :132111 as key
<143>1 2018-12-07T13:32:09.943446+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="379"] 155:dtm/dtmnd/dtmnodedb.cc:298 << dtmnodedelete
<143>1 2018-12-07T13:32:09.943452+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="380"] 155:dtm/dtmnd/dtmnodedb.cc:254 >> dtmnodedelete: DTM:value of
i 2
<143>1 2018-12-07T13:32:09.943457+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="381"] 155:dtm/dtmnd/dtmnodedb.cc:298 << dtmnodedelete
<143>1 2018-12-07T13:32:09.943495+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="382"] 155:dtm/dtmnd/dtmnodesockets.cc:194 << dtmcommsocketclose
<143>1 2018-12-07T13:32:09.943502+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="383"] 155:dtm/dtmnd/dtmnodesockets.cc:644 << dtmprocessconnect:
sockdesc :-1
<142>1 2018-12-07T13:32:09.94351+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="384"] 155:dtm/dtmnd/dtmnode.cc:484 IN DTM: add New incoming
connection to fd : 34
<139>1 2018-12-07T13:32:09.943527+01:00 SC-2 osafdtmd 150 osafdtmd [meta
sequenceId="385"] 155:dtm/dtmnd/dtmnode.cc:548 ER DTM: epollctl(6, EPOLLCTLADD,
34) failed: 9*
--------------------------------------------------------------------------------------------------------------------------------
PL4 dtm trace:
*
<143>1 2018-12-07T13:32:09.942314+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33579"] 155:dtm/dtmnd/dtmnodesockets.cc:522 >> dtmprocessconnect
<143>1 2018-12-07T13:32:09.942331+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33580"] 155:dtm/dtmnd/dtmnodesockets.cc:553 TR mcast flag: 0
<143>1 2018-12-07T13:32:09.942336+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33581"] 155:dtm/dtmnd/dtmnodedb.cc:135 >> dtmnodegetbyid
<143>1 2018-12-07T13:32:09.94234+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33582"] 155:dtm/dtmnd/dtmnodedb.cc:146 << dtmnodegetbyid
<143>1 2018-12-07T13:32:09.942343+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33583"] 155:dtm/dtmnd/dtmnodedb.cc:38 >> dtmnodenew
<143>1 2018-12-07T13:32:09.942361+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33584"] 155:dtm/dtmnd/dtmnodedb.cc:45 << dtmnodenew
<143>1 2018-12-07T13:32:09.942365+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33585"] 155:dtm/dtmnd/dtmnodesockets.cc:242 >> commsocketsetupnew
<143>1 2018-12-07T13:32:09.942368+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33586"] 155:dtm/dtmnd/dtmnodesockets.cc:245 TR DTM:dgramportrcvr
:6900
<143>1 2018-12-07T13:32:09.942376+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33587"] 155:dtm/dtmnd/dtmnodesockets.cc:269 TR DTM:foreignaddress :
10.0.3.102 localportstr :6700
<143>1 2018-12-07T13:32:09.94238+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33588"] 155:dtm/dtmnd/dtmnodesockets.cc:289 TR DTM : family : 2,
socktype : 1, protocol :6
<143>1 2018-12-07T13:32:09.942392+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33589"] 155:dtm/dtmnd/dtmnodesockets.cc:73 >> setkeepalive
<143>1 2018-12-07T13:32:09.942399+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33590"] 155:dtm/dtmnd/dtmnodesockets.cc:143 << setkeepalive: rc :1
<143>1 2018-12-07T13:32:09.94254+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33591"] 155:dtm/dtmnd/dtmnodesockets.cc:344 << commsocketsetupnew:
sockdesc : 22
<143>1 2018-12-07T13:32:09.942548+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33592"] 155:dtm/dtmnd/dtmnodesockets.cc:623 TR DTM: dtmnodeadd
.nodeip: 10.0.3.102 nodeid: 131599, commsocket 22
<143>1 2018-12-07T13:32:09.942552+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33593"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.942555+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33594"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 0
<143>1 2018-12-07T13:32:09.942558+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33595"] 155:dtm/dtmnd/dtmnodedb.cc:203 TR DTM:Adding nodeid to the
database with nodeid :131599 as key
<143>1 2018-12-07T13:32:09.942561+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33596"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 1
<143>1 2018-12-07T13:32:09.942565+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33597"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.942567+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33598"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 2
<143>1 2018-12-07T13:32:09.942571+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33599"] 155:dtm/dtmnd/dtmnodedb.cc:219 TR DTM:Adding nodeip to the
database with nodeip :10.0.3.102 as key
<143>1 2018-12-07T13:32:09.942574+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33600"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 1
<143>1 2018-12-07T13:32:09.942577+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33601"] 155:dtm/dtmnd/dtmnodesockets.cc:640 TR DTM: dtmnodeadd add
.nodeip: 10.0.3.102, nodeid: 131599
<143>1 2018-12-07T13:32:09.94258+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33602"] 155:dtm/dtmnd/dtmnodesockets.cc:644 << dtmprocessconnect:
sockdesc :22
<142>1 2018-12-07T13:32:09.942585+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33603"] 155:dtm/dtmnd/dtmnode.cc:484 IN DTM: add New incoming
connection to fd : 22
<143>1 2018-12-07T13:32:09.942592+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33604"] 155:dtm/dtmnd/multicast.cc:822 >> Receive
<143>1 2018-12-07T13:32:09.942597+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33605"] 155:dtm/dtmnd/multicast.cc:832 << Receive: rc :-1
<143>1 2018-12-07T13:32:09.942608+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33606"] 150:dtm/dtmnd/multicast.cc:789 >> Send
<143>1 2018-12-07T13:32:09.942667+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33607"] 155:dtm/dtmnd/multicast.cc:822 >> Receive
<143>1 2018-12-07T13:32:09.942674+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33608"] 155:dtm/dtmnd/multicast.cc:832 << Receive: rc :60
<143>1 2018-12-07T13:32:09.942681+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33609"] 155:dtm/dtmnd/dtmnodesockets.cc:522 >> dtmprocessconnect
<143>1 2018-12-07T13:32:09.942687+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33610"] 155:dtm/dtmnd/dtmnodesockets.cc:533 TR DTM: received the
self nodeid bcast message, dropping message clusterid: 1 nodeid: 132111
<143>1 2018-12-07T13:32:09.94269+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33611"] 155:dtm/dtmnd/dtmnodesockets.cc:539 << dtmprocessconnect
<143>1 2018-12-07T13:32:09.942693+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33612"] 155:dtm/dtmnd/multicast.cc:822 >> Receive
<143>1 2018-12-07T13:32:09.942697+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33613"] 155:dtm/dtmnd/multicast.cc:832 << Receive: rc :-1
<143>1 2018-12-07T13:32:09.942713+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33614"] 155:dtm/dtmnd/dtmnode.cc:294 >>
dtminternodeprocesspollrcvmsg
<143>1 2018-12-07T13:32:09.942731+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33615"] 155:dtm/dtmnd/dtmnode.cc:101 >> dtmprocessnodeinfo
<143>1 2018-12-07T13:32:09.942735+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33616"] 155:dtm/dtmnd/dtmnodesockets.cc:208 >> dtmcommsocketsend
<143>1 2018-12-07T13:32:09.942762+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33617"] 155:dtm/dtmnd/dtmnodesockets.cc:218 << dtmcommsocketsend:
rc :1
<143>1 2018-12-07T13:32:09.942766+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33618"] 155:dtm/dtmnd/dtmnode.cc:158 TR DTM: dtmprocessnodeinfo
nodename: 'SC-2' nodeip:10.0.3.102, nodeid:131599 iaddrfamily:2
<143>1 2018-12-07T13:32:09.94277+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33619"] 155:dtm/dtmnd/dtmnode.cc:191 TR DTM: dtmprocessnodeupdown
nodeip:10.0.3.102, nodeid:131599 iaddrfamily:2
<143>1 2018-12-07T13:32:09.942773+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33620"] 155:dtm/dtmnd/dtmintersvc.cc:127 >> dtmnodeup
<141>1 2018-12-07T13:32:09.942797+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33621"] 155:dtm/dtmnd/dtmintersvc.cc:128 NO Established contact
with 'SC-2'
<143>1 2018-12-07T13:32:09.942801+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33622"] 155:dtm/dtmnd/dtmintersvc.cc:143 TR DTM: nodeip:10.0.3.102,
nodeid:131599 iaddrfamily:2
<143>1 2018-12-07T13:32:09.942815+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33623"] 155:dtm/dtmnd/dtmintersvc.cc:154 TR DTM : Intranode IPCSEND
: NODE UP EVENT : SUCC
<143>1 2018-12-07T13:32:09.942819+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33624"] 155:dtm/dtmnd/dtmintersvc.cc:484 >>
dtmprepareandsendsvcupmsgfornodeup
<143>1 2018-12-07T13:32:09.942823+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33625"] 155:dtm/dtmnd/dtmintertrans.cc:239 >>
dtminternodesndmsgtonode
<143>1 2018-12-07T13:32:09.942826+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33626"] 155:dtm/dtmnd/dtmnodedb.cc:135 >> dtmnodegetbyid
<143>1 2018-12-07T13:32:09.94283+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33627"] 155:dtm/dtmnd/dtmnodedb.cc:146 << dtmnodegetbyid
<143>1 2018-12-07T13:32:09.942833+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33628"] 155:dtm/dtmnd/dtmintertrans.cc:178 >>
dtminternodesndmsgcommon
<143>1 2018-12-07T13:32:09.942858+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33629"] 155:dtm/dtmnd/dtmintertrans.cc:186 <<
dtminternodesndmsgcommon
<143>1 2018-12-07T13:32:09.942862+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33630"] 155:dtm/dtmnd/dtmintertrans.cc:254 <<
dtminternodesndmsgtonode
<143>1 2018-12-07T13:32:09.942866+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33631"] 155:dtm/dtmnd/dtmintersvc.cc:521 <<
dtmprepareandsendsvcupmsgfornodeup
<143>1 2018-12-07T13:32:09.942869+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33632"] 155:dtm/dtmnd/dtmintersvc.cc:160 << dtmnodeup
<143>1 2018-12-07T13:32:09.942872+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33633"] 155:dtm/dtmnd/dtmnode.cc:172 << dtmprocessnodeinfo
<143>1 2018-12-07T13:32:09.942877+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33634"] 155:dtm/dtmnd/dtmnode.cc:335 <<
dtminternodeprocesspollrcvmsg
<143>1 2018-12-07T13:32:09.942888+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33635"] 150:dtm/dtmnd/multicast.cc:808 << Send: success: 1 ? why
sned here?
<143>1 2018-12-07T13:32:09.943301+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33636"] 154:dtm/dtmnd/dtmintra.cc:705 TR DTM: nodeip:10.0.3.102,
nodeid:131599 iaddrfamily:2
<143>1 2018-12-07T13:32:09.943365+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33637"] 155:dtm/dtmnd/dtmnode.cc:418 TR DTM :Listening socket is
readable
<143>1 2018-12-07T13:32:09.943389+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33638"] 154:dtm/dtmnd/dtmintrasvc.cc:1492 >>
dtmintranodeprocessnodeup
<143>1 2018-12-07T13:32:09.9434+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33639"] 155:dtm/dtmnd/dtmnodesockets.cc:670 >> dtmprocessaccept
<143>1 2018-12-07T13:32:09.943425+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33640"] 154:dtm/dtmnd/dtmintrasvc.cc:1504 TR nodename:SC-2,
nodeip:10.0.3.102, nodeid:131599 iaddrfamily:2
<143>1 2018-12-07T13:32:09.943441+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33641"] 155:dtm/dtmnd/dtmnodesockets.cc:73 >> setkeepalive
<143>1 2018-12-07T13:32:09.94346+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33642"] 154:dtm/dtmnd/dtmintrasvc.cc:1262 >> dtmaddtonodedblist
<143>1 2018-12-07T13:32:09.943473+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33643"] 155:dtm/dtmnd/dtmnodesockets.cc:143 << setkeepalive: rc :1
<143>1 2018-12-07T13:32:09.943483+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33644"] 154:dtm/dtmnd/dtmintrasvc.cc:1264 TR nodeip:10.0.3.102,
nodeid:131599 iaddrfamily:2
<143>1 2018-12-07T13:32:09.943492+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33645"] 155:dtm/dtmnd/dtmnodedb.cc:38 >> dtmnodenew
<143>1 2018-12-07T13:32:09.943553+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33646"] 154:dtm/dtmnd/dtmintrasvc.cc:1272 << dtmaddtonodedblist
<143>1 2018-12-07T13:32:09.943617+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33647"] 155:dtm/dtmnd/dtmnodedb.cc:45 << dtmnodenew
<143>1 2018-12-07T13:32:09.94365+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33648"] 154:dtm/dtmnd/dtmintrasvc.cc:1530 TR DTM:
nodeip:10.0.3.102, nodeid:131599 iaddrfamily:2
<143>1 2018-12-07T13:32:09.943682+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33649"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.943715+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33650"] 154:dtm/dtmnd/dtmintrasvc.cc:1433 >> dtmlibpreparenodeupmsg
<143>1 2018-12-07T13:32:09.94374+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33651"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 2
<143>1 2018-12-07T13:32:09.943766+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33652"] 154:dtm/dtmnd/dtmintrasvc.cc:1435 TR nodeip:10.0.3.102,
nodeid:131599 iaddrfamily:2
<143>1 2018-12-07T13:32:09.943791+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33653"] 155:dtm/dtmnd/dtmnodedb.cc:219 TR DTM:Adding nodeip to the
database with nodeip :10.0.3.103 as key
<143>1 2018-12-07T13:32:09.943821+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33654"] 154:dtm/dtmnd/dtmintrasvc.cc:1446 << dtmlibpreparenodeupmsg
<143>1 2018-12-07T13:32:09.943844+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33655"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 1
<143>1 2018-12-07T13:32:09.943876+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33656"] 154:dtm/dtmnd/dtmintrasvc.cc:1548 <<
dtmintranodeprocessnodeup
<143>1 2018-12-07T13:32:09.943903+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33657"] 155:dtm/dtmnd/dtmnodesockets.cc:760 << dtmprocessaccept:
DTM: newconnsd :23
<143>1 2018-12-07T13:32:09.943921+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33658"] 155:dtm/dtmnd/dtmnodesockets.cc:208 >> dtmcommsocketsend
<143>1 2018-12-07T13:32:09.943999+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33659"] 155:dtm/dtmnd/dtmnodesockets.cc:218 << dtmcommsocketsend:
rc :1
<143>1 2018-12-07T13:32:09.944017+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33660"] 155:dtm/dtmnd/dtmnode.cc:508 TR DTM: add New incoming
connection to fd: 23
<143>1 2018-12-07T13:32:09.944029+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33661"] 155:dtm/dtmnd/dtmnodesockets.cc:670 >> dtmprocessaccept
<143>1 2018-12-07T13:32:09.944064+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33662"] 155:dtm/dtmnd/dtmnodesockets.cc:73 >> setkeepalive
<143>1 2018-12-07T13:32:09.944085+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33663"] 155:dtm/dtmnd/dtmnodesockets.cc:143 << setkeepalive: rc :1
<143>1 2018-12-07T13:32:09.9441+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33664"] 155:dtm/dtmnd/dtmnodedb.cc:38 >> dtmnodenew
<143>1 2018-12-07T13:32:09.944132+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33665"] 155:dtm/dtmnd/dtmnodedb.cc:45 << dtmnodenew
<143>1 2018-12-07T13:32:09.944146+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33666"] 155:dtm/dtmnd/dtmnodedb.cc:195 >> dtmnodeadd
<143>1 2018-12-07T13:32:09.944154+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33667"] 155:dtm/dtmnd/dtmnodedb.cc:196 TR DTM:value of i 2
<143>1 2018-12-07T13:32:09.944167+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33668"] 155:dtm/dtmnd/dtmnodedb.cc:219 TR DTM:Adding nodeip to the
database with nodeip :10.0.3.102 as key
<143>1 2018-12-07T13:32:09.944176+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33669"] 155:dtm/dtmnd/dtmnodedb.cc:225 TR DTM:ncspatriciatreeadd
for nodeip FAILED for :10.0.3.102 :2
<143>1 2018-12-07T13:32:09.944188+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33670"] 155:dtm/dtmnd/dtmnodedb.cc:238 << dtmnodeadd: rc : 2
<139>1 2018-12-07T13:32:09.944233+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33671"] 155:dtm/dtmnd/dtmnodesockets.cc:753 ER DTM: dtmnodeadd
failed .nodeip: 10.0.3.102, nodeid: 0
<143>1 2018-12-07T13:32:09.944246+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33672"] 155:dtm/dtmnd/dtmnodesockets.cc:157 >> dtmcommsocketclose
<143>1 2018-12-07T13:32:09.944254+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33673"] 155:dtm/dtmnd/dtmnodesockets.cc:161 TR DTM: node deleting
enty
<143>1 2018-12-07T13:32:09.944268+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33674"] 155:dtm/dtmnd/dtmnodedb.cc:254 >> dtmnodedelete: DTM:value
of i 0
<143>1 2018-12-07T13:32:09.944275+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33675"] 155:dtm/dtmnd/dtmnodedb.cc:298 << dtmnodedelete
<143>1 2018-12-07T13:32:09.944283+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33676"] 155:dtm/dtmnd/dtmnodedb.cc:254 >> dtmnodedelete: DTM:value
of i 2
<143>1 2018-12-07T13:32:09.944296+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33677"] 155:dtm/dtmnd/dtmnodedb.cc:298 << dtmnodedelete
<143>1 2018-12-07T13:32:09.944379+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33678"] 155:dtm/dtmnd/dtmnodesockets.cc:194 << dtmcommsocketclose
<143>1 2018-12-07T13:32:09.944412+01:00 PL-4 osafdtmd 150 osafdtmd [meta
sequenceId="33679"] 155:dtm/dtmnd/dtmnodesockets.cc:0 << dtmprocessaccept*
---
Sent from sourceforge.net because [email protected] is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list._______________________________________________
Opensaf-tickets mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
