[
https://issues.apache.org/jira/browse/ZOOKEEPER-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bozhen Liu updated ZOOKEEPER-3819:
----------------------------------
Description:
We run our static race detector on ZooKeeper and discovered three methods in
class org/apache/zookeeper/server/DataTree may have concurrent accesses on
ephemerals without proper lock protection. The three methods are createNode(),
killSession() and deserialize(). They all read and write the ephemerals in the
similar way without common lock from get() to put():
{code:java}
HashSet<String> list = ephemerals.get(eowner);//from deserialize
if (list == null){
list = new HashSet<String>();
ephemerals.put(eowner, list);
}
list.add(path);
{code}
Both createNode() and deserialize() can add to the list of a session, one with
lock, but the other has no lock.
This is similar to ZOOKEEPER-3102.
The possible traces that lead to the race are (v3.5.4, probably the same in
v3.6.1):
=> Race: org/apache/zookeeper/server/DataTree.java/util/Map
(org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
org/apache/zookeeper/server/DataTree:1215 (TID: 9592) )
Trace 1st node: (TID: 2190)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
*> Thread (2190) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.createNode from
org/apache/zookeeper/server/DataTree.processTxn (line 786)
=> Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.createNode (line 478)
Trace 2st node: (TID: 9592)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/Leader.lead from
org/apache/zookeeper/server/quorum/QuorumPeer.run (line 1227)
*> Thread (6402) created by
Application,Lorg/apache/zookeeper/server/quorum/Leader>.lead (line 448)
-> Call org/apache/zookeeper/server/quorum/Leader$LearnerCnxAcceptor.run from
java/lang/Thread.start (line -1)
*> Thread (7676) created by
Application,Lorg/apache/zookeeper/server/quorum/Leader$LearnerCnxAcceptor>.run
(line 386)
-> Call org/apache/zookeeper/server/quorum/LearnerHandler.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/Leader.processAck from
org/apache/zookeeper/server/quorum/LearnerHandler.run (line 559)
-> Call org/apache/zookeeper/server/quorum/Leader.tryToCommit from
org/apache/zookeeper/server/quorum/Leader.processAck (line 863)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.processReconfig from
org/apache/zookeeper/server/quorum/Leader.tryToCommit (line 779)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.restartLeaderElection
from org/apache/zookeeper/server/quorum/QuorumPeer.processReconfig (line 1871)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.startLeaderElection from
org/apache/zookeeper/server/quorum/QuorumPeer.restartLeaderElection (line 1530)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.getLastLoggedZxid from
org/apache/zookeeper/server/quorum/QuorumPeer.startLeaderElection (line 917)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.loadDataBase from
org/apache/zookeeper/server/quorum/QuorumPeer.getLastLoggedZxid (line 1009)
-> Call org/apache/zookeeper/server/ZKDatabase.loadDataBase from
org/apache/zookeeper/server/quorum/QuorumPeer.loadDataBase (line 867)
-> Call org/apache/zookeeper/server/persistence/FileTxnSnapLog.restore from
org/apache/zookeeper/server/ZKDatabase.loadDataBase (line 240)
-> Call org/apache/zookeeper/server/persistence/FileSnap.deserialize from
org/apache/zookeeper/server/persistence/FileTxnSnapLog.restore (line 200)
-> Call org/apache/zookeeper/server/persistence/FileSnap.deserialize from
org/apache/zookeeper/server/persistence/FileSnap.deserialize (line 87)
-> Call org/apache/zookeeper/server/util/SerializeUtils.deserializeSnapshot
from org/apache/zookeeper/server/persistence/FileSnap.deserialize (line 122)
-> Call org/apache/zookeeper/server/DataTree.deserialize from
org/apache/zookeeper/server/util/SerializeUtils.deserializeSnapshot (line 141)
=> Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.deserialize (line 1215)
--------------------------------------------------------------------------------------------------------------------------------
=> Race: org/apache/zookeeper/server/DataTree.java/util/Map
(org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
org/apache/zookeeper/server/DataTree:1005 (TID: 2835) )
Trace 1st node: (TID: 2190)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
*> Thread (2190) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.createNode from
org/apache/zookeeper/server/DataTree.processTxn (line 786)
=> Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.createNode (line 478)
Trace 2st node: (TID: 2835)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
*> Thread (2190) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer$1.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.startup
from org/apache/zookeeper/server/quorum/QuorumPeer$1.run (line 1157)
-> Call org/apache/zookeeper/server/ZooKeeperServer.startup from
org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.startup (line 70)
-> Call
org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.setupRequestProcessors
from org/apache/zookeeper/server/ZooKeeperServer.startup (line 453)
*> Thread (2835) created by
Application,Lorg/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer>.setupRequestProcessors
(line 59)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.killSession from
org/apache/zookeeper/server/DataTree.processTxn (line 854)
=> Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.killSession (line 1005)
--------------------------------------------------------------------------------------------------------------------------------
=> Race: org/apache/zookeeper/server/DataTree.java/util/Map
(org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
org/apache/zookeeper/server/DataTree:481 (TID: 13154) )
Trace 1st node: (TID: 2190)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
*> Thread (2190) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.createNode from
org/apache/zookeeper/server/DataTree.processTxn (line 786)
=> Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.createNode (line 478)
Trace 2st node: (TID: 13154)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/Observer.observeLeader from
org/apache/zookeeper/server/quorum/QuorumPeer.run (line 1201)
-> Call org/apache/zookeeper/server/quorum/Learner.syncWithLeader from
org/apache/zookeeper/server/quorum/Observer.observeLeader (line 74)
-> Call org/apache/zookeeper/server/ZooKeeperServer.startup from
org/apache/zookeeper/server/quorum/Learner.syncWithLeader (line 559)
-> Call
org/apache/zookeeper/server/quorum/ObserverZooKeeperServer.setupRequestProcessors
from org/apache/zookeeper/server/ZooKeeperServer.startup (line 453)
*> Thread (8653) created by
Application,Lorg/apache/zookeeper/server/quorum/ObserverZooKeeperServer>.setupRequestProcessors
(line 97)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/CommitProcessor.start (line 263)
-> Call org/apache/zookeeper/server/quorum/CommitProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/CommitProcessor.sendToNextProcessor
from org/apache/zookeeper/server/quorum/CommitProcessor.run (line 180)
-> Call org/apache/zookeeper/server/WorkerService.schedule from
org/apache/zookeeper/server/quorum/CommitProcessor.sendToNextProcessor (line
272)
*> Thread (13154) created by
Application,Lorg/apache/zookeeper/server/WorkerService>.schedule (line 128)
-> Call org/apache/zookeeper/server/WorkerService$ScheduledWorkRequest.run
from java/lang/Thread.start (line -1)
-> Call
org/apache/zookeeper/server/quorum/CommitProcessor$CommitWorkRequest.doWork
from org/apache/zookeeper/server/WorkerService$ScheduledWorkRequest.run (line
162)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/quorum/CommitProcessor$CommitWorkRequest.doWork
(line 297)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.createNode from
org/apache/zookeeper/server/DataTree.processTxn (line 786)
=> Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.createNode (line 481)
was:
We run our static race detector on ZooKeeper and discovered three methods in
class org/apache/zookeeper/server/DataTree may have concurrent accesses on
ephemerals without proper lock protection. The three methods are createNode(),
killSession() and deserialize(). They all read and write the ephemerals in the
similar way without common lock from get() to put():
{code:java}
HashSet<String> list = ephemerals.get(eowner);
if (list == null){
list = new HashSet<String>();
ephemerals.put(eowner, list);
}
list.add(path);
{code}
This is similar to ZOOKEEPER-3102.
The possible traces that lead to the race are (v3.5.4, probably the same in
v3.6.1):
=> Race: org/apache/zookeeper/server/DataTree.java/util/Map
(org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
org/apache/zookeeper/server/DataTree:1215 (TID: 9592) )
Trace 1st node: (TID: 2190)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
*> Thread (2190) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.createNode from
org/apache/zookeeper/server/DataTree.processTxn (line 786)
=> Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.createNode (line 478)
Trace 2st node: (TID: 9592)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/Leader.lead from
org/apache/zookeeper/server/quorum/QuorumPeer.run (line 1227)
*> Thread (6402) created by
Application,Lorg/apache/zookeeper/server/quorum/Leader>.lead (line 448)
-> Call org/apache/zookeeper/server/quorum/Leader$LearnerCnxAcceptor.run from
java/lang/Thread.start (line -1)
*> Thread (7676) created by
Application,Lorg/apache/zookeeper/server/quorum/Leader$LearnerCnxAcceptor>.run
(line 386)
-> Call org/apache/zookeeper/server/quorum/LearnerHandler.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/Leader.processAck from
org/apache/zookeeper/server/quorum/LearnerHandler.run (line 559)
-> Call org/apache/zookeeper/server/quorum/Leader.tryToCommit from
org/apache/zookeeper/server/quorum/Leader.processAck (line 863)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.processReconfig from
org/apache/zookeeper/server/quorum/Leader.tryToCommit (line 779)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.restartLeaderElection
from org/apache/zookeeper/server/quorum/QuorumPeer.processReconfig (line 1871)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.startLeaderElection from
org/apache/zookeeper/server/quorum/QuorumPeer.restartLeaderElection (line 1530)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.getLastLoggedZxid from
org/apache/zookeeper/server/quorum/QuorumPeer.startLeaderElection (line 917)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.loadDataBase from
org/apache/zookeeper/server/quorum/QuorumPeer.getLastLoggedZxid (line 1009)
-> Call org/apache/zookeeper/server/ZKDatabase.loadDataBase from
org/apache/zookeeper/server/quorum/QuorumPeer.loadDataBase (line 867)
-> Call org/apache/zookeeper/server/persistence/FileTxnSnapLog.restore from
org/apache/zookeeper/server/ZKDatabase.loadDataBase (line 240)
-> Call org/apache/zookeeper/server/persistence/FileSnap.deserialize from
org/apache/zookeeper/server/persistence/FileTxnSnapLog.restore (line 200)
-> Call org/apache/zookeeper/server/persistence/FileSnap.deserialize from
org/apache/zookeeper/server/persistence/FileSnap.deserialize (line 87)
-> Call org/apache/zookeeper/server/util/SerializeUtils.deserializeSnapshot
from org/apache/zookeeper/server/persistence/FileSnap.deserialize (line 122)
-> Call org/apache/zookeeper/server/DataTree.deserialize from
org/apache/zookeeper/server/util/SerializeUtils.deserializeSnapshot (line 141)
=> Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.deserialize (line 1215)
--------------------------------------------------------------------------------------------------------------------------------
=> Race: org/apache/zookeeper/server/DataTree.java/util/Map
(org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
org/apache/zookeeper/server/DataTree:1005 (TID: 2835) )
Trace 1st node: (TID: 2190)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
*> Thread (2190) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.createNode from
org/apache/zookeeper/server/DataTree.processTxn (line 786)
=> Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.createNode (line 478)
Trace 2st node: (TID: 2835)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
*> Thread (2190) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer$1.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.startup
from org/apache/zookeeper/server/quorum/QuorumPeer$1.run (line 1157)
-> Call org/apache/zookeeper/server/ZooKeeperServer.startup from
org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.startup (line 70)
-> Call
org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.setupRequestProcessors
from org/apache/zookeeper/server/ZooKeeperServer.startup (line 453)
*> Thread (2835) created by
Application,Lorg/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer>.setupRequestProcessors
(line 59)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.killSession from
org/apache/zookeeper/server/DataTree.processTxn (line 854)
=> Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.killSession (line 1005)
--------------------------------------------------------------------------------------------------------------------------------
=> Race: org/apache/zookeeper/server/DataTree.java/util/Map
(org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
org/apache/zookeeper/server/DataTree:481 (TID: 13154) )
Trace 1st node: (TID: 2190)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
*> Thread (2190) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.createNode from
org/apache/zookeeper/server/DataTree.processTxn (line 786)
=> Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.createNode (line 478)
Trace 2st node: (TID: 13154)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
-> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
*> Thread (153) created by
Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
(line 200)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
-> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/Observer.observeLeader from
org/apache/zookeeper/server/quorum/QuorumPeer.run (line 1201)
-> Call org/apache/zookeeper/server/quorum/Learner.syncWithLeader from
org/apache/zookeeper/server/quorum/Observer.observeLeader (line 74)
-> Call org/apache/zookeeper/server/ZooKeeperServer.startup from
org/apache/zookeeper/server/quorum/Learner.syncWithLeader (line 559)
-> Call
org/apache/zookeeper/server/quorum/ObserverZooKeeperServer.setupRequestProcessors
from org/apache/zookeeper/server/ZooKeeperServer.startup (line 453)
*> Thread (8653) created by
Application,Lorg/apache/zookeeper/server/quorum/ObserverZooKeeperServer>.setupRequestProcessors
(line 97)
-> Call java/lang/Thread.start from
org/apache/zookeeper/server/quorum/CommitProcessor.start (line 263)
-> Call org/apache/zookeeper/server/quorum/CommitProcessor.run from
java/lang/Thread.start (line -1)
-> Call org/apache/zookeeper/server/quorum/CommitProcessor.sendToNextProcessor
from org/apache/zookeeper/server/quorum/CommitProcessor.run (line 180)
-> Call org/apache/zookeeper/server/WorkerService.schedule from
org/apache/zookeeper/server/quorum/CommitProcessor.sendToNextProcessor (line
272)
*> Thread (13154) created by
Application,Lorg/apache/zookeeper/server/WorkerService>.schedule (line 128)
-> Call org/apache/zookeeper/server/WorkerService$ScheduledWorkRequest.run
from java/lang/Thread.start (line -1)
-> Call
org/apache/zookeeper/server/quorum/CommitProcessor$CommitWorkRequest.doWork
from org/apache/zookeeper/server/WorkerService$ScheduledWorkRequest.run (line
162)
-> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest from
org/apache/zookeeper/server/quorum/CommitProcessor$CommitWorkRequest.doWork
(line 297)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
-> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
-> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
-> Call org/apache/zookeeper/server/DataTree.processTxn from
org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
-> Call org/apache/zookeeper/server/DataTree.createNode from
org/apache/zookeeper/server/DataTree.processTxn (line 786)
=> Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
org/apache/zookeeper/server/DataTree.createNode (line 481)
> Potential Races on DataTree
> ---------------------------
>
> Key: ZOOKEEPER-3819
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3819
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.5.4, 3.6.1
> Reporter: Bozhen Liu
> Priority: Minor
>
> We run our static race detector on ZooKeeper and discovered three methods in
> class org/apache/zookeeper/server/DataTree may have concurrent accesses on
> ephemerals without proper lock protection. The three methods are
> createNode(), killSession() and deserialize(). They all read and write the
> ephemerals in the similar way without common lock from get() to put():
> {code:java}
> HashSet<String> list = ephemerals.get(eowner);//from deserialize
> if (list == null){
> list = new HashSet<String>();
> ephemerals.put(eowner, list);
> }
> list.add(path);
> {code}
> Both createNode() and deserialize() can add to the list of a session, one
> with lock, but the other has no lock.
> This is similar to ZOOKEEPER-3102.
> The possible traces that lead to the race are (v3.5.4, probably the same in
> v3.6.1):
> => Race: org/apache/zookeeper/server/DataTree.java/util/Map
> (org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
> org/apache/zookeeper/server/DataTree:1215 (TID: 9592) )
> Trace 1st node: (TID: 2190)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
> from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
> org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
> *> Thread (153) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
> (line 200)
> -> Call java/lang/Thread.start from
> org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
> java/lang/Thread.start (line -1)
> *> Thread (2190) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
> -> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
> java/lang/Thread.start (line -1)
> -> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
> org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
> -> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest
> from org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
> -> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
> -> Call org/apache/zookeeper/server/DataTree.processTxn from
> org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
> -> Call org/apache/zookeeper/server/DataTree.createNode from
> org/apache/zookeeper/server/DataTree.processTxn (line 786)
> => Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
> org/apache/zookeeper/server/DataTree.createNode (line 478)
> Trace 2st node: (TID: 9592)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
> from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
> org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
> *> Thread (153) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
> (line 200)
> -> Call java/lang/Thread.start from
> org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
> java/lang/Thread.start (line -1)
> -> Call org/apache/zookeeper/server/quorum/Leader.lead from
> org/apache/zookeeper/server/quorum/QuorumPeer.run (line 1227)
> *> Thread (6402) created by
> Application,Lorg/apache/zookeeper/server/quorum/Leader>.lead (line 448)
> -> Call org/apache/zookeeper/server/quorum/Leader$LearnerCnxAcceptor.run
> from java/lang/Thread.start (line -1)
> *> Thread (7676) created by
> Application,Lorg/apache/zookeeper/server/quorum/Leader$LearnerCnxAcceptor>.run
> (line 386)
> -> Call org/apache/zookeeper/server/quorum/LearnerHandler.run from
> java/lang/Thread.start (line -1)
> -> Call org/apache/zookeeper/server/quorum/Leader.processAck from
> org/apache/zookeeper/server/quorum/LearnerHandler.run (line 559)
> -> Call org/apache/zookeeper/server/quorum/Leader.tryToCommit from
> org/apache/zookeeper/server/quorum/Leader.processAck (line 863)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.processReconfig from
> org/apache/zookeeper/server/quorum/Leader.tryToCommit (line 779)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.restartLeaderElection
> from org/apache/zookeeper/server/quorum/QuorumPeer.processReconfig (line 1871)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.startLeaderElection
> from org/apache/zookeeper/server/quorum/QuorumPeer.restartLeaderElection
> (line 1530)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.getLastLoggedZxid from
> org/apache/zookeeper/server/quorum/QuorumPeer.startLeaderElection (line 917)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.loadDataBase from
> org/apache/zookeeper/server/quorum/QuorumPeer.getLastLoggedZxid (line 1009)
> -> Call org/apache/zookeeper/server/ZKDatabase.loadDataBase from
> org/apache/zookeeper/server/quorum/QuorumPeer.loadDataBase (line 867)
> -> Call org/apache/zookeeper/server/persistence/FileTxnSnapLog.restore from
> org/apache/zookeeper/server/ZKDatabase.loadDataBase (line 240)
> -> Call org/apache/zookeeper/server/persistence/FileSnap.deserialize from
> org/apache/zookeeper/server/persistence/FileTxnSnapLog.restore (line 200)
> -> Call org/apache/zookeeper/server/persistence/FileSnap.deserialize from
> org/apache/zookeeper/server/persistence/FileSnap.deserialize (line 87)
> -> Call org/apache/zookeeper/server/util/SerializeUtils.deserializeSnapshot
> from org/apache/zookeeper/server/persistence/FileSnap.deserialize (line 122)
> -> Call org/apache/zookeeper/server/DataTree.deserialize from
> org/apache/zookeeper/server/util/SerializeUtils.deserializeSnapshot (line 141)
> => Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals)
> in org/apache/zookeeper/server/DataTree.deserialize (line 1215)
>
> --------------------------------------------------------------------------------------------------------------------------------
> => Race: org/apache/zookeeper/server/DataTree.java/util/Map
> (org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
> org/apache/zookeeper/server/DataTree:1005 (TID: 2835) )
> Trace 1st node: (TID: 2190)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
> from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
> org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
> *> Thread (153) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
> (line 200)
> -> Call java/lang/Thread.start from
> org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
> java/lang/Thread.start (line -1)
> *> Thread (2190) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
> -> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
> java/lang/Thread.start (line -1)
> -> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
> org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
> -> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest
> from org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
> -> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
> -> Call org/apache/zookeeper/server/DataTree.processTxn from
> org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
> -> Call org/apache/zookeeper/server/DataTree.createNode from
> org/apache/zookeeper/server/DataTree.processTxn (line 786)
> => Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
> org/apache/zookeeper/server/DataTree.createNode (line 478)
> Trace 2st node: (TID: 2835)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
> from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
> org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
> *> Thread (153) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
> (line 200)
> -> Call java/lang/Thread.start from
> org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
> java/lang/Thread.start (line -1)
> *> Thread (2190) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer$1.run from
> java/lang/Thread.start (line -1)
> -> Call org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.startup
> from org/apache/zookeeper/server/quorum/QuorumPeer$1.run (line 1157)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.startup from
> org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.startup (line 70)
> -> Call
> org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.setupRequestProcessors
> from org/apache/zookeeper/server/ZooKeeperServer.startup (line 453)
> *> Thread (2835) created by
> Application,Lorg/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer>.setupRequestProcessors
> (line 59)
> -> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
> java/lang/Thread.start (line -1)
> -> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
> org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
> -> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest
> from org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
> -> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
> -> Call org/apache/zookeeper/server/DataTree.processTxn from
> org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
> -> Call org/apache/zookeeper/server/DataTree.killSession from
> org/apache/zookeeper/server/DataTree.processTxn (line 854)
> => Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals)
> in org/apache/zookeeper/server/DataTree.killSession (line 1005)
>
> --------------------------------------------------------------------------------------------------------------------------------
> => Race: org/apache/zookeeper/server/DataTree.java/util/Map
> (org/apache/zookeeper/server/DataTree:478 (TID: 2190) ,
> org/apache/zookeeper/server/DataTree:481 (TID: 13154) )
> Trace 1st node: (TID: 2190)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
> from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
> org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
> *> Thread (153) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
> (line 200)
> -> Call java/lang/Thread.start from
> org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
> java/lang/Thread.start (line -1)
> *> Thread (2190) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeer>.run (line 1167)
> -> Call org/apache/zookeeper/server/PrepRequestProcessor.run from
> java/lang/Thread.start (line -1)
> -> Call org/apache/zookeeper/server/PrepRequestProcessor.pRequest from
> org/apache/zookeeper/server/PrepRequestProcessor.run (line 145)
> -> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest
> from org/apache/zookeeper/server/PrepRequestProcessor.pRequest (line 906)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
> -> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
> -> Call org/apache/zookeeper/server/DataTree.processTxn from
> org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
> -> Call org/apache/zookeeper/server/DataTree.createNode from
> org/apache/zookeeper/server/DataTree.processTxn (line 786)
> => Read on org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals) in
> org/apache/zookeeper/server/DataTree.createNode (line 478)
> Trace 2st node: (TID: 13154)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun
> from org/apache/zookeeper/server/quorum/QuorumPeerMain.main (line 82)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeerMain.runFromConfig from
> org/apache/zookeeper/server/quorum/QuorumPeerMain.initializeAndRun (line 123)
> *> Thread (153) created by
> Application,Lorg/apache/zookeeper/server/quorum/QuorumPeerMain>.runFromConfig
> (line 200)
> -> Call java/lang/Thread.start from
> org/apache/zookeeper/server/quorum/QuorumPeer.start (line 862)
> -> Call org/apache/zookeeper/server/quorum/QuorumPeer.run from
> java/lang/Thread.start (line -1)
> -> Call org/apache/zookeeper/server/quorum/Observer.observeLeader from
> org/apache/zookeeper/server/quorum/QuorumPeer.run (line 1201)
> -> Call org/apache/zookeeper/server/quorum/Learner.syncWithLeader from
> org/apache/zookeeper/server/quorum/Observer.observeLeader (line 74)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.startup from
> org/apache/zookeeper/server/quorum/Learner.syncWithLeader (line 559)
> -> Call
> org/apache/zookeeper/server/quorum/ObserverZooKeeperServer.setupRequestProcessors
> from org/apache/zookeeper/server/ZooKeeperServer.startup (line 453)
> *> Thread (8653) created by
> Application,Lorg/apache/zookeeper/server/quorum/ObserverZooKeeperServer>.setupRequestProcessors
> (line 97)
> -> Call java/lang/Thread.start from
> org/apache/zookeeper/server/quorum/CommitProcessor.start (line 263)
> -> Call org/apache/zookeeper/server/quorum/CommitProcessor.run from
> java/lang/Thread.start (line -1)
> -> Call
> org/apache/zookeeper/server/quorum/CommitProcessor.sendToNextProcessor from
> org/apache/zookeeper/server/quorum/CommitProcessor.run (line 180)
> -> Call org/apache/zookeeper/server/WorkerService.schedule from
> org/apache/zookeeper/server/quorum/CommitProcessor.sendToNextProcessor (line
> 272)
> *> Thread (13154) created by
> Application,Lorg/apache/zookeeper/server/WorkerService>.schedule (line 128)
> -> Call org/apache/zookeeper/server/WorkerService$ScheduledWorkRequest.run
> from java/lang/Thread.start (line -1)
> -> Call
> org/apache/zookeeper/server/quorum/CommitProcessor$CommitWorkRequest.doWork
> from org/apache/zookeeper/server/WorkerService$ScheduledWorkRequest.run (line
> 162)
> -> Call org/apache/zookeeper/server/FinalRequestProcessor.processRequest
> from
> org/apache/zookeeper/server/quorum/CommitProcessor$CommitWorkRequest.doWork
> (line 297)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/FinalRequestProcessor.processRequest (line 104)
> -> Call org/apache/zookeeper/server/ZooKeeperServer.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1190)
> -> Call org/apache/zookeeper/server/ZKDatabase.processTxn from
> org/apache/zookeeper/server/ZooKeeperServer.processTxn (line 1199)
> -> Call org/apache/zookeeper/server/DataTree.processTxn from
> org/apache/zookeeper/server/ZKDatabase.processTxn (line 434)
> -> Call org/apache/zookeeper/server/DataTree.createNode from
> org/apache/zookeeper/server/DataTree.processTxn (line 786)
> => Write to org/apache/zookeeper/server/DataTree.java/util/Map(ephemerals)
> in org/apache/zookeeper/server/DataTree.createNode (line 481)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
