Server shutdown due a deadlock in BerkelyDB between flush thread and GC thread.
-------------------------------------------------------------------------------
Key: CDV-502
URL: https://jira.terracotta.org/jira//browse/CDV-502
Project: Community Development
Issue Type: Bug
Affects Versions: 2.4.5
Reporter: Saravanan Subbiah
Assignee: Saravanan Subbiah
This is from the forum post
http://forums.terracotta.org/forums/posts/list/588.page
>From the logs :
2007-11-04 19:42:24,540 [WorkerThread(managed_object_flush_stage,3)] ERROR
com.tc.server.TCServerMain -
Thread:Thread[WorkerThread(managed_object_flush_stage,3),5,TC Thread Group] got
an uncaught exception. About to sleep then exit.
com.tc.objectserver.persistence.sleepycat.TCDatabaseException:
com.sleepycat.je.DeadlockException: (JE 3.2.23) Lock expired. Locker
30970876_WorkerThread(managed_object_flush_stage,3)_Txn: waited for lock on
database=objects node=3409750 type=WRITE grant=WAIT_NEW timeoutMillis=180000
startTime=1194233786667 endTime=1194234144540
Owners: [<LockInfo locker="30970875_GC_Txn" type="WRITE"/>]
Waiters: []
Transaction 30970875_GC_Txn waits for node 1464631
Transaction 30970876_WorkerThread(managed_object_flush_stage,3)_Txn owns
1464631 <LockInfo
locker="30970876_WorkerThread(managed_object_flush_stage,3)_Txn" type="WRITE"/>
Transaction 30970876_WorkerThread(managed_object_flush_stage,3)_Txn waits for
node 3409750
at
com.tc.objectserver.persistence.sleepycat.ManagedObjectPersistorImpl.basicSaveObject(ManagedObjectPersistorImpl.java:280)
at
com.tc.objectserver.persistence.sleepycat.ManagedObjectPersistorImpl.saveAllObjects(ManagedObjectPersistorImpl.java:315)
at
com.tc.objectserver.impl.PersistentManagedObjectStore.commitAllObjects(PersistentManagedObjectStore.java:81)
at
com.tc.objectserver.impl.ObjectManagerImpl.flushAll(ObjectManagerImpl.java:614)
at
com.tc.objectserver.impl.ObjectManagerImpl.flushAndEvict(ObjectManagerImpl.java:801)
at
com.tc.objectserver.handler.ManagedObjectFlushHandler.handleEvent(ManagedObjectFlushHandler.java:22)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:140)
Caused by: com.sleepycat.je.DeadlockException: (JE 3.2.23) Lock expired.
Locker 30970876_WorkerThread(managed_object_flush_stage,3)_Txn: waited for lock
on database=objects node=3409750 type=WRITE grant=WAIT_NEW timeoutMillis=180000
startTime=1194233786667 endTime=1194234144540
Owners: [<LockInfo locker="30970875_GC_Txn" type="WRITE"/>]
Waiters: []
Transaction 30970875_GC_Txn waits for node 1464631
Transaction 30970876_WorkerThread(managed_object_flush_stage,3)_Txn owns
1464631 <LockInfo
locker="30970876_WorkerThread(managed_object_flush_stage,3)_Txn" type="WRITE"/>
Transaction 30970876_WorkerThread(managed_object_flush_stage,3)_Txn waits for
node 3409750
at com.sleepycat.je.txn.LockManager.lock(LockManager.java:266)
at com.sleepycat.je.txn.Txn.lockInternal(Txn.java:295)
at com.sleepycat.je.txn.Locker.lock(Locker.java:257)
at
com.sleepycat.je.dbi.CursorImpl.lockLNDeletedAllowed(CursorImpl.java:2349)
at com.sleepycat.je.tree.Tree.insert(Tree.java:2567)
at com.sleepycat.je.dbi.CursorImpl.putLN(CursorImpl.java:988)
at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:1011)
at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:861)
at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:798)
at com.sleepycat.je.Cursor.putInternal(Cursor.java:752)
at com.sleepycat.je.Database.putInternal(Database.java:662)
at com.sleepycat.je.Database.put(Database.java:608)
at
com.tc.objectserver.persistence.sleepycat.ManagedObjectPersistorImpl.basicSaveObject(ManagedObjectPersistorImpl.java:270)
... 6 more
2007-11-04 19:42:24,712 [GC] ERROR com.tc.server.TCServerMain -
Thread:Thread[GC,5,TC Thread Group] got an uncaught exception. About to sleep
then exit.
com.sleepycat.je.DeadlockException: (JE 3.2.23) Lock expired. Locker
30970875_GC_Txn: waited for lock on database=objects node=1464631 type=WRITE
grant=WAIT_NEW timeoutMillis=180000 startTime=1194233786839
endTime=1194234144712
Owners: [<LockInfo
locker="30970876_WorkerThread(managed_object_flush_stage,3)_Txn" type="WRITE"/>]
Waiters: []
at com.sleepycat.je.txn.LockManager.lock(LockManager.java:266)
at com.sleepycat.je.txn.Txn.lockInternal(Txn.java:295)
at com.sleepycat.je.txn.Locker.lock(Locker.java:257)
at
com.sleepycat.je.dbi.CursorImpl.lockLNDeletedAllowed(CursorImpl.java:2349)
at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:2271)
at
com.sleepycat.je.dbi.CursorImpl.searchAndPosition(CursorImpl.java:1963)
at com.sleepycat.je.Cursor.searchInternal(Cursor.java:1178)
at com.sleepycat.je.Cursor.searchAllowPhantoms(Cursor.java:1148)
at com.sleepycat.je.Cursor.search(Cursor.java:1014)
at com.sleepycat.je.Database.deleteInternal(Database.java:473)
at com.sleepycat.je.Database.delete(Database.java:405)
at
com.tc.objectserver.persistence.sleepycat.ManagedObjectPersistorImpl.deleteObjectByID(ManagedObjectPersistorImpl.java:358)
at
com.tc.objectserver.persistence.sleepycat.ManagedObjectPersistorImpl.deleteAllObjectsByID(ManagedObjectPersistorImpl.java:372)
at
com.tc.objectserver.impl.PersistentManagedObjectStore.removeAllObjectsByIDNow(PersistentManagedObjectStore.java:86)
at
com.tc.objectserver.impl.ObjectManagerImpl.removeFromStore(ObjectManagerImpl.java:600)
at
com.tc.objectserver.impl.ObjectManagerImpl.notifyGCComplete(ObjectManagerImpl.java:585)
at
com.tc.objectserver.core.impl.MarkAndSweepGarbageCollector.gc(MarkAndSweepGarbageCollector.java:176)
at
com.tc.objectserver.impl.ObjectManagerImpl$1.run(ObjectManagerImpl.java:686)
2007-11-04 19:42:27,556 [CommonShutDownHook - [EMAIL PROTECTED] INFO
com.tc.net.protocol.TCNetworkMessage - No of times Buffers wasted = 0 Buffers
wasted count = 0
2007-11-04 19:42:31,712 [Timer-4] INFO com.terracottatech.console - Server
exiting...
2007-11-04 19:42:31,712 [Timer-4] INFO com.terracottatech.console - Server
exiting...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.terracotta.org/jira//secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
_______________________________________________
tc-dev mailing list
[email protected]
http://lists.terracotta.org/mailman/listinfo/tc-dev
