Hi all,
I'm attempting to use sequoia (2.10.9) to cluster some Apache Derby
databases, which generally seems to work as expected.
However, there is a problem that seems to keep cropping up, and appears like it
might be related to Priority Inversion - and I'm not quite sure what that is.
The problem I'm seeing is that sometimes when two queries (which are sequential
& from the same client) both update the same row, they are hitting a locking
situation which wouldn't normally occur.
Normally, the requests from sequoia come through to derby looking like this
(these are Derby Logs, for transaction #40738):
2008-02-21 16:13:34.190 GMT Thread[DRSCluster - BackendWorkerThread for backend
'DRSPRI-BACKUP' with RAIDb level:1,5,main] (XID = 40738), (SESSIONID = 21),
(DATABASE = DRSPRI-BACKUP), (DRDAID = null), Executing prepared statement:
update users set failcount = 0 where id = ? :End prepared statement with 1
parameters begin parameter #1: 611 :end parameter
2008-02-21 16:13:34.193 GMT Thread[DRSCluster - BackendWorkerThread for backend
'DRSPRI-BACKUP' with RAIDb level:1,5,main] (XID = 40738), (SESSIONID = 21),
(DATABASE = DRSPRI-BACKUP), (DRDAID = null), Committing
2008-02-21 16:13:34.295 GMT Thread[DRSCluster - BackendWorkerThread for backend
'DRSPRI-BACKUP' with RAIDb level:1,5,main] (XID = 40738), (SESSIONID = 21),
(DATABASE = DRSPRI-BACKUP), (DRDAID = null), Committing
2008-02-21 16:13:34.295 GMT Thread[DRSCluster - BackendWorkerThread for backend
'DRSPRI-BACKUP' with RAIDb level:1,5,main] (XID = 40738), (SESSIONID = 21),
(DATABASE = DRSPRI-BACKUP), (DRDAID = null), Committing
However, every so often, all I get is the execution, without any commit:
2008-02-21 16:13:34.295 GMT Thread[DRSCluster - BackendWorkerThread for backend
'DRSPRI-BACKUP' with RAIDb level:1,5,main] (XID = 40739), (SESSIONID = 21),
(DATABASE = DRSPRI-BACKUP), (DRDAID = null), Executing prepared statement:
update users set failcount = 0 where id = ? :End prepared statement with 1
parameters begin parameter #1: 611 :end parameter
(these is the only activity visible in derby for transaction #40739)
but at the same time, Sequoia performs Priority Inversion on (presumably) this
query:
2008-02-21 16:13:34,295 DEBUG
org.continuent.sequoia.controller.backend.DatabaseBackend.DRSCluster.DRSPRI-BACKUP
- Priority inversion for already started request update users set failcount =
0 where id = ?/<!%I|611|!%>
2008-02-21 16:13:34,295 DEBUG
org.continuent.sequoia.controller.backend.DatabaseBackend.DRSCluster.DRSPRI-BACKUP
- Moving StatementExecuteUpdateTask from transaction 4222124650660804 (update
users set failcount = 0 where id = ?/<!%I|611|!%>) to non conflicting queue
when this happens, it appears that the commit (which is issued by the client)
never gets presented to derby, so that a lock is now kept on the updated row,
and subsequent updates of this row fail with lock timeout errors (confirmed
from Derby lock diagnostics).
I've reproduced this several times, and every time there is a Priority
Inversion when the commit disappears. Without sequoia, I've had no locking
issues.
I've also just noticed that in all cases, the commit for the previous
transaction, the executing of the failing transaction and the priority
inversion are all timestamped to the same millisecond (although the previous
commit always comes through to derby before the excecution)
So, can anyone explain what Prority Inversion is, and where the problem might
be?
If it has any relevance, my setup has two or three distributed controllers
(using appia w/tcp sequencer), each of which has two derby backends. Although
there are distributed controllers, these failures are all local (e.g. the
failures occur in the backends attached to the controller which receives the
request).
many thanks in advance,
Andrew Lawrenson
_______________________________________________
Sequoia mailing list
[email protected]
https://forge.continuent.org/mailman/listinfo/sequoia
