Hi Markus,
we are using Sequoia 2.10.10 and have two controllers on dedicated
systems each with one backend database.
Yesterday I started a backup on controller2 and after that the database
was not enabled again. So I restarted the virtual-database but since
that the controller hangs.
After searching for some info about the problem (no log file entries) I
dicovered that there are 7 pending transactions on controller1 which
seem to block the controller. They do not seem to timeout and I have
found no option to rollback/timeout them with the console or JMX.
Yes, it is true that there is no option in the console to rollback a
transaction, this is probably a feature we should add. If you close the
client connections (can probably easily be done by resetting the app.
side connection pool) that will automatically rollback the connections.
Open transactions usually come from an application that does not
commit/rollback transactions in all cases (and thus transactions remain
uncommitted). Another caveat of the JDBC API is that a commit/rollback
automatically starts a new transaction. So if you don't force the
connection to setAutoCommit(true) after a commit/rollback they will be
in a new transaction. And Sequoia can only perform
enable.disable/backup/shutdown operations on transaction boundaries.
We never implemented a kill/abort command because it cannot be
implemented deterministically (what do you do is the transaction has
already committed on a backend and you try to cancel cluster-wide?).
This could however be implemented for open transaction with no active
request. Feel free to add a feature request in JIRA if you think this
could be a useful tool to have.
Am I missing something? Can someone give me a hint on how to stabilize
the system again. Or is the only solution to kill both controllers,
initialize the database again (or restore from dump) and synchronize the
controllers again afterwards?
In your case, the 2nd backend was still in backup mode waiting for
transactions to complete. If you are not able to terminate the
transactions by closing your application connections, you can reset the
virtual database by using a force shutdown (this will abort pending
transactions). You will probably have to resynchronize the 2nd
controller with its backend upon restart to make sure that they are
properly synchronized.
Hope this helps,
Emmanuel
--
Emmanuel Cecchet
FTO @ Frog Thinker
Open Source Development & Consulting
--
Web: http://www.frogthinker.org
email: [EMAIL PROTECTED]
Skype: emmanuel_cecchet
_______________________________________________
Sequoia mailing list
[email protected]
https://forge.continuent.org/mailman/listinfo/sequoia
