[
https://issues.apache.org/jira/browse/QPID-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13808080#comment-13808080
]
ASF subversion and git services commented on QPID-5139:
-------------------------------------------------------
Commit 1536754 from [~aconway] in branch 'qpid/trunk'
[ https://svn.apache.org/r1536754 ]
QPID-5139: HA transactions block a thread, can deadlock the broker
PrimaryTxObserver::prepare used to block pending responses from each backup.
With
concurrent transactions this can deadlock the broker: once all worker threads
are blocked in prepare, responses from backups cannot be received.
This commit generalizes the async completion mechanism for messages to allow
async completion of arbitrary commands. It leaves the special-case code for
messages undisturbed but adds a second path (starting from
SessionState::handleCommand) for async completion of other commands.
In particular it implements tx.commit to allow async completion.
TxBuffer is now an AsyncCompletion and commitLocal() is split into
- startCommit() called by SemanticState::commit()
- endCommit() called when the commit command completes
TxAccept no longer holds pre-computed ranges, compute fresh each time.
- Avoid range iterators going out of date during a delayed commit.
> HA transactions block a thread, can deadlock the broker.
> --------------------------------------------------------
>
> Key: QPID-5139
> URL: https://issues.apache.org/jira/browse/QPID-5139
> Project: Qpid
> Issue Type: Bug
> Components: C++ Clustering
> Affects Versions: 0.24
> Reporter: Alan Conway
> Assignee: Alan Conway
> Priority: Critical
> Fix For: 0.25
>
>
> When the client sends a "prepare" command for a transaction, the thread
> handling that command is blocked until all backups have responded with their
> prepare status. This can easily deadlock the broker if there are more
> concurrent transactions than worker threads.
> To reproduce you need a way to delay completion of a transaction. The
> following regression test illustrates, it will be added to TransactionTests
> in ha_tests.py as part of the fix:
> {noformat}
> def test_tx_block_threads(self):
> """Verify that TXs blocked in commit don't block broker threads."""
> cluster = HaCluster(self, 2, args=["--worker-threads=2"])
> sessions = [cluster[0].connect().session(transactional=True) for i in
> xrange(2)]
> for s in sessions: s.sender("foo;{create:always}").send("foo")
> self.assertEqual(2, len(cluster[1].agent().tx_queues()))
> os.kill(cluster[1].pid, signal.SIGSTOP) # Freeze backup so tx can't
> complete.
> threads = [ Thread(target=s.commit) for s in sessions]
> for t in threads: t.start()
> cluster[0].ready(timeout=1) # Should not block
> os.kill(cluster[1].pid, signal.SIGCONT) # Allow tx to complete.
> for t in threads: t.join()
> c.close()
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
