[
https://issues.apache.org/jira/browse/AMQ-7067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638417#comment-16638417
]
Jamie goodyear edited comment on AMQ-7067 at 10/4/18 5:40 PM:
--------------------------------------------------------------
Please see the amq7067 unit test.
This patch is for AMQ 5.15.x branch.
You can execute using the following command:
mvn -Dtest=AMQ7067Test test
Keeping original comment for history:
I've attached a stand alone test case to reproduce the issue.
Here's how to use the test case:
0. Setup AMQ 5.15.6 broker on local system. Use default configuration, no
special settings required. Start broker.
1. Extract AMQ7067 zip file.
2. Run XATransactionTest while Broker is live using : mvn clean compile
exec:java -Dexec.mainClass="XATransactionTest"
The test case will output:
[INFO] — exec-maven-plugin:1.6.0:java (default-cli) @ transaction —
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/Users/icbts/.m2/repository/org/apache/activemq/activemq-all/5.15.6/activemq-all-5.15.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Users/icbts/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
createDanglingTransaction txid:
XID:[55,globalId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1,branchId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1]
createXATransaction txid: XID:[55,globalId=000015333ec,branchId=000015333ec]
Pending Transactions 1:
[XID:[55,globalId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1,branchId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1]]
INFO | I/O exception (org.apache.http.NoHttpResponseException) caught when
processing request to {}->[http://localhost:8161:] The target server failed to
respond
INFO | Retrying request to {}->[http://localhost:8161|http://localhost:8161/]
Pending Transactions 2:
[XID:[55,globalId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1,branchId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1],
XID:[55,globalId=000015333ec,branchId=000015333ec]]
[INFO] ------------------------------------------------------------------------
In the Broker log you'll see:
INFO | KahaDB is version 6
INFO | Recovering from the journal @1:741
INFO | Recovery replayed 429 operations from the journal in 0.081 seconds.
WARN | Recovered prepared XA TX:
[XID:[55,globalId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1,branchId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1]]
WARN | Recovered prepared XA TX:
[XID:[55,globalId=000015333ec,branchId=000015333ec]]
On JConsole you'll see two messages in org.apache.activemq > Broker >
RecoveredXaTransaction MBean.
Given we commit the transaction, I'd expect there to not be a message in
Recovered state.
Note: If we do not send additional messages to force a new page file, than we
do not see the message in Recovered state after broker restart.
It appears to be that the prepared statement in db-1.log,. and commit in
db-2.log will result in the commit getting GC'd (hence no pare to match) and
having the message being seen as being in prepared state, and hence getting
recovered.
was (Author: jgoodyear):
I've attached a stand alone test case to reproduce the issue.
Here's how to use the test case:
0. Setup AMQ 5.15.6 broker on local system. Use default configuration, no
special settings required. Start broker.
1. Extract AMQ7067 zip file.
2. Run XATransactionTest while Broker is live using : mvn clean compile
exec:java -Dexec.mainClass="XATransactionTest"
The test case will output:
[INFO] — exec-maven-plugin:1.6.0:java (default-cli) @ transaction —
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/Users/icbts/.m2/repository/org/apache/activemq/activemq-all/5.15.6/activemq-all-5.15.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Users/icbts/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
createDanglingTransaction txid:
XID:[55,globalId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1,branchId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1]
createXATransaction txid: XID:[55,globalId=000015333ec,branchId=000015333ec]
Pending Transactions 1:
[XID:[55,globalId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1,branchId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1]]
INFO | I/O exception (org.apache.http.NoHttpResponseException) caught when
processing request to {}->[http://localhost:8161:] The target server failed to
respond
INFO | Retrying request to {}->[http://localhost:8161|http://localhost:8161/]
Pending Transactions 2:
[XID:[55,globalId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1,branchId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1],
XID:[55,globalId=000015333ec,branchId=000015333ec]]
[INFO] ------------------------------------------------------------------------
In the Broker log you'll see:
INFO | KahaDB is version 6
INFO | Recovering from the journal @1:741
INFO | Recovery replayed 429 operations from the journal in 0.081 seconds.
WARN | Recovered prepared XA TX:
[XID:[55,globalId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1,branchId=ffffffffffffffffffffffffffffffffffffffe9ffffffb8fffffff7ffffffa1]]
WARN | Recovered prepared XA TX:
[XID:[55,globalId=000015333ec,branchId=000015333ec]]
On JConsole you'll see two messages in org.apache.activemq > Broker >
RecoveredXaTransaction MBean.
Given we commit the transaction, I'd expect there to not be a message in
Recovered state.
Note: If we do not send additional messages to force a new page file, than we
do not see the message in Recovered state after broker restart.
It appears to be that the prepared statement in db-1.log,. and commit in
db-2.log will result in the commit getting GC'd (hence no pare to match) and
having the message being seen as being in prepared state, and hence getting
recovered.
> KahaDB Recovery can experience a dangling transaction when prepare and commit
> occur on different pagefiles.
> -----------------------------------------------------------------------------------------------------------
>
> Key: AMQ-7067
> URL: https://issues.apache.org/jira/browse/AMQ-7067
> Project: ActiveMQ
> Issue Type: Bug
> Components: KahaDB, XA
> Affects Versions: 5.15.6
> Reporter: Jamie goodyear
> Priority: Critical
> Fix For: 5.16.0, 5.15.7
>
>
> KahaDB Recovery can experience a dangling transaction when prepare and commit
> occur on different pagefiles.
> Scenario:
> A XA Transaction is started, message is prepared and sent into Broker.
> We then send into broker enough messages to file page file (100 message with
> 512 * 1024 characters in message payload). This forces a new pagefile to be
> created.
> Commit the XA transaction. Commit will land on the new page file.
> Restart the Broker.
> Upon restart a KahaDB recovery is executed.
> The prepare in PageFile 1 is not matched to Commit on PageFile 2, as such, it
> will appear in recovered message state.
> Looking deeper into this scenario, it appears that the commit message is
> GC'd, hence the prepare & commit can not be matched.
> The MessageDatabase only checks the following for GC:
> {color:#808080}// Don't GC files referenced by in-progress tx
> {color}{color:#cc7832}if {color}(inProgressTxRange[{color:#6897bb}0{color}]
> != {color:#cc7832}null{color}) {
> {color:#cc7832}for {color}({color:#cc7832}int
> {color}pendingTx=inProgressTxRange[{color:#6897bb}0{color}].getDataFileId(){color:#cc7832};
> {color}pendingTx <=
> inProgressTxRange[{color:#6897bb}1{color}].getDataFileId(){color:#cc7832};
> {color}pendingTx++) {
> gcCandidateSet.remove(pendingTx){color:#cc7832};
> {color} }
> }
> We need to become aware of where the prepare & commits occur in pagefiles
> with respect to GCing files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
