lordgarrish commented on code in PR #13339: URL: https://github.com/apache/ignite/pull/13339#discussion_r3945029454
########## docs/_docs/key-value-api/transactions.adoc: ########## @@ -322,6 +341,89 @@ A transaction might fail with the following exceptions: == Long Running Transactions Termination +The `tx` command of the `control.sh` utility allows you to search for transactions based on various criteria (such as ID, duration, or label) and roll back transactions. Since transactions in Apache Ignite are generally distributed (multiple nodes are involved in processing a single transaction), this command may display the same transaction as present on multiple nodes simultaneously. + +Rolling back a transaction using the `tx` command is only possible for the `ACTIVE`, `PREPARING`, and `PREPARED` statuses. A transaction in statuses related to the `rollback` procedure is already in the process of rolling back. Transactions in the `COMMITTING` and `COMMITTED` statuses cannot be rolled back due to 2PC protocol restrictions. + +* First termination stage — cancelling transactions using `tx kill`. To cancel transactions, use the `control.sh` utility to minimize the impact on the cluster: +. Execute the `control.sh --tx --min-duration <transaction_timeout_in_ms>` command in the cluster. Example output: ++ +---- +Command [TX] started +Arguments: --tx --min-duration 30 +-------------------------------------------------------------------------------- +Matching transactions: +TcpDiscoveryNode [id=<id>, addrs=[<address>], order=1, ver=2.18.0#YYYYMMDD-sha1:00000000, isClient=false, consistentId=gridCommandHandlerTest0] + Tx: [xid=<id>, label=null, state=ACTIVE, startTime=YYYY-MM-DD 15:15:16.852, duration=3 sec, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=100, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]] + Tx: [xid=<id>, label=null, state=ACTIVE, startTime=YYYY-MM-DD 15:15:16.852, duration=3 sec, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=100, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]] +Command [TX] finished with code: 0 +---- +. Extract the `xid` from all transactions across all nodes and remove duplicate `xid` values. Duplicates occur because the same transaction can exist on multiple nodes — in this case, its `xid` will appear in the command output multiple times. +. For each `xid`, run the `tx kill` command to roll back that transaction. Example call: `control.sh --tx --xid <id> --kill`. + +* Second termination stage (optional) — restarting nodes that block transaction completion. Check if the first stage was sufficient to handle LRT: +. After completing the first stage, wait for a period equal to the configured transaction timeout. +. Run the `control.sh --tx --min-duration <transaction_timeout>` command. +If the list is empty, all LRTs have been successfully canceled, and the LRT impact should be resolved. +If the list is not empty, the command output will contain information about nodes and the transactions running on them: ++ +---- +Command [TX] started +Arguments: --tx +-------------------------------------------------------------------------------- +Matching transactions: +TcpDiscoveryNode [id=<id>, addrs=[<address>], order=1, ver=2.18.0#YYYYMMDD-sha1:00000000, isClient=false, consistentId=gridCommandHandlerTest0] + Tx: [xid=<id>, label=null, state=ACTIVE, startTime=YYYY-MM-DD 16:58:45.348, duration=3 sec, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=100, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]] + Tx: [xid=<id>, label=label1, state=ACTIVE, startTime=YYYY-MM-DD 16:58:48.451, duration=0 sec, isolation=READ_COMMITTED, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=2147483 sec, size=111, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]] +TcpDiscoveryNode [id=<id>, addrs=[<address>], order=2, ver=2.18.0#YYYYMMDD-sha1:00000000, isClient=false, consistentId=gridCommandHandlerTest1] + Tx: [xid=<id>, label=null, state=ACTIVE, startTime=YYYY-MM-DD 16:58:45.348, duration=3 sec, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=1, dhtNodes=[<id>], nearXid=<id>, parentNodeIds=[<id>]] +TcpDiscoveryNode [id=<id>, addrs=[<address>], order=3, ver=2.18.0#YYYYMMDD-sha1:00000000, isClient=true, consistentId=client] + Tx: [xid=<id>, label=label2, state=PREPARING, startTime=YYYY-MM-DD 16:58:45.348, duration=3 sec, isolation=READ_COMMITTED, concurrency=OPTIMISTIC, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=2], timeout=0 sec, size=11, dhtNodes=[<id>, <id>], nearXid=<id>, parentNodeIds=[<id>]] +Command [TX] finished with code: 0 +---- +. For each transaction, extract the value of the `dhtNodes` field. +. For each ID in `dhtNodes`, obtain detailed node information using the `control.sh --system-view NODES` command. +. In the command output, find the node whose `nodeId` starts with the string from `dhtNodes`. +. Restart this node to unblock the transaction. + +== Causes of LRT +* High system load, which leads to slower transaction processing. Indicators: Review Comment: Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
